[meteorite-list] Mars Express Observations Temporarily Suspended

From: Ron Baalke <baalke_at_meteoritecentral.com>
Date: Tue, 1 Nov 2011 09:31:46 -0700 (PDT)
Message-ID: <201111011631.pA1GVkaM002429_at_zagami.jpl.nasa.gov>

http://sci.esa.int/science-e/www/object/index.cfm?fobjectid=49549

Mars Express observations temporarily suspended
European Space Agency
31 Oct 2011

Anomalies in the operation of the solid-state mass memory system on
board Mars Express have caused science observations to be temporarily
halted. A technical work-around is being investigated that will enable
the resumption of a number of observations and should evolve into a
long-term solution.

In mid-August, Mars Express autonomously entered safe mode, an
operational mode designed to safeguard both the spacecraft itself and
its instrument payload in the event of faults or errors.

The cause of entering the safe mode was a complex combination of events
relating to reading from and writing to memory modules in the
Solid-State Mass Memory (SSMM) system. This is used to store data
acquired by the instruments and housekeeping data from the spacecraft's
subsystems, prior to its transmission to Earth, and is also used to
store commands for the spacecraft that have been received from the
ground stations, while awaiting execution.

Switch to spare memory controller

As the previous safe mode was three years ago and the current event
looked like a 'normal' transition to safe mode, the flight control team
executed the standard recovery procedure and restarted observations. A
few days later, another fairly similar set of SSMM problems occurred.
The decision was then taken to switch over to the cold-redundant spare,
or 'B-side', SSMM controller, as this was virtually the only subsystem
common to the two events. The decision to act was triggered by both the
need to achieve stable science performance and the need to reduce the
consumption of fuel caused by transitions to safe mode.

An important element in trying to reduce the number of safe mode
transitions, besides the science observing time being lost, is the
amount of fuel used when entering safe mode, which is approximately
equal to the amount of fuel required for six months of nominal
operations. The reason for this fuel consumption when entering safe mode
is the 'Sun acquisition' process, which requires a significant amount of
spacecraft manoeuvring.

Full science observations were resumed on 15 September, using the B-side
SSMM controller. Eight days later, the spacecraft again entered safe
mode. The cause was a checksum error during communication between two
subsystems within the SSMM. As this event exhibited no similarities with
the two events in August, it was decided to treat this as an isolated
incident and return to nominal operations; this took place on 29 September.

On 11 October, a similar problem occurred, but no safe mode transition
was triggered. As all operations continued normally, it was decided to
take no action, but to find a slot to execute a warm reset of the SSMM
at the earliest opportunity.

Mars Express mass memory anomaly -- event timeline

Date Event
13 August 2011 Safe mode transition following SSMM read/write error
18 August 2011 Observations restarted
23 August 2011 Re-occurrence of SSMM read/write error - without safe
                  mode transition
24 August 2011 Decision to switch to B-side SSMM controller
15 September 2011 Observations restarted using B-side SSMM controller
23 September 2011 Safe mode transition following SSMM internal communications error
29 September 2011 Return to nominal operations
11 October 2011 SSMM internal communications error - without safe mode transition
16 October 2011 Safe mode transition following SSMM internal communications error
                  Observations temporarily suspended

Observations halted

On Sunday, 16 October, Mars Express again entered safe mode. The cause
was a sequence of events very similar to the one that triggered the safe
mode transition on 23 September, following an internal communication error.

Resuming use of the A-side SSMM controller is not considered a viable
option; it would most probably lead to further safe mode transitions
caused by memory module read/write errors and the additional fuel
consumption caused by these events cannot be sustained. Faced with these
circumstances, the Mission Manager decided to halt science observations.

Possible work-around

The SSMM is a critical subsystem on board Mars Express, central to all
platform and instrument operations. The ESOC flight control team is
preparing a work-around to address the current problems and allow at
least partial resumption of science observations.

During normal operations, telecommands for the platform and instrument
payload that have been received from the ground stations are stored in
the Long Mission TimeLine (L-MTL), a special file in the SSMM that is
used to fill the platform's command cache. In order to cope with
anomalous operation scenarios, a hardware-based timeline store - the
Short Mission TimeLine (S-MTL) - is available outside the SSMM subsystem.

The work-around now being investigated uses the S-MTL to store
telecommands before they are executed.

Good progress is being made with the implementation of this alternative
approach to commanding the Mars Express platform and its payload. A full
test on the spacecraft will be executed in the near future.

ESA's mission control team and other experts are working closely with
the designers and manufacturers of the SSMM subsystem to investigate the
cause of the read/write anomalies and to attempt to find a solution that
will enable a return to normal operations.
      
Contact

F. Jansen, Mars Express Mission Manager
Directorate of Science and Robotic Exploration, ESA, The Netherlands
Email: fjansenrssd.esa.int
Received on Tue 01 Nov 2011 12:31:46 PM PDT