Testing the Robustness of Spacecraft Control Software

by Olaf Maibaum

Software robustness is a key factor for long-term space missions. To ensure mission success, the testing of software and systems is of utmost importance. This paper describes the testing of software robustness in a software-in-the-loop test bed as realized in the project SiLEST.

Space is an inhospitable environment for electronic and mechanical components. Radiation and thermal conditions cause accelerated aging of sensors and actuators, and the applied loads during take-off and manoeuvring can cause damage. Such conditions – unusual on Earth – are common in spacecraft operations, and the on-board software must be robust enough to cope with them autonomously as far as possible.

Intensive testing is necessary to demonstrate the robustness of software. Such evaluation must be performed at the system level as an ‘in-the-loop’ testing of the software. The traditional test approach is a ‘hardware in the loop’ (HiL) test. In this approach, the software runs on a controller board that mimics the target hardware. Using the native interfaces it is coupled with devices, which can be real hardware or simulations that reproduce the behaviour of sensors and actuators in the system. The dynamic behaviour of the environment is also implemented. The main disadvantage of an HiL test is the need for real-time simulation. With the increasing complexity of systems, the effort required for effective testing increases drastically.

Another approach is to test the software in a ‘software-in-the-loop’ (SiL) test bed. This method makes use of interfaces, sensors, actuators and an environment that are completely realized in software, ie simulated. Since the software being evaluated communicates with other system elements via simulated interfaces, no specialized hardware environment is needed. Tests can be run during the system design phase, and are not restricted - as with HiL test beds - to the integration phase. In addition, the integration of the software as part of the simulation is feasible. Further debugging is also facilitated in an SiL test bed.

However, the coupling in an SiL test bed generates a change in timing. This is the result of changes in the software that are necessary to establish the coupling, and means that timing failures can be masked. This is a drawback of SiL test beds, and requires further efforts to analyse the timing behaviour of the software.

To prove and demonstrate the advantages of SiL tests, an SiL test bed has been developed in the cooperative project SiLEST (Software in the Loop for Embedded Software Tests), with the additional objective of analysing the timing behaviour. SiLEST is a collaborative effort involving the German Aerospace Center (DLR), IAV GmbH, Webdynamix GmbH, the Technical University of Berlin, and Fraunhofer Institute for Computer Architecture and Software Technology (FIRST).

Records from the BIRD micro satellite are used for testing the control softare.

The aim of SiLEST is to test the robustness of embedded control software with simulated flaws in the electrical environment. Test objects are the Attitude Control Software (ACS) of the BIRD micro satellite from the space domain, and an engine control unit from the automotive domain.

The proper performance of the ACS is tested with noisy or lost sensor signals or jerky actuator movement. In such cases the ACS will at least switch to ‘satellite save mode’ to ensure the survival of the satellite until troubleshooting is available from the ground.
The engine control unit is tested in an SiL test bed with nominal behaviour and with faults such as the cable break of sensors, short circuits, signal noise, blocked mechanical components and so on.

During development, the SiL test bed will prove its adequacy for robustness tests in a closed loop. To achieve this, the test results of the SiL test bed are compared to the results of HiL test beds, and additionally in the case of the ACS with records from operations of the BIRD micro satellite.

The tests of the ACS are performed by the German Aerospace Center (DLR), while the tests for the engine control system are executed by the cooperation partner IAV.

A further objective of SiLEST is to increase the automation level and the efficiency of the tests. The test cases are described in XML. An adaptable XML test-case editor has been developed by the cooperation partner Webdynamix. This editor allows the creation of test cases for SiL and HiL tests and fulfils the requirements of the partners from both the space and automotive domains. Using plug-ins it is possible to integrate specialized data editors for the data input into the test cases. For example, a graphical editor allows control curves to be drawn. The intended test automation environment offers scripting facilities and the integration of different components at different locations by Web Services, in order to satisfy the requirements of several application domains.

By the end of the project, in mid-2007, it will be possible to determine the extent to which an SiL test bed is adequate for robustness tests of embedded control software. Recommendations will be made regarding the appropriate use of SiL or HiL test beds, and an adaptable test automation environment and a test case editor for SiL and HiL test beds will be available.

The SiLEST project is funded by the German Federal Ministry of Education and Research under code 01ISC12A. The author of the publication is responsible for the content.

Links:
SiLEST home page: http://www.silest.de/
BIRD home page: http://www.dlr.de/os/forschung/projekte/bird
Research program: http://www.softwarefoerderung.de/

Please contact:
Olaf Maibaum, German Aerospace Center (DLR), Germany
Tel: +49 531 295 2974
E-mail: Olaf.Maibaumdlr.de