ERCIM News No.37 - April 1999

SFT - Smart File Transfer

by Markus Laakso

---

SFT, Smart File Transfer, is an application developed by VTT Information Technology for automatic transfer and distribution of files over mobile networks. The need for this emerged from the Finnish Maritime Administration as the Finnish icebreaker fleet had problems in transferring data to and from the icebreakers. Reliable data transfer is essential for optimal fleet management. Using SFT, the nine Finnish icebreakers get satellite images and ice charts daily and use this information in combination with information about the ship traffic to co-ordinate the assistance of ships through ice. The most cost-effective network in the areas of operation is still the NMT-450 analogue cellular network. In the Gulf of Finland GSM-data has been used and the newest multi-purpose icebreakers are equipped with satellite communication facilities.

Using SFT, the information is automatically distributed to the icebreakers. Controlled by its parameters, SFT scans predefined input directories and transfers any files found in these directories to the remote nodes and vice versa. As the connection is unreliable, and may break in the middle of a file transfer, SFT uses an acknowledging protocol with resendings if needed, to guarantee that a file, once queued for sending, is received once and only once at the receiving node.

SFT uses standard transfer protocols on top of which additional layers have been built. In the present implementation, RCP (Remote Copy) has been used for file transfer, but could be substituted by FTP or some secure transfer protocol if needed. In this way standard routers, with dial-on-demand features, can be used for establishing the connection.

The applications using the services provided by SFT, may have different types of transfer requirements: small files related to operational information should be transferred with as little delay as possible, but maps and satellite images are large and do not have as stringent delay requirements. SFT implements two types of queues: a priority queue and an ordinary queue. Files in the priority queue are sent before files in the ordinary queue. In addition, new priority files suspend the transfer of an ordinary file. This multiplexing makes it possible to send even large files as background transfer without blocking the channel for time sensitive priority files.

Automatic file compression

To save transfer time, the files are compressed before sending using lossless compression (Lempel - Ziw) and expanded at the reception. This enables simple formats to be used by different applications without the need of separate compression prior to sending. To guarantee that the file is correctly reassembled an MD5 checksum algorithm is used to check that the original file has been transferred unchanged.

Because SFT is used unattended 24 h/day 7days/week, special logfiles are automatically created, which gives information about the performance of the system as well as trace information in case of unexpected errors.

To summarise, SFT atomises the information exchange between the co-ordinating centre and the icebreakers providing reliable unattended file transfer over unreliable connections. Although the use of standard protocols does not necessarily maximise the net transfer rate, the robustness and modularity of this approach are considered to be more valuable for operational use than maximised channel capacity utilisation.

Please contact:

Markus Laakso - VTT Information Technology
Tel: +358 9 456 4568
E-mail: Markus.Laakso@vtt.fi