ERCIM News No.26 - July 1996 - FORTH

Using Remote Memory
to Avoid Disk Thrashing

by Evangelos P. Markatos and George Dramitinos

Traditional operating systems use magnetic disks to store filesystems and memory pages even though the cost of a page transfer between disk and main memory, measured in processor cycles, continues to increase. Moreover, applications like multimedia, windowing systems, scientific computations, engineering simulations, etc, running on workstation clusters (or networks of PCs) require an ever increasing amount of main memory, usually more than any single workstation has available. To alleviate the memory shortage problem, an application could use the virtual memory paging provided by the operating system, and have some of its data in main memory and the rest on the disk.

Unfortunately, as the disparity between processor and disk speeds becomes ever increasing, the cost of frequently accessing a magnetic disk becomes unacceptable. While some workstations are overloaded with memory-starved applications, other workstations (within the same cluster) stay idle, effectively wasting their processors and memories. Profiling the unused memory of the workstations in our lab, we found that the amount of free memory rarely falls under 50% of the total amount of the cluster's memory. This huge amount of unused memory could be used as a global cache for the cluster's disk partitions, both for filesystem caching and for VM page storage, if the appropriate OS support is provided.

To demonstrate the benefits and ease of use of remote memory, we have designed and implemented a pager that uses remote memory as backing store. The workstations that devote part of their memory to store pages of remote applications are called remote memory servers, while the workstations that use this memory are called remote memory clients. A workstation may act as a server or as a client depending on its memory load, as shown in the figure.

Figure:Workstations experiencing high memory load act as clients
while workstations that have enough free memory act as servers.

All server workstations run a remote memory server that handles requests for pageins, pageouts and informs the clients about the amount of its free memory. When a client wants to swap out a page it picks the most promising server. When a server runs out of memory, it informs its clients and stores some of its least recently used pages on its local disk. In this way, the servers act as a global LRU cache for the swap partitions of the cluster.

In order to be able to recover from a single server crash, some form of redundancy must be used, so that lost pages can be reconstructed. We have implemented a new reliability technique, parity logging, that minimizes the memory and runtime overhead imposed to paging.

Our performance results are very encouraging. Our test applications show that using the remote memory for paging results in a twofold improvement in execution time over an Ethernet interconnection network. Simulating faster networks we find that even higher performance improvements are realizable.

More information on the project may be found at

Please contact:
Evangelos P. Markatos - ICS-FORTH
Tel: +30 81 391655

return to the contents page