Cover ERCIM News 62

This issue in pdf
(64 pages; 10,4 Mb)


Cover ERCIM News 61
previous issue:
Number 61
January 2005:
Special theme:
Biomedical Informatics

previous issues online

Next issue:
October 2005

Next Special theme:
Security and Trust Management

Call for the next issue

About ERCIM News

< Contents ERCIM News No. 62, July 2005
SPECIAL THEME: Multimedia Informatics

Managing Digital Photo Collections

by Lynn Wilcox

The FXPAL photo application automatically organizes digital photo collections based on date, event, person, or place.

The increased use of digital cameras is creating a need to manage large collections of photos. People want to organize their photos, browse their collections, search for a particular photo, and create slide shows to share with friends. However, people are typically unwilling to manually classify their photos for organization, and most often simply upload the images from the camera into folders on their computer.

The FXPAL photo application is a tool that allows users to automatically organize their photo collections on the fly. The user first tells the application where the photos are stored on the computer. He then selects the desired view and the system displays the photos accordingly. For example, if the user wants to see the photos organized by date, he can select the dates view to see photos grouped hierarchically by year, month, and day. Similarly, he can select the people view to see photos grouped according to who is in the picture and the place view to see photos grouped according to location. There is also an event view that automatically groups the photos into meaningful events, such as birthdays or weddings.

We achieve these groupings by automatically analyzing the photos’ content and the metadata associated with the photos. By metadata, we mean information such as the date, time, and location the picture was taken. Grouping by time and date is straightforward — we simply create a hierarchy that allows users to select all photos by year, month, or day. The FXPAL photo application provides a calendar interface that visualizes the photos within the calendar for easier selection. The use of space is optimized in sparse calendar views by expanding the days of the calendar where there are photos and shrinking the days where there are none.

Figure 1
Figure 1: Event View in the photo organizer. Events have been automatically determined. Semantic event labels are assigned by the user.
Figure 1
Figure 2: Face view in the photo organizer, showing cropped faces from photos and their similarity to the photo labeled ‘Andreas’.

Grouping photos by events such as holiday gatherings or vacations is a common practice among photographers. The FXPAL system automatically detects events by clustering photos according to the time they were taken and their content. The technique is based on similarity analysis, in which the self-similarity matrix of the photos, computed from temporal and content similarity, is partitioned into disjoint events. After photos have been automatically grouped into events, users can attach semantic labels, so that when the photos are displayed in the event view the contents of the collection are easily understood.

Another common method for grouping photos is according to people. Manually assigning names to people in photos is tedious and time consuming. We make use of the FSCA face detection and recognition system to make this task easier. Face detection is highly accurate and is run automatically. Faces are cropped from the photos by orienting an ellipse based on the line segment between the eye positions. Users can view only the cropped faces from the photos by selecting the face view.

To determine the identity of people in the photos, we use the FSCA face recognition system. Unlike face detection, face recognition is not reliable. Variations in lighting, pose, and eyewear cause the system to make errors. Thus we provide a user interface for semi-automatic classification of faces according to person. Using the face view, the user selects one or more faces of a particular person to create a model. The user then asks the system to find similar faces. This results in a display of faces ordered by similarity to the model. The user selects correct faces, usually found near the top of the list, and adds them to the set of faces corresponding to the person. These faces are used to update the model, thus increasing the accuracy when more faces of the person. This process is repeated until all faces for people in the collection have been labeled.

The FXPAL photo application also provides interesting ways to view photos. The Pan‘n Scan animated slide show displays full screen photo images with background music. The slide show is animated by panning and zooming through each image using rules based on location of faces. Another way to view photos is as a Stained Glass image. Selected photos are cropped using detected faces and assembled in a collage with irregular boundaries similar to stained glass windows.

The photo application created by FXPAL and FSCA is available for user testing. Please contact the author for use permission.


Please contact:
Lynn Wilcox, FX Palo Alto Laboratory, USA
Tel: +1 650 813 7574