Motion and shape can hardly be described in real-life applications of noisy video surveillance. Using a greater number of cameras can help to obtain super-resolution or registered tracks, but this entails calibration of the cameras. The definition of shape or motion in outdoor applications is ambiguous. Unfixed cameras and transient motion of objects together present the challenge of applying statistical inference and optimisation to simultaneously estimate the relative positions of cameras and moving objects. In developing a multi-camera surveillance system, our task is the detection of motion, and the segmentation and characterisation of moving objects. The information coming from the various camera units is interchanged to obtain more precise data concerning direction and vectors of motion, and to recognise the object and its behaviour. These tasks need fast data transmission, comparison of images, and continuous evaluation and explanation of image characteristics. This in turn requires high-speed processing to ensure real-time operation. We are working on relaxation-based object-tracking methods for indefinite object shapes, where moving objects following different paths may obscure each other, or partially fade into other objects in low contrast areas.
This research has strong connections with theoretical work being done at the Laboratory of Image Processing in Veszprém University (led by Tamás Szirányi), where a new semi-automatic digital restoration system is being introduced for motion-picture restoration for film archives. The automation is controlled by occasional operator interactions. For this purpose, the film analysis is supported by cut detection and film indexing based on colour information and motion activity. Data representation in XML also aims to create well-defined and controllable processes. When restoring defective films, corresponding scene sequences must be registered with each other by scene-based indexing.
Our surveillance work is supported by appropriate hardware and network tools. These include intelligent camera units with processing and optimised networking capabilities, ultra-fast image-analysis engines of Cellular Neural Network processors, and wired and radio transmission of images and data. The camera system is robust in arbitrary connections and geometry. We are developing the system for automatic grouping and calibration.
Another important issue involves human sensation and the information content of the digital image, including the artistic interpretation of a scene. We have run several human tests for qualifying methods in which artefacts and objects must be sequestered.
Our project is partly supported by the Hungarian National Research and Development Program: TeleSense NKFP 2001/02/035. With this activity, we joined a new Network of Excellence project run by ERCIM: MUSCLE (Multimedia Understanding through Semantics, Computation and Learning).