Fischlár-News: Multimedia Access to Broadcast TV News

by Alan F. Smeaton, Noel E. O’Connor and Hyowon Lee

Fischlár-News is an operational system which provides content-based access to a growing archive of broadcast TV news.

The Centre for Digital Video Processing at Dublin City University has been carrying out leading research into video analysis and structuring since the mid-1990s and this has been done with a view towards supporting content-based operations such as summarisation (movies), highlight detection (sports) and searching/browsing (TV News). Part of this work has culminated in the Físchlár-News system as part of a project which commenced in 2000. The resulting system captures broadcast TV news, nightly, and automatically analyses and structures the entire broadcast into an MPEG-7 annotation. These analysis processes include shot boundary detection, detection of keyframes, identification of the exact start/end of the program, TV advertisement detection, speech/music discrimination and automatic detection of anchorperson shots. The outputs of these analyses are fed into a trained Support Vector Machine (SVM) which segments the broadcast into discrete news stories which we use as the units for retrieval. The analysis of a broadcast news program takes about real time and the program is available on the system about 30 minutes after its transmission.

A user uses Físchlár-News to access TV news video through a conventional web browser and when a user logs on he/she is presented with several ways in which to locate TV news stories. The most popular modality is access by date and a calendar option provides fast access to all the news stories for a given date in this way. Users can also enter keywords into a text search box which are then matched against a text representation for each news story using standard information retrieval approaches, captured via the closed captions associated with the broadcast. Once a story is located it is presented as a set of keyframes, including an anchorperson shot plus keyframes for any outside footage for that story, inter-twined with the text dialogue of that story as taken from the closed captions. For each story presented we also present a list of “related” stories which are those from the archive (currently about 10,000 stories, and growing daily) which are most similar to the story in question. This allows a user to follow a thread of stories of, for example, a criminal event, the tracking down and capture of the criminals, their trial, sentencing, etc. It also allows a user to follow links to related stories. For example, when viewing a story of a murder in Dublin city a user would be shown links to stories of other murders in Dublin from other dates, etc. Finally, we also incorporate a personalisation and recommender system into Físchlar-News which tracks users’ viewing of stories, as well as their explicit story ratings, and we use this to provide a “recommended stories” feature which is used when users have been away from the system for some time. This is also useful to bring older stories which users may have missed, or forgotten, to their attention. The diagram in the figure shows the process or news video analysis in the system and apart from some periodic sanity-checking of the performance of the SVM, the process is entirely automatic.

The Físchlár-News system has been shown to be very useful for users but it is primarily a showcase for our underlying research in multimedia content analysis and automatic annotation using MPEG–7. It demonstrates a variety of ways in which multimedia content (digital video in our case) should be indexed and subsequently accessed. The system has been operational on the University campus for nearly 3 years and over 1,500 unique users have used the system to keep themselves informed of TV news. It is especially useful for people when traveling as it allows them to access their own TV local news, from abroad using only a web browser and their password.

Although the research project which funded the development of Físchlár-News was finished in late 2003 we have kept the system operational, mostly because of the demands of our users who find it too useful to be without ! The system also acts as an operational showcase of how video analysis and multimedia analysis and annotation can lead to useful systems to allow searching and browsing of that same multimedia content. Several extensions to the system have also been suggested including recording more than 1 TV news program per day, recording TV news from more than 1 broadcaster and allowing more explicit temporal browsing through topic threads. These suggestions came from an extensive user study we performed recently based on analysis of usage data, user diaries and pre- and post-study questionnaires. It is our intention to incorporate as many of these suggestions as we can.

The support of the Informatics Directorate of Enterprise Ireland is gratefully acknowledged.

Link:
Centre for Digital Video Processing: http://www.cdvp.dcu.ie/

Please contact:
Alan F. Smeaton, Centre for Digital Video Processing, Dublin City University /
Irish Universities Consortium, Ireland
E-mail: Alan.Smeatondcu.ie