ERCIM News No.47, October 2001 [contents]

A Flood of Intelligence - the Living Room Project

by Jukka Vanhala

The Living Room project studies the smart home environment. The project is unique in its approach of developing user interface paradigms and communication techniques for large collections of intelligent small objects.

Today we are able to add intelligence into an object with about 20 Euros. We are interested in what happens when the cost is eventually closer to 20 cents. The research is conducted at the Institute of Electronics of the Tampere University of Technology in Finland by a research group called Personal Electronics Group, PEG. The project started in summer 1999 and is continuing.

Let’s get Physical
The concept of digital convergence is expanding. We share the vision that the ambient intelligence arises from the convergence of three key technologies: Ubiquitous Computing, Ubiquitous communication, and Intelligent User Friendly Interfaces. But there is also the fourth component, which is the real world physical objects.

The boundary between real world and the information world will vanish. Traditionally things have existed on their own sides. Food and tools clearly are material, books and email are mostly information. The new technology will connect the two worlds. A new car has dozens of micro controllers and a computer and a mobile phone are primarily information appliances. It becomes possible to control the real world trough the network and make use of information contents by manipulating real world objects.

In addition to information processing, communication, and contents the everyday living environment gets a digital form. Eg imagine a tee shirt suitable for children’s games. There would be a computer with a game contents downloaded from the network and sensors measuring the actions of the player. The shirt communicates with the other players’ shirts. Clearly the tee shirt is a real physical object, which has been augmented with digital contents.

The everyday environment at the home or at the office will still be a real physical environment. The physical objects will get new digital content and functions, but they still have their traditional functions. How people will see the effect of the Moore’s law is not in the high-end computers doing spectacular things, but rather in the very small processors embedded in every imaginable object forming a very fine grain parallel multicomputer with a user interface spread across the whole environment.

The interior of the Living Room
The interior of the Living Room showing spotlights, the floor with pressure sensors, and the smart flowerpot.

the Living Room project studies the smart home as a place where intelligence is embedded in practically all physical objects ranging from kitchen appliances to toothpicks. A home of the future might have thousands of active smart objects. Today the cost of embedding some simple sensors, a microcontroller, a short-range radio link, and a battery into an object is about 20 euros. The size of the device is about the size of a matchbox. In about ten years it is possible to integrate all electronics onto a single chip and the cost and the size will drop dramatically. We reach an interesting point when the intelligence adds less than 20 cents to the cost of the product. Then it is possible to have smart things costing about 2 euros total, a price tag on a say toothbrush or on a cone of ice cream.

Natural and Control Interfaces
A smart object has a natural user interface and a control user interface, which can be physically separated. All objects have a physical appearance and functionality that represents its natural purpose. Tools have handles and a refrigerator has a door and shelves. We call this the natural user interface. The design of it has traditionally been the job of the industrial designers and the mechanical engineers. Now they too have to learn new skills, as the embedded electronics will give new functionality to old objects. The enhanced natural user interfaces utilizes sensors to measure the user action and the object reacts with actuators typical to each object.

As electronics is embedded into objects, they will get also information content and ability to process information. The information is in symbolic form and therefore a smart object needs also an interface capable to represent and input symbols. Without the communication capability each of the objects would need its own small control interface with probably a few buttons and a couple of text lines on an LCD-display. It is difficult to design a good interface with such a limited set of controls. It is also clear that most of these interfaces would have different logic and different appearance. Thus the user must learn to use dozens of badly designed interfaces.

When the ability to communicate with the home network is an integral part of the devices the situation improves. All control interfaces can be detached from the actual device much in the same manner as a TV has a remote controller. Here we run into another well-known problem. It is no better to have dozens of loose remote controllers instead of the control interfaces on each device. The solution is to have a universal remote controller, which can control all possible devices. There are already such devices on the market although they are pretty limited in their design. A future control device, the universal information appliance, UIA, should have a large screen, a pointing device, speech input and a standard method to describe the appearance and functionality of the interface, which can be eg loaded from the web page of the appliance manufacturer. Designing sets of ‘skins’ to control interfaces for smart objects could become a new value added service. Service provider could be independent of appliance manufacturers and provide matching designs for all devices. There will also be a need for different kinds of terminals, thus the interfaces must scale to meet the abilities of the underlying hardware.

The Living Room
Living Room is besides the name of the projects also the name of the actual laboratory where the new systems can be evaluated. The 40 square meter laboratory resembles a flat with a living room and a bar kitchen. During the summer 2002 a new laboratory will be opened with a two room fully equipped flat including a full kitchen and a sauna.

At the moment the infrastructure is ready comprising of a short range RF-network, the design for the controller to be embedded into smart objects, the master control computer and the prototype system software with a control user interface. There are also a number of smart objects including a flowerpot, a door with locks, an array of spotlights, shades, refrigerator, pressure sensitive floor, and even a smart wine bottle rack.

Communication is done wirelessly with a narrow band control network, which has been designed just for this purpose. It is based on the Nordic VLSI transceiver chip, which operates on the license-free 433 MHz IMS band. The throughput is 10 kbits/s and the reliable range is over 30 meters, which is enough for the normal rooms at home. Of course it is not possible to transmit images or speech over the control network. For that purpose there is the IEEE 802.11 wireless LAN connecting the control computer and the user interface terminals, which are both PC-class computers.

Many legacy products such as TVs and audio equipment use infra red transmission for communication. Since those devices will be in use for at least another twenty years, there must be a way of controlling them. An IR-receiver/transmitter is connected to the control computer via a serial link so that the same unified user interface as is used to control devices via the RF-network can be used.

The controller to be embedded into smart objects consists of an Atmel AVR-series microcontroller, sensor interface electronics, the RF-link, and a battery. Software is easily customizable for different requirements. The size of the controller is about 2 times 4 times 6 centimeters.

At the moment we are using a normal office-PC with the Windows-98 operating system as the control computer. Eventually the controller will be a home gateway type standalone computer but the prototyping work is much easier with the good development tools available for Windows-98. The user control interface is implemented on an industrial PC with a touch screen. The computer is embedded into the surface of a coffee table. Similar control terminals can be installed on walls and other surfaces.

Conclusions and Future Development
Traditional user interface research has concentrated on the design of the control interface. When all objects in a smart space can be used to collect and distribute information the collection of the natural user interfaces will dominate the behavior. We call the collection of the natural user interfaces a collective user interface. There is not much empirical knowledge or experience on this because it would require a full implementation of a smart home, which has not yet been built anywhere. The Living Room –project is working towards implementing a prototype environment of a collective user interface. Our approach is very practical. We are interested both in the overall architectural design and in the implementation details. Real world experiments and prototype installations are used to test all designs.


Please contact:
Jukka Vanhala - Tampere University of Technology, Institute of Electronics, Finland
Tel: +358 3 365 3383
E-mail: jukka.vanhala@tut.fi