Interactive Systems

Overall Goal of this Field of Research

The concept of interaction is relevant for our research in a twofold way. On the one hand, it is relevant from the application point of view. Here we are concerned with the development of computer systems which are able to interact with humans in a natural way and to learn autonomously from interactions within their environment. On the other hand, the concept of interaction is relevant for basic research as well. Recent results from the fields of cognitive robotics, artificial consciousness or the philosophy of mind suggest that the interaction of an embodied agent with its environment is crucial for consciousness to arise.

For instance, in the context of self-learning systems we investigate the general possibility of artificial systems to learn their own behavior or abilities such as scene recognition autonomously from interactions with their environment. More basic subjects of our investigation such as the cognitive skill of self-learning by interaction as a matter of principle are covered in the section on Cognitive Systems. The remainder of this section covers the application point of view of interaction, that is the development of interactive computer systems.

Major Challenges - and Links to Our Other Core Areas

The major demand made on computer systems which are supposed to interact with humans is their ability to adapt to their environment and especially to human needs, rather than expecting an adaption of humans to the restrictions of an artifical system. To this end an interactive system has to be adaptive, autonomous, and self-learning, and it has to be able to master at least three major challenges:

1. It has to be capable to actively explore, perceive, and understand its environment (see core area Computer Vision). This can be accomplished, for example, with biologically inspired methods. One yet unsolved subproblem here consists in the requested property of location awareness: an interactive system is supposed to be aware of the locations of the system components, the users, and the objects in its environment including other living beings (see core areas Image and Scene Synthesis and also Computational Neuroscience).
2. An interactive system should be able to evaluate its perceptions and to learn strategies of data processing and behavior in an autonomous way (see core area Cognitive Systems). This can be realized, for instance, with methods inspired by insights from cognitive psychology.
3. An interactive system has to present data to be communicated to humans in the most intuitive way. As vision is considered our dominant sensory modality, particular emphasis is laid on the visual display of data (see core area Image and Scene Synthesis). Results from the science of art and aesthetics can be helpful here to design interfaces of interactive systems (see core area New Media Art).

Unsolved Problems

Yet unsolved problems in the development of interactive systems consist in

• the fusion of methods and techniques formerly separated into the distinct fields of Computer Vision, Computer Graphics, or Machine Learning and
• the development of multimodal forms of interaction between humans and computers.
  This comprises the development of new sensors, the integration of information from multiple sensors on the input side of a system, the seamless integration of these different modalities within the system, and the preparation of multiple output channels for the information transfer from the system to humans.

Our Expertise

Our group has multiple activities in this field of research. For example, as part of a project funded by the German Research Foundation (DFG) we developed a systems that is able to autonomously learn strategies for object acquisition by a fusion of methods from Computer Vision and Machine Learning. Within a framework of another project we developed a flexible display realized by a Projector-Camera System. It projects images of objects on plain surfaces such as table tops or walls and takes the position of a user into account to convey a 3D impression of the displayed objects. Furthermore, we are active in the field of Freehand Acquisition of Objects as well as Eye Gesture- and Head Gesture-Controlled Interfaces.