Image and Scene Synthesis

Overall Goal of this Field of Research

Whereas one of the main goals of the core area of Computer Vision is the analysis of visual presentations of information in the form of images or videos, here we are concerned with the opposite direction, namely the generation of visual presentations of information. Information can be presented in different visual forms, for example in the form of 3D scenes or in the form of videos. But also the visual display of data in the classic form of 2D images is still a vivid field of research.

One purpose of image systhesis can be the creation of a special visual experience. Another reason consists in data analysis, at which the visual presentation should allow for the recognition of patterns or relations between the data by the human eye. The major domain of scene synthesis, on the other hand, is interaction. To interact with the world Interactive Systems need, at least to some extent, internal representations of the world. To this end, this field of research aims at capturing the real world and transforming it into a virtual world.

There are two main reasons for a sythesis of scenes. First, 3D scene reconstruction provides the basis for a system to interact with its environment. As such it serves as an auxiliary technology for scene understanding. In this function scene reconstruction constitutes the foundation for key elements of an interactive system such as navigation, or localization and manipulation of objects in the environment. The second reason for image or scene synthesis applies to the field of interface design. Here the synthesis of visual information is important from the users' points of view who interact with a system. As interfaces are supposed to ensure a safe and intuitive interaction, questions of information visualization and aesthetics play an important role (see also core area New Media Art). The visual presentation of information to humans comprises the synthesis of 3D objects in a virtual world as well as the generation of 2D visual information in the form of images or videos.

 

Major Challenges

One of the most important technologies in the future will be 3D reconstruction techniques, which map the real world to a virtual 3D world. Thereby, also in this field of research a major challenge consists again in the handling of huge data sets and appropriate ways of processing them. Future research areas are represented by questions concerning an intelligent acquisition of the real world, appropriate parametrizations of 3D scenes, and object descriptions suitable for efficient synthesis, display, and manipulation. Another huge, interdisciplinary, and to a larger extend unexplored field concerns questions of pleasing and aesthetic qualities of the visual display of data in interactive systems. The visual appearance of virtual worlds still too often reflects the conceptions of system developers rather than meeting the aesthetic sense of their prospective users or taking knowledge of design experts into account. Here a more intense collaboration between computer scientists and artists or designers seems necessary.

 

Our Expertise

Many of our activities in this field of research are concerned with the topic of 3D reconstruction. Our students are repeatedly successful on this topic receiving awards or nominations for their theses on object acquisition or scene reconstruction.
In collaboration with the California Institute of Technology we were engaged in the question how to represent the surrounding world on a 2D interface in a most realistic way. This resulted in a technique for generating panorama photographs without the usual distortions.
In another project, for instance, we developed a virtual camera operator for the production of video streams of scenes with moving people. It autonomously zooms, pans, and chooses shots inspired by cinematographic techniques and thus produces a video stream that appears interesting and keeps the attention of the observers focused on the content of the video.

 

Images: Human-Computer Interaction

Downloads: