To the computer science researcher, multimedia consists of a wide variety of topic:
Multimedia processing and coding. This includes multimedia content analysis, content – based multimedia retrieval, multimedia security, audio / image / video processing, compression, and so on. Multimedia system support and networking.People look at such topics as network protocols, Internet, operating systems, servers and clients, quality of service (QoS), and databases. Multimedia tools, end systems, and applications. These include hypermedia systems, user interfaces, authoring systems, multimodal interaction, and integration: “ubiquity” — web – everywhere devices, multimedia education, including computer supported collaborative learning and design, and applications of virtual environments.
The concerns of multimedia researchers also impact researchers in almost every other branch of computer science. For example, data mining is an important current research area, and a large database of multimedia data objects is a good example of just what we may be interested in mining. Telemedicine applications, such as “telemedical patient consultative encounters,” are multimedia applications that place a heavy burden on existing network architectures.
Current Multimedia Projects Many exciting research projects are currently underway in multimedia, and we’d like to introduce a few of them here.
For example, researchers are interested in camera – based object tracking technology. One aim is to develop control systems for industrial control, gaming, and so on that rely on moving scale models (toys) around a real environment (a board game, say). Tracking the control objects (toys) provides user control of the process.
3D motion capture can also be used for multiple actor capture, so that multiple real actors in a virtual studio can be used to automatically produce realistic animated models with natural movement.
Multiple views from several cameras or from a single camera under differing lighting can accurately acquire data that gives both the shape and surface properties of materials, thus automatically generating synthetic graphics models. This allows photo – realistic (video – quality) synthesis of virtual actors.
3D capture technology is next to fast enough now to allow acquiring dynamic characteristics of human facial expression during speech, to synthesize highly realistic facial animation from speech.
Multimedia applications aimed at handicapped persons, particularly those with poor vision and the elderly, are a rich field of endeavor in current research.
“Digital fashion” aims to develop smart clothing that can communicate with other such enhanced clothing using wireless communication, so as to artificially enhance human interaction in a social setting. The vision here is to use technology to allow individuals to allow certain thoughts and feelings to be broadcast automatically, for exchange with others equipped with similar technology.
Georgia Tech’s Electronic Housecall system, an initiative for providing interactive health monitoring services to patients in their homes, relies on networks for delivery, challenging current capabilities.
Behavioral science models can be brought into play to model interaction between people, which can then be extended to enable natural interaction by virtual characters. Such “augmented interaction” applications can be used to develop interfaces between real and virtual humans for tasks such as augmented storytelling.
Each of these application areas pushes the development of computer science generally, stimulates new applications, and fascinates practitioners.