Progress of the 3DTV Project: Some Intermediate Results

Progress of the 3DTV Project: Some Intermediate Results

It is already over 20 months since the 3DTV project has started. This 19 partner NoE has been focusing on all aspects of 3DTV, from capturing 3D scenes to displaying them and everything in between. Furthermore, applications of 3D video technology to many different areas are also investigated. The multidisciplinary nature of the topic, and the breadth of the scope resulted in a large number of results each in a different aspect of the 3DTV, as listed below. Naturally, the focus of the consortium is the 3DTV and developements in functional components described below are essential for a successful final integration. Such integration is among the plans of the consortium during the next phase of operations.

Here are the highlights of some intermediate results:

Capture:

Many different candidate technologies are comparatively assessed. It is concluded that multiple synchoronized video recordings are currently the most promising technology. Other alternatives like single camera techniques, pattern projection based approaches and holographic cameras are also under investigation.
A robot equipped with a laser scanner and an omnidirectional camera captures 3D structure of the environements as it travels. The system is based on a stereo technique and calculates dense depth fields.
Many experimental multi-camera capture systems are designed and tested. Synchronization among the cameras is achieved. 3D models of fire and smoke are developed and reconstructed.
Many techniques are developed to generate automated 3D personalized human avatars from multi-camera video input.
Image-based methods are developed for for surface reconstruction of moving garment from multiple calibrated video cameras.
A method, based on synthetic aperture radar techniques is developed to increase the resolution of CCD based holographic recording.
Algorithms are developed for denoising interference patterns to improve the accuracy of reconstructed 3D scenes from holographic recordings.
Signal processing methods are developed for automated detection of face, facial parts, facial features and facial motion in recorded video.
A method for generating and animating a 3D model of a human face is developed.
A method to track gesture features and speech-correlated gesture analysis is develolped.
Representation:

A technique for interactive view-dependent streaming of progressively encoded 3D models over lossy networks is developed. The merthod is based on progressive octree representation.
A method to represent 3D objects using multiresolution tetrahedral meshes is developed.
An agorithm for resonstructing a 3D environment from images recorded by multiple calibrated cameras is developed.
A novel approach for digitizing archeological excavation sites in 3D from multiple views are developed.
Comparative quality assessments for different 3D reconstruction methods are conducted.
Methods for filtering to eliminate the jitter in captured motion data are developed.
A technique is developed to recognize head and hand gestures; the method is then used to synthesize speech-synchoronized gestures.
A method for representing scalable 3D image-based video objects is developed.
Objective quality assessment methods for 3D video objects are developed.
Software tools for easy description of 3D video objects are developed.
Different 3D video synthesis methods are evaluated.
Coding and Compression:

A technique to automatically segment stereo 3D video sequences is developed.
A method for optimal rate and input format control for content and context adaptive streaming video is developed.
An approach for minimum delay content adaptive video streaming over variable bit-rate channels is developed.
Rate control techniques for 3D video streaming are developed.
A full end-to-end multi-view video codec is implemented and tested.
A comparative study for investigating the effects of different disparity maps and their properties in an embedded JPEG2000 based disparity compansated stereo image coder is completed.
The effects of various lossy coding techniques on stereo images are investigated.
A storage format for 3D video is developed.
Statistical properties of spatio-temporal prediction for multi-view video coding are evaluated.
NoE partners actively participated in MPEG stgandardization activities for 3D video.
Techniques for coding multi-view video are developed.
A proposal submitted to MPEG for multiview video coding by a Partner of our project was performed best in subjective tests among eight other proposals from different parts of the world.
Available MPEG tools are evaluated for multi-view video synthesis.
Multi-view test data sets using arrays of eight camera have been produced and made available to MPEG and general scienific community.
Various 3D mesh compression techniques are developed and tested.
Methods for coding and rendering free-view point video are developed.
Representation for 3D scenes for interactive applications is investigated.
Compression techniques for holograms are developed.
Multiple description coding techniques for 3D are investigated.
Watermarking techniques for 3D are developed.
Transmission:

An optimal cross-layer scheduling for video streaming is developed.
An optimal streaming strategy under rate and quality constraints are developed.
Different approaches for error concealment in stereoscopic images are developed and compared.
Color and depth representation based end-to-end 3DTV is further developed and tested.
Effects of noisy channels on 3D video transmission are investigated.
Application of turbo codes to 3D are investigated.
Signal Processing Issues in Difraction and Holography:

Diffraction and holography is revisited from a signal processing point of view.
Analytical solutions for complex coherent light field generation by a deflectable mirror array device are developed.
Methods to compress holographic data are developed.
Fast methods to compute diffraction between tilted planes are developed and tested.
Algorithms to compute 3D optical fields from data distributed over 3D space are developed and tested.
Phase-retreival methods for 3D measurements are investigated.
Display:

Autostereoscopic displays for 3DTV are further developed.
Viewer tracking autostereoscopic displays are further developed.
Characterization and calibration techniques for various spatial light modulator based holographic displays are developed.
Polymer dispersed liquid crystals are investigated for display applications.
Switchable materials are investigated for dynamic holographic displays.
Laser scanning

Go back to the web show-case