Mucha system for acquisition and processing of the lightfield representation

Today, there is a great interest in fuller recording of the surrounding world than a flat 2D image. Many leading research centres and industrial companies are developing techniques of recording, processing and displaying of three-dimensional image. 3D cameras and 3D monitors have been successfully deployed on the market.

However, the ubiquitous 3D technology is based on the registration, processing and displaying of stereoscopic images. The stereoscopic pair allows to convey depth perception to the viewer, but only from one, very specific viewpoint, in which a pair of “digital eyes” of the camera were located. Stereoscopic technique is just a substitute for full registration of 3D information (3D light field) about the scene.

Companies are already developing more advanced technology for displaying 3D/holographic images, Companies such as Holografika, Ostendo or Dimenco [1, 2] developed and introduced to the market a new generation of ultra-realistic Super Multi View (SMV) 3D monitors, that allow observation of the scene like “through the window”. The complex system of lenticular lenses or mirrors and projectors reconstructs all the information about the captures light wave. This allows the viewer to freely moves in front of the monitor and enjoying a ultra-realistic three dimensional image from any place in the room without the need to wear any special glasses or other devices. In addition, when moving, the viewer observes the image with changing perspective, which gives the impression of a scene “suspended” at front of and in the monitor. Such a monitors reconstruct the full parallax of the scene, which allow observation of objects almost from both sides.

Great progress in the techniques of ultra-realistic 3D video displaying has been also recognized by the biggest standards organizations – the International Organization for Standardization (ISO) and International Telecommunication Union (ITU), and they began the works on the development of appropriate standards. For this purpose, a Motion Picture Expert Group (MPEG) of ISO has established a new working group named MPEG-I (MPEG Immersive), whose task is to develop the standards for the registration, processing and displaying of 3D light field video in the 3-4 years’.

Nevertheless, all the novel ultra-realistic 3D monitors that are present on the market today are experiencing the same one problem – the lack of appropriate content for displaying. Almost all currently available content for these commercially available monitors has been artificially computer generated. Other words, there is lack of appropriate 3D light field cameras that enable registration of high quality, wide angle, 3D light field video.
These observations and facts led to develop a system called MUCHA (FLY in English) for 3D light filed video capturing and processing.

3D light filed video camera allows for fuller, more complete, registration of the surrounding world than the classic (2D) camera. Due to it’s unique construction, 3D light filed video camera registers not only intensity of the incoming light in three subbands (RGB), but also its direction. This enables fuller registration of the surrounding world with the use of four dimensional space (position and direction of the incoming light ray) by capturing all information about light waves that reaches the camera. Such cameras are sometimes called holographic cameras (but to be able given light field camera a holographic camera it must be able to record a light filed with wide viewing angle).

Except typical parameters used to describe a classical cameras (image resolution – commonly expressed in MegaPixels and a frame rate – commonly expressed in frames per second, fps) light field cameras are also characterized by a number of registered simultaneously light rays (commonly in MegaRays).

Technical parameters

The proposed MUCHA system is characterized by the following parameters:

• number of registered light rays: 192 Mega Rays,

• angular resolution, number of captured light rays per pixel: 100 light rays per pixel,

• resolution of the reconstructed 2D image: 20 Mega Pixels with at least 300 depth layers,

• frame rate: 25 frames per second (fps),

• focus range 10 times wider in comparison with conventional camera with the same effective aperture size,

• 3D scene reconstruction accuracy: 1mm x 1mm x 1mm within the range from 10cm to 2m and 1cm x 1cm x 1cm within the range from 2m to 20m.

• the perspective change: 10 degrees for objects at 1 meter distance.