Combining Shape from Shading and Stereo Depth Maps

The Package

This program is packaged in combine.tar.Z

Abstract

Stereo algorithms suffer from the lack of local surface texture due to smoothness of depth constraint, or local miss-matches in disparity estimates. Thus, the stereo methods only provide a coarse depth map which can be associated with a low pass image of the depth map. On the other hand, shape from shading algorithms produce better estimates of local surface areas, but some of them have problems with variable albedo and spherical surfaces. Thus, shape from shading methods produce better detailed depth information, and can be associated with the high pass image of the depth map image. In order to compute a better depth map, we present a method for integrating the high frequency information from the shape from shading and the low frequency information from stereo. In this paper, we present a method for integrating the high frequency information from the shape from shading, and the low frequency information from the stereo in order to produce a better depth map. Our method is motivated by the human vision system, and follows Hall and Hall's model. The proposed algorithm is very simple, takes about .7 seconds for a 128 x 128 image on a Sun SparcStation-1, is non-iterative, and does not use any thresholds. The results obtained with a variety of synthetic and real images are discussed. The quality of depth obtained by integrating shading and stereo is compared with the ground truth (range image) using average surface gradient error measure, and improvement ranging from 30% to 50% over stereo, and from 65% to 98% over shading is demonstrated.

What's In It?

 Program1 name : combine
  Input images :
           mozart-sh.rimg : depth map from shading
           mozart-st.rimg : depth map from stereo
       or
           bread-sh.rimg : depth map from shading
           bread-st.rimg : depth map from stereo

  Output :
           Depth.rimg : combined result
           L.rimg : Low freq. info of Hall's Model
           H.rimg : High freq. info. of Hall's Model
           combine-depth.ascii : ascii data file of the combined
                                 result (for 3-D plotting)

 Program2 name : lambert
  Input image : depth map (range image)
                Light source direction (any)

  Output : `lambert.iimg' Gray level image created using
            the lambertian reflectance model.

 Program3 name : disp3
            This will display the images with the
            extensions .rimg and .iimg to the screen of a
            SUN spark station.

 Program4 name : stereo
            Barnard's stereo algorithm.

The Technical Report

The algorithm is described in Technical Report tech25.ps.gz