KEYWORDS: Image segmentation, 3D modeling, Optical flow, 3D image processing, Clouds, Stereoscopic cameras, 3D acquisition, 3D image reconstruction, Data modeling, Optical tracking
This paper improves the authors' conventional method for reconstructing the 3D structure of moving and still objects
that are tracked in the video and/or depth image sequences acquired by moving cameras and/or range finder. The authors
proposed a Temporal Modified-RANSAC based method [1] that (1) can discriminate each moving object from the still
background in color image and depth image sequences acquired by moving stereo cameras or moving range finder, (2)
can compute the stereo cameras' egomotion, (3) can compute the motion of each moving object, and (4) can reconstruct
the 3D structure of each moving object and the background. However, the TMR based method has the following two
problems concerning the 3D reconstruction: lack of accuracy of segmenting into each object's region and sparse 3D
reconstructed points in each object's region. To solve these problems of our conventional method, this paper proposes a
new 3D segmentation method that utilizes Graph-cut, which is frequently used for segmentation tasks. First, the
proposed method tracks feature points in the color and depth image sequences so that 3D optical flows of the feature
points in every N frames are obtained. Then, TMR classifies all the obtained 3D optical flows into regions (3D flow set)
for the background and each moving object; simultaneously, the rotation matrix and the translation vector for each 3D
flow set are computed. Next, Graph-Cut using the energy function that consists of color probability, structure probability
and a-priori probability is performed so that pixels in each frame are segmented into object regions and the background
region. Finally, 3D point clouds are obtained from the segmentation result image and depth image, and then the point
clouds are merged using the rotation and translation from the N-th frame prior to the current frame so that 3D models for
the background and each moving object are constructed with dense 3D point data.
This paper proposes a Temporal Modified-RANSAC based method that can discriminate each moving object from the
still background in the stereo video sequences acquired by moving stereo cameras, can compute the stereo cameras'
egomotion, and can reconstruct the 3D structure of each moving object and the background. We compute 3D optical
flows from the depth map and results of tracking feature points. We define "3D flow region" as a set of connected pixels
whose 3D optical flows have a common rotation matrix and translation vector. Our Temporal Modified-RANSAC
segments the detected 3D optical flows into 3D flow regions and computes the rotation matrix and translation vector for
each 3D flow region. As opposed to the conventional Modified-RANSAC for only two frames, The Temporal Modified-
RANSAC can handle temporal images with arbitrary length by performing the Modified-RANSAC to the set of a 3D
flow region that classified in the latest frame and new 3D optical flows detected in the current frame iteratively. Finally,
the 3D points computed from the depth map in all the frames are registered using each 3D flow region's matrix to the
initial positions in the initial frame so that the 3D structures of the moving objects and still background are reconstructed.
Experiments using multiple moving objects and real stereo sequences demonstrate promising results of our proposed
method.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.