A Probabilistic Framework for Object Search w/ 6-DOF Pose Estimation
International Journal of Robotics Research, 2010 (submitted)
Dr. Jeremy Ma, Jet Propulsion Laboratory
Dr. Timothy Chung, Naval Postgraduate School
Dr. Joel Burdick, California Institute of Technology
|
|
|
Robustness to Varying Objects and Elevations
|
|
The following videos illustrate successful runs with objects of various shapes and colors and at various elevations.
| Object |
Video Link (right click on image and save as) |
Description |
Size |
 |
 |
Object placed ahead of robot on
box at level elevation with robot height. This is a
straightforward application of the proposed
algorithm. More difficult cases will be shown in the
videos below.
Shown is the grid-based
probability map generated from initial scan of room,
the planned path in blue, the actual path in
magenta, and the classified stereo points (red for
obstacle, green for ground plane) superimposed. Not
shown is the costmap populated from classified
stereo for obstacle avoidance. |
11.7 MB |
 |
 |
Object placed to the front left
of robot on box at high elevation. Note that as the
robot applies the local-search method for 6DOF pose
estimation, it goes through a predefined sequence of
search patterns covering a search hemisphere. If
there exist enough feature correspondences with some
area of the scene, the search sequence on the
hemisphere is abandoned and the robot centers the
camera on the detected object origin.
Shown
is the grid-based probability map generated from
initial scan of room, the planned path in blue, the
actual path in magenta, and the classified stereo
points (red for obstacle, green for ground plane)
superimposed. Not shown is the costmap populated
from classified stereo for obstacle
avoidance. |
9.4 MB |
 |
 |
Object placed to the front left
of robot on box at level elevation with the robot. In
this example, the object was trained in the upright
position yet due to the chosen features (SIFT), the
object is still detectable in other
orientations.
Shown is the grid-based
probability map generated from initial scan of room,
the planned path in blue, the actual path in
magenta, and the classified stereo points (red for
obstacle, green for ground plane) superimposed. Not
shown is the costmap populated from classified
stereo for obstacle avoidance. |
16.1 MB |
 |
 |
In this experiment, the object
is placed to the far left of robot on box at level
elevation with the robot. Note that the local search
with 6-DOF pose estimation struggles to localize the
object. However, after several attempts and
autonomous re-poisitioning of the camera (via
pan-tilt capabilities), the object is properly
localized.
Shown is the grid-based
probability map generated from initial scan of room,
the planned path in blue, the actual path in
magenta, and the classified stereo points (red for
obstacle, green for ground plane) superimposed. Not
shown is the costmap populated from classified
stereo for obstacle avoidance. |
12.8 MB |
 |
 |
The object is placed to the far
left of robot on a box at a high elevation. The
robot is able to find the object quite easily,
primarily because of the strong color signature in
the color histogram of the yellow mustard
bottle. 6-DOF localization via the local search
method is shown as well.
Shown is the
grid-based probability map generated from initial
scan of room, the planned path in blue, the actual
path in magenta, and the classified stereo points
(red for obstacle, green for ground plane)
superimposed. Not shown is the costmap populated
from classified stereo for obstacle
avoidance. |
13.3 MB |
 |
 |
The object is placed to the front
of the robot at level elevation with the height of
the robot. The robot is able to find the object
quite easily, primarily because of the strong color
signature in the color histogram of the orange
clock. 6-DOF localization via the local search
method is shown as well.
Shown is the
grid-based probability map generated from initial
scan of room, the planned path in blue, the actual
path in magenta, and the classified stereo points
(red for obstacle, green for ground plane)
superimposed. Not shown is the costmap populated
from classified stereo for obstacle
avoidance. |
15.1 MB |
 |
 |
The object is placed to the front
right of the robot at level elevation with the
height of the robot. The robot is able to find the
object quite easily, primarily because of the strong
color signature in the color histogram of the yellow
lunch box. 6-DOF localization via the local
search method is shown as well.
Shown is the
grid-based probability map generated from initial
scan of room, the planned path in blue, the actual
path in magenta, and the classified stereo points
(red for obstacle, green for ground plane)
superimposed. Not shown is the costmap populated
from classified stereo for obstacle
avoidance. |
12.0 MB |
|
|
Robustness to Replanning (after failed object detection/search)
|
|
These are some videos of experiments where the robot
fails to find the object on the first attempt, yet
replans to the next likely location (using the
probability map) and successfully detects and localizes
the object.
| Object |
Video Link (right click on image and save as) |
Description |
Size |
 |
 |
In this experiment, the robot
initially investigates an area of the environment
that contains an object with a color histogram
distribution that matches that of the desired
object. Upon failing to detect the object in that
cell location, the robot successfully replans to the
next probable location and finds the object
there.
Shown is the grid-based probability
map generated from initial scan of room, the planned
path in blue, the actual path in magenta, and the
classified stereo points (red for obstacle, green
for ground plane) superimposed. Not shown is the
costmap populated from classified stereo for
obstacle avoidance. |
21.6 MB |
 |
 |
In this experiment, the initial
setup was an obstacle avoidance test. However, the
robot detects the fire hydrant on the wall to have a
color distribution which matches that of the desired
object (Bob's Big Boy model) and goes there
first. Upon failing to detect the object in that
cell location, the robot successfully replans to the
next probable location and finds the object
there.
Shown is the grid-based probability
map generated from initial scan of room, the planned
path in blue, the actual path in magenta, and the
classified stereo points (red for obstacle, green
for ground plane) superimposed. Not shown is the
costmap populated from classified stereo for
obstacle avoidance. |
31.0 MB |
|
|
Robustness to Obstacle Avoidance (costmap generation)
|
|
The following videos illustrate successful runs,
illustrating costmap generation using stereo. Keep in
mind that most objects are placed on pedestals or boxes
that often get detected as obstacles. In order to allow
the robot to plan a path to a goal location that may
contain an obstacle, the A* planner
suppresses the cost of cells in a neighborhood about the
goal location.
| Object |
Video Link (right click on image and save as) |
Description |
Size |
 |
 |
In this experiment, an obstacle
(white box) is placed directly in front of the
robot, with the object to be detected placed just
beyond the obstacle. This setup requires the robot
to maneuver around the obstacle to successfully
detect the object.
Shown in the video is the
grid-based costmap (10cm x 10cm resolution)
generated by using full stereo classified as either
ground-plane or not
ground-plane. |
22.5 MB |
 |
 |
In this experiment, the object
(Raisin-Bran cereal box) is placed to the far left
of the robot, with the object to be detected placed
on a stool at level elevation with the robot
height. Note that due to the reflectivity of the
waxed floor, false stereo returns apear at the
fringe of the stereo detection range. Nonetheless,
the wall and chair are picked up as obstacles
successfully.
Shown in the video is the
grid-based costmap (10cm x 10cm resolution)
generated by using full stereo classified as either
ground-plane or not
ground-plane. |
21.8 MB |
|