Image processing apparatus for estimating three-dimensional position of object and method therefor

Patent metadata

Field	Value
Publication number	US-9877012-B2
Application number	US-201615084381-A
Country	US
Kind code	B2
Filing date	Mar 29, 2016
Priority date	Apr 1, 2015
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

An image processing apparatus includes a holding unit configured to hold a positional relationship of a plurality of image capturing units, an acquisition unit configured to detect objects from respective images captured by the plurality of image capturing units and acquire positions of the objects on the captured images and geometric attributes of the objects, an associating unit configured to associate the detected objects, based on the positional relationship, the positions and the geometric attributes, and an estimation unit configured to estimate a three-dimensional position of the objects based on the positional relationship and the positions of the detected objects.

First claim

What is claimed is:

1. An image processing apparatus comprising:
a holding unit configured to hold a positional relationship of a plurality of image capturing units;
an acquisition unit configured to detect objects from respective images captured by the plurality of image capturing units and acquire positions of the objects on the captured images and geometric attributes of the objects;
a selection unit configured to select a combination of objects to be associated, based on the positional relationship and the positions of the objects on the respective captured images;
a determination unit configured to determine a consistency of the selected combination based on a degree of similarity between the geometric attributes of the respective objects belonging to the combination;

an associating unit configured to associate the objects in the combination based on the consistency of the combination; and
an estimation unit configured to estimate a three-dimensional position of the objects based on the positional relationship and the positions of the detected objects.

2. The image processing apparatus according to claim 1 , wherein the determination unit calculates a degree of similarity between a geometric attribute of an object detected from an image captured by an image capturing unit of interest in the plurality of image capturing units and a geometric attribute of a projection of an object detected from an image captured by another one of the plurality of image capturing units onto the image captured by the image capturing unit of interest, and the determination unit determines the consistency of the combination based on the degree of similarity.

3. The image processing apparatus according to claim 1 , wherein the determination unit calculates a degree of consistency of the three-dimensional position.

4. The image processing apparatus according to claim 3 , wherein based on an assumption that the objects exist in the estimated three-dimensional position, the determination unit calculates a degree of similarity between the geometric attributes of the objects that are acquired from the respective captured images, and a degree of consistency of the three-dimensional position based on the calculated degree of similarity.

5. The image processing apparatus according to claim 3 , wherein the associating unit associates a combination having a high degree of consistency.

6. The image processing apparatus according to claim 3 , wherein the associating unit eliminates a combination having a low degree of consistency.

7. The image processing apparatus according to claim 1 , wherein based on the positional relationship, the selection unit obtains a straight line on a captured image of a first image capturing unit that connects a second image capturing unit to an object detected at the second image capturing unit, and the selection unit selects a combination of an object having a distance from the straight line that is equal to or smaller than a threshold value among objects detected at the first image capturing unit and the object detected at the second image capturing unit.

8. The image processing apparatus according to claim 1 , wherein the geometric attribute is an object size.

9. The image processing apparatus according to claim 1 , wherein the geometric attribute is an object direction.

10. The image processing apparatus according to claim 1 , wherein the geometric attribute is an object motion.

11. The image processing apparatus according to claim 1 , wherein the plurality of image capturing units includes a first image capturing unit, a second image capturing unit, and a third image capturing unit,
wherein the estimation unit estimates the three-dimensional position of the objects based on positions of the objects on respective captured images of the first image capturing unit and the second image capturing unit, and
wherein the image processing apparatus further comprises a complementation unit configured to complement a detection failure by the acquisition unit by projecting a geometric attribute of the object acquired from the captured image of the first image capturing unit or the second image capturing unit onto a captured image of the third image capturing unit, based on an assumption that the objects exist in the estimated three-dimensional position.

12. The image processing apparatus according to claim 1 , further comprising an image display unit configured to superimpose on the captured image a symbol indicating the position on the acquired captured image.

13. An image processing apparatus comprising:
a holding unit configured to hold a positional relationship of a plurality of image capturing units;
an acquisition unit configured to detect objects from respective images captured by the plurality of image capturing units and acquire positions of the objects on the captured images and geometric attributes of the objects;
an associating unit configured to associate the detected objects, based on the positional relationship, the positions, and the geometric attributes;
an estimation unit configured to estimate a three-dimensional position of the objects based on the positional relationship and the positions of the detected objects; and

a map display unit configured to superimpose on a two-dimensional or three-dimensional map a symbol indicating the estimated three-dimensional position of the objects.

14. An image processing method comprising:
detecting objects from respective images captured by a plurality of image capturing units and acquiring positions of the objects on the captured images and geometric attributes of the objects;
selecting a combination of objects to be associated, based on a positional relationship of the plurality of image capturing units that is held in a holding unit and the positions of the objects on the respective captured images;
determining a consistency of the selected combination based on a degree of similarity between the geometric attributes of the respective objects belonging to the combination;

associating the objects in the combination based on consistency of the combination; and
estimating a three-dimensional position of the objects based on the positional relationship and the positions of the associated objects on the respective captured images.

15. A non-transitory computer-readable storage medium storing a program that causes a computer to function as:
a holding unit configured to hold a positional relationship of a plurality of image capturing units;
an acquisition unit configured to detect objects from respective images captured by the plurality of image capturing units and acquire positions of the objects on the captured images and geometric attributes of the objects;
a selection unit configured to select a combination of objects to be associated, based on the positional relationship and the positions of the objects on the respective captured images;
a determination unit configured to determine a consistency of the selected combination based on a degree of similarity between the geometric attributes of the respective objects belonging to the combination;

an associating unit configured to associate the objects in the combination based on the consistency of the combination; and
an estimation unit configured to estimate a three-dimensional position of the objects based on the positional relationship and the positions of the objects associated by the associating unit on the respective captured images.

16. An image processing apparatus comprising:
an acquisition unit configured to acquire a group of frame images captured consecutively from ea

Assignees

• Canon Kk

Inventors

• Uchiyama Hiroyuki
• Guo Muling
• Yano Kotaro
• Umeda Ichiro

Classifications

• G06T7/55

  from multiple images · CPC title

• H04N13/0468

  Electricity · mapped topic

• H04N13/0282 Primary

  Electricity · mapped topic

• G06T7/593

  from stereo images · CPC title

• G06T2207/20088

  Trinocular vision calculations; trifocal tensor · CPC title