Object recognition device and object recognition method

Patent metadata

Field	Value
Publication number	US-10482341-B2
Application number	US-201715704083-A
Country	US
Kind code	B2
Filing date	Sep 14, 2017
Priority date	Sep 29, 2016
Publication date	Nov 19, 2019
Grant date	Nov 19, 2019

Abstract

The object recognition device includes an imaging unit that captures images of a predetermined monitoring area to acquire a three-dimensional image and a two-dimensional image, an object extraction unit that extracts an area having pixels whose pixel values are within a predetermined range from the three-dimensional image acquired by the imaging unit, an image searching unit that searches the two-dimensional image, acquired by the imaging unit, for a reference image registered in advance according to the type of an object, and a determination unit that determines the type of the object depending on whether or not the reference image searched for by the image searching unit exists in the area extracted by the object extraction unit.

First claim

The invention claimed is:

1. An object recognition device comprising:
an imaging unit that captures images of a predetermined monitoring area to acquire a three-dimensional image and a two-dimensional image; and
a controller which has positional-relation-correspondence information between pixels of the three dimensional image and pixels of the two dimensional image, wherein the controller is configured to conduct:
an object extraction process that extracts, in the three dimensional image whose pixels respectively have their height information, an area having pixels whose pixel values are within a predetermined height range, and that determines a search area in the two-dimensional image so as to correspond to the extracted area based on the positional-relation-correspondence information;
an image searching process that searches the two-dimensional image, acquired by the imaging unit, for a reference image registered in advance according to a type of an object; and
a determination process that determines the type of the object depending on whether or not the reference image searched for in the image searching process exists in the search area determined in the two dimensional image; and
a control unit that controls a robot based on the determination process.

2. The object recognition device according to claim 1 , wherein the controller is configured to further conduct
a search area setting process that sets a search area to be searched in the image searching process so as to contain the area extracted in the object extraction process.

3. The object recognition device according to claim 1 , wherein
in the image searching process, the controller is configured to perform the search while changing scale of the reference image based on a distance, from the imaging unit, of each part in the three-dimensional image acquired by the imaging unit.

4. The object recognition device according to claim 1 , wherein
in the image searching process, the controller is configured to perform the search while deforming the reference image based on an inclination angle, relative to an optical axis of the imaging unit, of each part in the three-dimensional image acquired by the imaging unit.

5. An object recognition method comprising:
an imaging step of capturing images of a predetermined monitoring area to acquire a three-dimensional image and a two-dimensional image;
an object extraction step of extracting, in the three dimensional image whose pixels respectively have their height information, an area having pixels whose pixel values are within a predetermined height range and that determines a search area in the two-dimensional image so as to correspond to the extracted area based on positional-relation-correspondence information;
an image searching step of searching the two-dimensional image, acquired by the imaging step, for a reference image registered in advance according to a type of an object; and
a determination step of determining the type of the object depending on whether or not the reference image searched for by the image searching step exists in the search area determined in the two dimensional image; and
a control step of controlling a robot based on the determination process.

6. An object recognition device comprising:
an imaging unit that captures images of a predetermined monitoring area to acquire a three-dimensional image and a two-dimensional image; and
a controller which has positional-relation-correspondence information between pixels of the three dimensional image and pixels of the two dimensional image, wherein the controller is configured to conduct:
an object extraction process that binarizes and extracts, in the three dimensional image whose pixels respectively have their height information, an area having pixels whose pixel values are within a predetermined height range, and that determines a search area in the two-dimensional image so as to correspond to the extracted area based on the positional-relation-correspondence information;
an image searching process that searches the two-dimensional image, acquired by the imaging unit, for a reference image registered in advance according to a type of an object; and
a determination process that determines the type of the object depending on whether or not the reference image searched for in the image searching process exists in the search area determined in the two dimensional image; and
a control unit that controls a robot based on the determination process.

Assignees

• Fanuc Corp

Inventors

• Takahashi Yuuki
• Watanabe Atsushi
• Nakamura Minoru

Classifications

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06T7/74 Primary

  involving reference images or patches · CPC title

• G06V10/225

  based on a marking or identifier characterising the area · CPC title

• G06V10/32

  Normalisation of the pattern dimensions · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title