Map creation apparatus, map creation method, and computer-readable recording medium

Patent metadata

Field	Value
Publication number	US-9846043-B2
Application number	US-201514967761-A
Country	US
Kind code	B2
Filing date	Dec 14, 2015
Priority date	Jan 7, 2015
Publication date	Dec 19, 2017
Grant date	Dec 19, 2017

Abstract

A map creation apparatus includes an image receiver configured to receive images in time series while moving; a first calculator configured to extract an image area indicating an object from the images and calculate a coordinate of the object in a world coordinate system; a second calculator configured to track the object in the extracted image area with the images and calculate an optical flow of the object; an eliminator configured to calculate a difference in coordinate between vanishing points generated by the movement of the image receiver and the object based on the optical flow, and eliminate the image area of the object from the images when determining the object as a moving object based on the calculated difference; and a storage controller configured to store map information including the coordinate of the object, not eliminated by the moving object eliminator, in the world coordinate system.

First claim

What is claimed is:

1. A map creation apparatus comprising a processor and a non-transitory storage medium storing one or more programs of instructions that, when executed by the processor of the map creation apparatus, cause the map creation apparatus to perform a method comprising:
(a) receiving images in time series while the apparatus is moving;
(b) extracting an image area indicating an object from the images received in (a) and calculating a coordinate of the object in a world coordinate system;
(c) tracking the object in the extracted image area with the images input in sequence and calculating an optical flow of the object;
(d) calculating a difference between a coordinate of a first vanishing point generated by the movement of the apparatus and a coordinate of a second vanishing point generated by a movement of the object based on the optical flow calculated in (c) and eliminating the image area of the object from the images when the object is determined to be a moving object based on the calculated difference;
(e) storing map information including the coordinate of the object, not eliminated in (d), in the world coordinate system;
(f) determining that an object newly extracted is same as the object under tracking, and eliminating the newly-extracted object and stopping the tracking, when it is determined that the object newly extracted is same as the object under tracking; and
(g) calculating an error between (i) the coordinate of the object in a camera coordinate system that indicates a coordinate system based on the apparatus and (ii) a coordinate which is re-projected onto the camera coordinate system from a coordinate of the object in the world coordinate system in the map information; and deleting the map information of the object stored in (e) and updating the map information when the calculated error is not less than a preset threshold.

2. The map creation apparatus according to claim 1 , wherein the method performed by the apparatus further comprises:
extracting feature points of a plurality of objects from the images and calculating coordinates of the feature points of the plurality of objects in the world coordinate system from a disparity image input from plural image sources;
extracting feature points of the plurality of objects with the images input in sequence and calculating optical flows of the plurality of objects;
dividing the images into plural image areas for each image kind and integrating the optical flows as one group for each of the divided image areas;
calculating a coordinate of a vanishing point based on the optical flow for each of the divided image areas; and
integrating further the optical flows for each of the divided image areas whose distance between the coordinates of the vanishing points is not more than a preset threshold, determining the further-integrated optical flows as a moving object, and eliminating an image area of the object from the images.

3. The map creation apparatus according to claim 1 , wherein the method performed by the apparatus further comprises:
using (i) a speed at which the apparatus moves and (ii) a steering angle of a car wheel, to calculate movement information including a moving speed and a moving direction of the apparatus; and
obtaining a coordinate of the first vanishing point generated by the movement of the apparatus from the images input in sequence based on the movement information; and
determining a moving object and eliminating the image area of the object from the images based on a value of a calculated inner product between a vector in a direction connecting the coordinate of the object in the images input in sequence and the obtained coordinate of the first vanishing point and a movement vector of the optical flow.

4. The map creation apparatus according to claim 2 , wherein the method performed by the apparatus further comprises:
storing an obstacle-masked image indicating an image area of an obstacle; and
using a disparity obtained from the disparity image to calculate a distance image including an obstacle image indicating the image area of the obstacle, wherein
determining the obstacle image as a moving object and eliminating the image area of the object from the images when the obstacle image overlaps the obstacle-masked image and when the distance image and the obstacle image locate at not more than a preset distance.

5. The map creation apparatus according to claim 2 , wherein the method performed by the apparatus further comprises:
identifying an image area of a road surface from the images; and
determining an image area which is in contact with the identified image is of the road surface as a moving object, and eliminating the image area of the object from the images.

6. The map creation apparatus according to claim 1 , wherein the method performed by the apparatus further comprises:
using a random sample consensus (RANSAC) algorithm to approximate a feature point of a background area included in the images by a quadratic curve, determining the feature point away from the quadratic curve as a moving object, and eliminating an image area of an object corresponding to the feature point from the images.

7. A map creation method comprising:
receiving images in time series by an image receiver while the image receiver moves;
extracting an image area indicating an object from the images received at the receiving and calculating a coordinate of the object in a world coordinate system;
tracking the object in the extracted image area with the images input in sequence and calculating an optical flow of the object;
calculating a difference between a coordinate of a first vanishing point generated by the movement of the image receiver and a coordinate of a second vanishing point generated by a movement of the object based on the optical flow calculated at the tracking, and eliminating the image area of the object from the images when determining the object as a moving object based on the calculated difference;
storing map information including the coordinate of the object, not eliminated at the calculating, in the world coordinate system;
eliminating, when determining that an object newly extracted at the extracting is same as the object under tracking at the tracking, the newly-extracted object and stop the tracking; and
calculating an error between the coordinate of the object in a camera coordinate system that indicates a coordinate system based on the image receiver and a coordinate which is re-projected onto the camera coordinate system from a coordinate of the object in the world coordinate system in the map information, and deleting the stored map information of the object and updating the map information when the calculated error is not less than a preset threshold.

8. A non-transitory computer-readable recording medium with an executable program stored thereon and executed by a computer, wherein the program instructs the computer to perform:
receiving images in time series by an image receiver while the image receiver moves;
extracting an image area indicating an object from the images received at the receiving and calculating a coordinate of the object in a world coordinate system;
tracking the object in the extracted image area with the images input in sequence and calculating an optical flow of the object;
calculating a difference between a coordinate of a first vanishing point generated by the movement of the image receiver and a coordinate of a second vanishing point generated by a movement of the object based on the optical flow calculated at the tracking, and eliminating the image area of the object from the images when determining the object as a moving object based on the calculated difference;
storing map information inclu

Assignees

• Aoki Satoshi
• Sadasue Tamon
• Kajiwara Yasuhiro
• Takazawa Kazuhiro
• Matsushita Kazufumi
• Hashimoto yasuko
• Shirataka Yasuko
• Ricoh Co Ltd

Inventors

• Aoki Satoshi
• Sadasue Tamon
• Kajiwara Yasuhiro
• Takazawa Kazuhiro
• Matsushita Kazufumi
• Hashimoto yasuko
• Shirataka Yasuko

Classifications

• G01C21/26 Primary

  specially adapted for navigation in a road network · CPC title

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V10/507

  Summing image-intensity values; Histogram projection analysis · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06T7/73

  using feature-based methods · CPC title