Method and apparatus for detecting objects of interest in an environment

Patent metadata

Field	Value
Publication number	US-11270162-B2
Application number	US-201816174866-A
Country	US
Kind code	B2
Filing date	Oct 30, 2018
Priority date	Oct 30, 2018
Publication date	Mar 8, 2022
Grant date	Mar 8, 2022

Abstract

A method is provided for generating training data to facilitate automatically locating an object of interest within an image. Methods may include: receiving sensor data including a plurality of images from at least one image sensor; receiving an identification, from a user, of an object visible within an image of the plurality of images, where at least a portion of the object is visible in one or more of the plurality of images; determining a predicted location of the object in the one or more of the remaining images of the plurality of images; identifying the object in the one or more of the remaining images of the plurality of images; and storing the plurality of images including an indication of the object at the object location within the one or more of the plurality of images.

First claim

That which is claimed:

1. An apparatus to facilitate autonomous or semi-autonomous control of a vehicle comprising at least one processor and at least one non-transitory memory including computer program code instructions, the computer program code instructions configured to, when executed, cause the apparatus to at least:
receive sensor data from at least one image sensor, wherein the sensor data is representative of a plurality of images, wherein each image is associated with sensor position data;
receive an identification, from a user, of an object visible within an image of the plurality of images, wherein at least a portion of the object is visible in one or more of the remaining images of the plurality of images;
determine a predicted location of the object in the one or more of the remaining images of the plurality of images based, at least in part, on the sensor position data of the respective image, identify the object in the one or more of the remaining images of the plurality of images based, at least in part, on the predicted location of the object in the one or more of the remaining images of the plurality of images and image feature analysis;
store the plurality of images including an indication of the object at an object location within the one or more of the plurality of images; and
identify an object in a subsequent image automatically based, at least in part, on machine learning from the plurality of stored images, wherein the machine learning comprises pre-computed training data, wherein the pre-computed training data having verified accuracy of the object in the plurality of stored images.

2. The apparatus of claim 1 , wherein the sensor position data comprises at least one of visual odometry or actual odometry from at least one sensor of the vehicle.

3. The apparatus of claim 2 , wherein causing the apparatus to identify the object in one or more of the remaining images of the plurality of images based, at least in part, on the predicted location of the object in the one or more of the remaining images of the plurality of images and image feature analysis comprises causing the apparatus to identify the object location in the one or more of the remaining images of the plurality of images based on the at least one of visual odometry or actual odometry from the at least one sensor of the vehicle.

4. The apparatus of claim 1 , wherein the apparatus is further caused to receive an indication of a correction, from the user, of the location of the object within at least one of the plurality of images relative to the predicted location of the object in the at least one of the plurality of images, wherein causing the apparatus to store the plurality of images including an indication of the object at the object location within the one or more of the plurality of images comprises causing the apparatus to store the at least one of the plurality of images including an indication of the object at a corrected location within the at least one of the plurality of images.

5. The apparatus of claim 1 , wherein the apparatus is further caused to facilitate autonomous control of the vehicle based on identification of the object in the subsequent image automatically based, at least in part, on the machine learning from the plurality of images.

6. The apparatus of claim 1 , wherein the at least one image sensor is associated with the vehicle traveling along a road segment, wherein the object comprises a road sign along the road segment.

7. The apparatus of claim 6 , wherein the road sign comprises information regarding travel restrictions along the road segment, wherein the apparatus is further caused to provide for autonomous control of the vehicle based, at least in part, on the information of the road sign.

8. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions to:
receive sensor data from at least one image sensor, wherein the sensor data is representative of a plurality of images, wherein each image is associated with sensor position data;
receive an identification, from a user, of an object visible within an image of the plurality of images, wherein at least a portion of the object is visible in one or more of the remaining images of the plurality of images;
determine a predicted location of the object in the one or more of the remaining images of the plurality of images based, at least in part, on the sensor position data of the respective image;
identify the object in the one or more of the remaining images of the plurality of images based, at least in part, on the predicted location of the object in the one or more of the remaining images of the plurality of images and image feature analysis;
store the plurality of images including an indication of the object at an object location within the one or more of the plurality of images; and
identify an object in a subsequent image automatically based, at least in part, on machine learning from the plurality of stored images, wherein the machine learning comprises a pre-computed training data, wherein the pre-computed training data having verified accuracy of the object in the plurality of stored images.

9. The computer program product of claim 8 , wherein the sensor position data comprises at least one of visual odometry or actual odometry from at least one sensor of the vehicle.

10. The computer program product of claim 9 , wherein the program code instructions to identify the object in one or more of the remaining images of the plurality of images based, at least in part, on the predicted location of the object in the one or more of the remaining images of the plurality of images and image feature analysis comprises program code instructions to identify the object location in the one or more of the remaining images of the plurality of images based on the at least one of visual odometry or actual odometry from the at least one sensor of the vehicle.

11. The computer program product of claim 8 , further comprising program code instructions to receive an indication of a correction, from the user, of the location of the object within at least one of the plurality of images relative to the predicted location of the object in the at least one of the plurality of images, wherein the program code instructions to store the plurality of images including an indication of the object at the object location within the one or more of the plurality of images comprises program code instructions to store the at least one of the plurality of images including an indication of the object at a corrected location within the at least one of the plurality of images.

12. The computer program product of claim 8 , further comprising program code instructions to facilitate autonomous control of the vehicle based on identification of the object in a subsequent image automatically based, at least in part, on the machine learning from the plurality of images.

13. The computer program product of claim 8 , wherein the at least one image sensor is associated with the vehicle traveling along a road segment, wherein the object comprises a road sign along the road segment.

14. The computer program product of claim 13 , wherein the road sign comprises information regarding travel restrictions along the road segment, wherein the computer program product further comprises program code instructions to provide for autonomous control of the vehicle based, at least in part, on the information of the road sign.

15. A

Assignees

• Here Global Bv

Inventors

• Viswanathan Anirudh

Classifications

• G06V10/7747

  Organisation of the process, e.g. bagging or boosting · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06F18/40

  Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor · CPC title

• G06F18/2178

  based on feedback of a supervisor · CPC title