Systems and methods of detecting moving obstacles

Patent metadata

Field	Value
Publication number	US-11436839-B2
Application number	US-201816179223-A
Country	US
Kind code	B2
Filing date	Nov 2, 2018
Priority date	Nov 2, 2018
Publication date	Sep 6, 2022
Grant date	Sep 6, 2022

Abstract

The present disclosure provides systems and methods to detect occluded objects using shadow information to anticipate moving obstacles that are occluded behind a corner or other obstacle. The system may perform a dynamic threshold analysis on enhanced images allowing the detection of even weakly visible shadows. The system may classify an image sequence as either “dynamic” or “static”, enabling an autonomous vehicle, or other moving platform, to react and respond to a moving, yet occluded object by slowing down or stopping.

First claim

What is claimed is:

1. A method for detecting an unseen object by a vehicle comprising:
capturing a sequence of images over a period with an image capture device;
storing the sequence in a cyclic buffer;
registering each image on the cyclic buffer to a projected image;
enhancing the projected image;
identifying a shadow of a non-line-of-sight object in the projected image;
comparing a sum of pixel values in the projected image to a threshold value to determine movement of the shadow;
classifying into a scene determination the projected image based on the movement of the shadow; and
issuing a control signal to the vehicle upon classifying the projected image.

2. The method of claim 1 wherein classifying the projected image comprises classifying the image as a dynamic scene.

3. The method of claim 1 wherein classifying the projected image comprises determining a probability of the scene as one of a static scene or a dynamic scene.

4. The method of claim 1 wherein the step of enhancing the projected image comprises color amplification.

5. The method of claim 1 wherein the step of enhancing the projected image further comprises resizing the projected image.

6. The method of claim 1 wherein the step of enhancing the projected image further comprises down-sampling the projected image.

7. The method of claim 1 wherein the step of enhancing the projected image further comprises computing a mean image.

8. The method of claim 7 wherein the step of enhancing the projected image further comprises subtracting the mean image from each image in the sequence.

9. The method of claim 1 wherein the step of enhancing the projected image further comprises applying a Gaussian blur.

10. The method of claim 1 further comprising determining a region of interest from the projected image.

11. The method of claim 1 further comprising applying a temporal low-pass filter on the enhanced projected image.

12. The method of claim 1 wherein the control signal comprises a command to halt a trajectory of the vehicle.

13. The method of claim 1 wherein the pixel values of the projected image are indicative of a change in illumination in the sequence of images.

14. The method of claim 1 wherein the classification of the projected image comprises determining a speed of an unseen object.

15. The method of claim 1 wherein the classification of the projected image comprises determining a direction of an unseen object.

16. The method of claim 1 wherein the classification of the projected image comprises determining a quantity of unseen objects.

17. The method of claim 1 wherein the classification of the projected image comprises determining a size of an unseen object.

18. A system for detecting an unseen object comprising:
an image capture device configured to capture a sequence of images of a period;
a cyclic buffer configured to receive the sequence of images;
a processor configured to:
register each of the images stored in the cyclic buffer into a projected image;
enhance the projected image;
identify a shadow of a non-line-of-sight object in the projected image; and
compare a sum of pixel values in the projected image to a threshold value to determine movement of the shadow;
classify the projected image as one of a static image or a dynamic image based on the movement of the shadow; and

a vehicle interface in communication with a vehicle control, the vehicle interface configured to transmit a control signal to the vehicle control upon the classification of the projected image as a dynamic image.

19. A non-transitory computer-readable medium having program code recorded thereon for controlling a robotic device, the program code executed by a processor and comprising:
program code to capture a sequence of images over a period with an image capture device;
program code to store the sequence in a cyclic buffer;
program code to register each image on the cyclic buffer to a projected image;
program code to enhance the projected image;
program code to identify a shadow of a non-line-of-sight object in the projected image;
program code to compare a sum of pixel values in the projected image to a threshold value to determine movement of the shadow;
program code to classify into a scene determination the projected image based on the movement of the shadow; and
program code to issue a control signal to the vehicle upon classifying the projected image.

Assignees

• Toyota Res Inst Inc
• Massachusetts Institue Of Tech

Inventors

• Naser Felix Maximilian
• Gilitschenski Igor
• Rosman Guy
• Amini Alexander Andre
• Durand Fredo
• Torralba Antonio
• Wornell Gregory
• Freeman William
• Karaman Sertac
• Rus Daniela

Classifications

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06T5/003

  Physics · mapped topic

• G06T5/009

  Physics · mapped topic

• G06T5/73

  Deblurring; Sharpening · CPC title