Method and system for analyzing surroundings of an autonomous or semi-autonomous vehicle

Patent metadata

Field	Value
Publication number	US-11741709-B2
Application number	US-201917057086-A
Country	US
Kind code	B2
Filing date	May 20, 2019
Priority date	May 22, 2018
Publication date	Aug 29, 2023
Grant date	Aug 29, 2023

Abstract

Disclosed are a system and methods for operating an autonomous or semi-autonomous vehicle. One method comprises travelling in an outdoor setting, capturing data related to the outdoor setting, processing captured data and identifying occlusion present in the preprocessed data. The system comprises an autonomous or semi-autonomous vehicle configured to travel in outdoor settings and comprising at least one first sensor and at least one processing component. The processing component is configured to process data captured by the first sensor and identify occlusion present in the preprocessed data.

First claim

The invention claimed is:

1. A method for operating a semi-autonomous or autonomous vehicle, the method comprising:
the vehicle traveling in an outdoor setting;
capturing data related to the outdoor setting via at least one first sensor;
preprocessing captured data to identify areas of interest corresponding to at least one segment of the outdoor setting and to produce preprocessed data;
identifying occlusion present in the preprocessed data;
computing a distance range between the vehicle and the identified occlusion; and
discarding the identified occlusion if a minimum possible distance between the vehicle and the identified occlusion is above a predetermined threshold.

2. The method according to claim 1 , further comprising identifying a type of occlusion present in the preprocessed data.

3. The method according to claim 2 , further comprising executing an action, wherein said action is selected based on the type of occlusion.

4. The method according to claim 1 , further comprising contacting a remote operator terminal if the identified occlusion exceeds a predetermined parameter.

5. The method according to claim 4 , further comprising the remote operator terminal assuming control of the vehicle after being contacted.

6. The method according to claim 1 , wherein the at least one first sensor comprises at least one of a visual camera and a time-of-flight (ToF) sensor and the captured data comprises a plurality of frames captured by the at least one first sensor.

7. The method according to claim 1 , wherein traveling in the outdoor setting comprises the vehicle traveling travelling on a pedestrian pathway.

8. The method of claim 1 , wherein the vehicle comprises a mobile robot.

9. The method according to claim 1 , further comprising comparing the preprocessed data with stored map data indicative of the at least one segment of the outdoor setting to identify occlusion present in the preprocessed data.

10. The method according to claim 9 , wherein the stored map data comprises traffic road data and wherein the stored map data further comprises a road difficulty parameter for at least some roads.

11. The method according to claim 10 , wherein comparing the preprocessed data with the stored map data further comprises identifying traffic roads with a road difficulty parameter exceeding a particular threshold in the preprocessed data.

12. The method according to claim 9 , wherein comparing the preprocessed data with the stored map data comprises projecting the stored map data onto coordinates of the preprocessed data.

13. The method according to claim 12 ,
wherein the stored map data further comprises infinity points; and
wherein comparing the preprocessed data with the stored map data further comprises identifying a position of infinity points in the preprocessed data.

14. The method according to claim 1 , wherein preprocessing captured data comprises segmenting the captured data, and wherein segmenting comprises dividing the captured data into a plurality of different area types.

15. The method according to claim 1 , wherein identifying the occlusion comprises
assigning an occlusion score to each potential detected occlusion; and
comparing the occlusion score with a predetermined occlusion threshold and keeping occlusions with a score above the predetermined occlusion threshold.

16. The method according to claim 15 , further comprising running a neural network-based algorithm on a combination of the preprocessed data and stored map data, and wherein the neural network-based algorithm is run on occlusions with an occlusion score above the predetermined occlusion threshold.

17. A system for operating a semi-autonomous or autonomous vehicle, the system comprising:
a vehicle configured to travel in outdoor settings and comprising at least one first sensor configured to capture data related to the outdoor settings; and
at least one processing component configured to:
preprocess data captured by the at least one first sensor to identify areas of interest corresponding to at least one section of an outdoor setting, and to produce preprocessed data; and
identify occlusion present in the preprocessed data;
compute a distance range between the vehicle and the identified occlusion; and
discard the identified occlusion if a minimum possible distance between the vehicle and the identified occlusion is above a predetermined threshold.

18. The system according to claim 17 further comprising a memory component configured to store map data indicative of the outdoor settings; and
wherein the processing component is further configured to compare the preprocessed data with the stored map data.

19. The system according to claim 17 , further comprising a remote operator terminal, and wherein the remote operator terminal is configured to assume control of the vehicle when contacted upon detecting an occlusion exceeding a certain predetermined threshold.

20. A method for operating a semi-autonomous or autonomous vehicle, the method comprising:
traveling in an outdoor setting; and
capturing data related to the outdoor setting via at least one first sensor;
preprocessing captured data to identify areas of interest corresponding to at least one segment of the outdoor setting and to produce preprocessed data; and
identifying occlusion present in the preprocessed data; and
comparing the preprocessed data with stored map data indicative of the at least one segment of the outdoor setting to identify the occlusion present in the preprocessed data,
wherein comparing the preprocessed data with the stored map data comprises projecting the stored map data onto coordinates of the preprocessed data,
wherein the stored map data further comprises infinity points; and
wherein comparing the preprocessed data with the stored map data further comprises identifying a position of infinity points in the preprocessed data.

21. The method according to claim 20 , wherein preprocessing captured data comprises segmenting the captured data, and wherein segmenting comprises dividing the captured data into a plurality of different area types.

22. The method according to claim 20 , wherein the stored map data comprises traffic road data and wherein the stored map data further comprises a road difficulty parameter for each road.

23. The method according to claim 22 , wherein comparing the preprocessed data with the stored map data further comprises identifying traffic roads with a road difficulty parameter exceeding a particular threshold in the preprocessed data.

24. The method according to claim 20 , wherein identifying the occlusion comprises
assigning an occlusion score to each potential detected occlusion; and
comparing the occlusion score with a predetermined occlusion threshold and keeping occlusions with a score above the predetermined occlusion threshold.

25. The method according to claim 24 , further comprising running a neural network-based algorithm on a combination of the preprocessed data and stored map data, and wherein the neural network-based algorithm is run on occlusions with an occlusion score above the predetermined occlusion threshold.

26. The method according to claim 20 , further comprising:
computing a distance range between the vehicle and the identified occlusion; and
discarding the identified occlusion if a minimum possible distance between the vehicle and the identified occlusion is above

Assignees

• Starship Tech Oue

Inventors

• Korjus Kristjan
• Kharagorgiiev Sergii
• Heinla Ahti
• Pärnamaa Tanel

Classifications

• G06V20/38 Primary

  Outdoor scenes · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title