Method and apparatus for the detection and labeling of features of an environment through contextual clues

Patent metadata

Field	Value
Publication number	US-11531348-B2
Application number	US-201816229389-A
Country	US
Kind code	B2
Filing date	Dec 21, 2018
Priority date	Dec 21, 2018
Publication date	Dec 20, 2022
Grant date	Dec 20, 2022

Abstract

Described herein are methods of detecting and labeling features within an image of an environment. Methods may include: receiving sensor data from an image sensor, where the sensor data is representative of a first image including an aerial view of a geographic region; detecting, using a perception module, at least one vehicle within the image of the geographic region; identifying an area around the at least one vehicle as a road segment in response to detecting the at least one vehicle; based on the identification of the area around the vehicle as a road segment, identifying features within the area as road features based on a context of the area; generating a map update for the road features of the road segment; and causing a map database to be updated with the road features of the road segment.

First claim

That which is claimed:

1. An apparatus to facilitate autonomous or semi-autonomous control of a vehicle comprising at least one processor and at least one non-transitory memory including computer program code instructions, the computer program code instructions configured to, when executed, cause the apparatus to at least:
receive sensor data from an image sensor, wherein the sensor data is representative of a first image including an aerial view of a geographic region;
detect, using a perception module, at least one vehicle within the image of the geographic region;
identify a location of the at least one vehicle;
determine, from map data, that a mapped road is within a predefined distance of the location of the at least one vehicle;
identify an area around the at least one vehicle as a road segment in response to detecting the at least one vehicle and the location of the at least one vehicle being within the predefined distance of the mapped road;
based on the identification of the area around the vehicle as a road segment, identify features within the area as road features based on a context of the area, wherein road features comprise one or more of road signs, lane lines, or road boundaries;
generate a map update for the road features of the road segment; and
cause a map database to be updated with the road features of the road segment.

2. The apparatus of claim 1 , wherein causing the apparatus to detect, using the perception module, at least one vehicle within the image of the geographic region comprises causing the apparatus to:
identify at least one object within the image of the environment as a vehicle in response to the at least one object corresponding to a learned template of a vehicle.

3. The apparatus of claim 1 , wherein the apparatus is further caused to:
provide for autonomous control of a vehicle based, at least in part, on the map update of the road features of the road segment.

4. The apparatus of claim 3 , wherein the road features of the road segment comprise information associated with driving restrictions along the road segment, wherein causing the apparatus to provide for autonomous control of the vehicle based, at least in part, on the map update comprises causing the apparatus to provide autonomous control of the vehicle along the road segment based on the driving restrictions.

5. The apparatus of claim 1 , wherein the perception module comprises an auto-encoder, wherein the auto-encoder is trained based on a plurality of manually identified vehicles.

6. The apparatus of claim 1 , wherein causing the apparatus to identify an area around the at least one vehicle as a road segment comprises causing the apparatus to:
apply a dilation algorithm to probe and expand the area around the at least one vehicle in identifying the area as a road segment; and
apply a spline-based curve fitting model to extract lane level geometry of the road segment.

7. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions to:
receive sensor data from an image sensor, wherein the sensor data is representative of a first image including an aerial view of a geographic region;
detect, using a perception module, at least one vehicle within the image of the geographic region;
identify a location of the at least one vehicle;
determine, from map data, that a mapped road is within a predefined distance of the location of the at least one vehicle;
identify an area around the at least one vehicle as a road segment in response to detecting the at least one vehicle and the location of the at least one vehicle being within the predefined distance of the mapped road;
based on the identification of the area around the vehicle as a road segment, identify features within the area as road features based on a context of the area, wherein road features comprise one or more of road signs, lane lines, or road boundaries;
generate a map update for the road features of the road segment; and
cause a map database to be updated with the road features of the road segment.

8. The computer program product of claim 7 , wherein the program code instructions to detect, using the perception module, at least one vehicle within the image of the geographic region comprises program code instructions to:
identify at least one object within the image of the environment as a vehicle in response to the at least one object corresponding to a learned template of a vehicle.

9. The computer program product of claim 7 , further comprising program code instructions to:
provide for autonomous control of a vehicle based, at least in part, on the map update of the road features of the road segment.

10. The computer program product of claim 9 , wherein the road features of the road segment comprise information associated with driving restrictions along the road segment, wherein the program code instructions to provide for autonomous control of the vehicle based, at least in part, on the map update comprises program code instructions to provide autonomous control of the vehicle along the road segment based on the driving restrictions.

11. The computer program product of claim 7 , wherein the perception module comprises an auto-encoder, wherein the auto-encoder is trained based on a plurality of manually identified vehicles.

12. The computer program product of claim 7 , wherein the program code instructions to identify an area around the at least one vehicle as a road segment comprises program code instructions to:
apply a dilation algorithm to probe and expand the area around the at least one vehicle in identifying the area as a road segment; and
apply a spline-based curve fitting model to extract lane level geometry of the road segment.

13. A method comprising:
receiving sensor data from an image sensor, wherein the sensor data is representative of a first image including an aerial view of a geographic region;
detecting, using a perception module, at least one vehicle within the image of the geographic region;
identifying a location of the at least one vehicle;
determining, from map data, that a mapped road is within a predefined distance of the location of the at least one vehicle;
identifying an area around the at least one vehicle as a road segment in response to detecting the at least one vehicle and the location of the at least one vehicle being within the predefined distance of the mapped road;
based on the identification of the area around the vehicle as a road segment, identifying features within the area as road features based on a context of the area, wherein road features comprise one or more of road signs, lane lines, or road boundaries;
generating a map update for the road features of the road segment; and
causing a map database to be updated with the road features of the road segment.

14. The method of claim 13 , wherein detecting, using the perception module, at least one vehicle within the image of the geographic region comprises:
identifying at least one object within the image of the environment as a vehicle in response to the at least one object corresponding to a learned template of a vehicle.

15. The method of claim 13 , further comprising:
providing for autonomous control of a vehicle based, at least in part, on the map update of the road features of the road segment.

16. The method of claim 15 , wherein the road features of the road segment comprise information associated with drivin

Assignees

• Here Global Bv

Inventors

• Viswanathan Anirudh

Classifications

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• G05D1/0274

  using mapping information stored in a memory device (navigation using map-matching G01C21/30) · CPC title

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G01C21/3852

  Data derived from aerial or satellite images · CPC title

• G01C21/3822

  Road feature data, e.g. slope data · CPC title