Processing sensor information for object detection

Patent metadata

Field	Value
Publication number	US-11443522-B2
Application number	US-201916701021-A
Country	US
Kind code	B2
Filing date	Dec 2, 2019
Priority date	Nov 30, 2018
Publication date	Sep 13, 2022
Grant date	Sep 13, 2022

Abstract

Methods of processing vehicle sensor information for object detection may include capturing generating a feature map based on captured sensor information, associating with each pixel of the feature map a prior box having a set of two or more width priors and a set of two or more height priors, determining a confidence value of each height prior and each width prior, outputting an indication of a detected object based on a highest confidence height prior and a highest confidence width prior, and performing a vehicle operation based on the output indication of a detected object. Embodiments may include determining for each pixel of the feature map one or more prior boxes having a center value, a size value, and a set of orientation priors, determining a confidence value for each orientation prior, and outputting an indication of the orientation of a detected object based on the highest confidence orientation.

First claim

What is claimed is:

1. A method of processing sensor information for object detection, comprising:
capturing, by a sensor, sensor information regarding an environment around the sensor;
generating, by a processor, a feature map based on the sensor information;
associating, by the processor, with each pixel of the feature map a prior box comprising:
a plurality of width priors that each include a width value,
a plurality of height priors that each include a height value, and
a plurality of orientation priors that each include an orientation value;

determining, by the processor, a confidence value of each height prior of the plurality of height priors, each width prior of the plurality of width priors, and each orientation prior of the plurality of orientation priors;
outputting, by the processor, an indication of a detected object based on a highest confidence height prior from among the plurality of height priors, a highest confidence width prior from among the plurality of width priors, and a highest confidence orientation prior from among the plurality of orientation priors; and
taking an action based on the output indication of a detected object.

2. The method of claim 1 , wherein taking an action based on the output indication of a detected object comprises:
performing, by the processor, a vehicle operation based on the output indication of a detected object.

3. The method of claim 1 , further comprising:
selecting the highest confidence height prior from the plurality of height priors; and
selecting the highest confidence width prior from the plurality of width priors.

4. The method of claim 1 , further comprising:
determining a height refinement value for each of the plurality of height priors; and
determining a width refinement value for each of the plurality of width priors.

5. The method of claim 4 , further comprising:
refining the highest confidence height prior with the height refinement value; and
refining the highest confidence width prior with the width refinement value.

6. The method of claim 5 , wherein outputting an indication of a detected object based on a highest confidence height prior from among the plurality of height priors and a highest confidence width prior from among the plurality of width priors comprises:
outputting the indication of a detected object comprising the refined highest confidence height prior and the refined highest confidence width prior.

7. The method of claim 1 , wherein the prior box associated with each pixel further comprises a center value.

8. The method of claim 7 , further comprising:
determining for each prior box a center value refinement; and
refining the center value with the determined center value refinement.

9. The method of claim 1 , further comprising:
determining for each prior box one or more object classifications; and
determining for each of the one or more object classifications a confidence value.

10. The method of claim 1 , wherein the output indication of a detected object comprises one or more of a height confidence value and a width confidence value.

11. A vehicle control unit, comprising:
a processor configured with processor-executable instructions to perform operations comprising:
receiving sensor information from a sensor;
generating a feature map based on the sensor information;

associating with each pixel of the feature map a prior box comprising:
a plurality of width priors that each include a width value,
a plurality of height priors that each include a height value, and
a plurality of orientation priors that each include an orientation value;
determining a confidence value of each height prior of the plurality of height priors, each width prior of the plurality of width priors for each prior box, and each orientation prior of the plurality of orientation priors;
outputting an indication of a detected object based on a highest confidence height prior from among the plurality of height priors a highest confidence width prior from among the plurality of width priors, and a highest confidence orientation prior from among the plurality of orientation priors; and
taking an action based on the output indication of a detected object.

12. The vehicle control unit of claim 11 , wherein the processor is configured with processor-executable instructions to perform operations such that taking an action based on the output indication of a detected object comprises:
performing a vehicle operation based on the output indication of a detected object.

13. The vehicle control unit system of claim 11 , wherein the processor is configured with processor-executable instructions to perform operations further comprising:
selecting the highest confidence height prior from the plurality of height priors; and
selecting the highest confidence width prior from the plurality of width priors.

14. The vehicle control unit of claim 11 , wherein the processor is configured with processor-executable instructions to perform operations further comprising:
determining a height refinement value for each of the plurality of height priors; and
determining a width refinement value for each of the plurality of width priors.

15. The vehicle control unit of claim 14 , wherein the processor is configured with processor-executable instructions to perform operations further comprising:
refining the highest confidence height prior with the height refinement value; and
refining the highest confidence width prior with the width refinement value.

16. The vehicle control unit of claim 15 , wherein the processor is configured with processor-executable instructions to perform operations such that outputting an indication of a detected object based on a highest confidence height prior from among the plurality of height priors and a highest confidence width prior from among the plurality of width priors comprises:
outputting the indication of a detected object comprising the refined highest confidence height prior and the refined highest confidence width prior.

17. The vehicle control unit of claim 11 , wherein the processor is configured with processor-executable instructions to perform operations such that each associated prior box further comprises a center value.

18. The vehicle control unit of claim 17 , wherein the processor is configured with processor-executable instructions to perform operations further comprising:
determining for each prior box a center value refinement; and
refining the center value with the determined center value refinement.

19. The vehicle control unit of claim 11 , wherein the processor is configured with processor-executable instructions to perform operations further comprising:
determining for each prior box one or more object classifications; and
determining for each of the one or more object classifications a confidence value.

20. The vehicle control unit of claim 11 , wherein the processor is configured with processor-executable instructions to perform operations such that the output indication of a detected object comprises one or more of a height confidence value and a width confidence value.

21. A method of processing vehicle sensor information for object detection, comprising:
capturing, by a sensor, sensor information regarding an environment around the sensor;
generating, by a processor, a feature map based on the sensor information;
associating, by the processor, with each pixel of the feature map, one or more prior boxes each comprising a center val

Assignees

• Qualcomm Inc

Inventors

• Major Bence
• Fontijne Daniel Hendricus Franciscus
• Sukhavasi Ravi Teja
• Ansari Amin

Classifications

• G06V10/25 Primary

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title