Line-based feature generation for vision-based driver assistance systems and methods

Patent metadata

Field	Value
Publication number	US-11295151-B2
Application number	US-201916527179-A
Country	US
Kind code	B2
Filing date	Jul 31, 2019
Priority date	Jul 31, 2019
Publication date	Apr 5, 2022
Grant date	Apr 5, 2022

Abstract

Embodiments provide line-based feature generation for vision-based driver assistance systems and methods. For one embodiment, a feature generator includes a circular buffer and a processor coupled to an image sensor. The circular buffer receives image data from the image sensor and stores N lines at a time of an image frame captured by the image sensor. The N lines of the image frame are less than all of the lines for the image frame. The processor receives the N lines from the circular buffer and stores one or more features generated from the N lines in a memory. Iterative blocks of N lines of image data are processed to complete processing of the full image frame, and multiple frames can be processed. The generated features are analyzed by a vision processor to identify, classify, and track objects for vision-based driver assistance and related vision-based assistance actions.

First claim

What is claimed is:

1. A system, comprising:
an image sensor;
a feature generator, comprising:
a circular buffer coupled to receive image data from the image sensor and to store N lines of an image frame at a time, the N lines being less than all of the lines for the image frame;
a processor coupled to receive the N lines from the circular buffer and having one or more features generated from the N lines as an output;
a controller for the processor coupled to the circular buffer to cause all lines for the image frame to be received and processed by the processor through the circular buffer;
a memory coupled to store the one or more features from the processor;
a pixel buffer coupled between the circular buffer and a feature engine within the processor; and
a feature buffer coupled between the feature engine and the memory, wherein:
the controller is configured to cause data transfers for the pixel buffer and for the feature buffer to occur in parallel to provide pipelined operation, and
a direct memory access (DMA) is used for data transfers from the circular buffer to the pixel buffer and from the feature buffer to the memory; and

a vision processor coupled to receive the one or more features from the memory and having one or more actions for vision-based assistance as an output.

2. The system of claim 1 , wherein the processor comprises hardware logic circuits configured to generate the one or more features from the N lines of the image.

3. The system of claim 1 , wherein the one or more actions comprises vibrating the steering wheel, vibrating the seat, generating an alarm sound, applying an emergency break, or adjusting the steering.

4. The system of claim 1 , wherein only a subset of pixels within the circular buffer are processed at a time by the processor.

5. A circuit to generate features for a vision-based assistance system, comprising:
a circular buffer coupled to receive image data for an image and to store N lines of an image frame at a time, the N lines being less than all of the lines for the image frame;
a processor coupled to receive the N lines from the circular buffer and having one or more features generated from the N lines as an output;
a controller ( 304 ) for the processor coupled to the circular buffer to cause all lines for the image frame to be received and processed by the processor through the circular buffer; and
a memory coupled to store the one or more features from the processor;
a pixel buffer coupled between the circular buffer and a feature engine within the processor; and
a feature buffer coupled between the feature engine and the memory, wherein:
the controller is configured to cause data transfers for the pixel buffer and for the feature buffer to occur in parallel to provide pipelined operation, and
a direct memory access (DMA) is used for data transfers from the circular buffer to the pixel buffer and from the feature buffer to the memory.

6. The circuit of claim 5 , wherein the processor comprises hardware logic circuits configured to generate the one or more features from the N lines of the image.

7. The circuit of claim 5 , wherein the processor further comprises a feature engine ( 310 ) including a plurality of feature extractors ( 316 ) coupled to receive and process the image data.

8. The circuit of claim 5 , wherein only a subset of pixels within the circular buffer are processed at a time.

9. The circuit of claim 5 , wherein the controller is configured to cause data transfers from the circular buffer to occur when the circular buffer is full.

Assignees

• Nxp Usa Inc

Inventors

• Naidu Sharath Subramanya
• Singh Ajit
• Staudenmaier Michael Andreas
• Surico Leonardo
• Herrmann Stephan Matthias

Classifications

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06V10/70

  using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) · CPC title

• G06V10/955

  using specific electronic processors · CPC title

• G06F18/21

  Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation · CPC title

• G06V2201/07

  Target detection · CPC title