Image synthesis method with DSP and GPU

Patent metadata

Field	Value
Publication number	US-9672586-B2
Application number	US-201514920328-A
Country	US
Kind code	B2
Filing date	Oct 22, 2015
Priority date	Oct 22, 2015
Publication date	Jun 6, 2017
Grant date	Jun 6, 2017

Abstract

This patent application focuses on partitioning the computation to perform the image processing on the DSP, communicate the results to the GPU in an optimized manner, and use the GPU to render 3D information to the display. This invention solves the problem of seamlessly merging live streams from plural automotive cameras into a 3D display that can be rotated to see different view points. This purpose is achieved by partitioning the computation task between a digital signal processor (DSP) for image processing and a graphics processing unit (GPU) for rendering and display. This invention also includes meta data structures for communication of results from the DSP to the GPU.

First claim

What is claimed is:

1. A method of data processing comprising steps of:
receiving a video image feed from each of a plurality of cameras having differing views of a scene;
dewarping each video image feed from a corresponding received video image to a predetermined display surface;
adjusting lighting of each dewarped video image to a uniform lighting level;
wherein said steps of receiving, dewarping, adjusting and synthesizing are performed via a digital signal processor;
synthesizing a combined image from the lighting adjusted video images into a single combined video image;
vertex processing;
pixel processing;
blending;
forming a birds eye surround view;
said steps of vertex processing, pixel processing, blending, shading and color conversion, and forming a birds eye surround view are performed on a graphics processing unit; and
further comprising a step of forming an index buffer data structure to transmit data from the digital signal processor to the graphics processing unit mapping per-pixel for the whole buffer as defined below:

typedef struct _gridpoint

{

  short x;
/1. X coordinate, signed 16bit

  short y;
/2. Y coordinate, signed 16bit

  q12_4_t z;
/3. Z coordinate (Q12.4

    format)

  unsigned short texelY;
/4. Input image gridpoint

    location vertical

  unsigned short texelX;
/5. Input image gridpoint

    location horizontal

  unsigned short imageindex1;/6. Input image index for

    blend layer1 (i.e 1

    - 4)

  unsigned char alpha1;
/ Alpha for layer 1

  unsigned short imageindex2;/6. Input image index for

    blend layer2 (i.e 1

    - 4)

  unsigned char alpha2;
/ Alpha for layer 2 (including

    priority)

  unsigned char coloradjust;
/ Color adjustment factor

}gridpoint.

2. The method of claim 1 , wherein:
said steps of receiving, dewarping, adjusting and synthesizing performed via a digital signal processor and said steps of vertex processing, pixel processing, blending, and forming a birds eye surround view on a graphics processing unit are controlled by varying a mesh count depending on digital signal processor and graphics processing unit computational capacities.

3. The method of claim 1 , further comprising:
generating attribute arrays from per-pixel information having an index array, texture coordinates and image ID for a given camera view point and mesh.

Assignees

• Texas Instruments Inc

Inventors

• Sundareson Prabindh
• Balagopalakrishnan Anand

Classifications

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06T3/0018 Primary

  Physics · mapped topic

• G06T15/503

  Blending, e.g. for anti-aliasing · CPC title

• G06T5/006

  Physics · mapped topic

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title