Image processing method in remote control, device, apparatus and program product

Patent metadata

Field	Value
Publication number	US-12114100-B2
Application number	US-202117519375-A
Country	US
Kind code	B2
Filing date	Nov 4, 2021
Priority date	Dec 22, 2020
Publication date	Oct 8, 2024
Grant date	Oct 8, 2024

Abstract

An image processing method in remote control, a device, an apparatus and a program product are provided and related to the field of automatic driving technologies. The specific implementation solution includes receiving image information sent by a vehicle, wherein the image information includes multiple-channel image data collected by the vehicle; performing a stitching process on the multiple-channel image data, to obtain a stitched image; sending the stitched image to a remote cockpit apparatus for controlling the vehicle.

First claim

What is claimed is:

1. An image processing method in remote control, performed by a cloud apparatus, comprising:
receiving image information sent by a vehicle, wherein the image information comprises multiple-channel image data collected by the vehicle;
performing a stitching process on the multiple-channel image data to obtain a stitched image; and
sending the stitched image to a remote cockpit apparatus for controlling the vehicle;
wherein the multiple-channel image data comprises a first channel image data, a second channel image data and a third channel image data, the performing the stitching process on the multiple-channel image data to obtain the stitched image comprises:
extracting a first common pixel point of the first channel image data and the second channel image data;
creating a first coordinate system based on the first common pixel point, mapping the first channel image data to the first coordinate system to obtain first image mapping data, and mapping the second channel image data to the first coordinate system to obtain second image mapping data;
replicating the first image mapping data and the second image mapping data into a first blank image, respectively, and performing weighted fusion of overlapping image data of the first image mapping data and the second image mapping data in the first blank image to obtain a first stitched sub-image;
extracting a second common pixel point of the third channel image data and the second channel image data;
creating a second coordinate system based on the second common pixel point, mapping the third channel image data to the second coordinate system to obtain third image mapping data, and mapping the second channel image data to the second coordinate system to obtain fourth image mapping data;
replicating the third image mapping data and the fourth image mapping data into a second blank image, respectively, and performing weighted fusion of overlapping image data of the third image mapping data and the fourth image mapping data in the second blank image to obtain a second stitched sub-image; and
stitching the first stitched sub-image and the second stitched sub-image to obtain the stitched image.

2. The method according to claim 1 , wherein:
the receiving the image information sent by the vehicle comprises receiving first image compression information sent by the vehicle via real-time communication (RTC), wherein the first image compression information comprises the multiple-channel image data collected by the vehicle; and
performing the stitching process on the multiple-channel image data comprises performing a decompression process on the first image compression information to obtain the multiple-channel image data, and performing the stitching process on the multiple-channel image data to obtain the stitched image.

3. The method according to claim 1 , wherein sending the stitched image to the remote cockpit apparatus for controlling the vehicle comprises:
compressing the stitched image to obtain second image compression information, and sending the second image compression information via RTC to the remote cockpit apparatus for controlling the vehicle.

4. An electronic apparatus, comprising:
at least one processor; and
a storage communicatively connected to the at least one processor, wherein,
instructions executable by the at least one processor are stored in the storage, the instructions are performed by the at least one processor to cause the at least one processor to perform an image processing method in remote control, the method comprising,
receiving image information sent by a vehicle, wherein the image information comprises multiple-channel image data collected by the vehicle;
performing a stitching process on the multiple-channel image data to obtain a stitched image;
sending the stitched image to a remote cockpit apparatus for controlling the vehicle;
wherein the multiple-channel image data comprises a first channel image data, a second channel image data and a third channel image data, the performing the stitching process on the multiple-channel image data to obtain the stitched image comprises:
extracting a first common pixel point of the first channel image data and the second channel image data;
creating a first coordinate system based on the first common pixel point, mapping the first channel image data to the first coordinate system to obtain first image mapping data, and mapping the second channel image data to the first coordinate system to obtain second image mapping data;
replicating the first image mapping data and the second image mapping data into a first blank image, respectively, and performing weighted fusion of overlapping image data of the first image mapping data and the second image mapping data in the first blank image to obtain a first stitched sub-image;
extracting a second common pixel point of the third channel image data and the second channel image data;
creating a second coordinate system based on the second common pixel point, mapping the third channel image data to the second coordinate system to obtain third image mapping data, and mapping the second channel image data to the second coordinate system to obtain fourth image mapping data;
replicating the third image mapping data and the fourth image mapping data into a second blank image, respectively, and performing weighted fusion of overlapping image data of the third image mapping data and the fourth image mapping data in the second blank image to obtain a second stitched sub-image; and
stitching the first stitched sub-image and the second stitched sub-image to obtain the stitched image.

5. The electronic apparatus according to claim 4 , wherein:
receiving the image information sent by the vehicle comprises receiving first image compression information sent by the vehicle via real-time communication (RTC), wherein the first image compression information comprises the multiple-channel image data collected by the vehicle; and
performing the stitching process on the multiple-channel image data comprises performing a decompression process on the first image compression information to obtain the multiple-channel image data, and performing the stitching process on the multiple-channel image data to obtain the stitched image.

6. The electronic apparatus according to claim 4 , wherein sending the stitched image to the remote cockpit apparatus for controlling the vehicle comprises:
compressing the stitched image to obtain second image compression information, and sending the second image compression information via RTC to the remote cockpit apparatus for controlling the vehicle.

7. A non-transitory computer readable storage medium, storing therein computer instructions to cause a computer to perform an image processing method in remote control, the method comprising:
receiving image information sent by a vehicle, wherein the image information comprises multiple-channel image data collected by the vehicle;
performing a stitching process on the multiple-channel image data to obtain a stitched image; and
sending the stitched image to a remote cockpit apparatus for controlling the vehicle;
wherein the multiple-channel image data comprises a first channel image data, a second channel image data and a third channel image data, the performing the stitching process on the multiple-channel image data to obtain the stitched image comprises:
extracting a first common pixel point of the first channel image data and the second channel image data;
creating a first coordinate system based on the first common pixel point, mapping the first channel image data to the first coordinate system to obtain first image mapping data, and mapping the second channel image data to the first coordinate system to obtain se

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Sun Qingrui
• Feng Jingchao
• Xia Liming
• Chen Zhuo

Classifications

• G05D1/249

  from positioning sensors located off-board the vehicle, e.g. from cameras · CPC title

• G06V10/80

  Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10) · CPC title

• H04L65/61

  for supporting one-way streaming services, e.g. Internet radio · CPC title

• G06T9/00

  Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title