Multi-viewpoint image processing system and method thereof

Patent metadata

Field	Value
Publication number	US-12355934-B2
Application number	US-202117765454-A
Country	US
Kind code	B2
Filing date	May 28, 2021
Priority date	May 28, 2021
Publication date	Jul 8, 2025
Grant date	Jul 8, 2025

Abstract

A multi-viewpoint image processing system includes a processing apparatus and a display apparatus, the processing apparatus includes an acquisition module and an encoding module, wherein the acquisition module is configured to acquire K viewpoint images, and a viewpoint image includes M rows and N columns of pixels; the encoding module is configured to receive the K viewpoint images, encode the K viewpoint images to generate a plurality of encoded images, and send the plurality of encoded images to the display apparatus; the display apparatus is configured to receive the plurality of encoded images and obtain M pieces of display information according to the plurality of encoded images, an i-th piece of display information including i-th rows of pixels of the K viewpoint images, i=1, 2, . . . , M; and perform stereoscopic display according to the M pieces of display information.

First claim

The invention claimed is:

1. A multi-viewpoint image processing system, comprising: a processing apparatus and a display apparatus, wherein the processing apparatus comprises an acquisition module and an encoding module,
the acquisition module is configured to acquire K viewpoint images, wherein a viewpoint image comprises M rows and N columns of pixels, K is a positive integer greater than or equal to 2, and M and N are positive integers greater than or equal to 1;
the encoding module is configured to receive the K viewpoint images, encode the K viewpoint images to generate a plurality of encoded images, and send the plurality of encoded images to the display apparatus;
the display apparatus is configured to receive the plurality of encoded images and obtain M pieces of display information according to the plurality of encoded images, an i-th piece of display information comprising i-th rows of pixel points of the K viewpoint images, i=1, 2, . . . , M; and perform stereoscopic display according to the M pieces of display information; and
the processing apparatus is arranged in the display apparatus, or the processing apparatus and the display apparatus are arranged separately,
the display apparatus comprises a decoding module and a display module, wherein
the decoding module is configured to receive the plurality of encoded images and obtain the M pieces of display information according to the plurality of encoded images; and
the display module is configured to receive the M pieces of display information and perform stereoscopic display according to the M pieces of display information,
wherein the decoding module is configured to obtain the i-th rows of pixels of the K viewpoint images according to the first viewpoint image and the K−1 second viewpoint images; and obtain the M pieces of display information according to first rows of pixels to M-th rows of pixels of the K viewpoint images; and
the display module is configured to display the i-th rows of the K viewpoint images according to the i-th piece of display information,
wherein the encoding module is configured to select an m-th viewpoint image from the K viewpoint images as a first viewpoint image, and take all viewpoint images except the m-th viewpoint image as second viewpoint images, a quantity of the second viewpoint images being K−1, 1≤m≤K; obtain K−1 first images according to the first viewpoint image and the K−1 second viewpoint images, wherein a k-th first image is obtained according to the first viewpoint image and a k-th second viewpoint image, k=1, 2, . . . , K−1; obtain K−1 encoded images according to the first viewpoint image, the K−1 second viewpoint images, and the K−1 first images, wherein a k-th encoded image is obtained according to the first viewpoint image, the k-th second viewpoint image, and the k-th first image; and send the K−1 encoded images and the first viewpoint image, the first images and the encoded images each comprising M rows and N columns of pixels, or
the encoding module is configured to extract sequentially an i-th row of pixels of each of the viewpoint images to form M second images, each of the second images comprising K rows and N columns of pixels, and an n-th row of pixels of an i-th second image being an i-th row of pixels of an n-th viewpoint image, n=1, 2, . . . , K; and encode the M second images to generate M encoded images.

2. The system according to claim 1 , wherein the acquisition module is configured to acquire an image to be displayed, and acquire the K viewpoint images according to the image to be displayed.

3. The system according to claim 1 , further comprising a multi-viewpoint collection apparatus, the multi-viewpoint collection apparatus comprises a plurality of Charge Coupled Device (CCD) cameras, wherein each of the CCD cameras in the multi-viewpoint collection apparatus performs dynamic scene collection to obtain the K viewpoint images; and
the acquisition module is configured to receive the K viewpoint images obtained by the multi-viewpoint collection apparatus.

4. The system according to claim 1 , wherein the encoding module is further configured to obtain information of a pixel in an i-th row and a j-th column of the k-th first image according to information of a pixel in an i-th row and a j-th column of the first viewpoint image and information of a pixel in an i-th row and a j-th column of the k-th second viewpoint image; and obtain the k-th first image according to information of all pixels of the k-th first image, j=1, 2, . . . , N.

5. The system according to claim 4 , wherein the information of the pixel comprises color component information, the color component information comprises a first color component value, a second color component value, and a third color component value, the first color, the second color, and the third color are one of red, green, and blue respectively and are different colors;
the encoding module is further configured to obtain a grayscale value of the pixel in the i-th row and the j-th column of the first viewpoint image according to color component information of the pixel in the i-th row and the j-th column of the first viewpoint image; and obtain a grayscale value of the pixel in the i-th row and the j-th column of the k-th second viewpoint image according to color component information of the pixel in the i-th row and the j-th column of the k-th second viewpoint image, wherein when a difference between the grayscale value of the pixel in the i-th row and the j-th column of the first viewpoint image and the grayscale value of the pixel in the i-th row and the j-th column of the k-th second viewpoint image is less than a threshold difference, the pixel in the i-th row and the j-th column of the k-th first image is a black pixel, and when the difference between the grayscale value of the pixel in the i-th row and the j-th column of the first viewpoint image and the grayscale value of the pixel in the i-th row and the j-th column of the k-th second viewpoint image is greater than the threshold difference, the pixel in the i-th row and the j-th column of the k-th first image is a white pixel;
or the encoding module is further configured to calculate a difference between a first color component value of the pixel in the i-th row and the j-th column of the first viewpoint image and a first color component value of the pixel in the i-th row and the j-th column of the k-th second viewpoint image to obtain a first color component difference; calculate a difference between a second color component value of the pixel in the i-th row and the j-th column of the first viewpoint image and a second color component value of the pixel in the i-th row and the j-th column of the k-th second viewpoint image to obtain a second color component difference; calculate a difference between a third color component value of the pixel in the i-th row and the j-th column of the first viewpoint image and a third color component value of the pixel in the i-th row and the j-th column of the k-th second viewpoint image to obtain a third color component difference; and obtain a grayscale difference according to the first color component difference, the second color component difference, and the third color component difference, wherein when the grayscale difference is less than a threshold difference, the pixel in the i-th row and the j-th column of the k-th first image is a black pixel, and when the grayscale difference is greater than the threshold difference, the pixel in the i-th row and the j-th column of the k-th first image is a white pixel; and
when a color of the pixel in the i-th row and the j-th column of the k-th second viewpoint image is different, a threshold difference during obtaining the k-th first image is different.

6. The system according to claim 1 , wherein

Assignees

• Boe Technology Group Co Ltd

Inventors

• Chu Minglei
• Duan Xin
• Zhang Shuo
• Sun Yan
• Shi Tiankuo
• Sun Wei
• Peng Kuanjun
• Dong Xue
• Liu Jiantao

Classifications

• H04N13/282

  for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems · CPC title

• H04N13/194

  Transmission of image signals · CPC title

• H04N13/302

  for viewing without the aid of special glasses, i.e. using autostereoscopic displays · CPC title

• H04N13/15

  for colour aspects of image signals · CPC title

• H04N13/243

  using three or more two-dimensional [2D] image sensors · CPC title