Imaging processing device, imaging system and imaging apparatus including the same, and image processing method

Patent metadata

Field	Value
Publication number	US-11290645-B2
Application number	US-201916694329-A
Country	US
Kind code	B2
Filing date	Nov 25, 2019
Priority date	Feb 6, 2015
Publication date	Mar 29, 2022
Grant date	Mar 29, 2022

Abstract

A video display method for a video display device is provided. The video display method comprises: acquiring a plurality of captured images from each of a plurality of cameras for each frame; generating a plurality of correction images from the plurality of captured images by performing parallax correction in each of the plurality of captured images for each frame, the parallax correction being performed between images captured by adjacent cameras; and compositing the plurality of correction images to generate a 360° panoramic composite image for each frame. An amount of the parallax correction of each captured image in each frame is limited so that an amount of change from a previous amount of the parallax correction in a previous frame is within a range defined by a predetermined limitation value.

First claim

What is claimed is:

1. A video display method for a video display device, the video display method comprising:
acquiring a plurality of captured images from each of a plurality of cameras, the plurality of captured images being acquired for each frame of a plurality of frames;
generating a plurality of correction images from the plurality of captured images by performing parallax correction in each of the plurality of captured images for each frame, the parallax correction being performed between images captured by adjacent cameras in a same frame; and
compositing the plurality of correction images to generate a panoramic composite image for each frame,
wherein, in the generating, a second amount of the parallax correction between a pair of adjacent captured images in a second frame of the plurality of frames is limited so that a difference between the second amount and a first amount of the parallax correction between the pair of adjacent captured images in a first frame of the plurality of frames is within a range defined by a predetermined limitation value, the first frame being immediately prior in time to the second frame.

2. The video display method of claim 1 ,
wherein in a first case where the difference between the second amount and the first amount is within the range defined by the predetermined limitation value,
a first correction image is generated based on the second amount of the parallax correction, and

wherein in a second case where the difference between the second amount and the first amount is outside the range defined by the predetermined limitation value,
a second correction image is generated by using a value obtained by adding or subtracting the predetermined limitation value to the second amount of parallax correction.

3. The video display method of claim 1 , wherein
each of the plurality of correction images is further generated by performing block matching between the images captured by the adjacent cameras in the same frame.

4. The video display method of claim 1 , wherein
each of the plurality of correction images is further generated based on a result of detection by a sensor that detects a distance to a photographic subject.

5. The video display method of claim 1 ,
wherein the plurality of cameras is arranged such that portions of image capture areas of the adjacent cameras overlap.

6. A video display method for a video display device, the video display method comprising:
acquiring a plurality of captured images from each of a plurality of cameras, the plurality of captured images being acquired for each frame of a plurality of frames;
generating a plurality of correction images from the plurality of captured images by performing parallax correction in each of the plurality of captured images for each frame, the parallax correction being performed between images captured by adjacent cameras in a same frame; and
compositing the plurality of correction images to generate a panoramic composite image for each frame,
wherein in a case where, in the panoramic composite image of a first frame of the plurality of frames, a first image of a first object at a first distance is displayed across a boundary portion between a captured image region of a first camera of the plurality of cameras and an adjacent captured image region of a second camera of the plurality of cameras adjacent to the first camera and a second image of a second object, which is different from the first object, at a second distance is displayed across the boundary portion in a second frame of the plurality of frames following the first frame,
in the second frame, a first correction image corresponding to the first camera and a second correction image corresponding to the second camera are generated by performing the parallax correction such that the second image of the second object in the boundary portion is displaced in a direction intersecting the boundary portion so as to be close to a display state in portions other than the boundary portion,
wherein, in the generating, a second amount of the parallax correction between the first image and the second image in the second frame is limited so that a difference between the second amount and a first amount of the parallax correction between the first image and the second image in the first frame is within a range defined by a predetermined limitation value.

7. The video display method of claim 6 ,
wherein in a first case where the difference between the second amount and the first amount is within the range defined by the predetermined limitation value,
a first correction image is generated based on the second amount of the parallax correction, and

wherein in a second case where the difference between the second amount and the first amount is outside the range defined by the predetermined limitation value,
a second correction image is generated by using a value obtained by adding or subtracting the predetermined limitation value to the second amount of parallax correction.

8. The video display method of claim 6 , wherein
each of the plurality of correction images is further generated by performing block matching between the images captured by the adjacent cameras in the same frame.

9. The video display method of claim 6 , wherein
each of the plurality of correction images is further generated based on a result of detection by a sensor that detects a distance to a photographic subject.

10. The video display method of claim 6 ,
wherein the plurality of cameras is arranged such that portions of image capture areas of the adjacent cameras overlap.

11. A video image processing device, comprising:
a processor, and
a memory including instructions that, when executed by the processor, cause the processor to perform operations, including:
acquiring a plurality of captured images from each of a plurality of cameras, the plurality of captured images being acquired for each frame of a plurality of frames;
generating a plurality of correction images from the plurality of captured images by performing parallax correction in each of the plurality of captured images for each frame, the parallax correction being performed between images captured by adjacent cameras in a same frame; and

compositing the plurality of the correction images to generate a panoramic composite image for each frame,
wherein, in the generating, a second amount of the parallax correction between a pair of adjacent captured images in a second frame of the plurality of frames is limited so that a difference between the second amount and a first amount of the parallax correction between the pair of adjacent captured images in a first frame of the plurality of frames is within a range defined by a predetermined limitation value, the first frame being immediately prior in time to the second frame.

Assignees

• Panasonic Ip Man Co Ltd

Inventors

• Kuwada Junya
• Ogino Ryoji
• Hirota Ryo
• Nishiyama Akio

Classifications

• H04N1/3876 Primary

  Recombination of partial images to recreate the original image · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/70

  Circuitry for compensating brightness variation in the scene · CPC title

• H04N23/88

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title