Imaging system, imaging apparatus, and system

Patent metadata

Field	Value
Publication number	US-10681268-B2
Application number	US-201514941152-A
Country	US
Kind code	B2
Filing date	Nov 13, 2015
Priority date	May 15, 2014
Publication date	Jun 9, 2020
Grant date	Jun 9, 2020

Abstract

An imaging system includes an imaging unit, a detecting unit, a recording unit, a correcting unit, and a video generating unit. The imaging unit captures images of a plurality frames. The detecting unit detects a tilt of the imaging unit with respect to a reference direction. The recording unit records time-series data of the tilt detected by the detecting unit. The correcting unit applies tilt correction to an image of each of the frames captured by the imaging unit, based on the time-series data of the tilt recorded by the recording unit and additional information unique to the imaging unit. The video generating unit generates video data based on the image of each of the frames corrected by the correcting unit.

First claim

What is claimed is:

1. An imaging system, comprising:
an omnidirectional camera including a plurality of lenses, each with a different image capturing direction, and first processing circuitry configured to
capture a plurality of associated images through the plurality of lenses over a period of time to capture a sequence of the plurality of associated images;
detect, while capturing the sequence of the plurality of associated images, a tilt angle of the omnidirectional camera with respect to a direction of gravity and generate tilt angle data; and
record a time series of the tilt angle data corresponding to the sequence of the plurality of associated images into a memory; and

an image processing apparatus including second processing circuitry configured to
access the sequence of the plurality of associated images and the time series of the tilt angle data corresponding to the plurality of associated images; and
apply, based on the time series of the tilt angle data, a tilt correction with respect to the direction of gravity to each of the images of the sequence of the plurality of associated images to generate omnidirectional video data including a plurality of frames of omnidirectional images with corrected tilt angles, so that a zenith direction of the omnidirectional images of the plurality of frames that are corrected coincide with the direction of gravity,

wherein:
the first processing circuitry is further configured to generate fisheye images, and the second processing circuitry is further configured to convert the fisheye images into the omnidirectional images by using a conversion table, the conversion table including design data of lens optical systems of the omnidirectional camera, and
the conversion table is modified to reflect the zenith direction when a direct upward direction of the omnidirectional camera does not coincide with a vertical line.

2. The imaging system according to claim 1 , wherein
the first processing circuitry is further configured to
compress the sequence of the plurality of associated images, and
output the sequence of the plurality of associated images which has been compressed to the image processing apparatus,

the second processing circuitry is further configured to receive the sequence of the plurality of associated images which has been compressed and output by the first processing circuitry, and
in applying the tilt correction, the second processing circuitry is configured to
decompress the sequence of the plurality of associated images which has been compressed,
apply the tilt correction to the sequence of the plurality of associated images which has been decompressed, and
generate the omnidirectional images with corrected tilt angles from the sequence of the plurality of associated images which has been decompressed.

3. The imaging system according to claim 1 , wherein
the first processing circuitry is configured to output the sequence of the plurality of associated images to the image processing apparatus, and
the second processing circuitry is configured to receive the sequence of the plurality of associated images output by the first processing circuitry.

4. The imaging system according to claim 1 , wherein
the omnidirectional camera includes a first wide-angle imaging optical system including one of the plurality of lenses and a second wide-angle imaging optical system including another of the plurality of lenses,
the first processing circuitry captures the sequence of the plurality of associated images of the plurality of frames by controlling the first wide-angle imaging optical system to capture first partial omnidirectional images and by controlling the second wide-angle imaging optical system to capture second partial omnidirectional images,
the second processing circuitry is configured to
decompress the sequence of the plurality of associated images into the first and second partial omnidirectional images,
modify, according to the time series of the tilt angle data, conversion data for converting the first partial omnidirectional images and the second partial omnidirectional images into data in a spherical coordinate system,
convert, using the modified conversion data, the first partial omnidirectional images and the second partial omnidirectional images into data in the spherical coordinate system,
generate adjusted omnidirectional images from the first partial omnidirectional images and the second partial omnidirectional images, and
generate adjusted video data based on the adjusted omnidirectional images.

5. The imaging system according to claim 1 , wherein the second processing circuitry is configured to load into a second memory, to apply the tilt correction, some of the images of the sequence of the plurality of associated images that are based on a starting frame.

6. The imaging system according to claim 1 , wherein
the omnidirectional camera includes an acceleration sensor, and
the first processing circuitry detects the tilt angle of the omnidirectional camera according to an output of the acceleration sensor.

7. An imaging apparatus, comprising:
processing circuitry configured to
capture a plurality of associated images through a plurality of lenses over a period of time to capture a sequence of the plurality of associated images of a subject;
detect a tilt of the imaging apparatus during capture of the sequence of the plurality of associated images with respect to a direction of gravity;
generate a time series of tilt angle data of the imaging apparatus according to the detected tilt;
record the sequence of the plurality of associated images; and
record, into a memory, the time series of the tilt angle data corresponding to the sequence of the plurality of associated images, wherein

an image processing apparatus accesses the sequence of the plurality of associated images and the time series of the tilt angle data corresponding to the sequence of the plurality of associated images and applies, based on corresponding tilt angle data, a tilt correction with respect to the direction of gravity to the sequence of the plurality of associated images to generate omnidirectional video data including adjusted omnidirectional images in a spherical coordinate system and having corrected tilt so that a zenith direction of the adjusted omnidirectional images coincide with the direction of gravity,
wherein:
the capture of the plurality of associated images captures fisheye images,
the processing circuitry further configured to convert the fisheye images into the omnidirectional images by using a conversion table, the conversion table including design data of lens optical systems of an omnidirectional camera that captures the fisheye images, and
the processing circuitry to modify the conversion table to reflect the zenith direction when a direct upward direction of the omnidirectional camera does not coincide with a vertical line.

8. The imaging apparatus according to claim 7 , wherein the processing circuitry is configured to associate each of the sequence of the plurality of associated images with corresponding tilt angle data based on the tilt of the imaging apparatus during capture of the sequence of the plurality of associated images.

9. The imaging apparatus according to claim 7 , further comprising:
a plurality of cameras, wherein
the processing circuitry is configured to generate the omnidirectional video data based on the sequence of the plurality of associated images output from the plurality of cameras, and
the processing circuitry is configured to output the omnidirectional video data.

10. The imaging apparatus according to claim 7 , further comprising:
a first wide-angle imaging optical syst

Assignees

• Ricoh Co Ltd

Inventors

• Yamamoto Hideaki

Classifications

• H04N5/23267

  Electricity · mapped topic

• H04N5/2628

  Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation · CPC title

• G03B37/00

  Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe · CPC title

• H04N5/2258

  Electricity · mapped topic

• H04N5/2254

  Electricity · mapped topic