Image processing device and method, and program for correcting an imaging direction

Patent metadata

Field	Value
Publication number	US-10110817-B2
Application number	US-201615391346-A
Country	US
Kind code	B2
Filing date	Dec 27, 2016
Priority date	Dec 26, 2012
Publication date	Oct 23, 2018
Grant date	Oct 23, 2018

Abstract

Provided is an image processing device including an acquisition unit configured to acquire information on an imaging position and an imaging direction in units of frame images that constitute a moving image obtained through capturing by an imaging unit, a converted image generation unit configured to generate converted images having different imaging directions for each frame image that constitutes the moving image based on the frame image itself and preceding and succeeding frame images of the frame image, an evaluation value calculation unit configured to calculate an evaluation value for each converted moving image constituted by combining the converted image and the original frame image, the evaluation value being used to evaluate a blur between the converted images or between the original frame images, and a selection unit configured to select a converted moving image with less blur based on an evaluation value calculated by the evaluation value calculation unit.

First claim

The invention claimed is:

1. An image processing device comprising:
at least one processor configured to:
acquire information on a first frame image, a second frame image, and a third frame image that constitute a moving image obtained through capturing by an imaging unit, the second frame image being between the first frame image and the third frame image;
correct a second imaging direction of the second frame image based on at least one of a first imaging direction of the first frame image and a third imaging direction of the third frame image to reduce differences between the first imaging direction, the second imaging direction, and the third imaging direction;
send the second frame image having the corrected second imaging direction to a display unit; and
control the display unit to display a route on a map,

wherein the route corresponds to frame images that constitute the moving image.

2. The image processing device according to claim 1 ,
wherein the at least one processor is further configured to:
acquire a correction input by a user to determine a correction amount of the second imaging direction; and
correct the second imaging direction based on the correction amount.

3. The image processing device according to claim 2 ,
wherein the at least one processor is further configured to control the display unit to display a correction indicator indicating the correction amount.

4. The image processing device according to claim 3 ,
wherein the at least one processor is further configured to control the display unit to display the route corresponding to the correction amount along with the correction indicator.

5. The image processing device according to claim 3 ,
the correction indicator is an operable scale receiving the correction input.

6. The image processing device according to claim 1 ,
wherein the route has colors corresponding to imaging directions of the frame images, and
wherein changes between the colors correspond to changes between the imaging directions.

7. The image processing device according to claim 1 ,
wherein the at least one processor is further configured to:
recognize at least one object included in the frame images; and
control the display unit to display the at least one object on the map as time series images.

8. The image processing device according to claim 7 ,
wherein the at least one object includes a plurality of objects classified into object types including at least two of a building, a landscape, a vehicle and a person, and
wherein the at least one processor is further configured to control the display unit to display the time series images in a first direction and display the plurality of objects classified into the object types in a second direction perpendicular to the first direction.

9. The image processing device according to claim 1 ,
wherein the image processing device is a network server further comprising a storage unit holding the moving image obtained through capturing by the imaging unit.

10. The image processing device according to claim 9 ,
wherein the at least one processor is further configured to:
determine whether the frame images that constitute the moving image contain a prohibited region to be prohibited from being reproduced; and
make the prohibited region to be processed incapable of viewing by a user in a case when determining that the prohibited region is contained in the frame images.

11. The image processing device according to claim 1 ,
wherein the at least one processor is further configured to:
correct the second imaging direction by generating a converted frame image based on an angle of view shift using the second frame image and the first frame image and the third frame image.

12. The image processing device according to claim 1 ,
wherein the at least one processor is further configured to:
correct the second imaging direction by generating a converted frame image based on transformation of a shape of the second frame image using the second frame image and the first frame image and the third frame image.

13. The image processing device according to claim 1 ,
wherein the at least one processor is further configured to:
correct the second imaging direction by generating a converted frame image based on composition of viewpoints of the first frame image, the second frame image, and the third frame image.

14. The image processing device according to claim 1 ,
wherein the imaging unit is a mobile camera.

15. The image processing device according to claim 14 ,
wherein the mobile camera is a wearable camera.

16. The image processing device according to claim 1 ,
wherein the at least one processor constitute at least one network server.

17. The image processing device according to claim 1 ,
wherein the at least one processor performs the correcting of the second imaging direction of the second frame image and the sending of the second frame image by a cloud computing.

18. An image processing method, performed via at least one processor, the method comprising:
acquiring information on a first frame image, a second frame image, and a third frame image that constitute a moving image obtained through capturing by an imaging unit, the second frame image being a frame image between the first frame image and the third frame image;
correcting a second imaging direction of the second frame image based on a first imaging direction of the first frame image and a third imaging direction of the third frame image to reduce differences between the first imaging direction, the second imaging direction, and the third imaging direction;
sending the second frame image having the corrected second imaging direction to a display unit; and
controlling the display unit to display a route on a map,
wherein the route corresponds to frame images that constitute the moving image.

19. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer, causes the computer to execute a method, the method comprising:
acquiring information on a first frame image, a second frame image, and a third frame image that constitute a moving image obtained through capturing by an imaging unit, the second frame image being a frame image between the first frame image and the third frame image;
correcting a second imaging direction of the second frame image based on a first imaging direction of the first frame image and a third imaging direction of the third frame image to reduce differences between the first imaging direction, the second imaging direction, and the third imaging direction;
sending the second frame image having the corrected second imaging direction to a display unit; and
controlling the display unit to display a route on a map,
wherein the route corresponds to frame images that constitute the moving image.

Assignees

• Sony Corp

Inventors

• Sutou Yasuhiro
• Shimomura Hideki
• Okubo Atsushi
• Aoyama Kazumi
• Tanaka Akichika

Classifications

• H04N23/682

  Vibration or motion blur correction · CPC title

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• G03B17/48 Primary

  adapted for combination with other photographic or optical apparatus · CPC title

• H04N23/631

  Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters · CPC title

• H04N23/683 Primary

  performed by a processor, e.g. controlling the readout of an image memory · CPC title