Image processing device, image processing method, and program

Patent metadata

Field	Value
Publication number	US-10547774-B2
Application number	US-201314655621-A
Country	US
Kind code	B2
Filing date	Nov 12, 2013
Priority date	Jan 9, 2013
Publication date	Jan 28, 2020
Grant date	Jan 28, 2020

Abstract

There is provided an image processing device including a subject distance change determination unit configured to detect a temporal change of a distance from an imaging position to each subject present in an image and determine a tendency toward approach or recession of the each subject with respect to the imaging position on the basis of the detection, and a main subject determination unit configured to determine a main subject on the basis of the tendency toward approach or recession of the each subject determined by the subject distance change determination unit.

First claim

The invention claimed is:

1. A hand-held camera comprising:
an image sensor configured to capture an image;
a subject distance change determination circuitry configured to
detect a temporal change of a distance from an imaging position to each subject of a plurality of subjects present in the image, and
determine a tendency toward recession of the each subject of the plurality of subjects with respect to the imaging position on a basis of the temporal change of the distance from the imaging position to the each subject; and

a main subject selection circuitry configured to
select a first main subject on a basis of the tendency toward recession of the each subject of the plurality of subjects that has been determined by the subject distance change determination circuitry, wherein the first main subject has a tendency towards recession, and
responsive to selecting the first main subject, control a function of the hand-held camera to change the image captured by the image sensor, the function is one selected from a group consisting of:
an auto-focus function,
an exposure function, or
a zoom function; and

a control circuitry configured to
receive a still image capture input, and
responsive to receiving the still image capture input, control a memory to store a still image of the first main subject based on the main subject selection circuitry controlling the function of the hand-held camera to change the image captured by the image sensor.

2. The hand-held camera according to claim 1 ,
wherein the subject distance change determination circuitry is further configured to determine a tendency toward approach of the each subject to the imaging position on a basis of information regarding the temporal change of the distance of the each subject, and wherein the main subject selection circuitry is further configured to select a second main subject on a basis of only the tendency toward approach of the each subject.

3. The hand-held camera according to claim 1 , wherein, to detect the temporal change of the distance, the subject distance change determination circuitry is further configured to detect the temporal change of a size of a subject image in the image.

4. The hand-held camera according to claim 1 , wherein, to detect the temporal change of the distance, the subject distance change determination circuitry is further configured to detect the temporal change of a size of a subject image in the image, the subject image being one or more of candidate images extracted in the image.

5. The hand-held camera according to claim 2 ,
wherein the subject distance change determination circuitry is configured to determine that the each subject that is approaching on average, cumulatively, or continuously is a subject with the tendency toward approach, as a detection result of the temporal change of the distance, and
wherein the main subject selection circuitry is configured to select a part or all of the subject that is determined to have the tendency toward approach as the second main subject.

6. The hand-held camera according to claim 1 , further comprising:
an attribute identification circuitry configured to
identify an attribute of the subject, and
output attribute information on a basis of the attribute that has been identified,

wherein the subject distance change determination circuitry is configured to change a determination condition of the tendency toward recession of the subject according to the attribute information.

7. The hand-held camera according to claim 2 , wherein the subject distance change determination circuitry is configured to detect the temporal change of the distance of the each subject in each division region in the image.

8. The hand-held camera according to claim 7 , wherein the subject distance change determination circuitry is configured to
determine a division region at which the each subject is located in a second image before a unit time, and
detect the temporal change of the distance of the each subject as a difference between a first subject distance of the division region that has been determined and a second subject distance of a division region of the image that is a current processing target image.

9. The hand-held camera according to claim 7 ,
wherein the subject distance change determination circuitry is configured to determine that the each subject of the division region that is approaching on average, cumulatively, or continuously is a subject with the tendency toward approach, as a detection result of the temporal change of the distance, and
wherein the main subject selection circuitry is further configured to select the second main subject on a basis of information regarding the division region in which the subject is determined to have the tendency toward approach.

10. The hand-held camera according to claim 9 , wherein the main subject selection circuitry is configured to
group a different division region in which a same subject as the subject contained in the division region that is determined to have the tendency toward approach, and
set an image range serving as the second main subject based on a range of one or more division regions that have been grouped.

11. The hand-held camera according to claim 10 , wherein the main subject selection circuitry is configured to group the division region and the different division region for which a distance difference of a subject distance is within a predetermined value and which is a region continuous with the division region.

12. The hand-held camera according to claim 1 , wherein responsive to selecting the first main subject, the main subject selection circuitry is further configured to control a second function of the hand-held camera to change the image captured by the hand-held camera, the second function is one selected from a second group comprising:
an edit function,
an image effect function, or
a tracking function.

13. The hand-held camera according to claim 1 , wherein the control circuitry is further configured to
receive a video capture input, and
responsive to receiving the video capture input, control the memory to store a video of the first main subject based on controlling the function of the hand-held camera to change the image captured by the image sensor and in response to receiving the video capture input.

14. The hand-held camera according to claim 1 ,
wherein, to select the first main subject on the basis of the tendency toward recession of the each subject of the plurality of subjects that has been determined by the subject distance change determination circuitry, the main subject selection circuitry is further configured to select only one subject from the plurality of subjects as the first main subject.

15. A method for controlling a hand-held camera, the method comprising:
capturing, with an image sensor of the hand-held camera, an image;
detecting, with control circuitry of the hand-held camera, a temporal change of a distance from an imaging position to each subject of a plurality of subjects present in the image captured by the image sensor of the hand-held camera;
determining, with the control circuitry, a tendency toward recession of the each subject of the plurality of subjects with respect to the imaging position on a basis of the temporal change of the distance from the imaging position to the each subject;
selecting, with the control circuitry, a main subject on a basis of the tendency toward recession of the each subject of the plurality of subjects, wherein the main subject has a tendency towards recession;
controlling, with the control circuitry, a func

Assignees

• Sony Corp

Inventors

• Kinoshita Masaya

Classifications

• G02B7/365

  by analysis of the spatial frequency components of the image · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• H04N5/144 Primary

  Movement detection (for video coding H04N19/503; analysis of motion in general G06T7/20) · CPC title

• H04N23/69

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title

• H04N23/67

  Focus control based on electronic image sensor signals · CPC title