Apparatus and method for estimating gazed position of person

Patent metadata

Field	Value
Publication number	US-2016247272-A1
Application number	US-201615050725-A
Country	US
Kind code	A1
Filing date	Feb 23, 2016
Priority date	Feb 25, 2015
Publication date	Aug 25, 2016
Grant date	—

Abstract

An image processing apparatus comprises: a detecting unit to detect, respectively from plural images obtained by imaging with plural imaging units a space in which plural persons exist, positions of the persons on the images; an associating unit to perform association of a same person in the plural images; a person position estimating unit to estimate positions of the persons in the space, based on the positions of the persons on the images detected from the plural images and a result of the association by the associating unit; a direction estimating unit to estimate eye directions of the persons, based on the plural images; and a distribution estimating unit to estimate a spatial distribution of positions gazed by the plural persons, based on the estimated positions of the plural persons in the space and the estimated eye directions of the plural persons.

First claim

What is claimed is:

1 . An image processing apparatus comprising;
a detecting unit configured to detect, respectively from plural images obtained by imaging with plural imaging units a space in which plural persons exist, positions of the persons on the images; an associating unit configured to perform association of a same person in the plural images; a person position estimating unit configured to estimate positions of the persons in the space, based on the positions of the persons on the images detected from the plural images and a result of the association by the associating unit; a direction estimating unit configured to estimate eye directions of the persons, based on the plural images; and a distribution estimating unit configured to estimate a spatial distribution of positions gazed by the plural persons, based on the estimated positions of the plural persons in the space and the estimated eye directions of the plural persons.

2 . The image processing apparatus according to claim 1 , further comprising a gazing level estimating unit configured to estimate a gazing level for each object, based on the estimated spatial distribution of the gazed positions.

3 . The image processing apparatus according to claim 1 , further comprising an obtaining unit configured to obtain respective imaging parameters of the plural imaging units.

4 . The image processing apparatus according to claim 3 , wherein the person position estimating unit estimates the position of the person in the space, based on the positions of the persons on the image and the respective imaging parameters of the plural imaging units.

5 . The image processing apparatus according to claim 3 , wherein the distribution estimating unit estimates the spatial distribution of the positions gazed by the persons, based on the positions of the persons in the space, the eye directions and the respective imaging parameters of the plural imaging units.

6 . The image processing apparatus according to claim 3 , wherein the obtaining unit obtains internal parameters and external parameters of the plural imaging units calculated from the plural images respectively imaged by the plural imaging units, as the imaging parameters.

7 . The image processing apparatus according to claim 6 , wherein the internal parameter includes at least one of information indicating a central position of the imaging unit in an image coordinate system, information indicating a magnification, and information indicating distortion of two axes of image.

8 . The image processing apparatus according to claim 6 , wherein the external parameter includes information indicating a position or an orientation of the imaging unit.

9 . The image processing apparatus according to claim. 1 , wherein the direction estimating unit estimates the eye direction of the person, based on a partial image representing a human body or a part thereof in each of the plural images.

10 . The image processing apparatus according to claim 9 , wherein the direction estimating unit uses a partial image of at least one of the human body, a face and eyes, as the partial image representing the human body or the part thereof in each of the plural images.

11 . The image processing apparatus according to claim 1 , further comprising a deciding unit configured to decide whether or not the association is correct, based on the position of the person in the space and the eye direction, wherein
the distribution estimating unit estimates the spatial distribution of the positions gazed by the persons, based on the positions of the persons in the space of which the association has been decided as being correct by the deciding unit, and the eye directions.

12 . The image processing apparatus according to claim 11 , wherein the deciding unit calculates a value indicating consistency of the association, and decides that the association is correct, in a case where the calculated value is larger than a threshold.

13 . The image processing apparatus according to claim 11 , wherein the deciding unit calculates a value indicating consistency of the association by integrating a value indicating consistency of the positions of the persons in the space and a value indicating consistency of the eye directions.

14 . An image processing method comprising:
detecting, respectively from plural images obtained by imaging with plural imaging units a space in which plural persons exist, positions of the persons on the images; performing association, of a same person in the plural images; estimating positions of the persons in the space, based on the positions of the persons on the images detected from the plural images and a result of the association; estimating eye directions of the persons, based on the plural images; and estimating a spatial distribution of positions gazed by the plural persons, based on the estimated positions of the plural persons in the space and the estimated eye directions of the plural persons.

15 . A non-transitory computer-readable storage medium which stores a program to cause a computer to perform:
a detecting step of detecting, respectively from plural images obtained by imaging with plural imaging units a space in which plural persons exist, positions of the persons on the images; an associating step of performing association of a same person in the plural images; a person position estimating step of estimating positions of the persons in the space, based on the positions of the persons on the images detected from the plural images and a result of the association in the associating step; a direction estimating step of estimating eye directions of the persons, based on the plural images; and a distribution estimating step of estimating a spatial distribution of positions gazed by the plural persons, based on the estimated positions of the plural persons in the space and the estimated eye directions of the plural persons.

Assignees

• Canon Kk

Inventors

• Nishino Daisuke
• Yano Kotaro
• Kaneda Yuji
• Uchiyama Hiroyuki

Classifications

• G06K9/00604

  Physics · mapped topic

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06T2207/30201

  Face · CPC title

• G06T7/004 Primary

  Physics · mapped topic

• G06T2207/30196

  Human being; Person · CPC title