Orientation estimation method, and orientation estimation device

Patent metadata

Field	Value
Publication number	US-2017154441-A1
Application number	US-201515322911-A
Country	US
Kind code	A1
Filing date	Jul 29, 2015
Priority date	Aug 6, 2014
Publication date	Jun 1, 2017
Grant date	—

Abstract

An orientation estimation device includes a processor. The processor receives the analysis target image and sets a plurality of reference positions including a head position and a waist position of a person with respect to an input analysis target image. A candidate region of a part region in an analysis target image is determined based on a joint base link model in which an orientation of a person is defined by an arrangement of a plurality of point positions including the head position and the waist position and a plurality of the part regions and the plurality of set reference positions. It is determined whether the person included in the analysis target image takes the orientation or not based on a part image feature which is an image feature of a part region in an image obtained by photographing a person and an image feature of the determined candidate region.

First claim

1 . An orientation estimation method comprising:
causing a processor that estimates an orientation of a person within an analysis target image, to receive the analysis target image, to set a plurality of reference positions including a head position and a waist position of a person with respect to an input analysis target image, to determine a candidate region of a part region in an analysis target image based on a joint base link model in which an orientation of a person is defined by an arrangement of a plurality of point positions including the head position and the waist position and a plurality of the part regions and the plurality of set reference positions, and to determine whether the person included in the analysis target image takes the orientation or not based on a part image feature which is an image feature of a part region in an image obtained by photographing a person and an image feature of the determined candidate region.

2 . The orientation estimation method of claim 1 , further comprising:
causing the processor to display the analysis target image, to receive a drag-and-drop operation with respect to the displayed analysis target image, and to respectively set a start point and an end point of the drag-and-drop operation with respect to the analysis target image as the head position and the waist position to set the reference position.

3 . The orientation estimation method of claim 1 ; further comprising:
causing the processor to determine the candidate region regarding each of the plurality of part regions, to calculate likelihood per part representing certainty that the candidate region is a corresponding part region for each of a plurality of the candidate regions, and to determine whether the person included in the analysis target image takes the orientation or not based on some or all of a plurality of the calculated likelihoods per part.

4 . The orientation estimation method of claim 1 ,
wherein the joint base link model includes a combination of a plurality of state variables that define the arrangement, and wherein the method further comprises causing the processor to change a value of the state variable and determine a relative positional relationship between the plurality of point positions and the plurality of part regions for each of a plurality of the orientations, to determine the plurality of candidate regions based on the determined relative positional relationship for each of the plurality of orientations and the plurality of set reference positions, and to determine, for each of the plurality of orientations, a candidate orientation which is an orientation having a high possibility that a person included in the analysis target image takes with respect to the plurality of determined candidate regions.

5 . The orientation estimation method of claim 4 , further comprising:
causing the processor to determine the candidate region using a hyperplane which is restrained by the plurality of reference positions of low-dimensional orientation state space obtained by reducing dimensions of orientation state space which has the plurality of state variables as axes by a main component analysis.

6 . The orientation estimation method of claim 4 , further comprising:
causing the processor to change the value of the state variable using the candidate orientation as a reference and determine the relative positional relationship of an additional candidate orientation approaching the candidate orientation, to determine an additional candidate region of each of the plurality of part regions in the analysis target image based on the relative positional relationship of the additional candidate orientation and the plurality of set reference positions, and to determine an orientation having a high possibility that the person included in the analysis target image takes with respect to the additional candidate region.

7 . The orientation estimation method of claim 6 , further comprising:
causing the processor to determine whether the values of the plurality of likelihoods per part satisfy a predetermined end condition, in a case where the predetermined end condition is not satisfied, to repeat processing of performing determination of the additional candidate region and determination of the additional orientation using the additional candidate orientation determined immediately before as a reference, in a case where the predetermined end condition is satisfied, to determine the additional candidate orientation determined lastly as the orientation that the person included in the analysis target image takes, and to output information indicating the determined orientation.

8 . An orientation estimation device comprising:
a processor, wherein the processor is configured to receive the analysis target image, set a plurality of reference positions including a head position and a waist position of a person with respect to an input analysis target image, determine a candidate region of a part region in an analysis target image based on a joint base link model in which an orientation of a person is defined by an arrangement of a plurality of point positions including the head position and the waist position and a plurality of the part regions and the plurality of set reference positions, and determine whether the person included in the analysis target image takes the orientation or not based on a part image feature which is an image feature of a part region in an image obtained by photographing a person and an image feature of the determined candidate region.

9 . The orientation estimation device of claim 8 ,
wherein the processor is configured to display the analysis target image, receive a drag-and-drop operation with respect to the displayed analysis target image, and respectively set a start point and an end point of the drag-and-drop operation with respect to the analysis target image as the head position and the waist position to set the reference position.

10 . The orientation estimation device of to claim 8 ,
wherein the processor is configured to determine the candidate region regarding each of the plurality of part regions, calculate likelihood per part representing certainty that the candidate region is a corresponding part region for each of a plurality of the candidate regions, and determine whether the person included in the analysis target image takes the orientation or not based on some or all of a plurality of the calculated likelihoods per part.

11 . The orientation estimation device of claim 8 ,
wherein the joint base link model includes a combination of a plurality of state variables that define the arrangement, and wherein the processor is configured to change a value of the state variable and determine a relative positional relationship between the plurality of point positions and the plurality of part regions for each of a plurality of the orientations, determine the plurality of candidate regions based on the determined relative positional relationship for each of the plurality of orientations and the plurality of set reference positions, and determine, for each of the plurality of orientations, a candidate orientation which is an orientation having a high possibility that a person included in the analysis target image takes with respect to the plurality of determined candidate regions.

12 . The orientation estimation device of claim 11 ,
wherein the processor is configured to determine the candidate region using a hyperplane which is restrained by the plurality of reference positions of low-dimensional orientation state space obtained by reducing dimensions of orien

Assignees

• Panasonic Corp

Inventors

• Kawaguchi Kyoko
• Sekii Taiki

Classifications

• G06T7/74 Primary

  involving reference images or patches · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06T2207/30221

  Sports video; Sports image · CPC title

• G06T7/277

  involving stochastic approaches, e.g. using Kalman filters · CPC title

• G06T2207/30196

  Human being; Person · CPC title