User intention analysis apparatus and method based on image information of three-dimensional space

Patent metadata

Field	Value
Publication number	US-2016335485-A1
Application number	US-201615092726-A
Country	US
Kind code	A1
Filing date	Apr 7, 2016
Priority date	May 13, 2015
Publication date	Nov 17, 2016
Grant date	—

Abstract

Provided are a user intention analysis apparatus and method based on image information of a three-dimensional (3D) space. The user intention analysis apparatus includes a 3D space generator configured to generate a 3D virtual space corresponding to an ambient environment, based on physical relative positions of a plurality of cameras and image information generated by photographing the ambient environment with the plurality of cameras, a 3D image analyzer configured to estimate a relative position between a first object and a second object included in the image information in the 3D virtual space and generate contact information of the first object and the second object, based on the relative positions of the first object and the second object, an action pattern recognizer configured to compare the contact information with a pre-learned action pattern to recognize an action pattern of a user who manipulates the first object or the second object, and a user intention recognizer configured to infer a user intention corresponding to the recognized action pattern, based on ontology.

First claim

What is claimed is:

1 . A user intention analysis apparatus based on image information of a three-dimensional (3D) space, the user intention analysis apparatus comprising:
a 3D space generator configured to generate a 3D virtual space corresponding to an ambient environment, based on physical relative positions of a plurality of cameras and image information generated by photographing the ambient environment with the plurality of cameras; a 3D image analyzer configured to estimate a relative position between a first object and a second object included in the image information in the 3D virtual space and generate contact information of the first object and the second object, based on the relative positions of the first object and the second object; an action pattern recognizer configured to compare the contact information with a pre-learned action pattern to recognize an action pattern of a user who manipulates the first object or the second object; and a user intention recognizer configured to infer a user intention corresponding to the recognized action pattern, based on ontology.

2 . The user intention analysis apparatus of claim 1 , wherein the 3D image generator generates a 3D virtual space corresponding to a 3D space where a voxel of a 3D space generated by a first camera which photographs the ambient environment in a first direction intersects a voxel of a 3D space generated by a second camera which photographs the ambient environment in a second direction.

3 . The user intention analysis apparatus of claim 1 , wherein the 3D image analyzer estimates 3D positions of the first object and the second object to detect a first feature point and a second feature point, which is to be used as landmarks, from the first object and the second object and matches the first feature point and the second feature point with a 3D model obtained by modeling the first object and the second object.

4 . The user intention analysis apparatus of claim 3 , wherein the 3D image analyzer traces positions of a 3D model of the first object and a 3D model of the second object obtained by matching the first feature point with the second feature point to generate contact information of the first object and the second object.

5 . The user intention analysis apparatus of claim 3 , wherein the 3D image analyzer assigns an identifier (ID) to an arbitrary point or region of each of the 3D model of the first object and the 3D model of the second object obtained by matching the first feature point with the second feature point, and generates n-dimensional contact information which includes IDs of corresponding nodes and information about whether the arbitrary point or region (hereinafter referred to as a node) to which the ID is assigned contacts other nodes.

6 . The user intention analysis apparatus of claim 3 , wherein the 3D image analyzer compares a predetermined threshold distance with a distance between nodes to generate contact information of the nodes which is classified into contact, adjacency, and noncontact.

7 . The user intention analysis apparatus of claim 1 , wherein the 3D image analyzer comprises:
a first sensing unit configured to generate first sensing information about an object, based on image information which is obtained by photographing the ambient environment in a first direction; a second sensing unit configured to generate second sensing information about the object, based on image information which is obtained by photographing the ambient environment in a second direction; a first scanner configured to generate first scan data corresponding to the first object; a second scanner configured to generate second scan data corresponding to the second object; a first matcher configured to match the first scan data with a feature point which is detected by combining the first and second sensing information and is used as a landmark in the first object; a second matcher configured to match the second scan data with a feature point which is detected by combining the first and second sensing information and is used as a landmark in the second object; and a contact determiner configured to generate contact information of a 3D model of the first object, which is output as a matched result from the first matcher, and a 3D model of the second object which is output as a matched result from the second matcher.

8 . The user intention analysis apparatus of claim 1 , wherein the action pattern recognizer comprises:
a user experience storage configured to store a learned user action pattern; an action pattern analyzer configured to classify the learned user action pattern from an n-dimensional vector included in the contact information; and an action pattern output unit configured to recognize and output the classified action pattern as a user action pattern corresponding to the contact information.

9 . The user intention analysis apparatus of claim 1 , wherein the user intention recognizer comprises:
a user experience analyzer configured to store information obtained by analyzing a correlation between a user action pattern and a user intention, based on ontology; a user intention determiner configured to determine a user intention corresponding to the user action pattern recognized by the action pattern recognizer, based on the correlation between the user action pattern and the user intention analyzed by the user experience analyzer; and a user experience learner configured to learn the correlation between the user intention determined by the user intention determiner and the user action pattern recognized by the action pattern recognizer and transfer a learned result to the user experience analyzer.

10 . A user intention analysis method based on image information of a three-dimensional (3D) space, the user intention analysis method comprising:
generating a 3D virtual space corresponding to an ambient environment, based on physical relative positions of a plurality of cameras and image information generated by photographing the ambient environment with the plurality of cameras; estimating a relative position between a first object and a second object included in the image information in the 3D virtual space to generate contact information of the first object and the second object, based on the relative positions of the first object and the second object; comparing the contact information with a pre-learned action pattern to recognize an action pattern of a user who manipulates the first object or the second object; and inferring a user intention corresponding to the recognized action pattern, based on ontology.

11 . The user intention analysis method of claim 10 , wherein the generating of the 3D virtual space comprises generating a 3D virtual space corresponding to a 3D space where a voxel of a 3D space generated by a first camera which photographs the ambient environment in a first direction intersects a voxel of a 3D space generated by a second camera which photographs the ambient environment in a second direction.

12 . The user intention analysis method of claim 10 , wherein the generating of the contact information comprises estimating 3D positions of the first object and the second object to detect a first feature point and a second feature point, which is to be used as landmarks, from the first object and the second object and matching the first feature point and the second feature point with a 3D model obtained by modeling the first object and the second object.

13 . The user intention analysis method of claim 12 , wherein the generating of the contact information comprises tracing positions of a 3D model of the first object and a

Assignees

• Electronics & Telecommunications Res Inst

Inventors

• Kim Jin Woo

Classifications

• G06V30/194

  References adjustable by an adaptive method, e.g. learning · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06K9/00335 Primary

  Physics · mapped topic

• G06K9/00845

  Physics · mapped topic

• G06K9/66

  Physics · mapped topic