Method for user function operation based on face recognition and mobile terminal supporting the same

Patent metadata

Field	Value
Publication number	US-9239949-B2
Application number	US-201314144950-A
Country	US
Kind code	B2
Filing date	Dec 31, 2013
Priority date	Jan 7, 2013
Publication date	Jan 19, 2016
Grant date	Jan 19, 2016

Abstract

A mobile device user interface method activates a camera module to support a video chat function and acquires an image of a target object using the camera module. In response to detecting a face in the captured image, the facial image data is analyzed to identify an emotional characteristic of the face by identifying a facial feature and comparing the identified feature with a predetermined feature associated with an emotion. The identified emotional characteristic is compared with a corresponding emotional characteristic of previously acquired facial image data of the target object. In response to the comparison, an emotion indicative image is generated and the generated emotion indicative image is transmitted to a destination terminal used in the video chat.

First claim

What is claimed is:

1. A method in a mobile terminal, comprising:
executing, by at least one processor, a chat function in communication with a destination terminal over a network;
acquiring, by a camera of the mobile terminal, an image of a target object;
detecting, by the at least one processor of the mobile terminal, whether a face is present in the acquired image, and if a face is present, analyzing by the processor the acquired image to extract facial image data and identify an emotional characteristic of the face;
comparing, by the at least one processor, the identified emotional characteristic with a corresponding emotional characteristic identified from a previously acquired facial image data for the target object;
generating by the at least one processor an emotion indicative image based on the comparing; and
displaying the generated emotion indicative image in the chat function and transmitting, by a communication unit of the mobile terminal, the generated emotion indicative image through the network to the destination terminal for display on a display screen of the destination terminal.

2. The method of claim 1 , further comprising
identifying, by the at least one processor, a facial feature from the facial image data and comparing the identified facial feature with a predetermined feature associated with an emotion by feature boundary edge detection comprising a determined transition in pixel luminance; and
comparing the identified facial feature with the predetermined feature associated with an emotion in mapping information, using a transformation of the identified facial features comprising at least one of, translation, scaling and rotation operations; and
transmitting by the communication unit, when the identified emotional characteristic is identical to a previous emotional characteristic, a message indicating similarity between the identified emotional characteristic and the previous emotional characteristics without generating and transmitting another emotion indicative image.

3. The method of claim 1 , wherein activating the camera is executed in response to at least one of: expiration of a time period duration in the chat, activation of a message application supporting the chat, generation of an input event for writing a message during the chat, generation of an input event for transmitting a written message during the chat, or generation of an input event for reading or replying to a received message.

4. The method of claim 1 , wherein the acquiring of the image of a target object comprises at least one of:
selecting an image from among a sequence of preview images provided by the camera; and
acquiring an image of the target object after output of a predetermined notification.

5. The method of claim 1 , wherein the emotion indicative image is generated with a predetermined size by adjusting resolution and size of the acquired image.

6. The method of claim 5 , wherein the generating of the emotion indicative image further comprises inserting, by the at least one processor, textual information concerning the identified emotional characteristic into the emotion indicative image.

7. The method of claim 1 , further comprising:
in response to a determination that the acquired image does not contain a face of the target object, generating a message indicating absence of the target object; and
transmitting the message indicating the absence of the target object to the destination terminal.

8. The method of claim 1 , further comprising:
transmitting an original emotion indicative image to the destination terminal in response to receiving an original image request.

9. The method of claim 2 ,
wherein the camera is activated in response to a predetermined command;
wherein the facial feature is identified by comparing the facial feature with a predetermined facial feature associated with an emotion;
wherein similarity is detected between the identified emotional characteristic of the face and t a previously determined emotional characteristic of a face in a previously acquired image; and
wherein the step of generating comprises in response to the identified emotional characteristic being dissimilar to the previously determined emotional characteristic, generating an emotion indicative image based on the acquired image and transmitting the generated emotion indicative image to a counterpart terminal, and in response to the identified emotional characteristic being similar to the previously determined emotional characteristic, transmitting a message indicating similarity between the identified emotional characteristic and the previously determined emotional characteristic.

10. The method of claim 1 , wherein the step of activating comprises activating the camera in response to generation of an event, the method further comprising outputting the emotional characteristic and an emotion indicative image in response to the analyzing the facial image data.

11. The method of claim 10 , further comprising:
analyzing the facial image data to identify an emotional characteristic of the face by identifying a facial feature;
comparing the identified feature with a predetermined feature associated with an emotion; and
transmitting the emotional characteristic and emotion indicative image to destination terminal.

12. The method of claim 10 , further comprising:
storing the emotional characteristic and emotion indicative image; and
temporarily storing the emotion indicative image as an original image.

13. The method of claim 10 , wherein the emotional characteristic and emotion indicative images are produced by the mobile terminal or are received from the destination terminal.

14. The method of claim 10 , wherein camera is activated in response to at least one of: activation of a user function, writing of a message during a chat session, issuance of a message transmission request during a chat session, and collection of a voice signal during a call.

15. A mobile terminal, comprising:
a camera to acquire an image of a target object;
a communication unit communicatively coupled to a destination terminal over a network; and
at least one processor configured to:
execute a chat function with the destination terminal over the network and acquire the image of the target object with the camera during the chat function,
detect whether a face is present in the acquired image,
if the face is present, analyze the acquired image to extract facial image data and identify an emotional characteristic of the face by identifying a facial feature of the face from the facial image data, and compare the identified facial feature with a predetermined facial feature associated with an emotion,
determining similarity between the identified emotional characteristic of the face and a previously determined emotional characteristic of the face in a previously acquired image,
in response to the identified emotional characteristic being dissimilar to the previously determined emotional characteristic, generate an emotion indicative image based on the acquired image, displaying the generated emotion indicative image in the chat function, and transmit by the communication unit the generated emotion indicative image over the network for display on a display screen of the destination terminal, and
in response to the identified emotional characteristic being similar to the previously determined emotional characteristic, transmitting by the communication unit a message indicating similarity between the identified emotional characteristic and the previously determined emotional characteristic over the network.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jung Jeehye
• Choi Bokun
• Kang Doosuk
• Lee Changho
• Yim Sae Mee
• Jung Euichang

Classifications

• H04L12/1827 Primary

  Network arrangements for conference optimisation or adaptation · CPC title

• H04N21/4788

  communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)} · CPC title

• B62J1/007

  Saddles with specific anatomical adaptations · CPC title

• H04N21/44008

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• B62J1/002

  Saddles having a seating area with a central cavity or depression · CPC title