Mobile terminal and controlling method thereof for extracting available personal information corresponding to recognized faces

Patent metadata

Field	Value
Publication number	US-9622056-B2
Application number	US-201313900213-A
Country	US
Kind code	B2
Filing date	May 22, 2013
Priority date	Jun 7, 2012
Publication date	Apr 11, 2017
Grant date	Apr 11, 2017

Abstract

A mobile terminal including a wireless communication unit configured to wirelessly communicate with at least one other terminal; a memory configured to store a phonebook database including personal information and a profile image of each of a plurality of counterparts; a memory; a display unit; and a controller configured to recognize faces of people in an image displayed on the display unit, extract available personal information from the phonebook database of counterparts corresponding to the recognized faces, and perform a group call or a group chat, via the wireless communication unit, with the corresponding counterparts using the extracted personal information.

First claim

What is claimed is:

1. A mobile terminal comprising:
a wireless communication unit configured to wirelessly communicate with at least one other terminal;
a memory configured to store a phonebook database including personal information and a profile image of each of a plurality of counterparts;
a display unit configured to display an image stored in the memory; and
a controller configured to:
recognize faces of people in the image displayed on the display unit,
extract available personal information from the phonebook database of counterparts corresponding to the recognized faces,
select a group call or a group chat, via the wireless communication unit, with the corresponding counterparts using the extracted personal information,
in response to the selection of the group chat, control the display unit to display a conversation box of each counterparts corresponding to the recognized faces in a first region within the displayed image and control the display unit to display a name of each counterparts corresponding to the recognized faces in a second region within the displayed image,
when personal information of a counterpart among the recognized faces is failed to be extracted from the phonebook database, transmitting via the controller, a request for the personal information to a cloud server, accessing the cloud server and retrieving the personal information of a counterpart determined as the same character or person of the face of a specific counterpart from another phonebook database saved in a storage of a cloud account, and
when a second icon is dragged to a first icon or a face of the counterpart corresponding to the first icon, the controller is further configured to send a request for personal information of the counterpart corresponding to the second icon to the counterpart corresponding to the first icon, wherein the first icon indicates the face is recognized while the second icon indicates a failure to recognize the face.

2. The mobile terminal of claim 1 , wherein the controller is further configured to determine whether the recognized faces correspond to counterparts included the phonebook database, and
wherein counterparts that are included in the phonebook database are first recognized faces and counterparts that are not included in the phonebook database are second recognized faces.

3. The mobile terminal of claim 2 , wherein the controller is further configured to identify the first recognized faces using a first icon displayed near the first recognized faces, and identify the second recognized faces using a second icon displayed near the second recognized faces.

4. The mobile terminal of claim 3 , wherein if a corresponding second icon is selected, the controller is further configured to create a new corresponding counterpart having the second recognized face as a profile image in the phonebook database.

5. The mobile terminal of claim 2 , wherein the controller is further configured to access the cloud account of at least one of the counterparts corresponding to the first recognized faces to retrieve the personal information of at least one counterpart corresponding to the second recognized faces.

6. The mobile terminal of claim 2 , wherein the controller is further configured to access an SNS account or a homepage of at least one of the counterparts corresponding to the first recognized faces to retrieve the personal information of at least one counterpart corresponding to the second recognized faces.

7. The mobile terminal of claim 1 , wherein the controller is further configured to display on the display unit the image including the faces of the people as a background image when performing the group call or group chat.

8. The mobile terminal of claim 7 , wherein the controller is further configured to display a success indication near a recognized face included in the image for which a call connection has been successful and to display a fail indication near a recognized face included in the image for which a call connection has not been successful.

9. The mobile terminal of claim 7 , wherein while the group call is in progress, the controller is further configured to control the display unit to visually indicate a person currently speaking.

10. The mobile terminal of claim 7 , wherein while the group call is in progress, the controller is further configured to control the display unit to display language captions for speech included in the call.

11. The mobile terminal of claim 7 , wherein while the group call is in progress, and if a specific recognized face is selected from the image, the controller is further configured to terminate the call with the person corresponding to the specific recognized face.

12. The mobile terminal of claim 7 , wherein while the group call is in progress, and if a specific recognized face is selected from the image, the controller is further configured to initiate a separate message communication with only the person corresponding to the specific recognized face.

13. The mobile terminal of claim 7 , wherein while the group call is in progress, and if a specific recognized face is selected from the image, the controller is further configured to initiate a separate message communication with only persons excluding the person corresponding to the recognized face.

14. The mobile terminal of claim 7 , wherein while the group call is in progress, the controller is further configured to control the display unit to visually distinguish a recognized face corresponding to a person having sent a message.

15. The mobile terminal of claim 1 , wherein the image includes a combination of profile images included in the phonebook database.

16. The mobile terminal of claim 1 , further comprising:
a camera configured to capture a real-time image as corresponding to the image including the recognized faces.

17. A method of controlling a mobile terminal, the method comprising:
storing, in a memory, a phonebook database including personal information and a profile image of each of a plurality of counterparts;
recognizing, via a controller, faces of people in an image displayed on a display unit;
extracting, via the controller, available personal information from the phonebook database of counterparts corresponding to the recognized faces;
selecting, via the controller controlling a wireless communication unit, a group call or a group chat with the corresponding counterparts using the extracted personal information;
in response to the selection of the group chat, control the display unit to display a conversation box of each counterparts corresponding to the recognized faces in a first region within the displayed image and control the display unit to display a name of each counterparts corresponding to the recognized faces in a second region within the displayed image,
when personal information of a counterpart among the recognized faces is failed to be extracted from the phonebook database, transmitting via the controller, a request for the personal information to a cloud server, accessing cloud server and retrieving personal information of a counterpart determined as the same character or person of the face of a specific counterpart from another phonebook database saved in a storage of a cloud account; and
when a second icon is dragged to a first icon or a face of the counterpart corresponding to the first icon, the controller is further configured to send a request for personal information of the counterpart corresponding to the second icon to the counterpart corresponding to the first icon, wherein the first icon indicates the fac

Assignees

• Lg Electronics Inc

Inventors

• Hwang Myunghee
• Lee Kimoon
• Hwang Soojeong
• Lee Nalin

Classifications

• H04M2250/62

  user interface aspects of conference calls · CPC title

• H04M1/2757

  by data transmission, e.g. downloading · CPC title

• H04M1/27475

  using interactive graphical means or pictorial representations · CPC title

• H04W4/16 Primary

  Communication-related supplementary services, e.g. call-transfer or call-hold · CPC title

• G06K9/00221

  Physics · mapped topic