User terminal device and method for recognizing user'S location using sensor-based behavior recognition

Patent metadata

Field	Value
Publication number	US-10341981-B2
Application number	US-201615754512-A
Country	US
Kind code	B2
Filing date	Jul 12, 2016
Priority date	Sep 2, 2015
Publication date	Jul 2, 2019
Grant date	Jul 2, 2019

Abstract

Disclosed is a system for recognizing a user location using sensor-based activity recognition. The system includes a mobile device which includes a sensor module configured to sense information about at least one of a user state and surrounding environments, a first processor configured to extract activity information based on the sensed information, and a first communication module configured to transmit the extracted activity information; and a user terminal device which includes a second communication module configured to receive the activity information from the first communication module, and a processor configured to determine a user location in a user space corresponding to a user activity based on the received activity information. By determining a user's location in a user space corresponding to a user activity, it is possible to control electronic devices corresponding to the user activity or provide a service of monitoring activities of the old and weak and the child.

First claim

The invention claimed is:

1. A user terminal device for recognizing a user location using sensor-based activity recognition, the user terminal device comprising:
a communication interface circuitry (CIC) configured to receive activity information from a mobile device that senses information about at least one of a user state and surrounding environments, and extracts the activity information based on the sensed information, and configured to perform Bluetooth communication and Wi-Fi communication; and
a processor configured to:
identify a user location in a user space corresponding to a user activity based on the received activity information,
measure a distance between the mobile device and the user terminal device based on signal strength of the Bluetooth communication with the mobile device, and
based on a user and the user terminal device being identified to be in the same location when the measured distance is within a predetermined range, identifying a user terminal location obtained by a Wi-Fi-based positioning system (WPS) as a user location.

2. The user terminal device according to claim 1 , wherein the mobile device comprises at least one first sensor comprising at least one of an acceleration sensor, a microphone, a heart-beat pulse sensor, a gyroscope, a geomagnetic sensor, a gravity sensor, an optical sensor, a digital compass, an odor sensor, or a proximity sensor.

3. The user terminal device according to claim 2 , wherein the user terminal device comprises at least one second sensor comprising at least one of an acceleration sensor, a microphone, a gyroscope, a geomagnetic sensor, a gravity sensor, an optical sensor, a digital compass, an odor sensor, or a proximity sensor.

4. The user terminal device according to claim 3 , wherein the mobile device extracts first activity information based on first sensed information sensed by the first sensor, and
the processor extracts second activity information based on second sensed information sensed by the second sensor, and identifies a user location based on the first activity information and the second activity information.

5. The user terminal device according to claim 2 , wherein the activity information comprises an activity fingerprint generated based on raw sense data obtained from the first sensor.

6. The user terminal device according to claim 5 , wherein the activity fingerprint is generated by a clustering algorithm based on features extracted from the raw sense data.

7. The user terminal device according to claim 5 , wherein the processor identifies a user location using a statistics Markov model based on the activity fingerprint.

8. The user terminal device according to claim 7 , further comprising a storage configured to store user activity history data in which user activity corresponding to the activity information is associated with activity time.

9. The user terminal device according to claim 1 , wherein the processor further comprises a storage configured to store an activity fingerprint corresponding to the user location as a learning model in association with the user location.

10. A system for recognizing a user location using sensor-based activity recognition, the system comprising:
a mobile device which comprises at least one sensor configured to sense information about at least one of a user state and surrounding environments, a first processor configured to extract activity information based on the sensed information, and a first communication interface circuitry (CIC) configured to transmit the extracted activity information and to perform Bluetooth communication; and
a user terminal device which comprises a second CIC configured to receive the activity information from the first CIC, and to perform the Bluetooth communication and Wi-Fi communication, and a second processor configured to:
identify a user location in a user space corresponding to a user activity based on the received activity information,
measure a distance between the mobile device and the user terminal device based on signal strength of the Bluetooth communication with the mobile device, and
based on a user and the user terminal device being identified to be in the same location when the measured distance is within a predetermined range, identifying a user terminal location obtained by a Wi-Fi-based positioning system (WPS) as a user location.

11. A system for recognizing a user location using sensor-based activity recognition, the system comprising:
a first mobile device which comprises at least one sensor configured to sense information about at least one of a user state and surrounding environments, a first processor configured to extract activity information based on the sensed information, and a first communication interface circuitry (CIC) configured to transmit the extracted activity information and to perform Bluetooth communication;
a second mobile device which comprises a second CIC configured to perform Wi-Fi communication and Bluetooth communication, and a second processor configured to obtain its own location information through the Wi-Fi communication and obtain distance information from the first mobile device through the Bluetooth communication; and
a user terminal device which comprises a third CIC configured to receive the activity information from the first CIC and the location information and the distance information from the second CIC, and a third processor configured to:
identify the location information as a user location when the first mobile device and the second mobile device are within a predetermined distance based on the distance information,
identify a user location in a user space corresponding to a user activity based on the received activity information when the first mobile device and the second mobile device are beyond a predetermined distance,
measure a distance between the first mobile device and the user terminal device based on signal strength of the Bluetooth communication with the first mobile device, and
based on a user and the user terminal device being identified to be in the same location when the measured distance is within a predetermined range, identifying a user terminal location obtained by a Wi-Fi-based positioning system (WPS) as a user location,

wherein the user terminal device is the same device as the second mobile device or a device different from the second mobile device.

12. A method of recognizing a user location using sensor-based activity recognition, the method comprising:
by a mobile device, sensing information about at least one of a user state and surrounding environments
extracting activity information based on the sensed information;
transmitting the activity information to a user terminal device; and
identifying a user location in a user space corresponding to the activity information based on the received activity information,
wherein the identifying comprises:
measuring a distance between the mobile device and the user terminal device based on signal strength of Bluetooth communication with the mobile device, and
based on a user and the user terminal device being identified to be in the same location when the measured distance is within a predetermined range, identifying a user terminal location obtained by a Wi-Fi-based positioning system (WPS) as a user location.

13. A non-transitory recording medium for recording a program of identifying the user location in the user space corresponding to the user activity based on the activity information according to claim 12 .

Assignees

• Samsung Electronics Co Ltd
• Univ Yonsei Iacf

Inventors

• Yu Seung-Hak
• Lee Seung-Woo
• Cha Ho-Jung
• Ko Yo-Han
• Kim Soo-Hwan
• Kim Hyun-Choong
• Shin Jong-Hoon
• Lee Kyoung-Woo
• Hahm Seong-Il

Classifications

• H04M2250/02

  including a Bluetooth® interface · CPC title

• H04M2250/12

  including a sensor for measuring a physical value, e.g. temperature or motion · CPC title

• H04W4/80

  Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication · CPC title

• H04L29/08

  Electricity · mapped topic

• H04W64/00 Primary

  Locating users or terminals {or network equipment} for network management purposes, e.g. mobility management · CPC title