Mobile home robot and controlling method of the mobile home robot

Patent metadata

Field	Value
Publication number	US-2019202062-A1
Application number	US-201916239898-A
Country	US
Kind code	A1
Filing date	Jan 4, 2019
Priority date	Jan 4, 2018
Publication date	Jul 4, 2019
Grant date	—

Abstract

A mobile home robot is provided. The mobile home robot includes a storage configured to store in-home map data, a communication interface comprising communication interface circuitry, a camera, a user interface, and a processor configured to, based on device information of a plurality of Internet of things (IoT) devices installed in the home and an image captured through the camera while the mobile home robot moves around in the home, generate location information of each of the plurality of IoT devices, to map the generated location information with the map data, and in response to a user command being received through the user interface, to provide an IoT device location-based service based on the map data with which the location information is mapped.

First claim

What is claimed is:

1 . A mobile home robot, comprising:
a storage configured to store in-home map data; a communication interface comprising communication interface circuitry; a camera; a user interface; and a processor configured to:
generate location information of each of a plurality of Internet of Things (IoT) devices based on device information of the plurality of IoT devices installed in the home and an image captured through the camera while the mobile home robot moves around in the home;
map the generated location information with the map data;
provide an IoT device location-based service based on the map data with which the location information is mapped in response to a user command being received through the user interface.

2 . The mobile home robot as claimed in claim 1 , wherein the processor is configured to:
obtain an image of each of the plurality of IoT devices from an external server based on the device information; recognize each of the plurality of IoT devices by comparing the obtained image with the captured image; and generate location information of the each of the recognized plurality of IoT devices based on the captured image.

3 . The mobile home robot as claimed in claim 1 , wherein the camera includes at least one of: a three-dimensional (3D) camera and a stereo camera, and
wherein the location information includes at least two of: coordinate information of each of the plurality of IoT devices on the map data, size information of each of the plurality of IoT devices, direction information of each of the plurality of IoT devices, and information about a height at which each of the plurality of IoT devices is located.

4 . The mobile home robot as claimed in claim 1 , wherein the user interface includes a microphone, and
wherein the processor is configured to:
map semantic information regarding the location information of each of the plurality of IoT devices with the map data; and
perform an operation corresponding to the user command using the map data with which the semantic information is mapped based on a user voice command being received through the microphone.

5 . The mobile home robot as claimed in claim 1 , wherein the processor is configured to update the map data and the location information of each of the plurality of IoT devices on a predetermined cycle based on information obtained while the mobile home robot moves around in the home.

6 . The mobile home robot as claimed in claim 1 , wherein the processor is configured to transmit the location information and/or the map data with which the location information is mapped to a smartphone of a user and/or at least one of the plurality of IoT devices.

7 . The mobile home robot as claimed in claim 1 , wherein the processor is configured to:
generate a control command corresponding to a user command based on a position of a user inputting the user command and location information of one IoT device based on a user command for controlling the one IoT device from among the plurality of IoT devices being received; and transmit the generated control command to the one IoT device.

8 . The mobile home robot as claimed in claim 1 , wherein the processor is configured to:
determine whether it is appropriate that one IoT device performs an operation corresponding to the user command based on a distance between a user inputting the user command and the one IoT device based on a user command for controlling the one IoT device from among the plurality of IoT devices being received; transmit a control command corresponding to the user command to the one IoT device based on determining that it is appropriate that the one IoT device performs an operation corresponding to the user command; and provide a notification that the one IoT device is not capable of performing the user command through the user interface based on determining that it is inappropriate that the one IoT device performs the operation corresponding to the user command.

9 . The mobile home robot as claimed in claim 1 , wherein the processor is configured to:
determine context information of a user based on a user image captured through the camera; and transmit a control command for changing an operation mode of at least one IoT device from among the plurality of IoT devices to the at least one IoT device.

10 . The mobile home robot as claimed in claim 1 , wherein the user interface includes a microphone and a speaker, and
wherein the processor is configured to:
obtain device information of a new IoT device from an external server based on an image of the new IoT device captured through the camera in response to a user voice command for registration of a new IoT device being received; and
provide an interactive service for registration of the new IoT device based on the obtained device information of the new IoT device.

11 . A method of controlling a mobile home robot, the method comprising:
generating location information of each of a plurality of Internet of Things (IoT) devices based on device information of a plurality of IoT devices installed in the home and an image captured through the camera while the mobile home robot moves around in the home; mapping the generated location information with the map data; in response to a user command, and providing an IoT device location-based service based on the map data with which the location information is mapped.

12 . The method as claimed in claim 11 , wherein the generating the location information comprises:
obtaining an image of each of the plurality of IoT devices from an external server based on the device information; recognizing each of the plurality of IoT devices by comparing the obtained image with the captured image; and generating location information of the recognized each of the plurality of IoT devices based on the captured image.

13 . The method as claimed in claim 11 , wherein the camera includes at least one of: a three-dimensional (3D) camera and a stereo camera, and
wherein the location information includes at least two of: coordinate information of each of the plurality of IoT devices on the map data, size information of each of the plurality of IoT devices, direction information of each of the plurality of IoT devices, and information about a height at which each of the plurality of IoT devices is located.

14 . The method as claimed in claim 11 , further comprising:
mapping semantic information regarding the location information of each of the plurality of IoT devices with the map data, wherein the providing the IoT device location-based service comprises, performing an operation corresponding to the user voice command using the map data with which the semantic information is mapped based on a user voice command being received.

15 . The method as claimed in claim 11 , further comprising:
updating the map data and the location information of each of the plurality of IoT devices on a predetermined cycle based on information obtained while the mobile home robot moves around in the home.

16 . The method as claimed in claim 11 , further comprising:
transmitting the location information and/or the map data with which the location information is mapped to a smartphone of a user and/or at least one of the plurality of IoT devices.

17 . The method as claimed in claim 11 , wherein the providing the IoT device location-based service comprises:
generating a control command corresponding to the user command based on a position of a user inputting the user command and location information of one

Assignees

• Samsung Electronics Co Ltd

Inventors

• Park Chan-Ju
• Kim Eun-Seok

Classifications

• H04W4/70

  Services for machine-to-machine communication [M2M] or machine type communication [MTC] · CPC title

• H04L67/12

  specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks · CPC title

• H04W4/33

  for indoor environments, e.g. buildings · CPC title

• H04W4/021

  Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences · CPC title

• H04W4/023

  using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds · CPC title