Robot, system comprising robot and user device and controlling method thereof

Patent metadata

Field	Value
Publication number	US-12533813-B2
Application number	US-202217570896-A
Country	US
Kind code	B2
Filing date	Jan 7, 2022
Priority date	Apr 20, 2021
Publication date	Jan 27, 2026
Grant date	Jan 27, 2026

Abstract

A robot is provided. The robot includes a plurality of sensors, a memory, a driving unit, and a processor configured to, based on identifying that a predetermined event occurs, control the driving unit to move the robot to a predetermined point, based on identifying that the robot has moved to the point, obtain a plurality of images through the sensors, identify whether to perform calibration for at least one sensor based on the obtained images, based on identifying to perform the calibration for the sensor, obtain calibration data for calibrating sensing data corresponding to the sensor based on the obtained images and store the obtained calibration data in the memory, based on the sensing data being obtained from the sensor, calibrate the obtained sensing data based on the calibration data stored in the memory, and control the driving unit based on the calibrated sensing data.

First claim

What is claimed is:

1 . A robot comprising:
a plurality of sensors including a camera for sensing optical signals; a memory configured to store a reference image; a driving unit; and a processor configured to:
based on identifying that a predetermined event has occurred, control the driving unit so that the robot moves to a predetermined point corresponding to the reference image, the predetermined event being related to calibration of the plurality of sensors,
based on identifying that the robot has moved to the predetermined point, obtain a plurality of images through the plurality of sensors,
obtain a similarity between the reference image and the obtained plurality of images,
identify whether it is necessary to perform calibration for at least one sensor of the plurality of sensors based on similarity between the reference image and the obtained plurality of images,
based on the similarity being equal to or less than a threshold value, obtain calibration data to calibrate sensing data corresponding to the at least one sensor based on the plurality of images and store the obtained calibration data in the memory,
based on the sensing data being obtained from the at least one sensor, calibrate the obtained sensing data based on the calibration data stored in the memory, and
control the driving unit based on the calibrated sensing data.

2 . The robot of claim 1 , wherein the processor is further configured to:
identify whether there is a mechanical distortion on at least one of the plurality of sensors based on the plurality of obtained images; and based on identifying that there is the mechanical distortion on the at least one sensor of the plurality of sensors, identify that it is necessary to perform calibration for the at least one sensor.

3 . The robot of claim 1 ,
wherein the processor is further configured to:
obtain a depth image based on a synthesis of the plurality of obtained images,
compare the reference image with the obtained depth image, and
identify whether it is necessary to perform the calibration for the at least one sensor of the plurality of sensors based on a result of the comparison.

4 . The robot of claim 1 , wherein the processor is further configured to:
obtain a depth image based on a synthesis of the plurality of obtained images; identify a flat surface based on the depth image; and identify whether it is necessary to perform the calibration for the at least one sensor of the plurality of sensors based on the identified flat surface.

5 . The robot of claim 4 , wherein the processor is further configured to identify the flat surface including a largest number of point clouds within a threshold distance of the robot by:
repeatedly extracting three random point clouds among a plurality of point clouds included in the depth image; identifying a flat surface including the three extracted point clouds; and calculating a number of point clouds located within the threshold distance from the identified flat surface.

6 . The robot of claim 1 , wherein the processor is further configured to, based on identifying that a predetermined time interval has elapsed or that an interval in which a predetermined number of tasks have been performed has arrived, identify that the predetermined event has occurred.

7 . The robot of claim 1 , wherein the processor is further configured to, based on identifying that the robot has docked at a charge station, identify that the robot has moved to the predetermined point.

8 . The robot of claim 1 , further comprising:
a distance sensor, wherein the processor is further configured to:
identify whether a dynamic object exists in a surrounding environment of the predetermined point based on sensing data obtained by the distance sensor, after identifying that the robot has moved to the predetermined point, and
based on identifying that the dynamic object exists, finish a calibration operation for the plurality of sensors.

9 . The robot of claim 1 , wherein the processor is further configured to:
identify whether any obstacles are within a specified range from the robot; and perform the calibration if it is identified that there are no obstacles within the specified range from the robot.

10 . The robot of claim 1 , further comprising:
a user interface; and a communication interface, wherein the processor is further configured to, based on a user command being received through at least one of the user interface or the communication interface, control the driving unit so that the robot moves to the predetermined point.

11 . The robot of claim 1 , wherein the processor is further configured to:
based on the calibration data being obtained, additionally obtain sensing data from the at least one sensor; obtain calibrated sensing data by applying the calibration data to the additionally obtained sensing data; and based on identifying that the calibrated sensing data is improved compared to the sensing data by a threshold value or more, store the obtained calibration data in the memory.

12 . The robot of claim 1 , wherein the processor is further configured to, based on an event in which a traveling mode of the robot is changed occurring after identifying that the robot has moved to the predetermined point, finish a calibration operation for the plurality of sensors.

13 . A system comprising:
a robot comprising a plurality of sensors including a camera for sensing optical signals; and a user terminal, wherein the user terminal is configured to, based on a user command for performing calibration for the plurality of sensors provided in the robot being input, transmit the user command to the robot; and wherein the robot is configured to:
based on the user command being received from the user terminal, obtain a similarity between a reference image stored in the robot and a plurality of images, identify whether it is necessary to perform calibration for at least one sensor of the plurality of sensors based on similarity between the plurality of images obtained from the plurality of sensors and the reference image for calibrating the at least one sensor while the robot moves to a predetermined point corresponding to the reference image for calibrating the at least one sensor and is located at the predetermined point, and
based on the similarity being equal to or less than a threshold value, obtain calibration data for calibrating sensing data corresponding to the at least one sensor based on the plurality of images and store the obtained calibration data in a memory.

14 . A method for controlling a robot including a camera for sensing optical signals and storing a reference image, the method comprising:
based on identifying that a predetermined event has occurred, moving the robot to a predetermined point corresponding to the reference image, the predetermined event being related to calibration of the plurality of sensors; based on identifying that the robot has moved to the predetermined point, obtaining a plurality of images through a plurality of sensors; obtaining a similarity between the reference image and the obtained plurality of images, identifying whether it is necessary to perform calibration for at least one sensor of the plurality of sensors based on similarity between the reference image and the obtained plurality of images; based on the similarity being equal to or less than a threshold value, obtaining and storing calibration data to calibrate sensing data corresponding to the at least one sensor based on the plurality of images; based on the sensing data being obtained from the at least on

Assignees

• Samsung Electronics Co Ltd

Inventors

• Ryu Minwoo
• Lee Donghun
• Kim Shin
• Chang Euiyoung
• Ha Changho

Classifications

• G05D1/0212

  with means for defining a desired trajectory (involving a plurality of land vehicles G05D1/0287) · CPC title

• B25J9/1692 Primary

  Calibration of manipulator · CPC title

• G05D1/661

  Docking at a base station (delivering or retrieving payloads G05D1/667) · CPC title

• G05D2111/64

  taken simultaneously from spaced apart sensors, e.g. stereovision · CPC title

• G05D2111/10

  Optical signals · CPC title