Method and apparatus for recognizing target device, for augmented reality

Patent metadata

Field	Value
Publication number	US-12217504-B2
Application number	US-202318112726-A
Country	US
Kind code	B2
Filing date	Feb 22, 2023
Priority date	Aug 25, 2020
Publication date	Feb 4, 2025
Grant date	Feb 4, 2025

Abstract

An electronic device may include a display, a camera, a communication circuitry, and a processor, wherein the processor may be configured to: control to transmit an image obtained through the camera to an external device through the communication circuitry, receive an AR image including at least one object from the external device through the communication circuitry and display same through the display, recognize a target object from among the at least one object, and map one of peripheral devices found on a data communication link through the communication circuitry to a target device corresponding to the recognized target object. Various embodiments are possible.

First claim

The invention claimed is:

1. An electronic device comprising:
a display;
a camera;
a communication circuitry; and
at least one processor comprising processing circuitry; and
memory storing instructions that, when executed by the at least one processor, cause the electronic device to:
control to transmit an image obtained through the camera to an external device through the communication circuitry;
receive an augmented reality (AR) image including at least one object related to at least one peripheral device that exists around the electronic device from the external device through at least the communication circuitry;
control to display the AR image including the at least one object through the display;
recognize a target object from among the at least one object;
determine, based on signals transmitted by a plurality of peripheral devices on a data communication link, location information of the plurality of peripheral devices on the data communication link;
determine location information of the target object in the AR image; and
map, based on the determined location information of the target object in the AR image and the determined location information of the plurality of peripheral devices on the data communication link a peripheral device among the plurality of peripheral devices found on the data communication link to a target device corresponding to the recognized target object in the AR image.

2. The electronic device of claim 1 , wherein the instructions, when executed by the at least one processor, cause electronic device to:
estimate a distance and direction to the target object on the AR image,
identify whether a signal transmitted from the target device on the data communication link is received from the estimated distance and direction, and
map the target object and the target device when the signal transmitted from the target device is received from the estimated distance and direction.

3. The electronic device of claim 2 , wherein the instructions, when executed by the at least one processor, cause electronic device to:
perform a first operation for an angle of arrival (AoA) estimation for the target device by applying a designated first estimation algorithm, and
perform a second operation for the angle of arrival estimation at least by applying a second estimation algorithm designated based on an estimation result according to the performing of the first operation.

4. The electronic device of claim 3 , wherein the instructions, when executed by the at least one processor, cause electronic device to perform a channel impulse response (CIR) tab selection based on the signal from the target device.

5. The electronic device of claim 3 , wherein the instructions, when executed by the at least one processor, cause electronic device to select a tab of interest to be used for the angle of arrival estimation from among a plurality of tabs sampled at a predetermined interval for the signal from the target device,
wherein the tab of interest comprises a tab which corresponds to a value obtained at least by dividing the corresponding distance by a speed of an electromagnetic wave based on the distance estimated on the AR image.

6. The electronic device of claim 5 , wherein the instructions, when executed by the at least one processor, cause electronic device to:
generate a plurality of outputs at least by applying the first estimation algorithm to the tab of interest, and
apply the second estimation algorithm to the plurality of outputs,
wherein the first estimation algorithm comprises an algorithm for estimating a signal in a specific direction by beamforming, and the second estimation algorithm comprises an algorithm for separating a designated signal and noise.

7. An electronic device comprising:
a camera;
a display;
a communication circuitry; and
at least one processor comprising processing circuitry; and
memory storing instructions that, when executed by the at least one processor, cause the electronic device to:
control to transmit an image obtained through the camera to an external device through the communication circuitry;
receive an augmented reality (AR) image based on the transmitted image from the external device;
control to display the received AR image through the display;
estimate a distance and direction to a target object on the AR image displayed through the display based on sensing information using at least one sensor;
perform a tab selection among a plurality of channel impulse response tabs sampled at a predetermined interval based on a signal transmitted by the target object and the distance and direction estimated to the target object on the AR image;
apply a first estimation algorithm to the selected tab to obtain, for each antenna subset of a plurality of antennas of the communication circuitry, an output by using at least the estimated distance and direction; and
estimate an angle of arrival (AoA) at least by applying a second estimation algorithm to the outputs obtained by applying the first estimation algorithm to the antenna subsets.

8. The electronic device of claim 7 , wherein the first estimation algorithm comprises an algorithm for estimating a signal in a specific direction by beamforming, and the second estimation algorithm comprises an algorithm for separating a designated signal and noise.

9. The electronic device of claim 7 , wherein the instructions, when executed by the at least one processor, cause electronic device to:
recognize the target object on the image obtained through the camera and displayed through the display,
estimate a distance and direction to the recognized target object,
select a tab corresponding to the estimated distance and direction on the image as a tab of interest to be used for the angle of arrival estimation among a plurality of tabs sampled at a predetermined interval, and
provide the selected tab of interest as an input for applying the first estimation algorithm.

10. The electronic device of claim 7 , wherein the instructions, when executed by the at least one processor, cause electronic device to generate the antenna subset including a designated number of antennas based on the plurality of antennas.

11. The electronic device of claim 10 , wherein the instructions, when executed by the at least one processor, cause electronic device to generate the antenna subset by overlapping at least one antenna in each of the antenna subsets to have a maximum and/or large number of antenna subsets.

12. The electronic device of claim 11 , wherein the instructions, when executed by the at least one processor, cause electronic device to:
generate a plurality of outputs corresponding to the number of antenna subsets at least by applying the first estimation algorithm to the antenna subsets, and
apply the second estimation algorithm to the plurality of outputs.

13. The electronic device of claim 12 , wherein the instructions, when executed by the at least one processor, cause the electronic device to:
separate eigenvectors constituting a signal subspace and eigenvectors constituting a noise subspace for the plurality of outputs, and
complete the angle of arrival estimation based on the signal subspace.

14. A method of operating an electronic device, the method comprising:
obtaining an image through a camera of the electronic device;
transmitting at least a part of the obtained image to an external device;
receiving an augmented reality (AR) image including at least one object related to at least one peripheral device that exist around the electronic device based on the transmitted image, from the external device;
di

Assignees

• Samsung Electronics Co Ltd

Inventors

• Choi Junsu
• Min Hyunkee
• Sung Jihoon
• Lee Sunkee
• Lee Junghun

Classifications

• G06F3/0483

  Interaction with page-structured environments, e.g. book metaphor · CPC title

• G01S5/0218

  Multipath in signal reception · CPC title

• G01S13/89

  for mapping or imaging · CPC title

• G01S13/72

  for two-dimensional [2D] tracking, e.g. combination of angle and range tracking, track-while-scan radar · CPC title

• G01S5/04

  Position of source determined by a plurality of spaced direction-finders · CPC title