Electronic device for providing augmented reality service and operating method thereof

Patent metadata

Field	Value
Publication number	US-11222477-B2
Application number	US-202017092853-A
Country	US
Kind code	B2
Filing date	Nov 9, 2020
Priority date	Nov 25, 2019
Publication date	Jan 11, 2022
Grant date	Jan 11, 2022

Abstract

A device and a method of providing an augmented reality service in an electronic device by interworking with another electronic device is provided. A first electronic device includes a communication circuit, a camera, a sensor, a display device, and a processor, the display device, the sensor, and the camera, the processor configured to establish communication with a second electronic device, obtain information on at least one object included in a field of view of a user wearing the first electronic device through the camera or the sensor, obtain information on a user's gaze, determine whether to interwork with the second electronic device for an augmented reality service, based on the information on the user's gaze and the information on the at least one object, and when it is determined to interwork with the second electronic device, transmit interworking information on the augmented reality service to the second electronic device.

First claim

What is claimed is:

1. A first electronic device comprising:
a communication circuit;
a camera;
a sensor;
a display device; and
a processor operably connected to the communication circuit, the display device, the sensor, and the camera,
wherein the processor is configured to:
establish communication with a second electronic device through the communication circuit,
obtain information on at least one object corresponding to a field of view of a user wearing the first electronic device through at least one of the camera or the sensor,
obtain information on a user's gaze,
based on the information on the user's gaze and the information on the at least one object, determine whether to use the second electronic device as an input device for an augmented reality service, and
when it is determined to use the second electronic device as the input device for an augmented reality service, transmit information related to the input device for the augmented reality service to the second electronic device through the communication circuit.

2. The first electronic device of claim 1 , wherein the processor is further configured to obtain the information on the user's gaze, based on at least one of a head gaze or an eye gaze of the first electronic device.

3. The first electronic device of claim 1 , wherein the processor is further configured to control the display device to display at least one virtual object on at least a portion of an area corresponding to the field of view of the user wearing the first electronic device.

4. The first electronic device of claim 3 , wherein the processor is further configured to:
determine to use the second electronic device as the input device for the augmented reality service when the user's gaze corresponding to at least one object corresponding to the field of view of the user wearing the first electronic device or any one object of the at least one virtual object is consistently maintained for a reference time.

5. The first electronic device of claim 1 , wherein the processor is further configured to:
when an object related to the second electronic device exists in an area corresponding to the field of view of the user wearing the first electronic device, identify whether the user's gaze is directed toward the object related to the second electronic device, and
when the user's gaze is directed toward an object other than the object related to the second electronic device, determine to use the second electronic device as the input device for the augmented reality service.

6. The first electronic device of claim 1 , wherein the processor is further configured to:
when it is determined to use the second electronic device as the input device for the augmented reality service, identify whether input information is received from the second electronic device through the communication circuit, and
when the input information is received from the second electronic device, perform an operation corresponding to the input information.

7. The first electronic device of claim 6 , wherein the input information received from the second electronic device comprises at least one of a touch input, a physical button input, or information on a motion of the second electronic device.

8. The first electronic device of claim 6 , wherein the processor is further configured to:
when the input information is received from the second electronic device, identify an object corresponding to the user's gaze, and
control the display device to display a control menu related to the object corresponding to the user's gaze.

9. A second electronic device comprising:
a communication circuit;
an input device; and
a processor operably connected to the communication circuit and the input device,
wherein the processor is configured to:
establish communication with a first electronic device through the communication circuit,
when an input is detected through the input device, identify an input processing method of the second electronic device,
when the input processing method is a first method for sharing input, transmit information on the input to the first electronic device through the communication circuit, and
when the input processing method is a second method in which input sharing is restricted, perform an operation corresponding to the information on the input.

10. The second electronic device of claim 9 , wherein the processor is further configured to:
configure the input processing method of the second electronic device as the first method when information related to an input device for an augmented reality service is received from the first electronic device through the communication circuit, and
configure the input processing method of the second electronic device as a second method when the information related to the input device for the augmented reality service is not received from the first electronic device through the communication circuit, or when non-interworking information is received therefrom.

11. The second electronic device of claim 9 , further comprising:
a display device,
wherein the processor is further configured to, when information related to an input device for an augmented reality service is received from the first electronic device through the communication circuit, control the display device to one of change display information or change a display method of the display device.

12. The second electronic device of claim 9 , wherein the input device comprises at least one of a touch circuit or physical buttons.

13. A second electronic device comprising:
a communication circuit;
a display device; and
a processor operably connected to the communication circuit and the display device,
wherein the processor is configured to:
receive, from a first electronic device, information on at least one object corresponding to a field of view of a user wearing the first electronic device through the communication circuit,
based on the information on the at least one object, identify an object corresponding to a gaze of the user wearing the first electronic device,
identify whether the object corresponding to the gaze of the user wearing the first electronic device is an object related to the second electronic device, and
when the object corresponding to the gaze of the user wearing the first electronic device is an object different from the object related to the second electronic device, determine to the second electronic device as an input device for an augmented reality service.

14. The second electronic device of claim 13 , wherein the processor is further configured to obtain information on the user's gaze, based on at least one of a head gaze or an eye gaze of the first electronic device.

15. The second electronic device of claim 13 , wherein the processor is further configured to control the first electronic device to display at least one virtual object on at least a portion of an area corresponding to the field of view of the user wearing the first electronic device.

16. The second electronic device of claim 13 , wherein the processor is further configured to, when the object related to the second electronic device is not included in the at least one object corresponding to the field of view of the user wearing the first electronic device, determine to use the second electronic device as the input device for the augmented reality service.

17. The second electronic device of claim 13 , wherein the processor is further configured to, when the user's gaze is directed toward the object related to the second electronic devi

Assignees

• Samsung Electronics Co Ltd

Inventors

• Moon Choonkyoung
• Jung Shinjae

Classifications

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• H04L67/75

  Indicating network or usage conditions on the user display · CPC title

• G06F9/451

  Execution arrangements for user interfaces · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title