Electronic device and method for representing contents based on gaze dwell time

Patent metadata

Field	Value
Publication number	US-12033382-B2
Application number	US-202217959494-A
Country	US
Kind code	B2
Filing date	Oct 4, 2022
Priority date	Sep 27, 2021
Publication date	Jul 9, 2024
Grant date	Jul 9, 2024

Abstract

Provided is an electronic device including a body part, a glass member disposed on the body part, a display disposed on the glass member, a support part rotatably connected to the body part, a sensor including an eye tracking camera and a front camera configured to capture an image of a front side of a user, and a processor operatively connected to the display and the sensor, wherein the processor is configured to output at least one content via the display, recognize at least one object by the front camera, obtain the user's gaze dwell time for the object by the eye tracking camera, based on the gaze dwell time being longer than or equal to a reference time, obtain an area occupied by the object in a field of view (FOV) of the front camera, based on the area, output the at least one content on a region having no overlap with the object, or reduce a size of the at least one content and output the at least one content on one side of the display, and based on the user's gaze dwell time for the object being less than the reference time, output the at least one content in a type identical to an initial output type.

First claim

What is claimed is:

1. An electronic device comprising:
a body part;
a glass member disposed on the body part;
a display disposed on the glass member;
a support part rotatably connected to the body part;
a sensor comprising an eye tracking camera and a front camera configured to capture an image of a front side of a user; and
a processor operatively connected to the display and the sensor,
wherein the processor is configured to:
output at least one content via the display;
recognize at least one object by the front camera;
obtain the user's gaze dwell time for the object by the eye tracking camera;
based on the gaze dwell time being longer than or equal to a reference time, obtain an area occupied by the object in a field of view (FOV) of the front camera;
determine an importance level of the at least one content;
in case of the importance level being less than a predetermined value, based on the area, output the at least one content on a region having no overlap with the object, or reduce a size of the at least one content and output the at least one content on one side of the display;
in case of the importance level being equal to or greater than the predetermined value, output the at least one content without reducing the size; and
based on the user's gaze dwell time for the object being less than the reference time, output the at least one content in a type identical to an initial output type.

2. The electronic device of claim 1 , wherein the processor is further configured to:
obtain position coordinates of the object in the FOV by the front camera;
determine, based on the position coordinates, a region on which the object is positioned in an edge region of the FOV or a center region of the FOV; and
output the at least one content, based on the area and the region on which the object is positioned in the FOV.

3. The electronic device of claim 2 , wherein the processor is further configured to:
based on the object being positioned in the edge region of the FOV, or the area being smaller than a predetermined area, output the at least one content on a region having no overlap with the object; and
based on the object being positioned in the center region of the FOV, and the area being equal to or greater than the predetermined area, reduce the size of the at least one content and output the at least one content on one side of the display.

4. The electronic device of claim 3 , wherein the processor is further configured to:
based on the at least one content being output on the region having no overlap with the object,
based on the area occupied by the object being smaller than the predetermined area, output the at least one content on a region remaining excluding a region displaying the object; and
based on the area occupied by the object being equal to or greater than the predetermined area, and the region displaying the object being included in the edge region, move a region outputting the at least one content so that the moved region does not overlap with the region displaying the object.

5. The electronic device of claim 1 , wherein the processor is further configured to:
determine whether the object is a pre-registered object based on an image of the object obtained using the front camera; and
based on the object being a pre-registered object, obtain the user's gaze dwell time for the object.

6. The electronic device of claim 5 , wherein the processor is further configured to:
based on the object being a pre-registered object, output a graphic user interface (UI) comprising additional information corresponding to the object on the display.

7. The electronic device of claim 1 , wherein the processor is further configured to:
based on the size of the at least one content being reduced and the at least one content being output on one side of the display temporarily stop reproduction of the content at a time point of reducing the size of the content.

8. The electronic device of claim 1 , wherein the processor is further configured to:
search for an image corresponding to the object and obtain related information corresponding to the image; and
output a graphic user interface (UI) comprising the related information at one side of the object in the FOV.

9. The electronic device of claim 1 , wherein the processor is further configured to:
obtain an image of a hand of the user with which the object is held by the front camera;
based on the gaze dwell time being equal to or longer than the reference time, and the object being held with one hand of the user, output the at least one content on a region having no overlap with the object, based on the area; and
based on the gaze dwell time being equal to or longer than the reference time, and the object being held with both hands of the user, reduce the size of the at least one content and output the at least one content on one side of the display.

10. The electronic device of claim 1 , wherein the processor is further configured to:
based on the front camera recognizing a first object and a second object,
determine, as an object of interest, at least one of the first object and the second object, based on a first gaze dwell time for the first object and a second gaze dwell time for the second object; and
output the at least one content, based on an area occupied by the at least one object of interest in the FOV.

11. The electronic device of claim 1 , wherein the processor is further configured to determine the user has recognized the object, based on recognizing a gesture of a hand of the user holding the object by the front camera.

12. The electronic device of claim 11 , wherein the processor is further configured to:
generate a unique gesture of an object, map the unique gesture to the object, and store the mapped unique gesture and object; and
based on the unique gesture being recognized, determine the user has recognized the object mapped to the unique gesture.

13. A content display method of an electronic device, the method comprising:
outputting at least one content on a display;
recognizing at least one object by a front camera;
obtaining a user's gaze dwell time for the object by an eye tracking camera;
based on the gaze dwell time being longer than or equal to a reference time, obtaining an area occupied by the object in a field of view (FOV) of the front camera;
determining an importance level of the at least one content;
in case of the importance level being less than a predetermined value, based on the area, outputting the at least one content on a region having no overlap with the object, or reducing a size of the at least one content and outputting the at least one content on one side of the display;
in case of the importance level being equal to or greater than the predetermined value, output the at least one content without reducing the size; and
based on the user's gaze dwell time for the object being less than the reference time, outputting the at least one content in a type identical to an initial output type.

14. The method of claim 13 , wherein the outputting of the content further comprises:
obtaining position coordinates of the object in the FOV by the front camera;
determining, based on the position coordinates, a region on which the object is positioned in an edge region or a center region of the FOV; and
outputting the at least one content, based on the area and the region on which the object is positioned in the FOV.

15. The method of claim 14 , wherein the outputting of the content further comprises:
based on the object being positioned in the edge region of the FOV,

Assignees

• Samsung Electronics Co Ltd

Inventors

• Ko Taehun
• Shin Jiyeon
• Han Hoon
• Ji Yunjeong

Classifications

• G06F2203/04803

  Split screen, i.e. subdividing the display area or the window area into separate subareas · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/03

  Arrangements for converting the position or the displacement of a member into a coded form · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06T3/40

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title