Wearable electronic device for displaying obstacle-related information, operating method, and storage medium

Patent metadata

Field	Value
Publication number	US-2025131612-A1
Application number	US-202418978844-A
Country	US
Kind code	A1
Filing date	Dec 12, 2024
Priority date	Aug 2, 2022
Publication date	Apr 24, 2025
Grant date	—

Abstract

A wearable electronic device is provided. The wearable electronic device includes a camera, a display, a microphone, a sensor, memory storing one or more computer programs, and one or more processors communicatively coupled with the camera, the display, the microphone, the sensor, and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the wearable electronic device to determine an environment level indicating a degree of danger of a surrounding environment for a user of the wearable electronic device based on an image obtained through the camera, determine whether to use the sensor, the camera, and the microphone depending on the determined environment level, obtain obstacle information and an estimated impulse between an obstacle and the user based on data input through at least one determined to be used among the sensor, the camera, or the microphone, determine a user interface (UI) level for displaying danger of collision between the obstacle and the user based on the environment level, the obstacle information, and the estimated impulse, generate a graphic entity indicating a direction and a danger degree of the obstacle depending on the UI level, and display an obstacle-related UI including the graphic entity on the display.

First claim

What is claimed is:

1 . A wearable electronic device comprising:
a camera; a display; a microphone; a sensor; memory storing one or more computer programs; and one or more processors communicatively coupled with the camera, the display, the microphone, the sensor, and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the wearable electronic device to:
determine an environment level indicating a danger degree of a surrounding environment for a user of the wearable electronic device based on an image obtained through the camera,
determine whether to use the sensor, the camera, and the microphone depending on the determined environment level,
obtain obstacle information and an estimated impulse between an obstacle and the user based on data input through at least one determined to be used among the sensor, the camera, or the microphone,
determine a user interface (UI) level for displaying danger of collision between the obstacle and the user based on the environment level, the obstacle information, and the estimated impulse,
generate a graphic entity indicating a direction and a danger degree of the obstacle depending on the UI level, and
display an obstacle-related UI including the graphic entity on the display.

2 . The wearable electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the wearable electronic device to:
determine a minimum level and a maximum level of the UI level based on the environment level, and determine the UI level within a range by the minimum level and the maximum level based on the obstacle information and the estimated impulse.

3 . The wearable electronic device of claim 1 , wherein at least one of whether it is possible to set whether to display the obstacle-related UI based on a user input or a relationship between the obstacle-related UI and another UI varies with the UI level.

4 . The wearable electronic device of claim 1 ,
wherein, it is possible to set whether to display the obstacle-related UI based on the user input, when the UI level is determined as a first UI level, and wherein, it is impossible to set whether to display the obstacle-related UI based on the user input, acknowledgement is requested from the user in response to a user input not to display the obstacle-related UI, or a notification of recommending displaying the obstacle-related UI is output on the display, when the UI level is determined as a second UI level higher than the first UI level.

5 . The wearable electronic device of claim 4 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the wearable electronic device to:
display only other UI and fail to display the obstacle-related UI, when the UI level is determined as the first UI level, display the obstacle-related UI behind the other UI, when the UI level is determined as the second UI level higher than the first UI level, display the obstacle-related UI in front of the other UI, when the UI level is determined as a third UI level higher than the second UI level, and display only the obstacle-related UI and fail to display the other UI, when the UI level is determined as a fourth UI level higher than the third UI level.

6 . The wearable electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the wearable electronic device to:
receive a user input for adjusting a minimum level or the maximum level, adjust the minimum level or a maximum level in response to the user input, and guide a UI level recommended according to the environment level, when adjusted minimum level or the adjusted maximum level is lower than a specified level.

7 . The wearable electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the wearable electronic device to:
identify whether the obstacle is an obstacle within a viewing angle range or an obstacle out of the viewing angle range, differently determine a UI level for starting to display the obstacle-related UI depending on the identified result, and differently determine a UI display method for a same UI level depending on the identified result.

8 . The wearable electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the wearable electronic device to:
determine a danger recognition area for the obstacle, assign a danger recognition time set according to the UI level to the obstacle, identify whether a gaze of the user is included in the danger recognition area within the danger recognition time, and suspend displaying the obstacle-related UI and reassign the danger recognition time, when the gaze of the user is included in the danger recognition area within the danger recognition time.

9 . The wearable electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the wearable electronic device to:
differently determine at least one of a size, transparency, or a color of the graphic entity based on the estimated impulse, and differently determine at least one of a direction indicated by the graphic entity, a direction range, or a number of graphic entities based on the obstacle information.

10 . The wearable electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the wearable electronic device to:
classify the image based on at least one of whether it is inside or outside, analysis of a vehicle-related element, or obstacle complexity, and determine the environment level depending on the classified result.

11 . A method of operating a wearable electronic device, the method comprising:
determining an environment level indicating a danger degree of a surrounding environment for a user of the wearable electronic device based on an image obtained through a camera; determining whether to use a sensor, the camera, and a microphone depending on the determined environment level; obtaining obstacle information and an estimated impulse between an obstacle and the user based on data input through at least one determined to be used among the sensor, the camera, or the microphone; determining a user interface (UI) level for displaying danger of collision between the obstacle and the user based on the environment level, the obstacle information, and the estimated impulse; generating a graphic entity indicating a direction and a danger degree of the obstacle depending on the UI level; and displaying an obstacle-related UI including the graphic entity on a display.

12 . The method of claim 11 , wherein the determining of the UI level includes:
determining a minimum level and a maximum level of the UI level based on the environment level; and determining the UI level within a range by the minimum level and the maximum level based on the obstacle information and the estimated impulse.

13 . The method of claim 12 , further comprising:
determining at least one of whether

Assignees

• Samsung Electronics Co Ltd

Inventors

• Lee Sanghun
• Kim Sungoh
• Kim Jihyun
• Prushinskiy Valeriy
• Yeom Donghyun
• Lee Dasom

Classifications

• G06V40/18

  Eye characteristics, e.g. of the iris · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title