Wearable device, method, and non-transitory computer-readable storage medium for setting depth value of pixels

Patent metadata

Field	Value
Publication number	US-12585331-B2
Application number	US-202519085516-A
Country	US
Kind code	B2
Filing date	Mar 20, 2025
Priority date	Apr 12, 2024
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

A wearable device includes a display assembly comprising one or more displays, cameras, sensors, memory comprising one or more storage media and storing instructions, and at least one processor comprising processing circuitry. The instructions, when executed by the at least one processor individually or collectively, cause the wearable device to obtain an image, identify depth values of pixels using obtained depth information, identify the number of pixels, identify a reference depth value among the depth values of the pixels, identify a first portion of the pixels, identify a second portion of the pixels, and based on setting the depth values of the first portion of the pixels to the reference depth value and based on setting depth values of the second portion of the pixels to a predetermined depth value, display, via the display assembly, a screen corresponding to the image.

First claim

What is claimed is:

1 . A wearable device comprising:
a display assembly comprising one or more displays; one or more cameras; one or more sensors; memory, comprising one or more storage media, configured to store instructions; and at least one processor comprising processing circuitry, wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:
obtain an image via the one or more cameras,
identify depth values of pixels in the image using depth information obtained via at least one of the one or more cameras or the one or more sensors,
identify the number of one or more pixels having each of the depth values of the pixels,
identify a reference depth value using a depth value associated with the number corresponding to a peak value among the depth values,
identify a first portion of the pixels having first depth values that are within a reference range with respect to the reference depth value,
identify a second portion of the pixels that are different from the first portion of the pixels, and
based on setting the first depth values of the first portion of the pixels to the reference depth value and setting second depth values of the second portion of the pixels to a predetermined depth value, display, via the display assembly, a screen corresponding to the image having the first portion of the pixels and the second portion of the pixels.

2 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, further cause the wearable device to:
identify another reference depth value using another depth value associated to the number corresponding to another peak value among the depth values of the pixels in the image, identify a third portion of the pixels that are different from the first portion of the pixels, wherein the third portion of the pixels has third depth values that are within the reference range with respect to the other reference depth value, and based on setting the first depth values of the first portion of the pixels to the reference depth value, setting the second depth values of the second portion of the pixels to the predetermined depth value, and setting the third depth values of the third portion of the pixels to the other reference depth value, display, via the display assembly, a screen corresponding to the image having the first portion of the pixels, the second portion of the pixels, and the third portion of the pixels.

3 . The wearable device of claim 1 , further comprising one or more other sensors configured to obtain eye tracking data regarding a gaze of one or more eyes,
wherein the predetermined depth value is a first predetermined depth value, and wherein the instructions, when executed by the at least one processor individually or collectively, further cause the wearable device to:
determine a foveated area of the image and a peripheral area of the image surrounding the foveated area of the image, using the eye tracking data obtained using the one or more other sensors,
identify the first portion of the pixels located within the foveated area of the image, wherein the first portion of the pixels has the first depth values that are within the reference range with respect to the reference depth value,
identify the second portion of the pixels located within the foveated area of the image, wherein the second portion of the pixels has the second depth values outside the reference range with respect to the reference depth value,
identify a third portion of the pixels located within the peripheral area of the image, and
based on setting the first depth values of the first portion of the pixels to the reference depth value, setting the second depth values of the second portion of the pixels to the first predetermined depth value, and setting third depth values of the third portion of the pixels to a second predetermined depth value, display, via the display assembly, a screen corresponding to the image having the first portion of the pixels, the second portion of the pixels, and the third portion of the pixels.

4 . The wearable device of claim 1 , further comprising:
one or more other sensors configured to obtain eye tracking data regarding a gaze of one or more eyes, wherein the predetermined depth value is a first predetermined depth value, and wherein the instructions, when executed by the at least one processor individually or collectively, further cause the wearable device to:
determine a foveated area of the image and a peripheral area of the image surrounding the foveated area of the image, using the eye tracking data obtained using the one or more other sensors,
identify the first portion of the pixels located within the foveated area of the image, wherein the first portion of the pixels has the first depth values that are outside another reference range and are within the reference range with respect to the reference depth value,
identify the second portion of the pixels located within the foveated area of the image, wherein the second portion of the pixels has the second depth values that are outside the other reference range and are outside the reference range with respect to the reference depth value,
identify a third portion of the pixels located within the foveated area of the image, wherein the third portion of the pixels has third depth values that are within the other reference range,
identify a fourth portion of the pixels located within the peripheral area of the image, and
based on setting the first depth values of the first portion of the pixels to the reference depth value, setting the second depth values of the second portion of the pixels to the first predetermined depth value, and setting fourth depth values of the fourth portion of the pixels to the second predetermined depth value, display, via the display assembly, a screen corresponding to the image having the first portion of the pixels, the second portion of the pixels and the fourth portion of the pixels, and
wherein depth values of an area within the screen corresponding to the third portion of the pixels correspond to the third depth values that are within the other reference range of the third portion of the pixels.

5 . The wearable device of claim 4 , wherein the one or more sensors comprise a direct time of flight (D-TOF) sensor and an indirect time of flight (I-TOF) sensor,
wherein the area within the screen comprises a first partial area and a second partial area, wherein the first partial area in the area within the screen have depth values corresponding to depth values identified using the D-TOF sensor among the D-TOF sensor and the I-TOF sensor, and wherein the second partial area in the area within the screen have depth values corresponding to depth values identified using the I-TOF sensor among the D-TOF sensor and the I-TOF sensor.

6 . The wearable device of claim 1 , wherein the predetermined depth value is obtained using an average value of the depth values of the pixels in the image.

7 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, further cause the wearable device to identify the reference depth value, using the depth value associated with the number and a difference between a position of the one or more cameras attached to the wearable device and a position of eyes of a user wearing the wearable device.

8 . A wearable device comprising:
a display assembly, including one or more displays; one or more cameras; one or more sensors; one or more other sensors configured to obtain eye tracking data regarding a gaze of one or more eyes; memory, com

Assignees

• Samsung Electronics Co Ltd

Inventors

• Yoo Dongheon
• Do Wonjoon
• Yeom Donghyun
• Kim Sungoh
• Kim Kihwan
• Lee Sanghun

Classifications

• G06T7/521

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title