Mitigating artifacts caused by an under-display light emitter

Patent metadata

Field	Value
Publication number	US-12154480-B2
Application number	US-202318184592-A
Country	US
Kind code	B2
Filing date	Mar 15, 2023
Priority date	Jun 17, 2022
Publication date	Nov 26, 2024
Grant date	Nov 26, 2024

Abstract

A light emitter that operates through a display may cause display artifacts, even when the light emitter operates using non-visible wavelengths. Display artifacts caused by a light emitter that operates through a display may be referred to as emitter artifacts. To mitigate emitter artifacts, operating conditions for a display frame may be used to determine an optimal firing time for the light emitter during that display frame. The operating conditions used to determine the optimal firing time may include emitter operating conditions, display content statistics, display brightness, temperature, and refresh rate. Operating conditions from one or more previous frames may be stored in a frame buffer and may be used to help determine the optimal firing time for the light emitter during a display frame. Pixel values for the display may be modified to mitigate emitter artifacts.

First claim

What is claimed is:

1. An electronic device, comprising:
a display comprising pixels;
a sensor comprising a light emitter that emits light through the display; and
control circuitry configured to:
based on a first set of operating conditions for a first display frame and a second set of operating conditions for a second display frame, determine a first optimal firing time for the light emitter in the sensor, wherein the first optimal firing time is a delay relative to the beginning of the first display frame and wherein the second display frame is previous to the first display frame;
based on the second set of operating conditions for the second display frame, determine a second optimal firing time for the light emitter in the sensor, wherein the second optimal firing time is a delay relative to the beginning of the second display frame and the second optimal firing time is different from the first optimal firing time; and
control the light emitter to emit light at the first and second optimal firing times, respectively, during the first and second display frames.

2. The electronic device defined in claim 1 , wherein the first set of operating conditions for the first display frame comprises operating conditions for the light emitter.

3. The electronic device defined in claim 2 , wherein the operating conditions for the light emitter comprise an operating condition selected from the group consisting of: a wavelength for the light emitter, an irradiation power for the light emitter, a pulse duration for the light emitter, and a firing frequency for the light emitter.

4. The electronic device defined in claim 1 , wherein the first set of operating conditions for the first display frame comprises statistics for a subset of the pixels.

5. The electronic device defined in claim 4 , wherein the statistics for the subset of the pixels comprise color and brightness information for the subset of the pixels.

6. The electronic device defined in claim 5 , wherein the light from the light emitter passes through the subset of the pixels.

7. The electronic device defined in claim 5 , wherein the control circuitry is configured to, for each pixel in the subset of the pixels, determine an emitter artifact profile for that pixel using a look-up table.

8. The electronic device defined in claim 7 , wherein the control circuitry is configured to use a weighted average of the emitter artifact profiles for the subset of the pixels to obtain a single representative emitter artifact profile for the subset of the pixels and wherein determining the first optimal firing time for the light emitter in the sensor comprises determining the first optimal firing time for the light emitter in the sensor based on the single representative emitter artifact profile.

9. The electronic device defined in claim 5 , wherein the control circuitry is configured to, for each pixel in the subset of the pixels, determine an emitter artifact profile for that pixel using a predictive model.

10. The electronic device defined in claim 9 , wherein the control circuitry is configured to use a weighted average of the emitter artifact profiles for the subset of the pixels to obtain a single representative emitter artifact profile for the subset of the pixels and wherein determining the first optimal firing time for the light emitter in the sensor comprises determining the first optimal firing time for the light emitter in the sensor based on the single representative emitter artifact profile.

11. The electronic device defined in claim 1 , wherein the first set of operating conditions for the first display frame comprises a display brightness or a refresh rate for the display.

12. The electronic device defined in claim 1 , further comprising:
a temperature sensor, wherein the first set of operating conditions for the first display frame comprises a temperature from the temperature sensor.

13. The electronic device defined in claim 1 , wherein the sensor comprises a proximity sensor.

14. The electronic device defined in claim 1 , further comprising:
a frame buffer configured to store at least the second set of operating conditions for the second display frame.

15. The electronic device defined in claim 1 , wherein the control circuitry is configured to:
determine a difference between an operating condition for the first display frame and the operating condition for the second display frame; and
in response to the difference being less than a threshold, control the light emitter to emit light at a previously determined first optimal firing time during the first display frame.

16. The electronic device defined in claim 1 , wherein the control circuitry is configured to:
determine a difference between an operating condition for the first display frame and the operating condition for the second display frame, wherein determining the first optimal firing time for the light emitter in the sensor based on the first set of operating conditions for the first display frame and the second set of operating conditions for the second display frame comprises determining the first optimal firing time for the light emitter in the sensor based on the first set of operating conditions for the first display frame in response to the difference being greater than a threshold.

17. The electronic device defined in claim 1 , wherein the light emitter is an infrared light emitter that emits infrared light.

18. The electronic device of claim 1 , wherein determining the second optimal firing time for the light emitter in the sensor based on the second set of operating conditions for the second display frame comprises determining the second optimal firing time for the light emitter in the sensor based on the second set of operating conditions for the second display frame and at least a third set of operating conditions for a third display frame, wherein the third display frame is previous to the second display frame.

19. An electronic device, comprising:
a display comprising pixels;
a proximity sensor comprising a light emitter that emits light through the display; and
control circuitry configured to, based on operating conditions for a given display frame, modify pixel data for the given display frame to mitigate artifacts caused by the light from the light emitter passing through the display, wherein at least one pixel in the display for the given display frame has an initial target brightness and a perceived brightness, wherein the perceived brightness is affected by the artifacts caused by the light from the light emitter passing through the display, and wherein modifying the pixel data for the given display frame comprises adjusting a brightness of the at least one pixel from a first magnitude to a second magnitude so that the perceived brightness of the at least one pixel matches the initial target brightness of the at least one pixel.

20. An electronic device, comprising:
a display comprising pixels;
a proximity sensor comprising a light emitter that emits light through the display; and
control circuitry configured to:
for each pixel in a subset of the pixels, determine an emitter artifact profile for that pixel using at least a brightness of that pixel for a given display frame and a look-up table;
spatially weight the emitter artifact profiles for the subset of the pixels to obtain a single representative emitter artifact profile for the subset of the pixels;
use the single representative emitter artifact profile to determine an optimal firing time for the light emitter in the

Assignees

• Apple Inc

Inventors

• Hu Jenny
• Wang Chaohao
• Glazowski Christopher E
• Perlaki Clint M
• Manly David R
• Wen Feng
• Williams Graeme M
• Kam Hei
• Boo Hyun H
• Choboter Kevin J
• Kim Kyounghwan
• Yan Lu
• Chappalli Mahesh B
• Winkler Mark T
• Zhu Na
• Holland Peter F
• Chen Tong
• Rieutort-Louis Warren S
• Cai Wenrui
• Guan Ximeng
• Tang Yingying
• Che Yuchi

Classifications

• G09G3/3233

  with pixel circuitry controlling the current through the light-emitting element · CPC title

• G09G3/3406

  Control of illumination source (illumination devices structurally associated with liquid crystal cells G02F1/1336) · CPC title

• G09G2360/145

  the light originating from the display screen · CPC title

• G09G2320/0285

  using tables for spatial correction of display data · CPC title

• G09G2360/14

  Detecting light within display terminals, e.g. using a single or a plurality of photosensors · CPC title