Transparent insert identification

Patent metadata

Field	Value
Publication number	US-11948043-B2
Application number	US-202217824339-A
Country	US
Kind code	B2
Filing date	May 25, 2022
Priority date	Jun 2, 2021
Publication date	Apr 2, 2024
Grant date	Apr 2, 2024

Abstract

Various implementations disclosed herein include devices, systems, and methods that use a marking on a transparent surface (e.g., a prescription lens insert for an HMD) to identify information (e.g., prescription parameters) about the transparent surface. In some implementations, the markings do not interfere with eye tracking through the transparent surface or using the transparent surface to view virtual content or a physical environment. In some implementations, image data is obtained from an image sensor of an electronic device, the image data corresponding to a transparent surface attached to the electronic device. Then, a code is identified in the image data, wherein the code is detectable on the transparent surface by the image sensor without interfering with a function of the electronic device involving the transparent surface. In some implementations, content is provided at the electronic device based on the identified code, wherein the content is viewable through the transparent surface.

First claim

What is claimed is:

1. A method comprising:
at a processor:
detecting an attachment of a transparent surface to an electronic device, wherein detecting the attachment comprises detecting that the transparent surface is mounted to the electronic device;
based on detecting the attachment, obtaining image data from an image sensor of an electronic device, the image data corresponding to a transparent surface attached to the electronic device;
detecting a blink of a user of the electronic device;
obtaining one or more frames of data from the image sensor during the blink;
identifying a code in the image data based on detecting the code on the transparent surface in the one or more frames of data from the image sensor, wherein the code is detectable on the transparent surface by the image sensor without interfering with a function of the electronic device involving the transparent surface; and
providing content at the device based on the identified code, wherein the content is viewable through the transparent surface.

2. The method of claim 1 , wherein the identified code identifies prescription parameters of the transparent surface, a user of the electronic device, or position and orientation of the transparent surface relative to the electronic device.

3. The method of claim 1 , further comprising detecting the code based on detecting an act of attaching the transparent surface to the electronic device, wherein detecting the attachment is based on audio data, IMU data, verbal communications, proximity sensor, a vision sensor, or an instruction.

4. The method of claim 1 , wherein the code comprises markings in the transparent surface that are not perceivable by an unaided human eye.

5. The method of claim 1 , wherein the code comprises excitable fluorescent markings visible at a preset emission wavelength or wavelength band.

6. The method of claim 1 , wherein the code comprises excitable photochromic markings viewable at a preset detection wavelength or wavelength band.

7. The method of claim 1 , wherein the code comprises reflection holographic markings viewable from a limited angular range of positions.

8. The method of claim 1 , wherein the code comprises computational caustic markings viewable at a preset detection wavelength or wavelength band on a projection surface spaced apart from the transparent surface.

9. The method of claim 1 , wherein the detecting comprises detecting a transition of the transparent surface from an unmounted state to a mounted state.

10. A method comprising:
at a processor:
detecting an attachment of a transparent surface to an electronic device, wherein detecting the attachment comprises detecting that the transparent surface is mounted to the electronic device;
based on detecting the attachment, obtaining image data from an image sensor of an electronic device, the image data corresponding to a transparent surface attached to the electronic device;
identifying a code in the image data, wherein the code is detectable on the transparent surface by the image sensor without interfering with a function of the electronic device involving the transparent surface;
periodically detecting the code, wherein the code is generated using active illumination, when eye tracking or blink detection is not being performed by the electronic device or the electronic device is not in use, and
providing content at the device based on the identified code, wherein the content is viewable through the transparent surface.

11. An electronic device comprising:
an image sensor configured to obtain image data corresponding to a transparent surface;
a display configured to present content at the electronic device for viewing; and
a processor configured to perform operations comprising:
detecting an attachment of the transparent surface to the electronic device, wherein detecting the attachment comprises detecting that the transparent surface is mounted to the electronic device;
based on detecting the attachment, obtaining image data from an image sensor of an electronic device, the image data corresponding to the transparent surface attached to the electronic device;
detecting a blink of a user of the electronic device;
obtaining one or more frames of data from the image sensor during the blink;
identifying the code in the image data based on detecting the code on the transparent surface in the one or more frames of data from the image sensor, wherein the code is detectable on the transparent surface by the image sensor without interfering with a function of the electronic device involving the transparent surface; and
providing the content at the electronic device based on the identified code, wherein the content is viewable through the transparent surface.

12. The electronic device of claim 11 , wherein the code comprises markings in the transparent surface that are not perceivable by an unaided human eye.

13. The electronic device of claim 11 , wherein the code comprises an excitable fluorescent material visible at a preset emission wavelength or wavelength band when the electronic device is not in use.

14. The electronic device of claim 11 , wherein the code comprises excitable photochromic material that is viewable at a preset detection wavelength or wavelength band.

15. The electronic device of claim 11 , wherein the code comprises a reflection holographic optical element viewable from a limited angular range of positions.

16. The electronic device of claim 11 , wherein the code comprises a computational caustic and that is viewable at a preset detection wavelength or wavelength band on a projection surface spaced apart from the transparent surface.

17. The electronic device of claim 11 , wherein the electronic device comprises a head-mounted device.

18. The electronic device of claim 11 , further comprising tracking eyes or a gaze of a user of the electronic device using the image sensor.

19. The electronic device of claim 11 , wherein the identified code identifies prescription parameters of the transparent surface, a user of the electronic device, or position and orientation of the transparent service relative to the electronic device.

20. A non-transitory computer-readable storage medium, storing program instructions executable via one or more processors to perform operations comprising:
detecting an attachment of a transparent surface to an electronic device, wherein detecting the attachment comprises detecting that the transparent surface is mounted to the electronic device;
based on detecting the attachment, obtaining image data from an image sensor of an electronic device, the image data corresponding to a transparent surface attached to the electronic device;
detecting a blink of a user of the electronic device;
obtaining one or more frames of data from the image sensor during the blink;
identifying a code in the image data based on detecting the code on the transparent surface in the one or more frames of data from the image sensor, wherein the code is detectable on the transparent surface by the image sensor without interfering with a function of the electronic device involving the transparent surface; and
providing content at the device based on the identified code, wherein the content is viewable through the transparent surface.

Assignees

• Apple Inc

Inventors

• Kurz Daniel
• Grundhoefer Anselm
• Gupta Tushar

Classifications

• G06K7/1404 Primary

  Methods for optical code recognition · CPC title

• G02B27/0101

  characterised by optical features (G02B27/0172 takes precedence) · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06V40/18 Primary

  Eye characteristics, e.g. of the iris · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title