Method and sytem for presenting a digital information related to a real object

Patent metadata

Field	Value
Publication number	US-12039644-B2
Application number	US-202117323542-A
Country	US
Kind code	B2
Filing date	May 18, 2021
Priority date	Jun 3, 2014
Publication date	Jul 16, 2024
Grant date	Jul 16, 2024

Abstract

The invention is related to a method of presenting a digital information related to a real object, comprising determining a real object, providing a plurality of presentation modes, wherein the plurality of presentation modes comprises an augmented reality mode, and at least one of a virtual reality mode and an audio mode, providing at least one representation of a digital information related to the real object, determining a spatial relationship between a camera and a reference coordinate system under consideration of an image captured by the camera, selecting a presentation mode from the plurality of presentation modes according to the spatial relationship, and presenting the at least one representation of the digital information using the selected presentation mode.

First claim

The invention claimed is:

1. A method of presenting a digital information related to a real object, comprising:
determining a field of view of an eye of a user of an electronic device through a display of the electronic device for which digital information is to be presented;
in response to a determination that at least part of a real object is visible within the field of view, selecting an augmented reality (“AR”) presentation mode from among a plurality of information presentation modes which support the presentation of the digital information;
presenting the digital information using the AR presentation mode; and
during the AR presentation mode and in accordance with a change in spatial relationship between the eye of the user and the real object with respect to the display:
determining that at least part of the real object is outside the field of view;
selecting a second information presentation mode from among the plurality of information presentation modes based on the determination that at least part of the real object is outside the field of view, and
switching the presentation of the digital information from the AR presentation mode to the second information presentation mode based on the change in the spatial relationship.

2. The method of claim 1 , wherein the second information presentation mode comprises at least one of a virtual reality mode and an audio mode.

3. The method according to claim 2 , wherein selecting the presentation mode comprises:
determining if a gaze of one or more eyes locates on a display device, and
in response to determining that the gaze of one or more eyes does not locate on the display device, selecting an audio mode as the presentation mode.

4. The method of claim 1 , further comprising capturing a live view of the real object by a camera of the electronic device,
wherein an image of the real object is captured by the camera, and
wherein the live view of the real object is based on a view of the image presented on a display of the electronic device.

5. The method of claim 1 , further comprising capturing a view of the real object through a semi-transparent display.

6. The method of claim 1 , wherein the spatial relationship is determined based on a spatial relationship between the eye and a coordinate system associated with the real object.

7. A non-transitory computer readable medium comprising computer readable code for presenting a digital information related to a real object, the computer readable code executable by one or more processors to:
determine a field of view of an eye of a user of an electronic device through a display of the electronic device for which digital information is to be presented;
in response to a determination that at least part of a real object is visible within the field of view, select an augmented reality (“AR”) presentation mode from among a plurality of information presentation modes which support the presentation of the digital information;
present the digital information using the AR presentation mode; and
during the AR presentation mode and in accordance with a change in spatial relationship between the eye of the user and the real object with respect to the display:
determine that at least part of the real object is outside the field of view;
select a second information presentation mode from among the plurality of information presentation modes based on the determination that at least part of the real object is outside the field of view, and
switch the presentation of the digital information from the AR presentation mode to the second information presentation mode based on the change in the spatial relationship.

8. The non-transitory computer readable medium of claim 7 , wherein the second information presentation mode comprises at least one of a virtual reality mode and an audio mode.

9. The non-transitory computer readable medium of claim 8 , wherein the computer readable code executable by one or more processors to select the presentation mode comprises computer readable code executable by one or more processors to:
determine if a gaze of one or more eyes locates on a display device, and
in response to determining that the gaze of one or more eyes does not locate on the display device, select an audio mode as the presentation mode.

10. The non-transitory computer readable medium of claim 7 , further comprising computer readable code executable by one or more processors to capture a live view of the real object by a camera of the electronic device,
wherein an image of the real object is captured by the camera, and
wherein the live view of the real object is based on a view of the image presented on a display of the electronic device.

11. The non-transitory computer readable medium of claim 7 , further comprising computer readable code to capture a view of the real object through a semi-transparent display.

12. The non-transitory computer readable medium of claim 7 , wherein the spatial relationship is determined based on a spatial relationship between the eye and a coordinate system associated with the real object.

13. A system for presenting a digital information related to a real object, comprising:
one or more processors; and
one or more computer readable media comprising computer readable code executable by one or more processors to:
determine a field of view of an eye of a user of an electronic device through a display of the electronic device for which digital information is to be presented;
in response to a determination that at least part of a real object is visible within the field of view, select an augmented reality (“AR”) presentation mode from among a plurality of information presentation modes which support the presentation of the digital information;
present the digital information using the AR presentation mode; and
during the AR presentation mode and in accordance with a change in spatial relationship between the eye of the user and the real object with respect to the display:
determine that at least part of the real object is outside the field of view;
select a second information presentation mode from among the plurality of information presentation modes based on the determination that at least part of the real object is outside the field of view, and
switch the presentation of the digital information from the AR presentation mode to the second information presentation mode based on the change in the spatial relationship.

14. The system of claim 13 , wherein the second information presentation mode comprises at least one of a virtual reality mode and an audio mode.

15. The system of claim 14 , wherein the computer readable code executable by one or more processors to select the presentation mode comprises computer readable code executable by one or more processors to:
determine if a gaze of one or more eyes locates on a display device, and
in response to determining that the gaze of one or more eyes does not locate on the display device, select an audio mode as the presentation mode.

16. The system of claim 13 , further comprising computer readable code executable by one or more processors to capture a live view of the real object by a camera of the electronic device,
wherein an image of the real object is captured by the camera, and
wherein the live view of the real object is based on a view of the image presented on a display of the electronic device.

17. The system of claim 13 , further comprising computer readable code to capture a view of the real object through a semi-transparent display.

18. The system

Assignees

• Apple Inc

Inventors

• Kurz Daniel
• Fedosov Anton

Classifications

• H04N7/183

  for receiving images from a single remote source · CPC title

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T2207/20068

  Projection on vertical or horizontal image axis · CPC title

• G06T2215/16

  Using real world measurements to influence rendering · CPC title