Artificial reality environment with glints displayed by an extra reality device

Patent metadata

Field	Value
Publication number	US-11636655-B2
Application number	US-202117397266-A
Country	US
Kind code	B2
Filing date	Aug 9, 2021
Priority date	Nov 17, 2020
Publication date	Apr 25, 2023
Grant date	Apr 25, 2023

Abstract

The present embodiments relate to display of glints associated with real-world objects in an environment displayed on an extra reality (XR) device. The glint can include a virtual object associated with a real-world object, such as an indication of a social interaction associated with a real-world object, a content item tagged to an object, etc. The system as described herein can present glints on a display of an XR device based on a distance between the XR device and a location associated with the glint. Responsive to selection of a glint in the environment, additional information can be presented relating to the glint or another action can be taken, such as to open an application. In some instances, a glint can include a series of search results relating to a corresponding real-world object to provide additional information relating to the real-world object.

First claim

We claim:

1. A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process comprising:
determining that a distance between a geographic position of an object and a location of an XR device is greater than a threshold distance and in response:
presenting an actionable virtual object, related to the object, a threshold amount above the object, wherein the threshold amount, that the actionable virtual object is displayed above the object related to the actionable virtual object, is automatically determined by the computing system to be an amount determined to make the actionable virtual object higher than an intervening object that is between the XR device and the object related to the actionable virtual object;

determining that the distance between the geographic position of the object and the location of the XR device has updated to be less than the threshold distance and in response:
presenting the actionable virtual object to be attached to the object that is related to the actionable virtual object; and

while the actionable virtual object is attached to the object, identifying a user selection on the actionable virtual object based on a user gesture being directed at the actionable virtual object or a user gaze being directed at the actionable virtual object and in response:
modifying the actionable virtual object to present a deep link interface that controls an application associated with the actionable virtual object, wherein the deep link interface provides one or more of: a preview of content from the associated application, routing of commands to the associated application, shortcuts to controls for the associated application, controls that open other virtual objects, or any combination thereof.

2. The computer-readable storage medium of claim 1 , wherein the process further comprises:
identifying the object as a first detected object by comparing detectable features of the object with characteristics of a series of known objects;
processing visual features of the first detected object to derive a series of search results corresponding to content and/or applications relating to the first detected object;
identifying one or more actionable virtual objects, including the actionable virtual object, based on the search results; and
presenting the one or more actionable virtual objects, in an environment depicted in the XR device, in spatial relation to the object.

3. The computer-readable storage medium of claim 1 , wherein presenting the actionable virtual object as attached to the object related to the actionable virtual object comprises:
presenting the actionable virtual object to be, from a point of view of a user of the XR device, a pre-determined amount to the left or right of the object related to the actionable virtual object.

4. The computer-readable storage medium of claim 1 , wherein the user selection on the actionable virtual object includes determining that an attention focal point of the XR device is directed at a location of the actionable virtual object, wherein the attention focal point is determined by the XR device based on any of eye tracking for a user, head tracking for the user, hand tracking for the user, or any combination thereof.

5. The computer-readable storage medium of claim 1 , wherein the deep link interface provides controls for interacting with the actionable virtual object, and wherein the controls provide access to actions for the application relating to the actionable virtual object, the actions being responsive to detecting a second selection action in relation to the controls.

6. The computer-readable storage medium of claim 1 ,
wherein the deep link interface includes displaying additional content including an indication of an interaction on a social network in relation to the object, and
wherein the object is a real-world object in an environment of the XR device.

7. The computer-readable storage medium of claim 1 , wherein the process further comprises:
determining that the actionable virtual object is one of a number of actionable virtual objects within a specified area; and
responsive to determining that the number of actionable virtual objects are within the specified area, updating an environment displayed on the XR device to replace the number of actionable virtual objects with a group actionable virtual object indicator indicative of the number of actionable virtual objects.

8. The computer-readable storage medium of claim 1 , wherein presenting the actionable virtual object as attached to the object includes presenting the actionable virtual object in a minimized form and wherein, responsive to detecting a first selection action of the actionable virtual object, the presentation of the minimized actionable virtual object is updated to a maximized version with A) controls for interacting with the actionable virtual object and/or B) additional content associated with the actionable virtual object.

9. A computing system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
determining that a distance between a geographic position of an object and a location of an XR device is greater than a threshold distance and in response:
presenting an actionable virtual object, related to the object, a threshold amount above the object, wherein the threshold amount, that the actionable virtual object is displayed above the object related to the actionable virtual object, is automatically determined by the computing system to be an amount determined to make the actionable virtual object higher than an intervening object that is between the XR device and the object related to the actionable virtual object;

determining that the distance between the geographic position of the object and the location of the XR device has updated to be less than the threshold distance and in response:
presenting the actionable virtual object as attached to the object that is related to the actionable virtual object; and

identifying a user selection on the actionable virtual object based on a user gesture being directed at the actionable virtual object or a user gaze being directed at the actionable virtual object and in response:
modifying the actionable virtual object to present a deep link interface that controls an application associated with the actionable virtual object, wherein the deep link interface provides one or more of: a preview of content from the associated application, routing of commands to the associated application, shortcuts to controls for the associated application, controls that open other virtual objects, or any combination thereof.

10. The computing system of claim 9 , wherein presenting the actionable virtual object as attached to the object includes presenting the actionable virtual object in a minimized form and wherein, responsive to detecting a first selection action of the actionable virtual object, the presentation of the minimized actionable virtual object is updated to a maximized version with A) controls for interacting with the actionable virtual object and/or B) additional content associated with the actionable virtual object.

11. The computing system of claim 9 , wherein the process further comprises:
identifying the object as a first detected object by comparing detectable features of the object with characteristics of a series of known objects;
processing visual features of the first detected object to derive a series of search results corresponding to content and/or applications relating to th

Assignees

• Meta Platforms Tech Llc

Inventors

• Ma Jing
• Hofmeester Gerrit Hendrik
• Blakeley John Jacob
• De Almeida E De Vincenzo Camila Cortes
• Mac Gagneet Singh
• Velez Jenna
• Char Cody
• Rodrigues Annika
• Luo michael

Classifications

• G06Q10/40 Primary

  Business processes related to social networking or social networking services · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title