Systems and methods for dynamic detection and presentation of obscured real-world objects in augmented or mixed reality virtual content

Patent metadata

Field	Value
Publication number	US-2023113125-A1
Application number	US-202117497519-A
Country	US
Kind code	A1
Filing date	Oct 8, 2021
Priority date	Oct 8, 2021
Publication date	Apr 13, 2023
Grant date	—

Abstract

A method for dynamic detection and presentation of obscured real-world objects in augmented or mixed reality virtual content may include an object of interest computer program executed by an electronic device: (1) receiving data for a virtual or augmented reality object to display on an image of a physical environment on a display of the electronic device; (2) receiving data for a physical object in the physical environment; (3) determining that the physical object is obscured by the virtual or augmented reality object; (4) generating a representation of the physical object; and (5) providing the representation of the physical object to a virtual reality or augmented reality computer program executed by the electronic device, wherein the virtual reality or augmented reality computer program is configured to display the physical object over the virtual or augmented reality object.

First claim

What is claimed is:

1 . A method for dynamic detection and presentation of obscured real-world objects in augmented or mixed reality virtual content, comprising:
receiving, by an object of interest computer program executed by an electronic device, data for a virtual or augmented reality object to display on an image of a physical environment on a display of the electronic device; receiving, by the object of interest computer program, data for a physical object in the physical environment; determining, by the object of interest computer program, that the physical object is at least partially obscured by the virtual or augmented reality object; generating, by the object of interest computer program, a representation of the physical object; and providing, by the object of interest computer program, the representation of the physical object to a virtual reality or augmented reality computer program executed by the electronic device, wherein the virtual reality or augmented reality computer program is configured to display the physical object over the virtual or augmented reality object.

2 . The method of claim 1 , wherein the object of interest computer program receives the virtual or augmented reality object from the virtual reality or augmented reality computer program.

3 . The method of claim 1 , further comprising:
determining, by the object of interest computer program, that the physical object is an object of interest.

4 . The method of claim 3 , wherein the object of interest comprises a dangerous, hazardous, or harmful object.

5 . The method of claim 1 , wherein the step of determining, by the object of interest computer program, that the physical object is at least partially obscured by the virtual or augmented reality object comprises:
determining a location of the physical object and a location of the virtual or augmented reality object; and determining that the location of the virtual or augmented reality object is between the electronic device and the physical object.

6 . The method of claim 5 , wherein the object of interest computer program determines a location of the physical object using a ray cast query.

7 . The method of claim 5 , wherein the object of interest computer program determines a location of the physical object using LiDAR.

8 . The method of claim 1 , wherein the representation of the physical object comprises an image of the physical object.

9 . A method for dynamic detection and presentation of obscured real-world objects in augmented or mixed reality virtual content, comprising:
receiving, by an object of interest computer program executed by an electronic device, data for a virtual or augmented reality object to display on an image of a physical environment on a display of the electronic device; receiving, by the object of interest computer program, data for a physical object in the physical environment; determining, by the object of interest computer program, that the physical object is at least partially obscured by the virtual or augmented reality object; detecting, by the object of interest computer program, a boundary of the physical object; and providing, by the object of interest computer program, to a virtual reality or augmented reality computer program executed by the electronic device, wherein the virtual reality or augmented reality computer program is configured to remove an area within the boundary from the virtual or augmented reality object, wherein the physical object is visible in the area of the virtual or augmented reality object.

10 . The method of claim 9 , wherein the object of interest computer program receives the virtual or augmented reality object from the virtual reality or augmented reality computer program.

11 . The method of claim 9 , further comprising:
determining, by the object of interest computer program, that the physical object is an object of interest; wherein the object of interest comprises a dangerous, hazardous, or harmful object.

12 . The method of claim 11 , wherein the step of determining, by the object of interest computer program, that the physical object is at least partially obscured by the virtual or augmented reality object comprises:
determining a location of the physical object and a location of the virtual or augmented reality object; and determining that the location of the virtual or augmented reality object is between the electronic device and the physical object.

13 . The method of claim 12 , wherein the object of interest computer program determines a location of the physical object using a ray cast query.

14 . The method of claim 12 , wherein the object of interest computer program determines a location of the physical object using LiDAR.

15 . An electronic device, comprising:
a display; an object of interest computer program; and a computer processor; wherein, when executed by the computer processor, the object of interest computer program causes the computer processor to: receive data for a virtual or augmented reality object to display on an image of a physical environment on the display of the electronic device; receive data for a physical object in the physical environment; determine that the physical object is at least partially obscured by the virtual or augmented reality object; generate a representation of the physical object; and provide the representation of the physical object to a virtual reality or augmented reality computer program executed by the electronic device, wherein the virtual reality or augmented reality computer program is configured to display the physical object over the virtual or augmented reality object.

16 . The electronic device of claim 15 , wherein the object of interest computer program receives the virtual or augmented reality object from the virtual reality or augmented reality computer program.

17 . The electronic device of claim 15 , wherein the object of interest comprises a dangerous, hazardous, or harmful object.

18 . The electronic device of claim 15 , wherein the object of interest computer program causes the computer processor to determine that the physical object is at least partially obscured by the virtual or augmented reality object by determining a location of the physical object and a location of the virtual or augmented reality object and determining that the location of the virtual or augmented reality object is between the electronic device and the physical object.

19 . The electronic device of claim 18 , wherein location of the physical object is determined using a ray cast query.

20 . The electronic device of claim 18 , further comprising a LiDAR transceiver, and wherein the location of the physical object is determined using LiDAR from the LiDAR transceiver.

Assignees

• Jpmorgan Chase Bank Na

Inventors

• Pistoia Marco
• Moriarty Bill
• Hu Shaohan
• Kalsi Hargun
• Arantes Aniella

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T11/00 Primary

  Two-dimensional [2D] image generation · CPC title

• G01S17/89

  for mapping or imaging · CPC title