Displaying items of interest in an augmented reality environment

Patent metadata

Field	Value
Publication number	US-11935169-B2
Application number	US-202318151184-A
Country	US
Kind code	B2
Filing date	Jan 6, 2023
Priority date	Nov 2, 2016
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

Computer program products, methods, systems, apparatus, and computing entities are provided for an augmented reality display using an actual image of the item. Additionally, the present disclosure provides for a proportionally dimensioned representation of the item in the augmented reality display. In some aspects, a beacon/tag/sensor-based approach may be used. In some aspects, a marker-based approach may be used.

First claim

The invention claimed is:

1. A method comprising:
generating a request for an image of an item, the image of the item being previously captured at a point during transportation of the item, the image being stored based on an item identifier associated with the item;
receiving the image of the item based on the request and a dispatch plan associated with the item; and
generating an augmented reality display layer comprising the image of the item superimposed over a real world background layer, wherein the image of the item is proportionally dimensioned relative to the real world background layer based on a set of dimensions determined for the item from the image of the item captured at the point during transportation of the item, and the dispatch plan is navigable from the augmented reality display layer.

2. The method of claim 1 , wherein the set of dimensions for the item are determined based on a sensor and a camera that are each positioned on a powered asset to capture the image of the item and to determine at least one of a length, a width, a height, or a weight of the item.

3. The method of claim 1 , wherein the item identifier transmits location data associated with the transportation of the item, and the method further comprises:
receiving another image of the item based on the location data from the item identifier; and
updating the augmented reality display layer based on receiving the other image of the item.

4. The method of claim 1 , further comprising:
receiving item data from a powered asset beacon corresponding to a powered asset that is associated with the transportation of the item;
determining a triggering event based on the item data; and
updating the augmented reality display layer based on the triggering event.

5. The method of claim 4 , wherein the item data includes temperature and moisture data.

6. An apparatus comprising:
at least one processor; and
at least one memory including program code, wherein the at least one memory is configured to, with the at least one processor, cause the apparatus to at least:
receive an image of an item based on a request for the image of the item, the image of the item being previously captured at a point during transportation of the item, the request corresponding to an item identifier of the item; and
generate a display layer comprising the image of the item superimposed over a real world background layer, wherein the image of the item is proportionally dimensioned relative to the real world background layer based on a set of dimensions determined for the item from the image of the item captured at the point during transportation of the item, and the display layer indicates a location of the item in a powered asset based on transmissions from a powered asset tag of the powered asset.

7. The apparatus of claim 6 , wherein the set of dimensions include a volumetric measurement.

8. The apparatus of claim 7 , wherein the apparatus is further caused to apply a classification to the item based on the volumetric measurement, and wherein the image of the item is stored and categorized based on the classification.

9. The apparatus of claim 8 , wherein the request includes the classification and item identifier, and the item identifier comprises a barcode.

10. The apparatus of claim 8 , wherein the powered asset tag includes a Wi-Fi access point, and the Wi-Fi access point is used for indicating the location of the item based on triangulation with another Wi-Fi access point located on the item.

11. The apparatus of claim 6 , wherein the apparatus further comprises a head-mounted display having a see-through lens, wherein the display layer is provided on the see-through lens.

12. One or more computer-readable media having stored thereon instructions that when executed by a processor, cause the processor to perform operations comprising:
identifying, via the computing entity, an item based on an item identifier associated with the item;
receiving, via the computing entity, an image of the item that was captured at a point during transportation of the item;
determining a set of dimensions for the item from the image;
applying a classification to the item based on a dimension of the item provided in the set of dimensions, wherein the image of the item is stored and categorized based on the classification; and
causing the computing entity to display a layer comprising the image of the item superimposed over a real world background layer, wherein the image of the item proportionally dimensioned relative to the real world background layer based on the set of dimensions.

13. The media of claim 12 , wherein the layer is oriented to a field of view of the computing entity and relative to the real world background layer being with the field of view of the computing entity.

14. The media of claim 12 , wherein the computing entity is a wearable device.

15. The media of claim 12 , wherein the computing entity includes an accelerometer, and the image of the item is oriented to the field of view based on a detection by the accelerometer.

16. The media of claim 12 , wherein the computing device is a head-mounted display.

17. The media of claim 12 , wherein the layer further includes a handling procedure for the item.

Assignees

• United parcel service america inc

Inventors

• Dotterweich Andrew
• Schenken Christopher T
• Cooper Jeffrey

Classifications

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06K7/10297

  arrangements for handling protocols designed for non-contact record carriers such as RFIDs NFCs, e.g. ISO/IEC 14443 and 18092 (cryptographic protocols H04L9/00; network security protocols H04L63/00; real-time communication protocols in data switching networks H04L65/00; network protocols for data switching network services H04L67/00) · CPC title

• G06K7/1413

  1D bar codes · CPC title

• G06K7/1417

  2D bar codes · CPC title