Imagery and annotations

Patent metadata

Field	Value
Publication number	US-9818150-B2
Application number	US-201414152925-A
Country	US
Kind code	B2
Filing date	Jan 10, 2014
Priority date	Apr 5, 2013
Publication date	Nov 14, 2017
Grant date	Nov 14, 2017

Abstract

A decade from now, a visit to the supermarket will be a very different experience than the familiar experiences of decades past. Product packaging will come alive with interactivity—each object a portal into a rich tapestry of experiences, with contributions authored by the product brand, by the store selling the product, and by other shoppers. The present technology concerns arrangements for authoring and delivering such experiences. A great variety of other features and technologies are also detailed.

First claim

The invention claimed is:

1. A method involving an on-screen display of differently-oriented views of a 3D product captured by a camera of a user's mobile device, the method comprising the acts:
receiving a first camera-captured view of the 3D product, captured when the product has a first pose relative to the camera, said first camera-captured view depicting the 3D product at a first orientation; and augmenting display of said first view on a screen of the user's mobile device with an annotation, the annotation being associated with a first anchoring feature on the 3D product, the anchoring feature being a particular location within a face of the product depicted in the first view, the annotation being positioned to the right of the displayed 3D product on said screen;
receiving a second camera-captured view of the 3D product, captured when the product has a second pose relative to the camera, said second camera-captured view depicting the 3D product at a second orientation and again depicting said first anchoring feature; and augmenting display of said second view on the screen of the user's mobile device with said annotation, said annotation still being associated with said first anchoring feature, but being positioned to the left of the displayed 3D product on said screen, the annotation thus moving from the right side of the displayed 3D product to the left side; and
receiving a third camera-captured view of the 3D product, captured when the product has a third pose relative to the camera, said third camera-captured view depicting the 3D product at a third orientation and again depicting said first anchoring feature; and displaying the third view on the screen of the user's mobile device without said annotation, despite depiction of said first anchoring feature in said third camera-captured view;
wherein varying presentation of the annotation with variations in product-camera pose helps serve the function of drawing the user's attention to the annotation.

2. The method of claim 1 that includes sensing said first and second orientations by analyzing image data depicting said product, using a hardware processor configured to perform such act.

3. The method of claim 1 in which the annotation is spatially associated with the first anchoring feature on the product.

4. The method of claim 1 in which the product includes a first face comprising said first anchoring feature, and the method includes:
when the product is displayed with the first face oriented within a threshold angular value of a viewpoint, augmenting the on-screen display with said annotation; and
when the product is displayed with the first face oriented outside said threshold angular value, displaying the product without said annotation;
wherein the annotation disappears when the first face is turned away from the viewpoint.

5. The method of claim 1 wherein the product comprises a retail package box.

6. A method involving an on-screen display of differently-oriented views of a 3D object that includes first and second anchoring features on a common planar face, each of said anchoring features being a particular location within said face of the 3D object, the method comprising the acts:
augmenting the on-screen display with a first annotation associated with the first anchoring feature, and with a second annotation associated with the second anchoring feature, while the orientation of the 3D object relative to the camera is varied between first, second and third orientations, wherein:
when the object is displayed at the first orientation, presenting both the first and second annotations to the right of the displayed 3D object;
when the object is displayed at the second orientation, presenting both the first and second annotations to the left of the displayed object, the annotations thus moving from the right side of the displayed 3D product to the left side; and
when the object is displayed at the third orientation, which is between the first and second orientations, presenting the first annotation to the right of the displayed object, and presenting the second annotation to the left of the displayed object;
wherein varying presentation of the first and second annotations with variations in object-camera orientation helps serve the function of drawing the user's attention to the annotations.

7. The method of claim 6 wherein the object comprises a retail package box.

8. The method of claim 6 that includes sensing said first and second orientations by analyzing image data depicting said object, using a hardware processor configured to perform such act.

9. A method involving an on-screen display of differently-oriented views of a 3D object having a planar face, the method comprising the acts:
determining orientation information indicating an orientation at which said face of the displayed 3D object is viewed;
comparing said determined orientation information with stored reference information, using a hardware processor configured to perform such act, the stored reference information serving to specify a threshold angular value datum, said datum defining orientations at which an annotation should be rendered; and
when said comparing has a first outcome, augmenting the on-screen display by presenting the annotation, and when said comparing has a second outcome, not augmenting the on-screen display with the annotation;
wherein presentation of the annotation varies depending both on said determined orientation information, and on said threshold angular value datum, wherein such varying presentation helps serve the function of drawing a user's attention to the annotation; and
wherein the annotation is associated with an anchoring feature on said planar face.

10. The method of claim 9 in which the object includes first and second features on a common planar face of the object, and the method includes:
determining information indicating viewing orientations of the first and second features;
comparing said determined information with stored first information for the first feature, and with stored second information for the second feature; and
at a first time, based on a result of said comparing, augmenting the on-screen display by presenting a first annotation corresponding to the first feature, but not augmenting the on-screen display by presenting a second annotation corresponding to the second feature.

11. The method of claim 10 that includes, at a different time, with a differently-oriented view of the object:
based on a result of said comparing, augmenting the on-screen display by presenting a first annotation corresponding to the first feature, and also augmenting the on-screen display by presenting a second annotation corresponding to the second feature.

12. The method of claim 9 wherein the object comprises a retail package box.

13. The method of claim 9 that includes sensing said first and second orientations by analyzing image data depicting said object, using a hardware processor configured to perform such act.

14. The method of claim 9 in which the object comprises a packaged food item, and the method further includes responding to user selection of the annotation by rendering a nutrition facts graphic on the screen, said rendering including a visual transition in which the annotation expands out from one of the corners of the display screen.

15. The method of claim 9 in which the object comprises a packaged food item that includes first and second features on a common planar face thereof, the first and second features having first and second annotations respectively associated therewith, the method further including:
when s

Assignees

• Digimarc Corp

Inventors

• Rhoads Geoffrey B
• Conwell William Y

Classifications

• G06F17/30

  Physics · mapped topic

• G06Q30/0643 Primary

  graphically representing goods, e.g. 3D product representation · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F16/00

  Information retrieval; Database structures therefor; File system structures therefor · CPC title