Determining pose for use with digital watermarking, fingerprinting and augmented reality

Patent metadata

Field	Value
Publication number	US-9684941-B2
Application number	US-201313789126-A
Country	US
Kind code	B2
Filing date	Mar 7, 2013
Priority date	Oct 29, 2012
Publication date	Jun 20, 2017
Grant date	Jun 20, 2017

Abstract

Image recognition and augmented reality experiences utilize auxiliary data extracted from an image or video, or image fingerprints, or a combination of both. One claim recites a method comprising: receiving a plurality of imagery frames captured by a device sensor; identifying a plurality of key points in each of the plurality of imagery frames; resolving image pose by utilizing relative relationships of the key points between different frames of the plurality of frames; modifying imagery of at least one of the plurality of frames based on resolved image pose; and deriving a fingerprint from modified imagery. Of course other claims and combinations are provided as well.

First claim

What is claimed is:

1. A method comprising:
receiving imagery captured by a device sensor, in which the device sensor comprises an image or video camera;
analyzing the imagery to decode digital watermarking therefrom, in which the digital watermarking comprises orientation attributes;
determining orientation information from the orientation attributes, the orientation information being associated with a capture position of the device sensor relative to an imaged subject;
providing the orientation information for use by an augmented reality (AR) system; and
using one or more configured processors, refining the orientation information determined from the orientation attributes with image homography, in which the orientation information is utilized as a starting pose for the image homography.

2. The method of claim 1 further comprising: identifying a plurality of key points in the imagery, in which the key points represent robust image characteristics that are independent of the digital watermarking, in which said refining utilizes pose homographies associated with the key points.

3. The method of claim 2 in which said refining further minimizes a cost function associated with the pose homographies.

4. The method of claim 1 in which the AR system overlays graphics or video on a display screen in an area corresponding to a spatial area in which the digital watermarking was detected from the imagery.

5. The method of claim 1 in which the AR system utilizes the orientation information to transform the imagery for display on a display screen.

6. The method of claim 1 in which the digital watermarking comprises a payload, in which said method further comprises initiating an announcement associated with the AR system utilizing at least a portion of the payload.

7. The method of claim 1 further comprising:
modifying the imagery according to the orientation information to yield modified imagery; and
deriving a fingerprint from the modified imagery, the fingerprint comprising a hash of the modified imagery.

8. The method of claim 7 further comprising providing the fingerprint to the AR system to help facilitate display of graphics or video on a display screen corresponding to a spatial area of modified imagery from which the fingerprint was derived from.

9. A mobile device comprising:
a device sensor comprising an image or video camera;
memory storing instructions;
one or more processors configured by the instructions for:
analyzing imagery captured by said device sensor to extract an encoded signal therefrom, in which the encoded signal comprises orientation attributes;
determining orientation information from the orientation attributes, the orientation information being associated with a capture position of said device sensor relative to an imaged subject;
providing the orientation information for use by an augmented reality (AR) system; and
adapting the orientation information determined from the orientation attributes with image homography, in which the orientation information is utilized as a starting pose for the image homography.

10. The mobile device of claim 9 further comprising the AR system.

11. The mobile device of claim 10 in which said AR system is configured for overlaying graphics or video on a display screen in an area corresponding to a spatial area in which the encoded signal was detected from the imagery.

12. The mobile device of claim 10 in which said mobile device comprises a display screen, and in which said AR system is configured for utilizing the orientation information to transform the imagery for display on said display screen.

13. The mobile device of claim 10 in which the encoded signal comprises a payload, in which at least a portion of the payload is used to initiate an announcement associated with said AR system.

14. The mobile device of claim 10 in which said instructions comprise instructions to configure said one or more processors for:
transforming the imagery according to the orientation information to yield modified imagery; and
extracting a fingerprint from the modified imagery, the fingerprint comprising a hash of the modified imagery.

15. The mobile device of claim 14 further comprising a display screen, and instructions to configure said one or more processors for providing the fingerprint to said AR system to help facilitate display of graphics or video on said display screen corresponding to a spatial area of modified imagery from which the fingerprint was derived from.

16. The mobile device of claim 9 in which said instructions comprise instructions to configure said one or more processors for: identifying a plurality of key points in the imagery, in which the key points represent robust image characteristics that are independent of the encoded signal, in which said adapting utilizes pose homographies associated with the key points.

17. The mobile device of claim 16 in which said instructions comprise instructions to configure said one or more processors for minimizing a cost function associated with the pose homographies.

Assignees

• Digimarc Corp

Inventors

• Filler Tomas

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T1/0064 Primary

  Geometric transfor invariant watermarking, e.g. affine transform invariant · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T2201/0065

  Extraction of an embedded watermark; Reliable detection · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title