Homography error correction

Patent metadata

Field	Value
Publication number	US-12272091-B2
Application number	US-202418431643-A
Country	US
Kind code	B2
Filing date	Feb 2, 2024
Priority date	Oct 25, 2019
Publication date	Apr 8, 2025
Grant date	Apr 8, 2025

Abstract

An object tracking system includes a sensor and a controller. The sensor captures frames for a global plane for at least a portion of a space. The controller has a memory that stores a homography associated with the sensor and a processor. The processor receives a first frame from the sensor and identifies a pixel location within the first frame. The processor determines an estimated sensor location for the sensor by applying the homography to the pixel location, and determines a location difference between the estimated sensor location and an actual sensor location for the sensor. The processor then compares the location difference to a difference threshold level, and recomputes the homography in response to determining that the location difference exceeds the difference threshold level.

First claim

The invention claimed is:

1. An object tracking system, comprising:
a sensor configured to capture frames of a global plane for at least a portion of a space, wherein:
the global plane represents (x,y) coordinates for the at least a portion of the space;
each frame comprises a plurality of pixels; and
each pixel from the plurality of pixels is associated with a pixel location comprising a pixel row and a pixel column; and

a controller comprising:
one or more memories operable to store a homography associated with the sensor, wherein the homography comprises coefficients that translate between pixel locations in a frame and (x,y) coordinates in the global plane; and
one or more processors operably coupled to the one or more memories, and configured to:
receive a first frame from the sensor;
identify a pixel location within the first frame;
determine an estimated sensor location for the sensor by applying the homography to the pixel location;
determine a location difference between the estimated sensor location and an actual sensor location for the sensor;
compare the location difference to a difference threshold level; and
recompute the homography in response to determining that the location difference exceeds the difference threshold level.

2. The system of claim 1 , wherein the pixel location corresponds with a pixel in the center of the frame.

3. The system of claim 1 , further comprising a position sensor operably coupled to the sensor, wherein the position sensor is configured to output the actual sensor location for the sensor.

4. The system of claim 1 , wherein:
identifying the pixel location within the first frame comprises detecting a first marker within the first frame; and
the pixel location corresponds with a location of the first marker within the first frame.

5. The system of claim 1 , wherein the sensor is further configured to capture depth information.

6. The system of claim 1 , wherein the global plane is parallel with a plane of the space.

7. The system of claim 1 , wherein recomputing the homography comprises:
receive a first (x,y) coordinate identifying a first x-value and a first y-value in the global plane where a first marker is located in the space, wherein the first marker is a first object identifying a first location in the space;
receive a second (x,y) coordinate identifying a second x-value and a second y-value in the global plane where a second marker is located in the space, wherein the second marker is a second object identifying a second location in the space;
receive a second frame from the sensor;
identify the first marker and the second marker within the second frame;
determine a first pixel location in the second frame for the first marker, wherein the first pixel location comprises a first pixel row and a first pixel column of the second frame;
determine a second pixel location in the second frame for the second marker, wherein the second pixel location comprises a second pixel row and a second pixel column of the second frame; and
recompute the homography based on the first (x,y) coordinate, the second (x,y) coordinate, the first pixel location, and the second pixel location.

8. A homography error correction method, comprising:
receiving a first frame from a sensor, wherein the sensor configured to capture frames of a global plane for at least a portion of a space, wherein:
the global plane represents (x,y) coordinates for the at least a portion of the space;
each frame comprises a plurality of pixels; and
each pixel from the plurality of pixels is associated with a pixel location comprising a pixel row and a pixel column;

identifying a pixel location within the first frame;
determining an estimated sensor location for the sensor by applying a homography to the pixel location, wherein:
the homography is associated with the sensor; and
the homography comprises coefficients that translate between pixel locations in a frame and (x,y) coordinates in the global plane;

determining a location difference between the estimated sensor location and an actual sensor location for the sensor;
comparing the location difference to a difference threshold level; and
recomputing the homography in response to determining that the location difference exceeds the difference threshold level.

9. The method of claim 8 , wherein the pixel location corresponds with a pixel in the center of the frame.

10. The method of claim 8 , wherein the actual sensor location is determined by a position sensor.

11. The method of claim 8 , wherein:
identifying the pixel location within the first frame comprises detecting a first marker within the first frame; and
the pixel location corresponds with a location of the first marker within the first frame.

12. The method of claim 8 , wherein the sensor is further configured to capture depth information.

13. The method of claim 8 , wherein the global plane is parallel with a plane of the space.

14. The method of claim 8 , wherein recomputing the homography comprises:
receiving a first (x,y) coordinate identifying a first x-value and a first y-value in the global plane where a first marker is located in the space, wherein the first marker is a first object identifying a first location in the space;
receiving a second (x,y) coordinate identifying a second x-value and a second y-value in the global plane where a second marker is located in the space, wherein the second marker is a second object identifying a second location in the space;
receiving a second frame from the sensor;
identifying the first marker and the second marker within the second frame;
determining a first pixel location in the second frame for the first marker, wherein the first pixel location comprises a first pixel row and a first pixel column of the second frame;
determining a second pixel location in the second frame for the second marker, wherein the second pixel location comprises a second pixel row and a second pixel column of the second frame; and
recomputing the homography based on the first (x,y) coordinate, the second (x,y) coordinate, the first pixel location, and the second pixel location.

15. A non-transitory computer-readable medium storing instructions that when executed by a processor cause the processor to:
receive a first frame from a sensor that captures frames of a global plane for at least a portion of a space, wherein:
the global plane represents (x,y) coordinates for the at least a portion of the space;
each frame comprises a plurality of pixels; and
each pixel from the plurality of pixels is associated with a pixel location comprising a pixel row and a pixel column;

identify a pixel location within the first frame;
determine an estimated sensor location for the sensor by applying a homography to the pixel location, wherein:
the homography is associated with the sensor; and
the homography comprises coefficients that translate between pixel locations in a frame and (x,y) coordinates in the global plane;

determine a location difference between the estimated sensor location and an actual sensor location for the sensor;
compare the location difference to a difference threshold level; and
recompute the homography in response to determining that the location difference exceeds the difference threshold level.

16. The non-transitory computer-readable medium of claim 15 , wherein the pixel location corresponds with a pixel in the center of the frame.

17. The non-transitory computer-readable medium of claim 15 , wherein the actual sensor location is determined by a pos

Assignees

• 7 Eleven Inc

Inventors

• Mirza Shahmeer Ali
• Krishnamurthy Sailesh Bharathwaaj
• Dalal Kyle

Classifications

• G06V40/107

  Static hand or arm · CPC title

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06Q30/0641

  utilising user interfaces specially adapted for shopping · CPC title