Learning-based ground position estimation

Patent metadata

Field	Value
Publication number	US-11276200-B2
Application number	US-202016985109-A
Country	US
Kind code	B2
Filing date	Aug 4, 2020
Priority date	Oct 30, 2018
Publication date	Mar 15, 2022
Grant date	Mar 15, 2022

Abstract

Operations of the present disclosure may include receiving a group of images taken by a camera over time in an environment. The operations may also include identifying a first position of an object in a target region of the environment in a first image of the group of images and identifying a second position of the object in a second image of the group of images. Additionally, the operations may include determining an estimated trajectory of the object based on the first position of the object and the second position of the object. The operations may further include, based on the estimated trajectory, estimating a ground position in the environment associated with a starting point of the estimated trajectory of the object. Additionally, the operations may include providing the ground position associated with the starting point of the estimated trajectory of the object for display in a graphical user interface.

First claim

What is claimed is:

1. A method comprising:
determining an estimated trajectory of an object in an environment based on a first position of the object and a second position of the object identified from images of the object in the environment;
based on the estimated trajectory, estimating a ground position in the environment associated with a starting point of the estimated trajectory of the object; and
providing the ground position associated with the starting point of the estimated trajectory of the object for display in a graphical user interface.

2. The method of claim 1 , further comprising:
identifying the object in a target region of the environment, thereby triggering the determining of the estimated trajectory of the object.

3. The method of claim 2 , further comprising:
finding the target region of the environment using a neural network to identify a plurality of definition points corresponding to features within the target region.

4. The method of claim 3 , wherein finding the target region of the environment is in response to movement of one or both of a camera and the target region.

5. The method of claim 1 , further comprising:
identifying the first position of the object in a first image and identifying the second position of the object in a second image, including identifying pixels that represent at least a portion of the object.

6. The method of claim 5 , wherein:
identifying pixels that represent at least a portion of the object includes identifying pixels using a K-nearest neighborhood algorithm; and
a center portion of the identified pixels represents a center of gravity of the object.

7. The method of claim 5 , wherein:
identifying positions of the object is performed in a backwards manner from the first image until reaching a stop image in which a center portion of pixels associated with the object is unidentifiable; and
the second image is positioned chronologically between the first image and the stop image.

8. The method of claim 5 , wherein the second position of the object corresponds to the starting point of the estimated trajectory of the object, and the method further comprises:
determining a flight time of the object in response to variation in the starting point of the estimated trajectory that exceeds a threshold amount of variation, the flight time determined by counting a number of images between the first image and the second image.

9. The method of claim 1 , wherein determining the estimated trajectory includes fitting the first position and the second position to a curve to identify curve coefficients.

10. The method of claim 9 , wherein estimating the ground position includes inputting into a neural network the curve coefficients and one or both of: a plurality of definition points corresponding to features within the target region and a flight time of the object.

11. A system comprising:
a memory; and
one or more processors communicatively coupled to the memory, the one or more processors being configured to execute instructions stored in the memory to cause the system to perform operations comprising:
determining an estimated trajectory of an object in an environment based on a first position of the object and a second position of the object identified from images of the object in the environment;
based on the estimated trajectory, estimating a ground position in the environment associated with a starting point of the estimated trajectory of the object; and
providing the ground position associated with the starting point of the estimated trajectory of the object for display in a graphical user interface.

12. The system of claim 11 , wherein the operations further comprise:
identifying the object in a target region of the environment, thereby triggering the determining of the estimated trajectory of the object.

13. The system of claim 12 , wherein the operations further comprise:
finding the target region of the environment using a neural network to identify a plurality of definition points corresponding to features within the target region.

14. The system of claim 13 , wherein finding the target region of the environment is in response to movement of one or both of a camera and the target region.

15. The system of claim 11 , wherein the operations further comprise:
identifying the first position of the object in a first image and identifying the second position of the object in a second image, including identifying pixels that represent at least a portion of the object.

16. The system of claim 15 , wherein:
identifying pixels that represent at least a portion of the object includes identifying pixels using a K-nearest neighborhood algorithm; and
a center portion of the identified pixels represents a center of gravity of the object.

17. The system of claim 15 , wherein:
identifying positions of the object is performed in a backwards manner from the first image until reaching a stop image in which a center portion of pixels associated with the object is unidentifiable; and
the second image is positioned chronologically between the first image and the stop image.

18. The system of claim 15 , wherein:
the second position of the object corresponds to the starting point of the estimated trajectory of the object; and
execution of the instructions further causes the system to determine a flight time of the object in response to variation in the starting point of the estimated trajectory that exceeds a threshold amount of variation, the flight time determined by counting a number of images between the first image and the second image.

19. The system of claim 11 , wherein determining the estimated trajectory includes fitting the first position and the second position to a curve to identify curve coefficients.

20. A system, comprising:
one or more processors; and
non-transitory computer readable media that include instructions thereon that, in response to execution by the one or more processors, control performance of operations comprising:
determining an estimated trajectory of an object in an environment based on a first position of the object and a second position of the object identified from images of the object in the environment;
based on the estimated trajectory, estimating a ground position in the environment associated with a starting point of the estimated trajectory of the object; and
providing the ground position associated with the starting point of the estimated trajectory of the object for display in a graphical user interface.

Assignees

• Rapsodo Pte Ltd

Inventors

• Jin Xu
• Tao Xi
• Zhang Boyi
• Okur Batuhan

Classifications

• G06T7/248

  involving reference images or patches · CPC title

• G06T2207/30241

  Trajectory · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/30224

  Ball; Puck · CPC title

• G06T2207/10024

  Color image · CPC title