Learning-based ground position estimation

Patent metadata

Field	Value
Publication number	US-10733758-B2
Application number	US-201816175749-A
Country	US
Kind code	B2
Filing date	Oct 30, 2018
Priority date	Oct 30, 2018
Publication date	Aug 4, 2020
Grant date	Aug 4, 2020

Abstract

Operations of the present disclosure may include receiving a group of images taken by a camera over time in an environment. The operations may also include identifying a first position of an object in a target region of the environment in a first image of the group of images and identifying a second position of the object in a second image of the group of images. Additionally, the operations may include determining an estimated trajectory of the object based on the first position of the object and the second position of the object. The operations may further include, based on the estimated trajectory, estimating a ground position in the environment associated with a starting point of the estimated trajectory of the object. Additionally, the operations may include providing the ground position associated with the starting point of the estimated trajectory of the object for display in a graphical user interface.

First claim

What is claimed is:

1. A method comprising:
receiving a plurality of images taken by a camera over time in an environment;
identifying a first position of an object in a target region of the environment in a first image of the plurality of images;
identifying a second position of the object in a second image of the plurality of images;
determining an estimated trajectory of the object based on the first position of the object and the second position of the object;
based on the estimated trajectory, estimating a ground position in the environment associated with a starting point of the estimated trajectory of the object; and
providing the ground position associated with the starting point of the estimated trajectory of the object for display in a graphical user interface.

2. The method of claim 1 , wherein identifying the object in the target region triggers the determining of the estimated trajectory of the object.

3. The method of claim 1 , wherein identifying the first position of the object in the first image and identifying the second position of the object in the second image include identifying pixels that represent at least a portion of the object.

4. The method of claim 3 , wherein:
identifying pixels that represent at least a portion of the object includes identifying pixels using a K-nearest neighborhood algorithm; and
a center portion of the identified pixels represents a center of gravity of the object.

5. The method of claim 3 , wherein:
identifying positions of the object is performed in a backwards manner from the first image until reaching a stop image in which a center portion of pixels associated with the object is unidentifiable; and
the second image is positioned chronologically between the first image and the stop image.

6. The method of claim 1 , further comprising finding the target region of the environment using a neural network to identify a plurality of definition points corresponding to features within the target region.

7. The method of claim 6 , wherein finding the target region of the environment is in response to movement of one or both of the camera and the target region.

8. The method of claim 1 , wherein the second position of the object corresponds to the starting point of the estimated trajectory of the object, and the method further comprises:
determining a flight time of the object in response to variation in the starting point of the estimated trajectory that exceeds a threshold amount of variation, the flight time determined by counting a number of images between the first image and the second image.

9. The method of claim 1 , wherein determining the estimated trajectory includes fitting the first position and the second position to a curve to identify curve coefficients.

10. The method of claim 9 , wherein estimating the ground position includes inputting into a neural network the curve coefficients and one or both of: a plurality of definition points corresponding to features within the target region and a flight time of the object.

11. A system comprising:
a memory; and
one or more processors communicatively coupled to the memory, the one or more processors being configured to execute instructions stored in the memory to cause the system to perform operations comprising:
receive a plurality of images taken by a camera over time in an environment;
identify a first position of an object in a target region of the environment in a first image of the plurality of images;
identify a second position of the object in a second image of the plurality of images;
determine an estimated trajectory of the object based on the first position of the object and the second position of the object;
based on the estimated trajectory, estimate a ground position in the environment associated with a starting point of the estimated trajectory of the object; and
provide the ground position associated with the starting point of the estimated trajectory of the object for display in a graphical user interface.

12. The system of claim 11 , wherein identifying the object in the target region triggers the determining of the estimated trajectory of the object.

13. The system of claim 11 , wherein identifying the first position of the object in the first image and identifying the second position of the object in the second image include identifying pixels that represent at least a portion of the object.

14. The system of claim 13 , wherein:
identifying pixels that represent at least a portion of the object includes identifying pixels using a K-nearest neighborhood algorithm; and
a center portion of the identified pixels represents a center of gravity of the object.

15. The system of claim 13 , wherein:
identifying positions of the object is performed in a backwards manner from the first image until reaching a stop image in which a center portion of pixels associated with the object is unidentifiable; and
the second image is positioned chronologically between the first image and the stop image.

16. The system of claim 11 , wherein execution of the instructions further causes the system to find the target region of the environment using a neural network to identify a plurality of definition points corresponding to features within the target region.

17. The system of claim 16 , wherein finding the target region of the environment is in response to movement of one or both of the camera and the target region.

18. The system of claim 11 , wherein:
the second position of the object corresponds to the starting point of the estimated trajectory of the object; and
execution of the instructions further causes the system to determine a flight time of the object in response to variation in the starting point of the estimated trajectory that exceeds a threshold amount of variation, the flight time determined by counting a number of images between the first image and the second image.

19. The system of claim 11 , wherein determining the estimated trajectory includes fitting the first position and the second position to a curve to identify curve coefficients.

20. A system, comprising:
one or more processors; and
non-transitory computer readable media that include instructions thereon that, in response to execution by the one or more processors, control performance of operations comprising:
receive a plurality of images taken by a camera over time in an environment;
identify a first position of an object in a target region of the environment in a first image of the plurality of images;
identify a second position of the object in a second image of the plurality of images;
determine an estimated trajectory of the object based on the first position of the object and the second position of the object;
based on the estimated trajectory, estimate a ground position in the environment associated with a starting point of the estimated trajectory of the object; and
provide the ground position associated with the starting point of the estimated trajectory of the object for display in a graphical user interface.

Assignees

• Rapsodo Pte Ltd

Inventors

• Jin Xu
• Tao Xi
• Zhang Boyi
• Okur Batuhan

Classifications

• G06T2207/30224

  Ball; Puck · CPC title

• G06T2207/30241

  Trajectory · CPC title

• G06T7/248

  involving reference images or patches · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title