Systems, methods and, media for determining object motion in three dimensions using speckle images

Patent metadata

Field	Value
Publication number	US-10152798-B2
Application number	US-201715483007-A
Country	US
Kind code	B2
Filing date	Apr 10, 2017
Priority date	Apr 10, 2017
Publication date	Dec 11, 2018
Grant date	Dec 11, 2018

Abstract

In accordance with some embodiments, systems, methods and media for determining object motion in three dimensions using speckle images are provided. In some embodiments, a system for three dimensional motion estimation is provided, comprising: a light source; an image sensor; and a hardware processor programmed to: cause the light source to emit light toward the scene; cause the image sensor to capture a first defocused speckle image of the scene at a first time and capture a second defocused speckle image of the scene at a second time; generate a first scaled version of the first defocused image; generate a second scaled version of the first defocused image; compare each of the first defocused image, the first scaled version, and the second scaled version to the second defocused image; and determine axial and lateral motion of the object based on the comparisons.

First claim

What is claimed is:

1. A system for three dimensional motion estimation, the system comprising:
a light source configured to emit light toward a scene, wherein the light is substantially temporally coherent around a center frequency λ;
an image sensor; and
a hardware processor that is programmed to:
cause the light source to emit light toward the scene;
cause the image sensor to capture a first defocused image of the scene at a first time, wherein the first defocused image includes a first speckle pattern generated by an object in the scene reflecting the light emitted by the light source;
cause the image sensor to capture a second defocused image of the scene at a second time, wherein the second defocused image includes a second speckle pattern generated by the object in the scene reflecting the light emitted by the light source;
generate a first scaled version of the first defocused image by expanding the first defocused image by a predetermined amount;
generate a second scaled version of the first defocused image by contracting the first defocused image by the predetermined amount;
compare the first defocused image to the second defocused image;
compare the first scaled version of the first defocused image to the second defocused image;
compare the second scaled version of the first defocused image to the second defocused image;
determine axial motion of the object between the first time and the second time based on the comparisons; and
determine lateral motion of the object between the first time and second time based on the comparisons.

2. The system of claim 1 , wherein the hardware processor is further programmed to:
calculate, for the first defocused image, a first two-dimensional cross-correlation with the second defocused image, wherein the first two-dimensional cross-correlation includes a peak with a first intensity at a first location, the first intensity indicative of how closely the first speckle pattern in the first defocused image matches the second speckle pattern in the second defocused image and the first location indicative of lateral motion of the object between the first time and the second time;
calculate, for the first scaled version of the first defocused image, a second two-dimensional cross-correlation with the second defocused image, wherein the second two-dimensional cross-correlation includes a peak with a second intensity at a second location, the second intensity indicative of how closely the first speckle pattern in the first scaled version of the first defocused image matches the second speckle pattern in the second defocused image and the second location indicative of lateral motion of the object between the first time and the second time; and
calculate, for the second scaled version of the first defocused image, a third two-dimensional cross-correlation with the second defocused image; wherein the third two-dimensional cross-correlation includes a peak with a third intensity at a third location, the third intensity indicative of how closely the first speckle pattern in the second scaled version of the first defocused image matches the second speckle pattern in the second defocused image and the third location indicative of lateral motion of the object between the first time and the second time.

3. The system of claim 2 , wherein the hardware processor is further programmed to:
compare at least the intensity of the first peak, the second peak, and the third peak; and
select a version of the first defocused image that includes the largest intensity peak; and
determine the axial motion of the object based on the scale of the selected version of the first defocused image.

4. The system of claim 1 , wherein the hardware processor is further programmed to:
receive information indicating that the motion of the object between the first time and the second corresponds to a first hand gesture;
generate motion information indicative of motion of the object between the first time and the second time based on the axial motion and the lateral motion;
provide the motion information as input to a classification model as training data for training the classification model to recognize the first hand gesture input;
generate a trained classification model using the input, wherein the trained classification model is configured to receive motion information of a scene as input and output a likelihood that the received motion information corresponds to the first hand gesture.

5. The system of claim 4 , wherein the hardware processor is further programmed to:
cause the light source to emit light toward a second scene that is different than the scene subsequent to generating the trained classification model;
cause the image sensor to capture a third defocused image of the second scene at a third time, wherein the third defocused image includes a third speckle pattern generated by an object in the second scene reflecting the light emitted by the light source;
cause the image sensor to capture a fourth defocused image of the second scene at a fourth time, wherein the fourth defocused image includes a fourth speckle pattern generated by the object in the second scene reflecting the light emitted by the light source;
generate a first scaled version of the third defocused image by expanding the first defocused image by a predetermined amount;
generate a second scaled version of the third defocused image by contracting the first defocused image by the predetermined amount;
compare the third defocused image to the fourth defocused image;
compare the first scaled version of the third defocused image to the fourth defocused image;
compare the second scaled version of the third defocused image to the fourth defocused image;
determine second axial motion of the object in the second scene between the third time and the fourth time based on the comparisons;
determine second lateral motion of the object in the second scene between the third time and fourth time based on the comparisons;
generate second motion information indicative of motion of the object in the second scene between the third time and the fourth time based on the second axial motion and the second lateral motion;
provide the second motion information as input to the trained classification model; and
receive output from the trained classification model indicating a likelihood that the motion in the second scene corresponds to the first hand gesture.

6. The system of claim 1 , wherein the light source comprises a laser diode.

7. The system of claim 1 , wherein the coherence area of the temporally coherent light at the object is less than 1 mm.

8. The system of claim 1 , wherein the first defocused image includes a first total speckle pattern with contributions from the first speckle pattern and a third speckle pattern generated by a second object in the scene, and
the second defocused image includes a second total speckle pattern with contributions from the second speckle pattern and a fourth speckle pattern generated by the second object in the scene,
wherein the hardware processor is further programmed to:
generate a third scaled version of the first defocused image by expanding the first defocused image by a second predetermined amount;
compare the third scaled version of the first defocused image to the second defocused image;
determine axial motion of the second object between the first time and the second time based on the comparisons; and
determine lateral motion of the second object between the first time and second time based on the comparisons.

9. A method for three dimensional motion estimation, the method comprising:
causing a light source to emit light toward a scene, wherein the light is substantially

Assignees

• Wisconsin Alumni Res Found

Inventors

• Gupta Mohit
• Smith Brandon M
• Desai Pratham H
• Agarwal Vishal R

Classifications

• G06V10/62 Primary

  relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/24

  Classification techniques · CPC title