Video stabilization

Patent metadata

Field	Value
Publication number	US-10616486-B2
Application number	US-201816059993-A
Country	US
Kind code	B2
Filing date	Aug 9, 2018
Priority date	Aug 9, 2018
Publication date	Apr 7, 2020
Grant date	Apr 7, 2020

Abstract

A computing system can receive a stabilized image stream, wherein the stabilized image stream is drift-corrected based on determining that an input image stream is stable and then applying drift correction to maintain a stabilized field of view, wherein the field of view is stabilized with respect to the real world. The computing system can operate a vehicle based on determining at least one moving object in the stabilized image stream.

First claim

We claim:

1. A method, comprising:
receiving a video data stream, wherein the video data stream is drift-corrected based on determining that an input video data stream is stable based on determining an eccentricity ε k at a time k and comparing the eccentricity ε k to an empirically determined threshold value, where eccentricity ε k measures a rate at which data points associated with a pixel location x k are changing as a function of time k, and then applying drift correction to maintain a stabilized field of view, wherein the field of view is stabilized with respect to real world coordinates; and
operating a vehicle based on determining at least one moving object in a stabilized image stream.

2. The method of claim 1 , wherein the eccentricity ε k is based on recursively processing the input video data stream based on an empirically determined constant α, an input video data stream data point at time k x k , a mean of x k at time k μ k , and variance of x k at time k σ k 2 .

3. The method of claim 2 , further comprising determining the moving object in the stabilized video data stream based on determining eccentricity ε k and comparing eccentricity ε k to a constant proportional to α.

4. The method of claim 2 , further comprising determining eccentricity ε k based on recursively updating mean μ k based on a previous mean μ k-1 and weighted video data stream data points αx k that assign decreasing weights to older video data stream data points x k .

5. The method of claim 2 , further comprising determining eccentricity ε k based on recursively updating variance σ k 2 for video data stream data points based on the constant α, the input video data stream data point x k , and the mean μ k .

6. The method of claim 1 , wherein operating the vehicle based on determining at least one moving object in the video data stream includes determining a 3D location of the moving object with respect to the vehicle.

7. The method of claim 6 , wherein the video data stream is acquired by a stationary video camera and determining the 3D location of the moving object with respect to the vehicle includes determining a location and a direction of the stationary video camera with respect to the vehicle.

8. The method of claim 7 , wherein determining location and a direction of the stationary video camera with respect to the vehicle includes determining a 3D pose of the stationary video camera.

9. A system, comprising a processor; and
a memory, the memory including instructions to be executed by the processor to:
receive a video data stream, wherein the video data stream is drift-corrected based on determining that an input video data stream is stable based on determining an eccentricity ε k at a time k and comparing the eccentricity ε k to an empirically determined threshold value, where eccentricity ε k measures a rate at which data points associated with a pixel location x k are changing as a function of time k, and then applying drift correction to maintain a stabilized field of view, wherein the field of view is stabilized with respect to real world coordinates; and
operate a vehicle based on determining at least one moving object in a stabilized video data stream.

10. The system of claim 9 , wherein the eccentricity ε k is based on recursively processing the input video data stream based on an empirically determined constant α, an input image stream data point at time k x k , a mean of x k at time k μ k , and variance of x k at time k σ k 2 .

11. The system of claim 10 , the instructions further comprising instructions to determine the moving object in the stabilized video data stream based on determining eccentricity ε k and comparing eccentricity ε k to a constant proportional to α.

12. The system of claim 10 , the instructions further comprising instructions to determine eccentricity ε k based on recursively updating mean μ k based on a previous mean μ k-1 and weighted stationary video data stream data points αx k that assign decreasing weights to older stationary video data stream data points x k .

13. The system of claim 10 , the instructions further comprising instructions to determine eccentricity ε k based on recursively updating variance σ k 2 for stabilized video data stream data points based on the constant α, the input video data stream data point x k , and the mean μ k .

14. The system of claim 9 , wherein operating the vehicle based on determining at least one moving object in the stabilized video data stream includes determining a 3D location of the moving object with respect to the vehicle.

15. The system of claim 14 , wherein the stationary video data stream is acquired by a stationary video camera and determining the 3D location of the moving object with respect to the vehicle includes determining a location and a direction of the stationary video camera with respect to the vehicle.

16. The system of claim 15 , wherein determining location and a direction of the stationary video camera with respect to the vehicle includes determining a 3D pose of the stationary video camera.

17. A system, comprising:
means for controlling vehicle steering, braking, and powertrain; and
computer means for:
receiving a stabilized video data stream, wherein the stabilized video data stream is drift-corrected based on determining that an input video data stream is stable based on determining an eccentricity ε k at a time k and comparing the eccentricity ε k to an empirically determined threshold value, where eccentricity ε k measures a rate at which data points associated with a pixel location x k are changing as a function of time k, and then applying drift correction to maintain a stabilized field of view, wherein the field of view is stabilized with respect to real world coordinates; and
operating a vehicle based on determining at least one moving object in a stabilized image stream and the means for controlling vehicle steering, braking, and powertrain.

Assignees

• Ford Global Tech Llc

Inventors

• Corona Enrique
• Giardinelli Stephen
• Jales Costa Bruno Sielly
• Parchami Mostafa
• Puskorius Gintaras Vincent

Classifications

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/30236

  Traffic on road, railway or crossing · CPC title

• G06T2207/30232

  Surveillance · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title