System and method for supporting three-dimensional display in first person view (FPV)

Patent metadata

Field	Value
Publication number	US-9894276-B2
Application number	US-201615349678-A
Country	US
Kind code	B2
Filing date	Nov 11, 2016
Priority date	Sep 8, 2015
Publication date	Feb 13, 2018
Grant date	Feb 13, 2018

Abstract

System and method can support three-dimensional display. The system can receive a plurality of image frames, which are captured by an imaging device on a movable object. Furthermore, the system can obtain state information of the imaging device on the movable object, and use the state information to configure a pair of image frames based on the plurality of image frames for supporting a three-dimensional first person view (FPV). Additionally, an image frame selected from the plurality of image frames can be used for a first image frame in the pair of image frames.

First claim

What is claimed is:

1. A method for supporting three-dimensional (3D) display, comprising:
receiving a plurality of image frames, which are captured by an imaging device on a movable object;
obtaining state information of the imaging device on the movable object;
configuring a pair of image frames based on the plurality of image frames using the state information of the imaging device on the movable object, wherein configuring the pair of image frames includes:
selecting an image frame from the plurality of image frames as a first image frame in a pair of image frames;
obtaining a transformation relationship between the first image frame in the pair of image frames and a first image frame in a base pair of image frames, wherein the first image frame in the pair of image frames was captured by the imaging device at a different time than either the first image frame or a second image frame in the base pair of image frames, wherein the transformation relationship includes transforming the first image frame in the base pair of image frames into the first image frame in the pair of image frames; and
generating a second image frame in the pair of image frames by transforming the second image frame in the base pair of image frames using the transformation relationship.

2. The method of claim 1 , wherein:
obtaining state information includes determining state information based on positional and attitude information of the movable object received from a control module for the movable object, and attitude information of the imaging device received from a payload stabilization control module, wherein the payload stabilization control module controls a stabilization system, which stabilizes the imaging device on the movable object.

3. The method of claim 1 , further comprising:
providing the pair of image frames to a displaying device that is adapted to display a three-dimensional (3D) first person view (FPV).

4. The method of claim 1 , wherein:
the first image frame in the pair of image frames is a latest image frame in a stream of image frames.

5. The method of claim 1 , further comprising:
determining a movement of a field of view (FOV) for a three-dimensional (3D) first person view (FPV), based on state information of the imaging device on the movable object.

6. The method of claim 5 , further comprising:
configuring the first image frame in the pair of image frames for a right image in the three-dimensional (3D) FPV, if the FOV for the three-dimensional FPV moves from left to right, and
configuring the first image frame in the pair of image frames for a left image in the three-dimensional (3D) FPV, if the FOV for the three-dimensional FPV moves from right to left.

7. The method of claim 1 , further comprising:
selecting another image frame in the plurality of image frames as a second image frame in the pair of image frames based on a frame count difference.

8. The method of claim 7 , further comprising:
dynamically reconfiguring the frame count difference between the first image frame and the second image frame in the pair of image frames based on the state information of the imaging device on the movable object.

9. The method of claim 7 , further comprising:
allowing a user to configure the frame count difference between the first image frame and the second image frame in the pair of image frames, while viewing a three-dimensional (3D) first person view (FPV).

10. The method of claim 1 , wherein:
the transformation relationship is a homography that is obtained based on matching feature points between the first image frame in the pair of image frames and the first image frame in the base pair of image frames.

11. The method of claim 1 , further comprising:
obtaining another pair of image frames based on the plurality of image frames, wherein an image frame selected from the plurality of image frames is used as a first image frame in said another pair of image frames and a second image frame in said another pair of image frames is obtained based on the second image frame in the pair of image frames and another transformation relationship.

12. The method of claim 1 , further comprising:
determining a second image frame in the pair of image frames based on an image stored in an image cache, wherein the image cache operates to store a portion of a stream of image frames and/or a buffer of image frames.

13. The method of claim 1 , further comprising:
configuring a size for an image cache based on an estimated frame count difference between the first image frame and a second image frame in the pair of image frames.

14. A system for supporting three-dimensional (3D) display, comprising:
one or more microprocessors;
a convertor running on the one or more microprocessors, wherein the convertor operates to:
receive a plurality of image frames, which are captured by an imaging device on a movable object;
obtain state information of the imaging device on the movable object; and
configure a pair of image frames based on the plurality of image frames using the state information of the imaging device on the movable object, wherein configuring the pair of image frames includes:
selecting an image frame from the plurality of image frames as a first image frame in a pair of image frames;
obtaining a transformation relationship between the first image frame in the pair of image frames and a first image frame in a base pair of image frames, wherein the first image frame in the pair of image frames was captured by the imaging device at a different time than either the first image frame or a second image frame in the base pair of image frames, wherein the transformation relationship includes transforming the first image frame in the base pair of image frames into the first image frame in the pair of image frames; and
generating a second image frame in the pair of image frames by transforming the second image frame in the base pair of image frames using the transformation relationship.

15. The system of claim 14 , wherein:
the state information is determined based on positional and attitude information of the movable object received from a control module for the movable object, and attitude information of the imaging device received from a payload stabilization control module, wherein the payload stabilization control module controls a stabilization system, which stabilizes the imaging device on the movable object.

16. The system of claim 14 , wherein:
the convertor operates to provide the pair of image frames to a displaying device that is adapted to display a three-dimensional (3D) first person view (FPV).

17. The system of claim 14 , wherein:
the first image frame in the pair of image frames is a latest image frame in the plurality of image frames.

18. The system of claim 14 , wherein:
the convertor operates to determine a movement of a field of view (FOV) for a three-dimensional first person view (FPV), based on state information of the imaging device on the movable object.

19. The system of claim 14 , wherein:
the convertor operates to
configure the first image frame in the pair of image frames for a right image in a three-dimensional (3D) first person view (FPV), if the FOV for the three-dimensional FPV moves from left to right, and
configure the first image frame in the pair of image frames for a left image in the three-dimensional (3D) FPV, if the FOV for the three-dimensional FPV moves from right to left.

20. The system of claim 14 , wherein:
the convertor operates to select

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Cao Zisheng
• Bao Linchao
• Hu Pan
• Wang Mingyu

Classifications

• H04N23/685

  performed by mechanical compensation · CPC title

• H04N23/68 Primary

  for stable pick-up of the scene, e.g. compensating for camera body vibrations · CPC title

• H04N13/221 Primary

  using the relative movement between cameras and objects · CPC title

• H04N7/183

  for receiving images from a single remote source · CPC title

• H04N13/0221

  Electricity · mapped topic