Method and apparatus for outputting pose information

Patent metadata

Field	Value
Publication number	US-10866427-B2
Application number	US-201916283898-A
Country	US
Kind code	B2
Filing date	Feb 25, 2019
Priority date	Oct 1, 2018
Publication date	Dec 15, 2020
Grant date	Dec 15, 2020

Abstract

A method and apparatus for outputting pose information selects first points from a frames captured by a first sensor, estimates rotation information between the frames based on motion information sensed by a second sensor, corrects the estimated rotation information based on third points, the third points being remaining points when second points corresponding to a dynamic object are excluded from the first points, obtains translation information between the frames based on the third points and the corrected rotation information, and outputs the corrected rotation information and the translation information.

First claim

What is claimed is:

1. A method of obtaining pose information, the method comprising:
selecting first points from frames captured by a first sensor;
estimating initial rotation information between the frames by:
estimating a current state of an ego-motion, using a filter, based on a previous state of the ego-motion and motion information sensed by a second sensor; and
calculating the initial rotation information as an amount of rotation change between a first output of the filter at the previous state and a second output of the filter at the current state;

generating corrected rotation information by,
prior to translation information between the frames being obtained, obtaining third points as a result of excluding, from the first points, based on the estimated initial rotation information, second points corresponding to a dynamic object;
estimating additional rotation information between the frames based on the third points;
estimating a corrected current state of the ego-motion, using the filter, based on the current state of the ego-motion and the additional rotation information; and
calculating the corrected rotation information as an amount of rotation change between the first output of the filter at the previous state and a third output of the filter at the corrected current state;

obtaining the translation information between the frames based on the third points and the corrected rotation information; and
outputting the corrected rotation information and the translation information.

2. The method of claim 1 , wherein the estimating of the initial rotation information comprises:
estimating the initial rotation information of the frames using one of a Kalman filter, an extended Kalman filter, an iterated extended Kalman filter, an unscented Kalman filter, or a particle filter.

3. The method of claim 1 , wherein the second points are identified based on the estimated initial rotation information.

4. The method of claim 1 , wherein the generating of the corrected rotation information further comprises:
correcting the estimated initial rotation information by using the estimated additional rotation information.

5. The method of claim 4 , wherein the correcting of the estimated initial rotation information comprises:
updating a state of an ego-motion corresponding to any one or any combination of the frames based on a difference between the estimated additional rotation information and the estimated initial rotation information.

6. The method of claim 1 , wherein the frames comprise a first frame captured at a first time and a second frame captured at a second time, and
wherein ere the removing of the second points comprises:
calculating a rotation change amount between first rotation information estimated at the first time and second rotation information estimated at the second time; and
removing the second points based on the rotation change amount.

7. The method of claim 1 , wherein the estimating of the additional rotation information comprises:
estimating the additional rotation information by applying one of a Perspective-n-Point (PNP) function, a Gauss-Newton optimization function, or a Levenberg-Marquardt optimization function to the third points.

8. The method of claim 1 , wherein the obtaining of the translation information comprises:
determining the translation information based on the corrected rotation information such that an energy function associated with a difference in intensity between frames to which the third points belong is less than a threshold.

9. The method of claim 8 , wherein the energy function comprises one of a Gauss-Newton optimization function or a Levenberg-Marquardt optimization function.

10. The method of claim 1 , wherein the selecting of the first points comprises either one or both of:
selecting the first points for each block into which the frames are divided and;
selecting the first points for each class, among classes, in the frames.

11. The method of claim 10 , wherein the selecting the first points for each block comprises:
equally dividing each of the frames into blocks; and
selecting the first points from each of the equally divided blocks.

12. The method of claim 10 , wherein the selecting of the first points for each class comprises:
determining candidate points corresponding to the classes based on any one or any combination of any two or more of degrees of intensity gradients of pixels in each of the frames, directions of the intensity gradients of the pixels, and intensity relationships between the pixels and adjacent pixels; and
selecting the first points from the candidate points in the each of the classes.

13. The method of claim 1 , wherein the selecting of the first points comprises:
assigning a first weight to points a, among a plurality of points, at a center of each of the frames;
assigning a second weight to points, among the plurality of points, away from the center and toward an outer edge of each of the frames; and
selecting the first points from the plurality of points based on the first weight and the second weight.

14. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .

15. An apparatus for outputting pose information, the apparatus comprising:
a first sensor configured to capture frames;
a second sensor configured to sense motion information; and
a processor configured to:
estimate initial rotation information between the frames by:
estimating a current state of an ego-motion, using a filter, based on a previous state of the ego-motion and motion information sensed by the second sensor; and
calculating the initial rotation information as an amount of rotation change between a first output of the filter at the previous state and a second output of the filter at the current state;

generating corrected rotation information by,
prior to translation information between the frames being obtained, obtaining third points as a result of excluding, from the first points, based on the estimated initial rotation information, second points corresponding to a dynamic object; and
estimating additional rotation information between the frames based on the third points;
estimating a corrected current state of the ego-motion, using the filter, based on the current state of the ego-motion and the additional rotation information; and
calculating the corrected rotation information as an amount of rotation change between the first output of the filter at the previous state and a third output of the filter at the corrected current state; and

obtaining the translation information between the frames based on the third points and the corrected rotation information.

16. The apparatus of claim 15 , wherein the generating of the corrected rotation information comprises:
correcting the estimated initial rotation information by using the estimated additional rotation information.

17. The apparatus of claim 15 , wherein the processor is further configured to determine the translation information based on the corrected rotation information such that an energy function associated with a difference in intensity between a frames to which the third points belong is less than a threshold.

18. The apparatus of claim 15 , wherein each of the frames comprises an image of a front view from a vehicle.

19. A pose detection apparatus comprising:
a first sensor configured to capture image frames;
a second sensor configured to sense mot

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jung Hwiryong
• Kang Chul Woo
• Moon Hyewon
• Jung Kyungboo
• Hong Sunghoon

Classifications

• G01C21/1656 Primary

  with passive imaging devices, e.g. cameras · CPC title

• G02B27/0179 Primary

  Display position adjusting means not related to the information to be displayed · CPC title

• G06T2207/30248

  Vehicle exterior or interior · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title