Apparatus and a method for obtaining a registration error map representing a level of sharpness of an image

Patent metadata

Field	Value
Publication number	US-11694349-B2
Application number	US-202017028180-A
Country	US
Kind code	B2
Filing date	Sep 22, 2020
Priority date	Jun 17, 2015
Publication date	Jul 4, 2023
Grant date	Jul 4, 2023

Abstract

The present invention generally relates to an apparatus and a method for obtaining a registration error map representing a level of sharpness of an image. Many methods are known which allow determining the position of a camera with respect to an object, based on the knowledge of a 3D model of the object and the intrinsic parameters of the camera. However, regardless of the visual servoing technique used, there is no control in the image space and the object may get out of the camera field of view during servoing. It is proposed to obtain a registration error map relating to an image of the object of interest generated by computing an intersection of a re-focusing surface obtained from a 3D model of said object of interest and a focal stack based on acquired four-dimensional light-field data relating to said object of interest.

First claim

The invention claimed is:

1. A method comprising:
obtaining a 3D model of an object of interest associated with a scene;
generating a re-focusing surface based on the 3D model;
obtaining a 4D light field data of the scene;
re-focusing the 4D light field data using the re-focusing surface;
generating a re-focused image by using the re-focused 4D light field data; and
determining whether an end user is correctly positioned relative to the scene by using the re-focused image.

2. The method of claim 1 , wherein the 3D model of the object of interest is obtained from a selection by the end user.

3. The method of claim 1 , wherein the 3D model of the object of interest is obtained from a selection of a code which identifies the scene.

4. The method of claim 1 , wherein generating a re-focusing surface based on the 3D model comprises:
positioning the 3D model in a reference position based on a pre-selected point of view of the object of interest; and
computing a distance map based on the positioned 3D model.

5. The method of claim 1 , wherein the end user is a human user.

6. The method of claim 5 , wherein the re-focused image is displayed on a mobile device such that the human user views the re-focused image.

7. The method of claim 1 , wherein the end user is a robot.

8. The method of claim 1 , wherein the 4D light field data of the scene is obtained by using a camera device from a current point of view of the end user.

9. The method of claim 1 , wherein determining whether an end user is correctly positioned relative to the scene by using the re-focused image comprises:
generating a registration error map wherein pixels of the registration error map indicate a degree of fuzziness of points in the re-focused image.

10. A non-transitory computer readable medium comprising computer-executable instructions to enable a processor to perform the method of claim 1 .

11. An apparatus comprising:
a processor configured for:
obtaining a 3D model of an object of interest associated with a scene;
generating a re-focusing surface based on the 3D model;
obtaining a 4D light field data of the scene;
re-focusing the 4D light field data using the re-focusing surface;
generating a re-focused image by using the re-focused 4D light field data; and
determining whether an end user is correctly positioned relative to the scene by using the re-focused image.

12. The apparatus of claim 11 , wherein the 3D model of the object of interest is obtained from a selection by the end user.

13. The apparatus of claim 11 , wherein the 3D model of the object of interest is obtained from a selection of a code which identifies the scene.

14. The apparatus of claim 11 , wherein generating a re-focusing surface based on the 3D model comprises:
positioning the 3D model in a reference position based on a pre-selected point of view of the object of interest; and
computing a distance map based on the positioned 3D model.

15. The apparatus of claim 11 , wherein the end user is a human user.

16. The apparatus of claim 15 , wherein the re-focused image is displayed on a mobile device such that the human user views the re-focused image.

17. The apparatus of claim 11 , wherein the end user is a robot.

18. The apparatus of claim 11 , wherein the 4D light field data of the scene is obtained by using a camera device from a current point of view of the end user.

19. The apparatus of claim 11 , wherein determining whether an end user is correctly positioned relative to the scene by using the re-focused image comprises:
generating a registration error map wherein pixels of the registration error map indicate a degree of fuzziness of points in the re-focused image.

20. A non-transitory computer readable medium storing data content generated by the apparatus of claim 11 .

Assignees

• Interdigital Ce Patent Holdings

Inventors

• Fleureau Julien
• Hellier Pierre
• Vandame Benoit

Classifications

• H04N13/232

  using fly-eye lenses, e.g. arrangements of circular lenses · CPC title

• G06T7/001

  using an image reference approach · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/30244

  Camera pose · CPC title