Removing reflection artifacts from point clouds

Patent metadata

Field	Value
Publication number	US-2024054621-A1
Application number	US-202318356864-A
Country	US
Kind code	A1
Filing date	Jul 21, 2023
Priority date	Aug 10, 2022
Publication date	Feb 15, 2024
Grant date	—

Abstract

A computer-implemented method is provided that includes detecting at least one reflective surface in at least one two-dimensional (2D) image of an environment. The method further includes generating bounding coordinates encompassing the at least one reflective surface in the 2D image. The method further includes projecting the bounding coordinates of the 2D image into a three-dimensional (3D) space of the environment. The method further includes identifying a reflection artifact encompassed by the bounding coordinates in the 3D space. The method further includes removing the reflection artifact identified in the bounding coordinates.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
detecting at least one reflective surface in at least one two-dimensional (2D) image of an environment; generating bounding coordinates encompassing the at least one reflective surface in the 2D image; projecting the bounding coordinates of the 2D image into a three-dimensional (3D) space of the environment; identifying a reflection artifact encompassed by the bounding coordinates in the 3D space; and removing the reflection artifact identified in the bounding coordinates.

2 . The computer-implemented method of claim 1 , wherein an artificial intelligence (AI) model is trained to detect the at least one reflection artifact and generate the bounding coordinates encompassing the at least one reflective surface.

3 . The computer-implemented method of claim 1 , wherein the AI model is trained on a dataset of a plurality of 2D images, the plurality of 2D images comprising a plurality of bounding coordinates respectively encompassing a plurality of reflective surfaces.

4 . The computer-implemented method of claim 1 , wherein removing the reflection artifact identified in the bounding coordinates comprises:
selecting candidate 3D points encompassed by the bounding coordinates in the 3D space; clustering the candidate 3D points by intensity values or reflectance values; and selecting at least one of the candidate 3D points as the reflection artifact based at least in part on a threshold associated with the intensity values or the reflectance values.

5 . The computer-implemented method of claim 1 , wherein removing the reflection artifact identified in the bounding coordinates comprises:
selecting candidate 3D points encompassed by the bounding coordinates in the 3D space; clustering the candidate 3D points by depth values; selecting at least one of the candidate 3D points as the reflection artifact based at least in part on a threshold associated with the depth values.

6 . The computer-implemented method of claim 1 , wherein removing the reflection artifact identified in the bounding coordinates comprises:
fit a plane on the bounding coordinates by using a normal vector of a center point of the bounding coordinates; find a maximum depth within the bounding coordinates; and fit a rectangular volume using the maximum depth.

7 . The computer-implemented method of claim 1 , wherein the 2D image is a panorama image.

8 . The computer-implemented method of claim 1 , wherein the 3D space is a 3D point cloud that is registered with at least one other 3D point cloud of the environment.

9 . A system comprising:
a memory having computer readable instructions; and one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:
detecting at least one reflective surface in at least one two-dimensional (2D) image of an environment;
generating bounding coordinates encompassing the at least one reflective surface in the 2D image;
projecting the bounding coordinates of the 2D image into a three-dimensional (3D) space of the environment;
identifying a reflection artifact encompassed by the bounding coordinates in the 3D space; and
removing the reflection artifact identified in the bounding coordinates.

10 . The system of claim 9 , wherein an artificial intelligence (AI) model is trained to detect the at least one reflection artifact and generate the bounding coordinates encompassing the at least one reflective surface.

11 . The system of claim 9 , wherein the AI model is trained on a dataset of a plurality of 2D images, the plurality of 2D images comprising a plurality of bounding coordinates respectively encompassing a plurality of reflective surfaces.

12 . The system of claim 9 , wherein removing the reflection artifact identified in the bounding coordinates comprises:
selecting candidate 3D points encompassed by the bounding coordinates in the 3D space; clustering the candidate 3D points by intensity values or reflectance values; and selecting at least one of the candidate 3D points as the reflection artifact based at least in part on a threshold associated with the intensity values or the reflectance values.

13 . The system of claim 9 , wherein removing the reflection artifact identified in the bounding coordinates comprises:
selecting candidate 3D points encompassed by the bounding coordinates in the 3D space; clustering the candidate 3D points by depth values; selecting at least one of the candidate 3D points as the reflection artifact based at least in part on a threshold associated with the depth values.

14 . The system of claim 9 , wherein removing the reflection artifact identified in the bounding coordinates comprises:
fit a plane on the bounding coordinates by using a normal vector of a center point of the bounding coordinates; find a maximum depth within the bounding coordinates; and fit a rectangular volume using the maximum depth.

15 . The system of claim 9 , wherein the 2D image is a panorama image.

16 . The system of claim 9 , wherein the 3D space is a 3D point cloud that is registered with at least one other 3D point cloud of the environment.

17 . A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform operations comprising:
detecting at least one reflective surface in at least one two-dimensional (2D) image of an environment; generating bounding coordinates encompassing the at least one reflective surface in the 2D image; projecting the bounding coordinates of the 2D image into a three-dimensional (3D) space of the environment; identifying a reflection artifact encompassed by the bounding coordinates in the 3D space; and removing the reflection artifact identified in the bounding coordinates.

18 . The computer program product of claim 17 , wherein an artificial intelligence (AI) model is trained to detect the at least one reflection artifact and generate the bounding coordinates encompassing the at least one reflective surface.

19 . The computer program product of claim 17 , wherein the AI model is trained on a dataset of a plurality of 2D images, the plurality of 2D images comprising a plurality of bounding coordinates respectively encompassing a plurality of reflective surfaces.

20 . The computer program product of claim 17 , wherein removing the reflection artifact identified in the bounding coordinates comprises:
selecting candidate 3D points encompassed by the bounding coordinates in the 3D space; clustering the candidate 3D points by intensity values or reflectance values; and selecting at least one of the candidate 3D points as the reflection artifact based at least in part on a threshold associated with the intensity values or the reflectance values.

Assignees

• Faro Tech Inc

Inventors

• Azam Raza Ul
• Dube-Dallaire Mathieu
• Pompe Daniel
• Ostapchuk Vitaliy
• Kalburgi Sagar

Classifications

• G06T5/005 Primary

  Physics · mapped topic

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T7/50

  Depth or shape recovery · CPC title

• G06T2210/12

  Bounding box · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title