Surface profile estimation and bump detection for autonomous machine

Patent metadata

Field	Value
Publication number	US-11900629-B2
Application number	US-202318174770-A
Country	US
Kind code	B2
Filing date	Feb 27, 2023
Priority date	Dec 11, 2019
Publication date	Feb 13, 2024
Grant date	Feb 13, 2024

Abstract

In various examples, surface profile estimation and bump detection may be performed based on a three-dimensional (3D) point cloud. The 3D point cloud may be filtered in view of a portion of an environment including drivable free-space, and within a threshold height to factor out other objects or obstacles other than a driving surface and protuberances thereon. The 3D point cloud may be analyzed—e.g., using a sliding window of bounding shapes along a longitudinal or other heading direction—to determine one-dimensional (1D) signal profiles corresponding to heights along the driving surface. The profile itself may be used by a vehicle—e.g., an autonomous or semi-autonomous vehicle—to help in navigating the environment, and/or the profile may be used to detect bumps, humps, and/or other protuberances along the driving surface, in addition to a location, orientation, and geometry thereof.

First claim

What is claimed is:

1. A method comprising:
generating, based at least on sensor data generated using one or more sensors of a machine, a point cloud associated with an environment;
determining, using one or more bounding shapes and based at least on one or more points of the point cloud, a location of a perturbation within the environment; and
causing, based at least on the location of the perturbation, the machine to perform one or more operations.

2. The method of claim 1 , further comprising:
determining a free-space region associated with the environment,
wherein the point cloud is associated with the free-space region associated with the environment.

3. The method of claim 1 , wherein the determining the location of the perturbation within the environment comprises:
determining, using the one or more bounding shapes and based at least on the one or more points of the point cloud, one or more heights relative to a surface; and
determining, based at least on the one or more heights relative to the surface, the location of the perturbation within the environment.

4. The method of claim 3 , wherein the determining the location of the perturbation within the environment comprises:
determining that the one or more heights satisfy a threshold height above the surface; and
determining, based at least on the one or more heights satisfying the threshold height above the surface, the location of the perturbation as corresponding to at least one location of one or more locations associated with the one or more bounding shapes within the environment.

5. The method of claim 3 , wherein the determining the location of the perturbation within the environment comprises:
determining that the one or more heights satisfy a threshold height below the surface; and
determining, based at least on the one or more heights satisfying the threshold height below the surface, the location of the perturbation as corresponding to at least one location of one or more locations associated with the one or more bounding shapes within the environment.

6. The method of claim 1 , wherein the determining the location of the perturbation within the environment comprises:
determining, using a first bounding shape of the one or more bounding shapes and based at least on one or more first points of the one or more points, a first height relative to a surface;
determining, using a second bounding shape of the one or more bounding shapes and based at least on one or more second points of the one or more points, a second height relative to the surface;
determining that a first location associated with the first bounding shape is within a threshold distance to a second location associated with the second bounding shape; and
determining, based at least on the first location being within the threshold distance to the second location, the location of the perturbation within the environment using the first height relative to the surface and the second height relative to the surface.

7. The method of claim 1 , further comprising:
determining, based at least on the location of the perturbation, a geometry associated with the perturbation,
wherein the causing the machine to perform the one or more operations is based at least on the geometry of the perturbation.

8. The method of claim 1 , wherein the determining the location of the perturbation within the environment comprises:
determining, using the one or more bounding shapes and based at least on the one or more points of the point cloud, a surface profile associated with the environment; and
determining, based at least on the surface profile, the location of the perturbation within the environment.

9. The method of claim 1 , further comprising:
generating a heatmap indicating at least the location of the perturbation within the environment,
wherein the causing the machine to perform the one or more operations is based at least on the heatmap.

10. The method of claim 1 , wherein the perturbation comprises at least one of a bump, a hump, a dip, or a hole associated with a road of the environment.

11. A system comprising:
one or more processing units to:
generate, based at least on sensor data generated using one or more sensors of a machine, a point cloud associated with an environment;
determine, using one or more bounding shapes and based at least on one or more points of the point cloud, a surface profile associated with the environment; and
cause, based at least on the surface profile, the machine to perform one or more operations.

12. The system of claim 11 , wherein the one or more processing units are further to:
determine a free-space region associated with the environment,
wherein the point cloud is associated with the free-space region associated with the environment.

13. The system of claim 11 , wherein the determination of the surface profile associated with the environment comprises:
determining, using the one or more bounding shapes and based at least on the one or more points of the point cloud, one or more heights relative to a surface; and
determining, based at least on the one or more heights relative to the surface, the surface profile associated with the environment.

14. The system of claim 11 , wherein the one or more bounding shapes include at least a first bounding shape located a first distance from the machine and a second bounding shape located a second distance from the machine.

15. The system of claim 11 , wherein the one or more processing units are further to:
determine, based at least on the surface profile, a location of a perturbation within the environment,
wherein the machine is caused to perform the one or more operations based at least on the location of the perturbation.

16. The system of claim 11 , wherein the one or more processing units are further to:
determine, based at least on the surface profile, a geometry of a perturbation within the environment,
wherein the machine is caused to perform the one or more operations based at least on the geometry of the perturbation.

17. The system of claim 11 , wherein the system is implemented within at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing deep learning operations;
a system implemented using an edge device;
a system incorporating one or more virtual machines (VMs);
a system implemented using a robot;
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.

18. A processor comprising:
one or more processing units to cause a machine to perform one or more operations based on at least one of a location of a perturbation within an environment or a geometry of the perturbation within the environment, wherein the at least one of the location or the geometry is determined using one or more bounding shapes and based at least on a point cloud associated with the environment.

19. The processor of claim 18 , wherein the one or more processing units are further to:
determine, using the one or more bounding shapes and based at least on one or more points of the point cloud, one or more heights relative to a surface,
wherein the at least one of the location or the geometry of the perturbation within the environment is determined based at least on the one or more heights relative to the surface.

20. The processor of claim

Assignees

• Nvidia Corp

Inventors

• Park Minwoo
• Wu Yue
• Grabner Michael
• Yang Cheng-Chieh

Classifications

• G06T7/60 Primary

  Analysis of geometric attributes · CPC title

• G06T7/579 Primary

  from motion · CPC title

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G06T2200/08

  involving all processing steps from image acquisition to 3D model generation · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title