Vehicle stain and trash detection systems and methods

Patent metadata

Field	Value
Publication number	US-10304165-B2
Application number	US-201715593880-A
Country	US
Kind code	B2
Filing date	May 12, 2017
Priority date	May 12, 2017
Publication date	May 28, 2019
Grant date	May 28, 2019

Abstract

Example vehicle stain and trash detection systems and methods are described. In one implementation, a method receives a clean image and clean Lidar data associated with a clean vehicle interior. The method also receives a second image and second Lidar data associated with the vehicle interior after a passenger has occupied the vehicle. Differences are identified between the clean image and the second image. Additionally, differences are identified between the clean Lidar data and the second Lidar data. The method determines whether the vehicle interior includes at least one of a stain, dirt, or trash based on the identified differences.

First claim

The invention claimed is:

1. A method comprising:
receiving a clean image and clean Lidar data associated with a clean vehicle interior;
receiving a second image and second Lidar data associated with the vehicle interior after a passenger has occupied the vehicle;
identifying differences between the clean image and the second image;
identifying differences between the clean Lidar data and the second Lidar data; and
determining whether the vehicle interior includes at least one of a stain, dirt, or trash;
wherein determining whether the vehicle interior includes at least one of a stain, dirt, or trash includes classifying the identified differences as two-dimensional or three-dimensional.

2. The method of claim 1 , further comprising quantizing the clean image and the second image to reduce noise.

3. The method of claim 1 , further comprising identifying at least one contour in the identified differences between the clean image and the second image.

4. The method of claim 3 , further comprising associating a bounding box with the identified contour.

5. The method of claim 1 , wherein determining whether the vehicle interior includes at least one of a stain, dirt, or trash is based on at least one of differences between the clean image and the second image, and differences between the clean Lidar data and the second Lidar data.

6. The method of claim 1 , wherein identifying differences between the clean image and the second image includes subtracting the second image from the clean image.

7. The method of claim 1 , wherein identifying differences between the clean Lidar data and the second Lidar data includes identifying differences in the point clouds of the clean Lidar data and the second Lidar data.

8. The method of claim 1 , further comprising classifying two-dimensional differences as stains and classifying three-dimensional differences as at least one of dirt or trash.

9. The method of claim 1 , further comprising determining a shape associated with the three-dimensional differences.

10. The method of claim 1 , further comprising determining dimensions associated with the three-dimensional differences.

11. The method of claim 1 , further comprising determining whether the vehicle interior needs to be cleaned based on determining whether the vehicle interior includes at least one of a stain, dirt, or trash.

12. The method of claim 1 , wherein the vehicle is an autonomous vehicle.

13. A method comprising:
receiving a clean image and clean Lidar data associated with a clean vehicle interior;
receiving a second image and second Lidar data associated with the vehicle interior after a passenger has occupied the vehicle;
identifying, by a vehicle stain and trash detection system, differences between the clean image and the second image;
identifying, by the vehicle stain and trash detection system, differences between the clean Lidar data and the second Lidar data; and
determining whether the vehicle interior includes at least one of a stain, dirt, or trash based on:
the identified differences between the clean image and the second image; and
the identified differences between the clean Lidar data and the second Lidar data;

wherein determining whether the vehicle interior includes at least one of a stain, dirt, or trash further includes classifying the identified differences as two-dimensional or three-dimensional.

14. The method of claim 13 , wherein identifying differences between the clean Lidar data and the second Lidar data includes identifying differences in the point clouds of the clean Lidar data and the second Lidar data.

15. The method of claim 13 , further comprising classifying two-dimensional differences as stains and classifying three-dimensional differences as at least one of dirt or trash.

16. An apparatus comprising:
a communication manager configured to receive a clean image and clean Lidar data associated with a clean vehicle interior, and configured to receive a second image and second Lidar data associated with the vehicle interior after a passenger has occupied the vehicle;
an image processing module configured to identify differences between the clean image and the second image;
a Lidar processing module configured to identify differences between the clean Lidar data and the second Lidar data; and
a classification module configured to classify an area within the vehicle as one of a stain, dirt, trash, or another item based on at least one of differences between the clean image and the second image, and differences between the clean Lidar data and the second Lidar data;
wherein the classification module is further configured to identify the identified the stain, dirt, trash, or other item as two-dimensional or three-dimensional.

17. The apparatus of claim 16 , wherein identifying differences between the clean Lidar data and the second Lidar data includes identifying differences in the point clouds of the clean Lidar data and the second Lidar data.

Assignees

• Ford Global Tech Llc

Inventors

• Tokatyan Shant

Classifications

• G01S17/89

  for mapping or imaging · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/0008 Primary

  checking presence/absence · CPC title

• G06T2207/30268

  Vehicle interior · CPC title

• G06T2207/10052

  Images from lightfield camera · CPC title