Stain and trash detection systems and methods

Patent metadata

Field	Value
Publication number	US-10509974-B2
Application number	US-201715494158-A
Country	US
Kind code	B2
Filing date	Apr 21, 2017
Priority date	Apr 21, 2017
Publication date	Dec 17, 2019
Grant date	Dec 17, 2019

Abstract

Example stain and trash detection systems and methods are described. In one implementation, a method receives a clean image of a vehicle interior and a second image of the vehicle interior after a passenger has occupied the vehicle. The method quantizes the clean image and the second image to reduce noise and identifies differences between the clean image and the second image. Additionally, the method identifies at least one contour in the identified differences and classifies an area defined by the contour as one of a stain, dirt, trash, or another item.

First claim

The invention claimed is:

1. A method comprising:
receiving a clean image of a vehicle interior of a vehicle;
receiving a second image of the vehicle interior after a passenger has occupied the vehicle;
quantizing the clean image and the second image to reduce noise by reducing pixels of the clean image and the second image to three levels including one black level, one gray level, and one white level according to intensity of the pixels to obtain a quantized clean image and a quantized second image;
identifying differences between the quantized clean image and the quantized second image;
identifying a contour in the identified differences; and
classifying an area defined by the contour as one of a stain, dirt, trash, or another item;
wherein the vehicle is an autonomous vehicle;
wherein the intensities of the pixels of the clean image and the second image have a first range of values; and
wherein quantizing the clean image and the second image to reduce noise comprises:
replacing pixels in the clean image and the second image having intensities in a second range of values with the one white level;
replacing pixels in the clean image and the second image having intensities in a third range of values with the one gray level; and
replacing pixels in the clean image and the second image having intensities in a fourth range of values with the one black level; and

wherein the second range of values is higher than the third range of values and the third range of values is higher than the fourth range of values.

2. The method of claim 1 , further comprising associating a bounding box with the contour.

3. The method of claim 1 wherein identifying differences between the quantized clean image and the quantized second image includes subtracting the quantized second image from the quantized clean image.

4. The method of claim 1 , wherein classifying the area defined by the contour includes determining a shape of the contour.

5. The method of claim 1 , wherein classifying the area defined by the contour includes determining a smoothness of edges of the contour.

6. The method of claim 1 , further comprising determining whether the vehicle interior needs to be cleaned based on the classification of the area defined by the contour.

7. The method of claim 6 , further comprising taking the vehicle out of service if the vehicle interior needs to be cleaned.

8. The method of claim 6 , further comprising, responsive to determining that the vehicle interior needs to be cleaned, arranging cleaning of the vehicle interior.

9. The method of claim 6 , further comprising awaiting instructions to pick up a new passenger if the vehicle interior does not need to be cleaned.

10. An apparatus comprising:
a communication manager configured to receive a clean image of a vehicle interior of a vehicle and configured to receive a second image of the vehicle interior after a passenger has occupied the vehicle;
an image quantization module configured to quantize the clean image and the second image by reducing pixels of the clean image and the second image to three levels including one black level, one gray level, and one white level according to intensity of the pixels to obtained a quantized clean image and a quantized second image;
an image processing module configured to identify differences between the quantized clean image and the quantized second image;
a contour identification module configured to identify a contour in the identified differences; and
a classification module configured to classify an area defined by the contour as one of a stain, dirt, trash, or another item
wherein the image quantization module is configured to quantize the clean image and the second image by:
replacing pixels in the clean image and the second image having intensities in a first range of values with the one white level;
replacing pixels in the clean image and the second image having intensities in a second range of values with the one gray level; and
replacing pixels in the clean image and the second image having intensities in a third range of values with the one black level; and
wherein the first range of values is higher than the second range of values and the second range of values is higher than the third range of values.

11. The apparatus of claim 10 , further comprising a vehicle maintenance manager configured to determine whether the vehicle interior needs to be cleaned based on the classification of the area defined by the contour.

12. The apparatus of claim 10 , wherein the apparatus is coupled to an automated driving system in the vehicle.

Assignees

• Ford Global Tech Llc

Inventors

• Tokatyan Shant

Classifications

• B60S1/64

  for cleaning vehicle interiors, e.g. built-in vacuum cleaners · CPC title

• G06T2207/30268

  Vehicle interior · CPC title

• G06T2207/30248

  Vehicle exterior or interior · CPC title

• G06T7/001

  using an image reference approach · CPC title

• G06T7/0008 Primary

  checking presence/absence · CPC title