Determining comfort settings in vehicles using computer vision

Patent metadata

Field	Value
Publication number	US-10850693-B1
Application number	US-201816118787-A
Country	US
Kind code	B1
Filing date	Aug 31, 2018
Priority date	Apr 5, 2018
Publication date	Dec 1, 2020
Grant date	Dec 1, 2020

Abstract

An apparatus includes a capture device and a processor. The capture device may be configured to generate a plurality of video frames corresponding to users of a vehicle. The processor may be configured to perform operations to detect objects in the video frames, detect users of the vehicle based on the objects detected in the video frames, determine a comfort profile for the users and select a reaction to adjust vehicle components according to the comfort profile of the detected users. The comfort profile may be determined in response to characteristics of the users. The characteristics of the users may be determined by performing the operations on each of the users.

First claim

The invention claimed is:

1. An apparatus comprising:
a capture device configured to generate a plurality of video frames corresponding to users of a vehicle; and
a processor configured to (i) perform computer vision operations to detect objects in said video frames by applying a feature detection window to each of a plurality of layers in each of the video frames, (ii) detect users of said vehicle based on said objects detected in said video frames, (iii) determine a comfort profile for said users and (iv) select a reaction to adjust vehicle components according to said comfort profile of said detected users, wherein (a) said comfort profile is determined in response to characteristics of said users, (b) said characteristics of said users are determined by performing said operations on each of said users and (c) said computer vision operations are performed by applying a convolution operation using matrix multiplication of said plurality of layers defined by said feature detection window.

2. The apparatus according to claim 1 , wherein said characteristics of said users comprise a shape and size of body parts of said users detected by said operations.

3. The apparatus according to claim 1 , wherein said characteristics of said users comprise facial features of said users detected by said operations.

4. The apparatus according to claim 3 , wherein (i) said processor is further configured to compare said facial features detected with stored face recognition profiles and (ii) said comfort profile comprises pre-defined settings for said vehicle components specific to one of said users corresponding to a matching one of said face recognition profiles.

5. The apparatus according to claim 1 , wherein (i) said video frames comprise images of said of at least one of said users before entering said vehicle and (ii) said reaction is performed before said one of said users sits in said vehicle.

6. The apparatus according to claim 1 , wherein (i) said video frames comprise images of at least one of said users sitting in said vehicle and (ii) said reaction is performed while at least one of said users is in said vehicle.

7. The apparatus according to claim 1 , wherein (i) said reaction is performed for each of said vehicle components corresponding to one of said users and (ii) and said reaction selected for each of said vehicle components is based on said comfort profile for a corresponding one of said users.

8. The apparatus according to claim 1 , wherein said reaction is further configured to adjust said vehicle components of said vehicle corresponding to said comfort profile for a specific seat of said vehicle.

9. The apparatus according to claim 1 , wherein said comfort profile comprises at least one of: a location of a seat, an angle of recline of said seat, an arm rest position for said seat, a temperature setting, fan settings, light settings, infotainment center settings, an angle of rotation of said seat, or a steering wheel position.

10. The apparatus according to claim 1 , wherein said comfort profile is further determined based on supplemental information received from a smart phone.

11. The apparatus according to claim 1 , wherein said computer vision operations are implemented by a convolutional neural network.

12. The apparatus according to claim 11 , wherein said convolutional neural network is trained using fleet learning.

13. The apparatus according to claim 12 , wherein (i) said fleet learning comprises capturing reference images using capture devices implemented in a plurality of vehicles, (ii) said reference images comprise occupied interiors of said plurality of vehicles, (iii) said reference images are used as training data for said convolutional neural network and (iv) said training data comprises said reference images from many different vehicles.

14. The apparatus according to claim 13 , wherein (i) said training data further comprises body type information of said users in said reference images and associated comfort profiles for said body type information and (ii) said associated comfort profiles are used to predict said comfort profile for said users with said characteristics that correspond to similar body types from said reference images.

15. The apparatus according to claim 1 , wherein said processor has a plurality of co-processors.

16. The apparatus according to claim 1 , wherein (i) said apparatus comprises a second capture device configured to implement a stereo camera pair with said capture device and (ii) said operations comprise performing stereo vision to determine depth information based on said video frames captured by said stereo camera pair.

17. The apparatus according to claim 1 , wherein said reaction selected by said processor is implemented autonomously by said vehicle.

18. The apparatus according to claim 1 , wherein (i) said vehicle is an autonomously driven vehicle and (ii) said comfort profile comprises rotational arrangement information for seats of said vehicle.

Assignees

• Ambarella Int Lp

Inventors

• Pertsel Shimon
• Martin Patrick

Classifications

• G06V20/597

  Recognising the driver's state or behaviour, e.g. attention or drowsiness · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• B60N2210/24

  Cameras · CPC title

• B60N2220/30

  using a neural network · CPC title