Virtual vehicle occupant rendering

Patent metadata

Field	Value
Publication number	US-10860872-B2
Application number	US-201616087121-A
Country	US
Kind code	B2
Filing date	Mar 21, 2016
Priority date	Mar 21, 2016
Publication date	Dec 8, 2020
Grant date	Dec 8, 2020

Abstract

Image data of a vehicle occupant are collected from a plurality of cameras. A dimensional model of substantially an entire body of the vehicle occupant is generated based on the image data. A gesture performed by the vehicle occupant is recognized based on the dimensional model. A vehicle subsystem is adjusted based on the gesture.

First claim

The invention claimed is:

1. A system, comprising a computer including a processor and a memory, the memory storing instructions executable by the computer to:
collect image data of substantially an entire body of a vehicle operator and substantially an entire body of a vehicle passenger from a plurality of cameras;
generate a dimensional model of substantially the entire body of the vehicle operator and a second dimensional model of substantially the entire body of the vehicle passenger based on the image data;
identify a right hand of the vehicle operator and a left hand of the vehicle passenger based on the dimensional models;
based on the dimensional models of the vehicle operator and the vehicle passenger, recognize a gesture performed by one of the right hand of the vehicle operator or the left hand of the vehicle passenger;
based on the gesture, accept, input to a vehicle human-machine interface from the left hand of the vehicle passenger and ignore input to the vehicle human-machine interface from the right hand of the vehicle operator; and
adjust a vehicle subsystem based on the gesture.

2. The system of claim 1 , wherein the instructions further include instructions to identify at least one occupant body part.

3. The system of claim 1 , wherein the instructions further include instructions to transform the image data into a vehicle coordinate system.

4. The system of claim 1 , wherein the gesture includes at least one of a hand moving to operate the vehicle human-machine interface and to grasp a steering wheel.

5. The system of claim 1 , wherein the instructions further include instructions to detect the gesture by detecting movement of a plurality of body parts.

6. The system of claim 1 , wherein the instructions further include instructions to collect at least some of the image data of the occupant when the occupant is outside the vehicle.

7. The system of claim 6 , wherein the instructions further include instructions to adjust a vehicle seat when detecting the occupant outside the vehicle.

8. The system of claim 1 , wherein the cameras are arranged in the vehicle to capture image data of substantially the entire body of the vehicle occupant.

9. A method, comprising:
collecting image data of substantially an entire body of a vehicle operator and substantially an entire body of a vehicle passenger from a plurality of cameras;
generating a dimensional model of substantially the entire body of the vehicle operator and a second dimensional model of substantially the entire body of the vehicle passenger based on the image data;
identifying a right hand of the vehicle operator and a left hand of the vehicle passenger based on the dimensional models;
based on the dimensional models of the vehicle operator and the vehicle passenger, recognizing a gesture performed by one of the right hand of the vehicle operator or the left hand of the vehicle passenger;
based on the gesture, accepting input to a vehicle human-machine interface from the left hand of the vehicle passenger and ignoring input to the vehicle human-machine interface from the right hand of the vehicle operator; and
adjust a vehicle subsystem based on the gesture.

10. The method of claim 9 , further comprising identifying at least one occupant body part.

11. The method of claim 9 , further comprising transforming the image data into a vehicle coordinate system.

12. The method of claim 9 , wherein the gesture includes at least one of a hand moving to operate the vehicle human-machine interface and to grasp a steering wheel.

13. The method of claim 9 , further comprising detecting the gesture by detecting movement of a plurality of body parts.

14. The method of claim 9 , further comprising collecting at least some of the image data of the occupant when the occupant is outside the vehicle.

15. The method of claim 14 , further comprising adjusting a vehicle seat when detecting the occupant outside the vehicle.

16. The method of claim 9 , wherein the cameras are arranged in the vehicle to capture image data of substantially the entire body of the vehicle occupant.

Assignees

• Ford Global Tech Llc

Inventors

• Wang Nanxin
• Wan Jian
• Strumolo Gary Steven

Classifications

• G06V10/85

  Markov-related models; Markov random fields · CPC title

• G06V20/597

  Recognising the driver's state or behaviour, e.g. attention or drowsiness · CPC title

• H04N7/181 Primary

  for receiving images from a plurality of remote sources · CPC title

• G06F18/295

  Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title