Vision-based airbag enablement

Patent metadata

Field	Value
Publication number	US-11807181-B2
Application number	US-202017081428-A
Country	US
Kind code	B2
Filing date	Oct 27, 2020
Priority date	Oct 27, 2020
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

Vision-based airbag enablement may include capturing two-dimensional images of a passenger, segmenting the image, classifying the image, and determining seated height of the passenger from the image. Enabling or disabling deployment of the airbag may be controlled based at least in part upon the determined seated height.

First claim

What is claimed is:

1. An apparatus, comprising:
a camera providing a two-dimensional image of a seating area of a vehicle;
a controller using a height attribute model to determine a seated height of a seated passenger in the seating area of the vehicle appearing in the two-dimensional image, the height attribute model having been trained on seated height extraction from three-dimensional skeletons training produced by projecting two-dimensional skeletons upon three-dimensional training images, wherein the two-dimensional skeletons are extracted from segmented two-dimensional training images; and
a controller enabling deployment of an airbag based upon the determined seated height of the seated passenger.

2. The apparatus of claim 1 , further comprising a seat weight sensor providing a sensed weight of the seated passenger, wherein the height attribute module further comprises fusion of the sensed weight of the seated passenger.

3. The apparatus of claim 1 , wherein the controller further uses a pose model to determine a pose of the seated passenger in the seating area of the vehicle appearing in the two-dimensional image, the pose model having been trained on mesh poses reconstructed from the segmented two-dimensional training images.

4. The apparatus of claim 1 , wherein the controller further uses a segmentation model performing background segmentation of the two-dimensional images to provide segmented two-dimensional images to the height attribute model, the segmentation model having been trained on the segmented two-dimensional training images, wherein the segmented two-dimensional training images are produced based upon pairs of simultaneously captured two-dimensional and three-dimensional training images.

5. The apparatus of claim 3 , wherein the controller further uses a segmentation model performing background segmentation of the two-dimensional images to provide segmented two-dimensional images to the height attribute model and the pose model, the segmentation model having been trained on the segmented two-dimensional training images, wherein the segmented two-dimensional training images are produced based upon pairs of simultaneously captured two-dimensional and three-dimensional training images.

6. The apparatus of claim 2 , wherein the controller enabling deployment of an airbag based upon the determined seated height of the seated passenger further enables deployment of the airbag based upon the seated passenger weight.

7. An apparatus, comprising:
a camera providing a two-dimensional image of a seating area of a vehicle;
a seat weight sensor providing a sensed weight on a seat in the seating area;
a controller comprising:
a segmentation model performing background segmentation of the two-dimensional image to provide a segmented two-dimensional image of a seated passenger in the seating area of the vehicle appearing in the two-dimensional image;
a pose model determining a pose of the seated passenger in the seating area of the vehicle based upon the segmented two-dimensional image;
a height and weight attribute model determining a seated height and a weight of the seated passenger in the seating area of the vehicle based upon the segmented two-dimensional image and the sensed weight of the seated passenger; and

a controller enabling deployment of an airbag based upon the determined seated height and weight of the seated passenger;
wherein each of the segmentation model, the pose model and the height and weight attribute model comprises an offline trained machine learning model.

8. The apparatus of claim 7 , wherein the machine learning models comprise neural networks.

9. An apparatus, comprising:
a camera providing a two-dimensional image of a seating area of a vehicle;
a seat weight sensor providing a sensed weight on a seat in the seating area;
a controller comprising:
a segmentation model performing background segmentation of the two-dimensional image to provide a segmented two-dimensional image of a seated passenger in the seating area of the vehicle appearing in the two-dimensional image;
a pose model determining a pose of the seated passenger in the seating area of the vehicle based upon the segmented two-dimensional image;
a height and weight attribute model determining a seated height and a weight of the seated passenger in the seating area of the vehicle based upon the segmented two-dimensional image and the sensed weight of the seated passenger; and

a controller enabling deployment of an airbag based upon the determined seated height and weight of the seated passenger;
wherein the segmentation model comprises a neural network trained on a training database of segmented two-dimensional training images, wherein the segmented two-dimensional training images are produced based upon pairs of simultaneously captured two-dimensional and three-dimensional training images.

10. An apparatus, comprising:
a camera providing a two-dimensional image of a seating area of a vehicle;
a seat weight sensor providing a sensed weight on a seat in the seating area;
a controller comprising:
a segmentation model performing background segmentation of the two-dimensional image to provide a segmented two-dimensional image of a seated passenger in the seating area of the vehicle appearing in the two-dimensional image;
a pose model determining a pose of the seated passenger in the seating area of the vehicle based upon the segmented two-dimensional image;
a height and weight attribute model determining a seated height and a weight of the seated passenger in the seating area of the vehicle based upon the segmented two-dimensional image and the sensed weight of the seated passenger; and

a controller enabling deployment of an airbag based upon the determined seated height and weight of the seated passenger;
wherein the pose model comprises a neural network trained on a training database of segmented two-dimensional training images, wherein the segmented two-dimensional training images are produced based upon pairs of simultaneously captured two-dimensional and three-dimensional training images.

11. An apparatus, comprising:
a camera providing a two-dimensional image of a seating area of a vehicle;
a seat weight sensor providing a sensed weight on a seat in the seating area;
a controller comprising:
a segmentation model performing background segmentation of the two-dimensional image to provide a segmented two-dimensional image of a seated passenger in the seating area of the vehicle appearing in the two-dimensional image;
a pose model determining a pose of the seated passenger in the seating area of the vehicle based upon the segmented two-dimensional image;
a height and weight attribute model determining a seated height and a weight of the seated passenger in the seating area of the vehicle based upon the segmented two-dimensional image and the sensed weight of the seated passenger; and

a controller enabling deployment of an airbag based upon the determined seated height and weight of the seated passenger;
wherein the height and weight attribute model comprises a neural network trained on three-dimensional skeletons corresponding to a training database of segmented two-dimensional training images, wherein the segmented two-dimensional training images are produced based upon pairs of simultaneously captured two-dimensional and three-dimensional training images.

12. The apparatus of claim 11 , wherein the segmented two-dimensional training images are subjected to a two-dimensional skeleton extraction to produce two-dimensional skeletons projected upon three-dimensional training images corresponding to the segmented two-dimensional training imag

Assignees

• Gm Global Tech Operations Llc

Inventors

• Druker Itai
• Kamhi Gila
• Baltaxe Michael
• Nistel Kobi
• Mergui Ruben
• Sala Dorel M
• Fallon Lisa A

Classifications

• G01S17/86

  Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• B60R21/01538 Primary

  for image processing, e.g. cameras or sensor arrays · CPC title

• B60R21/01516

  using force or pressure sensing means · CPC title

• B60R21/16

  Inflatable occupant restraints or confinements designed to inflate upon impact or impending impact, e.g. air bags ({protective garments with automatically inflatable shock-absorbing means A41D13/018; } connection of valves to inflatable elastic bodies B60C29/00) · CPC title