Detecting user interactions with a computing system of a vehicle

Patent metadata

Field	Value
Publication number	US-2018024695-A1
Application number	US-201615218873-A
Country	US
Kind code	A1
Filing date	Jul 25, 2016
Priority date	Jul 25, 2016
Publication date	Jan 25, 2018
Grant date	—

Abstract

A computing system of a vehicle for controlling a vehicle or systems within a vehicle. The computing system comprises a presence-sensitive panel within the vehicle, an infrared camera configured to capture images in a three-dimensional space within the vehicle, at least one processor; and at least one storage device that stores instructions. When the instructions are executed, they case the at least one processor to: receive, from the presence-sensitive panel, a first indication of input, receive, from the infrared camera, a second indication of input, and determine, based on at least one of the first indication of input or the second indication of input, an operation to be performed.

First claim

What is claimed is:

1 . A computing system of a vehicle comprising:
a presence-sensitive panel within the vehicle; an infrared camera configured to capture images in a three-dimensional space within the vehicle; at least one processor; and at least one storage device that stores instructions that, when executed, cause the at least one processor to:
receive, from the presence-sensitive panel, a first indication of input,
receive, from the infrared camera, a second indication of input, and
determine, based on at least one of the first indication of input or the second indication of input, an operation to be performed.

2 . The computing system of claim 1 ,
wherein the presence-sensitive panel is positioned between the infrared camera and the three-dimensional space and is transparent to infrared; and wherein the infrared camera is positioned under the presence-sensitive panel and is oriented to capture images in the three-dimensional space above the presence-sensitive panel.

3 . The computing system of claim 2 , wherein the presence-sensitive panel is positioned substantially horizontally on a central console of the vehicle.

4 . The computing system of claim 1 ,
wherein the presence-sensitive panel is configured to detect two-dimensional gesture inputs performed within a threshold distance of the presence-sensitive panel; and wherein the infrared camera is configured to detect three-dimensional gesture inputs performed beyond the threshold distance of an input surface of the presence-sensitive panel and in the three-dimensional space.

5 . The computing system of claim 1 , wherein determining the operation to be performed is based on both the first indication of input and the second indication of input.

6 . The computing system of claim 1 , wherein determining the operation to be performed is based on the first indication of input.

7 . The computing system of claim 6 , wherein the operation to be performed is a first operation to be performed, and wherein the instructions, when executed, further cause the at least one processor to:
responsive to determining the first operation to be performed, output, for display, information about the first operation to be performed; determine, based on the second indication of input, a second operation to be performed; and responsive to determining the second operation to be performed, output, for display, information about the second operation to be performed.

8 . The computing system of claim 6 , further comprising a display screen, wherein the operation to be performed is a first operation to be performed, and wherein the instructions, when executed, further cause the at least one processor to:
responsive to determining the first operation to be performed, display on the display screen information about the first operation to be performed; determine, based on the second indication of input, a second operation to be performed; and responsive to determining the second operation to be performed, display on the display screen information about the second operation to be performed.

9 . The computing system of claim 6 , further comprising a display screen, and wherein the instructions, when executed, further cause the at least one processor to:
responsive to determining the operation to be performed and prior to receiving the second indication of input, display on the display screen information requesting confirmation to perform the operation; determine, based on the second indication of input, that the operation should be performed; and perform the operation.

10 . The computing system of claim 6 ,
wherein the three-dimensional space has high ambient light conditions, and wherein determining the operation to be performed is based on the second indication of input and includes determining gestures from images captured by the infrared camera in the high ambient light conditions.

11 . The computing system of claim 6 ,
wherein the three-dimensional space has low ambient light conditions, and wherein determining the operation to be performed is based on the second indication of input and includes determining gestures from images captured by the infrared camera in the low ambient light conditions.

12 . The computing system of claim 6 ,
wherein the three-dimensional space includes structure in a field of view of the infrared camera, and wherein determining the operation to be performed is based on the second indication of input and includes discerning gestures from the structure in the field of view.

13 . A method comprising:
receiving, by a computing system of a vehicle, from a presence-sensitive panel positioned within the vehicle, a first indication of input; receiving, by the computing system, from an infrared camera configured to capture images in a three-dimensional space within the vehicle, a second indication of input; and determining, based on at least one of the first indication of input or the second indication of input, an operation to be performed.

14 . The method of claim 13 ,
wherein the presence-sensitive panel is positioned between the infrared camera and the three-dimensional space and is transparent to infrared; and wherein the infrared camera is positioned under the presence-sensitive panel and is oriented to capture images in the three-dimensional space above the presence-sensitive panel.

15 . The method of claim 14 , wherein the presence-sensitive panel is positioned substantially horizontally on a central console of the vehicle.

16 . The method of claim 13 ,
wherein the presence-sensitive panel is configured to detect two-dimensional gesture inputs performed within a threshold distance of the presence-sensitive panel; and wherein the infrared camera is configured to detect three-dimensional gesture inputs performed beyond the threshold distance of an input surface of the presence-sensitive panel and in the three-dimensional space.

17 . The method of claim 13 , wherein determining the operation to be performed is based on both the first indication of input and the second indication of input.

18 . The method of claim 13 , wherein determining the operation to be performed is based on the first indication of input.

19 . The method of claim 18 , wherein the operation to be performed is a first operation to be performed, the method further comprising:
responsive to determining the first operation to be performed, outputting, for display, information about the first operation to be performed; determining, based on the second indication of input, a second operation to be performed; and responsive to determining the second operation to be performed, outputting, for display, information about the second operation to be performed.

20 . A computer-readable storage medium comprising instructions that, when executed by a computing system of a vehicle, cause at least one processor of the computing system to:
receive, from a presence-sensitive panel positioned within the vehicle, a first indication of input; receive, from an infrared camera configured to capture images in a three-dimensional space within the vehicle, a second indication of input; and determine, based on at least one of the first indication of input or the second indication of input, an operation to be performed.

Assignees

• Google Inc

Inventors

• Iyer Rakesh Narayan
• Onorato Joseph

Classifications

• B60K35/80

  Arrangements for controlling instruments · CPC title

• B60K35/60

  Instruments characterised by their location or relative disposition in or on vehicles (arrangements of lighting devices on dashboards B60Q3/10) · CPC title

• B60K35/50

  Instruments characterised by their means of attachment to or integration in the vehicle (B60K35/231 takes precedence) · CPC title

• B60K35/28

  characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver · CPC title

• B60K35/90

  Calibration of instruments, e.g. setting initial or reference parameters; Testing of instruments, e.g. detecting malfunction · CPC title