Interpretation of ambiguous vehicle instructions

Patent metadata

Field	Value
Publication number	US-9747898-B2
Application number	US-201414465049-A
Country	US
Kind code	B2
Filing date	Aug 21, 2014
Priority date	Mar 15, 2013
Publication date	Aug 29, 2017
Grant date	Aug 29, 2017

Abstract

Various exemplary embodiments relate to a command interpreter for use in a vehicle control system in a vehicle for interpreting user commands, a vehicle interaction system including such a command interpreter, a vehicle including such a vehicle interaction system, and related method and non-transitory machine-readable storage medium, including: a memory and a processor, the processor being configured to: receive, from at least one human via a first input device, a first input having a first type; receive a second input having a second type via a second input device, wherein the second type comprises at least one of sensed information describing a surrounding environment of the vehicle and input received from at least one human; interpret both the first input and the second input to generate a system instruction; and transmit the system instruction to a different system of the vehicle.

First claim

What is claimed is:

1. A command interpreter of a vehicle control system comprising:
a memory and a processor, the memory includes command interpretation instructions, the processor executes the command interpretation instructions, the command interpretation instructions include:
receiving from at least one of a human vehicle driver and a human vehicle passenger via a first input device, a first input having a first input type;
receiving a second input having a second input type different from the first input type via a second input device, wherein the second input type comprises at least one of sensed information describing a surrounding environment of the vehicle and input received from at least one of the human vehicle driver and the human vehicle passenger;
interpreting both the first input and the second input to generate a system instruction, wherein interpreting both the first input and the second input to generate the system instruction includes correlating the first input and the second input to determine at least one instruction command, wherein upon determining the at least one instruction command, the processor retrieves metadata associated with the at least one instruction command and determines one or more parameter sets for the at least one instruction command based on the metadata,
wherein the processor determines if a sufficient confidence level exists in the one or more parameter sets based on a vehicle state indicative of a current vibration level of a cabin of the vehicle received from a vehicle state sensing subsystem and the processor generates the system instruction based on the at least one instruction command and the one or more parameter sets in which the sufficient confidence level exists, wherein the system instruction includes a change lane command to move the vehicle to a lane associated with a vibration level lower than the current vibration level; and
controlling an operation of an autonomous control system to control motion of the vehicle based on execution of the system instruction.

2. The command interpreter of claim 1 , wherein interpreting both the first input and the second input to generate the system instruction includes:
upon determining the at least one instruction command the processor analyzes the at least one instruction command to determine an instruction command that includes a sufficient confidence level that represents an intention of the at least one of the human vehicle driver and the human passenger, wherein upon determining that the at least one instruction command does not include the sufficient confidence level, a request for command clarification is output.

3. The command interpreter of claim 1 , wherein interpreting both the first input and the second input to generate the system instruction includes upon determining that the sufficient confidence level does not exist in the one or more parameter sets, a request for parameter clarification is output.

4. The command interpreter of claim 1 , wherein, interpreting both the first input and the second input to generate the system instruction includes, utilizing the second input to select from a plurality of potential instructions identified as being associated with the first input.

5. The command interpreter of claim 1 , wherein the processor utilizes machine learning to generate a plurality of rules.

6. The command interpreter of claim 1 , wherein:
the second input device is a camera; the processor receives the second input from a gesture recognition system; and the second input describes a gesture performed by the at least one of the human vehicle driver and the human vehicle passenger,
wherein the processor determines the at least one instruction command based on a correlation of the first input and the gesture performed by the at least one of the human vehicle driver and the human vehicle passenger and retrieving metadata associated with the at least one instruction command to determine the one or more parameter sets for the instruction command based on the metadata.

7. The command interpreter of claim 1 , wherein:
the processor receives the second input from an environment sensing system that identifies objects present in the environment outside of the vehicle; and the second input describes at least one object present in the environment outside the vehicle.

8. The command interpreter of claim 1 , wherein the system instruction is associated with at least one parameter, wherein the at least one parameter is based on a correlation of the first input, the second input, and the one or more parameter sets, wherein the at least one parameter defines an action associated with the system instruction to be performed by the autonomous control system.

9. The command interpreter of claim 1 , wherein the system instruction indicates a tactical maneuver for the autonomous control system to perform.

10. The command interpreter of claim 1 , further including controlling an operation of a heads-up-display positioned for viewing by at least one of the human vehicle driver and the human vehicle passenger and the system instruction is associated with a graphic to be displayed via the heads-up-display.

11. A vehicle interaction system comprising:
a first input system that receives a first input having a first input type from at least one of a human vehicle driver and a human passenger;
a second input system that receives a second input having a second input type that is different from the first input type, wherein the second input type comprises sensed information describing a surrounding environment of a vehicle and input received from at least one of the human vehicle driver and the human passenger;
a storage device that stores the first input, the second input, and command interpretation instructions;
an output system that accepts instructions from other systems;
a command interpreter that executes the command interpretation instructions that include:
obtaining the first input and the second input stored in the storage device,
interpreting both the first input and the second input to generate a system instruction, wherein interpreting both the first input and the second input to generate the system instruction includes correlating the first input and the second input to determine at least one instruction command, wherein upon determining the at least one instruction command the command interpreter retrieves metadata associated with the at least one instruction command and determines one or more parameter sets for the at least one instruction command based on the metadata,
wherein the command interpreter determines if a sufficient confidence level exists in the one or more parameter sets based on a vehicle state indicative of a current vibration level of a cabin of the vehicle received from a vehicle state sensing subsystem and the command interpreter generates the system instruction based on the at least one instruction command and the one or more parameter sets in which the sufficient confidence level exists, wherein the system instruction includes a change lane command to move the vehicle to a lane associated with a vibration level lower than the current vibration level, and
controlling an operation of an autonomous control system to control motion of the vehicle based on execution of the system instruction.

12. The command interpreter of claim 11 , wherein, in interpreting both the first input and the second input to generate the system instruction, the command interpreter:
analyzes the at least one instruction command to determine an instruction command that includes a sufficient confidence level that represents an intention of the at least one of the human vehicle driver

Assignees

• Honda Motor Co Ltd

Inventors

• Ng-Thow-Hing Victor
• Bark Karlin
• Tran Cuong

Classifications

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G10L15/20 Primary

  Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence) · CPC title

• G06N99/005

  Physics · mapped topic