Vehicle control device, vehicle control method, and storage medium

Patent metadata

Field	Value
Publication number	US-11377124-B2
Application number	US-202016807207-A
Country	US
Kind code	B2
Filing date	Mar 3, 2020
Priority date	Mar 7, 2019
Publication date	Jul 5, 2022
Grant date	Jul 5, 2022

Abstract

A vehicle control device includes a recognizer configured to recognize at least a surroundings situation and an in-vehicle situation of a vehicle, and a driving controller configured to perform at least one of speed control and steering control of the vehicle on the basis of a recognition result of the recognizer, the recognizer is configured to recognize an object to be loaded including at least any one of an occupant candidate present outside the vehicle and scheduled to board the vehicle and a baggage candidate present outside the vehicle and scheduled to be loaded into the vehicle, and to recognize that the object to be loaded has been loaded into the vehicle, and the driving controller is configured to permit start of travel of the vehicle in a case where the recognizer has recognized that all of the objects to be loaded have been loaded into the vehicle.

First claim

What is claimed is:

1. A vehicle control device comprising a processor configured to execute a program to:
recognize, as a recognition result, at least a surroundings situation and an in-vehicle situation of a vehicle;
recognize at least one object to be loaded including at least any one of an occupant candidate present outside the vehicle and scheduled to board the vehicle and a baggage candidate present outside the vehicle and scheduled to be loaded into the vehicle on the basis of an image of an outside-vehicle camera configured to image the vicinity of the vehicle;
recognize that the at least one object to be loaded has been loaded into the vehicle on the basis of a comparison result between the at least one object to be loaded recognized on the basis of the image of the outside-vehicle camera and a detection result of an in-vehicle detector configured to detect an in-vehicle status of the vehicle;
initiate travel of the vehicle in a case where it has been recognized that the at least one object to be loaded has been loaded into the vehicle;
derive a first number of the at least one object to be loaded based on the image of the outside-vehicle camera;
determine that the first number matches a second number of the at least one object to be loaded which has been detected by the in-vehicle detector and loaded into the vehicle;
recognize that the at least one object to be loaded has been loaded into the vehicle in a case where it has been recognized that the first number matches the second number; and
perform one of speed control and steering control of the vehicle based on recognizing that the at least one object to be loaded has been loaded into the vehicle in the case where it has been recognized that the first number matches the second number.

2. The vehicle control device according to claim 1 , wherein the processor is further configured to execute the program to:
control an output to notify occupants of the vehicle that it has been recognized that all of objects to be loaded are not loaded into the vehicle in a case where it has been recognized that all of objects to be loaded are not loaded into the vehicle.

3. The vehicle control device according to claim 2 , further comprising:
a receiver configured to receive travel start permission from a user,
wherein the processor is configured to execute the program to permit the start of travel of the vehicle in a case where the receiver receives a travel start permission of the user after the output is controlled.

4. The vehicle control device according to claim 1 ,
wherein the processor is configured to execute the program to recognize that the at least one object to be loaded has been loaded into the vehicle on the basis of the detection result of the in-vehicle detector, and
the in-vehicle detector includes at least one of a sensor configured to detect whether or not a seat belt of the vehicle has been used, a camera configured to image the inside of the vehicle, or a pressure sensor mounted on a seat of the vehicle.

5. The vehicle control device according to claim 1 , wherein the processor is configured to execute the program to recognize a non-boarding person performing a predetermined action among persons recognized around the vehicle and to exclude the non-boarding person from the objects to be loaded.

6. The vehicle control device according to claim 1 , further comprising:
the outside-vehicle camera,
wherein the processor is configured to execute the program to recognize that all of the objects to be loaded have been loaded into the vehicle on the basis of the image of the outside-vehicle camera.

7. The vehicle control device according to claim 6 , wherein the processor is configured to execute the program to recognize that all of the objects to be loaded have been loaded into the vehicle on the basis of an image of the vicinity of a door of the vehicle or the vicinity of a baggage compartment of the vehicle.

8. The vehicle control device according to claim 6 , wherein the processor is configured to execute the program to recognize that all of the objects to be loaded have not been loaded into the vehicle in a case where a predetermined item has been recognized near a door of the vehicle after the occupant candidates have boarded.

9. The vehicle control device according to claim 1 , wherein the processor is configured to execute the program to recognize a constantly present object among objects recognized around the vehicle and to exclude the constantly present object from the objects to be loaded.

10. A vehicle control method using an in-vehicle computer, comprising:
recognizing, as a recognition result, at least a surroundings situation and an in-vehicle situation of a vehicle;
recognizing at least one object to be loaded including at least any one of an occupant candidate present outside the vehicle and scheduled to board the vehicle and a baggage candidate present outside the vehicle and scheduled to be loaded into the vehicle on the basis of an image of an outside-vehicle camera configured to image the vicinity of the vehicle;
recognizing that the at least one object to be loaded has been loaded into the vehicle on the basis of a comparison result between the at least object to be loaded recognized on the basis of the image of the outside-vehicle camera and a detection result of an in-vehicle detector configured to detect an in-vehicle status of the vehicle;
initiating travel of the vehicle in a case where it has been recognized that the at least one object to be loaded has been loaded into the vehicle;
deriving a first number of the at least one object to be loaded based on the image of the outside-vehicle camera;
determining that the first number matches a second number of the at least one object to be loaded which has been detected by the in-vehicle detector and loaded into the vehicle;
recognizing that the at least one object to be loaded has been loaded into the vehicle in a case where it has been recognized that the first number matches the second number; and
performing one of speed control and steering control of the vehicle based on recognizing that the at least one object to be loaded has been loaded into the vehicle in the case where it has been recognized that the first number matches the second number.

11. A non-transitory computer-readable storage medium storing a program, the program causing an in-vehicle computer to:
recognize, as a recognition result, at least a surroundings situation and an in-vehicle situation of a vehicle;
recognize at least one object to be loaded including at least any one of an occupant candidate present outside the vehicle and scheduled to board the vehicle and a baggage candidate present outside the vehicle and scheduled to be loaded into the vehicle on the basis of an image of an outside-vehicle camera configured to image the vicinity of the vehicle;
recognize that the at least one object to be loaded has been loaded into the vehicle on the basis of a comparison result between the at least one object to be loaded recognized on the basis of the image of the outside-vehicle camera and a detection result of an in-vehicle detector configured to detect an in-vehicle status of the vehicle;
initiate travel of the vehicle in a case where it has been recognized that the at least one object to be loaded has been loaded into the vehicle;
derive a first number of the at least one object to be loaded based on the image of the outside-vehicle camera;
determine that the first number matches a second number of the at least one object to be loaded which has been detected by the in-vehicle detector and loaded into the vehicle;
recognize that the at least one object to be loaded has

Assignees

• Honda Motor Co Ltd

Inventors

• Mimura Yoshitaka
• Yamane Katsuyasu
• Yamanaka Hiroshi
• Sugihara Chie
• Motegi Yuki
• Shibauchi Tsubasa

Classifications

• G06V20/59

  inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• B60W10/04

  including control of propulsion units · CPC title

• B60W10/20

  including control of steering systems · CPC title