Real time driving difficulty categorization

Patent metadata

Field	Value
Publication number	US-9686451-B2
Application number	US-201514602214-A
Country	US
Kind code	B2
Filing date	Jan 21, 2015
Priority date	Jan 21, 2015
Publication date	Jun 20, 2017
Grant date	Jun 20, 2017

Abstract

The disclosure includes a system and method for determine a real time driving difficulty category. The method may include determining image feature vector data based on one or more features depicted in a real time image of a road scene. The image feature vector data may describe an image feature vector for an edited version of the real time image. The method may include determining offline road map data for the road scene, which includes a static label for a road included in the road scene and offline road information describing a regulatory speed limit for the road. The method may include selecting, based on the static label, a classifier for analyzing the image feature vector. The method may include executing the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene.

First claim

What is claimed is:

1. A method comprising:
receiving real time image data describing a real time image depicting a road scene;
determining region of interest data describing a version of the real time image configured to remove redundant features from the real time image;
determining dense grid sampling data describing one or more features included in the version of the real time image described by the region of interest data;
determining image feature vector data based on the one or more features included in the dense grid sampling data, the image feature vector data describing an image feature vector for the version of the real time image described by the region of interest data;
receiving client location data describing a geographic location of a client device associated with the road scene included in the real time image;
determining offline road map data for the road scene based on the geographic location of the client device, the offline road map data describing (1) a static label for a road included in the road scene and (2) offline road information describing a regulatory speed limit for the road included in the road scene;
selecting, based on the static label, a classifier for analyzing the image feature vector; and
executing, by a processor, the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene depicted in the real time image, wherein the real time driving difficulty category is determined based on the image feature vector, offline road information and runtime road description data describing the real time driving conditions of the road scene depicted in the real time image.

2. A method comprising:
determining dense grid sampling data describing one or more features included in a version of a real time image of a road scene which is configured to remove one or more redundant features from the real time image;
determining image feature vector data based on one or more features described by the dense grid sampling data, the image feature vector data describing an image feature vector for the version of the real time image which is configured to remove the one or more redundant features from the real time image;
determining offline road map data for the road scene based on a geographic location of a client device associated with the road scene included in the real time image, the offline road map data describing (1) a static label for a road included in the road scene and (2) offline road information describing a regulatory speed limit for the road included in the road scene;
selecting, based on the static label, a classifier for analyzing the image feature vector; and
executing, by a processor, the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene depicted in the real time image, wherein the real time driving difficulty category is determined based on the image feature vector and the offline road information.

3. The method of claim 2 , wherein the classifier is executed in real time relative to when the real time image depicting the road scene is captured.

4. The method of claim 2 , wherein the real time driving difficulty category is determined based in part on runtime road description data describing the real time driving conditions of the road scene depicted in the real time image.

5. The method of claim 2 , wherein the client device is a mobile device.

6. The method of claim 2 , wherein the real time image is captured by an onboard image system.

7. The method of claim 6 , wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be frontward facing.

8. The method of claim 6 , wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be rearward facing.

9. The method of claim 2 , wherein the offline road map data is determined during an offline process.

10. The method of claim 2 , wherein the real time image is captured during a runtime process.

11. A non-transitory computer-readable medium having computer instructions stored thereon that are executable by a processing device to perform or control performance of operations comprising:
determining dense grid sampling data describing one or more features included in a version of a real time image of a road scene which is configured to remove one or more redundant features from the real time image;
determining image feature vector data based on one or more features described by the dense grid sampling data, the image feature vector data describing an image feature vector for flail the version of the real time image which is configured to remove the one or more redundant features from the real time image;
determining offline road map data for the road scene based on a geographic location of a client device associated with the road scene included in the real time image, the offline road map data describing (1) a static label for a road included in the road scene and (2) offline road information describing a regulatory speed limit for the road included in the road scene;
selecting, based on the static label, a classifier analyzing the image feature vector; and
executing, by a processor, the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene depicted in the real time image, wherein the real time driving difficulty category is determined based on the image feature vector and the offline road information.

12. The non-transitory computer-readable medium of claim 11 , wherein the classifier is executed in real time relative to when the real time image depicting the road scene is captured.

13. The non-transitory computer-readable medium of claim 11 , wherein the client device is a robot.

14. The non-transitory computer-readable medium of claim 11 , wherein the client device is an IoT device.

15. The non-transitory computer-readable medium of claim 11 , wherein the real time image is captured by an onboard image system.

16. The non-transitory computer-readable medium of claim 15 , wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be frontward facing.

17. The non-transitory computer-readable medium of claim 15 , wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be rearward facing.

18. The non-transitory computer-readable medium of claim 15 or 17 , wherein: the offline road map data is determined during an offline process; and the real time image is captured during a runtime process.

Assignees

• Toyota Motor Co Ltd

Inventors

• Pillai Preeti Jayagopi
• Yalla Veeraganesh
• Oguchi Kentaro
• Nomoto Hirokazu

Classifications

• G06V20/38 Primary

  Outdoor scenes · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06K9/00791

  Physics · mapped topic

• G06K9/46

  Physics · mapped topic

• H04N5/225 Primary

  Electricity · mapped topic