Enhanced front curb viewing system

Patent metadata

Field	Value
Publication number	US-10093247-B2
Application number	US-201414280395-A
Country	US
Kind code	B2
Filing date	May 16, 2014
Priority date	May 23, 2013
Publication date	Oct 9, 2018
Grant date	Oct 9, 2018

Abstract

A system and method for determining when to display frontal curb view images to a driver of a vehicle, and what types of images to display. A variety of factors—such as vehicle speed, GPS/location data, the existence of a curb in forward-view images, and vehicle driving history—are evaluated as potential triggers for the curb view display, which is intended for situations where the driver is pulling the vehicle into a parking spot which is bounded in front by a curb or other structure. When forward curb-view display is triggered, a second evaluation is performed to determine what image or images to display which will provide the best view of the vehicle's position relative to the curb. The selected images are digitally synthesized or enhanced, and displayed on a console-mounted or in-dash display device.

First claim

What is claimed is:

1. A system for providing enhanced images of an area in front of a vehicle, said system comprising:
a first camera mounted at the left front of the vehicle;
a second camera mounted at the right front of the vehicle;
a video acquisition device in communication with the first and second cameras, said video acquisition device receiving raw images from the first and second cameras;
a processor including a memory module, said processor being configured to determine whether to activate a front curb view display based on a detected frontal parking maneuver, said processor being further configured to determine what images to include in the front curb view display, and to provide the images including enhanced images synthesized from the raw images from the first and second cameras, where the processor includes a learning module configured to:
(A) determine when to activate the front curb view display based on (1) preprogrammed learning based on parking view display preferences of a general population of drivers and (2) adaptive learning based on parking view display preferences of the driver of the vehicle; and
(B) determine what views to include in the front curb view display based on (1) the preprogrammed learning based on the parking view display preferences of the general population of drivers and (2) the adaptive learning based on the parking view display preferences of the driver of the vehicle; and

a display unit in a cockpit area of the vehicle for displaying the front curb view display from the processor for viewing by a driver of the vehicle.

2. The system of claim 1 further comprising a user input device for allowing the driver of the vehicle to signal that he or she wants to activate a front curb view display.

3. The system of claim 2 wherein the user input device is a microphone and the driver utters a spoken command to activate the front curb view display.

4. The system of claim 1 wherein the processor includes a heuristics module for detecting a frontal parking maneuver, said heuristics module including a vehicle state module for evaluating vehicle dynamic conditions, an object detection module for detecting a curb or parking space ahead of the vehicle, and a map module for determining from Global Positioning System data and digital map data that the vehicle is in a parking lot.

5. The system of claim 4 wherein the heuristics module is configured to activate the front curb view display when a vehicle speed is below a predetermined threshold and a curb or parking space is detected ahead of the vehicle.

6. The system of claim 4 wherein the vehicle state module detects vehicle speed, steering angle, yaw rate, longitudinal acceleration and lateral acceleration, and the vehicle state module determines when a parking maneuver is underway or possible.

7. The system of claim 4 wherein the object detection module uses image processing techniques to identify one or more parking spaces or structures ahead of the vehicle, and the parking spaces or structures are included in the front curb view display.

8. The system of claim 7 wherein the object detection module also uses signals from an object detection sensor to detect a curb or structure ahead of the vehicle, where the heuristics module is configured to activate the front curb view display when a vehicle speed is below a predetermined threshold and the curb or structure is detected ahead of the vehicle.

9. The system of claim 8 wherein the object detection sensor is selected from the group consisting of an ultrasonic sensor, a short-range radar sensor and a LIDAR sensor.

10. The system of claim 4 wherein the processor includes a mode selection module which is configured to select an optimal display mode in the front curb view display based on data from the heuristics module, where the optimal display mode includes one or more views in front of the vehicle which depict a parking space or structure toward which the vehicle is moving.

11. The system of claim 1 wherein the adaptive learning includes using driving history and Global Positioning System data to identify locations where the front curb view display was activated in the past.

12. The system of claim 11 wherein the adaptive learning also includes recognition of frequent destinations of the vehicle, where the frequent destinations include a home location, a work location and a school location.

13. The system of claim 1 wherein the front curb view display includes a top-down display which is synthesized from the raw images from the first and second cameras.

14. A system for providing enhanced images of an area in front of a vehicle, said system comprising:
a first camera mounted at the left front of the vehicle;
a second camera mounted at the right front of the vehicle;
a video acquisition device in communication with the first and second cameras, said video acquisition device receiving raw images from the first and second cameras;
a user input device for allowing a driver of the vehicle to signal that he or she wants to activate a front curb view display;
a heuristics module for detecting a frontal parking maneuver, said heuristics module including a vehicle state module for evaluating vehicle speed and steering angle, an image analysis module for detecting a curb or parking space ahead of the vehicle, and a map module for determining from Global Positioning System and digital map data that the vehicle is in a parking lot;
a learning module in communication with the heuristics module and the user input device, the learning module configured to:
(A) determine when to activate a front curb view display based on (1) preprogrammed learning based on parking view display preferences of a general population of drivers and (2) adaptive learning based on parking view display preferences of the driver of the vehicle; and
(B) determine what views to include in the front curb view display based on (1) the preprogrammed learning based on the parking view display preferences of the general population of drivers and (2) the adaptive learning based on the parking view display preferences of the driver of the vehicle; and

a processor including a memory module, said processor being configured to determine activate the front curb view display in response to signals from the user input device, the heuristics module, and the learning module and to generate the front curb view display including enhanced images synthesized from the raw images from the first and second cameras; and
a display unit in a cockpit area of the vehicle for displaying the front curb view display from the processor for viewing by a driver of the vehicle.

15. The system of claim 14 wherein the processor is configured to activate the front curb view display when the vehicle speed is below a predetermined threshold, a curb or parking space is detected ahead of the vehicle, and the vehicle is determined to be located in a parking lot.

16. The system of claim 14 wherein the processor includes a mode selection module which is configured to select an optimal display mode in the front curb view display based on data from the heuristics module, where the optimal display mode includes one or more views in front of the vehicle which depict a parking space or structure toward which the vehicle is moving.

17. A method for providing enhanced front curb viewing in a vehicle, said method comprising:
determining a vehicle speed by reading a vehicle speed signal from a vehicle speed sensor;
determining whether to activate a front curb view display based on whether the vehicle speed is below a speed

Assignees

• Gm Global Tech Operations Llc

Inventors

• Zhang Wende
• Wang Jinsong
• Lybecker Kent S
• Piasecki Jeffrey S
• Litkouhi Bakhtiar Brian
• Frakes Ryan M

Classifications

• B60R2300/605

  the adjustment being automatic · CPC title

• B60R2300/607

  from a bird's eye viewpoint · CPC title

• B60R2300/302

  combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data · CPC title

• B60R1/00

  Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles · CPC title

• B60R11/04 Primary

  Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle · CPC title