Situational awareness in a vehicle

Patent metadata

Field	Value
Publication number	US-2022410931-A1
Application number	US-202217844942-A
Country	US
Kind code	A1
Filing date	Jun 21, 2022
Priority date	Jun 29, 2021
Publication date	Dec 29, 2022
Grant date	—

Abstract

Enhancing situational awareness of an advanced driver assistance system in a host vehicle can be provided by acquiring, with an image sensor, an image data stream comprising a plurality of image frames. Analyzing A vision processor can analyze the image data stream to detect objects, shadows and/or lighting in the image frames. Recognizing A situation recognition engine can recognize at least one most probable traffic situation out of a set of predetermined traffic situations taking into account the detected objects, shadows and/or lighting. A processor can then control the host vehicle taking into account the at least one most probable traffic situation.

First claim

1 - 14 . (canceled)

15 . A computing device for a vehicle, including a processor and a memory configured such that the computing device is programmed to:
acquire, with an image sensor, an image data stream comprising a plurality of image frames; analyze, with a vision processor, the image data stream to detect objects, shadows and/or lighting in the image frames, wherein analyzing the image data stream comprises detecting artificial lighting in the image frames, the artificial lighting being dynamic lighting that includes at least one of moving or changing size in relation to surfaces or objects being illuminated; recognize, with a situation recognition engine, at least one most probable traffic situation out of a set of predetermined traffic situations taking into account the detected objects, shadows and/or lighting; and control the vehicle taking into account the at least one most probable traffic situation.

16 . The computing device of claim 15 , further programmed to analyze the image data stream by detecting shadows in the image frames.

17 . The computing device of claim 16 , further programmed to analyze the image data stream by detecting dynamic shadows, wherein movements and/or changes in size of the dynamic shadows are detected with reference to the surfaces the respective shadows are cast on.

18 . The computing device of claim 16 , wherein the set of predetermined traffic situations include traffic situations for anticipating an out-of-sight traffic participant, the traffic situations including the out-of-sight traffic participant casting a shadow into the field-of-view of the image sensor, wherein the shadow is a moving shadow.

19 . The computing device of claim 16 , wherein the set of predetermined traffic situations include traffic situations comprising a row of parking slots, wherein a plurality, but not all, of the parking slots are occupied by respective cars, the cars in the occupied parking slots respectively casting a shadow into the field-of-view of the image sensor, an unoccupied parking slot being identifiable by lack of a corresponding shadow even if the unoccupied parking slot itself is still out-of-sight.

20 . The computing device of claim 15 , wherein recognizing at least one most probable traffic situation out of a set of predetermined traffic situations includes reducing a probability value relating to traffic situations comprising an object as it has been detected by the vision processor if the detected object lacks a corresponding shadow although such shadow would be expected due to existing lighting conditions, including when all other objects detected by the vision processor in vicinity of the detected object cast a respective shadow.

21 . The computing device of claim 15 , wherein the set of predetermined traffic situations includes traffic situations comprising a traffic participant with active vehicle lighting, in particular wherein the active vehicle lighting comprises at least one out of brake lights, reversing lights, direction indicator lights, hazard warning lights, low beams and high beams, emitting the artificial lighting to be detected in the image frames.

22 . The computing device of claim 15 , wherein the set of predetermined traffic situations includes traffic situations for anticipating an out-of-sight traffic participant, the traffic situations comprising the out-of-sight traffic participant with active vehicle lighting, the active vehicle lighting being detectable in the field-of-view of the image sensor and illuminating one or more of objects, surfaces, or airborne particles in the field-of-view of the image sensor.

23 . The computing device of claim 15 , wherein the set of predetermined traffic situations includes traffic situations for anticipating traffic participants suddenly moving into a path of the host vehicle, the traffic situations respectively comprising the traffic participant with active vehicle lighting in the field-of-view of the image sensor, the active vehicle lighting being of a defined type.

24 . A method, comprising:
acquiring, with an image sensor, an image data stream comprising a plurality of image frames; analyzing, with a vision processor, the image data stream to detect objects, shadows and/or lighting in the image frames, wherein analyzing the image data stream comprises detecting artificial lighting in the image frames, the artificial lighting being dynamic lighting that includes at least one of moving or changing size in relation to surfaces or objects being illuminated; recognizing, with a situation recognition engine, at least one most probable traffic situation out of a set of predetermined traffic situations taking into account the detected objects, shadows and/or lighting; and controlling the vehicle taking into account the at least one most probable traffic situation.

25 . The method of claim 24 , further comprising analyzing the image data stream by detecting shadows in the image frames.

26 . The method of claim 25 , further comprising analyzing the image data stream by detecting dynamic shadows, wherein movements and/or changes in size of the dynamic shadows are detected with reference to the surfaces the respective shadows are cast on.

27 . The method of claim 25 , wherein the set of predetermined traffic situations include traffic situations for anticipating an out-of-sight traffic participant, the traffic situations including the out-of-sight traffic participant casting a shadow into the field-of-view of the image sensor, wherein the shadow is a moving shadow.

28 . The method of claim 25 , wherein the set of predetermined traffic situations include traffic situations comprising a row of parking slots, wherein a plurality, but not all, of the parking slots are occupied by respective cars, the cars in the occupied parking slots respectively casting a shadow into the field-of-view of the image sensor, an unoccupied parking slot being identifiable by lack of a corresponding shadow even if the unoccupied parking slot itself is still out-of-sight.

29 . The method of claim 24 , wherein recognizing at least one most probable traffic situation out of a set of predetermined traffic situations includes reducing a probability value relating to traffic situations comprising an object as it has been detected by the vision processor if the detected object lacks a corresponding shadow although such shadow would be expected due to existing lighting conditions, including when all other objects detected by the vision processor in vicinity of the detected object cast a respective shadow.

30 . The method of claim 24 , wherein the set of predetermined traffic situations includes traffic situations comprising a traffic participant with active vehicle lighting, in particular wherein the active vehicle lighting comprises at least one out of brake lights, reversing lights, direction indicator lights, hazard warning lights, low beams and high beams, emitting the artificial lighting to be detected in the image frames.

31 . The method of claim 24 , wherein the set of predetermined traffic situations includes traffic situations for anticipating an out-of-sight traffic participant, the traffic situations comprising the out-of-sight traffic participant with active vehicle lighting, the active vehicle lighting being detectable in the field-of-view of the image sensor and illuminating one or more of objects, surfaces, or airborne particles in the field-of-view of the image sensor.

32 . The method of claim 24 , wherein the set of predetermined traffic situations includes

Assignees

• Ford Global Tech Llc

Inventors

• Jost Gabrielle
• Benmimoun Ahmed
• Nowak Thomas
• Ma Chenhao
• Golgiri Hamid M
• Pak Tony Tae-Jin
• Elminyawi Youssef
• Mishra Anoop

Classifications

• G06V20/584

  of vehicle lights or traffic lights · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• B60W2554/4045

  Intention, e.g. lane change or imminent movement · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• B60W30/09

  Taking automatic action to avoid collision, e.g. braking and steering · CPC title