Picture processing device and method

Patent metadata

Field	Value
Publication number	US-9787946-B2
Application number	US-201213683601-A
Country	US
Kind code	B2
Filing date	Nov 21, 2012
Priority date	May 25, 2010
Publication date	Oct 10, 2017
Grant date	Oct 10, 2017

Abstract

The disclosed video processing device contains: a video acquisition unit that acquires surroundings information including video taken of the surroundings of a vehicle; a line-of-sight acquisition unit that acquires the origin and direction of the line of sight of the driver of the aforementioned vehicle; a line-of-sight video generation unit which generates, from the surroundings information, line-of-sight video corresponding to the origin of the line of sight; a blocking-information computation unit that computes, on the basis of the origin of the line of sight, blocking information including video or a region of the body of the aforementioned vehicle that blocks the driver's line of sight; and a display-video generation unit that generates display video on the basis of the line-of-sight video and the blocking information.

First claim

What is claimed is:

1. A picture processing device, comprising:
a memory; and
a processor coupled to the memory and configured to:
acquire peripheral information including a picture obtained by shooting a periphery of a driver's vehicle,
acquire a line-of-sight origin and a direction of the line of sight of a driver of the driver's vehicle, the line-of-sight origin being a central position between a left eyeball and a right eyeball of the driver,
generate from the peripheral information which is acquired at a certain point in time, a line-of-sight picture, which corresponds to the line-of-sight origin, at the certain point in time by extracting the line-of-sight picture from the picture included in the peripheral information at the certain point in time based on the line-of-sight origin and the direction of the line of sight,
calculate cutoff information including a vehicle body area or a vehicle body picture of the driver's vehicle which cuts off a line of sight of the driver based on the line-of-sight origin; and
generate a display picture according to the line-of-sight picture and the cutoff information,

wherein:
the line-of-sight picture includes a mirror picture corresponding to a visual mirror confirmation range in which the driver may perform a visual confirmation through at least one mirror of the driver's vehicle,
the cutoff information includes mirror cutoff information containing a vehicle body area and/or a vehicle body picture of the driver's vehicle which cuts off a mirror line-of-sight of the driver to the visual mirror confirmation range,
the display picture includes a mirror display picture for observation by the driver through at least one mirror of the driver's vehicle,

the processor is further configured to:
calculate an effective vision range having a specified angle range in which the driver may perform a visual confirmation based on the line-of-sight origin and the direction of the line of sight, and determine whether there is at least one mirror of the driver's vehicle in the effective vision range,
calculate the visual mirror confirmation range in which the driver may perform a visual confirmation through the mirror of the driver's vehicle according to mirror information including the line-of-sight origin and a position of the mirror when at least one mirror is located in the effective vision range, and generate the mirror picture corresponding to the visual mirror confirmation range from a peripheral picture of the driver's vehicle,
calculate the mirror cutoff information based on the line-of-sight origin and the visual mirror confirmation range, and
generate the mirror display picture according to the mirror picture and the mirror cutoff information.

2. The picture processing device according to claim 1 , wherein:
the direction of the line of sight includes a vehicle window line-of-sight direction from the line-of-sight origin through a window of the driver's vehicle;
the line-of-sight picture includes a vehicle window picture corresponding to the line-of-sight origin and the vehicle window line-of-sight direction;
the cutoff information includes vehicle window cutoff information containing a vehicle body area and/or a vehicle body picture of the driver's vehicle about cutoff of the sight-of-line in the vehicle window line-of-sight direction of the driver;
the display picture includes a vehicle window display picture for observation by the driver through a window of the driver's vehicle;
wherein the processor is further configured to:
generate the vehicle window picture from a peripheral picture of the driver's vehicle,
calculate the vehicle window cutoff information based on the line-of-sight origin, and
generate the vehicle window display picture according to the vehicle window picture and the vehicle window cutoff information.

3. The picture processing device according to claim 1 , wherein
the processor superposes the mirror display picture corresponding to a mirror in the effective vision range on the vehicle window display picture when at least one mirror is located in the effective vision range.

4. The picture processing device according to claim 1 , wherein the processor is further configured to:
calculate a vision range of the driver according to at least one piece of biological information including a pupil, a number of pulses, perspiration, and a retention time in the direction of the line of sight; and
process the display picture based on the vision range.

5. The picture processing device according to claim 1 , wherein the processor is further configured to:
fade an area outside the vision range in the display picture.

6. The picture processing device according to claim 1 , wherein the processor is further configured to:
generate a sight-of-line locus in the line-of-sight picture which is designated by the line-of-sight origin and the direction of the line of sight; and
superpose the sight-of-line locus on the display picture.

7. The picture processing device according to claim 1 , wherein
the line-of-sight origin is an average line-of-sight origin obtained by averaging line-of-sight origin of the driver in a specified time; and
the direction of the line of sight is an average direction of the line of sight obtained by averaging a direction of the line of sight in the specified time.

8. The picture processing device according to claim 1 , wherein the processor is further configured to:
generate a projection picture by projecting a peripheral picture of the driver's vehicle on a 3-dimensional projection surface as a virtual space, and generate as the sight-of-line picture a picture corresponding to the line-of-sight origin and the direction of the line of sight in the projection picture.

9. The picture processing device according to claim 1 , wherein the processor is further configured to:
generate the display picture by removing a picture of an area of a body of the driver's vehicle included in the cutoff information from the line-of-sight picture.

10. The picture processing device according to claim 1 , wherein the processor is further configured to:
generate the display picture by superposing a vehicle body picture of the driver's vehicle included in the cutoff information on the line-of-sight picture.

11. A non-transitory medium which stores a picture processing program used to direct a processor to perform a process, the process comprising: acquiring peripheral information including a picture obtained by shooting a periphery of a driver's vehicle; acquiring a line-of-sight origin and a direction of the line of sight of a driver of the driver's vehicle, the line-of-sight origin being a central position between a left eyeball and a right eyeball of the driver; generating from the peripheral information, which is acquired by the acquiring the peripheral information at a certain point in time, a line-of-sight picture, which corresponds to the line-of-sight origin, at the certain point in time by extracting the line-of-sight picture from the picture included in the peripheral information at the certain point in time based on the line-of-sight origin and the direction of the line of sight; calculating cutoff information including a car vehicle body area or a car vehicle body picture of the driver's vehicle which cuts off a line of sight of the driver based on the line-of-sight origin; and generating a display picture according to the line-of-sight picture and the cutoff information, wherein: the line-of-sight picture includes a mirror picture corresponding to a visual mirror confirmation range in which the driver may perform a visual confirmation through at least one mirror of the driver's vehicle, the cutoff in

Assignees

• Fujitsu Ltd

Inventors

• Yamakage Yuzuru
• Hamaguchi Shingo
• Ozaki Kazuyuki

Classifications

• G06T7/73

  using feature-based methods · CPC title

• G09B19/167

  Control of land vehicles · CPC title

• G06T2207/30248

  Vehicle exterior or interior · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G09B9/052

  characterised by provision for recording or measuring trainee's performance (devices for psychotechnics, e.g. for vehicle drivers A61B5/16, A61B5/18) · CPC title