Surround view system

Patent metadata

Field	Value
Publication number	US-10076997-B2
Application number	US-201213563408-A
Country	US
Kind code	B2
Filing date	Jul 31, 2012
Priority date	Aug 5, 2011
Publication date	Sep 18, 2018
Grant date	Sep 18, 2018

Abstract

A system operable to generate a surround view based on image data, where the image data may include image data associated with surroundings of an object, such as a vehicle. The system may also be operable to project the surround view inversely onto a bowl-shaped projection surrounding the object. Further, the system may be operable to generate a virtual user view via a virtual camera position, where the virtual camera position is on a first horizontal ellipse that is about the bowl-shaped projection at a first height. Also, the system is operable to determine a viewing direction from the virtual camera position that points at a location on a second horizontal ellipse that is about the bowl-shaped projection at a second height. The second height may be lower than the first height and the second horizontal ellipse may be smaller than the first horizontal ellipse.

First claim

We claim:

1. A surround view system comprising:
an image processing device including a processor and memory storing instructions executable to:
generate a surround view based on image data collected from image sensors, where the surround view projects the image data inversely on a bowl-shaped projection surrounding an object; and
generate a virtual user view from a virtual camera position at a viewing direction, where the virtual camera position is on a first horizontal ellipse parallel to the ground at a first height above the ground, and the viewing direction directs from the virtual camera position to a point located on a second horizontal ellipse having a major axis and a minor axis that is parallel to the ground at a second height above the ground, where the major axis is longer than the minor axis, where the second height is lower than the first height, where the second horizontal ellipse is smaller than the first horizontal ellipse, and where each of the first horizontal ellipse and the second horizontal ellipse is smaller than the bowl-shaped projection at associated respective vertical locations of the bowl-shaped projection and each of the first horizontal ellipse and the second horizontal ellipse extends around the object; and

a control device including one or more operating elements to change the viewing direction and to change the virtual camera position at least in a horizontal direction,
where the virtual camera position and the viewing direction are derived from the first and second horizontal ellipses, and where the image processing device is operable to:
maintain the virtual camera position and to change the viewing direction by moving the point located on the second horizontal ellipse in response to a command of changing the viewing direction.

2. The surround view system according to claim 1 , where the instructions are further executable to:
maintain the virtual camera position and to change the viewing direction, so that an opening angle of the viewing direction is within a predefined maximum opening angle, wherein the opening angle of the viewing direction is defined by a projection of a direct line of the virtual camera position to a center of the second horizontal ellipse and by a direct line of the center of the second horizontal ellipse to the point located on the second horizontal ellipse, and wherein, when changing the viewing direction, the viewing direction follows points located on the second horizontal ellipse to continue directing from the virtual camera position to the second horizontal ellipse.

3. The surround view system according to claim 2 , where the instructions are further executable to:
adjust the viewing direction of the virtual camera position on the first horizontal ellipse to maintain the opening angle within the predefined maximum opening angle.

4. The surround view system according to claim 1 , where the instructions are further executable to determine a perimeter of the second horizontal ellipse so that the perimeter of the second horizontal ellipse corresponds to a perimeter of the object, and where a size of the first horizontal ellipse is within a range of ten to thirty percent greater than a size of the second horizontal ellipse.

5. The surround view system according to claim 1 , where:
the instructions are further executable to provide a starting object view in which the virtual camera position is behind the object at a point of the first horizontal ellipse that is located on a first axis of the first horizontal ellipse, and
where the viewing direction points towards a location on the second horizontal ellipse that is located on a first axis of the second horizontal ellipse, where the first axis of the first horizontal ellipse is a major axis of the first horizontal ellipse and the first axis of the second horizontal ellipse is the major axis of the second horizontal ellipse.

6. The surround view system according to claim 1 , where the instructions are further executable to position the first horizontal ellipse and the second horizontal ellipse coaxially and parallel to each other and wherein the first height of the first horizontal ellipse is within a range of four and a half feet to seven and a half feet above the ground and the viewing direction is tilted downward toward the second horizontal ellipse.

7. The surround view system according to claim 1 , where the control device contains a turning element that is a part of a vehicle electronic system used for controlling another device in a vehicle, the turning element operable to be:
turned in a first direction, to indicate a change of the virtual camera position in a first horizontal direction; and
turned in a second, different direction, to indicate a change of the virtual camera position in a second horizontal direction, where the object remains centered with respect to the virtual user view as the virtual camera position changes in each of the first and second horizontal directions.

8. The surround view system according to claim 1 , where:
the instructions are further executable to change the virtual camera position vertically from a first height to a second height above the object, and
where a perimeter of the second horizontal ellipse and a perimeter of the first horizontal ellipse are based on the second height above the object.

9. The surround view system according to claim 1 , where:
the instructions are further executable to change the virtual camera position in a direction about a first ellipsoidally shaped body defined by a rotation of the first horizontal ellipse, and
where the viewing direction points towards a location on a second ellipsoidally shaped body defined by a rotation of the second horizontal ellipse.

10. A method comprising:
generating via an image processing device of a surround view system, a surround view based on image data corresponding to surroundings of an object;
projecting the surround view inversely onto a bowl-shaped projection surrounding the object;
generating a virtual user view via a virtual camera position and a viewing direction, where the virtual camera position is on a first horizontal ellipse located parallel to the ground at a first height above the ground, and the viewing direction directs from the virtual camera position to a point located on a second horizontal ellipse located parallel to the ground at a second height above the ground, where the second horizontal ellipse has a major axis and a minor axis, the major axis being longer than the minor axis, where the second height is lower than the first height, where the second horizontal ellipse is smaller than the first horizontal ellipse, and where each of the first horizontal ellipse and the second horizontal ellipse is smaller than the bowl-shaped projection at associated respective vertical locations of the bowl-shaped projection and each of the first horizontal ellipse and the second horizontal ellipse extends around the object; and
adjusting, via a control device, the viewing direction and the virtual camera position at least in a horizontal direction,
wherein the virtual camera position and the viewing direction are determined based on the first and second horizontal ellipses, the method further comprising:
maintaining the virtual camera position; and
changing the viewing direction by moving the point located on the second horizontal ellipse responsive to a command for a horizontal view change.

11. The method according to claim 10 , further comprising:
maintaining the virtual camera position; and
changing the viewing direction until a predefined maximum opening angle of the viewing direction is obtained, wherein an opening angle of the viewing direction is defin

Assignees

• Quast Johannes
• Scholl Kay Ulrich
• Gassmann Bernd
• Harman Becker Automotive Systems Gmbh

Inventors

• Quast Johannes
• Scholl Kay-Ulrich
• Gassmann Bernd

Classifications

• B60R2300/607

  from a bird's eye viewpoint · CPC title

• B60R2300/105

  using multiple cameras · CPC title

• B60R2300/602

  with an adjustable viewpoint · CPC title

• B60R2300/303

  using joined images, e.g. multiple camera images · CPC title

• B60R1/00 Primary

  Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles · CPC title