Image processing device and image processing method

Patent metadata

Field	Value
Publication number	US-12592004-B2
Application number	US-202318389932-A
Country	US
Kind code	B2
Filing date	Dec 20, 2023
Priority date	Dec 27, 2022
Publication date	Mar 31, 2026
Grant date	Mar 31, 2026

Abstract

An image processing device connected to a display device having a non-planar screen. The image processing device generates a parameter for generating a display-target image in accordance with information on an eye position; generates a display-target image from an outside-world image based on the generated parameter; and outputs the generated display-target image to the display device. The parameter includes: an element of a projective transformation matrix for transforming the coordinates of the outside-world image from those in the first coordinate system corresponding to the real space into those in the coordinate system (a second coordinate system) of the outside-world image; and an element of a projective transformation matrix for transforming, into the coordinates of the screen, the coordinates of a part of the outside-world image corresponding to the area of the reference plane to be displayed on the screen.

First claim

What is claimed is:

1 . An image processing device disposed in a moving body and connected to a display device having a screen at least a part of which is non-planar, the image processing device comprising:
a controller; and an image capturing device configured to capture an outside-world image showing an outside world of the moving body; wherein the controller is configured to: receive information on an eye position of an occupant of the moving body; generate a parameter for generating a display-target image to be displayed on the display device from the outside-world image in accordance with the information on the eye position; generate the display-target image from the outside-world image based on the parameter; and output the display-target image to the display device; wherein the parameter includes: an element of a projective transformation matrix for transforming coordinates on a reference plane set in a real space from a first coordinate system corresponding to the real space to a second coordinate system that is a coordinate system of the outside-world image; and an element of a projective transformation matrix that transforms coordinates on the screen into coordinates on the reference plane corresponding to an area to be displayed on the screen, by specifying an intersection coordinate on the reference plane, where an extension line of the line segment connecting the eye position in the first coordinate system and the coordinate on the screen disposed in the first coordinate system based on screen data indicating a shape and a position of the screen intersecting with the reference plane set in the first coordinate system at the intersection coordinate on the reference plane.

2 . The image processing device according to claim 1 , wherein
the parameter is used to: transform the coordinates of the outside-world image from those in the first coordinate system into those in the second coordinate system, and to transform the coordinates within the area into the coordinates of the screen, provided that a target of the transformations is some of the pixels of the screen; and the controller performs interpolation processing based on the information on some of the pixels included in the display-target image generated based on the parameter, thereby acquiring information on all pixels of the display-target image.

3 . The image processing device according to claim 1 , wherein,
the screen is a substantially rectangular screen formed in a curved-surface shape that is curved in a first direction; and the screen data includes information on a curved-line shape in the first direction, a straight-line shape in a second direction perpendicular to the first direction, and an installation angle of the screen.

4 . The image processing device according to claim 1 , wherein
the display device is installed in a pillar portion of the moving body.

5 . The image processing device according to claim 1 , wherein,
the outside-world image is an image captured by the image capturing device including a wide-angle lens; and the parameter further includes an element for removing a barrel distortion of the outside-world image to transform the coordinates of the external image from those in the first coordinate system into those in the second coordinate system.

6 . The image processing device according to claim 1 , wherein
the image processing device is connected to a plurality of display devices installed in the moving body; the controller generates, in accordance with the information on the eye position, parameters corresponding respectively to the plurality of display devices; and the controller generates the display-target images corresponding respectively to the plurality of display devices based on the respective parameters generated by the controller.

7 . An image processing device disposed in a moving body and connected to a display device having a screen at least a part of which is non-planar, the image processing device comprising:
an eye-point information acquisition unit configured to acquire information on an eye position of an occupant of the moving body; an image acquisition unit configured to acquire an outside-world image showing an outside world of the moving body; a parameter generation unit configured to generate a parameter for generating a display-target image to be displayed on the display device from the outside-world image in accordance with the information on the eye position; an image generation unit configured to generate the display-target image from the outside-world image based on the parameter generated by the parameter generation unit; and an image output unit configured to output the display-target image generated by the image generation unit to the display device; wherein the parameter includes: an element of a projective transformation matrix for transforming coordinates on a reference plane set in a real space from a first coordinate system corresponding to the real space to a second coordinate system that is a coordinate system of the outside-world image; and an element of a projective transformation matrix that transforms coordinates of the screen into coordinates of the outside-world image within an area of the reference plane displayed on the screen, by specifying the coordinate on a reference plane, where the extension line of the line segment connecting the eye position in the first coordinate system and the coordinate on the screen disposed in the first coordinate system based on screen data indicating a shape and a position of the screen intersects with the reference plane set in the first coordinate system; and wherein the display device is movably installed in the moving body, and the parameter is updated in accordance with motions of the display device.

8 . An image processing method executed by an image processing device disposed in a moving body and connected to a display device having a screen at least a part of which is non-planar, the method comprising the steps of:
acquiring information on an eye position of an occupant of the moving body; acquiring an outside-world image showing an outside world of the moving body; generating a parameter for generating a display-target image to be displayed on the display device from the outside-world image in accordance with the information on the eye position; generating the display-target image from the outside-world image based on the generated parameter; and outputting the generated display-target image to the display device, wherein the parameter includes: an element of a projective transformation matrix for transforming coordinates on a reference plane set in a real space from a first coordinate system corresponding to the real space to a second coordinate system that is a coordinate system of the outside-world image; and an element of a projective transformation matrix that transforms coordinates on the screen into coordinates on the reference plane corresponding to an area to be displayed on the screen, by specifying an intersection coordinate on the reference plane, where an extension line of the line segment connecting the eye position in the first coordinate system and the coordinate on the screen disposed in the first coordinate system based on screen data indicating a shape and a position of the screen intersecting with the reference plane set in the first coordinate system at the intersection coordinate on the reference plane.

9 . The image processing method according to claim 8 , wherein
the parameter is used to: transform the coordinates of the outside-world image from those in the first coordinate system into those in the second coordinate system, and to transform the coordinates within the area

Assignees

• Faurecia Clarion Electronics Co Ltd

Inventors

• Murakami Tetsuro
• Sakata Naoto
• Koga Masashi

Classifications

• G06T5/80

  Geometric correction · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/14 Primary

  Digital output to display device {; Cooperation and interconnection of the display device with other functional units} · CPC title

• G06T11/00 Primary

  Two-dimensional [2D] image generation · CPC title