Image processing device and image processing method

Patent metadata

Field	Value
Publication number	US-12581024-B2
Application number	US-202318531998-A
Country	US
Kind code	B2
Filing date	Dec 7, 2023
Priority date	Dec 22, 2022
Publication date	Mar 17, 2026
Grant date	Mar 17, 2026

Abstract

An image processing device that enables an occupant of a mobile body, such as a vehicle, to intuitively understand the relationship between an outside image displayed on a display device and an eye position. The device includes: a 3D position information acquisition unit that acquires information on the position of an occupant's eye; an image acquisition unit that acquires an outside image; a cropping unit that crops a region image from the outside image, based on the eye position information; a projection image generation unit that generates a projection image of a 3D model having an opening, as viewed from the position of the eye; an image compositing unit that composites the region image and the projection image such that the region image appears in the opening of the projection image on the screen; and an image output unit that outputs a composite image to the display device.

First claim

What is claimed is:

1 . An image processing device, comprising:
a 3D-point information acquisition unit that acquires information on a position of an eye of an occupant of a mobile body; an image acquisition unit that acquires an outside image of the outside of the mobile body; a cropping unit that crops a region image of a region displayed on a screen of a display device provided in the mobile body from the outside image acquired from the image acquisition unit, based on the eye position information; a projection image generation unit that generates a projection image of a 3D model having an opening, as viewed from the position of the eye on the screen; an image compositing unit that composites the region image and the projection image such that the region image appears in the opening of the projection image on the screen; and an image output unit that outputs a composite image of the image compositing unit to the display device.

2 . The image processing device according to claim 1 , wherein the 3D model imitates a frame enclosing the screen, and the projection image generation unit deforms the shape of the frame to match the shape of the screen.

3 . The image processing device according to claim 2 , further comprising:
an object detection unit that detects an object in a periphery of the mobile body; and an alert display unit that displays an alert in a portion of the frame corresponding to a direction in which the object is positioned when the object detected by the object detection unit is in a position not appearing in the region image.

4 . The image processing device according to claim 1 , wherein the cropping unit:
crops a projection range of the outside image projected on the screen as the region image when the eye position is set as the projection center; and deforms the region image to match the shape of the screen.

5 . The image processing device according to claim 1 , wherein the outside image is an image of the outside of the mobile body on an opposite side from the occupant across the display device and is captured by an image capture device provided with a wide-angle lens.

6 . The image processing device according to claim 1 , further comprising:
an information acquisition unit that acquires information related to the mobile body; and a superposition unit that superimposes the information acquired by the information acquisition unit on the composite image, wherein the image output unit outputs the composite image on which the information is superimposed by the superposition unit to the display device.

7 . The image processing device according to claim 6 , further comprising:
a shape deformation unit that deforms the projection image according to the information acquired by the information acquisition unit, wherein the superposition unit superimposes the information on the projection image after being deformed by the shape deformation unit, which is included in the composite image.

8 . An image processing method, comprising the steps of:
acquiring information on a position of an eye of an occupant of a mobile body; acquiring an outside image of the outside of the mobile body; cropping a region image of a region displayed on a screen of a display device provided in the mobile body from the acquired outside image, based on the eye position information; generating a projection image of a 3D model having an opening, as viewed from the position of the eye on the screen; compositing the region image and the projection image such that the region image appears in the opening of the projection image on the screen; and outputting a composite image of the region image and projection image to the display device.

9 . The image processing method according to claim 8 , wherein the 3D model imitates a frame enclosing the screen, and the shape of the frame is deformed to match the shape of the screen to generate the projection image in the step of generating the projection image.

10 . The image processing method according to claim 9 , further comprising the steps of:
detecting an object in a periphery of the mobile body; and displaying an alert in a portion of the frame corresponding to a direction in which the object is positioned when the detected object is in a position not appearing in the region image.

11 . The image processing method according to claim 8 , wherein the region image is cropped in the step of cropping the region image by cropping a projection range of the outside image projected on the screen as the region image when the eye position is set as the projection center, and deforming the region image to match the shape of the screen.

12 . The image processing method according to claim 8 , wherein the outside image is an image of the outside of the mobile body on an opposite side from the occupant across the display device and is captured by an image capture device provided with a wide-angle lens.

13 . The image processing method according to claim 8 , further comprising the steps of:
acquiring information related to the mobile body; and superimposing the information related to the mobile body on the composite image, wherein the composite image on which the information related to the mobile body is superimposed is output to the display device in the step of outputting the composite image.

14 . The image processing method according to claim 13 , further comprising the step of:
deforming the projection image according to the eye position information, wherein the information related to the mobile body is superimposed on the projection image after being deformed, which is included in the composite image in the step of superimposing the information related to the mobile body in the composite image.

Assignees

• Faurecia Clarion Electronics Co Ltd

Inventors

• Murakami Tetsuro
• Sakata Naoto
• Koga Masashi

Classifications

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/30268

  Vehicle interior · CPC title

• G06T2210/22

  Cropping · CPC title

• G06T2207/30201

  Face · CPC title