Correction of displayed images for users with vision abnormalities

Patent metadata

Field	Value
Publication number	US-9916690-B2
Application number	US-201514679316-A
Country	US
Kind code	B2
Filing date	Apr 6, 2015
Priority date	Jun 19, 2013
Publication date	Mar 13, 2018
Grant date	Mar 13, 2018

Abstract

This invention solves the problem that objects in the field of view of reading glasses using multifocal lenses for correcting ocular refractive errors such as presbyopia appear distorted. This image display device is provided with the following: a distance computation unit that computes the distance to a subject in a field-of-view image; a visual-acuity-information acquisition unit that stores visual-acuity information for the user in advance; a corrected-image generation unit that generates and outputs a corrected image on the basis of the field-of-view image and an eye image generated from the field-of-view image, the distance information, and the visual-acuity information; and a display unit that displays the corrected image by superimposing same onto the user's field of view. Since the corrected image is superimposed without the use of lenses, lenses being the cause of the abovementioned distortion, the user can see close objects clearly.

First claim

The invention claimed is:

1. A head mounted type or spectacle type image display device which displays images in a visual field of a user in an overlapping manner so that the user can clearly see a subject within the visual field, the image display device comprising:
a visual field imaging unit that captures and outputs a visual field image which is an image of the visual field of the user; and
a processor comprising:
a distance calculation unit that receives the visual field image, and calculates a distance between the subject in the visual field image and the visual field imaging unit so as to output as distance information;
an eyesight information acquisition unit that acquires eyesight information which is information regarding eyesight of the user;
a correction image generation unit that generates and outputs a correction image on the basis of the visual field image, the distance information, and the eyesight information; and
a naked-eye image generation unit that receives the visual field image, the distance information, and the eyesight information, and generates and outputs a naked-eye image, which is a predicted value of an image which is projected on a retina of the user when the user sees the subject without using the image display device, and which is not directly captured from the retina of the user; and

a display unit that receives the correction image, and displays the correction image of the subject within the visual field of the user in the overlapping manner,
wherein the correction image generation unit receives the visual field image and the naked-eye image, and generates and outputs the correction image on the basis of an image obtained by adding a constant value C off to a difference image between the visual field image and the naked-eye image, the C off is a constant which is added so that the difference image does not become a negative value, and
wherein the overlapping manner indicates that displaying at the display unit is performed so that the user can visually recognize the subject in a state in which the correction image overlaps an image of the subject which is directly seen by the user.

2. The image display device according to claim 1 , the processor further comprising:
an eye position calculation unit that calculates a position and a posture of each eye of the user including a visual line of the user so as to output as eye position information,
wherein the naked-eye image generation unit receives the visual field image, the distance information, the eyesight information, and the eye position information, calculates a gaze point distance which is a distance between a gaze point of the subject on the visual line of the user and each eye of the user, and generates and outputs the naked-eye image in a case where the user sees the gaze point.

3. The image display device according to claim 2 , wherein the display unit includes a projection optical system that displays an image at the gaze point distance within the visual field of the user in an overlapping manner,
wherein the naked-eye image generation unit outputs the gaze point distance,
wherein the correction image generation unit generates and outputs the correction image on the basis of the visual field image, the naked-eye image, and the gaze point distance, and
wherein the projection optical system controls an optical system so that the correction image is displayed so as to overlap the subject located at the gaze point.

4. The image display device according to claim 2 , wherein the visual field imaging unit is a light field camera which captures multi-viewpoint images and can reconfigure an image from any viewpoint, and
wherein the visual field imaging unit receives visual line information of the user calculated by the eye position calculation unit so as to reconfigure an image of the visual line of the user and to output as the visual field image.

5. The image display device according to claim 1 , wherein the correction image generation unit makes the correction image bright in a case where the visual field of the user is dark.

6. The image display device according to claim 1 , wherein the eyesight information includes information regarding a change in the eyesight of the user over time.

7. The image display device according to claim 6 , wherein the eyesight information includes information regarding a relationship between an elapsed time after the user starts to gaze at the subject at a predetermined distance and a change in an extent of focus blurring, and
wherein the naked-eye image generation unit generates the naked-eye image on the basis of the eyesight information and a change in the gaze point distance.

8. The image display device according to claim 6 , wherein the eyesight information includes information regarding a relationship between an elapsed time after the visual field of the user changes from a bright state to a dark state and a change in brightness of an image visually recognized by the user, and
wherein the naked-eye image generation unit generates the naked-eye image on the basis of the eyesight information and a change in the brightness of the image within the visual field.

9. An image display method of displaying images in a visual field of a user in an overlapping manner so that the user can clearly see a subject within the visual field, the method comprising:
a visual field imaging step of capturing and outputting a visual field image which is an image of the visual field of the user;
a distance calculation step of receiving the visual field image, and calculating a distance between the subject in the visual field image and a visual field imaging unit so as to output as distance information;
an eyesight information acquisition step of acquiring eyesight information which is information regarding eyesight of the user;
a correction image generation step of generating and outputting a correction image on the basis of the visual field image, the distance information, and the eyesight information; and
a naked-eye image generation step of receiving the visual field image, the distance information, and the eyesight information, and generating and outputting a naked-eye image, which is a predicted value of an image which is projected on a retina of the user when the user sees the subject without using an image display device, and which is not directly captured from the retina of the user; and
a display step of receiving the correction image, and displaying the correction image of the subject within the visual field of the user in the overlapping manner,
wherein the correction image generation step includes a step of receiving the visual field image and the naked-eye image, and generating and outputting the correction image on the basis of an image obtained by adding a constant value C off to a difference image between the visual field image and the naked-eye image, the C off is a constant which is added so that the difference image does not become a negative value, and
wherein the overlapping manner indicates that displaying at a display unit is performed so that the user can visually recognize the subject in a state in which the correction image overlaps an image of the subject which is directly seen by the user.

10. A computer program product comprising a non-transitory computer-readable storage media and program instructions stored on the non-transitory computer-readable storage media for causing a computer to execute each of the steps included in the image display method according to claim 9 .

Assignees

• Panasonic Ip Man Co Ltd

Inventors

• Nobori Kunio
• Maruyama Ayako
• Kato Yumiko
• Inoue Tsuyoshi

Classifications

• G06T2207/10012

  Stereo images · CPC title

• G02C11/10

  Electronic devices other than hearing aids · CPC title

• G06T5/006

  Physics · mapped topic

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T2207/30201

  Face · CPC title