System and method for assisting a colorblind user

Patent metadata

Field	Value
Publication number	US-10049599-B2
Application number	US-201515122633-A
Country	US
Kind code	B2
Filing date	Oct 23, 2015
Priority date	May 15, 2015
Publication date	Aug 14, 2018
Grant date	Aug 14, 2018

Abstract

The disclosure proposes a system and method for assisting a colorblind user. The system comprises: a perspective display device for displaying an electronic image and/or allowing the user to view an actual image through the perspective display device; an image acquisition device for acquiring an electronic image corresponding to an actual image which can be viewed through the perspective device; and a processing system for, when the current processing mode of the system is a first display mode, processing the electronic image acquired by the image acquisition device to obtain a processed electronic image, and superimposing the processed electronic image on an actual image viewed through the perspective display device in a way of being aligned with the actual image and displaying it; wherein the processing the electronic image comprises: conducting an HSV space transformation for a target part in the electronic image corresponding to an area of a color in the actual image which can not be distinguished.

First claim

The invention claimed is:

1. A system for assisting a colorblind user, comprising:
a perspective display device for displaying an electronic image and/or allowing the user to view an actual image through the perspective display device;
an image acquisition device for acquiring an electronic image corresponding to an actual image which can be viewed through the perspective display device; and
a processing system for, when a current processing mode of the system is a first display mode, processing the electronic image acquired by the image acquisition device to obtain a processed electronic image, and superimposing the processed electronic image on an actual image viewed through the perspective display device in a way of being aligned with the actual image and displaying it, wherein the processing the electronic image comprises: conducting an HSV space transformation for a target part in the electronic image corresponding to an area of a color in the actual image which can not be distinguished;

wherein a way of the HSV space transformation comprises at least:
y 1 =2/3(360− x )+45, x∈[ 315,360);

y 2 =2/3 x+ 45, x∈[ 0,45];
y 3 =5/6 x+ 102.5, x∈[ 75,165];

wherein the range of the red hue is [315, 360] and [0, 45], the range of the green hue is [75, 165], y 1 , y 2 and y 3 are the hues of the pixels of the target part in the processed electronic image, and x is the hue of a pixel of the target part in the electronic image.

2. The system as claimed in claim 1 , wherein the processing the electronic image further comprises marking the color of the target part with characters.

3. The system as claimed in claim 1 , wherein
the system further comprises a first user interface, wherein the processing system is further configured to conduct reduction of 1-10 degrees of y 1 , y 2 and y 3 in response to a fine adjustment instruction of the user received via the first user interface; and/or
the processing system is further configured to adjust the brightness and/or saturation in response to the received fine adjustment instruction of the user.

4. The system as claimed in claim 1 , wherein the processing system is further used for, when the current processing mode of the system is a second display mode independent of other modes, processing the electronic image acquired by the image acquisition device to obtain a processed electronic image, and superimposing the processed electronic image on an actual image viewed through the perspective display device in a way of being aligned with the actual image and displaying it, wherein the processing the electronic image is:
marking the color of the target part with characters.

5. The system as claimed in claim 2 , wherein the characters are marked inside or outside an outline of the color in the target part.

6. The system as claimed in claim 1 , wherein the system further comprises a pupil positioning device for tracing the convergence point of the lines of sight of the left and/or right eye of the user so as to determine the focal plane, such that the image acquisition device acquires an image of the focal plane, and the system performs coordinate matching for an image displayed by the perspective display device and an actual image viewed through it at least partially based on the positioned convergence point and the pupil position positioned by the pupil positioning device, thereby aligning the two.

7. The system as claimed in claim 4 , wherein the system further comprises a voice output device, wherein the processing system is further used for, in a case in which the current processing mode of the system is the first display mode, the second display mode or a broadcast mode independent of the other modes, recognizing a color in the electronic image which can not be distinguished and outputting it to the user in the form of voice via the voice output device.

8. The system as claimed in claim 7 , wherein the voice output device is a bone conduction headphone.

9. The system as claimed in claim 1 , further comprising a second user interface, wherein the second user interface is further used for recognizing selection of the current processing mode by the user by recognizing different gestures of the user, so that the processor performs corresponding processing.

10. A method for assisting a colorblind user, comprising:
acquiring an electronic image corresponding to an actual image which can be viewed through a perspective display device;
when a current processing mode is a first display mode,
processing the electronic image to obtain a processed electronic image; and
superimposing the processed electronic image on an actual image viewed through the perspective display device in a way of being aligned with the actual image and displaying it on the perspective display device;
wherein the processing the electronic image comprises: conducting an HSV space transformation for a target part in the electronic image corresponding to an area of a color in the actual image which can not be distinguished,

wherein a way of the HSV space transformation comprises at least:
y 1 =2/3(360− x )+45, x∈[ 315,360);

y 2 =2/3 x+ 45, x∈[ 0,45];
y 3 =5/6 x+ 102.5, x∈[ 75,165];
wherein the range of the red hue is [315, 360] and [0, 45], the range of the green hue is [75, 165], y 1 , y 2 and y 3 are the hues of the pixels of the target part in the processed electronic image, and x is the hue of a pixel of the target part in the electronic image.

11. The method as claimed in claim 10 , wherein the processing further comprises marking the color of the target part with characters.

12. The method as claimed in claim 10 , wherein the method further comprises: conducting reduction of 1-10 degrees of y 1 , y 2 and y 3 in response to a received fine adjustment instruction of the user;
and/or
further comprises: adjusting the brightness and/or saturation in response to the received fine adjustment instruction of the user.

13. The method as claimed in claim 10 , further comprising:
when the current processing mode is a second display mode independent of other modes, processing the electronic image to obtain a processed electronic image; and
superimposing the processed electronic image on an actual image viewed through the perspective display device in a way of being aligned with the actual image and displaying it on the perspective display device;
wherein the processing the electronic image is: marking the color of the target part with characters.

14. The method as claimed in claim 11 , wherein the characters are marked inside or outside an outline of the color in the target part.

15. The method as claimed in claim 10 , wherein the method further comprises:
tracing the convergence point of the lines of sight of the left and/or right eye of the user so as to determine the focal plane, such that the image acquisition device acquires an image of the focal plane, and performing coordinate matching for an image displayed by the perspective display device and an actual image viewed through it at least partially based on the positioned convergence point and the positioned pupil position of the user thereby aligning the two.

16. The method as claimed in claim 13 , further comprising: in a case in which the current processing mode is the first display mode, the second display mode or a broadcast mode independent of the other modes,
recognizing a color in the electronic image which can not be distinguished and outputting it to the user in the form of voice via a voice output device.

17. The method as claime

Assignees

• Boe Technology Group Co Ltd
• Boe Technology Group Co Ltd

Inventors

• Sun Mengmeng

Classifications

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G09B21/008 Primary

  using visual presentation of the information for the partially sighted · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title