Face detection, augmentation, spatial cueing and clutter reduction for the visually impaired

Patent metadata

Field	Value
Publication number	US-10013599-B2
Application number	US-201615206453-A
Country	US
Kind code	B2
Filing date	Jul 11, 2016
Priority date	Jul 13, 2015
Publication date	Jul 3, 2018
Grant date	Jul 3, 2018

Abstract

An apparatus for improving performance of a retinal implant may include processing circuitry. The processing circuitry may be configured to receive image data corresponding to a camera field of view, determine whether a particular object is detected within the camera field of view, perform image data processing to enable a representation of a portion of the image data corresponding to an implant field of view to be provided on a retinal implant where the implant field of view is smaller than the camera field of view, and, responsive to the particular object being located outside the implant field of view, provide a directional indicator in the implant field of view to indicate a location of the particular object relative to the implant field of view.

First claim

What is claimed is:

1. An apparatus comprising:
a video processing unit operably coupled to a camera defining a camera field of view, the video processing unit including processing circuitry, the processing circuitry being configured to:
receive image data corresponding to the camera field of view;
determine whether a particular object is detected within the camera field of view, wherein the particular object comprises a face;
perform image data processing to enable a representation of a portion of the image data corresponding to an implant field of view to be provided on a retinal implant, the implant field of view being smaller than the camera field of view; and
responsive to the particular object being located outside the implant field of view, provide a directional indicator in the implant field of view to indicate a location of the particular object relative to the implant field of view, wherein the video processing unit is configured to transition between a face detection mode and a background clutter reduction mode based on detection of the face in the camera field of view.

2. The apparatus of claim 1 , wherein the directional indicator employs a selected indication paradigm.

3. The apparatus of claim 2 , wherein the selected indication paradigm is identified by a characteristic of the directional indicator that indicates a type of spatial cue provided by the directional indicator.

4. The apparatus of claim 3 , wherein the characteristic comprises a size, shape, intensity, blink pattern, blink frequency, or display pattern of the directional indicator.

5. The apparatus of claim 2 , wherein the selected indication paradigm indicates a direction outside the implant at which the particular object can be found.

6. The apparatus of claim 2 , wherein the selected indication paradigm indicates a location of the particular object relative to the camera field of view.

7. The apparatus of claim 1 , wherein the directional indicator is overlaid over other image data in the implant field of view.

8. The apparatus of claim 1 , wherein the video processing unit is operably coupled to glasses.

9. The apparatus of claim 1 , wherein the video processing unit is operably coupled to the retinal implant, the retinal implant being defined by an electrode array.

10. The apparatus of claim 1 , wherein the video processing unit is configured to determine whether the face is in the implant field of view, and perform segmentation of the image data around the face in response to the face being in the implant field of view.

11. The apparatus of claim 10 , wherein the segmentation comprises defining a shape around the face, and wherein the video processing unit is configured to perform image enhancement inside the shape and perform filtering on background data outside the ellipse.

12. The apparatus of claim 11 , wherein the image enhancement comprises histogram equalization, intensity adjustment, or contrast adjustment.

13. The apparatus of claim 11 , wherein the video processing unit is further configured to downsample an output from the image enhancement prior to display at the retinal implant.

14. The apparatus of claim 10 , wherein the video processing unit is configured to determine that the face is no longer in the field of view and continue to display the face for at least a predetermined time.

15. The apparatus of claim 1 , further comprising a mapping agent and a positioning module, wherein the mapping agent constructs a map and the positioning module determines location information relative to the map.

16. The apparatus of claim 15 , wherein the processing circuitry is configured to receive a query regarding the particular object from a wearer of the apparatus, and wherein the processing circuitry is further configured to provide the directional indicator in the implant field of view to indicate the location of the particular object relative to the implant field of view.

17. The apparatus of claim 16 , wherein the particular object is a specific face selected by the wearer.

18. A method comprising:
receiving, by processing circuitry, image data corresponding to a camera field of view, wherein the processing circuitry is part of a video processing unit that is operably coupled to a camera defining the camera field of view;
determining, by the processing circuitry, whether a face is detected within the camera field of view;
performing, by the processing circuitry, image data processing to enable a representation of a portion of the image data corresponding to an implant field of view to be provided on a retinal implant, the implant field of view being smaller than the camera field of view;
providing, by the processing circuitry and responsive to the face being located outside the implant field of view, a directional indicator in the implant field of view to indicate a location of the face relative to the implant field of view; and
transitioning, by the video processing unit, between a face detection mode and a background clutter reduction mode based on detection of the face in the camera field of view.

Assignees

• Univ Johns Hopkins

Inventors

• Rollend Derek M
• Katyal Kapil D
• Wolfe Kevin C
• Kleissas Dean M
• Para Matthew P
• Rosendall Paul E
• Helder John B
• Burlina Philippe M
• Cornish Duane C
• Murphy Ryan J
• Johannes Matthew S
• Roy Arup
• Billings Seth D
• Oben Jonathan M
• Greenberg Robert J

Classifications

• G06V40/161

  Detection; Localisation; Normalisation · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06K9/00228 Primary

  Physics · mapped topic

• G06T7/11

  Region-based segmentation · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title