Methods and systems for detecting obstacles for a visual prosthesis

Patent metadata

Field	Value
Publication number	US-10773083-B2
Application number	US-201514854916-A
Country	US
Kind code	B2
Filing date	Sep 15, 2015
Priority date	Sep 15, 2014
Publication date	Sep 15, 2020
Grant date	Sep 15, 2020

Abstract

Systems and methods for detecting and displaying obstacles to visually impaired persons are disclosed. Images from a captured scene are analyzed to detect obstacles. A V-Disparity algorithm is employed to detect the ground plane and remove it from the images, thereby enhancing the capacity of retinal implants to display only obstacles. The system includes cameras to capture images; a processor with memory to detect obstacles in the captured images, to remove a ground plane, and to calculate a map of pixels representing the obstacles; and a retinal implant configured to receive the map of pixels and display the obstacles to a system user by electrically stimulating retinal nerves. The method includes synchronizing captured images and calculating a disparity map between the captured images to detect obstacles and remove a ground plane from processed images. A map of pixels representing the obstacle is transmitted to a retinal implant. Depth information may derive from stereo cameras, a time of flight camera or a structured light camera.

First claim

What is claimed is:

1. A visual prosthesis comprising:
stereo cameras sending visual information, and mounted on a pair of glasses;
an accelerometer detecting acceleration and movement of the glasses, and generating movement data;
a video processor deriving depth information from the video information and combining the depth information with the video information to provide indicia of depth within stimulation patterns, and combining depth information with movement data to correct the stereo cameras to horizontal to remove a ground plane;
a transmitter receiving the stimulation patterns and transmitting the stimulation patterns; and
an implantable neurostimulator receiving the stimulation patterns from the transmitter, and stimulating neural tissue according to the stimulation patterns.

2. The visual prosthesis according to claim 1 , wherein the source of depth information includes a time-of-flight camera.

3. The visual prosthesis according to claim 1 , wherein the source of depth information includes a structured light camera.

4. The visual prosthesis according to claim 1 , wherein the video processor is configured to detect obstacles based on the visual information and the depth information, to remove a ground plane, to calculate a map of pixels representing the obstacles, and to alter the stimulation patterns based on the map of pixels.

5. The visual prosthesis according to claim 1 , wherein the stereo cameras are configured to generate captured images; and the video processor is configured to:
synchronize the captured images;
calculate a disparity map between the captured images;
detect a ground plane based on the disparity map;
generate a processed image representative of a view in front of the system user, based on the captured images and the disparity map;
remove the ground plane from the processed image;
generate a map of pixels representing the obstacles; and
alter the stimulation patterns based on the map of pixels.

6. The visual prosthesis according to claim 1 , wherein the stereo cameras are mounted in temple portions of the glasses.

7. The visual prosthesis according to claim 1 , further comprising an eye tracking camera mounted on the glasses.

8. The visual prosthesis according to claim 1 , wherein the video processor adds greater intensity to closer objects and less intensity to more distant objects.

Assignees

• Second Sight Medical Products Inc

Inventors

• Greenberg Robert J
• Roy Arup
• Caspi Avraham I
• Patel Uday
• Welch Mark D

Classifications

• H04N23/51

  Housings · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G09B21/008

  using visual presentation of the information for the partially sighted · CPC title

• A61N1/36046 Primary

  of the eye · CPC title