Intelligent glasses for the visually impaired

Patent metadata

Field	Value
Publication number	US-9805619-B2
Application number	US-201615254486-A
Country	US
Kind code	B2
Filing date	Sep 1, 2016
Priority date	Feb 7, 2014
Publication date	Oct 31, 2017
Grant date	Oct 31, 2017

Abstract

An approach for communicating navigation information on a physical environment to a user. The approach includes a computer receiving digital images of a physical environment of the user captured by digital video devices and converting the digital images into a three-dimensional image. The approach includes the computer analyzing the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment and determining at least one device associated with the user. The approach includes the computer formatting the output data for use with at least one device, wherein the at least one device is capable of providing to the user a spatial map of the physical environment created by an electrical stimulation pad, and receiving a touch from a user on a surface of the electrical stimulation pad corresponding to an object represented in the spatial map of the physical environment.

First claim

What is claimed is:

1. A method for communicating navigation information on a physical environment to a user, the method comprising:
receiving, by a server computer, at least one digital image of a physical environment of the user captured by one or more digital video devices;
converting, by the server computer, the at least one digital image into a three-dimensional image;
analyzing, by the server computer, the three-dimensional image using object analysis to generate output data corresponding to the physical environment, wherein the object analysis includes determining a motion of one or more objects based, at least in part, on a comparison of a digital image to a previous digital image from digital image data captured by the one or more digital video devices in the physical environment for identification to a user, where the motion of the one or more objects is at least one of a pulse of a plurality of electrical stimulation electrodes moving across an area of the electrical stimulation pad of the at least one device corresponding to the motion of each of the one or more objects in the physical environment or is a pitch of one or more beeps that corresponds to the motion of each of the one or more object in the physical environment;

determining, by the server computer, at least one device associated with the user, wherein the at least one device includes a wearable device with a first side of the at least one device including an electrical stimulation pad and a second side of the at least one device including a touch pad;
formatting, by the server computer, the output data for use with the at least one device, wherein the at least one device is capable of providing to the user a spatial map of the physical environment created by the electrical stimulation pad on the first side of the at least one device and providing the identification of the motion of the one or more objects on the first side of the at least one device; and
receiving, by the server computer, a user query for an identification of an object from a user touch on the second side of the at least one device.

2. The method of claim 1 , wherein receiving the user query for the identification of the object from the user touch on the second side of the at least one device, further comprises:
receiving, by the server computer, from the second side of the at least one device, an indication of a user request for a name of the object based, at least in part, on a touch from the user on a location on a surface of the second side of the at least one device corresponding to a representation of the object in the spatial map of the physical environment provided on the first side of the at least one device;
retrieving, by the server computer, a name of the object in the physical environment from a database, the database containing the at least one digital image associated with the name of the object; and
providing, by the server computer, the name of the object to the user by one of an audio device, a smartphone, or an intelligent computing device.

3. The method of claim 1 , wherein formatting the output data for use with the at least one device further comprises:
transmitting, by the server computer, the output data to the at least one device associated with the user, wherein the output data is formatted to create the spatial map of the physical environment using the electrical stimulation pad on the first side of the at least one device and the output data is formatted to create audio responses from one of an audio device, a smartphone and an intelligent computing device representing information on the object in the physical environment.

4. The method of claim 1 , wherein the at least one device associated with the user includes at least one of: an audio device, a smart phone, and an intelligent computing device.

5. The method of claim 1 , wherein formatting the output data for use with the at least one device further comprises:
determining, by the server computer, a level of intensity of the electrical stimulation associated with the object represented in the spatial map of the physical environment, the level corresponding to a motion of the object, a proximity of the object to the user, and a terrain in the physical environment.

6. The method of claim 1 , further comprising:
creating, by the server computer, a representation of the physical environment using a matrix of electrical stimulation electrodes in the electrical stimulation pad providing varying levels of intensity of electrical stimulation to represent a terrain, one or more objects in the terrain, a distance of the one or more objects to the user, and motion of the one or more objects in the physical environment.

7. A computer program product for communicating navigation information on a physical environment to a user, the computer program product comprising:
one or more non-transitory computer readable storage device and program instructions stored on the one or more non-transitory computer readable storage device, the program instructions executable by a processor, the program instructions comprising instructions for:
receiving at least one digital image of a physical environment of the user captured by one or more digital video devices;
converting the at least one digital image into a three-dimensional image;
analyzing the three-dimensional image using object analysis to generate output data corresponding to the physical environment, wherein the object analysis includes determining a motion of one or more objects based, at least in part, on a comparison of a digital image to a previous digital image from digital image data captured by the one or more digital video devices in the physical environment for identification to a user, where the motion of the one or more objects is at least one of a pulse of a plurality of electrical stimulation electrodes moving across an area of the electrical stimulation pad of the at least one device corresponding to the motion of each of the one or more objects in the physical environment or is a pitch of one or more beeps that corresponds to the motion of each of the one or more object in the physical environment;
determining at least one device associated with the user, wherein the at least one device includes a wearable device with a first side of the at least one device including an electrical stimulation pad and a second side of the at least one device including a touch pad;
formatting the output data for use with the at least one device, wherein the at least one device is capable of providing to the user a spatial map of the physical environment created by the electrical stimulation pad on the first side of the at least one device and providing the identification of the motion of the one or more objects on the first side of the at least one device; and
receiving a user query for an identification of an object from a user touch on the second side of the at least one device.

8. The computer program product of claim 7 , wherein receiving the user query for the identification of the object from the user touch on the second side of the at least one device, further comprises:
receiving from the second side of the at least one device, an indication of a user request for a name of the object based, at least in part, on a touch from the user on a location on a surface of the second side of the at least one device corresponding to a representation of the object in the spatial map of the physical environment provided on the first side of the at least one device;
retrieving a name of the object in the physical environment from a database, the database containing the at least one digital image associated with the name of the object; and
providing the name of the object to the user by one of an audio

Assignees

• IBM

Inventors

• Cervantes Humberto Orozco

Classifications

• H04N23/57

  Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G09B5/125

  the stations being mobile · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G09B21/006

  using audible presentation of the information · CPC title