System and method to assist users having reduced visual capability utilizing lighting device provided information

Patent metadata

Field	Value
Publication number	US-9536452-B2
Application number	US-201514711236-A
Country	US
Kind code	B2
Filing date	May 13, 2015
Priority date	May 13, 2015
Publication date	Jan 3, 2017
Grant date	Jan 3, 2017

Abstract

A lighting device obtains data related to objects and boundaries in an area in the vicinity of the lighting device, and a user wearable device provides a display (e.g. an augmented reality display based on the data related to the objects and the area boundaries) for a user/wearer. The lighting device includes a mapping sensor that collects data related to the objects and boundaries in the area. The user wearable device includes a camera or other optical sensor and wireless communication capability. The user wearable device is provided with mapping data that is presented on a display of the user wearable device. The communications and display capabilities allow the user wearable device to obtain room mapping information related to area in the vicinity of the lighting device in order to provide navigational assistance to a visually impaired person in the area.

First claim

What is claimed is:

1. A system, comprising:
a lighting device in an area of a premises including a processor, a wireless communication transceiver, a light source, a memory and a mapping sensor, wherein the lighting device processor is coupled to the wireless communication transceiver, the memory, the light source, and the mapping sensor, the processor upon execution of programming stored in the memory is configured to control operations of the lighting device including the mapping sensor, the mapping sensor being configured to collect mapping data related to the area of the premises in which the lighting device is located; and
a data storage for maintaining area mapping data collected by the mapping sensor of an area in which the lighting device is located in the premises, processed mapping data which is area mapping data that has been processed using image processing techniques, and mapping information including navigational information or indications, wherein the database is accessible by the lighting device processor;
wherein the lighting device memory stores programming, and execution of the programming by the lighting device processor configures the lighting device processor to perform functions, including functions to:
receive a request from a user wearable device for processed mapping data for the area of the premises in which the lighting device is located;
in response to the request, deliver processed mapping data for the area of the premises in which the lighting device is located to the user wearable device;
update the mapping data for the area of the premises to identify locations of moveable objects within the area as the moveable objects move within the area; and
use the updated mapping data to identify objects in the area.

2. The system of claim 1 , wherein the delivered processed mapping data includes one or more of object identifications, object boundaries indications, or navigation instructions or navigation indicators.

3. The system of claim 1 , further comprising:
a server, coupled to the lighting device processor and the database, the server configured to provide image processing services, the image processing services including identifying objects, object boundaries, and inserting as area mapping information navigation instructions and indicators in the processed mapping data; and
wherein the lighting device memory stores programming, and execution of the programming by the lighting device processor configures the lighting device processor to perform further functions, including functions to:
instruct the mapping sensor to collect mapping data of the area of the premises in which the lighting device is located; and
send the collected mapping data to the server for image processing.

4. The system of claim 1 , wherein the lighting device memory stores programming, and execution of the programming by the lighting device processor configures the lighting device processor to perform further functions, including functions to:
instruct the mapping sensor to collect mapping data of the area of the premises in which the lighting device is located;
generate processed mapping data by image processing the collected mapping data; and
store the processed mapping data in the lighting device memory.

5. The system of claim 4 , wherein the lighting device memory stores programming, and execution of the programming by the lighting device processor configures the lighting device processor to perform further functions, including functions to:
after generating the processed mapping data, forward a copy of the processed mapping data to the data storage.

6. The system of claim 1 , wherein the wireless communication transceiver is configured to send or receive data via radio frequency wireless communication.

7. The system of claim 1 , wherein the wireless communication transceiver is configured to perform as least one of sending or receiving data as a communication via light.

8. The system of claim 3 , wherein the lighting device memory stores programming, and execution of the programming by the lighting device processor configures the lighting device to perform functions, including functions to:
instruct the mapping sensor to collect mapping data of the area;
receive the collected mapping data from the mapping sensor; and
forwarding the collected mapping data including a lighting device identifier to the server for image processing, wherein the lighting device identifier uniquely identifies the location of the lighting device in the premises.

9. The system of claim 1 , further comprising:
the user wearable device, the user wearable device including an output device, a user wearable device processor, a user wearable device wireless communication transceiver, and a user wearable device memory,
wherein the user wearable device processor is coupled to the user wearable device wireless communication transceiver and the user wearable device memory, the user wearable device processor being configured to control operations of the user wearable device, wherein the user wearable device memory stores programming, and execution of the programming by the user wearable device processor configures the user wearable device to perform functions, including functions to:
send a request for mapping data, via the user wearable device wireless communication transceiver, related to a premises location of the user wearable device from the lighting device processor, wherein the premises location of the user wearable device is the area in which the lighting device is located;
receive, from the lighting device, the processed mapping data specific to the determined location of the user wearable device; and
based on the processed mapping data, output, via the output device, user detectable stimuli suitable for navigating a user through the area of the premises in which the lighting device is located.

10. The system of claim 9 , wherein the output device of the user wearable device includes:
an audio output device, and the outputted user detectable stimuli is an audio prompt indicating a direction of travel for a user wearing the user wearable device.

11. The system of claim 9 , wherein the output device of the user wearable device comprises:
an audio output device, and the outputted user detectable stimuli is an audio prompt indicating objects in an area surrounding the determined location of the user wearable device.

12. The system of claim 9 , wherein the output device of the user wearable device comprises:
a display configured to provide an augmented reality display of an area surrounding the determined location of the user wearable device, and the outputted user detectable stimuli are visual prompts indicating locations of objects with the area surrounding the determined location of the user wearable device.

13. The system of claim 9 , wherein the user wearable device includes:
a tactile device that outputs tactile stimuli indicative of a navigation path around objects in an area surrounding the determined location of the user wearable device.

14. The system of claim 9 , wherein the user wearable device comprises an eyewear frame having a structure to configure the user interface device to be worn as goggles or eyeglasses.

15. A method, comprising:
collecting, by a lighting device sensor, mapping data of an area of a premises to be illuminated by the lighting device;
identifying from the collected mapping data objects in the area of the premises;
storing the collected mapping data and an indication of the identified objects in memory associated with a lighting device;
in response to a request from a u

Assignees

• Abl Ip Holding Llc

Inventors

• Lydecker Stephen H
• Ramer David P
• Rains Jr Jack C
• Aggarwal Januk

Classifications

• G02B2027/014

  comprising information/image processing systems · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G09B21/008 Primary

  using visual presentation of the information for the partially sighted · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title