3D mapping of internet of things devices

Patent metadata

Field	Value
Publication number	US-10091015-B2
Application number	US-201414572301-A
Country	US
Kind code	B2
Filing date	Dec 16, 2014
Priority date	Dec 16, 2014
Publication date	Oct 2, 2018
Grant date	Oct 2, 2018

Abstract

A method for mapping and controlling network-enabled appliances with a control device may comprise collecting spatial information in three dimensions as at least one sensor of the control device is moved within an environment, receiving registration signals from network-enabled appliances in the environment, and associating each network-enabled appliance with a respective 3D position in the environment based on the spatial information. The method may include generating a digital map of the environment from the spatial information and placing a representation of each network-enabled appliance on the digital map based on the respective 3D position. The method may also include generating a spatially aware rule set that applies to one or more rooms within the environment. The method may also include detecting a trigger event, and in response to the trigger event, commanding the network-enabled appliances according to the spatially aware rule set.

First claim

The invention claimed is:

1. A method for mapping and controlling network-enabled appliances with a control device, the method comprising:
collecting, by at least one sensor of the control device, spatial information in three dimensions as the at least one sensor of the control device is moved within an environment containing network-enabled appliances, the at least one sensor including at least one camera;
recognizing at least one of the network-enabled appliances within the environment with machine vision techniques including applying text recognition algorithms to read physical tags to identify and recognize network enabled appliances in image data captured by the at least one camera;
receiving registration signals from at least one of the network-enabled appliances in the environment;
associating each network-enabled appliance with a respective three-dimensional (3D) position in the environment based on the spatial information;
registering each network-enabled appliance with the control device to create a registration log;
generating a digital map of the environment from the spatial information;
identifying rooms within the environment;
placing a representation of each network-enabled appliance on the digital map based on the respective 3D position;
generating a spatially aware rule set that applies to one or more of the rooms within the environment based upon the spatial information and the registration log;
detecting a trigger event; and
in response to the trigger event, commanding the network-enabled appliances within the room or rooms according to the spatially aware rule set.

2. The method of claim 1 , wherein identifying the rooms comprises using object recognition to compare the spatial information to a typical room arrangement.

3. The method of claim 1 ,
wherein identifying the rooms comprises receiving user input in the form of a spoken room name, a typed room name, or a manually selected room name, while collecting the spatial information.

4. The method of claim 1 , wherein the at least one camera comprises at least one depth camera or color camera.

5. The method of claim 1 , wherein the registration signals are infrared signals.

6. The method of claim 1 , wherein the registration signals are visible light signals.

7. The method of claim 1 , further comprising, before the registration signals are received, soliciting the registration signals from the network-enabled appliances via near field communication, wireless Internet, or a personal area network.

8. The method of claim 1 , wherein the at least one sensor includes an additional sensor that comprises an array microphone and the registration signals include audio signals.

9. The method of claim 1 , further comprising reading visible tags to recognize network-enabled appliances.

10. An automation system comprising a control device for mapping and controlling network-enabled appliances, the control device comprising:
at least one sensor configured to collect spatial information in three dimensions as the at least one sensor is moved within an environment containing network-enabled appliances, wherein the at least one sensor includes at least one camera;
a communication unit, including at least the one sensor or at least one second sensor, and further including a transmitter, the communication unit being configured to receive registration signals from at least one of the network-enabled appliances in the environment; and
a processor configured to execute an automation program to:
recognize at least one of the network-enabled appliances within the environment with machine vision techniques, the machine vision techniques including applying text recognition algorithms to read physical tags to identify and recognize network enabled appliances in image data captured by the at least one camera;
associate each network-enabled appliance with a respective three-dimensional (3D) position in the environment based on the spatial information;
register each network-enabled appliance with the control device to create a registration log;
generate a digital map of the environment from the spatial information; identify rooms within the environment;
place a representation of each network-enabled appliance on the digital map based on the respective 3D position;
generate a spatially aware rule set that applies to one or more of the rooms within the environment based upon the spatial information and the registration log; and
in response to a trigger event, command the network-enabled appliances within the room or rooms via the communication unit according to the spatially aware rule set.

11. The system of claim 10 , wherein the automation program identifies the rooms by using object recognition to compare the spatial information to a typical room arrangement.

12. The system of claim 10 , wherein the automation program identifies the rooms by receiving user input in the form of a spoken room name, a typed room name, or a manually selected room name, while the spatial information is collected.

13. The system of claim 10 , wherein the at least one camera comprises at least one depth camera or color camera.

14. The system of claim 10 , wherein the registration signals are infrared signals.

15. The system of claim 10 , wherein the registration signals are visible light signals.

16. The system of claim 10 , wherein the communication unit is configured to, before the registration signals are received, solicit the registration signals from the network-enabled appliances via near field communication, wireless Internet, or a personal area network.

17. The system of claim 10 , wherein the at least one sensor includes an additional sensor that comprises an array microphone and the registration signals include audio signals.

18. The system of claim 10 , wherein the automation program is configured to read visible tags to recognize the network-enabled appliances.

19. An automation system comprising a control device for mapping and controlling network-enabled appliances, the control device comprising:
at least one sensor configured to collect spatial information in three dimensions as the at least one sensor is moved within an environment containing network-enabled appliances, wherein the at least one sensor includes at least one camera that captures image data;
a communication unit, including at least the one sensor or at least one second sensor, and further including a transmitter, the communication unit being configured to receive registration signals from at least one of the network-enabled appliances in the environment; and
a processor configured to execute an automation program to:
use the image data captured by the at least one camera and/or an additional camera of the control device to recognize at least one of the network-enabled appliances with machine vision techniques, the machine vision techniques including text recognition algorithms to read physical tags to identify and recognize network enabled appliances in the image data captured by the at least one camera;
associate each network-enabled appliance with a respective three-dimensional (3D) position in the environment based on the spatial information;
register each network-enabled appliance with the control device to create a registration log;
generate a digital map of the environment from the spatial information; identify rooms within the environment;
place a representation of each network-enabled appliance on the digital map based on the respective 3D position;
generate a spatially aware rule set that applies

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Kennedy Kevin Alan

Classifications

• H04W4/70

  Services for machine-to-machine communication [M2M] or machine type communication [MTC] · CPC title

• H04L12/2814 Primary

  Exchanging control software or macros for controlling appliance services in a home automation network (arrangements for maintenance or administration involving configuration of the network and network elements H04L41/08) · CPC title

• H04W4/80

  Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication · CPC title

• H04W4/33 Primary

  for indoor environments, e.g. buildings · CPC title

• H04L41/0806

  for initial configuration or provisioning, e.g. plug-and-play · CPC title