Spatial remote control

Patent metadata

Field	Value
Publication number	US-10475332-B2
Application number	US-201715793232-A
Country	US
Kind code	B2
Filing date	Oct 25, 2017
Priority date	Oct 25, 2017
Publication date	Nov 12, 2019
Grant date	Nov 12, 2019

Abstract

The disclosure provides methods and systems for spatially remote controlling a network controllable device in a physical user environment. An exemplary method may include tracking, by a processor, a position and orientation of a remote control device based on sensor data. The sensors may be one or more of a gyroscope, accelerometer, and a magnetometer. Based on the tracking, a ray extending from the remote control device in a direction the remote control device is pointing is determined. The determined ray is compared to a predefined map, the predefined map including a spatial location associated with each of one or more network controllable devices. The method may include, based on the comparing, selecting a network controllable device of the one or more network controllable devices corresponding to the determined ray. In response to receiving a user input, a control signal is then transmitted to the selected network controllable device.

First claim

What is claimed is:

1. A method for spatially remote controlling a network controllable device in a physical user environment, the method comprising:
tracking, by a processor, a position and orientation of a remote control device based on sensor data;
based on the tracking, determining a geometric ray extending from the remote control device in a direction the remote control device is pointing;
comparing, by the processor, the determined geometric ray to a predefined map, the predefined map including a spatial location associated with each of one or more network controllable devices, constructing of the predefined map including:
determining, based on the position and orientation of the remote control device, a plurality of boundaries of the physical user environment; and
determining, based on the position and orientation of the remote control device in response to a user pointing the remote control device at each of the one or more network controllable devices in turn, the spatial location associated with each of the one or more network controllable devices;

based on the comparing, selecting one of the one or more network controllable devices corresponding to the determined geometric ray;
receiving, by the processor, a user input; and
in response to receiving the user input, transmitting a control signal to the selected one of the one or more network controllable devices.

2. The method as recited in claim 1 , wherein determining a plurality of boundaries includes:
determining a floor boundary; and
determining a wall boundary based on an intersection of the geometric ray and the floor boundary.

3. The method as recited in claim 1 , further comprising receiving a list of the one or more network controllable devices coupled to a network, wherein the list of the one or more network controllable devices includes a device identifier and a type of device for each of the one or more network controllable devices.

4. The method as recited in claim 1 , wherein the constructing the predefined map further includes activating each of the one or more network controllable devices.

5. The method as recited in claim 1 , wherein determining the spatial location associated with each of the one or more network controllable devices includes determining a point along the geometric ray corresponding to a physical location of the each of the network controllable devices.

6. The method as recited in claim 1 , further comprising generating a graphical user interface to display the predefined map as the predefined map is being constructed.

7. The method as recited in claim 6 , wherein the graphical user interface includes at least one of the plurality of boundaries.

8. The method as recited in claim 6 , wherein the graphical user interface includes currently calculated positions of the geometric ray and the spatial location associated with each of the one or more network controllable devices in real time.

9. The method as recited in claim 1 , wherein the comparing further includes calculating a point-line distance between the geometric ray and each spatial location.

10. The method as recited in claim 9 , wherein the selecting further includes determining which of the one or more network controllable devices is disposed closest to the geometric ray, wherein the selected one of the one or more network controllable devices is the closest determined network controllable device.

11. The method as recited in claim 1 , wherein the comparing further includes calculating a volume centered about the geometric ray and determining whether each of the spatial locations stored in the predefined map is disposed within the volume.

12. The method as recited in claim 11 , wherein the selecting further includes determining which of the one or more network controllable devices disposed within the volume is disposed closest to the remote control device, wherein the selected one of the one or more network controllable devices is the closest determined network controllable device.

13. The method as recited in claim 1 , wherein the received user input includes one or more of a button actuation, voice control, movement of the remote control device, and a gesture.

14. The method as recited in claim 13 , wherein the remote control device comprises a smart phone that includes a number of executable applications, and wherein, in response to receiving the user input, transmitting a control signal to the selected network controllable device comprises:
causing launching of an application on the smart phone that corresponds to the selected network controllable device.

15. The method of claim 1 , wherein the sensor data is acquired by one or more camera systems and at least one of a gyroscope of the remote control device, an accelerometer of the remote control device, or a magnetometer of the remote control device.

16. The method of claim 1 , further comprising generating a graphical user interface to display the predefined map as the predefined map is being constructed,
wherein the graphical user interface includes at least one of the plurality of boundaries, and
wherein the graphical user interface includes currently calculated positions of the geometric ray and the spatial location associated with each of the one or more network controllable devices in real time.

17. A system for spatially remote controlling a network controllable device in a physical user environment, the system comprising:
a remote control device;
a network coupled to one or more network controllable devices;
at least one processor; and
a memory storing processor-executable instructions, wherein the at least one processor is configured to implement the following operations upon executing the processor-executable instructions:
tracking, by the at least one processor, a position and orientation of the remote control device based on sensor data acquired by one or more camera systems, a gyroscope, an accelerometer, or a magnetometer of the remote control device;
based on the tracking, determining a geometric ray extending from the remote control device in a direction the remote control device is pointing;
comparing, by the processor, the determined geometric ray to a predefined map, the predefined map including a spatial location associated with each of the one or more network controllable devices, constructing of the predefined map including:
determining, based on the position and orientation of the remote control device, a plurality of boundaries of the physical user environment;
receiving a list of the one or more network controllable devices coupled to a network; and
determining, based on the position and orientation of the remote control device in response to a user pointing the remote control device at each of the one or more network controllable devices in turn, the spatial location associated with each of the one or more network controllable devices;

based on the comparing, selecting one of the one or more network controllable devices corresponding to the determined geometric ray;
receiving, by the processor, a user input; and
in response to receiving the user input, transmitting a control signal to the selected one of the one or more network controllable devices.

18. The system of claim 17 , wherein the one or more network controllable devices includes one or more of a smart light, a speaker, a television, a thermostat, a camera, a meter, an irrigation system, and a door lock.

19. The system of claim 17 , wherein the predefined map includes, for each of the one or more network controllable d

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Marks Richard
• Eads Joshua Michael

Classifications

• G08C17/02 Primary

  using a radio link · CPC title

• H04B1/20

  for coupling gramophone pick-up, recorder output, or microphone to receiver · CPC title

• G08C2201/42

  Transmitting or receiving remote control signals via a network · CPC title

• G08C2201/92

  Universal remote control · CPC title

• G08C2201/91

  Remote control based on location and proximity · CPC title