Smart-home device placement and installation using augmented-reality visualizations

Patent metadata

Field	Value
Publication number	US-10839608-B2
Application number	US-201916442879-A
Country	US
Kind code	B2
Filing date	Jun 17, 2019
Priority date	Jun 15, 2018
Publication date	Nov 17, 2020
Grant date	Nov 17, 2020

Abstract

A method for optimizing the placement of smart-home devices may include receiving, by a mobile computing device, a location for a smart-home device, where the mobile computing device comprises a display and a camera; rendering a view of a virtual object that represents a field-of-view of the smart-home device, where the view of the virtual object is rendered based on a position corresponding to a position of the mobile computing device; and displaying, by the mobile computing device, the view of a virtual object that represents a field-of-view of the smart-home device on the display of the mobile computing device.

First claim

What is claimed is:

1. A method for optimizing the placement of smart-home devices, the method comprising:
capturing, by a camera of a mobile computing device, a view of a smart-home device, wherein the mobile computing device comprises a display;
determining the location for the smart-home device relative to a location of the mobile computing device;
rendering, by the mobile computing device, a view of a virtual object that represents a field-of-view of the smart-home device, wherein the view of the virtual object is rendered based on a position corresponding to a position of the mobile computing device; and
displaying, by the mobile computing device, the view of a virtual object that represents a field-of-view of the smart-home device on the display of the mobile computing device.

2. The method of claim 1 , wherein the virtual object comprises a cone projecting away from the location for the smart-home device.

3. The method of claim 1 , wherein the virtual object is rendered as part of a virtual 3-D scene from a perspective of a virtual camera that corresponds to the position of the mobile computing device in the 3-D virtual scene.

4. The method of claim 1 , wherein rendering the view of the virtual object comprises clipping a portion of the view of the virtual object based on an obstruction within the field-of-view of the smart-home device.

5. The method of claim 1 , wherein the smart-home device comprises a motion sensor, and the field-of-view of the smart-home device comprises an area in which the motion sensor can detect motion.

6. The method of claim 1 , further comprising:
receiving a view of the smart-home device during installation;
generating a view of instructions for installing the smart-home device; and
displaying the instructions for installing the smart-home device on the display of the mobile computing device layered over the view of the smart-home device during installation.

7. The method of claim 6 , further comprising:
connecting to a second computing device; and
receiving live installation instructions from a user through the second computing device.

8. A system comprising:
one or more processors; and
one or more memory devices comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving a first position of a mobile computing device, wherein the mobile computing device comprises a display and a camera;
receiving a second position of the mobile computing device relative to the first position;
using the first position as the location for a smart-home device;
rendering, by the mobile computing device, a view of a virtual object that represents a field-of-view of the smart-home device, wherein the view of the virtual object is rendered based on a position corresponding to a position of the mobile computing device; and
displaying, by the mobile computing device, the view of a virtual object that represents a field-of-view of the smart-home device on the display of the mobile computing device.

9. The system of claim 8 , wherein the smart-home device comprises a magnetic sensor, and the field-of-view of the smart-home device comprises an area in which the magnetic sensor can detect a magnet.

10. The system of claim 8 , wherein the smart-home device comprises a radio, and the field-of-view of the smart-home device comprises a range of the radio.

11. The system of claim 8 , wherein the smart-home device comprises a motion sensor, and the field-of-view of the smart-home device comprises an area in which the motion sensor can detect motion.

12. The system of claim 8 , wherein the smart-home device comprises a camera, and the field-of-view of the smart-home device comprises an area that can be captured by the camera.

13. The system of claim 8 , wherein the view of the virtual object is rendered in a first color when a user and/or the mobile computing device is not in the field-of-view of the smart-home device, and wherein the view of the virtual object is rendered in a second color when the user and/or mobile computing device is in the field-of-view of the smart-home device.

14. The system of claim 8 , wherein the view of the virtual object is rendered in a first color for a first portion of the field-of-view and is rendered in a second color for a second portion of the field-of-view.

15. A non-transitory, computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a finger-tap input on a display of a mobile computing device, wherein the mobile computing device comprises a camera;
determining a location for a smart-home device based on the view of the camera of the mobile computing device and the finger-tap input;
rendering, by the mobile computing device, a view of a virtual object that represents a field-of-view of the smart-home device, wherein the view of the virtual object is rendered based on a position corresponding to a position of the mobile computing device; and
displaying, by the mobile computing device, the view of a virtual object that represents a field-of-view of the smart-home device on the display of the mobile computing device.

16. The non-transitory, computer-readable medium of claim 15 , wherein a shape of the virtual object changes according to sensitivity settings on the smart-home device.

17. The non-transitory, computer-readable medium of claim 15 , comprising additional instructions that, when executed by the one or more processors, cause the one or more processors to perform additional operations comprising:
identifying a source in the smart-home environment that interferes with the field of view of the smart-home device;
rendering a view of a second virtual object that represents an area of interference generated by the source; and
displaying, by the mobile computing device, the view of the second virtual object with the view of the virtual object.

18. The non-transitory, computer-readable medium of claim 15 , comprising additional instructions that, when executed by the one or more processors, cause the one or more processors to perform additional operations comprising:
receiving information descriptive of at least a portion of a smart-home environment;
determining locations in the smart-home environment where the smart-home device should or should not be installed;
rendering a view of a second virtual object that represents the locations in the smart-home environment where the smart-home device should or should not be installed; and
displaying, by the mobile computing device, the view of the second virtual object with the view of the virtual object.

19. The non-transitory, computer-readable medium of claim 15 , comprising additional instructions that, when executed by the one or more processors, cause the one or more processors to perform additional operations comprising:
receiving a view of the smart-home device during installation;
generating a view of instructions for installing the smart-home device; and
displaying the instructions for installing the smart-home device on the display of the mobile computing device layered over the view of the smart-home device during installation.

20. The non-transitory, computer-readable medium of claim 19 , comprising additional instructions that, when executed by the one or more processors, cause the one or more processors to perform additional operations comprising:
connecting to a second computing

Assignees

• Google Llc

Inventors

• Mittleman Adam
• Chamberlain Jason
• Grillo Jacobi
• Biran Daniel
• Kraz Mark
• Chanen Lauren
• Foran Daniel
• Fichou David
• Dong William
• Nguyen Bao-Tram Phan
• Silverstein Brian
• Modi Yash
• Finlayson Alex
• Shin Dongeek

Classifications

• G06T2210/04

  Architectural design, interior design · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/0488

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title