Systems and methods of presenting appropriate actions for responding to a visitor to a smart home environment

Patent metadata

Field	Value
Publication number	US-11134227-B2
Application number	US-201715809900-A
Country	US
Kind code	B2
Filing date	Nov 10, 2017
Priority date	Sep 20, 2017
Publication date	Sep 28, 2021
Grant date	Sep 28, 2021

Abstract

A method of presenting appropriate actions for responding to a visitor to a smart home environment via an electronic greeting system of the smart home environment, including detecting a visitor of the smart home environment; obtaining context information from the smart home environment regarding the visitor; based on the context information, identifying a plurality of appropriate actions available to a user of a client device for interacting with the visitor via the electronic greeting system; and causing the identified actions to be presented to the user of the client device.

First claim

What is claimed is:

1. A method comprising:
at an electronic greeting system of a smart home environment, wherein the electronic greeting system includes one or more processors and a memory:
detecting a visitor of the smart home environment;
obtaining context information from the smart home environment, wherein the context information indicates at least one of an action performed by the visitor or a characteristic of the visitor, and wherein the characteristic of the visitor indicates at least one of a status of the visitor or an identity of the visitor;
selecting, based on the at least one of the action performed by the visitor or the characteristic of the visitor indicated by the context information, only a first subset of actions from a plurality of context-based, user-selectable actions, wherein the plurality of context-based, user-selectable actions includes at least the first subset of actions and a second subset of actions different than the first subset of actions, and wherein the first subset of actions includes multiple actions;
outputting, for display at a client device, a graphical user interface to interact with a user of the client device; and
outputting, for display at the client device and within the graphical user interface, the first subset of actions, thereby allowing the user to select at least one of the first subset of actions in response to the electronic greeting system detecting the visitor.

2. The method of claim 1 , wherein detecting the visitor comprises obtaining and analyzing presence information indicative of the visitor approaching or in proximity to an entryway of the smart home environment.

3. The method of claim 2 , wherein the presence information is derived from motion data of one or more motion sensors of the smart home environment.

4. The method of claim 2 ,
wherein the presence information is derived from video data obtained from a camera having a field of view corresponding to the entryway, and
wherein deriving the presence information from the video data comprises analyzing a plurality of image frames to determine whether a difference in position of an object in the plurality of image frames exceeds a threshold.

5. The method of claim 2 ,
wherein the presence information is derived from audio data obtained from an audio sensor in proximity to the entryway, and
wherein deriving the presence information from the audio data comprises identifying at least one of a door knock, a doorbell sound, a footstep, or a verbal announcement.

6. The method of claim 1 , wherein the context information further indicates one or more of:
a facial recognition analysis result;
one or more behavioral characteristics of the visitor;
one or more physical characteristics of the visitor;
one or more clothing or accessory characteristics of the visitor;
a time of day during which the visitor approaches an entryway of the smart home environment;
a day of the week during which the visitor approaches the entryway;
audio data from the smart home environment;
proximity in time to a prescheduled event;
proximity in time to a prescheduled status of the smart home environment;
a known or an unknown status of the user; or
an expected or an unexpected status of the user.

7. The method of claim 1 , wherein one or more actions of the plurality of context-based, user-selectable actions comprises conveying a communication to the visitor, the communication including:
a communication conveying a status of the user;
a communication conveying a request for the visitor to perform an action;
a communication conveying a request for the visitor to leave a message;
a preprogrammed customized communication;
a user-composed text message for conversion to an audio message;
an audio message conveyed by a synthesized voice; or
a visual message that is output for display at the client device.

8. The method of claim 1 , wherein one or more actions of the plurality of context-based, user-selectable actions comprises initiating a security action including:
activating a light or adjusting a lighting level of the smart home environment;
locking or unlocking a door of the smart home environment;
activating an alarm or adjusting an alarm sensitivity of the smart home environment;
activating a sprinkler system of the smart home environment;
activating a simulated dog bark;
activating a security system or adjusting a security status of the smart home environment;
transmitting a notification or an alert to public or private law enforcement personnel;
transmitting a notification or an alert to a preselected contact of the user; or
recording an image or video of the visitor.

9. An electronic greeting system of a smart home environment, the system comprising one or more processors and a memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors to:
detect a visitor of the smart home environment;
obtain context information from the smart home environment, wherein the context information indicates at least one of an action performed by the visitor or a characteristic of the visitor, and wherein the characteristic of the visitor indicates at least one of a status of the visitor or an identity of the visitor;
select, based on the at least one of the action performed by the visitor or the characteristic of the visitor indicated by the context information, only the first subset of actions from a plurality of context-based, user-selectable actions, wherein the plurality of context-based, user-selectable actions includes at least the first subset of actions and a second subset of actions different than the first subset of actions, and wherein the first subset of actions includes multiple actions;
output, for display at a client device, a graphical user interface to interact with a user of the client device; and
output, for display at the client device and within the graphical user interface, the first subset of actions, thereby allowing the user to select at least one of the first subset of actions in response to the electronic greeting system detecting the visitor.

10. The electronic greeting system of claim 9 , wherein the plurality of context-based, user-selectable actions includes an action specific to a location of the smart home environment, an action specific to a building type of the smart home environment, or an action specific to a user disposition.

11. The electronic greeting system of claim 9 , wherein the first subset of actions includes conveying a communication to the visitor, and wherein the second subset of actions initiating a security action.

12. The electronic greeting system of claim 9 , wherein the one or more programs are further configured to be executed by the one or more processors to:
rank, based on the context information, one or more actions in the first subset of actions; and
order the one or more actions,
wherein a number of the one or more actions to be output at the client device is based on an amount of display space available in a quick action area of the graphical user interface of the client device.

13. The electronic greeting system of claim 9 , wherein the context information comprises a detected announcement event associated with the visitor, and wherein the detected announcement event includes a doorbell button press, a door knock, or a verbal announcement.

14. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by a computing system, cause the computing system to:
detect a visitor of the s

Assignees

• Google Llc

Inventors

• Goulden Jason Evans
• Aravamudhan Rengarajan
• Jeong Hae Rim
• Dixon Michael
• Stewart James Edward
• Shafi Sayed Yusef
• Mysore Sahana
• Yang Seungho
• Lien Yu-An
• Burns Christopher Charles
• Nongpiur Rajeev
• Boyd Jeffrey

Classifications

• G08B3/10 Primary

  using electric transmission; using electromagnetic transmission · CPC title

• H04N7/186 Primary

  Video door telephones · CPC title

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G08B13/19684

  Portable terminal, e.g. mobile phone, used for viewing video remotely · CPC title

• H04M11/025

  Door telephones (adapted for television H04N7/186) · CPC title