Video integration with home assistant

Patent metadata

Field	Value
Publication number	US-11736760-B2
Application number	US-202217656788-A
Country	US
Kind code	B2
Filing date	Mar 28, 2022
Priority date	Jul 19, 2017
Publication date	Aug 22, 2023
Grant date	Aug 22, 2023

Abstract

Methods, devices, systems, and means for video integration with a home assistant device are described herein. The home assistant device interacts with a person in a video stream by capturing, using a network-enabled outdoor video camera, a video stream of an outdoor location of a premises at which the person is present and analyzing the person appearing in the captured video stream to determine an identity of the person. Based on determining the identity of the person, the home assistant device announces a presence of the person that is outdoors and outputs instructions to the person.

First claim

What is claimed is:

1. A method for using a home assistant device to interact with a person in a video stream, the method comprising:
capturing, using a network-enabled outdoor video camera, a video stream of an outdoor location of a premises at which the person is present;
analyzing the person appearing in the captured video stream to determine an identity of the person;
based on determining the identity of the person, announcing, by the home assistant device, a presence of the person that is outdoors; and
outputting instructions to the person.

2. The method of claim 1 , wherein the analyzing the person appearing in the captured video stream to determine the identity of the person comprises:
identifying the person in the captured video stream.

3. The method of claim 2 , further comprising:
determining that no one is within premises; and
based on determining that no one is within premises, providing a first voice prompt to the identified person.

4. The method of claim 3 , further comprising:
in response to the providing the first voice prompt, recoding a voice message from the identified person; and
sending the voice message and the identity of the person to one or more authorized users of the home assistant device.

5. The method of claim 2 , wherein the outputting the instructions to the identified person comprises:
outputting instructions based on the identity of the identified person.

6. The method of claim 1 , wherein the analyzing the person appearing in the captured video stream to determine the identity of the person comprises:
determining that the person in the captured video stream is unidentified.

7. The method of claim 6 , further comprising:
determining that no one is within premises; and
based on determining that no one is within premises, providing a second voice prompt to the unidentified person.

8. The method of claim 7 , further comprising:
in response to the providing the second voice prompt, recoding a voice message from the unidentified person; and
sending the voice message and an indication of the unidentified person to one or more authorized users of the home assistant device.

9. The method of claim 1 , further comprising:
tracking a location of any item left by the person.

10. The method of claim 9 , wherein the tracking the location of the item comprises:
using video from the network-enabled outdoor video camera to monitor any activity occurring near the item.

11. The method of claim 1 , wherein the outputting the instructions to the person comprises:
outputting the instructions by the network-enabled outdoor video camera; or
outputting the instructions by a smart doorbell.

12. The method of claim 1 , wherein the analyzing the person appearing in the captured video stream to determine the identity of the person comprises:
performing facial recognition to determine if the person matches an authorized user based on stored user data, the stored user data including a name and facial features of the person.

13. The method of claim 1 , wherein the announcing the presence of the person that is outdoors comprises:
announcing the presence of the person based on detecting the person in the video stream; or
announcing the presence of the person based on the person ringing a doorbell.

14. A system for interacting with a person in a video stream, the system comprising:
a network-enabled outdoor video camera that captures a video stream of an outdoor location of a premises and streams the video stream to a cloud-based home assistant integration platform;
the cloud-based home assistant integration platform configured to:
analyze the person appearing in the captured video stream to determine an identity of the person;
a home assistant device comprising a microphone and wireless network interface, the home assistant device configured to:
based on the determination of the identity of the person, announce a presence of the person that is outdoors; and
the network-enabled outdoor video camera configured to:
output instructions to the person.

15. The system of claim 14 , the cloud-based home assistant integration platform configured to:
identify the person in the captured video stream;
determine that no one is within premises; and
generate a first voice prompt; and
the network-enabled outdoor video camera configured to:
output a first voice prompt to the identified person.

16. The system of claim 15 ,
the network-enabled outdoor video camera configured to:
in response to the output of the first voice prompt, record a voice message from the identified person; and
send the voice message and the identity of the person to one or more authorized users of the home assistant device via the cloud-based home assistant integration platform.

17. The system of claim 16 ,
the cloud-based home assistant integration platform configured to:
based on the identity of the person, generate instruction for the identified person; and
the network-enabled outdoor video camera configured to:
output the instructions based on the identity of the identified person.

18. The system of claim 14 , the cloud-based home assistant integration platform configured to:
track a location of any item left by the person.

19. The system of claim 18 , wherein the location of the presence of the item configures the cloud-based home assistant integration platform to:
use video from the network-enabled outdoor video camera to monitor any activity occurring near the item.

20. The system of claim 14 , wherein the network-enabled outdoor video camera is a doorbell.

Assignees

• Google Llc

Inventors

• Yuan Jessica
• Stewart James
• Nongpiur Rajeev
• Lister Patrick
• Ng Chi Yeung Jonathan

Classifications

• H04N21/43615 Primary

  Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals (home Audio Video Interoperability [HAVI] data switching networks H04L12/2805) · CPC title

• G06Q10/109

  Time management, e.g. calendars, reminders, meetings or time accounting · CPC title

• G10L17/22

  Interactive procedures; Man-machine interfaces · CPC title

• H04L12/281

  indicating a format for calling an appliance service function in a home automation network (for remote control or remote monitoring of applications H04L67/025) · CPC title

• H04L12/282

  based on user interaction within the home (receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204) · CPC title