Two-way communication interface for vision-based monitoring system

Patent metadata

Field	Value
Publication number	US-10735916-B2
Application number	US-201715628236-A
Country	US
Kind code	B2
Filing date	Jun 20, 2017
Priority date	Jun 20, 2017
Publication date	Aug 4, 2020
Grant date	Aug 4, 2020

Abstract

A method for enabling communicating between users, using a vision-based monitoring system. The method includes monitoring an environment for events, by the vision-based monitoring system and making a determination that a detected event involves a local person, in the monitored environment. The method further includes, based on the determination, identifying at least one remote person to be contacted, identifying a communication channel to the at least one remote person, and providing a notification to the at least one remote person, via the identified communication channel.

First claim

What is claimed is:

1. A method for enabling communicating between users, the method comprising:
storing, by a monitoring device, a plurality of action definitions corresponding to visual gestures;
monitoring, by a camera of the monitoring device, an environment for events;
determining, by the monitoring device, that a detected event of the events involves a local person in the monitored environment;
identifying, by the monitoring device, a profile associated with the local person;
identifying, by the monitoring device, a communication action associated with the profile;
detecting, by the monitoring device, a visual gesture performed by the local person, wherein the visual gesture is captured by the camera;
determining, by the monitoring device, that the detected visual gesture corresponds to at least one action definition of the stored plurality of action definitions;
determining, by the monitoring device, whether the at least one stored action definition corresponds to the communication action;
in accordance with a determination that the communication action corresponds to the at least one stored action definition, recording, by the camera of the monitoring device and based at least in part on the detected visual gesture, a message corresponding to the local person;
storing, by the monitoring device, the recorded message;
based at least in part on the determination, identifying, by the monitoring device, at least one remote person to be contacted;
identifying, by the monitoring device, a communication channel to the at least one remote person; and
providing, by the monitoring device, the stored message to the at least one remote person via the identified communication channel.

2. The method of claim 1 , further comprising:
identifying the local person, and wherein the provided message comprises an identity of the local person.

3. The method of claim 1 , wherein the at least one remote person to be contacted comprises a person with a profile associated with a site.

4. The method of claim 1 , wherein the visual gesture is recognized as an indication of the local person's intention to communicate.

5. The method of claim 1 , wherein the at least one remote person to be contacted is a person specified by the local person, in a communication request, received by the monitoring device and from the local person.

6. The method of claim 5 , wherein the at least one remote person to be contacted is a person specified by a name or a relationship, in the communication request, received by the monitoring device and from the local person.

7. The method of claim 1 , further comprising, after providing the message to the at least one remote person:
receiving a request, from one of the at least one remote person, to communicate with the local person; and
establishing a communication connection between the local person and the remote person.

8. The method of claim 7 , wherein the communication connection between the local person and the remote person comprises a two-way audio communication channel.

9. The method of claim 8 , wherein the communication connection between the local person and the remote person further comprises a one-way video channel from the local user to the remote user.

10. A monitoring device, comprising:
a memory for storing computer-readable instructions
a camera; and
one or more processors for executing the computer-readable instructions to at least:
store a plurality of action definitions corresponding to visual gestures;
monitor, via the camera, an environment for events;
determine that a detected event of the events involves a local person in the monitored environment;
identify a profile associated with the local person;
identify a communication action associated with the profile;
detecting a visual gesture performed by the local person, wherein the visual gesture is captured by the camera;
determine that the detected visual gesture corresponds to at least one action definition of the stored plurality of action definitions;
determine whether the at least one stored action definition corresponds to the communication action;
in accordance with a determination that the communication action corresponds to the at least one stored action definition, record, via the camera of the monitoring device and based at least in part on the detected visual gesture, a message corresponding to the local person;
store the recorded message;
identify a remote person to which the stored message is to be sent; and
provide the stored message to the remote person.

11. The monitoring device of claim 10 , further comprising, prior to providing the stored message to the remote person:
notifying the identified remote person of the availability of the stored message; and
receiving a request, from the remote person, to obtain the stored message.

12. The monitoring device of claim 10 , further comprising identifying the local person, by querying profiles of persons specific to the site where the local person was detected, in a monitoring system database.

13. The monitoring device of claim 12 wherein identifying the remote person to which the message is to be sent comprises at least one of obtaining the identity of the remote person from the local person or obtaining the identity of the remote person from the profile of the local person.

14. A non-transitory computer readable medium comprising instructions that enable a monitoring device to perform operations comprising:
storing, by a monitoring device, a plurality of action definitions corresponding to visual gestures;
monitoring, by a camera of the monitoring device, an environment for events;
determining that a detected event of the events involves a local person in the monitored environment;
identifying, by the monitoring device, a profile associated with the local person;
identifying, by the monitoring device, a communication action associated with the profile;
detecting a visual gesture performed by the local person, wherein the visual gesture is captured by the camera;
determining, by the monitoring device, that the detected visual gesture corresponds to at least one action definition of the stored plurality of action definitions;
determining, by the monitoring device, whether the at least one stored action definition corresponds to the communication action;
in accordance with a determination that the communication action corresponds to the at least one stored action definition, recording, by the camera of the monitoring device and based at least in part on the detected visual gesture, a message corresponding to the local person;
storing the recorded message;
identifying a remote person to be contacted; and
providing the recorded message to the remote person.

15. The non-transitory computer readable medium of claim 14 , wherein the visual gesture is recognized as an indication of the local person's intention to communicate.

16. The non-transitory computer readable medium of claim 14 , wherein the at least one remote person to be contacted is a person specified by the local person, in a communication request, received by the monitoring device and from the local person.

17. The non-transitory computer readable medium of claim 16 , wherein the at least one remote person to be contacted is a person specified by a name, in the communication request, received by the monitoring device and from the local person.

18. The non-transitory computer readable medium of claim 16 , wherein the at least one remote person to be contacted is a person specified by a relationship,

Assignees

• Apple Inc

Inventors

• Gilmartin Jessica
• Jin Jonghoon
• Scherb Tomas Alberto Meraz
• Kryze Tiffany
• Teichman Alexander William

Classifications

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06V40/20

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06V20/44

  Event detection · CPC title

• G06V20/40

  in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) · CPC title