Synchronizing computer vision interactions with a computer kiosk

Patent metadata

Field	Value
Publication number	US-11748465-B2
Application number	US-202217666360-A
Country	US
Kind code	B2
Filing date	Feb 7, 2022
Priority date	Jun 21, 2017
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

A system and method for linking observed human activity on video to a user account of a preferred embodiment that includes: through a computer vision monitoring system, detecting and tracking a human as a computer vision modeled person within an environment, the computer vision monitoring system being part of a computing platform managing a user interaction experience; through at least one associative mechanism, establishing an association between the computer vision modeled person and at least one associative element and thereby associating the computer vision modeled person and a user-record linked through the associative element; and directing the user interaction experience at least in part based on the combination of the modeled person and at least the user-record.

First claim

We claim:

1. A method comprising:
through a computer vision monitoring system, detecting and tracking a computer vision modeled person within an environment;
establishing an association between the computer vision modeled person and a user-record through use of a shared computer kiosk in the environment, which comprises:
receiving registration of the user-record at the shared computer kiosk, and
during a time of registration of the user record, detecting the shared computer kiosk being coincident in proximity to the computer vision modeled person and thereby associating the computer vision modeled person and the user-record; and

augmenting a user interaction experience based in part on the combination of the computer vision modeled person and the user-record.

2. The method of claim 1 , further comprising establishing a second association between the computer vision modeled person and a personal computing device, which comprises detecting the personal computing device instance with coincident proximity to the computer vision modeled person and associating the computer vision modeled person, the personal computing device, and the user-record.

3. The method of claim 1 , establishing a second association between a personal computing device and the computer vision modeled person which comprises authenticating a user-account in the personal computing device, wherein the user-account is associated with the user-record.

4. The method of claim 1 , wherein augmenting the user interaction experience comprises communicating a modeled state of a checkout list of the computer vision modeled person to an application instance of a personal computing device when the application instance is associated with the user-record.

5. The method of claim 4 , wherein augmenting the user interaction experience further comprises receiving user input on the application instance to delete a product from the checkout list and resolving a conflict between the computer vision model of the checkout list and the application instance model of the checkout list.

6. The method of claim 1 , further comprising receiving user input communicated from the personal computing device associated with the user-record; and wherein augmenting the user interaction experience comprises executing the user interactive experience based in part on the combination of modeled state of the computer vision monitoring system and the user input.

7. The method of claim 2 , wherein detecting the personal computing device comprises detecting a device signature through wireless communication.

8. The method of claim 2 , wherein detecting the personal computing device comprises detecting location of the personal computing device within the environment.

9. The method of claim 2 , wherein detecting the personal computing device comprises entering the personal computing device into a pairing mode configured to generate an identifying signal through a user interface output of the personal computing device.

10. The method of claim 2 , wherein detecting the personal computing device comprises, at the personal computing device, directing a physical action of the user and detecting a corresponding physical action of the modeled person and thereby associating the personal computing device with the computer vision modeled person.

11. The method of claim 2 , further comprising, at the personal computing device, sensing device motion using an inertial measurement unit of the personal computing device and wherein establishing the second association between the computer vision modeled person and the personal computing device comprises detecting the personal computing device having device motion that satisfies a synchronization condition with computer vision modeled person activity.

12. The method of claim 2 , further comprising at the personal computing device receiving biometric identification; and associating human identity confirmation with the computer vision modeled person and user-record.

13. The method of claim 2 , further comprising establishing a third association between the computer vision modeled person and a human identity, which comprises biometrically identifying a human through the computer vision system and associating a human identity with the computer vision modeled person, the personal computing device, and the user-record.

14. The method of claim 2 , further comprising, prior to establishing the second association between the computer vision modeled person and the personal computing device, requesting confirmation of the association of the personal computing device, and establishing the second association upon receiving affirmative confirmation of the association.

15. The method of claim 1 , wherein the user-record is an anonymous user-record.

16. The method of claim 1 , wherein receiving registration of the user-record comprises, at the shared computer kiosk, authenticating account credentials associated with the user-record.

17. The method of claim 1 , further comprising setting permissions based in part on properties of established associations with the CV-person; and wherein augmenting the user interaction experience comprises enforcing permissions during the user interaction experience.

18. The method of claim 1 , further comprising: receiving a document of human identification; prompting permission validation; receiving validation of human identification; and wherein augmenting the user interaction experience comprises permitting actions based in part on the validation.

19. The method of claim 18 , wherein permitting actions based in part on the validation comprises enabling purchase of restricted items.

20. The method of claim 18 , wherein the validation of human identification validates age of the computer vision modeled person.

21. The method of claim 1 , wherein receiving registration of the user-record at the shared computer kiosk comprises scanning a card.

22. The method of claim 21 , wherein the card is a payment card.

23. The method of claim 1 , wherein receiving registration of the user-record at the shared computer kiosk comprises reading a near field communication (NFC) device.

24. The method of claim 1 , wherein receiving registration of the user-record at the shared computer kiosk comprises reading a graphical code.

25. The method of claim 1 , wherein receiving registration of the user-record at the shared computer kiosk comprises reading biometric information.

26. The method of claim 1 , wherein establishing the association between the computer vision modeled person and the user-record occurs at an entrance of the environment.

27. The method of claim 1 , wherein establishing the association between the computer vision modeled person and the user-record occurs at an entrance of the environment.

28. The method of claim 1 , wherein augmenting the user interaction experience further comprises communicating modeled state of a checkout list of the computer vision modeled person to the shared computer kiosk.

29. The method of claim 28 , wherein augmenting the user interaction experience further comprises receiving user input at the shared computer kiosk to delete a product from the checkout list on the shared computer kiosk, and resolving a conflict between the computer vision model of the checkout list and the checkout list on the shared computer kiosk.

30. The method of cl

Assignees

• Grabango Co

Inventors

• Glaser William
• Van Osdol Brian

Classifications

• G06Q20/206 Primary

  comprising security or operator identification provisions, e.g. password entry · CPC title

• G06Q20/20

  Point-of-sale [POS] network systems · CPC title

• G06Q20/4016

  involving fraud or risk level assessment in transaction processing · CPC title

• G06V40/20

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• G06F21/32 Primary

  using biometric data, e.g. fingerprints, iris scans or voiceprints · CPC title