Systems and methods for automated object recognition

Patent metadata

Field	Value
Publication number	US-11610387-B2
Application number	US-202117175271-A
Country	US
Kind code	B2
Filing date	Feb 12, 2021
Priority date	Jun 23, 2016
Publication date	Mar 21, 2023
Grant date	Mar 21, 2023

Abstract

A method for recognizing an object in a video stream may include receiving a video stream from a video source, the video stream comprising a plurality of video frames. The method may also include selecting at least one video frame from the video frames according to a frame selection rate. The method may also include partitioning the selected video frame into a first plurality of image blocks. The method may also include recognizing, out of the first plurality of image blocks, a second plurality of image blocks which comprise an image of an object, the recognition being based on an image recognition parameter determined by a machine-learning algorithm. The method may also include determining that at least one of the second plurality of image blocks corresponds to the object based on a likelihood metric, the likelihood metric being determined by the processor based on at least the frame selection rate. The method may further include displaying, on a display, information identifying the object. A system and non-transitory computer-readable medium may also be provided.

First claim

What is claimed is:

1. A system for providing an object list based on objects identified in a video stream, comprising:
a memory storing instructions; and
a processor configured to execute the stored instructions to:
receive the video stream from a video source, the video stream comprising a first set of video frames;
select one or more video frames from the first set of video frames according to a frame selection rate;
partition the one or more selected video frames into one or more sets of image blocks, each set of image blocks corresponding to a respective video frame;
identify, within the one or more sets of image blocks, regions comprising an image of one or more objects, based on a machine-learning algorithm for determining regions characterized by an image recognition parameter;
calculate a likelihood metric that each of the identified regions correspond to the one or more objects;
update the object list, over a predetermined period of time, with the one or more objects, wherein the object list is updated based on the likelihood metric; and
display, on a display, information related to the object list.

2. The system of claim 1 , wherein the processor is further configured to execute the stored instructions to apply the machine-learning algorithm to a second set of video frames.

3. The system of claim 1 , wherein the object list appears at a predetermined interval during the video stream.

4. The system of claim 1 , wherein calculating that each of the identified regions corresponds to the one or more objects comprises:
comparing the likelihood metric to a predetermined threshold; and
determining that the identified regions correspond to the one or more objects when the likelihood metric exceeds or equals the predetermined threshold.

5. The system of claim 1 , wherein the processor is further configured to execute the stored instructions to adjust the frame selection rate in response to a user input.

6. The system of claim 1 , wherein the processor is further configured to execute the stored instructions to determine the frame selection rate based on at least one of an image quality of the video stream, a location of the object in the one or more selected video frames, or a viewable angle of the object in the one or more selected video frames.

7. The system of claim 1 , wherein the processor is further configured to execute the stored instructions to determine the likelihood metric based on the information identifying the object.

8. The system of claim 1 , wherein the processor is further configured to execute the stored instructions to receive the information related to the one or more objects, wherein the information includes descriptions of the one or more objects.

9. The system of claim 8 , wherein the description of the one or more objects includes information related to one or more of a price of the object, an availability of the object, or a location of the object.

10. The system of claim 1 , wherein:
the processor is further configured to execute the stored instructions to display the video stream; and displaying the information identifying the object comprises displaying the object list overlayed atop of the displayed video stream.

11. A computer-implemented method for recognizing an object in a video stream, comprising:
receiving the video stream from a video source, the video stream comprising a first set of video frames;
selecting one or more video frames from the first set of video frames according to a frame selection rate;
partitioning the one or more selected video frames into one or more sets of image blocks, each set of image blocks corresponding to a respective video frame;
identifying, within the one or more sets of image blocks, regions comprising an image of one or more objects, based on a machine-learning algorithm for determining regions characterized by an image recognition parameter;
calculating a likelihood metric that each of the identified regions correspond to the one or more objects;
updating an object list, over a predetermined period of time, with the one or more objects, wherein the object list is updated based on the likelihood metric; and
displaying, on a display, information related to the object list.

12. The computer-implemented method of claim 11 , wherein a processor is further configured to execute stored instructions to apply the machine-learning algorithm to a second set of video frames.

13. The computer-implemented method of claim 11 , wherein the object list appears at a predetermined interval during the video stream.

14. The computer-implemented method of claim 11 , wherein a processor is further configured to execute stored instructions to adjust the frame selection rate in response to a user input.

15. The computer-implemented method of claim 11 , wherein calculating that each of the identified regions corresponds to the one or more objects comprises:
comparing the likelihood metric to a predetermined threshold; and

determining that the identified regions correspond to the one or more objects when the likelihood metric exceeds or equals the predetermined threshold.

16. The computer-implemented method of claim 15 , further comprising adjusting the frame selection rate when the likelihood metric is less than the predetermined threshold.

17. The computer-implemented method of claim 11 , further comprising determining the frame selection rate based on at least one of image quality of the video stream, a location of the first set of objects in the one or more selected video frames, or a viewable angle of a second set of objects in the one or more selected video frames.

18. The computer-implemented method of claim 11 , wherein a processor is further configured to execute stored instructions to receive the information related to the one or more objects, wherein the information includes descriptions of the one or more objects.

19. A non-transitory computer-readable medium storing instructions which, when executed, cause at least one processor to perform operations for recognizing an object in a video stream, the operations comprising:
receiving the video stream from a video source, the video stream comprising a first set of video frames;
selecting one or more video frames from the first set of video frames according to a frame selection rate;
partitioning the one or more selected video frames into one or more sets of image blocks, each set of image blocks corresponding to a respective video frame;
identifying, within the one or more sets of image blocks, regions comprising an image of one or more objects, based on a machine-learning algorithm for determining regions characterized by an image recognition parameter;
calculating a likelihood metric that each of the identified regions correspond to the one or more objects;
updating an object list, over a predetermined period of time, with the one or more objects, wherein the object list is updated based on the likelihood metric; and
displaying, on a display, information related to the object list.

20. The non-transitory computer-readable medium of claim 19 , wherein calculating that each of the identified regions corresponds to the one or more objects comprises:
comparing the likelihood metric to a predetermined threshold; and
determining that the identified regions correspond to the one or more objects when the likelihood metric exceeds or equals the predetermined threshold.

Assignees

• Capital One Services Llc

Inventors

• Jouhikainen Hannes Mikael
• Jacobs Drew

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06Q30/0241

  Advertisements · CPC title

• H04N21/47815

  Electronic shopping (payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12) · CPC title

• H04N21/812

  involving advertisement data (advertising per se G06Q30/02) · CPC title