Method and computer readable medium for presentation of content items synchronized with media display

Patent metadata

Field	Value
Publication number	US-10643264-B2
Application number	US-201615218319-A
Country	US
Kind code	B2
Filing date	Jul 25, 2016
Priority date	Jul 25, 2016
Publication date	May 5, 2020
Grant date	May 5, 2020

Abstract

An online system displays relevant products along with a video watched by a user in a synchronized manner, which allows the user to explore relevant content (e.g., products recommended to the user) while the user is watching the video. The online system receives a video including a sequence of video frames to be displayed to the user, and the video may be associated with a specific theme, e.g., a soccer game, and one or more video frames may include points of interest (referred as “Q points”). The online system detects or receives Q points associated with corresponding video frames, where a Q point in a video frame identifies an object of interest in the video frame. The online system also correlates user profile information of the user with the detected points to select relevant products tailored to the user.

First claim

What is claimed is:

1. A computer-implemented method, the method comprising:
receiving a video for display to a user, the video comprising a plurality of video frames;
for each video frame:
analyzing visual features of the video frame; and
detecting one or more points of interest in the video frame based on the analysis of the visual features of the video frame, a point of interest in the video frame being associated with an object in the video frame, and the object in the video frame being related to one or more different products;

for each of the identified point of interests in the plurality of video frames, selecting a product of the one or more different products related to the object associated with the point of interest, by:
analyzing user profile information associated with the user to derive a user preference; and
selecting the product from the one or more different products based on the derived user preference; and

providing for display at a display interface:
a video frame of the plurality of video frames in a first portion of the display interface shown to the user, and
a scrolling carousel of products located in a second portion of the display interface, the carousel displaying content associated with a selected product for an identified point of interest of the identified points of interest corresponding to the video frame being concurrently displayed to the user in the first portion of the display interface, and wherein the scrolling carousel further concurrently displays a portion of content associated with a next selected product associated with a next video frame of the plurality of video frames containing a point of interest.

2. The method of claim 1 , wherein providing for display at the display interface comprises:
for each video frame currently being displayed to the user:
determining whether the video frame has a detected point of interest; and
selecting one or more products associated with the detected point of interest in the video frame.

3. The method of claim 1 ,
wherein the selected product is same or similar to a product shown in the video frame.

4. The method of claim 1 , wherein providing for display the scrolling carousel of products comprises:
displaying one or more tools in the second portion of the display interface, wherein the user interacts with the displayed products through the one or more tools.

5. The method of claim 4 , wherein the one or more tools displayed in the second portion of the display interface comprise one of the following:
a purchase tool for the user to purchase the displayed product; and
a bookmark tool for the user to bookmark the displayed product.

6. The method of claim 1 , wherein a point of interest in a video frame is identified by one of a timestamp associated with the video frame and frame number of the video frame.

7. The method of claim 1 , wherein the selected product associated with the identified point of interest shares one or more characteristics with the object in the video frame associated with the identified point of interest.

8. The method of claim 1 , wherein analyzing visual features of the video frame comprises:
applying a trained image feature extraction model to the video frame to extract a plurality of visual features;
identifying an object in the video frame based on the extracted visual features; and
identifying one or more items associated with the identified object, each identified item corresponding to a product that is likely to be of interest to the user.

9. The method of claim 1 , wherein the video displayed to the user is one of a pre-recorded video and a broadcast video streamed to the user in real time.

10. A non-transitory computer readable storage medium storing computer program instructions, the computer program instructions when executed by a computer processor causes the processor to perform the steps of:
receiving a video for display to a user, the video comprising a plurality of video frames;
for each video frame:
analyzing visual features of the video frame; and
detecting one or more points of interest in the video frame based on the analysis of the visual features of the video frame, a point of interest in the video frame being associated with an object in the video frame, and the object in the video frame being related to one or more different products;

for each of the identified point of interests in the plurality of video frames, selecting a product of the one or more different products related to the object associated with the point of interest, by:
analyzing user profile information associated with the user to derive a user preference;
and
selecting the product from the one or more different products based on the derived user preference; and

providing for display at a display interface:
a video frame of the plurality of video frames in a first portion of the display interface shown to the user, and
a scrolling carousel of products located in a second portion of the display interface, the carousel displaying content associated with a selected product for an identified point of interest of the identified points of interest corresponding to the video frame being concurrently displayed to the user in the first portion of the display interface, and wherein the scrolling carousel further concurrently displays a portion of content associated with a next selected product associated with a next video frame of the plurality of video frames containing a point of interest.

11. The non-transitory computer readable storage medium of claim 10 , wherein providing for display at the display interface comprises:
for each video frame currently being displayed to the user:
determining whether the video frame has a detected point of interest; and
selecting one or more products associated with the detected point of interest in the video frame.

12. The non-transitory computer readable storage medium of claim 10 ,
wherein the selected product is same or similar to a product shown in the video frame.

13. The non-transitory computer readable storage medium of claim 10 , wherein providing for display the scrolling carousel of products comprises:
displaying one or more tools in the second portion of the display interface, wherein the user interacts with the displayed products through one or more tools.

14. The non-transitory computer readable storage medium of claim 13 , wherein the one or more tools displayed in the second portion of the display interface comprise one of the following:
a purchase tool for the user to purchase the displayed product; and
a bookmark tool for the user to bookmark the displayed product.

15. The non-transitory computer readable storage medium of claim 10 , wherein a point of interest in a video frame is identified by one of a timestamp associated with the video frame and frame number of the video frame.

16. The non-transitory computer readable storage medium of claim 10 , wherein the selected product associated with the identified point of interest shares one or more characteristics with the object in the video frame associated with the identified point of interest.

17. The non-transitory computer readable storage medium of claim 10 , wherein analyzing visual features of the video frame comprises:
applying a trained image feature extraction model to the video frame to extract a plurality of visual features;
identifying an object in the video frame based on the extracted visual features; and
identifying one or more items associated with the identified object, each identified item corresponding to a pro

Assignees

• Facebook Inc

Inventors

• Jones Christopher William
• D'Arcy Mark Gerard
• Lee Ji Byol

Classifications

• G06Q30/0631 Primary

  Recommending goods or services · CPC title

• H04N21/4728

  for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region · CPC title

• G06Q30/0643

  graphically representing goods, e.g. 3D product representation · CPC title

• G06Q30/0633

  Managing shopping lists, e.g. compiling or processing purchase lists (shipping orders G06Q10/083; order filling G06Q10/087) · CPC title

• H04N21/44008

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title