Generating a shoppable video

Patent metadata

Field	Value
Publication number	US-10354290-B2
Application number	US-201514741111-A
Country	US
Kind code	B2
Filing date	Jun 16, 2015
Priority date	Jun 16, 2015
Publication date	Jul 16, 2019
Grant date	Jul 16, 2019

Abstract

Embodiments of the present invention provide systems and methods for automatically generating a shoppable video. A video is parsed into one or more scenes. Products and their corresponding product information are automatically associated with the one or more scenes. The shoppable video is then generated using the associated products and corresponding product information such that the products are visible in the shoppable video based on a scene in which the products are found.

First claim

What is claimed is:

1. One or more non-transitory computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to generate a shoppable video, comprising:
parsing, by a scene parsing module, a video into one or more video scenes;
automatically identifying, by a product determining module, one or more products in a first video scene of the one or more video scenes, the identifying comprising computing a product score for a plurality of products having corresponding product images stored in a database by calculating a feature vector for each of a plurality of tiles corresponding to the first video scene and comparing values of the feature vector corresponding to each of the plurality of tiles to values of feature vectors of product images stored in a database, wherein the product score indicates a likelihood that a product is visible in the first video scene;
retrieving, by a video generation module, product information corresponding to the one or more products in the first video scene based on the automatically identified one or more products;
automatically associating, by the video generation module, the one or more products and the corresponding product information with the first video scene by storing an indication that the one or more products in the first video scene corresponds to the retrieved product information; and
generating the shoppable video using the automatically associated one or more products and the corresponding product information, the shoppable video comprising at least the first video scene having the product information corresponding to the one or more products.

2. The one or more computer storage media of claim 1 , wherein the identifying of the one or more products in the first video scene is based on a plurality of features identified in the video frames of the first video scene compared to a plurality of features identified in a plurality of product images stored in a database.

3. The one or more computer storage media of claim 1 , wherein automatically associating the one or more products and the corresponding product information with the first video scene further comprises:
parsing the one or more video scenes into a plurality of frames;
for a first frame of the plurality of frames, producing one or more copy frames having different resolutions; and
partitioning the first frame and the one or more copy frames into a plurality of tiles.

4. The one or more computer storage media of claim 3 , further comprising:
for each of the plurality of tiles in the first frame and the one or more copy frames,
(1) computing a feature vector,
(2) comparing values of the feature vector corresponding to each of the plurality of tiles to values of feature vectors of product images stored in a database, and
(3) based on the comparison, computing the product score for the plurality of products having corresponding product images stored in the database.

5. The one or more computer storage media of claim 4 , further comprising based at least on the computed product score, determining the one or more products to associate with the at least the portion of the one or more video scenes.

6. The one or more computer storage media of claim 4 , further comprising:
for a second frame of the plurality of frames, producing one or more copy frames having different resolutions than the second frame;
partitioning the second frame and the one or more copy frames into a plurality of tiles; and
for each of the plurality of tiles in the second frame and the one or more copy frames,
(1) computing a feature vector,
(2) comparing values of the feature vector corresponding to each of the plurality of tiles to values of the feature vectors of the product images stored in the database, and
(3) based on the comparison, computing the product score for the plurality of products having corresponding product images stored in the database.

7. The one or more computer storage media of claim 4 , wherein the feature vector is an n-dimensional vector of numerical features that represent an object.

8. The one or more computer storage media of claim 4 , wherein the feature vector numerically represents one or more of color components, length, area, shape description, gradient magnitude, or gradient direction of the object in the plurality of tiles and the product images.

9. The one or more computer storage media of claim 4 , wherein the feature vectors are computed using a deep convolutional neural network.

10. The one or more computer storage media of claim 1 , further comprising:
determining the one or more products to automatically associate with the at least the portion of the one or more scenes, wherein the determining is based, at least, on a computed product scores for a plurality of products having corresponding product images stored in a database.

11. The one or more computer storage media of claim 10 , wherein the computed product scores are based on, at least, monitoring votes received for the plurality of products, such that a first product having a higher vote count than a second product indicates an increased likelihood that the first product is visible in the video.

12. The one or more computer storage media of claim 3 , wherein at least a portion of the plurality of tiles overlap with another tile.

13. The one or more computer storage media of claim 1 , wherein the shoppable video presents the one or more products based on a scene in which each product is found.

14. A computerized method for generating a shoppable video, the computerized method comprising:
computing, by an image feature vector module, a feature vector for each of a plurality of product images stored in a database;
automatically identifying, by a product determining module, one or more products in a first video scene of the one or more video scenes, the identifying comprising:
computing a feature vector using a tile feature vector module for a first tile in a first frame of a video, the first frame included in a first video scene;
comparing values, by the product determining module, of the feature vector of the first tile to values of the feature vectors for the plurality of product images; and
based on the comparing, determining that the first tile includes an image of a first product that corresponds to a first set of images of the plurality of product images stored in the database wherein the determining comprises computing a product score for the first product based on the feature vector of the first tile and the feature vectors for the plurality of product images, wherein the product score indicates a likelihood that the first product is visible in the first video scene; and

generating the shoppable video based on the automatically identified one or more products, the shoppable video comprising the first video scene having product information corresponding to the first product.

15. The computerized method of claim 14 , further comprising:
computing a feature vector for a second tile in a first frame of a video;
comparing values of the feature vector of the second tile to values of the feature vectors for the plurality of product images;
based on the comparing, determining that the second tile includes an image of a second product that corresponds to a second set of images of the plurality of product images stored in the database; and
generating the shoppable video that comprises the product information corresponding to the first product and the second product.

16. The computerized method of claim 14 , further comprisin

Assignees

• Adobe Inc

Inventors

• Yadav Vikas
• Krishnamurthy Balaji
• Sarkar Mausoom
• Mangla Rajiv
• Malik Gitesh

Classifications

• G06V10/757

  Matching configurations of points or features · CPC title

• H04N5/7605

  on discs or drums (H04N5/781, H04N5/805, H04N5/83, H04N5/85 take precedence) · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• H04N5/93

  Regeneration of the television signal or of selected parts thereof · CPC title

• G11B27/034

  on discs (G11B27/036, G11B27/038 take precedence) · CPC title