Methods and systems for processing imagery

Patent metadata

Field	Value
Publication number	US-11947591-B2
Application number	US-201817049452-A
Country	US
Kind code	B2
Filing date	Sep 18, 2018
Priority date	Sep 18, 2018
Publication date	Apr 2, 2024
Grant date	Apr 2, 2024

Abstract

The present disclosure is directed to processing imagery using one or more machine learning (ML) models. In particular, data describing imagery comprising a plurality of different and distinct frames can be received; and based at least in part on one or more ML models and the data describing the imagery, and for each frame of the plurality of different and distinct frames, one or more scores can be determined for the frame. Each score of the score(s) can indicate a determined measure of suitability of the frame with respect to one or more of various different and distinct uses for which the ML model(s) are configured to determine suitability of imagery.

First claim

What is claimed is:

1. A computer-implemented method comprising:
providing, by one or more computing devices, an application programming interface (API) for processing imagery using one or more machine learning (ML) models to identify imagery determined to be suitable for one or more different and distinct uses;
receiving, by the one or more computing devices, from a requesting application, and via the API, data describing imagery comprising a plurality of different and distinct frames for processing using the one or more ML models;
determining, by the one or more computing devices, based at least in part on the one or more ML models and the data describing the imagery, and for each frame of the plurality of different and distinct frames, one or more scores for the frame, each score of the one or more scores indicating a determined measure of suitability of the frame with respect to a use of the one or more different and distinct uses; and
communicating, by the one or more computing devices, to the requesting application, and via the API, data indicating, for each frame of one or more frames of the plurality of different and distinct frames, at least one of the one or more scores for the frame.

2. The computer-implemented method of claim 1 , wherein:
the plurality of different and distinct frames comprises a series of contemporaneously generated similar frames, each frame in the series comprising: one or more subjects shared with each other frame in the series, and a contextual background, of the one or more subjects, shared with each other frame in the series; and
the determining comprises determining, for one or more frames in the series, one or more scores indicating the one or more frames in the series are determined to be better suited for a particular use of the one or more different and distinct uses than each other frame in the series.

3. The computer-implemented method of claim 1 , wherein:
the plurality of different and distinct frames comprises a set of different and distinct frames including frames from multiple different and distinct series of frames, each frame in the set comprising one or more of:
one or more subjects different and distinct from each other frame in the set, or a contextual background, of one or more subjects of the frame in the set, different and distinct from each other frame in the set; and
the determining comprises determining, for one or more frames in the set, one or more scores indicating the one or more frames in the set are determined to be better suited for a particular use of the one or more different and distinct uses than each other frame in the set.

4. The computer-implemented method of claim 1 , wherein:
the method comprises receiving, by the one or more computing devices, from the requesting application, and via the API, data indicating a particular use of the one or more different and distinct uses; and
the determining comprises determining, for one or more frames in a set of frames included in the plurality of different and distinct frames, one or more scores indicating the one or more frames in the set are determined to be better suited for the particular use of the one or more different and distinct uses than each other frame in the set.

5. The computer-implemented method of claim 1 , wherein:
the one or more different and distinct uses include a use with a particular application; and
the determining comprises determining, for one or more frames in a set of frames included in the plurality of different and distinct frames, one or more scores indicating the one or more frames in the set are determined to be better suited for use with the particular application than each other frame in the set.

6. The computer-implemented method of claim 1 , wherein:
the one or more different and distinct uses include a use with a particular audience; and
the determining comprises determining, for one or more frames in a set of frames included in the plurality of different and distinct frames, one or more scores indicating the one or more frames in the set are determined to be better suited for use with the particular audience than each other frame in the set.

7. The computer-implemented method of claim 1 , wherein:
the method comprises receiving, by the one or more computing devices, from the requesting application, and via the API, data indicating one or more particular subjects of the plurality of different and distinct frames; and
the determining comprises:
identifying, from amongst the plurality of different and distinct frames, a set of frames that each include the one or more particular subjects; and
determining, for one or more frames in the set of frames, one or more scores indicating the one or more frames in the set are determined to be better suited for a particular use of the one or more different and distinct uses than each other frame in the set.

8. The computer-implemented method of claim 1 , wherein:
the method comprises receiving, by the one or more computing devices, from the requesting application, and via the API, data indicating one or more particular expressions exhibited by subjects of the plurality of different and distinct frames; and
the determining comprises:
identifying, from amongst the plurality of different and distinct frames, a set of frames that each include the one or more particular expressions; and
determining, for one or more frames in the set of frames, one or more scores indicating the one or more frames in the set are determined to be better suited for a particular use of the one or more different and distinct uses than each other frame in the set.

9. The computer-implemented method of claim 1 , wherein:
the method comprises receiving, by the one or more computing devices, from the requesting application, and via the API, data indicating one or more particular events depicted by the plurality of different and distinct frames; and
the determining comprises:
identifying, from amongst the plurality of different and distinct frames, a set of frames that each depict the one or more particular events; and
determining, for one or more frames in the set of frames, one or more scores indicating the one or more frames in the set are determined to be better suited for a particular use of the one or more different and distinct uses than each other frame in the set.

10. The computer-implemented method of claim 1 , comprising generating, by the one or more computing devices and based at least in part on the data indicating the at least one of the one or more scores, data describing an interface comprising a curation of at least a portion of the one or more frames.

11. The computer-implemented method of claim 1 , wherein receiving the data describing the imagery comprises receiving data:
generated based at least in part on one or more arrangements of pixels included in the imagery; and
that is not sufficient to enable reconstruction of the one or more arrangements of the pixels included in the imagery.

12. The computer-implemented method of claim 1 , comprising:
determining, by the one or more computing devices, for each frame of the one or more frames, and based at least in part on the at least one of the one or more scores for the frame, a position of the frame in a series of frames for sequential presentation; and
communicating, by the one or more computing devices, to the requesting application, and via the API, data indicating, for each frame of one or more frames, the position of the frame in the series of frames for sequential presentation.

13. The computer-implemented method of claim 1 , wherein:
receiving the data describing the imagery comprise

Assignees

• Google Llc

Inventors

• Karam David
• Zhang Li
• Gilder Ariel
• Watanabe Yuzo
• Penner Eric
• Ahmad Farooq
• Adam Hartwig

Classifications

• G06N20/00

  Machine learning · CPC title

• G06F9/451 Primary

  Execution arrangements for user interfaces · CPC title

• G06F16/583 Primary

  using metadata automatically derived from the content · CPC title

• G06F16/535

  Filtering based on additional data, e.g. user or group profiles · CPC title

• G06F16/55

  Clustering; Classification · CPC title