Cross-asset media analysis and processing

Patent metadata

Field	Value
Publication number	US-10282633-B2
Application number	US-201615298777-A
Country	US
Kind code	B2
Filing date	Oct 20, 2016
Priority date	Oct 20, 2015
Publication date	May 7, 2019
Grant date	May 7, 2019

Abstract

A method for processing media assets includes, given a first media asset, deriving characteristics from the first media asset, searching for other media assets having characteristics that correlate to the characteristics of the first media asset, when a match is found, deriving content corrections for the first media asset or a matching media asset from the other of the first media asset or the matching media asset, and correcting content of the first media asset or the matching media asset based on the content corrections.

First claim

The invention claimed is:

1. A method for processing media assets, comprising:
given a first media asset, deriving characteristics from the first media asset;
searching for other media assets having characteristics that correlate to the characteristics of the first media asset;
when a matching media asset is found, deriving content corrections for one of the group of the first media asset and a matching media asset from the other of the first media asset and the matching media asset; and
correcting content of the first media asset or the matching media asset based on the content corrections.

2. The method of claim 1 , wherein:
the deriving content corrections comprises deriving content corrections for the first media asset from the matching media asset, and
the correcting content comprises correcting content of the first media asset.

3. The method of claim 1 , wherein:
the deriving content corrections comprises deriving content corrections for the matching media asset from the first media asset, and
the correcting content comprises correcting content of the matching media asset.

4. The method of claim 1 , wherein:
the deriving characteristics comprises detecting an object from content of the first media asset or the matching media asset, and
the searching comprises searching for other media assets in which the detected object is represented.

5. The method of claim 1 , wherein
the deriving characteristics comprises determining a location where the first media asset or
the matching media asset was captured, and the searching comprises searching for other media assets captured at the location.

6. The method of claim 5 , wherein:
the searching further comprises searching for other media assets having similar fields of view of the location, and
the deriving content corrections comprises combining at least one of the first media asset and the matching media asset with the other media assets to create a multi-dimensional model.

7. The method of claim 1 , wherein:
the first media asset and the matching media asset are captured by different devices,
the deriving characteristics comprises estimating a capture time and a location of the first media asset or the matching media asset, and
the searching comprises searching for other media assets captured at the location at a matching capture time.

8. The method of claim 1 , wherein:
the first media asset and the matching media asset are captured by a common device at different times,
the deriving characteristics comprises estimating exposure settings of the first media asset or the matching media asset, and
the searching comprises searching for other media assets with matching exposure settings.

9. The method of claim 1 , wherein:
the first media asset and the matching media asset are captured with different capture settings by a common device; and
the correcting content comprises altering the capture settings of the first media asset or the matching media asset based on the capture settings of the other of the first media asset and the matching media asset.

10. The method of claim 1 , wherein:
the first media asset is an image or a video sequence and the matching media asset is the other of an image or a video sequence,
the deriving characteristics comprises detecting an object in one of the first media asset or the matching media asset, and
the correcting content comprises applying data associated with the object to the other of the first media asset or the matching media asset.

11. The method of claim 1 , wherein:
the first media asset is an image or a video sequence and the matching media asset is the other of an image or a video sequence,
the deriving characteristics comprises identifying exposure settings associated with the first media asset or the matching media asset, and
the correcting content comprises adjusting blur in the first media asset or the matching media asset based on the identified exposure settings.

12. The method of claim 1 , wherein:
the first media asset is an image or a video sequence and the matching media asset is the other of an image or a video sequence,
the deriving characteristics comprises identifying a spatial resolution associated with the first media asset or the matching media asset and a higher spatial resolution associated with the other of the first media asset and the matching media asset, and
the correcting content comprises upsampling content of the first media asset or the matching media asset using higher spatial resolution content of the other of the first media asset and the matching media asset.

13. The method of claim 1 , wherein the correcting content comprises applying panoramic image content of the first media asset or the matching media asset to the other of the first media asset and the matching media asset.

14. The method of claim 1 , wherein the correcting content comprises replacing image data of the first media asset or the matching media asset with higher quality image data from the other of the first media asset and the matching media asset.

15. The method of claim 1 , wherein:
the deriving characteristics comprises identifying foreground image content and background image content within the first media asset,
the searching comprises searching for other media assets with similar content,
when the similar content is background image content, the correcting content comprises creating a composite background image using the similar background content and the background image content, and placing the foreground image content over the composite background image, and
when the similar content is foreground image content, the correcting content comprises creating a composite foreground image using the similar foreground content and foreground image content, and placing the background image content over the composite foreground image.

16. The method of claim 1 , wherein the searching comprises searching for an audio asset related to the first media asset or the matching media asset, and wherein the method further comprises indicating a significance of the first media asset or the matching media asset based the audio asset.

17. The method of claim 1 , wherein the deriving characteristics comprises estimating a noise content in image data.

18. The method of claim 1 , wherein the deriving characteristics comprises estimating a motion blur in image data.

19. The method of claim 18 , wherein the correcting content comprises correcting the motion blur.

20. The method of claim 1 , wherein the deriving characteristics comprises detecting an object in image data.

21. The method of claim 1 , wherein the deriving characteristics comprises detecting a face.

22. The method of claim 1 , wherein the content corrections include correcting coloration.

23. The method of claim 1 , wherein the content corrections include correcting brightness.

24. The method of claim 1 , wherein the content corrections include correcting noise content.

25. The method of claim 1 , wherein the content corrections include correcting scene classification.

26. A system for processing media assets, comprising:
an image signal processor to capture; and
a processor to:
create and store media assets;
given a first media asset, derive characteristics from the first media asset;
search for other media assets that correlate to the characteristics of the first media asset;
when a matching me

Assignees

• Apple Inc

Inventors

• Liu Shujie
• Zhai Jiefu
• Chung Chris Y
• Wu Hsi-Jung
• Zheng Yunfei
• Keinath Albert E
• Zhou Xiaosong
• Zhang Ke

Classifications

• G06V10/80

  Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10) · CPC title

• G06F16/5838 Primary

  using colour · CPC title

• G06F18/25

  Fusion techniques · CPC title

• G06V10/993

  Evaluation of the quality of the acquired pattern · CPC title

• G06T5/003

  Physics · mapped topic