Filtering wind noises in video content

Patent metadata

Field	Value
Publication number	US-10356469-B2
Application number	US-201715826622-A
Country	US
Kind code	B2
Filing date	Nov 29, 2017
Priority date	May 5, 2016
Publication date	Jul 16, 2019
Grant date	Jul 16, 2019

Abstract

Implementations disclose filtering wind noises in video content. A method includes receiving video content comprising an audio component and a video component, detecting, by a processing device, occurrence of a wind noise artifact in a segment of the audio component, identifying an intensity of the wind noise artifact, wherein the intensity is based on a signal-to-noise ratio of the wind noise artifact, selecting, by the processing device, a wind noise replacement operation based on the identified intensity of the wind noise artifact, and applying, by the processing device, the selected wind noise replacement operation to the segment of the audio component to remove the wind noise artifact from the segment.

First claim

What is claimed is:

1. A method comprising:
receiving video content comprising an audio component and a video component;
detecting, by a processing device, occurrence of a wind noise artifact in a segment of the audio component;
identifying intensity of the wind noise artifact, wherein the intensity is based on a signal-to-noise ratio of the wind noise artifact;
selecting, by the processing device, a wind noise replacement operation based on the identified intensity of the wind noise artifact; and
applying, by the processing device, the selected wind noise replacement operation to the segment of the audio component to remove the wind noise artifact from the segment.

2. The method of claim 1 , wherein the wind noise replacement operation comprises in-filling the segment with an interpolation of audio signal extracted from other segments of the audio component surrounding the segment.

3. The method of claim 1 , wherein the wind noise replacement operation comprises filtering the segment to remove the wind noise artifact from the segment.

4. The method of claim 1 , wherein the wind noise replacement operation comprises replacing the segment with silence.

5. The method of claim 1 , wherein the wind noise replacement operation comprises replacing the wind noise artifact with audio signal extracted from another segment of the audio component.

6. The method of claim 1 , wherein the wind noise replacement operation comprises replacing the audio component with a different audio component.

7. The method of claim 6 , wherein a user is prompted to select the different audio component.

8. The method of claim 1 , wherein selecting the wind noise replacement operation further comprises:
deriving a plurality of signals from the identified intensity of the wind noise artifact;
mapping the derived signals to a corresponding set of threshold values; and
selecting the wind noise replacement operation that corresponds to the set of thresholds values mapped to the derived signals.

9. The method of claim 1 , further comprising identifying a duration of the wind noise artifact, the duration comprising a time length of the wind noise artifact in the segment, wherein selecting the wind noise replacement operation is further based on the identified duration of the wind noise artifact.

10. The method of claim 1 , wherein machine learning is used to detect the occurrence of the wind noise artifact.

11. The method of claim 1 , wherein deep learning is used to detect the occurrence of the wind noise artifact.

12. The method of claim 1 , wherein spectrogram analysis is used to detect the occurrence of the wind noise artifact.

13. A system comprising:
a memory; and
a processing device coupled to the memory, wherein the processing device is to:
extract an audio component from video content;
analyze the audio component to identify occurrence of a wind noise artifact in a segment of the audio component;
identify characteristics of the segment, wherein the characteristics comprise an intensity of the wind noise artifact in the segment, and wherein the intensity is based on a signal-to-noise ratio of the wind noise artifact;
select a wind noise replacement operation based on the identified characteristics; and
remove the wind noise artifact from the segment via application of the selected wind noise replacement operation to the segment.

14. The system of claim 13 , wherein the wind noise replacement operation comprises in-filling the segment with an interpolation of audio signal extracted from other segments of the audio component surrounding the segment.

15. The system of claim 13 , wherein the wind noise replacement operation comprises filtering the segment to remove the wind noise artifact from the segment.

16. The system of claim 13 , wherein the wind noise replacement operation comprises replacing the wind noise artifact with audio signal extracted from another segment of the audio component.

17. The system of claim 13 , wherein the wind noise replacement operation comprises replacing the segment with silence.

18. The system of claim 13 , wherein the wind noise replacement operation comprises replacing the audio component with a different audio component.

19. The system of claim 13 , wherein the characteristics of the segment further comprise at least one of a duration of the wind noise artifact, time markers of the segment, or an amplitude of the segment.

20. The system of claim 13 , wherein the processing device to select the wind noise replacement operation further comprises:
deriving a plurality of signals from the identified characteristics;
mapping the derived signals to corresponding set of threshold values; and
selecting the wind noise replacement operation that corresponds to the set of thresholds values mapped to the derived signals.

21. A non-transitory machine-readable storage medium storing instructions which, when executed, cause a processing device to perform operations comprising:
extracting an audio component from video content;
detecting, by the processing device, occurrence of a wind noise artifact in a segment of the audio component;
identifying, by the processing device, characteristics of the wind noise artifact, wherein the characteristics comprise an intensity of the wind noise artifact in the segment, and wherein the intensity is based on a signal-to-noise ratio of the wind noise artifact;
selecting, by the processing device, a wind noise replacement operation based on the identified characteristics of the wind noise artifact;
applying the selected wind noise replacement operation to the segment of the audio component to remove the wind noise artifact from the segment and generate a modified audio component;
combining the modified audio component with the video content; and
transmitting, by the processing device, the video content to a content sharing platform.

22. The non-transitory machine-readable storage medium of claim 21 , wherein the wind noise replacement operation comprises in-filling the segment with an interpolation of audio signal extracted from other segments of the audio component surrounding the segment.

23. The non-transitory machine-readable storage medium of claim 21 , wherein the wind noise replacement operation comprises filtering the segment to remove the wind noise artifact from the segment.

24. The non-transitory machine-readable storage medium of claim 21 , wherein the wind noise replacement operation comprises replacing the segment with silence.

25. The non-transitory machine-readable storage medium of claim 21 , wherein the wind noise replacement operation comprises replacing the wind noise artifact with audio signal extracted from another segment of the audio component.

26. The non-transitory machine-readable storage medium of claim 21 , wherein the wind noise replacement operation comprises replacing the audio component with a different audio component.

Assignees

• Google Inc
• Google Llc

Inventors

• Eban Elad
• Jansen Aren
• Chaudhuri Sourish

Classifications

• H04H60/12

  wherein another information is substituted for the portion of broadcast information · CPC title

• H04H60/65

  for using the result on users' side · CPC title

• H04N9/802

  involving processing of the sound signal (H04N9/806, H04N9/835 take precedence) · CPC title

• H04N21/4398 Primary

  involving reformatting operations of audio signals (details of audio signal transcoding G10L19/173) · CPC title

• H04N21/4394 Primary

  involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58) · CPC title