Multi-mode audio recognition and auxiliary data encoding and decoding

Patent metadata

Field	Value
Publication number	US-9401153-B2
Application number	US-201313841727-A
Country	US
Kind code	B2
Filing date	Mar 15, 2013
Priority date	Oct 15, 2012
Publication date	Jul 26, 2016
Grant date	Jul 26, 2016

Abstract

Audio signal processing enhances audio watermark embedding and detecting processes. Audio signal processes include audio classification and adapting watermark embedding and detecting based on classification. Advances in audio watermark design include adaptive watermark signal structure data protocols, perceptual models, and insertion methods. Perceptual and robustness evaluation is integrated into audio watermark embedding to optimize audio quality relative the original signal, and to optimize robustness or data capacity. These methods are applied to audio segments in audio embedder and detector configurations to support real time operation. Feature extraction and matching are also used to adapt audio watermark embedding and detecting.

First claim

We claim:

1. A method of embedding a watermark in an electronic audio signal, the method comprising:
with a programmed processor, classifying the audio signal according to audio type; the classifying including analyzing the audio signal to detect a voiced and an unvoiced sound;
based on the audio type, selecting with a programmed processor an audio perceptual model adapted for a detected voiced or unvoiced sound and insertion method; and
with a programmed processor, inserting a watermark of an audio watermark type in the audio signal according to the selected perceptual model.

2. The method of claim 1 wherein the classifying comprises discriminating audio segments based on types, including speech and music.

3. The method of claim 1 including embedding a code conveying the watermark type in the audio signal.

4. The method of claim 3 wherein the code comprises a Hadamard code.

5. The method of claim 1 wherein classifying comprises computing a feature vector of an audio segment, and determining audio type by submitting the feature vector to a database, where feature vectors are classified by audio type.

6. The method of claim 1 wherein classifying comprises transforming an audio segment according to an ear model that models human auditory response to the audio segment and provides a measure of perceptible features of the audio segment, and from the measure of perceptible features, selecting a watermark type.

7. The method of 6 including analyzing ear model output variables providing perceptible energy in bands to discern audio class.

8. The method of claim 1 wherein classifying comprises determining whether an audio segment is stationary or non-stationary, and adapting resolution of the perceptual model based on whether the audio segment is stationary or non-stationary.

9. The method of claim 1 wherein classifying comprises detecting spectral peaks and classifying the audio based on the detected spectral peaks; and applying an insertion method in which spectral peaks are adjusted to correspond to a bump structure of a corresponding watermark signal.

10. The method of claim 1 in which the classifying is performed on audio segments which are being transmitted, the classifying being performed at or near real time to limit delay introduced in transmission of the audio signal.

11. A method of embedding a watermark in an electronic audio signal, the method comprising:
with a programmed processor, classifying the audio signal according to audio type;
based on the audio type, selecting with a programmed processor an audio watermark type and insertion method; and
with a programmed processor, inserting a watermark of the selected audio watermark type in the audio signal according to the selected insertion method;
wherein classifying comprises transforming an audio segment according to an ear model that models human auditory response to the audio segment and provides a measure of perceptible features of the audio segment, and from the measure of perceptible features, selecting a watermark type; including analyzing ear model output variables providing perceptible energy in bands to discern audio class;
wherein analyzing comprises mapping a feature vector derived from the perceptible energy in the bands to an audio class in a feature vector database.

12. An audio processing system comprising:
a classifier for classifying an electronic audio signal according to audio type; the classifying including analyzing the audio signal to detect a voiced and an unvoiced sound;
a watermark embedder, in communication with the classifier for receiving the audio type, and based on the audio type, selecting an audio perceptual model adapted for a detected voiced or unvoiced sound and insertion method; and for inserting a watermark of an audio watermark type in the audio signal according to the selected perceptual model.

13. The system of claim 12 wherein the classifier discriminates audio segments based on types, including speech and music.

14. A method of detecting a watermark in an electronic audio signal, the method comprising:
with a programmed processor, classifying the audio signal according to audio type, the classifying including analyzing the audio signal to detect a voiced and an unvoiced sound;
based on the audio type, determining with a programmed processor an audio watermark type and insertion method; and
with a programmed processor, detecting a watermark of the selected audio watermark type in the audio signal according to the selected insertion method, the detecting including transforming the audio signal into a state or domain from which message symbols are extracted.

15. The method of claim 14 wherein the classifying comprises discriminating audio segments based on types, including speech and music.

16. An audio processing system comprising:
a classifier for classifying the audio signal according to audio type, the classifying including analyzing the audio signal to detect a voiced and an unvoiced sound;
a watermark detector, in communication with the classifier for receiving the audio type, and based on the audio type, determining an audio watermark type and insertion method; and for detecting a watermark of the selected audio watermark type in the audio signal according to the selected insertion method, the detector configured to transform the audio signal into a state or domain and extract message symbols from the transformed state or domain of the audio signal.

17. A device for embedding a watermark in an electronic audio signal, the device comprising:
means for classifying the audio signal according to audio type; the classifying including means for processing the audio signal to detect a voiced and an unvoiced sound;
means for selecting based on the audio type an audio perceptual model adapted for a detected voiced or unvoiced sound and insertion method; and
means for embedding a watermark of an audio watermark type in the audio signal according to the selected perceptual model.

18. The device of claim 17 wherein the classifying comprises discriminating audio segments based on types, including speech and music.

19. The device of claim 17 wherein the means for embedding is configured to embed a code conveying the watermark type in the audio signal.

20. The device of claim 17 wherein the means for classifying comprises a programmed processor configured to compute a feature vector of an audio segment, and the programmed processor is configured to determine audio type by submitting the feature vector to a database, where feature vectors are classified by audio type.

21. The device of claim 17 wherein the means for classifying comprises a programmed processor configured to transform an audio segment according to an ear model that models human auditory response to the audio segment and provides a measure of perceptible features of the audio segment, and the programmed processor configured to select a watermark type from the measure of perceptible features.

22. The device of claim 21 the programmed processor is configured to analyze ear model output variables providing perceptible energy in bands to discern audio class.

Assignees

• Digimarc Corp

Inventors

• Sharma Ravi K
• Bradley Brett A
• Thagadur Shivappa Shankar

Classifications

• G10L19/02

  using spectral analysis, e.g. transform vocoders or subband vocoders · CPC title

• G10L19/018 Primary

  Audio watermarking, i.e. embedding inaudible data in the audio signal · CPC title

• G10L19/028

  Noise substitution, i.e. substituting non-tonal spectral components by noisy source (comfort noise for discontinuous speech transmission G10L19/012) · CPC title

• G10L25/87

  Detection of discrete points within a voice signal · CPC title