Audio processing method and terminal

Patent metadata

Field	Value
Publication number	US-2024388866-A1
Application number	US-202418785171-A
Country	US
Kind code	A1
Filing date	Jul 26, 2024
Priority date	Jan 28, 2022
Publication date	Nov 21, 2024
Grant date	—

Abstract

This application disclose an audio processing method and a terminal. The method includes: decoding an audio bitstream to obtain audio optimization metadata, basic audio metadata, and M pieces of decoded audio data, the audio optimization metadata includes first metadata of a first optimized listening area and a first decoding audio mixing parameter corresponding to the first optimized listening area; rendering M pieces of decoded audio data based on a current location of a user and the basic audio metadata, to obtain M pieces of rendered audio data; when the current location is in the first optimized listening area, performing first audio mixing on the M pieces of rendered audio data based on the first decoding audio mixing parameter, to obtain M pieces of first audio mixing data; and mixing the M pieces of first audio mixing data, to obtain mixed audio data corresponding to the first optimized listening area.

First claim

What is claimed is:

1 . An audio processing method, comprising:
decoding an audio bitstream to obtain audio optimization metadata, basic audio metadata, and M pieces of decoded audio data, wherein the audio optimization metadata comprises first metadata of a first optimized listening area and a first decoding audio mixing parameter corresponding to the first optimized listening area, and M is a positive integer; rendering M pieces of decoded audio data based on a current location of a user and the basic audio metadata, to obtain M pieces of rendered audio data; when the current location is in the first optimized listening area, performing first audio mixing on the M pieces of rendered audio data based on the first decoding audio mixing parameter, to obtain M pieces of first audio mixing data; and mixing the M pieces of first audio mixing data, to obtain mixed audio data corresponding to the first optimized listening area.

2 . The method according to claim 1 , wherein the audio optimization metadata further comprises a second decoding audio mixing parameter corresponding to the first optimized listening area; and
the method further comprises: performing second audio mixing on the mixed audio data based on the second decoding audio mixing parameter, to obtain second audio mixing data corresponding to the first optimized listening area.

3 . The method according to claim 2 , wherein the audio optimization metadata further comprises: N−1 difference parameters of N−1 second decoding audio mixing parameters corresponding to N−1 optimized listening areas other than the first optimized listening area in N optimized listening areas with respect to the second decoding audio mixing parameter corresponding to the first optimized listening area, wherein N is a positive integer.

4 . The method according to claim 1 , wherein the method further comprises:
decoding a video image bitstream to obtain decoded video image data and video image metadata, wherein the video image metadata comprises video metadata and image metadata; rendering the decoded video image data based on the video image metadata, to obtain rendered video image data; establishing a virtual scene based on the rendered video image data; and identifying the first optimized listening area in the virtual scene based on the rendered video image data and the audio optimization metadata.

5 . The method according to claim 1 , wherein the audio optimization metadata comprises: N−1 difference parameters of N−1 first decoding audio mixing parameters corresponding to the N−1 optimized listening areas other than the first optimized listening area in the N optimized listening areas with respect to the first decoding audio mixing parameter corresponding to the first optimized listening area, wherein N is a positive integer.

6 . The method according to claim 1 , wherein the audio optimization metadata further comprises: the central location coordinates of the first optimized listening area in the N optimized listening areas, and a location offset of central location coordinates of the N−1 optimized listening areas other than the first optimized listening area in the N optimized listening areas with respect to the central location coordinates of the first optimized listening area, wherein N is a positive integer.

7 . An audio processing method, comprising:
receiving audio optimization metadata, basic audio metadata, and M pieces of first audio data, wherein the audio optimization metadata comprises first metadata of a first optimized listening area and a first audio mixing parameter corresponding to the first optimized listening area, and M is a positive integer; performing compression encoding on the audio optimization metadata, the basic audio metadata, and the M pieces of first audio data, to obtain an audio bitstream; and sending the audio bitstream.

8 . The method according to claim 7 , wherein the audio optimization metadata further comprises a second audio mixing parameter change identifier, wherein
the second audio mixing parameter change identifier indicates whether a second audio mixing parameter corresponding to first audio data of a current frame changes compared with a second audio mixing parameter corresponding to first audio data of a previous frame.

9 . The method according to claim 7 , wherein the audio optimization metadata further comprises the second audio mixing parameter corresponding to the first optimized listening area in N optimized listening areas, and N−1 difference parameters of N−1 second audio mixing parameters corresponding to N−1 optimized listening areas other than the first optimized listening area in the N optimized listening areas with respect to the second audio mixing parameter corresponding to the first optimized listening area, wherein N is a positive integer.

10 . The method according to claim 7 , wherein the audio optimization metadata further comprises: N−1 difference parameters of N−1 first audio mixing parameters corresponding to the N−1 optimized listening areas other than the first optimized listening area in the N optimized listening areas with respect to the first audio mixing parameter corresponding to the first optimized listening area.

11 . The method according to claim 7 , wherein the audio optimization metadata further comprises: central location coordinates of the first optimized listening area in the N optimized listening areas, and a location offset of central location coordinates of the N−1 optimized listening areas other than the first optimized listening area in the N optimized listening areas with respect to the central location coordinates of the first optimized listening area, wherein N is a positive integer.

12 . The method according to claim 7 , wherein the audio optimization metadata further comprises: an optimized listening area change identifier, and/or a first audio mixing parameter change identifier, wherein
the optimized listening area change identifier indicates whether the first optimized listening area changes; and the first audio mixing parameter change identifier indicates whether a first audio mixing parameter corresponding to the first audio data of the current frame changes compared with a first audio mixing parameter corresponding to the first audio data of the previous frame.

13 . An audio processing method, comprising:
obtaining basic audio metadata and metadata of N optimized listening areas, wherein N is a positive integer, and the N optimized listening areas comprise a first optimized listening area; rendering M pieces of to-be-processed audio data based on the first optimized listening area and the basic audio metadata, to obtain M pieces of rendered audio data corresponding to the first optimized listening area, wherein M is a positive integer; performing first audio mixing on the M pieces of rendered audio data, to obtain M pieces of first audio mixing data and a first audio mixing parameter corresponding to the first optimized listening area; and generating audio optimization metadata based on first metadata of the first optimized listening area and the first audio mixing parameter, wherein the audio optimization metadata comprises the first metadata and the first audio mixing parameter.

14 . The method according to claim 13 , wherein the method further comprises: mixing the M pieces of first audio mixing data, to obtain mixed audio data corresponding to the first optimized listening area; and performing second audio mixing on the mixed audio data, to obtain second audio mixing data corresponding to the first optimized listening area and a second audio mixing parameter corres

Assignees

• Huawei Tech Co Ltd

Inventors

• Ding Jiance

Classifications

• H04S2420/03

  Application of parametric coding in stereophonic audio systems · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• G10L19/008 Primary

  Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing · CPC title

• H04N19/44

  Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder · CPC title

• H04S2400/03

  Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1 (H04S2400/01 takes precedence) · CPC title