Method of generating vibration feedback signal, electronic device and storage medium

Patent metadata

Field	Value
Publication number	US-12593189-B2
Application number	US-202318334335-A
Country	US
Kind code	B2
Filing date	Jun 13, 2023
Priority date	Dec 7, 2022
Publication date	Mar 31, 2026
Grant date	Mar 31, 2026

Abstract

A method of generating a vibration feedback signal is provided and includes: generating an energy change trend curve (with a frame number as a horizontal axis and an energy change value as a vertical axis) of audio based on audio data; comparing an energy change value in the energy change trend curve with a preset energy change threshold; dividing the audio into audio segments based on a comparison result (indicating a tempo change trend of audio); matching a vibration feedback signal to the respective audio segment based on the tempo change trend; performing splicing process on all vibration feedback signals to obtain a target vibration feedback signal. The target vibration feedback signal obtained by splicing the vibration feedback signals finally matched to the audio segments conforms to the overall tempo change trend of the audio, thus effectively improving the rationality on matching between the audio and the vibration feedback signal.

First claim

What is claimed is:

1 . An electronic device comprising:
a memory and at least one processor; and a vibration feedback device; wherein the memory is configured to store at least one program, causing when executed by the at least one processor, the at least one processor to implement a method of generating a target vibration feedback signal; wherein the vibration feedback device is configured to vibrate synchronously during audio playback in response to the target vibration feedback signal generated by the at least one processor; wherein the method of generating the target vibration feedback signal comprising:
generating an energy change trend curve of audio based on audio data; wherein the energy change trend curve has a horizontal axis of frame number and a vertical axis of energy change value;
comparing an energy change value in the energy change trend curve with a preset energy change threshold and obtaining a comparison result;
dividing the audio into a plurality of audio segments based on the comparison result; wherein the comparison result indicates a tempo change trend corresponding to a respective audio segment of the plurality of audio segments;
matching a vibration feedback signal to each of the plurality of audio segments based on the tempo change trend corresponding to the respective audio segment;
performing a splicing process on all the vibration feedback signals to obtain the target vibration feedback signal.

2 . The electronic device according to claim 1 , wherein generating the energy change trend curve of the audio based on the audio data comprises:
generating an energy accumulation curve of the audio based on the audio data; wherein the energy accumulation curve has a horizontal axis of frame number and a vertical axis of energy accumulation value; and
generating the energy change trend curve of the audio based on the energy accumulation curve.

3 . The electronic device according to claim 2 , wherein generating the energy accumulation curve of the audio based on the audio data comprises:
generating the energy accumulation curve of the audio based on a number of frames, duration, a number of beats per minute and an energy curve of the audio; wherein the energy curve has a horizontal axis of frame number and a vertical axis of energy value.

4 . The electronic device according to claim 3 , wherein the energy curve is a Mel spectrum energy curve, with a number of frame as a horizontal axis and a Mel spectrum energy value as a vertical axis.

5 . The electronic device according to claim 3 , wherein generating the energy change trend curve of the audio based on the audio data further comprises:
obtaining an energy envelope of an original spectral energy of the audio data; and
correcting the energy change trend curve based on the energy envelope.

6 . The electronic device according to claim 5 , wherein the method comprises:
adjusting the preset energy change threshold based on a largest energy change value in a corrected energy change trend curve after correcting the energy change trend curve based on the energy envelope.

7 . The electronic device according to claim 1 , wherein dividing the audio into the plurality of audio segments based on the comparison result comprises:
dividing the audio into at least one first audio segment and at least one second audio segment based on the comparison result; wherein a corresponding energy change value in the energy change trend curve of each of the at least one first audio segment is greater than or equal to the preset energy change threshold, a corresponding energy change value in the energy change trend curve of each of the at least one second audio segment is less than the preset energy change threshold, tempo change trends of the at least one first audio segment is greater than that of the at least one second audio segment.

8 . The electronic device according to claim 1 , wherein matching the vibration feedback signal to each of the plurality of audio segments based on the tempo change trend corresponding to the respective audio segment comprises:
matching a corresponding vibration feedback signal to the respective audio segment from a pre-created signal library based on the tempo change trend corresponding to the respective audio segment.

9 . The electronic device according to claim 1 , wherein the method further comprises:
prior to matching the vibration feedback signal to each of the plurality of audio segments based on the tempo change trend corresponding to the respective audio segment:
merging an audio segment with duration less than or equal to a preset duration threshold into a previous audio segment or a next audio segment; or,
dividing an audio segment with duration less than or equal to the preset duration threshold into two sub-audio segments and merging the two sub-audio segments into the previous audio segment and the next audio segment respectively.

Assignees

• Aac Acoustic Tech Shanghai Co Ltd

Inventors

• Pei Shiyu
• Meng Zengyou
• Zheng Yajun
• Cao Mengya

Classifications

• H04S7/302

  Electronic adaptation of stereophonic sound system to listener position or orientation (H04S7/301 takes precedence) · CPC title

• H04R5/04

  Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)} · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G10L2015/027

  Syllables being the recognition units · CPC title

• G10L15/04

  Segmentation; Word boundary detection · CPC title