Data processing device and data processing method

Patent metadata

Field	Value
Publication number	US-12033660-B2
Application number	US-202318446775-A
Country	US
Kind code	B2
Filing date	Aug 9, 2023
Priority date	May 25, 2018
Publication date	Jul 9, 2024
Grant date	Jul 9, 2024

Abstract

A data processing device includes: a digital signal processor; at least one processor; and at least one memory device configured to store a plurality of instructions, which when executed by the at least one processor, cause the at least one processor to operate to: output a first determination result relating to a scene of content through use of sound data; select processing for the sound data by a first selection method based on the first determination result; determine an attribute of the content from among a plurality of attribute candidates; and select the processing by a second selection method, which is different from the first selection method, based on a determination result of the attribute, wherein the digital signal processor is configured to execute the processing selected by the at least one processor on the sound data.

First claim

What is claimed is:

1. A data processing device, comprising:
a digital signal processor;
at least one processor; and
at least one memory device configured to store a plurality of instructions, which when executed by the at least one processor, cause the at least one processor to operate to:
obtain, based on a scene determination model generated through a machine learning process relating to a first attribute among a plurality of attribute candidates but not to a second attribute among the plurality of attribute candidates, a first determination result relating to a scene that is a magnitude of change in amplitude of a low frequency effect (LFE) signal included in a sound data and is included in a content through use of the sound data associated with the scene;
determine an attribute of the content, in which the scene is included, from among the plurality of attribute candidates;
select processing to be applied to the sound data associated with the scene using a first selection method, which selects the processing to be applied to the sound data from among a plura lity of processing candidates respectively corresponding to a plurality of scene candidates, in a first case where the determined attribute of the content is the first attribute among the plurality of attribute candidates; and
select processing to be applied to the sound data associated with the scene using a second selection method, which selects the processing to be applied to the sound data from among the plurality of processing candidates respectively corresponding to the plurality of scene candidates and which differs from the first selection method, in a second case where the determined attribute of the content is the second attribute among the plurality of attribute candidates different from the first attribute,
wherein the digital signal processor is configured to execute the selected processing for the sound data associated with the scene on the sound data.

2. The data processing device according to claim 1 , wherein the at least one processor operates to:
obtain the first determination result comprising a score for each of the plurality of scene candidates,
multiply the score for each of the plurality of scene candidates by a coefficient in the second case where the determined attribute of the content is the second attribute, and
select the processing to be applied to the sound data associated with the scene based on the scores multiplied by the coefficient, in the second selection method.

3. The data processing device according to claim 1 , wherein the at least one processor operates to:
obtain the first determination result a score for each of the plurality of scene candidates, and
(iii) select a processing corresponding to a second scene candidate among the plurality of scene candidates when a first scene candidate among the plurality of scene candidates has the highest score in the first determination result, or (iv) select a processing, which is different from the plurality of processing candidates respectively corresponding to the plurality of scene candidates, in the second selection method.

4. The data processing device according to claim 1 , wherein
the plurality of instructions cause the at least one processor to output the first determination result based on a scene determination model generated through machine learning relating only to a part of the plurality of attribute candidates.

5. The data processing device according to claim 4 , wherein the plurality of instructions cause the at least one processor to extract a feature from the sound data associated with the scene, and perform classification based on the scene determination model, to thereby output a score relating to each of a plurality of scene candidates as the first determination result.

6. The data processing device according to claim 5 , wherein the plurality of instructions cause the at least one processor to select, in the second selection method, processing corresponding to one of the plurality of scene candidates that has a highest score among the plurality of scene candidates except a predetermined scene candidate is not selected even in a case where the predetermined scene candidate has the highest score.

7. The data processing device according to claim 5 , wherein the plurality of instructions cause the at least one processor to multiply, in the second selection method, the score relating to each of the plurality of scene candidates by a coefficient.

8. The data processing device according to claim 1 , wherein the plurality of instructions cause the at least one processor to select, in the second selection method, predetermined processing.

9. The data processing device according to claim 1 ,
wherein the plurality of instructions cause the at least one processor to select a sound field as the processing for the sound data associated with the scene, and
wherein the digital signal processor is configured to apply an effect of the sound field selected by the at least one processor to the sound data associated with the scene.

10. A data processing method, comprising:
obtaining, (i) with at least one processor operating with at least one memory device in a device and (ii) based on a scene determination model generated through a machine learning process relating to a first attribute among a plurality of attribute candidates but not to a second attribute among the plurality of attribute candidates, a first determination result relating to a scene that is a magnitude of change in amplitude of a low frequency effect (LFE) signal included in a sound data and is included in a content through use of the sound data associated with the scene;
determining, with the at least one processor operating with the at least one memory device in the device, an attribute of the content, in which the scene is included, from among the plurality of attribute candidates;
selecting, with the at least one processor operating with the at least one memory device in the device, processing to be applied to the sound data associated with the scene using a first selection method, which selects the processing to be applied to the sound data from among a plurality of processing candidates respectively corresponding to a plurality of scene candidates, in a first case where the determined attribute of the content is the first attribute among the plurality of attribute candidates;
selecting, with the at least one processor operating with the at least one memory device in the device, processing to be applied to the sound data associated with the scene using a second selection method, which selects the processing to be applied to the sound data from among the plurality of processing candidates respectively corresponding to the plurality of scene candidates and which differs from the first selection method, in a second case where the determined attribute of the content is the second attribute among the plurality of attribute candidates different from the first attribute; and
executing the selected processing for the sound data associated with the scene on the sound data.

11. The data processing method according to claim 10 , further comprising:
obtaining the first determination result a score for each of the plurality of scene candidates,
multiplying the score for each of the plurality of scene candidates by a coefficient in the second case where the determined attribute of the content is the second attribute, and
selecting the processing to be applied to the sound data associated with the scene based on the scores multiplied by the coefficient, in the second selection method.

12. The data processing method according to claim 10 , further comprising:

Assignees

• Yamaha Corp

Inventors

• Yuyama Yuta
• Kumagai Kunihiro
• Aoki Ryotaro

Classifications

• H04N21/4852

  for modifying audio parameters, e.g. switching between mono and stereo · CPC title

• H04N21/4394

  involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58) · CPC title

• H04N5/147

  Scene change detection · CPC title

• H04S3/008

  in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02) · CPC title

• H04S7/30

  Control circuits for electronic adaptation of the sound field · CPC title