Data processing device and data processing method

Patent metadata

Field	Value
Publication number	US-11004460-B2
Application number	US-201916418164-A
Country	US
Kind code	B2
Filing date	May 21, 2019
Priority date	May 25, 2018
Publication date	May 11, 2021
Grant date	May 11, 2021

Abstract

A data processing device includes: a digital signal processor; at least one processor; and at least one memory device configured to store a plurality of instructions, which when executed by the at least one processor, cause the at least one processor to operate to: output a first determination result relating to a scene of content through use of sound data; select processing for the sound data by a first selection method based on the first determination result; determine an attribute of the content from among a plurality of attribute candidates; and select the processing by a second selection method, which is different from the first selection method, based on a determination result of the attribute, wherein the digital signal processor is configured to execute the processing selected by the at least one processor on the sound data.

First claim

What is claimed is:

1. A data processing device, comprising:
a digital signal processor;
at least one processor; and
at least one memory device configured to store a plurality of instructions, which when executed by the at least one processor, cause the at least one processor to operate to:
obtain, based on a scene determination model generated through a machine learning process, a first determination result relating to a scene included in a content through use of sound data associated with the scene, wherein the first determination result comprises a score for each of a plurality of scene candidates;
determine, based on a frequency analysis of the sound data, an attribute of the content, in which the scene is included, from among a plurality of attribute candidates; and
select processing to be applied to the sound data associated with the scene based on a determination result of the attribute such that:
(i) in a case where the determined attribute of the content is a first attribute among the plurality of attribute candidates, sound field processing corresponding to a scene candidate having the highest score among the plurality of scene candidates is selected as a first selection method, and
(ii) in a case where the determined attribute of the content is a second attribute among the plurality of attribute candidates different from the first attribute, sound field processing corresponding to one of the plurality of scene candidates except the scene candidate having the highest score among the plurality of scene candidates is selected as a second selection method different from the first selection method,
wherein the digital signal processor is configured to execute the selected sound field processing for the sound data associated with the scene on the sound data.

2. The data processing device according to claim 1 , wherein
the plurality of instructions cause the at least one processor to output the first determination result based on the scene determination model generated through the machine learning process relating only to a part of the plurality of attribute candidates.

3. The data processing device according to claim 2 , wherein
the first attribute is included in the part of the plurality of attribute candidates; and
the second attribute is not included in the part of the plurality of attribute candidates.

4. The data processing device according to claim 2 , wherein the plurality of instructions cause the at least one processor to extract a feature from the sound data associated with the scene, and perform classification based on the scene determination model, to thereby output the score for each of the plurality of scene candidates as the first determination result.

5. The data processing device according to claim 4 , wherein the plurality of instructions cause the at least one processor to select, in the second selection method, the sound field processing corresponding to one of the plurality of scene candidates that has a next highest score among the plurality of scene candidates after the scene candidate having the highest score among the plurality of scene candidates.

6. The data processing device according to claim 4 , wherein the plurality of instructions cause the at least one processor to multiply, in the second selection method, the score for each of the plurality of scene candidates by a coefficient based on the determination result of the attribute.

7. The data processing device according to claim 1 , wherein the plurality of instructions cause the at least one processor to select, in the second selection method, predetermined sound field processing based on the determination result of the attribute.

8. The data processing device according to claim 1 ,
wherein the digital signal processor is configured to apply an effect of the selected sound field processing to the sound data associated with the scene.

9. A data processing method, comprising:
obtaining, with at least one processor operating with a memory device in a device and based on a scene determination model generated through a machine learning process, a first determination result relating to a scene included in a content through use of sound data associated with the scene, wherein the first determination result comprises a score for each of a plurality of scene candidates;
determining, with the at least one processor operating with the memory device in the device and based on a frequency analysis of the sound data, an attribute of the content, in which the scene is included, from among a plurality of attribute candidates;
selecting, with the at least one processor operating with the memory device in the device, processing to be applied to the sound data associated with the scene based on a determination result of the attribute such that:
(i) in a case where the determined attribute of the content is a first attribute among the plurality of attribute candidates, sound field processing corresponding to a scene candidate having the highest score among the plurality of scene candidates is selected as a first selection method, and
(ii) in a case where the determined attribute of the content is a second attribute among the plurality of attribute candidates different from the first attribute, sound field processing corresponding to one of the plurality of scene candidates except the scene candidate having the highest score among the plurality of scene candidates is selected as a second selection method different from the first selection method; and
executing the selected sound field processing for the sound data associated with the scene on the sound data.

10. The data processing method according to claim 9 , further comprising outputting, with the at least one processor operating with the memory device in the device, the first determination result based on the scene determination model generated through the machine learning process relating only to a part of the plurality of attribute candidates.

11. The data processing method according to claim 10 , wherein:
the first attribute is included in the part of the plurality of attribute candidates; and
the second attribute is not included in the part of the plurality of attribute candidates.

12. The data processing method according to claim 10 , further comprising extracting, with the at least one processor operating with the memory device in the device, a feature from the sound data associated with the scene, and performing classification based on the scene determination model, to thereby output the score for each of the plurality of scene candidates as the first determination result.

13. The data processing method according to claim 12 , further comprising selecting, with the at least one processor operating with the memory device in the device, in the second selection method, the sound field processing corresponding to one of the plurality of scene candidates that has a next highest score among the plurality of scene candidates after the scene candidate having the highest score among the plurality of scene candidates.

14. The data processing method according to claim 12 , further comprising multiplying, with the at least one processor operating with the memory device in the device, in the second selection method, the score for each of the plurality of scene candidates by a coefficient based on the determination result of the attribute.

15. The data processing method according to claim 9 , further comprising selecting, with the at least one processor operating with the memory device in the device, in the second selection method, predetermined sound field processing based on the determ

Assignees

• Yamaha Corp

Inventors

• Yuyama Yuta
• Kumagai Kunihiro
• Aoki Ryotaro

Classifications

• H04S7/30

  Control circuits for electronic adaptation of the sound field · CPC title

• H04N21/439

  Processing of audio elementary streams · CPC title

• H04N21/4852

  for modifying audio parameters, e.g. switching between mono and stereo · CPC title

• G10L25/51 Primary

  for comparison or discrimination · CPC title

• H04S3/008

  in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02) · CPC title