Acquiring public opinion and training word viscosity model

Patent metadata

Field	Value
Publication number	US-11610401-B2
Application number	US-202117206967-A
Country	US
Kind code	B2
Filing date	Mar 19, 2021
Priority date	Sep 30, 2020
Publication date	Mar 21, 2023
Grant date	Mar 21, 2023

Abstract

A public opinion acquisition method and device, a word viscosity model training method and device, a server, and a medium are provided in the present disclosure. And the present disclosure relates to the technical field of artificial intelligence, specifically to image recognition and natural language processing, which can be used in a cloud platform. A video public opinion acquisition method includes: receiving a public opinion acquisition request, the public opinion acquisition request including a public opinion keyword to be acquired; matching the public opinion keyword to be acquired with video data including a recognition result, wherein the recognition result is obtained by performing predefined content recognition on the video data, the predefined content recognition including text recognition and image recognition; and determining video data that matches with the public opinion keyword to be acquired as result video data.

First claim

The invention claimed is:

1. A public opinion acquisition method, comprising:
receiving a public opinion acquisition request, the public opinion acquisition request including a public opinion keyword to be acquired;
matching the public opinion keyword to be acquired with video data including a recognition result, wherein the recognition result is obtained by performing predefined content recognition on the video data, the predefined content recognition including text recognition and image recognition; and
determining video data that matches with the public opinion keyword to be acquired as result video data,
wherein the text recognition comprises:
acquiring a surrounding text of a video of the video data as text information, wherein the surrounding text includes one or more of: a title, a description text, and a speech text;
performing word segmentation processing on the text information;
inputting words obtained through the word segmentation processing into a trained word viscosity model, so as to obtain, for each word of the words, a probability of the word being connected with a latter word thereof;
filtering words having probabilities greater than a threshold probability; and
forming one or more key phrases using the words having the probabilities greater than the threshold probability as the recognition result.

2. The method according to claim 1 , wherein the performing the predefined content recognition on the video data comprises:
regularly acquiring source video data from a video source;
filtering the acquired source video data according to a predefined condition; and
transcoding the filtered source video data into a predefined format for the predefined content recognition.

3. The method according to claim 2 , wherein the predefined condition comprises one or more of: a video duration, a video type, and a publication date.

4. The method according to claim 1 , wherein the text recognition comprises:
extracting frames from a video of the video data to obtain a picture of each extracted frame;
recognizing a text in the picture as text information; and
extracting keywords from the text information as the recognition result.

5. The method according to claim 1 , wherein the forming one or more key phrases using the words having the probabilities greater than the threshold probability as the recognition result comprises:
obtaining an inverse document frequency of each word in a formed key phrase;
calculating a sum of the inverse document frequencies of all the words in the key phrase to be the inverse document frequency of the key phrase; and
selecting a predetermined number of key phrases with higher inverse document frequencies than other key phrase of the one or more key phrases as the recognition result.

6. The method according to claim 4 , further comprising:
performing an emotion analysis on the text information, the emotion analysis including analyzing one or more of positive emotion, neutral emotion or negative emotion; and
performing sensitivity recognition on the text information.

7. The method according to claim 1 , wherein the image recognition comprises face recognition, and the performing the predefined content recognition on the video data comprises:
extracting frames from a video of the video data to obtain a picture of each extracted frame;
recognizing a face in the picture; and
determining, based on a face database, a name corresponding to the face.

8. The method according to claim 1 , wherein the image recognition comprises scenario recognition, entity recognition and identity recognition, and the performing the predefined content recognition on the video data comprises:
extracting frames from a video of the video data to obtain a picture of each extracted frame;
recognizing a scenario in the picture;
recognizing an entity in the picture; and
recognizing an identity in the picture.

9. The method according to claim 1 , wherein the public opinion acquisition request further comprises a public opinion keyword to be filtered out, and the determining the video data that matches with the public opinion keyword to be acquired comprises:
filtering the video data that includes the recognition result based on the public opinion keyword to be filtered out; and
determining the filtered video data as the result video data.

10. The method according to claim 1 , wherein the trained word viscosity model was trained by performing steps comprising:
performing word segmentation on a text corpus to obtain a plurality of word pairs as a training sample, a word pair of the plurality of word pairs including a previous word and a latter word;
training the word viscosity model based on the training sample to cause the word viscosity model to output, for each word pair of the plurality of word pairs, a probability that the previous word and the latter word of the word pair form a key phrase; and
training the word viscosity model by gradient descent, until the word viscosity model meets a preset condition, wherein the preset condition includes a preset precision or a preset number of training.

11. The method according to claim 10 , wherein the performing word segmentation on the text corpus to obtain the plurality of word pairs comprises:
performing word segmentation processing on the text corpus;
setting a latter word window, wherein the latter word window represents a number of words that can form word pairs with previous words and are behind positions of the previous words in the text corpus;
for each word in the latter word window, setting a probability that the word in the latter word window form a word pair with a previous word, wherein the probability decreases in turn according to an order in the text corpus; and
acquiring the word pairs, as the training sample, according to the latter word window and the probability.

12. The method according to claim 10 , wherein the training the word viscosity model based on the training sample comprises:
respectively converting the previous words and the latter words in the input word pairs into vector data;
calculating a cosine similarity between the converted vector data; and
converting the cosine similarity into the probability.

13. A server, comprising:
a processor; and
a memory storing programs, the programs comprising instructions, which, when executed by the processor, cause the processor to carry out acts, including:
receiving a public opinion acquisition request, the public opinion acquisition request including a public opinion keyword to be acquired;
matching the public opinion keyword to be acquired with video data including a recognition result, wherein the recognition result is obtained by performing predefined content recognition on the video data, the predefined content recognition including text recognition and image recognition; and
determining video data that matches with the public opinion keyword to be acquired as result video data,
wherein the text recognition comprises:
acquiring a surrounding text of a video of the video data as text information, wherein the surrounding text includes one or more of: a title, a description text, and a speech text;
performing word segmentation processing on the text information;
inputting words obtained through the word segmentation processing into a trained word viscosity model, so as to obtain, for each word of the words, a probability of the word being connected with a latter word thereof;
filtering words having probabilities greater than a threshold probability; and
forming one or more key phrases using the words having the probabilities greater than the threshold probability as the re

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Zhang Wenjun
• Huang Qiang
• Fang Jun
• Pan Xu
• Li Yuncong
• Yang Zhe
• Hong Saiding

Classifications

• G06V20/43 Primary

  of news video content · CPC title

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• G06F16/784

  the detected or recognised objects being people · CPC title

• G06V20/635

  Overlay text, e.g. embedded captions in a TV programme · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title