Analysis of content written on a board

Patent metadata

Field	Value
Publication number	US-10062384-B1
Application number	US-201815897212-A
Country	US
Kind code	B1
Filing date	Feb 15, 2018
Priority date	May 25, 2017
Publication date	Aug 28, 2018
Grant date	Aug 28, 2018

Abstract

A computer-implemented method for analyzing content written on a board, on which a text and/or a drawing are made, is disclosed. The method includes obtaining content data including a series of images, which captures content being written on the board. The method also includes obtaining utterance data representing a series of utterances, which is associated with the series of the images. The method further includes extracting a section from the series of the utterances based on a change in topics and recognizing a content block for the section from the content data. The content block includes one or more content parts written during the section. The method includes further calculating evaluation value for the content block by using one or more utterances included in the section.

First claim

What is claimed is:

1. A computer-implemented method for analyzing content written on a board, the method comprising:
obtaining content data including a series of images, wherein the series of the images captures content written on the board;
obtaining utterance data representing a series of utterances, wherein the series of the utterances is associated with the series of the images;
extracting a section from the series of the utterances based on a change in topics;
recognizing a content block for the section from the content data, wherein the content block includes one or more content parts written during the section;
calculating an evaluation value for the content block by using one or more utterances included in the section, wherein the content data is a video data having audio data associated therewith, wherein the video data is generated by capturing the board on which a text is made and the one or more utterances is spoken by one or more speakers, wherein the obtaining of the utterance data further comprising converting from the audio data to the utterance data in textual form by using speech-to-text conversion;
identifying a speaker for each of the one or more utterances from a corresponding audio segment in the audio data by using given voice print data for one or more registered speakers;
wherein the calculating comprises calculating a density of discussion for the section based on an amount of the one or more utterances included in the section and a time length of the section, wherein the density of the discussion is used for calculating the evaluation value, wherein the calculating comprises obtaining one or more tones for the section by using a result of a linguistic analysis applied to the one or more utterances included in the section, wherein the one or more tones are used for calculating the evaluation value, wherein the calculating comprises counting a number of the respective speakers involved in one or more communications in the section, wherein the number of the respective speakers is used for calculating the evaluation value, wherein each speaker of the respective speakers is associated with a social attribute and the evaluation value is calculated by using the social attribute of the speaker for each of the one or more utterances in the section, wherein the utterance data is converted from audio data associated with the content data;
detecting the change in the topics, wherein the change is identified by using a result of a taxonomy classification applied to the utterance data;
performing the extracting, the recognizing, and the calculating two or more times for each section so as to obtain a plurality of sets of the evaluation value and the content block;
reading a preceding image and a subsequent image from the video data;
calculating a difference between the preceding and the subsequent images as an image segment;
merging a first image segment with a second image segment, associated with same section, to obtain the one or more content parts for the section, wherein the first image segment, at least partially, overlaps the second image segment;
defining a region encompassing the one or more content parts associated with the section;
recognizing one or more content elements based on an image corresponding to the region, wherein each content element is selected from the group consisting of a character, a string, a graphic, and a picture;
creating, for the section, a graphical object having the one or more content elements arranged therein, wherein the one or more content elements of the graphical object is configured to be editable;
storing an analysis result including the graphical object; and
creating a graphical representation where a difference in the evaluation value is represented by a visual expression, wherein the visual expression makes a difference in an appearance of the one or more content parts included in each content block.

Assignees

• IBM

Inventors

• Sugiura Tomonori
• MOCHIZUKI Tomoka
• WEN Lianzi
• Sato Munehiko

Classifications

• G10L15/26 Primary

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L21/16

  Transforming into a non-visible representation (devices or methods enabling ear patients to replace direct auditory perception by another kind of perception A61F11/04) · CPC title

• G10L17/22

  Interactive procedures; Man-machine interfaces · CPC title

• G06F40/30

  Semantic analysis · CPC title