In-application video navigation system

Patent metadata

Field	Value
Publication number	US-2020380030-A1
Application number	US-201916428308-A
Country	US
Kind code	A1
Filing date	May 31, 2019
Priority date	May 31, 2019
Publication date	Dec 3, 2020
Grant date	—

Abstract

Embodiments of the present invention provide systems, methods, and computer storage media for in-app video navigation in which videos including answers to user provided queries are presented within an application. And portions of the videos that specifically include the answer to the query are highlighted to allow for efficient and effective tutorial utilization. Upon receipt of a text or verbal query, top candidate videos including an answer to the query are determined. Within the top candidate videos, a video span with a starting sentence location and an ending location is identified based on the query and contextual information within each candidate video. The video span with the highest overall score calculated based on a video score and a span score is presented to the user.

First claim

What is claimed is:

1 . One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations including:
receiving, via a workspace of an application, a query including a question related to the application, the workspace of the application presented within an application interface of the application; for a set of candidate videos, identifying a set of spans of the corresponding videos that include a potential answer to the question related to the application; determining an answer span from the identified set of spans, the answer span including a best potential answer to the question related to the application; and causing presentation of the answer span within the application interface of the application.

2 . The media of claim 1 , wherein the answer span is determined based on a best span score associated with each span of the set of spans.

3 . The media of claim 1 , wherein the answer span is determined based on an aggregate score associated with each span of the set of spans, the aggregate score being based on a candidate video score associated with a corresponding candidate video of the set of candidate videos and a best span score associated with a corresponding span of the set of spans.

4 . The media of claim 1 , wherein the set of candidate videos is selected from a video repository of application tutorials.

5 . The media of claim 1 , wherein the set of candidate videos is determined based on contextual information associated with the application.

6 . The media of claim 5 , wherein the contextual information includes at least one of past user commands, application status, user information, localization and geographical information.

7 . The media of claim 1 , wherein the set of candidate videos is selected based on a sentence segmentation of each candidate video of the set of candidate videos.

8 . The media of claim 1 , wherein each span of the set of spans includes a starting sentence and an ending sentence within a corresponding candidate video of the set of candidate videos.

9 . The media of claim 1 , wherein the operations further comprise:
generating, for each span of the set of spans, a span embedding; generating a question embedding based on the question; and determining a best span score for each span of the set of spans based on the corresponding span embedding and the question embedding.

10 . The media of claim 1 , wherein the answer span is caused to present in conjunction with the workspace of the application within the application interface.

11 . A computerized method for presenting a video including an answer to a query within an application, the method including:
receiving a query including a question related to the application, the application including contextual features; determining a video tutorial that includes a span of content having an answer to the question, the video tutorial determined based on the query and contextual features associated with the application; and presenting within the application, via a user interface, the video tutorial having the span, such that the video tutorial and a workspace of the application are presented simultaneously via the user interface.

12 . The method of claim 11 , wherein the query is received via voice command and the method further comprises a speech recognition technique to identify the query based on the voice command.

13 . The method of claim 12 , wherein the video further includes a table of contents associated with the video tutorial.

14 . The method of claim 12 , wherein the table of contents is associated with a timeline of the video tutorial.

15 . The method of claim 11 , wherein the span is defined as a portion of the video tutorial including the answer to the question, the portion of the video indicated by a starting location and an ending location of the answer to the question within a timeline of the video, where the starting location and the ending location are based on a location of the answer to the question in the video tutorial.

16 . The method of claim 15 , wherein the portion of the video is highlighted within the timeline of the video from the starting location to the ending location, and the timeline is presented on the user interface.

17 . The method of claim 16 , further comprising playing the video tutorial from the starting location.

18 . An in-application video navigation system comprising:
one or more hardware processors and memory configured to provide computer program instructions to the one or more hardware processors; an in-application video navigation environment configured to use the one or more hardware processors to:
generate a query embedding based on a received query including a question;
search, using the query, for candidate videos based on a video embeddings associated with a plurality of videos, each of the candidate videos including a potential answer to the question;
a means for identifying a span within each of the candidate videos, the span being most likely to include the potential answer to the question; and a means for identifying one of the spans associated with the candidate videos as an answer span, the answer span including a best answer to the question from the spans.

19 . The in-app video navigation system of claim 18 , the in-app video navigation system further comprising a means for causing presentation of the answer span via an interactive user interface, the answer span presented in conjunction with a workspace of an application associated with the in-application video navigation environment.

20 . The in-app navigation system of claim 19 , wherein the means for presenting the answer span via an interactive user interface may further cause presentation of a table of contents associated with candidate video associated with the answer span.

Assignees

• Adobe Inc

Inventors

• Colas Anthony Michael
• Kim Doo Soon
• Dernoncourt Franck
• Kim Seokhwan

Classifications

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G09B7/02

  of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student · CPC title

• G06F16/73 Primary

  Querying · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title