View playback to enhance collaboration and comments

Patent metadata

Field	Value
Publication number	US-11237848-B2
Application number	US-201916278616-A
Country	US
Kind code	B2
Filing date	Feb 18, 2019
Priority date	Feb 18, 2019
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

Collaborator content selection and manipulation actions and comments during interaction with a document are recorded to enhance asynchronous collaboration. The collaborator content selection and manipulation actions and comments are recorded during interaction with a document in a first collaboration viewport at a first time and played back in a second collaboration viewport at a second time while the same or a different instance of the document is displayed. The collaborator content selection and manipulation actions and comments include at least text selection, mouse or pen movements, and voice comments or ink annotations that are played back to recreate the collaborator changes and comments as though they were occurring in real-time, thereby creating a user experience that is more like the “over the shoulder” collaboration that would be available to collaborators that are at the same physical location.

First claim

What is claimed is:

1. A method of recording and playing back collaborator content selection and manipulation actions and comments during interaction with a document to enhance asynchronous collaboration, comprising:
storing a representation of a document as a string of primitive data structures and command operations expressing changes to the primitive data structures;
recording collaborator content selection and manipulation actions and comments during interaction with the document in a first collaboration viewport at a first time, the collaborator content selection and manipulation actions and comments including at least one of an audio or video recording and text selection, mouse or pen movements, or ink annotations formatted as strings of primitive data structures and command operations expressing changes to the primitive data structures;
inserting a recording of collaborator content selection and manipulation actions and comments during interaction with the document by at least one collaborator in the first collaboration viewport at the first time into the document as an interactable element, the interactable element being activatable by a collaborator to playback the at least one audio or video recording and the text selection, mouse or pen movements, or ink annotations during interaction with the document by the at least one collaborator in the first collaboration viewport at the first time; and
upon activation of the interactable element by the collaborator, retrieving selected strings of the primitive data structures and command operations expressing changes to the primitive data structures for the document for playback of the at least one audio or video recording while the command operations are executed for the text selection, mouse or pen movements or ink annotations during interaction with the document by the at least one collaborator to illustrate changes made to the document in a second collaboration viewport at a second time.

2. A method as in claim 1 , further comprising capturing a screen view and playing back the collaborator content selection and manipulation actions and comments while displaying the screen view in the second collaboration viewport.

3. A method as in claim 2 , wherein the collaborator content selection and manipulation actions and comments are recorded during interaction with the document in the first collaboration viewport for a first form factor on a first device, the method further comprising formatting the screen view for a second form factor on a second device to enable playing back the recorded collaborator content selection and manipulation actions while viewing the document in the second collaboration viewport on the second device.

4. A method as in claim 2 , wherein the collaborator content selection and manipulation actions and comments are recorded during interaction with the document in the first collaboration viewport in a first window, the method further comprising formatting the screen view for a second window to enable playing back the recorded collaborator content selection and manipulation actions while viewing the document in the second collaboration viewport in the second window.

5. A method as in claim 4 , further comprising playing back the recorded collaborator content selection and manipulation actions in the second window while viewing a second instance of the document.

6. A method as in claim 1 , wherein recording collaborator content selection and manipulation actions and comments during interaction with a document comprises transcribing a collaborator's voice as content selection and manipulation actions are being made by the collaborator, and wherein playing back the recorded collaborator content selection and manipulation actions while viewing the document comprises playing back the content selection and manipulation actions on the document as the transcribed collaborator's voice is played back.

7. A method as in claim 1 , further comprising playing back the recorded collaborator content selection and manipulation actions for a particular collaborator independent of content selection and manipulation actions of other collaborators after completion of editing at least a portion of the document.

8. A device that records and plays back collaborator content selection and manipulation actions and comments during interaction with a document to enhance asynchronous collaboration, comprising:
an instruction memory that stores instructions;
a memory that stores a representation of a document as a string of primitive data structures and command operations expressing changes to the primitive data structures; and
one or more processing devices that execute the instructions to perform operations including:
implementing a document editor that records collaborator content selection and manipulation actions and comments during interaction with the document in a first collaboration viewport at a first time and inserts a recording of collaborator content selection and manipulation actions and comments during interaction with the document by at least one collaborator in the first collaboration viewport at the first time into the document as an interactable element, the interactable element being activatable by a collaborator to playback at least one audio or video recording and a text selection, mouse or pen movements, or ink annotations during interaction with the document by the at least one collaborator in the first collaboration viewport at the first time, and the collaborator content selection and manipulation actions and comments including at least one of the audio or video recording and text selection, mouse or pen movements, or ink annotations formatted as strings of primitive data structures and command operations expressing changes to the primitive data structures; and
implementing a viewer that, upon activation of the interactable element by the collaborator, retrieves selected strings of the primitive data structures and command operations expressing changes to the primitive data structures for the document from the memory for playback of the at least one audio or video recording while the command operations are executed for the text selection, mouse or pen movements, or ink annotations during interaction with the document by the at least one collaborator to illustrate changes made to the document in a second collaboration viewport at a second time.

9. A system as in claim 8 , wherein the document editor captures a screen view and the viewer plays back the collaborator content selection and manipulation actions and comments while displaying the screen view in the second collaboration viewport.

10. A system as in claim 9 , wherein the document editor and viewer are included in a first device having a first form factor, the document editor records collaborator content selection and manipulation actions and comments during interaction with the document in the first collaboration viewport for the first form factor on the first device while the viewer adjusts the screen view for playing back recorded collaborator content selection and manipulation actions recorded in a second device having a second form factor for display on the first device with the first form factor.

11. A system as in claim 9 , wherein the document editor records the collaborator content selection and manipulation actions and comments during interaction with the document in the first collaboration viewport in a first window and the viewer formats the screen view for a second window to enable playing back the recorded collaborator content selection and manipulation actions while viewing the document in the second collaboration viewport in the second window.

12. A sys

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Fox Seth
• Kikin Gil Erez
• Costenaro Daniel P
• Jung Christopher Andrews
• Smith Benjamin D

Classifications

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F9/451 Primary

  Execution arrangements for user interfaces · CPC title

• G06Q10/103

  Workflow collaboration or project management · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F40/166 Primary

  Editing, e.g. inserting or deleting · CPC title