Enhanced video interaction for a user interface of a telepresence network

Patent metadata

Field	Value
Publication number	US-9571789-B2
Application number	US-201514747839-A
Country	US
Kind code	B2
Filing date	Jun 23, 2015
Priority date	Nov 26, 2012
Publication date	Feb 14, 2017
Grant date	Feb 14, 2017

Abstract

A telepresence device may relay video, audio, and/or measurement data to a user operating a control device. A user interface may permit the user to quickly view and/or understand temporally and/or spatially disparate information. The telepresence device may pre-gather looped video of spatially disparate areas in an environment. A temporal control mechanism may start video playback at a desired point in a current or historical video segment. Notations may be associated with time spans in a video and recalled by capturing an image similar to a frame in the time span of the video. An area of interest may be selected and video containing the area of interest may be automatically found. Situational data may be recorded and used to recall video segments of interest. The telepresence device may synchronize video playback and movement. A series of videos may be recorded at predetermined time intervals to capture visually trending information.

First claim

The invention claimed is:

1. A non-transitory computer-readable storage medium including computer-readable instruction code for performing a method of generating pre-session video loops, the method comprising:
receiving, at a telepresence device that includes a camera, a display device, a microphone, and a speaker, an indication to navigate to an indicated location;
adjusting a camera on the telepresence device to identify one or more areas of interest at the indicated location;
sequentially targeting each area of interest with the camera;
recording video of each area of interest for a predetermined time period; and
transmitting the recorded video to a remotely located control device, wherein the control device can establish a two-way audio/video communication session with the telepresence device.

2. The non-transitory computer-readable storage medium of claim 1 , wherein recording video of each area of interest comprises continuously cycling through the one or more areas of interest until receiving a connection request from the remotely located control device.

3. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more areas of interest contain an object selected from the group consisting of a patient's face, an EKG monitor, a chart, and a fluid bag.

4. The non-transitory computer-readable storage medium of claim 1 , wherein the method further comprises recording a time code indicating when the video of the one or more areas of interest was recorded.

5. A non-transitory computer-readable storage medium including computer-readable instruction code for performing a method of providing segmented temporal video control of video received from a telepresence device that includes a camera, a display device, a microphone, and a speaker, the method comprising:
receiving, at a control device, streaming video from the telepresence device;
determining a plurality of video segments corresponding to a plurality of telepresence device activities;
displaying a first video segment associated with a first telepresence device activity to a user; and
providing a temporal control mechanism to the user permitting the user to select a first playback point within the first video segment, wherein the control device can establish a two-way audio/video communication session with the telepresence device.

6. The non-transitory computer-readable storage medium of claim 5 , wherein the method further comprises receiving an indication of a desired video segment from the user.

7. The non-transitory computer-readable storage medium of claim 5 , wherein the method further comprises: receiving an indication of a desired time span from the user, wherein none of the plurality of determined video segments wholly correspond to the desired time span; determining a desired video segment corresponding to the desired time span; and displaying the desired video segment to the user.

8. The non-transitory computer-readable storage medium of claim 7 , wherein the method further comprises receiving an indication from the user to play the desired video segment in a loop.

9. A non-transitory computer-readable storage medium including computer-readable instruction code for performing a method of annotating video segments received from a telepresence device that includes a camera, a display device, a microphone, and a speaker, the method comprising:
receiving, at a control device, streaming video from the telepresence device;
displaying a video segment from the streaming video to a user;
receiving an indication of a desired time span from the user;
receiving an annotation corresponding to the desired time span from the user;
storing the received annotation in association with the desired time span; and
displaying an interactive indication in a location corresponding to the desired time span, wherein the user may access the received annotation using the interactive indication, wherein the control device can establish a two-way audio/video communication session with the telepresence device.

10. The non-transitory computer-readable storage medium of claim 9 , wherein receiving an indication of a desired time span comprises: receiving a user indicated start time; and automatically selecting an end time based on a default time span length.

11. The non-transitory computer-readable storage medium of claim 9 , wherein receiving an annotation comprises: receiving a user indicated title and a user indicated note; and automatically inserting an author name into the annotation, wherein the author name corresponds to the user.

12. The non-transitory computer-readable storage medium of claim 9 , wherein the method further comprises: receiving a drawn object to be overlaid on the video segment during the desired time span; and displaying the drawn object overlaid on the video segment when the desired time span is displayed.

13. A non-transitory computer-readable storage medium including computer-readable instruction code for performing a method of recalling annotations based on a captured image, the method comprising:
receiving, at a control device, video from a telepresence device that includes a camera, display device, a microphone, and a speaker;
receiving an annotation from a first user corresponding to a frame in the video;
storing first location data corresponding to the frame;
storing the annotation;
receiving an image from a second device at a later time;
identifying second location data corresponding to the image;
retrieving the first location data and the annotation; and
displaying at least a portion of the annotation if the second location data is within a predetermined threshold of the first location data, wherein the control device can establish a two-way audio/video communication session with the telepresence device.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the first location data comprises at least one data element selected from the group consisting of a room number, a position of the telepresence device, an orientation of the telepresence device, a camera tilt, a camera pan, a camera zoom, and a set of image descriptors, and wherein the first location data corresponds to the frame in the video.

15. The non-transitory computer-readable storage medium of claim 13 , wherein the first location data comprises general position information and a set of image descriptors.

16. The non-transitory computer-readable storage medium of claim 13 , wherein receiving an annotation comprises receiving an annotation corresponding to an object in the frame, and wherein displaying comprises displaying at least the portion of the received annotation if a second set of image descriptors are within the predetermined threshold of a first set of image descriptors corresponding to the object.

17. The non-transitory computer-readable storage medium of claim 13 , wherein the telepresence device comprises a robotic platform, wherein the first location data includes position and orientation data determined by a telepresence device component, and wherein inverse kinematics is used to compare the first location data to the second location data.

18. A non-transitory computer-readable storage medium including computer-readable instruction code for performing a method of spatially searching video, the method comprising:
receiving, at a control device, the video from a telepresence device that includes a camera, a display device, a microphone, and a speaker;
storing the video;
receiving an indication of a spatial area of interest in a frame of the video

Assignees

• Intouch Tech Inc

Inventors

• Pinter Marco
• Jordan Charles S
• Sanchez Daniel
• Hanrahan Kevin
• Temby Kelton
• Lambrecht Christopher

Classifications

• G11B27/34

  Indicating arrangements  {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} · CPC title

• H04N7/142

  Constructional details of the terminal equipment, e.g. arrangements of the camera and the display · CPC title

• H04N7/185

  from a mobile camera, e.g. for remote control · CPC title

• G16H40/67

  for remote operation · CPC title

• H04N5/272

  Means for inserting a foreground image in a background image, i.e. inlay, outlay · CPC title