Manipulation of content on display surfaces via augmented reality

Patent metadata

Field	Value
Publication number	US-2019212901-A1
Application number	US-201815986572-A
Country	US
Kind code	A1
Filing date	May 22, 2018
Priority date	Jan 8, 2018
Publication date	Jul 11, 2019
Grant date	—

Abstract

In one embodiment, a system comprising camera(s) adapted to capture first image(s) including display surface(s), touchscreen(s) adapted to detect user input, and processor(s) adapted to display, simultaneously on the touchscreen(s), an augmented reality user interface including the first image(s), and other item(s) not captured by the camera(s), the other item(s) including at least one of: control item(s), or content item(s), interpret the user input detected by the touchscreen(s) to include selection of a first item in the first image(s), and a second content item, determine that the first item is a first display surface of the display surface(s) in the first image(s), and cause the second content item to be moved to, or duplicated to, or moved from, or duplicated from, the first display surface.

First claim

What is claimed is:

1 . A system comprising:
at least one camera adapted to capture at least one first image including one or more display surfaces; at least one touchscreen adapted to detect user input; and at least one processor adapted to:
display, simultaneously on the at least one touchscreen, an augmented reality user interface including:
the at least one first image; and
at least one other item not captured by the at least one camera, the at least one other item including at least one of: one or more control items, or one or more content items;

interpret the user input detected by the at least one touchscreen to include selection of a first item in the at least one first image, and a second content item;
determine that the first item is a first display surface of the one or more display surfaces in the at least one first image; and
cause the second content item to be moved to, or duplicated to, or moved from, or duplicated from, the first display surface.

2 . The system of claim 1 , wherein the system comprises a portable computer, the portable computer including the at least one camera, the at least one touchscreen, and at least one of the at least one processor.

3 . The system of claim 2 , wherein the at least one processor includes at least one other processor that is not comprised in the portable computer.

4 . The system of claim 1 , wherein the at least one camera is adapted to capture at least one second image after capturing the at least one first image, and wherein the second content item is moved from, or duplicated from, the first display surface, and moved to, or duplicated to, a second display surface, the second display surface being included in at least one of: the one or more display surfaces in the at least one first image, or the at least one second image.

5 . The system of claim 1 , wherein the one or more display surfaces includes a plurality of display surfaces.

6 . The system of claim 5 , wherein the plurality of display surfaces includes two or more types of display surfaces selected from a group of types comprising: telepresence unit, television, monitor, and projection surface.

7 . The system of claim 1 , wherein the at least one touchscreen is further adapted to detect further user input, and wherein the at least one processor is further adapted to interpret the further user input to include selection of the first item in the at least one first image, and the second content item; determine that the first item is the first display surface of the one or more display surfaces in the at least one first image; and cause resizing or deletion of the second content item that is being displayed on the first display surface.

8 . The system of claim 1 , wherein the one or more control items include at least one content control item, wherein the at least one touchscreen is further adapted to detect further user input, and wherein the at least one processor is further adapted to interpret the further user input to include selection of the first item in the at least one first image, and the second content item;
determine that the first item is the first display surface of the one or more display surfaces in the at least one first image; and cause at least one displaying property for displaying the second content item on the first display surface to be toggled.

9 . The system of claim 1 , wherein the at least one processor being adapted to interpret, includes the at least one processor being adapted to interpret the user input to include selection of the second content item from among the one or more content items, the second content item being moved from, or duplicated from, the at least one touchscreen.

10 . The system of claim 1 , wherein the at least one processor being adapted to interpret, includes the at least one processor being adapted to interpret the user input to include selection of content that is being displayed on the first display surface, and wherein the second content item is the content that is being displayed on the first display surface.

11 . The system of claim 1 , wherein the second content item is moved to, or duplicated to, the at least one touchscreen, and wherein the at least one processor is further adapted to display a focused user interface which includes additional content, in addition to the second content item.

12 . The system of claim 1 , wherein the at least one touchscreen is further adapted to detect further user input, wherein the at least one processor is further adapted to display a template user interface on the at least one touchscreen, the template user interface including a list of templates, to interpret the further user input to include selection of a template from the list, and to cause content to be positioned in accordance with the template.

13 . The system of claim 1 , wherein the at least one processor being adapted to determine, includes the at least one processor being adapted to determine that a ray representative of the user input, that is sent from at least one virtual camera representing the at least one camera in a virtual model, intersects a virtual display surface representing the first display surface in the virtual model.

14 . The system of claim 13 , wherein the at least one camera is further adapted to scan a physical environment, and wherein the at least one processor is further adapted to compute characteristics of display surfaces in the physical environment, including the one or more display surfaces, and to construct the virtual model.

15 . A method comprising:
displaying simultaneously, on at least one touchscreen, an augmented reality user interface including:
at least one first image captured by at least one camera, the at least one first image including one or more display surfaces; and
at least one other item that was not captured by the at least one camera, the at least one other item including at least one of: one or more control items, or one or more content items;
interpreting user input detected by the at least one touchscreen to include selection of a first item in the at least one first image, and a second content item; determining that the first item is a first display surface of the one or more display surfaces in the at least one first image; and causing the second content item to be moved to, or duplicated to, or moved from, or duplicated from, the first display surface.

16 . The method of claim 15 , wherein the method is performed by a portable computer.

17 . The method of claim 15 , wherein the second content item is moved from, or duplicated from, the first display surface, and moved to, or duplicated to, a second display surface, the second display surface being included in at least one of: the one or more display surfaces in the at least one first image, or at least one second image captured by the at least one camera after capturing the at least one first image.

18 . The method of claim 15 , wherein the one or more display surfaces includes a plurality of display surfaces, the plurality of display surfaces including two or more types of display surfaces.

19 . The method of claim 15 , further comprising:
displaying a template user interface, the template user interface including a list of templates; detecting further user input; interpreting the further user input to include selection of a template from the list; and causing content to be positioned in accordance with the template.

20 . A computer program product, comprising a non-transitory computer rea

Assignees

• Cisco Tech Inc

Inventors

• Garrison Daniel
• Sutton Joseph E
• Hart Michael
• Ebtekar Ali
• Sarbin Brian

Classifications

• G09G2354/00

  Aspects of interface with display user · CPC title

• G06F3/1423

  controlling a plurality of local displays, e.g. CRT and flat panel display · CPC title

• G09G5/12

  Synchronisation between the display unit and other units, e.g. other display units, video-disc players · CPC title

• G06F3/04815 Primary

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title