Interactive digital displays

Patent metadata

Field	Value
Publication number	US-9665259-B2
Application number	US-201414302249-A
Country	US
Kind code	B2
Filing date	Jun 11, 2014
Priority date	Jul 12, 2013
Publication date	May 30, 2017
Grant date	May 30, 2017

Abstract

The description relates to an interactive digital display. One example includes a display device configured to receive user input and recognize commands relative to data visualizations. The system also includes a graphical user interface configured to be presented on the display device that allows users to interact with the data visualizations via the user commands.

First claim

The invention claimed is:

1. A device, comprising:
a screen;
a camera; and,
a processor configured to cause a graphical user interface to be presented on the screen and wherein the graphical user interface is configured to:
receive visual input from the camera, the visual input related to a non-touch user command performed by a user and detected by the camera,
interpret the visual input as a bump gesture indicating selection of a first data visualization and a second data visualization and movement of the first and second data visualizations toward one another to combine the first and second data visualizations,
responsive to interpreting the visual input as the bump gesture, without further input from the user beyond the bump gesture:
analyze data associated with the first and second data visualizations to automatically determine a logical way to process the data and automatically determine a format of a third data visualization, and
automatically generate the third data visualization with the automatically determined format, wherein the third data visualization represents a combination of the data from the first and second data visualizations.

2. The device of claim 1 , wherein the bump gesture performed by the user consists of the user selecting the first data visualization and the second data visualization with two hands or one or more fingers from each hand and simultaneously moving the two hands or the one or more fingers toward one another, and wherein the graphical user interface is further configured to show the first and second data visualizations simultaneously moving toward one another as the bump gesture is performed.

3. The device of claim 1 , wherein the graphical user interface is further configured to automatically generate the third data visualization by examining a first set of data from the first data visualization and a second set of data from the second data visualization for a common category.

4. The device of claim 3 , wherein the third data visualization is based at least in part on the common category.

5. The device of claim 1 , wherein the graphical user interface is configured to display the third data visualization.

6. The device of claim 1 , wherein the graphical user interface is further configured to:
receive another input from the camera,
interpret the another input as an extraction gesture to extract a subset of data from the third data visualization;
responsive to the extraction gesture, automatically create a fourth data visualization that displays the subset of data; and
display the fourth data visualization on the graphical user interface.

7. The device of claim 6 , wherein the extraction gesture is performed by the user in relation to one or more selectable objects in the third data visualization, the selectable objects representing the subset of data.

8. The device of claim 7 , wherein the fourth data visualization presents the subset of data in a different form than the third data visualization, and the fourth data visualization includes new objects that are individually selectable.

9. The device of claim 6 , wherein the fourth data visualization is displayed on the graphical user interface with the third data visualization.

10. A method implemented by one or more computing devices, the method comprising:
displaying a first image and a second image on a graphical user interface (GUI);
detecting a two-handed, non-touch user gesture relative to the first and second images, wherein a first hand indicates selection of the first image and a second hand indicates selection of the second image, and the two-handed, non-touch user gesture further comprises simultaneous independent actions performed by the user with the first hand and second hand relative to the first and second images;
analyzing data associated with the first and second images to determine a logical way to process the data based on the two-handed, non-touch user gesture and a type of the data;
creating a new image by processing the data according to the logical way; and
displaying the new image on the GUI.

11. The method of claim 10 , wherein the first and second images are pictures, and the new image created by processing the data according to the logical way includes at least one object from each of the pictures.

12. A system, comprising:
storage configured to store computer-readable instructions;
a processor configured to execute the computer-readable instructions;
a gesture recognition component configured to recognize a two-handed gesture entered by a user relative to selectable objects of a display of one or more visualizations of data, wherein the two-handed gesture comprises selecting a first individual selectable object with a first hand of the user and selecting a second individual selectable object with a second hand of the user, and the two-handed gesture further comprises an independent action performed by the user with the first hand relative to the first individual selectable object and a simultaneous independent action performed by the user with the second hand relative to the second individual selectable object;
a data analysis and processing component configured to:
analyze the data associated with the one or more visualizations in response to the two-handed gesture, and
determine a logical way to process at least some of the data from the one or more visualizations in accordance with the two-handed gesture; and,

a visualization rendering component configured to generate a new visualization that represents the at least some of the data from the one or more visualizations,
wherein the processor is configured to execute the computer-readable instructions associated with at least one of the gesture recognition component, the data analysis and processing component, or the visualization rendering component.

13. The system of claim 12 , wherein the system further comprises sensors that receive the two-handed gesture.

14. The system of claim 13 , wherein the sensors are touch sensors associated with the display.

15. The system of claim 14 , wherein the gesture recognition component is further configured to receive output from the touch sensors and derive the two-handed gesture from the output.

16. The system of claim 12 , wherein the visualization rendering component is further configured to communicate the new visualization as output to the display.

17. The system of claim 12 , wherein the system further comprises the display.

18. The system of claim 12 , wherein the one or more visualizations comprises multiple visualizations, and the new visualization represents combined data from the multiple visualizations.

19. The system of claim 12 , wherein the new visualization represents extracted data from the one or more visualizations.

20. The system of claim 19 , wherein the selectable objects are the one or more visualizations and the selectable objects are also elements within the one or more visualizations, the two-handed gesture is related to an individual visualization and an individual element, and the extracted data is associated with the individual element.

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Lee Bongshin
• Smith Greg
• Netz Amir
• Longley Matthew J
• Tran Allison
• Petculescu Cristian
• Prish Shahar
• Oppenheimer Diego
• Wilson Adam
• Baumgartner Patrick
• Rezaei Pedram Faghihi
• Forstrom Amy
• Megiddo Eran

Classifications

• G06T11/26

  Drawing of charts or graphs · CPC title

• G06F3/0486

  Drag-and-drop · CPC title

• G06F3/04845 Primary

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06F3/0488

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title