Computer with graphical user interface for interaction

Patent metadata

Field	Value
Publication number	US-9740364-B2
Application number	US-77286410-A
Country	US
Kind code	B2
Filing date	May 3, 2010
Priority date	May 3, 2010
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

Different techniques of processing user interactions with a computing system are described. In one implementation, an interactive display is configured to depict a graphical user interface which includes a plurality of different types of user interface elements (e.g., button-type element, scroll bar-type element). A user may use one or more user input object (e.g., finger, hand, stylus) to simultaneously interact with the interactive display. A plurality of different user input processing methods are used to process user inputs received by the graphical user interface differently and in accordance with the types of the user interface elements which are displayed. The processing of the user inputs is implemented to determine whether the user inputs control the respective user interface elements. The processing may determine whether the user inputs activate and/or manipulate the displayed user interface elements in but one example.

First claim

What is claimed is:

1. A user interaction method implemented by a system comprising:
receiving a user input image via a graphical user interface displayed on a display screen, wherein the graphical user interface includes a plurality of user interface elements, the user input image defining a plurality of pixel locations of the display screen, the plurality of pixel locations activated by one or more input objects interacting with the graphical user interface, the plurality of activated pixel locations defining one or more regions of activated pixel locations of the display screen; and
processing the user input image to determine whether the user input image controls at least one user interface element displayed in the graphical user interface, the processing including:
identifying a number of first pixels activated by the user input image and which are within an area defined by a geometric boundary of the user interface element,
weighting the number of first pixels to provide a first value,
identifying a number of second pixels which are proximate to the user interface element but outside the geometric boundary of the user interface element,
combining the number of first pixels without the weighting and the number of second pixels to provide a second value,
subtracting the second value from the first value to provide a third value, and
comparing the third value with a threshold to determine whether the user input image controls the user interface element.

2. The method of claim 1 , wherein the receiving comprises receiving the user input image resulting from a plurality of user input objects interacting with the graphical user interface at substantially the same moment in time.

3. The method of claim 1 further comprising processing the user input image to identify at least one of a plurality of different types of user input objects which interacted with the graphical user interface to create the user input image.

4. The method of claim 1 , wherein the processing comprises determining a first region comprising a plurality of pixels of the graphical user interface selected by the user input and comparing the first region with the at least one user interface element displayed in the graphical user interface.

5. The method of claim 1 , wherein the at least one user interface element comprises a first user interface element and a second user interface element in the graphical user interface, wherein the second user interface element is different from the first user interface element, and wherein a first input processing method corresponds to the first user interface element and a second input processing method corresponds to the second user interface element.

6. The method of claim 1 further comprising changing the at least one user interface element as a result of the processing determining that the user input image controls the at least one user interface elements which is displayed in the graphical user interface.

7. The method of claim 1 , wherein the at least one user interface element is a button-type element, and wherein the processing comprises:
identifying a number of first pixels activated by the user input image and which are within an area defined by a geometric boundary of the button-type element;
weighting the number of first pixels to provide a first value;
identifying a number of second pixels which are proximate to the button-type element but outside the geometric boundary of the button-type element;
combining the number of first pixels without the weighting and the number of second pixels to provide a second value;
subtracting the second value from the first value to provide a third value; and
comparing the third value with a threshold to determine whether the user input image controls the button-type element.

8. The method of claim 5 , wherein a second user interface element is a scroll bar-type element, and wherein the processing comprises identifying a motion vector of the user input image and comparing the motion vector with respect to the scroll bar-type element to determine whether the user input image controls the scroll bar-type element.

9. The method of claim 5 , wherein the first user interface element is a button-type element, and wherein the processing comprises identifying a number of pixels activated by the user input image and which are within an area corresponding to the button-type element and comparing the number of pixels with respect to a threshold to determine whether the user input image controls the button-type element.

10. The method of claim 1 , wherein each of the plurality of user interface elements corresponds to a type of a user interface element, and the method further comprising accessing information regarding the type of the at least one user interface element displayed in the graphical user interface.

11. A system comprising:
processing circuitry;
at least one computer-readable storage medium comprising programming stored thereon that is configured to cause the processing circuitry to perform processing comprising:
first processing a user input image received by a graphical user interface using a first user input processing method which corresponds to a first type of user interface element, the first processing including:
identifying a number of first pixels activated by the user input image and which are within an area defined by a geometric boundary of the user interface element,
weighting the number of first pixels to provide a first value,
identifying a number of second pixels which are proximate to the user interface element but outside the geometric boundary of the user interface element,
combining the number of first pixels without the weighting and the number of second pixels to provide a second value,
subtracting the second value from the first value to provide a third value, and
comparing the third value with a threshold to determine whether the user input image controls the user interface element;
second processing the user input image using a second user input processing method which is different than the first user input processing method and which corresponds to a second type of user interface element, the second type different from the first type;
using the first processing, determining whether the user input image controls one of the user interface elements that is displayed on the display screen, which is the first type of user interface element; and
using the second processing, determining whether the user input image controls another user interface element that is displayed on the display screen which is the second type of user interface element.

12. The system of claim 11 , wherein the programming is configured to cause the processing circuitry to perform processing that further comprises receiving the user input image resulting from a plurality of user input objects interacting with the graphical user interface at substantially the same moment in time.

13. The system of claim 11 , wherein the programming is configured to cause the processing circuitry to perform processing that further comprises processing the user input image to identify at least one of a plurality of different types of user input objects interacting with the graphical user interface.

14. The system of claim 11 , wherein the first processing comprises determining a region of a plurality of pixels of the graphical user interface selected by the user input image and comparing the region with the one of the user interface elements.

15. The system of claim 11 , wherein the programming is configured to cause the processing circuitry to perform processing that

Assignees

• Wilson Andrew D
• Microsoft Technology Licensing Llc

Inventors

• Wilson Andrew D

Classifications

• G06F3/0488

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/04812 Primary

  Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects · CPC title

• G06F2203/04808

  Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen · CPC title

• G06F3/0481

  based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance · CPC title

• G06F3/04855

  Interaction with scrollbars · CPC title