Enhanced target selection for a touch-based input enabled user interface

Patent metadata

Field	Value
Publication number	US-10684768-B2
Application number	US-201113274253-A
Country	US
Kind code	B2
Filing date	Oct 14, 2011
Priority date	Oct 14, 2011
Publication date	Jun 16, 2020
Grant date	Jun 16, 2020

Abstract

Systems and methods for providing graphical user interface elements optimized for touch-based input in connection with an application that is designed for conventional input received from a keyboard and a mouse. The touch-based graphical user interface elements are displayed when the computer system detects that a user is about to switch from conventional input devices to touch-sensitive input devices. The touch-based graphical user interface elements are hidden when the user provides input with the conventional input devices such that the touch-based graphical user interface elements do not distract from the applications normal operation. The display device includes a sensing capability that enables the computer system to detect when an object, such as a user's finger or a stylus, is proximate to, but not in contact with, the display device.

First claim

What is claimed is:

1. A computer-implemented method for displaying one or more touch-based graphical user interface (GUI) elements, the method comprising:
executing an application to cause a GUI to be displayed on a touch screen display associated with a computing device, wherein the GUI includes a plurality of conventional GUI elements in a conventional toolbar and configured to interact with at least one of a keyboard or a mouse device;
detecting a first location of an object that is located proximate to, but not yet in contact with, a surface of the touch screen display, wherein the first location comprises a first x-coordinate, a first y-coordinate, and a first z-coordinate;
detecting a second location of the object that is located proximate to, but not yet in contact with, the surface of the touch screen display, wherein the second location comprises a second x-coordinate, a second y-coordinate, and a second z-coordinate;
calculating a motion vector associated with the object based on the first location and the second location, wherein the motion vector indicates a trajectory of the object relative to the surface of the touch screen display;
calculating a target location of an intersection point of the motion vector with the surface;
determining a center of expansion for a touch-based toolbar based on the target location, wherein the center of expansion defines an anchor location for the touch-based toolbar relative to the conventional toolbar displayed within the GUI, and wherein the center of expansion is associated with a predefined point for a group of GUI elements within the conventional toolbar;
determining a first percentage of the one or more touch-based GUI elements to display left of the center of expansion based on a horizontal position of the center of expansion;
determining a second percentage of the one or more touch-based GUI elements to display right of the center of expansion based on the horizontal position of the center of expansion;
displaying, based on the target location, the center of expansion, the first percentage, and the second percentage, the one or more touch-based GUI elements on top of the plurality of conventional GUI elements;
after displaying the one or more touch-based GUI elements, determining that a user intends to provide input to the computing device via the at least one of the keyboard or the mouse device; and
in response, hiding the one or more touch-based GUI elements,
wherein each touch-based GUI element included in the one or more touch-based GUI elements is configured to perform a function associated with a different conventional GUI element included in the plurality of conventional GUI elements.

2. The method of claim 1 , wherein the one or more touch-based GUI elements are displayed on top of the GUI in response to detecting that the object is within a threshold distance from the surface.

3. The method of claim 1 , wherein the first x-coordinate is different than the second x-coordinate, the first y-coordinate is different than the second y-coordinate, and the first z-coordinate is different than the second z-coordinate.

4. The method of claim 1 , further comprising detecting a third location of the object that is located proximate to, but not yet in contact with, the surface of the touch screen display, wherein the third location comprises a third x-coordinate, a third y-coordinate, and a third z-coordinate, and wherein the motion vector is calculated based on the first location, the second location, and the third location.

5. The method of claim 1 , wherein the touch-based toolbar includes the one or more touch-based GUI elements.

6. The method of claim 1 , wherein the center of expansion for the touch-based toolbar is located at the target location.

7. A non-transitory computer-readable storage medium containing a program that, when executed by a processor, performs an operation for displaying one or more touch-based graphical user interface (GUI) elements, the operation comprising:
executing an application to cause a GUI to be displayed on a touch screen display associated with a computing device, wherein the GUI includes a plurality of conventional GUI elements in a conventional toolbar and configured to interact with at least one of a keyboard or a mouse device;
detecting a first location of an object that is located proximate to, but not yet in contact with, a surface of the touch screen display, wherein the first location comprises a first x-coordinate, a first y-coordinate, and a first z-coordinate;
detecting a second location of the object that is located proximate to, but not yet in contact with, the surface of the touch screen display, wherein the second location comprises a second x-coordinate, a second y-coordinate, and a second z-coordinate;
calculating a motion vector associated with the object based on the first location and the second location, wherein the motion vector indicates a trajectory of the object relative to the surface of the touch screen display;
calculating a target location of an intersection point of the motion vector with the surface;
determining a center of expansion for a touch-based toolbar based on the target location, wherein the center of expansion defines an anchor location for the touch-based toolbar relative to the conventional toolbar displayed within the GUI, and wherein the center of expansion is associated with a predefined point for a group of GUI elements within the conventional toolbar;
determining a first percentage of the one or more touch-based GUI elements to display left of the center of expansion based on a horizontal position of the center of expansion;
determining a second percentage of the one or more touch-based GUI elements to display right of the center of expansion based on the horizontal position of the center of expansion;
displaying, based on the target location, the center of expansion, the first percentage, and the second percentage, the one or more touch-based GUI elements on top of the plurality of conventional GUI elements;
after displaying the one or more touch-based GUI elements, determining that a user intends to provide input to the computing device via the at least one of the keyboard or the mouse device; and
in response, hiding the one or more touch-based GUI elements,
wherein each touch-based GUI element included in the one or more touch-based GUI elements is configured to perform a function associated with a different conventional GUI element included in the plurality of conventional GUI elements.

8. The non-transitory computer-readable storage medium of claim 7 , wherein the one or more touch-based GUI elements are displayed on top of the GUI in response to detecting that the object is within a threshold distance from the surface.

9. A system for displaying one or more touch-based graphical user interface (GUI) elements, the system comprising:
a touch screen display;
a memory that includes an application that causes a GUI to be displayed on the touch screen display, wherein the GUI includes a plurality of conventional GUI elements in a conventional toolbar and configured to interact with at least one of a keyboard or a mouse device; and
a processor configured to:
detect a first location of an object that is located proximate to, but not yet in contact with, a surface of the touch screen display, wherein the first location comprises a first x-coordinate, a first y-coordinate, and a first z-coordinate;
detect a second location of the object that is located proximate to, but not yet in contact with, the surface of the touch screen display, wherein the second location comprises a second x-coordinate, a second y-coordinate, and a second z-coordinate;
calculate a motion vector associated

Assignees

• Grossman Tovi
• Fitzmaurice George
• Yang Xing Dong
• Irani Pourang Polad
• Autodesk Inc

Inventors

• Grossman Tovi
• Fitzmaurice George
• Yang Xing-Dong
• Irani Pourang Polad

Classifications

• G06F3/0488 Primary

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title