Drag-and-drop on a mobile device

Patent metadata

Field	Value
Publication number	US-9606716-B2
Application number	US-201414522927-A
Country	US
Kind code	B2
Filing date	Oct 24, 2014
Priority date	Oct 24, 2014
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

Implementations provide an improved drag-and-drop operation on a mobile device. For example, a method includes identifying a drag area in a user interface of a first mobile application in response to a drag command, identifying an entity from a data store based on recognition performed on content in the drag area, receiving a drop location associated with a second mobile application, determining an action to perform in the second mobile application based on the drop location, and performing the action in the second mobile action using the entity. Another method may include receiving a selection of a smart copy control for a text input control in a first mobile application, receiving a selected area of a display generated by a second mobile application, identifying an entity in the selected area, automatically navigating back to the text input control, and pasting a description of the entity in the text input control.

First claim

What is claimed is:

1. A method comprising:
identifying, on a touchscreen display, a drag area in a user interface of a first mobile application in response to a drag command from a user of a mobile device;
identifying an entity from a graph-based data store based on recognition performed on content in the drag area;
receiving a drop location associated with a second mobile application, the second mobile application differing from the first mobile application;
determining an action to be performed in the second mobile application based on the drop location; and
performing the action in the second mobile application using the entity.

2. The method of claim 1 , wherein the action includes sharing the entity via a new message when the second mobile application includes a communications application.

3. The method of claim 1 , wherein the action includes initiating a search using a description of the entity when the second mobile application has a search user interface.

4. The method of claim 1 , further comprising identifying the drop in response to a voice command that identifies a user interface of the second mobile application.

5. The method of claim 1 , wherein the drop location includes an icon that launches the second mobile application.

6. The method of claim 5 , wherein the action includes launching a particular user interface in the second mobile application based on a type of the entity.

7. The method of claim 1 , further comprising:
determining that the drop location includes a text input control,
wherein performing the action includes pasting a text description for the entity from the graph in the text input control.

8. The method of claim 1 , further comprising:
displaying an indication of the drag area from the first mobile application on the display while the user navigates to the drop location in the second mobile application.

9. The method of claim 1 , wherein the entity is a first entity and the method further comprises:
storing the first entity and the second mobile application in a data store;
identifying at least a second entity from the graph in a second drag area in response to a second drag command from the user;
determining that an entity type for the second entity matches an entity type for the first entity; and
based on the determination, providing the second mobile application as a suggested drop location.

10. The method of claim 1 , further comprising:
sending the drag area to a server, the server identifying the at least one entity.

11. The method of claim 1 , wherein the drag area includes an image and the entity is recognized in the image.

12. A mobile device comprising:
a display;
at least one processor; and
memory storing instructions that, when executed by the at least one processor, cause the mobile device to:
identify a drag area in a user interface of a first mobile application in response to a drag command from a user of the mobile device;
determine, based on recognition performed on content in the drag area, that the drag area includes at least one entity from a graph-based data store storing a graph having entities as nodes of the graph;
receive a drop location;
determine the drop location is associated with a contact entity; and
automatically initiate a communication to the contact entity, the communication including information about the entity from the graph.

13. The mobile device of claim 12 , wherein the drop location is identified in a voice command that identifies the contact entity.

14. The mobile device of claim 12 , wherein determining that the drop location is associated with a contact entity includes:
determine that the drop location occurs on an email address,
wherein the communication is an email message.

15. The mobile device of claim 14 , wherein the mobile device includes a camera that generates a picture preview user interface and wherein the email address appears in the picture preview user interface.

16. The mobile device of claim 12 , wherein determining that the drop location is associated with a contact entity includes:
determine that the drop location occurs on a social media identifier,
wherein the communication includes sharing the entity with the social media identifier via a social media application.

17. The mobile device of claim 12 , wherein the mobile device includes a camera that generates a picture preview user interface and the drag area appears in the picture preview user interface.

18. The mobile device of claim 12 , wherein the memory further stores instructions that, when executed by the at least one processor, cause the mobile device to:
display an indication of the drag area on the display while the user navigates to the drop location.

19. The mobile device of claim 12 , wherein the drag area is image-based and the memory further stores instructions that, when executed by the at least one processor, cause the mobile device to:
search for a public URL that matches the drag area; and
provide the public URL in the communication.

20. The mobile device of claim 12 , wherein the drag area is image-based and the memory further stores instructions that, when executed by the at least one processor, cause the mobile device to:
upload the drag area to a server;
receive a URL from the server, the URL being a location of the drag area on the server; and
provide the URL in the communication.

21. The mobile device of claim 12 , wherein determining that the drag area includes the at least one entity includes:
sending the drag area to a server, the server storing the graph-based data store; and
receiving information about the entity from the server.

22. A method comprising:
receiving a selection of a smart copy control for a text input control on a mobile device, the text input control being in a first mobile application;
receiving a selected area of a display generated by a second mobile application;
identifying at least one entity of a graph-based data store in the selected area based on recognition performed on content of the selected area;
automatically navigating back to the text input control in the first mobile application; and
pasting a text description of the entity from the graph in the text input control.

23. The method of claim 22 , further comprising:
providing at least a second entity as a suggestion, an entity type of the second entity being compatible with the first mobile application.

24. The method of claim 22 , further comprising:
providing a third mobile application as a suggestion, the third mobile application selected based on prior smart copy operations.

25. The method of claim 22 , wherein the entity is identified from an image in the selected area.

Assignees

• Google Inc

Inventors

• Sharifi Matthew
• Petrou David

Classifications

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/0488

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F9/543

  User-generated data transfer, e.g. clipboards, dynamic data exchange [DDE], object linking and embedding [OLE] · CPC title

• G06F3/0486 Primary

  Drag-and-drop · CPC title