Actionable content displayed on a touch screen

Patent metadata

Field	Value
Publication number	US-11003349-B2
Application number	US-201816217513-A
Country	US
Kind code	B2
Filing date	Dec 12, 2018
Priority date	Sep 27, 2013
Publication date	May 11, 2021
Grant date	May 11, 2021

Abstract

Some implementations may present a media file that includes video on a touchscreen display. A user gesture performed on the touchscreen display may be detected. The user gesture may include one of a tap gesture, a swipe gesture, or a tap and hold and drag while holding gesture. Text selected by the user gesture may be determined. One or more follow-up actions may be performed automatically based at least partly on the text selected by the user gesture.

First claim

What is claimed is:

1. A system comprising:
one or more processors; and
a memory storing instructions that are executable by the one or more processors to perform operations comprising:
displaying one or more portions of a first image on a touchscreen display;
receiving, by the touchscreen display, input comprising a user gesture;
identifying selected text in the first image based on the user gesture;
determining that the selected text comprises a plurality of words included in the first image and a second image; and
automatically performing at least one follow-up action based at least partly on the selected text.

2. The system of claim 1 , further comprising:
determining that the user gesture comprises at least one of:
a tap and hold gesture; and
a drag while holding gesture.

3. The system of claim 2 , further comprising:
receiving, by the touchscreen display, a second input comprising a tap gesture; and
determining that the selected text in the first image comprises a word.

4. The system of claim 2 , further comprising:
receiving, by the touchscreen display, a third input comprising a swipe gesture; and
determining that the selected text comprises two or more words included in the first image.

5. The system of claim 2 , wherein:
the tap and hold gesture causes the first image to be selected; and
the drag while holding gesture causes selection of text in the first image and the second image to create the selected text.

6. The system of claim 1 , further comprising:
translating the selected text from a source language to a target language to create translated text.

7. The system of claim 1 , further comprising:
submitting a search query that includes the selected text to a search engine.

8. A computer implemented method comprising:
displaying one or more portions of a first image on a touchscreen display;
receiving, by the touchscreen display, input comprising a user gesture;
identifying selected text in the first image-based on the user gesture;
determining that the selected text comprises a plurality of words included in the first image and a second image; and
automatically performing at least one follow-up action based at least partly on the selected text.

9. The computer implemented method of claim 8 , further comprising:
determining that the user gesture comprises at least one of:
a tap and hold gesture; and
a drag while holding gesture.

10. The computer implemented method of claim 9 , further comprising:
receiving, by the touchscreen display, a second input comprising a tap gesture; and
determining that the selected text in the first image comprises a word.

11. The computer implemented method of claim 9 , further comprising:
receiving, by the touchscreen display, a third input comprising a swipe gesture; and
determining that the selected text comprises two or more words included in the first image.

12. The computer implemented method of claim 9 , wherein:
the tap and hold gesture causes the first image to be selected; and
the drag while holding gesture causes selection of text in the first image and the second image to create the selected text.

13. The computer implemented method of claim 8 , further comprising:
translating the selected text from a source language to a target language to create translated text.

14. The computer implemented method of claim 8 , further comprising:
submitting a search query that includes the selected text to a search engine.

15. The computer implemented method of claim 13 , further comprising:
submitting a search query that includes translated text to a search engine.

16. The computer implemented method of claim 8 , further comprising:
displaying results from automatically performing the at least one follow-up action in a pop-up window that at least partially overlays the first image on the touchscreen display.

17. A computing device comprising:
a touchscreen display;
one or more processors; and
a memory storing instructions that are executable by the one or more processors to perform operations comprising:
displaying one or more portions of a first image on a touchscreen display;
receiving, by the touchscreen display, input comprising a user gesture;
identifying selected text in the first image based on the user gesture;
determining that the selected text comprises a plurality of words included in the first image and a second image; and
automatically performing at least one follow-up action based at least partly on the selected text.

18. The computing device of claim 17 , the operations further comprising:
determining that the user gesture comprises at least one of:
a tap and hold gesture; and
a drag while holding gesture.

19. The computing device of claim 18 , the operations further comprising:
receiving, by the touchscreen display, a second input comprising a tap gesture; and
determining that the selected text in the first image comprises a word.

20. The computing device of claim 18 , the operations further comprising:
receiving, by the touchscreen display, a third input comprising a swipe gesture; and
determining that the selected text comprises two or more words included in the first image.

21. The computing device of claim 17 , the operations further comprising:
translating the selected text from a source language to a target language to create translated text; and
submitting, to a search engine, a search query that includes at least one of the text or the translated text.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Bai Peng
• Du Jun
• Sun Lei
• Huo Qiang

Classifications

• H04N21/4402 Primary

  involving reformatting operations of video signals for household redistribution, storage or real-time display {(details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764)} · CPC title

• G06F16/7844

  using original textual content or text extracted from visual content or transcript of audio data · CPC title

• H04N21/4622

  Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859) · CPC title

• H04N21/84

  Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} · CPC title

• G06F3/04883 Primary

  for inputting data by handwriting, e.g. gesture or text · CPC title