Systems and methods for context-based optical character recognition

Patent metadata

Field	Value
Publication number	US-11501736-B2
Application number	US-202017065837-A
Country	US
Kind code	B2
Filing date	Oct 8, 2020
Priority date	Nov 7, 2019
Publication date	Nov 15, 2022
Grant date	Nov 15, 2022

Abstract

Methods, systems, and apparatus, including computer programs stored on computer-readable media, for displaying contextually relevant information, comprising receiving cursor data comprising a location of a cursor on an electronic display, and determining a screenshot of at least a portion of the electronic display. One or more proximate alphanumeric characters may be determined in at least a portion of the screenshot based on the location of the cursor, and at least one of the proximate alphanumeric characters may be matched with one or more terms from a predetermined list of terms. An information card may be caused to be displayed on the electronic display based on the location of the cursor, the information card corresponding to the one or more terms from the predetermined list of terms.

First claim

What is claimed is:

1. A method for displaying contextually relevant information, comprising:
receiving cursor data comprising a location of a cursor on an electronic display;
determining a screenshot of at least a portion of the electronic display;
determining one or more proximate alphanumeric characters in at least a portion of the screenshot based on the location of the cursor;
determining one or more contextual terms associated with at least one of the proximate alphanumeric characters, the one or more contextual terms being within a threshold proximity of the at least one of the proximate alphanumeric characters;
determining a semantic meaning of the at least one of the proximate alphanumeric characters based on at least one of the contextual terms, the semantic meaning based on a semantic classification;
matching the at least one of the proximate alphanumeric characters with one or more terms from a predetermined list of terms;
determining one or more indicators based on the matching the at least one of the proximate alphanumeric characters with one or more terms from a predetermined list of terms; and
causing an information card to be displayed on the electronic display based on the location of the cursor, the information card corresponding to the one or more terms from the predetermined list of terms, the one or more indicators, and the semantic meaning.

2. The method of claim 1 , wherein the screenshot comprises a subportion of the electronic display located within a predetermined radius of the cursor.

3. The method of claim 1 , wherein a size of the screenshot comprises a predetermined subportion of the electronic display, the predetermined subportion of the electronic display being based on the location of the cursor.

4. The method of claim 1 , further comprising:
upon determining that the one or more proximate alphanumeric characters in at least the portion of the screenshot are above a predetermined size threshold, determining a second screenshot of at least a portion of the electronic display, the second screenshot being larger than the screenshot.

5. The method of claim 1 , wherein the screenshot is determined when a predetermined event is detected.

6. The method of claim 1 , wherein the screenshot is determined when a predetermined event is detected, the predetermined event comprising a hover event of the cursor.

7. The method of claim 1 , wherein determining a screenshot further comprises:
determining an active window, the active window comprising a currently focused application window on the electronic display, wherein the screenshot excludes any portion of the electronic display that is not the active window.

8. The method of claim 1 , wherein determining one or more proximate alphanumeric characters in at least a portion of the screenshot comprises:
determining if a background of at least the portion of the screenshot is lighter than or darker than a predetermined threshold; and
in response to determining that the background of at least the portion of the screenshot is darker than the predetermined threshold, inverting colors of at least the portion of the screenshot.

9. The method of claim 1 , further comprising:
determining a font size associated with the one or more proximate alphanumeric characters; and
in response to determining that the font size is below a predetermined threshold, scaling to enlarge the screenshot.

10. The method of claim 1 , wherein the determining of the one or more proximate alphanumeric characters in at least the portion of the screenshot utilizes optical character recognition.

11. A system for displaying contextually relevant information comprising:
one or more processors; and
one or more machine-readable media storing software including instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
receiving cursor data comprising a location of a cursor on an electronic display;
determining a screenshot of at least a portion of the electronic display;
determining one or more proximate alphanumeric characters in at least a portion of the screenshot based on the location of the cursor;
determining one or more contextual terms associated with at least one of the proximate alphanumeric characters, the one or more contextual terms being within a threshold proximity of the at least one of the proximate alphanumeric characters;
determining a semantic meaning of the at least one of the proximate alphanumeric characters based on at least one of the contextual terms, the semantic meaning based on a semantic classification;
matching the at least one of the proximate alphanumeric characters with one or more terms from a predetermined list of terms;
determining one or more indicators based on the matching the at least one of the proximate alphanumeric characters with one or more terms from a predetermined list of terms; and
causing an information card to be displayed on the electronic display based on the location of the cursor, the information card corresponding to the one or more terms from the predetermined list of terms, the one or more indicators, and the semantic meaning.

12. The system of claim 11 , wherein the screenshot comprises a subportion of the electronic display located within a predetermined radius of the cursor.

13. The system of claim 11 , wherein a size of the screenshot comprises a predetermined subportion of the electronic display, the predetermined subportion of the electronic display being based on the location of the cursor.

14. The system of claim 11 , the operations further comprising:
upon determining that the one or more proximate alphanumeric characters in at least the portion of the screenshot are above a predetermined size threshold, determining a second screenshot of at least a portion of the electronic display, the second screenshot being larger than the screenshot.

15. The system of claim 11 , wherein the screenshot is determined when a predetermined event is detected.

16. The system of claim 11 , wherein the screenshot is determined when a predetermined event is detected, the predetermined event comprising a hover event of the cursor.

17. The system of claim 11 , wherein determining a screenshot further comprises:
determining an active window, the active window comprising a currently focused application window on the electronic display, wherein the screenshot excludes any portion of the electronic display that is not the active window.

18. The system of claim 11 , wherein determining one or more proximate alphanumeric characters in at least a portion of the screenshot comprises:
determining if a background of at least the portion of the screenshot is lighter than or darker than a predetermined threshold; and
in response to determining that the background of at least the portion of the screenshot is darker than the predetermined threshold, inverting colors of at least the portion of the screenshot.

19. The system of claim 11 , the operations further comprising:
determining a font size associated with the one or more proximate alphanumeric characters; and
in response to determining that the font size is below a predetermined threshold, scaling to enlarge the screenshot.

20. One or more non-transitory machine-readable media storing instructions that, when executed by one or more processors, cause performance of operations for displaying contextually relevant information comprising:
receiving cursor data comprising a location of a cursor on an electronic d

Assignees

• Microstrategy Inc

Inventors

• Notani Shweta
• Chen Liang
• Zeng Jinliang
• Li Yan
• Dong Ting
• Nocedal De La Garza Jose Manuel
• Imtiaz Nida

Classifications

• G09G5/08 Primary

  Cursor circuits · CPC title

• G06F3/04845 Primary

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06V20/62

  Text, e.g. of license plates, overlay texts or captions on TV images · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V30/10

  Character recognition · CPC title