Eye event detection

Patent metadata

Field	Value
Publication number	US-9256784-B1
Application number	US-201313792953-A
Country	US
Kind code	B1
Filing date	Mar 11, 2013
Priority date	Mar 11, 2013
Publication date	Feb 9, 2016
Grant date	Feb 9, 2016

Abstract

The techniques and systems described herein track gaze movement of a user while eyes of the user read a document on a computing device. The techniques may then analyze or evaluate the gaze movement to determine if a reading interruption occurs (e.g., a reading pause or irregularity in a regular reading rate for the user). The reading interruption may occur when the user, while reading, encounters or notices a problem in the text. The reading interruption may also occur when the user encounters or notices text that the user has a strong interest in. When the reading interruption occurs, the techniques may evaluate a gaze direction and associate the reading interruption with a text that is currently displayed (e.g., word, sentence, paragraph, page, etc.). Moreover, the techniques may map the displayed text to a location in the document (e.g., a page or other identifiable section), and report the location to a centralized entity where statistical analysis can be performed to determine if there is a problem in the document.

First claim

What is claimed is:

1. A method comprising:
receiving, at a computing device, a user selection of a document;
displaying, on the computing device, displayed text of the document, the displayed text being displayed based at least in part on a first formatted version of the document;
capturing, using one or more front-facing cameras of the computing device, a plurality of images of at least a portion of eyes of a user as the user reads the displayed text of the document;
tracking, based at least in part on an analysis of individual ones of the plurality of captured images, a rate at which a gaze of the eyes moves as the user reads the displayed text of the document;
determining that the rate at which the gaze moves is less than a reading rate of the user;
determining a display screen location that contains a portion of the displayed text based on the determining that the rate at which the gaze moves is less than the reading rate of the user;
associating the display screen location that contains the portion of the displayed text with an identifiable section within a second formatted version of the document that is different than the first formatted version of the document, wherein the identifiable section comprises a page; and
sending, to one or more network-accessible devices, an indication that a reading interruption has occurred at the identifiable section within the second formatted version of the document.

2. The method as recited in claim 1 , wherein:
the first formatted version of the document is based at least in part on at least one setting of the computing device comprising at least one of text size, line spacing, text spacing, display screen size, or font type.

3. The method as recited in claim 1 , wherein the reading interruption is indicative of at least one of a typographical error, a grammatically incorrect phrase, or a confusing phrase in the portion of the displayed text.

4. The method as recited in claim 1 , further comprising accessing a reading model that defines the reading rate for the user based on average movement of the gaze of the user.

5. One or more non-transitory computer-readable media having computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
displaying text of a document at a computing device, the text being displayed based at least in part on a first formatted version of the document;
tracking gaze movement of a user while eyes of the user are directed toward the text of the document;
detecting an interruption to the gaze movement;
determining that the interruption to the gaze movement occurs in association with a display screen location that contains a portion of the text;
associating the display screen location that contains the portion of the text with an identifiable section within a second formatted version of the document that is different than the first formatted version of the document, wherein the identifiable section comprises a page of the second formatted version of the document; and
sending, to one or more network-accessible devices, information associated with the identifiable section within the second formatted version of the document.

6. The one or more non-transitory computer-readable media as recited in claim 5 , wherein the gaze movement comprises a rate of change in gaze direction of the eyes in relation to the text of the document.

7. The one or more non-transitory computer-readable media as recited in claim 6 , wherein the operations further comprise:
accessing reading information that defines a regular rate of change in gaze direction of the eyes of the user; and
comparing the rate of change in gaze direction to the regular rate of change in gaze direction, and wherein the interruption to the gaze movement is detected based at least in part on a determination that the rate of change in gaze direction is a threshold amount less than the regular rate of change in gaze direction.

8. The one or more non-transitory computer-readable media as recited in claim 5 , wherein:
the first formatted version of the document is based at least in part on at least one setting of the computing device.

9. The one or more non-transitory computer-readable media as recited in claim 5 , wherein the associating considers at least one setting of the computing device, the at least one setting comprising at least one of text size, line spacing, text spacing, display screen size, or a type of font.

10. The one or more non-transitory computer-readable media as recited in claim 5 , wherein the interruption to the gaze movement is indicative of at least one of a typographical error, a grammatically incorrect phrase, or a confusing phrase associated with the portion of the text.

11. The one or more non-transitory computer-readable media as recited in claim 5 , wherein the interruption to the gaze movement is indicative of a user interest in the portion of the text, and wherein the operations further comprise:
parsing the portion of the text to determine the user interest;
sending the user interest as part of the information associated with the identifiable section; and
receiving a recommendation based at least in part on the user interest.

12. The one or more non-transitory computer-readable media as recited in claim 11 , wherein the recommendation is one of an advertisement, an offer, or a coupon.

13. A method comprising:
receiving, at one or more network-accessible devices, information on a plurality of reading events from computing devices associated with a plurality of users reading a document, wherein an individual reading event is indicative of an interruption to eye gaze movement as a user reads text of a first formatted version of the document;
determining, based at least in part on the information, a location within a second formatted version of the document associated with a threshold number of reading events, wherein the second formatted version is different than at least one first formatted version and wherein the location comprises a page; and
communicating the location to at least one of:
a first computing device associated with a person that is responsible for writing, producing, or publishing the document; or
a second computing device associated with an entity that is responsible for writing, producing, or publishing the document.

14. The method as recited in claim 13 , wherein at least some of the plurality of reading events are indicative of at least one of grammatically incorrect text or confusing text in the document, the method further comprising receiving new text of the document that corrects the grammatically incorrect text or the confusing text.

15. The method as recited in claim 13 , wherein the interruption to the eye gaze movement occurs when a rate of change in eye gaze direction is a threshold amount less than a regular rate of change in eye gaze direction of a user.

16. The method as recited in claim 15 , wherein the regular rate of change in the eye gaze direction is determined based on tracking other eye gaze movement while the user reads other documents.

17. The method as recited in claim 13 , wherein the one or more network-accessible devices are associated with an account based entity that distributes the document to the network-accessible devices.

18. A system comprising:
one or more processors;
an eye sensor device; and
one or more computer-readable storage media storing computer-executable instructions that, when executed by the one or more processors, cause the one or mor

Assignees

• Amazon Tech Inc

Inventors

• Taylor Dennis James
• Wright Joshua Daniel
• Reinitz Jed Baraban
• Bennett Tommy Lee

Classifications

• H04N13/383

  for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/0483

  Interaction with page-structured environments, e.g. book metaphor · CPC title

• G06F15/0291

  for reading, e.g. e-books (constructional details of portable computers G06F1/1613) · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title