Line-of-sight processing method, line-of-sight processing system and wearable device

Patent metadata

Field	Value
Publication number	US-9760169-B2
Application number	US-201514740861-A
Country	US
Kind code	B2
Filing date	Jun 16, 2015
Priority date	Nov 14, 2014
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

The present disclosure provides a line-of-sight (LOS) processing method, an LOS processing system and a wearable device. The LOS processing method includes steps of: collecting a head deflection angle and an eyeball deflection angle of a user; determining a position of the user's LOS at a display interface in accordance with the head deflection angle and the eyeball deflection angle of the user; acquiring action information of the user's eyeball; identifying an action command corresponding to the action information of the user's eyeball; and performing a corresponding operation on a display image at the position of the user's LOS in accordance with the identified action command.

First claim

What is claimed is:

1. A line-of-sight (LOS) processing method, comprising:
a setting step of setting an LOS reference point of a user, and when the user fixes the user's eyes on the LOS reference point, acquiring initial LOS data of the user, wherein the initial LOS data comprises an initial junction between an iris and a sclera of the user's eyeball;
a collecting step of collecting a head deflection angle and an eyeball deflection angle of the user, wherein detecting the head deflection angle of the user includes using a microelectronic mechanical gyroscope located at the user's head, so as to acquire the head deflection angle of the user, and collecting an image of the user's eyeball using an image collector, so as to determine a junction between the iris and the sclera of the user's eyeball, thereby to calculate the eyeball deflection angle of the user;
a positioning step of determining a position of the user's LOS at a display interface in accordance with the head deflection angle and the eyeball deflection angle of the user;
an acquiring step of acquiring action information of the user's eyeball, wherein the action information of the user's eyeball includes, where or not the user's LOS remains at an identical region for a period of time exceeding a predetermined time, a blinking action of the user, pupil contraction, pupil dilation and a squinting action;
an identifying step of identifying an action command corresponding to the action information of the user's eyeball; and
a processing step of performing a corresponding operation on a display image at the position of the user's LOS in accordance with the identified action command, wherein when the user's LOS remains at an identical region for a period of time exceeding the predetermined time, an image at the region is zoomed in, and after the user's LOS is moved away, the image at the region is recovered; during the blinking action, the image at the position of the user's LOS is cancelled; during the pupil contraction or the squinting action, brightness of the image at the user's LOS is reduced; and during the pupil dilation, the brightness of the image at the user's LOS is increased.

2. The LOS processing method according to claim 1 , wherein the collecting step further comprises:
when the image is collected by the image collector, supplementing light for the image collector using an infrared lighting circuit.

3. The LOS processing method according to claim 1 , wherein the collecting step, the positioning step, the acquiring step and the identifying step are performed by a wearable device, and the processing step is performed by a terminal device,
wherein between the identifying step and the processing step, the method further comprises:
transmitting, by the wearable device, the identified action command and the position of the user's LOS to the terminal device.

4. A line-of-sight (LOS) processing system, comprising:
a setting circuit, configured to set an LOS reference point of a user, and when the user fixes the user's eyes on the LOS reference point, acquiring initial LOS data of the user;
a collecting circuit, configured to collect a head deflection angle and an eyeball deflection angle of the user, wherein the collecting circuit comprises:
a first collecting circuit, configured to detect the head deflection angle of the user using a microelectronic mechanical gyroscope located at the user's head, so as to acquire the head deflection angle of the user; and
a second collecting circuit, configured to collect an image of the user's eyeball using an image collector, so as to determine a junction between an iris and a sclera of the user's eyeball, thereby to calculate the eyeball deflection angle of the user;
a positioning circuit, configured to determine a position of the user's LOS at a display interface in accordance with the head deflection angle and the eyeball deflection angle of the user;
an acquiring circuit, configured to acquire action information of the user's eyeball, wherein the action information of the user's eyeball includes, where or not the user's LOS remains at an identical region for a period of time exceeding a predetermined time, a blinking action of the user, pupil contraction, pupil dilation and a squinting action;
an identifying circuit, configured to identify an action command corresponding to the action information of the user's eyeball; and
a processing circuit, configured to perform a corresponding operation on a display image at the position of the user's LOS in accordance with the identified action command, wherein when the user's LOS remains at an identical region for a period of time exceeding the predetermined time, an image at the region is zoomed in by the processing circuit, and after the user's LOS is moved away, the image at the region is recovered by the processing circuit; during the blinking action, the image at the position of the user's LOS is cancelled by the processing circuit; during the pupil contraction or the squinting action, brightness of the image at the user's LOS is reduced by the processing circuit; and during the pupil dilation, the brightness of the image at the user's LOS is increased by the processing circuit.

5. The LOS processing system according to claim 4 , wherein
the collecting circuit, the positioning circuit, the acquiring circuit and the identifying circuit are arranged in a wearable device, and the processing circuit is arranged in a terminal device;
the wearable device further comprises a primary communication circuit;
the terminal device further comprises a secondary communication circuit and a display circuit;
the primary communication circuit is configured to transmit the identified action command and the position of the user's LOS to the secondary communication circuit;
the secondary communication circuit is configured to receive the identified action command and the position of the user's LOS from the primary communication circuit; and
the display circuit is configured to provide the display interface.

6. The LOS processing system according to claim 4 , further comprising:
an infrared lighting circuit configured to supplement light for the image collector.

7. A wearable device, comprising:
a setting circuit, configured to set an LOS reference point of a user, and when the user fixes the user's eyes on the LOS reference point, acquiring initial LOS data of the user;
a collecting circuit, configured to collect a head deflection angle and an eyeball deflection angle of a user, wherein the collecting circuit comprises:
a first collecting circuit, configured to detect the head deflection angle of the user using a microelectronic mechanical gyroscope located at the user's head, so as to acquire the head deflection angle of the user;
a second collecting circuit, configured to collect an image of the user's eyeball using an image collector, so as to determine a junction between an iris and a sclera of the user's eyeball, thereby to calculate the eyeball deflection angle of the user;
a positioning circuit, configured to determine a position of the user's LOS at a display interface in accordance with the head deflection angle and the eyeball deflection angle of the user;
an acquiring circuit, configured to acquire action information of the user's eyeball, wherein the action information of the user's eyeball includes, where or not the user's LOS remains at an identical region for a period of time exceeding a predetermined time, a blinking action of the user, pupil contraction, pupil dilation and a squinting action;
an identifying circuit, configured to identify an action command corresponding to the action information of the user's eyeball; and
a processing circuit, configured to perform a corresponding operation on a display

Assignees

• Boe Technology Group Co Ltd
• Chengdu Boe Optoelect Tech Co

Inventors

• Dai Weinan
• Jin Fujiang

Classifications

• G06F3/014

  Hand-worn input/output arrangements, e.g. data gloves · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title