Self-mixing based 2D/3D user input detection and scanning laser system

Patent metadata

Field	Value
Publication number	US-10871820-B2
Application number	US-201916408295-A
Country	US
Kind code	B2
Filing date	May 9, 2019
Priority date	May 9, 2019
Publication date	Dec 22, 2020
Grant date	Dec 22, 2020

Abstract

Disclosed herein are electronic devices, and methods for their operation, that identify user inputs based on interaction of an object with input surfaces separate from the electronic devices. The electronic devices may include one or more self-mixing interferometry sensors that scan a field of view containing the input surface with a light beam, such as a laser beam emitted laser diode. Self-mixing of the emitted light with reflections can generate a self-mixing interferometry signal. Analysis of the self-mixing interferometry signal can allow for identification of an object, such as a user's finger, in the field of view. Deformation of the finger can be detected with the self-mixing interferometry sensor, and a user input identified therefrom.

First claim

What is claimed is:

1. A method of operating an electronic device, comprising:
scanning a field of view with a beam of light emitted by a self-mixing interferometry sensor;
identifying, using a self-mixing interferometry signal generated by the self-mixing interferometry sensor, an input surface in the field of view and an object in the field of view;
detecting a first distance between the input surface and the self-mixing interferometry sensor;
detecting a second distance between the object and the self-mixing interferometry sensor, the second distance shorter than the first distance;
detecting a third distance between the object and the self-mixing interferometry sensor, the third distance longer than the second distance and corresponding to an interaction between the object and the input surface; and
identifying a user input to the electronic device by comparing the second distance and the third distance.

2. The method of claim 1 , further comprising:
determining, from the self-mixing interferometry signal and a scan pattern, a location of the interaction between the object and the input surface;
mapping the location of the interaction to a portion of an image projected onto the input surface; and
altering an operation of the electronic device based on the user input; wherein altering the operation of the electronic device comprises at least one of:
changing the image;
executing an action of the electronic device associated with the portion of the image; or
providing a feedback to a user of the electronic device.

3. The method of claim 2 , wherein mapping the location of the interaction includes:
determining at least one of a topography or an orientation of the input surface; and
determining a correspondence between points of the image projected onto the input surface and the topography or the orientation of the input surface.

4. The method of claim 1 , wherein:
the beam of light is produced by a laser diode;
the self-mixing interferometry signal is an output of a photodetector associated with the laser diode;
identifying the input surface and the object is based at least on:
a first set of distances from the input surface to the laser diode determined using at least the self-mixing interferometry signal; and
a second set of distances from the object to the laser diode determined using at least the self-mixing interferometry signal;

the first distance corresponds to the first set of distances; and
the second distance and the third distance correspond to the second set of distances.

5. The method of claim 1 , further comprising determining that the object is a finger of a user; wherein a difference between the second distance and the third distance corresponds to a deformation of the finger caused by the finger applying a force against the input surface.

6. The method of claim 5 , further comprising:
determining a magnitude of the force applied by the finger based on the deformation;
determining that the user input is a first user input when the magnitude exceeds a first threshold but is less than a second threshold; and
determining that the user input is a second user input when the magnitude exceeds the second threshold.

7. The method of claim 5 , further comprising determining that the finger ceases applying the force against the input surface based on a reduction of the deformation.

8. The method of claim 1 , further comprising:
determining that the object is a finger of a user; wherein detecting the deformation of the finger includes:
determining a height of the finger above the input surface; and
comparing the height of the finger above the input surface to a finger thickness.

9. The method of claim 1 , further comprising:
determining a motion of the object within the field of view; and
identifying the user input based at least on the motion.

10. An electronic device, comprising:
a self-mixing interferometry sensor operable to scan a field of view with a beam of light, the field of view including an input surface; and
a processor operably linked with the self-mixing interferometry sensor and operable to:
determine first distances from the self-mixing interferometry sensor to the input surface using a self-mixing interferometry signal generated by the self-mixing interferometry sensor;
determine, at a first time, second distances from the self-mixing interferometry sensor to an object in the field of view using the self-mixing interferometry signal generated by the self-mixing interferometry sensor;
determine, at a second time after the first time, third distances from the self-mixing interferometry sensor to the object using the self-mixing interferometry signal generated by the self-mixing interferometry sensor;
identify a deformation of the object due to an interaction of the object with the input surface using at least the first distances, the second distances, and the third distances; and
identify a user input to the electronic device from the identified deformation.

11. The electronic device of claim 10 , further comprising:
an image projector operable to project an image onto the input surface in the field of view; wherein:
the processor is further operable to:
determine, from the self-mixing interferometry signal and a scan pattern of the field of view, a position of the interaction of the object with the input surface;
map the position of the interaction to a portion of the image projected onto the input surface by the image projector; and
alter an operation of the electronic device based on the user input; and

altering the operation of the electronic device comprises at least one of:
changing the image;
executing an action of the electronic device associated with the portion of the image; or
providing feedback to a user of the electronic device.

12. The electronic device of claim 10 , wherein:
the beam of light is a laser light emitted from a laser diode; and
the self-mixing interferometry signal is an output of a photodetector associated with the laser diode.

13. The electronic device of claim 12 , wherein the laser diode emits infrared electromagnetic radiation.

14. The electronic device of claim 12 , wherein the processor is operable to cause a modulation of a bias current to the laser diode, the modulation causing a change in the self-mixing interferometry signal.

15. The electronic device of claim 14 , wherein:
the modulation of the bias current of the laser diode is a triangle wave modulation; and
the processor is operable to determine each of the first distances from the self-mixing interferometry sensor to first locations on the input surface and the second distances or the third distances from the self-mixing interferometry sensor to second locations or third locations on the object using a spectrum analysis of the self-mixing interferometry signal.

16. The electronic device of claim 14 , wherein:
the modulation of the bias current of the laser diode is a sinusoidal wave modulation; and
the processor is operable to determine each of the first distances from the self-mixing interferometry sensor to first locations on the input surface and the second distances or the third distances from the self-mixing interferometry sensor to second locations or third locations on the object using an I/Q analysis of the self-mixing interferometry signal.

17. The method of claim 10 , wherein the first time occurs during a calibration procedure, the calibration procedure comprising:
generating a baseline topographical map of the object with respect to the input surface;
storing

Assignees

• Apple Inc

Inventors

• Mutlu Mehmet
• Cihan Ahmet Fatih

Classifications

• G01B2290/65

  Spatial scanning object beam · CPC title

• G01B9/02092

  Self-mixing interferometers, i.e. feedback of light from object into laser cavity · CPC title

• G01B9/02027

  Two or more interferometric channels or interferometers · CPC title

• G01B9/02007

  Two or more frequencies or sources used for interferometric measurement (using only beat G01B9/02003) · CPC title

• H04N9/3129

  scanning a light beam on the display screen (scanning a light beam on a screen in displays other than projection devices G09G3/02; scanning systems in general G02B26/10; projectors using laser light sources in general H04N9/3161) · CPC title