Radar system facilitating ease and accuracy of user interactions with a user interface

Patent metadata

Field	Value
Publication number	US-11204694-B2
Application number	US-202117148414-A
Country	US
Kind code	B2
Filing date	Jan 13, 2021
Priority date	Aug 24, 2018
Publication date	Dec 21, 2021
Grant date	Dec 21, 2021

Abstract

This document describes techniques and systems that enable a radar system facilitating ease and accuracy of user interactions with a user interface. The techniques and systems can be implemented in an electronic device, such as a smartphone, and use a radar field to accurately determine three-dimensional (3D) gestures that can be used in combination with other inputs, such as touch or voice inputs, to interact with the user interface. These techniques allow the user to make 3D gestures from a distance and enable seamless integration of touch and voice commands with 3D gestures to improve functionality and user enjoyment.

First claim

What is claimed is:

1. An electronic device, comprising:
a display;
a radar system, implemented at least partially in hardware, configured to:
provide a radar field;
sense reflections from an object in the radar field;
analyze the reflections from the object in the radar field; and
provide, based on the analysis of the reflections, radar data; and

one or more computer processors that cause the electronic device to:
present a user interface through the display of the electronic device, the user interface capable of receiving a touch input;
receive a touch input through the user interface;
perform an action that corresponds to the touch input, the action causing a result;
determine, based on the radar data, a gesture by the object in the radar field; and
perform another action that corresponds to the determined gesture, the other action associated with the result of the action corresponding to the touch input.

2. The electronic device of claim 1 , wherein the object in the radar field comprises a user.

3. The electronic device of claim 2 , wherein:
the action that corresponds to the touch input comprises an action by a drawing-based application; and
the other action that corresponds to the determined gesture is related to the drawing-based application.

4. The electronic device of claim 3 , wherein the drawing-based application comprises one of:
a graphical design application;
a sketching application; or
a computer-aided drafting program.

5. The electronic device of claim 3 , wherein the other action that corresponds to the determined gesture and is related to the drawing-based application comprises one or more of:
creation of a new element of a drawing produced with the drawing-based application;
a change to an existing element of the drawing produced with the drawing-based application;
an operation performed on a data file that represents the drawing produced with the drawing-based application; or
an interaction with a menu of the drawing-based application.

6. The electronic device of claim 2 , wherein:
the touch input comprises a touch and a slide of a finger along the user interface;
the result of the action that corresponds to the touch input comprises displaying a line on the display, the line having a shape that corresponds to a shape of the slide; and
the other action that corresponds to the determined gesture comprises a change to the line, the change including at least one of:
increasing or decreasing a thickness of the line;
changing a color of the line;
increasing or decreasing a length of the line; or
changing a pattern of the line.

7. The electronic device of claim 2 , wherein the determined gesture comprises a three-dimensional (3D) gesture comprising:
a dial-turning gesture; or
a spindle-twisting gesture.

8. The electronic device of claim 2 , wherein:
the action that corresponds to the touch input comprises an action by a text-based application; and
the other action that corresponds to the determined gesture is related to the text-based application.

9. The electronic device of claim 8 , wherein the text-based application comprises one of:
a word-processing application;
an e-mail application; or
a spreadsheet application.

10. The electronic device of claim 8 , wherein the other action that corresponds to the determined gesture comprises one or more of:
a change to existing text in the text-based application;
creation of new text in the text-based application;
an operation performed on a data file that represents the text in the text-based application; or
an interaction with a menu of the text-based application.

11. The electronic device of claim 2 , wherein:
the touch input comprises a touch of two or more fingers of the user to the user interface near a portion of displayed text and then a slide of one or more of the fingers effective to separate the fingers while maintaining the touch to the user interface;
the result of the action that corresponds to the touch input comprises selecting the portion of displayed text over which the touch input is performed; and
the determined gesture comprises a three-dimensional (3D) gesture that performs an operation on the selected text.

12. The electronic device of claim 11 , wherein:
the 3D gesture comprises a first pinch gesture;
the operation performed on the selected text comprises copying the selected text; and
the instructions are further configured to:
receive another touch input through the user interface, the other touch input effective to place a cursor on the user interface at a location corresponding to the other touch input;
determine, based on the radar data, another gesture by the user, the other gesture comprising a second pinch gesture; and
responsive to determining the second pinch gesture, perform another operation on the selected text, the other operation comprising pasting the selected text at the location corresponding to the other touch input.

13. The electronic device of claim 1 , wherein the electronic device comprises a smartphone.

14. The electronic device of claim 1 , wherein the electronic device comprises at least one of:
a tablet computer;
a laptop computer;
a desktop computer;
a computing watch;
computing spectacles;
a gaming system;
a home appliance;
a vehicle;
a television;
an entertainment system;
a home-automation and control system; or
a home security system.

15. An electronic device, comprising:
a display;
a radar system, implemented at least partially in hardware, configured to:
provide a radar field;
sense reflections from an object in the radar field;
analyze the reflections from the object in the radar field; and
provide, based on the analysis of the reflections, radar data;

one or more computer processors; and
one or more computer-readable media having instructions stored thereon that, responsive to execution by the one or more computer processors, are configured to:
present a user interface through the display of the electronic device;
receive a voice input through a microphone of the electronic device;
perform an action that corresponds to the voice input, the action causing a result;
determine, based on the radar data, a gesture by the object in the radar field; and
perform another action that corresponds to the determined gesture, the other action associated with the result of the action corresponding to the voice input.

16. The electronic device of claim 15 , wherein the object in the radar field comprises a user.

17. The electronic device of claim 16 , wherein:
the action that corresponds to the voice input comprises an action by a drawing-based application; and
the other action that corresponds to the determined gesture is related to the drawing-based application.

18. The electronic device of claim 17 , wherein the drawing-based application comprises one of:
a graphical design application;
a sketching application; or
a computer-aided drafting program.

19. The electronic device of claim 17 , wherein the other action that corresponds to the determined gesture comprises one or more of:
creation of a new element of a drawing produced with the drawing-based application;
a change to an existing element of the drawing produced with the drawing-based application;
an operation performed on a data file that represents the drawing produced with the drawing-based application; or
an interaction with a menu of the drawing-based application.

20. The electronic device of claim 1

Assignees

• Google Llc

Inventors

• Giusti Leonardo
• Poupyrev Ivan
• Amihood Patrick M

Classifications

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F2203/04101

  2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup · CPC title

• G01S7/415

  Identification of targets based on measurements of movement associated with the target · CPC title

• G01S13/88

  Radar or analogous systems specially adapted for specific applications (electromagnetic prospecting or detecting of objects, e.g. near-field detection, G01V3/00) · CPC title