System and method of determining input characters based on swipe input

Patent metadata

Field	Value
Publication number	US-11188158-B2
Application number	US-201815995841-A
Country	US
Kind code	B2
Filing date	Jun 1, 2018
Priority date	Jun 2, 2017
Publication date	Nov 30, 2021
Grant date	Nov 30, 2021

Abstract

Provided are an artificial intelligence (AI) system and an application thereof, which simulate functions of a human brain, such as recognition and determination, by using a machine learning algorithm, such as deep-learning. A method of processing, by a device, a keyboard input, based on training, may include: displaying a keyboard on a screen of the device; receiving a swipe input of a user, the swipe input connecting a plurality of keys on a displayed keyboard; extracting a trajectory connecting the plurality of keys; applying, to a trained model for a keyboard input, based on the trajectory, trajectory information indicating a shape of the trajectory and a relative position of the trajectory with respect to the keyboard; and determining at least one character corresponding to the trajectory, based on a result of the applying the trajectory information.

First claim

What is claimed is:

1. A method of processing, by an electronic device, a keyboard input, the method comprising:
outputting, via a display, a first virtual keyboard;
obtaining a first drawing input on the output first virtual keyboard;
normalizing the first drawing input for applying the normalized first drawing input to a machine learning model;
obtaining, by applying the normalized first drawing input to the machine learning model, a first word corresponding to the normalized first drawing input;
receiving a user input to replace the first virtual keyboard with a second virtual keyboard;
based on the received user input, replacing the first virtual keyboard with the second virtual keyboard, the second virtual keyboard being displayed on the display differently from the first virtual keyboard;
obtaining a second drawing input on the output second virtual keyboard;
normalizing the second drawing input for applying the normalized second drawing input to the machine learning model which is used for the normalized first drawing input; and
obtaining, by applying the normalized second drawing input to the machine learning model which is used for the normalized first drawing input, a second word corresponding to the normalized second drawing input,
wherein the first drawing input on the first virtual keyboard is different from the second drawing input on the second virtual keyboard that is displayed differently from the first virtual keyboard, and the obtained first word is a same word as the obtained second word, and
wherein the machine learning model is trained to recognize trajectory information of different drawing inputs, corresponding to a same word, with respect to a plurality of types of virtual keyboards including the first virtual keyboard and the second virtual keyboard.

2. The method of claim 1 , wherein the first drawing input forms a first trajectory passing through a plurality of keys on the first virtual keyboard.

3. The method of claim 2 , wherein the obtaining the first word comprises applying normalized coordinate information of the first trajectory and normalized coordinate information of the first virtual keyboard to the machine learning model.

4. The method of claim 3 , wherein the obtaining the first word further comprises applying information indicating a formation direction of the first trajectory to the machine learning model.

5. The method of claim 2 , wherein the normalizing the first drawing input comprises normalizing the first trajectory, and
wherein the applying the normalized first drawing input to the machine learning model comprises applying the normalized first trajectory to the machine learning model.

6. The method of claim 5 , wherein the normalizing the first trajectory comprises resizing the first virtual keyboard and the first trajectory on the first virtual keyboard to a pre-set size and a pre-set shape.

7. The method of claim 1 , further comprising:
displaying a plurality of recommended words obtained from the first drawing input on the first virtual keyboard;
receiving a user's selection to select one of the plurality of the recommended words;
identifying the selected recommended word as the first word.

8. The method of claim 1 , wherein the machine learning model is trained by using at least one of a gated recurrent unit (GRU) algorithm or a connectionist temporal classification (CTC) algorithm.

9. An electronic device comprising:
a display;
a memory storing a machine learning model; and
at least one processor configured to:
control to output, via the display, a first virtual keyboard;
obtain a first drawing input on the output first virtual keyboard;
normalize the first drawing input for applying the normalized first drawing input to the machine learning model;
obtain, by applying the normalized first drawing input to the machine learning model, a first word corresponding to the normalized first drawing input;
receive a user input to replace the first virtual keyboard with a second virtual keyboard;
based on the received user input, replace the first virtual keyboard with the second virtual keyboard, the second virtual keyboard being displayed on the display differently from the first virtual keyboard;
obtain a second drawing input on the output second virtual keyboard;
normalize the second drawing input for applying the normalized second drawing input to the machine learning model which is used for the normalized first drawing input; and
obtain, by applying the normalized second drawing input to the machine learning model which is used for the normalized first drawing input, a second word corresponding to the normalized second drawing input,

wherein the first drawing input on the first virtual keyboard is different from the second drawing input on the second virtual keyboard that is displayed differently from the first virtual keyboard, and the first word obtained via the machine learning model is a same word as the second word obtained via the machine learning model, and
wherein the machine learning model is trained to recognize trajectory information of different drawing inputs, corresponding to a same word, with respect to a plurality of types of virtual keyboards including the first virtual keyboard and the second virtual keyboard.

10. The electronic device of claim 9 , wherein the first drawing input forms a first trajectory passing through a plurality of keys on the first virtual keyboard.

11. The electronic device of claim 10 , wherein the at least one processor is configured to obtain the first word by applying normalized coordinate information of the first trajectory and normalized coordinate information of the first virtual keyboard to the machine learning model.

12. The electronic device of claim 11 , wherein the at least one processor is configured to obtain the first word by applying information indicating a formation direction of the first trajectory to the machine learning model.

13. The electronic device of claim 10 , wherein the at least one processor is further configured to normalize the first drawing input by normalizing the first trajectory, and apply the normalized first drawing input to the machine learning model by applying the normalized first trajectory to the machine learning model.

14. The electronic device of claim 13 , wherein the at least one processor is configured to resize the first virtual keyboard and the first trajectory on the first virtual keyboard to a pre-set size and a pre-set shape for normalizing the first trajectory.

15. The electronic device of claim 9 , wherein the at least one processor is further configured to:
display a plurality of recommended words obtained from the first drawing input on the first virtual keyboard;
receive a user's selection to select one of the plurality of the recommended words;
identify the selected recommended word as the first word.

16. The electronic device of claim 9 , wherein the machine learning model is trained by using at least one of a gated recurrent unit (GRU) algorithm or a connectionist temporal classification (CTC) algorithm.

17. The electronic device of claim 9 , wherein at least one of sizes, shapes or arrangements of keys of the first virtual keyboard and the second virtual keyboard are different from each other.

18. A computer program product comprising a computer readable storage medium comprising a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:
output, via a display, a first virtual

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Jung-Wook
• Yun Hui-Won
• Lee Hae-Jun
• Jung Ho-Jin

Classifications

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/092

  Reinforcement learning · CPC title