Method for recognizing handwriting on a physical surface

Patent metadata

Field	Value
Publication number	US-10126825-B2
Application number	US-201615190484-A
Country	US
Kind code	B2
Filing date	Jun 23, 2016
Priority date	Jun 26, 2015
Publication date	Nov 13, 2018
Grant date	Nov 13, 2018

Abstract

The invention relates to a method for recognizing handwriting on a physical surface on the basis of three-dimensional signals originating from sensors of a terminal, the method being characterized in that the signals are obtained on the basis of at least 3 different types of sensors, and in that it comprises steps of sampling, according to 3 axes and over a sliding time window, of inertial signals originating from the sensors, fusing the sampled signals into a 9-dimensional vector for each sampling period, converting the fused signals into a sequence of characteristic 9-dimensional vectors, and, when a signal characteristic of an input start has been detected, storing the sequence of characteristic vectors in a list of sequences of characteristic vectors, the preceding steps being repeated until the detection of a signal characteristic of an input end, the method furthermore comprising, on detection of said signal characteristic of an input end, a step of recognizing a word on the basis of the list of sequences of characteristic vectors created over the time window.

First claim

The invention claimed is:

1. A method for recognizing handwriting on a physical surface on the basis of three-dimensional signals originating from sensors of a terminal, the method being characterized in that the signals are obtained on the basis of at least 3 different types of sensors, and in that it comprises the following steps:
sampling, according to 3 axes and over a sliding time window, of inertial signals originating from the sensors,
fusing the sampled signals into a 9-dimensional vector for each sampling period,
converting the fused signals into a sequence of characteristic 9-dimensional vectors, and,
when a signal characteristic of an input start has been detected:
storing the sequence of characteristic vectors in a list of sequences of characteristic vectors,
the preceding steps being repeated until the detection of a signal characteristic of an input end, the method furthermore comprising, on detection of said signal characteristic of an input end, a step of:
recognizing a word on the basis of the list of sequences of characteristic vectors created between the detection of the input start and end signals;
wherein the signals characteristic of an input start and end are identified by means of a comparison between a sequence of characteristic vectors and at least one previously recorded sequence of characteristic vectors, and wherein the signal characteristic of an input start is an inertial signal characteristic of a micro-impact on a physical surface.

2. The method as claimed in claim 1 , characterized in that it furthermore includes a step of validation of the recognized word by a linguistic model.

3. The method as claimed in claim 1 , characterized in that the step of recognizing a word is performed by a recurrent bidirectional neural network.

4. The method as claimed in claim 3 , characterized in that the neural network is trained in advance on the basis of sequences of characteristic vectors corresponding to words from the vocabulary of the language concerned.

5. The method as claimed in claim 1 , characterized in that the step of converting the signals into a sequence of characteristic vectors includes steps of:
calibrating the signals originating from the sensors,
denoising the calibrated signal by applying a low-pass filter, and
approximating and normalizing the filtered signals.

6. A device for recognizing handwriting on a physical surface on the basis of three-dimensional signals originating from sensors, the device being characterized in that the signals are obtained from at least 3 different types of sensors of the device, and in that it comprises:
a module for sampling, over a sliding time window, the signals originating from the sensors,
a module for fusing the sampled signals into a 9-dimensional vector for each sampling period,
a module for converting the fused signals into a sequence of 9-dimensional characteristic vectors,
a module for detecting signals characteristic of an input start on the basis of the converted sequence of characteristic vectors,
a module for storing the sequences of characteristic vectors obtained between the detection of the input start signal and the detection of the input end signal in a list of sequences of characteristic vectors when a signal characteristic of an input start has been detected, the signals characteristic of an input start and end being identified by means of a comparison between a sequence of characteristic vectors and at least one previously recorded sequence of characteristic vectors, the signal characteristic of an input start being an inertial signal characteristic of a micro-impact on a physical surface, and
a module for recognizing a word on the basis of the stored list of sequences of characteristic vectors when a signal characteristic of an input end has been detected, and
a module for validating a word, implemented when a word is recognized.

7. A terminal, characterized in that it comprises a device for recognizing handwriting on a physical surface on the basis of three-dimensional signals originating from sensors, the device being characterized in that the signals are obtained from at least 3 different types of sensors of the device, and in that it comprises:
a module for sampling, over a sliding time window, the signals originating from the sensors,
a module for fusing the sampled signals into a 9-dimensional vector for each sampling period,
a module for converting the fused signals into a sequence of 9-dimensional characteristic vectors,
a module for detecting signals characteristic of an input start and end on the basis of the converted sequence of characteristic vectors,
a module for storing the sequences of characteristic vectors obtained between the detection of the input start signal and the detection of the input end signal in a list of sequences of characteristic vectors when a signal characteristic of an input start has been detected by the detection module, the signals characteristic of an input start and end being identified by means of a comparison between a sequence of characteristic vectors and at least one previously recorded sequence of characteristic vectors, the signal characteristic of an input start being an inertial signal characteristic of a micro-impact on a physical surface, and
a module for recognizing a word on the basis of the stored list of sequences of characteristic vectors when a signal characteristic of an input end has been detected by the detection module, and
a module for validating a word, implemented when a word is recognized.

8. A non-transitory computer-readable information medium on which a computer program is recorded, including instructions for carrying out steps of a handwriting recognition method for recognizing handwriting on a physical surface on the basis of three-dimensional signals originating from sensors of a terminal, the method being characterized in that the signals are obtained on the basis of at least 3 different types of sensors, and in that it comprises the following steps:
sampling, according to 3 axes and over a sliding time window, of inertial signals originating from the sensors,
fusing the sampled signals into a 9-dimensional vector for each sampling period,
converting the fused signals into a sequence of characteristic 9-dimensional vectors, and,
when a signal characteristic of an input start has been detected:
storing the sequence of characteristic vectors in a list of sequences of characteristic vectors,
the preceding steps being repeated until the detection of a signal characteristic of an input end, the method furthermore comprising, on detection of said signal characteristic of an input end, a step of:
recognizing a word on the basis of the list of sequences of characteristic vectors
created between the detection of the input start and end signals; wherein the signals characteristic of an input start and end are identified by means of a comparison between a sequence of characteristic vectors and at least one previously recorded sequence of characteristic vectors, and wherein the signal characteristic of an input start is an inertial signal characteristic of a micro-impact on a physical surface.

Assignees

• Orange

Inventors

• Lefebvre Gregoire

Classifications

• G06V10/46 Primary

  Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features (colour feature extraction G06V10/56) · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F18/251

  of input or preprocessed data · CPC title

• G06V30/228

  of three-dimensional [3D] handwriting, e.g. writing in the air · CPC title

• G06V30/32

  Digital ink · CPC title