Stroke extraction in free space

Patent metadata

Field	Value
Publication number	US-11093769-B2
Application number	US-201816166202-A
Country	US
Kind code	B2
Filing date	Oct 22, 2018
Priority date	Nov 30, 2015
Publication date	Aug 17, 2021
Grant date	Aug 17, 2021

Abstract

An approach for extracting strokes in a free space environment is described. Boundaries are displayed in a free space environment describing at least one two-dimensional surface area. One or more language movements are extracted from the free space environment by a paired ring device and transmitted as images for processing. Haptic feedback is provided to the paired ring device in response to detecting at least one language movement occurring outside of at least one two-dimensional surface area. At least one extracted language movement is input into a character training model.

First claim

What is claimed is:

1. A method for stroke extraction in free space utilizing a paired ring device and a paired display device connected to the paired ring device, the method comprising:
displaying, by one or more computer processors, on the paired display device, a modified free space environment including one or more bounded surface areas, with the one or more bounded surface areas including an area of two-dimensional space framed by a boundary within the free space environment;
receiving, by the one or more computer processors, one or more images transmitted from the paired ring device, wherein the one or more images are transcribed sequentially from one or more language movements recorded by the paired ring device utilizing a magnitude value and a direction value of proper acceleration of the one or more language movements, and wherein the one or more images include one or more of a plurality of vector points, a plurality of coordinates, and a plurality of dots interconnected by a plurality of lines;
determining, by the one or more computer processors, at least one of the one or more language movements crossed at least one side of at least one bounded surface area of the one or more bounded surface areas;
responsive to determining that at least one of the one or more language movements crossed at least one side of at least one bounded surface area of the one or more bounded surface areas, providing, by the one or more computer processors, haptic feedback to the paired ring device;
inputting, by the one or more computer processors, the one or more images into a character training model;
mapping, by the one or more computer processors, the one or more images into one or more characters in the character training model, wherein mapping includes determining whether the one or more images indicate at least one of the one or more characters in the character training model utilizing a pre-determined range of motion of the paired ring device during a time period of movement in free space, where the pre-determined range of motion creates a specific image; and
responsive to a determination that the one or more images indicate at least one of the one or more characters in the character training model, transcribing, by the one or more computer processors, the at least one of the one or more characters into a digital document, wherein the digital document is a visual augmentation for displaying in a heads up display of the paired display device.

2. The method of claim 1 , further comprising:
determining, by the one or more computer processors, a representation of the at least one of the one or more characters as a written language or a gesture output derived from a series of strokes in free space; and
encoding, by the one or more computer processors, the at least one of the one or more characters into a visual augmentation.

3. The method of claim 1 , wherein:
the paired ring device includes an accelerometer and a gyroscope for monitoring the one or more language movements; and
the one or more language movements include one or more of the following: (i) a change in a magnitude value and a direction value of proper acceleration of the paired ring device, (ii) an orientation of the paired ring device, (iii) a position of the paired ring device, (iv) a location of the paired ring device, (v) a change in the orientation of the paired ring device, (vi) a change in the position of the paired ring device, and (vii) a change in the location of the paired ring device in the modified free space environment.

4. The method of claim 1 , wherein the paired ring device is sized to be worn on a human finger digit.

5. A computer program product (CPP) for stroke extraction in free space, the CPP comprising:
a paired ring device;
a paired display device connected to the paired ring device;
a computer readable storage medium; and
program instructions stored on the computer readable storage medium, with the program instructions including instructions for causing one or more computer processors to perform operations including the following:
displaying, by the one or more computer processors, on the paired display device, a modified free space environment including one or more bounded surface areas, with the one or more bounded surface areas including an area of two-dimensional space framed by a boundary within the free space environment,
receiving, by the one or more computer processors, one or more images transmitted from the paired ring device, wherein the one or more images are transcribed sequentially from one or more language movements recorded by the paired ring device utilizing a magnitude value and a direction value of proper acceleration of the one or more language movements, and wherein the one or more images include one or more of a plurality of vector points, a plurality of coordinates, and a plurality of dots interconnected by a plurality of lines,
determining, by the one or more computer processors, at least one of the one or more language movements crossed at least one side of at least one bounded surface area of the one or more bounded surface areas,
responsive to determining that at least one of the one or more language movements crossed at least one side of at least one bounded surface area of the one or more bounded surface areas, providing, by the one or more computer processors, haptic feedback to the paired ring device,
inputting, by the one or more computer processors, the one or more images into a character training model,
mapping, by the one or more computer processors, the one or more images into one or more characters in the character training model, wherein mapping includes determining whether the one or more images indicate at least one of the one or more characters in the character training model utilizing a pre-determined range of motion of the paired ring device during a time period of movement in free space, where the pre-determined range of motion creates a specific image, and
responsive to a determination that the one or more images indicate at least one of the one or more characters in the character training model, transcribing, by the one or more computer processors, the at least one of the one or more characters into a digital document, wherein the digital document is a visual augmentation for displaying in a heads up display of the paired display device.

6. The CPP of claim 5 , wherein the program instructions further includes instructions for causing the one or more computer processors to perform the following operations:
determining, by the one or more computer processors, a representation of the at least one or more characters as a written language or a gesture output derived from a series of strokes in free space; and
encoding, by the one or more computer processors, the at least one or more characters into a visual augmentation.

7. The CPP of claim 5 , wherein:
the paired ring device includes an accelerometer and a gyroscope for monitoring the one or more language movements; and
the one or more language movements include one or more of the following: (i) a change in a magnitude value and a direction value of proper acceleration of the paired ring device, (ii) an orientation of the paired ring device, (iii) a position of the paired ring device, (iv) a location of the paired ring device, (v) a change in the orientation of the paired ring device, (vi) a change in the position of the paired ring device, and (vii) a change in the location of the paired ring device in the modified free space environment.

8. The CPP of claim 5 , wherein the paired ring device is sized to be worn on a human finger digit.

9. A computer system (CS) for stroke extraction in free space, the CS comprising:
a paired ring device;
a

Assignees

• IBM

Inventors

• Dow Eli M
• Fitzsimmons Thomas D
• Garrett Tynan J
• Metruck Emily M

Classifications

• G06V30/2276

  with probabilistic networks, e.g. hidden Markov models · CPC title

• G06F3/0484 Primary

  for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range · CPC title

• G06V30/36

  Matching; Classification · CPC title

• G06V30/10

  Character recognition · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title