System and methods of camera-based fingertip tracking

Patent metadata

Field	Value
Publication number	US-9791938-B2
Application number	US-201615239501-A
Country	US
Kind code	B2
Filing date	Aug 17, 2016
Priority date	Dec 20, 2007
Publication date	Oct 17, 2017
Grant date	Oct 17, 2017

Abstract

Systems and methods for camera-based fingertip tracking are disclosed. One such method includes identifying at least one location of a fingertip in at least one of the video frames, and mapping the location to a user input based on the location of the fingertip relative to a virtual user input device.

First claim

What is claimed is:

1. A method of translating sign language, the method comprising
receiving a series of images from a camera pertaining to a motion of a provider's hand, finger, fingertip, and/or arm corresponding to a first unit of sign language information;
identifying from the images pixels corresponding to the hand, finger, fingertip and/or arm;
tracking motion of the hand, finger, fingertips and/or arm based on the pixels in the images;
processing the images to correlate the motion of the first unit of sign language information to a stored unit of sign language information of a different sign language type stored on a computer readable medium; and
outputting the stored unit of information of a different sign language type to a receiver, wherein said outputting comprises providing image information comprising motion of a hand, finger, fingertip and/or arm pertaining to sign language of a type different than said first unit of information.

2. The method of claim 1 , wherein said outputting further comprises providing an aural signal.

3. The method of claim 2 , wherein said aural signal comprises speaking a word to which the first unit of information pertains.

4. The method of claim 1 , wherein said outputting further comprises providing a word in text form.

5. The method of claim 1 , wherein said outputting further comprises providing a word in tactile form.

6. The method of claim 5 , wherein said tactile form is Braille.

7. The method of claim 1 , wherein said outputting comprises further displaying a visual image of an object to which the first unit of sign language information pertains.

8. A system for translating sign language, the system comprising a camera in communication with a computer, the computer comprising a computer readable media and being programmed to receive video images of a unit of sign language motion, identifying from the images pixels corresponding to the hand, finger, fingertip and/or arm; tracking motion of the hand, finger, fingertips and/or arm based on the pixels in the images; and correlating the sign language motion to unit of information stored on the computer readable media; and an output device for outputting an output unit of information, wherein the computer is programmed to output the output unit of information via said output device, and wherein said output unit of information comprises providing image information comprising motion of a hand, finger, fingertip, and/or arm pertaining to sign language of a type different than said unit of sign language motion.

9. The system of claim 8 , wherein said output device comprises a speaker, display, or Braille machine.

Assignees

• Sugaya Kiminobu
• Sugaya Andrew
• Univ Central Florida Res Found Inc

Inventors

• Sugaya Kiminobu
• Sugaya Andrew

Classifications

• G06F3/0304 Primary

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G02B27/017

  Head mounted · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title