Aggregating and identifying new sign language signs

Patent metadata

Field	Value
Publication number	US-11977853-B2
Application number	US-202117486848-A
Country	US
Kind code	B2
Filing date	Sep 27, 2021
Priority date	Sep 27, 2021
Publication date	May 7, 2024
Grant date	May 7, 2024

Abstract

A system for receiving a corpus of sign language data in which a plurality of known signs each correspond to known meanings, generate a model for identifying new sign language signs using the corpus, and identifying, using the model, a new sign language sign that does not match any of the plurality of known signs.

First claim

What is claimed is:

1. A system comprising:
a processor; and
a memory in communication with the processor, the memory containing program instructions that, when executed by the processor, are configured to cause the processor to perform a method, the method comprising:
receiving a corpus of sign language data in which a plurality of known signs each correspond to known meanings;
generating a model for identifying new sign language signs using the corpus;
receiving a visual media file;
processing the visual media file with the model;
identifying, using the model, a new sign language sign in the visual media file that does not match any of the plurality of known signs;

converting signs in the visual media file to text;
applying contextual correlation and validation to the text; and
extracting a meaning, from the text, associated to the new sign.

2. The system of claim 1 , wherein the memory contains additional program instructions that, when executed by the processor, are configured to cause the processor to perform the method further comprising:
converting finger spellings in the visual media file to additional text.

3. The system of claim 1 , wherein the memory contains additional program instructions that, when executed by the processor, are configured to cause the processor to perform the method further comprising:
applying sign codification to determine a prevalence of an association.

4. The system of claim 3 , wherein the memory contains additional program instructions that, when executed by the processor, are configured to cause the processor to perform the method further comprising:
determining that the prevalence of the association surpasses a threshold, and
including the association in a corpus.

5. The system of claim 3 , wherein the applying sign codification is based upon a number of instances the association is used.

6. The system of claim 1 , wherein the memory contains additional program instructions that, when executed by the processor, are configured to cause the processor to perform the method further comprising:
determining if the association of the meaning and the new sign exists in the corpus, and
creating, based upon a finding that the association does not exist, a new entry for the association.

7. A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
receiving a corpus of sign language data in which a plurality of known signs each correspond to known meanings;
generating a model for identifying new sign language signs using the corpus;
receiving a visual media file;
processing the visual media file with the model;
identifying, using the model, a new sign language sign in the visual media file that does not match any of the plurality of known signs;
converting signs in the visual media file to text;
applying contextual correlation and validation to the text; and
extracting a meaning, from the text, associated to the new sign.

8. The computer program product of claim 7 , wherein the computer readable storage medium comprises additional program instructions executable by the computer to cause the computer to perform the method further comprising:
converting finger spellings in the visual media file to additional text.

9. The computer program product of claim 8 , wherein the computer readable storage medium comprises additional program instructions executable by the computer to cause the computer to perform the method further comprising:
applying sign codification to determine a prevalence of an association.

10. The computer program product of claim 9 , wherein the computer readable storage medium comprises additional program instructions executable by the computer to cause the computer to perform the method further comprising:
determining that the prevalence of the association surpasses a threshold, and
including the association in a corpus.

11. The computer program product of claim 9 , wherein the applying sign codification is based upon a number of instances the association is used.

12. The computer program product of claim 8 , wherein the computer readable storage medium comprises additional program instructions executable by the computer to cause the computer to perform the method further comprising:
determining if the association of the meaning and the new sign exists in the corpus, and
creating, based upon a finding that the association does not exist, a new entry for the association.

13. A method comprising:
receiving a corpus of sign language data in which a plurality of known signs each correspond to known meanings;
generating a model for identifying new sign language signs using the corpus;
receiving a visual media file;
processing the visual media file with the model;
identifying, using the model, a new sign language sign in the visual media file that does not match any of the plurality of known signs;
converting signs in the visual media file to text;
applying contextual correlation and validation to the text; and
extracting a meaning, from the text, associated to the new sign.

14. The method of claim 13 further comprising:
converting finger spellings in the visual media file to additional text.

15. The method of claim 14 further comprising:
applying sign codification to determine a prevalence of an association.

16. The method of claim 15 further comprising:
determining that the prevalence of the association surpasses a threshold, and
including the association in a corpus.

17. The method of claim 15 , wherein the applying sign codification is based upon a number of instances the association is used.

Assignees

• IBM

Inventors

• Decrop Clement
• Beller Charles E
• Silverstein Zachary A
• Fox Jeremy R

Classifications

• G06F40/47 Primary

  Machine-assisted translation, e.g. using translation memory · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06V40/113

  Recognition of static hand signs · CPC title

• G06V40/28 Primary

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06F2218/12

  Classification; Matching · CPC title