Using machine learning to flag gender biased words within free-form text, such as job descriptions

Patent metadata

Field	Value
Publication number	US-10936642-B2
Application number	US-201916267788-A
Country	US
Kind code	B2
Filing date	Feb 5, 2019
Priority date	Nov 2, 2017
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

Under one aspect, first user input including free-form text is received in a first graphical user interface (GUI). A classification engine of the computer system incorporating a machine learning model classifies words of the free-form text into a male-biased class, a female-biased class, or a neutral class. At least one of the words is classified into the male-biased class or the female-biased class. At least one of the words classified into the male-biased class or the female-biased class is flagged in the first GUI. Second user input is received in the first GUI including at least one revision to at least one of the words of the free-form text classified into the male-biased class or the female-biased class responsive to the flagging. The revised free-form text is posted to a web site for display in a second GUI.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, in a first graphical user interface (GUI) forming part of an end-user layer of a computer system, first user input comprising free-form text;
respectively classifying, by a machine learning model being executed by the computer system, words of the free-form text into a male-biased class, a female-biased class, or a neutral class, at least one of the words being classified into the male-biased class or the female-biased class, the machine learning model being trained using a plurality of historical job descriptions for which a ratio of the gender of the applicants is known and with a plurality of words labeled as being a male-biased class, a female-biased class, or a neutral class derived from a historical job description having a highest ratio and a historical job description having a lowest ratio;
flagging, in the first GUI, at least one of the words classified into the male-biased class or the female-biased class;
receiving, in the first GUI, second user input comprising at least one revision to at least one of the words of the free-form text classified into the male-biased class or the female-biased class responsive to the flagging; and
posting, by the computer system, the revised free-form text to a web site for display in a second GUI forming part of the end-user layer of the computer system.

2. The method of claim 1 , further comprising:
generating, by the computer system, a gender bias score for the free-form text based on the classification of the words of the free-form text; and
displaying, in the first GUI, the gender bias score.

3. The method of claim 2 , further comprising:
respectively classifying, by the machine learning model engine of the computer system, each word of the revised free-form text into the male-biased class, the female-biased class, or the neutral class, at least one of the words being classified into the male-biased class or the female-biased class;
generating, by the computer system, a revised gender bias score for the revised free-form text based on the classification of the words of the revised free-form text; and
displaying, in the first GUI, the revised gender bias score.

4. The method of claim 1 , wherein a first subset of words of the corpus are labeled as male-biased, female-biased, or neutral using operations comprising:
generating, from a plurality of job descriptions, a first subset of the job descriptions that comprises data regarding the respective genders of applicants to that job, the jobs being within job families;
generating, for each job description of the first subset, a gender ratio based on the data regarding the respective genders of applicants to that job;
selecting, from each job family, a job description that has the highest gender ratio and a job description that has the lowest gender ratio;
generating a second subset of the job descriptions comprising the selected job descriptions of the job families that have the highest gender ratios;
generating a third subset of the job descriptions comprising the selected job descriptions of the job families that have the lowest gender ratios;
comparing frequencies of words of the job descriptions of the second and third subsets of the job descriptions; and
based on the comparing:
labeling at least one word of the job descriptions of the second subset of the job descriptions as being one of male-biased and female-biased, and
labeling at least one word of the job descriptions of the third set of the job descriptions as being the other of male-biased and female-biased.

5. The method of claim 1 , wherein the flagging comprises highlighting the word, changing a color of the word, changing an emphasis of the word, or changing a font of the word.

6. The method of claim 1 , wherein the flagging comprises displaying the word in an area of the first GUI that is separate from the free-form text.

7. The method of claim 1 , wherein the machine learning model comprises a support vector machine, a random forest algorithm, a stochastic gradient descent classifier, or a logistic regression model.

8. A computer system comprising:
at least one data processor; and
memory storing instructions which, when executed by the at least one data processor, result in operations comprising:
receiving, in a first graphical user interface (GUI) forming part of an end-user layer of the computer system, first user input comprising free-form text;
respectively classifying, by a machine learning model being executed by the computer system, words of the free-form text into a male-biased class, a female-biased class, or a neutral class, at least one of the words being classified into the male-biased class or the female-biased class, the machine learning model being trained using a plurality of historical job descriptions for which a ratio of the gender of the applicants is known and with a plurality of words labeled as being a male-biased class, a female-biased class, or a neutral class derived from a historical job description having a highest ratio and a historical job description having a lowest ratio;
flagging, in the first GUI, at least one of the words classified into the male-biased class or the female-biased class;
receiving, in the first GUI, second user input comprising at least one revision to at least one of the words of the free-form text classified into the male-biased class or the female-biased class responsive to the flagging; and

posting, by the computer system, the revised free-form text to a web site for display in a second GUI forming part of the end-user layer of the computer system.

9. The computer system of claim 8 , the memory further storing instructions which, when executed by the at least one data processor, result in operations comprising:
generating, by the computer system, a gender bias score for the free-form text based on the classification of the words of the free-form text; and
displaying, in the first GUI, the gender bias score.

10. The computer system of claim 9 , the memory further storing instructions which, when executed by the at least one data processor, result in operations comprising:
respectively classifying, by the machine learning model, each word of the revised free-form text into the male-biased class, the female-biased class, or the neutral class, at least one of the words being classified into the male-biased class or the female-biased class;
generating, by the computer system, a revised gender bias score for the revised free-form text based on the classification of the words of the revised free-form text; and
displaying, in the first GUI, the revised gender bias score.

11. The computer system of claim 8 , wherein a first subset of words of the corpus are labeled as male-biased, female-biased, or neutral using operations comprising:
generating, from a plurality of job descriptions, a first subset of the job descriptions that comprises data regarding the respective genders of applicants to that job, the jobs being within job families;
generating, for each job description of the first subset, a gender ratio based on the data regarding the respective genders of applicants to that job;
selecting, from each job family, a job description that has the highest gender ratio and a job description that has the lowest gender ratio;
generating a second subset of the job descriptions comprising the selected job descriptions of the job families that have the highest gender ratios;
generating a third subset of the job descriptions comprising the selected job descriptions of the job families that have the lowest gender ratios;
comparing frequencies of words of the job descriptions of t

Assignees

• Sap Se

Inventors

• Shen Weiwei
• Tripathi Manish

Classifications

• G06F18/2155

  characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling · CPC title

• G06F18/2431

  Multiple classes · CPC title

• G06F16/353 Primary

  into predefined classes · CPC title

• G06V30/418

  Document matching, e.g. of document images · CPC title

• G06F40/30

  Semantic analysis · CPC title