Account classification using a trained model and sign-in data

Patent metadata

Field	Value
Publication number	US-12348559-B2
Application number	US-202117557254-A
Country	US
Kind code	B2
Filing date	Dec 21, 2021
Priority date	Dec 21, 2021
Publication date	Jul 1, 2025
Grant date	Jul 1, 2025

Abstract

A trained machine learning model distinguishes between human-driven accounts and machine-driven accounts by performing anomaly detection based on sign-in data and optionally also based on directory data. This machine versus human distinction supports security improvements that apply security controls and other risk management tools and techniques which are specifically tailored to the kind of account being secured. Formulation heuristics can improve account classification accuracy by supplementing a machine learning model anomaly detection result, e.g., based on directory information, kind of IP address, kind of authentication, or various sign-in source characteristics. Machine-driven accounts masquerading as human-driven may be identified as machine-driven. Reviewed classifications may serve as feedback to improve the model's accuracy. A precursor machine learning model may generate training data for training a production account classification machine learning model.

First claim

What is claimed is:

1. A computing system configured to classify an account in a computing environment as machine-driven or as human-driven, the computing system comprising:
a digital memory;
a processor in operable communication with the digital memory, the processor configured to perform account classification steps including (a) submitting sign-in data to a trained machine learning model, the sign-in data representing at least one attempt to sign-in to the account, the trained machine learning model tailored for account classification by at least one of the following: human-driven account sign-in data which trained the machine learning model to detect machine-driven accounts as anomalies, or machine-driven account sign-in data which trained the machine learning model to detect human-driven accounts as anomalies, (b) receiving from the trained machine learning model an anomaly detection result, (c) formulating an account classification based at least in part on the anomaly detection result, and (d) supplying the account classification for use by a cybersecurity risk management mechanism, the cybersecurity risk management mechanism configured to manage a cybersecurity risk associated with the account based at least in part on the account classification, thereby improving security by distinguishing the machine-driven accounts from the human-driven accounts;

wherein the trained machine learning model is tailored for account classification which classifies the account as a machine-driven account or a human-driven account at least in that the trained machine learning model has been trained to perform the account classification, and thereby configured, using training data which includes, represents, or is a calculation basis of at least three of:
an indication whether an IP address of a source of a sign-in attempt is hosted or residential;
an indication whether an autonomous system number of a source of a sign-in attempt represents hosted IPs or residential IPs;
an indication whether a source of a sign-in attempt is a browser; an indication whether a source of a sign-in attempt is a command line interpreter;
an indication whethera source of a sign-in attempt resides on a mobile device;
an indication whethera source of a sign-in attempt resides on an organizationally managed device;
an indication whethera sign-in attempt included or followed a successful multifactor authentication;
an indication whethera sign-in attempt included or followed a successful biometric authentication;
an indication whether a sign-in attempt included or followed a successful removable hardware security key device authentication;
an indication of which one or more operating systems are present on a source of a sign-in attempt;
an indication of how many operating systems are present on a source of a sign-in attempt; or an error code generated in response to the sign-in attempt.

2. The computing system of claim 1 , further comprising the trained machine learning model.

3. The computing system of claim 2 , wherein the trained machine learning model is tailored for account classification by at least a random forest algorithm implementation.

4. The computing system of claim 1 , wherein the trained machine learning model is tailored for account classification at least in that the trained machine learning model has been trained and thereby configured using training data which includes, represents, or is a calculation basis of at least four of the following features:
an indication whether an IP address of a source of a sign-in attempt is hosted or residential;
an indication whether an autonomous system number of a source of a sign-in attempt represents hosted IPs or residential IPs;
an indication whether a source of a sign-in attempt is a browser;
an indication whether a source of a sign-in attempt is a command line interpreter;
an indication whether a source of a sign-in attempt resides on a mobile device;
an indication whether a source of a sign-in attempt resides on an organizationally managed device;
an indication whether a sign-in attempt included or followed a successful multifactor authentication;
an indication whether a sign-in attempt included or followed a successful biometric authentication;
an indication whether a sign-in attempt included or followed a successful removable hardware security key device authentication;
an indication of which one or more operating systems are present on a source of a sign-in attempt;
an indication of how many operating systems are present on a source of a sign-in attempt; or
an error code generated in response to the sign-in attempt.

5. The computing system of claim 1 , wherein the trained machine learning model is tailored for account classification at least in that the trained machine learning model has been trained and thereby configured using training data which includes, represents, or is a calculation basis of at least two of the following features:
a sign-in attempt success measure of a source of a sign-in attempt;
an indication of an extent to which timestamps of respective sign-in attempts vary over a period of at least five days;
a signed-in duration indicating a length of a session after a successful sign-in attempt;
an active days indication, which indicates on how many consecutive days a successful sign-in attempt occurred, or indicates on which days a successful sign-in attempt occurred, or both;
an allocated resources count indicating how many resources are allocated to the account;
an indication whether a directory entry for the account includes contact information;
an indication whether a directory entry for the account includes employee information; or
a label accuracy feedback.

6. A method for classifying an account in a computing environment as machine-driven or as human-driven, the method performed by a computing system, the method comprising:
submitting sign-in data to a trained machine learning model, the sign-in data representing at least one attempt to sign-in to the account, the trained machine learning model tailored for account classification by at least one of the following: human-driven account sign-in data which trained the machine learning model to detect machine-driven accounts as anomalies, or machine-driven account sign-in data which trained the machine learning model to detect human-driven accounts as anomalies;
receiving from the trained machine learning model an anomaly detection result;
formulating an account classification based at least in part on the anomaly detection result;
supplying the account classification for use by a cybersecurity risk management mechanism, the cybersecurity risk management mechanism configured to manage a cybersecurity risk associated with the account based at least in part on the account classification, thereby improving security by distinguishing the machine-driven accounts from the human-driven accounts; and
wherein the trained machine learning model is tailored for account classification at least in that the trained machine learning model has been trained and thereby configured using training data which includes, represents, or is a calculation basis of at least two of:
a sign-in attempt success measure of a source of a sign-in attempt;
an indication of an extent to which timestamps of respective sign-in attempts vary over a period of at least five days;
a signed-in duration indicating a length of a session after a successful sign-in attempt;
an active days indication, which indicates on how many consecutive days a successful sign-in attempt occurred, or indicates on which days a successful sign-in attempt occurred, or both;
an allocated resources count indicating how many resources are allocated to the account;
an indication whethera directory entry for the acc

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Xu Ye
• Basseri Etan Micah
• Tan Biying
• Templeton Caroline Katherine
• Kanherkar Prithviraj Sanjeev

Classifications

• H04L63/1425

  Traffic logging, e.g. anomaly detection · CPC title

• H04L63/0869

  for achieving mutual authentication (cryptographic mechanisms or cryptographic arrangements for mutual authentication H04L9/3273) · CPC title

• H04L63/0861

  using biometrical features, e.g. fingerprint, retina-scan (cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231) · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N20/00

  Machine learning · CPC title