Software quality ticket enrichment

Patent metadata

Field	Value
Publication number	US-12561294-B2
Application number	US-202318242464-A
Country	US
Kind code	B2
Filing date	Sep 5, 2023
Priority date	Sep 5, 2023
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

Searches based on an incoming ticket identify quality ticket enrichment data using a vector database. Language model prompts target particular kinds of quality ticket data. The incoming quality ticket, or a search result ticket, or both, are enriched using enrichment data, such as a user intent identification, a workaround suggestion, a resolution description, a target audience description, a relevance description, an impact description, a description of missing resolution facilitation information, an association between the incoming quality ticket and the search result ticket, a user sentiment identification, a tag suggestion, or a feedback utility estimate. The enrichment reduces engineering and support burdens, and facilitates faster more effective resolution of the problem or the request that is stated or implied in the incoming quality ticket. Duplicate tickets are merged or removed. Tickets are prioritized. Missing problem resolution information is identified and requested sooner.

First claim

What is claimed is:

1 . A quality ticket enrichment method performed by a computing system, the method comprising:
calculating an embedding vector from at least a portion of an incoming quality ticket; searching a vector database, the vector database having associated quality ticket data, wherein the vector database searching comprises searching for quality ticket data which relates to the incoming quality ticket, and the vector database searching is based on at least the embedding vector; and enriching, with a data enrichment, the incoming quality ticket by displaying at least a portion of the data enrichment in the incoming quality ticket; wherein the data enrichment is based at least on the quality ticket data which relates to the incoming quality ticket; and wherein the data enrichment comprises at least one of: a user intent identification, a workaround suggestion, a resolution description, a target audience description, a relevance description, an impact description, a description of missing resolution facilitation information, an association between the incoming quality ticket and a search result ticket which is a result of the vector database searching, a user sentiment identification, a tag suggestion, or a feedback utility estimate.

2 . The method of claim 1 , further comprising:
harvesting at least a portion of the data enrichment from the search result ticket; wherein the data enrichment comprises at least one of: the workaround suggestion; the resolution description; the target audience description; the relevance description; the impact description; or the description of missing resolution facilitation information.

3 . The method of claim 1 , wherein enriching the incoming quality ticket comprises:
submitting at least a portion of the incoming quality ticket to a language model with a prompt that targets data enrichment, and in response receiving at least a portion of the data enrichment from the language model.

4 . The method of claim 3 , further comprising:
filtering an output of the language model and excluding a portion of the output from the data enrichment.

5 . The method of claim 1 , wherein calculating the embedding vector comprises:
basing the embedding vector on at least one of:
a user set identification; or
a ticket set identification.

6 . The method of claim 1 , wherein enriching the incoming quality ticket comprises:
harvesting at least a portion of the data enrichment from the search result ticket; and wherein the method further comprises:
prioritizing the incoming quality ticket or prioritizing the search result ticket or prioritizing both, wherein prioritizing is based on at least a result of harvesting.

7 . The method of claim 1 , comprising:
getting at least a portion of the incoming quality ticket via a user interface, wherein the embedding vector calculating and the vector database searching each chronologically overlap the getting; and displaying at least a portion of the search result ticket.

8 . The method of claim 1 , wherein enriching the incoming quality ticket comprises:
submitting at least a portion of the incoming quality ticket to a language model with a set of prompts that individually or collectively target an aspect of the data enrichment, and in response receiving at least a portion of the data enrichment from the language model; and wherein the set of prompts targets at least three of:
a set of steps for reproducing a problem;
a condition for reproducing a problem;
a feature title;
a feature description;
an offering identifier;
a problem description;
a suggestion;
a description of missing resolution facilitation information, the description including a log request;
a description of missing resolution facilitation information, the description including a screenshot request;
a request that missing resolution facilitation information be obtained when signed into a product;
a description of missing resolution facilitation information; or
a guide to collecting missing resolution facilitation information.

9 . The method of claim 1 , wherein the incoming quality ticket has a problem summary;
wherein the data enrichment comprises a remedy, the remedy comprises a workaround suggestion, a resolution description, or both; and wherein the method further comprises:
submitting at least a portion of the remedy and at least a portion of the problem summary to a language model with a prompt to assess the remedy relative to the problem summary, and in response receiving an assessment of the remedy from the language model.

10 . The method of claim 1 , wherein the data enrichment comprises the association between the incoming quality ticket and the search result ticket; and
wherein enriching the incoming quality ticket comprises at least one of:
displaying a suggestion to merge the incoming quality ticket and the search result ticket; or
merging the incoming quality ticket and the search result ticket.

11 . A quality ticket enrichment computing system, comprising:
a digital memory; a processor set comprising at least one processor, the processor set in operable communication with the digital memory; a vector database interface which upon execution by the processor set accesses a vector database which is associated with quality ticket data; and a data enricher which is configured to, upon execution by the processor set, calculate an embedding vector from at least a portion of an incoming quality ticket, submit the embedding vector to the vector database interface, receive a search result from the vector database interface, and enrich with a data enrichment the incoming quality ticket using the search result by adding at least a portion of the data enrichment in the incoming quality ticket; wherein the data enrichment comprises at least one of: a user intent identification, a workaround suggestion, a resolution description, a target audience description, a relevance description, an impact description, a description of missing resolution facilitation information, an association between the incoming quality ticket and the search result ticket, a user sentiment identification, a tag suggestion, or a feedback utility estimate.

12 . The quality ticket enrichment computing system of claim 11 , wherein the data enrichment comprises at least one of:
the user intent identification; the user sentiment identification; or a user identification.

13 . The quality ticket enrichment computing system of claim 11 , wherein the digital memory stores a list of predefined user intent identifications; and
wherein the data enricher is configured to, upon execution by the processor set, enrich the incoming quality ticket with at least one of the predefined user intent identifications.

14 . The quality ticket enrichment computing system of claim 11 , further comprising:
a language model interface; wherein the system upon execution submits at least a portion of the incoming quality ticket to the language model interface with a prompt that targets an aspect of the data enrichment, and in response receives at least a portion of the data enrichment from the language model interface.

15 . The quality ticket enrichment computing system of claim 14 , wherein the prompt targets at least one of:
the feedback utility estimate; the user sentiment identification; a set of steps for reproducing a problem; a condition for reproducing a problem; a feature title; a feature description; an offering identifier; a problem description; a workaround; a solution; a suggestion; a description

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Groenewegen Peter
• Mihaylov Nikola Minkov
• Cox Larissa Marie
• Mullen Nicholas Taylor
• Wilson-Thomas Mark Alistair
• Chapman Paul
• Bafna Kshama Gajraj
• Kumar Abhishek
• Chlus Jason
• Mitchell Holly

Classifications

• G06F16/24578

  using ranking · CPC title

• G06F16/951 Primary

  Indexing; Web crawling techniques · CPC title

• G06F16/215 Primary

  Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors · CPC title