Generating and using a semantic index

Patent metadata

Field	Value
Publication number	US-12001437-B2
Application number	US-202217953048-A
Country	US
Kind code	B2
Filing date	Sep 26, 2022
Priority date	Sep 26, 2022
Publication date	Jun 4, 2024
Grant date	Jun 4, 2024

Abstract

Methods and systems for generating and using a semantic index are provided. In some examples, content data is received. The content data includes a plurality of subsets of content data. Each of the plurality of subsets of content data are labelled, based on a semantic context corresponding to the content data. The plurality of subsets of content data and their corresponding labels are stored. The plurality of subsets of content data are grouped, based on their labels, thereby generating one or more groups of subsets of content data. Further, a computing device is adapted to perform an action, based on the one or more groups of subsets of content data.

First claim

What is claimed is:

1. A method for generating a semantic index, the method comprising:
receiving content data, the content data comprising a plurality of subsets of content data;
labeling each of the plurality of subsets of content data, based on a semantic context corresponding to the content data;
generating a feature vector for each of the labels corresponding to a respective one of the plurality of subsets of content data;
storing the plurality of subsets of content data and their corresponding labels;
grouping the plurality of subsets of content data, based on their labels, thereby generating one or more groups of subsets of content data, wherein the grouping of the plurality of subsets of content data, based on their labels, comprises:
determining a distance between each of the feature vectors;
determining that one or more of the distances are less than a predetermined threshold; and
grouping together the feature vectors with distances therebetween that are less than the predetermined threshold, thereby grouping together the respective subsets of the plurality of subsets of content data to which the feature vectors correspond; and

adapting a computing device to perform an action, based on the one or more groups of subsets of content data.

2. The method of claim 1 , wherein the plurality of subsets of content data and their corresponding labels are stored in a database, and wherein the action comprises returning the database comprising the one or more groups of subsets of content data.

3. The method of claim 1 , wherein the plurality of subsets of content data comprise a first type of content data and a second type of content data.

4. The method of claim 3 , wherein the first type of content data and the second type of content data are different types of content data from the group of: audio content data, visual content data, gaze content data, weather content data, news content data, time content data, people content data, and location content data.

5. The method of claim 1 , wherein the labeling of each of the plurality of subsets of content data comprises:
providing each of the plurality of subsets of content data and the semantic context to a machine-learning model, the machine-learning model comprising at least one of a natural language processor or a vision processor; and
receiving, from the machine-learning model, a respective label corresponding to each of the plurality of subsets of content data.

6. The method of claim 1 , wherein a timestamp is stored with each of the plurality of subsets of content data and their corresponding labels.

7. The method of claim 1 , wherein the action comprises annotating one or more elements on a display of a computing device.

8. The method of claim 1 , wherein the action comprises generating an email corresponding to the received content data.

9. The method of claim 1 , wherein the action comprises generating a calendar entry corresponding to the received content data.

10. The method of claim 1 , wherein the action comprises populating a clipboard with a document corresponding to the received content data.

11. A method for retrieving a search result from a semantic index, the method comprising:
generating a user-interface;
receiving, via the user-interface, a query, the query comprising information corresponding to at least two different content types; and
receiving, from a semantic index, a search result corresponding to the query,
wherein the semantic index is generated using content data that is stored and labelled, based on semantic context, the content data comprising a plurality of subsets of content data corresponding to the at least two different content types, and the storing and labelling comprising grouping the plurality of subsets of content data by:
determining a distance between feature vectors generated for labels corresponding to the plurality of subsets of content data;
determining that one or more of the distances are less than a predetermined threshold; and
grouping together the feature vectors with distances therebetween that are less than the predetermined threshold, thereby grouping together the respective subsets of the plurality of subsets of content data to which the feature vectors correspond.

12. The method of claim 11 , further comprising:
prior to receiving the search result, providing a suggested query to reduce an expected number of search results for the query.

13. The method of claim 11 , wherein the at least two different content types are from the group of: a person, a time, a location, audio content, visual content, weather, and a device.

14. A system for generating a semantic index, the system comprising:
at least one processor;
memory storing instructions that, when executed by the at least one processor, causes the system to perform a set of operations, the set of operations comprising:
receiving content data, the content data comprising a plurality of subsets of content data, the plurality of subsets of content data comprising a first subset of content data corresponding to a virtual type of content and a second subset of content data corresponding to a physical type of content;
labeling each of the plurality of subsets of content data, based on a semantic context corresponding to the content data;
generating a feature vector for each of the labels corresponding to a respective one of the plurality of subsets of content data;
grouping the plurality of subsets of content data, based on their labels, thereby generating one or more groups of subsets of content data, wherein the grouping of the plurality of subsets of content data, based on their labels, comprises:
determining a distance between each of the feature vectors;
determining that one or more of the distances are less than a predetermined threshold; and
grouping together the feature vectors with distances therebetween that are less than the predetermined threshold, thereby grouping together the respective subsets of the plurality of subsets of content data to which the feature vectors correspond; and

performing an action, based on the one or more groups of subsets of content data.

15. The system of claim 14 , further comprising one or more of a microphone, a camera, or a global positioning system (GPS), and wherein the second subset of content data comprise one or more from the group of: audio content data, visual content data, gaze content data, and location content data.

16. The system of claim 14 , wherein the at least one processor comprises a first processor of a first device and a second processor of a second device, wherein the first subset of the plurality of subsets is received via the first device, and wherein the second subset of the plurality of subsets is received via the second device.

17. The system of claim 14 , wherein the labeling of each of the plurality of subsets of content data comprises:
providing each of the plurality of subsets of content data and the semantic context to a machine-learning model, the machine-learning model comprising at least one of a natural language processor or a vision processor; and
receiving, from the machine-learning model, a respective label corresponding to each of the plurality of subsets of content data.

18. The system of claim 14 , wherein the set of operations further comprises:
generating a user-interface;
receiving, via the user-interface, a query, the query comprising information corresponding to two or more elements, the two or more elements being from at least two different content types; and
receiving, from a

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Sommerlade Eric Chris Wolfgang
• Pradeep Vivek
• Bathiche Steven N
• Luquetta-Fish Nathan

Classifications

• G06F16/24575 Primary

  using context · CPC title

• G06F16/2228

  Indexing structures · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title