Computer implemented methods for the automated analysis or use of data, including use of a large language model

Patent metadata

Field	Value
Publication number	US-11977854-B2
Application number	US-202318301639-A
Country	US
Kind code	B2
Filing date	Apr 17, 2023
Priority date	Aug 24, 2021
Publication date	May 7, 2024
Grant date	May 7, 2024

Abstract

Methods are provided, such as a method of interacting with a large language model (LLM), including the step of a processing system using a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language, to provide new context data for the LLM, in order to improve the output, such as continuation text output, generated by the LLM in response to a prompt; and such as a method of interacting with a LLM, including the step of providing continuation data generated by the LLM to a processing system that uses a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language, in which the processing system is configured to analyse the continuation output generated by the LLM in response to a prompt to enable an improved version of that continuation output to be provided to a user. Related computer systems are provided.

First claim

The invention claimed is:

1. A method of interacting with a large language model (LLM)-based system, including the step of using, in conjunction with the LLM-based system, a non-LLM processing system programmed to use an unambiguous machine language for one or more of the following:
(a) to enable non-statistical reasoning, including non-statistical abstract reasoning that is open and explainable, to initiate a real-world action; or
(b) to represent a model of the world broadly corresponding to how people describe the world, and to reason in a way that corresponds to how people understand the real world and reason in the real world; or
(c) to enable new facts or new reasoning to be acquired or learnt; or
(d) to enable facts or reasoning to be represented and stored for long-term re-use; or
(e) to perform non-statistical reasoning, including non-statistical abstract reasoning; or
(f) to enable outputs and reasoning to be open and explainable; or
(g) to enable outputs to be accurate, trusted and reliable, when based on inputs and reasoning that are accurate and reliable;
the method including a method of interacting with the large language model (LLM), including the step of providing continuation output data generated by the LLM to the non-LLM processing system that uses a structured, machine-readable representation of data that conforms to the unambiguous machine-readable language, in which the processing system is configured to analyse the continuation output data generated by the LLM in response to a prompt to enable an improved version of the continuation output data to be provided to a user.

2. The method of claim 1 , including the step of using, in conjunction with the LLM-based system, the non-LLM processing system programmed to use the unambiguous machine language to enable new facts or new reasoning to be acquired or learnt.

3. The method of claim 2 , in which the non-LLM processing system learns by representing the continuation generated by the LLM-based system in the unambiguous machine language.

4. The method of claim 2 , in which the LLM-based system generates natural language and the method includes an additional step of translating the natural language into the unambiguous machine language.

5. The method of claim 4 , including
(i) generating a semantic graph from the natural language;
(ii) attempting to match the semantic graph against templates with a graph of similar shape, that already has associated universal language (UL), and finally
(iii) applying direct translation to only those nodes in the graph that match variables in the UL.

6. The method of claim 2 , in which the LLM-based system generates natural language and the method includes an additional step of translating the natural language into the unambiguous machine language, using an intermediate representation (IR) as a middle stage between the source natural language and the destination language.

7. The method of claim 2 , in which the LLM-based system generates natural language and the method includes an additional step of translating the natural language into the unambiguous machine language, using an intermediate representation (IR) as a middle stage between the source natural language and the destination language, using a semantic graph.

8. The method of claim 2 , in which the method includes a step to validate the continuation for factual accuracy.

9. The method of claim 8 , in which the step to validate factual accuracy comprises the step of asking a question in the unambiguous machine language to the non-LLM processing system.

10. The method of claim 8 , in which the step to validate the continuation for factual accuracy includes a check to see if the continuation is contradicted by other facts represented in the unambiguous machine language.

11. The method of claim 1 , wherein the unambiguous machine-readable language is a universal language.

12. The method of claim 1 , in which the processing system translates at least some of the continuation data into the structured, machine-readable representation of data to analyse the translated continuation data.

13. The method of claim 12 , in which the processing system analyses the translated continuation data for at least one of factual inaccuracies, contradictions and scope, and, where necessary or beneficial, provides corrected output for use in the continuation output provided by the LLM, through the processing system providing an updated prompt, that is presented to a user.

14. The method of claim 12 , in which the processing system analyses the translated continuation data and, where necessary or beneficial, provides corrected output for use in the continuation output that is presented to a user.

15. The method of claim 1 , in which the processing system translates the continuation data into the structured, machine-readable representation of data to analyse the translated continuation data.

16. The method of claim 1 , in which the processing system analyses the internal, logical self-consistency, including a correct time ordering of events, of the continuation output and, where necessary or beneficial, provides logically self-consistent output, for use in the continuation output that is provided to a user.

17. The method of claim 1 , in which the processing system analyses internal, logical self-consistency of the continuation output and, where necessary or beneficial, provides logically self-consistent output, for use in the continuation output that is provided to a user.

18. The method of claim 1 , in which the processing system analyses correspondence of the continuation output generated by the LLM to how people understand the real world or reason in the real world and, where necessary or beneficial, provides reasoned output, corresponding to how people understand the real world or reason in the real world, for use in the continuation output that is provided to a user.

19. The method of claim 1 , in which the processing system analyses the continuation output for bias and, where necessary or beneficial, provides reduced bias or bias-free output, for use in the continuation output that is provided to a user.

20. The method of claim 1 , in which the processing system analyses the continuation output for an absence of relevant dynamic or real-time information and, where necessary or beneficial, provides any missing or useful dynamic or real-time information for use in the continuation output that is provided to a user.

21. The method of claim 1 , in which the processing system analyses the continuation output for any reasoned text and, where necessary or beneficial, provides improved reasoned text for use in the continuation output that is provided to a user.

22. The method of claim 1 , in which the LLM is answering a question and in which the processing system analyses a answer to that question in the continuation output and where necessary or beneficial, provides an improved answer for use in the continuation output that is provided to a user.

23. The method of claim 1 , in which a classifier operates to identify when a prompt is likely to result in a continuation output where accuracy is important, and/or when accuracy is important in the continuation output, and to then use the processing system to improve factual accuracy and/or factual scope of that continuation output.

24. The method of claim 1 , in which the continuation output from the LLM is a partial continuation, namely an output made before the

Assignees

• Unlikely Artificial Intelligence Ltd

Inventors

• Tunstall-Pedoe William
• Heywood Robert
• Warren Seth
• Benn Paul
• Reynolds Duncan
• Shah Ayush
• Krnic Luci
• Zhu Ziyi

Classifications

• G06F40/56 Primary

  Natural language generation · CPC title

• G06F40/205

  Parsing · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title