Generation and implementation of geospatial workflows

Patent metadata

Field	Value
Publication number	US-2025077566-A1
Application number	US-202418816539-A
Country	US
Kind code	A1
Filing date	Aug 27, 2024
Priority date	Aug 28, 2023
Publication date	Mar 6, 2025
Grant date	—

Abstract

Implementations are described herein for automatically generating multimodal geospatial workflows for accomplishing geospatial tasks. In various implementations, a natural language request may be processed based on generative model(s) such as LLM(s) to generate workflow output tokens that identify high-level actions for completing a geospatial task conveyed in the natural language request. First data indicative of the high-level actions may be processed using one or more of the generative models to generate dataset output tokens that identify responsive dataset(s) that likely contain data responsive to the geospatial task. Second data indicative of both the high-level actions and the responsive dataset(s) may be processed based on one or more of the generative models to generate data manipulation output tokens that identify data manipulation instructions for assembling data from the responsive dataset(s) into a response that fulfills the geospatial task.

First claim

What is claimed is:

1 . A method implemented using one or more processors and comprising:
processing a natural language request based on one or more generative models to generate workflow output tokens that identify high-level actions for completing a geospatial task conveyed in the natural language request; processing first data indicative of the high-level actions for completing the geospatial task based on one or more of the generative models to generate dataset output tokens that identify one or more responsive datasets that likely contain data responsive to the geospatial task conveyed in the natural language request; processing second data indicative of both the high-level actions for completing the geospatial task and the one or more responsive datasets based on one or more of the generative models to generate data manipulation output tokens that identify data manipulation instructions for assembling data from the one or more responsive datasets into a response that fulfills the geospatial task; causing the data manipulation instructions to be executed using the one or more responsive datasets to generate the response that fulfills the geospatial task; and causing the response that fulfills the geospatial task to be rendered at one or more computing devices.

2 . The method of claim 1 , wherein the natural language request is processed based on a different generative model than the first data.

3 . The method of claim 1 , wherein the natural language request is processed based on a different generative model than the second data.

4 . The method of claim 1 , wherein the first data is processed based on a different generative model than the second data.

5 . The method of claim 1 , wherein the first data further comprises metadata about a plurality of candidate datasets from which the one or more responsive datasets are identified.

6 . The method of claim 5 , wherein the metadata includes human-curated content describing one or more of the candidate datasets or data indicative of a database schema of one or more of the candidate datasets.

7 . The method of claim 1 , wherein the dataset output tokens indicate, for a plurality candidate datasets from which the one or more responsive datasets are identified, respective measures of usefulness for performing the high-level actions for completing the geospatial task conveyed in the natural language request.

8 . The method of claim 1 , wherein the one or more responsive datasets include a first overhead digital depiction of a geographic area that includes annotations identifying land-based features of the geographic area.

9 . The method of claim 8 , wherein the first overhead digital depiction comprises a raster image.

10 . The method of claim 9 , wherein the raster image is captured by a satellite or a drone.

11 . The method of claim 8 , wherein the first overhead digital depiction comprises a vector-based map of the geographic area.

12 . The method of claim 8 , wherein the one or more responsive datasets comprises a second overhead digital depiction of at least part of the geographic area.

13 . The method of claim 12 , wherein the data manipulation instructions comprise:
instructions to join the first and second overhead digital depictions; or instructions to overlay one of the first and second overhead digital depictions in relation to the other.

14 . The method of claim 1 , wherein the data manipulation instructions comprise source code composed in a high-level programming language, wherein the source code is configured to be executed to obtain data from the one or more responsive datasets and assemble the response to the natural language request.

15 . The method of claim 1 , wherein the one or more responsive datasets comprise at least first and second responsive databases, and wherein the data manipulation instructions comprise instructions to join data from first and second responsive databases.

16 . The method of claim 15 , wherein the instructions to join data from the first and second responsive databases comprise:
structured query language (SQL) instructions; or source code composed in a high-level programming language.

17 . The method of claim 1 , further comprising:
evaluating the data manipulation instructions based on one or more criteria; based on a determination that the data manipulation instructions fail to satisfy one or more of the criteria, assembling an input prompt for one or more of the generative models, wherein the input prompt comprises at least some of the data manipulation instructions and additional information about the determination that the data manipulation instructions fail to satisfy one or more of the criteria; and processing the input prompt using one or more of the generative models to generate new data manipulation instructions.

18 . The method of claim 17 , wherein the data manipulation instructions comprise source code in a high-level programming language, and the one or more criteria include a capability of compiling the source code without error.

19 . The method of claim 1 , wherein one or more of the generative models comprises a large language model (LLM).

20 . A system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to:
process a natural language request based on one or more generative models to generate workflow output tokens that identify high-level actions for completing a geospatial task conveyed in the natural language request; process first data indicative of the high-level actions for completing the geospatial task based on one or more of the generative models to generate dataset output tokens that identify one or more responsive datasets that likely contain data responsive to the geospatial task conveyed in the natural language request; process second data indicative of both the high-level actions for completing the geospatial task and the one or more responsive datasets based on one or more of the generative models to generate data manipulation output tokens that identify data manipulation instructions for assembling data from the one or more responsive datasets into a response that fulfills the geospatial task; cause the data manipulation instructions to be executed using the one or more responsive datasets to generate the response that fulfills the geospatial task; and cause the response that fulfills the geospatial task to be rendered at one or more computing devices.

Assignees

• X Dev Llc

Inventors

• Gupta Ananya
• Murphy Gearoid
• Goncharuk Artem
• Gupta Akshina
• Zhang Haoyu
• Walker Adrian

Classifications

• G06F40/284

  Lexical analysis, e.g. tokenisation or collocates · CPC title

• G06F16/909

  using geographical or spatial information, e.g. location (spatiotemporally dependent retrieval from the web G06F16/9537) · CPC title

• G06F16/387 Primary

  using geographical or spatial information, e.g. location · CPC title

• G06F16/90332 Primary

  Natural language query formulation or dialogue systems · CPC title