Lakehouse metadata tag creation method, lakehouse metadata tag query method and electronic device

Patent metadata

Field	Value
Publication number	US-12430365-B2
Application number	US-202418942247-A
Country	US
Kind code	B2
Filing date	Nov 8, 2024
Priority date	Nov 9, 2023
Publication date	Sep 30, 2025
Grant date	Sep 30, 2025

Abstract

A lakehouse metadata tag creation method, a lakehouse metadata tag query method and an electronic device are provided. The method includes: after a metadata service gateway determines a metadata tag creation request based on a metadata tag creation parameter provided by a data engine, sending, by the metadata service gateway, the metadata tag creation request to a metadata tag service system, so that the metadata tag service system executes a tag creation logic based on the metadata tag creation parameter carried in the metadata tag creation request, to implement a requirement of tagging one or more pieces of metadata.

First claim

The invention claimed is:

1. A lakehouse metadata tag creation method, applied to a metadata tag service system, the method comprising:
receiving a metadata tag creation request sent by a metadata service gateway, wherein the metadata tag creation request is determined by the metadata service gateway based on a metadata tag creation parameter provided by a data engine, the metadata tag creation parameter comprises a first tag name, a first tag value, and first tag object description information, and the data engine uses a lakehouse metadata management system by means of interface calling; and
executing a tag creation logic based on the metadata tag creation parameter carried in the metadata tag creation request, wherein the tag creation logic comprises:
writing tag name record content corresponding to the first tag name into a tag name record table, wherein the tag name record content comprises a tag name identifier corresponding to the first tag name;
writing tag object record content corresponding to the first tag object description information into a tag object record table, wherein the tag object record content comprises a tag object identifier corresponding to the first tag object description information; and
writing tag value record content corresponding to the first tag value into a tag value record table, wherein the tag value record content is determined based on the first tag value, the tag name identifier, and the tag object identifier.

2. The method according to claim 1 , wherein the metadata tag creation parameter further comprises a first tag set name; and
before the writing tag name record content corresponding to the first tag name into a tag name record table, the tag creation logic further comprises:
writing tag set record content corresponding to the first tag set name into a tag set record table, wherein the tag set record content comprises a set identifier corresponding to the first tag set name; and
determining the tag name record content corresponding to the first tag name based on the set identifier and the first tag name.

3. The method according to claim 1 , wherein the writing tag object record content corresponding to the first tag object description information into a tag object record table comprises:
in response to a non-existence of tag object record content corresponding to the first tag object description information in the tag object record table, writing the tag object record content corresponding to the first tag object description information into the tag object record table.

4. The method according to claim 1 , further comprising:
in response to detecting that the tag creation logic has been executed, generating feedback information corresponding to the metadata tag creation request; and
sending the feedback information to the metadata service gateway, wherein the metadata service gateway is configured to forward the feedback information to the data engine.

5. The method according to claim 4 , wherein
the generating feedback information corresponding to the metadata tag creation request comprises:
combining the first tag value, a set identifier corresponding to the first tag set name, the tag name identifier corresponding to the first tag name, and the tag object identifier corresponding to the first tag object description information to obtain the feedback information.

6. The method according to claim 1 , wherein the metadata tag creation request is a request for creating a tag corresponding to one piece of metadata or a plurality of pieces of metadata;
the one piece of metadata or the plurality of pieces of metadata are managed by a metadata storage system; and
the metadata service gateway is configured to route a metadata processing request sent by the data engine to the metadata storage system, and the metadata storage system is configured to execute data processing logic corresponding to the metadata processing request.

7. A lakehouse metadata tag query method, applied to a metadata tag service system, wherein the metadata tag service system is configured to query a tag created by using the lakehouse metadata tag creation method according to claim 1 ; and
the query method comprises:
receiving a metadata tag query request sent by a metadata service gateway, wherein the metadata tag query request is determined by the metadata service gateway based on a metadata tag query parameter provided by a data engine;
performing tag query processing based on the metadata tag query parameter carried in the metadata tag query request, to obtain a tag query result corresponding to the metadata tag query request; and
sending the tag query result to the metadata service gateway, wherein the metadata service gateway is configured to feed back the tag query result to the data engine.

8. The method according to claim 7 , wherein the metadata tag query parameter comprises a second tag name and second tag object description information; and
a process of determining the tag query result comprises:
searching for a tag name identifier corresponding to the second tag name in a tag name record table;
searching for a tag object identifier corresponding to the second tag object description information in the tag object record table;
querying the tag value record table based on the tag name identifier and the tag object identifier, to obtain a second tag value; and
determining the tag query result based on the second tag value.

9. The method according to claim 8 , wherein the metadata tag creation parameter further comprises a second tag set name; and
before the searching for a tag name identifier corresponding to the second tag name in a tag name record table, the process of determining the tag query result further comprises:
searching for a set identifier corresponding to the second tag set name in a tag set record table; and
the searching for a tag name identifier corresponding to the second tag name in a tag name record table comprises:
querying the tag name record table based on the set identifier and the second tag name, to obtain the tag name identifier corresponding to the second tag name.

10. The method according to claim 9 , wherein the determining the tag query result based on the second tag value comprises:
combining the set identifier, the tag name identifier, the tag object identifier, and the second tag value to obtain the tag query result.

11. An electronic device, comprising:
at least one processor and a memory, the memory being configured to store an instruction or a computer program, and the at least one processor being configured to execute the instruction or the computer program in the memory, so that the electronic device executes a lakehouse metadata tag creation method, comprising:
receiving a metadata tag creation request sent by a metadata service gateway, wherein the metadata tag creation request is determined by the metadata service gateway based on a metadata tag creation parameter provided by a data engine, the metadata tag creation parameter comprises a first tag name, a first tag value, and first tag object description information, and the data engine uses a lakehouse metadata management system by means of interface calling; and
executing a tag creation logic based on the metadata tag creation parameter carried in the metadata tag creation request, wherein the tag creation logic comprises:
writing tag name record content corresponding to the first tag name into a tag name record table, wherein the tag name record content comprises a tag name identifier corresponding to the first tag name;
writing tag object record content corresponding to the first tag object description information into a tag object record tab

Assignees

• Beijing Volcano Engine Technology Co Ltd

Inventors

• Li Mengjun
• Guo Jun
• Sun Ke
• Xu Qing

Classifications

• G06F16/2455

  Query execution · CPC title

• G06F16/284

  Relational databases · CPC title

• G06F16/24553

  of query operations · CPC title

• G06F16/242

  Query formulation · CPC title

• G06F16/2282

  Tablespace storage structures; Management thereof · CPC title