Caching techniques for a database change stream

Patent metadata

Field	Value
Publication number	US-11650985-B2
Application number	US-202117446197-A
Country	US
Kind code	B2
Filing date	Aug 27, 2021
Priority date	May 31, 2019
Publication date	May 16, 2023
Grant date	May 16, 2023

Abstract

Techniques are disclosed relating to caching techniques for processing a database change stream. A computer system may receive change records from a change stream that includes a plurality of records indicating changes to a database table. The change stream may include change records for multiple shards and be accessible by providing one or more position indicators for one or more of the multiple shards to request one or more change records and an updated position indicator. The system may store, for changes to a set of one or more shards, one or more cache entries that include respective groups of change records. The system may request a portion of the change stream by providing a received position indicator. The system may provide one or more cached change records from a cache entry that matches the provided position indicator.

First claim

What is claimed is:

1. A method, comprising:
receiving, by a computer system, change records from a change stream, wherein the change stream includes a plurality of records that indicate changes to a database table, wherein the change stream includes change records for multiple shards, and wherein the change stream is accessible by providing one or more position indicators for one or more of the multiple shards to request one or more change records and an updated position indicator;
storing, by the computer system for changes to a set of one or more shards, one or more cache entries that include respective groups of change records; and
in response to a request for a portion of the change stream, wherein the request includes a position indicator, the computer system providing one or more cached change records from a cache entry that matches the position indicator.

2. The method of claim 1 , further comprising:
storing, by the computer system, fields for multiple different tenants in a database table of a multi-tenant database system; and
converting, by the computer system, one or more received tenant-based requests to requests that are supported by the multi-tenant database system.

3. The method of claim 1 , further comprising:
determining, by the computer system, multiple tenant checkpoints based on processing change records for multiple tenants; and
determining, by the computer system, a combined checkpoint based on the multiple tenant checkpoints.

4. The method of claim 1 , further comprising:
accessing the change stream using multiple different stream processes executed by the computer system.

5. The method of claim 1 , further comprising:
storing, by the computer system, an address for a change processor in a lease table of a database system in association with one or more shards of the change stream handled by the change processor;
wherein the position indicator included in the request is forwarded based on the stored address in the lease table.

6. The method of claim 1 , further comprising:
polling, by the computer system, the change stream using a position indicator, wherein the position indicator includes shard identifiers for one or more shards in the change stream and a sequence number for each shard;
determining, in response to the polling, one or more new change records and an updated position indicator; and
storing, by the computer system, the one or more new change records in one or more cache entries.

7. The method of claim 1 , wherein the match between the cache entry and the provided position indicator is based on a shard identifier and sequence number included in the position indicator and corresponding tag information for the cache entry.

8. The method of claim 1 , wherein the receiving is performed by a caching module of the computer system and the request for the portion of the change stream is initiated by a change processor module.

9. The method of claim 1 , wherein the group of change records in a given cache entry includes change records up to a threshold group size.

10. A non-transitory computer-readable medium having instructions stored thereon that are capable of execution by a computing device to perform operations comprising:
receiving change records from a change stream, wherein the change stream includes a plurality of records that indicate changes to a database table, wherein the change stream includes change records for multiple shards, and wherein the change stream is accessible by providing one or more position indicators for one or more of the multiple shards to request one or more change records and an updated position indicator;
storing, for changes to a set of one or more shards, one or more cache entries that include respective groups of change records; and
in response to a request for a portion of the change stream, wherein the request includes a position indicator, providing one or more cached change records from a cache entry that matches the position indicator.

11. The non-transitory computer-readable medium of claim 10 , wherein the operations further comprise:
storing fields for multiple different tenants in a database table of a multi-tenant database system; and
converting one or more received tenant-based requests to requests that are supported by the multi-tenant database system.

12. The non-transitory computer-readable medium of claim 11 , wherein the operations further comprise:
determining multiple tenant checkpoints based on processing change records for multiple tenants; and
determining a combined checkpoint based on the multiple tenant checkpoints.

13. The non-transitory computer-readable medium of claim 10 , wherein the operations further comprise:
accessing the change stream using multiple different stream processes.

14. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise:
storing an address for a change processor in a lease table of a database system in association with one or more shards of the change stream handled by the change processor.

15. The non-transitory computer-readable medium of claim 14 , wherein the operations further comprise:
polling the change stream using a position indicator, wherein the position indicator includes shard identifiers for one or more shards in the change stream and a sequence number for each shard;
determining, in response to the polling, one or more new change records and an updated position indicator; and
storing the one or more new change records in one or more cache entries.

16. The non-transitory computer-readable medium of claim 10 , wherein the match between the cache entry and the provided position indicator is based on a shard identifier and sequence number included in the position indicator and corresponding tag information for the cache entry.

17. The non-transitory computer-readable medium of claim 10 , wherein the receiving is performed by a caching module and the request for the portion of the change stream is initiated by a change processor module.

18. The non-transitory computer-readable medium of claim 10 , wherein the group of change records in a given cache entry includes change records up to a threshold group size.

19. A system, comprising:
processor circuitry; and
one or more memories having instructions stored thereon that are capable of being executed by the processor circuitry to:
receive change records from a change stream, wherein the change stream includes a plurality of records that indicate changes to a database table, wherein the change stream includes change records for multiple shards, and wherein the change stream is accessible by providing one or more position indicators for one or more of the multiple shards to request one or more change records and an updated position indicator;
store, for changes to a set of one or more shards, one or more cache entries that include respective groups of change records; and
in response to a request for a portion of the change stream, wherein the request includes a position indicator, provide one or more cached change records from a cache entry that matches the position indicator.

20. The system of claim 19 , wherein the instructions are further capable of being executed by the processor circuitry to:
store fields for multiple different tenants in a database table of a multi-tenant database system;
convert one or more received tenant-based requests to requests that are supported by the multi-tenant database system.
determine multiple tenant checkpoints based on p

Assignees

• Salesforce Com Inc
• Salesforce Inc

Inventors

• Busjaeger Benjamin
• Sgroi Michael

Classifications

• G06F16/24552 Primary

  Database cache management · CPC title

• G06F16/2358 Primary

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F16/2282

  Tablespace storage structures; Management thereof · CPC title

• G06F16/24568

  Data stream processing; Continuous queries · CPC title