Method and device for storing high-concurrency data

Patent metadata

Field	Value
Publication number	US-11327945-B2
Application number	US-201615542088-A
Country	US
Kind code	B2
Filing date	Jan 7, 2016
Priority date	Jan 8, 2015
Publication date	May 10, 2022
Grant date	May 10, 2022

Abstract

A method and device for storing high-concurrency data. The method comprises: receiving high-concurrency data sent from a plurality of clients (110); pushing the high-concurrency data into a primary data queue, and responding to a corresponding client (120); consuming the high-concurrency data in the primary data queue by using multithreading (130); fragmenting the high-concurrency data according to the number of local queues (140); pushing the fragmented high-concurrency data into the local queues (150); and consuming the high-concurrency data in the local queues and storing the high-concurrency data in a database (160). By means of data asynchronous storage, high-concurrency data is temporarily stored in the primary data queue and the local queues, thereby alleviating pressure on the database, avoiding the problem of a database crash in the case of a high-concurrency storage, and improving the efficiency of storing high-concurrency data.

First claim

The invention claimed is:

1. A method for storing data, the method comprising:
receiving, from a plurality of clients, data to store at a database; and
when requests to store data at the database are received at a predetermined rate
pushing the data in a primary data queue, and responding to the plurality of clients the data was received from;
removing the data from the primary data queue using multithreading assigned by a thread pool;
fragmenting the data into a number of fragments, the number of fragments being the same as a number of local queues;
pushing the fragmented data into the local queues; and
removing the data from the local queues and storing the data in the database,
wherein fragmenting the data based on a number of local queues comprises:
performing a modulo operation on a primary key of the data by the number of local queues; and
fragmenting the data based on a result of the modulo operation, and

wherein removing the data from the local queues and storing the data in the database comprises:
removing the data from the local queues using timed scheduling; and
storing the removed data in the database.

2. The method according to claim 1 , wherein pushing the fragmented data into the local queues comprises:
pushing the fragmented data into a local queue having a serial number corresponding to the result of the modulo operation.

3. A hardware server for storing data, the hardware server configured to:
receive, from a plurality of clients, data to store at a database; and
when requests to store data at the database are received at a predetermined rate
push the data into a primary data queue, and respond to the plurality of clients the data was received from;
remove the data from the primary data queue using multithreading assigned by a thread pool;
fragment the data into a number of fragments, the number of fragments being the same as a number of local queues;
push the fragmented data into the local queues; and
remove the data from the local queues and store the data in the database,

wherein the hardware server is configured to fragment the data based on a number of local queues by
performing a modulo operation on a primary key of the data by the number of local queues; and
fragmenting the data based on a result of the modulo operation, and

wherein the hardware server is configured to remove the data from the local queues and store the data in the database by
removing the data from the local queues using timed scheduling; and
storing the removed data in the database.

4. The hardware server according to claim 3 , wherein the hardware server is configured to push the fragmented data into the local queues by pushing the fragmented data into a local queue having a serial number corresponding to the result of the modulo operation.

Assignees

• Beijing Jingdong Shangke Information Technology Co Ltd
• Beijing Jingdong Century Trading Co Ltd

Inventors

• Lan Jian

Classifications

• G06F9/485

  Task life-cycle, e.g. stopping, restarting, resuming execution (G06F9/4881 takes precedence) · CPC title

• G06F9/461

  Saving or restoring of program or task context · CPC title

• G06F9/45504

  Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators · CPC title

• G06F16/24578 Primary

  using ranking · CPC title

• G06F9/546

  Message passing systems or structures, e.g. queues · CPC title