Storage device and operating method of memory controller

Patent metadata

Field	Value
Publication number	US-11989125-B2
Application number	US-202017060737-A
Country	US
Kind code	B2
Filing date	Oct 1, 2020
Priority date	Feb 11, 2020
Publication date	May 21, 2024
Grant date	May 21, 2024

Abstract

A storage device includes a memory device including a plurality of memory blocks and a memory controller. The memory controller is configured to control a garbage collection operation of the memory device. The memory controller is also configured to select a victim block and a target block for performing the garbage collection operation among the plurality of memory blocks, according to state information of each of the plurality of memory blocks and feature information of data written in each of the plurality of memory blocks, by using a reinforcement learning model.

First claim

What is claimed is:

1. A storage device comprising:
a memory device comprising a plurality of memory blocks; and
a memory controller configured to
control a garbage collection operation of the memory device,
select a victim block and a target block for performing the garbage collection operation among the plurality of memory blocks, according to state information of each of the plurality of memory blocks and feature information of data written in each of the plurality of memory blocks, by using a reinforcement learning model that includes selecting one action that maximizes a reward based on a write amplification factor (WAF), wherein the WAF is calculated based on a ratio of data requested by a host to data written to the memory device, and
provide an address, a command, and a control signal to the memory device based on the selected victim block and the selected target block,

wherein data stored in the plurality of memory blocks are rearranged based on the provided address, the command, and the control signal, and
wherein the memory controller includes
a garbage collection (GC) manager configured to determine whether to perform the garbage collection operation, and
a GC policy generator that selects the victim block and the target block by using the reinforcement learning model and outputs the control signal to the selected victim block and the selected target block to the GC manager.

2. The storage device of claim 1 , wherein the memory controller further comprises an input/output unit configured to generate the feature information of write data received from the host and output the generated feature information to the GC manager.

3. The storage device of claim 1 , wherein the GC manager performs the garbage collection operation when the number of free memory blocks capable of writing data is less than a threshold value.

4. The storage device of claim 1 , wherein each of the plurality of memory blocks comprises at least one page, and
the memory device copies a valid page of the victim block to a free page of the target block based on a garbage collection signal received from the memory controller.

5. The storage device of claim 1 , wherein each of the plurality of memory blocks comprises at least one page, and
the state information comprises at least one of: the number of valid pages among pages of the plurality of memory blocks, a performance of the memory device, expected life times of the plurality of memory blocks, and erasing elapsed times of the plurality of memory blocks.

6. The storage device of claim 1 , wherein the feature information comprises at least one of: an expected life time of data input from the host, a continuity of the data input from the host, a multistream identifier (ID) assigned according to a use frequency of the data.

7. The storage device of claim 1 , wherein the memory controller further comprises a multistream manager configured to classify data input from the host according to a use frequency.

8. The storage device of claim 1 , wherein the victim block and the target block are selected based on a ratio of data requested by the host to data actually written to the memory device.

9. A storage device comprising:
a memory device comprising a plurality of memory blocks; and
a memory controller configured to control the memory device and perform garbage collection by performing reinforcement learning that includes selecting one action that maximizes a reward based on a write amplification factor (WAF), based on first information indicating a state of the memory device before writing data and second information indicating a state of the memory device after writing data, and to include
a host interface for interfacing between a host and the memory controller; and
a flash interface for interfacing between the memory controller and the memory device,
wherein the WAF is calculated based on a ratio of data requested by the host to data written to the memory device,
wherein the garbage collection includes rearranging data stored in the plurality of memory blocks, and
wherein the memory controller includes
a garbage collection (GC) manager configured to determine whether to perform the garbage collection, and
a GC policy generator that selects a victim block and a target block by using the reinforcement learning model and outputs the control signal to the selected victim block and the selected target block to the GC manager.

10. The storage device of claim 9 , wherein each of the plurality of memory blocks comprises at least one page, and
the first information comprises at least one of: the number of valid pages among pages, a performance of the memory device, a continuity of data input from the host, and a multistream identifier (ID) assigned according to a use frequency of the data.

11. The storage device of claim 9 , wherein the second information comprises at least one of: an expected life time of data input from the host, an expected life time of the memory block, and an erasing elapsed time of the memory block.

12. An operating method of a memory controller, the operating method comprising:
receiving write data;
establishing a policy as a result of performing reinforcement learning that includes selecting one action that maximizes a reward based on a write amplification factor (WAF), based on state information of a memory device;
determining, by a garbage collection (GC) manager, whether to perform a garbage collection operation based on the policy; and
selecting, using a GC policy generator, a victim block and a target block by using the reinforcement learning model and outputting the control signal to the selected victim block and the selected target block to the GC manager based on the policy,
wherein the WAF is calculated based on a ratio of data requested by a host to data written to the memory device.

13. The operating method of claim 12 , wherein the establishing a policy comprises:
assigning the reward for a result of an action according to a state based on the state information;
calculating an expected value of the reward; and
establishing the policy based on a maximum value of the expected value.

14. The operating method of claim 13 , wherein the reward is assigned based on a ratio of data requested by the host to data actually written to the memory device.

15. The operating method of claim 12 , wherein the establishing a policy comprises determining the victim block and the target block according to the policy.

16. The operating method of claim 15 , wherein each of the victim block and target block comprises at least one page, and
the garbage collection operation comprises:
searching for a valid page of the victim block;
searching for a free page of the target block; and
copying the valid page to the free page.

17. The operating method of claim 12 , further comprising:
comparing the number of free memory blocks of the memory device capable of writing the write data to a threshold value; and
performing the garbage collection operation when the threshold value is greater than the number of free memory blocks.

18. The operating method of claim 12 , further comprising:
generating first information indicating a state before writing data among the state information; and
generating second information indicating a state after writing data among the state information.

19. The operating method of claim 18 , wherein the establishing a policy comprises performing the reinforcement learning based on the first information and the second information.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jang Sungmin
• Jung Kibeen

Classifications

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06F12/0253 Primary

  Garbage collection, i.e. reclamation of unreferenced memory · CPC title

• G06F3/0604

  Improving or facilitating administration, e.g. storage management · CPC title

• G06F3/064

  Management of blocks · CPC title