Direct access to local memory in a PCI-E device

Patent metadata

Field	Value
Publication number	US-10120832-B2
Application number	US-201514721009-A
Country	US
Kind code	B2
Filing date	May 26, 2015
Priority date	May 27, 2014
Publication date	Nov 6, 2018
Grant date	Nov 6, 2018

Abstract

A method includes communicating between at least first and second devices over a bus in accordance with a bus address space, including providing direct access over the bus to a local address space of the first device by mapping at least some of the addresses of the local address space to the bus address space. In response to indicating, by the first device or the second device, that the second device requires to access a local address in the local address space that is not currently mapped to the bus address space, the local address is mapped to the bus address space, and the local address is accessed directly, by the second device, using the mapping.

First claim

The invention claimed is:

1. A method, comprising:
allocating, to a first device coupled to a bus that operates in accordance with a bus address space, multiple bus addresses of the bus address space;
providing direct access over the bus to a local address space of the first device by defining a mapping between at least some of the addresses of the local address space and the allocated bus addresses of the bus address space;
in response to detecting that a second device requires to access a given local address in the local address space of the first device that is not currently mapped by the defined mapping to any of the allocated bus addresses, modifying the defined mapping to also map the given local address of the first device to a respective bus address among the allocated bus addresses; and
accessing the given local address of the first device directly, by the second device, using the modified mapping of the defined mapping.

2. The method according to claim 1 , wherein modifying the mapping comprises re-mapping one or more addresses of the local address space that were previously mapped to the allocated bus addresses of the bus address space.

3. The method according to claim 1 , wherein accessing the given local address comprises translating an allocated bus address in the bus address space that is mapped to the given local address by the modified mapping.

4. The method according to claim 1 , wherein the second device uses a second address space that includes a second address corresponding to a respective intermediate address in the bus address space that is mapped to the given local address, wherein accessing the given local address comprises translating the second address to the intermediate address and translating the intermediate address to the given local address using the modified mapping.

5. The method according to claim 4 , wherein translating at least one of the second address and the intermediate address comprises holding a respective mapping table.

6. The method according to claim 4 , wherein translating at least one of the second address and the intermediate address comprises requesting translation from an address translation service (ATS).

7. The method according to claim 1 , wherein accessing the given local address comprises sending, over the bus, a command from the first device to the second device indicating that the given local address is mapped within the modified mapping and ready for access.

8. The method according to claim 1 , wherein the second device comprises a network adapter that communicates with a peer network adapter over a communication network, wherein the peer network adapter communicates with the first device using remote direct memory access (RDMA).

9. The method according to claim 1 , wherein the first device comprises a first local memory that is accessed using the local address space, and the second device comprises a second local memory, wherein accessing the given local address comprises transferring data directly between the first local memory and the second local memory.

10. Apparatus, comprising:
first and second devices, which are configured to communicate over a bus in accordance with a bus address space, wherein multiple bus addresses of the bus address space are allocated to the first device,
wherein the first device is configured to provide direct access over the bus to a local address space by defining a mapping between at least some of the addresses in the local address space and the allocated bus addresses of the bus address space,
and wherein, in response to detecting by the first device or the second device that the second device requires to access a given local address in the local address space of the first device that is not currently mapped by the defined mapping to any of the allocated bus addresses, the first device is configured to modify the mapping to also map the given local address to a respective bus address among the allocated bus addresses, and the second device is configured to access the given local address directly, using the modified mapping of the defined mapping.

11. The apparatus according to claim 10 , wherein the first device is configured to modify the mapping by re-mapping one or more addresses of the local address space that were previously mapped to the allocated bus addresses of the bus address space.

12. The apparatus according to claim 10 , wherein the first device is configured to translate an allocated bus address in the bus address space that is mapped to the given local address by the modified mapping.

13. The apparatus according to claim 10 , wherein the second device uses a second address space that includes a second address corresponding to a respective intermediate address in the bus address space that is mapped to the given local address, wherein the second device is configured to access the given local address by translating the second address to the intermediate address, and wherein the first device is configured to translate the intermediate address to the given local address using the modified mapping.

14. The apparatus according to claim 13 , wherein at least one of the first device and the second device is configured to translate the second address or the intermediate address, respectively, by holding a respective mapping table.

15. The apparatus according to claim 13 , wherein at least one of the first device and the second device is configured to translate the second address or the intermediate address, respectively, by requesting translation from an address translation service (STA).

16. The apparatus according to claim 10 , wherein the first device is configured to send a command to the second device, over the bus, indicating that the given local address is mapped within the modified mapping and ready for access.

17. The apparatus according to claim 10 , wherein the second device comprises a network adapter that communicates with a peer network adapter over a communication network, wherein the peer network adapter communicates with the first device using remote direct memory access (RDMA).

18. The apparatus according to claim 10 , wherein the first device comprises a first local memory that is accessed using the local address space, and the second device comprises a second local memory, wherein the first and second devices are configured to transfer data directly between the first local memory and the second local memory.

19. A method, comprising:
communicating between a first device and a second device over a bus that operates in accordance with a bus address space, and allocating for the first device multiple bus addresses of the bus address space, wherein the first device stores data in a storage area in accordance with a logical address space and provides direct access over the bus to a local memory of the first device by defining a mapping between at least some of the addresses of the local memory and the allocated bus addresses of the bus address space;
in response to indicating, by the second device, that the second device requires to access a logical address that stores requested data that resides in the storage area but is not currently available in the local memory, fetching the requested data from the logical address to a local address of the local memory that is mapped within the mapping to a respective bus address allocated in the bus address space; and
accessing the local address directly, by the second device, using the mapping.

20. The method according to claim 19 , wherein fetching the requested data

Assignees

• Mellanox Technologies Ltd
• Mellanox Technologies Ltd

Inventors

• Raindel Shachar
• Burstein Idan
• Bloch Noam
• Raikin Shlomo

Classifications

• G06F15/17331 Primary

  Distributed shared memory [DSM], e.g. remote direct memory access [RDMA] · CPC title

• G06F13/4221

  being an input/output bus, e.g. ISA bus, EISA bus, PCI bus, SCSI bus · CPC title

• G06F13/28 Primary

  using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence) · CPC title