Description
What is the DBI gateway and how can I use it in my design?
Configuration
The data bus interface (DBI) gateway is a soft logic solution for serialized access to the DBI register space of the PCIe controller. The DBI gateway manages access to the DBI such that multiple sources could independently access the PCIe registers without collision. The implementation of the DBI gateway is featured as part of the Speedster7t Device Manager for AC7t1500 devices.
The DBI gateway can be enabled in the Achronix Device Manager (ADM) by selecting the appropriate options in the Speedster7t Device Manager IP file and then generating the RTL file that instantiates the Device Manager accordingly. As of ACE 10.2, the DBI gateway can be enabled for PCIE_1 or PCIE_0 by selecting the Enable PCIE_1 Support option or the Enable PCIE_0 Support option respectively. The DBI gateway can be enabled for both PCIE_0 and PCIE_1 at the same time as well.
Note
In ACE releases previous to ACE 10.2, to enable the DBI gateway in the Device Manager, select the Enable PCIE_0 DBI Gateway option or Enable PCIE_1 DBI Gateway option in the ACE Speedster7t Device Manager Configuration Editor. Enable PCIE_0 DBI Gateway option must be selected for PCIE_0 (PCIe x8) DBI access, while Enable PCIE_1 DBI Gateway option must be selected for PCIE_1 (PCIe ×16) DBI access.
Figure: Device Manager with DBI Gateway Enabled for PCIE_1
Addressing
The sections below describes the addressing scheme for the DBI gateway. All accesses should use 4-byte aligned addresses.
DBI Gateway Address
The DBI gateway is addressable using the NoC address of the NAP that the Device Manager is placed at. The address space of the DBI gateway can be determined according to table shown below.
Table: NAP Address Translation
| Address Bit | 41 | 40 | 39 | 38 | 37 | 36 | 35 | 34 | 33 | 32 | 31 | 30 | 29 | 28 | 27 | 26 | 25 | 24 | … | 0 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NAP | 0 | 0 | 0 | 1 | 0 | 0 | 0 | NAP Column | NAP Row | Memory Address | ||||||||||
- Addr[41:35] =
7'b0001000 –Accesses any NAP endpoint in the device. - Addr[34:31] = NAP Column
–For AC7t1500, valid values for this field are 0 to 9, west to east. Columns on the 2D NoC are numbered 1 to 10, west to east. In order to save bits in the address, the number for a column "N" becomes "N-1" for this value (column 3 uses the value of 2). - Addr[30:28] = NAP Row
–For AC7t1500, valid values for this field are 0 to 3, south to north. Rows on the 2D NoC are numbered 1 to 8, south to north. In order to save bits in the address, the number for a row "N" becomes "N-1" for this value (row 5 uses the value of 4).Note
Although there are 8 rows in the AC7t1500, only the southern 4 rows are valid for use by the Speedster7t Device Manager.
- Addr[27:0] = Memory Address
–Passed to the FPGA fabric logic. For details on the DBI gateway memory addresses, see the Memory Map below.Note
The row and column fields in the address for AXI transactions start numbering from 0, whereas placement constraints in ACE use the actual row and column numbers, starting from 1.
To convert from a NAP location to a 2D NoC address, on the Speedster7t AC7t1500 FPGA, use the following equation:
Address = 0x40_0000_0000 + ((COLUMN – 1) << 31) + ((ROW–1) << 28)
Memory Map
The memory address bits[27:0] of the DBI gateway's NAP address space are mapped into sections according to the table below.
Table: DBI Gateway Address Mapping
| DBI Gateway Address Space | [27:23] | 22 | 21 | 20 | 19 | 18 | 17 | 16 | 15 | 14 | 13 | 12 | 11 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PCIe Configuration Header Space and Capability Structures | 0x0 | PCIE_1=0, PCIE_0=1(1) | 0 | 0 | PF[1:0](2) | 0x0 | Register Offset(3) | |||||||||||||||||
| MSI-X Table | 0x0 | PCIE_1=0, PCIE_0=1(1) | 0 | 0 | PF[1:0](2) | 0x0 | 0xD | Register Offset | ||||||||||||||||
| MSI-X PBA | 0x0 | PCIE_1=0, PCIE_0=1(1) | 0 | 0 | PF[1:0](2) | 0x0 | 0xE | Register Offset | ||||||||||||||||
| ATU Registers | 0x0 | PCIE_1=0, PCIE_0=1(1) | 1 | 1 | 0 | 0 | 0 | 0 | Register Offset within the ATU Space | |||||||||||||||
| DMA Register | 0x0 | PCIE_1=0, PCIE_0=1(1) | 1 | 1 | 0 | 0 | 0 | 1 | Register Offset within the DMA Space | |||||||||||||||
|
Table Notes
|
||||||||||||||||||||||||
Accessing PCIe Registers
The PCIe registers are accessible by issuing AXI read or write transactions to the address space of the DBI gateway's NAP. Read and write transactions to the DBI gateway can be initiated from JTAG, AXI initiator logic connected to a NAP, or memory transaction over PCIe. To access the DBI gateway from PCIe requires that a PCIe BAR be configured to map to the NAP address space being used by the DBI gateway. For more information on configuring the PCIe IP to access the DBI gateway see the knowledge base article, " VectorPath PCIe Design Configuration" (requires login to view) and the Bar Mapping section below.
The DBI gateway uses the address space of the NAP that the Device Manager is placed on. Only single, 32-bit transactions to the DBI gateway are supported.
Read Transaction
A read to the PCIe registers can be completed by issuing an AXI read transaction to the DBI gateway address space. Supported AXI read transactions must have a burst size of 32 bits and a burst length of 1. The read transaction completes with an AXI read response that is received by the AXI initiator.
Example
Consider a design with the DBI gateway enabled in the Device Manager and placed on the NAP located at row 4, column 6. The DBI gateway address space will use the NAP address space for row 4, column 6, which is 0x042b0000000. The PCIe configuration header space for the PCIE ×16 (PCIE_1) interface begins at address 0x0 of the DBI gateway address space indicated in the previous table. Therefore, a read to the Vendor ID and Device ID register (offset 0x0 from the base address of the PCIe configuration header space) will be at NoC address 0x042b0000000.
The snippet below shows a 32-bit read to the vendor ID and device ID register using a Tcl read command to the DBI gateway.
cmd> ac7t1500:noc_read 0x042b0000000
00101b69
In this example, consider the DBI gateway mapped to BAR3 as described in BAR Mapping below. The example below shows a 32-bit read to the vendor ID and device ID register using a PCIe read (via acx_pcie_peek_poke in the Achronix SDK) to the DBI gateway.
Note: The acx_pcie_peek_poke tool can be found in the /tools/acx_pcie_peek_poke/ directory of the Achronix SDK v2.1.1 release.
32-bit Read to DBI Gateway via PCIe
$ ./acx_pcie_peek_poke read bar3 0x0
0x00101b59
Error Response
A read transaction that has failed in the DBI gateway will return a SLVERR response on the write response channel as defined by the AXI protocol.
Write Transaction
A write to the PCIe registers can be completed by issuing an AXI write transaction to the DBI gateway address space. Supported AXI write transactions must have a burst size of 32 bits and a burst length of 1. The write transaction is complete once the AXI write response has been received by the AXI initiator.
Example
Consider a design with the DBI gateway enabled in the Device Manager and placed on the NAP located at row 4, column 6. The DBI gateway address space will use the NAP address space for row 4, column 6, which is 0x042b0000000. The PCIe DMA register space for the PCIE ×16 (PCIE_1) interface begins at address 0x0310000 of the DBI gateway address space indicated by the previous table. Therefore, a write to the DMA read engine enable register (offset 0x2c from the base address of the PCIe DMA register space) will be at NoC address
0x042b031002c
.
The snippet below shows a 32-bit write to the DMA read engine enable register using a Tcl write command to the DBI gateway.
cmd> ac7t1500:noc_read 0x042b031002c
00000000
cmd> ac7t1500:noc_write 0x042b031002c 00000001
cmd> ac7t1500:noc_read 0x042b031002c
00000001
In this example, consider the DBI gateway mapped to BAR3 as as described in BAR Mapping below. The second ATU region maps BAR3 offset 0x10000 to NoC address 0x042b0300000. If the target NoC address is 0x042b031002c, then the respective BAR3 offset would be 0x2002c (0x042b031002c − 0x042b0300000 + 0x10000). The example below shows a 32-bit write to the DMA read engine enable register using a PCIe write (via acx_pcie_peek_poke in the Achronix SDK) to the DBI gateway.
32-bit Write to DBI Gateway via PCIe
$ ./acx_pcie_peek_poke write bar3 0x2002c 0x00000001
Error Response
A write transaction that has failed in the DBI gateway will return a SLVERR response on the write response channel as defined by the AXI protocol.
BAR Mapping
For a host system to access the DBI gateway over the PCIe link, the DBI gateway address space must be mapped to a PCIe BAR. Achronix uses BAR3 as the typical BAR for access to the DBI gateway, although any BAR could be used. The full DBI gateway address space can be mapped into a single PCIe BAR or ATU regions can be used to map subsets of the address space. Achronix SDK compatible DBI gateway BAR mappings are shown in the sections below using the ACE 10.2 GUI.
Full Mapping
The full DBI gateway mapping refers to the mapping of the whole DBI gateway address space for an individual PCIe IP to a single BAR. With this mapping all the registers for each physical function can be accessed; mapping this address space will use a BAR size of 4 MB. The Achronix SDK can use this mapping scheme by modifying the Achronix_IP_config.c file in the src/Achronix_IP/ directory of the Achronix SDK. The bar_index field for the ACX_IP_DBI_GATEWAY_FULL constant should be changed to the target BAR.
The full DBI gateway mapping consists of a 4 MB memory size region. The 4 MB region with base address at the target PCIe IP's PCIe configuration header space for the DBI gateway
Example
In this example, the ADM is set to column 6, row 4. The corresponding NAP address is 0x42_b000_0000. The NAP address space is used by the DBI gateway as explained in the Addressing section. An example of BAR3 using the full DBI gateway mapping for each PCIe IP is shown below.
Figure: Full DBI Gateway Mapping for PCIE_1
Figure: Full DBI Gateway Mapping for PCIE_0
Compressed Mapping
By default the Achronix SDK expects a particular mapping of the BAR used for DBI gateway access. The expected mapping is a subset of the DBI gateway address space for a single physical function which may be referenced as the compressed DBI gateway mapping. In the compressed DBI gateway mapping, only the PCIe configuration header space and capability structures for a single physical function, the ATU registers, and the DMA registers are mapped to the BAR. To use a different, user-defined DBI gateway BAR mapping with the Achronix SDK, the Achronix_DBI_gateway.c file (in the src/Achronix_IP/ directory of the Achronix SDK) must be modified to match the user-defined scheme.
The compressed DBI gateway mapping consists of two ATU regions:
- 64 KB region with base address at the target physical function's PCIe configuration header space for the DBI gateway
- 128 KB region with base address at the ATU register space for the DBI gateway
Using the two ATU regions will result in the address table from the PCIe perspective shown below.
Table: DBI Gateway Compressed Address Mapping from PCIe Perspective
| DBI Gateway Address Space | 17 | 16 | 15 | 14 | 13 | 12 | 11 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PCIe Configuration Header Space and Capability Structures | 0 | 0 | 0x0 | Register Offset(†) | ||||||||||||||
| ATU Registers | 0 | 1 | Register Offset within the ATU Space | |||||||||||||||
| DMA Register | 1 | 0 | Register Offset within the DMA Space | |||||||||||||||
|
† Register space for PCIe configuration header and capability structure is within 0x0 to 0x700 |
||||||||||||||||||
Example Using Physical Function 0
In this example, the ADM is set to column 6, row 4 and the target PCIe interface is PCIE_1. The corresponding NAP address is 0x42_b000_0000. The NAP address space is used by the DBI gateway as explained in the Addressing section. An example of BAR3 using the compressed DBI gateway mapping for Physical Function 0 is shown below.
In the PCIe IP Memory Map BAR configurations, set the "Address Mode" to Address Match Mode in order to use the ATU regions with the BAR. Set the number of ATU Regions to 2 for the regions used for compressed DBI gateway mapping.
Figure: BAR3 Setting in Physical Function 0 Memory Map page
In the BAR3 ATU region page, configure the ATU regions with the addresses used for the compressed DBI gateway mapping. To access the PCIe configuration header space and capability structures for Physical Function 0 set the physical function field bits[19:18] to 0.
Figure: DBI Gateway Compressed Mapping for Physical Function 0 on BAR3
Example Using All Physical Functions
In this example, the ADM is set to column 6, row 4 and the target PCIe interface is PCIE_1. The corresponding NAP address is 0x42_b000_0000. The NAP address space is used by the DBI gateway as explained in the "Addressing" section. An example of BAR3 using the compressed DBI gateway mapping for each physical function is shown below. The compressed DBI gateway mapping for a particular physical function will be set in BAR3 for the respective physical function.
Note: If a physical function is not used, then the compressed DBI gateway mapping for that physical function is not required to be mapped to a BAR. Additionally, all physical functions can have BARs that map and access the PCIe Configuration Header Space and Capability Structures for any physical function. Therefore, the compressed DBI gateway mapping for physical function 0 (or any other physical function) can be mapped in a BAR for physical function 0, 1, 2, or 3 as well.
Physical Function 0
In the PCIe IP Memory Map BAR configurations, set the Address Mode to Address Match Mode in order to use the ATU regions with the BAR. Set the number of ATU Regions to 2 for the regions used for the compressed DBI gateway mapping.
Figure: BAR3 Setting in Physical Function 0 Memory Map page
In the BAR3 ATU Region page configure the ATU regions with the addresses used for the compressed DBI gateway mapping. To access the PCIe Configuration Header Space and Capability Structures for physical function 0 set the physical function field bits[19:18] to 0.
Figure: DBI gateway Compressed Mapping for Physical Function 0 on BAR3
Physical Function 1
In the PCIe IP Memory Map BAR configurations, set the Address Mode to Address Match Mode in order to use the ATU regions with the BAR. Set the number of ATU Regions to 2 for the regions used for the compressed DBI gateway mapping.
Figure: BAR3 Setting in Physical Function 1 Memory Map page
In the BAR3 ATU Region page configure the ATU regions with the addresses used for the compressed DBI gateway mapping. To access the PCIe configuration header space and capability structures for physical function 1 set the physical function field bits[19:18] to 1.
Figure: DBI Gateway Compressed Mapping for Physical Function 1 on BAR3
Physical Function 2
In the PCIe IP Memory Map BAR configurations, set the Address Mode to Address Match Mode in order to use the ATU regions with the BAR. Set the number of ATU Regions to 2 for the regions used for the compressed DBI gateway mapping.
Figure: BAR3 Setting in Physical Function 2 Memory Map page
In the BAR3 ATU Region page configure the ATU regions with the addresses used for the compressed DBI gateway mapping. To access the PCIe configuration header space and capability structures for physical function 2 set the physical function field bits[19:18] to 2.
Figure: DBI Gateway Compressed Mapping for Physical Function 2 on BAR3
Physical Function 3
In the PCIe IP Memory Map BAR configurations, set the Address Mode to Address Match Mode in order to use the ATU regions with the BAR. Set the number of ATU Regions to 2 for the regions used for the compressed DBI gateway mapping.
Figure: BAR3 Setting in Physical Function 3 Memory Map page
In the BAR3 ATU Region page configure the ATU regions with the addresses used for the compressed DBI gateway mapping. To access the PCIe configuration header space and capability structures for physical function 3 set the physical function field bits[19:18] to 3.
Figure: DBI Gateway Compressed Mapping for Physical Function 3 on BAR3