1. PCI Engineering Change Notice – MSI-X
TITLE: MSI-X
DATE: June 10, 2003
AFFECTED DOCUMENT(S): PCI Local Bus Specification, Revision 2.3 & 3.0 Draft SPONSOR: Joe Cowan; Hewlett-Packard Company
1.1. Summary of the Functional Changes
2018高考英语答案Changes are to the PCI Local Bus Specification Revision 2.3, relead March 29, 2002. The changes planned for MSI in PCI 2.3 will also be integrated into the draft PCI 3.0 specification, which has already undergone membership review.
Extend the current MSI functionality to support a larger number of MSI vectors, plus a parate and independent Message Address and Message Data for each MSI vector. Allow more flexibility when SW allocates fewer MSI vectors than requested by HW. Enable per-vector masking capability.
Compared to the MSI-X ECN that underwent membership review in September 2002, this version has t
he following notable changes: (1) allows the MSI-X Table to be placed into general purpo read/write memory on a device, (2) collects the per-vector Pending Bits into parate Pending Bit Array (PBA), (3) allows different MSI-X vectors to have different Upper Message Address values, (4) restricts the programming model to permit only full DWORD or QWORD transactions to access the MSI-X Table and PBA, (5) adds a new MSI capability structure with 32-bit Message Address and per-vector masking, and (6) adds Function Masking capability.
Compared to the MSI-X ECN that underwent membership review in February 2003, this version primarily has only clarifications and additional implementation notes. There is only one mantic change, that of changing the MSI-X Capability ID from 0Dh to 11h.
1.2. Benefits
1. An advanced device can deliver interrupts to any processor in an SMP platform, even when
the number of processors exceeds 32 (the current MSI vector limit).
2. By supporting parate and independent Message Address/Data for each MSI vector, an
advanced device can target interrupts to different processors in an SMP without relying on a re-vecto
ring table in the chip t.
dark moon
1.3. Asssment of the Impact
majoring
Defines an optional new "MSI-X" capability structure. New devices can implement both the old and new MSI capability structures. Old SW can u the old MSI capability with complete backward compatibility. See attached pages, which show all required changes to the MSI ction in Chapter 6.
1.4. Analysis of the Hardware Implications
New devices that require the extended MSI capabilities can implement the new functionality. Requires new MSI-X capability structure, MSI-X Table, Pending Bit Array, and associated control logic. Can leverage existing MSI implementation logic. Does not require u of any rerved pins, nor define any new commands.
1.5. Analysis of the Software Implications
New devices can support both old and new MSI capabilities. Old SW can continue to u old MSI capability for full backward compatibility. Requires new SW to u new MSI-X capabilities.
1.6. Additional Description and Rationale
There are at least two class of upcoming adapters that require more flexibility in delivering interrupts over what MSI provides today. First there's the class that supports many (i.e., hundreds or thousands of) request and completion queues, with granularity down to a per-process level or finer. On SMP systems that support process affinity, it's of great benefit for a completion queue that's owned by a given process to have its associated completion interrupts delivered to the processor that typically runs that process. To support this well, an adapter function needs to support at least as many MSI vectors as there are processors in the SMP system, so that each processor can have at least one dedicated MSI vector by the adapter function. SMPs supporting 64 processors are becoming common today, and much larger ones are envisioned for the future. The current MSI limit of 32 vectors per adapter function isn't adequate to support current and future large SMP systems for this class of adapters. InfiniBand Architecture (IBA) Host Channel Adapters (HCAs) are a good example of this adapter class. IBA HCAs typically support many thousands of completion queues (CQs). With apps doing OS-bypass with IBA messaging and/or remote DMA, each CQ is typically owned by a single process. When a process allocates a CQ, it's reasonable on an OS that supports process affinity for the driver to assign a specific MSI vector to that CQ bad upon OS policy.
Another class of adapters benefiting from extended MSI functionality are tho who external links
support multiple Quality-of-Service (QoS) levels, such as Ethernet NICs with special support for 802.1p. An adapter function from this class may not need a large number of distinct MSI vectors (maybe only 4 or 8), but it's important for the different MSI vectors to be directed to different processors in an SMP system so that different processors can be assigned to handling different QoS levels.
While current MSIs can be directed to different processors by appropriate re-vectoring logic in chipts, not all chipts implement such re-vectoring logic. Some chipts deliver interrupts by merely "forwarding" MSIs from a PCI bus over to the processor "front-side bus" or equivalent. Having all MSIs associated with a single adapter function u the same message address means that all of the function's MSI vectors on such systems will go to the same processor. A lack of control over the full message data value can be limiting as well. Allowing different MSI vectors for the same adapter function each to have an arbitrary address and arbitrary data allows an adapter to target interrupts to different processors on such systems. Moreover, removing the dependence on non-standardized re-vectoring logic in chipts makes it more worthwhile for adapter vendors to implement support for multiple MSI vectors per function, and OSVs to implement support for features such as interrupt assignment bad upon processor affinity. Details of MSI-X ECN Changes
For details of the changes (with changes highlighted) e attached “Section 6.8 - Message Signaled Interrupts”, as it would appear in “Chapter 6 - Configuration Space” for the Conventional PCI 2.3 Specification.
Revision 2.3ldy lickem
189Chapter 6 Configuration Space
6.8. Message Signaled Interrupts
Message Signaled Interrupts (MSI) is an optional feature that enables a device function to request rvice by writing a system-specified message data value to a system-specified address (using a PCI DWORD memory write transaction). The transaction address
specifies the message destination and the transaction data specifies the message. System software initializes the message destination address and message data (from here on referred to as the “vector”) during device configuration, allocating one or more non-shared messagevector s to each MSI capable function.
Since the target of the transaction cannot distinguish between an MSI write transaction and any othe
r write transaction, all transaction termination conditions are supported. Therefore, a n MSI write transaction can be terminated with a Retry, Master-Abort, Target-Abort, or normal completion (refer to Section 3.3.3.2.).
It is recommended that devices implement interrupt pins to provide compatibility in systems that do not support MSI (devices default to interrupt pins). However, it is expected that the need for interrupt pins will diminish over time. Devices that do not support interrupt pins due to pin constraints (rely on polling for device rvice) may implement messages to increa performance without adding additional pins. Therefore, system configuration software must not assume that a message capable device has an interrupt pin.
cramerInterrupt latency (the time from interrupt signaling to interrupt rvicing) is system dependent. Consistent with current interrupt architectures, message signaled interrupts do not provide interrupt latency time guarantees. MSI-X defines a parate optional extension to basic MSI functionality. Compared to MSI, MSI-X supports a larger maximum number of vectors per function, the ability for software to control aliasing when fewer vectors are allocated than requested, plus the ability for each vector to u an independent address and data value, specified by a table that resides in Memory Space. However, most of the other characteristics of MSI-X are identical to tho of MSI. MSI and M
SI-X each support per-vector masking. Per-vector masking is an optional extension to MSI, and a standard feature with MSI-X. A function that supports the per-vector masking extension to MSI is still backward compatible with system software that is unaware of the extension. MSI-X also supports a Function Mask bit, which when t masks all of the vectors associated with a function.
一分钟趣味学习法
Per-vector masking is managed through a Mask and Pending bit pair per MSI vector or MSI-X Table entry. An MSI vector is masked when its associated Mask bit is t. An MSI-X vector is masked when its associated MSI-X Table entry Mask bit or the MSI-X Function Mask bit is t. While a vector is masked, the function is prohibited from nding the associated message, and the function must t the associated Pending bit whenever the function would otherwi nd the message. When software unmasks a vector who associated Pending bit is t, the function must schedule nding the associated message, and clear the Pending bit as soon as the message has been nt.
Revision 2.3harper
A function is permitted to implement both MSI and MSI-X, but system software is
prohibited from enabling both at the same time. If system software enables both at the
same time, the result is undefined.
For the sake of software backward compatibility, MSI and MSI-X u parate and
independent capability structures. On functions that support both MSI and MSI-X,
system software that supports only MSI can still enable and u MSI without any
modification. MSI functionality is managed exclusively through the MSI Capability
Structure, and MSI-X functionality is managed exclusively through the MSI-X Capability Structure.
6.8.1. Message MSI Capability Structure
The capabilities mechanism (refer to Section 6.7.) is ud to identify and configure a n
MSI or MSI-X capable device. The MSI capability structure is described in the current
ction. The MSI-X capability structure is described in Section 6.8.2.
The message MSI capability structure is illustrated in Figure 6-1. Each device function
trick or treat是什么意思that supports MSI (in a multi-function device) must implement its own MSI capability
structure. More then than one MSI capability structure per function is prohibited, but a
Capability Structure for 32-bit Message Address and Per-vector Masking
31 16 15 8 7 0
Message Control Next Pointer Capability ID Capability Pointer
Message Address Capability Pointer + 04h
Rerved Message Data Capability Pointer + 08h
Mask Bits Capability Pointer + 0Ch
Pending Bits Capability Pointer + 10h
190
Revision 2.3
191 Capability Structure for 64-bit Message Address and Per-vector Masking 31 16 15 8 7 0 Message Control Next Pointer Capability ID Capability Pointer Message Address Capability Pointer + 04h Message Upper Address Capability Pointer + 08h Rerved Message Data Capability Pointer + 0Ch Mask Bits Capability Pointer + 10h Pending Bits Capability Pointer + 14h Figure 6-1: Message Signaled InterruptMSI Capability Structure s
To request rvice, an MSI function writes the contents of the Message Data register to the address specified by the contents of the Message Address register (and, optionally, the Message Upper Addr
ess register for a 64-bit message address). A read of the address specified by the contents of the Message Address register produces undefined results.
A function supporting MSI implements one of four MSI Capability Structure layouts illustrated in Figure 6-1, depending upon which optional features are supported. If a function supports 64-bit addressing (DAC) when acting as a master, the function is required to implement 64-bit addressing. The capability structure for a 32-bit message address (illustrated in Figure 6-1) is
implemented if the function supports a 32-bit message address. The capability structure for a 64-bit message address (illustrated in Figure 6-1) is implemented if the function supports a 64-bit message address. If a device supports MSI and the device supports
64-bit addressing (DAC) when acting as a master, the device is required to implement the 64-bit message address structure.
The message control register indicates the function’s capabilities and provides system software control over MSI.
Each field is further described in the following sub-ctions. Rerved registers and bits always retu
rn 0 when read and write operations have no effect. Read-only registers return valid data when read and write operations have no effect.
6.8.1.1. Capability ID for MSI
7::0 CAP_ID The value of 05h in this field identifies the function as
message signaled interruptbeing MSI capable. This
field is read only.
plmn
6.8.1.2. Next Pointer for MSIflavor
7::0 NXT_PTR Pointer to the next item in the capabilities list. Must
be NULL for the final item in the list. This field is
read only .
