Re: MSI prototype

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

:On 2/21/06, Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx> wrote:
:..
:>     I think (3) isn't a big deal if the MSI irq abstraction is given
:>     the 'number of vectors' required and returns just the base vector,
:>     letting the driver do base+0, base+1, base+2 for the individual
:>     vectors.  One resource would thus represent all N requested contiguous
:>     IRQs.
:
:That sounds fine for MSI, but how do you handle MSI-X which doesn't
:have the same restriction for contiguous vectors? I can easily imagine
:having a fragmented vector space in which there are enough available
:vectors to satisfy a MSI-X request, but none of the vectors are
:contiguous so the allocation fails. This would defeat one of the big
:advantages of MSI-X over MSI. So I'm suspicious that we would still
:want multiple interrupt resources.

    I wouldn't call it an advantage, at least not in regards to contiguous
    vs non-contiguous.  Since we have a large 256 entry vector space to work
    with there is no reason to complicate matters by allowing devices to
    reserve/allocate non-contiguous blocks of vectors.

:>     Reservation and allocation can be solved by removing the software
:>     limits on the number of supported IRQs.  In 1.4 I separated out the
:
:I'm not totally sure I follow how this fixes the problem, so let me
:ask a follow up question. Are you thinking that the bus driver would
:pre-assign all the vectors each device can use? I can imagine this
:being one approach to (1), but does this fix (2) as well?

    I was thinking that we would use the same resource reservation/allocation
    model we use now.

:...
:>     My understanding of MSI(X) is that it basically treats the interrupt
:>     as edge-triggered.  On trigger it writes the specified data to the
:
:Yup
:
:>     specified address which, I assume, would be an IOAPIC or LAPIC address?
:
:LAPIC (0xfee00000 + 0xXXX)
:
:>     I can't imagine how it would work with an IOAPIC with its stupid
:>     register window crap, so I assume the MSI(X) address and data would
:>     inject an ICR command into the LAPIC.  I don't quite understand how
:>     the device is able to do that without polling the 'S'end Pending
:>     bit in the ICR, though.
:
:My guess is that the memory controller clues the processor into what
:is happening out-of-band. Since all the messages are written to
:essentially the same address, the memory controller decodes this
:address as an interrupt (vs a normal memory access) and passes the 16
:bit data (which includes the vector) to the processor.
:
:---chuck

    I looked at the linux code but couldn't figure out what they were
    pointing the devices to.  The code is pretty enough, but almost
    entirely undocumented.

    The MSI vector is either doing an actual write into the LAPIC's ICR,
    or it is using a separate message-signaled PCI bus protocol transaction
    and just supplying the vector and delivery mode, and that is automatically
    being routed to the LAPIC.

					-Matt
					Matthew Dillon 
					<dillon@xxxxxxxxxxxxx>