Re: Multiple MSI, take 3

[ebiederm@xmission.com (Eric W. Biederman)]

Matthew Wilcox <matthew@wil.cx> writes:

> On Fri, Jul 11, 2008 at 03:06:33AM -0700, Eric W. Biederman wrote:
>> Matthew Wilcox <matthew@wil.cx> writes:
>> 
>> > I'd like to thank Michael Ellerman for his feedback.  This is a much
>> > better patchset than it used to be.
>> 
>> There is a reason we don't have an API to support this.  Linux can not
>> reasonably support this, especially not on current X86.  The designers
>> of the of the AHCI were idiots and should have used MSI-X.
>
> Thank you for your constructive feedback, Eric.  Unfortunately, we have
> to deal with the world as it is, not how we would like it to be.  Since
> I have it running, I'd like to know what is unreasonable about the
> implementation.

At the very least it is setting all kinds of expectations that it
doesn't meet.

In addition the MSI-X spec predates the AHCI device by a long shot.
In general my experience has been that the hardware designers who
really care and have done their homework and can actually take
advantage of multiple irqs have implemented MSI-X.

>> mask/unmask is will likely break because the mask bit is optional
>> and when it is not present we disable the msi capability.
>
> I believe this is fixable.  I will attempt to do so.

Assuming AHCI implements the mask bits.  In the general case this
is not fixable.  I know of several devices that do not implement 
the optional mask bits.

>> We can not set the affinity individually so we can not allow
>> different queues to be processed on different cores.
>
> This is true, and yet, it is still useful.  Just not as useful as one
> would want.

Also a case of mismatched expectations.  The linux irq API allows
irqs to be bound to different cpus individually.  Multi msi does
not meet that contract.

>> So unless the performance of the AHCI is better by a huge amount I don't
>> see the point, and even then I am extremely sceptical.
>
> I don't have performance numbers yet, but surely you can see that
> avoiding a register read in the interrupt path is a large win?

No.  It is not obvious to me.

Eric