regarding your patch "AHCI: Support multiple MSIs"

regarding your patch "AHCI: Support multiple MSIs"

[Jan Beulich]

Alexander,

I'm a bit puzzled by the way ahci_init_interrupts() works, when
looking at a particular hardware implementation having 6 ports
but allowing up to 16 MSIs - obviously, with one IRQ per port,
obtaining 8 MSIs would be sufficient in this case, conserving on
resources. Is there any reason this can't be done (i.e. by the
caller of pci_enable_msi_block_auto() passing in the maximum
desired number)?

Besides that, would it be possible to tweak things so that
/proc/interrupts could show distinct names for the individual
IRQs?

Thanks, Jan

Re: regarding your patch "AHCI: Support multiple MSIs"

[Alexander Gordeev]

On Wed, Apr 10, 2013 at 04:05:30PM +0100, Jan Beulich wrote:
> Alexander,
> 
> I'm a bit puzzled by the way ahci_init_interrupts() works, when
> looking at a particular hardware implementation having 6 ports
> but allowing up to 16 MSIs - obviously, with one IRQ per port,
> obtaining 8 MSIs would be sufficient in this case, conserving on
> resources. Is there any reason this can't be done (i.e. by the
> caller of pci_enable_msi_block_auto() passing in the maximum
> desired number)?

There are few points here.

1. Some (all?) hardware implementations (i.e. Intel) have stupid
logic that falls back to single MSI mode as if MRSM bit was set,
in case # of allocated MSIs is less than # of available MSIs,
even though # of allocated MSIs >= # of ports. I did not find any
chips which could perform otherwise and stuck with this simpler
mode for now.

2. I did implement the approach you described - you can look at it
in my earlier versions of the patchset. But I withdrew the change,
since an update to the generic MSI code was needed, and it would be
hard to convince people to accept changes that nobody consumes.

3. If you aware of any hardware that is able sending multiple MSIs
while the condition holds true:
	# of ports <= # of allocated MSIs < # of available MSIs
...I can revive my changes and post it upstream.

> Besides that, would it be possible to tweak things so that
> /proc/interrupts could show distinct names for the individual
> IRQs?

I am not sure what do you mean here. Once a separate MSI vector is
allocated to a port (to anything in fact) it will show up in
/proc/interrupts as a distinct record.

> Thanks, Jan

HTH

> 

-- 
Regards,
Alexander Gordeev
agordeev@redhat.com

Re: regarding your patch "AHCI: Support multiple MSIs"

[Jan Beulich]

>>> On 10.04.13 at 17:42, Alexander Gordeev <agordeev@redhat.com> wrote:
> On Wed, Apr 10, 2013 at 04:05:30PM +0100, Jan Beulich wrote:
>> Alexander,
>> 
>> I'm a bit puzzled by the way ahci_init_interrupts() works, when
>> looking at a particular hardware implementation having 6 ports
>> but allowing up to 16 MSIs - obviously, with one IRQ per port,
>> obtaining 8 MSIs would be sufficient in this case, conserving on
>> resources. Is there any reason this can't be done (i.e. by the
>> caller of pci_enable_msi_block_auto() passing in the maximum
>> desired number)?
> 
> There are few points here.
> 
> 1. Some (all?) hardware implementations (i.e. Intel) have stupid
> logic that falls back to single MSI mode as if MRSM bit was set,
> in case # of allocated MSIs is less than # of available MSIs,
> even though # of allocated MSIs >= # of ports. I did not find any
> chips which could perform otherwise and stuck with this simpler
> mode for now.
> 
> 2. I did implement the approach you described - you can look at it
> in my earlier versions of the patchset. But I withdrew the change,
> since an update to the generic MSI code was needed, and it would be
> hard to convince people to accept changes that nobody consumes.
> 
> 3. If you aware of any hardware that is able sending multiple MSIs
> while the condition holds true:
> 	# of ports <= # of allocated MSIs < # of available MSIs
> ...I can revive my changes and post it upstream.

I'll have to see if I can check the two or three capable machines
I have, but considering what you describe I'm sure the same will
apply there. Thanks for explaining!

>> Besides that, would it be possible to tweak things so that
>> /proc/interrupts could show distinct names for the individual
>> IRQs?
> 
> I am not sure what do you mean here. Once a separate MSI vector is
> allocated to a port (to anything in fact) it will show up in
> /proc/interrupts as a distinct record.

They have separate lines, but all say just "ahci" at the end (as
opposed to e.g. all the USB consumed ones, which are nicely
distinguishable).

Jan

Re: regarding your patch "AHCI: Support multiple MSIs"

[Alexander Gordeev]

On Wed, Apr 10, 2013 at 04:50:48PM +0100, Jan Beulich wrote:
> They have separate lines, but all say just "ahci" at the end (as
> opposed to e.g. all the USB consumed ones, which are nicely
> distinguishable).

That is seems possible. Will have to check side effects though.

If *-ahci in case of single interrupt and *-ahciN where N is port #
works for you?

> Jan
> 

-- 
Regards,
Alexander Gordeev
agordeev@redhat.com

Re: regarding your patch "AHCI: Support multiple MSIs"

[Jan Beulich]

>>> On 10.04.13 at 18:32, Alexander Gordeev <agordeev@redhat.com> wrote:
> On Wed, Apr 10, 2013 at 04:50:48PM +0100, Jan Beulich wrote:
>> They have separate lines, but all say just "ahci" at the end (as
>> opposed to e.g. all the USB consumed ones, which are nicely
>> distinguishable).
> 
> That is seems possible. Will have to check side effects though.
> 
> If *-ahci in case of single interrupt and *-ahciN where N is port #
> works for you?

Something like that would surely do.

Jan

Re: regarding your patch "AHCI: Support multiple MSIs"

[Jan Beulich]

>>> On 10.04.13 at 17:42, Alexander Gordeev <agordeev@redhat.com> wrote:
> On Wed, Apr 10, 2013 at 04:05:30PM +0100, Jan Beulich wrote:
>> Alexander,
>> 
>> I'm a bit puzzled by the way ahci_init_interrupts() works, when
>> looking at a particular hardware implementation having 6 ports
>> but allowing up to 16 MSIs - obviously, with one IRQ per port,
>> obtaining 8 MSIs would be sufficient in this case, conserving on
>> resources. Is there any reason this can't be done (i.e. by the
>> caller of pci_enable_msi_block_auto() passing in the maximum
>> desired number)?
> 
> There are few points here.
> 
> 1. Some (all?) hardware implementations (i.e. Intel) have stupid
> logic that falls back to single MSI mode as if MRSM bit was set,
> in case # of allocated MSIs is less than # of available MSIs,
> even though # of allocated MSIs >= # of ports. I did not find any
> chips which could perform otherwise and stuck with this simpler
> mode for now.
> 
> 2. I did implement the approach you described - you can look at it
> in my earlier versions of the patchset. But I withdrew the change,
> since an update to the generic MSI code was needed, and it would be
> hard to convince people to accept changes that nobody consumes.

Just checked this, and indeed it behaves the way you describe.
And actually, the ICH specification also describes this in sufficient
detail, in particular stating that the value 011 (for 8 MSIs) is
reserved, causing undefined behavior when used. As odd as it
is, it looks like we need to live with the 10 wasted vectors...

> 3. If you aware of any hardware that is able sending multiple MSIs
> while the condition holds true:
> 	# of ports <= # of allocated MSIs < # of available MSIs
> ...I can revive my changes and post it upstream.

We'll have to see how other devices behave - many express root
ports allow for using 2 MSIs (but clearly there's no room for
wasting vectors here).

Jan