Re: [PATCH 8/25] msi: Simplify the msi irq limit policy.

[ebiederm@xmission.com (Eric W. Biederman)]

Roland Dreier <rdreier@cisco.com> writes:

>  > Only the s2io driver even takes advantage of this feature
>  > all other drivers have a fixed number of irqs they need and
>  > bail if they can't get them.
>
> My todo list for the mthca (InfiniBand HCA) driver includes adding
> support for more event queues.  When I do that, I'll likely want to
> try to get something on the order of number_of_cpus plus two or three
> MSI-X vectors, and fall back to a lower number of vectors if that
> allocation fails.

Allowing drivers that can take advantage of large numbers of
irqs is part of this patch.  To be clear the method still supports
allocate a bunch of irqs and then falling back.

We have to kinds of drivers.  Those that allocate a lot of irqs and
allocate fewer if they can't get them, and those just allocate a
couple and fail if they can't get them. 

The policy I deleted tries to be fair, and give all drivers the
same number of irqs.  What I left us with is a simple first
come first serve policy.

To do better you need an accurate count and you need to separate
out the various kinds of drivers, and you need a shortage of
irqs.

Currently x86_64 hardware supports up to 244*NR_CPUS irqs.  I
don't expect we will exceed that limit any time soon even with
a first come first serve policy.  

The worst case I can think of with your proposed irq allocation
policy in the mthca driver is a 128 cpu machine with 128 IB cards
in it.  That would just barely fail with every driver allocating
3 IRQs per cpu, and it could be trivially fixed by putting in
dual core cpus :)

So when we exceed the limit on the number of IRQs we actually
have then it probably makes sense to see if a policy more
aggressive than first come first serve makes sense.  But until
then it is a waste of time and we should be concentrating
our efforts on making more IRQs usable.

Eric