Re: infiniband limit of 32 cards per system?

Roland Dreier
2010-01-27T00:09:13Z

> It looks like we have a hole from [128, 192).
>
> Would it be something as simple as this?

> - IB_UVERBS_BASE_MINOR = 192,
> - IB_UVERBS_MAX_DEVICES = 32
> + IB_UVERBS_BASE_MINOR = 128,
> + IB_UVERBS_MAX_DEVICES = 64

I don't think this is a good idea for two reasons:

- It doesn't take into account the fact that the infiniband_mad and
infiniband_cm drivers will take up more minors if more devices appear
(in the best case, you would only be able to run opensm on the first
32 HCAs or something like that).

- It changes the minor of the first uverbs device, so something like a
system with hardcoded static /dev would break in a mysterious way.

I think unfortunately we have to extend the device # assignment so the
first 32 HCAs get the same minors they would have and then overflow into
some dynamic region.

- R.