Re: [PATCH 0/7] Increase maximum Infiniband HCAs per-system

[Hal Rosenstock <hal.rosenstock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

On Fri, Jan 29, 2010 at 6:41 PM, Alex Chiang <achiang-VXdhtT5mjnY@public.gmane.org> wrote:
> * Roland Dreier <rdreier-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org>:
>>
>> Have you considered drivers/infiniband/core/user_mad.c and ucm.c?
>
> Ah, darn. I had not considered those drivers. You're gonna make
> me learn a lot more about IB than I'd originally intended. ;)
>
>> I think user_mad.c is somewhat more important, as that is what
>> allows an adapter to be used for running the SM.  So I think
>> we're still left with some potential issues around lots of
>> adapters in one system.  (I think use of ucm by real apps is
>> minimal to nonexistent, but someday we should deal with that
>> too)
>
> Ok, a quick glance through those drivers shows:
>
>        enum {
>                IB_UMAD_MAX_PORTS  = 64,
>                IB_UMAD_MAX_AGENTS = 32,
>
>                IB_UMAD_MAJOR      = 231,
>                IB_UMAD_MINOR_BASE = 0
>        };
>
> and
>
>        enum {
>                IB_UCM_MAJOR = 231,
>                IB_UCM_BASE_MINOR = 224,
>                IB_UCM_MAX_DEVICES = 32
>        };
>
> They're all sharing the same major number, so they'll all have to
> get the same treatment as the uverbs driver wrt overflow (to
> prevent minor number overlap).
>
> What I'm a little unsure of is, does IB_UMAD_MAX_AGENTS need to
> double too? We don't export the agent id in the filesystem
> anywhere, but we do give it to the user via an ioctl. That's just
> used for book keeping purposes but...
>
> Currently, there are 2x as many ports as there are agents. Do we
> want to keep that ratio, or would it be ok to have 4x as many
> ports as there are agents?

I think it's 2x as many ports as devices (based on the common HCAs
being 2 ports max). This should be maintained as that is still the
case.

-- Hal

>
> Thanks (and sorry for the n00b questions).
>
> /ac
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html