Re: [PATCH 0/7] Increase maximum Infiniband HCAs per-system

Alex Chiang
2010-01-29T23:41:48Z

* Roland Dreier <rdreier@cisco.com>:
>
> Have you considered drivers/infiniband/core/user_mad.c and ucm.c?

Ah, darn. I had not considered those drivers. You're gonna make
me learn a lot more about IB than I'd originally intended. ;)

> I think user_mad.c is somewhat more important, as that is what
> allows an adapter to be used for running the SM. So I think
> we're still left with some potential issues around lots of
> adapters in one system. (I think use of ucm by real apps is
> minimal to nonexistent, but someday we should deal with that
> too)

Ok, a quick glance through those drivers shows:

enum {
IB_UMAD_MAX_PORTS = 64,
IB_UMAD_MAX_AGENTS = 32,

IB_UMAD_MAJOR = 231,
IB_UMAD_MINOR_BASE = 0
};

and

enum {
IB_UCM_MAJOR = 231,
IB_UCM_BASE_MINOR = 224,
IB_UCM_MAX_DEVICES = 32
};

They're all sharing the same major number, so they'll all have to
get the same treatment as the uverbs driver wrt overflow (to
prevent minor number overlap).

What I'm a little unsure of is, does IB_UMAD_MAX_AGENTS need to
double too? We don't export the agent id in the filesystem
anywhere, but we do give it to the user via an ioctl. That's just
used for book keeping purposes but...

Currently, there are 2x as many ports as there are agents. Do we
want to keep that ratio, or would it be ok to have 4x as many
ports as there are agents?

Thanks (and sorry for the n00b questions).

/ac