Re: [PATCH 2/4] SGI Altix cross partition functionality (1st revision)

Robin Holt
2004-09-05T11:45:13Z

On Sat, Sep 04, 2004 at 11:35:11AM -0500, Russ Anderson wrote:
> Christoph Hellwig wrote:
> >
> > > I like keeping the lock protecting as little as possible. This has been
> > > drilled into peoples heads here at SGI since the early Cray days. We have
> > > always been told to keep locks protecting a single cohesive group of data.
> >
> > Keep it as simple as possible and optimize where optimization is needed.
> > Needless complexity is the root of all evil.
>
> IMHO, Dean's code is simple and not complex. There is a lock per channel.
> What's so complex about that?
>
> Dean's code> + for (ch_number = 0; ch_number < XPC_NCHANNELS; ch_number++) {
> Dean's code> + sema_init(&xpc_registrations[ch_number].sema, 1); /* mutex */
> Dean's code> + }
>
...
>
> What Dean is doing, and what the Cray and SGI people have learned over
> the last couple decades of hard work, is that it is simpler and less complex
> to design in fine grain locks to avoid scaling problems. We know that CPUs
> will get faster, the number of CPUs will increase, as will the number of nodes
> and amount of memory. And as they increase, we know that big locks will get
> hot and need to broken up. So that is why you will find people that
> believe that it is simpler and less complex to design in fine grain locks,
> to avoid having to track down and fix scaling bugs.

I would add that I think a lock that is protecting a channel is clearer to
understand than a lock per partition protecting all the channels in it. When
I look at the partition structure and see a lock for the channels, I would,
at first glance, assume it is protecting addition or removal of the channels.

I remember someone once saying "Speaking clearly is saying what the other
person understands." I think that applies here.

Robin