Re: [patch 0/3] j_state_lock, j_list_lock, remove-bitlocks

Andrew Morton
2005-03-16T10:27:13Z

Ingo Molnar <mingo@elte.hu> wrote:
>
> > ooh, I'd rather not. I spent an intense three days removing all the
> > sleeping locks from ext3 (and three months debugging the result).
> > Ended up gaining 1000% on 16-way.
> >
> > Putting them back in will really hurt the SMP performance.
>
> ah. Yeah. Sniff.
>
> if we gain 1000% on a 16-way then there's something really wrong about
> semaphores (or scheduling) though. A semaphore is almost a spinlock, in
> the uncontended case - and even under contention we really (should) just
> spend the cycles that we'd spend spinning. There will be some
> intermediate contention level where semaphores hurt, but 1000% sounds
> truly excessive.

I forget how much of the 1000% came from that, but it was quite a lot.

Removing the BKL was the first step. That took the context switch rate
under high load from ~10,000/sec up to ~300,000/sec. Because the first
thing a CPU hit on entry to the fs was then a semaphore. Performance rather
took a dive.

Of course the locks also became much finer-grained, so the contention
opportunities lessened. But j_list_lock and j_state_lock have fs-wide
scope, so I'd expect the context switch rate to go up quite a lot again.

The hold times are short, and a context switch hurts rather ore than a quick
spin.