Re: ext3 IO latency measurements (was: Linux 2.6.29)

Ingo Molnar
2009-03-26T23:03:56Z

* Theodore Tso <tytso@mit.edu> wrote:

> On Thu, Mar 26, 2009 at 03:03:12PM +0100, Ingo Molnar wrote:
> >
> > I still see similarly bad latencies in Vim:
> >
>
> When you say "similarly bad", how many seconds were you seeing? I
> understand that from the user's perspective, the 120 seconds you
> saw with ext3 isn't going to be that different from 15 seconds
> (which seems to be the maximum commit time in the jbd2 history
> file you sent me), but I'm curious if what you saw was just as bad
> with ext4, or was it somewhat better (i.e., 120 seconds vs 15 or
> so). Or were you also seeing a net time to save the file using
> vim of around 120 seconds with ext4?

It was in the minute range, iirc. It was totally unusable
interactively. I wrote the vim-test script during that workload and
i'm still getting annoyed thinking back at the experience. Is that
enough to consider it bad? :-)

This isnt me streaming gigs of data in and out of the system
dirtying 90% of all RAM. This is a trivial workload barely
scratching the RAM and CPU capabilities of the system.

Do you have a non-tweaked default Fedora install somewhere? These
kinds of delays in Vim were easily reproducible in the last 5 years
and i saw it reported frequently on various lists.

Have you tried to reproduce it? Have you tried CONFIG_LATENCYTOP? We
implemented that kernel feature specifically to make it easy for
developers to instrument their kernel and keep system latencies
down. This isnt some oddball workload or oddball system. These
latencies are reproducible on just about any Linux development
system i ever tried with ext3.

And the thing is, to 99.9% of the people it doesnt matter how
scalable we are to 16000 CPUs or whether a directory with 1 million
files in it takes 10 or 200 msecs to parse.

But it gives a permanent impression how much delay basic everyday
operations on the system have. So latency optimizations (and i use
the term losely here) have to be the primary development metric in
Linux IMHO. ( If i were doing filesystem development i'd sure
already have my low-latency filesystem patchset ;-)

Ingo