Re: [ext3][kernels >= 2.6.20.7 at least] KDE going comatose when FS is under heavy write load (massive starvation)

[Alex Tomas <alex@clusterfs.com>]

Andrew Morton wrote:
> Sort-of.  But the per-superpblock, per-inode writeback code is pretty
> careful to avoid livelocks.  The per-inode writeback is a strict single
> linear sweep across the file.  It'll basically write out anything which was
> dirty when it was called.  The per-superblock inode walk isn't as accurate
> as that, becuase of the difficulties of juggling list_heads.  But we're
> slowly working on that, and I suspect it'll be ggod enough for ext3
> purposes already.

I'd say that these are two different mechanism solving different problems:
1) VFS/MM does periodic updates and uses regular writeback
2) data=ordered is to avoid metadata pointing to not-written-yet data

we can't use regular writeback in commit thread as long as it can fall into
allocation. so, we'd have to add one more WB mode (btw, i have a patch which
skips non-allocated blocks in writeback if special WB mode is requested).

OTOH, the faster we go through data sync part of commit, the better. given
that lots of inodes can be dirty with no data to sync, it's going to take
long in some cases. it's especially bad because commit doesn't scale to many
CPUs.

also, why would we need to flush *everything* every 5s? just because ext3 does
this? sounds strange. if somebody really need this we could add this possibility
to regular writeback path (making it tunable). but I'd rather prefer to have
a separate (fast, lightweight, scalable) mechanism to support data=ordered.

thanks, Alex