ext4 change in v3.3-rc2 broke user space

ext4 change in v3.3-rc2 broke user space

[Bryan Whitehead]

Specifically this change:
http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=d1f5273e9adb40724a85272f248f210dc4ce919a

This change has broken gluster when using ext4 as the backing
filesystem. Details are here:
http://www.gluster.org/2012/08/glusterfs-bit-by-ext4-structure-change/

The change is about a year old now but it is finally making its way
into distribution binaries. As a user it sucks because "ext4" the
"stable" fs basically breaks. Fortunately for myself I've used xfs as
suggested by GlusterFS (older versions stated ext4/ext3 was fine).
However, there are many installs (particularly older installs) that
used ext4 and are finding they must now run outdated old kernels until
they can completely reinstall their gluster cluster with non-ext4
underlying filesystems. Another side effect is users like myself
dealing with the fallout on mailinglists. I've run across this issue
on glusterfs maillinglist (obviously) and also cloudstack. It is so
damn annoying to tell people a kernel change screwed up gluster and
that is why their cluster is busted. To be frank, most will just
assume kernel would never do such a bad thing and it is nonsense that
the kernel is to blame. This sucks.

I'm not hear to bitch (well maybe a little) but I hope changes like
this STOP happening. I think the biggest blame is on RedHat for
backporting the above patch and pushing out kernel binaries before a
workaround in kernel-land was found. On the other hand - a work-around
should have been part of the original patch. I'm resigned to the fact
there is NO hope of this being fixed since everyone has moved on and
it has been accepted for so long. Damnit.

If ext4 is seriously this much in flux - I'd like to politely suggest
it is marked experimental, non-prod, or dangerous.

-Bryan

Re: ext4 change in v3.3-rc2 broke user space

[Theodore Ts'o]

On Mon, Mar 11, 2013 at 02:03:51PM -0700, Bryan Whitehead wrote:
> 
> The change is about a year old now but it is finally making its way
> into distribution binaries. As a user it sucks because "ext4" the
> "stable" fs basically breaks. Fortunately for myself I've used xfs as
> suggested by GlusterFS (older versions stated ext4/ext3 was fine).
> However, there are many installs (particularly older installs) that
> used ext4 and are finding they must now run outdated old kernels until
> they can completely reinstall their gluster cluster with non-ext4
> underlying filesystems.

GlusterFS is doing something fundamentally buggy; it is assuming that
high bits of the value returned "telldir" are zero.  This is
inherently non-portable.  This change was used to avoid hash collision
causing problems for applications using telldir/seekdir() on very
large file systems.

So the question was whether it was important to fix a bug for well
behaved, non-buggy applications, or to not fix this bug to accomodate
a buggy application (namely, Gluster).

There were some questions going back and forth about whether Gluster
could work around this non-potable, buggy use of telldir()/seekdir(),
and that's why nothing had been done for a while.

What we'll probably do to accomodate Gluster is to create a new
ext4-specifical ioctl(), which allows the process to get the old
telldir cookie format (i.e., EXT4_IOC_IM_A_BUGGY_APPLICATION :-).

						- Ted