XFS / Quota Bug in 2.6.17.x and 2.6.18x

XFS / Quota Bug in 2.6.17.x and 2.6.18x

[ProfiHost - Stefan Priebe]

Hello!

The crash only occurs if you use quota and IDE without barrier support.

The Problem is, that on a new mount of a root filesystem - the flag 
VFS_RDONLY is set - and so no barrier check is done before checking 
quota. With this patch barrier check is done always. The partition 
should not be mounted at that moment. For mount -o remount, rw or 
something like this it uses another function where VFS_RDONLY is checked.

Error Message:
ns2 Wed Jul 26 14:22:58 2006 "I/O error in filesystem ("hda6") meta-data 
dev
hda6 block 0x23db5ab       ("xlog_iodone") error 5 buf count 1024"
ns2 Wed Jul 26 14:22:58 2006 "xfs_force_shutdown(hda6,0x2) called from line
959 of file fs/xfs/xfs_log.c.  Return address = 0xc0211535"
ns2 Wed Jul 26 14:22:58 2006 "Filesystem "hda6": Log I/O Error Detected.
Shutting down filesystem: hda6"
ns2 Wed Jul 26 14:22:58 2006 "Please umount the filesystem, and rectify the
problem(s)"
ns2 Wed Jul 26 14:22:58 2006 "xfs_force_shutdown(hda6,0x1) called from line
338 of file fs/xfs/xfs_rw.c.  Return address = 0xc0211535"
ns2 Wed Jul 26 14:22:58 2006 "xfs_force_shutdown(hda6,0x1) called from line
338 of file fs/xfs/xfs_rw.c.  Return address = 0xc0211535"

Patch:
*** fs/xfs/xfs_vfsops.c.orig	Thu Jul 27 13:10:23 2006
--- fs/xfs/xfs_vfsops.c	Thu Jul 27 13:11:17 2006
*************** xfs_mount(
*** 524,528 ****
   		goto error2;

! 	if ((mp->m_flags & XFS_MOUNT_BARRIER) && !(vfsp->vfs_flag &
VFS_RDONLY))
   		xfs_mountfs_check_barriers(mp);

--- 524,528 ----
   		goto error2;

! 	if (mp->m_flags & XFS_MOUNT_BARRIER)
   		xfs_mountfs_check_barriers(mp);


Best regards,
Ihr ProfiHost Team
------------------------------------------
ProfiHost e.K.
Lindener Str 15
38300 Wolfenbüttel

Tel.: 05331 996890
Fax: 05331 996899
URL: http://www.profihost.com
E-Mail: support@profihost.com

Re: XFS / Quota Bug in 2.6.17.x and 2.6.18x

[Alexey Dobriyan]

On Thu, Jul 27, 2006 at 01:39:29PM +0200, ProfiHost - Stefan Priebe wrote:

Please regenerate patch with "diff -up" and sent it to
Nathan Scott <nathans@sgi.com>, CC'ing linux-fsdevel@vger.kernel.org,
xfs@oss.sgi.com

Also, read section 11 of Documentation/SubmittingPatches and if one of
clauses is applicable to you add appropriate "Signed-off-by" line.

> The crash only occurs if you use quota and IDE without barrier support.
> 
> The Problem is, that on a new mount of a root filesystem - the flag 
> VFS_RDONLY is set - and so no barrier check is done before checking 
> quota. With this patch barrier check is done always. The partition 
> should not be mounted at that moment. For mount -o remount, rw or 
> something like this it uses another function where VFS_RDONLY is checked.
> 
> Error Message:
> ns2 Wed Jul 26 14:22:58 2006 "I/O error in filesystem ("hda6") meta-data 
> dev
> hda6 block 0x23db5ab       ("xlog_iodone") error 5 buf count 1024"
> ns2 Wed Jul 26 14:22:58 2006 "xfs_force_shutdown(hda6,0x2) called from line
> 959 of file fs/xfs/xfs_log.c.  Return address = 0xc0211535"
> ns2 Wed Jul 26 14:22:58 2006 "Filesystem "hda6": Log I/O Error Detected.
> Shutting down filesystem: hda6"
> ns2 Wed Jul 26 14:22:58 2006 "Please umount the filesystem, and rectify the
> problem(s)"
> ns2 Wed Jul 26 14:22:58 2006 "xfs_force_shutdown(hda6,0x1) called from line
> 338 of file fs/xfs/xfs_rw.c.  Return address = 0xc0211535"
> ns2 Wed Jul 26 14:22:58 2006 "xfs_force_shutdown(hda6,0x1) called from line
> 338 of file fs/xfs/xfs_rw.c.  Return address = 0xc0211535"
> 
> Patch:
> *** fs/xfs/xfs_vfsops.c.orig	Thu Jul 27 13:10:23 2006
> --- fs/xfs/xfs_vfsops.c	Thu Jul 27 13:11:17 2006
> *************** xfs_mount(
> *** 524,528 ****
>   		goto error2;
> 
> ! 	if ((mp->m_flags & XFS_MOUNT_BARRIER) && !(vfsp->vfs_flag &
> VFS_RDONLY))
>   		xfs_mountfs_check_barriers(mp);
> 
> --- 524,528 ----
>   		goto error2;
> 
> ! 	if (mp->m_flags & XFS_MOUNT_BARRIER)
>   		xfs_mountfs_check_barriers(mp);
> 
> 

Re: XFS / Quota Bug in 2.6.17.x and 2.6.18x

[Nathan Scott]

Hi Stefan,

On Thu, Jul 27, 2006 at 01:39:29PM +0200, ProfiHost - Stefan Priebe wrote:
> Hello!
> 
> The crash only occurs if you use quota and IDE without barrier support.
> 
> The Problem is, that on a new mount of a root filesystem - the flag 
> VFS_RDONLY is set - and so no barrier check is done before checking 
> quota. With this patch barrier check is done always. The partition 
> should not be mounted at that moment. For mount -o remount, rw or 
> something like this it uses another function where VFS_RDONLY is checked.

Ah, I see.  The patch isn't quite right, I think we will now need to
also add a test to xfs_mountfs_check_barriers() to ensure the device
beneath us is not bdev_read_only().

I'll add that and get the fix merged, thanks.

cheers.

-- 
Nathan

Re: XFS / Quota Bug in 2.6.17.x and 2.6.18x

[Jan Engelhardt]

> The crash only occurs if you use quota and IDE without barrier support.

I don't quite get this. I do use quota, and have barriers turned 
off (either explicitly or because the drive does not support it),
but yet no error message like you posted. Do I just have luck?

> The Problem is, that on a new mount of a root filesystem - the flag VFS_RDONLY
> is set - and so no barrier check is done before checking quota. With this patch
> barrier check is done always. The partition should not be mounted at that
> moment. For mount -o remount, rw or something like this it uses another
> function where VFS_RDONLY is checked.

Jan Engelhardt
-- 

Re: XFS / Quota Bug in 2.6.17.x and 2.6.18x

[Nathan Scott]

On Fri, Jul 28, 2006 at 07:11:31PM +0200, Jan Engelhardt wrote:
> 
> > The crash only occurs if you use quota and IDE without barrier support.
> 
> I don't quite get this. I do use quota, and have barriers turned 
> off (either explicitly or because the drive does not support it),
> but yet no error message like you posted. Do I just have luck?

Heh, no - its more likely you just haven't needed to do a quotacheck
on a filesystem thats initially mounted readonly (like root often is).
I'm guessing you had quota enabled from earlier barrier-unaware kernels
and quotacheck only needs to be run during that initial mount.

cheers.

-- 
Nathan

Re: XFS / Quota Bug in 2.6.17.x and 2.6.18x

[Jan Engelhardt]

>> > The crash only occurs if you use quota and IDE without barrier support.
>> 
>> I don't quite get this. I do use quota, and have barriers turned 
>> off (either explicitly or because the drive does not support it),
>> but yet no error message like you posted. Do I just have luck?
>
>Heh, no - its more likely you just haven't needed to do a quotacheck
>on a filesystem thats initially mounted readonly (like root often is).

Well I "sometimes" do that, i.e. intentionally turning off quota on the 
running system, to force a recheck on boot. The mount options essentially
are /bin/mount /dev/hda2 / -o ro,usrquota,grpquota and then /bin/mount / -o 
remount,rw
No breakage so far, which is why I wondered. Is it limited to a specific 
kernel version?

>I'm guessing you had quota enabled from earlier barrier-unaware kernels
>and quotacheck only needs to be run during that initial mount.


Jan Engelhardt
-- 

Re: XFS / Quota Bug in 2.6.17.x and 2.6.18x

[ProfiHost - Stefan Priebe]

Hi!

This only happens, if your partition is the root partition of the whole 
system and it is mounted read only on system start up. The pre barrier 
check for normal mount works without any problems.

Stefan

Jan Engelhardt schrieb:
>>>>The crash only occurs if you use quota and IDE without barrier support.
>>>
>>>I don't quite get this. I do use quota, and have barriers turned 
>>>off (either explicitly or because the drive does not support it),
>>>but yet no error message like you posted. Do I just have luck?
>>
>>Heh, no - its more likely you just haven't needed to do a quotacheck
>>on a filesystem thats initially mounted readonly (like root often is).
> 
> 
> Well I "sometimes" do that, i.e. intentionally turning off quota on the 
> running system, to force a recheck on boot. The mount options essentially
> are /bin/mount /dev/hda2 / -o ro,usrquota,grpquota and then /bin/mount / -o 
> remount,rw
> No breakage so far, which is why I wondered. Is it limited to a specific 
> kernel version?
> 
> 
>>I'm guessing you had quota enabled from earlier barrier-unaware kernels
>>and quotacheck only needs to be run during that initial mount.
> 
> 
> 
> Jan Engelhardt

Re: XFS / Quota Bug in 2.6.17.x and 2.6.18x

[Jan Engelhardt]

> This only happens, if your partition is the root partition of the whole system
>
It is..

> and it is mounted read only on system start up.
>
It is...

> The pre barrier check for normal mount works without any problems.
>
Dang, I forgot that hda is a little older and does not support barriers. 
/me slaps himself.



Jan Engelhardt
-- 

Re: XFS / Quota Bug in 2.6.17.x and 2.6.18x

[ProfiHost - Stefan Priebe]

Hi!

 >>The pre barrier check for normal mount works without any problems.
 > Dang, I forgot that hda is a little older and does not support
 > barriers. /me slaps himself.
That matches for me too. My drives are also too old to support barriers. 
It occours on about 35 servers - so not all filesystems could be damaged.

So please recheck the following:

1.) You need a Kernel, where barriers are on by default:
so something like: 2.6.16.2x or 2.6.17.x
I'm not shure of the minimal version for the 2.6.16.x Kernel tree.

2.) that hda2 is / on your system

3.) you boot your system with kernel option rootflags=quota

4.) do not use mount, to deactivate quota - use quotaoff / command

5.) then reboot your system...


Stefan

Jan Engelhardt schrieb:
>>This only happens, if your partition is the root partition of the whole system
>>
> 
> It is..
> 
> 
>>and it is mounted read only on system start up.
>>
> 
> It is...
> 
> 
>>The pre barrier check for normal mount works without any problems.
> Dang, I forgot that hda is a little older and does not support barriers. 
> /me slaps himself.
> 
> 
> 
> Jan Engelhardt

Re: XFS / Quota Bug in 2.6.17.x and 2.6.18x

[Jan Engelhardt]

> 1.) You need a Kernel, where barriers are on by default:
> so something like: 2.6.16.2x or 2.6.17.x
> I'm not shure of the minimal version for the 2.6.16.x Kernel tree.

2.6.17.6

> 3.) you boot your system with kernel option rootflags=quota

Ah I think there it is. I do boot with rootflags=, but since the initrd 
does the interpreting of rootflags= and mounting of /dev/hda2(/), that 
should pose a different situation?


Jan Engelhardt
-- 

Re: XFS / Quota Bug in 2.6.17.x and 2.6.18x

[ProfiHost - Stefan Priebe]

Hi!

Correct - it only happens, if you use rootflags=quota without initrd.

Stefan


Jan Engelhardt schrieb:
>>1.) You need a Kernel, where barriers are on by default:
>>so something like: 2.6.16.2x or 2.6.17.x
>>I'm not shure of the minimal version for the 2.6.16.x Kernel tree.
> 
> 
> 2.6.17.6
> 
> 
>>3.) you boot your system with kernel option rootflags=quota
> 
> 
> Ah I think there it is. I do boot with rootflags=, but since the initrd 
> does the interpreting of rootflags= and mounting of /dev/hda2(/), that 
> should pose a different situation?
> 
> 
> Jan Engelhardt