From: Dmitry Monakhov <dmonakhov@openvz.org>
To: "Buehl, Reiner" <reiner.buehl@hp.com>
Cc: "linux-ide@vger.kernel.org" <linux-ide@vger.kernel.org>,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>
Subject: Re: ext3 filesystem corruption on md RAID1 device
Date: Thu, 20 May 2010 15:27:01 +0400 [thread overview]
Message-ID: <871vd624qi.fsf@openvz.org> (raw)
In-Reply-To: <BA8A2107B0FD8A48AEB0405BDC36CE4F3F48B1DDA7@GVW1115EXC.americas.hpqcorp.net> (Reiner Buehl's message of "Thu, 20 May 2010 11:10:52 +0000")
"Buehl, Reiner" <reiner.buehl@hp.com> writes:
> Hi Dimitry,
>
>> How does it happen? Do you remember any power failures happened
>> previously?
>
> I had no power failures. The first such error appears always approx. 2
> minutes after booting the system. This behavior started after I moved
> the md RAID fro
Hm.. So may be this is not a barrier related.
>m two smaller Seagate disks to the currently used 1TB disks. There for my first thought was a failed disk or memory but they seem to be ok based on the SMART and memtest results.
>
You may possibly force this to happen sooner like follows:
find /your-mnt/ -ls > /dev/null #fore fstat for all inodes
>> What mount options do you used? Probably you use default options
>> which means that fs was mounted w/o barrier support. Even if was
>> mounted
>> with barriers, your raid driver may simply ignore it.
>
> I use default options and software raid (mdadm). Does mdadm handle barriers correct?
>
>> Can you please post following info:
>> 1) mount options and cat /proc/mount
>
> /dev/md1 / ext3 defaults 01
>
> bilbo:~# cat /proc/mounts
> rootfs / rootfs rw 0 0
> none /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
> none /proc proc rw,nosuid,nodev,noexec,relatime 0 0
> udev /dev tmpfs rw,relatime,size=10240k,mode=755 0 0
> /dev/disk/by-uuid/a059fdf2-4ff6-4c30-ba7f-77c85e7f5d1b / ext3 rw,relatime,errors=continue,data=ordered 0 0
> tmpfs /lib/init/rw tmpfs rw,nosuid,relatime,mode=755 0 0
> usbfs /proc/bus/usb usbfs rw,nosuid,nodev,noexec,relatime 0 0
> tmpfs /dev/shm tmpfs rw,nosuid,nodev,relatime 0 0
> devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
> /dev/mapper/vg_data-lv_video /video ext3 rw,relatime,errors=continue,data=ordered 0 0
> /dev/mapper/vg_data-lv_mpg /video/film/mpg ext3 rw,relatime,errors=continue,data=ordered 0 0
> rpc_pipefs /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
> automount(pid3007) /mnt/auto autofs rw,relatime,fd=4,pgrp=3007,timeout=120,minproto=2,maxproto=4,indirect 0 0
> automount(pid2964) /mnt/usb autofs rw,relatime,fd=4,pgrp=2964,timeout=2,minproto=2,maxproto=4,indirect 0 0
> automount(pid3062) /net autofs rw,relatime,fd=4,pgrp=3062,timeout=300,minproto=2,maxproto=4,indirect 0 0
> nfsd /proc/fs/nfsd nfsd rw,relatime 0 0
>
>
>> 2) Write something to your fs and sync;
>> like follows "echo test > /path_to_your_mnt/test ; sync"
>
> Done.
>
>> 3) dmesg log after stage(2)
>
> Nothing got written to dmesg during step 2.
This test was necessary to check that your md has barriers support
but this may happen only if you passed '-obarrier=1' explicitly
Since you use default mount options, then you use ext3 without barriers
so this test is not necessary
>
> I currently have 2 of the EXT3-fs errors in dmesg but writing and syncing did not produce another one.
>
> Best regards,
> Reiner.
WARNING: multiple messages have this Message-ID (diff)
From: Dmitry Monakhov <dmonakhov@openvz.org>
To: "Buehl\, Reiner" <reiner.buehl@hp.com>
Cc: "linux-ide\@vger.kernel.org" <linux-ide@vger.kernel.org>,
"linux-fsdevel\@vger.kernel.org" <linux-fsdevel@vger.kernel.org>
Subject: Re: ext3 filesystem corruption on md RAID1 device
Date: Thu, 20 May 2010 15:27:01 +0400 [thread overview]
Message-ID: <871vd624qi.fsf@openvz.org> (raw)
In-Reply-To: <BA8A2107B0FD8A48AEB0405BDC36CE4F3F48B1DDA7@GVW1115EXC.americas.hpqcorp.net> (Reiner Buehl's message of "Thu, 20 May 2010 11:10:52 +0000")
"Buehl, Reiner" <reiner.buehl@hp.com> writes:
> Hi Dimitry,
>
>> How does it happen? Do you remember any power failures happened
>> previously?
>
> I had no power failures. The first such error appears always approx. 2
> minutes after booting the system. This behavior started after I moved
> the md RAID fro
Hm.. So may be this is not a barrier related.
>m two smaller Seagate disks to the currently used 1TB disks. There for my first thought was a failed disk or memory but they seem to be ok based on the SMART and memtest results.
>
You may possibly force this to happen sooner like follows:
find /your-mnt/ -ls > /dev/null #fore fstat for all inodes
>> What mount options do you used? Probably you use default options
>> which means that fs was mounted w/o barrier support. Even if was
>> mounted
>> with barriers, your raid driver may simply ignore it.
>
> I use default options and software raid (mdadm). Does mdadm handle barriers correct?
>
>> Can you please post following info:
>> 1) mount options and cat /proc/mount
>
> /dev/md1 / ext3 defaults 01
>
> bilbo:~# cat /proc/mounts
> rootfs / rootfs rw 0 0
> none /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
> none /proc proc rw,nosuid,nodev,noexec,relatime 0 0
> udev /dev tmpfs rw,relatime,size=10240k,mode=755 0 0
> /dev/disk/by-uuid/a059fdf2-4ff6-4c30-ba7f-77c85e7f5d1b / ext3 rw,relatime,errors=continue,data=ordered 0 0
> tmpfs /lib/init/rw tmpfs rw,nosuid,relatime,mode=755 0 0
> usbfs /proc/bus/usb usbfs rw,nosuid,nodev,noexec,relatime 0 0
> tmpfs /dev/shm tmpfs rw,nosuid,nodev,relatime 0 0
> devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
> /dev/mapper/vg_data-lv_video /video ext3 rw,relatime,errors=continue,data=ordered 0 0
> /dev/mapper/vg_data-lv_mpg /video/film/mpg ext3 rw,relatime,errors=continue,data=ordered 0 0
> rpc_pipefs /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
> automount(pid3007) /mnt/auto autofs rw,relatime,fd=4,pgrp=3007,timeout=120,minproto=2,maxproto=4,indirect 0 0
> automount(pid2964) /mnt/usb autofs rw,relatime,fd=4,pgrp=2964,timeout=2,minproto=2,maxproto=4,indirect 0 0
> automount(pid3062) /net autofs rw,relatime,fd=4,pgrp=3062,timeout=300,minproto=2,maxproto=4,indirect 0 0
> nfsd /proc/fs/nfsd nfsd rw,relatime 0 0
>
>
>> 2) Write something to your fs and sync;
>> like follows "echo test > /path_to_your_mnt/test ; sync"
>
> Done.
>
>> 3) dmesg log after stage(2)
>
> Nothing got written to dmesg during step 2.
This test was necessary to check that your md has barriers support
but this may happen only if you passed '-obarrier=1' explicitly
Since you use default mount options, then you use ext3 without barriers
so this test is not necessary
>
> I currently have 2 of the EXT3-fs errors in dmesg but writing and syncing did not produce another one.
>
> Best regards,
> Reiner.
next prev parent reply other threads:[~2010-05-20 11:27 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-05-20 10:08 ext3 filesystem corruption on md RAID1 device Buehl, Reiner
2010-05-20 10:56 ` Dmitry Monakhov
2010-05-20 10:56 ` Dmitry Monakhov
2010-05-20 11:10 ` Buehl, Reiner
2010-05-20 11:27 ` Dmitry Monakhov [this message]
2010-05-20 11:27 ` Dmitry Monakhov
2010-05-20 11:35 ` Buehl, Reiner
2010-05-20 11:49 ` Tim Small
2010-05-20 12:04 ` Buehl, Reiner
2010-05-20 14:30 ` tytso
2010-05-21 14:40 ` Buehl, Reiner
2010-05-23 3:21 ` Buehl, Reiner
2010-05-23 5:46 ` Buehl, Reiner
2010-05-27 20:12 ` Jan Kara
2010-05-29 13:48 ` Buehl, Reiner
2010-05-31 20:55 ` Jan Kara
2010-06-01 7:25 ` Buehl, Reiner
[not found] ` <20100601102240.GA4275@quack.suse.cz>
2010-06-18 7:12 ` Buehl, Reiner
2010-06-18 11:09 ` Bug#582275: " Theodore Tso
2010-06-18 11:25 ` Theodore Tso
2010-05-21 4:40 ` Robert Hancock
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=871vd624qi.fsf@openvz.org \
--to=dmonakhov@openvz.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-ide@vger.kernel.org \
--cc=reiner.buehl@hp.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.