* Re: [PATCH] xfs_mkfs: wipe old signatures from the device [not found] ` <alpine.LFD.2.00.1302131127070.2315@(none)> @ 2013-02-13 12:16 ` Karel Zak 2013-02-13 22:17 ` Dave Chinner 0 siblings, 1 reply; 8+ messages in thread From: Karel Zak @ 2013-02-13 12:16 UTC (permalink / raw) To: Lukáš Czerner Cc: Dave Chinner, xfs, sandeen, Zach Brown, linux-btrfs On Wed, Feb 13, 2013 at 11:41:08AM +0100, Lukáš Czerner wrote: > However > > mkfs.btrfs /dev/sda > mkfs.ext4 -F /dev/sda > > works well, however I am not sure why that is. Is that some kind of > mount(8) magic ? This is bug in libmount. Fixed in upstream tree. The libmount in some cases ignores the ambivalent probing result from libblkid and tries stuff from /etc/filesystems (where is for example ext4). > So I think that liblkid _and_ btrfs tools has to be fixed not to treat > backup superblocks as primary! The problem with additional btrfs superblocks has been already reported to btrfs guys: http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg20957.html and I don't see a reply "yep, this is btrfs-progs bug" :-) Karel -- Karel Zak <kzak@redhat.com> http://karelzak.blogspot.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] xfs_mkfs: wipe old signatures from the device 2013-02-13 12:16 ` [PATCH] xfs_mkfs: wipe old signatures from the device Karel Zak @ 2013-02-13 22:17 ` Dave Chinner 2013-02-14 7:29 ` Chris Murphy 0 siblings, 1 reply; 8+ messages in thread From: Dave Chinner @ 2013-02-13 22:17 UTC (permalink / raw) To: Karel Zak; +Cc: Lukáš Czerner, xfs, sandeen, Zach Brown, linux-btrfs On Wed, Feb 13, 2013 at 01:16:55PM +0100, Karel Zak wrote: > On Wed, Feb 13, 2013 at 11:41:08AM +0100, Lukáš Czerner wrote: > > However > > > > mkfs.btrfs /dev/sda > > mkfs.ext4 -F /dev/sda > > > > works well, however I am not sure why that is. Is that some kind of > > mount(8) magic ? > > This is bug in libmount. Fixed in upstream tree. The libmount in some > cases ignores the ambivalent probing result from libblkid and tries > stuff from /etc/filesystems (where is for example ext4). > > > So I think that liblkid _and_ btrfs tools has to be fixed not to treat > > backup superblocks as primary! > > The problem with additional btrfs superblocks has been already reported > to btrfs guys: > > http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg20957.html > > and I don't see a reply "yep, this is btrfs-progs bug" :-) Oh, it's most definitely a btrfs-progs bug. Using stale metadata data on the block device in preference to current, primary metadata identifying the device to belong to a different filesystem is, well, rather unfriendly. In the above case, mkfs.ext4 overwrites the btrfs magic block field so that it is not zero (as it is after wipefs runs), and hence the btrfs tools see it as a broken superblock and continue to the next location for recovery rather than see the device as containing a valid but foreign filesystem. None of the filesystem tools from ext4, btrfs and XFS handle this sort of thing consistently, even though it is their responsibility to do so. XFS is probably the best of them, in that *some* of the tools will refuse to run unless you provide a force flag when the device appears to contain primary identifiers for a different filesystem. e.g. mkfs.xfs won't overwrite existing filesystems without a force flag, but mkfs.ext4 and mkfs.btrfs will: $ sudo mkfs.xfs /dev/vdb mkfs.xfs: /dev/vdb appears to contain an existing filesystem (btrfs). mkfs.xfs: Use the -f option to force overwrite. $ sudo mkfs.xfs -f /dev/vdb meta-data=/dev/vdb isize=256 agcount=4, agsize=720896 blks = sectsz=512 attr=2, projid32bit=0 = crc=0 data = bsize=4096 blocks=2883584, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 $ sudo mkfs.xfs /dev/vdb mkfs.xfs: /dev/vdb appears to contain an existing filesystem (xfs). mkfs.xfs: Use the -f option to force overwrite. $ sudo mkfs.btrfs /dev/vdb WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL WARNING! - see http://btrfs.wiki.kernel.org before using fs created label (null) on /dev/vdb nodesize 4096 leafsize 4096 sectorsize 4096 size 11.00GB Btrfs Btrfs v0.19 $ sudo mkfs.ext4 /dev/vdb mke2fs 1.42.5 (29-Jul-2012) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 720896 inodes, 2883584 blocks 144179 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=2952790016 88 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208 Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done $ sudo mkfs.xfs /dev/vdb mkfs.xfs: /dev/vdb appears to contain an existing filesystem (ext4). mkfs.xfs: Use the -f option to force overwrite. $ IOWs, it is the responsibility of the filesystem tools to correctly identify the filesystem being operated on, and not to modify/trash/recover anything unless specifically directed by the user. When a filesystem tool gets it wrong, then that specific tool needs to be fixed. i.e. it is the responsibility of filesystem tools to behave sanely, not for the rest of the world to have to work around the dangerous behaviour of a specific filesystems' toolset. And that makes it a btrfs-progs bug that needs to be fixed. Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] xfs_mkfs: wipe old signatures from the device 2013-02-13 22:17 ` Dave Chinner @ 2013-02-14 7:29 ` Chris Murphy [not found] ` <alpine.LFD.2.00.1302140929570.2315@(none)> 0 siblings, 1 reply; 8+ messages in thread From: Chris Murphy @ 2013-02-14 7:29 UTC (permalink / raw) To: Dave Chinner Cc: Karel Zak, Lukáš Czerner, xfs, sandeen, Zach Brown, linux-btrfs On Feb 13, 2013, at 3:17 PM, Dave Chinner <david@fromorbit.com> wrote: > it is the responsibility of filesystem > tools to behave sanely, not for the rest of the world to have to > work around the dangerous behaviour of a specific filesystems' > toolset. I appreciate this, and in particular that mkfs.xfs doesn't nerf a file system without the use of -f; even an existing XFS file system. Considering most data loss is user induced, I'd appreciate it if other file systems's tools weren't so easily made belligerent by (hopefully temporarily) confused apes wearing pants. Chris Murphy ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <alpine.LFD.2.00.1302140929570.2315@(none)>]
* Re: [PATCH] xfs_mkfs: wipe old signatures from the device [not found] ` <alpine.LFD.2.00.1302140929570.2315@(none)> @ 2013-02-14 11:04 ` Dave Chinner 2013-02-14 14:48 ` Martin Steigerwald 2013-02-14 14:54 ` Hugo Mills 2013-02-14 17:25 ` Eric Sandeen 1 sibling, 2 replies; 8+ messages in thread From: Dave Chinner @ 2013-02-14 11:04 UTC (permalink / raw) To: Lukáš Czerner Cc: Chris Murphy, Karel Zak, xfs, sandeen, Zach Brown, linux-btrfs On Thu, Feb 14, 2013 at 09:36:38AM +0100, Lukáš Czerner wrote: > On Thu, 14 Feb 2013, Chris Murphy wrote: > > > Date: Thu, 14 Feb 2013 00:29:59 -0700 > > From: Chris Murphy <lists@colorremedies.com> > > To: Dave Chinner <david@fromorbit.com> > > Cc: Karel Zak <kzak@redhat.com>, Lukáš Czerner <lczerner@redhat.com>, > > xfs@oss.sgi.com, sandeen@redhat.com, Zach Brown <zabrown@redhat.com>, > > linux-btrfs@vger.kernel.org > > Subject: Re: [PATCH] xfs_mkfs: wipe old signatures from the device > > > > > > On Feb 13, 2013, at 3:17 PM, Dave Chinner <david@fromorbit.com> wrote: > > > it is the responsibility of filesystem > > > tools to behave sanely, not for the rest of the world to have to > > > work around the dangerous behaviour of a specific filesystems' > > > toolset. > > > > I appreciate this, and in particular that mkfs.xfs doesn't nerf a file system without the use of -f; even an existing XFS file system. Considering most data loss is user induced, I'd appreciate it if other file systems's tools weren't so easily made belligerent by (hopefully temporarily) confused apes wearing pants. > > > > Chris Murphy > > I would not be so optimistic about it. The reason being that there > are almost _always_ old file system signatures on the device. That assumption is way off the mark. What you do as a filesystem developer (remake filesystems on the same block device hundreds of times a day) is not at all typical, so you cannot extrapolate from your usage habits to what typically happens in production environments. Admins don't tend to use "force" options by default (especially for destructive comands like mkfs) as 1) they are rarely necessary in the real world and 2) the consequences of errors are severe. The most common filesystem creation pattern in production systems (be it desktop, workstation or server) is that storage, devices and filesystems are set up once and then used for the entire lifetime ofthe system without ever having mkfs run on them again. i.e on pristine, empty hardware. Hence users rarely, if ever, need to use the force option for mkfs.xfs. > So I > think that it got to the point where users will usually use mkfs.xfs > -f all the time. And even if they did not and they would use a wrong > device they would probably get the same warning even for the device > they wanted to use in the first place. I get a couple of queries a year from people saying they accidentally ran mkfs.ext4 on the wrong device and want to know if they can recover their XFS filesystem. The next question is usually "why didn't mkfs.ext4 warn me there was an existing filesystem on the device like mkfs.xfs does?". That is why the "don't overwrite an existing filesystem by default" behaviour is important. Users like to be protected from mistakes they weren't aware they made, and far too few of our filesystem utilities provide that safety net. A couple of users a year losing data like this is a couple of users too many. Especially when it would only take a couple of hours of your time to implement.... > So even thoug it might help in some cases I do not think that we > should go and change all file systems to do that as well, it would > not be very useful anyway. Tell that to the next user that trashes their data because a filesystem tool simply assumed in correctly that it owned the block device. Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] xfs_mkfs: wipe old signatures from the device 2013-02-14 11:04 ` Dave Chinner @ 2013-02-14 14:48 ` Martin Steigerwald 2013-02-14 18:35 ` Eric Sandeen 2013-02-14 14:54 ` Hugo Mills 1 sibling, 1 reply; 8+ messages in thread From: Martin Steigerwald @ 2013-02-14 14:48 UTC (permalink / raw) To: linux-btrfs Cc: Dave Chinner, Lukáš Czerner, Chris Murphy, Karel Zak, xfs, sandeen, Zach Brown Am Donnerstag, 14. Februar 2013 schrieb Dave Chinner: > > So I > > think that it got to the point where users will usually use mkfs.xfs > > -f all the time. And even if they did not and they would use a wrong > > device they would probably get the same warning even for the device > > they wanted to use in the first place. > > I get a couple of queries a year from people saying they > accidentally ran mkfs.ext4 on the wrong device and want to know if > they can recover their XFS filesystem. The next question is usually > "why didn't mkfs.ext4 warn me there was an existing filesystem on > the device like mkfs.xfs does?". > > That is why the "don't overwrite an existing filesystem by default" > behaviour is important. Users like to be protected from mistakes > they weren't aware they made, and far too few of our filesystem > utilities provide that safety net. > > A couple of users a year losing data like this is a couple of users > too many. Especially when it would only take a couple of hours of > your time to implement.... > > > So even thoug it might help in some cases I do not think that we > > should go and change all file systems to do that as well, it would > > not be very useful anyway. > > Tell that to the next user that trashes their data because a > filesystem tool simply assumed in correctly that it owned the block > device. Full ACK. I always loved that mkfs.xfs asks in that case. IMO its just sane to do so. -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] xfs_mkfs: wipe old signatures from the device 2013-02-14 14:48 ` Martin Steigerwald @ 2013-02-14 18:35 ` Eric Sandeen 0 siblings, 0 replies; 8+ messages in thread From: Eric Sandeen @ 2013-02-14 18:35 UTC (permalink / raw) To: Martin Steigerwald Cc: linux-btrfs, Dave Chinner, Lukáš Czerner, Chris Murphy, Karel Zak, xfs, Zach Brown On 2/14/13 8:48 AM, Martin Steigerwald wrote: > Am Donnerstag, 14. Februar 2013 schrieb Dave Chinner: >>> So I >>> think that it got to the point where users will usually use mkfs.xfs >>> -f all the time. And even if they did not and they would use a wrong >>> device they would probably get the same warning even for the device >>> they wanted to use in the first place. >> >> I get a couple of queries a year from people saying they >> accidentally ran mkfs.ext4 on the wrong device and want to know if >> they can recover their XFS filesystem. The next question is usually >> "why didn't mkfs.ext4 warn me there was an existing filesystem on >> the device like mkfs.xfs does?". >> >> That is why the "don't overwrite an existing filesystem by default" >> behaviour is important. Users like to be protected from mistakes >> they weren't aware they made, and far too few of our filesystem >> utilities provide that safety net. >> >> A couple of users a year losing data like this is a couple of users >> too many. Especially when it would only take a couple of hours of >> your time to implement.... >> >>> So even thoug it might help in some cases I do not think that we >>> should go and change all file systems to do that as well, it would >>> not be very useful anyway. >> >> Tell that to the next user that trashes their data because a >> filesystem tool simply assumed in correctly that it owned the block >> device. > > Full ACK. > > I always loved that mkfs.xfs asks in that case. > > IMO its just sane to do so. I just sent a patch to do so for btrfs-progs, FWIW. :) -Eric ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] xfs_mkfs: wipe old signatures from the device 2013-02-14 11:04 ` Dave Chinner 2013-02-14 14:48 ` Martin Steigerwald @ 2013-02-14 14:54 ` Hugo Mills 1 sibling, 0 replies; 8+ messages in thread From: Hugo Mills @ 2013-02-14 14:54 UTC (permalink / raw) To: Dave Chinner Cc: Lukáš Czerner, Chris Murphy, Karel Zak, xfs, sandeen, Zach Brown, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 2454 bytes --] On Thu, Feb 14, 2013 at 10:04:23PM +1100, Dave Chinner wrote: > On Thu, Feb 14, 2013 at 09:36:38AM +0100, Lukáš Czerner wrote: > > On Thu, 14 Feb 2013, Chris Murphy wrote: > > > > > Date: Thu, 14 Feb 2013 00:29:59 -0700 > > > From: Chris Murphy <lists@colorremedies.com> > > > To: Dave Chinner <david@fromorbit.com> > > > Cc: Karel Zak <kzak@redhat.com>, Lukáš Czerner <lczerner@redhat.com>, > > > xfs@oss.sgi.com, sandeen@redhat.com, Zach Brown <zabrown@redhat.com>, > > > linux-btrfs@vger.kernel.org > > > Subject: Re: [PATCH] xfs_mkfs: wipe old signatures from the device > > So I > > think that it got to the point where users will usually use mkfs.xfs > > -f all the time. And even if they did not and they would use a wrong > > device they would probably get the same warning even for the device > > they wanted to use in the first place. > > I get a couple of queries a year from people saying they > accidentally ran mkfs.ext4 on the wrong device and want to know if > they can recover their XFS filesystem. The next question is usually > "why didn't mkfs.ext4 warn me there was an existing filesystem on > the device like mkfs.xfs does?". > > That is why the "don't overwrite an existing filesystem by default" > behaviour is important. Users like to be protected from mistakes > they weren't aware they made, and far too few of our filesystem > utilities provide that safety net. > > A couple of users a year losing data like this is a couple of users > too many. Especially when it would only take a couple of hours of > your time to implement.... > > > So even thoug it might help in some cases I do not think that we > > should go and change all file systems to do that as well, it would > > not be very useful anyway. > > Tell that to the next user that trashes their data because a > filesystem tool simply assumed in correctly that it owned the block > device. We had someone on IRC a day or two ago who had done exactly this. They're not the only one -- I can recall seeing at least one other person who managed to mkfs.btrfs on an existing filesystem. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- I am but mad north-north-west: when the wind is southerly, I --- know a hawk from a handsaw. [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 828 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] xfs_mkfs: wipe old signatures from the device [not found] ` <alpine.LFD.2.00.1302140929570.2315@(none)> 2013-02-14 11:04 ` Dave Chinner @ 2013-02-14 17:25 ` Eric Sandeen 1 sibling, 0 replies; 8+ messages in thread From: Eric Sandeen @ 2013-02-14 17:25 UTC (permalink / raw) To: Lukáš Czerner Cc: Chris Murphy, Dave Chinner, Karel Zak, xfs, Zach Brown, linux-btrfs On 2/14/13 2:36 AM, Lukáš Czerner wrote: > On Thu, 14 Feb 2013, Chris Murphy wrote: > >> Date: Thu, 14 Feb 2013 00:29:59 -0700 >> From: Chris Murphy <lists@colorremedies.com> >> To: Dave Chinner <david@fromorbit.com> >> Cc: Karel Zak <kzak@redhat.com>, Lukáš Czerner <lczerner@redhat.com>, >> xfs@oss.sgi.com, sandeen@redhat.com, Zach Brown <zabrown@redhat.com>, >> linux-btrfs@vger.kernel.org >> Subject: Re: [PATCH] xfs_mkfs: wipe old signatures from the device >> >> >> On Feb 13, 2013, at 3:17 PM, Dave Chinner <david@fromorbit.com> wrote: >>> it is the responsibility of filesystem >>> tools to behave sanely, not for the rest of the world to have to >>> work around the dangerous behaviour of a specific filesystems' >>> toolset. >> >> I appreciate this, and in particular that mkfs.xfs doesn't nerf a file system without the use of -f; even an existing XFS file system. Considering most data loss is user induced, I'd appreciate it if other file systems's tools weren't so easily made belligerent by (hopefully temporarily) confused apes wearing pants. >> >> Chris Murphy > > I would not be so optimistic about it. The reason being that there > are almost _always_ old file system signatures on the device. So I > think that it got to the point where users will usually use mkfs.xfs > -f all the time. I know I do ;) But as Dave points out, fs developers are odd ducks. > And even if they did not and they would use a wrong > device they would probably get the same warning even for the device > they wanted to use in the first place. I was thinking, as annoying as it might be, requiring the user to use "-f $OLD_FS_TYPE" might require a bit more positive action and cognition on the admin's part. OTOH, that could get annoying, and break old scripts. :) -Eric > So even thoug it might help in some cases I do not think that we > should go and change all file systems to do that as well, it would > not be very useful anyway. > > -Lukas > ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2013-02-14 18:36 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1360667215-14701-1-git-send-email-lczerner@redhat.com>
[not found] ` <20130212202753.GC26694@dastard>
[not found] ` <20130213080154.GC18597@x2.net.home>
[not found] ` <alpine.LFD.2.00.1302131127070.2315@(none)>
2013-02-13 12:16 ` [PATCH] xfs_mkfs: wipe old signatures from the device Karel Zak
2013-02-13 22:17 ` Dave Chinner
2013-02-14 7:29 ` Chris Murphy
[not found] ` <alpine.LFD.2.00.1302140929570.2315@(none)>
2013-02-14 11:04 ` Dave Chinner
2013-02-14 14:48 ` Martin Steigerwald
2013-02-14 18:35 ` Eric Sandeen
2013-02-14 14:54 ` Hugo Mills
2013-02-14 17:25 ` Eric Sandeen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox