btrfs problems and fedora 14

btrfs problems and fedora 14

[david grant]

I thought I would try btrfs on a new installation of f14. yes, I know
its experimental but stable so it seemed to be a good time to try it.
I am not sure if I have missed something out of all my searching but am
I correct in thinking that currently: 
     I. it is not possible to boot from a snapshot of the operating
        system and, in particular, the yum snapshots cannot be used for
        that purpose 
    II. it is so easy to create raid arrays of btrfs partitions but they
        cannot be read by f13 or f14 
   III. it is not possible to copy btrfs partitions with snapshots
        except possibly by the use of dd.
This is not meant to be a put down of btrfs but a plea to have some
clarification and in particular the ability to boot snapshots.

Hope I can get a response 

Re: btrfs problems and fedora 14

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1579 bytes --]

   Hi,

On Tue, Nov 23, 2010 at 10:19:43AM +1100, david grant wrote:
> I thought I would try btrfs on a new installation of f14. yes, I know
> its experimental but stable so it seemed to be a good time to try it.
> I am not sure if I have missed something out of all my searching but am
> I correct in thinking that currently: 
>      I. it is not possible to boot from a snapshot of the operating
>         system and, in particular, the yum snapshots cannot be used for
>         that purpose 

   You can use btrfs subvolume set-default to set the default
subvolume that is mounted if no "subvol=" or "subvolid=" parameter is
given to mount. (And you can then subsequently access the original
root of the filesystem using mount -o subvolid=0).

>     II. it is so easy to create raid arrays of btrfs partitions but they
>         cannot be read by f13 or f14 

   There's no particular reason that this should be the case. How do
you come to this conclusion? What did you try, what did you expect to
happen, and what actually happened?

>    III. it is not possible to copy btrfs partitions with snapshots
>         except possibly by the use of dd.

   Again, I can't see a reason that this shouldn't work. What are you
trying to do, exactly, and how is it failing?

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
           --- I believe that it's closely correlated with ---           
                       the aeroswine coefficient.                        

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: btrfs problems and fedora 14

[Wenyi Liu]

2010/11/23, david grant <dg@david-grant.com>:
> I thought I would try btrfs on a new installation of f14. yes, I know
> its experimental but stable so it seemed to be a good time to try it.
> I am not sure if I have missed something out of all my searching but am
> I correct in thinking that currently:
>      I. it is not possible to boot from a snapshot of the operating
>         system and, in particular, the yum snapshots cannot be used for
>         that purpose

Is the Fedora grub support btrfs now?
In this page http://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs
I got the following information:
(deferred) a patch to grub1 -- on top of the already existing patch to
support btrfs in grub1 -- to allow selecting between snapshots of the
boot partition.

>     II. it is so easy to create raid arrays of btrfs partitions but they
>         cannot be read by f13 or f14
>    III. it is not possible to copy btrfs partitions with snapshots
>         except possibly by the use of dd.
> This is not meant to be a put down of btrfs but a plea to have some
> clarification and in particular the ability to boot snapshots.
>
> Hope I can get a response
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: btrfs problems and fedora 14

[C Anthony Risinger]

On Mon, Nov 22, 2010 at 10:47 PM, Wenyi Liu <qingshenlwy@gmail.com> wro=
te:
> 2010/11/23, david grant <dg@david-grant.com>:
>> I thought I would try btrfs on a new installation of f14. yes, I kno=
w
>> its experimental but stable so it seemed to be a good time to try it=
=2E
>> I am not sure if I have missed something out of all my searching but=
 am
>> I correct in thinking that currently:
>> =A0 =A0 =A0I. it is not possible to boot from a snapshot of the oper=
ating
>> =A0 =A0 =A0 =A0 system and, in particular, the yum snapshots cannot =
be used for
>> =A0 =A0 =A0 =A0 that purpose
>
> Is the Fedora grub support btrfs now?
> In this page http://fedoraproject.org/wiki/Features/SystemRollbackWit=
hBtrfs
> I got the following information:
> (deferred) a patch to grub1 -- on top of the already existing patch t=
o
> support btrfs in grub1 -- to allow selecting between snapshots of the
> boot partition.

all you need to do is add:

subvol=3D<name of the snapshot>

-- or --

subvolid=3D<id of the snapshot>

to your kernel boot line (edit in grub on the fly)... however, if
fedora is like archlinux in this respect (brief google search seems to
agree), you will actually need to add this:

rootflags=3Dsubvol=3D<name of the snapshot>

where `rootflags` are the mount options passed to the initramfs/root
device.  also, you reeeeally don't need grub, whatsoever[1]; in arch,
we use an initramfs hook to perform system rollback by dynamically
modifying the rootflags in accordance with the user's choice:

http://aur.archlinux.org/packages/mkinitcpio-btrfs/mkinitcpio-btrfs/btr=
fs_hook

perhaps someone in fedora can adapt that script... it's rather simple,
and it's MUCH easier and safer than fiddling with grub legacy[1].

C Anthony

[1] note however, that a proper grub2/extlinux solution is ideal to
support kernel-level rollbacks.  in the link above, everything is
rolled back except the kernel (residing on /boot... non-btrfs).
though, a kexec solution may be possible.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs problems and fedora 14

[david grant]

Thank you all for your responses to my boot snapshot problem but it
still exists.
.
Hugo, you told me how to mount a snapshot. Thank you, that works but you
didn't tell me how to boot into it.

Anthony, I really hoped that you had provided the answer using grub but
all combinations of your suggestions result in a boot failure with
standard error message of unable to mount root because of of wrong fs
type etc. I assume that with your suggestion I need a standard fstab
entry with default options but it doesn't work even with subvol options.
I am always nervous of messing with the MBR so I want to stick with
grub.

Perhaps this is a fedora problem but I have to say I find it very
strange that they tout btrfs as the future, particularly with respect to
rollbacks but provide no guide to doing this. I assume it is a
combination of grub boot parameters and fstab but nobody seems to know
what to do.

I am not a techo so I just need simple instructions. Is there any other
site, I should be posting this on?

Thanks in anticipation


On Tue, 2010-11-23 at 00:45 -0600, C Anthony Risinger wrote: 
> On Mon, Nov 22, 2010 at 10:47 PM, Wenyi Liu <qingshenlwy@gmail.com> wrote:
> > 2010/11/23, david grant <dg@david-grant.com>:
> >> I thought I would try btrfs on a new installation of f14. yes, I know
> >> its experimental but stable so it seemed to be a good time to try it.
> >> I am not sure if I have missed something out of all my searching but am
> >> I correct in thinking that currently:
> >>      I. it is not possible to boot from a snapshot of the operating
> >>         system and, in particular, the yum snapshots cannot be used for
> >>         that purpose
> >
> > Is the Fedora grub support btrfs now?
> > In this page http://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs
> > I got the following information:
> > (deferred) a patch to grub1 -- on top of the already existing patch to
> > support btrfs in grub1 -- to allow selecting between snapshots of the
> > boot partition.
> 
> all you need to do is add:
> 
> subvol=<name of the snapshot>
> 
> -- or --
> 
> subvolid=<id of the snapshot>
> 
> to your kernel boot line (edit in grub on the fly)... however, if
> fedora is like archlinux in this respect (brief google search seems to
> agree), you will actually need to add this:
> 
> rootflags=subvol=<name of the snapshot>
> 
> where `rootflags` are the mount options passed to the initramfs/root
> device.  also, you reeeeally don't need grub, whatsoever[1]; in arch,
> we use an initramfs hook to perform system rollback by dynamically
> modifying the rootflags in accordance with the user's choice:
> 
> http://aur.archlinux.org/packages/mkinitcpio-btrfs/mkinitcpio-btrfs/btrfs_hook
> 
> perhaps someone in fedora can adapt that script... it's rather simple,
> and it's MUCH easier and safer than fiddling with grub legacy[1].
> 
> C Anthony
> 
> [1] note however, that a proper grub2/extlinux solution is ideal to
> support kernel-level rollbacks.  in the link above, everything is
> rolled back except the kernel (residing on /boot... non-btrfs).
> though, a kexec solution may be possible.

Re: btrfs problems and fedora 14

[cwillu]

On Wed, Nov 24, 2010 at 1:32 AM, david grant <dg@david-grant.com> wrote:
> Hugo, you told me how to mount a snapshot. Thank you, that works but you
> didn't tell me how to boot into it.

He also gave you the command to set the default subvolume/snapshot
used if you don't provide one:  "btrfs subvolume set-default <id>
<path>".  There's also a standard way to send mount options for the
root filesystem, which would allow you to use the mount options he
provided (which Anthony pointed out in his email).

> Anthony, I really hoped that you had provided the answer using grub but
> all combinations of your suggestions result in a boot failure with
> standard error message of unable to mount root because of of wrong fs
> type etc. I assume that with your suggestion I need a standard fstab
> entry with default options but it doesn't work even with subvol options.
> I am always nervous of messing with the MBR so I want to stick with
> grub.

He meant that you distribution uses an initial ram filesystem loaded
into memory with necessary modules, placed in the same place as the
kernel image that grub loads.  This is unrelated to the MBR.

> Perhaps this is a fedora problem but I have to say I find it very
> strange that they tout btrfs as the future, particularly with respect to
> rollbacks but provide no guide to doing this. I assume it is a
> combination of grub boot parameters and fstab but nobody seems to know
> what to do.

The future != the present.  Btrfs will make things like rollback easy
to implement, but it's not implemented yet in useful way for an
untechnical user.  The hard technical bits are over and done with by
the time there are guides on the various forums about "how to do
rollback with btrfs".

> I am not a techo so I just need simple instructions. Is there any other
> site, I should be posting this on?

Not to belabour the point, but a more careful reading of what people
told you would have gotten you up and running.  If those instructions
were too technical, then you probably shouldn't be using btrfs yet:
it's very much at a "some assembly required" stage, and if you don't
understand how your system boots at a basic-but-technical level,
you're either going to come away frustrated, or you're going to have
to learn at least some "linux administrator 101".  :)

Understand what the commands people are giving you actually do, and
you'll have this working in no time.

[sorry for sending this twice David, I consistently fail to hit "reply
to all" when replying to mailing lists]  :(

Re: btrfs problems and fedora 14

[david grant]

Thank you all for your help and in particular you cwillu (sounds
strangely formal!).

Yes, I can now boot into a snapshot but I thought it might be helpful to
explain why I thought otherwise.

I am totally anal about having backups of a current operating systems
and using those for testing I thought tat the best way to do this with
btrfs was to rsync the file system to another partition but exclude all
snapshots. This worked very well as long as I mounted only the root file
system of the copy but what I did was add snapshots to the copy and at
some point (probably at the start) the btree system was corrupted but I
only saw this on backtracking and checking all messages. Also, I didn't
want to boot from a snapshot of my working operating system for fear I
could screw things up and have to re-install from scratch.  In order to
try again, I deleted all snapshots from the original system, did an
rsync and checked the copy. I then made a snapshot of the copy via yum,
used rootflags and it worked!!

So, cwillu, after your scolding of me and your (perfectly reasonable)
questioning of my understanding, I did get it together for booting.

BUT I am still left with the problem that caused it for me: how do I
backup (clone?) a btrfs file system with snapshots to another btrfs
partition (apart from using dd). I just hope I don't get scolded again
and told I am not up to it.




On Wed, 2010-11-24 at 03:19 -0600, cwillu wrote: 
> On Wed, Nov 24, 2010 at 1:32 AM, david grant <dg@david-grant.com> wrote:
> > Hugo, you told me how to mount a snapshot. Thank you, that works but you
> > didn't tell me how to boot into it.
> 
> He also gave you the command to set the default subvolume/snapshot
> used if you don't provide one:  "btrfs subvolume set-default <id>
> <path>".  There's also a standard way to send mount options for the
> root filesystem, which would allow you to use the mount options he
> provided (which Anthony pointed out in his email).
> 
> > Anthony, I really hoped that you had provided the answer using grub but
> > all combinations of your suggestions result in a boot failure with
> > standard error message of unable to mount root because of of wrong fs
> > type etc. I assume that with your suggestion I need a standard fstab
> > entry with default options but it doesn't work even with subvol options.
> > I am always nervous of messing with the MBR so I want to stick with
> > grub.
> 
> He meant that you distribution uses an initial ram filesystem loaded
> into memory with necessary modules, placed in the same place as the
> kernel image that grub loads.  This is unrelated to the MBR.
> 
> > Perhaps this is a fedora problem but I have to say I find it very
> > strange that they tout btrfs as the future, particularly with respect to
> > rollbacks but provide no guide to doing this. I assume it is a
> > combination of grub boot parameters and fstab but nobody seems to know
> > what to do.
> 
> The future != the present.  Btrfs will make things like rollback easy
> to implement, but it's not implemented yet in useful way for an
> untechnical user.  The hard technical bits are over and done with by
> the time there are guides on the various forums about "how to do
> rollback with btrfs".
> 
> > I am not a techo so I just need simple instructions. Is there any other
> > site, I should be posting this on?
> 
> Not to belabour the point, but a more careful reading of what people
> told you would have gotten you up and running.  If those instructions
> were too technical, then you probably shouldn't be using btrfs yet:
> it's very much at a "some assembly required" stage, and if you don't
> understand how your system boots at a basic-but-technical level,
> you're either going to come away frustrated, or you're going to have
> to learn at least some "linux administrator 101".  :)
> 
> Understand what the commands people are giving you actually do, and
> you'll have this working in no time.
> 
> [sorry for sending this twice David, I consistently fail to hit "reply
> to all" when replying to mailing lists]  :(

Re: btrfs problems and fedora 14

[Oystein Viggen]

* [david grant]=20

> BUT I am still left with the problem that caused it for me: how do I
> backup (clone?) a btrfs file system with snapshots to another btrfs
> partition (apart from using dd). I just hope I don't get scolded agai=
n
> and told I am not up to it.

I don't think you can conveniently clone the filesystem including the
snapshots to another computer or partition using traditional userspace
tools like tar or rsync, since they'd end up de-linking the reflink-nes=
s
of the snapshots, so that all the snapshots end up taking the full
space.

However, I can think of one or two strategies that might help you
achieve something close to what you actually want:

1. If the snapshots are just for "online backup", you could backup only
what you consider the live subvol (or even better: a very recent
snapshot of it), and then make snapshots on the target filesystem after
each backup.  While this isn't really a backup including the snapshots,
it might serve the purpose you want.

2. You could rsync the oldest snapshot, make a snapshot of it on the
target filesystem named the same as your second-oldest snapshot, rsync
(--inplace) the second-oldest snapshot into that newly created snapshot=
,
and repeat until you've done all the snapshots.  My head is already
spinning, but it seems to me that it should be possible to automate thi=
s
in a not-too-ugly shell script that also handles updates in a sane way.
This falls to bits, however, if the various snapshots are regularly
written to, or if you can't be sure of their creation order.  (for date=
d
backup snapshots, there shouldn't be a problem).

What would be really awesome is some sort of "btrfs-send" program that
handles all this the best way for you, but I don't think that exists
(yet).  User friendly tools will undoubtedly appear as btrfs is more
used, but I guess it's still partly in the "roll your own" early adopte=
r
stage.  :)

=D8ystein
--=20
"Windows is too dangerous to be left to Windows admins."
                 -- James Riden in the monastery

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs problems and fedora 14

[David Pottage]

On 26/11/10 10:11, Oystein Viggen wrote:
> What would be really awesome is some sort of "btrfs-send" program that
> handles all this the best way for you, but I don't think that exists
> (yet).  User friendly tools will undoubtedly appear as btrfs is more
> used, but I guess it's still partly in the "roll your own" early adopter
> stage.  :)
>    
I agree.

In the past this sort of thing has been handled by adding features to 
tar, so that tar 'knows' how to pack up a filesystem with the latest new 
features (extended meta data for example), and how to restore that data 
at the other end.

-- 
David Pottage.

Re: btrfs problems and fedora 14

[cwillu]

On Fri, Nov 26, 2010 at 3:40 AM, david grant <dg@david-grant.com> wrote=
:
> I am totally anal about having backups of a current operating systems
> and using those for testing I thought tat the best way to do this wit=
h
> btrfs was to rsync the file system to another partition but exclude a=
ll
> snapshots. This worked very well as long as I mounted only the root f=
ile
> system of the copy but what I did was add snapshots to the copy and a=
t
> some point (probably at the start) the btree system was corrupted but=
 I
> only saw this on backtracking and checking all messages. Also, I didn=
't
> want to boot from a snapshot of my working operating system for fear =
I
> could screw things up and have to re-install from scratch. =A0In orde=
r to
> try again, I deleted all snapshots from the original system, did an
> rsync and checked the copy. I then made a snapshot of the copy via yu=
m,
> used rootflags and it worked!!
>
> So, cwillu, after your scolding of me and your (perfectly reasonable)
> questioning of my understanding, I did get it together for booting.
>
> BUT I am still left with the problem that caused it for me: how do I
> backup (clone?) a btrfs file system with snapshots to another btrfs
> partition (apart from using dd).

I use rsync myself, and explicitly list the subvolumes and mirrors I
want copied, which sounds pretty much like what you were doing.  The
corruption you saw definitely wasn't supposed to happen, but depending
on which kernel's you've used and (to a lessor extent) whether a few
particular kernel options are set, isn't too surprising.  Things
_have_ been pretty stable for me for the last little while, basically
since 2.6.35+btrfs_git, and I use snapshots quite a bit.

What I use in a nutshell is:

mountpoint "${BACKUP_TO}/${TODAY}" || {
        btrfs subvolume snapshot "${BACKUP_TO}/${YESTERDAY}"
"${BACKUP_TO}/${TODAY}" || exit 1
}
btrfs subvolume snapshot / /backup-snap && {
        rsync -vaxR --inplace --delete --ignore-errors /backup-snap/./
${BACKUP_TO}/${TODAY}/
        btrfs subvolume delete /backup-snap
}

This will give you incremental backups while avoiding the worst of the
duplication.  I haven't verified that rsync actually does anything
useful COW-wise at the file level, but that's the idea behind the
--inplace option (without it, rsync writes to a copy, and replaces the
original, which COW can't help with).

This is a still a little ways from actually making new snapshots to
fully reproduce the existing filesystem, but I'm not certain that's
what you were after.

> I just hope I don't get scolded again and told I am not up to it.

In point of fact, I said that you _were_ up to it (you were), and that
you'd have it running in no time (you did) once you understood things
better (you do).  I win?  (I always win)  =3DD

[Also, please post your replies under the quoted original, not on top.
 Easier to follow the thread that way.]
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html