Mounting raid without a btrfsctl scan

Mounting raid without a btrfsctl scan

[Matt Brown]

Hi,

Would it be possible and feasible to support mounting btrfs
raid/multi-device filesystems without having to run 'btrfsctl -a'?

Currently, as you may know, if a one wants to attach a btrfs raid
filesystem to a system (usb, hotswap, reboot, etc), the user or program
has to run:

btrfsctl -a (or similar)
mount /dev/sdb1 /mount/point

While this works, it will require patching of various subsystems
involved with managing disks, such as udev, mkinitrd, dracut, hal, and
others. Each one will have to know to scan, then mount.

For example, I have a system that has a btrfs raid1 as root. However, I
had to patch the boot loader (dracut) so during boot it would scan just
before mounting the root filesystem.

I filed a bug with dracut, but the more I think of it, the more it seems
that either mount.btrfs should be taking care of this, or another part
of btrfs.

Any thoughts or plans on the matter?

Thanks,
Matt

Re: Mounting raid without a btrfsctl scan

[Goffredo Baroncelli]

On Saturday 15 May 2010, Matt Brown wrote:
> Hi,
Hi Matt,
> 
> Would it be possible and feasible to support mounting btrfs
> raid/multi-device filesystems without having to run 'btrfsctl -a'?
> 
> Currently, as you may know, if a one wants to attach a btrfs raid
> filesystem to a system (usb, hotswap, reboot, etc), the user or program
> has to run:
> 
> btrfsctl -a (or similar)
> mount /dev/sdb1 /mount/point
> 
> While this works, it will require patching of various subsystems
> involved with managing disks, such as udev, mkinitrd, dracut, hal, and
> others. Each one will have to know to scan, then mount.

In a my previous post ([RFC] btrfs, udev and btrfs - 16/April 2010), I 
suggested a solution for this kind of problem. I a Debian/Ubuntu system it is 
not necessary to patch anything, it is only required to put some files in the 
initramfs-tool configure directories.

IIRC, also the md (RAID) subsystem require a scan from the user space to find 
and activate the volumes. I think also dm (LVM) subsystem requires the same.

> 
> For example, I have a system that has a btrfs raid1 as root. However, I
> had to patch the boot loader (dracut) so during boot it would scan just
> before mounting the root filesystem.
> 
> I filed a bug with dracut, but the more I think of it, the more it seems
> that either mount.btrfs should be taking care of this, or another part
> of btrfs.

If it would be mount.btrfs to perform the scan, that means a scan for every 
mounting. I think that is better to separate the two function. The scan has to 
be performed a device discovery time, and not a the mounting time.


> 
> Any thoughts or plans on the matter?
> 
> Thanks,
> Matt

BR
Goffredo
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


-- 
gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo) <kreijackATinwind.it>
Key fingerprint = 4769 7E51 5293 D36C 814E  C054 BF04 F161 3DC5 0512

Re: Mounting raid without a btrfsctl scan

[Sebastian 'gonX' Jensen]

On 15 May 2010 16:36, Goffredo Baroncelli <kreijack@gmail.com> wrote:
> On Saturday 15 May 2010, Matt Brown wrote:
>> Hi,
> Hi Matt,
Hey guys, first of all, apologize for the double post for Goffredo, I
forgot to add linux-btrfs to my recipient list.

>>
>> Would it be possible and feasible to support mounting btrfs
>> raid/multi-device filesystems without having to run 'btrfsctl -a'?
>>
>> Currently, as you may know, if a one wants to attach a btrfs raid
>> filesystem to a system (usb, hotswap, reboot, etc), the user or prog=
ram
>> has to run:
>>
>> btrfsctl -a (or similar)
>> mount /dev/sdb1 /mount/point
>>
>> While this works, it will require patching of various subsystems
>> involved with managing disks, such as udev, mkinitrd, dracut, hal, a=
nd
>> others. Each one will have to know to scan, then mount.
>
> In a my previous post ([RFC] btrfs, udev and btrfs - 16/April 2010), =
I
> suggested a solution for this kind of problem. I a Debian/Ubuntu syst=
em it is
> not necessary to patch anything, it is only required to put some file=
s in the
> initramfs-tool configure directories.
>
> IIRC, also the md (RAID) subsystem require a scan from the user space=
 to find
> and activate the volumes. I think also dm (LVM) subsystem requires th=
e same.
>

md does not require a scan from userspace with at least both RAID-0
and RAID-1. There's a setting in the kernel config you can set, which
will automatically detect all RAID drives with the FD partition flag
set. DM may however be true.

I think the idea of this, is to avoid having initramfs deal with this?
To be honest that's the only thing I use it for now, and it slows my
boot time by about 5 seconds.

>>
>> For example, I have a system that has a btrfs raid1 as root. However=
, I
>> had to patch the boot loader (dracut) so during boot it would scan j=
ust
>> before mounting the root filesystem.
>>
>> I filed a bug with dracut, but the more I think of it, the more it s=
eems
>> that either mount.btrfs should be taking care of this, or another pa=
rt
>> of btrfs.
>
> If it would be mount.btrfs to perform the scan, that means a scan for=
 every
> mounting. I think that is better to separate the two function. The sc=
an has to
> be performed a device discovery time, and not a the mounting time.

Would it not be possible to have a cache of sorts for btrfs that check
each device/partition based on an UUID and drive serial number (hdparm
should have this) every time a mount with selected devices is done?
e.g:
/dev/sda1 is attempted to be mounted. This drive is not in the btrfs
drive cache, and will then be probed for btrfs RAID arrays. The cache
should eventually be volatile, but could also be on a certain place on
the partition, e.g. together with metadata for a cache of all btrfs
drives used.

Anybody not with me?

>
>
>>
>> Any thoughts or plans on the matter?
>>
>> Thanks,
>> Matt
>
> BR
> Goffredo

Regards,
Sebastian J.

>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrf=
s" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at =C2=A0http://vger.kernel.org/majordomo-info.h=
tml
>>
>
>
> --
> gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo) <kreijackATi=
nwind.it>
> Key fingerprint =3D 4769 7E51 5293 D36C 814E =C2=A0C054 BF04 F161 3DC=
5 0512
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =C2=A0http://vger.kernel.org/majordomo-info.ht=
ml
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mounting raid without a btrfsctl scan

[Tomasz Torcz]

On Sat, May 15, 2010 at 06:47:22PM +0200, Sebastian 'gonX' Jensen wrote:
> >> Would it be possible and feasible to support mounting btrfs
> >> raid/multi-device filesystems without having to run 'btrfsctl -a'?
> >>
> >> Currently, as you may know, if a one wants to attach a btrfs raid
> >> filesystem to a system (usb, hotswap, reboot, etc), the user or program
> >> has to run:
> >>
> >> btrfsctl -a (or similar)
> >> mount /dev/sdb1 /mount/point
> >>
> >> While this works, it will require patching of various subsystems
> >> involved with managing disks, such as udev, mkinitrd, dracut, hal, and
> >> others. Each one will have to know to scan, then mount.
> >
> > In a my previous post ([RFC] btrfs, udev and btrfs - 16/April 2010), I
> > suggested a solution for this kind of problem. I a Debian/Ubuntu system it is
> > not necessary to patch anything, it is only required to put some files in the
> > initramfs-tool configure directories.
> >
> > IIRC, also the md (RAID) subsystem require a scan from the user space to find
> > and activate the volumes. I think also dm (LVM) subsystem requires the same.
> >
> 
> md does not require a scan from userspace with at least both RAID-0
> and RAID-1. There's a setting in the kernel config you can set, which
> will automatically detect all RAID drives with the FD partition flag
> set. DM may however be true.


  MD will only detect RAID on MS-DOS partitions (seldom used when RAIDing
whole devices, not possible to use with devices >2 TiB) and metadata
0.90, which is not default. Userspace probing is general way kernel 
subsystem are going, it is more flexible.
  And dracut already has btrfs module.

-- 
Tomasz Torcz              ,,If you try to upissue this patchset I shall be seeking
xmpp: zdzichubg@chrome.pl   an IP-routable hand grenade.'' -- Andrew Morton (LKML)

Re: Mounting raid without a btrfsctl scan

[Sebastian 'gonX' Jensen]

On 16 May 2010 02:02, Tomasz Torcz <tomek@pipebreaker.pl> wrote:
> On Sat, May 15, 2010 at 06:47:22PM +0200, Sebastian 'gonX' Jensen wro=
te:
>> >> Would it be possible and feasible to support mounting btrfs
>> >> raid/multi-device filesystems without having to run 'btrfsctl -a'=
?
>> >>
>> >> Currently, as you may know, if a one wants to attach a btrfs raid
>> >> filesystem to a system (usb, hotswap, reboot, etc), the user or p=
rogram
>> >> has to run:
>> >>
>> >> btrfsctl -a (or similar)
>> >> mount /dev/sdb1 /mount/point
>> >>
>> >> While this works, it will require patching of various subsystems
>> >> involved with managing disks, such as udev, mkinitrd, dracut, hal=
, and
>> >> others. Each one will have to know to scan, then mount.
>> >
>> > In a my previous post ([RFC] btrfs, udev and btrfs - 16/April 2010=
), I
>> > suggested a solution for this kind of problem. I a Debian/Ubuntu s=
ystem it is
>> > not necessary to patch anything, it is only required to put some f=
iles in the
>> > initramfs-tool configure directories.
>> >
>> > IIRC, also the md (RAID) subsystem require a scan from the user sp=
ace to find
>> > and activate the volumes. I think also dm (LVM) subsystem requires=
 the same.
>> >
>>
>> md does not require a scan from userspace with at least both RAID-0
>> and RAID-1. There's a setting in the kernel config you can set, whic=
h
>> will automatically detect all RAID drives with the FD partition flag
>> set. DM may however be true.
>
>
> =C2=A0MD will only detect RAID on MS-DOS partitions (seldom used when=
 RAIDing
> whole devices, not possible to use with devices >2 TiB) and metadata
> 0.90, which is not default. Userspace probing is general way kernel
> subsystem are going, it is more flexible.
> =C2=A0And dracut already has btrfs module.
>

You're right, only metadata 0.9 is supported with autodetection
kernel-side. I had forgotten about that.

On a more related note, I agree that userspace probing is a lot more
flexible, but for mounting root partitions it is also extremely
inconvenient. I don't know if I mentioned it before, but my system
spends at least 6 seconds in the initramfs before passing control over
to my distributions initscripts. Part of it is obviously because of
btrfs having to spend approximately 3 seconds probing for devices (I
have a decent amount of devices). However, btrfs does not have that
many functions yet. Would it be possible to at least have partial
functionality kernel side, so that an initramfs is not required for
mounting RAID devices?

> --
> Tomasz Torcz =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0,,If you=
 try to upissue this patchset I shall be seeking
> xmpp: zdzichubg@chrome.pl =C2=A0 an IP-routable hand grenade.'' -- An=
drew Morton (LKML)
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =C2=A0http://vger.kernel.org/majordomo-info.ht=
ml
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mounting raid without a btrfsctl scan

[Goffredo Baroncelli]

On Sunday 16 May 2010, Sebastian 'gonX' Jensen wrote:
> On 16 May 2010 02:02, Tomasz Torcz <tomek@pipebreaker.pl> wrote:

[...]
> I don't know if I mentioned it before, but my system
> spends at least 6 seconds in the initramfs before passing control over
> to my distributions initscripts. Part of it is obviously because of
> btrfs having to spend approximately 3 seconds probing for devices (I
> have a decent amount of devices). 


> However, btrfs does not have that
> many functions yet. Would it be possible to at least have partial
> functionality kernel side, so that an initramfs is not required for
> mounting RAID devices?

I think that the good question is: why a kernel probing should be faster than 
a user space probing ? 

I think that some reasons of the time required by the btrfsctl scan in user 
space are:
1) a scan performed via 'btrfsctl -a' scans every block device (cdrom and 
floppy included). I suggest to perform a scan via udev (see my previous post); 
that should be reduce the boot time.
2) the actual initramfs tools pay attention to wait that _all_ devices 
appeared. This requires a bit of time which is not related to the "user space" 
scan.

Comments ?

G.Baroncelli

> 
> > --
> > Tomasz Torcz              ,,If you try to upissue this patchset I shall be 
seeking
> > xmpp: zdzichubg@chrome.pl   an IP-routable hand grenade.'' -- Andrew 
Morton (LKML)
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


-- 
gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo) <kreijackATinwind.it>
Key fingerprint = 4769 7E51 5293 D36C 814E  C054 BF04 F161 3DC5 0512

Re: Mounting raid without a btrfsctl scan

[Sebastian 'gonX' Jensen]

Resending because I forgot to add linux-btrfs... AGAIN!

On 16 May 2010 20:44, Goffredo Baroncelli <kreijack@gmail.com> wrote:
> On Sunday 16 May 2010, Sebastian 'gonX' Jensen wrote:
>> On 16 May 2010 02:02, Tomasz Torcz <tomek@pipebreaker.pl> wrote:
>
> [...]
>> I don't know if I mentioned it before, but my system
>> spends at least 6 seconds in the initramfs before passing control over
>> to my distributions initscripts. Part of it is obviously because of
>> btrfs having to spend approximately 3 seconds probing for devices (I
>> have a decent amount of devices).
>
>
>> However, btrfs does not have that
>> many functions yet. Would it be possible to at least have partial
>> functionality kernel side, so that an initramfs is not required for
>> mounting RAID devices?
>
> I think that the good question is: why a kernel probing should be faster than
> a user space probing ?

It's unlikely that it is (it's also harder to code, probably), but
having to initialize the entire ramfs (3 seconds on my reasonably fast
desktop computer) before being able to scan for devices with btrfs is
something that could probably be avoided. A luxury, if you will.

> I think that some reasons of the time required by the btrfsctl scan in user
> space are:
> 1) a scan performed via 'btrfsctl -a' scans every block device (cdrom and
> floppy included). I suggest to perform a scan via udev (see my previous post);
> that should be reduce the boot time.

I agree, and I think this is a good idea. This could also be
implemented if kernel-side probing is to be implemented.
If UDev scanning isn't feasible yet, or just a luxury, would it at
least not be possible to select what partitions/drives to probe, to
save time? This seems (from my limited programming perspective) fairly
easy to implement.

> 2) the actual initramfs tools pay attention to wait that _all_ devices
> appeared. This requires a bit of time which is not related to the "user space"
> scan.
>
> Comments ?
>
> G.Baroncelli
>

- Sebastian J.

[...]