RAID5 doesn't mount on boot, but you can afterwards?

RAID5 doesn't mount on boot, but you can afterwards?

[Sjoerd]

Hi All,

A RAID5 setup on raw devices doesn't want to automount on boot. After I skip 
mounting I can log in (Ubuntu server 14.04 on kernel 4.1.8) and just do a 
"sudo mount -a" to get all mounted fine. So the array doesn't seem to be 
broken. "btrfs fi show /data" doesn't show anything wrong either.

The only weird thing I see in the syslog is :

BTRFS info (device sdd): disk space caching is enabled
BTRFS: has skinny extents
BTRFS: failed to read the system array on sdd
BTRFS: open_ctree failed
 
If I reboot the machine the drive in the log changed and looks random (i've 
seen in 3 boots sda, sdc and sde passing by)

I am using btrfs-progs 4.2.1 if that matters in this case...

Anyone have a clue whyt it's not automounting? Or something I can do to 
troubleshoot?

Thanks in advance.

Cheers,
Sjoerd

Re: RAID5 doesn't mount on boot, but you can afterwards?

[Leonidas Spyropoulos]

Hello,

On 30/09/15, Sjoerd wrote:
> Hi All,
> 
> A RAID5 setup on raw devices doesn't want to automount on boot.  
> [..]

Post your /etc/fstab file please.

Thanks
-- 
Sent using mutt

Re: RAID5 doesn't mount on boot, but you can afterwards?

[Sjoerd]

hi my fstab looks as follows (nb: i added the recovery option to see if 
that would help, which didn't) the bootdisk (and @home)is a ssd and the 
label STORAGE represents the RAID5 array:

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sdc1 during installation
UUID=0ea60d4d-3f34-4451-8272-442fcccb7f2e /               btrfs   
recovery,noatime,nodiratime,subvol=@ 0       1
# /home was on /dev/sdc1 during installation
UUID=0ea60d4d-3f34-4451-8272-442fcccb7f2e /home           btrfs   
recovery,noatime,nodiratime,subvol=@home 0       2

# STORAGE
LABEL=STORAGE   /data/HOME              btrfs   
recovery,noatime,nodiratime,compress=zlib,subvol=@home_dir      0       2
LABEL=STORAGE   /data/Pictures          btrfs   
recovery,noatime,nodiratime,compress=zlib,subvol=@pictures      0       2
LABEL=STORAGE   /data/Multimedia        btrfs   
recovery,noatime,nodiratime,compress=zlib,subvol=@multimedia    0       2
LABEL=STORAGE   /data/docker            btrfs   
recovery,noatime,nodiratime,compress=zlib,subvol=@docker                0   
   2LABEL=STORAGE   /data/vms               btrfs   
recovery,noatime,nodiratime,compress=zlib,subvol=@vms           0       2
LABEL=STORAGE   /data/Downloads         btrfs   
recovery,noatime,nodiratime,compress=zlib,subvol=@downloads     0       2
LABEL=STORAGE   /data/Backups           btrfs   
recovery,noatime,nodiratime,compress=zlib,subvol=@backups       0       2
LABEL=STORAGE   /data/Software          btrfs   
recovery,noatime,nodiratime,compress=zlib,subvol=@software      0       2



On September 30, 2015 9:04:39 PM Leonidas Spyropoulos <artafinde@gmail.com> 
wrote:

> Hello,
>
> On 30/09/15, Sjoerd wrote:
>> Hi All,
>>
>> A RAID5 setup on raw devices doesn't want to automount on boot.
>> [..]
>
> Post your /etc/fstab file please.
>
> Thanks
> --
> Sent using mutt
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RAID5 doesn't mount on boot, but you can afterwards?

[Duncan]

Sjoerd posted on Wed, 30 Sep 2015 18:49:21 +0200 as excerpted:

> A RAID5 setup on raw devices doesn't want to automount on boot. After I
> skip mounting I can log in (Ubuntu server 14.04 on kernel 4.1.8) and
> just do a "sudo mount -a" to get all mounted fine. So the array doesn't
> seem to be broken. "btrfs fi show /data" doesn't show anything wrong
> either.
> 
> The only weird thing I see in the syslog is :
> 
> BTRFS info (device sdd): disk space caching is enabled BTRFS: has skinny
> extents BTRFS: failed to read the system array on sdd BTRFS: open_ctree
> failed
>  
> If I reboot the machine the drive in the log changed and looks random
> (i've seen in 3 boots sda, sdc and sde passing by)
> 
> I am using btrfs-progs 4.2.1 if that matters in this case...
> 
> Anyone have a clue whyt it's not automounting? Or something I can do to
> troubleshoot?

That's very likely because unlike traditional single-device filesystems 
(including single-device btrfs), multi-device btrfs has multiple devices 
it must know about before it can mount the device, while mount only feeds 
it one device.

There are two ways to tell btrfs (the kernel side) about the other 
devices.

1) Do a btrfs device scan before trying to mount.

2) Name the component devices in the mount options, using the device= 
option (multiple times as necessary to list all devices).

For various reasons including dynamic device discovery effectively 
randomizing device sd* assignment, btrfs device scan is the normally used 
option.

What's probably happening is that at some point in the boot process, 
btrfs device scan is being automatically run, but it's after the attempt 
to mount the filesystem during boot, so the boot attempt to mount fails, 
but doing a manual mount succeeds, because the scan has already been done 
by the time you get a prompt in ordered to run the command.

So what you need to do is find the service that runs the btrfs device 
scan, and make the mount depend on it, so the scan is done before the 
attempt to mount.  Then it should work.

Or if it's easier, simply create a new service that runs the scan, and 
have it run before the mount, since rerunning the scan twice won't hurt 
anything, it simply needs to run before the mount is attempted in ordered 
for btrfs to know what devices compose the filesystem, so it can be 
mounted.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: RAID5 doesn't mount on boot, but you can afterwards?

[Sjoerd]

On Thursday 01 October 2015 02:21:23 Duncan wrote:

> That's very likely because unlike traditional single-device filesystems
> (including single-device btrfs), multi-device btrfs has multiple devices
> it must know about before it can mount the device, while mount only feeds
> it one device.
> 
> There are two ways to tell btrfs (the kernel side) about the other
> devices.
> 
> 1) Do a btrfs device scan before trying to mount.
> 
> 2) Name the component devices in the mount options, using the device=
> option (multiple times as necessary to list all devices).
> 
Option 2 was to simplest to check and that works. Thanks for the tip!
Still weird that my single devide SSD BTRFS bootdisk just worked fine 
(althought it's using the uuid offcourse)...But it would imply to me that 
there's a btrfs device scan run before mounting it.
Before I created the RAID5 I just had the four disks configures as JBOD in 
btrfs (I used partitions that time though) and mounted it just fine with the 
LABEL only, so it might be something with RAID5.

Anyway it's working now with just a much longer fstab line ;)

Cheers,
Sjoerd

Re: RAID5 doesn't mount on boot, but you can afterwards?

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1419 bytes --]

On Thu, Oct 01, 2015 at 07:04:43PM +0200, Sjoerd wrote:
> On Thursday 01 October 2015 02:21:23 Duncan wrote:
> 
> > That's very likely because unlike traditional single-device filesystems
> > (including single-device btrfs), multi-device btrfs has multiple devices
> > it must know about before it can mount the device, while mount only feeds
> > it one device.
> > 
> > There are two ways to tell btrfs (the kernel side) about the other
> > devices.
> > 
> > 1) Do a btrfs device scan before trying to mount.
> > 
> > 2) Name the component devices in the mount options, using the device=
> > option (multiple times as necessary to list all devices).
> > 
> Option 2 was to simplest to check and that works. Thanks for the tip!
> Still weird that my single devide SSD BTRFS bootdisk just worked fine 
> (althought it's using the uuid offcourse)...But it would imply to me that 
> there's a btrfs device scan run before mounting it.

   Not really. A single deice FS doesn't need the scan.

> Before I created the RAID5 I just had the four disks configures as JBOD in 
> btrfs (I used partitions that time though) and mounted it just fine with the 
> LABEL only, so it might be something with RAID5.
> 
> Anyway it's working now with just a much longer fstab line ;)
> 
> Cheers,
> Sjoerd
> 

-- 
Hugo Mills             | Vote early, vote often
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: RAID5 doesn't mount on boot, but you can afterwards?

[Duncan]

Hugo Mills posted on Thu, 01 Oct 2015 17:46:15 +0000 as excerpted:

> On Thu, Oct 01, 2015 at 07:04:43PM +0200, Sjoerd wrote:
>> On Thursday 01 October 2015 02:21:23 Duncan wrote:
>> 
>> > That's very likely because unlike traditional single-device
>> > filesystems (including single-device btrfs), multi-device btrfs has
>> > multiple devices it must know about before it can mount the device,
>> > while mount only feeds it one device.
>> > 
>> > There are two ways to tell btrfs (the kernel side) about the other
>> > devices.
>> > 
>> > 1) Do a btrfs device scan before trying to mount.
>> > 
>> > 2) Name the component devices in the mount options, using the device=
>> > option (multiple times as necessary to list all devices).
>> > 
>> Option 2 was to simplest to check and that works. Thanks for the tip!
>> Still weird that my single devide SSD BTRFS bootdisk just worked fine
>> (althought it's using the uuid offcourse)...But it would imply to me
>> that there's a btrfs device scan run before mounting it.
> 
> Not really. A single deice FS doesn't need the scan.

Yes.

A mount command takes a single device pointer either on the commandline, 
or from fstab.  For traditional single-device filesystems, that pointer, 
whether via traditional /dev/* path, or by (udev-mediated) LABEL=, UUID=, 
etc, is all that's needed, one device, and the kernel knows what it was 
because it was supplied in that pointer.

But for non-traditional multi-device filesystems, like btrfs in multi-
device mode (as opposed to btrfs used on only a single device where the 
single device pointer works fine), a single device pointer only provides 
part of the necessary information, the kernel has to figure out what 
other devices are needed by some other method.  With btrfs, there are two 
such other methods, btrfs device scan, or supplying the other devices via 
device= mount option, with as many such device= options used as necessary 
to list all filesystem component devices.

... Which is what I was trying to explain in the earlier reply as well, 
when I specifically included the "including single-device btrfs" 
parenthetical in the traditional device class, contrasted with multi-
device btrfs, but apparently that specific bit didn't transfer.

Well, at least the practical solution, use device scan or name the 
devices in mount options, did. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

RE: RAID5 doesn't mount on boot, but you can afterwards?

[guido_kuenne]

> -----Original Message-----
> From: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-
> owner@vger.kernel.org] On Behalf Of Duncan
> Sent: Friday, October 02, 2015 3:12 AM
> To: linux-btrfs@vger.kernel.org
> Subject: Re: RAID5 doesn't mount on boot, but you can afterwards?
> 
> Hugo Mills posted on Thu, 01 Oct 2015 17:46:15 +0000 as excerpted:
> 
> > On Thu, Oct 01, 2015 at 07:04:43PM +0200, Sjoerd wrote:
> >> On Thursday 01 October 2015 02:21:23 Duncan wrote:
> >>
> >> > That's very likely because unlike traditional single-device
> >> > filesystems (including single-device btrfs), multi-device btrfs has
> >> > multiple devices it must know about before it can mount the device,
> >> > while mount only feeds it one device.
> >> >
> >> > There are two ways to tell btrfs (the kernel side) about the other
> >> > devices.
> >> >
> >> > 1) Do a btrfs device scan before trying to mount.
> >> >
> >> > 2) Name the component devices in the mount options, using the
> >> > device= option (multiple times as necessary to list all devices).
> >> >
> >> Option 2 was to simplest to check and that works. Thanks for the tip!
> >> Still weird that my single devide SSD BTRFS bootdisk just worked fine
> >> (althought it's using the uuid offcourse)...But it would imply to me
> >> that there's a btrfs device scan run before mounting it.
> >
> > Not really. A single deice FS doesn't need the scan.
> 
> Yes.
> 
> A mount command takes a single device pointer either on the
> commandline, or from fstab.  For traditional single-device filesystems, that
> pointer, whether via traditional /dev/* path, or by (udev-mediated) LABEL=,
> UUID=, etc, is all that's needed, one device, and the kernel knows what it
> was because it was supplied in that pointer.
> 
> But for non-traditional multi-device filesystems, like btrfs in multi- device
> mode (as opposed to btrfs used on only a single device where the single
> device pointer works fine), a single device pointer only provides part of the
> necessary information, the kernel has to figure out what other devices are
> needed by some other method.  With btrfs, there are two such other
> methods, btrfs device scan, or supplying the other devices via device=
> mount option, with as many such device= options used as necessary to list
> all filesystem component devices.
> 
> ... Which is what I was trying to explain in the earlier reply as well, when I
> specifically included the "including single-device btrfs"
> parenthetical in the traditional device class, contrasted with multi- device
> btrfs, but apparently that specific bit didn't transfer.
> 
> Well, at least the practical solution, use device scan or name the devices in
> mount options, did. =:^)
> 
Beginner here, so just if it helps: My two-device raid 1 mounts on boot in Fedora 22 (uuid in fstab, no further devices specified) but I mount the fs via uuid while Sjoerd mounted subvolumes. From what I understand (not much) it's either an subvolume issue or Fedora must then somehow perform a device scan before handling fstab. 

Re: RAID5 doesn't mount on boot, but you can afterwards?

[Duncan]

guido_kuenne posted on Sat, 03 Oct 2015 22:53:46 +0200 as excerpted:

> Beginner here, so just if it helps: My two-device raid 1 mounts on boot
> in Fedora 22 (uuid in fstab, no further devices specified) but I mount
> the fs via uuid while Sjoerd mounted subvolumes. From what I understand
> (not much) it's either an subvolume issue or Fedora must then somehow
> perform a device scan before handling fstab.

I'm guessing fedora has btrfs device scan in its initr*.

I believe dracut is a fedora based project and what they use to generate 
their initr*s.  I too use dracut for my initr* generation, and btrfs is 
an optional module which I obviously load (in a site-specific custom 
initr* here, on gentoo), and I know it has the btrfs device scan in the 
initr* here, so presumably it's in the general purpose initr* they use as 
well.  The only way it wouldn't would be if they didn't include the btrfs 
dracut module at all in their initr*, and I think btrfs is popular enough 
that they include it, so...

... So even if Fedora doesn't include btrfs device scan in the real-root 
init (which I'd guess it does as well), it's going to be in the initr*, 
which means unless the devices for that btrfs aren't plugged in until 
after boot, it should see them due to the scan at the initr* stage, and 
"just work", mounting the btrfs without problems at any time after that.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: RAID5 doesn't mount on boot, but you can afterwards?

[Sjoerd]

On Sunday 04 October 2015 02:28:29 Duncan wrote:
> guido_kuenne posted on Sat, 03 Oct 2015 22:53:46 +0200 as excerpted:
> > Beginner here, so just if it helps: My two-device raid 1 mounts on boot
> > in Fedora 22 (uuid in fstab, no further devices specified) but I mount
> > the fs via uuid while Sjoerd mounted subvolumes. From what I understand
> > (not much) it's either an subvolume issue or Fedora must then somehow
> > perform a device scan before handling fstab.
> 
> I'm guessing fedora has btrfs device scan in its initr*.

I think so too, since I was wondering why it did mount my multidevice JBOD 
correctly before mounting it in the same way as written in the first mail.
Later I found the probable answer that the JBOD setup was with Ubuntu 15.04 
and most likely they also put a btrfs scan somewhere in the init before 
mounting. So there're some approvements btrfs wise when not using the latest 
LTS release..

Sjoerd