btrfs and systemd

btrfs and systemd

[Stefan Priebe - Profihost AG]

Hi,

i'm trying to get my 60TB btrfs volume to mount with systemd at boot.
But this always fails with: "mounting timed out. Stopping." after 90s.

I can't find any fstab setting for systemd to higher this timeout.
There's just  the x-systemd.device-timeout but this controls how long to
wait for the device and not for the mount command.

Is there any solution for big btrfs volumes and systemd?

Greets,
Stefan

Re: btrfs and systemd

[Duncan]

Stefan Priebe - Profihost AG posted on Sun, 28 Aug 2016 22:15:21 +0200 as
excerpted:

> Hi,
> 
> i'm trying to get my 60TB btrfs volume to mount with systemd at boot.
> But this always fails with: "mounting timed out. Stopping." after 90s.
> 
> I can't find any fstab setting for systemd to higher this timeout.
> There's just  the x-systemd.device-timeout but this controls how long to
> wait for the device and not for the mount command.
> 
> Is there any solution for big btrfs volumes and systemd?

Yes.

The key here is to realize that systemd creates mount units dynamically 
based on the fstab file, placing them in /run/systemd/generator/ (take a 
look =:^), and then uses them to process mounts.  So the normal method 
for reconfiguring/overriding systemd units applies.  See the systemd.unit 
manpage, example section, example 2 (as of systemd 230), for override 
details, but in general you choose whether you want to override the 
entire default unit (probably not, as you'd replace the generated file 
and override the fstab options) or just specific options (likely what you 
want), and place your override file in the appropriate subdir of 
/etc/systemd/ accordingly.


Now that you know where to put the file with the override, take a look at 
the systemd.mount manpage, options section.  Based on that (IOW, I've not 
actually tried this), the setting you need to change is TimeoutSec.


You can also consider changing the global timeout setting by setting 
DefaultTimeoutStart in /etc/systemd/system.conf, tho that's going to 
affect other units as well.  But it may be easier, if your other units 
start up fine without getting anywhere close to the timeout, and thus 
never need to use the timeout setting.  Of course if they break, you'll 
then be dealing with the longer timeouts, but you may find it easier to 
simply set that default and deal with the long timeouts on anything else 
when and if some other unit does actually break and start following the 
longer default timeouts.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: btrfs and systemd

[Qu Wenruo]

At 08/29/2016 04:15 AM, Stefan Priebe - Profihost AG wrote:
> Hi,
>
> i'm trying to get my 60TB btrfs volume to mount with systemd at boot.
> But this always fails with: "mounting timed out. Stopping." after 90s.

60TB is quite large, and under most case it will already cause mount 
speed problem.

In our test environment, filling a fs with 16K small files to 2T (just 
128K files)will already slow the mount process to 10s.

For larger fs, or more specifically, large extent tree, will slow the 
mount process obviously.

The root fix will need a rework of extent tree.
AFAIK Josef is working on the rework.

So the btrfs fix will need some time.

Thanks,
Qu
>
> I can't find any fstab setting for systemd to higher this timeout.
> There's just  the x-systemd.device-timeout but this controls how long to
> wait for the device and not for the mount command.
>
> Is there any solution for big btrfs volumes and systemd?
>
> Greets,
> Stefan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>

Re: btrfs and systemd

[Stefan Priebe - Profihost AG]

Hi Qu,

Am 29.08.2016 um 03:48 schrieb Qu Wenruo:
> 
> 
> At 08/29/2016 04:15 AM, Stefan Priebe - Profihost AG wrote:
>> Hi,
>>
>> i'm trying to get my 60TB btrfs volume to mount with systemd at boot.
>> But this always fails with: "mounting timed out. Stopping." after 90s.
> 
> 60TB is quite large, and under most case it will already cause mount
> speed problem.
> 
> In our test environment, filling a fs with 16K small files to 2T (just
> 128K files)will already slow the mount process to 10s.
> 
> For larger fs, or more specifically, large extent tree, will slow the
> mount process obviously.
> 
> The root fix will need a rework of extent tree.
> AFAIK Josef is working on the rework.
> 
> So the btrfs fix will need some time.

thanks but i've no problem with the long mount time (in my case 6
minutes) i'm just wondering how to live with it with systemd. As it
always cancels the mount process after 90s and i see no fstab option to
change this.

Greets,
Stefan

> 
> Thanks,
> Qu
>>
>> I can't find any fstab setting for systemd to higher this timeout.
>> There's just  the x-systemd.device-timeout but this controls how long to
>> wait for the device and not for the mount command.
>>
>> Is there any solution for big btrfs volumes and systemd?
>>
>> Greets,
>> Stefan
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
> 
> 

Re: btrfs and systemd

[Imran Geriskovan]

>>> I can't find any fstab setting for systemd to higher this timeout.
>>> There's just  the x-systemd.device-timeout but this controls how long to
>>> wait for the device and not for the mount command.
>>> Is there any solution for big btrfs volumes and systemd?
>>> Stefan

Switch to Runit.

First time I seriously consider another init on my
notebook is when I have a problem like yours.

Even when / (root) is mounted just fine, if there is
any problem with any other fstab entry, you'll
get into such a situation on systemd.

Give it a try, appending "init=/usr/bin/runit-init"
to your kernel command line on your bootloader.
You dont need to uninstall any package until
getting Runit behave "exactly" as you like.

RE: btrfs and systemd

[Paul Jones]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1446 bytes --]

> -----Original Message-----
> From: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-
> owner@vger.kernel.org] On Behalf Of Imran Geriskovan
> Sent: Monday, 29 August 2016 9:19 PM
> To: Stefan Priebe - Profihost AG <s.priebe@profihost.ag>
> Cc: Qu Wenruo <quwenruo@cn.fujitsu.com>; linux-btrfs@vger.kernel.org
> Subject: Re: btrfs and systemd
> 
> >>> I can't find any fstab setting for systemd to higher this timeout.
> >>> There's just  the x-systemd.device-timeout but this controls how
> >>> long to wait for the device and not for the mount command.
> >>> Is there any solution for big btrfs volumes and systemd?
> >>> Stefan
> 
> Switch to Runit.
> 
> First time I seriously consider another init on my notebook is when I have a
> problem like yours.
> 
> Even when / (root) is mounted just fine, if there is any problem with any
> other fstab entry, you'll get into such a situation on systemd.
> 
> Give it a try, appending "init=/usr/bin/runit-init"
> to your kernel command line on your bootloader.
> You dont need to uninstall any package until getting Runit behave "exactly"
> as you like.

Why not just create a Systemd unit (or whatever the proper term is) that runs on boot and runs the mount command manually and doesn't wait for it to return? Seems easier than messing with init systems.


Paul.
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±ý»k~ÏâžØ^n‡r¡ö¦zË\x1aëh™¨è­Ú&£ûàz¿äz¹Þ—ú+€Ê+zf£¢·hšˆ§~†­†Ûiÿÿïêÿ‘êçz_è®\x0fæj:+v‰¨þ)ߣøm

Re: btrfs and systemd

[Imran Geriskovan]

> Why not just create a Systemd unit (or whatever the proper term is) that
> runs on boot and runs the mount command manually and doesn't wait for it to
> return? Seems easier than messing with init systems.

Exactly: Never "mess" with inits..

Re: btrfs and systemd

[Austin S. Hemmelgarn]

On 2016-08-29 07:18, Imran Geriskovan wrote:
>>>> I can't find any fstab setting for systemd to higher this timeout.
>>>> There's just  the x-systemd.device-timeout but this controls how long to
>>>> wait for the device and not for the mount command.
>>>> Is there any solution for big btrfs volumes and systemd?
>>>> Stefan
>
> Switch to Runit.
>
> First time I seriously consider another init on my
> notebook is when I have a problem like yours.
>
> Even when / (root) is mounted just fine, if there is
> any problem with any other fstab entry, you'll
> get into such a situation on systemd.
>
> Give it a try, appending "init=/usr/bin/runit-init"
> to your kernel command line on your bootloader.
> You dont need to uninstall any package until
> getting Runit behave "exactly" as you like.
I hate to say this, but based on my own experience, I have to say that 
switching to a different init system is usually the simplest and 
quickest solution to almost any 'systemd does X that I don't like' problems.

I can't personally give any advice on runit as I've never used it 
myself, but I can say that the stack I'm using (sysv-init, OpenRC, and 
monit) provides most of the good things about systemd except for socket 
activation and interruptible parallel startup, and it's not hard to get 
similarly complete configurations with other tools too.

The downside to this of course is that there are some projects that 
shouldn't need to depend on a particular init system that do, so some 
software may need to be replaced on your system.

As far as this specific case, mount units are what you want to look 
into, as mentioned by Duncan.

Re: btrfs and systemd

[Timofey Titovets]

Do you try: nofail,noauto,x-systemd.automount ?

2016-08-29 9:28 GMT+03:00 Stefan Priebe - Profihost AG <s.priebe@profihost.ag>:
> Hi Qu,
>
> Am 29.08.2016 um 03:48 schrieb Qu Wenruo:
>>
>>
>> At 08/29/2016 04:15 AM, Stefan Priebe - Profihost AG wrote:
>>> Hi,
>>>
>>> i'm trying to get my 60TB btrfs volume to mount with systemd at boot.
>>> But this always fails with: "mounting timed out. Stopping." after 90s.
>>
>> 60TB is quite large, and under most case it will already cause mount
>> speed problem.
>>
>> In our test environment, filling a fs with 16K small files to 2T (just
>> 128K files)will already slow the mount process to 10s.
>>
>> For larger fs, or more specifically, large extent tree, will slow the
>> mount process obviously.
>>
>> The root fix will need a rework of extent tree.
>> AFAIK Josef is working on the rework.
>>
>> So the btrfs fix will need some time.
>
> thanks but i've no problem with the long mount time (in my case 6
> minutes) i'm just wondering how to live with it with systemd. As it
> always cancels the mount process after 90s and i see no fstab option to
> change this.
>
> Greets,
> Stefan
>
>>
>> Thanks,
>> Qu
>>>
>>> I can't find any fstab setting for systemd to higher this timeout.
>>> There's just  the x-systemd.device-timeout but this controls how long to
>>> wait for the device and not for the mount command.
>>>
>>> Is there any solution for big btrfs volumes and systemd?
>>>
>>> Greets,
>>> Stefan
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>>
>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Have a nice day,
Timofey.

Re: btrfs and systemd

[Stefan Priebe - Profihost AG]

Am 29.08.2016 um 13:33 schrieb Timofey Titovets:
> Do you try: nofail,noauto,x-systemd.automount ?

sure this fails too as it has the same timeout in systemd.

Mr. Poettering has recommanded me todo the following:
# mkdir -p /etc/systemd/system/$(systemd-escape --suffix=mount -p
/foo/bar/baz).d/
# cat > /etc/systemd/system/$(systemd-escape --suffix=mount -p
/foo/bar/baz).d/timeout.conf <<EOF
[Mount]
TimeoutSec=2h
EOF

he has also accepted my wish to make this configurable through fstab.

Stefan

> 2016-08-29 9:28 GMT+03:00 Stefan Priebe - Profihost AG <s.priebe@profihost.ag>:
>> Hi Qu,
>>
>> Am 29.08.2016 um 03:48 schrieb Qu Wenruo:
>>>
>>>
>>> At 08/29/2016 04:15 AM, Stefan Priebe - Profihost AG wrote:
>>>> Hi,
>>>>
>>>> i'm trying to get my 60TB btrfs volume to mount with systemd at boot.
>>>> But this always fails with: "mounting timed out. Stopping." after 90s.
>>>
>>> 60TB is quite large, and under most case it will already cause mount
>>> speed problem.
>>>
>>> In our test environment, filling a fs with 16K small files to 2T (just
>>> 128K files)will already slow the mount process to 10s.
>>>
>>> For larger fs, or more specifically, large extent tree, will slow the
>>> mount process obviously.
>>>
>>> The root fix will need a rework of extent tree.
>>> AFAIK Josef is working on the rework.
>>>
>>> So the btrfs fix will need some time.
>>
>> thanks but i've no problem with the long mount time (in my case 6
>> minutes) i'm just wondering how to live with it with systemd. As it
>> always cancels the mount process after 90s and i see no fstab option to
>> change this.
>>
>> Greets,
>> Stefan
>>
>>>
>>> Thanks,
>>> Qu
>>>>
>>>> I can't find any fstab setting for systemd to higher this timeout.
>>>> There's just  the x-systemd.device-timeout but this controls how long to
>>>> wait for the device and not for the mount command.
>>>>
>>>> Is there any solution for big btrfs volumes and systemd?
>>>>
>>>> Greets,
>>>> Stefan
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>>
>>>
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
>