Re: [Samba] Fw: Btrfs Samba and Quotas

Re: [Samba] Fw: Btrfs Samba and Quotas

[Hendrik Friedel]

Hello,

I a bit surprised to get no replies at all...
How come? Lack of information? Lack of clarity?

Greetings,
Hendrik


------ Originalnachricht ------
Von: "Hendrik Friedel via samba" <samba@lists.samba.org>
An: "samba@lists.samba.org" <samba@lists.samba.org>
Gesendet: 14.05.2019 20:01:41
Betreff: [Samba] Fw: Btrfs Samba and Quotas

>Hello,
>
>by suggestion from linux-btrfs I post this to samba@lists.samba.org.
>I think, thiss is a bug in Samba. Can you confirm and suggest a workaround?
>
>Regards,
>Hendrik
>
>------ Weitergeleitete Nachricht ------
>Von: "Hendrik Friedel" <hendrik@friedels.name>
>An: "Btrfs BTRFS" <linux-btrfs@vger.kernel.org>
>Gesendet: 12.05.2019 13:27:00
>Betreff: Btrfs Samba and Quotas
>
>Hello,
>
>I was wondering, whether anyone of you has experience with this samba in conjunction with BTRFS and quotas.
>
>I am using Openmediavault (debian based NAS distribution), which is not actively supporting btrfs. It uses quotas by default, and I think, that me using btrfs is causing troubles...
>
>In the logs I find:
>May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for path [.]!
>May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.879166,  0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for path [.]!
>May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.879356,  0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for path [.]!
>May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.879688,  0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for path [Hendrik]!
>May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.879888,  0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for path [Hendrik]!
>May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.880093,  0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for path [Hendrik]!
>May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.880287,  0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for path [Hendrik]!
>
>As you can see, this is quite frequent.
>
>Searching for this, I find some bug-reports:
>https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1735953
>https://bugzilla.samba.org/show_bug.cgi?id=10541
>
>Now, I am sure that I am not the first to use Samba with btrfs. What's special about me? How's your experience?
>
>Greetings,
>Hendrik
>
>
>-- To unsubscribe from this list go to the following URL and read the
>instructions:  https://lists.samba.org/mailman/options/samba

Re: [Samba] Fw: Btrfs Samba and Quotas

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 3411 bytes --]



On 2019/5/18 下午5:22, Hendrik Friedel wrote:
> Hello,
> 
> I a bit surprised to get no replies at all...

Samba mail list needs subscription to send mail, so I'm afraid this mail
may not reach samba list if you're not subscribing.

For btrfs part, I think I have replied before, and it should not be a
problem for btrfs.

Thanks,
Qu

> How come? Lack of information? Lack of clarity?
> 
> Greetings,
> Hendrik
> 
> 
> ------ Originalnachricht ------
> Von: "Hendrik Friedel via samba" <samba@lists.samba.org>
> An: "samba@lists.samba.org" <samba@lists.samba.org>
> Gesendet: 14.05.2019 20:01:41
> Betreff: [Samba] Fw: Btrfs Samba and Quotas
> 
>> Hello,
>>
>> by suggestion from linux-btrfs I post this to samba@lists.samba.org.
>> I think, thiss is a bug in Samba. Can you confirm and suggest a
>> workaround?
>>
>> Regards,
>> Hendrik
>>
>> ------ Weitergeleitete Nachricht ------
>> Von: "Hendrik Friedel" <hendrik@friedels.name>
>> An: "Btrfs BTRFS" <linux-btrfs@vger.kernel.org>
>> Gesendet: 12.05.2019 13:27:00
>> Betreff: Btrfs Samba and Quotas
>>
>> Hello,
>>
>> I was wondering, whether anyone of you has experience with this samba
>> in conjunction with BTRFS and quotas.
>>
>> I am using Openmediavault (debian based NAS distribution), which is
>> not actively supporting btrfs. It uses quotas by default, and I think,
>> that me using btrfs is causing troubles...
>>
>> In the logs I find:
>> May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for
>> path [.]!
>> May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.879166, 
>> 0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>> May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for
>> path [.]!
>> May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.879356, 
>> 0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>> May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for
>> path [.]!
>> May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.879688, 
>> 0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>> May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for
>> path [Hendrik]!
>> May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.879888, 
>> 0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>> May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for
>> path [Hendrik]!
>> May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.880093, 
>> 0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>> May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for
>> path [Hendrik]!
>> May 12 09:34:06 homeserver smbd[4116]: [2019/05/12 09:34:06.880287, 
>> 0] ../source3/lib/sysquotas.c:461(sys_get_quota)
>> May 12 09:34:06 homeserver smbd[4116]:   sys_path_to_bdev() failed for
>> path [Hendrik]!
>>
>> As you can see, this is quite frequent.
>>
>> Searching for this, I find some bug-reports:
>> https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1735953
>> https://bugzilla.samba.org/show_bug.cgi?id=10541
>>
>> Now, I am sure that I am not the first to use Samba with btrfs. What's
>> special about me? How's your experience?
>>
>> Greetings,
>> Hendrik
>>
>>
>> -- To unsubscribe from this list go to the following URL and read the
>> instructions:  https://lists.samba.org/mailman/options/samba
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re[2]: [Samba] Fw: Btrfs Samba and Quotas

[Hendrik Friedel]

Hello

>No, probably a lack of users using your combination of Samba, btrfs and quotas.
I would have thought that btrfs is becoming more mainstream now. And 
then, Samba and Quotas should be rather common...
>

>Of course, more info may help, what is in your smb.conf etc.
Added at the end of this mail.

>Why are you using a filesystem that your OS doesn't officially support ?
That implies, that it would be better with any other OS. Are you saying, 
that this is a known samba bug, that has been fixed, but the fix has 
just not yet made it into Debian? Otherwise, I don't understand the 
background of the question.
Appart from that:
Btrfs support was introduced in DebianSqueeze 
<https://wiki.debian.org/DebianSqueeze>.

>
 > I tried brtfs several times together with samba and (sorry to say this)
 > it just a pain in the a.. Never use it together with quotas or CTDB it
 > will crash after short time. I only take xfs and have no problem at all.
 > I don't know wy, but it's not good idea to user brtfs with samba.

Well, as long as this is not being reported and being improved, it will remain that way...

 > I wonder, are your clients Linux or Windows systems?
Windows (10)

 > I wonder if upgrading servers may not be so wise.
What do you mean by that?

Regards,
Hendrik


>
>


cat /etc/samba/smb.conf
#======================= Global Settings =======================
[global]
workgroup = WORKGROUP
server string = %h server
dns proxy = no
log level = 1
log file = /var/log/samba/log.%m
max log size = 1000
logging = syslog
panic action = /usr/share/samba/panic-action %d
encrypt passwords = true
passdb backend = tdbsam
obey pam restrictions = no
unix password sync = no
passwd program = /usr/bin/passwd %u
passwd chat = *Enter\snew\s*\spassword:* %n\n 
*Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
pam password change = yes
socket options = TCP_NODELAY IPTOS_LOWDELAY
guest account = nobody
load printers = no
disable spoolss = yes
printing = bsd
printcap name = /dev/null
unix extensions = yes
wide links = no
create mask = 0777
directory mask = 0777
use sendfile = yes
aio read size = 16384
aio write size = 16384
local master = yes
time server = no
wins support = no


[Dokumente]
path = /srv/dev-disk-by-label-DataPool1/Dokumente
guest ok = no
read only = no
browseable = yes
inherit acls = yes
inherit permissions = no
ea support = no
store dos attributes = no
vfs objects =
printable = no
create mask = 0664
force create mode = 0664
directory mask = 0775
force directory mode = 0775
hide special files = yes
follow symlinks = yes
hide dot files = yes
valid users = "henfri"
invalid users =
read list =
write list = "henfri"



>

Re: [Samba] Fw: Btrfs Samba and Quotas

[Jeremy Allison]

On Mon, May 20, 2019 at 05:54:13PM +0000, Hendrik Friedel via samba wrote:
> Hello,
> 
> > Is btrfs becoming more common ?
> In my impression: Yes. Also, this problem seems to affect also zfs and thus
> all (?) file systems that support checksums and scrubbing in linux;
> consequently all filesystems that are the choice of users who need this for
> ensuring data consistency.
> > You posted this:
> > 
> > I am using Openmediavault (debian based NAS distribution), which is not actively supporting btrfs
> > 
> > It is this that I was referring to.
> Ah, yes.
> OMV intended to move to btrfs as the only choice with the next version. In
> order to pave the way, I intended to be an early adopter. The problem I
> report here, that there is good reason to.
> 
> > > > it just a pain in the a.. Never use it together with quotas or CTDB it
> > > > will crash after short time. I only take xfs and have no problem at all.
> > > > I don't know wy, but it's not good idea to user brtfs with samba.
> > > 
> > > Well, as long as this is not being reported and being improved, it will remain that way...
> > > 
> > Possibly, but it works great with ext4
> Glad to hear that.
> > I suggest you sit down with a copy of 'man smb.conf' and remove all the default
> My intent is not to solve *my* problem, but to make developers aware of this
> issue and help getting this issue fixed.
> 
> I feel a bit helpless though, as I perceive a lack of interest...
> I mean... This Bug is now celebrating its 5th aniversary.
> https://bugzilla.samba.org/show_bug.cgi?id=10541

That's because the concept of a btrfs "subvolume" completely
breaks the POSIX idioms that smbd depends on.

We absolutely identify a file by a dev/ino pair, and
expect the dev to remain consistent under an exported
share path.

If you sub-mount this also breaks smbd dfree/quotas, and
that's a lot more common.

This identity is baked into Samba in order to implement
leases/oplocks and it's not going to change.

If you want to do this (subvolumes/submounts) I think
you should get familiar with the:

dfree command:

set quota command:

set quota command:

scripts in smb.conf.

Re[2]: [Samba] Fw: Btrfs Samba and Quotas

[Hendrik Friedel]

Hello,

>If you want to do this (subvolumes/submounts) I think
>you should get familiar with the:
sorry, that's well beyond my skill-set. Honestly, I fear I cannot do any 
good solving this issue.
My intent is/was to make you aware -although frankly, for *me* this is 
no problem.

Greetings,
Hendrik


>

Re[2]: [Samba] Fw: Btrfs Samba and Quotas

[Hendrik Friedel]

Hello,

>>>You posted this:
>>>I am using Openmediavault (debian based NAS distribution), which is not actively supporting btrfs
>>>It is this that I was referring to.
>>Ah, yes.
>>OMV intended to move to btrfs as the only choice with the next version. In order to pave the way, I intended to be an early adopter. The problem I report here, that there is good reason to.
>For you perhaps, but if OMV will not help, well......
What do you mean? [there was a 'shows' missing. It shoult have read 
'shows that there is good reas on to']

>That sort of says a lot, if it was that big a problem and multiple users were complaining, then it probably would get fixed, but there are only so many developers and they have only so much time available to fix things and add new features, so the most important things get fixed first. We do, however, accept patches to fix things, so perhaps you can get together with OMV and fix it from your end ;-)
I fear, that's beyond my skills.

Well, I did my best.

Greetings,
Hendrik

Re[2]: [Samba] Fw: Btrfs Samba and Quotas

[Hendrik Friedel]

Hello,

 >> In my impression: Yes. Also, this problem seems to affect also zfs 
and
 >
 > I'm mostly interested in the claim that ZFS is affected.
 > I haven't followed this thread carefully, but what exactly is the problem we're
 > talking about, and how do we know it impacts ZFS?
 > [Something more than a single one-liner in that bug report?]

Indeed, I only find that one line. I can try to find out.

 > Is the extent of the issue that quotas won't work, while enforced from Samba
 > against a ZFS volume?
 >
 > Can someone perhaps enlighten me? :)

The explaination is:

 > That's because the concept of a btrfs "subvolume" completely
 > breaks the POSIX idioms that smbd depends on.

And wouldn't that also be applicable to zfs?

>  At least I hope you can understand why some bug reports seem to take forever to get fixed, it is all down to priorities, the highest  priority ones get fixed first,

Yes, I understand that.

 > What I was trying to say was (and failing, it would seem), this is a 
two way street
 > and if OMV cannot/will not help you, then it is hard to fix,

What is OMV specific here? Isn't the problem fully included already in linux (=kernel) and samba?

 > especially now that Jeremy has pointed out that it cannot be fixed as is. Now this
 > doesn't mean it can never be fixed, throw enough money and man hours at it
 > and a workaround can probably be found

Here, I could imagine that linking with linux-btrfs would be worthwhile.

 > but this would undoubtedly entail OMV getting involved

Why? OMV merely writes the smb.conf...

Greetings,
Hendrik

Re: Re[2]: [Samba] Fw: Btrfs Samba and Quotas

[Chris Murphy]

On Tue, May 21, 2019 at 2:01 PM Hendrik Friedel <hendrik@friedels.name> wrote:
>
> Hello,
>
>  >> In my impression: Yes. Also, this problem seems to affect also zfs
> and
>  >
>  > I'm mostly interested in the claim that ZFS is affected.
>  > I haven't followed this thread carefully, but what exactly is the problem we're
>  > talking about, and how do we know it impacts ZFS?
>  > [Something more than a single one-liner in that bug report?]
>
> Indeed, I only find that one line. I can try to find out.
>
>  > Is the extent of the issue that quotas won't work, while enforced from Samba
>  > against a ZFS volume?
>  >
>  > Can someone perhaps enlighten me? :)
>
> The explaination is:
>
>  > That's because the concept of a btrfs "subvolume" completely
>  > breaks the POSIX idioms that smbd depends on.
>
> And wouldn't that also be applicable to zfs?
>
> >  At least I hope you can understand why some bug reports seem to take forever to get fixed, it is all down to priorities, the highest  priority ones get fixed first,
>
> Yes, I understand that.
>
>  > What I was trying to say was (and failing, it would seem), this is a
> two way street
>  > and if OMV cannot/will not help you, then it is hard to fix,
>
> What is OMV specific here? Isn't the problem fully included already in linux (=kernel) and samba?
>
>  > especially now that Jeremy has pointed out that it cannot be fixed as is. Now this
>  > doesn't mean it can never be fixed, throw enough money and man hours at it
>  > and a workaround can probably be found
>
> Here, I could imagine that linking with linux-btrfs would be worthwhile.
>
>  > but this would undoubtedly entail OMV getting involved
>
> Why? OMV merely writes the smb.conf...

If project C wants to use storage technology A and B together, then C
is best off helping upstreams in order to make downstream integration
easier. I don't think it's anything beyond that.

Anyway, I use Btrfs and Samba together just fine. But I mount a
particular subvolume to a mountpoint and then Samba shares that
mountpoint. I do not have any subvolumes nested within the shared
subvolume. That might explain why I'm not affected as Samba only sees
one set of inodes, no duplicates, per mount.

From the same Btrfs volume, I do share other subvolumes, and therefore
there's a repeat of inodes, but they're each in their own
mountpoints+shares. So far I've seen no evidence of Samba confusion.
But I also don't use quotas.


-- 
Chris Murphy