Can BTRFS handle XATTRs larger than 4K?

Can BTRFS handle XATTRs larger than 4K?

[Richard Sharpe]

Hi folks,

I need a Linux file system that supports XATTRs up to 64K.

Can BTRFS support that or is XFS the only Linux file system with such support?

-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)

Re: Can BTRFS handle XATTRs larger than 4K?

[Chris Murphy]

On Fri, Dec 19, 2014 at 7:07 PM, Richard Sharpe
<realrichardsharpe@gmail.com> wrote:
> Hi folks,
>
> I need a Linux file system that supports XATTRs up to 64K.
>
> Can BTRFS support that or is XFS the only Linux file system with such support?

I'm feeling too lazy to actually go look for evidence of this; but I
made a file system a while ago with 64KB node/leafsize and ran into a
bug I reported for send/receive. I'm pretty sure this was related to
xattr being limited to nodesize, and I'm also pretty sure it got
fixed. But I'm not sure whether the problem was on the send or receive
side, and I'm not sure when.



-- 
Chris Murphy

Re: Can BTRFS handle XATTRs larger than 4K?

[Chris Samuel]

On Sat, 20 Dec 2014 01:38:43 AM Chris Murphy wrote:

> I'm feeling too lazy to actually go look for evidence of this; but I
> made a file system a while ago with 64KB node/leafsize and ran into a
> bug I reported for send/receive. I'm pretty sure this was related to
> xattr being limited to nodesize, and I'm also pretty sure it got
> fixed. But I'm not sure whether the problem was on the send or receive
> side, and I'm not sure when.

I suspect it's this commit and it implies that it's limited to the leafsize:

commit 4395e0c4da486f007dcb45b0336427be7ec08ab1
Author: Filipe Manana <fdmanana@suse.com>
Date:   Wed Aug 20 10:45:45 2014 +0100

    Btrfs: send, lower mem requirements for processing xattrs
    
    Maximum xattr size can be up to nearly the leaf size. For an fs with a
    leaf size larger than the page size, using kmalloc requires allocating
    multiple pages that are contiguous, which might not be possible if
    there's heavy memory fragmentation. Therefore fallback to vmalloc if
    we fail to allocate with kmalloc. Also start with a smaller buffer size,
    since xattr values typically are smaller than a page.
    
    Reported-by: Chris Murphy <lists@colorremedies.com>
    Signed-off-by: Filipe Manana <fdmanana@suse.com>
    Signed-off-by: Chris Mason <clm@fb.com>


-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

Re: Can BTRFS handle XATTRs larger than 4K?

[Chris Samuel]

On Mon, 22 Dec 2014 10:38:04 PM Chris Samuel wrote:

> I suspect it's this commit and it implies that it's limited to the leafsize:

Forgot to mention that looks like it should be in 3.17 and later.

-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

Re: Can BTRFS handle XATTRs larger than 4K?

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 1052 bytes --]

On 2014-12-19 21:07, Richard Sharpe wrote:
> Hi folks,
>
> I need a Linux file system that supports XATTRs up to 64K.
>
> Can BTRFS support that or is XFS the only Linux file system with such support?
>
At the moment, BTRFS is limited to xattrs that fit inline in the 
metadata nodes (so ~3900 bytes for a 4k leafsize).

XFS, however, isn't the only Linux filesystem that supports xattrs that 
size.  Assuming that you are using a recent kernel, you can also use 
such xattrs on at least:
  * XFS
  * JFS
  * ext4
  * reiserfs (I think, not 100% certain about this one though)
  * OCFS2 (even though it is technically a cluster fs, it can be run 
single node without the clustering)
  * ZFS (IIRC, ZFS supports unlimited xattr size)
  * NTFS (no limit on xattr size, though you should use NTFS-3G instead 
of the in-kernel driver)
  * SquashFS (read-only)
  * HFS+ (Also no limit on xattr size)
Of these, I'd personally suggest using XFS unless you need to be able to 
shrink the filesystem, in which case I'd suggest ext4.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2455 bytes --]

Re: Can BTRFS handle XATTRs larger than 4K?

[Richard Sharpe]

On Mon, Dec 22, 2014 at 6:28 AM, Austin S Hemmelgarn
<ahferroin7@gmail.com> wrote:
> On 2014-12-19 21:07, Richard Sharpe wrote:
>>
>> Hi folks,
>>
>> I need a Linux file system that supports XATTRs up to 64K.
>>
>> Can BTRFS support that or is XFS the only Linux file system with such
>> support?
>>
> At the moment, BTRFS is limited to xattrs that fit inline in the metadata
> nodes (so ~3900 bytes for a 4k leafsize).
>
> XFS, however, isn't the only Linux filesystem that supports xattrs that
> size.  Assuming that you are using a recent kernel, you can also use such
> xattrs on at least:
>  * XFS
>  * JFS
>  * ext4
>  * reiserfs (I think, not 100% certain about this one though)
>  * OCFS2 (even though it is technically a cluster fs, it can be run single
> node without the clustering)
>  * ZFS (IIRC, ZFS supports unlimited xattr size)
>  * NTFS (no limit on xattr size, though you should use NTFS-3G instead of
> the in-kernel driver)
>  * SquashFS (read-only)
>  * HFS+ (Also no limit on xattr size)
> Of these, I'd personally suggest using XFS unless you need to be able to
> shrink the filesystem, in which case I'd suggest ext4.

Thanks for the info. I hadn't realized that ext4 had lifted the restriction.

-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)

Re: Can BTRFS handle XATTRs larger than 4K?

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 1740 bytes --]

On 2014-12-22 12:27, Richard Sharpe wrote:
> On Mon, Dec 22, 2014 at 6:28 AM, Austin S Hemmelgarn
> <ahferroin7@gmail.com> wrote:
>> On 2014-12-19 21:07, Richard Sharpe wrote:
>>>
>>> Hi folks,
>>>
>>> I need a Linux file system that supports XATTRs up to 64K.
>>>
>>> Can BTRFS support that or is XFS the only Linux file system with such
>>> support?
>>>
>> At the moment, BTRFS is limited to xattrs that fit inline in the metadata
>> nodes (so ~3900 bytes for a 4k leafsize).
>>
>> XFS, however, isn't the only Linux filesystem that supports xattrs that
>> size.  Assuming that you are using a recent kernel, you can also use such
>> xattrs on at least:
>>   * XFS
>>   * JFS
>>   * ext4
>>   * reiserfs (I think, not 100% certain about this one though)
>>   * OCFS2 (even though it is technically a cluster fs, it can be run single
>> node without the clustering)
>>   * ZFS (IIRC, ZFS supports unlimited xattr size)
>>   * NTFS (no limit on xattr size, though you should use NTFS-3G instead of
>> the in-kernel driver)
>>   * SquashFS (read-only)
>>   * HFS+ (Also no limit on xattr size)
>> Of these, I'd personally suggest using XFS unless you need to be able to
>> shrink the filesystem, in which case I'd suggest ext4.
>
> Thanks for the info. I hadn't realized that ext4 had lifted the restriction.
>
Yeah, it would be nice if there was more clarity in the documentation.

Personally, I'd love to see unlimited length xattr's like NTFS and HFS+ 
do, as that would greatly improve interoperability (both Windows and OS 
X use xattrs, although they call them 'alternative data streams' and 
'forks' respectively), and provide a higher likelihood that xattrs would 
start getting used more.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2455 bytes --]

Re: Can BTRFS handle XATTRs larger than 4K?

[Chris Murphy]

On Mon, Dec 22, 2014 at 11:09 AM, Austin S Hemmelgarn
<ahferroin7@gmail.com> wrote:

> Personally, I'd love to see unlimited length xattr's like NTFS and HFS+ do,
> as that would greatly improve interoperability (both Windows and OS X use
> xattrs, although they call them 'alternative data streams' and 'forks'
> respectively), and provide a higher likelihood that xattrs would start
> getting used more.

This is two years old, but it looks like NFS will not support xattr.
http://comments.gmane.org/gmane.linux.nfs/53259

It looks like SMB does support xattr (and sometimes requires it) but I
have no idea to what degree, including the host/client preservation on
different filesystems. [1] It would still be helpful for cp and rsync
to be able to preserve xattr, however Apple has moved to a new on-disk
format that makes the future of reading OS X volumes on Linux an open
question. [2]




[1]e.g. Btrfs > Samba- server > SMB over TCP > "Apple SMB2" > HFS+.
And then the OS X client pushing a file to the server is a separate
test. Next test would be OS X as server and Linux as client to do HFS+
> "Apple SMB2" > SMB over TCP > Samba-client > Btrfs, and then Linux
client pushing a file to OS X is a separate test. So it's four tests
for any combination of filesystems.

[2] Apple has a logical volume manager called CoreStorage. Until
recently it's mainly used to implement full disk (volume) encryption,
but encryption is actually optional. It's also used to combine SSD+HDD
partitions into a single logical volume using the SSD as a cache.
Starting with 10.10 "Yosemite" it's used by default for the main
HFSJ/X volume for system/apps/user data, and even legacy OS X only
installations are converted to a CoreStorage logical volume upon
upgrading. There's no pre-baked support on linux for this right now,
and I'm not really sure if/when we'd ever see this in a distribution
by default.
http://www.cl.cam.ac.uk/~osc22/docs/cl_fv2_presentation_2012.pdf
https://github.com/libyal/libfvde/


-- 
Chris Murphy

Re: Can BTRFS handle XATTRs larger than 4K?

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 2740 bytes --]

On 2014-12-22 13:43, Chris Murphy wrote:
> On Mon, Dec 22, 2014 at 11:09 AM, Austin S Hemmelgarn
> <ahferroin7@gmail.com> wrote:
>
>> Personally, I'd love to see unlimited length xattr's like NTFS and HFS+ do,
>> as that would greatly improve interoperability (both Windows and OS X use
>> xattrs, although they call them 'alternative data streams' and 'forks'
>> respectively), and provide a higher likelihood that xattrs would start
>> getting used more.
>
> This is two years old, but it looks like NFS will not support xattr.
> http://comments.gmane.org/gmane.linux.nfs/53259
>
> It looks like SMB does support xattr (and sometimes requires it) but I
> have no idea to what degree, including the host/client preservation on
> different filesystems. [1] It would still be helpful for cp and rsync
> to be able to preserve xattr, however Apple has moved to a new on-disk
> format that makes the future of reading OS X volumes on Linux an open
> question. [2]
>
>
>
>
> [1]e.g. Btrfs > Samba- server > SMB over TCP > "Apple SMB2" > HFS+.
> And then the OS X client pushing a file to the server is a separate
> test. Next test would be OS X as server and Linux as client to do HFS+
>> "Apple SMB2" > SMB over TCP > Samba-client > Btrfs, and then Linux
> client pushing a file to OS X is a separate test. So it's four tests
> for any combination of filesystems.
>
> [2] Apple has a logical volume manager called CoreStorage. Until
> recently it's mainly used to implement full disk (volume) encryption,
> but encryption is actually optional. It's also used to combine SSD+HDD
> partitions into a single logical volume using the SSD as a cache.
> Starting with 10.10 "Yosemite" it's used by default for the main
> HFSJ/X volume for system/apps/user data, and even legacy OS X only
> installations are converted to a CoreStorage logical volume upon
> upgrading. There's no pre-baked support on linux for this right now,
> and I'm not really sure if/when we'd ever see this in a distribution
> by default.
> http://www.cl.cam.ac.uk/~osc22/docs/cl_fv2_presentation_2012.pdf
> https://github.com/libyal/libfvde/
>
>
I'm more thinking along the lines of not having to jump through hoops to 
get _ALL_ the data in a tar file from OS X to extract on a Linux box. 
Also, Windows has been using it's 'alternative data streams' 
functionality from NTFS more in recent years (the new 'file history' 
functionality in Win8/8.1 uses ADS for storing old copies of files), and 
these are essentially just forks with a extra compatibility interface. 
There are other potentially interesting uses though, for example, 
storing multiple localized versions of a text file as a single 
user-visible 'file'.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2455 bytes --]

Re: Can BTRFS handle XATTRs larger than 4K?

[Richard Sharpe]

On Mon, Dec 22, 2014 at 10:43 AM, Chris Murphy <lists@colorremedies.com> wrote:
> On Mon, Dec 22, 2014 at 11:09 AM, Austin S Hemmelgarn
> <ahferroin7@gmail.com> wrote:
>
>> Personally, I'd love to see unlimited length xattr's like NTFS and HFS+ do,
>> as that would greatly improve interoperability (both Windows and OS X use
>> xattrs, although they call them 'alternative data streams' and 'forks'
>> respectively), and provide a higher likelihood that xattrs would start
>> getting used more.
>
> This is two years old, but it looks like NFS will not support xattr.
> http://comments.gmane.org/gmane.linux.nfs/53259
>
> It looks like SMB does support xattr (and sometimes requires it) but I
> have no idea to what degree, including the host/client preservation on
> different filesystems. [1] It would still be helpful for cp and rsync
> to be able to preserve xattr, however Apple has moved to a new on-disk
> format that makes the future of reading OS X volumes on Linux an open
> question. [2]

Those are the old OS 2 XATTRs, better known as EAs, and NTFS says that
you can support EAs or you can have Reparse Points, but not both
(basically, they re-used the EA Length field as the reparse tag).
Also, Windows (of any flavor) does not make it easy to access EAs.

-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)

Re: Can BTRFS handle XATTRs larger than 4K?

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 2428 bytes --]

On 2014-12-22 15:06, Richard Sharpe wrote:
> On Mon, Dec 22, 2014 at 10:43 AM, Chris Murphy <lists@colorremedies.com> wrote:
>> On Mon, Dec 22, 2014 at 11:09 AM, Austin S Hemmelgarn
>> <ahferroin7@gmail.com> wrote:
>>
>>> Personally, I'd love to see unlimited length xattr's like NTFS and HFS+ do,
>>> as that would greatly improve interoperability (both Windows and OS X use
>>> xattrs, although they call them 'alternative data streams' and 'forks'
>>> respectively), and provide a higher likelihood that xattrs would start
>>> getting used more.
>>
>> This is two years old, but it looks like NFS will not support xattr.
>> http://comments.gmane.org/gmane.linux.nfs/53259
>>
>> It looks like SMB does support xattr (and sometimes requires it) but I
>> have no idea to what degree, including the host/client preservation on
>> different filesystems. [1] It would still be helpful for cp and rsync
>> to be able to preserve xattr, however Apple has moved to a new on-disk
>> format that makes the future of reading OS X volumes on Linux an open
>> question. [2]
>
> Those are the old OS 2 XATTRs, better known as EAs, and NTFS says that
> you can support EAs or you can have Reparse Points, but not both
> (basically, they re-used the EA Length field as the reparse tag).
> Also, Windows (of any flavor) does not make it easy to access EAs.
>
But OS/2 style XATTRS are not the same as NTFS Alternative Data Streams, 
which technically (because of Windows backward compatibility interfaces) 
don't need a huge amount of support from SMB.  They were originally 
added to support SFM in NT3.1, so that windows could store resource 
forks.  The two primary uses on windows today are the file history 
interface in Win8/8.1 and the 'zone_identifier' saved with downloads by 
most modern browsers.  They're actually pretty easy to get to, you just 
append the ADS name to the end of the filename with a : separating them, 
and you can access it like a regular file (which is part of why : isn't 
a legal character in a windows filename).  Most people don't know about 
them because they don't get listed in windows explorer, even with hidden 
files and protected OS file visible.  The actual on-disk format for them 
is actually kind of interesting, the file data itself (what in the Apple 
world is called the Data Fork) is actually stored as an unnamed ADS 
associated with the filename.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2455 bytes --]

Re: Can BTRFS handle XATTRs larger than 4K?

[Richard Sharpe]

On Mon, Dec 22, 2014 at 12:44 PM, Austin S Hemmelgarn
<ahferroin7@gmail.com> wrote:
> On 2014-12-22 15:06, Richard Sharpe wrote:
>>
>> On Mon, Dec 22, 2014 at 10:43 AM, Chris Murphy <lists@colorremedies.com>
>> wrote:
>>>
>>> On Mon, Dec 22, 2014 at 11:09 AM, Austin S Hemmelgarn
>>> <ahferroin7@gmail.com> wrote:
>>>
>>>> Personally, I'd love to see unlimited length xattr's like NTFS and HFS+
>>>> do,
>>>> as that would greatly improve interoperability (both Windows and OS X
>>>> use
>>>> xattrs, although they call them 'alternative data streams' and 'forks'
>>>> respectively), and provide a higher likelihood that xattrs would start
>>>> getting used more.
>>>
>>>
>>> This is two years old, but it looks like NFS will not support xattr.
>>> http://comments.gmane.org/gmane.linux.nfs/53259
>>>
>>> It looks like SMB does support xattr (and sometimes requires it) but I
>>> have no idea to what degree, including the host/client preservation on
>>> different filesystems. [1] It would still be helpful for cp and rsync
>>> to be able to preserve xattr, however Apple has moved to a new on-disk
>>> format that makes the future of reading OS X volumes on Linux an open
>>> question. [2]
>>
>>
>> Those are the old OS 2 XATTRs, better known as EAs, and NTFS says that
>> you can support EAs or you can have Reparse Points, but not both
>> (basically, they re-used the EA Length field as the reparse tag).
>> Also, Windows (of any flavor) does not make it easy to access EAs.
>>
> But OS/2 style XATTRS are not the same as NTFS Alternative Data Streams,

I agree with you and was not trying to claim that EAs are ADSs.
Indeed, they have worse functionality than unix XATTRs, IMO.

> which technically (because of Windows backward compatibility interfaces)
> don't need a huge amount of support from SMB.  They were originally added to
> support SFM in NT3.1, so that windows could store resource forks.  The two
> primary uses on windows today are the file history interface in Win8/8.1 and
> the 'zone_identifier' saved with downloads by most modern browsers.  They're
> actually pretty easy to get to, you just append the ADS name to the end of
> the filename with a : separating them, and you can access it like a regular
> file (which is part of why : isn't a legal character in a windows filename).
> Most people don't know about them because they don't get listed in windows
> explorer, even with hidden files and protected OS file visible.  The actual
> on-disk format for them is actually kind of interesting, the file data
> itself (what in the Apple world is called the Data Fork) is actually stored
> as an unnamed ADS associated with the filename.
>

-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)

Re: Can BTRFS handle XATTRs larger than 4K?

[Robert White]

On 12/22/2014 12:44 PM, Austin S Hemmelgarn wrote:
> On 2014-12-22 15:06, Richard Sharpe wrote:
>> On Mon, Dec 22, 2014 at 10:43 AM, Chris Murphy
>> <lists@colorremedies.com> wrote:
>>> On Mon, Dec 22, 2014 at 11:09 AM, Austin S Hemmelgarn
>>> <ahferroin7@gmail.com> wrote:
>>>
>>>> Personally, I'd love to see unlimited length xattr's like NTFS and
>>>> HFS+ do,
>>>> as that would greatly improve interoperability (both Windows and OS
>>>> X use
>>>> xattrs, although they call them 'alternative data streams' and 'forks'
>>>> respectively), and provide a higher likelihood that xattrs would start
>>>> getting used more.
>>>
>>> This is two years old, but it looks like NFS will not support xattr.
>>> http://comments.gmane.org/gmane.linux.nfs/53259
>>>
>>> It looks like SMB does support xattr (and sometimes requires it) but I
>>> have no idea to what degree, including the host/client preservation on
>>> different filesystems. [1] It would still be helpful for cp and rsync
>>> to be able to preserve xattr, however Apple has moved to a new on-disk
>>> format that makes the future of reading OS X volumes on Linux an open
>>> question. [2]
>>
>> Those are the old OS 2 XATTRs, better known as EAs, and NTFS says that
>> you can support EAs or you can have Reparse Points, but not both
>> (basically, they re-used the EA Length field as the reparse tag).
>> Also, Windows (of any flavor) does not make it easy to access EAs.
>>
> But OS/2 style XATTRS are not the same as NTFS Alternative Data Streams,
> which technically (because of Windows backward compatibility interfaces)
> don't need a huge amount of support from SMB.  They were originally
> added to support SFM in NT3.1, so that windows could store resource
> forks.  The two primary uses on windows today are the file history
> interface in Win8/8.1 and the 'zone_identifier' saved with downloads by
> most modern browsers.  They're actually pretty easy to get to, you just
> append the ADS name to the end of the filename with a : separating them,
> and you can access it like a regular file (which is part of why : isn't
> a legal character in a windows filename).  Most people don't know about
> them because they don't get listed in windows explorer, even with hidden
> files and protected OS file visible.  The actual on-disk format for them
> is actually kind of interesting, the file data itself (what in the Apple
> world is called the Data Fork) is actually stored as an unnamed ADS
> associated with the filename.
>

My stupid two cents...

Wouldn't keeping a file history be "better" done with something git-like 
(monotonish? 8-) combined with an incron type file-watcher?

So like a small xattr to link the file to the repository or something...

setfattr --name=user.history_repo --value=/path/to/repository file

and some not-in-the-kernel subsystems?

What is the practical use case for really large XATTRS that isn't solved 
by indirection to non-kernel facilities.

(That's not snark, I've been trying to figure out why _I_ would want 
that expanse of auxiliary data so inconveniently stored and I've come up 
with nothing. Maybe I lack imagination.)

I see the ADS thing now that you mention it. Kinda neat way to recycle 
the otherwise much-disparaged colon-is-for-devices thing. But would that 
sort of thing match the Linux/POSIX expectation at all? Everything in 
*ix being a file, the chaos of expectation from the equivalent 
/dev/sda1:ADS (to make it a portmantu of sorts) becomes unfriendly.


I've thought of some interesting things to do whit XATTRS (like a kernel 
patch to let an executable carry around environment overrides like a 
restricted/overridden PATH) or include the intended editor for use in a 
GUI/file-manager. I'm pretty sure I could get into the 32K range with 
that stuff. But "unbounded" seems a little high. 8-)

So skipping the full ADS, what's the current demand/payoff for large 
XATTR space?

Re: Can BTRFS handle XATTRs larger than 4K?

[Richard Sharpe]

On Mon, Dec 22, 2014 at 2:52 PM, Robert White <rwhite@pobox.com> wrote:
>
> On 12/22/2014 12:44 PM, Austin S Hemmelgarn wrote:
>>
>> On 2014-12-22 15:06, Richard Sharpe wrote:
>>>
>>> On Mon, Dec 22, 2014 at 10:43 AM, Chris Murphy
>>> <lists@colorremedies.com> wrote:
>>>>
>>>> On Mon, Dec 22, 2014 at 11:09 AM, Austin S Hemmelgarn
>>>> <ahferroin7@gmail.com> wrote:
>>>>
>>>>> Personally, I'd love to see unlimited length xattr's like NTFS and
>>>>> HFS+ do,
>>>>> as that would greatly improve interoperability (both Windows and OS
>>>>> X use
>>>>> xattrs, although they call them 'alternative data streams' and 'forks'
>>>>> respectively), and provide a higher likelihood that xattrs would start
>>>>> getting used more.
>>>>
>>>>
>>>> This is two years old, but it looks like NFS will not support xattr.
>>>> http://comments.gmane.org/gmane.linux.nfs/53259
>>>>
>>>> It looks like SMB does support xattr (and sometimes requires it) but I
>>>> have no idea to what degree, including the host/client preservation on
>>>> different filesystems. [1] It would still be helpful for cp and rsync
>>>> to be able to preserve xattr, however Apple has moved to a new on-disk
>>>> format that makes the future of reading OS X volumes on Linux an open
>>>> question. [2]
>>>
>>>
>>> Those are the old OS 2 XATTRs, better known as EAs, and NTFS says that
>>> you can support EAs or you can have Reparse Points, but not both
>>> (basically, they re-used the EA Length field as the reparse tag).
>>> Also, Windows (of any flavor) does not make it easy to access EAs.
>>>
>> But OS/2 style XATTRS are not the same as NTFS Alternative Data Streams,
>> which technically (because of Windows backward compatibility interfaces)
>> don't need a huge amount of support from SMB.  They were originally
>> added to support SFM in NT3.1, so that windows could store resource
>> forks.  The two primary uses on windows today are the file history
>> interface in Win8/8.1 and the 'zone_identifier' saved with downloads by
>> most modern browsers.  They're actually pretty easy to get to, you just
>> append the ADS name to the end of the filename with a : separating them,
>> and you can access it like a regular file (which is part of why : isn't
>> a legal character in a windows filename).  Most people don't know about
>> them because they don't get listed in windows explorer, even with hidden
>> files and protected OS file visible.  The actual on-disk format for them
>> is actually kind of interesting, the file data itself (what in the Apple
>> world is called the Data Fork) is actually stored as an unnamed ADS
>> associated with the filename.
>>
>
> My stupid two cents...
>
> Wouldn't keeping a file history be "better" done with something git-like (monotonish? 8-) combined with an incron type file-watcher?
>
> So like a small xattr to link the file to the repository or something...
>
> setfattr --name=user.history_repo --value=/path/to/repository file
>
> and some not-in-the-kernel subsystems?
>
> What is the practical use case for really large XATTRS that isn't solved by indirection to non-kernel facilities.
>
> (That's not snark, I've been trying to figure out why _I_ would want that expanse of auxiliary data so inconveniently stored and I've come up with nothing. Maybe I lack imagination.)
>
> I see the ADS thing now that you mention it. Kinda neat way to recycle the otherwise much-disparaged colon-is-for-devices thing. But would that sort of thing match the Linux/POSIX expectation at all? Everything in *ix being a file, the chaos of expectation from the equivalent /dev/sda1:ADS (to make it a portmantu of sorts) becomes unfriendly.
>
>
> I've thought of some interesting things to do whit XATTRS (like a kernel patch to let an executable carry around environment overrides like a restricted/overridden PATH) or include the intended editor for use in a GUI/file-manager. I'm pretty sure I could get into the 32K range with that stuff. But "unbounded" seems a little high. 8-)
>
> So skipping the full ADS, what's the current demand/payoff for large XATTR space?

Windows Security Descriptors (sometimes incorrectly called ACLs)
stored by Samba.

-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)

Re: Can BTRFS handle XATTRs larger than 4K?

[Robert White]

On 12/22/2014 02:55 PM, Richard Sharpe wrote:
> On Mon, Dec 22, 2014 at 2:52 PM, Robert White <rwhite@pobox.com> wrote:
>> So skipping the full ADS, what's the current demand/payoff for large XATTR space?
>
> Windows Security Descriptors (sometimes incorrectly called ACLs)
> stored by Samba.

Ah.

I know that Linux ACLs are fairly small per entry, I take it Windows' 
can be much bigger?

Having just gone off an read a lot about the many ADS possible in 
windows, they've sort of treated ever file as if it were the name of a 
phantom directory limited depth... That is you seem to be able to create 
any name as a stream name but you can't create any pathname as same.

The system-level API -- that is the complete retooling of SYS_open et al 
-- and the requsite departure from POSIX -- seems unlikely.

On the antipode, it seems like being able to put an inode reference key 
type (e.g. a name,inode pair as one of the metadata entries) could 
relieve the space constraint for a limited number of entires. The 
contents of that inode's data region would become the value of the 
single attribute.

Does that relieve Security Descriptor burden? Is each descriptor a 
separate attribute or are all the descriptors held in one attribute as a 
list-of?

Going full "phantom directory" to match Windows just seems like we get 
into the business of replacing whole kernel tidbits a la the 
inner-system effect.

Re: Can BTRFS handle XATTRs larger than 4K?

[Richard Sharpe]

On Mon, Dec 22, 2014 at 4:08 PM, Robert White <rwhite@pobox.com> wrote:
>
> On 12/22/2014 02:55 PM, Richard Sharpe wrote:
>>
>> On Mon, Dec 22, 2014 at 2:52 PM, Robert White <rwhite@pobox.com> wrote:
>>>
>>> So skipping the full ADS, what's the current demand/payoff for large XATTR space?
>>
>>
>> Windows Security Descriptors (sometimes incorrectly called ACLs)
>> stored by Samba.
>
>
> Ah.
>
> I know that Linux ACLs are fairly small per entry, I take it Windows' can be
> much bigger?

An NTFS SD can be up to 64kiB in size. That allows for up to about
1,800 ACEs, although I have not seen that many. However, I have seen
some ACLs with several hundred ACEs. ACEs are around 30 bytes in
length (S-1-5-21-x-y-z-rid, plus perms etc)

--
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)

Re: Can BTRFS handle XATTRs larger than 4K?

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 2531 bytes --]

On 2014-12-22 19:08, Robert White wrote:
> On 12/22/2014 02:55 PM, Richard Sharpe wrote:
>> On Mon, Dec 22, 2014 at 2:52 PM, Robert White <rwhite@pobox.com> wrote:
>>> So skipping the full ADS, what's the current demand/payoff for large
>>> XATTR space?
>>
>> Windows Security Descriptors (sometimes incorrectly called ACLs)
>> stored by Samba.
>
> Ah.
>
> I know that Linux ACLs are fairly small per entry, I take it Windows'
> can be much bigger?
>
> Having just gone off an read a lot about the many ADS possible in
> windows, they've sort of treated ever file as if it were the name of a
> phantom directory limited depth... That is you seem to be able to create
> any name as a stream name but you can't create any pathname as same.
>
> The system-level API -- that is the complete retooling of SYS_open et al
> -- and the requsite departure from POSIX -- seems unlikely.
>
> On the antipode, it seems like being able to put an inode reference key
> type (e.g. a name,inode pair as one of the metadata entries) could
> relieve the space constraint for a limited number of entires. The
> contents of that inode's data region would become the value of the
> single attribute.
>
> Does that relieve Security Descriptor burden? Is each descriptor a
> separate attribute or are all the descriptors held in one attribute as a
> list-of?
>
> Going full "phantom directory" to match Windows just seems like we get
> into the business of replacing whole kernel tidbits a la the
> inner-system effect.
>

Personally, I'm thinking something more like the OS X Forks API than the 
Windows ADS API.  Forks are actually the original reason that windows 
added ADS to NTFS, and their API is much cleaner than the windows one. 
The standard there is that forks are accessible as 
'<filename>/..namedfork/<forkname>'  ..namedfork doesn't get listed by 
ls -a, but you can explicitly list '<filename>/..namedfork/' to get a 
list of forks other than the data fork.  Such an API needs almost zero 
modification to existing programs (just enough to get tar and friends to 
check if there are any named forks for each file).

The significant advantages I see to having forks are:
  1. Windows Security Descriptors
  2. Macintosh Resource Forks
  3. BeOS (and others) style associated metadata (ie, associated file 
icons/thumbnails, per-file program associations, in other words, the 
same sort of stuff that OSX uses resource forks for)
  4. Better flexibility for stuff like IMA and EVM



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2455 bytes --]

Re: Can BTRFS handle XATTRs larger than 4K?

[ronnie sahlberg]

On Mon, Dec 22, 2014 at 2:52 PM, Robert White <rwhite@pobox.com> wrote:
> On 12/22/2014 12:44 PM, Austin S Hemmelgarn wrote:
>>
>> On 2014-12-22 15:06, Richard Sharpe wrote:
>>>
>>> On Mon, Dec 22, 2014 at 10:43 AM, Chris Murphy
>>> <lists@colorremedies.com> wrote:
>>>>
>>>> On Mon, Dec 22, 2014 at 11:09 AM, Austin S Hemmelgarn
>>>> <ahferroin7@gmail.com> wrote:
>>>>
>>>>> Personally, I'd love to see unlimited length xattr's like NTFS and
>>>>> HFS+ do,
>>>>> as that would greatly improve interoperability (both Windows and OS
>>>>> X use
>>>>> xattrs, although they call them 'alternative data streams' and 'forks'
>>>>> respectively), and provide a higher likelihood that xattrs would start
>>>>> getting used more.
>>>>
>>>>
>>>> This is two years old, but it looks like NFS will not support xattr.
>>>> http://comments.gmane.org/gmane.linux.nfs/53259
>>>>
>>>> It looks like SMB does support xattr (and sometimes requires it) but I
>>>> have no idea to what degree, including the host/client preservation on
>>>> different filesystems. [1] It would still be helpful for cp and rsync
>>>> to be able to preserve xattr, however Apple has moved to a new on-disk
>>>> format that makes the future of reading OS X volumes on Linux an open
>>>> question. [2]
>>>
>>>
>>> Those are the old OS 2 XATTRs, better known as EAs, and NTFS says that
>>> you can support EAs or you can have Reparse Points, but not both
>>> (basically, they re-used the EA Length field as the reparse tag).
>>> Also, Windows (of any flavor) does not make it easy to access EAs.
>>>
>> But OS/2 style XATTRS are not the same as NTFS Alternative Data Streams,
>> which technically (because of Windows backward compatibility interfaces)
>> don't need a huge amount of support from SMB.  They were originally
>> added to support SFM in NT3.1, so that windows could store resource
>> forks.  The two primary uses on windows today are the file history
>> interface in Win8/8.1 and the 'zone_identifier' saved with downloads by
>> most modern browsers.  They're actually pretty easy to get to, you just
>> append the ADS name to the end of the filename with a : separating them,
>> and you can access it like a regular file (which is part of why : isn't
>> a legal character in a windows filename).  Most people don't know about
>> them because they don't get listed in windows explorer, even with hidden
>> files and protected OS file visible.  The actual on-disk format for them
>> is actually kind of interesting, the file data itself (what in the Apple
>> world is called the Data Fork) is actually stored as an unnamed ADS
>> associated with the filename.
>>
>
> My stupid two cents...
>
> Wouldn't keeping a file history be "better" done with something git-like
> (monotonish? 8-) combined with an incron type file-watcher?
>
> So like a small xattr to link the file to the repository or something...
>
> setfattr --name=user.history_repo --value=/path/to/repository file
>
> and some not-in-the-kernel subsystems?

Atomicity.
NTFS/ADS snapshots are created atomically.

There is of course nothing that prevents the "separate file" +
setfattr "the path to the file"
from working, but it is inconvenient and somewhat ugly since you have
to implement your own
transaction and rollback mechanism every time :-(

What about if someone else modifies/renames/deletes '/path/to/repository file'
Can you prevent that? If not, how do you detect this and handle it
when 'user.history_repo'
no longer points to the right data?
You have to write code to handle it somehow.

It is a lot more convenient when this is a first class filesystem api
since you don't have to deal with it. The guy writing the filesystem
dealt with it for you.



Second really big benefit with the ADS approach is that once it is a
first class API
you don't need to rewrite all third party apps to understand the
mapping 'user.history_repo' => '/path/to/repository file'
If it is a standard filesystem feature, then you just update 'tar',
'rsync', 'scp', 'cp', 'mv', ...
once and it will work. You don;t need to re-implement this API in
these tools everytime
a new mapping is invented.

I.e. by having it as a standard API for the filesystem, you update the
external tools once and it will work reliably always,a cross all
filesystems (that support it).



>
> What is the practical use case for really large XATTRS that isn't solved by
> indirection to non-kernel facilities.

metadata
and
snapshots

>
> (That's not snark, I've been trying to figure out why _I_ would want that
> expanse of auxiliary data so inconveniently stored and I've come up with
> nothing. Maybe I lack imagination.)
>
> I see the ADS thing now that you mention it. Kinda neat way to recycle the
> otherwise much-disparaged colon-is-for-devices thing. But would that sort of
> thing match the Linux/POSIX expectation at all? Everything in *ix being a
> file, the chaos of expectation from the equivalent /dev/sda1:ADS (to make it
> a portmantu of sorts) becomes unfriendly.
>
>
> I've thought of some interesting things to do whit XATTRS (like a kernel
> patch to let an executable carry around environment overrides like a
> restricted/overridden PATH) or include the intended editor for use in a
> GUI/file-manager. I'm pretty sure I could get into the 32K range with that
> stuff. But "unbounded" seems a little high. 8-)
>
> So skipping the full ADS, what's the current demand/payoff for large XATTR
> space?
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Can BTRFS handle XATTRs larger than 4K?

[Robert White]

So I'll ask again...

On 12/22/2014 03:15 PM, ronnie sahlberg wrote:
>On Mon, Dec 22, 2014 at 2:52 PM, Robert White <rwhite@pobox.com> wrote:
>> So skipping the full ADS, what's the current demand/payoff for large XATTR
>> space?

^^^^^^^^^^^^

So skipping the full ADS, what is the current demand/payoff for large 
XATTR space?

Re: Can BTRFS handle XATTRs larger than 4K?

[Richard Sharpe]

On Mon, Dec 22, 2014 at 3:55 PM, Robert White <rwhite@pobox.com> wrote:
> So I'll ask again...
>
> On 12/22/2014 03:15 PM, ronnie sahlberg wrote:
>>
>> On Mon, Dec 22, 2014 at 2:52 PM, Robert White <rwhite@pobox.com> wrote:
>>>
>>> So skipping the full ADS, what's the current demand/payoff for large
>>> XATTR
>>> space?
>
>
> ^^^^^^^^^^^^
>
> So skipping the full ADS, what is the current demand/payoff for large XATTR
> space?

As I said, NTFS Security Descriptors (sometimes incorrectly called
ACLs) as set by Samba.

I know of at least two products that use that approach, one on CentOS,
one on FreeBSD. There are likely others and there will be one more in
the near future.

-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)

Re: Can BTRFS handle XATTRs larger than 4K?

[Robert White]

On 12/22/2014 03:58 PM, Richard Sharpe wrote:
> On Mon, Dec 22, 2014 at 3:55 PM, Robert White <rwhite@pobox.com> wrote:
>> So I'll ask again...
>>
>> On 12/22/2014 03:15 PM, ronnie sahlberg wrote:
>>>
>>> On Mon, Dec 22, 2014 at 2:52 PM, Robert White <rwhite@pobox.com> wrote:
>>>>
>>>> So skipping the full ADS, what's the current demand/payoff for large
>>>> XATTR
>>>> space?
>>
>>
>> ^^^^^^^^^^^^
>>
>> So skipping the full ADS, what is the current demand/payoff for large XATTR
>> space?
>
> As I said, NTFS Security Descriptors (sometimes incorrectly called
> ACLs) as set by Samba.
>

yep, other branch of thread, my re-ask was towards Ronie Sahlberg as 
quoted... we passed each other on the backstretch... 8-)

Re: Can BTRFS handle XATTRs larger than 4K?

[Richard Sharpe]

On Mon, Dec 22, 2014 at 10:09 AM, Austin S Hemmelgarn
<ahferroin7@gmail.com> wrote:
> On 2014-12-22 12:27, Richard Sharpe wrote:
>>
>> On Mon, Dec 22, 2014 at 6:28 AM, Austin S Hemmelgarn
>> <ahferroin7@gmail.com> wrote:
>>>
>>> On 2014-12-19 21:07, Richard Sharpe wrote:
>>>>
>>>>
>>>> Hi folks,
>>>>
>>>> I need a Linux file system that supports XATTRs up to 64K.
>>>>
>>>> Can BTRFS support that or is XFS the only Linux file system with such
>>>> support?
>>>>
>>> At the moment, BTRFS is limited to xattrs that fit inline in the metadata
>>> nodes (so ~3900 bytes for a 4k leafsize).
>>>
>>> XFS, however, isn't the only Linux filesystem that supports xattrs that
>>> size.  Assuming that you are using a recent kernel, you can also use such
>>> xattrs on at least:
>>>   * XFS
>>>   * JFS
>>>   * ext4
>>>   * reiserfs (I think, not 100% certain about this one though)
>>>   * OCFS2 (even though it is technically a cluster fs, it can be run
>>> single
>>> node without the clustering)
>>>   * ZFS (IIRC, ZFS supports unlimited xattr size)
>>>   * NTFS (no limit on xattr size, though you should use NTFS-3G instead
>>> of
>>> the in-kernel driver)
>>>   * SquashFS (read-only)
>>>   * HFS+ (Also no limit on xattr size)
>>> Of these, I'd personally suggest using XFS unless you need to be able to
>>> shrink the filesystem, in which case I'd suggest ext4.
>>
>>
>> Thanks for the info. I hadn't realized that ext4 had lifted the
>> restriction.
>>
> Yeah, it would be nice if there was more clarity in the documentation.
>
> Personally, I'd love to see unlimited length xattr's like NTFS and HFS+ do,
> as that would greatly improve interoperability (both Windows and OS X use
> xattrs, although they call them 'alternative data streams' and 'forks'
> respectively), and provide a higher likelihood that xattrs would start
> getting used more.

Well, there is a big difference in the semantics of Alternate Data
Streams (ADSs) and XATTRs.

For example, you can seek on an ADS and read at any offset. You cannot
do that on an XATTR (at least, not with the semantics provided by the
UNIX interface.)

-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)

Re: Can BTRFS handle XATTRs larger than 4K?

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 2476 bytes --]

On 2014-12-22 15:04, Richard Sharpe wrote:
> On Mon, Dec 22, 2014 at 10:09 AM, Austin S Hemmelgarn
> <ahferroin7@gmail.com> wrote:
>> On 2014-12-22 12:27, Richard Sharpe wrote:
>>>
>>> On Mon, Dec 22, 2014 at 6:28 AM, Austin S Hemmelgarn
>>> <ahferroin7@gmail.com> wrote:
>>>>
>>>> On 2014-12-19 21:07, Richard Sharpe wrote:
>>>>>
>>>>>
>>>>> Hi folks,
>>>>>
>>>>> I need a Linux file system that supports XATTRs up to 64K.
>>>>>
>>>>> Can BTRFS support that or is XFS the only Linux file system with such
>>>>> support?
>>>>>
>>>> At the moment, BTRFS is limited to xattrs that fit inline in the metadata
>>>> nodes (so ~3900 bytes for a 4k leafsize).
>>>>
>>>> XFS, however, isn't the only Linux filesystem that supports xattrs that
>>>> size.  Assuming that you are using a recent kernel, you can also use such
>>>> xattrs on at least:
>>>>    * XFS
>>>>    * JFS
>>>>    * ext4
>>>>    * reiserfs (I think, not 100% certain about this one though)
>>>>    * OCFS2 (even though it is technically a cluster fs, it can be run
>>>> single
>>>> node without the clustering)
>>>>    * ZFS (IIRC, ZFS supports unlimited xattr size)
>>>>    * NTFS (no limit on xattr size, though you should use NTFS-3G instead
>>>> of
>>>> the in-kernel driver)
>>>>    * SquashFS (read-only)
>>>>    * HFS+ (Also no limit on xattr size)
>>>> Of these, I'd personally suggest using XFS unless you need to be able to
>>>> shrink the filesystem, in which case I'd suggest ext4.
>>>
>>>
>>> Thanks for the info. I hadn't realized that ext4 had lifted the
>>> restriction.
>>>
>> Yeah, it would be nice if there was more clarity in the documentation.
>>
>> Personally, I'd love to see unlimited length xattr's like NTFS and HFS+ do,
>> as that would greatly improve interoperability (both Windows and OS X use
>> xattrs, although they call them 'alternative data streams' and 'forks'
>> respectively), and provide a higher likelihood that xattrs would start
>> getting used more.
>
> Well, there is a big difference in the semantics of Alternate Data
> Streams (ADSs) and XATTRs.
>
> For example, you can seek on an ADS and read at any offset. You cannot
> do that on an XATTR (at least, not with the semantics provided by the
> UNIX interface.)
>
I'm not trying to say the semantics are identical, just that the 
functionality is pretty similar.  A better way to put it would be that 
xattrs are an extremely limited form of ADS/Forks.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2455 bytes --]