XAttrs limited to 100 bytes

XAttrs limited to 100 bytes

[Joaquim Rocha]

Hi folks,

I found out recently that xattributes are limited to 100 chars in length 
so I saw the commits that introduced this change [1] but I don't 
understand why this is needed if we take into account that by default 
xattributes are going into the omap.

In those changes it also prevents that limit to go over 100 even 
configured that way so I would like to know the reason for this limit 
and if there is any other way we can override it I might be missing.

For my use case I will need a much larger length so I think that there 
should be a way to easily increase the limit (or even have a much larger 
value as the default).

[1] https://github.com/ceph/ceph/pull/2111


Cheers,

--
Joaquim Rocha

Re: XAttrs limited to 100 bytes

[Gregory Farnum]

On Fri, Nov 14, 2014 at 6:24 AM, Joaquim Rocha <joaquim.rocha@cern.ch> wrote:
> Hi folks,
>
> I found out recently that xattributes are limited to 100 chars in length so
> I saw the commits that introduced this change [1] but I don't understand why
> this is needed if we take into account that by default xattributes are going
> into the omap.
>
> In those changes it also prevents that limit to go over 100 even configured
> that way so I would like to know the reason for this limit and if there is
> any other way we can override it I might be missing.
>
> For my use case I will need a much larger length so I think that there
> should be a way to easily increase the limit (or even have a much larger
> value as the default).
>
> [1] https://github.com/ceph/ceph/pull/2111

These are just setting limits on the length of the xattr *names*, not
the xattr contents. This is because filesystems impose limits on name
size and we do store xattrs in the filesystem in the default
configuration with most releases of Ceph (they overflow into leveldb
if there are too many).
If that's really going to be a problem for you...I don't know what to
say, sorry. These are close to limits you should have been following
already fi you want your stuff to work on e.g. ext4. :)
-Greg

Re: XAttrs limited to 100 bytes

[Dan van der Ster]

On Fri, Nov 14, 2014 at 4:35 PM, Dan van der Ster
<daniel.vanderster@cern.ch> wrote:
> Hi Greg,
>
> On Fri Nov 14 2014 at 4:07:41 PM Gregory Farnum <greg@gregs42.com> wrote:
>>
>> On Fri, Nov 14, 2014 at 6:24 AM, Joaquim Rocha <joaquim.rocha@cern.ch>
>> wrote:
>> > Hi folks,
>> >
>> > I found out recently that xattributes are limited to 100 chars in length
>> > so
>> > I saw the commits that introduced this change [1] but I don't understand
>> > why
>> > this is needed if we take into account that by default xattributes are
>> > going
>> > into the omap.
>> >
>> > In those changes it also prevents that limit to go over 100 even
>> > configured
>> > that way so I would like to know the reason for this limit and if there
>> > is
>> > any other way we can override it I might be missing.
>> >
>> > For my use case I will need a much larger length so I think that there
>> > should be a way to easily increase the limit (or even have a much larger
>> > value as the default).
>> >
>> > [1] https://github.com/ceph/ceph/pull/2111
>>
>> These are just setting limits on the length of the xattr *names*, not
>> the xattr contents. This is because filesystems impose limits on name
>> size and we do store xattrs in the filesystem in the default
>> configuration with most releases of Ceph (they overflow into leveldb
>> if there are too many).
>> If that's really going to be a problem for you...I don't know what to
>> say, sorry. These are close to limits you should have been following
>> already fi you want your stuff to work on e.g. ext4. :)
>
>
>
> Do you know where the limits are? This limitation was a surprise to us
> because with latest dumpling on RHEL6.6 XFS we can set and get very long
> xattr names. IOW, we cannot reproduce the truncation issue reported by
> yuyuyu in #2111. With a new object asdf we set the first xattr and it
> already overflowed into the omap.
>

Nevermind, I realized this is because we had
filestore_max_inline_xattrs = 2. With filestore_max_inline_xattrs = 5
for example, I can reproduce the truncation issue.

Cheers, Dan