JFS extended attributes and ACLs - New Patches

JFS extended attributes and ACLs - New Patches

[Dave Kleikamp]

Andreas,

Thanks for the test cases.  I now have new patches that pass the ACL 
tests cleanly, and pass the extended attributes cases that are not 
ext2/3-specific.

This code allows extended attributes to be set on symlinks.  Should this 
be disallowed?

The patches are available here:
http://www-124.ibm.com/developerworks/patch/?func=detailpatch&patch_id=351&group_id=35
http://www-124.ibm.com/developerworks/patch/?func=detailpatch&patch_id=352&group_id=35

On Friday 06 September 2002 10:18, Andreas Gruenbacher wrote:
> Hi David,
>
> concerning the EA and ACL jfs code: Could you please check if the
> current test scripts work correctly. Some of the EA tests are
> ext2/ext3 specific, so it's okay if they fail. The test scripts
> should shortly appear under <http://www.suse.de/~agruen/test/>; if
> that takes too long I can send them per e-mail.
>
> Regards,
> Andreas.

-- 
David Kleikamp
IBM Linux Technology Center

Re: JFS extended attributes and ACLs - New Patches

[Andreas Gruenbacher]

On Monday 09 September 2002 16:58, Dave Kleikamp wrote:
> Andreas,
>
> Thanks for the test cases.  I now have new patches that pass the ACL
> tests cleanly, and pass the extended attributes cases that are not
> ext2/3-specific.
>
> This code allows extended attributes to be set on symlinks.  Should this
> be disallowed?

Extended user attributes should be allowed only for regular files and for 
directories with the sticky bet cleared. Rationale: See the attr(5) manual 
page.

> The patches are available here:
> http://www-124.ibm.com/developerworks/patch/?func=detailpatch&patch_id=351&
>group_id=35
> http://www-124.ibm.com/developerworks/patch/?func=detailpatch&patch_id=352&
>group_id=35

Are these links permanent?

--Andreas.

Re: JFS extended attributes and ACLs - New Patches

[Dave Kleikamp]

On Monday 09 September 2002 10:02, Andreas Gruenbacher wrote:

> Extended user attributes should be allowed only for regular files and
> for directories with the sticky bet cleared. Rationale: See the
> attr(5) manual page.

Hmm.  If different policies are to be applied to the different 
namespaces, I think it would be a good thing to move that policy to the 
vfs layer, rather than have each filesystem redundantly enforce these 
policies.

If the VFS is requesting that an extended attribute be set, the 
filesystem-specific code should be able to assume it's allowed.  I've 
tried to keep the JFS code simple.  It really doesn't care and doesn't 
want to care about namespaces.  I've special cased the setting of acl's 
to make sure the permissions get set correctly, but I'd prefer this 
happened at a higher level as well.

Would you consider a patch to migrate the namespace-specific code into 
the vfs layer, and just let the file systems worry about the storage?

> > The patches are available here:
> > http://www-124.ibm.com/developerworks/patch/?func=detailpatch&patch
> >_id=351& group_id=35
> > http://www-124.ibm.com/developerworks/patch/?func=detailpatch&patch
> >_id=352& group_id=35
>
> Are these links permanent?

In the sense that they aren't going away.  Later versions will probably 
have different urls.  If you want a static link, you can use: 
http://oss.software.ibm.com/developerworks/patch/?group=35

>
> --Andreas.

-- 
David Kleikamp
IBM Linux Technology Center

Re: JFS extended attributes and ACLs - New Patches

[Dave Kleikamp]

On Monday 09 September 2002 10:28, Dave Kleikamp wrote:

> In the sense that they aren't going away.  Later versions will
> probably have different urls.  If you want a static link, you can
> use: http://oss.software.ibm.com/developerworks/patch/?group=35

http://oss.software.ibm.com/developerworks/patch/?group_id=35

-- 
David Kleikamp
IBM Linux Technology Center

RE: JFS extended attributes and ACLs - New Patches

[Luka Renko]

> On Monday 09 September 2002 10:02, Andreas Gruenbacher wrote:
> 
> > Extended user attributes should be allowed only for regular 
> files and 
> > for directories with the sticky bet cleared. Rationale: See the
> > attr(5) manual page.
> 
> Hmm.  If different policies are to be applied to the different 
> namespaces, I think it would be a good thing to move that 
> policy to the 
> vfs layer, rather than have each filesystem redundantly enforce these 
> policies.
> 
> If the VFS is requesting that an extended attribute be set, the 
> filesystem-specific code should be able to assume it's allowed.  I've 
> tried to keep the JFS code simple.  It really doesn't care 
> and doesn't 
> want to care about namespaces.  I've special cased the 
> setting of acl's 
> to make sure the permissions get set correctly, but I'd prefer this 
> happened at a higher level as well.

I think that is the right solution. FS should not care about namespaces and
potential limitations they should just provide storage for EA.

I anyway do not like the fact that EA functionality is limited for different
type of files. 
If the only limitation is quata system, then we may just have a additional
mount option to support EA on any file type - this way user can select what
they want to have (strict quota or EA on special files and symbolic links).

Regards,
Luka

Re: JFS extended attributes and ACLs - New Patches

[Andreas Gruenbacher]

On Tuesday 10 September 2002 09:40, Luka Renko wrote:
> On Monday 09 September 2002 10:02, Andreas Gruenbacher wrote:
> > Extended user attributes should be allowed only for regular files and
> > for directories with the sticky bet cleared. Rationale: See the
> > attr(5) manual page.
>
> Hmm. If different policies are to be applied to the different
> namespaces, I think it would be a good thing to move that
> policy to the
> vfs layer, rather than have each filesystem redundantly enforce these
> policies.
>
> If the VFS is requesting that an extended attribute be set, the
> filesystem-specific code should be able to assume it's allowed. I've
> tried to keep the JFS code simple. It really doesn't care and doesn't
> want to care about namespaces. I've special cased the setting of acl's
> to make sure the permissions get set correctly, but I'd prefer this
> happened at a higher level as well.
> I think that is the right solution. FS should not care about namespaces
> and potential limitations they should just provide storage for EA.

>From a filesystem point of view I can understand this position. From the VFS 
point of view this doesn't make too much sense though: For each namespace 
there are at least three kinds of limitations: Policy/permissions, 
restrictions on the valid values, and storage related limitations. 
Filesystems may support different namespaces, and within namespaces like 
`system', different attributes with individual restrictions.

If we implement the policy in the VFS, then the VFS would have to know about 
all possible kinds of namespaces and attributes, and about their generic 
limitations, while the file systems would still need to check for their 
specific limitations (i.e., some tasks would have to be done twice). The 
current interface (the xattr VFS operations) wouldn't fit.

Some filesystems (e.g., XFS) have legacy support for things like ACLs with 
their own specific limitations. JFS apparently has some sort of legacy 
extended attributes (the OS/2 stuff that's mentioned in the patch), NTFS has 
very different Windows type ACLs. ACLs will start to differ from each other 
once additional features are implemented. Some file systems will find it easy 
to support additions/extensions, while others may find it more difficult or 
impossible. There are many examples why enforcing a common policy doesn't 
make a lot of sense at the moment.

Note that the VFS also doesn't enforce the permission scheme filesystems 
implement, but only has a mechanism to test if a certain type of access is 
allowed. Extended attributes are not used in the kernel outside the file 
systems, except for kernel nfsd, which is a notorious exception from many 
rules anyway.

> I anyway do not like the fact that EA functionality is limited for
> different types of files.
> If the only limitation is quata system, then we may just have a additional
> mount option to support EA on any file type - this way user can select what
> they want to have (strict quota or EA on special files and symbolic links).

Roughly the current xattr interface is defined like this (a more detailed 
description is found in the attr(5) manual page): There are different 
namespaces. Each namespace determines the rules for accessing the xatts in 
this namespace; the rules are consistent inside the namespace.

Currently the `user' and `system' namespaces are defined. `User' means 
permissions to xattrs are similar to permissions for file contents. `System' 
means there are specific limitations depending on the full attribute name 
(i.e., the rules for `system.posix_acl_*' will likely differ from 
`system.posix_capabilities', etc.).

If you take this as the definition, Extended User Attributes (`user' 
namespace) make no sense for device special files, symlinks, or for sticky 
directories. For these, file permissions work too differently from the usual 
case, so the usual permissions used on UNIX systems would lead to peculiar 
`user.*' permissions. (Which sysadmin would expect user xattrs of /dev/null 
to be the cause for a full disk?)

The reasoning in the attr(5) manual page is terse, because a verbose 
description didn't seem to make much sense there. If you want more 
information/discussion on this issue you are invited to ask, or contact 
Nathan Scott <nathans@sgi.com> who was also involved in thinking out the 
rules.

Cheers, Andreas.


------------------------------------------------------------------
 Andreas Gruenbacher                                SuSE Linux AG
 mailto:agruen@suse.de                     Deutschherrnstr. 15-19
 http://www.suse.de/                   D-90429 Nuernberg, Germany

Re: JFS extended attributes and ACLs - New Patches

[Dave Kleikamp]

On Thursday 12 September 2002 10:13, Andreas Gruenbacher wrote:

> From a filesystem point of view I can understand this position. From
> the VFS point of view this doesn't make too much sense though: For
> each namespace there are at least three kinds of limitations:
> Policy/permissions, restrictions on the valid values, and storage
> related limitations. Filesystems may support different namespaces,
> and within namespaces like `system', different attributes with
> individual restrictions.

Let the filesystems just worry about the storage related limitations.  
If the filesystems support another namespace, let them register it at a 
filesystem-independent layer so all of this policy is enforced in one 
place.  I think its a bad idea to have 5 different file systems all 
duplicating the same code to check permissions, etc. for the system and 
user namespaces.

> If we implement the policy in the VFS, then the VFS would have to
> know about all possible kinds of namespaces and attributes, and about
> their generic limitations, while the file systems would still need to
> check for their specific limitations (i.e., some tasks would have to
> be done twice). The current interface (the xattr VFS operations)
> wouldn't fit.

Yes the VFS would have to know.  That is preferable to all of the file 
systems having to duplicate the work.  If some kind of namespace 
registration were permitted in the VFS layer, the xattr VFS operations 
wouldn't need to be changed.  Those operations would be used to store 
and retrieve the attributes.  That's all.  If a filesystem chose not to 
support a namespace, it could simply return -ENOTSUPP.

> Some filesystems (e.g., XFS) have legacy support for things like ACLs
> with their own specific limitations.

If the filesystems want to convert the posix acl to another format, or 
restrict the size, etc.  It can still do that.  Let a common layer 
handle the common tasks.

> JFS apparently has some sort of
> legacy extended attributes (the OS/2 stuff that's mentioned in the
> patch), NTFS has very different Windows type ACLs. ACLs will start to
> differ from each other once additional features are implemented.

JFS could add an "os2." namespace to handle the legacy attributes.  
Different ACLs could be stored in a different system.* attribute, or 
converted to posix ACLs. within before being stored.

> Some
> file systems will find it easy to support additions/extensions, while
> others may find it more difficult or impossible.

It would be much easier to support extensions if the code were only in 
one place and not duplicated with every file system.

> There are many
> examples why enforcing a common policy doesn't make a lot of sense at
> the moment.

So instead we enforce a common policy in 5 different places (ext2, ext3, 
XFS, JFS, reiserfs).

> Note that the VFS also doesn't enforce the permission scheme
> filesystems implement, but only has a mechanism to test if a certain
> type of access is allowed.

???  In all other operations the VFS DOES enforce the permission scheme.  
It doesn't expect fs_ops->mkdir() to explicitly call permission() 
before creating a directory.  The VFS does a very good job of providing 
common code whenever file systems do similar things, only requiring the 
file system to perform tasks that actually are specific to the file 
system.

If modifying system.posix_acl_access, may modify i_mode, let the VFS do 
that and just ask the file system to store the xattr.

> Extended attributes are not used in the
> kernel outside the file systems, except for kernel nfsd, which is a
> notorious exception from many rules anyway.
>
> > I anyway do not like the fact that EA functionality is limited for
> > different types of files.
> > If the only limitation is quata system, then we may just have a
> > additional mount option to support EA on any file type - this way
> > user can select what they want to have (strict quota or EA on
> > special files and symbolic links).
>
> Roughly the current xattr interface is defined like this (a more
> detailed description is found in the attr(5) manual page): There are
> different namespaces. Each namespace determines the rules for
> accessing the xatts in this namespace; the rules are consistent
> inside the namespace.

Then why implement each namespace's rules in each file system?

> Currently the `user' and `system' namespaces are defined. `User'
> means permissions to xattrs are similar to permissions for file
> contents. 

Then have the VFS enforce it!

> `System' means there are specific limitations depending on
> the full attribute name (i.e., the rules for `system.posix_acl_*'
> will likely differ from `system.posix_capabilities', etc.).

Then let's handle these in one place!

> If you take this as the definition, Extended User Attributes (`user'
> namespace) make no sense for device special files, symlinks, or for
> sticky directories. For these, file permissions work too differently
> from the usual case, so the usual permissions used on UNIX systems
> would lead to peculiar `user.*' permissions. (Which sysadmin would
> expect user xattrs of /dev/null to be the cause for a full disk?)

I understand the reason that the policies differ between namespaces.  I 
don't agree that the VFS should leave it to the file systems to enforce 
the policies.

> The reasoning in the attr(5) manual page is terse, because a verbose
> description didn't seem to make much sense there. If you want more
> information/discussion on this issue you are invited to ask, or
> contact Nathan Scott <nathans@sgi.com> who was also involved in
> thinking out the rules.

The rules are fine.  Let's have some common code to enforce them.

-- 
David Kleikamp
IBM Linux Technology Center

Re: JFS extended attributes and ACLs - New Patches

[Andreas Gruenbacher]

On Thursday 12 September 2002 18:16, Dave Kleikamp wrote:
> On Thursday 12 September 2002 10:13, Andreas Gruenbacher wrote:
> > From a filesystem point of view I can understand this position. From
> > the VFS point of view this doesn't make too much sense though: For
> > each namespace there are at least three kinds of limitations:
> > Policy/permissions, restrictions on the valid values, and storage
> > related limitations. Filesystems may support different namespaces,
> > and within namespaces like `system', different attributes with
> > individual restrictions.
>
> Let the filesystems just worry about the storage related limitations.
> If the filesystems support another namespace, let them register it at a
> filesystem-independent layer so all of this policy is enforced in one
> place.  I think its a bad idea to have 5 different file systems all
> duplicating the same code to check permissions, etc. for the system and
> user namespaces.

We could take the restriction checking code (which is in fact only a couple of 
lines), put these functions in the VFS, and let the file systems invoke these 
functions (as partially done for ACLs, e.g., fs/posix_acl.c's 
posix_acl_valid()). Not a lot of code would remain though.

I don't think a VFS xattr namespace/name/policy registry stands a chance of 
getting accepted into the standard kernel.

Part of the checking code may be to convert xattrs into some internal 
representation (e.g., for ACLs). I'd like to avoid enforcing in the VFS the 
representation that is used.

[...]

> > JFS apparently has some sort of
> > legacy extended attributes (the OS/2 stuff that's mentioned in the
> > patch), NTFS has very different Windows type ACLs. ACLs will start to
> > differ from each other once additional features are implemented.
>
> JFS could add an "os2." namespace to handle the legacy attributes.
> Different ACLs could be stored in a different system.* attribute, or
> converted to posix ACLs. within before being stored.

Yes, this is what I wanted to propose, unless the semantics are exactly the 
same as for user xattrs. What are the exact rules for accessing these 
attributes under OS/2? How about AIX?

Can you implement the `os2.' namespace in the JFS patches? I'd be much happier 
with the patch.


Regards,
Andreas.

------------------------------------------------------------------
 Andreas Gruenbacher                                SuSE Linux AG
 mailto:agruen@suse.de                     Deutschherrnstr. 15-19
 http://www.suse.de/                   D-90429 Nuernberg, Germany

Re: JFS extended attributes and ACLs - New Patches

[Dave Kleikamp]

On Thursday 12 September 2002 12:04, Andreas Gruenbacher wrote:
> We could take the restriction checking code (which is in fact only a
> couple of lines), put these functions in the VFS, and let the file
> systems invoke these functions (as partially done for ACLs, e.g.,
> fs/posix_acl.c's posix_acl_valid()). Not a lot of code would remain
> though.

Helper functions would be an improvement.

> I don't think a VFS xattr namespace/name/policy registry stands a
> chance of getting accepted into the standard kernel.

Maybe not, but I don't see any benefit of doing it per fs.

> Part of the checking code may be to convert xattrs into some internal
> representation (e.g., for ACLs). I'd like to avoid enforcing in the
> VFS the representation that is used.

Would any other internal representation of a posix acl make sense?  This 
wouldn't be tied to the way the acl was stored.  Only to the format 
coming in from the caller.  Of course I would propose that i_acl and 
i_default_acl be moved to the generic inode.

> > JFS could add an "os2." namespace to handle the legacy attributes.
> > Different ACLs could be stored in a different system.* attribute,
> > or converted to posix ACLs. within before being stored.
>
> Yes, this is what I wanted to propose, unless the semantics are
> exactly the same as for user xattrs. What are the exact rules for
> accessing these attributes under OS/2?

I don't remember exactly, but OS/2 doesn't have unix-like permissions.  
I'm really not interested in the behavior being OS/2-like.  I we may as 
well be able to access the OS/2 attributes as long as they exist.  The 
permissions may as well be the same as 'user.' permissions.

> How about AIX?

Don't know.

> Can you implement the `os2.' namespace in the JFS patches? I'd be
> much happier with the patch.

I believe I can.  I will use the 'os2.' prefix externally, but store it 
without the prefix.  On the other side, any attribute that doesn't 
begin with a known prefix will get 'os2.' prepended when listing the 
xattrs.

I could even go as far as mapping them to user.os2.<os2name> to avoid 
creating a new namespace.

-- 
David Kleikamp
IBM Linux Technology Center

Re: JFS extended attributes and ACLs - New Patches

[Nathan Scott]

hi Dave,

On Thu, Sep 12, 2002 at 12:34:29PM -0500, Dave Kleikamp wrote:
> On Thursday 12 September 2002 12:04, Andreas Gruenbacher wrote:
> 
> > Can you implement the `os2.' namespace in the JFS patches? I'd be
> > much happier with the patch.
> 
> I believe I can.  I will use the 'os2.' prefix externally, but store it 
> without the prefix.  On the other side, any attribute that doesn't 
> begin with a known prefix will get 'os2.' prepended when listing the 
> xattrs.

This is basically what we do in XFS - we have a separate namespace
for accessing the actual ondisk names &/ values of what on IRIX is
called the ROOT namespace - on Linux the "xfsroot" namespace.  This
is how xfsdump and xfsrestore are able to work with the same on-tape
format that is used in IRIX - they manipulate attributes in this
XFS-specific namespace while the generic ACL tools (setfacl, getfacl,
and co) access the system.posix_acl names, which are then independent
of the underlying filesystem type.

I would shy away from a "hierarchical" namespace approach; if I grok
what you're saying, I think it would mean that the value for the name
"user.os2.a" would be different between filesystems (for JFS it would
mean "a" in the user.os2 namespace, for everyone else it would mean
"os2.a" in the user namespace) - that'd be pretty confusing.

cheers.

-- 
Nathan

Re: JFS extended attributes and ACLs - New Patches

[Andreas Gruenbacher]

On Friday 13 September 2002 07:06, Nathan Scott wrote:
> hi Dave,
>
> On Thu, Sep 12, 2002 at 12:34:29PM -0500, Dave Kleikamp wrote:
> > On Thursday 12 September 2002 12:04, Andreas Gruenbacher wrote:
> > > Can you implement the `os2.' namespace in the JFS patches? I'd be
> > > much happier with the patch.
> >
> > I believe I can.  I will use the 'os2.' prefix externally, but store it
> > without the prefix.  On the other side, any attribute that doesn't
> > begin with a known prefix will get 'os2.' prepended when listing the
> > xattrs.

Good. The namespace should still be flat, as Nathan said already. Which is 
what I think you had in mind anyway.

--Andreas.

Re: JFS extended attributes and ACLs - New Patches

[Andreas Gruenbacher]

On Thursday 12 September 2002 19:34, Dave Kleikamp wrote:
> I could even go as far as mapping them to user.os2.<os2name> to avoid
> creating a new namespace.

A new namespace is not a big deal, but if all OS/2 attributes can reasonably 
well live under `user.' that would be the simples solution.

--Andreas.

Re: JFS extended attributes and ACLs - New Patches

[Dave Kleikamp]

On Friday 13 September 2002 00:06, Nathan Scott wrote:
> I would shy away from a "hierarchical" namespace approach; if I grok
> what you're saying, I think it would mean that the value for the name
> "user.os2.a" would be different between filesystems (for JFS it would
> mean "a" in the user.os2 namespace, for everyone else it would mean
> "os2.a" in the user namespace) - that'd be pretty confusing.

It really wouldn't appear different to user-space.  Even on JFS, 
"user.os2.a" would appear to be os2.a in the user namespace.  The only 
difference would be how the kernel-space code stores is on disk, and 
that it would appear in os/2's flat namespace.

I threw that out as an idea, but I think using "os2.a" makes more sense.  
Note many of OS/2 xattrs start with a leading period, so the these will 
look like "os2..SUBJECT" or something similar.

>
> cheers.

-- 
David Kleikamp
IBM Linux Technology Center

Re: [Acl-Devel] Re: JFS extended attributes and ACLs - New Patches

[Andreas Gruenbacher]

On Fri, 13 Sep 2002, Dave Kleikamp wrote:

> On Friday 13 September 2002 00:06, Nathan Scott wrote:
> > I would shy away from a "hierarchical" namespace approach; if I grok
> > what you're saying, I think it would mean that the value for the name
> > "user.os2.a" would be different between filesystems (for JFS it would
> > mean "a" in the user.os2 namespace, for everyone else it would mean
> > "os2.a" in the user namespace) - that'd be pretty confusing.
>
> It really wouldn't appear different to user-space.  Even on JFS,
> "user.os2.a" would appear to be os2.a in the user namespace.  The only
> difference would be how the kernel-space code stores is on disk, and
> that it would appear in os/2's flat namespace.

So there are different ways of storing xattrs? I wasn't aware of that.

> I threw that out as an idea, but I think using "os2.a" makes more sense.
> Note many of OS/2 xattrs start with a leading period, so the these will
> look like "os2..SUBJECT" or something similar.

Yes. "os2..*" seems alright.

--Andreas.

Re: [Acl-Devel] Re: JFS extended attributes and ACLs - New Patches

[Dave Kleikamp]

On Friday 13 September 2002 08:20, Andreas Gruenbacher wrote:

> > It really wouldn't appear different to user-space.  Even on JFS,
> > "user.os2.a" would appear to be os2.a in the user namespace.  The
> > only difference would be how the kernel-space code stores is on
> > disk, and that it would appear in os/2's flat namespace.
>
> So there are different ways of storing xattrs? I wasn't aware of
> that.

Not different ways, actually.  In order for OS/2 to see the attribute 
names as it expects, the "user.os2." attributes would be stored without 
a prefix.  The attribute name for any other namespace would include the 
namespace prefix.

Example:

>From user-land:
user.test
user.os2.test

On disk:
user.test
test

OS/2 would see both attributes as they are stored on disk.  It has no 
concept of namespaces, other than the convention that "system" 
attributes begin with a period.

Of course it will work the same with either the "os2." or "user.os2." 
prefix.


> Yes. "os2..*" seems alright.

I didn't think there'd be a problem.  I just thought it was interesting.


-- 
David Kleikamp
IBM Linux Technology Center