Immutable vs read-only for Windows compatibility

Immutable vs read-only for Windows compatibility

[Pali Rohár]

Hello,

Few months ago I discussed with Steve that Linux SMB client has some
problems during removal of directory which has read-only attribute set.

I was looking what exactly the read-only windows attribute means, how it
is interpreted by Linux and in my opinion it is wrongly used in Linux at
all.

Windows filesystems NTFS and ReFS, and also exported over SMB supports
two ways how to present some file or directory as read-only. First
option is by setting ACL permissions (for particular or all users) to
GENERIC_READ-only. Second option is by setting the read-only attribute.
Second option is available also for (ex)FAT filesystems (first option via
ACL is not possible on (ex)FAT as it does not have ACLs).

First option (ACL) is basically same as clearing all "w" bits in mode
and ACL (if present) on Linux. It enforces security permission behavior.
Note that if the parent directory grants for user delete child
permission then the file can be deleted. This behavior is same for Linux
and Windows (on Windows there is separate ACL for delete child, on Linux
it is part of directory's write permission).

Second option (Windows read-only attribute) means that the file/dir
cannot be opened in write mode, its metadata attribute cannot be changed
and the file/dir cannot be deleted at all. But anybody who has
WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
wants.

Linux filesystems has similar thing to Windows read-only attribute
(FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
which can be set by the "chattr" tool. Seems that the only difference
between Windows read-only and Linux immutable is that on Linux only
process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
it can be anybody who has write ACL.

Now I'm thinking, how should be Windows read-only bit interpreted by
Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:

0) Simply ignored. Disadvantage is that over network fs, user would not
   be able to do modify or delete such file, even as root.

1) Smartly ignored. Meaning that for local fs, it is ignored and for
   network fs it has to be cleared before any write/modify/delete
   operation.

2) Translated to Linux mode/ACL. So the user has some ability to see it
   or change it via chmod. Disadvantage is that it mix ACL/mode.

3) Translated to Linux FS_IMMUTABLE_FL. So the user can use lsattr /
   chattr to see or change it. Disadvantage is that this bit can be
   changed only by root or by CAP_LINUX_IMMUTABLE process.

4) Exported via some new xattr. User can see or change it. But for
   example recursive removal via rm -rf would be failing as rm would not
   know about this special new xattr.

In any case, in my opinion, all Linux fs drivers for these filesystems
(FAT, exFAT, NTFS, SMB, are there some others?) should handle this
windows read-only bit in the same way.

What do you think, what should be the best option?

I have another idea. What about introducing a new FS_IMMUTABLE_USER_FL
bit which have same behavior as FS_IMMUTABLE_FL, just it would be
possible to set it for any user who has granted "write" permission?
Instead of requiring CAP_LINUX_IMMUTABLE. I see a nice usecase that even
ordinary user could be able to mark file as protected against removal or
modification (for example some backup data).

Pali

Re: Immutable vs read-only for Windows compatibility

[Jan Kara]

Hello!

On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> Few months ago I discussed with Steve that Linux SMB client has some
> problems during removal of directory which has read-only attribute set.
> 
> I was looking what exactly the read-only windows attribute means, how it
> is interpreted by Linux and in my opinion it is wrongly used in Linux at
> all.
> 
> Windows filesystems NTFS and ReFS, and also exported over SMB supports
> two ways how to present some file or directory as read-only. First
> option is by setting ACL permissions (for particular or all users) to
> GENERIC_READ-only. Second option is by setting the read-only attribute.
> Second option is available also for (ex)FAT filesystems (first option via
> ACL is not possible on (ex)FAT as it does not have ACLs).
> 
> First option (ACL) is basically same as clearing all "w" bits in mode
> and ACL (if present) on Linux. It enforces security permission behavior.
> Note that if the parent directory grants for user delete child
> permission then the file can be deleted. This behavior is same for Linux
> and Windows (on Windows there is separate ACL for delete child, on Linux
> it is part of directory's write permission).
> 
> Second option (Windows read-only attribute) means that the file/dir
> cannot be opened in write mode, its metadata attribute cannot be changed
> and the file/dir cannot be deleted at all. But anybody who has
> WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> wants.

I guess someone with more experience how to fuse together Windows & Linux
permission semantics should chime in here but here are my thoughts.

> Linux filesystems has similar thing to Windows read-only attribute
> (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> which can be set by the "chattr" tool. Seems that the only difference
> between Windows read-only and Linux immutable is that on Linux only
> process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> it can be anybody who has write ACL.
> 
> Now I'm thinking, how should be Windows read-only bit interpreted by
> Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> 
> 0) Simply ignored. Disadvantage is that over network fs, user would not
>    be able to do modify or delete such file, even as root.
> 
> 1) Smartly ignored. Meaning that for local fs, it is ignored and for
>    network fs it has to be cleared before any write/modify/delete
>    operation.
> 
> 2) Translated to Linux mode/ACL. So the user has some ability to see it
>    or change it via chmod. Disadvantage is that it mix ACL/mode.

So this option looks sensible to me. We clear all write permissions in
file's mode / ACL. For reading that is fully compatible, for mode
modifications it gets a bit messy (probably I'd suggest to just clear
FILE_ATTRIBUTE_READONLY on modification) but kind of close.

> 3) Translated to Linux FS_IMMUTABLE_FL. So the user can use lsattr /
>    chattr to see or change it. Disadvantage is that this bit can be
>    changed only by root or by CAP_LINUX_IMMUTABLE process.
> 
> 4) Exported via some new xattr. User can see or change it. But for
>    example recursive removal via rm -rf would be failing as rm would not
>    know about this special new xattr.
> 
> In any case, in my opinion, all Linux fs drivers for these filesystems
> (FAT, exFAT, NTFS, SMB, are there some others?) should handle this
> windows read-only bit in the same way.
> 
> What do you think, what should be the best option?
> 
> I have another idea. What about introducing a new FS_IMMUTABLE_USER_FL
> bit which have same behavior as FS_IMMUTABLE_FL, just it would be
> possible to set it for any user who has granted "write" permission?

Uh, in unix, write permission is for accessing file data. Modifying access
permissions etc. is always limited to inode owner (or user with special
capabilities). So this would be very confusing in Unix permission model.

> Instead of requiring CAP_LINUX_IMMUTABLE. I see a nice usecase that even
> ordinary user could be able to mark file as protected against removal or
> modification (for example some backup data).

So I don't see us introducing another immutable bit in VFS. Special
"protected file" bit modifiable by whoever can modify file permissions is
not really different from clearing write permission bits in mode. So that
doesn't seem as a terribly useful feature to me. That being said nothing
really stops individual filesystems from introducing their own inode flags,
get / set them via ioctl and implement whatever permission checking needed
with them. After all this is how the flags like IMMUTABLE or APPEND started
and only later on when they propagated into many filesystems we've lifted
them into VFS.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: Immutable vs read-only for Windows compatibility

[Chuck Lever]

On 1/2/25 9:37 AM, Jan Kara wrote:
> Hello!
> 
> On Fri 27-12-24 13:15:08, Pali Rohár wrote:
>> Few months ago I discussed with Steve that Linux SMB client has some
>> problems during removal of directory which has read-only attribute set.
>>
>> I was looking what exactly the read-only windows attribute means, how it
>> is interpreted by Linux and in my opinion it is wrongly used in Linux at
>> all.
>>
>> Windows filesystems NTFS and ReFS, and also exported over SMB supports
>> two ways how to present some file or directory as read-only. First
>> option is by setting ACL permissions (for particular or all users) to
>> GENERIC_READ-only. Second option is by setting the read-only attribute.
>> Second option is available also for (ex)FAT filesystems (first option via
>> ACL is not possible on (ex)FAT as it does not have ACLs).
>>
>> First option (ACL) is basically same as clearing all "w" bits in mode
>> and ACL (if present) on Linux. It enforces security permission behavior.
>> Note that if the parent directory grants for user delete child
>> permission then the file can be deleted. This behavior is same for Linux
>> and Windows (on Windows there is separate ACL for delete child, on Linux
>> it is part of directory's write permission).
>>
>> Second option (Windows read-only attribute) means that the file/dir
>> cannot be opened in write mode, its metadata attribute cannot be changed
>> and the file/dir cannot be deleted at all. But anybody who has
>> WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
>> wants.
> 
> I guess someone with more experience how to fuse together Windows & Linux
> permission semantics should chime in here but here are my thoughts.
> 
>> Linux filesystems has similar thing to Windows read-only attribute
>> (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
>> which can be set by the "chattr" tool. Seems that the only difference
>> between Windows read-only and Linux immutable is that on Linux only
>> process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
>> it can be anybody who has write ACL.
>>
>> Now I'm thinking, how should be Windows read-only bit interpreted by
>> Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
>>
>> 0) Simply ignored. Disadvantage is that over network fs, user would not
>>     be able to do modify or delete such file, even as root.
>>
>> 1) Smartly ignored. Meaning that for local fs, it is ignored and for
>>     network fs it has to be cleared before any write/modify/delete
>>     operation.
>>
>> 2) Translated to Linux mode/ACL. So the user has some ability to see it
>>     or change it via chmod. Disadvantage is that it mix ACL/mode.
> 
> So this option looks sensible to me. We clear all write permissions in
> file's mode / ACL. For reading that is fully compatible, for mode
> modifications it gets a bit messy (probably I'd suggest to just clear
> FILE_ATTRIBUTE_READONLY on modification) but kind of close.

IMO Linux should store the Windows-specific attribute information but
otherwise ignore it. Modifying ACLs based seems like a road to despair.
Plus there's no ACL representation for OFFLINE and some of the other
items that we'd like to be able to support.


If I were king-for-a-day (tm) I would create a system xattr namespace
just for these items, and provide a VFS/statx API for consumers like
Samba, ksmbd, and knfsd to set and get these items. Each local
filesystem can then implement storage with either the xattr or (eg,
ntfs) can store them directly.

Semantics like READONLY or IMMUTABLE might be provided in the VFS if
we care to expose these semantics to POSIX consumers.


>> 3) Translated to Linux FS_IMMUTABLE_FL. So the user can use lsattr /
>>     chattr to see or change it. Disadvantage is that this bit can be
>>     changed only by root or by CAP_LINUX_IMMUTABLE process.
>>
>> 4) Exported via some new xattr. User can see or change it. But for
>>     example recursive removal via rm -rf would be failing as rm would not
>>     know about this special new xattr.
>>
>> In any case, in my opinion, all Linux fs drivers for these filesystems
>> (FAT, exFAT, NTFS, SMB, are there some others?) should handle this
>> windows read-only bit in the same way.
>>
>> What do you think, what should be the best option?
>>
>> I have another idea. What about introducing a new FS_IMMUTABLE_USER_FL
>> bit which have same behavior as FS_IMMUTABLE_FL, just it would be
>> possible to set it for any user who has granted "write" permission?
> 
> Uh, in unix, write permission is for accessing file data. Modifying access
> permissions etc. is always limited to inode owner (or user with special
> capabilities). So this would be very confusing in Unix permission model.
> 
>> Instead of requiring CAP_LINUX_IMMUTABLE. I see a nice usecase that even
>> ordinary user could be able to mark file as protected against removal or
>> modification (for example some backup data).
> 
> So I don't see us introducing another immutable bit in VFS. Special
> "protected file" bit modifiable by whoever can modify file permissions is
> not really different from clearing write permission bits in mode. So that
> doesn't seem as a terribly useful feature to me. That being said nothing
> really stops individual filesystems from introducing their own inode flags,
> get / set them via ioctl and implement whatever permission checking needed
> with them. After all this is how the flags like IMMUTABLE or APPEND started
> and only later on when they propagated into many filesystems we've lifted
> them into VFS.
> 
> 								Honza


-- 
Chuck Lever

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Thursday 02 January 2025 15:37:50 Jan Kara wrote:
> Hello!
> 
> On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > Few months ago I discussed with Steve that Linux SMB client has some
> > problems during removal of directory which has read-only attribute set.
> > 
> > I was looking what exactly the read-only windows attribute means, how it
> > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > all.
> > 
> > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > two ways how to present some file or directory as read-only. First
> > option is by setting ACL permissions (for particular or all users) to
> > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > Second option is available also for (ex)FAT filesystems (first option via
> > ACL is not possible on (ex)FAT as it does not have ACLs).
> > 
> > First option (ACL) is basically same as clearing all "w" bits in mode
> > and ACL (if present) on Linux. It enforces security permission behavior.
> > Note that if the parent directory grants for user delete child
> > permission then the file can be deleted. This behavior is same for Linux
> > and Windows (on Windows there is separate ACL for delete child, on Linux
> > it is part of directory's write permission).
> > 
> > Second option (Windows read-only attribute) means that the file/dir
> > cannot be opened in write mode, its metadata attribute cannot be changed
> > and the file/dir cannot be deleted at all. But anybody who has
> > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > wants.
> 
> I guess someone with more experience how to fuse together Windows & Linux
> permission semantics should chime in here but here are my thoughts.

It is important to know that read-only attribute is not a permission
(like UNIX mode or POSIX ACL or Windows ACL) and neither it is not a
security mechanism. It is just an attribute which affects execution of
modification operations.

> > Linux filesystems has similar thing to Windows read-only attribute
> > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > which can be set by the "chattr" tool. Seems that the only difference
> > between Windows read-only and Linux immutable is that on Linux only
> > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > it can be anybody who has write ACL.
> > 
> > Now I'm thinking, how should be Windows read-only bit interpreted by
> > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > 
> > 0) Simply ignored. Disadvantage is that over network fs, user would not
> >    be able to do modify or delete such file, even as root.
> > 
> > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> >    network fs it has to be cleared before any write/modify/delete
> >    operation.
> > 
> > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> >    or change it via chmod. Disadvantage is that it mix ACL/mode.
> 
> So this option looks sensible to me. We clear all write permissions in
> file's mode / ACL. For reading that is fully compatible, for mode
> modifications it gets a bit messy (probably I'd suggest to just clear
> FILE_ATTRIBUTE_READONLY on modification) but kind of close.

This this option, there are lot of open questions regarding the
behavior. For example:

How the change mode or change ACL operation executed on Linux should
behave? Should it always clear or set read-only attribute when calling
chmod or setfacl? And what if the user does not have capability or
permission to change mode? Then the user would not be able to clear
read-only attribute (IIRC this attribute can be set/clear by having
just write permission).

And when reading, to which mode / ACL part should be read-only attribute
propagated? Only to user, to all 3 parts (user, group, others), or also
into all ACL entries? For example SMB protocol has extension support for
POSIX ACL, so how it should be handled?

Also how to handle unlink operation? With this option, the user first
would have to call chmod +w and after that would be able to call rm -f.
It it quite non-Linux way that it is needed to have write permission on
the file itself for deleting it. Normally it is needed just write
permission on the parent directory.

I do not have answers for these questions, it seems to be really messy
and not intuitive behavior.

> > 3) Translated to Linux FS_IMMUTABLE_FL. So the user can use lsattr /
> >    chattr to see or change it. Disadvantage is that this bit can be
> >    changed only by root or by CAP_LINUX_IMMUTABLE process.
> > 
> > 4) Exported via some new xattr. User can see or change it. But for
> >    example recursive removal via rm -rf would be failing as rm would not
> >    know about this special new xattr.
> > 
> > In any case, in my opinion, all Linux fs drivers for these filesystems
> > (FAT, exFAT, NTFS, SMB, are there some others?) should handle this
> > windows read-only bit in the same way.
> > 
> > What do you think, what should be the best option?
> > 
> > I have another idea. What about introducing a new FS_IMMUTABLE_USER_FL
> > bit which have same behavior as FS_IMMUTABLE_FL, just it would be
> > possible to set it for any user who has granted "write" permission?
> 
> Uh, in unix, write permission is for accessing file data. Modifying access
> permissions etc. is always limited to inode owner (or user with special
> capabilities). So this would be very confusing in Unix permission model.

(Windows) read-only attribute is not a permission, it is an attribute,
that is probably confusing. Similarly FS_IMMUTABLE_FL is not a
permission if I understand it correctly.

And about attributes, Now I have tested that on Linux it is possible to
change time attributes (utimensat) also for files which are not owned by
calling user/process, it is just enough for caller to have write
permission.

Read-only is similar to those time attributes.

> > Instead of requiring CAP_LINUX_IMMUTABLE. I see a nice usecase that even
> > ordinary user could be able to mark file as protected against removal or
> > modification (for example some backup data).
> 
> So I don't see us introducing another immutable bit in VFS. Special
> "protected file" bit modifiable by whoever can modify file permissions is
> not really different from clearing write permission bits in mode.

It differs for unlink operations. Immutable does not allow to to remove
the file. File without write permission can be removed.

I see benefit in FS_IMMUTABLE_FL for that purpose of backup file, but
FS_IMMUTABLE_FL cannot be used by normal user.

> So that
> doesn't seem as a terribly useful feature to me. That being said nothing
> really stops individual filesystems from introducing their own inode flags,
> get / set them via ioctl and implement whatever permission checking needed
> with them. After all this is how the flags like IMMUTABLE or APPEND started
> and only later on when they propagated into many filesystems we've lifted
> them into VFS.
> 
> 								Honza
> -- 
> Jan Kara <jack@suse.com>
> SUSE Labs, CR

Ok, fair enough.

As there are more filesystems which supports this read-only attribute
feature, it make sense to have same API for all of them. So what is the
better option for exporting it to userspace. Via ioctl or via new xattr?
Or is there any other option?

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Thursday 02 January 2025 10:52:51 Chuck Lever wrote:
> On 1/2/25 9:37 AM, Jan Kara wrote:
> > Hello!
> > 
> > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > Few months ago I discussed with Steve that Linux SMB client has some
> > > problems during removal of directory which has read-only attribute set.
> > > 
> > > I was looking what exactly the read-only windows attribute means, how it
> > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > all.
> > > 
> > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > two ways how to present some file or directory as read-only. First
> > > option is by setting ACL permissions (for particular or all users) to
> > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > Second option is available also for (ex)FAT filesystems (first option via
> > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > 
> > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > Note that if the parent directory grants for user delete child
> > > permission then the file can be deleted. This behavior is same for Linux
> > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > it is part of directory's write permission).
> > > 
> > > Second option (Windows read-only attribute) means that the file/dir
> > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > and the file/dir cannot be deleted at all. But anybody who has
> > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > wants.
> > 
> > I guess someone with more experience how to fuse together Windows & Linux
> > permission semantics should chime in here but here are my thoughts.
> > 
> > > Linux filesystems has similar thing to Windows read-only attribute
> > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > which can be set by the "chattr" tool. Seems that the only difference
> > > between Windows read-only and Linux immutable is that on Linux only
> > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > it can be anybody who has write ACL.
> > > 
> > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > 
> > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > >     be able to do modify or delete such file, even as root.
> > > 
> > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > >     network fs it has to be cleared before any write/modify/delete
> > >     operation.
> > > 
> > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > >     or change it via chmod. Disadvantage is that it mix ACL/mode.
> > 
> > So this option looks sensible to me. We clear all write permissions in
> > file's mode / ACL. For reading that is fully compatible, for mode
> > modifications it gets a bit messy (probably I'd suggest to just clear
> > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> 
> IMO Linux should store the Windows-specific attribute information but
> otherwise ignore it.

Ignoring attribute which affects fs operations for network filesystems
is a problem. And read-only attribute is such example. If Linux network
fs drivers are going to ignore them, then "rm -f" would not work for
files with read-only attribute set.

But attributes which does not affect fs operations like, "system",
"hidden", "archived", "offline", ... and whatever else is used, should
be ignored at all (well, what else such attributes could do if they do
not affect anything?).

> Modifying ACLs based seems like a road to despair.
> Plus there's no ACL representation for OFFLINE and some of the other
> items that we'd like to be able to support.

I mostly agree.

Just OFFLINE is not a permission which grants some access, right? So it
should not be in ACL at all, which controls and grants/deny access.

> If I were king-for-a-day (tm) I would create a system xattr namespace
> just for these items, and provide a VFS/statx API for consumers like
> Samba, ksmbd, and knfsd to set and get these items. Each local
> filesystem can then implement storage with either the xattr or (eg,
> ntfs) can store them directly.
> 
> Semantics like READONLY or IMMUTABLE might be provided in the VFS if
> we care to expose these semantics to POSIX consumers.

For me it sounds like a good idea to export these attributes as xattrs
in system namespace.

Another "big" consumer of them can be wine.

Re: Immutable vs read-only for Windows compatibility

[Christian Brauner]

On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
> On 1/2/25 9:37 AM, Jan Kara wrote:
> > Hello!
> > 
> > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > Few months ago I discussed with Steve that Linux SMB client has some
> > > problems during removal of directory which has read-only attribute set.
> > > 
> > > I was looking what exactly the read-only windows attribute means, how it
> > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > all.
> > > 
> > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > two ways how to present some file or directory as read-only. First
> > > option is by setting ACL permissions (for particular or all users) to
> > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > Second option is available also for (ex)FAT filesystems (first option via
> > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > 
> > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > Note that if the parent directory grants for user delete child
> > > permission then the file can be deleted. This behavior is same for Linux
> > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > it is part of directory's write permission).
> > > 
> > > Second option (Windows read-only attribute) means that the file/dir
> > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > and the file/dir cannot be deleted at all. But anybody who has
> > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > wants.
> > 
> > I guess someone with more experience how to fuse together Windows & Linux
> > permission semantics should chime in here but here are my thoughts.
> > 
> > > Linux filesystems has similar thing to Windows read-only attribute
> > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > which can be set by the "chattr" tool. Seems that the only difference
> > > between Windows read-only and Linux immutable is that on Linux only
> > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > it can be anybody who has write ACL.
> > > 
> > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > 
> > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > >     be able to do modify or delete such file, even as root.
> > > 
> > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > >     network fs it has to be cleared before any write/modify/delete
> > >     operation.
> > > 
> > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > >     or change it via chmod. Disadvantage is that it mix ACL/mode.
> > 
> > So this option looks sensible to me. We clear all write permissions in
> > file's mode / ACL. For reading that is fully compatible, for mode
> > modifications it gets a bit messy (probably I'd suggest to just clear
> > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> 
> IMO Linux should store the Windows-specific attribute information but
> otherwise ignore it. Modifying ACLs based seems like a road to despair.
> Plus there's no ACL representation for OFFLINE and some of the other
> items that we'd like to be able to support.
> 
> 
> If I were king-for-a-day (tm) I would create a system xattr namespace
> just for these items, and provide a VFS/statx API for consumers like
> Samba, ksmbd, and knfsd to set and get these items. Each local
> filesystem can then implement storage with either the xattr or (eg,
> ntfs) can store them directly.

Introducing a new xattr namespace for this wouldn't be a problem imho.
Why would this need a new statx() extension though? Wouldn't the regular
xattr apis to set and get xattrs be enough?

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Saturday 04 January 2025 09:52:44 Christian Brauner wrote:
> On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
> > On 1/2/25 9:37 AM, Jan Kara wrote:
> > > Hello!
> > > 
> > > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > > Few months ago I discussed with Steve that Linux SMB client has some
> > > > problems during removal of directory which has read-only attribute set.
> > > > 
> > > > I was looking what exactly the read-only windows attribute means, how it
> > > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > > all.
> > > > 
> > > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > > two ways how to present some file or directory as read-only. First
> > > > option is by setting ACL permissions (for particular or all users) to
> > > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > > Second option is available also for (ex)FAT filesystems (first option via
> > > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > > 
> > > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > > Note that if the parent directory grants for user delete child
> > > > permission then the file can be deleted. This behavior is same for Linux
> > > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > > it is part of directory's write permission).
> > > > 
> > > > Second option (Windows read-only attribute) means that the file/dir
> > > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > > and the file/dir cannot be deleted at all. But anybody who has
> > > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > > wants.
> > > 
> > > I guess someone with more experience how to fuse together Windows & Linux
> > > permission semantics should chime in here but here are my thoughts.
> > > 
> > > > Linux filesystems has similar thing to Windows read-only attribute
> > > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > > which can be set by the "chattr" tool. Seems that the only difference
> > > > between Windows read-only and Linux immutable is that on Linux only
> > > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > > it can be anybody who has write ACL.
> > > > 
> > > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > > 
> > > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > > >     be able to do modify or delete such file, even as root.
> > > > 
> > > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > > >     network fs it has to be cleared before any write/modify/delete
> > > >     operation.
> > > > 
> > > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > > >     or change it via chmod. Disadvantage is that it mix ACL/mode.
> > > 
> > > So this option looks sensible to me. We clear all write permissions in
> > > file's mode / ACL. For reading that is fully compatible, for mode
> > > modifications it gets a bit messy (probably I'd suggest to just clear
> > > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> > 
> > IMO Linux should store the Windows-specific attribute information but
> > otherwise ignore it. Modifying ACLs based seems like a road to despair.
> > Plus there's no ACL representation for OFFLINE and some of the other
> > items that we'd like to be able to support.
> > 
> > 
> > If I were king-for-a-day (tm) I would create a system xattr namespace
> > just for these items, and provide a VFS/statx API for consumers like
> > Samba, ksmbd, and knfsd to set and get these items. Each local
> > filesystem can then implement storage with either the xattr or (eg,
> > ntfs) can store them directly.
> 
> Introducing a new xattr namespace for this wouldn't be a problem imho.
> Why would this need a new statx() extension though? Wouldn't the regular
> xattr apis to set and get xattrs be enough?

I can imagine that alternative application to "ls -l" or "stat" could
want to show some of those new flags. So to prevent storm of stat and
getxattr syscalls, there can be one syscall (e.g. part of the statx)
which could serve all information. But this is just my imagination...

Anyway, there is already such application and it is wine. WinAPI
functions which are equivalent to Linux stat function are already
returning all those flags.

In the same way flags are part of the basic SMB stat operation, so Samba
or ksmbd would have to always ask for all of them.

Re: Immutable vs read-only for Windows compatibility

[Chuck Lever]

On 1/4/25 3:52 AM, Christian Brauner wrote:
> On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
>> On 1/2/25 9:37 AM, Jan Kara wrote:
>>> Hello!
>>>
>>> On Fri 27-12-24 13:15:08, Pali Rohár wrote:
>>>> Few months ago I discussed with Steve that Linux SMB client has some
>>>> problems during removal of directory which has read-only attribute set.
>>>>
>>>> I was looking what exactly the read-only windows attribute means, how it
>>>> is interpreted by Linux and in my opinion it is wrongly used in Linux at
>>>> all.
>>>>
>>>> Windows filesystems NTFS and ReFS, and also exported over SMB supports
>>>> two ways how to present some file or directory as read-only. First
>>>> option is by setting ACL permissions (for particular or all users) to
>>>> GENERIC_READ-only. Second option is by setting the read-only attribute.
>>>> Second option is available also for (ex)FAT filesystems (first option via
>>>> ACL is not possible on (ex)FAT as it does not have ACLs).
>>>>
>>>> First option (ACL) is basically same as clearing all "w" bits in mode
>>>> and ACL (if present) on Linux. It enforces security permission behavior.
>>>> Note that if the parent directory grants for user delete child
>>>> permission then the file can be deleted. This behavior is same for Linux
>>>> and Windows (on Windows there is separate ACL for delete child, on Linux
>>>> it is part of directory's write permission).
>>>>
>>>> Second option (Windows read-only attribute) means that the file/dir
>>>> cannot be opened in write mode, its metadata attribute cannot be changed
>>>> and the file/dir cannot be deleted at all. But anybody who has
>>>> WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
>>>> wants.
>>>
>>> I guess someone with more experience how to fuse together Windows & Linux
>>> permission semantics should chime in here but here are my thoughts.
>>>
>>>> Linux filesystems has similar thing to Windows read-only attribute
>>>> (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
>>>> which can be set by the "chattr" tool. Seems that the only difference
>>>> between Windows read-only and Linux immutable is that on Linux only
>>>> process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
>>>> it can be anybody who has write ACL.
>>>>
>>>> Now I'm thinking, how should be Windows read-only bit interpreted by
>>>> Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
>>>>
>>>> 0) Simply ignored. Disadvantage is that over network fs, user would not
>>>>      be able to do modify or delete such file, even as root.
>>>>
>>>> 1) Smartly ignored. Meaning that for local fs, it is ignored and for
>>>>      network fs it has to be cleared before any write/modify/delete
>>>>      operation.
>>>>
>>>> 2) Translated to Linux mode/ACL. So the user has some ability to see it
>>>>      or change it via chmod. Disadvantage is that it mix ACL/mode.
>>>
>>> So this option looks sensible to me. We clear all write permissions in
>>> file's mode / ACL. For reading that is fully compatible, for mode
>>> modifications it gets a bit messy (probably I'd suggest to just clear
>>> FILE_ATTRIBUTE_READONLY on modification) but kind of close.
>>
>> IMO Linux should store the Windows-specific attribute information but
>> otherwise ignore it. Modifying ACLs based seems like a road to despair.
>> Plus there's no ACL representation for OFFLINE and some of the other
>> items that we'd like to be able to support.
>>
>>
>> If I were king-for-a-day (tm) I would create a system xattr namespace
>> just for these items, and provide a VFS/statx API for consumers like
>> Samba, ksmbd, and knfsd to set and get these items. Each local
>> filesystem can then implement storage with either the xattr or (eg,
>> ntfs) can store them directly.
> 
> Introducing a new xattr namespace for this wouldn't be a problem imho.
> Why would this need a new statx() extension though? Wouldn't the regular
> xattr apis to set and get xattrs be enough?

My thought was to have a consistent API to access these attributes, and
let the filesystem implementers decide how they want to store them. The
Linux implementation of ntfs, for example, probably wants to store these
on disk in a way that is compatible with the Windows implementation of
NTFS.

A common API would mean that consumers (like NFSD) wouldn't have to know
those details.


-- 
Chuck Lever

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Saturday 04 January 2025 10:30:26 Chuck Lever wrote:
> On 1/4/25 3:52 AM, Christian Brauner wrote:
> > On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
> > > On 1/2/25 9:37 AM, Jan Kara wrote:
> > > > Hello!
> > > > 
> > > > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > > > Few months ago I discussed with Steve that Linux SMB client has some
> > > > > problems during removal of directory which has read-only attribute set.
> > > > > 
> > > > > I was looking what exactly the read-only windows attribute means, how it
> > > > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > > > all.
> > > > > 
> > > > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > > > two ways how to present some file or directory as read-only. First
> > > > > option is by setting ACL permissions (for particular or all users) to
> > > > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > > > Second option is available also for (ex)FAT filesystems (first option via
> > > > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > > > 
> > > > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > > > Note that if the parent directory grants for user delete child
> > > > > permission then the file can be deleted. This behavior is same for Linux
> > > > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > > > it is part of directory's write permission).
> > > > > 
> > > > > Second option (Windows read-only attribute) means that the file/dir
> > > > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > > > and the file/dir cannot be deleted at all. But anybody who has
> > > > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > > > wants.
> > > > 
> > > > I guess someone with more experience how to fuse together Windows & Linux
> > > > permission semantics should chime in here but here are my thoughts.
> > > > 
> > > > > Linux filesystems has similar thing to Windows read-only attribute
> > > > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > > > which can be set by the "chattr" tool. Seems that the only difference
> > > > > between Windows read-only and Linux immutable is that on Linux only
> > > > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > > > it can be anybody who has write ACL.
> > > > > 
> > > > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > > > 
> > > > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > > > >      be able to do modify or delete such file, even as root.
> > > > > 
> > > > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > > > >      network fs it has to be cleared before any write/modify/delete
> > > > >      operation.
> > > > > 
> > > > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > > > >      or change it via chmod. Disadvantage is that it mix ACL/mode.
> > > > 
> > > > So this option looks sensible to me. We clear all write permissions in
> > > > file's mode / ACL. For reading that is fully compatible, for mode
> > > > modifications it gets a bit messy (probably I'd suggest to just clear
> > > > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> > > 
> > > IMO Linux should store the Windows-specific attribute information but
> > > otherwise ignore it. Modifying ACLs based seems like a road to despair.
> > > Plus there's no ACL representation for OFFLINE and some of the other
> > > items that we'd like to be able to support.
> > > 
> > > 
> > > If I were king-for-a-day (tm) I would create a system xattr namespace
> > > just for these items, and provide a VFS/statx API for consumers like
> > > Samba, ksmbd, and knfsd to set and get these items. Each local
> > > filesystem can then implement storage with either the xattr or (eg,
> > > ntfs) can store them directly.
> > 
> > Introducing a new xattr namespace for this wouldn't be a problem imho.
> > Why would this need a new statx() extension though? Wouldn't the regular
> > xattr apis to set and get xattrs be enough?
> 
> My thought was to have a consistent API to access these attributes, and
> let the filesystem implementers decide how they want to store them. The
> Linux implementation of ntfs, for example, probably wants to store these
> on disk in a way that is compatible with the Windows implementation of
> NTFS.
> 
> A common API would mean that consumers (like NFSD) wouldn't have to know
> those details.
> 
> 
> -- 
> Chuck Lever

So, what about introducing new xattrs for every attribute with this pattern?

system.attr.readonly
system.attr.hidden
system.attr.system
system.attr.archive
system.attr.temporary
system.attr.offline
system.attr.not_content_indexed

All those attributes can be set by user, I took names from SMB, which
matches NTFS and which subsets are used by other filesystems like FAT,
exFAT, NFS4, UDF, ...

Every xattr would be in system.attr namespace and would contain either
value 0 or 1 based on that fact if is set or unset. If the filesystem
does not support particular attribute then xattr get/set would return
error that it does not exist.

This would be possible to use by existing userspace getfattr/setfattr
tools and also by knfsd/ksmbd via accessing xattrs directly.

Re: Immutable vs read-only for Windows compatibility

[Chuck Lever]

On 1/14/25 4:10 PM, Pali Rohár wrote:
> On Saturday 04 January 2025 10:30:26 Chuck Lever wrote:
>> On 1/4/25 3:52 AM, Christian Brauner wrote:
>>> On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
>>>> On 1/2/25 9:37 AM, Jan Kara wrote:
>>>>> Hello!
>>>>>
>>>>> On Fri 27-12-24 13:15:08, Pali Rohár wrote:
>>>>>> Few months ago I discussed with Steve that Linux SMB client has some
>>>>>> problems during removal of directory which has read-only attribute set.
>>>>>>
>>>>>> I was looking what exactly the read-only windows attribute means, how it
>>>>>> is interpreted by Linux and in my opinion it is wrongly used in Linux at
>>>>>> all.
>>>>>>
>>>>>> Windows filesystems NTFS and ReFS, and also exported over SMB supports
>>>>>> two ways how to present some file or directory as read-only. First
>>>>>> option is by setting ACL permissions (for particular or all users) to
>>>>>> GENERIC_READ-only. Second option is by setting the read-only attribute.
>>>>>> Second option is available also for (ex)FAT filesystems (first option via
>>>>>> ACL is not possible on (ex)FAT as it does not have ACLs).
>>>>>>
>>>>>> First option (ACL) is basically same as clearing all "w" bits in mode
>>>>>> and ACL (if present) on Linux. It enforces security permission behavior.
>>>>>> Note that if the parent directory grants for user delete child
>>>>>> permission then the file can be deleted. This behavior is same for Linux
>>>>>> and Windows (on Windows there is separate ACL for delete child, on Linux
>>>>>> it is part of directory's write permission).
>>>>>>
>>>>>> Second option (Windows read-only attribute) means that the file/dir
>>>>>> cannot be opened in write mode, its metadata attribute cannot be changed
>>>>>> and the file/dir cannot be deleted at all. But anybody who has
>>>>>> WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
>>>>>> wants.
>>>>>
>>>>> I guess someone with more experience how to fuse together Windows & Linux
>>>>> permission semantics should chime in here but here are my thoughts.
>>>>>
>>>>>> Linux filesystems has similar thing to Windows read-only attribute
>>>>>> (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
>>>>>> which can be set by the "chattr" tool. Seems that the only difference
>>>>>> between Windows read-only and Linux immutable is that on Linux only
>>>>>> process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
>>>>>> it can be anybody who has write ACL.
>>>>>>
>>>>>> Now I'm thinking, how should be Windows read-only bit interpreted by
>>>>>> Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
>>>>>>
>>>>>> 0) Simply ignored. Disadvantage is that over network fs, user would not
>>>>>>       be able to do modify or delete such file, even as root.
>>>>>>
>>>>>> 1) Smartly ignored. Meaning that for local fs, it is ignored and for
>>>>>>       network fs it has to be cleared before any write/modify/delete
>>>>>>       operation.
>>>>>>
>>>>>> 2) Translated to Linux mode/ACL. So the user has some ability to see it
>>>>>>       or change it via chmod. Disadvantage is that it mix ACL/mode.
>>>>>
>>>>> So this option looks sensible to me. We clear all write permissions in
>>>>> file's mode / ACL. For reading that is fully compatible, for mode
>>>>> modifications it gets a bit messy (probably I'd suggest to just clear
>>>>> FILE_ATTRIBUTE_READONLY on modification) but kind of close.
>>>>
>>>> IMO Linux should store the Windows-specific attribute information but
>>>> otherwise ignore it. Modifying ACLs based seems like a road to despair.
>>>> Plus there's no ACL representation for OFFLINE and some of the other
>>>> items that we'd like to be able to support.
>>>>
>>>>
>>>> If I were king-for-a-day (tm) I would create a system xattr namespace
>>>> just for these items, and provide a VFS/statx API for consumers like
>>>> Samba, ksmbd, and knfsd to set and get these items. Each local
>>>> filesystem can then implement storage with either the xattr or (eg,
>>>> ntfs) can store them directly.
>>>
>>> Introducing a new xattr namespace for this wouldn't be a problem imho.
>>> Why would this need a new statx() extension though? Wouldn't the regular
>>> xattr apis to set and get xattrs be enough?
>>
>> My thought was to have a consistent API to access these attributes, and
>> let the filesystem implementers decide how they want to store them. The
>> Linux implementation of ntfs, for example, probably wants to store these
>> on disk in a way that is compatible with the Windows implementation of
>> NTFS.
>>
>> A common API would mean that consumers (like NFSD) wouldn't have to know
>> those details.
>>
>>
>> -- 
>> Chuck Lever
> 
> So, what about introducing new xattrs for every attribute with this pattern?
> 
> system.attr.readonly
> system.attr.hidden
> system.attr.system
> system.attr.archive
> system.attr.temporary
> system.attr.offline
> system.attr.not_content_indexed

Yes, all of them could be stored as xattrs for file systems that do
not already support these attributes.

But I think we don't want to expose them directly to users, however.
Some file systems, like NTFS, might want to store these on-disk in a way
that is compatible with Windows.

So I think we want to create statx APIs for consumers like user space
and knfsd, who do not care to know the specifics of how this information
is stored by each file system.

The xattrs would be for file systems that do not already have a way to
represent this information in their on-disk format.


> All those attributes can be set by user, I took names from SMB, which
> matches NTFS and which subsets are used by other filesystems like FAT,
> exFAT, NFS4, UDF, ...
> 
> Every xattr would be in system.attr namespace and would contain either
> value 0 or 1 based on that fact if is set or unset. If the filesystem
> does not support particular attribute then xattr get/set would return
> error that it does not exist.

Or, if the xattr exists, then that means the equivalent Windows
attribute is asserted; and if it does not, the equivalent Windows
attribute is clear. But again, I think each file system should be
able to choose how they implement these, and that implementation is
then hidden by statx.


> This would be possible to use by existing userspace getfattr/setfattr
> tools and also by knfsd/ksmbd via accessing xattrs directly.


-- 
Chuck Lever

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Tuesday 14 January 2025 16:44:55 Chuck Lever wrote:
> On 1/14/25 4:10 PM, Pali Rohár wrote:
> > On Saturday 04 January 2025 10:30:26 Chuck Lever wrote:
> > > On 1/4/25 3:52 AM, Christian Brauner wrote:
> > > > On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
> > > > > On 1/2/25 9:37 AM, Jan Kara wrote:
> > > > > > Hello!
> > > > > > 
> > > > > > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > > > > > Few months ago I discussed with Steve that Linux SMB client has some
> > > > > > > problems during removal of directory which has read-only attribute set.
> > > > > > > 
> > > > > > > I was looking what exactly the read-only windows attribute means, how it
> > > > > > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > > > > > all.
> > > > > > > 
> > > > > > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > > > > > two ways how to present some file or directory as read-only. First
> > > > > > > option is by setting ACL permissions (for particular or all users) to
> > > > > > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > > > > > Second option is available also for (ex)FAT filesystems (first option via
> > > > > > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > > > > > 
> > > > > > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > > > > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > > > > > Note that if the parent directory grants for user delete child
> > > > > > > permission then the file can be deleted. This behavior is same for Linux
> > > > > > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > > > > > it is part of directory's write permission).
> > > > > > > 
> > > > > > > Second option (Windows read-only attribute) means that the file/dir
> > > > > > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > > > > > and the file/dir cannot be deleted at all. But anybody who has
> > > > > > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > > > > > wants.
> > > > > > 
> > > > > > I guess someone with more experience how to fuse together Windows & Linux
> > > > > > permission semantics should chime in here but here are my thoughts.
> > > > > > 
> > > > > > > Linux filesystems has similar thing to Windows read-only attribute
> > > > > > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > > > > > which can be set by the "chattr" tool. Seems that the only difference
> > > > > > > between Windows read-only and Linux immutable is that on Linux only
> > > > > > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > > > > > it can be anybody who has write ACL.
> > > > > > > 
> > > > > > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > > > > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > > > > > 
> > > > > > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > > > > > >       be able to do modify or delete such file, even as root.
> > > > > > > 
> > > > > > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > > > > > >       network fs it has to be cleared before any write/modify/delete
> > > > > > >       operation.
> > > > > > > 
> > > > > > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > > > > > >       or change it via chmod. Disadvantage is that it mix ACL/mode.
> > > > > > 
> > > > > > So this option looks sensible to me. We clear all write permissions in
> > > > > > file's mode / ACL. For reading that is fully compatible, for mode
> > > > > > modifications it gets a bit messy (probably I'd suggest to just clear
> > > > > > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> > > > > 
> > > > > IMO Linux should store the Windows-specific attribute information but
> > > > > otherwise ignore it. Modifying ACLs based seems like a road to despair.
> > > > > Plus there's no ACL representation for OFFLINE and some of the other
> > > > > items that we'd like to be able to support.
> > > > > 
> > > > > 
> > > > > If I were king-for-a-day (tm) I would create a system xattr namespace
> > > > > just for these items, and provide a VFS/statx API for consumers like
> > > > > Samba, ksmbd, and knfsd to set and get these items. Each local
> > > > > filesystem can then implement storage with either the xattr or (eg,
> > > > > ntfs) can store them directly.
> > > > 
> > > > Introducing a new xattr namespace for this wouldn't be a problem imho.
> > > > Why would this need a new statx() extension though? Wouldn't the regular
> > > > xattr apis to set and get xattrs be enough?
> > > 
> > > My thought was to have a consistent API to access these attributes, and
> > > let the filesystem implementers decide how they want to store them. The
> > > Linux implementation of ntfs, for example, probably wants to store these
> > > on disk in a way that is compatible with the Windows implementation of
> > > NTFS.
> > > 
> > > A common API would mean that consumers (like NFSD) wouldn't have to know
> > > those details.
> > > 
> > > 
> > > -- 
> > > Chuck Lever
> > 
> > So, what about introducing new xattrs for every attribute with this pattern?
> > 
> > system.attr.readonly
> > system.attr.hidden
> > system.attr.system
> > system.attr.archive
> > system.attr.temporary
> > system.attr.offline
> > system.attr.not_content_indexed
> 
> Yes, all of them could be stored as xattrs for file systems that do
> not already support these attributes.
> 
> But I think we don't want to expose them directly to users, however.
> Some file systems, like NTFS, might want to store these on-disk in a way
> that is compatible with Windows.
> 
> So I think we want to create statx APIs for consumers like user space
> and knfsd, who do not care to know the specifics of how this information
> is stored by each file system.
> 
> The xattrs would be for file systems that do not already have a way to
> represent this information in their on-disk format.
> 
> 
> > All those attributes can be set by user, I took names from SMB, which
> > matches NTFS and which subsets are used by other filesystems like FAT,
> > exFAT, NFS4, UDF, ...
> > 
> > Every xattr would be in system.attr namespace and would contain either
> > value 0 or 1 based on that fact if is set or unset. If the filesystem
> > does not support particular attribute then xattr get/set would return
> > error that it does not exist.
> 
> Or, if the xattr exists, then that means the equivalent Windows
> attribute is asserted; and if it does not, the equivalent Windows
> attribute is clear. But again, I think each file system should be
> able to choose how they implement these, and that implementation is
> then hidden by statx.
> 
> 
> > This would be possible to use by existing userspace getfattr/setfattr
> > tools and also by knfsd/ksmbd via accessing xattrs directly.
> 
> 
> -- 
> Chuck Lever

With this xattr scheme I mean that API would be xattr between fs and
vfs/userspace/knfsd/smbd. So NTFS would take that xattr request and
translate it to its own NTFS attributes. Other non-windows fs stores
them as xattrs.

I think that you understood it quite differently as I thought because
you are proposing statx() API for fetching them. I thought that they
would be exported via getxattr()/setxattr().

This is also a good idea, just would need to write new userspace tools
for setting and gettting... And there is still one important thing. How
to modify those attribute? Because statx() is GET-only API.

Re: Immutable vs read-only for Windows compatibility

[Darrick J. Wong]

On Tue, Jan 14, 2025 at 10:53:50PM +0100, Pali Rohár wrote:
> On Tuesday 14 January 2025 16:44:55 Chuck Lever wrote:
> > On 1/14/25 4:10 PM, Pali Rohár wrote:
> > > On Saturday 04 January 2025 10:30:26 Chuck Lever wrote:
> > > > On 1/4/25 3:52 AM, Christian Brauner wrote:
> > > > > On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
> > > > > > On 1/2/25 9:37 AM, Jan Kara wrote:
> > > > > > > Hello!
> > > > > > > 
> > > > > > > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > > > > > > Few months ago I discussed with Steve that Linux SMB client has some
> > > > > > > > problems during removal of directory which has read-only attribute set.
> > > > > > > > 
> > > > > > > > I was looking what exactly the read-only windows attribute means, how it
> > > > > > > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > > > > > > all.
> > > > > > > > 
> > > > > > > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > > > > > > two ways how to present some file or directory as read-only. First
> > > > > > > > option is by setting ACL permissions (for particular or all users) to
> > > > > > > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > > > > > > Second option is available also for (ex)FAT filesystems (first option via
> > > > > > > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > > > > > > 
> > > > > > > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > > > > > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > > > > > > Note that if the parent directory grants for user delete child
> > > > > > > > permission then the file can be deleted. This behavior is same for Linux
> > > > > > > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > > > > > > it is part of directory's write permission).
> > > > > > > > 
> > > > > > > > Second option (Windows read-only attribute) means that the file/dir
> > > > > > > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > > > > > > and the file/dir cannot be deleted at all. But anybody who has
> > > > > > > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > > > > > > wants.
> > > > > > > 
> > > > > > > I guess someone with more experience how to fuse together Windows & Linux
> > > > > > > permission semantics should chime in here but here are my thoughts.
> > > > > > > 
> > > > > > > > Linux filesystems has similar thing to Windows read-only attribute
> > > > > > > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > > > > > > which can be set by the "chattr" tool. Seems that the only difference
> > > > > > > > between Windows read-only and Linux immutable is that on Linux only
> > > > > > > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > > > > > > it can be anybody who has write ACL.
> > > > > > > > 
> > > > > > > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > > > > > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > > > > > > 
> > > > > > > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > > > > > > >       be able to do modify or delete such file, even as root.
> > > > > > > > 
> > > > > > > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > > > > > > >       network fs it has to be cleared before any write/modify/delete
> > > > > > > >       operation.
> > > > > > > > 
> > > > > > > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > > > > > > >       or change it via chmod. Disadvantage is that it mix ACL/mode.
> > > > > > > 
> > > > > > > So this option looks sensible to me. We clear all write permissions in
> > > > > > > file's mode / ACL. For reading that is fully compatible, for mode
> > > > > > > modifications it gets a bit messy (probably I'd suggest to just clear
> > > > > > > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> > > > > > 
> > > > > > IMO Linux should store the Windows-specific attribute information but
> > > > > > otherwise ignore it. Modifying ACLs based seems like a road to despair.
> > > > > > Plus there's no ACL representation for OFFLINE and some of the other
> > > > > > items that we'd like to be able to support.
> > > > > > 
> > > > > > 
> > > > > > If I were king-for-a-day (tm) I would create a system xattr namespace
> > > > > > just for these items, and provide a VFS/statx API for consumers like
> > > > > > Samba, ksmbd, and knfsd to set and get these items. Each local
> > > > > > filesystem can then implement storage with either the xattr or (eg,
> > > > > > ntfs) can store them directly.
> > > > > 
> > > > > Introducing a new xattr namespace for this wouldn't be a problem imho.
> > > > > Why would this need a new statx() extension though? Wouldn't the regular
> > > > > xattr apis to set and get xattrs be enough?
> > > > 
> > > > My thought was to have a consistent API to access these attributes, and
> > > > let the filesystem implementers decide how they want to store them. The
> > > > Linux implementation of ntfs, for example, probably wants to store these
> > > > on disk in a way that is compatible with the Windows implementation of
> > > > NTFS.
> > > > 
> > > > A common API would mean that consumers (like NFSD) wouldn't have to know
> > > > those details.
> > > > 
> > > > 
> > > > -- 
> > > > Chuck Lever
> > > 
> > > So, what about introducing new xattrs for every attribute with this pattern?
> > > 
> > > system.attr.readonly
> > > system.attr.hidden
> > > system.attr.system
> > > system.attr.archive
> > > system.attr.temporary
> > > system.attr.offline
> > > system.attr.not_content_indexed
> > 
> > Yes, all of them could be stored as xattrs for file systems that do
> > not already support these attributes.
> > 
> > But I think we don't want to expose them directly to users, however.
> > Some file systems, like NTFS, might want to store these on-disk in a way
> > that is compatible with Windows.
> > 
> > So I think we want to create statx APIs for consumers like user space
> > and knfsd, who do not care to know the specifics of how this information
> > is stored by each file system.
> > 
> > The xattrs would be for file systems that do not already have a way to
> > represent this information in their on-disk format.
> > 
> > 
> > > All those attributes can be set by user, I took names from SMB, which
> > > matches NTFS and which subsets are used by other filesystems like FAT,
> > > exFAT, NFS4, UDF, ...
> > > 
> > > Every xattr would be in system.attr namespace and would contain either
> > > value 0 or 1 based on that fact if is set or unset. If the filesystem
> > > does not support particular attribute then xattr get/set would return
> > > error that it does not exist.
> > 
> > Or, if the xattr exists, then that means the equivalent Windows
> > attribute is asserted; and if it does not, the equivalent Windows
> > attribute is clear. But again, I think each file system should be
> > able to choose how they implement these, and that implementation is
> > then hidden by statx.
> > 
> > 
> > > This would be possible to use by existing userspace getfattr/setfattr
> > > tools and also by knfsd/ksmbd via accessing xattrs directly.
> > 
> > 
> > -- 
> > Chuck Lever
> 
> With this xattr scheme I mean that API would be xattr between fs and
> vfs/userspace/knfsd/smbd. So NTFS would take that xattr request and
> translate it to its own NTFS attributes. Other non-windows fs stores
> them as xattrs.
> 
> I think that you understood it quite differently as I thought because
> you are proposing statx() API for fetching them. I thought that they
> would be exported via getxattr()/setxattr().
> 
> This is also a good idea, just would need to write new userspace tools
> for setting and gettting... And there is still one important thing. How
> to modify those attribute? Because statx() is GET-only API.

statx/FS_IOC_FSGETXATTR to retrieve and FS_IOC_FSSETXATTR to set?

Last time this came up hch said no to random magic xattrs that every
filesystem then has to filter.

--D

Re: Immutable vs read-only for Windows compatibility

[ronnie sahlberg]

On Wed, 15 Jan 2025 at 07:54, Pali Rohár <pali@kernel.org> wrote:
>
> On Tuesday 14 January 2025 16:44:55 Chuck Lever wrote:
> > On 1/14/25 4:10 PM, Pali Rohár wrote:
> > > On Saturday 04 January 2025 10:30:26 Chuck Lever wrote:
> > > > On 1/4/25 3:52 AM, Christian Brauner wrote:
> > > > > On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
> > > > > > On 1/2/25 9:37 AM, Jan Kara wrote:
> > > > > > > Hello!
> > > > > > >
> > > > > > > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > > > > > > Few months ago I discussed with Steve that Linux SMB client has some
> > > > > > > > problems during removal of directory which has read-only attribute set.
> > > > > > > >
> > > > > > > > I was looking what exactly the read-only windows attribute means, how it
> > > > > > > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > > > > > > all.
> > > > > > > >
> > > > > > > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > > > > > > two ways how to present some file or directory as read-only. First
> > > > > > > > option is by setting ACL permissions (for particular or all users) to
> > > > > > > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > > > > > > Second option is available also for (ex)FAT filesystems (first option via
> > > > > > > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > > > > > >
> > > > > > > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > > > > > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > > > > > > Note that if the parent directory grants for user delete child
> > > > > > > > permission then the file can be deleted. This behavior is same for Linux
> > > > > > > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > > > > > > it is part of directory's write permission).
> > > > > > > >
> > > > > > > > Second option (Windows read-only attribute) means that the file/dir
> > > > > > > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > > > > > > and the file/dir cannot be deleted at all. But anybody who has
> > > > > > > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > > > > > > wants.
> > > > > > >
> > > > > > > I guess someone with more experience how to fuse together Windows & Linux
> > > > > > > permission semantics should chime in here but here are my thoughts.
> > > > > > >
> > > > > > > > Linux filesystems has similar thing to Windows read-only attribute
> > > > > > > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > > > > > > which can be set by the "chattr" tool. Seems that the only difference
> > > > > > > > between Windows read-only and Linux immutable is that on Linux only
> > > > > > > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > > > > > > it can be anybody who has write ACL.
> > > > > > > >
> > > > > > > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > > > > > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > > > > > >
> > > > > > > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > > > > > > >       be able to do modify or delete such file, even as root.
> > > > > > > >
> > > > > > > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > > > > > > >       network fs it has to be cleared before any write/modify/delete
> > > > > > > >       operation.
> > > > > > > >
> > > > > > > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > > > > > > >       or change it via chmod. Disadvantage is that it mix ACL/mode.
> > > > > > >
> > > > > > > So this option looks sensible to me. We clear all write permissions in
> > > > > > > file's mode / ACL. For reading that is fully compatible, for mode
> > > > > > > modifications it gets a bit messy (probably I'd suggest to just clear
> > > > > > > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> > > > > >
> > > > > > IMO Linux should store the Windows-specific attribute information but
> > > > > > otherwise ignore it. Modifying ACLs based seems like a road to despair.
> > > > > > Plus there's no ACL representation for OFFLINE and some of the other
> > > > > > items that we'd like to be able to support.
> > > > > >
> > > > > >
> > > > > > If I were king-for-a-day (tm) I would create a system xattr namespace
> > > > > > just for these items, and provide a VFS/statx API for consumers like
> > > > > > Samba, ksmbd, and knfsd to set and get these items. Each local
> > > > > > filesystem can then implement storage with either the xattr or (eg,
> > > > > > ntfs) can store them directly.
> > > > >
> > > > > Introducing a new xattr namespace for this wouldn't be a problem imho.
> > > > > Why would this need a new statx() extension though? Wouldn't the regular
> > > > > xattr apis to set and get xattrs be enough?
> > > >
> > > > My thought was to have a consistent API to access these attributes, and
> > > > let the filesystem implementers decide how they want to store them. The
> > > > Linux implementation of ntfs, for example, probably wants to store these
> > > > on disk in a way that is compatible with the Windows implementation of
> > > > NTFS.
> > > >
> > > > A common API would mean that consumers (like NFSD) wouldn't have to know
> > > > those details.
> > > >
> > > >
> > > > --
> > > > Chuck Lever
> > >
> > > So, what about introducing new xattrs for every attribute with this pattern?
> > >
> > > system.attr.readonly
> > > system.attr.hidden
> > > system.attr.system
> > > system.attr.archive
> > > system.attr.temporary
> > > system.attr.offline
> > > system.attr.not_content_indexed
> >
> > Yes, all of them could be stored as xattrs for file systems that do
> > not already support these attributes.
> >
> > But I think we don't want to expose them directly to users, however.
> > Some file systems, like NTFS, might want to store these on-disk in a way
> > that is compatible with Windows.
> >
> > So I think we want to create statx APIs for consumers like user space
> > and knfsd, who do not care to know the specifics of how this information
> > is stored by each file system.
> >
> > The xattrs would be for file systems that do not already have a way to
> > represent this information in their on-disk format.
> >
> >
> > > All those attributes can be set by user, I took names from SMB, which
> > > matches NTFS and which subsets are used by other filesystems like FAT,
> > > exFAT, NFS4, UDF, ...
> > >
> > > Every xattr would be in system.attr namespace and would contain either
> > > value 0 or 1 based on that fact if is set or unset. If the filesystem
> > > does not support particular attribute then xattr get/set would return
> > > error that it does not exist.
> >
> > Or, if the xattr exists, then that means the equivalent Windows
> > attribute is asserted; and if it does not, the equivalent Windows
> > attribute is clear. But again, I think each file system should be
> > able to choose how they implement these, and that implementation is
> > then hidden by statx.
> >
> >
> > > This would be possible to use by existing userspace getfattr/setfattr
> > > tools and also by knfsd/ksmbd via accessing xattrs directly.
> >
> >
> > --
> > Chuck Lever
>
> With this xattr scheme I mean that API would be xattr between fs and
> vfs/userspace/knfsd/smbd. So NTFS would take that xattr request and
> translate it to its own NTFS attributes. Other non-windows fs stores
> them as xattrs.

I am not sure if for the cifs client doing this by emulating xattrs.
I have bad memories of the emulated xattrs.

What about extending ioctl(FS_IOC_GETFLAGS)? There are plenty of spare
flags there
and you even have NTFS.readonly ~= Linux.immutable so ... :-)

To me to feels like the flags you want to implement would fit
"somewhat naturally" there.

regards
ronnie s

>
> I think that you understood it quite differently as I thought because
> you are proposing statx() API for fetching them. I thought that they
> would be exported via getxattr()/setxattr().
>
> This is also a good idea, just would need to write new userspace tools
> for setting and gettting... And there is still one important thing. How
> to modify those attribute? Because statx() is GET-only API.
>

Re: Immutable vs read-only for Windows compatibility

[Dave Chinner]

On Tue, Jan 14, 2025 at 04:44:55PM -0500, Chuck Lever wrote:
> On 1/14/25 4:10 PM, Pali Rohár wrote:
> > > My thought was to have a consistent API to access these attributes, and
> > > let the filesystem implementers decide how they want to store them. The
> > > Linux implementation of ntfs, for example, probably wants to store these
> > > on disk in a way that is compatible with the Windows implementation of
> > > NTFS.
> > > 
> > > A common API would mean that consumers (like NFSD) wouldn't have to know
> > > those details.
> > > 
> > > 
> > > -- 
> > > Chuck Lever
> > 
> > So, what about introducing new xattrs for every attribute with this pattern?
> > 
> > system.attr.readonly
> > system.attr.hidden
> > system.attr.system
> > system.attr.archive
> > system.attr.temporary
> > system.attr.offline
> > system.attr.not_content_indexed

"attr" is a poor choice for an attribute class (yes, naming is
hard...). It's a windows file attribute class, the name should
reflect that.

However, my main problem with this approach is that it will be
pretty nasty in terms of performance regressions. xattr lookup is
*much* more expensive than reading a field out of the inode itself.

If you want an example of the cost of how a single xattr per file
can affect the performance of CIFS servers, go run dbench (a CIFS
workload simulator) with and without xattrs. The difference in
performance storing a single xattr per file is pretty stark, and it
only gets worse as we add more xattrs. i.e. xattrs like this will
have significant performance implications for all file create,
lookup/stat and unlink operations.

IOWs, If this information is going to be stored as an xattr, it
needs to be a single xattr with a well defined bit field as it's
value (i.e. one bit per attribute). The VFS inode can then cache
that bitfield with minimal addition overhead during the first
lookup/creation/modification for the purpose of fast, low overhead,
statx() operation.

> Yes, all of them could be stored as xattrs for file systems that do
> not already support these attributes.
> 
> But I think we don't want to expose them directly to users, however.
> Some file systems, like NTFS, might want to store these on-disk in a way
> that is compatible with Windows.
> 
> So I think we want to create statx APIs for consumers like user space
> and knfsd, who do not care to know the specifics of how this information
> is stored by each file system.
> 
> The xattrs would be for file systems that do not already have a way to
> represent this information in their on-disk format.

Even the filesystems that store this information natively should
support the xattr interface - they just return the native
information in the xattr format, and then every application has a
common way to change these attributes. (i.e. change the xattr to
modify the attributes, statx to efficiently sample them.

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: Immutable vs read-only for Windows compatibility

[ronnie sahlberg]

On Wed, 15 Jan 2025 at 09:32, Dave Chinner <david@fromorbit.com> wrote:
>
> On Tue, Jan 14, 2025 at 04:44:55PM -0500, Chuck Lever wrote:
> > On 1/14/25 4:10 PM, Pali Rohár wrote:
> > > > My thought was to have a consistent API to access these attributes, and
> > > > let the filesystem implementers decide how they want to store them. The
> > > > Linux implementation of ntfs, for example, probably wants to store these
> > > > on disk in a way that is compatible with the Windows implementation of
> > > > NTFS.
> > > >
> > > > A common API would mean that consumers (like NFSD) wouldn't have to know
> > > > those details.
> > > >
> > > >
> > > > --
> > > > Chuck Lever
> > >
> > > So, what about introducing new xattrs for every attribute with this pattern?
> > >
> > > system.attr.readonly
> > > system.attr.hidden
> > > system.attr.system
> > > system.attr.archive
> > > system.attr.temporary
> > > system.attr.offline
> > > system.attr.not_content_indexed
>
> "attr" is a poor choice for an attribute class (yes, naming is
> hard...). It's a windows file attribute class, the name should
> reflect that.
>
> However, my main problem with this approach is that it will be
> pretty nasty in terms of performance regressions. xattr lookup is
> *much* more expensive than reading a field out of the inode itself.
>
> If you want an example of the cost of how a single xattr per file
> can affect the performance of CIFS servers, go run dbench (a CIFS
> workload simulator) with and without xattrs. The difference in
> performance storing a single xattr per file is pretty stark, and it
> only gets worse as we add more xattrs. i.e. xattrs like this will
> have significant performance implications for all file create,
> lookup/stat and unlink operations.
>
> IOWs, If this information is going to be stored as an xattr, it
> needs to be a single xattr with a well defined bit field as it's
> value (i.e. one bit per attribute). The VFS inode can then cache
> that bitfield with minimal addition overhead during the first
> lookup/creation/modification for the purpose of fast, low overhead,
> statx() operation.

For this use case I don't think he means to store them on the cifs
server as xattr
(or case-insensitive extended attributes as cifs does).
They can already be read/written using SMB2 GetInfo/SetInfo commands.

What I think he means is to read these attributes using SMB2 GetInfo
but then present this to the application via a synthetic xattr.
Application reads a magic xattr and then the driver just makes it up based on
other information it has. (cifs does this for other things already afaik)

Correct me if I am wrong Pali, but you mean translate the SMB2 attribute field
into a magic xattr?  But that means it is not storage of the
attributes anymore but rather
the API for applications to read these attributes.

>
> > Yes, all of them could be stored as xattrs for file systems that do
> > not already support these attributes.
> >
> > But I think we don't want to expose them directly to users, however.
> > Some file systems, like NTFS, might want to store these on-disk in a way
> > that is compatible with Windows.
> >
> > So I think we want to create statx APIs for consumers like user space
> > and knfsd, who do not care to know the specifics of how this information
> > is stored by each file system.
> >
> > The xattrs would be for file systems that do not already have a way to
> > represent this information in their on-disk format.
>
> Even the filesystems that store this information natively should
> support the xattr interface - they just return the native
> information in the xattr format, and then every application has a
> common way to change these attributes. (i.e. change the xattr to
> modify the attributes, statx to efficiently sample them.
>
> -Dave.
> --
> Dave Chinner
> david@fromorbit.com
>

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Wednesday 15 January 2025 09:29:14 ronnie sahlberg wrote:
> On Wed, 15 Jan 2025 at 07:54, Pali Rohár <pali@kernel.org> wrote:
> >
> > On Tuesday 14 January 2025 16:44:55 Chuck Lever wrote:
> > > On 1/14/25 4:10 PM, Pali Rohár wrote:
> > > > On Saturday 04 January 2025 10:30:26 Chuck Lever wrote:
> > > > > On 1/4/25 3:52 AM, Christian Brauner wrote:
> > > > > > On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
> > > > > > > On 1/2/25 9:37 AM, Jan Kara wrote:
> > > > > > > > Hello!
> > > > > > > >
> > > > > > > > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > > > > > > > Few months ago I discussed with Steve that Linux SMB client has some
> > > > > > > > > problems during removal of directory which has read-only attribute set.
> > > > > > > > >
> > > > > > > > > I was looking what exactly the read-only windows attribute means, how it
> > > > > > > > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > > > > > > > all.
> > > > > > > > >
> > > > > > > > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > > > > > > > two ways how to present some file or directory as read-only. First
> > > > > > > > > option is by setting ACL permissions (for particular or all users) to
> > > > > > > > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > > > > > > > Second option is available also for (ex)FAT filesystems (first option via
> > > > > > > > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > > > > > > >
> > > > > > > > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > > > > > > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > > > > > > > Note that if the parent directory grants for user delete child
> > > > > > > > > permission then the file can be deleted. This behavior is same for Linux
> > > > > > > > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > > > > > > > it is part of directory's write permission).
> > > > > > > > >
> > > > > > > > > Second option (Windows read-only attribute) means that the file/dir
> > > > > > > > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > > > > > > > and the file/dir cannot be deleted at all. But anybody who has
> > > > > > > > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > > > > > > > wants.
> > > > > > > >
> > > > > > > > I guess someone with more experience how to fuse together Windows & Linux
> > > > > > > > permission semantics should chime in here but here are my thoughts.
> > > > > > > >
> > > > > > > > > Linux filesystems has similar thing to Windows read-only attribute
> > > > > > > > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > > > > > > > which can be set by the "chattr" tool. Seems that the only difference
> > > > > > > > > between Windows read-only and Linux immutable is that on Linux only
> > > > > > > > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > > > > > > > it can be anybody who has write ACL.
> > > > > > > > >
> > > > > > > > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > > > > > > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > > > > > > >
> > > > > > > > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > > > > > > > >       be able to do modify or delete such file, even as root.
> > > > > > > > >
> > > > > > > > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > > > > > > > >       network fs it has to be cleared before any write/modify/delete
> > > > > > > > >       operation.
> > > > > > > > >
> > > > > > > > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > > > > > > > >       or change it via chmod. Disadvantage is that it mix ACL/mode.
> > > > > > > >
> > > > > > > > So this option looks sensible to me. We clear all write permissions in
> > > > > > > > file's mode / ACL. For reading that is fully compatible, for mode
> > > > > > > > modifications it gets a bit messy (probably I'd suggest to just clear
> > > > > > > > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> > > > > > >
> > > > > > > IMO Linux should store the Windows-specific attribute information but
> > > > > > > otherwise ignore it. Modifying ACLs based seems like a road to despair.
> > > > > > > Plus there's no ACL representation for OFFLINE and some of the other
> > > > > > > items that we'd like to be able to support.
> > > > > > >
> > > > > > >
> > > > > > > If I were king-for-a-day (tm) I would create a system xattr namespace
> > > > > > > just for these items, and provide a VFS/statx API for consumers like
> > > > > > > Samba, ksmbd, and knfsd to set and get these items. Each local
> > > > > > > filesystem can then implement storage with either the xattr or (eg,
> > > > > > > ntfs) can store them directly.
> > > > > >
> > > > > > Introducing a new xattr namespace for this wouldn't be a problem imho.
> > > > > > Why would this need a new statx() extension though? Wouldn't the regular
> > > > > > xattr apis to set and get xattrs be enough?
> > > > >
> > > > > My thought was to have a consistent API to access these attributes, and
> > > > > let the filesystem implementers decide how they want to store them. The
> > > > > Linux implementation of ntfs, for example, probably wants to store these
> > > > > on disk in a way that is compatible with the Windows implementation of
> > > > > NTFS.
> > > > >
> > > > > A common API would mean that consumers (like NFSD) wouldn't have to know
> > > > > those details.
> > > > >
> > > > >
> > > > > --
> > > > > Chuck Lever
> > > >
> > > > So, what about introducing new xattrs for every attribute with this pattern?
> > > >
> > > > system.attr.readonly
> > > > system.attr.hidden
> > > > system.attr.system
> > > > system.attr.archive
> > > > system.attr.temporary
> > > > system.attr.offline
> > > > system.attr.not_content_indexed
> > >
> > > Yes, all of them could be stored as xattrs for file systems that do
> > > not already support these attributes.
> > >
> > > But I think we don't want to expose them directly to users, however.
> > > Some file systems, like NTFS, might want to store these on-disk in a way
> > > that is compatible with Windows.
> > >
> > > So I think we want to create statx APIs for consumers like user space
> > > and knfsd, who do not care to know the specifics of how this information
> > > is stored by each file system.
> > >
> > > The xattrs would be for file systems that do not already have a way to
> > > represent this information in their on-disk format.
> > >
> > >
> > > > All those attributes can be set by user, I took names from SMB, which
> > > > matches NTFS and which subsets are used by other filesystems like FAT,
> > > > exFAT, NFS4, UDF, ...
> > > >
> > > > Every xattr would be in system.attr namespace and would contain either
> > > > value 0 or 1 based on that fact if is set or unset. If the filesystem
> > > > does not support particular attribute then xattr get/set would return
> > > > error that it does not exist.
> > >
> > > Or, if the xattr exists, then that means the equivalent Windows
> > > attribute is asserted; and if it does not, the equivalent Windows
> > > attribute is clear. But again, I think each file system should be
> > > able to choose how they implement these, and that implementation is
> > > then hidden by statx.
> > >
> > >
> > > > This would be possible to use by existing userspace getfattr/setfattr
> > > > tools and also by knfsd/ksmbd via accessing xattrs directly.
> > >
> > >
> > > --
> > > Chuck Lever
> >
> > With this xattr scheme I mean that API would be xattr between fs and
> > vfs/userspace/knfsd/smbd. So NTFS would take that xattr request and
> > translate it to its own NTFS attributes. Other non-windows fs stores
> > them as xattrs.
> 
> I am not sure if for the cifs client doing this by emulating xattrs.
> I have bad memories of the emulated xattrs.
> 
> What about extending ioctl(FS_IOC_GETFLAGS)? There are plenty of spare
> flags there

Are FS_IOC_GETFLAGS/FS_IOC_SETFLAGS flags preserved across regular
"cp -a" or "rsync -someflag" commands? I'm just worried to not invent
new way how to get or set flags which would not be understood by
existing backup or regular "copy" applications. Because the worst thing
which can happen is adding new API which nobody would use and basically
will not gain those benefits which should have them... Like if I move or
copy file from one filesystem to another to not loose all those
attributes.

> and you even have NTFS.readonly ~= Linux.immutable so ... :-)

I know it :-) I have not explicitly written it in the email, but put
this information into one of the options what can be possible to do.
The bad thing about this option for remote filesystems is that
Linux.immutable can be cleared only by root (or process which privilege
which nobody does not normally have), but on Windows system (and also
when exported over SMB) it can be cleared by anybody who can modify file
(based on ACL). So with this Linux will start blocking to do some
operation with file, which Windows fully allows. And this very user
unfriendly, specially if also wine wants to benefit from it, as wine
normally is not going to be run under root (or special capabilities).

> To me to feels like the flags you want to implement would fit
> "somewhat naturally" there.

So thank you and others for this FS_IOC_GETFLAGS opinion. Maybe this
looks like a better solution?

> regards
> ronnie s
> 
> >
> > I think that you understood it quite differently as I thought because
> > you are proposing statx() API for fetching them. I thought that they
> > would be exported via getxattr()/setxattr().
> >
> > This is also a good idea, just would need to write new userspace tools
> > for setting and gettting... And there is still one important thing. How
> > to modify those attribute? Because statx() is GET-only API.
> >

Re: Immutable vs read-only for Windows compatibility

[Darrick J. Wong]

On Wed, Jan 15, 2025 at 12:55:47AM +0100, Pali Rohár wrote:
> On Wednesday 15 January 2025 09:29:14 ronnie sahlberg wrote:
> > On Wed, 15 Jan 2025 at 07:54, Pali Rohár <pali@kernel.org> wrote:
> > >
> > > On Tuesday 14 January 2025 16:44:55 Chuck Lever wrote:
> > > > On 1/14/25 4:10 PM, Pali Rohár wrote:
> > > > > On Saturday 04 January 2025 10:30:26 Chuck Lever wrote:
> > > > > > On 1/4/25 3:52 AM, Christian Brauner wrote:
> > > > > > > On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
> > > > > > > > On 1/2/25 9:37 AM, Jan Kara wrote:
> > > > > > > > > Hello!
> > > > > > > > >
> > > > > > > > > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > > > > > > > > Few months ago I discussed with Steve that Linux SMB client has some
> > > > > > > > > > problems during removal of directory which has read-only attribute set.
> > > > > > > > > >
> > > > > > > > > > I was looking what exactly the read-only windows attribute means, how it
> > > > > > > > > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > > > > > > > > all.
> > > > > > > > > >
> > > > > > > > > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > > > > > > > > two ways how to present some file or directory as read-only. First
> > > > > > > > > > option is by setting ACL permissions (for particular or all users) to
> > > > > > > > > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > > > > > > > > Second option is available also for (ex)FAT filesystems (first option via
> > > > > > > > > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > > > > > > > >
> > > > > > > > > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > > > > > > > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > > > > > > > > Note that if the parent directory grants for user delete child
> > > > > > > > > > permission then the file can be deleted. This behavior is same for Linux
> > > > > > > > > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > > > > > > > > it is part of directory's write permission).
> > > > > > > > > >
> > > > > > > > > > Second option (Windows read-only attribute) means that the file/dir
> > > > > > > > > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > > > > > > > > and the file/dir cannot be deleted at all. But anybody who has
> > > > > > > > > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > > > > > > > > wants.
> > > > > > > > >
> > > > > > > > > I guess someone with more experience how to fuse together Windows & Linux
> > > > > > > > > permission semantics should chime in here but here are my thoughts.
> > > > > > > > >
> > > > > > > > > > Linux filesystems has similar thing to Windows read-only attribute
> > > > > > > > > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > > > > > > > > which can be set by the "chattr" tool. Seems that the only difference
> > > > > > > > > > between Windows read-only and Linux immutable is that on Linux only
> > > > > > > > > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > > > > > > > > it can be anybody who has write ACL.
> > > > > > > > > >
> > > > > > > > > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > > > > > > > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > > > > > > > >
> > > > > > > > > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > > > > > > > > >       be able to do modify or delete such file, even as root.
> > > > > > > > > >
> > > > > > > > > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > > > > > > > > >       network fs it has to be cleared before any write/modify/delete
> > > > > > > > > >       operation.
> > > > > > > > > >
> > > > > > > > > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > > > > > > > > >       or change it via chmod. Disadvantage is that it mix ACL/mode.
> > > > > > > > >
> > > > > > > > > So this option looks sensible to me. We clear all write permissions in
> > > > > > > > > file's mode / ACL. For reading that is fully compatible, for mode
> > > > > > > > > modifications it gets a bit messy (probably I'd suggest to just clear
> > > > > > > > > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> > > > > > > >
> > > > > > > > IMO Linux should store the Windows-specific attribute information but
> > > > > > > > otherwise ignore it. Modifying ACLs based seems like a road to despair.
> > > > > > > > Plus there's no ACL representation for OFFLINE and some of the other
> > > > > > > > items that we'd like to be able to support.
> > > > > > > >
> > > > > > > >
> > > > > > > > If I were king-for-a-day (tm) I would create a system xattr namespace
> > > > > > > > just for these items, and provide a VFS/statx API for consumers like
> > > > > > > > Samba, ksmbd, and knfsd to set and get these items. Each local
> > > > > > > > filesystem can then implement storage with either the xattr or (eg,
> > > > > > > > ntfs) can store them directly.
> > > > > > >
> > > > > > > Introducing a new xattr namespace for this wouldn't be a problem imho.
> > > > > > > Why would this need a new statx() extension though? Wouldn't the regular
> > > > > > > xattr apis to set and get xattrs be enough?
> > > > > >
> > > > > > My thought was to have a consistent API to access these attributes, and
> > > > > > let the filesystem implementers decide how they want to store them. The
> > > > > > Linux implementation of ntfs, for example, probably wants to store these
> > > > > > on disk in a way that is compatible with the Windows implementation of
> > > > > > NTFS.
> > > > > >
> > > > > > A common API would mean that consumers (like NFSD) wouldn't have to know
> > > > > > those details.
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Chuck Lever
> > > > >
> > > > > So, what about introducing new xattrs for every attribute with this pattern?
> > > > >
> > > > > system.attr.readonly
> > > > > system.attr.hidden
> > > > > system.attr.system
> > > > > system.attr.archive
> > > > > system.attr.temporary
> > > > > system.attr.offline
> > > > > system.attr.not_content_indexed
> > > >
> > > > Yes, all of them could be stored as xattrs for file systems that do
> > > > not already support these attributes.
> > > >
> > > > But I think we don't want to expose them directly to users, however.
> > > > Some file systems, like NTFS, might want to store these on-disk in a way
> > > > that is compatible with Windows.
> > > >
> > > > So I think we want to create statx APIs for consumers like user space
> > > > and knfsd, who do not care to know the specifics of how this information
> > > > is stored by each file system.
> > > >
> > > > The xattrs would be for file systems that do not already have a way to
> > > > represent this information in their on-disk format.
> > > >
> > > >
> > > > > All those attributes can be set by user, I took names from SMB, which
> > > > > matches NTFS and which subsets are used by other filesystems like FAT,
> > > > > exFAT, NFS4, UDF, ...
> > > > >
> > > > > Every xattr would be in system.attr namespace and would contain either
> > > > > value 0 or 1 based on that fact if is set or unset. If the filesystem
> > > > > does not support particular attribute then xattr get/set would return
> > > > > error that it does not exist.
> > > >
> > > > Or, if the xattr exists, then that means the equivalent Windows
> > > > attribute is asserted; and if it does not, the equivalent Windows
> > > > attribute is clear. But again, I think each file system should be
> > > > able to choose how they implement these, and that implementation is
> > > > then hidden by statx.
> > > >
> > > >
> > > > > This would be possible to use by existing userspace getfattr/setfattr
> > > > > tools and also by knfsd/ksmbd via accessing xattrs directly.
> > > >
> > > >
> > > > --
> > > > Chuck Lever
> > >
> > > With this xattr scheme I mean that API would be xattr between fs and
> > > vfs/userspace/knfsd/smbd. So NTFS would take that xattr request and
> > > translate it to its own NTFS attributes. Other non-windows fs stores
> > > them as xattrs.
> > 
> > I am not sure if for the cifs client doing this by emulating xattrs.
> > I have bad memories of the emulated xattrs.
> > 
> > What about extending ioctl(FS_IOC_GETFLAGS)? There are plenty of spare
> > flags there
> 
> Are FS_IOC_GETFLAGS/FS_IOC_SETFLAGS flags preserved across regular
> "cp -a" or "rsync -someflag" commands? I'm just worried to not invent

No, none of them are.  We should perhaps talk to the util-linux folks
about fixing cp.

> new way how to get or set flags which would not be understood by
> existing backup or regular "copy" applications. Because the worst thing
> which can happen is adding new API which nobody would use and basically
> will not gain those benefits which should have them... Like if I move or
> copy file from one filesystem to another to not loose all those
> attributes.
> 
> > and you even have NTFS.readonly ~= Linux.immutable so ... :-)
> 
> I know it :-) I have not explicitly written it in the email, but put
> this information into one of the options what can be possible to do.
> The bad thing about this option for remote filesystems is that
> Linux.immutable can be cleared only by root (or process which privilege
> which nobody does not normally have), but on Windows system (and also
> when exported over SMB) it can be cleared by anybody who can modify file
> (based on ACL). So with this Linux will start blocking to do some
> operation with file, which Windows fully allows. And this very user
> unfriendly, specially if also wine wants to benefit from it, as wine
> normally is not going to be run under root (or special capabilities).
> 
> > To me to feels like the flags you want to implement would fit
> > "somewhat naturally" there.
> 
> So thank you and others for this FS_IOC_GETFLAGS opinion. Maybe this
> looks like a better solution?

FS_IOC_FS[GS]ETXATTR captures a superset of file attributes from
FS_IOC_[GS]ETFLAGS, please use the former if available.

--D

> > regards
> > ronnie s
> > 
> > >
> > > I think that you understood it quite differently as I thought because
> > > you are proposing statx() API for fetching them. I thought that they
> > > would be exported via getxattr()/setxattr().
> > >
> > > This is also a good idea, just would need to write new userspace tools
> > > for setting and gettting... And there is still one important thing. How
> > > to modify those attribute? Because statx() is GET-only API.
> > >
> 

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Wednesday 15 January 2025 09:42:26 ronnie sahlberg wrote:
> On Wed, 15 Jan 2025 at 09:32, Dave Chinner <david@fromorbit.com> wrote:
> >
> > On Tue, Jan 14, 2025 at 04:44:55PM -0500, Chuck Lever wrote:
> > > On 1/14/25 4:10 PM, Pali Rohár wrote:
> > > > > My thought was to have a consistent API to access these attributes, and
> > > > > let the filesystem implementers decide how they want to store them. The
> > > > > Linux implementation of ntfs, for example, probably wants to store these
> > > > > on disk in a way that is compatible with the Windows implementation of
> > > > > NTFS.
> > > > >
> > > > > A common API would mean that consumers (like NFSD) wouldn't have to know
> > > > > those details.
> > > > >
> > > > >
> > > > > --
> > > > > Chuck Lever
> > > >
> > > > So, what about introducing new xattrs for every attribute with this pattern?
> > > >
> > > > system.attr.readonly
> > > > system.attr.hidden
> > > > system.attr.system
> > > > system.attr.archive
> > > > system.attr.temporary
> > > > system.attr.offline
> > > > system.attr.not_content_indexed
> >
> > "attr" is a poor choice for an attribute class (yes, naming is
> > hard...). It's a windows file attribute class, the name should
> > reflect that.

Sure. This was just to show how pattern can look like. Does not have to
be the final name. I agree that naming is hard, but in this case I think
the harder part would be to design API.

Btw, first 4 attributes were inherited from dos, so they are not
originally from windows. But that is not important and it is better to
have consistent naming.

> > However, my main problem with this approach is that it will be
> > pretty nasty in terms of performance regressions. xattr lookup is
> > *much* more expensive than reading a field out of the inode itself.
> >
> > If you want an example of the cost of how a single xattr per file
> > can affect the performance of CIFS servers, go run dbench (a CIFS
> > workload simulator) with and without xattrs. The difference in
> > performance storing a single xattr per file is pretty stark, and it
> > only gets worse as we add more xattrs. i.e. xattrs like this will
> > have significant performance implications for all file create,
> > lookup/stat and unlink operations.
> >
> > IOWs, If this information is going to be stored as an xattr, it
> > needs to be a single xattr with a well defined bit field as it's
> > value (i.e. one bit per attribute). The VFS inode can then cache
> > that bitfield with minimal addition overhead during the first
> > lookup/creation/modification for the purpose of fast, low overhead,
> > statx() operation.
> 
> For this use case I don't think he means to store them on the cifs
> server as xattr
> (or case-insensitive extended attributes as cifs does).
> They can already be read/written using SMB2 GetInfo/SetInfo commands.
> 
> What I think he means is to read these attributes using SMB2 GetInfo
> but then present this to the application via a synthetic xattr.
> Application reads a magic xattr and then the driver just makes it up based on
> other information it has. (cifs does this for other things already afaik)
> 
> Correct me if I am wrong Pali, but you mean translate the SMB2 attribute field
> into a magic xattr?  But that means it is not storage of the
> attributes anymore but rather
> the API for applications to read these attributes.

Exactly, thank you for better explanation. If filesystem supports
"hidden" attribute then request from userspace call (setxattr
system.attr.hidden) would be translated to filesystem native "hidden"
attribute. Which for SMB2+ is SetInfo(), for NFS4 is op_setattr(hidden),
for FAT/exFAT/NTFS is updating hidden bit in attrbit field, for UDF is
updating hidden bit in FileCharacteristics. And for other filesystem
which supports "hidden" attribute natively, it would be stored as that
xattr system.attr.hidden as is.

Performance for sure needs to checked. I have already wrote it that it
could be a problem for userspace applications when they would need to
call lot of syscall to just list directory entries with all attributes.
So for getting attributes maybe some statx() interface could be useful.
In my opinion, applications would probably ask for attributes more times
than trying to set or change them. It is like stat() which is used more
times than utimes() (or other syscalls for changing stat stuff).

In any case, Linux SMB client is already fetching all those attributes
because basically SMB protocol always sends them. So SMB communication
between client and server should not be affected at all by any solution
which is going to be chosen.

> >
> > > Yes, all of them could be stored as xattrs for file systems that do
> > > not already support these attributes.
> > >
> > > But I think we don't want to expose them directly to users, however.
> > > Some file systems, like NTFS, might want to store these on-disk in a way
> > > that is compatible with Windows.
> > >
> > > So I think we want to create statx APIs for consumers like user space
> > > and knfsd, who do not care to know the specifics of how this information
> > > is stored by each file system.
> > >
> > > The xattrs would be for file systems that do not already have a way to
> > > represent this information in their on-disk format.
> >
> > Even the filesystems that store this information natively should
> > support the xattr interface - they just return the native
> > information in the xattr format, and then every application has a
> > common way to change these attributes. (i.e. change the xattr to
> > modify the attributes, statx to efficiently sample them.
> >
> > -Dave.
> > --
> > Dave Chinner
> > david@fromorbit.com
> >

Re: Immutable vs read-only for Windows compatibility

[Maciej W. Rozycki]

On Tue, 14 Jan 2025, Darrick J. Wong wrote:

> > Are FS_IOC_GETFLAGS/FS_IOC_SETFLAGS flags preserved across regular
> > "cp -a" or "rsync -someflag" commands? I'm just worried to not invent
> 
> No, none of them are.  We should perhaps talk to the util-linux folks
> about fixing cp.

 FWIW `cp' comes from GNU coreutils rather than util-linux.

  Maciej

Re: Immutable vs read-only for Windows compatibility

[Amir Goldstein]

On Wed, Jan 15, 2025 at 12:59 AM Darrick J. Wong <djwong@kernel.org> wrote:
>
> On Wed, Jan 15, 2025 at 12:55:47AM +0100, Pali Rohár wrote:
> > On Wednesday 15 January 2025 09:29:14 ronnie sahlberg wrote:
> > > On Wed, 15 Jan 2025 at 07:54, Pali Rohár <pali@kernel.org> wrote:
> > > >
> > > > On Tuesday 14 January 2025 16:44:55 Chuck Lever wrote:
> > > > > On 1/14/25 4:10 PM, Pali Rohár wrote:
> > > > > > On Saturday 04 January 2025 10:30:26 Chuck Lever wrote:
> > > > > > > On 1/4/25 3:52 AM, Christian Brauner wrote:
> > > > > > > > On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
> > > > > > > > > On 1/2/25 9:37 AM, Jan Kara wrote:
> > > > > > > > > > Hello!
> > > > > > > > > >
> > > > > > > > > > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > > > > > > > > > Few months ago I discussed with Steve that Linux SMB client has some
> > > > > > > > > > > problems during removal of directory which has read-only attribute set.
> > > > > > > > > > >
> > > > > > > > > > > I was looking what exactly the read-only windows attribute means, how it
> > > > > > > > > > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > > > > > > > > > all.
> > > > > > > > > > >
> > > > > > > > > > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > > > > > > > > > two ways how to present some file or directory as read-only. First
> > > > > > > > > > > option is by setting ACL permissions (for particular or all users) to
> > > > > > > > > > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > > > > > > > > > Second option is available also for (ex)FAT filesystems (first option via
> > > > > > > > > > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > > > > > > > > >
> > > > > > > > > > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > > > > > > > > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > > > > > > > > > Note that if the parent directory grants for user delete child
> > > > > > > > > > > permission then the file can be deleted. This behavior is same for Linux
> > > > > > > > > > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > > > > > > > > > it is part of directory's write permission).
> > > > > > > > > > >
> > > > > > > > > > > Second option (Windows read-only attribute) means that the file/dir
> > > > > > > > > > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > > > > > > > > > and the file/dir cannot be deleted at all. But anybody who has
> > > > > > > > > > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > > > > > > > > > wants.
> > > > > > > > > >
> > > > > > > > > > I guess someone with more experience how to fuse together Windows & Linux
> > > > > > > > > > permission semantics should chime in here but here are my thoughts.
> > > > > > > > > >
> > > > > > > > > > > Linux filesystems has similar thing to Windows read-only attribute
> > > > > > > > > > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > > > > > > > > > which can be set by the "chattr" tool. Seems that the only difference
> > > > > > > > > > > between Windows read-only and Linux immutable is that on Linux only
> > > > > > > > > > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > > > > > > > > > it can be anybody who has write ACL.
> > > > > > > > > > >
> > > > > > > > > > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > > > > > > > > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > > > > > > > > >
> > > > > > > > > > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > > > > > > > > > >       be able to do modify or delete such file, even as root.
> > > > > > > > > > >
> > > > > > > > > > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > > > > > > > > > >       network fs it has to be cleared before any write/modify/delete
> > > > > > > > > > >       operation.
> > > > > > > > > > >
> > > > > > > > > > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > > > > > > > > > >       or change it via chmod. Disadvantage is that it mix ACL/mode.
> > > > > > > > > >
> > > > > > > > > > So this option looks sensible to me. We clear all write permissions in
> > > > > > > > > > file's mode / ACL. For reading that is fully compatible, for mode
> > > > > > > > > > modifications it gets a bit messy (probably I'd suggest to just clear
> > > > > > > > > > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> > > > > > > > >
> > > > > > > > > IMO Linux should store the Windows-specific attribute information but
> > > > > > > > > otherwise ignore it. Modifying ACLs based seems like a road to despair.
> > > > > > > > > Plus there's no ACL representation for OFFLINE and some of the other
> > > > > > > > > items that we'd like to be able to support.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > If I were king-for-a-day (tm) I would create a system xattr namespace
> > > > > > > > > just for these items, and provide a VFS/statx API for consumers like
> > > > > > > > > Samba, ksmbd, and knfsd to set and get these items. Each local
> > > > > > > > > filesystem can then implement storage with either the xattr or (eg,
> > > > > > > > > ntfs) can store them directly.
> > > > > > > >
> > > > > > > > Introducing a new xattr namespace for this wouldn't be a problem imho.
> > > > > > > > Why would this need a new statx() extension though? Wouldn't the regular
> > > > > > > > xattr apis to set and get xattrs be enough?
> > > > > > >
> > > > > > > My thought was to have a consistent API to access these attributes, and
> > > > > > > let the filesystem implementers decide how they want to store them. The
> > > > > > > Linux implementation of ntfs, for example, probably wants to store these
> > > > > > > on disk in a way that is compatible with the Windows implementation of
> > > > > > > NTFS.
> > > > > > >
> > > > > > > A common API would mean that consumers (like NFSD) wouldn't have to know
> > > > > > > those details.
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Chuck Lever
> > > > > >
> > > > > > So, what about introducing new xattrs for every attribute with this pattern?
> > > > > >
> > > > > > system.attr.readonly
> > > > > > system.attr.hidden
> > > > > > system.attr.system
> > > > > > system.attr.archive
> > > > > > system.attr.temporary
> > > > > > system.attr.offline
> > > > > > system.attr.not_content_indexed
> > > > >
> > > > > Yes, all of them could be stored as xattrs for file systems that do
> > > > > not already support these attributes.
> > > > >
> > > > > But I think we don't want to expose them directly to users, however.
> > > > > Some file systems, like NTFS, might want to store these on-disk in a way
> > > > > that is compatible with Windows.
> > > > >
> > > > > So I think we want to create statx APIs for consumers like user space
> > > > > and knfsd, who do not care to know the specifics of how this information
> > > > > is stored by each file system.
> > > > >
> > > > > The xattrs would be for file systems that do not already have a way to
> > > > > represent this information in their on-disk format.
> > > > >
> > > > >
> > > > > > All those attributes can be set by user, I took names from SMB, which
> > > > > > matches NTFS and which subsets are used by other filesystems like FAT,
> > > > > > exFAT, NFS4, UDF, ...
> > > > > >
> > > > > > Every xattr would be in system.attr namespace and would contain either
> > > > > > value 0 or 1 based on that fact if is set or unset. If the filesystem
> > > > > > does not support particular attribute then xattr get/set would return
> > > > > > error that it does not exist.
> > > > >
> > > > > Or, if the xattr exists, then that means the equivalent Windows
> > > > > attribute is asserted; and if it does not, the equivalent Windows
> > > > > attribute is clear. But again, I think each file system should be
> > > > > able to choose how they implement these, and that implementation is
> > > > > then hidden by statx.
> > > > >
> > > > >
> > > > > > This would be possible to use by existing userspace getfattr/setfattr
> > > > > > tools and also by knfsd/ksmbd via accessing xattrs directly.
> > > > >
> > > > >
> > > > > --
> > > > > Chuck Lever
> > > >
> > > > With this xattr scheme I mean that API would be xattr between fs and
> > > > vfs/userspace/knfsd/smbd. So NTFS would take that xattr request and
> > > > translate it to its own NTFS attributes. Other non-windows fs stores
> > > > them as xattrs.
> > >
> > > I am not sure if for the cifs client doing this by emulating xattrs.
> > > I have bad memories of the emulated xattrs.
> > >
> > > What about extending ioctl(FS_IOC_GETFLAGS)? There are plenty of spare
> > > flags there
> >
> > Are FS_IOC_GETFLAGS/FS_IOC_SETFLAGS flags preserved across regular
> > "cp -a" or "rsync -someflag" commands? I'm just worried to not invent
>
> No, none of them are.  We should perhaps talk to the util-linux folks
> about fixing cp.
>

I don't think it is a good idea to start copying these attributes with existing
cp -a without any opt-in with mount option or new cp command line option.

After all, smb client already exports the virtual xattr "smb3.dosattrib", but
it is not listed by listxattr, so cp -a does not pick it up anyway.

You could just as well define a standard virtual xattr "system.fs.fsxattr"
that implements an alternative interface for FS_IOC_FS[GS]ETXATTR
but it would have to be opt-in to show up in listxattr().

> > new way how to get or set flags which would not be understood by
> > existing backup or regular "copy" applications. Because the worst thing
> > which can happen is adding new API which nobody would use and basically
> > will not gain those benefits which should have them... Like if I move or
> > copy file from one filesystem to another to not loose all those
> > attributes.
> >
> > > and you even have NTFS.readonly ~= Linux.immutable so ... :-)
> >
> > I know it :-) I have not explicitly written it in the email, but put
> > this information into one of the options what can be possible to do.
> > The bad thing about this option for remote filesystems is that
> > Linux.immutable can be cleared only by root (or process which privilege
> > which nobody does not normally have), but on Windows system (and also
> > when exported over SMB) it can be cleared by anybody who can modify file
> > (based on ACL). So with this Linux will start blocking to do some
> > operation with file, which Windows fully allows. And this very user
> > unfriendly, specially if also wine wants to benefit from it, as wine
> > normally is not going to be run under root (or special capabilities).
> >
> > > To me to feels like the flags you want to implement would fit
> > > "somewhat naturally" there.
> >
> > So thank you and others for this FS_IOC_GETFLAGS opinion. Maybe this
> > looks like a better solution?
>
> FS_IOC_FS[GS]ETXATTR captures a superset of file attributes from
> FS_IOC_[GS]ETFLAGS, please use the former if available.
>

I agree. Flags that you define in the FS_XFLAG_* namespace,
should also be defined in the STATX_ATTR_* namepsace.

Looking at the FILE_ATTRIBUTE_* flags defined in SMB protocol
 (fs/smb/common/smb2pdu.h) I wonder how many of them will be
needed for applications beyond the obvious ones that were listed.

Thanks,
Amir.

Re: Immutable vs read-only for Windows compatibility

[Darrick J. Wong]

On Fri, Jan 17, 2025 at 05:53:34PM +0100, Amir Goldstein wrote:
> On Wed, Jan 15, 2025 at 12:59 AM Darrick J. Wong <djwong@kernel.org> wrote:
> >
> > On Wed, Jan 15, 2025 at 12:55:47AM +0100, Pali Rohár wrote:
> > > On Wednesday 15 January 2025 09:29:14 ronnie sahlberg wrote:
> > > > On Wed, 15 Jan 2025 at 07:54, Pali Rohár <pali@kernel.org> wrote:
> > > > >
> > > > > On Tuesday 14 January 2025 16:44:55 Chuck Lever wrote:
> > > > > > On 1/14/25 4:10 PM, Pali Rohár wrote:
> > > > > > > On Saturday 04 January 2025 10:30:26 Chuck Lever wrote:
> > > > > > > > On 1/4/25 3:52 AM, Christian Brauner wrote:
> > > > > > > > > On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
> > > > > > > > > > On 1/2/25 9:37 AM, Jan Kara wrote:
> > > > > > > > > > > Hello!
> > > > > > > > > > >
> > > > > > > > > > > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > > > > > > > > > > Few months ago I discussed with Steve that Linux SMB client has some
> > > > > > > > > > > > problems during removal of directory which has read-only attribute set.
> > > > > > > > > > > >
> > > > > > > > > > > > I was looking what exactly the read-only windows attribute means, how it
> > > > > > > > > > > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > > > > > > > > > > all.
> > > > > > > > > > > >
> > > > > > > > > > > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > > > > > > > > > > two ways how to present some file or directory as read-only. First
> > > > > > > > > > > > option is by setting ACL permissions (for particular or all users) to
> > > > > > > > > > > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > > > > > > > > > > Second option is available also for (ex)FAT filesystems (first option via
> > > > > > > > > > > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > > > > > > > > > >
> > > > > > > > > > > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > > > > > > > > > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > > > > > > > > > > Note that if the parent directory grants for user delete child
> > > > > > > > > > > > permission then the file can be deleted. This behavior is same for Linux
> > > > > > > > > > > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > > > > > > > > > > it is part of directory's write permission).
> > > > > > > > > > > >
> > > > > > > > > > > > Second option (Windows read-only attribute) means that the file/dir
> > > > > > > > > > > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > > > > > > > > > > and the file/dir cannot be deleted at all. But anybody who has
> > > > > > > > > > > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > > > > > > > > > > wants.
> > > > > > > > > > >
> > > > > > > > > > > I guess someone with more experience how to fuse together Windows & Linux
> > > > > > > > > > > permission semantics should chime in here but here are my thoughts.
> > > > > > > > > > >
> > > > > > > > > > > > Linux filesystems has similar thing to Windows read-only attribute
> > > > > > > > > > > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > > > > > > > > > > which can be set by the "chattr" tool. Seems that the only difference
> > > > > > > > > > > > between Windows read-only and Linux immutable is that on Linux only
> > > > > > > > > > > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > > > > > > > > > > it can be anybody who has write ACL.
> > > > > > > > > > > >
> > > > > > > > > > > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > > > > > > > > > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > > > > > > > > > >
> > > > > > > > > > > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > > > > > > > > > > >       be able to do modify or delete such file, even as root.
> > > > > > > > > > > >
> > > > > > > > > > > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > > > > > > > > > > >       network fs it has to be cleared before any write/modify/delete
> > > > > > > > > > > >       operation.
> > > > > > > > > > > >
> > > > > > > > > > > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > > > > > > > > > > >       or change it via chmod. Disadvantage is that it mix ACL/mode.
> > > > > > > > > > >
> > > > > > > > > > > So this option looks sensible to me. We clear all write permissions in
> > > > > > > > > > > file's mode / ACL. For reading that is fully compatible, for mode
> > > > > > > > > > > modifications it gets a bit messy (probably I'd suggest to just clear
> > > > > > > > > > > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> > > > > > > > > >
> > > > > > > > > > IMO Linux should store the Windows-specific attribute information but
> > > > > > > > > > otherwise ignore it. Modifying ACLs based seems like a road to despair.
> > > > > > > > > > Plus there's no ACL representation for OFFLINE and some of the other
> > > > > > > > > > items that we'd like to be able to support.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > If I were king-for-a-day (tm) I would create a system xattr namespace
> > > > > > > > > > just for these items, and provide a VFS/statx API for consumers like
> > > > > > > > > > Samba, ksmbd, and knfsd to set and get these items. Each local
> > > > > > > > > > filesystem can then implement storage with either the xattr or (eg,
> > > > > > > > > > ntfs) can store them directly.
> > > > > > > > >
> > > > > > > > > Introducing a new xattr namespace for this wouldn't be a problem imho.
> > > > > > > > > Why would this need a new statx() extension though? Wouldn't the regular
> > > > > > > > > xattr apis to set and get xattrs be enough?
> > > > > > > >
> > > > > > > > My thought was to have a consistent API to access these attributes, and
> > > > > > > > let the filesystem implementers decide how they want to store them. The
> > > > > > > > Linux implementation of ntfs, for example, probably wants to store these
> > > > > > > > on disk in a way that is compatible with the Windows implementation of
> > > > > > > > NTFS.
> > > > > > > >
> > > > > > > > A common API would mean that consumers (like NFSD) wouldn't have to know
> > > > > > > > those details.
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Chuck Lever
> > > > > > >
> > > > > > > So, what about introducing new xattrs for every attribute with this pattern?
> > > > > > >
> > > > > > > system.attr.readonly
> > > > > > > system.attr.hidden
> > > > > > > system.attr.system
> > > > > > > system.attr.archive
> > > > > > > system.attr.temporary
> > > > > > > system.attr.offline
> > > > > > > system.attr.not_content_indexed
> > > > > >
> > > > > > Yes, all of them could be stored as xattrs for file systems that do
> > > > > > not already support these attributes.
> > > > > >
> > > > > > But I think we don't want to expose them directly to users, however.
> > > > > > Some file systems, like NTFS, might want to store these on-disk in a way
> > > > > > that is compatible with Windows.
> > > > > >
> > > > > > So I think we want to create statx APIs for consumers like user space
> > > > > > and knfsd, who do not care to know the specifics of how this information
> > > > > > is stored by each file system.
> > > > > >
> > > > > > The xattrs would be for file systems that do not already have a way to
> > > > > > represent this information in their on-disk format.
> > > > > >
> > > > > >
> > > > > > > All those attributes can be set by user, I took names from SMB, which
> > > > > > > matches NTFS and which subsets are used by other filesystems like FAT,
> > > > > > > exFAT, NFS4, UDF, ...
> > > > > > >
> > > > > > > Every xattr would be in system.attr namespace and would contain either
> > > > > > > value 0 or 1 based on that fact if is set or unset. If the filesystem
> > > > > > > does not support particular attribute then xattr get/set would return
> > > > > > > error that it does not exist.
> > > > > >
> > > > > > Or, if the xattr exists, then that means the equivalent Windows
> > > > > > attribute is asserted; and if it does not, the equivalent Windows
> > > > > > attribute is clear. But again, I think each file system should be
> > > > > > able to choose how they implement these, and that implementation is
> > > > > > then hidden by statx.
> > > > > >
> > > > > >
> > > > > > > This would be possible to use by existing userspace getfattr/setfattr
> > > > > > > tools and also by knfsd/ksmbd via accessing xattrs directly.
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Chuck Lever
> > > > >
> > > > > With this xattr scheme I mean that API would be xattr between fs and
> > > > > vfs/userspace/knfsd/smbd. So NTFS would take that xattr request and
> > > > > translate it to its own NTFS attributes. Other non-windows fs stores
> > > > > them as xattrs.
> > > >
> > > > I am not sure if for the cifs client doing this by emulating xattrs.
> > > > I have bad memories of the emulated xattrs.
> > > >
> > > > What about extending ioctl(FS_IOC_GETFLAGS)? There are plenty of spare
> > > > flags there
> > >
> > > Are FS_IOC_GETFLAGS/FS_IOC_SETFLAGS flags preserved across regular
> > > "cp -a" or "rsync -someflag" commands? I'm just worried to not invent
> >
> > No, none of them are.  We should perhaps talk to the util-linux folks
> > about fixing cp.
> >
> 
> I don't think it is a good idea to start copying these attributes with existing
> cp -a without any opt-in with mount option or new cp command line option.
> 
> After all, smb client already exports the virtual xattr "smb3.dosattrib", but
> it is not listed by listxattr, so cp -a does not pick it up anyway.
> 
> You could just as well define a standard virtual xattr "system.fs.fsxattr"
> that implements an alternative interface for FS_IOC_FS[GS]ETXATTR
> but it would have to be opt-in to show up in listxattr().
> 
> > > new way how to get or set flags which would not be understood by
> > > existing backup or regular "copy" applications. Because the worst thing
> > > which can happen is adding new API which nobody would use and basically
> > > will not gain those benefits which should have them... Like if I move or
> > > copy file from one filesystem to another to not loose all those
> > > attributes.
> > >
> > > > and you even have NTFS.readonly ~= Linux.immutable so ... :-)
> > >
> > > I know it :-) I have not explicitly written it in the email, but put
> > > this information into one of the options what can be possible to do.
> > > The bad thing about this option for remote filesystems is that
> > > Linux.immutable can be cleared only by root (or process which privilege
> > > which nobody does not normally have), but on Windows system (and also
> > > when exported over SMB) it can be cleared by anybody who can modify file
> > > (based on ACL). So with this Linux will start blocking to do some
> > > operation with file, which Windows fully allows. And this very user
> > > unfriendly, specially if also wine wants to benefit from it, as wine
> > > normally is not going to be run under root (or special capabilities).
> > >
> > > > To me to feels like the flags you want to implement would fit
> > > > "somewhat naturally" there.
> > >
> > > So thank you and others for this FS_IOC_GETFLAGS opinion. Maybe this
> > > looks like a better solution?
> >
> > FS_IOC_FS[GS]ETXATTR captures a superset of file attributes from
> > FS_IOC_[GS]ETFLAGS, please use the former if available.
> >
> 
> I agree. Flags that you define in the FS_XFLAG_* namespace,
> should also be defined in the STATX_ATTR_* namepsace.
> 
> Looking at the FILE_ATTRIBUTE_* flags defined in SMB protocol
>  (fs/smb/common/smb2pdu.h) I wonder how many of them will be
> needed for applications beyond the obvious ones that were listed.

Well they only asked for seven of them. ;)

I chatted with Ted about this yesterday, and ... some of the attributes
(like read only) imply that you'd want the linux server to enforce no
writing to the file; some like archive seem a little superfluous since
on linux you can compare cmtime from the backup against what's in the
file now; and still others (like hidden/system) might just be some dorky
thing that could be hidden in some xattr because a unix filesystem won't
care.

And then there are other attrs like "integrity stream" where someone
with more experience with windows would have to tell me if fsverity
provides sufficient behaviors or not.

But maybe we should start by plumbing one of those bits in?  I guess the
gross part is that implies an ondisk inode format change or (gross)
xattr lookups in the open path.

(This is the point where I declare that except for applying a mandatory
Thunderbolt firmware update a month ago I haven't touched Windows since
2002, and this is the extent of my interest in the topic.)

--D

> Thanks,
> Amir.

Re: Immutable vs read-only for Windows compatibility

[Steve French]

On Fri, Jan 17, 2025 at 11:39 AM Darrick J. Wong <djwong@kernel.org> wrote:
>
> On Fri, Jan 17, 2025 at 05:53:34PM +0100, Amir Goldstein wrote:
> > On Wed, Jan 15, 2025 at 12:59 AM Darrick J. Wong <djwong@kernel.org> wrote:
<...>
> > Looking at the FILE_ATTRIBUTE_* flags defined in SMB protocol
> >  (fs/smb/common/smb2pdu.h) I wonder how many of them will be
> > needed for applications beyond the obvious ones that were listed.
>
> Well they only asked for seven of them. ;)
>
> I chatted with Ted about this yesterday, and ... some of the attributes
> (like read only) imply that you'd want the linux server to enforce no
> writing to the file; some like archive seem a little superfluous since
> on linux you can compare cmtime from the backup against what's in the
> file now; and still others (like hidden/system) might just be some dorky
> thing that could be hidden in some xattr because a unix filesystem won't
> care.
>
> And then there are other attrs like "integrity stream" where someone
> with more experience with windows would have to tell me if fsverity
> provides sufficient behaviors or not.
>
> But maybe we should start by plumbing one of those bits in?  I guess the
> gross part is that implies an ondisk inode format change or (gross)
> xattr lookups in the open path.
>

We have talked about some of these missing flags in the past, but the
obvious ones that would be helpful i (e.g. is used in other operating
systems when view directories in the equivalent of the "Files" GUI is
checking FILE_ATTRIBUTE_OFFLINE to determine whether to query icons,
and additional metadata for files).  In the past Unix used to have
various ways to determine this, but it is fairly common for files to
be tiered (where the data is in very slow storage offline - so should
only be opened and read by apps that really need to - not things like
GUIs browsing lists of files) so that attribute could be helpful.

The other two obvious ones (missing in Linux but that some other OS
have filesystems which support) discussed before were
FILE_ATTRIBUTE_INTEGRITY_STREAM which could be set for files that need
stronger data integrity guarantees (if a filesystem allows files to be
marked for stronger data integrity guarantees) , and
FILE_ATTRIBUTE_NO_SCRUB_DATA that indicates integrity checks can be
skipped for this particular file.
-- 
Thanks,

Steve

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Friday 17 January 2025 17:53:34 Amir Goldstein wrote:
> On Wed, Jan 15, 2025 at 12:59 AM Darrick J. Wong <djwong@kernel.org> wrote:
> >
> > On Wed, Jan 15, 2025 at 12:55:47AM +0100, Pali Rohár wrote:
> > > On Wednesday 15 January 2025 09:29:14 ronnie sahlberg wrote:
> > > > On Wed, 15 Jan 2025 at 07:54, Pali Rohár <pali@kernel.org> wrote:
> > > > >
> > > > > On Tuesday 14 January 2025 16:44:55 Chuck Lever wrote:
> > > > > > On 1/14/25 4:10 PM, Pali Rohár wrote:
> > > > > > > On Saturday 04 January 2025 10:30:26 Chuck Lever wrote:
> > > > > > > > On 1/4/25 3:52 AM, Christian Brauner wrote:
> > > > > > > > > On Thu, Jan 02, 2025 at 10:52:51AM -0500, Chuck Lever wrote:
> > > > > > > > > > On 1/2/25 9:37 AM, Jan Kara wrote:
> > > > > > > > > > > Hello!
> > > > > > > > > > >
> > > > > > > > > > > On Fri 27-12-24 13:15:08, Pali Rohár wrote:
> > > > > > > > > > > > Few months ago I discussed with Steve that Linux SMB client has some
> > > > > > > > > > > > problems during removal of directory which has read-only attribute set.
> > > > > > > > > > > >
> > > > > > > > > > > > I was looking what exactly the read-only windows attribute means, how it
> > > > > > > > > > > > is interpreted by Linux and in my opinion it is wrongly used in Linux at
> > > > > > > > > > > > all.
> > > > > > > > > > > >
> > > > > > > > > > > > Windows filesystems NTFS and ReFS, and also exported over SMB supports
> > > > > > > > > > > > two ways how to present some file or directory as read-only. First
> > > > > > > > > > > > option is by setting ACL permissions (for particular or all users) to
> > > > > > > > > > > > GENERIC_READ-only. Second option is by setting the read-only attribute.
> > > > > > > > > > > > Second option is available also for (ex)FAT filesystems (first option via
> > > > > > > > > > > > ACL is not possible on (ex)FAT as it does not have ACLs).
> > > > > > > > > > > >
> > > > > > > > > > > > First option (ACL) is basically same as clearing all "w" bits in mode
> > > > > > > > > > > > and ACL (if present) on Linux. It enforces security permission behavior.
> > > > > > > > > > > > Note that if the parent directory grants for user delete child
> > > > > > > > > > > > permission then the file can be deleted. This behavior is same for Linux
> > > > > > > > > > > > and Windows (on Windows there is separate ACL for delete child, on Linux
> > > > > > > > > > > > it is part of directory's write permission).
> > > > > > > > > > > >
> > > > > > > > > > > > Second option (Windows read-only attribute) means that the file/dir
> > > > > > > > > > > > cannot be opened in write mode, its metadata attribute cannot be changed
> > > > > > > > > > > > and the file/dir cannot be deleted at all. But anybody who has
> > > > > > > > > > > > WRITE_ATTRIBUTES ACL permission can clear this attribute and do whatever
> > > > > > > > > > > > wants.
> > > > > > > > > > >
> > > > > > > > > > > I guess someone with more experience how to fuse together Windows & Linux
> > > > > > > > > > > permission semantics should chime in here but here are my thoughts.
> > > > > > > > > > >
> > > > > > > > > > > > Linux filesystems has similar thing to Windows read-only attribute
> > > > > > > > > > > > (FILE_ATTRIBUTE_READONLY). It is "immutable" bit (FS_IMMUTABLE_FL),
> > > > > > > > > > > > which can be set by the "chattr" tool. Seems that the only difference
> > > > > > > > > > > > between Windows read-only and Linux immutable is that on Linux only
> > > > > > > > > > > > process with CAP_LINUX_IMMUTABLE can set or clear this bit. On Windows
> > > > > > > > > > > > it can be anybody who has write ACL.
> > > > > > > > > > > >
> > > > > > > > > > > > Now I'm thinking, how should be Windows read-only bit interpreted by
> > > > > > > > > > > > Linux filesystems drivers (FAT, exFAT, NTFS, SMB)? I see few options:
> > > > > > > > > > > >
> > > > > > > > > > > > 0) Simply ignored. Disadvantage is that over network fs, user would not
> > > > > > > > > > > >       be able to do modify or delete such file, even as root.
> > > > > > > > > > > >
> > > > > > > > > > > > 1) Smartly ignored. Meaning that for local fs, it is ignored and for
> > > > > > > > > > > >       network fs it has to be cleared before any write/modify/delete
> > > > > > > > > > > >       operation.
> > > > > > > > > > > >
> > > > > > > > > > > > 2) Translated to Linux mode/ACL. So the user has some ability to see it
> > > > > > > > > > > >       or change it via chmod. Disadvantage is that it mix ACL/mode.
> > > > > > > > > > >
> > > > > > > > > > > So this option looks sensible to me. We clear all write permissions in
> > > > > > > > > > > file's mode / ACL. For reading that is fully compatible, for mode
> > > > > > > > > > > modifications it gets a bit messy (probably I'd suggest to just clear
> > > > > > > > > > > FILE_ATTRIBUTE_READONLY on modification) but kind of close.
> > > > > > > > > >
> > > > > > > > > > IMO Linux should store the Windows-specific attribute information but
> > > > > > > > > > otherwise ignore it. Modifying ACLs based seems like a road to despair.
> > > > > > > > > > Plus there's no ACL representation for OFFLINE and some of the other
> > > > > > > > > > items that we'd like to be able to support.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > If I were king-for-a-day (tm) I would create a system xattr namespace
> > > > > > > > > > just for these items, and provide a VFS/statx API for consumers like
> > > > > > > > > > Samba, ksmbd, and knfsd to set and get these items. Each local
> > > > > > > > > > filesystem can then implement storage with either the xattr or (eg,
> > > > > > > > > > ntfs) can store them directly.
> > > > > > > > >
> > > > > > > > > Introducing a new xattr namespace for this wouldn't be a problem imho.
> > > > > > > > > Why would this need a new statx() extension though? Wouldn't the regular
> > > > > > > > > xattr apis to set and get xattrs be enough?
> > > > > > > >
> > > > > > > > My thought was to have a consistent API to access these attributes, and
> > > > > > > > let the filesystem implementers decide how they want to store them. The
> > > > > > > > Linux implementation of ntfs, for example, probably wants to store these
> > > > > > > > on disk in a way that is compatible with the Windows implementation of
> > > > > > > > NTFS.
> > > > > > > >
> > > > > > > > A common API would mean that consumers (like NFSD) wouldn't have to know
> > > > > > > > those details.
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Chuck Lever
> > > > > > >
> > > > > > > So, what about introducing new xattrs for every attribute with this pattern?
> > > > > > >
> > > > > > > system.attr.readonly
> > > > > > > system.attr.hidden
> > > > > > > system.attr.system
> > > > > > > system.attr.archive
> > > > > > > system.attr.temporary
> > > > > > > system.attr.offline
> > > > > > > system.attr.not_content_indexed
> > > > > >
> > > > > > Yes, all of them could be stored as xattrs for file systems that do
> > > > > > not already support these attributes.
> > > > > >
> > > > > > But I think we don't want to expose them directly to users, however.
> > > > > > Some file systems, like NTFS, might want to store these on-disk in a way
> > > > > > that is compatible with Windows.
> > > > > >
> > > > > > So I think we want to create statx APIs for consumers like user space
> > > > > > and knfsd, who do not care to know the specifics of how this information
> > > > > > is stored by each file system.
> > > > > >
> > > > > > The xattrs would be for file systems that do not already have a way to
> > > > > > represent this information in their on-disk format.
> > > > > >
> > > > > >
> > > > > > > All those attributes can be set by user, I took names from SMB, which
> > > > > > > matches NTFS and which subsets are used by other filesystems like FAT,
> > > > > > > exFAT, NFS4, UDF, ...
> > > > > > >
> > > > > > > Every xattr would be in system.attr namespace and would contain either
> > > > > > > value 0 or 1 based on that fact if is set or unset. If the filesystem
> > > > > > > does not support particular attribute then xattr get/set would return
> > > > > > > error that it does not exist.
> > > > > >
> > > > > > Or, if the xattr exists, then that means the equivalent Windows
> > > > > > attribute is asserted; and if it does not, the equivalent Windows
> > > > > > attribute is clear. But again, I think each file system should be
> > > > > > able to choose how they implement these, and that implementation is
> > > > > > then hidden by statx.
> > > > > >
> > > > > >
> > > > > > > This would be possible to use by existing userspace getfattr/setfattr
> > > > > > > tools and also by knfsd/ksmbd via accessing xattrs directly.
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Chuck Lever
> > > > >
> > > > > With this xattr scheme I mean that API would be xattr between fs and
> > > > > vfs/userspace/knfsd/smbd. So NTFS would take that xattr request and
> > > > > translate it to its own NTFS attributes. Other non-windows fs stores
> > > > > them as xattrs.
> > > >
> > > > I am not sure if for the cifs client doing this by emulating xattrs.
> > > > I have bad memories of the emulated xattrs.
> > > >
> > > > What about extending ioctl(FS_IOC_GETFLAGS)? There are plenty of spare
> > > > flags there
> > >
> > > Are FS_IOC_GETFLAGS/FS_IOC_SETFLAGS flags preserved across regular
> > > "cp -a" or "rsync -someflag" commands? I'm just worried to not invent
> >
> > No, none of them are.  We should perhaps talk to the util-linux folks
> > about fixing cp.
> >
> 
> I don't think it is a good idea to start copying these attributes with existing
> cp -a without any opt-in with mount option or new cp command line option.

Just to note that cp's "-a" argument is an alias for "-dR --preserve=all"
and "--preserve=all" is an alias --preserve=<everythingsupported>
(context, links, xattr, ...). There is also --no-preserve= to explicitly
mention which parts are not going to be copied.

IMHO, when somebody is going extend cp to copy also FS_IOC_FS[GS]ETXATTR,
I guess it would be by adding a new option which can be passed to
--preserve= and --no-preserve= arguments, plus "all" would include also
that new option. Hence there would be opt-in and also opt-out
possibility at cp level. But this is just my option.

> After all, smb client already exports the virtual xattr "smb3.dosattrib", but
> it is not listed by listxattr, so cp -a does not pick it up anyway.

Yes, but it is SMB specific and issue here which we are discussing is
filesystem generic. And I do not think that it is a good idea to use
"smb3.dosattrib" xattr for example by FAT32 or NTFS or UDF fs drivers.

> You could just as well define a standard virtual xattr "system.fs.fsxattr"
> that implements an alternative interface for FS_IOC_FS[GS]ETXATTR
> but it would have to be opt-in to show up in listxattr().

Ok. But I think that translating FS_IOC_FS[GS]ETXATTR to xattr
"system.fs.fsxattr" is less usable. It cannot be easily modified by
existing userspace tools like setfattr as it would need to use bitmask
and also it would require get-set roundtrip with with modification at
bit-level. So it would be needed to write completely new tool for it,
and in this case FS_IOC_FS[GS]ETXATTR API tools to be more usable. No
need to define new opt-in/opt-out API for listxattr(), no need to define
virtual xattr "system.fs.fsxattr" and translate it to FS_IOC_FS[GS]ETXATTR.

My original idea was to use one xattr per one attribute, which allows
directly to use setfattr to change just one attribute without need to
mess by binary bitfields. This can have benefits in having those new
attributes encoded in xattrs.

> > > new way how to get or set flags which would not be understood by
> > > existing backup or regular "copy" applications. Because the worst thing
> > > which can happen is adding new API which nobody would use and basically
> > > will not gain those benefits which should have them... Like if I move or
> > > copy file from one filesystem to another to not loose all those
> > > attributes.
> > >
> > > > and you even have NTFS.readonly ~= Linux.immutable so ... :-)
> > >
> > > I know it :-) I have not explicitly written it in the email, but put
> > > this information into one of the options what can be possible to do.
> > > The bad thing about this option for remote filesystems is that
> > > Linux.immutable can be cleared only by root (or process which privilege
> > > which nobody does not normally have), but on Windows system (and also
> > > when exported over SMB) it can be cleared by anybody who can modify file
> > > (based on ACL). So with this Linux will start blocking to do some
> > > operation with file, which Windows fully allows. And this very user
> > > unfriendly, specially if also wine wants to benefit from it, as wine
> > > normally is not going to be run under root (or special capabilities).
> > >
> > > > To me to feels like the flags you want to implement would fit
> > > > "somewhat naturally" there.
> > >
> > > So thank you and others for this FS_IOC_GETFLAGS opinion. Maybe this
> > > looks like a better solution?
> >
> > FS_IOC_FS[GS]ETXATTR captures a superset of file attributes from
> > FS_IOC_[GS]ETFLAGS, please use the former if available.
> >
> 
> I agree. Flags that you define in the FS_XFLAG_* namespace,
> should also be defined in the STATX_ATTR_* namepsace.

Ok, so this could solve the problem that application would be able to
read all attributes (time modifications, file size, hidden attribute...)
by just one statx call, right?

> Looking at the FILE_ATTRIBUTE_* flags defined in SMB protocol
>  (fs/smb/common/smb2pdu.h) I wonder how many of them will be
> needed for applications beyond the obvious ones that were listed.

Please do not take it SMB specific, as this new API should be fs
agnostic, or at least agnostic for filesystems used by windows.

Also looking at SMB definitions without understanding its meaning is not
a good starting point because SMB bitfield flags mix user-settable
attributes (e.g. hidden) which has no effect together with status
information which are get-only and not directly settable by user (e.g.
inode is directory, or file is spare, or inode does not have set any
other bit in this bitfield = normal).

Attributes which covers those application usage I wrote in some
previous email, I'm quoting it below:

| system.attr.readonly
| system.attr.hidden
| system.attr.system
| system.attr.archive
| system.attr.temporary
| system.attr.offline
| system.attr.not_content_indexed

But it is possible that some new version of windows or filesystems
starts using some other yet unassigned bits. So API should be prepared
for extending.

> Thanks,
> Amir.

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Friday 17 January 2025 11:51:54 Steve French wrote:
> On Fri, Jan 17, 2025 at 11:39 AM Darrick J. Wong <djwong@kernel.org> wrote:
> >
> > On Fri, Jan 17, 2025 at 05:53:34PM +0100, Amir Goldstein wrote:
> > > On Wed, Jan 15, 2025 at 12:59 AM Darrick J. Wong <djwong@kernel.org> wrote:
> <...>
> > > Looking at the FILE_ATTRIBUTE_* flags defined in SMB protocol
> > >  (fs/smb/common/smb2pdu.h) I wonder how many of them will be
> > > needed for applications beyond the obvious ones that were listed.
> >
> > Well they only asked for seven of them. ;)
> >
> > I chatted with Ted about this yesterday, and ... some of the attributes
> > (like read only) imply that you'd want the linux server to enforce no
> > writing to the file; some like archive seem a little superfluous since
> > on linux you can compare cmtime from the backup against what's in the
> > file now; and still others (like hidden/system) might just be some dorky
> > thing that could be hidden in some xattr because a unix filesystem won't
> > care.
> >
> > And then there are other attrs like "integrity stream" where someone
> > with more experience with windows would have to tell me if fsverity
> > provides sufficient behaviors or not.
> >
> > But maybe we should start by plumbing one of those bits in?  I guess the
> > gross part is that implies an ondisk inode format change or (gross)
> > xattr lookups in the open path.
> >
> 
> We have talked about some of these missing flags in the past, but the
> obvious ones that would be helpful i (e.g. is used in other operating
> systems when view directories in the equivalent of the "Files" GUI is
> checking FILE_ATTRIBUTE_OFFLINE to determine whether to query icons,
> and additional metadata for files).  In the past Unix used to have
> various ways to determine this, but it is fairly common for files to
> be tiered (where the data is in very slow storage offline - so should
> only be opened and read by apps that really need to - not things like
> GUIs browsing lists of files) so that attribute could be helpful.
> 
> The other two obvious ones (missing in Linux but that some other OS
> have filesystems which support) discussed before were
> FILE_ATTRIBUTE_INTEGRITY_STREAM which could be set for files that need
> stronger data integrity guarantees (if a filesystem allows files to be
> marked for stronger data integrity guarantees) , and
> FILE_ATTRIBUTE_NO_SCRUB_DATA that indicates integrity checks can be
> skipped for this particular file.
> -- 
> Thanks,
> 
> Steve

Thank you for information about integrity stream and these new things
around. I have not included them into my initial list because I have not
used them yet. That it why I listed only seven. But as I wrote in the
other email, whatever API is chosen, it should be prepared for extending
and integrity stream sounds like something could be included there.

Re: Immutable vs read-only for Windows compatibility

[Amir Goldstein]

> > Looking at the FILE_ATTRIBUTE_* flags defined in SMB protocol
> >  (fs/smb/common/smb2pdu.h) I wonder how many of them will be
> > needed for applications beyond the obvious ones that were listed.
>
> Well they only asked for seven of them. ;)
>
> I chatted with Ted about this yesterday, and ... some of the attributes
> (like read only) imply that you'd want the linux server to enforce no
> writing to the file; some like archive seem a little superfluous since
> on linux you can compare cmtime from the backup against what's in the
> file now; and still others (like hidden/system) might just be some dorky
> thing that could be hidden in some xattr because a unix filesystem won't
> care.
>
> And then there are other attrs like "integrity stream" where someone
> with more experience with windows would have to tell me if fsverity
> provides sufficient behaviors or not.
>
> But maybe we should start by plumbing one of those bits in?  I guess the
> gross part is that implies an ondisk inode format change or (gross)
> xattr lookups in the open path.
>

I may be wrong, but I think there is a confusion in this thread.
I don't think that Pali was looking for filesystems to implement
storing those attributes. I read his email as a request to standardize
a user API to get/set those attributes for the filesystems that
already support them and possibly for vfs to enforce some of them
(e.g. READONLY) in generic code.

Nevertheless, I understand the confusion because I know there
is also demand for storing those attributes by file servers in a
standard way and for vfs to respect those attributes on the host.

Full disclosure - I have an out of tree xfs patch that implements
ioctls XFS_IOC_[GS]ETDOSATTRAT and stashes these
attributes in the unused di_dmevmask space.

Compared to the smb server alternative of storing those attributes
as xattrs on the server, this saves a *lot* of IO in an SMB file browsing
workload, where most of the inodes have large (ACL) xattrs that do
not fit into the inode, because SMB protocol needs to return
those attributes in a response to READDIR(PLUSPLUS), so
it needs to read all the external xattr blocks.

So yeh, I would love to have proper support in xfs...

Thanks,
Amir.

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Friday 17 January 2025 19:46:30 Amir Goldstein wrote:
> > > Looking at the FILE_ATTRIBUTE_* flags defined in SMB protocol
> > >  (fs/smb/common/smb2pdu.h) I wonder how many of them will be
> > > needed for applications beyond the obvious ones that were listed.
> >
> > Well they only asked for seven of them. ;)
> >
> > I chatted with Ted about this yesterday, and ... some of the attributes
> > (like read only) imply that you'd want the linux server to enforce no
> > writing to the file; some like archive seem a little superfluous since
> > on linux you can compare cmtime from the backup against what's in the
> > file now; and still others (like hidden/system) might just be some dorky
> > thing that could be hidden in some xattr because a unix filesystem won't
> > care.
> >
> > And then there are other attrs like "integrity stream" where someone
> > with more experience with windows would have to tell me if fsverity
> > provides sufficient behaviors or not.
> >
> > But maybe we should start by plumbing one of those bits in?  I guess the
> > gross part is that implies an ondisk inode format change or (gross)
> > xattr lookups in the open path.
> >
> 
> I may be wrong, but I think there is a confusion in this thread.
> I don't think that Pali was looking for filesystems to implement
> storing those attributes. I read his email as a request to standardize
> a user API to get/set those attributes for the filesystems that
> already support them and possibly for vfs to enforce some of them
> (e.g. READONLY) in generic code.

Yes, you understood it correctly. I was asking for standardizing API how
to get/set these attributes from userspace. And Chuck wrote that would
like to have also standardized it for kernel consumers like nfsd or
ksmbd. Which makes sense.

> Nevertheless, I understand the confusion because I know there
> is also demand for storing those attributes by file servers in a
> standard way and for vfs to respect those attributes on the host.

Userspace fileserver, like Samba, would just use that standardized
userspace API for get/set attributes. And in-kernel fileservers like
nfsd or ksmbd would like to use that API too.

And there were some proposals how filesystems without native
support for these attributes could store and preserve these attributes.
So this can be a confusion in this email thread discussion.

> Full disclosure - I have an out of tree xfs patch that implements
> ioctls XFS_IOC_[GS]ETDOSATTRAT and stashes these
> attributes in the unused di_dmevmask space.
> 
> Compared to the smb server alternative of storing those attributes
> as xattrs on the server, this saves a *lot* of IO in an SMB file browsing
> workload, where most of the inodes have large (ACL) xattrs that do
> not fit into the inode, because SMB protocol needs to return
> those attributes in a response to READDIR(PLUSPLUS), so
> it needs to read all the external xattr blocks.
> 
> So yeh, I would love to have proper support in xfs...
> 
> Thanks,
> Amir.

So you would also benefit from standardizing of API for these attributes
as then you could implement that API for xfs.

Re: Immutable vs read-only for Windows compatibility

[Darrick J. Wong]

On Fri, Jan 17, 2025 at 07:46:30PM +0100, Amir Goldstein wrote:
> > > Looking at the FILE_ATTRIBUTE_* flags defined in SMB protocol
> > >  (fs/smb/common/smb2pdu.h) I wonder how many of them will be
> > > needed for applications beyond the obvious ones that were listed.
> >
> > Well they only asked for seven of them. ;)
> >
> > I chatted with Ted about this yesterday, and ... some of the attributes
> > (like read only) imply that you'd want the linux server to enforce no
> > writing to the file; some like archive seem a little superfluous since
> > on linux you can compare cmtime from the backup against what's in the
> > file now; and still others (like hidden/system) might just be some dorky
> > thing that could be hidden in some xattr because a unix filesystem won't
> > care.
> >
> > And then there are other attrs like "integrity stream" where someone
> > with more experience with windows would have to tell me if fsverity
> > provides sufficient behaviors or not.
> >
> > But maybe we should start by plumbing one of those bits in?  I guess the
> > gross part is that implies an ondisk inode format change or (gross)
> > xattr lookups in the open path.
> >
> 
> I may be wrong, but I think there is a confusion in this thread.
> I don't think that Pali was looking for filesystems to implement
> storing those attributes. I read his email as a request to standardize
> a user API to get/set those attributes for the filesystems that
> already support them and possibly for vfs to enforce some of them
> (e.g. READONLY) in generic code.
> 
> Nevertheless, I understand the confusion because I know there
> is also demand for storing those attributes by file servers in a
> standard way and for vfs to respect those attributes on the host.
> 
> Full disclosure - I have an out of tree xfs patch that implements
> ioctls XFS_IOC_[GS]ETDOSATTRAT and stashes these
> attributes in the unused di_dmevmask space.

[cc linux-xfs]

Urrrrk, please don't fork the xfs ondisk format!

--D

> Compared to the smb server alternative of storing those attributes
> as xattrs on the server, this saves a *lot* of IO in an SMB file browsing
> workload, where most of the inodes have large (ACL) xattrs that do
> not fit into the inode, because SMB protocol needs to return
> those attributes in a response to READDIR(PLUSPLUS), so
> it needs to read all the external xattr blocks.
> 
> So yeh, I would love to have proper support in xfs...
> 
> Thanks,
> Amir.
> 

Re: Immutable vs read-only for Windows compatibility

[Christoph Hellwig]

On Fri, Jan 17, 2025 at 12:21:12PM -0800, Darrick J. Wong wrote:
> > Full disclosure - I have an out of tree xfs patch that implements
> > ioctls XFS_IOC_[GS]ETDOSATTRAT and stashes these
> > attributes in the unused di_dmevmask space.
> 
> [cc linux-xfs]
> 
> Urrrrk, please don't fork the xfs ondisk format!

Yeah, adding your own bits to any file system on-disk format is a huge
no-go.

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Friday 17 January 2025 19:59:47 Pali Rohár wrote:
> On Friday 17 January 2025 19:46:30 Amir Goldstein wrote:
> > > > Looking at the FILE_ATTRIBUTE_* flags defined in SMB protocol
> > > >  (fs/smb/common/smb2pdu.h) I wonder how many of them will be
> > > > needed for applications beyond the obvious ones that were listed.
> > >
> > > Well they only asked for seven of them. ;)
> > >
> > > I chatted with Ted about this yesterday, and ... some of the attributes
> > > (like read only) imply that you'd want the linux server to enforce no
> > > writing to the file; some like archive seem a little superfluous since
> > > on linux you can compare cmtime from the backup against what's in the
> > > file now; and still others (like hidden/system) might just be some dorky
> > > thing that could be hidden in some xattr because a unix filesystem won't
> > > care.
> > >
> > > And then there are other attrs like "integrity stream" where someone
> > > with more experience with windows would have to tell me if fsverity
> > > provides sufficient behaviors or not.
> > >
> > > But maybe we should start by plumbing one of those bits in?  I guess the
> > > gross part is that implies an ondisk inode format change or (gross)
> > > xattr lookups in the open path.
> > >
> > 
> > I may be wrong, but I think there is a confusion in this thread.
> > I don't think that Pali was looking for filesystems to implement
> > storing those attributes. I read his email as a request to standardize
> > a user API to get/set those attributes for the filesystems that
> > already support them and possibly for vfs to enforce some of them
> > (e.g. READONLY) in generic code.
> 
> Yes, you understood it correctly. I was asking for standardizing API how
> to get/set these attributes from userspace. And Chuck wrote that would
> like to have also standardized it for kernel consumers like nfsd or
> ksmbd. Which makes sense.
> 
> > Nevertheless, I understand the confusion because I know there
> > is also demand for storing those attributes by file servers in a
> > standard way and for vfs to respect those attributes on the host.
> 
> Userspace fileserver, like Samba, would just use that standardized
> userspace API for get/set attributes. And in-kernel fileservers like
> nfsd or ksmbd would like to use that API too.
> 
> And there were some proposals how filesystems without native
> support for these attributes could store and preserve these attributes.
> So this can be a confusion in this email thread discussion.

So to recap, for set-API there are possible options:

1) extending FS_IOC_FSSETXATTR / FS_IOC_SETFLAGS for each individual bit

2) creating one new xattr in system namespace which will contain all bit
   flags in one structure

3) creating new xattr in system namespace for each individual flag

Disadvantages for option 1) is that "cp -a" or "rsync -a" does not
preserve them. If in option 2) or 3) those xattrs would be visible in
listxattr() call then this can be advantage, as all flags are properly
preserved when doing "archive" backup.

Do you have any preference which of those options should be used?


And how many bit flags are needed? I have done some investigation. Lets
start with table which describes all 32 possible bit flags which are
used by Windows system and also by filesystems FAT / exFAT / NTFS / ReFS
and also by SMB over network:

bit / attrib.exe flag / SDK constant / description

 0 - R - FILE_ATTRIBUTE_READONLY              - writing to file or deleting it is disallowed
 1 - H - FILE_ATTRIBUTE_HIDDEN                - inode is hidden
 2 - S - FILE_ATTRIBUTE_SYSTEM                - inode is part of operating system
 3 -   - FILE_ATTRIBUTE_VOLUME                - inode is the disk volume label entry
 4 -   - FILE_ATTRIBUTE_DIRECTORY             - inode is directory
 5 - A - FILE_ATTRIBUTE_ARCHIVE               - inode was not archived yet (when set)
 6 -   - FILE_ATTRIBUTE_DEVICE                - inode represents  in-memory device (e.g. C:\), flag not stored on filesystem
 7 -   - FILE_ATTRIBUTE_NORMAL                - no other flag is set (value 0 means to not change flags, bit 7 means to clear all flags)
 8 -   - FILE_ATTRIBUTE_TEMPORARY             - inode data do not have to be flushed to disk
 9 -   - FILE_ATTRIBUTE_SPARSE_FILE           - file is sparse with holes
10 -   - FILE_ATTRIBUTE_REPARSE_POINT         - inode has attached reparse point (symlink is also reparse point)
11 -   - FILE_ATTRIBUTE_COMPRESSED            - file is compressed, for directories it means that newly created inodes would have this flag set
12 - O - FILE_ATTRIBUTE_OFFLINE               - HSM - inode is used by HSM
13 - I - FILE_ATTRIBUTE_NOT_CONTENT_INDEXED   - inode will not be indexed by content indexing service
14 -   - FILE_ATTRIBUTE_ENCRYPTED             - file is encrypted, for directories it means that newly created inodes would have this flag set
15 - V - FILE_ATTRIBUTE_INTEGRITY_STREAM      - fs does checksumming of data and metadata when reading inode, read-only
16 -   - FILE_ATTRIBUTE_VIRTUAL               - inode is in %LocalAppData%\VirtualStore, flag not stored on filesystem
17 - X - FILE_ATTRIBUTE_NO_SCRUB_DATA         - do not use scrubber (proactive background data integrity scanner) on this file, for directories it means that newly created inodes would have this flag set
18 -   - FILE_ATTRIBUTE_EA                    - inode has xattrs, (not in readdir output, shares same bit with FILE_ATTRIBUTE_RECALL_ON_OPEN)
18 -   - FILE_ATTRIBUTE_RECALL_ON_OPEN        - HSM - inode is not stored locally (only in readdir output, shares same bit with FILE_ATTRIBUTE_EA)
19 - P - FILE_ATTRIBUTE_PINNED                - HSM - inode data content must be always stored on locally
20 - U - FILE_ATTRIBUTE_UNPINNED              - HSM - inode data content can be removed from local storage
21 -   -                                      - reserved
22 -   - FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS - HSM - inode data content is not stored locally
23 -   -                                      - reserved
24 -   -                                      - reserved
25 -   -                                      - reserved
26 -   -                                      - reserved
27 -   -                                      - reserved
28 -   -                                      - reserved
29 - B - FILE_ATTRIBUTE_STRICTLY_SEQUENTIAL   - SMR Blob, unknown meaning, read-only
30 -   -                                      - reserved
31 -   -                                      - reserved

(HSM means Hierarchical Storage Management software, which uses reparse
points to make some remote file/folder available on the local
filesystem, for example OneDrive or DropBox)

From above list only following bit flags are suitable for modification
over some Linux API:
- FILE_ATTRIBUTE_READONLY
- FILE_ATTRIBUTE_HIDDEN
- FILE_ATTRIBUTE_SYSTEM
- FILE_ATTRIBUTE_ARCHIVE
- FILE_ATTRIBUTE_TEMPORARY
- FILE_ATTRIBUTE_COMPRESSED
- FILE_ATTRIBUTE_OFFLINE
- FILE_ATTRIBUTE_NOT_CONTENT_INDEXED
- FILE_ATTRIBUTE_ENCRYPTED
- FILE_ATTRIBUTE_NO_SCRUB_DATA
- FILE_ATTRIBUTE_PINNED
- FILE_ATTRIBUTE_UNPINNED

And if I'm looking correctly the FILE_ATTRIBUTE_COMPRESSED can be
already mapped to Linux FS_COMPR_FL / STATX_ATTR_COMPRESSED, which has
same meaning. Also FILE_ATTRIBUTE_ENCRYPTED can be mapped to
FS_ENCRYPT_FL / STATX_ATTR_ENCRYPTED. Note that these two flags cannot
be set over WinAPI or SMB directly and it is required to use special
WinAPI or SMB ioctl.

So totally are needed 10 new bit flags. And for future there are 9
reserved bits which could be introduced by MS in future.

Additionally there are get-only attributes which can be useful for statx
purposes (for example exported by cifs.ko SMB client):
- FILE_ATTRIBUTE_REPARSE_POINT
- FILE_ATTRIBUTE_INTEGRITY_STREAM
- FILE_ATTRIBUTE_RECALL_ON_OPEN
- FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS
- FILE_ATTRIBUTE_STRICTLY_SEQUENTIAL

From the above list of flags suitable for modification, following bit
flags have no meaning for kernel and it is up to userspace how will use
them. What is needed from kernel and/or filesystem driver is to preserve
those bit flags.
- FILE_ATTRIBUTE_HIDDEN
- FILE_ATTRIBUTE_SYSTEM
- FILE_ATTRIBUTE_ARCHIVE
- FILE_ATTRIBUTE_NOT_CONTENT_INDEXED

Following are bit flags which kernel / VFS / fsdriver would have to
handle specially, to provide enforcement or correct behavior of them:
- FILE_ATTRIBUTE_READONLY - enforce that data modification or unlink is disallowed when set
- FILE_ATTRIBUTE_COMPRESSED - enforce compression on filesystem when set
- FILE_ATTRIBUTE_ENCRYPTED - enforce encryption on filesystem when set

Then there are HSM flags which for local filesystem would need some
cooperation with userspace synchronization software. For network
filesystems (SMB / NFS4) they need nothing special, just properly
propagating them over network:
- FILE_ATTRIBUTE_OFFLINE
- FILE_ATTRIBUTE_PINNED
- FILE_ATTRIBUTE_UNPINNED

About following 2 flags, I'm not sure if the kernel / VFS / fs driver
has to do something or it can just store bits to fs:
- FILE_ATTRIBUTE_TEMPORARY
- FILE_ATTRIBUTE_NO_SCRUB_DATA


=======================================================================

And there is still unresolved issue with FILE_ATTRIBUTE_READONLY.
Its meaning is similar to existing Linux FS_IMMUTABLE_FL, just
FILE_ATTRIBUTE_READONLY does not require root / CAP_LINUX_IMMUTABLE.

I think that for proper support, to enforce FILE_ATTRIBUTE_READONLY
functionality, it is needed to introduce new flag e.g.
FS_IMMUTABLE_FL_USER to allow setting / clearing it also for normal
users without CAP_LINUX_IMMUTABLE. Otherwise it would be unsuitable for
any SMB client, SMB server or any application which would like to use
it, for example wine.

Just to note that FreeBSD has two immutable flags SF_IMMUTABLE and
UF_IMMUTABLE, one settable only by superuser and second for owner.

Any opinion?

Re: Immutable vs read-only for Windows compatibility

[Amir Goldstein]

On Sun, Feb 2, 2025 at 4:23 PM Pali Rohár <pali@kernel.org> wrote:
>
> On Friday 17 January 2025 19:59:47 Pali Rohár wrote:
> > On Friday 17 January 2025 19:46:30 Amir Goldstein wrote:
> > > > > Looking at the FILE_ATTRIBUTE_* flags defined in SMB protocol
> > > > >  (fs/smb/common/smb2pdu.h) I wonder how many of them will be
> > > > > needed for applications beyond the obvious ones that were listed.
> > > >
> > > > Well they only asked for seven of them. ;)
> > > >
> > > > I chatted with Ted about this yesterday, and ... some of the attributes
> > > > (like read only) imply that you'd want the linux server to enforce no
> > > > writing to the file; some like archive seem a little superfluous since
> > > > on linux you can compare cmtime from the backup against what's in the
> > > > file now; and still others (like hidden/system) might just be some dorky
> > > > thing that could be hidden in some xattr because a unix filesystem won't
> > > > care.
> > > >
> > > > And then there are other attrs like "integrity stream" where someone
> > > > with more experience with windows would have to tell me if fsverity
> > > > provides sufficient behaviors or not.
> > > >
> > > > But maybe we should start by plumbing one of those bits in?  I guess the
> > > > gross part is that implies an ondisk inode format change or (gross)
> > > > xattr lookups in the open path.
> > > >
> > >
> > > I may be wrong, but I think there is a confusion in this thread.
> > > I don't think that Pali was looking for filesystems to implement
> > > storing those attributes. I read his email as a request to standardize
> > > a user API to get/set those attributes for the filesystems that
> > > already support them and possibly for vfs to enforce some of them
> > > (e.g. READONLY) in generic code.
> >
> > Yes, you understood it correctly. I was asking for standardizing API how
> > to get/set these attributes from userspace. And Chuck wrote that would
> > like to have also standardized it for kernel consumers like nfsd or
> > ksmbd. Which makes sense.
> >
> > > Nevertheless, I understand the confusion because I know there
> > > is also demand for storing those attributes by file servers in a
> > > standard way and for vfs to respect those attributes on the host.
> >
> > Userspace fileserver, like Samba, would just use that standardized
> > userspace API for get/set attributes. And in-kernel fileservers like
> > nfsd or ksmbd would like to use that API too.
> >
> > And there were some proposals how filesystems without native
> > support for these attributes could store and preserve these attributes.
> > So this can be a confusion in this email thread discussion.
>
> So to recap, for set-API there are possible options:
>
> 1) extending FS_IOC_FSSETXATTR / FS_IOC_SETFLAGS for each individual bit
>
> 2) creating one new xattr in system namespace which will contain all bit
>    flags in one structure
>
> 3) creating new xattr in system namespace for each individual flag
>
> Disadvantages for option 1) is that "cp -a" or "rsync -a" does not
> preserve them. If in option 2) or 3) those xattrs would be visible in
> listxattr() call then this can be advantage, as all flags are properly
> preserved when doing "archive" backup.
>
> Do you have any preference which of those options should be used?
>

Darrick said in this thread twice:
statx/FS_IOC_FSGETXATTR to retrieve and FS_IOC_FSSETXATTR to set.
(NOT FS_IOC_SETFLAGS)
and I wrote that I agree with him.

I suggest that you let go of the cp -a out-of-the-box requirement.
It is not going to pass muster - maybe for a specific filesystem but
as a filesystem agnostic feature, unless you change cp tool.

>
> And how many bit flags are needed? I have done some investigation. Lets
> start with table which describes all 32 possible bit flags which are
> used by Windows system and also by filesystems FAT / exFAT / NTFS / ReFS
> and also by SMB over network:
>
> bit / attrib.exe flag / SDK constant / description
>
>  0 - R - FILE_ATTRIBUTE_READONLY              - writing to file or deleting it is disallowed
>  1 - H - FILE_ATTRIBUTE_HIDDEN                - inode is hidden
>  2 - S - FILE_ATTRIBUTE_SYSTEM                - inode is part of operating system
>  3 -   - FILE_ATTRIBUTE_VOLUME                - inode is the disk volume label entry
>  4 -   - FILE_ATTRIBUTE_DIRECTORY             - inode is directory
>  5 - A - FILE_ATTRIBUTE_ARCHIVE               - inode was not archived yet (when set)
>  6 -   - FILE_ATTRIBUTE_DEVICE                - inode represents  in-memory device (e.g. C:\), flag not stored on filesystem
>  7 -   - FILE_ATTRIBUTE_NORMAL                - no other flag is set (value 0 means to not change flags, bit 7 means to clear all flags)
>  8 -   - FILE_ATTRIBUTE_TEMPORARY             - inode data do not have to be flushed to disk
>  9 -   - FILE_ATTRIBUTE_SPARSE_FILE           - file is sparse with holes
> 10 -   - FILE_ATTRIBUTE_REPARSE_POINT         - inode has attached reparse point (symlink is also reparse point)
> 11 -   - FILE_ATTRIBUTE_COMPRESSED            - file is compressed, for directories it means that newly created inodes would have this flag set
> 12 - O - FILE_ATTRIBUTE_OFFLINE               - HSM - inode is used by HSM
> 13 - I - FILE_ATTRIBUTE_NOT_CONTENT_INDEXED   - inode will not be indexed by content indexing service
> 14 -   - FILE_ATTRIBUTE_ENCRYPTED             - file is encrypted, for directories it means that newly created inodes would have this flag set
> 15 - V - FILE_ATTRIBUTE_INTEGRITY_STREAM      - fs does checksumming of data and metadata when reading inode, read-only
> 16 -   - FILE_ATTRIBUTE_VIRTUAL               - inode is in %LocalAppData%\VirtualStore, flag not stored on filesystem
> 17 - X - FILE_ATTRIBUTE_NO_SCRUB_DATA         - do not use scrubber (proactive background data integrity scanner) on this file, for directories it means that newly created inodes would have this flag set
> 18 -   - FILE_ATTRIBUTE_EA                    - inode has xattrs, (not in readdir output, shares same bit with FILE_ATTRIBUTE_RECALL_ON_OPEN)
> 18 -   - FILE_ATTRIBUTE_RECALL_ON_OPEN        - HSM - inode is not stored locally (only in readdir output, shares same bit with FILE_ATTRIBUTE_EA)
> 19 - P - FILE_ATTRIBUTE_PINNED                - HSM - inode data content must be always stored on locally
> 20 - U - FILE_ATTRIBUTE_UNPINNED              - HSM - inode data content can be removed from local storage
> 21 -   -                                      - reserved
> 22 -   - FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS - HSM - inode data content is not stored locally
> 23 -   -                                      - reserved
> 24 -   -                                      - reserved
> 25 -   -                                      - reserved
> 26 -   -                                      - reserved
> 27 -   -                                      - reserved
> 28 -   -                                      - reserved
> 29 - B - FILE_ATTRIBUTE_STRICTLY_SEQUENTIAL   - SMR Blob, unknown meaning, read-only
> 30 -   -                                      - reserved
> 31 -   -                                      - reserved
>

I suspect that we need to reserve expansion for more than 7 bits if we
design a proper API.
The fsx_xflags field is already too crowded for adding so many flags
We can use the padding at the end of fsxattr to add __u32 fsx_dosattrib
or fsx_dosflags field.

> (HSM means Hierarchical Storage Management software, which uses reparse
> points to make some remote file/folder available on the local
> filesystem, for example OneDrive or DropBox)
>

I am quite interested in supporting those HSM flags for fanotify.

> From above list only following bit flags are suitable for modification
> over some Linux API:
> - FILE_ATTRIBUTE_READONLY
> - FILE_ATTRIBUTE_HIDDEN
> - FILE_ATTRIBUTE_SYSTEM
> - FILE_ATTRIBUTE_ARCHIVE
> - FILE_ATTRIBUTE_TEMPORARY
> - FILE_ATTRIBUTE_COMPRESSED
> - FILE_ATTRIBUTE_OFFLINE
> - FILE_ATTRIBUTE_NOT_CONTENT_INDEXED
> - FILE_ATTRIBUTE_ENCRYPTED
> - FILE_ATTRIBUTE_NO_SCRUB_DATA
> - FILE_ATTRIBUTE_PINNED
> - FILE_ATTRIBUTE_UNPINNED
>
> And if I'm looking correctly the FILE_ATTRIBUTE_COMPRESSED can be
> already mapped to Linux FS_COMPR_FL / STATX_ATTR_COMPRESSED, which has
> same meaning. Also FILE_ATTRIBUTE_ENCRYPTED can be mapped to
> FS_ENCRYPT_FL / STATX_ATTR_ENCRYPTED. Note that these two flags cannot
> be set over WinAPI or SMB directly and it is required to use special
> WinAPI or SMB ioctl.
>

There is a standard way to map from fileattr::flags
to fileattr::fsx_xflags, so that you could set/get COMPR,ENCRYPT using
FS_IOC_FS[GS]ETXATTR ioctl.
see fileattr_fill_flags/fileattr_fill_xflags.
but I think that xfs_fileattr_set() will need to have a supported xflags mask
check if you start adding xflags that xfs does not currently support and
other filesystems do support.

> So totally are needed 10 new bit flags. And for future there are 9
> reserved bits which could be introduced by MS in future.
>
> Additionally there are get-only attributes which can be useful for statx
> purposes (for example exported by cifs.ko SMB client):
> - FILE_ATTRIBUTE_REPARSE_POINT
> - FILE_ATTRIBUTE_INTEGRITY_STREAM
> - FILE_ATTRIBUTE_RECALL_ON_OPEN
> - FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS
> - FILE_ATTRIBUTE_STRICTLY_SEQUENTIAL
>
> From the above list of flags suitable for modification, following bit
> flags have no meaning for kernel and it is up to userspace how will use
> them. What is needed from kernel and/or filesystem driver is to preserve
> those bit flags.
> - FILE_ATTRIBUTE_HIDDEN
> - FILE_ATTRIBUTE_SYSTEM
> - FILE_ATTRIBUTE_ARCHIVE
> - FILE_ATTRIBUTE_NOT_CONTENT_INDEXED
>
> Following are bit flags which kernel / VFS / fsdriver would have to
> handle specially, to provide enforcement or correct behavior of them:
> - FILE_ATTRIBUTE_READONLY - enforce that data modification or unlink is disallowed when set
> - FILE_ATTRIBUTE_COMPRESSED - enforce compression on filesystem when set
> - FILE_ATTRIBUTE_ENCRYPTED - enforce encryption on filesystem when set
>
> Then there are HSM flags which for local filesystem would need some
> cooperation with userspace synchronization software. For network
> filesystems (SMB / NFS4) they need nothing special, just properly
> propagating them over network:
> - FILE_ATTRIBUTE_OFFLINE
> - FILE_ATTRIBUTE_PINNED
> - FILE_ATTRIBUTE_UNPINNED
>
> About following 2 flags, I'm not sure if the kernel / VFS / fs driver
> has to do something or it can just store bits to fs:
> - FILE_ATTRIBUTE_TEMPORARY
> - FILE_ATTRIBUTE_NO_SCRUB_DATA
>
>
> =======================================================================
>
> And there is still unresolved issue with FILE_ATTRIBUTE_READONLY.
> Its meaning is similar to existing Linux FS_IMMUTABLE_FL, just
> FILE_ATTRIBUTE_READONLY does not require root / CAP_LINUX_IMMUTABLE.
>
> I think that for proper support, to enforce FILE_ATTRIBUTE_READONLY
> functionality, it is needed to introduce new flag e.g.
> FS_IMMUTABLE_FL_USER to allow setting / clearing it also for normal
> users without CAP_LINUX_IMMUTABLE. Otherwise it would be unsuitable for
> any SMB client, SMB server or any application which would like to use
> it, for example wine.
>
> Just to note that FreeBSD has two immutable flags SF_IMMUTABLE and
> UF_IMMUTABLE, one settable only by superuser and second for owner.
>
> Any opinion?

For filesystems that already support FILE_ATTRIBUTE_READONLY,
can't you just set S_IMMUTABLE on the inode and vfs will do the correct
enforcement?

The vfs does not control if and how S_IMMUTABLE is set by filesystems,
so if you want to remove this vfs flag without CAP_LINUX_IMMUTABLE
in smb client, there is nothing stopping you (I think).

How about tackling this one small step at a time, not in that order
necessarily:

1. Implement the standard API with FS_IOC_FS[GS]ETXATTR ioctl
    and with statx to get/set some non-controversial dosattrib flags on
    ntfs/smb/vfat
2. Wire some interesting dosattrib flags (e.g. compr/enrypt) to local
    filesystems that already support storing those bits
3. Wire network servers (e.g. Samba) to use the generic API if supported
4. Add on-disk support for storing the dosattrib flags to more local fs
5. Update S_IMMUTABLE inode flag if either FS_XFLAG_IMMUTABLE
    or FS_DOSATTRIB_READONLY are set on the file

Thoughts?

Thanks,
Amir.

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Monday 03 February 2025 22:59:46 Amir Goldstein wrote:
> On Sun, Feb 2, 2025 at 4:23 PM Pali Rohár <pali@kernel.org> wrote:
> > And there is still unresolved issue with FILE_ATTRIBUTE_READONLY.
> > Its meaning is similar to existing Linux FS_IMMUTABLE_FL, just
> > FILE_ATTRIBUTE_READONLY does not require root / CAP_LINUX_IMMUTABLE.
> >
> > I think that for proper support, to enforce FILE_ATTRIBUTE_READONLY
> > functionality, it is needed to introduce new flag e.g.
> > FS_IMMUTABLE_FL_USER to allow setting / clearing it also for normal
> > users without CAP_LINUX_IMMUTABLE. Otherwise it would be unsuitable for
> > any SMB client, SMB server or any application which would like to use
> > it, for example wine.
> >
> > Just to note that FreeBSD has two immutable flags SF_IMMUTABLE and
> > UF_IMMUTABLE, one settable only by superuser and second for owner.
> >
> > Any opinion?
> 
> For filesystems that already support FILE_ATTRIBUTE_READONLY,
> can't you just set S_IMMUTABLE on the inode and vfs will do the correct
> enforcement?
> 
> The vfs does not control if and how S_IMMUTABLE is set by filesystems,
> so if you want to remove this vfs flag without CAP_LINUX_IMMUTABLE
> in smb client, there is nothing stopping you (I think).

Function fileattr_set_prepare() checks for CAP_LINUX_IMMUTABLE when
trying to change FS_IMMUTABLE_FL bit. This function is called from
ioctl(FS_IOC_SETFLAGS) and also from ioctl(FS_IOC_FSSETXATTR).
And when function fileattr_set_prepare() fails then .fileattr_set
callback is not called at all. So I think that it is not possible to
remove the IMMUTABLE flag from userspace without capability for smb
client.

And it would not solve this problem for local filesystems (ntfs or ext4)
when Samba server or wine would want to set this bit.

> How about tackling this one small step at a time, not in that order
> necessarily:
> 
> 1. Implement the standard API with FS_IOC_FS[GS]ETXATTR ioctl
>     and with statx to get/set some non-controversial dosattrib flags on
>     ntfs/smb/vfat
> 2. Wire some interesting dosattrib flags (e.g. compr/enrypt) to local
>     filesystems that already support storing those bits
> 3. Wire network servers (e.g. Samba) to use the generic API if supported
> 4. Add on-disk support for storing the dosattrib flags to more local fs
> 5. Update S_IMMUTABLE inode flag if either FS_XFLAG_IMMUTABLE
>     or FS_DOSATTRIB_READONLY are set on the file
> 
> Thoughts?
> 
> Thanks,
> Amir.

Re: Immutable vs read-only for Windows compatibility

[Amir Goldstein]

On Mon, Feb 3, 2025 at 11:20 PM Pali Rohár <pali@kernel.org> wrote:
>
> On Monday 03 February 2025 22:59:46 Amir Goldstein wrote:
> > On Sun, Feb 2, 2025 at 4:23 PM Pali Rohár <pali@kernel.org> wrote:
> > > And there is still unresolved issue with FILE_ATTRIBUTE_READONLY.
> > > Its meaning is similar to existing Linux FS_IMMUTABLE_FL, just
> > > FILE_ATTRIBUTE_READONLY does not require root / CAP_LINUX_IMMUTABLE.
> > >
> > > I think that for proper support, to enforce FILE_ATTRIBUTE_READONLY
> > > functionality, it is needed to introduce new flag e.g.
> > > FS_IMMUTABLE_FL_USER to allow setting / clearing it also for normal
> > > users without CAP_LINUX_IMMUTABLE. Otherwise it would be unsuitable for
> > > any SMB client, SMB server or any application which would like to use
> > > it, for example wine.
> > >
> > > Just to note that FreeBSD has two immutable flags SF_IMMUTABLE and
> > > UF_IMMUTABLE, one settable only by superuser and second for owner.
> > >
> > > Any opinion?
> >
> > For filesystems that already support FILE_ATTRIBUTE_READONLY,
> > can't you just set S_IMMUTABLE on the inode and vfs will do the correct
> > enforcement?
> >
> > The vfs does not control if and how S_IMMUTABLE is set by filesystems,
> > so if you want to remove this vfs flag without CAP_LINUX_IMMUTABLE
> > in smb client, there is nothing stopping you (I think).
>
> Function fileattr_set_prepare() checks for CAP_LINUX_IMMUTABLE when
> trying to change FS_IMMUTABLE_FL bit. This function is called from
> ioctl(FS_IOC_SETFLAGS) and also from ioctl(FS_IOC_FSSETXATTR).
> And when function fileattr_set_prepare() fails then .fileattr_set
> callback is not called at all. So I think that it is not possible to
> remove the IMMUTABLE flag from userspace without capability for smb
> client.
>

You did not understand what I meant.

You cannot relax the CAP_LINUX_IMMUTABLE for setting FS_IMMUTABLE_FL
and there is no reason that you will need to relax it.

The vfs does NOT enforce permissions according to FS_IMMUTABLE_FL
The vfs enforces permissions according to the S_IMMUTABLE in-memory
inode flag.

There is no generic vfs code that sets S_IMMUTABLE inode flags, its
the filesystems that translate the on-disk FS_IMMUTABLE_FL to
in-memory S_IMMUTABLE inode flag.

So if a filesystem already has an internal DOSATTRIB flags set, this
filesystem can set the in-memory S_IMMUTABLE inode flag according
to its knowledge of the DOSATTRIB_READONLY flag and the
CAP_LINUX_IMMUTABLE rules do not apply to the DOSATTRIB_READONLY
flag, which is NOT the same as the FS_IMMUTABLE_FL flag.

> And it would not solve this problem for local filesystems (ntfs or ext4)
> when Samba server or wine would want to set this bit.
>

The Samba server would use the FS_IOC_FS[GS]ETXATTR ioctl
API to get/set dosattrib, something like this:

struct fsxattr fsxattr;
ret = ioctl_get_fsxattr(fd, &fsxattr);
if (!ret && fsxattr.fsx_xflags & FS_XFLAG_HASDOSATTR) {
    fsxattr.fsx_dosattr |= FS_DOSATTRIB_READONLY;
    ret = ioctl_set_fsxattr(fd, &fsxattr);
}

For ntfs/ext4, you will need to implement on-disk support for
set/get the dosattrib flags.

I can certainly not change the meaning of existing on-disk
flag of FS_IMMUTABLE_FL to a flag that can be removed
without CAP_LINUX_IMMUTABLE. that changes the meaning
of the flag.

If ext4 maintainers agrees, you may be able to reuse some
old unused on-disk flags (e.g.  EXT4_UNRM_FL) as storage
place for FS_DOSATTRIB_READONLY, but that would be
quite hackish.

> > How about tackling this one small step at a time, not in that order
> > necessarily:
> >
> > 1. Implement the standard API with FS_IOC_FS[GS]ETXATTR ioctl
> >     and with statx to get/set some non-controversial dosattrib flags on
> >     ntfs/smb/vfat
> > 2. Wire some interesting dosattrib flags (e.g. compr/enrypt) to local
> >     filesystems that already support storing those bits
> > 3. Wire network servers (e.g. Samba) to use the generic API if supported
> > 4. Add on-disk support for storing the dosattrib flags to more local fs
> > 5. Update S_IMMUTABLE inode flag if either FS_XFLAG_IMMUTABLE
> >     or FS_DOSATTRIB_READONLY are set on the file
> >
> > Thoughts?
> >

Anything wrong with the plan above?
It seems that you are looking for shortcuts and I don't think that
it is a good way to make progress.

Thanks,
Amir.

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Tuesday 04 February 2025 00:02:44 Amir Goldstein wrote:
> On Mon, Feb 3, 2025 at 11:20 PM Pali Rohár <pali@kernel.org> wrote:
> >
> > On Monday 03 February 2025 22:59:46 Amir Goldstein wrote:
> > > On Sun, Feb 2, 2025 at 4:23 PM Pali Rohár <pali@kernel.org> wrote:
> > > > And there is still unresolved issue with FILE_ATTRIBUTE_READONLY.
> > > > Its meaning is similar to existing Linux FS_IMMUTABLE_FL, just
> > > > FILE_ATTRIBUTE_READONLY does not require root / CAP_LINUX_IMMUTABLE.
> > > >
> > > > I think that for proper support, to enforce FILE_ATTRIBUTE_READONLY
> > > > functionality, it is needed to introduce new flag e.g.
> > > > FS_IMMUTABLE_FL_USER to allow setting / clearing it also for normal
> > > > users without CAP_LINUX_IMMUTABLE. Otherwise it would be unsuitable for
> > > > any SMB client, SMB server or any application which would like to use
> > > > it, for example wine.
> > > >
> > > > Just to note that FreeBSD has two immutable flags SF_IMMUTABLE and
> > > > UF_IMMUTABLE, one settable only by superuser and second for owner.
> > > >
> > > > Any opinion?
> > >
> > > For filesystems that already support FILE_ATTRIBUTE_READONLY,
> > > can't you just set S_IMMUTABLE on the inode and vfs will do the correct
> > > enforcement?
> > >
> > > The vfs does not control if and how S_IMMUTABLE is set by filesystems,
> > > so if you want to remove this vfs flag without CAP_LINUX_IMMUTABLE
> > > in smb client, there is nothing stopping you (I think).
> >
> > Function fileattr_set_prepare() checks for CAP_LINUX_IMMUTABLE when
> > trying to change FS_IMMUTABLE_FL bit. This function is called from
> > ioctl(FS_IOC_SETFLAGS) and also from ioctl(FS_IOC_FSSETXATTR).
> > And when function fileattr_set_prepare() fails then .fileattr_set
> > callback is not called at all. So I think that it is not possible to
> > remove the IMMUTABLE flag from userspace without capability for smb
> > client.
> >
> 
> You did not understand what I meant.
> 
> You cannot relax the CAP_LINUX_IMMUTABLE for setting FS_IMMUTABLE_FL
> and there is no reason that you will need to relax it.
> 
> The vfs does NOT enforce permissions according to FS_IMMUTABLE_FL
> The vfs enforces permissions according to the S_IMMUTABLE in-memory
> inode flag.
> 
> There is no generic vfs code that sets S_IMMUTABLE inode flags, its
> the filesystems that translate the on-disk FS_IMMUTABLE_FL to
> in-memory S_IMMUTABLE inode flag.
> 
> So if a filesystem already has an internal DOSATTRIB flags set, this
> filesystem can set the in-memory S_IMMUTABLE inode flag according
> to its knowledge of the DOSATTRIB_READONLY flag and the
> CAP_LINUX_IMMUTABLE rules do not apply to the DOSATTRIB_READONLY
> flag, which is NOT the same as the FS_IMMUTABLE_FL flag.
> 
> > And it would not solve this problem for local filesystems (ntfs or ext4)
> > when Samba server or wine would want to set this bit.
> >
> 
> The Samba server would use the FS_IOC_FS[GS]ETXATTR ioctl
> API to get/set dosattrib, something like this:
> 
> struct fsxattr fsxattr;
> ret = ioctl_get_fsxattr(fd, &fsxattr);
> if (!ret && fsxattr.fsx_xflags & FS_XFLAG_HASDOSATTR) {
>     fsxattr.fsx_dosattr |= fs_dosattrib_readonly;
>     ret = ioctl_set_fsxattr(fd, &fsxattr);
> }

Thanks for more explanation. First time I really did not understood it.
But now I think I understood it. So basically there would be two flags
which would result in setting S_IMMUTABLE on inode. One is the existing
FS_IMMUTABLE_FL which requires the capability and some new flag (e.g.
FS_XFLAG_HASDOSATTR) which would not require it and can be implemented
for cifs, vfat, ntfs, ... Right?

> For ntfs/ext4, you will need to implement on-disk support for
> set/get the dosattrib flags.

ntfs has already on-disk support for FILE_ATTRIBUTE_READONLY.

On-disk support for ext4 and other linux filesystems can be discussed
later. I think that this could be more controversial.

> I can certainly not change the meaning of existing on-disk
> flag of FS_IMMUTABLE_FL to a flag that can be removed
> without CAP_LINUX_IMMUTABLE. that changes the meaning
> of the flag.
> 
> If ext4 maintainers agrees, you may be able to reuse some
> old unused on-disk flags (e.g.  EXT4_UNRM_FL) as storage
> place for FS_DOSATTRIB_READONLY, but that would be
> quite hackish.
> 
> > > How about tackling this one small step at a time, not in that order
> > > necessarily:
> > >
> > > 1. Implement the standard API with FS_IOC_FS[GS]ETXATTR ioctl
> > >     and with statx to get/set some non-controversial dosattrib flags on
> > >     ntfs/smb/vfat
> > > 2. Wire some interesting dosattrib flags (e.g. compr/enrypt) to local
> > >     filesystems that already support storing those bits
> > > 3. Wire network servers (e.g. Samba) to use the generic API if supported
> > > 4. Add on-disk support for storing the dosattrib flags to more local fs
> > > 5. Update S_IMMUTABLE inode flag if either FS_XFLAG_IMMUTABLE
> > >     or FS_DOSATTRIB_READONLY are set on the file
> > >
> > > Thoughts?
> > >
> 
> Anything wrong with the plan above?
> It seems that you are looking for shortcuts and I don't think that
> it is a good way to make progress.
> 
> Thanks,
> Amir.

If other developers agree that the FS_IOC_FS[GS]ETXATTR ioctl is the
right direction then for me it looks good.

Re: Immutable vs read-only for Windows compatibility

[Amir Goldstein]

On Tue, Feb 4, 2025 at 12:34 AM Pali Rohár <pali@kernel.org> wrote:
>
> On Tuesday 04 February 2025 00:02:44 Amir Goldstein wrote:
> > On Mon, Feb 3, 2025 at 11:20 PM Pali Rohár <pali@kernel.org> wrote:
> > >
> > > On Monday 03 February 2025 22:59:46 Amir Goldstein wrote:
> > > > On Sun, Feb 2, 2025 at 4:23 PM Pali Rohár <pali@kernel.org> wrote:
> > > > > And there is still unresolved issue with FILE_ATTRIBUTE_READONLY.
> > > > > Its meaning is similar to existing Linux FS_IMMUTABLE_FL, just
> > > > > FILE_ATTRIBUTE_READONLY does not require root / CAP_LINUX_IMMUTABLE.
> > > > >
> > > > > I think that for proper support, to enforce FILE_ATTRIBUTE_READONLY
> > > > > functionality, it is needed to introduce new flag e.g.
> > > > > FS_IMMUTABLE_FL_USER to allow setting / clearing it also for normal
> > > > > users without CAP_LINUX_IMMUTABLE. Otherwise it would be unsuitable for
> > > > > any SMB client, SMB server or any application which would like to use
> > > > > it, for example wine.
> > > > >
> > > > > Just to note that FreeBSD has two immutable flags SF_IMMUTABLE and
> > > > > UF_IMMUTABLE, one settable only by superuser and second for owner.
> > > > >
> > > > > Any opinion?
> > > >
> > > > For filesystems that already support FILE_ATTRIBUTE_READONLY,
> > > > can't you just set S_IMMUTABLE on the inode and vfs will do the correct
> > > > enforcement?
> > > >
> > > > The vfs does not control if and how S_IMMUTABLE is set by filesystems,
> > > > so if you want to remove this vfs flag without CAP_LINUX_IMMUTABLE
> > > > in smb client, there is nothing stopping you (I think).
> > >
> > > Function fileattr_set_prepare() checks for CAP_LINUX_IMMUTABLE when
> > > trying to change FS_IMMUTABLE_FL bit. This function is called from
> > > ioctl(FS_IOC_SETFLAGS) and also from ioctl(FS_IOC_FSSETXATTR).
> > > And when function fileattr_set_prepare() fails then .fileattr_set
> > > callback is not called at all. So I think that it is not possible to
> > > remove the IMMUTABLE flag from userspace without capability for smb
> > > client.
> > >
> >
> > You did not understand what I meant.
> >
> > You cannot relax the CAP_LINUX_IMMUTABLE for setting FS_IMMUTABLE_FL
> > and there is no reason that you will need to relax it.
> >
> > The vfs does NOT enforce permissions according to FS_IMMUTABLE_FL
> > The vfs enforces permissions according to the S_IMMUTABLE in-memory
> > inode flag.
> >
> > There is no generic vfs code that sets S_IMMUTABLE inode flags, its
> > the filesystems that translate the on-disk FS_IMMUTABLE_FL to
> > in-memory S_IMMUTABLE inode flag.
> >
> > So if a filesystem already has an internal DOSATTRIB flags set, this
> > filesystem can set the in-memory S_IMMUTABLE inode flag according
> > to its knowledge of the DOSATTRIB_READONLY flag and the
> > CAP_LINUX_IMMUTABLE rules do not apply to the DOSATTRIB_READONLY
> > flag, which is NOT the same as the FS_IMMUTABLE_FL flag.
> >
> > > And it would not solve this problem for local filesystems (ntfs or ext4)
> > > when Samba server or wine would want to set this bit.
> > >
> >
> > The Samba server would use the FS_IOC_FS[GS]ETXATTR ioctl
> > API to get/set dosattrib, something like this:
> >
> > struct fsxattr fsxattr;
> > ret = ioctl_get_fsxattr(fd, &fsxattr);
> > if (!ret && fsxattr.fsx_xflags & FS_XFLAG_HASDOSATTR) {
> >     fsxattr.fsx_dosattr |= fs_dosattrib_readonly;
> >     ret = ioctl_set_fsxattr(fd, &fsxattr);
> > }
>
> Thanks for more explanation. First time I really did not understood it.
> But now I think I understood it. So basically there would be two flags
> which would result in setting S_IMMUTABLE on inode. One is the existing
> FS_IMMUTABLE_FL which requires the capability and some new flag (e.g.
> FS_XFLAG_HASDOSATTR) which would not require it and can be implemented
> for cifs, vfat, ntfs, ... Right?
>

Well, almost right.
The flag that would correspond to FILE_ATTRIBUTE_READONLY
is FS_DOSATTRIB_READONLY from the new field fsx_dosattrib
(see below)

--- a/include/uapi/linux/fs.h
+++ b/include/uapi/linux/fs.h
@@ -145,7 +145,8 @@ struct fsxattr {
        __u32           fsx_nextents;   /* nextents field value (get)   */
        __u32           fsx_projid;     /* project identifier (get/set) */
        __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
-       unsigned char   fsx_pad[8];
+       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
+       unsigned char   fsx_pad[4];
 };

 /*
@@ -167,7 +168,16 @@ struct fsxattr {
 #define FS_XFLAG_FILESTREAM    0x00004000      /* use filestream allocator */
 #define FS_XFLAG_DAX           0x00008000      /* use DAX for IO */
 #define FS_XFLAG_COWEXTSIZE    0x00010000      /* CoW extent size
allocator hint */
-#define FS_XFLAG_HASATTR       0x80000000      /* no DIFLAG for this   */
+#define FS_XFLAG_HASATTR       0x80000000      /* has extended attributes */
+
+/*
+ * Flags for the fsx_dosattrib field
+ */
+#define FS_DOSATTRIB_READONLY  0x00000001      /* R - read-only */
+#define FS_DOSATTRIB_HIDDEN    0x00000002      /* H - hidden */
+#define FS_DOSATTRIB_SYSTEM    0x00000004      /* S - system */
+#define FS_DOSATTRIB_ARCHIVE   0x00000020      /* A - archive */
+#define FS_DOSATTRIB_HASATTR   0x80000000      /* has dos attributes */

This last special flag is debatable and I am not really sure that we need it.

It is needed for proper backward compat with existing userspace tools.
For example, if there was a backup tool storing the fsxattr blob result of
FS_IOC_FSGETXATTR and sets it later during restore with
FS_IOC_FSSETXATTR, then it would be better to ignore a zero
value of fsx_dosattrib instead of resetting all of the on-disk dosattrib flags
if the restore happens after ntfs gained support for setting dosattrib flags
via FS_IOC_FSSETXATTR.

When using the standard tools to set fsxattr (chattr and xfs_io -c chattr)
the tool does FS_IOC_FSGETXATTR + modify + FS_IOC_FSSETXATTR,
so those tools are expected to leave new bits in fsx_dosattrib at their
value if ntfs gains support for get/set fsx_dosattrib.

Setting the auxiliary FS_DOSATTRIB_HASATTR flag can help the
kernel/fs to explicitly state that the values returned in fsx_dosattrib
are valid and the tool to state that values set in fsx_dosattrib are valid.
But using a single flag will not help expanding ntfs support for more
fsx_dosattrib flags later, so I am not sure if it is useful (?).


> > For ntfs/ext4, you will need to implement on-disk support for
> > set/get the dosattrib flags.
>
> ntfs has already on-disk support for FILE_ATTRIBUTE_READONLY.
>

This is interesting.

fat/ntfs both already have a mount option sys_immutable to map
FILE_ATTRIBUTE_SYSTEM to S_IMMUTABLE in-memory.

fat does not support fileattr_set(), but has a proprietary ioctl
FAT_IOCTL_SET_ATTRIBUTES which enforces
CAP_LINUX_IMMUTABLE for changing S_IMMUTABLE.

ntfs also maps FILE_ATTRIBUTE_SYSTEM to S_IMMUTABLE
and it allows changing FILE_ATTRIBUTE_SYSTEM via ntfs_setxattr
of system.{dos,ntfs}_attrib without enforcing CAP_LINUX_IMMUTABLE,
or any other permissions at all (?)
This does not change S_IMMUTABLE in-memory, so change will
only apply on the next time inode is loaded from disk.
Bottom line: seems like *any user at all* can change the READONLY
and SYSTEM attributes on ntfs.

OTOH, ntfs does support fileattr_set() - it allows changing
S_IMMUTABLE and S_APPEND in-memory, but as far as I can
tell, this change is not stored on-disk (?).

Also in ntfs, FILE_ATTRIBUTE_READONLY is mapped
to not having posix write permissions on-disk:
                /* Linux 'w' -> Windows 'ro'. */
                if (0222 & inode->i_mode)
                        ni->std_fa &= ~FILE_ATTRIBUTE_READONLY;
                else
                        ni->std_fa |= FILE_ATTRIBUTE_READONLY;

So for ntfs, S_IMMUTABLE could be updated depending on three
independent flags: SYSTEM, READONLY and  FS_XFLAG_IMMUTABLE.

Having ntfs treat FILE_ATTRIBUTE_READONLY as S_IMMUTABLE
internally, is completely confined to ntfs and has nothing to do with vfs
or with a new standard API.

> On-disk support for ext4 and other linux filesystems can be discussed
> later. I think that this could be more controversial.
>

Obviously there are existing users that need this.
Samba has its own xattr user.DOSATTRIB and if people really want
to be able to export those attributes in a standard way, I doubt there
will be objection to adding on-disk support (e.g. to ext4/xfs).
But somebody has to do the work and adding new on-disk support
is not so easy.

I can help with that when the time comes.
First thing first, try to propose patches to extend fsx_dosattrib and
support them in ntfs/fat/smb.

> > I can certainly not change the meaning of existing on-disk
> > flag of FS_IMMUTABLE_FL to a flag that can be removed
> > without CAP_LINUX_IMMUTABLE. that changes the meaning
> > of the flag.
> >
> > If ext4 maintainers agrees, you may be able to reuse some
> > old unused on-disk flags (e.g.  EXT4_UNRM_FL) as storage
> > place for FS_DOSATTRIB_READONLY, but that would be
> > quite hackish.
> >
> > > > How about tackling this one small step at a time, not in that order
> > > > necessarily:
> > > >
> > > > 1. Implement the standard API with FS_IOC_FS[GS]ETXATTR ioctl
> > > >     and with statx to get/set some non-controversial dosattrib flags on
> > > >     ntfs/smb/vfat
> > > > 2. Wire some interesting dosattrib flags (e.g. compr/enrypt) to local
> > > >     filesystems that already support storing those bits
> > > > 3. Wire network servers (e.g. Samba) to use the generic API if supported
> > > > 4. Add on-disk support for storing the dosattrib flags to more local fs
> > > > 5. Update S_IMMUTABLE inode flag if either FS_XFLAG_IMMUTABLE
> > > >     or FS_DOSATTRIB_READONLY are set on the file
> > > >
> > > > Thoughts?
> > > >
> >
> > Anything wrong with the plan above?
> > It seems that you are looking for shortcuts and I don't think that
> > it is a good way to make progress.
> >
> > Thanks,
> > Amir.
>
> If other developers agree that the FS_IOC_FS[GS]ETXATTR ioctl is the
> right direction then for me it looks good.

This thread has been going on for a while.
I did not see any objections to this idea that Darrick proposed,
so I think next step for you is to post patches, because some
developers will only engage when there are patches to discuss.

Thanks,
Amir.

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Tuesday 04 February 2025 12:54:01 Amir Goldstein wrote:
> On Tue, Feb 4, 2025 at 12:34 AM Pali Rohár <pali@kernel.org> wrote:
> >
> > On Tuesday 04 February 2025 00:02:44 Amir Goldstein wrote:
> > > On Mon, Feb 3, 2025 at 11:20 PM Pali Rohár <pali@kernel.org> wrote:
> > > >
> > > > On Monday 03 February 2025 22:59:46 Amir Goldstein wrote:
> > > > > On Sun, Feb 2, 2025 at 4:23 PM Pali Rohár <pali@kernel.org> wrote:
> > > > > > And there is still unresolved issue with FILE_ATTRIBUTE_READONLY.
> > > > > > Its meaning is similar to existing Linux FS_IMMUTABLE_FL, just
> > > > > > FILE_ATTRIBUTE_READONLY does not require root / CAP_LINUX_IMMUTABLE.
> > > > > >
> > > > > > I think that for proper support, to enforce FILE_ATTRIBUTE_READONLY
> > > > > > functionality, it is needed to introduce new flag e.g.
> > > > > > FS_IMMUTABLE_FL_USER to allow setting / clearing it also for normal
> > > > > > users without CAP_LINUX_IMMUTABLE. Otherwise it would be unsuitable for
> > > > > > any SMB client, SMB server or any application which would like to use
> > > > > > it, for example wine.
> > > > > >
> > > > > > Just to note that FreeBSD has two immutable flags SF_IMMUTABLE and
> > > > > > UF_IMMUTABLE, one settable only by superuser and second for owner.
> > > > > >
> > > > > > Any opinion?
> > > > >
> > > > > For filesystems that already support FILE_ATTRIBUTE_READONLY,
> > > > > can't you just set S_IMMUTABLE on the inode and vfs will do the correct
> > > > > enforcement?
> > > > >
> > > > > The vfs does not control if and how S_IMMUTABLE is set by filesystems,
> > > > > so if you want to remove this vfs flag without CAP_LINUX_IMMUTABLE
> > > > > in smb client, there is nothing stopping you (I think).
> > > >
> > > > Function fileattr_set_prepare() checks for CAP_LINUX_IMMUTABLE when
> > > > trying to change FS_IMMUTABLE_FL bit. This function is called from
> > > > ioctl(FS_IOC_SETFLAGS) and also from ioctl(FS_IOC_FSSETXATTR).
> > > > And when function fileattr_set_prepare() fails then .fileattr_set
> > > > callback is not called at all. So I think that it is not possible to
> > > > remove the IMMUTABLE flag from userspace without capability for smb
> > > > client.
> > > >
> > >
> > > You did not understand what I meant.
> > >
> > > You cannot relax the CAP_LINUX_IMMUTABLE for setting FS_IMMUTABLE_FL
> > > and there is no reason that you will need to relax it.
> > >
> > > The vfs does NOT enforce permissions according to FS_IMMUTABLE_FL
> > > The vfs enforces permissions according to the S_IMMUTABLE in-memory
> > > inode flag.
> > >
> > > There is no generic vfs code that sets S_IMMUTABLE inode flags, its
> > > the filesystems that translate the on-disk FS_IMMUTABLE_FL to
> > > in-memory S_IMMUTABLE inode flag.
> > >
> > > So if a filesystem already has an internal DOSATTRIB flags set, this
> > > filesystem can set the in-memory S_IMMUTABLE inode flag according
> > > to its knowledge of the DOSATTRIB_READONLY flag and the
> > > CAP_LINUX_IMMUTABLE rules do not apply to the DOSATTRIB_READONLY
> > > flag, which is NOT the same as the FS_IMMUTABLE_FL flag.
> > >
> > > > And it would not solve this problem for local filesystems (ntfs or ext4)
> > > > when Samba server or wine would want to set this bit.
> > > >
> > >
> > > The Samba server would use the FS_IOC_FS[GS]ETXATTR ioctl
> > > API to get/set dosattrib, something like this:
> > >
> > > struct fsxattr fsxattr;
> > > ret = ioctl_get_fsxattr(fd, &fsxattr);
> > > if (!ret && fsxattr.fsx_xflags & FS_XFLAG_HASDOSATTR) {
> > >     fsxattr.fsx_dosattr |= fs_dosattrib_readonly;
> > >     ret = ioctl_set_fsxattr(fd, &fsxattr);
> > > }
> >
> > Thanks for more explanation. First time I really did not understood it.
> > But now I think I understood it. So basically there would be two flags
> > which would result in setting S_IMMUTABLE on inode. One is the existing
> > FS_IMMUTABLE_FL which requires the capability and some new flag (e.g.
> > FS_XFLAG_HASDOSATTR) which would not require it and can be implemented
> > for cifs, vfat, ntfs, ... Right?
> >
> 
> Well, almost right.
> The flag that would correspond to FILE_ATTRIBUTE_READONLY
> is FS_DOSATTRIB_READONLY from the new field fsx_dosattrib
> (see below)

Thank you for example, it is for sure good starting point for me.

> --- a/include/uapi/linux/fs.h
> +++ b/include/uapi/linux/fs.h
> @@ -145,7 +145,8 @@ struct fsxattr {
>         __u32           fsx_nextents;   /* nextents field value (get)   */
>         __u32           fsx_projid;     /* project identifier (get/set) */
>         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> -       unsigned char   fsx_pad[8];
> +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> +       unsigned char   fsx_pad[4];
>  };
> 
>  /*
> @@ -167,7 +168,16 @@ struct fsxattr {
>  #define FS_XFLAG_FILESTREAM    0x00004000      /* use filestream allocator */
>  #define FS_XFLAG_DAX           0x00008000      /* use DAX for IO */
>  #define FS_XFLAG_COWEXTSIZE    0x00010000      /* CoW extent size
> allocator hint */
> -#define FS_XFLAG_HASATTR       0x80000000      /* no DIFLAG for this   */
> +#define FS_XFLAG_HASATTR       0x80000000      /* has extended attributes */
> +
> +/*
> + * Flags for the fsx_dosattrib field
> + */
> +#define FS_DOSATTRIB_READONLY  0x00000001      /* R - read-only */
> +#define FS_DOSATTRIB_HIDDEN    0x00000002      /* H - hidden */
> +#define FS_DOSATTRIB_SYSTEM    0x00000004      /* S - system */
> +#define FS_DOSATTRIB_ARCHIVE   0x00000020      /* A - archive */
> +#define FS_DOSATTRIB_HASATTR   0x80000000      /* has dos attributes */

Should these FS_DOSATTRIB_* constants follows the Windows
FILE_ATTRIBUTE_* constants? Because I see that you put a gap between
system and archive.

> This last special flag is debatable and I am not really sure that we need it.

This constant has very similar meaning to FILE_ATTRIBUTE_NORMAL. Both
has some compatibility meaning that "field is valid or something is set".
Just FILE_ATTRIBUTE_NORMAL is not 31th bit.

> It is needed for proper backward compat with existing userspace tools.
> For example, if there was a backup tool storing the fsxattr blob result of
> FS_IOC_FSGETXATTR and sets it later during restore with
> FS_IOC_FSSETXATTR, then it would be better to ignore a zero
> value of fsx_dosattrib instead of resetting all of the on-disk dosattrib flags
> if the restore happens after ntfs gained support for setting dosattrib flags
> via FS_IOC_FSSETXATTR.
> 
> When using the standard tools to set fsxattr (chattr and xfs_io -c chattr)
> the tool does FS_IOC_FSGETXATTR + modify + FS_IOC_FSSETXATTR,
> so those tools are expected to leave new bits in fsx_dosattrib at their
> value if ntfs gains support for get/set fsx_dosattrib.
> 
> Setting the auxiliary FS_DOSATTRIB_HASATTR flag can help the
> kernel/fs to explicitly state that the values returned in fsx_dosattrib
> are valid and the tool to state that values set in fsx_dosattrib are valid.
> But using a single flag will not help expanding ntfs support for more
> fsx_dosattrib flags later, so I am not sure if it is useful (?).

If the fsx_dosattrib would match all FILE_ATTRIBUTE_* then we can do it
as the ntfs matches FILE_ATTRIBUTE_* and no extension is needed for
future.

And I think that this backward compatibility sounds good.

What could be useful for userspace is also ability to figure out which
FS_DOSATTRIB_* are supported by the filesystem. Because for example UDF
on-disk format supports only FS_DOSATTRIB_HIDDEN bit. And FAT only those
attributes which are in the lowest byte.

> 
> > > For ntfs/ext4, you will need to implement on-disk support for
> > > set/get the dosattrib flags.
> >
> > ntfs has already on-disk support for FILE_ATTRIBUTE_READONLY.
> >
> 
> This is interesting.

I mean that ntfs filesystem has support for FILE_ATTRIBUTE_READONLY.
I did not mean linux ntfs implementation.

But I'm aware of some of those details in linux fs implementations, but
I did not wanted to mentioned it as basically every linux fs
implementation has its own way how flags are handled or exported to
userspace. It is good to know, but not important when designing or
discussing the unified/generic standard API.

> fat/ntfs both already have a mount option sys_immutable to map
> FILE_ATTRIBUTE_SYSTEM to S_IMMUTABLE in-memory.
> 
> fat does not support fileattr_set(), but has a proprietary ioctl
> FAT_IOCTL_SET_ATTRIBUTES which enforces
> CAP_LINUX_IMMUTABLE for changing S_IMMUTABLE.
> 
> ntfs also maps FILE_ATTRIBUTE_SYSTEM to S_IMMUTABLE
> and it allows changing FILE_ATTRIBUTE_SYSTEM via ntfs_setxattr
> of system.{dos,ntfs}_attrib without enforcing CAP_LINUX_IMMUTABLE,
> or any other permissions at all (?)
> This does not change S_IMMUTABLE in-memory, so change will
> only apply on the next time inode is loaded from disk.
> Bottom line: seems like *any user at all* can change the READONLY
> and SYSTEM attributes on ntfs.
> 
> OTOH, ntfs does support fileattr_set() - it allows changing
> S_IMMUTABLE and S_APPEND in-memory, but as far as I can
> tell, this change is not stored on-disk (?).
> 
> Also in ntfs, FILE_ATTRIBUTE_READONLY is mapped
> to not having posix write permissions on-disk:
>                 /* Linux 'w' -> Windows 'ro'. */
>                 if (0222 & inode->i_mode)
>                         ni->std_fa &= ~FILE_ATTRIBUTE_READONLY;
>                 else
>                         ni->std_fa |= FILE_ATTRIBUTE_READONLY;
> 
> So for ntfs, S_IMMUTABLE could be updated depending on three
> independent flags: SYSTEM, READONLY and  FS_XFLAG_IMMUTABLE.
> 
> Having ntfs treat FILE_ATTRIBUTE_READONLY as S_IMMUTABLE
> internally, is completely confined to ntfs and has nothing to do with vfs
> or with a new standard API.
> 
> > On-disk support for ext4 and other linux filesystems can be discussed
> > later. I think that this could be more controversial.
> >
> 
> Obviously there are existing users that need this.
> Samba has its own xattr user.DOSATTRIB and if people really want
> to be able to export those attributes in a standard way, I doubt there
> will be objection to adding on-disk support (e.g. to ext4/xfs).
> But somebody has to do the work and adding new on-disk support
> is not so easy.

Yes, it is not easy and on-disk support can be done later or basically
independently of this work here. So I will let it for other people.

> I can help with that when the time comes.
> First thing first, try to propose patches to extend fsx_dosattrib and
> support them in ntfs/fat/smb.

Ok. Thanks.

> > > I can certainly not change the meaning of existing on-disk
> > > flag of FS_IMMUTABLE_FL to a flag that can be removed
> > > without CAP_LINUX_IMMUTABLE. that changes the meaning
> > > of the flag.
> > >
> > > If ext4 maintainers agrees, you may be able to reuse some
> > > old unused on-disk flags (e.g.  EXT4_UNRM_FL) as storage
> > > place for FS_DOSATTRIB_READONLY, but that would be
> > > quite hackish.
> > >
> > > > > How about tackling this one small step at a time, not in that order
> > > > > necessarily:
> > > > >
> > > > > 1. Implement the standard API with FS_IOC_FS[GS]ETXATTR ioctl
> > > > >     and with statx to get/set some non-controversial dosattrib flags on
> > > > >     ntfs/smb/vfat
> > > > > 2. Wire some interesting dosattrib flags (e.g. compr/enrypt) to local
> > > > >     filesystems that already support storing those bits
> > > > > 3. Wire network servers (e.g. Samba) to use the generic API if supported
> > > > > 4. Add on-disk support for storing the dosattrib flags to more local fs
> > > > > 5. Update S_IMMUTABLE inode flag if either FS_XFLAG_IMMUTABLE
> > > > >     or FS_DOSATTRIB_READONLY are set on the file
> > > > >
> > > > > Thoughts?
> > > > >
> > >
> > > Anything wrong with the plan above?
> > > It seems that you are looking for shortcuts and I don't think that
> > > it is a good way to make progress.
> > >
> > > Thanks,
> > > Amir.
> >
> > If other developers agree that the FS_IOC_FS[GS]ETXATTR ioctl is the
> > right direction then for me it looks good.
> 
> This thread has been going on for a while.
> I did not see any objections to this idea that Darrick proposed,
> so I think next step for you is to post patches, because some
> developers will only engage when there are patches to discuss.
> 
> Thanks,
> Amir.

Ok, I will try to prepare something. Just give me some weeks as would
not have time for this right now. I just wanted to be sure that this
is really the right direction and also that this is something which
makes sense. I did not wanted to start doing something which could be
completely useless... That is why I started rather longer discussion
first.

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Monday 03 February 2025 22:59:46 Amir Goldstein wrote:
> On Sun, Feb 2, 2025 at 4:23 PM Pali Rohár <pali@kernel.org> wrote:
> >
> > On Friday 17 January 2025 19:59:47 Pali Rohár wrote:
> > > On Friday 17 January 2025 19:46:30 Amir Goldstein wrote:
> > > > > > Looking at the FILE_ATTRIBUTE_* flags defined in SMB protocol
> > > > > >  (fs/smb/common/smb2pdu.h) I wonder how many of them will be
> > > > > > needed for applications beyond the obvious ones that were listed.
> > > > >
> > > > > Well they only asked for seven of them. ;)
> > > > >
> > > > > I chatted with Ted about this yesterday, and ... some of the attributes
> > > > > (like read only) imply that you'd want the linux server to enforce no
> > > > > writing to the file; some like archive seem a little superfluous since
> > > > > on linux you can compare cmtime from the backup against what's in the
> > > > > file now; and still others (like hidden/system) might just be some dorky
> > > > > thing that could be hidden in some xattr because a unix filesystem won't
> > > > > care.
> > > > >
> > > > > And then there are other attrs like "integrity stream" where someone
> > > > > with more experience with windows would have to tell me if fsverity
> > > > > provides sufficient behaviors or not.
> > > > >
> > > > > But maybe we should start by plumbing one of those bits in?  I guess the
> > > > > gross part is that implies an ondisk inode format change or (gross)
> > > > > xattr lookups in the open path.
> > > > >
> > > >
> > > > I may be wrong, but I think there is a confusion in this thread.
> > > > I don't think that Pali was looking for filesystems to implement
> > > > storing those attributes. I read his email as a request to standardize
> > > > a user API to get/set those attributes for the filesystems that
> > > > already support them and possibly for vfs to enforce some of them
> > > > (e.g. READONLY) in generic code.
> > >
> > > Yes, you understood it correctly. I was asking for standardizing API how
> > > to get/set these attributes from userspace. And Chuck wrote that would
> > > like to have also standardized it for kernel consumers like nfsd or
> > > ksmbd. Which makes sense.
> > >
> > > > Nevertheless, I understand the confusion because I know there
> > > > is also demand for storing those attributes by file servers in a
> > > > standard way and for vfs to respect those attributes on the host.
> > >
> > > Userspace fileserver, like Samba, would just use that standardized
> > > userspace API for get/set attributes. And in-kernel fileservers like
> > > nfsd or ksmbd would like to use that API too.
> > >
> > > And there were some proposals how filesystems without native
> > > support for these attributes could store and preserve these attributes.
> > > So this can be a confusion in this email thread discussion.
> >
> > So to recap, for set-API there are possible options:
> >
> > 1) extending FS_IOC_FSSETXATTR / FS_IOC_SETFLAGS for each individual bit
> >
> > 2) creating one new xattr in system namespace which will contain all bit
> >    flags in one structure
> >
> > 3) creating new xattr in system namespace for each individual flag
> >
> > Disadvantages for option 1) is that "cp -a" or "rsync -a" does not
> > preserve them. If in option 2) or 3) those xattrs would be visible in
> > listxattr() call then this can be advantage, as all flags are properly
> > preserved when doing "archive" backup.
> >
> > Do you have any preference which of those options should be used?
> >
> 
> Darrick said in this thread twice:
> statx/FS_IOC_FSGETXATTR to retrieve and FS_IOC_FSSETXATTR to set.
> (NOT FS_IOC_SETFLAGS)
> and I wrote that I agree with him.
> 
> I suggest that you let go of the cp -a out-of-the-box requirement.
> It is not going to pass muster - maybe for a specific filesystem but
> as a filesystem agnostic feature, unless you change cp tool.

Ok. I was not sure that this is the decided way. So we are going to
take this direction. I hope that cp and rsync tools would accept
changes/patches in future for ability to copy these flags too.

> >
> > And how many bit flags are needed? I have done some investigation. Lets
> > start with table which describes all 32 possible bit flags which are
> > used by Windows system and also by filesystems FAT / exFAT / NTFS / ReFS
> > and also by SMB over network:
> >
> > bit / attrib.exe flag / SDK constant / description
> >
> >  0 - R - FILE_ATTRIBUTE_READONLY              - writing to file or deleting it is disallowed
> >  1 - H - FILE_ATTRIBUTE_HIDDEN                - inode is hidden
> >  2 - S - FILE_ATTRIBUTE_SYSTEM                - inode is part of operating system
> >  3 -   - FILE_ATTRIBUTE_VOLUME                - inode is the disk volume label entry
> >  4 -   - FILE_ATTRIBUTE_DIRECTORY             - inode is directory
> >  5 - A - FILE_ATTRIBUTE_ARCHIVE               - inode was not archived yet (when set)
> >  6 -   - FILE_ATTRIBUTE_DEVICE                - inode represents  in-memory device (e.g. C:\), flag not stored on filesystem
> >  7 -   - FILE_ATTRIBUTE_NORMAL                - no other flag is set (value 0 means to not change flags, bit 7 means to clear all flags)
> >  8 -   - FILE_ATTRIBUTE_TEMPORARY             - inode data do not have to be flushed to disk
> >  9 -   - FILE_ATTRIBUTE_SPARSE_FILE           - file is sparse with holes
> > 10 -   - FILE_ATTRIBUTE_REPARSE_POINT         - inode has attached reparse point (symlink is also reparse point)
> > 11 -   - FILE_ATTRIBUTE_COMPRESSED            - file is compressed, for directories it means that newly created inodes would have this flag set
> > 12 - O - FILE_ATTRIBUTE_OFFLINE               - HSM - inode is used by HSM
> > 13 - I - FILE_ATTRIBUTE_NOT_CONTENT_INDEXED   - inode will not be indexed by content indexing service
> > 14 -   - FILE_ATTRIBUTE_ENCRYPTED             - file is encrypted, for directories it means that newly created inodes would have this flag set
> > 15 - V - FILE_ATTRIBUTE_INTEGRITY_STREAM      - fs does checksumming of data and metadata when reading inode, read-only
> > 16 -   - FILE_ATTRIBUTE_VIRTUAL               - inode is in %LocalAppData%\VirtualStore, flag not stored on filesystem
> > 17 - X - FILE_ATTRIBUTE_NO_SCRUB_DATA         - do not use scrubber (proactive background data integrity scanner) on this file, for directories it means that newly created inodes would have this flag set
> > 18 -   - FILE_ATTRIBUTE_EA                    - inode has xattrs, (not in readdir output, shares same bit with FILE_ATTRIBUTE_RECALL_ON_OPEN)
> > 18 -   - FILE_ATTRIBUTE_RECALL_ON_OPEN        - HSM - inode is not stored locally (only in readdir output, shares same bit with FILE_ATTRIBUTE_EA)
> > 19 - P - FILE_ATTRIBUTE_PINNED                - HSM - inode data content must be always stored on locally
> > 20 - U - FILE_ATTRIBUTE_UNPINNED              - HSM - inode data content can be removed from local storage
> > 21 -   -                                      - reserved
> > 22 -   - FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS - HSM - inode data content is not stored locally
> > 23 -   -                                      - reserved
> > 24 -   -                                      - reserved
> > 25 -   -                                      - reserved
> > 26 -   -                                      - reserved
> > 27 -   -                                      - reserved
> > 28 -   -                                      - reserved
> > 29 - B - FILE_ATTRIBUTE_STRICTLY_SEQUENTIAL   - SMR Blob, unknown meaning, read-only
> > 30 -   -                                      - reserved
> > 31 -   -                                      - reserved
> >
> 
> I suspect that we need to reserve expansion for more than 7 bits if we
> design a proper API.
> The fsx_xflags field is already too crowded for adding so many flags
> We can use the padding at the end of fsxattr to add __u32 fsx_dosattrib
> or fsx_dosflags field.
> 
> > (HSM means Hierarchical Storage Management software, which uses reparse
> > points to make some remote file/folder available on the local
> > filesystem, for example OneDrive or DropBox)
> >
> 
> I am quite interested in supporting those HSM flags for fanotify.

Ok. I heard that more people are interesting in these flags, so seems
that this is some area which is missing on Linux.

> > From above list only following bit flags are suitable for modification
> > over some Linux API:
> > - FILE_ATTRIBUTE_READONLY
> > - FILE_ATTRIBUTE_HIDDEN
> > - FILE_ATTRIBUTE_SYSTEM
> > - FILE_ATTRIBUTE_ARCHIVE
> > - FILE_ATTRIBUTE_TEMPORARY
> > - FILE_ATTRIBUTE_COMPRESSED
> > - FILE_ATTRIBUTE_OFFLINE
> > - FILE_ATTRIBUTE_NOT_CONTENT_INDEXED
> > - FILE_ATTRIBUTE_ENCRYPTED
> > - FILE_ATTRIBUTE_NO_SCRUB_DATA
> > - FILE_ATTRIBUTE_PINNED
> > - FILE_ATTRIBUTE_UNPINNED
> >
> > And if I'm looking correctly the FILE_ATTRIBUTE_COMPRESSED can be
> > already mapped to Linux FS_COMPR_FL / STATX_ATTR_COMPRESSED, which has
> > same meaning. Also FILE_ATTRIBUTE_ENCRYPTED can be mapped to
> > FS_ENCRYPT_FL / STATX_ATTR_ENCRYPTED. Note that these two flags cannot
> > be set over WinAPI or SMB directly and it is required to use special
> > WinAPI or SMB ioctl.
> >
> 
> There is a standard way to map from fileattr::flags
> to fileattr::fsx_xflags, so that you could set/get COMPR,ENCRYPT using
> FS_IOC_FS[GS]ETXATTR ioctl.
> see fileattr_fill_flags/fileattr_fill_xflags.
> but I think that xfs_fileattr_set() will need to have a supported xflags mask
> check if you start adding xflags that xfs does not currently support and
> other filesystems do support.

Ok. I will try to play with it.

> > So totally are needed 10 new bit flags. And for future there are 9
> > reserved bits which could be introduced by MS in future.
> >
> > Additionally there are get-only attributes which can be useful for statx
> > purposes (for example exported by cifs.ko SMB client):
> > - FILE_ATTRIBUTE_REPARSE_POINT
> > - FILE_ATTRIBUTE_INTEGRITY_STREAM
> > - FILE_ATTRIBUTE_RECALL_ON_OPEN
> > - FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS
> > - FILE_ATTRIBUTE_STRICTLY_SEQUENTIAL
> >
> > From the above list of flags suitable for modification, following bit
> > flags have no meaning for kernel and it is up to userspace how will use
> > them. What is needed from kernel and/or filesystem driver is to preserve
> > those bit flags.
> > - FILE_ATTRIBUTE_HIDDEN
> > - FILE_ATTRIBUTE_SYSTEM
> > - FILE_ATTRIBUTE_ARCHIVE
> > - FILE_ATTRIBUTE_NOT_CONTENT_INDEXED
> >
> > Following are bit flags which kernel / VFS / fsdriver would have to
> > handle specially, to provide enforcement or correct behavior of them:
> > - FILE_ATTRIBUTE_READONLY - enforce that data modification or unlink is disallowed when set
> > - FILE_ATTRIBUTE_COMPRESSED - enforce compression on filesystem when set
> > - FILE_ATTRIBUTE_ENCRYPTED - enforce encryption on filesystem when set
> >
> > Then there are HSM flags which for local filesystem would need some
> > cooperation with userspace synchronization software. For network
> > filesystems (SMB / NFS4) they need nothing special, just properly
> > propagating them over network:
> > - FILE_ATTRIBUTE_OFFLINE
> > - FILE_ATTRIBUTE_PINNED
> > - FILE_ATTRIBUTE_UNPINNED
> >
> > About following 2 flags, I'm not sure if the kernel / VFS / fs driver
> > has to do something or it can just store bits to fs:
> > - FILE_ATTRIBUTE_TEMPORARY
> > - FILE_ATTRIBUTE_NO_SCRUB_DATA

Re: Immutable vs read-only for Windows compatibility

[Amir Goldstein]

On Tue, Feb 4, 2025 at 10:26 PM Pali Rohár <pali@kernel.org> wrote:
>
> On Tuesday 04 February 2025 12:54:01 Amir Goldstein wrote:
> > On Tue, Feb 4, 2025 at 12:34 AM Pali Rohár <pali@kernel.org> wrote:
> > >
> > > On Tuesday 04 February 2025 00:02:44 Amir Goldstein wrote:
> > > > On Mon, Feb 3, 2025 at 11:20 PM Pali Rohár <pali@kernel.org> wrote:
> > > > >
> > > > > On Monday 03 February 2025 22:59:46 Amir Goldstein wrote:
> > > > > > On Sun, Feb 2, 2025 at 4:23 PM Pali Rohár <pali@kernel.org> wrote:
> > > > > > > And there is still unresolved issue with FILE_ATTRIBUTE_READONLY.
> > > > > > > Its meaning is similar to existing Linux FS_IMMUTABLE_FL, just
> > > > > > > FILE_ATTRIBUTE_READONLY does not require root / CAP_LINUX_IMMUTABLE.
> > > > > > >
> > > > > > > I think that for proper support, to enforce FILE_ATTRIBUTE_READONLY
> > > > > > > functionality, it is needed to introduce new flag e.g.
> > > > > > > FS_IMMUTABLE_FL_USER to allow setting / clearing it also for normal
> > > > > > > users without CAP_LINUX_IMMUTABLE. Otherwise it would be unsuitable for
> > > > > > > any SMB client, SMB server or any application which would like to use
> > > > > > > it, for example wine.
> > > > > > >
> > > > > > > Just to note that FreeBSD has two immutable flags SF_IMMUTABLE and
> > > > > > > UF_IMMUTABLE, one settable only by superuser and second for owner.
> > > > > > >
> > > > > > > Any opinion?
> > > > > >
> > > > > > For filesystems that already support FILE_ATTRIBUTE_READONLY,
> > > > > > can't you just set S_IMMUTABLE on the inode and vfs will do the correct
> > > > > > enforcement?
> > > > > >
> > > > > > The vfs does not control if and how S_IMMUTABLE is set by filesystems,
> > > > > > so if you want to remove this vfs flag without CAP_LINUX_IMMUTABLE
> > > > > > in smb client, there is nothing stopping you (I think).
> > > > >
> > > > > Function fileattr_set_prepare() checks for CAP_LINUX_IMMUTABLE when
> > > > > trying to change FS_IMMUTABLE_FL bit. This function is called from
> > > > > ioctl(FS_IOC_SETFLAGS) and also from ioctl(FS_IOC_FSSETXATTR).
> > > > > And when function fileattr_set_prepare() fails then .fileattr_set
> > > > > callback is not called at all. So I think that it is not possible to
> > > > > remove the IMMUTABLE flag from userspace without capability for smb
> > > > > client.
> > > > >
> > > >
> > > > You did not understand what I meant.
> > > >
> > > > You cannot relax the CAP_LINUX_IMMUTABLE for setting FS_IMMUTABLE_FL
> > > > and there is no reason that you will need to relax it.
> > > >
> > > > The vfs does NOT enforce permissions according to FS_IMMUTABLE_FL
> > > > The vfs enforces permissions according to the S_IMMUTABLE in-memory
> > > > inode flag.
> > > >
> > > > There is no generic vfs code that sets S_IMMUTABLE inode flags, its
> > > > the filesystems that translate the on-disk FS_IMMUTABLE_FL to
> > > > in-memory S_IMMUTABLE inode flag.
> > > >
> > > > So if a filesystem already has an internal DOSATTRIB flags set, this
> > > > filesystem can set the in-memory S_IMMUTABLE inode flag according
> > > > to its knowledge of the DOSATTRIB_READONLY flag and the
> > > > CAP_LINUX_IMMUTABLE rules do not apply to the DOSATTRIB_READONLY
> > > > flag, which is NOT the same as the FS_IMMUTABLE_FL flag.
> > > >
> > > > > And it would not solve this problem for local filesystems (ntfs or ext4)
> > > > > when Samba server or wine would want to set this bit.
> > > > >
> > > >
> > > > The Samba server would use the FS_IOC_FS[GS]ETXATTR ioctl
> > > > API to get/set dosattrib, something like this:
> > > >
> > > > struct fsxattr fsxattr;
> > > > ret = ioctl_get_fsxattr(fd, &fsxattr);
> > > > if (!ret && fsxattr.fsx_xflags & FS_XFLAG_HASDOSATTR) {
> > > >     fsxattr.fsx_dosattr |= fs_dosattrib_readonly;
> > > >     ret = ioctl_set_fsxattr(fd, &fsxattr);
> > > > }
> > >
> > > Thanks for more explanation. First time I really did not understood it.
> > > But now I think I understood it. So basically there would be two flags
> > > which would result in setting S_IMMUTABLE on inode. One is the existing
> > > FS_IMMUTABLE_FL which requires the capability and some new flag (e.g.
> > > FS_XFLAG_HASDOSATTR) which would not require it and can be implemented
> > > for cifs, vfat, ntfs, ... Right?
> > >
> >
> > Well, almost right.
> > The flag that would correspond to FILE_ATTRIBUTE_READONLY
> > is FS_DOSATTRIB_READONLY from the new field fsx_dosattrib
> > (see below)
>
> Thank you for example, it is for sure good starting point for me.
>
> > --- a/include/uapi/linux/fs.h
> > +++ b/include/uapi/linux/fs.h
> > @@ -145,7 +145,8 @@ struct fsxattr {
> >         __u32           fsx_nextents;   /* nextents field value (get)   */
> >         __u32           fsx_projid;     /* project identifier (get/set) */
> >         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> > -       unsigned char   fsx_pad[8];
> > +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> > +       unsigned char   fsx_pad[4];
> >  };
> >
> >  /*
> > @@ -167,7 +168,16 @@ struct fsxattr {
> >  #define FS_XFLAG_FILESTREAM    0x00004000      /* use filestream allocator */
> >  #define FS_XFLAG_DAX           0x00008000      /* use DAX for IO */
> >  #define FS_XFLAG_COWEXTSIZE    0x00010000      /* CoW extent size
> > allocator hint */
> > -#define FS_XFLAG_HASATTR       0x80000000      /* no DIFLAG for this   */
> > +#define FS_XFLAG_HASATTR       0x80000000      /* has extended attributes */
> > +
> > +/*
> > + * Flags for the fsx_dosattrib field
> > + */
> > +#define FS_DOSATTRIB_READONLY  0x00000001      /* R - read-only */
> > +#define FS_DOSATTRIB_HIDDEN    0x00000002      /* H - hidden */
> > +#define FS_DOSATTRIB_SYSTEM    0x00000004      /* S - system */
> > +#define FS_DOSATTRIB_ARCHIVE   0x00000020      /* A - archive */
> > +#define FS_DOSATTRIB_HASATTR   0x80000000      /* has dos attributes */
>
> Should these FS_DOSATTRIB_* constants follows the Windows
> FILE_ATTRIBUTE_* constants? Because I see that you put a gap between
> system and archive.
>

Well, no, they do not need to follow Windows contestants,
but then again, why not?
I mean if we only ever needed the 4 RHSA bits above, we could
have used the FS_XFLAG_* flags space, but if we extend the API
with a new 32bit field, why not use 1-to-1 mapping at least as a starting point.

You can see that it is quite common that filesystems re-define the
same constants for these flags (e.g. EXT4_IMMUTABLE_FL).
I am a bit surprised that there is no build time assertion
BUILD_BUG_ON(EXT4_IMMUTABLE_FL != FS_IMMUTABLE_FL)
which would be the standard way to make sure that the constants
stay in sync if they need to be in sync, but some filesystems don't
even assume that these constants are in sync (e.g. f2fs_fsflags_map)

> > This last special flag is debatable and I am not really sure that we need it.
>
> This constant has very similar meaning to FILE_ATTRIBUTE_NORMAL. Both
> has some compatibility meaning that "field is valid or something is set".
> Just FILE_ATTRIBUTE_NORMAL is not 31th bit.
>

No it does not. I don't think that you understood the meaning of
FS_DOSATTRIB_HASATTR.
Nevermind it was a bad idea anyway. see more below.

> > It is needed for proper backward compat with existing userspace tools.
> > For example, if there was a backup tool storing the fsxattr blob result of
> > FS_IOC_FSGETXATTR and sets it later during restore with
> > FS_IOC_FSSETXATTR, then it would be better to ignore a zero
> > value of fsx_dosattrib instead of resetting all of the on-disk dosattrib flags
> > if the restore happens after ntfs gained support for setting dosattrib flags
> > via FS_IOC_FSSETXATTR.
> >
> > When using the standard tools to set fsxattr (chattr and xfs_io -c chattr)
> > the tool does FS_IOC_FSGETXATTR + modify + FS_IOC_FSSETXATTR,
> > so those tools are expected to leave new bits in fsx_dosattrib at their
> > value if ntfs gains support for get/set fsx_dosattrib.
> >
> > Setting the auxiliary FS_DOSATTRIB_HASATTR flag can help the
> > kernel/fs to explicitly state that the values returned in fsx_dosattrib
> > are valid and the tool to state that values set in fsx_dosattrib are valid.
> > But using a single flag will not help expanding ntfs support for more
> > fsx_dosattrib flags later, so I am not sure if it is useful (?).
>
> If the fsx_dosattrib would match all FILE_ATTRIBUTE_* then we can do it
> as the ntfs matches FILE_ATTRIBUTE_* and no extension is needed for
> future.
>
> And I think that this backward compatibility sounds good.
>

That's only true if you can support ALL the dosattrib flags from the
first version and that Windows will not add any of the reserved
flags in the future, which is hard to commit to.

> What could be useful for userspace is also ability to figure out which
> FS_DOSATTRIB_* are supported by the filesystem. Because for example UDF
> on-disk format supports only FS_DOSATTRIB_HIDDEN bit. And FAT only those
> attributes which are in the lowest byte.
>

Exactly.
statx has this solved with the stx_attributes_mask field.

We could do the same for FS_IOC_FS[GS]ETXATTR, but because
right now, this API does not verify that fsx_pad is zero, we will need to
define a new set of ioctl consants FS_IOC_[GS]ETFSXATTR2
with the exact same functionality but that userspace knows that they
publish and respect the dosattrib mask:

--- a/fs/ioctl.c
+++ b/fs/ioctl.c
@@ -868,9 +868,11 @@ static int do_vfs_ioctl(struct file *filp, unsigned int fd,
        case FS_IOC_SETFLAGS:
                return ioctl_setflags(filp, argp);

+       case FS_IOC_GETFSXATTR2:
        case FS_IOC_FSGETXATTR:
                return ioctl_fsgetxattr(filp, argp);

+       case FS_IOC_SETFSXATTR2:
        case FS_IOC_FSSETXATTR:
                return ioctl_fssetxattr(filp, argp);
--- a/include/uapi/linux/fs.h
+++ b/include/uapi/linux/fs.h
@@ -145,7 +145,8 @@ struct fsxattr {
        __u32           fsx_nextents;   /* nextents field value (get)   */
        __u32           fsx_projid;     /* project identifier (get/set) */
        __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
-       unsigned char   fsx_pad[8];
+       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
+       __u32           fsx_dosattrib_mask; /* dosattrib field validity mask */
 };

 /*
@@ -238,6 +248,9 @@ struct fsxattr {
 #define FS_IOC32_SETFLAGS              _IOW('f', 2, int)
 #define FS_IOC32_GETVERSION            _IOR('v', 1, int)
 #define FS_IOC32_SETVERSION            _IOW('v', 2, int)
+#define FS_IOC_GETFSXATTR2              _IOR('x', 31, struct fsxattr)
+#define FS_IOC_SETFSXATTR2              _IOW('x', 32, struct fsxattr)
+/* Duplicate legacy ioctl numbers for backward compact */
 #define FS_IOC_FSGETXATTR              _IOR('X', 31, struct fsxattr)
 #define FS_IOC_FSSETXATTR              _IOW('X', 32, struct fsxattr)
 #define FS_IOC_GETFSLABEL              _IOR(0x94, 49, char[FSLABEL_MAX])

We could also use this opportunity to define a larger fsxattr2 struct
that also includes an fsx_xflags_mask field, so that the xflags namespace
could also be extended in a backward compat way going forward:

@@ -145,7 +145,21 @@ struct fsxattr {
        __u32           fsx_nextents;   /* nextents field value (get)   */
        __u32           fsx_projid;     /* project identifier (get/set) */
        __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
        unsigned char   fsx_pad[8];

 };
+
+/*
+ * Structure for FS_IOC_[GS]ETFSXATTR2.
+ */
+struct fsxattr2 {
+       __u32           fsx_xflags;     /* xflags field value (get/set) */
+       __u32           fsx_extsize;    /* extsize field value (get/set)*/
+       __u32           fsx_nextents;   /* nextents field value (get)   */
+       __u32           fsx_projid;     /* project identifier (get/set) */
+       __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
+       __u32           fsx_xflags_mask; /* xflags field validity mask */
+       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
+       __u32           fsx_dosattrib_mask; /* dosattrib field validity mask */
+};

And you'd also need to flug those new mask and dosattrib
via struct fileattr into filesystems - too much to explain.
try to figure it out (unless someone objects) and if you can't figure
it out let me know.

> >
> > > > For ntfs/ext4, you will need to implement on-disk support for
> > > > set/get the dosattrib flags.
> > >
> > > ntfs has already on-disk support for FILE_ATTRIBUTE_READONLY.
> > >
> >
> > This is interesting.
>
> I mean that ntfs filesystem has support for FILE_ATTRIBUTE_READONLY.
> I did not mean linux ntfs implementation.
>
> But I'm aware of some of those details in linux fs implementations, but
> I did not wanted to mentioned it as basically every linux fs
> implementation has its own way how flags are handled or exported to
> userspace. It is good to know, but not important when designing or
> discussing the unified/generic standard API.
>

A good API will describe what the existing filesystems already support
and what they do not support.

> > fat/ntfs both already have a mount option sys_immutable to map
> > FILE_ATTRIBUTE_SYSTEM to S_IMMUTABLE in-memory.
> >
> > fat does not support fileattr_set(), but has a proprietary ioctl
> > FAT_IOCTL_SET_ATTRIBUTES which enforces
> > CAP_LINUX_IMMUTABLE for changing S_IMMUTABLE.
> >
> > ntfs also maps FILE_ATTRIBUTE_SYSTEM to S_IMMUTABLE
> > and it allows changing FILE_ATTRIBUTE_SYSTEM via ntfs_setxattr
> > of system.{dos,ntfs}_attrib without enforcing CAP_LINUX_IMMUTABLE,
> > or any other permissions at all (?)
> > This does not change S_IMMUTABLE in-memory, so change will
> > only apply on the next time inode is loaded from disk.
> > Bottom line: seems like *any user at all* can change the READONLY
> > and SYSTEM attributes on ntfs.
> >
> > OTOH, ntfs does support fileattr_set() - it allows changing
> > S_IMMUTABLE and S_APPEND in-memory, but as far as I can
> > tell, this change is not stored on-disk (?).
> >
> > Also in ntfs, FILE_ATTRIBUTE_READONLY is mapped
> > to not having posix write permissions on-disk:
> >                 /* Linux 'w' -> Windows 'ro'. */
> >                 if (0222 & inode->i_mode)
> >                         ni->std_fa &= ~FILE_ATTRIBUTE_READONLY;
> >                 else
> >                         ni->std_fa |= FILE_ATTRIBUTE_READONLY;
> >
> > So for ntfs, S_IMMUTABLE could be updated depending on three
> > independent flags: SYSTEM, READONLY and  FS_XFLAG_IMMUTABLE.
> >
> > Having ntfs treat FILE_ATTRIBUTE_READONLY as S_IMMUTABLE
> > internally, is completely confined to ntfs and has nothing to do with vfs
> > or with a new standard API.
> >
> > > On-disk support for ext4 and other linux filesystems can be discussed
> > > later. I think that this could be more controversial.
> > >
> >
> > Obviously there are existing users that need this.
> > Samba has its own xattr user.DOSATTRIB and if people really want
> > to be able to export those attributes in a standard way, I doubt there
> > will be objection to adding on-disk support (e.g. to ext4/xfs).
> > But somebody has to do the work and adding new on-disk support
> > is not so easy.
>
> Yes, it is not easy and on-disk support can be done later or basically
> independently of this work here. So I will let it for other people.
>
> > I can help with that when the time comes.
> > First thing first, try to propose patches to extend fsx_dosattrib and
> > support them in ntfs/fat/smb.
>
> Ok. Thanks.
>
> > > > I can certainly not change the meaning of existing on-disk
> > > > flag of FS_IMMUTABLE_FL to a flag that can be removed
> > > > without CAP_LINUX_IMMUTABLE. that changes the meaning
> > > > of the flag.
> > > >
> > > > If ext4 maintainers agrees, you may be able to reuse some
> > > > old unused on-disk flags (e.g.  EXT4_UNRM_FL) as storage
> > > > place for FS_DOSATTRIB_READONLY, but that would be
> > > > quite hackish.
> > > >
> > > > > > How about tackling this one small step at a time, not in that order
> > > > > > necessarily:
> > > > > >
> > > > > > 1. Implement the standard API with FS_IOC_FS[GS]ETXATTR ioctl
> > > > > >     and with statx to get/set some non-controversial dosattrib flags on
> > > > > >     ntfs/smb/vfat
> > > > > > 2. Wire some interesting dosattrib flags (e.g. compr/enrypt) to local
> > > > > >     filesystems that already support storing those bits
> > > > > > 3. Wire network servers (e.g. Samba) to use the generic API if supported
> > > > > > 4. Add on-disk support for storing the dosattrib flags to more local fs
> > > > > > 5. Update S_IMMUTABLE inode flag if either FS_XFLAG_IMMUTABLE
> > > > > >     or FS_DOSATTRIB_READONLY are set on the file
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > >
> > > > Anything wrong with the plan above?
> > > > It seems that you are looking for shortcuts and I don't think that
> > > > it is a good way to make progress.
> > > >
> > > > Thanks,
> > > > Amir.
> > >
> > > If other developers agree that the FS_IOC_FS[GS]ETXATTR ioctl is the
> > > right direction then for me it looks good.
> >
> > This thread has been going on for a while.
> > I did not see any objections to this idea that Darrick proposed,
> > so I think next step for you is to post patches, because some
> > developers will only engage when there are patches to discuss.
> >
> > Thanks,
> > Amir.
>
> Ok, I will try to prepare something. Just give me some weeks as would
> not have time for this right now. I just wanted to be sure that this
> is really the right direction and also that this is something which
> makes sense. I did not wanted to start doing something which could be
> completely useless... That is why I started rather longer discussion
> first.

No rush on my side.
You'd better wait a while to let other people comment before
going to implement anything.

People often have many and different opinions when it comes
to APIs design, so you will also need some patience to get to
consensus.

Thanks,
Amir.

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Wednesday 05 February 2025 17:33:30 Amir Goldstein wrote:
> On Tue, Feb 4, 2025 at 10:26 PM Pali Rohár <pali@kernel.org> wrote:
> >
> > On Tuesday 04 February 2025 12:54:01 Amir Goldstein wrote:
> > > On Tue, Feb 4, 2025 at 12:34 AM Pali Rohár <pali@kernel.org> wrote:
> > > >
> > > > On Tuesday 04 February 2025 00:02:44 Amir Goldstein wrote:
> > > > > On Mon, Feb 3, 2025 at 11:20 PM Pali Rohár <pali@kernel.org> wrote:
> > > > > >
> > > > > > On Monday 03 February 2025 22:59:46 Amir Goldstein wrote:
> > > > > > > On Sun, Feb 2, 2025 at 4:23 PM Pali Rohár <pali@kernel.org> wrote:
> > > > > > > > And there is still unresolved issue with FILE_ATTRIBUTE_READONLY.
> > > > > > > > Its meaning is similar to existing Linux FS_IMMUTABLE_FL, just
> > > > > > > > FILE_ATTRIBUTE_READONLY does not require root / CAP_LINUX_IMMUTABLE.
> > > > > > > >
> > > > > > > > I think that for proper support, to enforce FILE_ATTRIBUTE_READONLY
> > > > > > > > functionality, it is needed to introduce new flag e.g.
> > > > > > > > FS_IMMUTABLE_FL_USER to allow setting / clearing it also for normal
> > > > > > > > users without CAP_LINUX_IMMUTABLE. Otherwise it would be unsuitable for
> > > > > > > > any SMB client, SMB server or any application which would like to use
> > > > > > > > it, for example wine.
> > > > > > > >
> > > > > > > > Just to note that FreeBSD has two immutable flags SF_IMMUTABLE and
> > > > > > > > UF_IMMUTABLE, one settable only by superuser and second for owner.
> > > > > > > >
> > > > > > > > Any opinion?
> > > > > > >
> > > > > > > For filesystems that already support FILE_ATTRIBUTE_READONLY,
> > > > > > > can't you just set S_IMMUTABLE on the inode and vfs will do the correct
> > > > > > > enforcement?
> > > > > > >
> > > > > > > The vfs does not control if and how S_IMMUTABLE is set by filesystems,
> > > > > > > so if you want to remove this vfs flag without CAP_LINUX_IMMUTABLE
> > > > > > > in smb client, there is nothing stopping you (I think).
> > > > > >
> > > > > > Function fileattr_set_prepare() checks for CAP_LINUX_IMMUTABLE when
> > > > > > trying to change FS_IMMUTABLE_FL bit. This function is called from
> > > > > > ioctl(FS_IOC_SETFLAGS) and also from ioctl(FS_IOC_FSSETXATTR).
> > > > > > And when function fileattr_set_prepare() fails then .fileattr_set
> > > > > > callback is not called at all. So I think that it is not possible to
> > > > > > remove the IMMUTABLE flag from userspace without capability for smb
> > > > > > client.
> > > > > >
> > > > >
> > > > > You did not understand what I meant.
> > > > >
> > > > > You cannot relax the CAP_LINUX_IMMUTABLE for setting FS_IMMUTABLE_FL
> > > > > and there is no reason that you will need to relax it.
> > > > >
> > > > > The vfs does NOT enforce permissions according to FS_IMMUTABLE_FL
> > > > > The vfs enforces permissions according to the S_IMMUTABLE in-memory
> > > > > inode flag.
> > > > >
> > > > > There is no generic vfs code that sets S_IMMUTABLE inode flags, its
> > > > > the filesystems that translate the on-disk FS_IMMUTABLE_FL to
> > > > > in-memory S_IMMUTABLE inode flag.
> > > > >
> > > > > So if a filesystem already has an internal DOSATTRIB flags set, this
> > > > > filesystem can set the in-memory S_IMMUTABLE inode flag according
> > > > > to its knowledge of the DOSATTRIB_READONLY flag and the
> > > > > CAP_LINUX_IMMUTABLE rules do not apply to the DOSATTRIB_READONLY
> > > > > flag, which is NOT the same as the FS_IMMUTABLE_FL flag.
> > > > >
> > > > > > And it would not solve this problem for local filesystems (ntfs or ext4)
> > > > > > when Samba server or wine would want to set this bit.
> > > > > >
> > > > >
> > > > > The Samba server would use the FS_IOC_FS[GS]ETXATTR ioctl
> > > > > API to get/set dosattrib, something like this:
> > > > >
> > > > > struct fsxattr fsxattr;
> > > > > ret = ioctl_get_fsxattr(fd, &fsxattr);
> > > > > if (!ret && fsxattr.fsx_xflags & FS_XFLAG_HASDOSATTR) {
> > > > >     fsxattr.fsx_dosattr |= fs_dosattrib_readonly;
> > > > >     ret = ioctl_set_fsxattr(fd, &fsxattr);
> > > > > }
> > > >
> > > > Thanks for more explanation. First time I really did not understood it.
> > > > But now I think I understood it. So basically there would be two flags
> > > > which would result in setting S_IMMUTABLE on inode. One is the existing
> > > > FS_IMMUTABLE_FL which requires the capability and some new flag (e.g.
> > > > FS_XFLAG_HASDOSATTR) which would not require it and can be implemented
> > > > for cifs, vfat, ntfs, ... Right?
> > > >
> > >
> > > Well, almost right.
> > > The flag that would correspond to FILE_ATTRIBUTE_READONLY
> > > is FS_DOSATTRIB_READONLY from the new field fsx_dosattrib
> > > (see below)
> >
> > Thank you for example, it is for sure good starting point for me.
> >
> > > --- a/include/uapi/linux/fs.h
> > > +++ b/include/uapi/linux/fs.h
> > > @@ -145,7 +145,8 @@ struct fsxattr {
> > >         __u32           fsx_nextents;   /* nextents field value (get)   */
> > >         __u32           fsx_projid;     /* project identifier (get/set) */
> > >         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> > > -       unsigned char   fsx_pad[8];
> > > +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> > > +       unsigned char   fsx_pad[4];
> > >  };
> > >
> > >  /*
> > > @@ -167,7 +168,16 @@ struct fsxattr {
> > >  #define FS_XFLAG_FILESTREAM    0x00004000      /* use filestream allocator */
> > >  #define FS_XFLAG_DAX           0x00008000      /* use DAX for IO */
> > >  #define FS_XFLAG_COWEXTSIZE    0x00010000      /* CoW extent size
> > > allocator hint */
> > > -#define FS_XFLAG_HASATTR       0x80000000      /* no DIFLAG for this   */
> > > +#define FS_XFLAG_HASATTR       0x80000000      /* has extended attributes */
> > > +
> > > +/*
> > > + * Flags for the fsx_dosattrib field
> > > + */
> > > +#define FS_DOSATTRIB_READONLY  0x00000001      /* R - read-only */
> > > +#define FS_DOSATTRIB_HIDDEN    0x00000002      /* H - hidden */
> > > +#define FS_DOSATTRIB_SYSTEM    0x00000004      /* S - system */
> > > +#define FS_DOSATTRIB_ARCHIVE   0x00000020      /* A - archive */
> > > +#define FS_DOSATTRIB_HASATTR   0x80000000      /* has dos attributes */
> >
> > Should these FS_DOSATTRIB_* constants follows the Windows
> > FILE_ATTRIBUTE_* constants? Because I see that you put a gap between
> > system and archive.
> >
> 
> Well, no, they do not need to follow Windows contestants,
> but then again, why not?

I have just few cons:
- there are gaps and unused bits (e.g. FILE_ATTRIBUTE_VOLUME)
- there are bits which have more meanings (FILE_ATTRIBUTE_EA / FILE_ATTRIBUTE_RECALL_ON_OPEN)
- there are bits used only by WinAPI and not by NT / SMB
- there are bits which have different NT meaning in readdir() and in stat()
- there are bits for which we already have FS_IOC_GETFLAGS API

I think it that is is bad a idea if bit N in our ioctl() would have
different meaning than in our statx().

Should we duplicate FS_IOC_GETFLAGS FS_COMPR_FL and FS_ENCRYPT_FL into
that new fsx_dosattrib? For me it sounds like that it is a better idea
to have compression bit and encryption bit only in one field and not
duplicated.

But we can slightly follow Windows constants. Not exactly, but remove
redundancy, remove reserved bits, modify bits to have exactly one
meaning, etc... So basically do some sane modifications.

> I mean if we only ever needed the 4 RHSA bits above, we could
> have used the FS_XFLAG_* flags space, but if we extend the API
> with a new 32bit field, why not use 1-to-1 mapping at least as a starting point.

I understand it, for me it looks also a good idea, just needs to resolve
those problems above... and it would result in incompatibility that it
would not be 1-to-1 mapping at the end.

> You can see that it is quite common that filesystems re-define the
> same constants for these flags (e.g. EXT4_IMMUTABLE_FL).
> I am a bit surprised that there is no build time assertion
> BUILD_BUG_ON(EXT4_IMMUTABLE_FL != FS_IMMUTABLE_FL)
> which would be the standard way to make sure that the constants
> stay in sync if they need to be in sync, but some filesystems don't
> even assume that these constants are in sync (e.g. f2fs_fsflags_map)
> 
> > > This last special flag is debatable and I am not really sure that we need it.
> >
> > This constant has very similar meaning to FILE_ATTRIBUTE_NORMAL. Both
> > has some compatibility meaning that "field is valid or something is set".
> > Just FILE_ATTRIBUTE_NORMAL is not 31th bit.
> >
> 
> No it does not. I don't think that you understood the meaning of
> FS_DOSATTRIB_HASATTR.
> Nevermind it was a bad idea anyway. see more below.
> 
> > > It is needed for proper backward compat with existing userspace tools.
> > > For example, if there was a backup tool storing the fsxattr blob result of
> > > FS_IOC_FSGETXATTR and sets it later during restore with
> > > FS_IOC_FSSETXATTR, then it would be better to ignore a zero
> > > value of fsx_dosattrib instead of resetting all of the on-disk dosattrib flags
> > > if the restore happens after ntfs gained support for setting dosattrib flags
> > > via FS_IOC_FSSETXATTR.
> > >
> > > When using the standard tools to set fsxattr (chattr and xfs_io -c chattr)
> > > the tool does FS_IOC_FSGETXATTR + modify + FS_IOC_FSSETXATTR,
> > > so those tools are expected to leave new bits in fsx_dosattrib at their
> > > value if ntfs gains support for get/set fsx_dosattrib.
> > >
> > > Setting the auxiliary FS_DOSATTRIB_HASATTR flag can help the
> > > kernel/fs to explicitly state that the values returned in fsx_dosattrib
> > > are valid and the tool to state that values set in fsx_dosattrib are valid.
> > > But using a single flag will not help expanding ntfs support for more
> > > fsx_dosattrib flags later, so I am not sure if it is useful (?).
> >
> > If the fsx_dosattrib would match all FILE_ATTRIBUTE_* then we can do it
> > as the ntfs matches FILE_ATTRIBUTE_* and no extension is needed for
> > future.
> >
> > And I think that this backward compatibility sounds good.
> >
> 
> That's only true if you can support ALL the dosattrib flags from the
> first version and that Windows will not add any of the reserved
> flags in the future, which is hard to commit to.

I see, it would not work. And Windows will for sure use some reserved
bits in future.

> > What could be useful for userspace is also ability to figure out which
> > FS_DOSATTRIB_* are supported by the filesystem. Because for example UDF
> > on-disk format supports only FS_DOSATTRIB_HIDDEN bit. And FAT only those
> > attributes which are in the lowest byte.
> >
> 
> Exactly.
> statx has this solved with the stx_attributes_mask field.
> 
> We could do the same for FS_IOC_FS[GS]ETXATTR, but because
> right now, this API does not verify that fsx_pad is zero, we will need to
> define a new set of ioctl consants FS_IOC_[GS]ETFSXATTR2
> with the exact same functionality but that userspace knows that they
> publish and respect the dosattrib mask:

I understand and this is a problem.

> --- a/fs/ioctl.c
> +++ b/fs/ioctl.c
> @@ -868,9 +868,11 @@ static int do_vfs_ioctl(struct file *filp, unsigned int fd,
>         case FS_IOC_SETFLAGS:
>                 return ioctl_setflags(filp, argp);
> 
> +       case FS_IOC_GETFSXATTR2:
>         case FS_IOC_FSGETXATTR:
>                 return ioctl_fsgetxattr(filp, argp);
> 
> +       case FS_IOC_SETFSXATTR2:
>         case FS_IOC_FSSETXATTR:
>                 return ioctl_fssetxattr(filp, argp);
> --- a/include/uapi/linux/fs.h
> +++ b/include/uapi/linux/fs.h
> @@ -145,7 +145,8 @@ struct fsxattr {
>         __u32           fsx_nextents;   /* nextents field value (get)   */
>         __u32           fsx_projid;     /* project identifier (get/set) */
>         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> -       unsigned char   fsx_pad[8];
> +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> +       __u32           fsx_dosattrib_mask; /* dosattrib field validity mask */
>  };
> 
>  /*
> @@ -238,6 +248,9 @@ struct fsxattr {
>  #define FS_IOC32_SETFLAGS              _IOW('f', 2, int)
>  #define FS_IOC32_GETVERSION            _IOR('v', 1, int)
>  #define FS_IOC32_SETVERSION            _IOW('v', 2, int)
> +#define FS_IOC_GETFSXATTR2              _IOR('x', 31, struct fsxattr)
> +#define FS_IOC_SETFSXATTR2              _IOW('x', 32, struct fsxattr)
> +/* Duplicate legacy ioctl numbers for backward compact */
>  #define FS_IOC_FSGETXATTR              _IOR('X', 31, struct fsxattr)
>  #define FS_IOC_FSSETXATTR              _IOW('X', 32, struct fsxattr)
>  #define FS_IOC_GETFSLABEL              _IOR(0x94, 49, char[FSLABEL_MAX])
> 
> We could also use this opportunity to define a larger fsxattr2 struct
> that also includes an fsx_xflags_mask field, so that the xflags namespace
> could also be extended in a backward compat way going forward:
> 
> @@ -145,7 +145,21 @@ struct fsxattr {
>         __u32           fsx_nextents;   /* nextents field value (get)   */
>         __u32           fsx_projid;     /* project identifier (get/set) */
>         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
>         unsigned char   fsx_pad[8];
> 
>  };
> +
> +/*
> + * Structure for FS_IOC_[GS]ETFSXATTR2.
> + */
> +struct fsxattr2 {
> +       __u32           fsx_xflags;     /* xflags field value (get/set) */
> +       __u32           fsx_extsize;    /* extsize field value (get/set)*/
> +       __u32           fsx_nextents;   /* nextents field value (get)   */
> +       __u32           fsx_projid;     /* project identifier (get/set) */
> +       __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> +       __u32           fsx_xflags_mask; /* xflags field validity mask */
> +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> +       __u32           fsx_dosattrib_mask; /* dosattrib field validity mask */
> +};
> 
> And you'd also need to flug those new mask and dosattrib
> via struct fileattr into filesystems - too much to explain.
> try to figure it out (unless someone objects) and if you can't figure
> it out let me know.

Yea, I think that this is thing which I should be able to figure out
once I start changing it.

Anyway, I have alternative idea to the problem with fsx_pad. What about
introducing new fsx_xflags flag which would say that fsx_pad=fsx_dosattrib
is present? E.g.

#define FS_XFLAG_HASDOSATTRIB 0x40000000

Then we would not need new FS_IOC_GETFSXATTR2/FS_IOC_SETFSXATTR2 ioctls.

Also fsx_pad has 8 bytes, which can store both attrib and mask, so new
struct fsxattr2 would not be needed too.

> > >
> > > > > For ntfs/ext4, you will need to implement on-disk support for
> > > > > set/get the dosattrib flags.
> > > >
> > > > ntfs has already on-disk support for FILE_ATTRIBUTE_READONLY.
> > > >
> > >
> > > This is interesting.
> >
> > I mean that ntfs filesystem has support for FILE_ATTRIBUTE_READONLY.
> > I did not mean linux ntfs implementation.
> >
> > But I'm aware of some of those details in linux fs implementations, but
> > I did not wanted to mentioned it as basically every linux fs
> > implementation has its own way how flags are handled or exported to
> > userspace. It is good to know, but not important when designing or
> > discussing the unified/generic standard API.
> >
> 
> A good API will describe what the existing filesystems already support
> and what they do not support.
> 
> > > fat/ntfs both already have a mount option sys_immutable to map
> > > FILE_ATTRIBUTE_SYSTEM to S_IMMUTABLE in-memory.
> > >
> > > fat does not support fileattr_set(), but has a proprietary ioctl
> > > FAT_IOCTL_SET_ATTRIBUTES which enforces
> > > CAP_LINUX_IMMUTABLE for changing S_IMMUTABLE.
> > >
> > > ntfs also maps FILE_ATTRIBUTE_SYSTEM to S_IMMUTABLE
> > > and it allows changing FILE_ATTRIBUTE_SYSTEM via ntfs_setxattr
> > > of system.{dos,ntfs}_attrib without enforcing CAP_LINUX_IMMUTABLE,
> > > or any other permissions at all (?)
> > > This does not change S_IMMUTABLE in-memory, so change will
> > > only apply on the next time inode is loaded from disk.
> > > Bottom line: seems like *any user at all* can change the READONLY
> > > and SYSTEM attributes on ntfs.
> > >
> > > OTOH, ntfs does support fileattr_set() - it allows changing
> > > S_IMMUTABLE and S_APPEND in-memory, but as far as I can
> > > tell, this change is not stored on-disk (?).
> > >
> > > Also in ntfs, FILE_ATTRIBUTE_READONLY is mapped
> > > to not having posix write permissions on-disk:
> > >                 /* Linux 'w' -> Windows 'ro'. */
> > >                 if (0222 & inode->i_mode)
> > >                         ni->std_fa &= ~FILE_ATTRIBUTE_READONLY;
> > >                 else
> > >                         ni->std_fa |= FILE_ATTRIBUTE_READONLY;
> > >
> > > So for ntfs, S_IMMUTABLE could be updated depending on three
> > > independent flags: SYSTEM, READONLY and  FS_XFLAG_IMMUTABLE.
> > >
> > > Having ntfs treat FILE_ATTRIBUTE_READONLY as S_IMMUTABLE
> > > internally, is completely confined to ntfs and has nothing to do with vfs
> > > or with a new standard API.
> > >
> > > > On-disk support for ext4 and other linux filesystems can be discussed
> > > > later. I think that this could be more controversial.
> > > >
> > >
> > > Obviously there are existing users that need this.
> > > Samba has its own xattr user.DOSATTRIB and if people really want
> > > to be able to export those attributes in a standard way, I doubt there
> > > will be objection to adding on-disk support (e.g. to ext4/xfs).
> > > But somebody has to do the work and adding new on-disk support
> > > is not so easy.
> >
> > Yes, it is not easy and on-disk support can be done later or basically
> > independently of this work here. So I will let it for other people.
> >
> > > I can help with that when the time comes.
> > > First thing first, try to propose patches to extend fsx_dosattrib and
> > > support them in ntfs/fat/smb.
> >
> > Ok. Thanks.
> >
> > > > > I can certainly not change the meaning of existing on-disk
> > > > > flag of FS_IMMUTABLE_FL to a flag that can be removed
> > > > > without CAP_LINUX_IMMUTABLE. that changes the meaning
> > > > > of the flag.
> > > > >
> > > > > If ext4 maintainers agrees, you may be able to reuse some
> > > > > old unused on-disk flags (e.g.  EXT4_UNRM_FL) as storage
> > > > > place for FS_DOSATTRIB_READONLY, but that would be
> > > > > quite hackish.
> > > > >
> > > > > > > How about tackling this one small step at a time, not in that order
> > > > > > > necessarily:
> > > > > > >
> > > > > > > 1. Implement the standard API with FS_IOC_FS[GS]ETXATTR ioctl
> > > > > > >     and with statx to get/set some non-controversial dosattrib flags on
> > > > > > >     ntfs/smb/vfat
> > > > > > > 2. Wire some interesting dosattrib flags (e.g. compr/enrypt) to local
> > > > > > >     filesystems that already support storing those bits
> > > > > > > 3. Wire network servers (e.g. Samba) to use the generic API if supported
> > > > > > > 4. Add on-disk support for storing the dosattrib flags to more local fs
> > > > > > > 5. Update S_IMMUTABLE inode flag if either FS_XFLAG_IMMUTABLE
> > > > > > >     or FS_DOSATTRIB_READONLY are set on the file
> > > > > > >
> > > > > > > Thoughts?
> > > > > > >
> > > > >
> > > > > Anything wrong with the plan above?
> > > > > It seems that you are looking for shortcuts and I don't think that
> > > > > it is a good way to make progress.
> > > > >
> > > > > Thanks,
> > > > > Amir.
> > > >
> > > > If other developers agree that the FS_IOC_FS[GS]ETXATTR ioctl is the
> > > > right direction then for me it looks good.
> > >
> > > This thread has been going on for a while.
> > > I did not see any objections to this idea that Darrick proposed,
> > > so I think next step for you is to post patches, because some
> > > developers will only engage when there are patches to discuss.
> > >
> > > Thanks,
> > > Amir.
> >
> > Ok, I will try to prepare something. Just give me some weeks as would
> > not have time for this right now. I just wanted to be sure that this
> > is really the right direction and also that this is something which
> > makes sense. I did not wanted to start doing something which could be
> > completely useless... That is why I started rather longer discussion
> > first.
> 
> No rush on my side.
> You'd better wait a while to let other people comment before
> going to implement anything.
> 
> People often have many and different opinions when it comes
> to APIs design, so you will also need some patience to get to
> consensus.
> 
> Thanks,
> Amir.

Yes, that is fine. And thank you for your input.

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

On Wednesday 05 February 2025 19:16:45 Pali Rohár wrote:
> On Wednesday 05 February 2025 17:33:30 Amir Goldstein wrote:
> > On Tue, Feb 4, 2025 at 10:26 PM Pali Rohár <pali@kernel.org> wrote:
> > >
> > > On Tuesday 04 February 2025 12:54:01 Amir Goldstein wrote:
> > > > On Tue, Feb 4, 2025 at 12:34 AM Pali Rohár <pali@kernel.org> wrote:
> > > > >
> > > > > On Tuesday 04 February 2025 00:02:44 Amir Goldstein wrote:
> > > > > > On Mon, Feb 3, 2025 at 11:20 PM Pali Rohár <pali@kernel.org> wrote:
> > > > > > >
> > > > > > > On Monday 03 February 2025 22:59:46 Amir Goldstein wrote:
> > > > > > > > On Sun, Feb 2, 2025 at 4:23 PM Pali Rohár <pali@kernel.org> wrote:
> > > > > > > > > And there is still unresolved issue with FILE_ATTRIBUTE_READONLY.
> > > > > > > > > Its meaning is similar to existing Linux FS_IMMUTABLE_FL, just
> > > > > > > > > FILE_ATTRIBUTE_READONLY does not require root / CAP_LINUX_IMMUTABLE.
> > > > > > > > >
> > > > > > > > > I think that for proper support, to enforce FILE_ATTRIBUTE_READONLY
> > > > > > > > > functionality, it is needed to introduce new flag e.g.
> > > > > > > > > FS_IMMUTABLE_FL_USER to allow setting / clearing it also for normal
> > > > > > > > > users without CAP_LINUX_IMMUTABLE. Otherwise it would be unsuitable for
> > > > > > > > > any SMB client, SMB server or any application which would like to use
> > > > > > > > > it, for example wine.
> > > > > > > > >
> > > > > > > > > Just to note that FreeBSD has two immutable flags SF_IMMUTABLE and
> > > > > > > > > UF_IMMUTABLE, one settable only by superuser and second for owner.
> > > > > > > > >
> > > > > > > > > Any opinion?
> > > > > > > >
> > > > > > > > For filesystems that already support FILE_ATTRIBUTE_READONLY,
> > > > > > > > can't you just set S_IMMUTABLE on the inode and vfs will do the correct
> > > > > > > > enforcement?
> > > > > > > >
> > > > > > > > The vfs does not control if and how S_IMMUTABLE is set by filesystems,
> > > > > > > > so if you want to remove this vfs flag without CAP_LINUX_IMMUTABLE
> > > > > > > > in smb client, there is nothing stopping you (I think).
> > > > > > >
> > > > > > > Function fileattr_set_prepare() checks for CAP_LINUX_IMMUTABLE when
> > > > > > > trying to change FS_IMMUTABLE_FL bit. This function is called from
> > > > > > > ioctl(FS_IOC_SETFLAGS) and also from ioctl(FS_IOC_FSSETXATTR).
> > > > > > > And when function fileattr_set_prepare() fails then .fileattr_set
> > > > > > > callback is not called at all. So I think that it is not possible to
> > > > > > > remove the IMMUTABLE flag from userspace without capability for smb
> > > > > > > client.
> > > > > > >
> > > > > >
> > > > > > You did not understand what I meant.
> > > > > >
> > > > > > You cannot relax the CAP_LINUX_IMMUTABLE for setting FS_IMMUTABLE_FL
> > > > > > and there is no reason that you will need to relax it.
> > > > > >
> > > > > > The vfs does NOT enforce permissions according to FS_IMMUTABLE_FL
> > > > > > The vfs enforces permissions according to the S_IMMUTABLE in-memory
> > > > > > inode flag.
> > > > > >
> > > > > > There is no generic vfs code that sets S_IMMUTABLE inode flags, its
> > > > > > the filesystems that translate the on-disk FS_IMMUTABLE_FL to
> > > > > > in-memory S_IMMUTABLE inode flag.
> > > > > >
> > > > > > So if a filesystem already has an internal DOSATTRIB flags set, this
> > > > > > filesystem can set the in-memory S_IMMUTABLE inode flag according
> > > > > > to its knowledge of the DOSATTRIB_READONLY flag and the
> > > > > > CAP_LINUX_IMMUTABLE rules do not apply to the DOSATTRIB_READONLY
> > > > > > flag, which is NOT the same as the FS_IMMUTABLE_FL flag.
> > > > > >
> > > > > > > And it would not solve this problem for local filesystems (ntfs or ext4)
> > > > > > > when Samba server or wine would want to set this bit.
> > > > > > >
> > > > > >
> > > > > > The Samba server would use the FS_IOC_FS[GS]ETXATTR ioctl
> > > > > > API to get/set dosattrib, something like this:
> > > > > >
> > > > > > struct fsxattr fsxattr;
> > > > > > ret = ioctl_get_fsxattr(fd, &fsxattr);
> > > > > > if (!ret && fsxattr.fsx_xflags & FS_XFLAG_HASDOSATTR) {
> > > > > >     fsxattr.fsx_dosattr |= fs_dosattrib_readonly;
> > > > > >     ret = ioctl_set_fsxattr(fd, &fsxattr);
> > > > > > }
> > > > >
> > > > > Thanks for more explanation. First time I really did not understood it.
> > > > > But now I think I understood it. So basically there would be two flags
> > > > > which would result in setting S_IMMUTABLE on inode. One is the existing
> > > > > FS_IMMUTABLE_FL which requires the capability and some new flag (e.g.
> > > > > FS_XFLAG_HASDOSATTR) which would not require it and can be implemented
> > > > > for cifs, vfat, ntfs, ... Right?
> > > > >
> > > >
> > > > Well, almost right.
> > > > The flag that would correspond to FILE_ATTRIBUTE_READONLY
> > > > is FS_DOSATTRIB_READONLY from the new field fsx_dosattrib
> > > > (see below)
> > >
> > > Thank you for example, it is for sure good starting point for me.
> > >
> > > > --- a/include/uapi/linux/fs.h
> > > > +++ b/include/uapi/linux/fs.h
> > > > @@ -145,7 +145,8 @@ struct fsxattr {
> > > >         __u32           fsx_nextents;   /* nextents field value (get)   */
> > > >         __u32           fsx_projid;     /* project identifier (get/set) */
> > > >         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> > > > -       unsigned char   fsx_pad[8];
> > > > +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> > > > +       unsigned char   fsx_pad[4];
> > > >  };
> > > >
> > > >  /*
> > > > @@ -167,7 +168,16 @@ struct fsxattr {
> > > >  #define FS_XFLAG_FILESTREAM    0x00004000      /* use filestream allocator */
> > > >  #define FS_XFLAG_DAX           0x00008000      /* use DAX for IO */
> > > >  #define FS_XFLAG_COWEXTSIZE    0x00010000      /* CoW extent size
> > > > allocator hint */
> > > > -#define FS_XFLAG_HASATTR       0x80000000      /* no DIFLAG for this   */
> > > > +#define FS_XFLAG_HASATTR       0x80000000      /* has extended attributes */
> > > > +
> > > > +/*
> > > > + * Flags for the fsx_dosattrib field
> > > > + */
> > > > +#define FS_DOSATTRIB_READONLY  0x00000001      /* R - read-only */
> > > > +#define FS_DOSATTRIB_HIDDEN    0x00000002      /* H - hidden */
> > > > +#define FS_DOSATTRIB_SYSTEM    0x00000004      /* S - system */
> > > > +#define FS_DOSATTRIB_ARCHIVE   0x00000020      /* A - archive */
> > > > +#define FS_DOSATTRIB_HASATTR   0x80000000      /* has dos attributes */
> > >
> > > Should these FS_DOSATTRIB_* constants follows the Windows
> > > FILE_ATTRIBUTE_* constants? Because I see that you put a gap between
> > > system and archive.
> > >
> > 
> > Well, no, they do not need to follow Windows contestants,
> > but then again, why not?
> 
> I have just few cons:
> - there are gaps and unused bits (e.g. FILE_ATTRIBUTE_VOLUME)
> - there are bits which have more meanings (FILE_ATTRIBUTE_EA / FILE_ATTRIBUTE_RECALL_ON_OPEN)
> - there are bits used only by WinAPI and not by NT / SMB
> - there are bits which have different NT meaning in readdir() and in stat()
> - there are bits for which we already have FS_IOC_GETFLAGS API
> 
> I think it that is is bad a idea if bit N in our ioctl() would have
> different meaning than in our statx().
> 
> Should we duplicate FS_IOC_GETFLAGS FS_COMPR_FL and FS_ENCRYPT_FL into
> that new fsx_dosattrib? For me it sounds like that it is a better idea
> to have compression bit and encryption bit only in one field and not
> duplicated.
> 
> But we can slightly follow Windows constants. Not exactly, but remove
> redundancy, remove reserved bits, modify bits to have exactly one
> meaning, etc... So basically do some sane modifications.
> 
> > I mean if we only ever needed the 4 RHSA bits above, we could
> > have used the FS_XFLAG_* flags space, but if we extend the API
> > with a new 32bit field, why not use 1-to-1 mapping at least as a starting point.
> 
> I understand it, for me it looks also a good idea, just needs to resolve
> those problems above... and it would result in incompatibility that it
> would not be 1-to-1 mapping at the end.
> 
> > You can see that it is quite common that filesystems re-define the
> > same constants for these flags (e.g. EXT4_IMMUTABLE_FL).
> > I am a bit surprised that there is no build time assertion
> > BUILD_BUG_ON(EXT4_IMMUTABLE_FL != FS_IMMUTABLE_FL)
> > which would be the standard way to make sure that the constants
> > stay in sync if they need to be in sync, but some filesystems don't
> > even assume that these constants are in sync (e.g. f2fs_fsflags_map)
> > 
> > > > This last special flag is debatable and I am not really sure that we need it.
> > >
> > > This constant has very similar meaning to FILE_ATTRIBUTE_NORMAL. Both
> > > has some compatibility meaning that "field is valid or something is set".
> > > Just FILE_ATTRIBUTE_NORMAL is not 31th bit.
> > >
> > 
> > No it does not. I don't think that you understood the meaning of
> > FS_DOSATTRIB_HASATTR.
> > Nevermind it was a bad idea anyway. see more below.
> > 
> > > > It is needed for proper backward compat with existing userspace tools.
> > > > For example, if there was a backup tool storing the fsxattr blob result of
> > > > FS_IOC_FSGETXATTR and sets it later during restore with
> > > > FS_IOC_FSSETXATTR, then it would be better to ignore a zero
> > > > value of fsx_dosattrib instead of resetting all of the on-disk dosattrib flags
> > > > if the restore happens after ntfs gained support for setting dosattrib flags
> > > > via FS_IOC_FSSETXATTR.
> > > >
> > > > When using the standard tools to set fsxattr (chattr and xfs_io -c chattr)
> > > > the tool does FS_IOC_FSGETXATTR + modify + FS_IOC_FSSETXATTR,
> > > > so those tools are expected to leave new bits in fsx_dosattrib at their
> > > > value if ntfs gains support for get/set fsx_dosattrib.
> > > >
> > > > Setting the auxiliary FS_DOSATTRIB_HASATTR flag can help the
> > > > kernel/fs to explicitly state that the values returned in fsx_dosattrib
> > > > are valid and the tool to state that values set in fsx_dosattrib are valid.
> > > > But using a single flag will not help expanding ntfs support for more
> > > > fsx_dosattrib flags later, so I am not sure if it is useful (?).
> > >
> > > If the fsx_dosattrib would match all FILE_ATTRIBUTE_* then we can do it
> > > as the ntfs matches FILE_ATTRIBUTE_* and no extension is needed for
> > > future.
> > >
> > > And I think that this backward compatibility sounds good.
> > >
> > 
> > That's only true if you can support ALL the dosattrib flags from the
> > first version and that Windows will not add any of the reserved
> > flags in the future, which is hard to commit to.
> 
> I see, it would not work. And Windows will for sure use some reserved
> bits in future.
> 
> > > What could be useful for userspace is also ability to figure out which
> > > FS_DOSATTRIB_* are supported by the filesystem. Because for example UDF
> > > on-disk format supports only FS_DOSATTRIB_HIDDEN bit. And FAT only those
> > > attributes which are in the lowest byte.
> > >
> > 
> > Exactly.
> > statx has this solved with the stx_attributes_mask field.
> > 
> > We could do the same for FS_IOC_FS[GS]ETXATTR, but because
> > right now, this API does not verify that fsx_pad is zero, we will need to
> > define a new set of ioctl consants FS_IOC_[GS]ETFSXATTR2
> > with the exact same functionality but that userspace knows that they
> > publish and respect the dosattrib mask:
> 
> I understand and this is a problem.
> 
> > --- a/fs/ioctl.c
> > +++ b/fs/ioctl.c
> > @@ -868,9 +868,11 @@ static int do_vfs_ioctl(struct file *filp, unsigned int fd,
> >         case FS_IOC_SETFLAGS:
> >                 return ioctl_setflags(filp, argp);
> > 
> > +       case FS_IOC_GETFSXATTR2:
> >         case FS_IOC_FSGETXATTR:
> >                 return ioctl_fsgetxattr(filp, argp);
> > 
> > +       case FS_IOC_SETFSXATTR2:
> >         case FS_IOC_FSSETXATTR:
> >                 return ioctl_fssetxattr(filp, argp);
> > --- a/include/uapi/linux/fs.h
> > +++ b/include/uapi/linux/fs.h
> > @@ -145,7 +145,8 @@ struct fsxattr {
> >         __u32           fsx_nextents;   /* nextents field value (get)   */
> >         __u32           fsx_projid;     /* project identifier (get/set) */
> >         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> > -       unsigned char   fsx_pad[8];
> > +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> > +       __u32           fsx_dosattrib_mask; /* dosattrib field validity mask */
> >  };
> > 
> >  /*
> > @@ -238,6 +248,9 @@ struct fsxattr {
> >  #define FS_IOC32_SETFLAGS              _IOW('f', 2, int)
> >  #define FS_IOC32_GETVERSION            _IOR('v', 1, int)
> >  #define FS_IOC32_SETVERSION            _IOW('v', 2, int)
> > +#define FS_IOC_GETFSXATTR2              _IOR('x', 31, struct fsxattr)
> > +#define FS_IOC_SETFSXATTR2              _IOW('x', 32, struct fsxattr)
> > +/* Duplicate legacy ioctl numbers for backward compact */
> >  #define FS_IOC_FSGETXATTR              _IOR('X', 31, struct fsxattr)
> >  #define FS_IOC_FSSETXATTR              _IOW('X', 32, struct fsxattr)
> >  #define FS_IOC_GETFSLABEL              _IOR(0x94, 49, char[FSLABEL_MAX])
> > 
> > We could also use this opportunity to define a larger fsxattr2 struct
> > that also includes an fsx_xflags_mask field, so that the xflags namespace
> > could also be extended in a backward compat way going forward:
> > 
> > @@ -145,7 +145,21 @@ struct fsxattr {
> >         __u32           fsx_nextents;   /* nextents field value (get)   */
> >         __u32           fsx_projid;     /* project identifier (get/set) */
> >         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> >         unsigned char   fsx_pad[8];
> > 
> >  };
> > +
> > +/*
> > + * Structure for FS_IOC_[GS]ETFSXATTR2.
> > + */
> > +struct fsxattr2 {
> > +       __u32           fsx_xflags;     /* xflags field value (get/set) */
> > +       __u32           fsx_extsize;    /* extsize field value (get/set)*/
> > +       __u32           fsx_nextents;   /* nextents field value (get)   */
> > +       __u32           fsx_projid;     /* project identifier (get/set) */
> > +       __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> > +       __u32           fsx_xflags_mask; /* xflags field validity mask */
> > +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> > +       __u32           fsx_dosattrib_mask; /* dosattrib field validity mask */
> > +};
> > 
> > And you'd also need to flug those new mask and dosattrib
> > via struct fileattr into filesystems - too much to explain.
> > try to figure it out (unless someone objects) and if you can't figure
> > it out let me know.
> 
> Yea, I think that this is thing which I should be able to figure out
> once I start changing it.
> 
> Anyway, I have alternative idea to the problem with fsx_pad. What about
> introducing new fsx_xflags flag which would say that fsx_pad=fsx_dosattrib
> is present? E.g.
> 
> #define FS_XFLAG_HASDOSATTRIB 0x40000000
> 
> Then we would not need new FS_IOC_GETFSXATTR2/FS_IOC_SETFSXATTR2 ioctls.
> 
> Also fsx_pad has 8 bytes, which can store both attrib and mask, so new
> struct fsxattr2 would not be needed too.

In case we decide to not do 1-to-1 mapping of Windows FILE_ATTRIBUTE_*
constants to these new Linux DOSATTRIB_* constants then we do not need
32-bit variable, but just 16-bit variable (I counted that we need just
10 bits for now). fsx_pad has currently 64 bits and we could use it for:
- 32 bits for fsx_xflags_mask
- 16 bits for fsx_dosattrib
- 16 bits for fsx_dosattrib_mask
And therefore no need for FS_IOC_GETFSXATTR2/FS_IOC_SETFSXATTR2 iocl or
struct fsxattr2. Just an idea how to simplify new API. Maybe together
with new fsx_xflags bit to indicate that fsx_xflags_mask field is present:
#define FS_XFLAG_HASXFLAGSMASK 0x20000000

> > > >
> > > > > > For ntfs/ext4, you will need to implement on-disk support for
> > > > > > set/get the dosattrib flags.
> > > > >
> > > > > ntfs has already on-disk support for FILE_ATTRIBUTE_READONLY.
> > > > >
> > > >
> > > > This is interesting.
> > >
> > > I mean that ntfs filesystem has support for FILE_ATTRIBUTE_READONLY.
> > > I did not mean linux ntfs implementation.
> > >
> > > But I'm aware of some of those details in linux fs implementations, but
> > > I did not wanted to mentioned it as basically every linux fs
> > > implementation has its own way how flags are handled or exported to
> > > userspace. It is good to know, but not important when designing or
> > > discussing the unified/generic standard API.
> > >
> > 
> > A good API will describe what the existing filesystems already support
> > and what they do not support.
> > 
> > > > fat/ntfs both already have a mount option sys_immutable to map
> > > > FILE_ATTRIBUTE_SYSTEM to S_IMMUTABLE in-memory.
> > > >
> > > > fat does not support fileattr_set(), but has a proprietary ioctl
> > > > FAT_IOCTL_SET_ATTRIBUTES which enforces
> > > > CAP_LINUX_IMMUTABLE for changing S_IMMUTABLE.
> > > >
> > > > ntfs also maps FILE_ATTRIBUTE_SYSTEM to S_IMMUTABLE
> > > > and it allows changing FILE_ATTRIBUTE_SYSTEM via ntfs_setxattr
> > > > of system.{dos,ntfs}_attrib without enforcing CAP_LINUX_IMMUTABLE,
> > > > or any other permissions at all (?)
> > > > This does not change S_IMMUTABLE in-memory, so change will
> > > > only apply on the next time inode is loaded from disk.
> > > > Bottom line: seems like *any user at all* can change the READONLY
> > > > and SYSTEM attributes on ntfs.
> > > >
> > > > OTOH, ntfs does support fileattr_set() - it allows changing
> > > > S_IMMUTABLE and S_APPEND in-memory, but as far as I can
> > > > tell, this change is not stored on-disk (?).
> > > >
> > > > Also in ntfs, FILE_ATTRIBUTE_READONLY is mapped
> > > > to not having posix write permissions on-disk:
> > > >                 /* Linux 'w' -> Windows 'ro'. */
> > > >                 if (0222 & inode->i_mode)
> > > >                         ni->std_fa &= ~FILE_ATTRIBUTE_READONLY;
> > > >                 else
> > > >                         ni->std_fa |= FILE_ATTRIBUTE_READONLY;
> > > >
> > > > So for ntfs, S_IMMUTABLE could be updated depending on three
> > > > independent flags: SYSTEM, READONLY and  FS_XFLAG_IMMUTABLE.
> > > >
> > > > Having ntfs treat FILE_ATTRIBUTE_READONLY as S_IMMUTABLE
> > > > internally, is completely confined to ntfs and has nothing to do with vfs
> > > > or with a new standard API.
> > > >
> > > > > On-disk support for ext4 and other linux filesystems can be discussed
> > > > > later. I think that this could be more controversial.
> > > > >
> > > >
> > > > Obviously there are existing users that need this.
> > > > Samba has its own xattr user.DOSATTRIB and if people really want
> > > > to be able to export those attributes in a standard way, I doubt there
> > > > will be objection to adding on-disk support (e.g. to ext4/xfs).
> > > > But somebody has to do the work and adding new on-disk support
> > > > is not so easy.
> > >
> > > Yes, it is not easy and on-disk support can be done later or basically
> > > independently of this work here. So I will let it for other people.
> > >
> > > > I can help with that when the time comes.
> > > > First thing first, try to propose patches to extend fsx_dosattrib and
> > > > support them in ntfs/fat/smb.
> > >
> > > Ok. Thanks.
> > >
> > > > > > I can certainly not change the meaning of existing on-disk
> > > > > > flag of FS_IMMUTABLE_FL to a flag that can be removed
> > > > > > without CAP_LINUX_IMMUTABLE. that changes the meaning
> > > > > > of the flag.
> > > > > >
> > > > > > If ext4 maintainers agrees, you may be able to reuse some
> > > > > > old unused on-disk flags (e.g.  EXT4_UNRM_FL) as storage
> > > > > > place for FS_DOSATTRIB_READONLY, but that would be
> > > > > > quite hackish.
> > > > > >
> > > > > > > > How about tackling this one small step at a time, not in that order
> > > > > > > > necessarily:
> > > > > > > >
> > > > > > > > 1. Implement the standard API with FS_IOC_FS[GS]ETXATTR ioctl
> > > > > > > >     and with statx to get/set some non-controversial dosattrib flags on
> > > > > > > >     ntfs/smb/vfat
> > > > > > > > 2. Wire some interesting dosattrib flags (e.g. compr/enrypt) to local
> > > > > > > >     filesystems that already support storing those bits
> > > > > > > > 3. Wire network servers (e.g. Samba) to use the generic API if supported
> > > > > > > > 4. Add on-disk support for storing the dosattrib flags to more local fs
> > > > > > > > 5. Update S_IMMUTABLE inode flag if either FS_XFLAG_IMMUTABLE
> > > > > > > >     or FS_DOSATTRIB_READONLY are set on the file
> > > > > > > >
> > > > > > > > Thoughts?
> > > > > > > >
> > > > > >
> > > > > > Anything wrong with the plan above?
> > > > > > It seems that you are looking for shortcuts and I don't think that
> > > > > > it is a good way to make progress.
> > > > > >
> > > > > > Thanks,
> > > > > > Amir.
> > > > >
> > > > > If other developers agree that the FS_IOC_FS[GS]ETXATTR ioctl is the
> > > > > right direction then for me it looks good.
> > > >
> > > > This thread has been going on for a while.
> > > > I did not see any objections to this idea that Darrick proposed,
> > > > so I think next step for you is to post patches, because some
> > > > developers will only engage when there are patches to discuss.
> > > >
> > > > Thanks,
> > > > Amir.
> > >
> > > Ok, I will try to prepare something. Just give me some weeks as would
> > > not have time for this right now. I just wanted to be sure that this
> > > is really the right direction and also that this is something which
> > > makes sense. I did not wanted to start doing something which could be
> > > completely useless... That is why I started rather longer discussion
> > > first.
> > 
> > No rush on my side.
> > You'd better wait a while to let other people comment before
> > going to implement anything.
> > 
> > People often have many and different opinions when it comes
> > to APIs design, so you will also need some patience to get to
> > consensus.
> > 
> > Thanks,
> > Amir.
> 
> Yes, that is fine. And thank you for your input.

Re: Immutable vs read-only for Windows compatibility

[Amir Goldstein]

> > > > What could be useful for userspace is also ability to figure out which
> > > > FS_DOSATTRIB_* are supported by the filesystem. Because for example UDF
> > > > on-disk format supports only FS_DOSATTRIB_HIDDEN bit. And FAT only those
> > > > attributes which are in the lowest byte.
> > > >
> > >
> > > Exactly.
> > > statx has this solved with the stx_attributes_mask field.
> > >
> > > We could do the same for FS_IOC_FS[GS]ETXATTR, but because
> > > right now, this API does not verify that fsx_pad is zero, we will need to
> > > define a new set of ioctl consants FS_IOC_[GS]ETFSXATTR2
> > > with the exact same functionality but that userspace knows that they
> > > publish and respect the dosattrib mask:
> >
> > I understand and this is a problem.
> >
> > > --- a/fs/ioctl.c
> > > +++ b/fs/ioctl.c
> > > @@ -868,9 +868,11 @@ static int do_vfs_ioctl(struct file *filp, unsigned int fd,
> > >         case FS_IOC_SETFLAGS:
> > >                 return ioctl_setflags(filp, argp);
> > >
> > > +       case FS_IOC_GETFSXATTR2:
> > >         case FS_IOC_FSGETXATTR:
> > >                 return ioctl_fsgetxattr(filp, argp);
> > >
> > > +       case FS_IOC_SETFSXATTR2:
> > >         case FS_IOC_FSSETXATTR:
> > >                 return ioctl_fssetxattr(filp, argp);
> > > --- a/include/uapi/linux/fs.h
> > > +++ b/include/uapi/linux/fs.h
> > > @@ -145,7 +145,8 @@ struct fsxattr {
> > >         __u32           fsx_nextents;   /* nextents field value (get)   */
> > >         __u32           fsx_projid;     /* project identifier (get/set) */
> > >         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> > > -       unsigned char   fsx_pad[8];
> > > +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> > > +       __u32           fsx_dosattrib_mask; /* dosattrib field validity mask */
> > >  };
> > >
> > >  /*
> > > @@ -238,6 +248,9 @@ struct fsxattr {
> > >  #define FS_IOC32_SETFLAGS              _IOW('f', 2, int)
> > >  #define FS_IOC32_GETVERSION            _IOR('v', 1, int)
> > >  #define FS_IOC32_SETVERSION            _IOW('v', 2, int)
> > > +#define FS_IOC_GETFSXATTR2              _IOR('x', 31, struct fsxattr)
> > > +#define FS_IOC_SETFSXATTR2              _IOW('x', 32, struct fsxattr)
> > > +/* Duplicate legacy ioctl numbers for backward compact */
> > >  #define FS_IOC_FSGETXATTR              _IOR('X', 31, struct fsxattr)
> > >  #define FS_IOC_FSSETXATTR              _IOW('X', 32, struct fsxattr)
> > >  #define FS_IOC_GETFSLABEL              _IOR(0x94, 49, char[FSLABEL_MAX])
> > >
> > > We could also use this opportunity to define a larger fsxattr2 struct
> > > that also includes an fsx_xflags_mask field, so that the xflags namespace
> > > could also be extended in a backward compat way going forward:
> > >
> > > @@ -145,7 +145,21 @@ struct fsxattr {
> > >         __u32           fsx_nextents;   /* nextents field value (get)   */
> > >         __u32           fsx_projid;     /* project identifier (get/set) */
> > >         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> > >         unsigned char   fsx_pad[8];
> > >
> > >  };
> > > +
> > > +/*
> > > + * Structure for FS_IOC_[GS]ETFSXATTR2.
> > > + */
> > > +struct fsxattr2 {
> > > +       __u32           fsx_xflags;     /* xflags field value (get/set) */
> > > +       __u32           fsx_extsize;    /* extsize field value (get/set)*/
> > > +       __u32           fsx_nextents;   /* nextents field value (get)   */
> > > +       __u32           fsx_projid;     /* project identifier (get/set) */
> > > +       __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> > > +       __u32           fsx_xflags_mask; /* xflags field validity mask */
> > > +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> > > +       __u32           fsx_dosattrib_mask; /* dosattrib field validity mask */
> > > +};
> > >
> > > And you'd also need to flug those new mask and dosattrib
> > > via struct fileattr into filesystems - too much to explain.
> > > try to figure it out (unless someone objects) and if you can't figure
> > > it out let me know.
> >
> > Yea, I think that this is thing which I should be able to figure out
> > once I start changing it.
> >
> > Anyway, I have alternative idea to the problem with fsx_pad. What about
> > introducing new fsx_xflags flag which would say that fsx_pad=fsx_dosattrib
> > is present? E.g.
> >
> > #define FS_XFLAG_HASDOSATTRIB 0x40000000
> >
> > Then we would not need new FS_IOC_GETFSXATTR2/FS_IOC_SETFSXATTR2 ioctls.
> >
> > Also fsx_pad has 8 bytes, which can store both attrib and mask, so new
> > struct fsxattr2 would not be needed too.
>
> In case we decide to not do 1-to-1 mapping of Windows FILE_ATTRIBUTE_*

The 1-to-1 is definitely not a requirement of the API.

> constants to these new Linux DOSATTRIB_* constants then we do not need
> 32-bit variable, but just 16-bit variable (I counted that we need just
> 10 bits for now).

The "for now" part is what worries me in this sentence.

> fsx_pad has currently 64 bits and we could use it for:
> - 32 bits for fsx_xflags_mask
> - 16 bits for fsx_dosattrib
> - 16 bits for fsx_dosattrib_mask

This is possible.

> And therefore no need for FS_IOC_GETFSXATTR2/FS_IOC_SETFSXATTR2 iocl or
> struct fsxattr2. Just an idea how to simplify new API. Maybe together

no need for struct fsxattr2.
but regarding no new ioctl I am not so sure.

> with new fsx_xflags bit to indicate that fsx_xflags_mask field is present:
> #define FS_XFLAG_HASXFLAGSMASK 0x20000000
>

I don't think that this flag is needed because there is no filesystem
with empty xflags_mask, so any non zero value of xflags_mask
is equivalent to setting this flag.

This is for the get side. For the set side things are more complicated.
A proper backward compat API should reject (-EINVAL) unknown flags.
As far as I can tell ioctl_fssetxattr() does not at any time verify that
fsx_pad is zero and as far as I can tell xfs_fileattr_set() does not
check for unsupported fsx_xflags.

So unless I am missing something, FS_IOC_FSSETXATTR will silently
ignore dosattrib flags (and mask) when called on an old kernel.
This is the justification of FS_IOC_SETFSXATTR2 - it will fail on an
old kernel, so application can trust that if it works - it also respects
dosattib and the masks.

Yes, applications can call FS_IOC_FSGETXATTR, check non zero
mask and deduce that FS_IOC_FSSETXATTR will respect the mask
This will probably work, but it is not a very clean API IMO.

Thanks,
Amir.

Re: Immutable vs read-only for Windows compatibility

[Amir Goldstein]

On Wed, Feb 5, 2025 at 7:16 PM Pali Rohár <pali@kernel.org> wrote:
>
> On Wednesday 05 February 2025 17:33:30 Amir Goldstein wrote:
> > On Tue, Feb 4, 2025 at 10:26 PM Pali Rohár <pali@kernel.org> wrote:
> > >
> > > On Tuesday 04 February 2025 12:54:01 Amir Goldstein wrote:
> > > > On Tue, Feb 4, 2025 at 12:34 AM Pali Rohár <pali@kernel.org> wrote:
> > > > >
> > > > > On Tuesday 04 February 2025 00:02:44 Amir Goldstein wrote:
> > > > > > On Mon, Feb 3, 2025 at 11:20 PM Pali Rohár <pali@kernel.org> wrote:
> > > > > > >
> > > > > > > On Monday 03 February 2025 22:59:46 Amir Goldstein wrote:
> > > > > > > > On Sun, Feb 2, 2025 at 4:23 PM Pali Rohár <pali@kernel.org> wrote:
> > > > > > > > > And there is still unresolved issue with FILE_ATTRIBUTE_READONLY.
> > > > > > > > > Its meaning is similar to existing Linux FS_IMMUTABLE_FL, just
> > > > > > > > > FILE_ATTRIBUTE_READONLY does not require root / CAP_LINUX_IMMUTABLE.
> > > > > > > > >
> > > > > > > > > I think that for proper support, to enforce FILE_ATTRIBUTE_READONLY
> > > > > > > > > functionality, it is needed to introduce new flag e.g.
> > > > > > > > > FS_IMMUTABLE_FL_USER to allow setting / clearing it also for normal
> > > > > > > > > users without CAP_LINUX_IMMUTABLE. Otherwise it would be unsuitable for
> > > > > > > > > any SMB client, SMB server or any application which would like to use
> > > > > > > > > it, for example wine.
> > > > > > > > >
> > > > > > > > > Just to note that FreeBSD has two immutable flags SF_IMMUTABLE and
> > > > > > > > > UF_IMMUTABLE, one settable only by superuser and second for owner.
> > > > > > > > >
> > > > > > > > > Any opinion?
> > > > > > > >
> > > > > > > > For filesystems that already support FILE_ATTRIBUTE_READONLY,
> > > > > > > > can't you just set S_IMMUTABLE on the inode and vfs will do the correct
> > > > > > > > enforcement?
> > > > > > > >
> > > > > > > > The vfs does not control if and how S_IMMUTABLE is set by filesystems,
> > > > > > > > so if you want to remove this vfs flag without CAP_LINUX_IMMUTABLE
> > > > > > > > in smb client, there is nothing stopping you (I think).
> > > > > > >
> > > > > > > Function fileattr_set_prepare() checks for CAP_LINUX_IMMUTABLE when
> > > > > > > trying to change FS_IMMUTABLE_FL bit. This function is called from
> > > > > > > ioctl(FS_IOC_SETFLAGS) and also from ioctl(FS_IOC_FSSETXATTR).
> > > > > > > And when function fileattr_set_prepare() fails then .fileattr_set
> > > > > > > callback is not called at all. So I think that it is not possible to
> > > > > > > remove the IMMUTABLE flag from userspace without capability for smb
> > > > > > > client.
> > > > > > >
> > > > > >
> > > > > > You did not understand what I meant.
> > > > > >
> > > > > > You cannot relax the CAP_LINUX_IMMUTABLE for setting FS_IMMUTABLE_FL
> > > > > > and there is no reason that you will need to relax it.
> > > > > >
> > > > > > The vfs does NOT enforce permissions according to FS_IMMUTABLE_FL
> > > > > > The vfs enforces permissions according to the S_IMMUTABLE in-memory
> > > > > > inode flag.
> > > > > >
> > > > > > There is no generic vfs code that sets S_IMMUTABLE inode flags, its
> > > > > > the filesystems that translate the on-disk FS_IMMUTABLE_FL to
> > > > > > in-memory S_IMMUTABLE inode flag.
> > > > > >
> > > > > > So if a filesystem already has an internal DOSATTRIB flags set, this
> > > > > > filesystem can set the in-memory S_IMMUTABLE inode flag according
> > > > > > to its knowledge of the DOSATTRIB_READONLY flag and the
> > > > > > CAP_LINUX_IMMUTABLE rules do not apply to the DOSATTRIB_READONLY
> > > > > > flag, which is NOT the same as the FS_IMMUTABLE_FL flag.
> > > > > >
> > > > > > > And it would not solve this problem for local filesystems (ntfs or ext4)
> > > > > > > when Samba server or wine would want to set this bit.
> > > > > > >
> > > > > >
> > > > > > The Samba server would use the FS_IOC_FS[GS]ETXATTR ioctl
> > > > > > API to get/set dosattrib, something like this:
> > > > > >
> > > > > > struct fsxattr fsxattr;
> > > > > > ret = ioctl_get_fsxattr(fd, &fsxattr);
> > > > > > if (!ret && fsxattr.fsx_xflags & FS_XFLAG_HASDOSATTR) {
> > > > > >     fsxattr.fsx_dosattr |= fs_dosattrib_readonly;
> > > > > >     ret = ioctl_set_fsxattr(fd, &fsxattr);
> > > > > > }
> > > > >
> > > > > Thanks for more explanation. First time I really did not understood it.
> > > > > But now I think I understood it. So basically there would be two flags
> > > > > which would result in setting S_IMMUTABLE on inode. One is the existing
> > > > > FS_IMMUTABLE_FL which requires the capability and some new flag (e.g.
> > > > > FS_XFLAG_HASDOSATTR) which would not require it and can be implemented
> > > > > for cifs, vfat, ntfs, ... Right?
> > > > >
> > > >
> > > > Well, almost right.
> > > > The flag that would correspond to FILE_ATTRIBUTE_READONLY
> > > > is FS_DOSATTRIB_READONLY from the new field fsx_dosattrib
> > > > (see below)
> > >
> > > Thank you for example, it is for sure good starting point for me.
> > >
> > > > --- a/include/uapi/linux/fs.h
> > > > +++ b/include/uapi/linux/fs.h
> > > > @@ -145,7 +145,8 @@ struct fsxattr {
> > > >         __u32           fsx_nextents;   /* nextents field value (get)   */
> > > >         __u32           fsx_projid;     /* project identifier (get/set) */
> > > >         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> > > > -       unsigned char   fsx_pad[8];
> > > > +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> > > > +       unsigned char   fsx_pad[4];
> > > >  };
> > > >
> > > >  /*
> > > > @@ -167,7 +168,16 @@ struct fsxattr {
> > > >  #define FS_XFLAG_FILESTREAM    0x00004000      /* use filestream allocator */
> > > >  #define FS_XFLAG_DAX           0x00008000      /* use DAX for IO */
> > > >  #define FS_XFLAG_COWEXTSIZE    0x00010000      /* CoW extent size
> > > > allocator hint */
> > > > -#define FS_XFLAG_HASATTR       0x80000000      /* no DIFLAG for this   */
> > > > +#define FS_XFLAG_HASATTR       0x80000000      /* has extended attributes */
> > > > +
> > > > +/*
> > > > + * Flags for the fsx_dosattrib field
> > > > + */
> > > > +#define FS_DOSATTRIB_READONLY  0x00000001      /* R - read-only */
> > > > +#define FS_DOSATTRIB_HIDDEN    0x00000002      /* H - hidden */
> > > > +#define FS_DOSATTRIB_SYSTEM    0x00000004      /* S - system */
> > > > +#define FS_DOSATTRIB_ARCHIVE   0x00000020      /* A - archive */
> > > > +#define FS_DOSATTRIB_HASATTR   0x80000000      /* has dos attributes */
> > >
> > > Should these FS_DOSATTRIB_* constants follows the Windows
> > > FILE_ATTRIBUTE_* constants? Because I see that you put a gap between
> > > system and archive.
> > >
> >
> > Well, no, they do not need to follow Windows contestants,
> > but then again, why not?
>
> I have just few cons:
> - there are gaps and unused bits (e.g. FILE_ATTRIBUTE_VOLUME)
> - there are bits which have more meanings (FILE_ATTRIBUTE_EA / FILE_ATTRIBUTE_RECALL_ON_OPEN)
> - there are bits used only by WinAPI and not by NT / SMB
> - there are bits which have different NT meaning in readdir() and in stat()
> - there are bits for which we already have FS_IOC_GETFLAGS API
>

valid points

> I think it that is is bad a idea if bit N in our ioctl() would have
> different meaning than in our statx().
>

definitely not, but I am not sure which bits you are referring to,

> Should we duplicate FS_IOC_GETFLAGS FS_COMPR_FL and FS_ENCRYPT_FL into
> that new fsx_dosattrib? For me it sounds like that it is a better idea
> to have compression bit and encryption bit only in one field and not
> duplicated.
>

No duplicates, of course, but notice there are no XFLAG for COMPR
and ENCRYPT.

The FS_IOC_[GS]ETFLAGS are a different alternative API.
It does not matter if you define them in fsx_xflags or on fsx_dosattrib
they will not be a duplicate of the FS_COMPR_FL flags, same as
FS_XFLAG_APPEND is not a duplicate of FS_APPEND_FL
it is a different API to control the same on-disk flag.
But you will need to include COMPRT/CRYPT in the
FS_XFLAG_COMMON masks, so the fsx_xflags probably makes
more sense.


> But we can slightly follow Windows constants. Not exactly, but remove
> redundancy, remove reserved bits, modify bits to have exactly one
> meaning, etc... So basically do some sane modifications.
>

Yep, conformity for the sake of conformity.
No hard rules.

> > I mean if we only ever needed the 4 RHSA bits above, we could
> > have used the FS_XFLAG_* flags space, but if we extend the API
> > with a new 32bit field, why not use 1-to-1 mapping at least as a starting point.
>
> I understand it, for me it looks also a good idea, just needs to resolve
> those problems above... and it would result in incompatibility that it
> would not be 1-to-1 mapping at the end.
>
> > You can see that it is quite common that filesystems re-define the
> > same constants for these flags (e.g. EXT4_IMMUTABLE_FL).
> > I am a bit surprised that there is no build time assertion
> > BUILD_BUG_ON(EXT4_IMMUTABLE_FL != FS_IMMUTABLE_FL)
> > which would be the standard way to make sure that the constants
> > stay in sync if they need to be in sync, but some filesystems don't
> > even assume that these constants are in sync (e.g. f2fs_fsflags_map)
> >
> > > > This last special flag is debatable and I am not really sure that we need it.
> > >
> > > This constant has very similar meaning to FILE_ATTRIBUTE_NORMAL. Both
> > > has some compatibility meaning that "field is valid or something is set".
> > > Just FILE_ATTRIBUTE_NORMAL is not 31th bit.
> > >
> >
> > No it does not. I don't think that you understood the meaning of
> > FS_DOSATTRIB_HASATTR.
> > Nevermind it was a bad idea anyway. see more below.
> >
> > > > It is needed for proper backward compat with existing userspace tools.
> > > > For example, if there was a backup tool storing the fsxattr blob result of
> > > > FS_IOC_FSGETXATTR and sets it later during restore with
> > > > FS_IOC_FSSETXATTR, then it would be better to ignore a zero
> > > > value of fsx_dosattrib instead of resetting all of the on-disk dosattrib flags
> > > > if the restore happens after ntfs gained support for setting dosattrib flags
> > > > via FS_IOC_FSSETXATTR.
> > > >
> > > > When using the standard tools to set fsxattr (chattr and xfs_io -c chattr)
> > > > the tool does FS_IOC_FSGETXATTR + modify + FS_IOC_FSSETXATTR,
> > > > so those tools are expected to leave new bits in fsx_dosattrib at their
> > > > value if ntfs gains support for get/set fsx_dosattrib.
> > > >
> > > > Setting the auxiliary FS_DOSATTRIB_HASATTR flag can help the
> > > > kernel/fs to explicitly state that the values returned in fsx_dosattrib
> > > > are valid and the tool to state that values set in fsx_dosattrib are valid.
> > > > But using a single flag will not help expanding ntfs support for more
> > > > fsx_dosattrib flags later, so I am not sure if it is useful (?).
> > >
> > > If the fsx_dosattrib would match all FILE_ATTRIBUTE_* then we can do it
> > > as the ntfs matches FILE_ATTRIBUTE_* and no extension is needed for
> > > future.
> > >
> > > And I think that this backward compatibility sounds good.
> > >
> >
> > That's only true if you can support ALL the dosattrib flags from the
> > first version and that Windows will not add any of the reserved
> > flags in the future, which is hard to commit to.
>
> I see, it would not work. And Windows will for sure use some reserved
> bits in future.
>
> > > What could be useful for userspace is also ability to figure out which
> > > FS_DOSATTRIB_* are supported by the filesystem. Because for example UDF
> > > on-disk format supports only FS_DOSATTRIB_HIDDEN bit. And FAT only those
> > > attributes which are in the lowest byte.
> > >
> >
> > Exactly.
> > statx has this solved with the stx_attributes_mask field.
> >
> > We could do the same for FS_IOC_FS[GS]ETXATTR, but because
> > right now, this API does not verify that fsx_pad is zero, we will need to
> > define a new set of ioctl consants FS_IOC_[GS]ETFSXATTR2
> > with the exact same functionality but that userspace knows that they
> > publish and respect the dosattrib mask:
>
> I understand and this is a problem.
>
> > --- a/fs/ioctl.c
> > +++ b/fs/ioctl.c
> > @@ -868,9 +868,11 @@ static int do_vfs_ioctl(struct file *filp, unsigned int fd,
> >         case FS_IOC_SETFLAGS:
> >                 return ioctl_setflags(filp, argp);
> >
> > +       case FS_IOC_GETFSXATTR2:
> >         case FS_IOC_FSGETXATTR:
> >                 return ioctl_fsgetxattr(filp, argp);
> >
> > +       case FS_IOC_SETFSXATTR2:
> >         case FS_IOC_FSSETXATTR:
> >                 return ioctl_fssetxattr(filp, argp);
> > --- a/include/uapi/linux/fs.h
> > +++ b/include/uapi/linux/fs.h
> > @@ -145,7 +145,8 @@ struct fsxattr {
> >         __u32           fsx_nextents;   /* nextents field value (get)   */
> >         __u32           fsx_projid;     /* project identifier (get/set) */
> >         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> > -       unsigned char   fsx_pad[8];
> > +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> > +       __u32           fsx_dosattrib_mask; /* dosattrib field validity mask */
> >  };
> >
> >  /*
> > @@ -238,6 +248,9 @@ struct fsxattr {
> >  #define FS_IOC32_SETFLAGS              _IOW('f', 2, int)
> >  #define FS_IOC32_GETVERSION            _IOR('v', 1, int)
> >  #define FS_IOC32_SETVERSION            _IOW('v', 2, int)
> > +#define FS_IOC_GETFSXATTR2              _IOR('x', 31, struct fsxattr)
> > +#define FS_IOC_SETFSXATTR2              _IOW('x', 32, struct fsxattr)
> > +/* Duplicate legacy ioctl numbers for backward compact */
> >  #define FS_IOC_FSGETXATTR              _IOR('X', 31, struct fsxattr)
> >  #define FS_IOC_FSSETXATTR              _IOW('X', 32, struct fsxattr)
> >  #define FS_IOC_GETFSLABEL              _IOR(0x94, 49, char[FSLABEL_MAX])
> >
> > We could also use this opportunity to define a larger fsxattr2 struct
> > that also includes an fsx_xflags_mask field, so that the xflags namespace
> > could also be extended in a backward compat way going forward:
> >
> > @@ -145,7 +145,21 @@ struct fsxattr {
> >         __u32           fsx_nextents;   /* nextents field value (get)   */
> >         __u32           fsx_projid;     /* project identifier (get/set) */
> >         __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> >         unsigned char   fsx_pad[8];
> >
> >  };
> > +
> > +/*
> > + * Structure for FS_IOC_[GS]ETFSXATTR2.
> > + */
> > +struct fsxattr2 {
> > +       __u32           fsx_xflags;     /* xflags field value (get/set) */
> > +       __u32           fsx_extsize;    /* extsize field value (get/set)*/
> > +       __u32           fsx_nextents;   /* nextents field value (get)   */
> > +       __u32           fsx_projid;     /* project identifier (get/set) */
> > +       __u32           fsx_cowextsize; /* CoW extsize field value (get/set)*/
> > +       __u32           fsx_xflags_mask; /* xflags field validity mask */
> > +       __u32           fsx_dosattrib;  /* dosattrib field value (get/set) */
> > +       __u32           fsx_dosattrib_mask; /* dosattrib field validity mask */
> > +};
> >
> > And you'd also need to flug those new mask and dosattrib
> > via struct fileattr into filesystems - too much to explain.
> > try to figure it out (unless someone objects) and if you can't figure
> > it out let me know.
>
> Yea, I think that this is thing which I should be able to figure out
> once I start changing it.
>
> Anyway, I have alternative idea to the problem with fsx_pad. What about
> introducing new fsx_xflags flag which would say that fsx_pad=fsx_dosattrib
> is present? E.g.
>
> #define FS_XFLAG_HASDOSATTRIB 0x40000000
>
> Then we would not need new FS_IOC_GETFSXATTR2/FS_IOC_SETFSXATTR2 ioctls.
>
> Also fsx_pad has 8 bytes, which can store both attrib and mask, so new
> struct fsxattr2 would not be needed too.
>

As I wrote in the other response, this can work, but has some pitfalls.

Thanks,
Amir.

Re: Immutable vs read-only for Windows compatibility

[Pali Rohár]

Some updates...

On Sunday 02 February 2025 16:23:43 Pali Rohár wrote:
> And how many bit flags are needed? I have done some investigation. Lets
> start with table which describes all 32 possible bit flags which are
> used by Windows system and also by filesystems FAT / exFAT / NTFS / ReFS
> and also by SMB over network:
> 
> bit / attrib.exe flag / SDK constant / description
> 
>  0 - R - FILE_ATTRIBUTE_READONLY              - writing to file or deleting it is disallowed
>  1 - H - FILE_ATTRIBUTE_HIDDEN                - inode is hidden
>  2 - S - FILE_ATTRIBUTE_SYSTEM                - inode is part of operating system
>  3 -   - FILE_ATTRIBUTE_VOLUME                - inode is the disk volume label entry
>  4 -   - FILE_ATTRIBUTE_DIRECTORY             - inode is directory
>  5 - A - FILE_ATTRIBUTE_ARCHIVE               - inode was not archived yet (when set)
>  6 -   - FILE_ATTRIBUTE_DEVICE                - inode represents  in-memory device (e.g. C:\), flag not stored on filesystem
>  7 -   - FILE_ATTRIBUTE_NORMAL                - no other flag is set (value 0 means to not change flags, bit 7 means to clear all flags)
>  8 -   - FILE_ATTRIBUTE_TEMPORARY             - inode data do not have to be flushed to disk
>  9 -   - FILE_ATTRIBUTE_SPARSE_FILE           - file is sparse with holes
> 10 -   - FILE_ATTRIBUTE_REPARSE_POINT         - inode has attached reparse point (symlink is also reparse point)
> 11 -   - FILE_ATTRIBUTE_COMPRESSED            - file is compressed, for directories it means that newly created inodes would have this flag set
> 12 - O - FILE_ATTRIBUTE_OFFLINE               - HSM - inode is used by HSM
> 13 - I - FILE_ATTRIBUTE_NOT_CONTENT_INDEXED   - inode will not be indexed by content indexing service
> 14 -   - FILE_ATTRIBUTE_ENCRYPTED             - file is encrypted, for directories it means that newly created inodes would have this flag set
> 15 - V - FILE_ATTRIBUTE_INTEGRITY_STREAM      - fs does checksumming of data and metadata when reading inode, read-only

FILE_ATTRIBUTE_INTEGRITY_STREAM can be enabled for individual inode via
FSCTL_SET_INTEGRITY_INFORMATION or FSCTL_SET_INTEGRITY_INFORMATION_EX
fs ioctl call, available on Windows and also via SMB protocol. So
de-facto it is read-write attribute, just over SMB requires separate
operation for changing it.

In similar way can be modified also FILE_ATTRIBUTE_COMPRESSED and
FILE_ATTRIBUTE_ENCRYPTED attributes.

> 16 -   - FILE_ATTRIBUTE_VIRTUAL               - inode is in %LocalAppData%\VirtualStore, flag not stored on filesystem
> 17 - X - FILE_ATTRIBUTE_NO_SCRUB_DATA         - do not use scrubber (proactive background data integrity scanner) on this file, for directories it means that newly created inodes would have this flag set
> 18 -   - FILE_ATTRIBUTE_EA                    - inode has xattrs, (not in readdir output, shares same bit with FILE_ATTRIBUTE_RECALL_ON_OPEN)
> 18 -   - FILE_ATTRIBUTE_RECALL_ON_OPEN        - HSM - inode is not stored locally (only in readdir output, shares same bit with FILE_ATTRIBUTE_EA)
> 19 - P - FILE_ATTRIBUTE_PINNED                - HSM - inode data content must be always stored on locally
> 20 - U - FILE_ATTRIBUTE_UNPINNED              - HSM - inode data content can be removed from local storage
> 21 -   -                                      - reserved
> 22 -   - FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS - HSM - inode data content is not stored locally
> 23 -   -                                      - reserved
> 24 -   -                                      - reserved
> 25 -   -                                      - reserved
> 26 -   -                                      - reserved
> 27 -   -                                      - reserved
> 28 -   -                                      - reserved
> 29 - B - FILE_ATTRIBUTE_STRICTLY_SEQUENTIAL   - SMR Blob, unknown meaning, read-only
> 30 -   -                                      - reserved
> 31 -   -                                      - reserved
> 
> (HSM means Hierarchical Storage Management software, which uses reparse
> points to make some remote file/folder available on the local
> filesystem, for example OneDrive or DropBox)
> 
> From above list only following bit flags are suitable for modification
> over some Linux API:
> - FILE_ATTRIBUTE_READONLY
> - FILE_ATTRIBUTE_HIDDEN
> - FILE_ATTRIBUTE_SYSTEM
> - FILE_ATTRIBUTE_ARCHIVE
> - FILE_ATTRIBUTE_TEMPORARY
> - FILE_ATTRIBUTE_COMPRESSED
> - FILE_ATTRIBUTE_OFFLINE
> - FILE_ATTRIBUTE_NOT_CONTENT_INDEXED
> - FILE_ATTRIBUTE_ENCRYPTED
> - FILE_ATTRIBUTE_NO_SCRUB_DATA
> - FILE_ATTRIBUTE_PINNED
> - FILE_ATTRIBUTE_UNPINNED

Hence this list needs to be extended by FILE_ATTRIBUTE_INTEGRITY_STREAM
attribute.

FILE_ATTRIBUTE_INTEGRITY_STREAM is interesting attribute as it allows to
enable checksumming of file content.

> And if I'm looking correctly the FILE_ATTRIBUTE_COMPRESSED can be
> already mapped to Linux FS_COMPR_FL / STATX_ATTR_COMPRESSED, which has
> same meaning. Also FILE_ATTRIBUTE_ENCRYPTED can be mapped to
> FS_ENCRYPT_FL / STATX_ATTR_ENCRYPTED. Note that these two flags cannot
> be set over WinAPI or SMB directly and it is required to use special
> WinAPI or SMB ioctl.
> 
> So totally are needed 10 new bit flags. And for future there are 9
> reserved bits which could be introduced by MS in future.
> 
> Additionally there are get-only attributes which can be useful for statx
> purposes (for example exported by cifs.ko SMB client):
> - FILE_ATTRIBUTE_REPARSE_POINT
> - FILE_ATTRIBUTE_INTEGRITY_STREAM
> - FILE_ATTRIBUTE_RECALL_ON_OPEN
> - FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS
> - FILE_ATTRIBUTE_STRICTLY_SEQUENTIAL
> 
> From the above list of flags suitable for modification, following bit
> flags have no meaning for kernel and it is up to userspace how will use
> them. What is needed from kernel and/or filesystem driver is to preserve
> those bit flags.
> - FILE_ATTRIBUTE_HIDDEN
> - FILE_ATTRIBUTE_SYSTEM
> - FILE_ATTRIBUTE_ARCHIVE
> - FILE_ATTRIBUTE_NOT_CONTENT_INDEXED
> 
> Following are bit flags which kernel / VFS / fsdriver would have to
> handle specially, to provide enforcement or correct behavior of them:
> - FILE_ATTRIBUTE_READONLY - enforce that data modification or unlink is disallowed when set
> - FILE_ATTRIBUTE_COMPRESSED - enforce compression on filesystem when set
> - FILE_ATTRIBUTE_ENCRYPTED - enforce encryption on filesystem when set
> 
> Then there are HSM flags which for local filesystem would need some
> cooperation with userspace synchronization software. For network
> filesystems (SMB / NFS4) they need nothing special, just properly
> propagating them over network:
> - FILE_ATTRIBUTE_OFFLINE
> - FILE_ATTRIBUTE_PINNED
> - FILE_ATTRIBUTE_UNPINNED
> 
> About following 2 flags, I'm not sure if the kernel / VFS / fs driver
> has to do something or it can just store bits to fs:
> - FILE_ATTRIBUTE_TEMPORARY
> - FILE_ATTRIBUTE_NO_SCRUB_DATA