Linux CIFS filesystem development
 help / color / mirror / Atom feed
* SMB2 DELETE vs UNLINK
@ 2024-10-06 10:31 Pali Rohár
  2024-10-07  4:18 ` Steve French
                   ` (2 more replies)
  0 siblings, 3 replies; 23+ messages in thread
From: Pali Rohár @ 2024-10-06 10:31 UTC (permalink / raw)
  To: Steve French, Paulo Alcantara, Namjae Jeon; +Cc: linux-cifs

Hello,

Windows NT systems and SMB2 protocol support only DELETE operation which
unlinks file from the directory after the last client/process closes the
opened handle.

So when file is opened by more client/processes and somebody wants to
unlink that file, it stay in the directory until the last client/process
stop using it.

This DELETE operation can be issued either by CLOSE request on handle
opened by DELETE_ON_CLOSE flag, or by SET_INFO request with class 13
(FileDispositionInformation) and with set DeletePending flag.


But starting with Windows 10, version 1709, there is support also for
UNLINK operation, via class 64 (FileDispositionInformationEx) [1] where
is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does UNLINK after
CLOSE and let file content usable for all other processes. Internally
Windows NT kernel moves this file on NTFS from its directory into some
hidden are. Which is de-facto same as what is POSIX unlink. There is
also class 65 (FileRenameInformationEx) which is allows to issue POSIX
rename (unlink the target if it exists).

What do you think about using & implementing this functionality for the
Linux unlink operation? As the class numbers are already reserved and
documented, I think that it could make sense to use them also over SMB
on POSIX systems.


Also there is another flag FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE
which can be useful for unlink. It allows to unlink also file which has
read-only attribute set. So no need to do that racy (unset-readonly,
set-delete-pending, set-read-only) compound on files with more file
hardlinks.

I think that this is something which SMB3 POSIX extensions can use and
do not have to invent new extensions for the same functionality.


[1] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ne-wdm-_file_information_class
[2] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntddk/ns-ntddk-_file_disposition_information_ex
[3] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_rename_information

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-10-06 10:31 SMB2 DELETE vs UNLINK Pali Rohár
@ 2024-10-07  4:18 ` Steve French
  2024-10-07 18:48   ` Pali Rohár
  2024-10-08  9:40 ` Ralph Boehme
  2024-12-25 14:47 ` Pali Rohár
  2 siblings, 1 reply; 23+ messages in thread
From: Steve French @ 2024-10-07  4:18 UTC (permalink / raw)
  To: Pali Rohár; +Cc: Steve French, Paulo Alcantara, Namjae Jeon, linux-cifs

On Sun, Oct 6, 2024 at 5:31 AM Pali Rohár <pali@kernel.org> wrote:
>
> Hello,
>
> Windows NT systems and SMB2 protocol support only DELETE operation which
> unlinks file from the directory after the last client/process closes the
> opened handle.
>
> So when file is opened by more client/processes and somebody wants to
> unlink that file, it stay in the directory until the last client/process
> stop using it.
>
> This DELETE operation can be issued either by CLOSE request on handle
> opened by DELETE_ON_CLOSE flag, or by SET_INFO request with class 13
> (FileDispositionInformation) and with set DeletePending flag.
>
>
> But starting with Windows 10, version 1709, there is support also for
> UNLINK operation, via class 64 (FileDispositionInformationEx) [1] where
> is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does UNLINK after
> CLOSE and let file content usable for all other processes. Internally
> Windows NT kernel moves this file on NTFS from its directory into some
> hidden are. Which is de-facto same as what is POSIX unlink. There is
> also class 65 (FileRenameInformationEx) which is allows to issue POSIX
> rename (unlink the target if it exists).
>
> What do you think about using & implementing this functionality for the
> Linux unlink operation? As the class numbers are already reserved and
> documented, I think that it could make sense to use them also over SMB
> on POSIX systems.
>
>
> Also there is another flag FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE
> which can be useful for unlink. It allows to unlink also file which has
> read-only attribute set. So no need to do that racy (unset-readonly,
> set-delete-pending, set-read-only) compound on files with more file
> hardlinks.

This is a really good point - but what about mkdir (where we have a
current bug relating to rmdir of a file after "chmod 0444 dir"


-- 
Thanks,

Steve

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-10-07  4:18 ` Steve French
@ 2024-10-07 18:48   ` Pali Rohár
  2024-10-08  0:07     ` Steve French
  0 siblings, 1 reply; 23+ messages in thread
From: Pali Rohár @ 2024-10-07 18:48 UTC (permalink / raw)
  To: Steve French; +Cc: Steve French, Paulo Alcantara, Namjae Jeon, linux-cifs

On Sunday 06 October 2024 23:18:28 Steve French wrote:
> On Sun, Oct 6, 2024 at 5:31 AM Pali Rohár <pali@kernel.org> wrote:
> >
> > Hello,
> >
> > Windows NT systems and SMB2 protocol support only DELETE operation which
> > unlinks file from the directory after the last client/process closes the
> > opened handle.
> >
> > So when file is opened by more client/processes and somebody wants to
> > unlink that file, it stay in the directory until the last client/process
> > stop using it.
> >
> > This DELETE operation can be issued either by CLOSE request on handle
> > opened by DELETE_ON_CLOSE flag, or by SET_INFO request with class 13
> > (FileDispositionInformation) and with set DeletePending flag.
> >
> >
> > But starting with Windows 10, version 1709, there is support also for
> > UNLINK operation, via class 64 (FileDispositionInformationEx) [1] where
> > is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does UNLINK after
> > CLOSE and let file content usable for all other processes. Internally
> > Windows NT kernel moves this file on NTFS from its directory into some
> > hidden are. Which is de-facto same as what is POSIX unlink. There is
> > also class 65 (FileRenameInformationEx) which is allows to issue POSIX
> > rename (unlink the target if it exists).
> >
> > What do you think about using & implementing this functionality for the
> > Linux unlink operation? As the class numbers are already reserved and
> > documented, I think that it could make sense to use them also over SMB
> > on POSIX systems.
> >
> >
> > Also there is another flag FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE
> > which can be useful for unlink. It allows to unlink also file which has
> > read-only attribute set. So no need to do that racy (unset-readonly,
> > set-delete-pending, set-read-only) compound on files with more file
> > hardlinks.
> 
> This is a really good point - but what about mkdir (where we have a
> current bug relating to rmdir of a file after "chmod 0444 dir"

I'm not sure what is doing "chmod 0444 dir". It is setting SMB/NT
read-only attribute?

If yes then FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE sounds like can
be something useful.


But anyway, I think that such bug could be fixed by sending SMB2
compound of following SMB2 commands:
* CREATE with DELETE desired access without DELETE_ON_CLOSE
* SET_INFO with clearing READ_ONLY attribute
* SET_INFO with setting DELETE_PENDING
* SET_INFO with setting READ_ONLY attribute
* CLOSE

CREATE with DELETE_ON_CLOSE fails on object with READ_ONLY attr, so
CREATE(open) has to be called without it. First SET_INFO will try to
remove the protection, to allow second SET_INFO to set DELETE_PENDING
flag. In case setting of it will fail, the third SET_INFO will restore
the protection.

Has SMB2 something like transaction support? NT kernel and its NTFS
subsystem provides transaction FS operations for applications. And I
think that Cygwin is using those FS transactions for race-free
implementation of removing file with read-only attribute.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-10-07 18:48   ` Pali Rohár
@ 2024-10-08  0:07     ` Steve French
  0 siblings, 0 replies; 23+ messages in thread
From: Steve French @ 2024-10-08  0:07 UTC (permalink / raw)
  To: Pali Rohár; +Cc: Steve French, Paulo Alcantara, Namjae Jeon, linux-cifs

On Mon, Oct 7, 2024 at 1:48 PM Pali Rohár <pali@kernel.org> wrote:
>
> On Sunday 06 October 2024 23:18:28 Steve French wrote:
> > On Sun, Oct 6, 2024 at 5:31 AM Pali Rohár <pali@kernel.org> wrote:
> > >
> > > Hello,
> > >
> > > Windows NT systems and SMB2 protocol support only DELETE operation which
> > > unlinks file from the directory after the last client/process closes the
> > > opened handle.
> > >
> > > So when file is opened by more client/processes and somebody wants to
> > > unlink that file, it stay in the directory until the last client/process
> > > stop using it.
> > >
> > > This DELETE operation can be issued either by CLOSE request on handle
> > > opened by DELETE_ON_CLOSE flag, or by SET_INFO request with class 13
> > > (FileDispositionInformation) and with set DeletePending flag.
> > >
> > >
> > > But starting with Windows 10, version 1709, there is support also for
> > > UNLINK operation, via class 64 (FileDispositionInformationEx) [1] where
> > > is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does UNLINK after
> > > CLOSE and let file content usable for all other processes. Internally
> > > Windows NT kernel moves this file on NTFS from its directory into some
> > > hidden are. Which is de-facto same as what is POSIX unlink. There is
> > > also class 65 (FileRenameInformationEx) which is allows to issue POSIX
> > > rename (unlink the target if it exists).
> > >
> > > What do you think about using & implementing this functionality for the
> > > Linux unlink operation? As the class numbers are already reserved and
> > > documented, I think that it could make sense to use them also over SMB
> > > on POSIX systems.
> > >
> > >
> > > Also there is another flag FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE
> > > which can be useful for unlink. It allows to unlink also file which has
> > > read-only attribute set. So no need to do that racy (unset-readonly,
> > > set-delete-pending, set-read-only) compound on files with more file
> > > hardlinks.
> >
> > This is a really good point - but what about mkdir (where we have a
> > current bug relating to rmdir of a file after "chmod 0444 dir"
>
> I'm not sure what is doing "chmod 0444 dir". It is setting SMB/NT
> read-only attribute?

"chmod 0444" (since that has the effect on Linux local fs of making a
file "read only) has the effect of setting the read only file
attribute for cases when ACLs are not supported (e.g. cifsacl or
modefromsid), or where POSIX/Linux extensions are not supported

-- 
Thanks,

Steve

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-10-06 10:31 SMB2 DELETE vs UNLINK Pali Rohár
  2024-10-07  4:18 ` Steve French
@ 2024-10-08  9:40 ` Ralph Boehme
  2024-10-08 18:18   ` Pali Rohár
  2024-12-25 14:47 ` Pali Rohár
  2 siblings, 1 reply; 23+ messages in thread
From: Ralph Boehme @ 2024-10-08  9:40 UTC (permalink / raw)
  To: Pali Rohár, Steve French, Paulo Alcantara, Namjae Jeon; +Cc: linux-cifs


[-- Attachment #1.1: Type: text/plain, Size: 1598 bytes --]

On 10/6/24 12:31 PM, Pali Rohár wrote:
> But starting with Windows 10, version 1709, there is support also
> for UNLINK operation, via class 64 (FileDispositionInformationEx)
> [1] where is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does
> UNLINK after CLOSE and let file content usable for all other
> processes. Internally Windows NT kernel moves this file on NTFS from
> its directory into some hidden are. Which is de-facto same as what
> is POSIX unlink. There is also class 65 (FileRenameInformationEx)
> which is allows to issue POSIX rename (unlink the target if it
> exists).

interesting. Thanks for pointing these out!

> What do you think about using & implementing this functionality for
> the Linux unlink operation? As the class numbers are already
> reserved and documented, I think that it could make sense to use
> them also over SMB on POSIX systems.

for SMB3 POSIX this will be the behaviour on POSIX handles so we don't
need an on the wire change. This is part of what will become POSIX-FSA.

> Also there is another flag
> FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE which can be useful for
> unlink. It allows to unlink also file which has read-only attribute
> set. So no need to do that racy (unset-readonly, set-delete-pending,
> set-read-only) compound on files with more file hardlinks.
> 
> I think that this is something which SMB3 POSIX extensions can use
> and do not have to invent new extensions for the same functionality.

same here (taking note to remember to add this to the POSIX-FSA and
check Samba behaviour).

-slow

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-10-08  9:40 ` Ralph Boehme
@ 2024-10-08 18:18   ` Pali Rohár
  2024-10-08 20:16     ` Ralph Boehme
  2024-10-09  5:03     ` Steve French
  0 siblings, 2 replies; 23+ messages in thread
From: Pali Rohár @ 2024-10-08 18:18 UTC (permalink / raw)
  To: Ralph Boehme; +Cc: Steve French, Paulo Alcantara, Namjae Jeon, linux-cifs

On Tuesday 08 October 2024 11:40:06 Ralph Boehme wrote:
> On 10/6/24 12:31 PM, Pali Rohár wrote:
> > But starting with Windows 10, version 1709, there is support also
> > for UNLINK operation, via class 64 (FileDispositionInformationEx)
> > [1] where is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does
> > UNLINK after CLOSE and let file content usable for all other
> > processes. Internally Windows NT kernel moves this file on NTFS from
> > its directory into some hidden are. Which is de-facto same as what
> > is POSIX unlink. There is also class 65 (FileRenameInformationEx)
> > which is allows to issue POSIX rename (unlink the target if it
> > exists).
> 
> interesting. Thanks for pointing these out!
> 
> > What do you think about using & implementing this functionality for
> > the Linux unlink operation? As the class numbers are already
> > reserved and documented, I think that it could make sense to use
> > them also over SMB on POSIX systems.
> 
> for SMB3 POSIX this will be the behaviour on POSIX handles so we don't
> need an on the wire change. This is part of what will become POSIX-FSA.
> 
> > Also there is another flag
> > FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE which can be useful for
> > unlink. It allows to unlink also file which has read-only attribute
> > set. So no need to do that racy (unset-readonly, set-delete-pending,
> > set-read-only) compound on files with more file hardlinks.
> > 
> > I think that this is something which SMB3 POSIX extensions can use
> > and do not have to invent new extensions for the same functionality.
> 
> same here (taking note to remember to add this to the POSIX-FSA and
> check Samba behaviour).
> 
> -slow

So the behavior when the POSIX extension is active would be same as if
every DELETE_ON_CLOSE and every DELETE_PENDING=true requests would set
those new NT flags FILE_DISPOSITION_POSIX_SEMANTICS and
FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE?

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-10-08 18:18   ` Pali Rohár
@ 2024-10-08 20:16     ` Ralph Boehme
  2024-10-09  5:03     ` Steve French
  1 sibling, 0 replies; 23+ messages in thread
From: Ralph Boehme @ 2024-10-08 20:16 UTC (permalink / raw)
  To: Pali Rohár; +Cc: Steve French, Paulo Alcantara, Namjae Jeon, linux-cifs


[-- Attachment #1.1: Type: text/plain, Size: 311 bytes --]

On 10/8/24 8:18 PM, Pali Rohár wrote:
> So the behavior when the POSIX extension is active would be same as if
> every DELETE_ON_CLOSE and every DELETE_PENDING=true requests would set
> those new NT flags FILE_DISPOSITION_POSIX_SEMANTICS and
> FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE?

yup.

-slow

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-10-08 18:18   ` Pali Rohár
  2024-10-08 20:16     ` Ralph Boehme
@ 2024-10-09  5:03     ` Steve French
  2024-10-14  9:49       ` Pali Rohár
  1 sibling, 1 reply; 23+ messages in thread
From: Steve French @ 2024-10-09  5:03 UTC (permalink / raw)
  To: Pali Rohár
  Cc: Ralph Boehme, Steve French, Paulo Alcantara, Namjae Jeon,
	linux-cifs

FILE_DISPOSITION_DO_NOT_DELETE  0x00000000 Specifies the system should
not delete a file.
FILE_DISPOSITION_DELETE  0x00000001 Specifies the system should delete a file.
FILE_DISPOSITION_POSIX_SEMANTICS  0x00000002 Specifies the system
should perform a POSIX-style delete.
FILE_DISPOSITION_FORCE_IMAGE_SECTION_CHECK  0x00000004 Specifies the
system should force an image section check.
FILE_DISPOSITION_ON_CLOSE  0x00000008Specifies if the system sets or
clears the on-close state.
FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE  0x00000010 Allows
read-only files to be deleted.

These are interesting flags, but are we certain they are passed
through over SMB3.1.1 by current Windows?  It is possible that they
are filtered and thus local only

Have you tried setting them remotely over an SMB3.1.1 mount to Windows server?

On Tue, Oct 8, 2024 at 1:18 PM Pali Rohár <pali@kernel.org> wrote:
>
> On Tuesday 08 October 2024 11:40:06 Ralph Boehme wrote:
> > On 10/6/24 12:31 PM, Pali Rohár wrote:
> > > But starting with Windows 10, version 1709, there is support also
> > > for UNLINK operation, via class 64 (FileDispositionInformationEx)
> > > [1] where is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does
> > > UNLINK after CLOSE and let file content usable for all other
> > > processes. Internally Windows NT kernel moves this file on NTFS from
> > > its directory into some hidden are. Which is de-facto same as what
> > > is POSIX unlink. There is also class 65 (FileRenameInformationEx)
> > > which is allows to issue POSIX rename (unlink the target if it
> > > exists).
> >
> > interesting. Thanks for pointing these out!
> >
> > > What do you think about using & implementing this functionality for
> > > the Linux unlink operation? As the class numbers are already
> > > reserved and documented, I think that it could make sense to use
> > > them also over SMB on POSIX systems.
> >
> > for SMB3 POSIX this will be the behaviour on POSIX handles so we don't
> > need an on the wire change. This is part of what will become POSIX-FSA.
> >
> > > Also there is another flag
> > > FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE which can be useful for
> > > unlink. It allows to unlink also file which has read-only attribute
> > > set. So no need to do that racy (unset-readonly, set-delete-pending,
> > > set-read-only) compound on files with more file hardlinks.
> > >
> > > I think that this is something which SMB3 POSIX extensions can use
> > > and do not have to invent new extensions for the same functionality.
> >
> > same here (taking note to remember to add this to the POSIX-FSA and
> > check Samba behaviour).
> >
> > -slow
>
> So the behavior when the POSIX extension is active would be same as if
> every DELETE_ON_CLOSE and every DELETE_PENDING=true requests would set
> those new NT flags FILE_DISPOSITION_POSIX_SEMANTICS and
> FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE?
>


-- 
Thanks,

Steve

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-10-09  5:03     ` Steve French
@ 2024-10-14  9:49       ` Pali Rohár
  2024-12-27 15:58         ` Pali Rohár
  0 siblings, 1 reply; 23+ messages in thread
From: Pali Rohár @ 2024-10-14  9:49 UTC (permalink / raw)
  To: Steve French
  Cc: Ralph Boehme, Steve French, Paulo Alcantara, Namjae Jeon,
	linux-cifs

I checked it and seems that both Windows SMB client and Windows SMB
server on Windows Server 2022 filter out this class 64.

Windows SMB client does not send request to server at all and
immediately return error to application which tried to use class 64. And
Windows SMB server returns STATUS_NOT_IMPLEMENTED, which indicates that
server recognized it, but filtered it. Older Windows servers returns
STATUS_INVALID_INFO_CLASS which indicates that they did not understand
class 64 at all.

So seems that against Windows SMB implementation, this class 64 is
unusable for now.


Anyway, can somebody ask in Microsoft if they can allow to use
class 64 (FileDispositionInformationEx) and class 65
(FileRenameInformationEx) in their SMB client and server?
This would really help with Linux/POSIX interop, which Windows already
provided for local filesystems.

Steve, are you able to ask somebody in Microsoft for this?

On Wednesday 09 October 2024 00:03:03 Steve French wrote:
> FILE_DISPOSITION_DO_NOT_DELETE  0x00000000 Specifies the system should
> not delete a file.
> FILE_DISPOSITION_DELETE  0x00000001 Specifies the system should delete a file.
> FILE_DISPOSITION_POSIX_SEMANTICS  0x00000002 Specifies the system
> should perform a POSIX-style delete.
> FILE_DISPOSITION_FORCE_IMAGE_SECTION_CHECK  0x00000004 Specifies the
> system should force an image section check.
> FILE_DISPOSITION_ON_CLOSE  0x00000008Specifies if the system sets or
> clears the on-close state.
> FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE  0x00000010 Allows
> read-only files to be deleted.
> 
> These are interesting flags, but are we certain they are passed
> through over SMB3.1.1 by current Windows?  It is possible that they
> are filtered and thus local only
> 
> Have you tried setting them remotely over an SMB3.1.1 mount to Windows server?
> 
> On Tue, Oct 8, 2024 at 1:18 PM Pali Rohár <pali@kernel.org> wrote:
> >
> > On Tuesday 08 October 2024 11:40:06 Ralph Boehme wrote:
> > > On 10/6/24 12:31 PM, Pali Rohár wrote:
> > > > But starting with Windows 10, version 1709, there is support also
> > > > for UNLINK operation, via class 64 (FileDispositionInformationEx)
> > > > [1] where is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does
> > > > UNLINK after CLOSE and let file content usable for all other
> > > > processes. Internally Windows NT kernel moves this file on NTFS from
> > > > its directory into some hidden are. Which is de-facto same as what
> > > > is POSIX unlink. There is also class 65 (FileRenameInformationEx)
> > > > which is allows to issue POSIX rename (unlink the target if it
> > > > exists).
> > >
> > > interesting. Thanks for pointing these out!
> > >
> > > > What do you think about using & implementing this functionality for
> > > > the Linux unlink operation? As the class numbers are already
> > > > reserved and documented, I think that it could make sense to use
> > > > them also over SMB on POSIX systems.
> > >
> > > for SMB3 POSIX this will be the behaviour on POSIX handles so we don't
> > > need an on the wire change. This is part of what will become POSIX-FSA.
> > >
> > > > Also there is another flag
> > > > FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE which can be useful for
> > > > unlink. It allows to unlink also file which has read-only attribute
> > > > set. So no need to do that racy (unset-readonly, set-delete-pending,
> > > > set-read-only) compound on files with more file hardlinks.
> > > >
> > > > I think that this is something which SMB3 POSIX extensions can use
> > > > and do not have to invent new extensions for the same functionality.
> > >
> > > same here (taking note to remember to add this to the POSIX-FSA and
> > > check Samba behaviour).
> > >
> > > -slow
> >
> > So the behavior when the POSIX extension is active would be same as if
> > every DELETE_ON_CLOSE and every DELETE_PENDING=true requests would set
> > those new NT flags FILE_DISPOSITION_POSIX_SEMANTICS and
> > FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE?
> >
> 
> 
> -- 
> Thanks,
> 
> Steve

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-10-06 10:31 SMB2 DELETE vs UNLINK Pali Rohár
  2024-10-07  4:18 ` Steve French
  2024-10-08  9:40 ` Ralph Boehme
@ 2024-12-25 14:47 ` Pali Rohár
  2024-12-27 16:21   ` Tom Talpey
  2 siblings, 1 reply; 23+ messages in thread
From: Pali Rohár @ 2024-12-25 14:47 UTC (permalink / raw)
  To: Steve French, Paulo Alcantara, Namjae Jeon; +Cc: linux-cifs

On Sunday 06 October 2024 12:31:27 Pali Rohár wrote:
> Hello,
> 
> Windows NT systems and SMB2 protocol support only DELETE operation which
> unlinks file from the directory after the last client/process closes the
> opened handle.
> 
> So when file is opened by more client/processes and somebody wants to
> unlink that file, it stay in the directory until the last client/process
> stop using it.
> 
> This DELETE operation can be issued either by CLOSE request on handle
> opened by DELETE_ON_CLOSE flag, or by SET_INFO request with class 13
> (FileDispositionInformation) and with set DeletePending flag.
> 
> 
> But starting with Windows 10, version 1709, there is support also for
> UNLINK operation, via class 64 (FileDispositionInformationEx) [1] where
> is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does UNLINK after
> CLOSE and let file content usable for all other processes. Internally
> Windows NT kernel moves this file on NTFS from its directory into some
> hidden are. Which is de-facto same as what is POSIX unlink. There is
> also class 65 (FileRenameInformationEx) which is allows to issue POSIX
> rename (unlink the target if it exists).
> 
> What do you think about using & implementing this functionality for the
> Linux unlink operation? As the class numbers are already reserved and
> documented, I think that it could make sense to use them also over SMB
> on POSIX systems.
> 
> 
> Also there is another flag FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE
> which can be useful for unlink. It allows to unlink also file which has
> read-only attribute set. So no need to do that racy (unset-readonly,
> set-delete-pending, set-read-only) compound on files with more file
> hardlinks.
> 
> I think that this is something which SMB3 POSIX extensions can use and
> do not have to invent new extensions for the same functionality.
> 
> 
> [1] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ne-wdm-_file_information_class
> [2] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntddk/ns-ntddk-_file_disposition_information_ex
> [3] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_rename_information

And now I figured out that struct FILE_FS_ATTRIBUTE_INFORMATION which
has member FileSystemAttributes contains new documented bit:

0x00000400 - FILE_SUPPORTS_POSIX_UNLINK_RENAME
The file system supports POSIX-style delete and rename operations.

See Windows NT spec:
https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_fs_attribute_information

Interesting is that this struct FILE_FS_ATTRIBUTE_INFORMATION is
available over SMB protocol too but bit value 0x00000400 is not
documented in [MS-FSCC] section 2.5.1 FileFsAttributeInformation:
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ebc7e6e5-4650-4e54-b17c-cf60f6fbeeaa

So it really looks like that POSIX unlink is prepared for SMB, just is
not documented or implemented in Windows yet.

Maybe somebody could ask Microsoft documentation team for more details?

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-10-14  9:49       ` Pali Rohár
@ 2024-12-27 15:58         ` Pali Rohár
  2024-12-27 16:30           ` Tom Talpey
  0 siblings, 1 reply; 23+ messages in thread
From: Pali Rohár @ 2024-12-27 15:58 UTC (permalink / raw)
  To: Steve French
  Cc: Ralph Boehme, Steve French, Paulo Alcantara, Namjae Jeon,
	linux-cifs

Hello, I have redone these tests again. And the results are that
Windows SMB3.1.1 server really filter out this class 64 and does not
allow to use it.

But Windows SMB1 server does not filter this class 64 when sent over
SMB PASSTHROUGH (sent as level 0x03e8+64, see [MS-SMB] 2.2.2.3.5), and
normally execute it.

I have checked that over first SMB connection I opened the file, then
over second SMB connection I sent that UNLINK command and check that
over second SMB connection the file is not available in the directory
listing anymore, and at the same time over first SMB connection it was
possible to use file handle (of that opened file) for other operations.

So it looks like that at least SMB1 can benefit of this POSIX UNLINK
support. I have also checked that if I used first connection over
SMB3.1.1 then it still worked correctly (opened file handle was usable
and the file was not in directory listing anymore).

Do you know if SMB3.1.1 has something like [MS-SMB] 2.2.2.3.5
Pass-through Information Level Codes? If yes then it could be
implemented also for SMB3.1.1.

On Monday 14 October 2024 11:49:13 Pali Rohár wrote:
> I checked it and seems that both Windows SMB client and Windows SMB
> server on Windows Server 2022 filter out this class 64.
> 
> Windows SMB client does not send request to server at all and
> immediately return error to application which tried to use class 64. And
> Windows SMB server returns STATUS_NOT_IMPLEMENTED, which indicates that
> server recognized it, but filtered it. Older Windows servers returns
> STATUS_INVALID_INFO_CLASS which indicates that they did not understand
> class 64 at all.
> 
> So seems that against Windows SMB implementation, this class 64 is
> unusable for now.
> 
> 
> Anyway, can somebody ask in Microsoft if they can allow to use
> class 64 (FileDispositionInformationEx) and class 65
> (FileRenameInformationEx) in their SMB client and server?
> This would really help with Linux/POSIX interop, which Windows already
> provided for local filesystems.
> 
> Steve, are you able to ask somebody in Microsoft for this?
> 
> On Wednesday 09 October 2024 00:03:03 Steve French wrote:
> > FILE_DISPOSITION_DO_NOT_DELETE  0x00000000 Specifies the system should
> > not delete a file.
> > FILE_DISPOSITION_DELETE  0x00000001 Specifies the system should delete a file.
> > FILE_DISPOSITION_POSIX_SEMANTICS  0x00000002 Specifies the system
> > should perform a POSIX-style delete.
> > FILE_DISPOSITION_FORCE_IMAGE_SECTION_CHECK  0x00000004 Specifies the
> > system should force an image section check.
> > FILE_DISPOSITION_ON_CLOSE  0x00000008Specifies if the system sets or
> > clears the on-close state.
> > FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE  0x00000010 Allows
> > read-only files to be deleted.
> > 
> > These are interesting flags, but are we certain they are passed
> > through over SMB3.1.1 by current Windows?  It is possible that they
> > are filtered and thus local only
> > 
> > Have you tried setting them remotely over an SMB3.1.1 mount to Windows server?
> > 
> > On Tue, Oct 8, 2024 at 1:18 PM Pali Rohár <pali@kernel.org> wrote:
> > >
> > > On Tuesday 08 October 2024 11:40:06 Ralph Boehme wrote:
> > > > On 10/6/24 12:31 PM, Pali Rohár wrote:
> > > > > But starting with Windows 10, version 1709, there is support also
> > > > > for UNLINK operation, via class 64 (FileDispositionInformationEx)
> > > > > [1] where is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does
> > > > > UNLINK after CLOSE and let file content usable for all other
> > > > > processes. Internally Windows NT kernel moves this file on NTFS from
> > > > > its directory into some hidden are. Which is de-facto same as what
> > > > > is POSIX unlink. There is also class 65 (FileRenameInformationEx)
> > > > > which is allows to issue POSIX rename (unlink the target if it
> > > > > exists).
> > > >
> > > > interesting. Thanks for pointing these out!
> > > >
> > > > > What do you think about using & implementing this functionality for
> > > > > the Linux unlink operation? As the class numbers are already
> > > > > reserved and documented, I think that it could make sense to use
> > > > > them also over SMB on POSIX systems.
> > > >
> > > > for SMB3 POSIX this will be the behaviour on POSIX handles so we don't
> > > > need an on the wire change. This is part of what will become POSIX-FSA.
> > > >
> > > > > Also there is another flag
> > > > > FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE which can be useful for
> > > > > unlink. It allows to unlink also file which has read-only attribute
> > > > > set. So no need to do that racy (unset-readonly, set-delete-pending,
> > > > > set-read-only) compound on files with more file hardlinks.
> > > > >
> > > > > I think that this is something which SMB3 POSIX extensions can use
> > > > > and do not have to invent new extensions for the same functionality.
> > > >
> > > > same here (taking note to remember to add this to the POSIX-FSA and
> > > > check Samba behaviour).
> > > >
> > > > -slow
> > >
> > > So the behavior when the POSIX extension is active would be same as if
> > > every DELETE_ON_CLOSE and every DELETE_PENDING=true requests would set
> > > those new NT flags FILE_DISPOSITION_POSIX_SEMANTICS and
> > > FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE?
> > >
> > 
> > 
> > -- 
> > Thanks,
> > 
> > Steve

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-12-25 14:47 ` Pali Rohár
@ 2024-12-27 16:21   ` Tom Talpey
  2024-12-27 16:32     ` Pali Rohár
  0 siblings, 1 reply; 23+ messages in thread
From: Tom Talpey @ 2024-12-27 16:21 UTC (permalink / raw)
  To: Pali Rohár, Steve French, Paulo Alcantara, Namjae Jeon; +Cc: linux-cifs

On 12/25/2024 9:47 AM, Pali Rohár wrote:
> On Sunday 06 October 2024 12:31:27 Pali Rohár wrote:
>> Hello,
>>
>> Windows NT systems and SMB2 protocol support only DELETE operation which
>> unlinks file from the directory after the last client/process closes the
>> opened handle.
>>
>> So when file is opened by more client/processes and somebody wants to
>> unlink that file, it stay in the directory until the last client/process
>> stop using it.
>>
>> This DELETE operation can be issued either by CLOSE request on handle
>> opened by DELETE_ON_CLOSE flag, or by SET_INFO request with class 13
>> (FileDispositionInformation) and with set DeletePending flag.
>>
>>
>> But starting with Windows 10, version 1709, there is support also for
>> UNLINK operation, via class 64 (FileDispositionInformationEx) [1] where
>> is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does UNLINK after
>> CLOSE and let file content usable for all other processes. Internally
>> Windows NT kernel moves this file on NTFS from its directory into some
>> hidden are. Which is de-facto same as what is POSIX unlink. There is
>> also class 65 (FileRenameInformationEx) which is allows to issue POSIX
>> rename (unlink the target if it exists).
>>
>> What do you think about using & implementing this functionality for the
>> Linux unlink operation? As the class numbers are already reserved and
>> documented, I think that it could make sense to use them also over SMB
>> on POSIX systems.
>>
>>
>> Also there is another flag FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE
>> which can be useful for unlink. It allows to unlink also file which has
>> read-only attribute set. So no need to do that racy (unset-readonly,
>> set-delete-pending, set-read-only) compound on files with more file
>> hardlinks.
>>
>> I think that this is something which SMB3 POSIX extensions can use and
>> do not have to invent new extensions for the same functionality.
>>
>>
>> [1] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ne-wdm-_file_information_class
>> [2] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntddk/ns-ntddk-_file_disposition_information_ex
>> [3] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_rename_information
> 
> And now I figured out that struct FILE_FS_ATTRIBUTE_INFORMATION which
> has member FileSystemAttributes contains new documented bit:
> 
> 0x00000400 - FILE_SUPPORTS_POSIX_UNLINK_RENAME
> The file system supports POSIX-style delete and rename operations.
> 
> See Windows NT spec:
> https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_fs_attribute_information
> 
> Interesting is that this struct FILE_FS_ATTRIBUTE_INFORMATION is
> available over SMB protocol too but bit value 0x00000400 is not
> documented in [MS-FSCC] section 2.5.1 FileFsAttributeInformation:
> https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ebc7e6e5-4650-4e54-b17c-cf60f6fbeeaa
> 
> So it really looks like that POSIX unlink is prepared for SMB, just is
> not documented or implemented in Windows yet.
> 
> Maybe somebody could ask Microsoft documentation team for more details?
We absolutely should do this, if the bit is visible remotely then it's
an obvious omission. If it can be set remotely, even better.

Feel free to raise the issue yourself! Simply email "dochelp@microsoft.com".
Send as much supporting evidence as you have gathered.

Tom.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-12-27 15:58         ` Pali Rohár
@ 2024-12-27 16:30           ` Tom Talpey
  0 siblings, 0 replies; 23+ messages in thread
From: Tom Talpey @ 2024-12-27 16:30 UTC (permalink / raw)
  To: Pali Rohár, Steve French
  Cc: Ralph Boehme, Steve French, Paulo Alcantara, Namjae Jeon,
	linux-cifs

On 12/27/2024 10:58 AM, Pali Rohár wrote:
> Hello, I have redone these tests again. And the results are that
> Windows SMB3.1.1 server really filter out this class 64 and does not
> allow to use it.

Not surprising. Windows is quite strict about poking holes in the
protocol, which is something Microsoft has a statutory requirement
to adhere to.

> But Windows SMB1 server does not filter this class 64 when sent over
> SMB PASSTHROUGH (sent as level 0x03e8+64, see [MS-SMB] 2.2.2.3.5), and
> normally execute it.

This is very likely a bug and a significant security issue. However,
it will be one of many SMB1 vulnerabilities and Microsoft's "don't use
SMB1" policy will probably prevail over any fix.

> I have checked that over first SMB connection I opened the file, then
> over second SMB connection I sent that UNLINK command and check that
> over second SMB connection the file is not available in the directory
> listing anymore, and at the same time over first SMB connection it was
> possible to use file handle (of that opened file) for other operations.
> 
> So it looks like that at least SMB1 can benefit of this POSIX UNLINK
> support. I have also checked that if I used first connection over
> SMB3.1.1 then it still worked correctly (opened file handle was usable
> and the file was not in directory listing anymore).

"Improving" SMB1 support is a fool's errand and we should not pursue it.

> Do you know if SMB3.1.1 has something like [MS-SMB] 2.2.2.3.5
> Pass-through Information Level Codes? If yes then it could be
> implemented also for SMB3.1.1.

It does not. By design, since SMB2.0.

Tom.


> On Monday 14 October 2024 11:49:13 Pali Rohár wrote:
>> I checked it and seems that both Windows SMB client and Windows SMB
>> server on Windows Server 2022 filter out this class 64.
>>
>> Windows SMB client does not send request to server at all and
>> immediately return error to application which tried to use class 64. And
>> Windows SMB server returns STATUS_NOT_IMPLEMENTED, which indicates that
>> server recognized it, but filtered it. Older Windows servers returns
>> STATUS_INVALID_INFO_CLASS which indicates that they did not understand
>> class 64 at all.
>>
>> So seems that against Windows SMB implementation, this class 64 is
>> unusable for now.
>>
>>
>> Anyway, can somebody ask in Microsoft if they can allow to use
>> class 64 (FileDispositionInformationEx) and class 65
>> (FileRenameInformationEx) in their SMB client and server?
>> This would really help with Linux/POSIX interop, which Windows already
>> provided for local filesystems.
>>
>> Steve, are you able to ask somebody in Microsoft for this?
>>
>> On Wednesday 09 October 2024 00:03:03 Steve French wrote:
>>> FILE_DISPOSITION_DO_NOT_DELETE  0x00000000 Specifies the system should
>>> not delete a file.
>>> FILE_DISPOSITION_DELETE  0x00000001 Specifies the system should delete a file.
>>> FILE_DISPOSITION_POSIX_SEMANTICS  0x00000002 Specifies the system
>>> should perform a POSIX-style delete.
>>> FILE_DISPOSITION_FORCE_IMAGE_SECTION_CHECK  0x00000004 Specifies the
>>> system should force an image section check.
>>> FILE_DISPOSITION_ON_CLOSE  0x00000008Specifies if the system sets or
>>> clears the on-close state.
>>> FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE  0x00000010 Allows
>>> read-only files to be deleted.
>>>
>>> These are interesting flags, but are we certain they are passed
>>> through over SMB3.1.1 by current Windows?  It is possible that they
>>> are filtered and thus local only
>>>
>>> Have you tried setting them remotely over an SMB3.1.1 mount to Windows server?
>>>
>>> On Tue, Oct 8, 2024 at 1:18 PM Pali Rohár <pali@kernel.org> wrote:
>>>>
>>>> On Tuesday 08 October 2024 11:40:06 Ralph Boehme wrote:
>>>>> On 10/6/24 12:31 PM, Pali Rohár wrote:
>>>>>> But starting with Windows 10, version 1709, there is support also
>>>>>> for UNLINK operation, via class 64 (FileDispositionInformationEx)
>>>>>> [1] where is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does
>>>>>> UNLINK after CLOSE and let file content usable for all other
>>>>>> processes. Internally Windows NT kernel moves this file on NTFS from
>>>>>> its directory into some hidden are. Which is de-facto same as what
>>>>>> is POSIX unlink. There is also class 65 (FileRenameInformationEx)
>>>>>> which is allows to issue POSIX rename (unlink the target if it
>>>>>> exists).
>>>>>
>>>>> interesting. Thanks for pointing these out!
>>>>>
>>>>>> What do you think about using & implementing this functionality for
>>>>>> the Linux unlink operation? As the class numbers are already
>>>>>> reserved and documented, I think that it could make sense to use
>>>>>> them also over SMB on POSIX systems.
>>>>>
>>>>> for SMB3 POSIX this will be the behaviour on POSIX handles so we don't
>>>>> need an on the wire change. This is part of what will become POSIX-FSA.
>>>>>
>>>>>> Also there is another flag
>>>>>> FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE which can be useful for
>>>>>> unlink. It allows to unlink also file which has read-only attribute
>>>>>> set. So no need to do that racy (unset-readonly, set-delete-pending,
>>>>>> set-read-only) compound on files with more file hardlinks.
>>>>>>
>>>>>> I think that this is something which SMB3 POSIX extensions can use
>>>>>> and do not have to invent new extensions for the same functionality.
>>>>>
>>>>> same here (taking note to remember to add this to the POSIX-FSA and
>>>>> check Samba behaviour).
>>>>>
>>>>> -slow
>>>>
>>>> So the behavior when the POSIX extension is active would be same as if
>>>> every DELETE_ON_CLOSE and every DELETE_PENDING=true requests would set
>>>> those new NT flags FILE_DISPOSITION_POSIX_SEMANTICS and
>>>> FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE?
>>>>
>>>
>>>
>>> -- 
>>> Thanks,
>>>
>>> Steve
> 
> 


^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-12-27 16:21   ` Tom Talpey
@ 2024-12-27 16:32     ` Pali Rohár
  2024-12-27 16:43       ` Tom Talpey
  0 siblings, 1 reply; 23+ messages in thread
From: Pali Rohár @ 2024-12-27 16:32 UTC (permalink / raw)
  To: Tom Talpey; +Cc: Steve French, Paulo Alcantara, Namjae Jeon, linux-cifs

On Friday 27 December 2024 11:21:49 Tom Talpey wrote:
> On 12/25/2024 9:47 AM, Pali Rohár wrote:
> > On Sunday 06 October 2024 12:31:27 Pali Rohár wrote:
> > > Hello,
> > > 
> > > Windows NT systems and SMB2 protocol support only DELETE operation which
> > > unlinks file from the directory after the last client/process closes the
> > > opened handle.
> > > 
> > > So when file is opened by more client/processes and somebody wants to
> > > unlink that file, it stay in the directory until the last client/process
> > > stop using it.
> > > 
> > > This DELETE operation can be issued either by CLOSE request on handle
> > > opened by DELETE_ON_CLOSE flag, or by SET_INFO request with class 13
> > > (FileDispositionInformation) and with set DeletePending flag.
> > > 
> > > 
> > > But starting with Windows 10, version 1709, there is support also for
> > > UNLINK operation, via class 64 (FileDispositionInformationEx) [1] where
> > > is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does UNLINK after
> > > CLOSE and let file content usable for all other processes. Internally
> > > Windows NT kernel moves this file on NTFS from its directory into some
> > > hidden are. Which is de-facto same as what is POSIX unlink. There is
> > > also class 65 (FileRenameInformationEx) which is allows to issue POSIX
> > > rename (unlink the target if it exists).
> > > 
> > > What do you think about using & implementing this functionality for the
> > > Linux unlink operation? As the class numbers are already reserved and
> > > documented, I think that it could make sense to use them also over SMB
> > > on POSIX systems.
> > > 
> > > 
> > > Also there is another flag FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE
> > > which can be useful for unlink. It allows to unlink also file which has
> > > read-only attribute set. So no need to do that racy (unset-readonly,
> > > set-delete-pending, set-read-only) compound on files with more file
> > > hardlinks.
> > > 
> > > I think that this is something which SMB3 POSIX extensions can use and
> > > do not have to invent new extensions for the same functionality.
> > > 
> > > 
> > > [1] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ne-wdm-_file_information_class
> > > [2] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntddk/ns-ntddk-_file_disposition_information_ex
> > > [3] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_rename_information
> > 
> > And now I figured out that struct FILE_FS_ATTRIBUTE_INFORMATION which
> > has member FileSystemAttributes contains new documented bit:
> > 
> > 0x00000400 - FILE_SUPPORTS_POSIX_UNLINK_RENAME
> > The file system supports POSIX-style delete and rename operations.
> > 
> > See Windows NT spec:
> > https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_fs_attribute_information
> > 
> > Interesting is that this struct FILE_FS_ATTRIBUTE_INFORMATION is
> > available over SMB protocol too but bit value 0x00000400 is not
> > documented in [MS-FSCC] section 2.5.1 FileFsAttributeInformation:
> > https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ebc7e6e5-4650-4e54-b17c-cf60f6fbeeaa
> > 
> > So it really looks like that POSIX unlink is prepared for SMB, just is
> > not documented or implemented in Windows yet.
> > 
> > Maybe somebody could ask Microsoft documentation team for more details?
> We absolutely should do this, if the bit is visible remotely then it's
> an obvious omission. If it can be set remotely, even better.

Now I check that Windows Server 2022 via both SMB3.1.1 FileFsAttributeInformation
and via SMB1 QUERY_FS_INFO/FS_ATTRIBUTES announce the 0x00000400 bit for
FILE_SUPPORTS_POSIX_UNLINK_RENAME.

See other email in this tread, I was able to send POSIX UNLINK as
FILE_DISPOSITION_POSIX_SEMANTICS via SMB1, but not over SMB3.1.1
(but it is possible that I did it in wrong way).

> Feel free to raise the issue yourself! Simply email "dochelp@microsoft.com".
> Send as much supporting evidence as you have gathered.
> 
> Tom.

Ok. I can do it. Should I include somebody else into copy?

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-12-27 16:32     ` Pali Rohár
@ 2024-12-27 16:43       ` Tom Talpey
  2024-12-27 18:51         ` Pali Rohár
  0 siblings, 1 reply; 23+ messages in thread
From: Tom Talpey @ 2024-12-27 16:43 UTC (permalink / raw)
  To: Pali Rohár; +Cc: Steve French, Paulo Alcantara, Namjae Jeon, linux-cifs

On 12/27/2024 11:32 AM, Pali Rohár wrote:
> On Friday 27 December 2024 11:21:49 Tom Talpey wrote:
>> On 12/25/2024 9:47 AM, Pali Rohár wrote:
>>> On Sunday 06 October 2024 12:31:27 Pali Rohár wrote:
>>>> Hello,
>>>>
>>>> Windows NT systems and SMB2 protocol support only DELETE operation which
>>>> unlinks file from the directory after the last client/process closes the
>>>> opened handle.
>>>>
>>>> So when file is opened by more client/processes and somebody wants to
>>>> unlink that file, it stay in the directory until the last client/process
>>>> stop using it.
>>>>
>>>> This DELETE operation can be issued either by CLOSE request on handle
>>>> opened by DELETE_ON_CLOSE flag, or by SET_INFO request with class 13
>>>> (FileDispositionInformation) and with set DeletePending flag.
>>>>
>>>>
>>>> But starting with Windows 10, version 1709, there is support also for
>>>> UNLINK operation, via class 64 (FileDispositionInformationEx) [1] where
>>>> is FILE_DISPOSITION_POSIX_SEMANTICS flag [2] which does UNLINK after
>>>> CLOSE and let file content usable for all other processes. Internally
>>>> Windows NT kernel moves this file on NTFS from its directory into some
>>>> hidden are. Which is de-facto same as what is POSIX unlink. There is
>>>> also class 65 (FileRenameInformationEx) which is allows to issue POSIX
>>>> rename (unlink the target if it exists).
>>>>
>>>> What do you think about using & implementing this functionality for the
>>>> Linux unlink operation? As the class numbers are already reserved and
>>>> documented, I think that it could make sense to use them also over SMB
>>>> on POSIX systems.
>>>>
>>>>
>>>> Also there is another flag FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE
>>>> which can be useful for unlink. It allows to unlink also file which has
>>>> read-only attribute set. So no need to do that racy (unset-readonly,
>>>> set-delete-pending, set-read-only) compound on files with more file
>>>> hardlinks.
>>>>
>>>> I think that this is something which SMB3 POSIX extensions can use and
>>>> do not have to invent new extensions for the same functionality.
>>>>
>>>>
>>>> [1] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ne-wdm-_file_information_class
>>>> [2] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntddk/ns-ntddk-_file_disposition_information_ex
>>>> [3] - https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_rename_information
>>>
>>> And now I figured out that struct FILE_FS_ATTRIBUTE_INFORMATION which
>>> has member FileSystemAttributes contains new documented bit:
>>>
>>> 0x00000400 - FILE_SUPPORTS_POSIX_UNLINK_RENAME
>>> The file system supports POSIX-style delete and rename operations.
>>>
>>> See Windows NT spec:
>>> https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_fs_attribute_information
>>>
>>> Interesting is that this struct FILE_FS_ATTRIBUTE_INFORMATION is
>>> available over SMB protocol too but bit value 0x00000400 is not
>>> documented in [MS-FSCC] section 2.5.1 FileFsAttributeInformation:
>>> https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ebc7e6e5-4650-4e54-b17c-cf60f6fbeeaa
>>>
>>> So it really looks like that POSIX unlink is prepared for SMB, just is
>>> not documented or implemented in Windows yet.
>>>
>>> Maybe somebody could ask Microsoft documentation team for more details?
>> We absolutely should do this, if the bit is visible remotely then it's
>> an obvious omission. If it can be set remotely, even better.
> 
> Now I check that Windows Server 2022 via both SMB3.1.1 FileFsAttributeInformation
> and via SMB1 QUERY_FS_INFO/FS_ATTRIBUTES announce the 0x00000400 bit for
> FILE_SUPPORTS_POSIX_UNLINK_RENAME.
> 
> See other email in this tread, I was able to send POSIX UNLINK as
> FILE_DISPOSITION_POSIX_SEMANTICS via SMB1, but not over SMB3.1.1
> (but it is possible that I did it in wrong way).
> 
>> Feel free to raise the issue yourself! Simply email "dochelp@microsoft.com".
>> Send as much supporting evidence as you have gathered.
>>
>> Tom.
> 
> Ok. I can do it. Should I include somebody else into copy?

Sure, you may include me, tell them I sent you. :)

Tom.


^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-12-27 16:43       ` Tom Talpey
@ 2024-12-27 18:51         ` Pali Rohár
  2025-04-08 22:43           ` Pali Rohár
  0 siblings, 1 reply; 23+ messages in thread
From: Pali Rohár @ 2024-12-27 18:51 UTC (permalink / raw)
  To: Tom Talpey; +Cc: Steve French, Paulo Alcantara, Namjae Jeon, linux-cifs

On Friday 27 December 2024 11:43:58 Tom Talpey wrote:
> On 12/27/2024 11:32 AM, Pali Rohár wrote:
> > On Friday 27 December 2024 11:21:49 Tom Talpey wrote:
> > > Feel free to raise the issue yourself! Simply email "dochelp@microsoft.com".
> > > Send as much supporting evidence as you have gathered.
> > > 
> > > Tom.
> > 
> > Ok. I can do it. Should I include somebody else into copy?
> 
> Sure, you may include me, tell them I sent you. :)
> 
> Tom.
> 

Just note for others that I have already sent email to dochelp.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: SMB2 DELETE vs UNLINK
  2024-12-27 18:51         ` Pali Rohár
@ 2025-04-08 22:43           ` Pali Rohár
  2025-04-09  6:50             ` Fwd: " Ralph Boehme
  0 siblings, 1 reply; 23+ messages in thread
From: Pali Rohár @ 2025-04-08 22:43 UTC (permalink / raw)
  To: linux-cifs
  Cc: Tom Talpey, Steve French, Paulo Alcantara, Namjae Jeon,
	Ralph Boehme

On Friday 27 December 2024 19:51:30 Pali Rohár wrote:
> On Friday 27 December 2024 11:43:58 Tom Talpey wrote:
> > On 12/27/2024 11:32 AM, Pali Rohár wrote:
> > > On Friday 27 December 2024 11:21:49 Tom Talpey wrote:
> > > > Feel free to raise the issue yourself! Simply email "dochelp@microsoft.com".
> > > > Send as much supporting evidence as you have gathered.
> > > > 
> > > > Tom.
> > > 
> > > Ok. I can do it. Should I include somebody else into copy?
> > 
> > Sure, you may include me, tell them I sent you. :)
> > 
> > Tom.
> > 
> 
> Just note for others that I have already sent email to dochelp.

Hello, I have good news!

dochelp on 04/07/2025 updated MS-FSCC documentation and now it contains
the structures to issue the POSIX UNLINK and RENAME operations.

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9
MS-FSCC 7 Change Tracking

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/2e860264-018a-47b3-8555-565a13b35a45
MS-FSCC 2.4.12 FileDispositionInformationEx has FILE_DISPOSITION_POSIX_SEMANTICS

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/4217551b-d2c0-42cb-9dc1-69a716cf6d0c
MS-FSCC 2.4.43 FileRenameInformationEx has FILE_RENAME_REPLACE_IF_EXISTS + FILE_RENAME_POSIX_SEMANTICS

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ebc7e6e5-4650-4e54-b17c-cf60f6fbeeaa
MS-FSCC 2.5.1 FileFsAttributeInformation has FILE_SUPPORTS_POSIX_UNLINK_RENAME

So now both classic Windows DELETE and POSIX UNLINK is available and
documented.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Fwd: SMB2 DELETE vs UNLINK
  2025-04-08 22:43           ` Pali Rohár
@ 2025-04-09  6:50             ` Ralph Boehme
  2025-04-09 15:57               ` [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564 Michael Bowen
  0 siblings, 1 reply; 23+ messages in thread
From: Ralph Boehme @ 2025-04-09  6:50 UTC (permalink / raw)
  To: Interoperability Documentation Help
  Cc: Steve French, Tom Talpey, Pali Rohár, CIFS


[-- Attachment #1.1: Type: text/plain, Size: 2401 bytes --]

Hello dochelp,

it seems the updates for POSIX unlink and rename made it into MS-FSCC

<https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9>

but I don't see accompanying updates to MS-FSA and, if supported over 
SMB, MS-SMB2.

Is this coming? If this is supported over SMB by Windows it is not 
sufficient to have it burried in MS-FSCC. :)

Thanks!
-slow

-------- Forwarded Message --------
Subject: Re: SMB2 DELETE vs UNLINK
Date: Wed, 9 Apr 2025 00:43:09 +0200
From: Pali Rohár <pali@kernel.org>
To: linux-cifs@vger.kernel.org
CC: Tom Talpey <tom@talpey.com>, Steve French <sfrench@samba.org>, Paulo 
Alcantara <pc@manguebit.com>, Namjae Jeon <linkinjeon@kernel.org>, Ralph 
Boehme <slow@samba.org>

On Friday 27 December 2024 19:51:30 Pali Rohár wrote:
> On Friday 27 December 2024 11:43:58 Tom Talpey wrote:
> > On 12/27/2024 11:32 AM, Pali Rohár wrote:
> > > On Friday 27 December 2024 11:21:49 Tom Talpey wrote:
> > > > Feel free to raise the issue yourself! Simply email "dochelp@microsoft.com".
> > > > Send as much supporting evidence as you have gathered.
> > > > 
> > > > Tom.
> > > 
> > > Ok. I can do it. Should I include somebody else into copy?
> > 
> > Sure, you may include me, tell them I sent you. :)
> > 
> > Tom.
> > 
> 
> Just note for others that I have already sent email to dochelp.

Hello, I have good news!

dochelp on 04/07/2025 updated MS-FSCC documentation and now it contains
the structures to issue the POSIX UNLINK and RENAME operations.

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9
MS-FSCC 7 Change Tracking

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/2e860264-018a-47b3-8555-565a13b35a45
MS-FSCC 2.4.12 FileDispositionInformationEx has 
FILE_DISPOSITION_POSIX_SEMANTICS

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/4217551b-d2c0-42cb-9dc1-69a716cf6d0c
MS-FSCC 2.4.43 FileRenameInformationEx has FILE_RENAME_REPLACE_IF_EXISTS 
+ FILE_RENAME_POSIX_SEMANTICS

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ebc7e6e5-4650-4e54-b17c-cf60f6fbeeaa
MS-FSCC 2.5.1 FileFsAttributeInformation has 
FILE_SUPPORTS_POSIX_UNLINK_RENAME

So now both classic Windows DELETE and POSIX UNLINK is available and
documented.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* RE: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564
  2025-04-09  6:50             ` Fwd: " Ralph Boehme
@ 2025-04-09 15:57               ` Michael Bowen
  2025-04-10  5:57                 ` Tom Talpey
  2025-04-10 11:07                 ` Obaid Farooqi
  0 siblings, 2 replies; 23+ messages in thread
From: Michael Bowen @ 2025-04-09 15:57 UTC (permalink / raw)
  To: Ralph Boehme; +Cc: smfrench, tom, Pali Rohár, CIFS, Microsoft Support

[DocHelp to bcc]

Hi Ralph,

Thanks for your question about SMB2. We've created case number 2504090040009564 to track this issue. One of our engineers will contact you soon.

Best regards,
Michael Bowen
Microsoft Open Specifications Support

-----Original Message-----
From: Ralph Boehme <slow@samba.org> 
Sent: Tuesday, April 8, 2025 11:51 PM
To: Interoperability Documentation Help <dochelp@microsoft.com>
Cc: smfrench <smfrench@gmail.com>; tom <tom@talpey.com>; Pali Rohár <pali@kernel.org>; CIFS <linux-cifs@vger.kernel.org>
Subject: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK

Hello dochelp,

it seems the updates for POSIX unlink and rename made it into MS-FSCC

<https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9>

but I don't see accompanying updates to MS-FSA and, if supported over SMB, MS-SMB2.

Is this coming? If this is supported over SMB by Windows it is not sufficient to have it burried in MS-FSCC. :)

Thanks!
-slow

-------- Forwarded Message --------
Subject: Re: SMB2 DELETE vs UNLINK
Date: Wed, 9 Apr 2025 00:43:09 +0200
From: Pali Rohár <pali@kernel.org>
To: linux-cifs@vger.kernel.org
CC: Tom Talpey <tom@talpey.com>, Steve French <sfrench@samba.org>, Paulo Alcantara <pc@manguebit.com>, Namjae Jeon <linkinjeon@kernel.org>, Ralph Boehme <slow@samba.org>

On Friday 27 December 2024 19:51:30 Pali Rohár wrote:
> On Friday 27 December 2024 11:43:58 Tom Talpey wrote:
> > On 12/27/2024 11:32 AM, Pali Rohár wrote:
> > > On Friday 27 December 2024 11:21:49 Tom Talpey wrote:
> > > > Feel free to raise the issue yourself! Simply email "dochelp@microsoft.com".
> > > > Send as much supporting evidence as you have gathered.
> > > > 
> > > > Tom.
> > > 
> > > Ok. I can do it. Should I include somebody else into copy?
> > 
> > Sure, you may include me, tell them I sent you. :)
> > 
> > Tom.
> > 
> 
> Just note for others that I have already sent email to dochelp.

Hello, I have good news!

dochelp on 04/07/2025 updated MS-FSCC documentation and now it contains the structures to issue the POSIX UNLINK and RENAME operations.

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9
MS-FSCC 7 Change Tracking

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/2e860264-018a-47b3-8555-565a13b35a45
MS-FSCC 2.4.12 FileDispositionInformationEx has FILE_DISPOSITION_POSIX_SEMANTICS

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/4217551b-d2c0-42cb-9dc1-69a716cf6d0c
MS-FSCC 2.4.43 FileRenameInformationEx has FILE_RENAME_REPLACE_IF_EXISTS 
+ FILE_RENAME_POSIX_SEMANTICS

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ebc7e6e5-4650-4e54-b17c-cf60f6fbeeaa
MS-FSCC 2.5.1 FileFsAttributeInformation has FILE_SUPPORTS_POSIX_UNLINK_RENAME

So now both classic Windows DELETE and POSIX UNLINK is available and documented.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564
  2025-04-09 15:57               ` [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564 Michael Bowen
@ 2025-04-10  5:57                 ` Tom Talpey
  2025-04-10 11:07                 ` Obaid Farooqi
  1 sibling, 0 replies; 23+ messages in thread
From: Tom Talpey @ 2025-04-10  5:57 UTC (permalink / raw)
  To: Michael Bowen, Ralph Boehme
  Cc: smfrench, Pali Rohár, CIFS, Microsoft Support

Thank you Michael and yes, processing rules in the SMB and filesystem
components are necessary. To which I'll add, which Windows SKUs actually
perform them (behavior notes).

Tom.

On 4/9/2025 5:57 PM, Michael Bowen wrote:
> [DocHelp to bcc]
> 
> Hi Ralph,
> 
> Thanks for your question about SMB2. We've created case number 2504090040009564 to track this issue. One of our engineers will contact you soon.
> 
> Best regards,
> Michael Bowen
> Microsoft Open Specifications Support
> 
> -----Original Message-----
> From: Ralph Boehme <slow@samba.org>
> Sent: Tuesday, April 8, 2025 11:51 PM
> To: Interoperability Documentation Help <dochelp@microsoft.com>
> Cc: smfrench <smfrench@gmail.com>; tom <tom@talpey.com>; Pali Rohár <pali@kernel.org>; CIFS <linux-cifs@vger.kernel.org>
> Subject: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK
> 
> Hello dochelp,
> 
> it seems the updates for POSIX unlink and rename made it into MS-FSCC
> 
> <https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9>
> 
> but I don't see accompanying updates to MS-FSA and, if supported over SMB, MS-SMB2.
> 
> Is this coming? If this is supported over SMB by Windows it is not sufficient to have it burried in MS-FSCC. :)
> 
> Thanks!
> -slow
> 
> -------- Forwarded Message --------
> Subject: Re: SMB2 DELETE vs UNLINK
> Date: Wed, 9 Apr 2025 00:43:09 +0200
> From: Pali Rohár <pali@kernel.org>
> To: linux-cifs@vger.kernel.org
> CC: Tom Talpey <tom@talpey.com>, Steve French <sfrench@samba.org>, Paulo Alcantara <pc@manguebit.com>, Namjae Jeon <linkinjeon@kernel.org>, Ralph Boehme <slow@samba.org>
> 
> On Friday 27 December 2024 19:51:30 Pali Rohár wrote:
>> On Friday 27 December 2024 11:43:58 Tom Talpey wrote:
>>> On 12/27/2024 11:32 AM, Pali Rohár wrote:
>>>> On Friday 27 December 2024 11:21:49 Tom Talpey wrote:
>>>>> Feel free to raise the issue yourself! Simply email "dochelp@microsoft.com".
>>>>> Send as much supporting evidence as you have gathered.
>>>>>
>>>>> Tom.
>>>>
>>>> Ok. I can do it. Should I include somebody else into copy?
>>>
>>> Sure, you may include me, tell them I sent you. :)
>>>
>>> Tom.
>>>
>>
>> Just note for others that I have already sent email to dochelp.
> 
> Hello, I have good news!
> 
> dochelp on 04/07/2025 updated MS-FSCC documentation and now it contains the structures to issue the POSIX UNLINK and RENAME operations.
> 
> https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9
> MS-FSCC 7 Change Tracking
> 
> https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/2e860264-018a-47b3-8555-565a13b35a45
> MS-FSCC 2.4.12 FileDispositionInformationEx has FILE_DISPOSITION_POSIX_SEMANTICS
> 
> https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/4217551b-d2c0-42cb-9dc1-69a716cf6d0c
> MS-FSCC 2.4.43 FileRenameInformationEx has FILE_RENAME_REPLACE_IF_EXISTS
> + FILE_RENAME_POSIX_SEMANTICS
> 
> https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ebc7e6e5-4650-4e54-b17c-cf60f6fbeeaa
> MS-FSCC 2.5.1 FileFsAttributeInformation has FILE_SUPPORTS_POSIX_UNLINK_RENAME
> 
> So now both classic Windows DELETE and POSIX UNLINK is available and documented.


^ permalink raw reply	[flat|nested] 23+ messages in thread

* RE: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564
  2025-04-09 15:57               ` [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564 Michael Bowen
  2025-04-10  5:57                 ` Tom Talpey
@ 2025-04-10 11:07                 ` Obaid Farooqi
  2025-05-06 19:00                   ` Obaid Farooqi
  1 sibling, 1 reply; 23+ messages in thread
From: Obaid Farooqi @ 2025-04-10 11:07 UTC (permalink / raw)
  To: Ralph Boehme; +Cc: smfrench, tom, Pali Rohár, CIFS, Microsoft Support

Hi Ralph, Tom:
I'll help you with this issue and will be in touch as soon as I have an answer.

Regards,
Obaid Farooqi
Escalation Engineer | Microsoft
-----Original Message-----
From: Michael Bowen <Mike.Bowen@microsoft.com>
Sent: Wednesday, April 9, 2025 8:57 PM
To: Ralph Boehme <slow@samba.org>
Cc: smfrench <smfrench@gmail.com>; tom <tom@talpey.com>; Pali Rohár <pali@kernel.org>; CIFS <linux-cifs@vger.kernel.org>; Microsoft Support <supportmail@microsoft.com>
Subject: RE: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564

[DocHelp to bcc]

Hi Ralph,

Thanks for your question about SMB2. We've created case number 2504090040009564 to track this issue. One of our engineers will contact you soon.

Best regards,
Michael Bowen
Microsoft Open Specifications Support

-----Original Message-----
From: Ralph Boehme <slow@samba.org>
Sent: Tuesday, April 8, 2025 11:51 PM
To: Interoperability Documentation Help <dochelp@microsoft.com>
Cc: smfrench <smfrench@gmail.com>; tom <tom@talpey.com>; Pali Rohár <pali@kernel.org>; CIFS <linux-cifs@vger.kernel.org>
Subject: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK

Hello dochelp,

it seems the updates for POSIX unlink and rename made it into MS-FSCC

<https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9>

but I don't see accompanying updates to MS-FSA and, if supported over SMB, MS-SMB2.

Is this coming? If this is supported over SMB by Windows it is not sufficient to have it burried in MS-FSCC. :)

Thanks!
-slow

-------- Forwarded Message --------
Subject: Re: SMB2 DELETE vs UNLINK
Date: Wed, 9 Apr 2025 00:43:09 +0200
From: Pali Rohár <pali@kernel.org>
To: linux-cifs@vger.kernel.org
CC: Tom Talpey <tom@talpey.com>, Steve French <sfrench@samba.org>, Paulo Alcantara <pc@manguebit.com>, Namjae Jeon <linkinjeon@kernel.org>, Ralph Boehme <slow@samba.org>

On Friday 27 December 2024 19:51:30 Pali Rohár wrote:
> On Friday 27 December 2024 11:43:58 Tom Talpey wrote:
> > On 12/27/2024 11:32 AM, Pali Rohár wrote:
> > > On Friday 27 December 2024 11:21:49 Tom Talpey wrote:
> > > > Feel free to raise the issue yourself! Simply email "dochelp@microsoft.com".
> > > > Send as much supporting evidence as you have gathered.
> > > >
> > > > Tom.
> > >
> > > Ok. I can do it. Should I include somebody else into copy?
> >
> > Sure, you may include me, tell them I sent you. :)
> >
> > Tom.
> >
>
> Just note for others that I have already sent email to dochelp.

Hello, I have good news!

dochelp on 04/07/2025 updated MS-FSCC documentation and now it contains the structures to issue the POSIX UNLINK and RENAME operations.

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9
MS-FSCC 7 Change Tracking

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/2e860264-018a-47b3-8555-565a13b35a45
MS-FSCC 2.4.12 FileDispositionInformationEx has FILE_DISPOSITION_POSIX_SEMANTICS

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/4217551b-d2c0-42cb-9dc1-69a716cf6d0c
MS-FSCC 2.4.43 FileRenameInformationEx has FILE_RENAME_REPLACE_IF_EXISTS
+ FILE_RENAME_POSIX_SEMANTICS

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ebc7e6e5-4650-4e54-b17c-cf60f6fbeeaa
MS-FSCC 2.5.1 FileFsAttributeInformation has FILE_SUPPORTS_POSIX_UNLINK_RENAME

So now both classic Windows DELETE and POSIX UNLINK is available and documented.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* RE: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564
  2025-04-10 11:07                 ` Obaid Farooqi
@ 2025-05-06 19:00                   ` Obaid Farooqi
  2025-08-31 12:55                     ` Pali Rohár
  0 siblings, 1 reply; 23+ messages in thread
From: Obaid Farooqi @ 2025-05-06 19:00 UTC (permalink / raw)
  To: Ralph Boehme; +Cc: smfrench, tom, Pali Rohár, CIFS, Microsoft Support

Hi Ralph:
FileDispositionInformationEx, is for use locally, as documented in MS-FSCC. MS-SMB2 and MS-SMB both do not support it but Pali have been able to use MS-SMB pass-through mechanism to send this information level to the object store who acts on it and deletes the file.
MS-FSCC is already updated and you can see the changes in the current release.
MS-SMB was updated as follows:
"
<133> Section 3.3.5.10.6:
...
If Information Level is set to an undefined value (0x0428) in the request, Windows Server 2022
does not process the request but sends a success response.
"
The above statement is to document the pass through behavior that Pali was able to produce. This does not tell the whole story, so I have filed a bug to make it more complete.

MS-FSA does not mention FileDispositionInformationEx. The reason is because FileDispositionInformationEx is local. But since Pali has been able to send FileDispositionInformationEx on the wire, I have filed a bug to add a section about FileDispositionInformationEx and information about POSIX semantics.

Please let me know if this does not answer your question.

Regards,
Obaid Farooqi
Escalation Engineer | Microsoft

-----Original Message-----
From: Obaid Farooqi
Sent: Thursday, April 10, 2025 6:07 AM
To: Ralph Boehme <slow@samba.org>
Cc: smfrench <smfrench@gmail.com>; tom <tom@talpey.com>; Pali Rohár <pali@kernel.org>; CIFS <linux-cifs@vger.kernel.org>; Microsoft Support <supportmail@microsoft.com>
Subject: RE: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564

Hi Ralph, Tom:
I'll help you with this issue and will be in touch as soon as I have an answer.

Regards,
Obaid Farooqi
Escalation Engineer | Microsoft
-----Original Message-----
From: Michael Bowen <Mike.Bowen@microsoft.com>
Sent: Wednesday, April 9, 2025 8:57 PM
To: Ralph Boehme <slow@samba.org>
Cc: smfrench <smfrench@gmail.com>; tom <tom@talpey.com>; Pali Rohár <pali@kernel.org>; CIFS <linux-cifs@vger.kernel.org>; Microsoft Support <supportmail@microsoft.com>
Subject: RE: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564

[DocHelp to bcc]

Hi Ralph,

Thanks for your question about SMB2. We've created case number 2504090040009564 to track this issue. One of our engineers will contact you soon.

Best regards,
Michael Bowen
Microsoft Open Specifications Support

-----Original Message-----
From: Ralph Boehme <slow@samba.org>
Sent: Tuesday, April 8, 2025 11:51 PM
To: Interoperability Documentation Help <dochelp@microsoft.com>
Cc: smfrench <smfrench@gmail.com>; tom <tom@talpey.com>; Pali Rohár <pali@kernel.org>; CIFS <linux-cifs@vger.kernel.org>
Subject: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK

Hello dochelp,

it seems the updates for POSIX unlink and rename made it into MS-FSCC

<https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9>

but I don't see accompanying updates to MS-FSA and, if supported over SMB, MS-SMB2.

Is this coming? If this is supported over SMB by Windows it is not sufficient to have it burried in MS-FSCC. :)

Thanks!
-slow

-------- Forwarded Message --------
Subject: Re: SMB2 DELETE vs UNLINK
Date: Wed, 9 Apr 2025 00:43:09 +0200
From: Pali Rohár <pali@kernel.org>
To: linux-cifs@vger.kernel.org
CC: Tom Talpey <tom@talpey.com>, Steve French <sfrench@samba.org>, Paulo Alcantara <pc@manguebit.com>, Namjae Jeon <linkinjeon@kernel.org>, Ralph Boehme <slow@samba.org>

On Friday 27 December 2024 19:51:30 Pali Rohár wrote:
> On Friday 27 December 2024 11:43:58 Tom Talpey wrote:
> > On 12/27/2024 11:32 AM, Pali Rohár wrote:
> > > On Friday 27 December 2024 11:21:49 Tom Talpey wrote:
> > > > Feel free to raise the issue yourself! Simply email "dochelp@microsoft.com".
> > > > Send as much supporting evidence as you have gathered.
> > > >
> > > > Tom.
> > >
> > > Ok. I can do it. Should I include somebody else into copy?
> >
> > Sure, you may include me, tell them I sent you. :)
> >
> > Tom.
> >
>
> Just note for others that I have already sent email to dochelp.

Hello, I have good news!

dochelp on 04/07/2025 updated MS-FSCC documentation and now it contains the structures to issue the POSIX UNLINK and RENAME operations.

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9
MS-FSCC 7 Change Tracking

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/2e860264-018a-47b3-8555-565a13b35a45
MS-FSCC 2.4.12 FileDispositionInformationEx has FILE_DISPOSITION_POSIX_SEMANTICS

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/4217551b-d2c0-42cb-9dc1-69a716cf6d0c
MS-FSCC 2.4.43 FileRenameInformationEx has FILE_RENAME_REPLACE_IF_EXISTS
+ FILE_RENAME_POSIX_SEMANTICS

https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ebc7e6e5-4650-4e54-b17c-cf60f6fbeeaa
MS-FSCC 2.5.1 FileFsAttributeInformation has FILE_SUPPORTS_POSIX_UNLINK_RENAME

So now both classic Windows DELETE and POSIX UNLINK is available and documented.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564
  2025-05-06 19:00                   ` Obaid Farooqi
@ 2025-08-31 12:55                     ` Pali Rohár
  0 siblings, 0 replies; 23+ messages in thread
From: Pali Rohár @ 2025-08-31 12:55 UTC (permalink / raw)
  To: Ralph Boehme, smfrench, tom, CIFS

Hello Ralph,

In [MS-SMB] 54.0 from 8/11/2025 in Change Tracking section
http://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-smb/4caa15a3-700f-479f-90ce-53e2ecca945d
is now documented that TRANS2_SET_FILE_INFORMATION supports
FileDispositionInformationEx which allows to do POSIX unlink.

And now also [MS-FSA] was updated for POSIX semantics support.
In version 41.0 from 8/11/2025 is in Change Tracking section
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fsa/78f36c85-889c-4804-ab11-6af3565e01b1
documented new volatile field DeleteUsingPosixSemantics and other
sections were extended for POSIX support, including the description how
to handle FileDispositionInformationEx requests.

Pali

On Tuesday 06 May 2025 19:00:17 Obaid Farooqi wrote:
> Hi Ralph:
> FileDispositionInformationEx, is for use locally, as documented in MS-FSCC. MS-SMB2 and MS-SMB both do not support it but Pali have been able to use MS-SMB pass-through mechanism to send this information level to the object store who acts on it and deletes the file.
> MS-FSCC is already updated and you can see the changes in the current release.
> MS-SMB was updated as follows:
> "
> <133> Section 3.3.5.10.6:
> ...
> If Information Level is set to an undefined value (0x0428) in the request, Windows Server 2022
> does not process the request but sends a success response.
> "
> The above statement is to document the pass through behavior that Pali was able to produce. This does not tell the whole story, so I have filed a bug to make it more complete.
> 
> MS-FSA does not mention FileDispositionInformationEx. The reason is because FileDispositionInformationEx is local. But since Pali has been able to send FileDispositionInformationEx on the wire, I have filed a bug to add a section about FileDispositionInformationEx and information about POSIX semantics.
> 
> Please let me know if this does not answer your question.
> 
> Regards,
> Obaid Farooqi
> Escalation Engineer | Microsoft
> 
> -----Original Message-----
> From: Obaid Farooqi
> Sent: Thursday, April 10, 2025 6:07 AM
> To: Ralph Boehme <slow@samba.org>
> Cc: smfrench <smfrench@gmail.com>; tom <tom@talpey.com>; Pali Rohár <pali@kernel.org>; CIFS <linux-cifs@vger.kernel.org>; Microsoft Support <supportmail@microsoft.com>
> Subject: RE: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564
> 
> Hi Ralph, Tom:
> I'll help you with this issue and will be in touch as soon as I have an answer.
> 
> Regards,
> Obaid Farooqi
> Escalation Engineer | Microsoft
> -----Original Message-----
> From: Michael Bowen <Mike.Bowen@microsoft.com>
> Sent: Wednesday, April 9, 2025 8:57 PM
> To: Ralph Boehme <slow@samba.org>
> Cc: smfrench <smfrench@gmail.com>; tom <tom@talpey.com>; Pali Rohár <pali@kernel.org>; CIFS <linux-cifs@vger.kernel.org>; Microsoft Support <supportmail@microsoft.com>
> Subject: RE: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564
> 
> [DocHelp to bcc]
> 
> Hi Ralph,
> 
> Thanks for your question about SMB2. We've created case number 2504090040009564 to track this issue. One of our engineers will contact you soon.
> 
> Best regards,
> Michael Bowen
> Microsoft Open Specifications Support
> 
> -----Original Message-----
> From: Ralph Boehme <slow@samba.org>
> Sent: Tuesday, April 8, 2025 11:51 PM
> To: Interoperability Documentation Help <dochelp@microsoft.com>
> Cc: smfrench <smfrench@gmail.com>; tom <tom@talpey.com>; Pali Rohár <pali@kernel.org>; CIFS <linux-cifs@vger.kernel.org>
> Subject: [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK
> 
> Hello dochelp,
> 
> it seems the updates for POSIX unlink and rename made it into MS-FSCC
> 
> <https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9>
> 
> but I don't see accompanying updates to MS-FSA and, if supported over SMB, MS-SMB2.
> 
> Is this coming? If this is supported over SMB by Windows it is not sufficient to have it burried in MS-FSCC. :)
> 
> Thanks!
> -slow
> 
> -------- Forwarded Message --------
> Subject: Re: SMB2 DELETE vs UNLINK
> Date: Wed, 9 Apr 2025 00:43:09 +0200
> From: Pali Rohár <pali@kernel.org>
> To: linux-cifs@vger.kernel.org
> CC: Tom Talpey <tom@talpey.com>, Steve French <sfrench@samba.org>, Paulo Alcantara <pc@manguebit.com>, Namjae Jeon <linkinjeon@kernel.org>, Ralph Boehme <slow@samba.org>
> 
> On Friday 27 December 2024 19:51:30 Pali Rohár wrote:
> > On Friday 27 December 2024 11:43:58 Tom Talpey wrote:
> > > On 12/27/2024 11:32 AM, Pali Rohár wrote:
> > > > On Friday 27 December 2024 11:21:49 Tom Talpey wrote:
> > > > > Feel free to raise the issue yourself! Simply email "dochelp@microsoft.com".
> > > > > Send as much supporting evidence as you have gathered.
> > > > >
> > > > > Tom.
> > > >
> > > > Ok. I can do it. Should I include somebody else into copy?
> > >
> > > Sure, you may include me, tell them I sent you. :)
> > >
> > > Tom.
> > >
> >
> > Just note for others that I have already sent email to dochelp.
> 
> Hello, I have good news!
> 
> dochelp on 04/07/2025 updated MS-FSCC documentation and now it contains the structures to issue the POSIX UNLINK and RENAME operations.
> 
> https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/f1f88b22-15c6-4081-a899-788511ae2ed9
> MS-FSCC 7 Change Tracking
> 
> https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/2e860264-018a-47b3-8555-565a13b35a45
> MS-FSCC 2.4.12 FileDispositionInformationEx has FILE_DISPOSITION_POSIX_SEMANTICS
> 
> https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/4217551b-d2c0-42cb-9dc1-69a716cf6d0c
> MS-FSCC 2.4.43 FileRenameInformationEx has FILE_RENAME_REPLACE_IF_EXISTS
> + FILE_RENAME_POSIX_SEMANTICS
> 
> https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ebc7e6e5-4650-4e54-b17c-cf60f6fbeeaa
> MS-FSCC 2.5.1 FileFsAttributeInformation has FILE_SUPPORTS_POSIX_UNLINK_RENAME
> 
> So now both classic Windows DELETE and POSIX UNLINK is available and documented.

^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2025-08-31 12:55 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-10-06 10:31 SMB2 DELETE vs UNLINK Pali Rohár
2024-10-07  4:18 ` Steve French
2024-10-07 18:48   ` Pali Rohár
2024-10-08  0:07     ` Steve French
2024-10-08  9:40 ` Ralph Boehme
2024-10-08 18:18   ` Pali Rohár
2024-10-08 20:16     ` Ralph Boehme
2024-10-09  5:03     ` Steve French
2024-10-14  9:49       ` Pali Rohár
2024-12-27 15:58         ` Pali Rohár
2024-12-27 16:30           ` Tom Talpey
2024-12-25 14:47 ` Pali Rohár
2024-12-27 16:21   ` Tom Talpey
2024-12-27 16:32     ` Pali Rohár
2024-12-27 16:43       ` Tom Talpey
2024-12-27 18:51         ` Pali Rohár
2025-04-08 22:43           ` Pali Rohár
2025-04-09  6:50             ` Fwd: " Ralph Boehme
2025-04-09 15:57               ` [EXTERNAL] Fwd: SMB2 DELETE vs UNLINK - TrackingID#2504090040009564 Michael Bowen
2025-04-10  5:57                 ` Tom Talpey
2025-04-10 11:07                 ` Obaid Farooqi
2025-05-06 19:00                   ` Obaid Farooqi
2025-08-31 12:55                     ` Pali Rohár

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox