v9fs does not honor open file handles on anonymous files

v9fs does not honor open file handles on anonymous files

[Richard Yao]

[-- Attachment #1: Type: text/plain, Size: 1573 bytes --]

I have a small shell script:

#!/bin/bash
cat <<-EOF
EOF

Running this causes bash to fork via clone(), set fd=0 to point to an
empty file in /tmp, unlink it and then execve cat. Specifically,
something like this;

[pid  3699] open("/tmp/sh-thd-1388524249",
O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600) = 3
[pid  3699] open("/tmp/sh-thd-1388524249", O_RDONLY) = 4
[pid  3699] close(3)                    = 0
[pid  3699] unlink("/tmp/sh-thd-1388524249") = 0
[pid  3699] dup2(4, 0)                  = 0
[pid  3699] close(4)                    = 0
[pid  3699] execve("/bin/cat", ["cat"], [/* 22 vars */]) = 0

One of the first things that cat does is fstat fd=0. This informs cat
that the file is empty and it will quit. If /tmp is on other
filesystems, cat will fstat fd=0, receive a return code of 0 from fstat,
print nothing and then quit normally. If /tmp is on 9pfs, cat will fstat
fd=0, receive ENOENT from fstat, print `cat: -: No such file or
directory` to stderr and die.

It seems that v9fs_vfs_unlink() is killing the file while we still have
open file handles. I have confirmed that this behavior occurs on Linux
3.13.0-rc6. This breaks several things when Gentoo is on a 9p rootfs
(e.g. gcc-config, any emerge command that involves a C compiler,
etcetera) inside QEMU. I have placed /tmp and /var/tmp/portage on a
tmpfs as a hack-fix, but it would be better to get this fixed.

I doubt that I will write a patch to fix this. I am sending the details
to the list so the 9p maintainers or any other interested individual can
fix it.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Re: v9fs does not honor open file handles on anonymous files

[Dominique Martinet]

Hi,

Richard Yao wrote on Tue, Dec 31, 2013 :
> #!/bin/bash
> cat <<-EOF
> EOF
> 
> Running this causes bash to fork via clone(), set fd=0 to point to an
> empty file in /tmp, unlink it and then execve cat. Specifically,
> something like this;
> 
> [pid  3699] open("/tmp/sh-thd-1388524249",
> O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600) = 3
> [pid  3699] open("/tmp/sh-thd-1388524249", O_RDONLY) = 4
> [pid  3699] close(3)                    = 0
> [pid  3699] unlink("/tmp/sh-thd-1388524249") = 0
> [pid  3699] dup2(4, 0)                  = 0
> [pid  3699] close(4)                    = 0
> [pid  3699] execve("/bin/cat", ["cat"], [/* 22 vars */]) = 0
> 
> It seems that v9fs_vfs_unlink() is killing the file while we still have
> open file handles. I have confirmed that this behavior occurs on Linux
> 3.13.0-rc6. This breaks several things when Gentoo is on a 9p rootfs
> (e.g. gcc-config, any emerge command that involves a C compiler,
> etcetera) inside QEMU. I have placed /tmp and /var/tmp/portage on a
> tmpfs as a hack-fix, but it would be better to get this fixed.
> 
> I doubt that I will write a patch to fix this. I am sending the details
> to the list so the 9p maintainers or any other interested individual can
> fix it.

I'm not sure if it is the client's job to remember the file has been
unlinked and only really unlink it after all the file handles are closed
or if we should expect the server to do it.

It might not matter in the case of qemu acting as a 9p server, but there
are a couple of network 9p2000.L servers out there (diod[1] and
nfs-ganesha[2]), and if the file is open on one client and unlinked on
another client.. How can the second client wait properly?

For what's it's worth, nfs-ganesha already behaves properly and this
will work with /tmp being a 9P mount off it.


It might be worth looking into qemu's code and see if it wouldn't be
easy to hold the unlink ? I've got to admit I honestly have no clue
there.
(or at least send them a copy of your mail :))


[1] diod "Distributed I/O daemon" - http://code.google.com/p/diod/
[2] nfs-ganesha - https://github.com/nfs-ganesha/nfs-ganesha


Cheers,
-- 
Dominique Martinet

Re: v9fs does not honor open file handles on anonymous files

[Richard Yao]

[-- Attachment #1: Type: text/plain, Size: 3100 bytes --]

On 12/31/2013 02:41 PM, Dominique Martinet wrote:
> Hi,

Thanks for the fast reply.

> Richard Yao wrote on Tue, Dec 31, 2013 :
>> #!/bin/bash
>> cat <<-EOF
>> EOF
>>
>> Running this causes bash to fork via clone(), set fd=0 to point to an
>> empty file in /tmp, unlink it and then execve cat. Specifically,
>> something like this;
>>
>> [pid  3699] open("/tmp/sh-thd-1388524249",
>> O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600) = 3
>> [pid  3699] open("/tmp/sh-thd-1388524249", O_RDONLY) = 4
>> [pid  3699] close(3)                    = 0
>> [pid  3699] unlink("/tmp/sh-thd-1388524249") = 0
>> [pid  3699] dup2(4, 0)                  = 0
>> [pid  3699] close(4)                    = 0
>> [pid  3699] execve("/bin/cat", ["cat"], [/* 22 vars */]) = 0
>>
>> It seems that v9fs_vfs_unlink() is killing the file while we still have
>> open file handles. I have confirmed that this behavior occurs on Linux
>> 3.13.0-rc6. This breaks several things when Gentoo is on a 9p rootfs
>> (e.g. gcc-config, any emerge command that involves a C compiler,
>> etcetera) inside QEMU. I have placed /tmp and /var/tmp/portage on a
>> tmpfs as a hack-fix, but it would be better to get this fixed.
>>
>> I doubt that I will write a patch to fix this. I am sending the details
>> to the list so the 9p maintainers or any other interested individual can
>> fix it.
> 
> I'm not sure if it is the client's job to remember the file has been
> unlinked and only really unlink it after all the file handles are closed
> or if we should expect the server to do it.

The answer to this is largely philosophical in nature, but it is a good
question to ask. Quite honestly, I have no preference, but I was not
sure who was responsible. Having caught another bug that I thought was
in QEMU that turned out to be in Linux, I decided to ping the Linux
folks first this time.

Speaking of which, that patch did not get much attention because I did
not send it to the right lists. I will resend it soon:

http://www.spinics.net/lists/linux-virtualization/msg21716.html

> It might not matter in the case of qemu acting as a 9p server, but there
> are a couple of network 9p2000.L servers out there (diod[1] and
> nfs-ganesha[2]), and if the file is open on one client and unlinked on
> another client.. How can the second client wait properly?
> 
> For what's it's worth, nfs-ganesha already behaves properly and this
> will work with /tmp being a 9P mount off it.
> 
> 
> It might be worth looking into qemu's code and see if it wouldn't be
> easy to hold the unlink ? I've got to admit I honestly have no clue
> there.
> (or at least send them a copy of your mail :))


If nfs-ganesha handles this properly, then I am going to say this is
QEMU's bug. I had CCed Aneesh, who I am told is responsible for the QEMU
9p code, so hopefully we will hear from him soon. Otherwise, I will
resend to the QEMU list.

> 
> [1] diod "Distributed I/O daemon" - http://code.google.com/p/diod/
> [2] nfs-ganesha - https://github.com/nfs-ganesha/nfs-ganesha
> 
> 
> Cheers,
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]