virtiofsd: sshfs as submount?

virtiofsd: sshfs as submount?

[Laszlo Ersek]

Hi Miklos,

the following 2019 presentation on Stefan's website:

  https://vmsplice.net/
  virtio-fs: A Shared File System for Virtual Machines at KVM Forum 2019
  https://vmsplice.net/~stefan/virtio-fs_%20A%20Shared%20File%20System%20for%20Virtual%20Machines.pdf

has a slide called "Use case: File system-as-a-service" (slide#4). It
seems to confirm my "grand" idea to expose an sshfs submount to the
guest, via virtiofsd. (The guest need not / should not know it's a
submount, just see the files.) Beyond the pure utility of this, it feels
exciting to chain FUSE to FUSE. :)

I've tried it; the FUSE_READDIRPLUS request fails.

[2020-12-20 00:32:08.64+0100] [ID: 00000006] unique: 83, opcode: READDIRPLUS (44), nodeid: 1, insize: 80, pid: 1
[2020-12-20 00:32:08.64+0100] [ID: 00000006]    unique: 83, error: -13 (Permission denied), outsize: 16

More precisely, it fails on the directory entry in the containing
directory that is the sshfs mount point, when listing the containing
directory.

I've skimmed the following thread:

  [PATCH] virtiofsd: Show submounts
  https://www.redhat.com/archives/virtio-fs/2020-April/msg00023.html

(which is now QEMU commit ace0829c0d08), and I vaguely suspect it should
work -- the MS_REC flag is present, and the MS_REC flag seems to be so
old that I think my host kernel (latest RHEL7) must support it too.

So... does the sshfs filesystem present itself as unshareable? Is it
supposed to work? Does it break for others too?

Thanks!
Laszlo

Re: virtiofsd: sshfs as submount?

[Max Reitz]

On 20.12.20 00:41, Laszlo Ersek wrote:
> Hi Miklos,

(I hope it’s OK for me not to be Miklos, even though I don’t have much 
to add)

> the following 2019 presentation on Stefan's website:
> 
>    https://vmsplice.net/
>    virtio-fs: A Shared File System for Virtual Machines at KVM Forum 2019
>    https://vmsplice.net/~stefan/virtio-fs_%20A%20Shared%20File%20System%20for%20Virtual%20Machines.pdf
> 
> has a slide called "Use case: File system-as-a-service" (slide#4). It
> seems to confirm my "grand" idea to expose an sshfs submount to the
> guest, via virtiofsd. (The guest need not / should not know it's a
> submount, just see the files.) Beyond the pure utility of this, it feels
> exciting to chain FUSE to FUSE. :)
> 
> I've tried it; the FUSE_READDIRPLUS request fails.
> 
> [2020-12-20 00:32:08.64+0100] [ID: 00000006] unique: 83, opcode: READDIRPLUS (44), nodeid: 1, insize: 80, pid: 1
> [2020-12-20 00:32:08.64+0100] [ID: 00000006]    unique: 83, error: -13 (Permission denied), outsize: 16
> 
> More precisely, it fails on the directory entry in the containing
> directory that is the sshfs mount point, when listing the containing
> directory.

I see the same.

> I've skimmed the following thread:
> 
>    [PATCH] virtiofsd: Show submounts
>    https://www.redhat.com/archives/virtio-fs/2020-April/msg00023.html
> 
> (which is now QEMU commit ace0829c0d08), and I vaguely suspect it should
> work -- the MS_REC flag is present, and the MS_REC flag seems to be so
> old that I think my host kernel (latest RHEL7) must support it too.

It works (for me) with other mounts (like XFS or ext4), so submounts 
shouldn’t be the problem.

> So... does the sshfs filesystem present itself as unshareable? Is it
> supposed to work? Does it break for others too?

I can share sshfs through sshfs, so it must be something virtiofs-specific.

I tried to debug it, but I could only find that the fstatat()/statx() on 
it (FD opened, then stat called with that FD, an empty pathname, and 
AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW) fails with EPERM.  I tried 
disabling all the sandboxing, but still got that error.

FWIW, I get the same error with virtiofsd-rs (and there, too, the 
fstatat64() yields the EPERM).

So far, I couldn’t reproduce it outside of virtiofsd, though...  (Like, 
just invoking stat on the command line works; and a simple program that 
opens the mount point FD and then stats it works, too.)

Max

Re: virtiofsd: sshfs as submount?

[Laszlo Ersek]

On 12/21/20 13:06, Max Reitz wrote:
> On 20.12.20 00:41, Laszlo Ersek wrote:
>> Hi Miklos,
> 
> (I hope it’s OK for me not to be Miklos, even though I don’t have much
> to add)

:)

Thank you, Max!
Laszlo

>> the following 2019 presentation on Stefan's website:
>>
>>    https://vmsplice.net/
>>    virtio-fs: A Shared File System for Virtual Machines at KVM Forum 2019
>>   
>> https://vmsplice.net/~stefan/virtio-fs_%20A%20Shared%20File%20System%20for%20Virtual%20Machines.pdf
>>
>>
>> has a slide called "Use case: File system-as-a-service" (slide#4). It
>> seems to confirm my "grand" idea to expose an sshfs submount to the
>> guest, via virtiofsd. (The guest need not / should not know it's a
>> submount, just see the files.) Beyond the pure utility of this, it feels
>> exciting to chain FUSE to FUSE. :)
>>
>> I've tried it; the FUSE_READDIRPLUS request fails.
>>
>> [2020-12-20 00:32:08.64+0100] [ID: 00000006] unique: 83, opcode:
>> READDIRPLUS (44), nodeid: 1, insize: 80, pid: 1
>> [2020-12-20 00:32:08.64+0100] [ID: 00000006]    unique: 83, error: -13
>> (Permission denied), outsize: 16
>>
>> More precisely, it fails on the directory entry in the containing
>> directory that is the sshfs mount point, when listing the containing
>> directory.
> 
> I see the same.
> 
>> I've skimmed the following thread:
>>
>>    [PATCH] virtiofsd: Show submounts
>>    https://www.redhat.com/archives/virtio-fs/2020-April/msg00023.html
>>
>> (which is now QEMU commit ace0829c0d08), and I vaguely suspect it should
>> work -- the MS_REC flag is present, and the MS_REC flag seems to be so
>> old that I think my host kernel (latest RHEL7) must support it too.
> 
> It works (for me) with other mounts (like XFS or ext4), so submounts
> shouldn’t be the problem.
> 
>> So... does the sshfs filesystem present itself as unshareable? Is it
>> supposed to work? Does it break for others too?
> 
> I can share sshfs through sshfs, so it must be something virtiofs-specific.
> 
> I tried to debug it, but I could only find that the fstatat()/statx() on
> it (FD opened, then stat called with that FD, an empty pathname, and
> AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW) fails with EPERM.  I tried
> disabling all the sandboxing, but still got that error.
> 
> FWIW, I get the same error with virtiofsd-rs (and there, too, the
> fstatat64() yields the EPERM).
> 
> So far, I couldn’t reproduce it outside of virtiofsd, though...  (Like,
> just invoking stat on the command line works; and a simple program that
> opens the mount point FD and then stats it works, too.)
> 
> Max

Re: virtiofsd: sshfs as submount?

[Laszlo Ersek]

On 12/21/20 13:06, Max Reitz wrote:

> I can share sshfs through sshfs, so it must be something virtiofs-specific.

Your insight proved crucial to solving the riddle.

Chaining sshfs with sshfs made me think that you must have used a normal
(non-root) user account on the first remote computer (where you ran the
2nd sshfs command).

And that reminded me of the "allow_root" option which I seemed to have
read somewhere around the FUSE manuals.

So indeed I set up another sshfs mount on my laptop, with my normal UID,
and tried to access the mount point from a plain root shell (with
virtiofsd completely out of the picture) -- it failed with "Permission
denied". :) It's apparently intentional on sshfs's / FUSE's part, to
protect the local root user from "remote nastiness injection".

Then I re-did the sshfs mount, but with "-o allow_root" this time. The
plain root shell can now access the mount point.

... So can virtiofsd :) It's *amazing* to see remote files in the UEFI
shell. I never thought "filesystem as a service" could feel this empowering.

Thanks, Max!
Laszlo

Re: virtiofsd: sshfs as submount?

[Max Reitz]

On 21.12.20 13:45, Laszlo Ersek wrote:
> On 12/21/20 13:06, Max Reitz wrote:
> 
>> I can share sshfs through sshfs, so it must be something virtiofs-specific.
> 
> Your insight proved crucial to solving the riddle.
> 
> Chaining sshfs with sshfs made me think that you must have used a normal
> (non-root) user account on the first remote computer (where you ran the
> 2nd sshfs command).
> 
> And that reminded me of the "allow_root" option which I seemed to have
> read somewhere around the FUSE manuals.

Oh, that makes sense.  Right.

> So indeed I set up another sshfs mount on my laptop, with my normal UID,
> and tried to access the mount point from a plain root shell (with
> virtiofsd completely out of the picture) -- it failed with "Permission
> denied". :) It's apparently intentional on sshfs's / FUSE's part, to
> protect the local root user from "remote nastiness injection".
> 
> Then I re-did the sshfs mount, but with "-o allow_root" this time. The
> plain root shell can now access the mount point.

Indeed, that works better.

> ... So can virtiofsd :) It's *amazing* to see remote files in the UEFI
> shell. I never thought "filesystem as a service" could feel this empowering.
> 
> Thanks, Max!

Er, *cough*, my pleasure! O:)

Max