Containers over 9p in 3.14

Containers over 9p in 3.14

[Alin Dobre]

Hello,

We are using 9p to run containers on top of remote filesystems, and it
works correctly using the 3.13.11 kernel. However, there were a bunch of
updates in the fs/9p area in 3.14 which seem to have broken the
namespaces support for containers.

A simple description of how we run the container over 9p is:
- on the remote source host we run diod as 9p server
- on the local host we mount the filesystem in /some/path
- also locally, we run
  contain /some/path /bin/bash

In 3.14 (3.14.4 to be exact), the contain command can no longer mount
the dev filesystem via mount("tmpfs", "dev", "tmpfs", 0, "mode=0755")
after unsharing IPC, NS, USER, UTS and NET, but before unsharing PID.
The above call returns an EPERM. You can look at the very simple code in
contain.c and mount.c at [1].

This call used to work fine in 3.13. I haven't tried to bisect and find
out the exact patch that introduces the problem, but it's one of "git
log --oneline b26d4cd.. fs/9p".

I can provide you with any additional information that might be needed.

Thank you for any feedback.

Cheers,
Ailn.

[1] https://github.com/arachsys/containers

Re: [V9fs-developer] Containers over 9p in 3.14

[Alin Dobre]

On 21/05/14 07:52, Dominique Martinet wrote:
> Hi,
> 
> Alin Dobre wrote on Tue, May 20, 2014 at 06:03:54PM +0100:
>> This call used to work fine in 3.13. I haven't tried to bisect and find
>> out the exact patch that introduces the problem, but it's one of "git
>> log --oneline b26d4cd.. fs/9p".
>>
>> I can provide you with any additional information that might be needed.
> 
> For curiosity, I wouldn't mind seeing your exact mount options ? (cache
> if any, access, posixacl if set, etc..)

Nothing fancy, really. I mounted it manually with:
mount -t 9p -o trans=tcp,aname=/some/path,version=9p2000.L 127.0.0.1
/mount/point. I can take a look at /proc/mounts to tell you all the
mount options, including default ones if you need.

> 
> 
> I've got no experience with containers/no setup I could try it with
> right now, but I recall 3.14 broke the xattr_set call - you might want
> to try to revert this one:
> * bdd5c28 9p: fix return value in case in v9fs_fid_xattr_set()
> 
> 
> Philippe Deniel posted about it on this list a while ago but nothing
> that I could see got done to actually fix it, I guess I'll submit
> something today or tomorrow with proper fix (e.g. keep negative value if
> there's an error to report, but 0 on success)
> 
> 
> I'll have a harder look if this isn't enough.
> 

I noticed you sent a patch, I will try that and come back with the result.

Thanks a lot for the support, Dominique!

Cheers,
Alin.

Re: [V9fs-developer] Containers over 9p in 3.14

[Dominique Martinet]

Hi,

Alin Dobre wrote on Tue, May 20, 2014 at 06:03:54PM +0100:
> This call used to work fine in 3.13. I haven't tried to bisect and find
> out the exact patch that introduces the problem, but it's one of "git
> log --oneline b26d4cd.. fs/9p".
> 
> I can provide you with any additional information that might be needed.

For curiosity, I wouldn't mind seeing your exact mount options ? (cache
if any, access, posixacl if set, etc..)


I've got no experience with containers/no setup I could try it with
right now, but I recall 3.14 broke the xattr_set call - you might want
to try to revert this one:
* bdd5c28 9p: fix return value in case in v9fs_fid_xattr_set()


Philippe Deniel posted about it on this list a while ago but nothing
that I could see got done to actually fix it, I guess I'll submit
something today or tomorrow with proper fix (e.g. keep negative value if
there's an error to report, but 0 on success)


I'll have a harder look if this isn't enough.
-- 
Dominique Martinet

Re: Containers over 9p in 3.14

[Eric W. Biederman]

Alin Dobre <alin.dobre-1hSFou9RDDldEee+Cai+ZQ@public.gmane.org> writes:

> Hello,
>
> We are using 9p to run containers on top of remote filesystems, and it
> works correctly using the 3.13.11 kernel. However, there were a bunch of
> updates in the fs/9p area in 3.14 which seem to have broken the
> namespaces support for containers.

That sounds very weird.  The filesystem used should be orthogonal.

> A simple description of how we run the container over 9p is:
> - on the remote source host we run diod as 9p server
> - on the local host we mount the filesystem in /some/path
> - also locally, we run
>   contain /some/path /bin/bash
>
> In 3.14 (3.14.4 to be exact), the contain command can no longer mount
> the dev filesystem via mount("tmpfs", "dev", "tmpfs", 0, "mode=0755")
> after unsharing IPC, NS, USER, UTS and NET, but before unsharing PID.
> The above call returns an EPERM. You can look at the very simple code in
> contain.c and mount.c at [1].
>
> This call used to work fine in 3.13. I haven't tried to bisect and find
> out the exact patch that introduces the problem, but it's one of "git
> log --oneline b26d4cd.. fs/9p".

That is weird.

There has been a bunch of work to keep from abusing the ability to
mount, that added some additional constraints.

For sys and proc those constraints pretty much mean other versions of
those filesystems need to be mounted somewhere or you are not allowd to
mount a fresh copy.

I can't think of any additional rules for tmpfs off of the top of my
head.

I can't imagine off the top of my head how 9p or any specific filesystem
would be involved.

> I can provide you with any additional information that might be
> needed.

If you are building your own kernels and this is easily reproducible
it may make sense to instrument up the mount call with printks can see
where the failure is coming from.

Unless it is a basic 9p bug where directories don't appear as
directories in some cases my deep hunch is that you have found some way
to run afoul of the restrictions on mount points.

Eric