Mount remount operations propagating from container to host and other containers.

Mount remount operations propagating from container to host and other containers.

[Michael H. Warfield]

[-- Attachment #1.1: Type: text/plain, Size: 3262 bytes --]

Hey all,

Been running into an ugly situation with LXC-Tools that seems to be
pointing up a real serious leakage from containers.  If you have a mount
inside a container (presumably a bind mount in this case), if the
container does a mount -o remount (say rw->ro or ro->rw) this propagates
to the host mount points (all the way to the primary mount point for
that partition in some cases) and is reflected in other containers.

This first show up with containers running full VM's running on a
mounted fs (aot the host / fs) were causing the real mounted fs to
become ro when they were shut down (the VM was remounting its rootfs as
ro and it was leaking out of the container).

I've since confirmed that and encountered it trying to have a shared ro
mounted fs in a container using bind mounts (bind mounts since 2.6.26
have allowed setting the ro flag on individual mount points) and
discovering that one container could make it rw and then all the other
containers would see it as rw as well!  If a container made a mount
point ro, all the other containers would see it as ro and the mount
point for the entire real fs in the host would become ro!  This is very
not good.  That's a pretty serious leakage from the containers out to
the host.

Is this a problem with the container isolation or some problem in
creating the container?

I'm running and testing on a Fedora 12 system with a 2.6.32 kernel.  Not
related (I don't think) but I have also noted that linux-utils-ng on F16
seems to also have a bug irt something similar here.  If I mount a
directory from a mounted partition onto another location and then make
that other location ro, the entire partition becomes ro.  BUT!  If I
then make the partition rw, that does not propagate back up and the bind
mount remains ro.

What should work is this:

Partition /export
Directory /export/readonly

mount --bind /export/readonly /srv/readonly

At this point, /export and /srv/readonly are both rw

mount -o remount,ro /srv/readonly

Now. both /export and /srv/readonly are ro!  This is wrong.
Only /srv/readonly is suppose to be ro!

Now, running...

mount -o remount,rw /export

now, /export is rw and /srv/readonly is readonly.

Back to containers...

If I have /srv/readonly mounted in several containers (same mount point)
it's ro in the host and in the containers...

Running this in one container:

mount -o remount,rw /srv/readonly

(I seriously wish this would NOT WORK AT ALL, but it does.  I don't want
the container to be able to write to that partition at all, like the
media was RO.  Anybody have any ideas on that one?)

Now /srv/readonly is rw in the host and all the containers!

(EVEN WORSE!)

Running this in one container:

mount -o remount,ro /srv/readonly

NOW /srv/readonly is ro in all the containers and /export is ro in the
host.  NOT GOOD.

Thoughts?

Regards,
Mike
-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw-BetbSzk+GohWk0Htik3J/w@public.gmane.org
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 482 bytes --]

[-- Attachment #2: Type: text/plain, Size: 206 bytes --]

_______________________________________________
Containers mailing list
Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

Re: Mount remount operations propagating from container to host and other containers.

[Serge E. Hallyn]

Quoting Michael H. Warfield (mhw-UGBql2FAF+1Wk0Htik3J/w@public.gmane.org):
> Hey all,
> 
> Been running into an ugly situation with LXC-Tools that seems to be
> pointing up a real serious leakage from containers.  If you have a mount
> inside a container (presumably a bind mount in this case), if the
> container does a mount -o remount (say rw->ro or ro->rw) this propagates
> to the host mount points (all the way to the primary mount point for
> that partition in some cases) and is reflected in other containers.

I think you'll want to mount --make-rslave or --make-rprivate
during container setup.  Distros vary with how they leave /
set up, and if it is all rshared from the start, then yeah
your containers' mount actions will propagate backward.

> This first show up with containers running full VM's running on a
> mounted fs (aot the host / fs) were causing the real mounted fs to
> become ro when they were shut down (the VM was remounting its rootfs as
> ro and it was leaking out of the container).
> 
> I've since confirmed that and encountered it trying to have a shared ro
> mounted fs in a container using bind mounts (bind mounts since 2.6.26
> have allowed setting the ro flag on individual mount points) and
> discovering that one container could make it rw and then all the other
> containers would see it as rw as well!  If a container made a mount
> point ro, all the other containers would see it as ro and the mount
> point for the entire real fs in the host would become ro!  This is very
> not good.  That's a pretty serious leakage from the containers out to
> the host.
> 
> Is this a problem with the container isolation or some problem in
> creating the container?
> 
> I'm running and testing on a Fedora 12 system with a 2.6.32 kernel.  Not
> related (I don't think) but I have also noted that linux-utils-ng on F16
> seems to also have a bug irt something similar here.  If I mount a
> directory from a mounted partition onto another location and then make
> that other location ro, the entire partition becomes ro.  BUT!  If I
> then make the partition rw, that does not propagate back up and the bind
> mount remains ro.
> 
> What should work is this:
> 
> Partition /export
> Directory /export/readonly
> 
> mount --bind /export/readonly /srv/readonly
> 
> At this point, /export and /srv/readonly are both rw
> 
> mount -o remount,ro /srv/readonly
> 
> Now. both /export and /srv/readonly are ro!  This is wrong.
> Only /srv/readonly is suppose to be ro!
> 
> Now, running...
> 
> mount -o remount,rw /export
> 
> now, /export is rw and /srv/readonly is readonly.
> 
> Back to containers...
> 
> If I have /srv/readonly mounted in several containers (same mount point)
> it's ro in the host and in the containers...
> 
> Running this in one container:
> 
> mount -o remount,rw /srv/readonly
> 
> (I seriously wish this would NOT WORK AT ALL, but it does.  I don't want
> the container to be able to write to that partition at all, like the
> media was RO.  Anybody have any ideas on that one?)
> 
> Now /srv/readonly is rw in the host and all the containers!
> 
> (EVEN WORSE!)
> 
> Running this in one container:
> 
> mount -o remount,ro /srv/readonly
> 
> NOW /srv/readonly is ro in all the containers and /export is ro in the
> host.  NOT GOOD.
> 
> Thoughts?
> 
> Regards,
> Mike
> -- 
> Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw-BetbSzk+GohWk0Htik3J/w@public.gmane.org
>    /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
>    NIC whois: MHW9          | An optimist believes we live in the best of all
>  PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!



> _______________________________________________
> Containers mailing list
> Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> https://lists.linux-foundation.org/mailman/listinfo/containers

Re: Mount remount operations propagating from container to host and other containers.

[Serge E. Hallyn]

Quoting Michael H. Warfield (mhw-UGBql2FAF+1Wk0Htik3J/w@public.gmane.org):
> Running this in one container:
> 
> mount -o remount,rw /srv/readonly
> 
> (I seriously wish this would NOT WORK AT ALL, but it does.  I don't want
> the container to be able to write to that partition at all, like the
> media was RO.  Anybody have any ideas on that one?)

That is what the work to make VFS respect hierarchical user namespaces
is for.  Alas it is a perennial victim of time shortages.  It would
let you mark that fs as being owned by root in the initial user
namespace.  Then root in a child user namespace would only get the
world access rights (like any other stranger user) to the fs and its
files.  If an fs or files should be owned by user 500 in a container,
then inode->i_uid will actually store 500, the fs will store a
user ns id (which only a particular uid in init_user_ns can associate
with a container) in which user 500 is valid, and optionally userids
to use for file access from higher user namespaces.  So the file is
owned by userid 500 in userns 5 which is a child of init_user_ns,
userns 5 is owned by userid 1000 in init_user_ns, so the file is owned
by userid 1000 in init_user_ns.

There are quite a few threads in the containers mailing list archive
between Eric and I about this, if you're interested in the background.

Part of the reason this work seems to refuse to get past 5-patch
prototypes is that only Eric and I seem to be interested.  Maybe now
that there is a serious growing user base around lxc that will change.

-serge