lxc configuration help - only network isolated?

lxc configuration help - only network isolated?

[Chris R. Jones]

I have a couple of basic configuration questions on linux containers.  I'm using lxc-0.6.1.

I'm trying to configure a setup where I have two containers, where the only virtualized/isolated resources are network resources, but I can still do IPC between processes in the two containers.

The lxc.conf man page indicates that, "by default, the pids, sysv ipc, and mount points are virtualized and isolated. "

Is there a way in the configuration to specify that those resources should NOT be isolated?  I'd really like to have communication between two processes running in different containers using sysV IPC and signals.  The only thing I really want to be isolated are two different network namespaces.

Is there a setting I use in the lxc.conf file to accomplish this?

Up to now, I've been doing some prototyping using lxc-unshare -n, but that doesn't really create a container, correct?  That mostly accomplishes my goals, but I can't find a way to spawn new processes into that same namespace.  Is there a way, without defining a container?

Any recommendations on how to properly configure a containers to allow IPC between processes in two different containers while still isolating the network resources?

Thanks,
Chris

Re: lxc configuration help - only network isolated?

[Daniel Lezcano]

Chris R. Jones wrote:
> I have a couple of basic configuration questions on linux containers.  I'm using lxc-0.6.1.
>
> I'm trying to configure a setup where I have two containers, where the only virtualized/isolated resources are network resources, but I can still do IPC between processes in the two containers.
>
> The lxc.conf man page indicates that, "by default, the pids, sysv ipc, and mount points are virtualized and isolated. "
>
> Is there a way in the configuration to specify that those resources should NOT be isolated?  I'd really like to have communication between two processes running in different containers using sysV IPC and signals.  The only thing I really want to be isolated are two different network namespaces.
>
> Is there a setting I use in the lxc.conf file to accomplish this?
>
>   
I thought no one would be interested by less isolation :)

I see you want to share the signals, that means no pid namespace, right ?

The design of the lxc is build around the pid namespace, if you kill the 
first process of the pid namespace, you kill all the process of the 
container. That allows to implement the 'lxc-stop' command.

So no pid namespace, no container :)
> Up to now, I've been doing some prototyping using lxc-unshare -n, but that doesn't really create a container, correct?  That mostly accomplishes my goals, but I can't find a way to spawn new processes into that same namespace.  Is there a way, without defining a container?
>   

No, except writing a forker and launch it inside the container and have 
a command outside to tell the forker to spawn a specific program. 
Dietmar Maurer is working on such component to be integrated in lxc.
> Any recommendations on how to properly configure a containers to allow IPC between processes in two different containers while still isolating the network resources?
>   

If you want to share the ipc to do some testing, you can hack the 
lxc_start function in start.c and remove the ipc cloning flag.

- clone_flags = CLONE_NEWPID|CLONE_NEWIPC|CLONE_NEWNS;
+clone_flags = CLONE_NEWPID|CLONE_NEWNS;

I hope that helps.

  -- Daniel

Re: lxc configuration help - only network isolated?

[Serge E. Hallyn]

Quoting Daniel Lezcano (daniel.lezcano-GANU6spQydw@public.gmane.org):
> Chris R. Jones wrote:
> > I have a couple of basic configuration questions on linux containers.  I'm using lxc-0.6.1.
> >
> > I'm trying to configure a setup where I have two containers, where the only virtualized/isolated resources are network resources, but I can still do IPC between processes in the two containers.
> >
> > The lxc.conf man page indicates that, "by default, the pids, sysv ipc, and mount points are virtualized and isolated. "
> >
> > Is there a way in the configuration to specify that those resources should NOT be isolated?  I'd really like to have communication between two processes running in different containers using sysV IPC and signals.  The only thing I really want to be isolated are two different network namespaces.
> >
> > Is there a setting I use in the lxc.conf file to accomplish this?
> >
> >   
> I thought no one would be interested by less isolation :)
> 
> I see you want to share the signals, that means no pid namespace, right ?
> 
> The design of the lxc is build around the pid namespace, if you kill the 
> first process of the pid namespace, you kill all the process of the 
> container. That allows to implement the 'lxc-stop' command.
> 
> So no pid namespace, no container :)

There has been discussion before about having a 'kill' or 'signal'
cgroup, analogous to the freezer, for sending signals to all tasks
in a cgroup.  We could push that, and have lxc-stop optionally use
that.

If there were interest.

> > Up to now, I've been doing some prototyping using lxc-unshare -n, but that doesn't really create a container, correct?  That mostly accomplishes my goals, but I can't find a way to spawn new processes into that same namespace.  Is there a way, without defining a container?
> >   
> 
> No, except writing a forker and launch it inside the container and have 
> a command outside to tell the forker to spawn a specific program. 
> Dietmar Maurer is working on such component to be integrated in lxc.
> > Any recommendations on how to properly configure a containers to allow IPC between processes in two different containers while still isolating the network resources?
> >   
> 
> If you want to share the ipc to do some testing, you can hack the 
> lxc_start function in start.c and remove the ipc cloning flag.
> 
> - clone_flags = CLONE_NEWPID|CLONE_NEWIPC|CLONE_NEWNS;
> +clone_flags = CLONE_NEWPID|CLONE_NEWNS;
> 
> I hope that helps.

I've heard this request in other places, especially about ipc, so maybe
it's a good idea to work such support into the config.  Maybe something
like:

keep_sharing=CLONE_NEWIPC

in lxc.conf, and mask keep_sharing out of clone_flags at lxc_start?

-serge

Re: lxc configuration help - only network isolated?

[Matt Helsley]

On Fri, Mar 27, 2009 at 08:08:59AM -0500, Serge E. Hallyn wrote:
> Quoting Daniel Lezcano (daniel.lezcano-GANU6spQydw@public.gmane.org):
> > Chris R. Jones wrote:
> > > I have a couple of basic configuration questions on linux containers.  I'm using lxc-0.6.1.
> > >
> > > I'm trying to configure a setup where I have two containers, where the only virtualized/isolated resources are network resources, but I can still do IPC between processes in the two containers.
> > >
> > > The lxc.conf man page indicates that, "by default, the pids, sysv ipc, and mount points are virtualized and isolated. "
> > >
> > > Is there a way in the configuration to specify that those resources should NOT be isolated?  I'd really like to have communication between two processes running in different containers using sysV IPC and signals.  The only thing I really want to be isolated are two different network namespaces.
> > >
> > > Is there a setting I use in the lxc.conf file to accomplish this?
> > >
> > >   
> > I thought no one would be interested by less isolation :)
> > 
> > I see you want to share the signals, that means no pid namespace, right ?
> > 
> > The design of the lxc is build around the pid namespace, if you kill the 
> > first process of the pid namespace, you kill all the process of the 
> > container. That allows to implement the 'lxc-stop' command.
> > 
> > So no pid namespace, no container :)
> 
> There has been discussion before about having a 'kill' or 'signal'
> cgroup, analogous to the freezer, for sending signals to all tasks
> in a cgroup.  We could push that, and have lxc-stop optionally use
> that.
> 
> If there were interest.

I've got some ancient patches for that! :) I think we'd want to
use Paul Menage's recent empty hierarchy, multiply-bound, and stateless
subsystem patches for a signal subsystem. In fact, it may be such a
simple task that using the old patches would be more of a hindrance...

Cheers,
	-Matt Helsley

RE: lxc configuration help - only network isolated?

[Elwin Stelzer Eliazer]

Removing CLONE_NEWIPC from clone_flags and the hack suggested below,
can this be assumed to be a safe feature, or is it only suggested for
testing?
I have a similar requirement like having network only isolation.

regards,
Elwin.




Chris R. Jones wrote:
>* I have a couple of basic configuration questions on linux containers.  I'm using lxc-0.6.1.
*>*
*>* I'm trying to configure a setup where I have two containers, where
the only virtualized/isolated resources are network resources, but I
can still do IPC between processes in the two containers.
*>*
*>* The lxc.conf man page indicates that, "by default, the pids, sysv
ipc, and mount points are virtualized and isolated. "
*>*
*>* Is there a way in the configuration to specify that those
resources should NOT be isolated?  I'd really like to have
communication between two processes running in different containers
using sysV IPC and signals.  The only thing I really want to be
isolated are two different network namespaces.
*>*
*>* Is there a setting I use in the lxc.conf file to accomplish this?
*>*
*>*
*I thought no one would be interested by less isolation :)

I see you want to share the signals, that means no pid namespace, right ?

The design of the lxc is build around the pid namespace, if you kill the
first process of the pid namespace, you kill all the process of the
container. That allows to implement the 'lxc-stop' command.

So no pid namespace, no container :)
>* Up to now, I've been doing some prototyping using lxc-unshare -n, but that doesn't really create a container, correct?  That mostly accomplishes my goals, but I can't find a way to spawn new processes into that same namespace.  Is there a way, without defining a container?
*>*
*
No, except writing a forker and launch it inside the container and have
a command outside to tell the forker to spawn a specific program.
Dietmar Maurer is working on such component to be integrated in lxc.
>* Any recommendations on how to properly configure a containers to allow IPC between processes in two different containers while still isolating the network resources?
*>*
*
If you want to share the ipc to do some testing, you can hack the
lxc_start function in start.c and remove the ipc cloning flag.

- clone_flags = CLONE_NEWPID|CLONE_NEWIPC|CLONE_NEWNS;
+clone_flags = CLONE_NEWPID|CLONE_NEWNS;

I hope that helps.

  -- Daniel

Re: lxc configuration help - only network isolated?

[Serge E. Hallyn]

It's safe.  So long as your applications don't get messed up with
some namespaces isolated and some not, at least.

-serge

Quoting Elwin Stelzer Eliazer (stelzere-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org):
> Removing CLONE_NEWIPC from clone_flags and the hack suggested below,
> can this be assumed to be a safe feature, or is it only suggested for
> testing?
> I have a similar requirement like having network only isolation.
> 
> regards,
> Elwin.
> 
> 
> 
> 
> Chris R. Jones wrote:
> >* I have a couple of basic configuration questions on linux containers.  I'm using lxc-0.6.1.
> *>*
> *>* I'm trying to configure a setup where I have two containers, where
> the only virtualized/isolated resources are network resources, but I
> can still do IPC between processes in the two containers.
> *>*
> *>* The lxc.conf man page indicates that, "by default, the pids, sysv
> ipc, and mount points are virtualized and isolated. "
> *>*
> *>* Is there a way in the configuration to specify that those
> resources should NOT be isolated?  I'd really like to have
> communication between two processes running in different containers
> using sysV IPC and signals.  The only thing I really want to be
> isolated are two different network namespaces.
> *>*
> *>* Is there a setting I use in the lxc.conf file to accomplish this?
> *>*
> *>*
> *I thought no one would be interested by less isolation :)
> 
> I see you want to share the signals, that means no pid namespace, right ?
> 
> The design of the lxc is build around the pid namespace, if you kill the
> first process of the pid namespace, you kill all the process of the
> container. That allows to implement the 'lxc-stop' command.
> 
> So no pid namespace, no container :)
> >* Up to now, I've been doing some prototyping using lxc-unshare -n, but that doesn't really create a container, correct?  That mostly accomplishes my goals, but I can't find a way to spawn new processes into that same namespace.  Is there a way, without defining a container?
> *>*
> *
> No, except writing a forker and launch it inside the container and have
> a command outside to tell the forker to spawn a specific program.
> Dietmar Maurer is working on such component to be integrated in lxc.
> >* Any recommendations on how to properly configure a containers to allow IPC between processes in two different containers while still isolating the network resources?
> *>*
> *
> If you want to share the ipc to do some testing, you can hack the
> lxc_start function in start.c and remove the ipc cloning flag.
> 
> - clone_flags = CLONE_NEWPID|CLONE_NEWIPC|CLONE_NEWNS;
> +clone_flags = CLONE_NEWPID|CLONE_NEWNS;
> 
> I hope that helps.
> 
>   -- Daniel
> _______________________________________________
> Containers mailing list
> Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> https://lists.linux-foundation.org/mailman/listinfo/containers