[Qemu-devel] simulating a chroot-like interface with qemu

[Qemu-devel] simulating a chroot-like interface with qemu

[Liraz Siri]

I'd like to run by you guys an idea I've been playing around with.

We've recently cut down in our use of qemu/kvm in our development
toolchain for TurnKey Linux and instead switched to using chroot for
many things, mainly because it is lighter and easier to script which
translates into reduced overhead during development.

On the flip side there are many downsides to using chroot:

* requires root privileges. You can get around this by giving a program
  suid privileges but that's dangerous because...
* root processes inside the chroot can easily break out
* processes inside the chroot share the network stack with processes
  outside the chroot.

  So for example, if mysql is running with the default configuration
  inside a VM and binds to port 3306 that will work even if the host is
  also running mysql listening to 3306. If you're using chroot  there is
  an additional overhead requiring you to reconfigure things.

* similarly, processes inside the chroot share the same abstract unix
  socket namespace, which complicates some usage scenarios...

I'm thinking maybe for some uses it would be useful to simulate an
interface that looks and functions like chroot but is magically
implemented with qemu/kvm behind the scenes (e.g., separate kernel,
network stack, etc.).  Sort of a power chroot that offers stronger
isolation/compartmentalization but with a similar unixish interface
(e.g., pipeable, scriptable, etc.)

Perhaps rather than mounting a block device, the guest could access its
root in the host filesystem transparently via a network filesystem of
some kind.

What do you think? Has anyone tried to use qemu to do something like
this before? Would it be difficult to implement?

Cheers,
Liraz

Re: [Qemu-devel] simulating a chroot-like interface with qemu

[Avi Kivity]

Liraz Siri wrote:
> I'd like to run by you guys an idea I've been playing around with.
>
> We've recently cut down in our use of qemu/kvm in our development
> toolchain for TurnKey Linux and instead switched to using chroot for
> many things, mainly because it is lighter and easier to script which
> translates into reduced overhead during development.
>
> On the flip side there are many downsides to using chroot:
>
> * requires root privileges. You can get around this by giving a program
>   suid privileges but that's dangerous because...
> * root processes inside the chroot can easily break out
> * processes inside the chroot share the network stack with processes
>   outside the chroot.
>
>   So for example, if mysql is running with the default configuration
>   inside a VM and binds to port 3306 that will work even if the host is
>   also running mysql listening to 3306. If you're using chroot  there is
>   an additional overhead requiring you to reconfigure things.
>
> * similarly, processes inside the chroot share the same abstract unix
>   socket namespace, which complicates some usage scenarios...
>
>   

All of these (and many more) will be fixed by the namespace work that's 
being merged into Linux bit by bit.

> I'm thinking maybe for some uses it would be useful to simulate an
> interface that looks and functions like chroot but is magically
> implemented with qemu/kvm behind the scenes (e.g., separate kernel,
> network stack, etc.).  Sort of a power chroot that offers stronger
> isolation/compartmentalization but with a similar unixish interface
> (e.g., pipeable, scriptable, etc.)
>
> Perhaps rather than mounting a block device, the guest could access its
> root in the host filesystem transparently via a network filesystem of
> some kind.
>   

You could easily use nfsroot in the guest together with -kernel and 
-initrd to load a guest without a block device.  With -append and 
'init=' you can have the guest start any script you like.


-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] simulating a chroot-like interface with qemu

[Liraz Siri]

Avi Kivity wrote:
> All of these (and many more) will be fixed by the namespace work that's
> being merged into Linux bit by bit.

Good point. I read up a bit on the progress the lxc folks have done
since I last checked and it looks like functionality similar to what I
am looking for has already been merged into the mainline kernel. The
userspace tools are also coming along nicely.

Making a super chroot like userspace tool would be much easier with lxc
primitives.

> You could easily use nfsroot in the guest together with -kernel and
> -initrd to load a guest without a block device.  With -append and
> 'init=' you can have the guest start any script you like.

Yep, the basics are there. You just have to tie it all together to make
it appear to work transparently. It seems much easier to do this with
the lxc primitives, but doing this with qemu still has some benefits
(e.g., super chroot into another emulated architecture).

Cheers,
Liraz