Using KVM from a process inside a Docker container

Using KVM from a process inside a Docker container

[Mike Spreitzer]

I am trying to do this, giving the container as few exceptional abilities 
as possible.  How can I accomplish this?

I know I could simply make the container "privileged", and that would 
work.  But I am trying to give less than that to the container.

Could I get what I need by giving the container some Linux "capabilities"? 
 If so, which ones?

It looks like the process inside the container needs to see a char-special 
file at "/dev/kvm".  So I tried making one, with `mknod /dev/kvm c 10 232` 
inside the container (matching the major and minor I found outside the 
container).  Here is what I saw from inside the container after that 
`mknod`:

# ls -l /dev/kvm
crw-rw---- 1 root 121 10, 232 Nov 27 01:31 /dev/kvm

But I still got these complaints when QEMU inside the container tried to 
use KVM:

Could not access KVM kernel module: Operation not permitted
failed to initialize KVM: Operation not permitted

I also tried bind-mounting the host's /dev/kvm to /dev/kvm inside the 
container.  But that also led to the two same complaints.  I also tried 
bind-mounting the host's /dev to /hostdev inside the container and then 
symlinking /hostdev/kvm to /dev/kvm inside the container, but that also 
produced the same two complaints.  What does it take to get an adequate 
/dev/kvm inside the container?  Is it just a matter of the right Linux 
capabilities, or is it something else?  I am running QEMU as root inside 
the container.

I am using Docker 1.12 and QEMU 2.6 on Ubuntu 16.10 (I know that's 
outdated).

Thanks,
Mike

Re: Using KVM from a process inside a Docker container

[Sean Christopherson]

On Fri, 2017-12-01 at 14:44 -0500, Mike Spreitzer wrote:
> I am trying to do this, giving the container as few exceptional abilities 
> as possible.  How can I accomplish this?
> 
> I know I could simply make the container "privileged", and that would 
> work.  But I am trying to give less than that to the container.
> 
> Could I get what I need by giving the container some Linux "capabilities"? 
>  If so, which ones?
> 
> It looks like the process inside the container needs to see a char-special 
> file at "/dev/kvm".  So I tried making one, with `mknod /dev/kvm c 10 232` 
> inside the container (matching the major and minor I found outside the 
> container).  Here is what I saw from inside the container after that 
> `mknod`:
> 
> # ls -l /dev/kvm
> crw-rw---- 1 root 121 10, 232 Nov 27 01:31 /dev/kvm
> 
> But I still got these complaints when QEMU inside the container tried to 
> use KVM:
> 
> Could not access KVM kernel module: Operation not permitted
> failed to initialize KVM: Operation not permitted
> 
> I also tried bind-mounting the host's /dev/kvm to /dev/kvm inside the 
> container.  But that also led to the two same complaints.  I also tried 
> bind-mounting the host's /dev to /hostdev inside the container and then 
> symlinking /hostdev/kvm to /dev/kvm inside the container, but that also 
> produced the same two complaints.  What does it take to get an adequate 
> /dev/kvm inside the container?  Is it just a matter of the right Linux 
> capabilities, or is it something else?  I am running QEMU as root inside 
> the container.
> 
> I am using Docker 1.12 and QEMU 2.6 on Ubuntu 16.10 (I know that's 
> outdated).
> 
> Thanks,
> Mike
> 

Try --device, e.g. 'docker run --device=/dev/kvm ...'.  I haven't used it
for KVM specifically, but have successfully used it to expose other IOCTL
char devices to an otherwise unprivileged container.

https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities

Re: Using KVM from a process inside a Docker container

[Mike Spreitzer]

> From: Sean Christopherson <sean.j.christopherson@intel.com>
...

> On Fri, 2017-12-01 at 14:44 -0500, Mike Spreitzer wrote:
> > I am trying to do this, giving the container as few exceptional 
abilities 
> > as possible.  How can I accomplish this?
> > 
...

> Try --device, e.g. 'docker run --device=/dev/kvm ...'.  I haven't used 
it
> for KVM specifically, but have successfully used it to expose other 
IOCTL
> char devices to an otherwise unprivileged container.

That worked!

Thanks,
Mike

Re: Using KVM from a process inside a Docker container

[Mike Rapoport]

On Sun, Dec 03, 2017 at 05:01:02PM -0500, Mike Spreitzer wrote:
> > From: Sean Christopherson <sean.j.christopherson@intel.com>
> ...
> 
> > On Fri, 2017-12-01 at 14:44 -0500, Mike Spreitzer wrote:
> > > I am trying to do this, giving the container as few exceptional 
> abilities 
> > > as possible.  How can I accomplish this?
> > > 
> ...
> 
> > Try --device, e.g. 'docker run --device=/dev/kvm ...'.  I haven't used 
> it
> > for KVM specifically, but have successfully used it to expose other 
> IOCTL
> > char devices to an otherwise unprivileged container.
 
You might also want to add /dev/vhost* to speedup the guest I/O.

> That worked!
> 
> Thanks,
> Mike
> 
> 

-- 
Sincerely yours,
Mike.