From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:43072) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RNTXt-0001BH-GE for qemu-devel@nongnu.org; Mon, 07 Nov 2011 13:03:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RNTXr-0001C4-53 for qemu-devel@nongnu.org; Mon, 07 Nov 2011 13:03:45 -0500 Received: from mail-iy0-f173.google.com ([209.85.210.173]:34482) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RNTXr-0001Bp-01 for qemu-devel@nongnu.org; Mon, 07 Nov 2011 13:03:43 -0500 Received: by iakk32 with SMTP id k32so2063056iak.4 for ; Mon, 07 Nov 2011 10:03:41 -0800 (PST) Message-ID: <4EB81D7A.4080409@codemonkey.ws> Date: Mon, 07 Nov 2011 12:03:38 -0600 From: Anthony Liguori MIME-Version: 1.0 References: <1320612020.3299.22.camel@lappy> <4EB8173F.9090008@codemonkey.ws> <1320688364.3500.12.camel@lappy> In-Reply-To: <1320688364.3500.12.camel@lappy> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] Secure KVM List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Sasha Levin Cc: Andrea Arcangeli , Pekka Enberg , Marcelo Tosatti , kvm , "Michael S. Tsirkin" , Corentin Chary , Asias He , Rusty Russell , qemu-devel , Cyrill Gorcunov , Avi Kivity , Ingo Molnar On 11/07/2011 11:52 AM, Sasha Levin wrote: > Hi Anthony, > > Thank you for your comments! > > On Mon, 2011-11-07 at 11:37 -0600, Anthony Liguori wrote: >> On 11/06/2011 02:40 PM, Sasha Levin wrote: >>> Hi all, >>> >>> I'm planning on doing a small fork of the KVM tool to turn it into a >>> 'Secure KVM' enabled hypervisor. Now you probably ask yourself, Huh? >>> >>> The idea was discussed briefly couple of months ago, but never got off >>> the ground - which is a shame IMO. >>> >>> It's easy to explain the problem: If an attacker finds a security hole >>> in any of the devices which are exposed to the guest, the attacker would >>> be able to either crash the guest, or possibly run code on the host >>> itself. >>> >>> The solution is also simple to explain: Split the devices into different >>> processes and use seccomp to sandbox each device into the exact set of >>> resources it needs to operate, nothing more and nothing less. >>> >>> Since I'll be basing it on the KVM tool, which doesn't really emulate >>> that many legacy devices, I'll focus first on the virtio family for the >>> sake of simplicity (and covering 90% of the options). >>> >>> This is my basic overview of how I'm planning on implementing the >>> initial POC: >>> >>> 1. First I'll focus on the simple virtio-rng device, it's simple enough >>> to allow us to focus on the aspects which are important for the POC >>> while still covering most bases (i.e. sandbox to single file >>> - /dev/urandom and such). >>> >>> 2. Do it on a one process per device concept, where for each device >>> (notice - not device *type*) requested, a new process which handles it >>> will be spawned. >>> >>> 3. That process will be limited exactly to the resources it needs to >>> operate, for example - if we run a virtio-blk device, it would be able >>> to access only the image file which it should be using. >>> >>> 4. Connection between hypervisor and devices will be based on unix >>> sockets, this should allow for better separation compared to other >>> approaches such as shared memory. >>> >>> 5. While performance is an aspect, complete isolation is more important. >>> Security is primary, performance is secondary. >>> >>> 6. Share as much code as possible with current implementation of virtio >>> devices, make it possible to run virtio devices either like it's being >>> done now, or by spawning them as separate processes - the amount of >>> specific code for the separate process case should be minimal. >>> >>> >>> Thats all I have for now, comments are *very* welcome. >> >> I thought about this a bit and have some ideas that may or may not help. >> >> 1) If you add device save/load support, then it's something you can potentially >> use to give yourself quite a bit of flexibility in changing the sandbox. At any >> point in run time, you can save the device model's state in the sandbox, destroy >> the sandbox, and then build a new sandbox and restore the device to its former >> state. >> >> This might turn out to be very useful in supporting things like device hotplug >> and/or memory hot plug. >> >> 2) I think it's largely possible to implement all device emulation without doing >> any dynamic memory allocation. Since memory allocation DoS is something you >> have to deal with anyway, I suspect most device emulation already uses a fixed >> amount of memory per device. This can potentially dramatically simplify things. >> >> 3) I think virtio can/should be used as a generic "backend to frontend" >> transport between the device model and the tool. > > virtio requires server and client to have shared memory, so if we > already go with shared memory we can just let the device manage the > actual virtio driver directly, no? Let's say you're implementing an IDE device model in the sandbox. You can try to implement the block layer in the sandbox but I think that quickly will become too difficult. You can do as Avi suggested and do all DMA accesses from the IDE device model as RPCs, or you can map guest memory as shared memory and utilize (1) in order to change that mapping as you need to. At some point, you end up with a struct iovec and an offset that you want to read/write to the virtual disk. You need a way to send that to the "frontend" that will then handle that as a raw/qcow2 request. Well, virtio is great at doing exactly that :-) So if you increase your shared memory to have a little bit extra to stick another vring, you can use that for device model -> front end communication without paying an extra memcpy. For notifications, the easiest thing to do is setup an "event channel" bitmap and use a single eventfd to multiplex that event channel bitmap. This is pretty much how Xen works btw. A single interrupt is reserved and a bitmap is used to dispatch the actual events. So the sandbox loop would look like: void main() { setup_devices(); read_from_event_channel(main_channel); for i in vrings: check_vring_notification(i); } Once vring would be used for dispatching PIO/MMIO. The remaining vrings could be used for anything really. Like I mentioned elsewhere, just think of the sandbox as just an extension of the guests firmware. The purpose of the sandbox is to reduce a very complicated, legacy device model, into a very simple and easy to audit, purely virtio based model. > > Also, things like interrupts would also require some sort of a different > IPC, which would complicate things a bit. > > >> 4) Lack of select() is really challenging. I understand why it's not there >> since it can technically be emulated but it seems like a no-risk syscall to >> whitelist and it would make programming in a sandbox so much easier. Maybe >> Andrea has some comments here? I might be missing something here. > > There are several of these which would be nice to have, and if we can > get seccomp filters we have good flexibility with which APIs we allow > for each device. Yeah, filters are nice but I fear that you lose some of the PR benefits of sandboxing. Once the first application claims to use sandboxing, whitelists a syscall it shouldn't, you'll start getting slashdot articles about "Linux sandbox broken, Linux security hopeless broken". Then what's the point of all of this? Regards, Anthony Liguori >> Regards, >> >> Anthony Liguori >> >>> >> >