[PATCHv3 0/2] vhost: a kernel-level virtio server

[PATCHv3 0/2] vhost: a kernel-level virtio server

[Michael S. Tsirkin]

This implements vhost: a kernel-level backend for virtio,
The main motivation for this work is to reduce virtualization
overhead for virtio by removing system calls on data path,
without guest changes. For virtio-net, this removes up to
4 system calls per packet: vm exit for kick, reentry for kick,
iothread wakeup for packet, interrupt injection for packet.

Some more detailed description attached to the patch itself.

The patches are against 2.6.31-rc4.  I'd like them to go into linux-next
and down the road 2.6.32 if possible.  Please comment.

Changelog from v2:
- Comments on RCU usage
- Compat ioctl support
- Make variable static
- Copied more idiomatic english from Rusty

Changes from v1:
- Move use_mm/unuse_mm from fs/aio.c to mm instead of copying.
- Reorder code to avoid need for forward declarations
- Kill a couple of debugging printks

Michael S. Tsirkin (2):
  mm: export use_mm/unuse_mm to modules
  vhost_net: a kernel-level virtio server

 MAINTAINERS                 |   10 +
 arch/x86/kvm/Kconfig        |    1 +
 drivers/Makefile            |    1 +
 drivers/vhost/Kconfig       |   11 +
 drivers/vhost/Makefile      |    2 +
 drivers/vhost/net.c         |  429 ++++++++++++++++++++++++++++
 drivers/vhost/vhost.c       |  663 +++++++++++++++++++++++++++++++++++++++++++
 drivers/vhost/vhost.h       |  108 +++++++
 fs/aio.c                    |   47 +---
 include/linux/Kbuild        |    1 +
 include/linux/miscdevice.h  |    1 +
 include/linux/mmu_context.h |    9 +
 include/linux/vhost.h       |  100 +++++++
 mm/Makefile                 |    2 +-
 mm/mmu_context.c            |   58 ++++
 15 files changed, 1396 insertions(+), 47 deletions(-)
 create mode 100644 drivers/vhost/Kconfig
 create mode 100644 drivers/vhost/Makefile
 create mode 100644 drivers/vhost/net.c
 create mode 100644 drivers/vhost/vhost.c
 create mode 100644 drivers/vhost/vhost.h
 create mode 100644 include/linux/mmu_context.h
 create mode 100644 include/linux/vhost.h
 create mode 100644 mm/mmu_context.c

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCHv3 0/2] vhost: a kernel-level virtio server

[Or Gerlitz]

Michael S. Tsirkin wrote:
> The patches are against 2.6.31-rc4.  I'd like them to go into linux-next
> and down the road 2.6.32 if possible.  Please comment.

Hi Michael,

Just wanted to make sure with you how this can be tested, is 2.6.31-rc4 
plus these two patches enough to form the kernel part? I wasn't sure if
both the irqfd/iosignalfd are needed and if both are merged to Linus tree...

As for the user space part, should I use upstream qemu plus patches 1-4?

Or.

Re: [PATCHv3 0/2] vhost: a kernel-level virtio server

[Michael S. Tsirkin]

On Wed, Aug 19, 2009 at 11:16:47AM +0300, Or Gerlitz wrote:
> Michael S. Tsirkin wrote:
> > The patches are against 2.6.31-rc4.  I'd like them to go into linux-next
> > and down the road 2.6.32 if possible.  Please comment.
> 
> Hi Michael,
> 
> Just wanted to make sure with you how this can be tested, is 2.6.31-rc4 
> plus these two patches enough to form the kernel part? I wasn't sure if
> both the irqfd/iosignalfd are needed and if both are merged to Linus tree...

No, these patches are on top of Avi's kvm.git

> As for the user space part, should I use upstream qemu plus patches 1-4?

qemu-kvm

> Or.

Re: [PATCHv3 0/2] vhost: a kernel-level virtio server

[Or Gerlitz]

Michael S. Tsirkin wrote:
> No, these patches are on top of Avi's kvm.git
>   
so are they on top of some branch in Avi's kvm.git which is planned to 
be merged for 2.6.32? what branch should I use?

Or.

Re: [PATCHv3 0/2] vhost: a kernel-level virtio server

[Michael S. Tsirkin]

On Wed, Aug 19, 2009 at 04:44:02PM +0300, Or Gerlitz wrote:
> Michael S. Tsirkin wrote:
>> No, these patches are on top of Avi's kvm.git
>>   
> so are they on top of some branch in Avi's kvm.git which is planned to  
> be merged for 2.6.32? what branch should I use?

Yes. master

> Or.

Re: [PATCHv3 0/2] vhost: a kernel-level virtio server

[Or Gerlitz]

Michael S. Tsirkin wrote:
> Yes. master
okay, will get testing this later next week. Any chance you can provide 
some packet-per-second numbers (netperf udp stream with small packets)?

Or.

Re: [PATCHv3 0/2] vhost: a kernel-level virtio server

[Michael S. Tsirkin]

On Thu, Aug 20, 2009 at 04:34:39PM +0300, Or Gerlitz wrote:
> Michael S. Tsirkin wrote:
>> Yes. master
> okay, will get testing this later next week. Any chance you can provide  
> some packet-per-second numbers (netperf udp stream with small packets)?
>
> Or.

Don't expect it to be good yet. I'm working on vm exit mitigation now,
before that only latency is worth checking.

-- 
MST

Re: [PATCHv3 0/2] vhost: a kernel-level virtio server

[Michael S. Tsirkin]

On Thu, Aug 20, 2009 at 04:34:39PM +0300, Or Gerlitz wrote:
> Michael S. Tsirkin wrote:
>> Yes. master
> okay, will get testing this later next week. Any chance you can provide  
> some packet-per-second numbers (netperf udp stream with small packets)?
>
> Or.

If you do, maybe you should apply the following patch on top
(seems to save 2 atomics in about 50% of cases for me).

---

mm: reduce atomic use on use_mm fast path

When mm switched to matches that of active mm, we don't need to
increment and then drop the mm count. Making that conditional reduces
contention on that cache line on SMP systems.

Acked-by: Andrea Arcangeli <aarcange@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

diff --git a/mm/mmu_context.c b/mm/mmu_context.c
index 9989c2f..0777654 100644
--- a/mm/mmu_context.c
+++ b/mm/mmu_context.c
@@ -27,13 +27,16 @@ void use_mm(struct mm_struct *mm)
 
 	task_lock(tsk);
 	active_mm = tsk->active_mm;
-	atomic_inc(&mm->mm_count);
+	if (active_mm != mm) {
+		atomic_inc(&mm->mm_count);
+		tsk->active_mm = mm;
+	}
 	tsk->mm = mm;
-	tsk->active_mm = mm;
 	switch_mm(active_mm, mm, tsk);
 	task_unlock(tsk);
 
-	mmdrop(active_mm);
+	if (active_mm != mm)
+		mmdrop(active_mm);
 }
 EXPORT_SYMBOL_GPL(use_mm);