From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony Liguori Subject: Re: [PATCH 1/2] kvm tools: Respect ISR status in virtio header Date: Sat, 07 May 2011 09:21:22 -0500 Message-ID: <4DC55562.4070304@codemonkey.ws> References: <1304735660-10844-1-git-send-email-asias.hejun@gmail.com> <20110507093027.GD27657@elte.hu> <4DC545BA.3030501@us.ibm.com> <20110507140251.GB2859@elte.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Asias He , "Michael S. Tsirkin" , Rusty Russell , Mark McLoughlin , Pekka Enberg , Cyrill Gorcunov , Sasha Levin , Prasad Joshi , kvm@vger.kernel.org To: Ingo Molnar Return-path: Received: from mail-yw0-f46.google.com ([209.85.213.46]:59110 "EHLO mail-yw0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755115Ab1EGOV0 (ORCPT ); Sat, 7 May 2011 10:21:26 -0400 Received: by ywj3 with SMTP id 3so1454928ywj.19 for ; Sat, 07 May 2011 07:21:25 -0700 (PDT) In-Reply-To: <20110507140251.GB2859@elte.hu> Sender: kvm-owner@vger.kernel.org List-ID: On 05/07/2011 09:02 AM, Ingo Molnar wrote: > Well the optimization also avoids unnecessary VM exits (due to the injection, > which interrupts a guest context immediately, even if it's running on another > CPU), not just system calls - so it could be more expensive than a system call, > right? If there's IRQ is already pending, the syscall should be a nop. If the IRQ is extraneous, then there's a pretty short window when the IRQ wouldn't already be pending. I took a look at the current virtio code and noticed a few things that are sub-optimal: 1) No MSI. MSI makes interrupts edge triggered and avoids the ISR read entirely. The ISR read is actually pretty expensive. 2) The access pattern is basically: while pending_requests: a = get_next_request(); process_next_request(a); But this is not the optimal pattern with virtio. The optimal pattern is: if pending_requests: disable_notifications(); again: while pending_requests: a = get_next_request(); process_next_request(); enable_notifications(); if pending_requests: goto again; You're currently taking exits way more than you should, particularly since you're using threads for processing the ring queues. 3) You aren't advertising VIRTIO_F_NOTIFY_ON_EMPTY. Particularly with TX, it's advantageous to inject an IRQ even if they're disabled when there are no longer packets in the ring. (2) and (1) are definitely the most important for performance. (2) should be a pretty simple change and should have a significant impact. Regards, Anthony Liguori > > Thanks, > > Ingo > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html