From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:45669) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QvqH8-0003tq-4Z for qemu-devel@nongnu.org; Tue, 23 Aug 2011 08:40:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QvqH6-0005r5-K8 for qemu-devel@nongnu.org; Tue, 23 Aug 2011 08:40:14 -0400 Received: from mail-yi0-f45.google.com ([209.85.218.45]:53673) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QvqH6-0005r0-HZ for qemu-devel@nongnu.org; Tue, 23 Aug 2011 08:40:12 -0400 Received: by yih10 with SMTP id 10so53447yih.4 for ; Tue, 23 Aug 2011 05:40:11 -0700 (PDT) Message-ID: <4E539FA9.3010507@codemonkey.ws> Date: Tue, 23 Aug 2011 07:40:09 -0500 From: Anthony Liguori MIME-Version: 1.0 References: <1313294689-21572-1-git-send-email-avi@redhat.com> <4E5291DF.1070603@siemens.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-7; format=flowed Content-Transfer-Encoding: 8bit Subject: Re: [Qemu-devel] [PATCH v2] posix-aio-compat: fix latency issues List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: Kevin Wolf , Jan Kiszka , Avi Kivity , kvm@vger.kernel.org, qemu-devel@nongnu.org On 08/23/2011 06:01 AM, Stefan Hajnoczi wrote: > On Mon, Aug 22, 2011 at 6:29 PM, Jan Kiszka wrote: >> On 2011-08-14 06:04, Avi Kivity wrote: >>> In certain circumstances, posix-aio-compat can incur a lot of latency: >>> - threads are created by vcpu threads, so if vcpu affinity is set, >>> aio threads inherit vcpu affinity. This can cause many aio threads >>> to compete for one cpu. >>> - we can create up to max_threads (64) aio threads in one go; since a >>> pthread_create can take around 30ìs, we have up to 2ms of cpu time >>> under a global lock. >>> >>> Fix by: >>> - moving thread creation to the main thread, so we inherit the main >>> thread's affinity instead of the vcpu thread's affinity. >>> - if a thread is currently being created, and we need to create yet >>> another thread, let thread being born create the new thread, reducing >>> the amount of time we spend under the main thread. >>> - drop the local lock while creating a thread (we may still hold the >>> global mutex, though) >>> >>> Note this doesn't eliminate latency completely; scheduler artifacts or >>> lack of host cpu resources can still cause it. We may want pre-allocated >>> threads when this cannot be tolerated. >>> >>> Thanks to Uli Obergfell of Red Hat for his excellent analysis and suggestions. >> >> At this chance: What is the state of getting rid of the remaining delta >> between upstream's version and qemu-kvm? > > That would be nice. qemu-kvm.git uses a signalfd to handle I/O > completion whereas qemu.git uses a signal, writes to a pipe from the > signal handler, and uses qemu_notify_event() to break the vcpu. Once > the force iothread patch is merged we should be able to move to > qemu-kvm.git's signalfd approach. No need to use a signal at all actually. The use of a signal is historic and was required to work around the TCG race that I referred to in another thread. You should be able to just use an eventfd or pipe. Better yet, we should look at using GThreadPool to replace posix-aio-compat. Regards, Anthony Liguori > > Stefan >