From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41708) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XuKzy-0008H9-FB for qemu-devel@nongnu.org; Fri, 28 Nov 2014 07:50:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XuKzl-0000Bo-4F for qemu-devel@nongnu.org; Fri, 28 Nov 2014 07:50:10 -0500 Received: from mx-v6.kamp.de ([2a02:248:0:51::16]:52699 helo=mx01.kamp.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XuKzk-0000BW-Q9 for qemu-devel@nongnu.org; Fri, 28 Nov 2014 07:49:57 -0500 Message-ID: <54786F71.2090706@kamp.de> Date: Fri, 28 Nov 2014 13:49:53 +0100 From: Peter Lieven MIME-Version: 1.0 References: <1417084026-12307-1-git-send-email-pl@kamp.de> <1417084026-12307-4-git-send-email-pl@kamp.de> <547753F7.2030709@redhat.com> <54782EC3.10005@kamp.de> <54784E55.6060405@redhat.com> <54785067.60905@kamp.de> <547858FF.5070602@redhat.com> <54785AA5.9070409@kamp.de> <54785B2E.9070203@redhat.com> <54785D60.1070306@kamp.de> <5478609B.8060503@kamp.de> <547869DE.3080907@redhat.com> <54786CF5.2060705@kamp.de> <54786E52.6050209@redhat.com> In-Reply-To: <54786E52.6050209@redhat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC PATCH 3/3] qemu-coroutine: use a ring per thread for the pool List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini , ming.lei@canonical.com, Kevin Wolf , Stefan Hajnoczi , "qemu-devel@nongnu.org" , Markus Armbruster Am 28.11.2014 um 13:45 schrieb Paolo Bonzini: > > On 28/11/2014 13:39, Peter Lieven wrote: >> Am 28.11.2014 um 13:26 schrieb Paolo Bonzini: >>> On 28/11/2014 12:46, Peter Lieven wrote: >>>>> I get: >>>>> Run operation 40000000 iterations 9.883958 s, 4046K operations/s, 247ns per coroutine >>>> Ok, understood, it "steals" the whole pool, right? Isn't that bad if we have more >>>> than one thread in need of a lot of coroutines? >>> Overall the algorithm is expected to adapt. The N threads contribute to >>> the global release pool, so the pool will fill up N times faster than if >>> you had only one thread. There can be some variance, which is why the >>> maximum size of the pool is twice the threshold (and probably could be >>> tuned better). >>> >>> Benchmarks are needed on real I/O too, of course, especially with high >>> queue depth. >> Yes, cool. The atomic operations are a bit tricky at the first glance ;-) >> >> Question: >> Why is the pool_size increment atomic and the set to zero not? > Because the set to zero is not a read-modify-write operation, so it is > always atomic. It's just not sequentially-consistent (see > docs/atomics.txt for some info on what that means). > >> Idea: >> If the release_pool is full why not put the coroutine in the thread alloc_pool instead of throwing it away? :-) > Because you can only waste 64 coroutines per thread. But numbers cannot > be sneezed at, so it's worth doing it as a separate patch. What do you mean by that? If I use dataplane I will fill the global pool and never use it okay, but then I use thread local storage only. So I get the same numbers as in my thread local storage only version. Maybe it is an idea to tweak the POOL_BATCH_SIZE * 2 according to what is really attached. If we have only dataplane or ioeventfd it can be POOL_BATCH_SIZE * 0 and we even won't waste those coroutines oxidating in the global pool. Peter