From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50211) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZFk2q-0008P3-NA for qemu-devel@nongnu.org; Thu, 16 Jul 2015 10:21:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZFk2m-0005F4-3N for qemu-devel@nongnu.org; Thu, 16 Jul 2015 10:21:52 -0400 Received: from e19.ny.us.ibm.com ([129.33.205.209]:59547) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZFk2l-0005ER-VY for qemu-devel@nongnu.org; Thu, 16 Jul 2015 10:21:48 -0400 Received: from /spool/local by e19.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 16 Jul 2015 10:21:46 -0400 Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by d01dlp02.pok.ibm.com (Postfix) with ESMTP id 9BA7E6E8059 for ; Thu, 16 Jul 2015 10:13:29 -0400 (EDT) Received: from d01av03.pok.ibm.com (d01av03.pok.ibm.com [9.56.224.217]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t6GELhs154067448 for ; Thu, 16 Jul 2015 14:21:43 GMT Received: from d01av03.pok.ibm.com (localhost [127.0.0.1]) by d01av03.pok.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t6GELgIg000390 for ; Thu, 16 Jul 2015 10:21:42 -0400 Message-ID: <55A7BDF4.4020509@linux.vnet.ibm.com> Date: Thu, 16 Jul 2015 10:21:40 -0400 From: "Jason J. Herne" MIME-Version: 1.0 References: <1435855010-30882-1-git-send-email-jjherne@linux.vnet.ibm.com> <1435855010-30882-2-git-send-email-jjherne@linux.vnet.ibm.com> <55956A2E.4020806@redhat.com> <55A3CEAF.6030504@linux.vnet.ibm.com> <55A3D5D8.7070902@redhat.com> <55A654D6.5000906@linux.vnet.ibm.com> <55A657ED.3070407@redhat.com> In-Reply-To: <55A657ED.3070407@redhat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v4 1/5] cpu: Provide vcpu throttling interface Reply-To: jjherne@linux.vnet.ibm.com List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini , afaerber@suse.de, amit.shah@redhat.com, dgilbert@redhat.com, borntraeger@de.ibm.com, quintela@redhat.com, qemu-devel@nongnu.org On 07/15/2015 08:54 AM, Paolo Bonzini wrote: > > > On 15/07/2015 14:40, Jason J. Herne wrote: >>>> I'm not sure how callbacks can pile up here. If the vcpus are >>>> running then their thread's will execute the callbacks. If they >>>> are not running then the use of QEMU_CLOCK_VIRTUAL_RT will >>>> prevent the callbacks from stacking because the timer is not >>>> running, right? >>> >>> Couldn't the iothread starve the VCPUs? They need to take the >>> iothread lock in order to process the callbacks. >> >> Yes, I can see the possibility here. I'm not sure what to do about >> it though. >> >> Maybe this is wishful thinking :) But if the iothread lock cannot be >> acquired then the cpu cannot run thereby preventing the guest from >> changing a ton of pages. This will have the effect of indirectly >> throttling the guest which will allow us to advance to the non-live >> phase of migration rather quickly. > > Makes sense. On the other hand this wouldn't prevent callbacks from > piling up for a short time because... > >> And again, if we are starving on >> the iothread lock then the guest vcpus are not executing and >> QEMU_CLOCK_VIRTUAL_RT is not ticking, right? > > ... you are talking about stolen time, and QEMU_CLOCK_VIRTUAL_RT does > count stolen time (stolen time is different for each VCPU, so you would > have a different clock for each VCPU). > > QEMU_CLOCK_VIRTUAL and QEMU_CLOCK_VIRTUAL_RT(*) only pause across > stop/cont. (By the way, the two are the same with KVM). > > However, something like > > if (!atomic_xchg(&cpu->throttle_thread_scheduled, 1)) { > async_run_on_cpu(cpu, cpu_throttle_thread, NULL); > } > > ... > atomic_set(&cpu->throttle_thread_scheduled, 0); > g_usleep(...); > > should be enough. You'd still have many timers that could starve the > VCPUs but, as you pointed out, in that case migration would hopefully > finish pretty fast. > > Paolo > > Paolo, Andreas & David, thanks for the review comments. Has this advanced enough for a reviewed-by? The only remaining objections I can find are: 1. Using atomic operations to manage throttle_percentage. I'm not sure where atomics are applicable here. If this is still a concern hopefully someone can explain. 2. Callback stacking. And it seems like we are convinced that it is not a big issue. Anyone disagree? -- -- Jason J. Herne (jjherne@linux.vnet.ibm.com)