From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: [Qemu-devel] [RFC] create a single workqueue for each vm to update vm irq routing table Date: Thu, 28 Nov 2013 12:47:10 +0200 Message-ID: <52971F2E.1050605@cloudius-systems.com> References: <52949847.6020908@redhat.com> <5294A68F.6060301@redhat.com> <5294B461.5000405@redhat.com> <5294B634.4050801@cloudius-systems.com> <20131126150357.GA20352@redhat.com> <5294BC3B.6070902@redhat.com> <5297050E.6000700@redhat.com> <20131128091903.GA4609@kernel.org> <5297118C.3050104@cloudius-systems.com> <529712A1.8090207@redhat.com> <5297181B.3090109@cloudius-systems.com> <52971D86.60601@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Gleb Natapov , "Zhanghaoyu (A)" , Gleb Natapov , Avi Kivity , "Huangweidong (C)" , KVM , "Michael S. Tsirkin" , "Jinxin (F)" , Luonengjun , "qemu-devel@nongnu.org" , Zanghongyong To: Paolo Bonzini Return-path: Received: from mail-bk0-f52.google.com ([209.85.214.52]:44047 "EHLO mail-bk0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751493Ab3K1KrP (ORCPT ); Thu, 28 Nov 2013 05:47:15 -0500 Received: by mail-bk0-f52.google.com with SMTP id u14so3698491bkz.39 for ; Thu, 28 Nov 2013 02:47:14 -0800 (PST) In-Reply-To: <52971D86.60601@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On 11/28/2013 12:40 PM, Paolo Bonzini wrote: > Il 28/11/2013 11:16, Avi Kivity ha scritto: >>> The QRCU I linked would work great latency-wise (it has roughly the same >>> latency of an rwsem but readers are lock-free). However, the locked >>> operations in the read path would hurt because of cache misses, so it's >>> not good either. >> I guess srcu would work. Do you know what's the typical write-side >> latency there? (in terms of what it waits for, not nanoseconds). > If there's no concurrent reader, it's zero if I read the code right. > Otherwise it depends: > > - if there are many callbacks, only 10 of them are processed per > millisecond. But unless there are concurrent synchronize_srcu calls > there should not be any callback at all. If all VCPUs were to furiously > change the MSIs, the latency could go up to #vcpu/10 milliseconds. > > - if there are no callbacks, but there are readers, synchronize_srcu > busy-loops for some time checking if the readers complete. After a > while (20 us for synchronize_srcu, 120 us for > synchronize_srcu_expedited) it gives up and starts using a workqueue to > poll every millisecond. This should never happen unless > > So, given the very restricted usage this SRCU would have, we probably > can expect synchronize_srcu_expedited to finish its job in the > busy-looping phase, and 120 us should be the expected maximum > latency---more likely to be an order of magnitude smaller, and in very > rare cases higher. > Looks like it's a good fit then. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42767) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vlz80-0000EE-DB for qemu-devel@nongnu.org; Thu, 28 Nov 2013 05:47:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Vlz7r-0002O2-76 for qemu-devel@nongnu.org; Thu, 28 Nov 2013 05:47:24 -0500 Received: from mail-bk0-f46.google.com ([209.85.214.46]:34794) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vlz7r-0002Ny-03 for qemu-devel@nongnu.org; Thu, 28 Nov 2013 05:47:15 -0500 Received: by mail-bk0-f46.google.com with SMTP id u15so3703093bkz.19 for ; Thu, 28 Nov 2013 02:47:14 -0800 (PST) Message-ID: <52971F2E.1050605@cloudius-systems.com> Date: Thu, 28 Nov 2013 12:47:10 +0200 From: Avi Kivity MIME-Version: 1.0 References: <52949847.6020908@redhat.com> <5294A68F.6060301@redhat.com> <5294B461.5000405@redhat.com> <5294B634.4050801@cloudius-systems.com> <20131126150357.GA20352@redhat.com> <5294BC3B.6070902@redhat.com> <5297050E.6000700@redhat.com> <20131128091903.GA4609@kernel.org> <5297118C.3050104@cloudius-systems.com> <529712A1.8090207@redhat.com> <5297181B.3090109@cloudius-systems.com> <52971D86.60601@redhat.com> In-Reply-To: <52971D86.60601@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC] create a single workqueue for each vm to update vm irq routing table List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: "Huangweidong (C)" , Gleb Natapov , KVM , "Michael S. Tsirkin" , "Zhanghaoyu (A)" , Luonengjun , "qemu-devel@nongnu.org" , Zanghongyong , Avi Kivity , Gleb Natapov , "Jinxin (F)" On 11/28/2013 12:40 PM, Paolo Bonzini wrote: > Il 28/11/2013 11:16, Avi Kivity ha scritto: >>> The QRCU I linked would work great latency-wise (it has roughly the same >>> latency of an rwsem but readers are lock-free). However, the locked >>> operations in the read path would hurt because of cache misses, so it's >>> not good either. >> I guess srcu would work. Do you know what's the typical write-side >> latency there? (in terms of what it waits for, not nanoseconds). > If there's no concurrent reader, it's zero if I read the code right. > Otherwise it depends: > > - if there are many callbacks, only 10 of them are processed per > millisecond. But unless there are concurrent synchronize_srcu calls > there should not be any callback at all. If all VCPUs were to furiously > change the MSIs, the latency could go up to #vcpu/10 milliseconds. > > - if there are no callbacks, but there are readers, synchronize_srcu > busy-loops for some time checking if the readers complete. After a > while (20 us for synchronize_srcu, 120 us for > synchronize_srcu_expedited) it gives up and starts using a workqueue to > poll every millisecond. This should never happen unless > > So, given the very restricted usage this SRCU would have, we probably > can expect synchronize_srcu_expedited to finish its job in the > busy-looping phase, and 120 us should be the expected maximum > latency---more likely to be an order of magnitude smaller, and in very > rare cases higher. > Looks like it's a good fit then.