From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:47205) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SiACJ-0003Jf-1d for qemu-devel@nongnu.org; Fri, 22 Jun 2012 16:11:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SiACH-00087Y-3G for qemu-devel@nongnu.org; Fri, 22 Jun 2012 16:11:14 -0400 Received: from mail-pz0-f45.google.com ([209.85.210.45]:45115) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SiACG-00085g-Ss for qemu-devel@nongnu.org; Fri, 22 Jun 2012 16:11:13 -0400 Received: by dadn2 with SMTP id n2so2834269dad.4 for ; Fri, 22 Jun 2012 13:11:11 -0700 (PDT) Message-ID: <4FE4D15C.2030708@codemonkey.ws> Date: Fri, 22 Jun 2012 15:11:08 -0500 From: Anthony Liguori MIME-Version: 1.0 References: <1340290158-11036-1-git-send-email-qemulist@gmail.com> <4FE33C61.9000509@siemens.com> <4FE44AE7.6050509@siemens.com> In-Reply-To: <4FE44AE7.6050509@siemens.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC] use little granularity lock to substitue qemu_mutex_lock_iothread List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jan Kiszka Cc: liu ping fan , Stefan Hajnoczi , "qemu-devel@nongnu.org" On 06/22/2012 05:37 AM, Jan Kiszka wrote: > On 2012-06-22 12:24, liu ping fan wrote: >> On Thu, Jun 21, 2012 at 11:23 PM, Jan Kiszka wrote: >>> On 2012-06-21 16:49, Liu Ping Fan wrote: >>>> Nowadays, we use qemu_mutex_lock_iothread()/qemu_mutex_unlock_iothread() to >>>> protect the race to access the emulated dev launched by vcpu threads& iothread. >>>> >>>> But this lock is too big. We can break it down. >>>> These patches separate the CPUArchState's protection from the other devices, so we >>>> can have a per-cpu lock for each CPUArchState, not the big lock any longer. >>> >>> Anything that reduces lock dependencies is generally welcome. But can >>> you specify in more details what you gain, and under which conditions? >>> >> In fact, there are several steps to break down the Qemu big lock. This >> step just aims to shrink the code area protected by >> qemu_mutex_lock_iothread()/qemu_mutex_unlock_iothread(). And I am >> working on the following steps, which focus on breaking down the big >> lock when calling handle_{io,mmio} > > Then let us discuss the strategy. This is important as it is unrealistic > to break up the lock for all code paths. We really need to focus on > goals that provide benefits for relevant use cases. Stefan put together a proof of concept that implemented the data-plane portion of virtio-blk in a separate thread. This is possible because of I/O eventfd (we were able to select() on that fd in a separate thread). The performance difference between virtio-blk-pci and virtio-blk-data-plane is staggering when dealing with a very large storage system. So we'd like to get the infrastructure in place where we can start multithreading devices in QEMU to we can integrate this work. The basic plan is introduce granular locking starting at the KVM dispatch level until we can get to MemoryRegion dispatch. We'll then have some way to indicate that a MemoryRegion's callbacks should be invoked without holding the qemu global mutex. We can then convert devices one at a time. While the threading in the KVM code is certainly complex, it's also relatively isolated from the rest of QEMU. So we don't have to worry about auditing large subsystems for re-entrancy safety. Once we have unlocked MemoryRegions, we can start writing some synthetic test cases to really stress the locking code too. Regards, Anthony Liguori > > Jan >