From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=59893 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Obvi0-0005Ro-I8 for qemu-devel@nongnu.org; Thu, 22 Jul 2010 09:21:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1Obvhx-0002Ht-WD for qemu-devel@nongnu.org; Thu, 22 Jul 2010 09:21:07 -0400 Received: from mail-vw0-f45.google.com ([209.85.212.45]:32890) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Obvhx-0002He-QO for qemu-devel@nongnu.org; Thu, 22 Jul 2010 09:21:05 -0400 Received: by vws19 with SMTP id 19so131201vws.4 for ; Thu, 22 Jul 2010 06:21:05 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <4C484105.5060607@siemens.com> References: <438ED7C1-AD7C-4946-99D2-B0E9A91B8DF1@gmail.com> <6BBDD0C9-39D5-435C-8CD7-4E3DD8BAF57D@gmail.com> <4C472889.5000407@mail.berlios.de> <2FAAD29F-6C3E-4128-A6E1-46EE15AF80FA@gmail.com> <4C484105.5060607@siemens.com> Date: Thu, 22 Jul 2010 14:21:04 +0100 Message-ID: From: Stefan Hajnoczi Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Subject: [Qemu-devel] Re: Release of COREMU, a scalable and portable full-system emulator List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jan Kiszka Cc: Chen Yufei , qemu-devel@nongnu.org 2010/7/22 Jan Kiszka : > Stefan Hajnoczi wrote: >> On Thu, Jul 22, 2010 at 9:48 AM, Chen Yufei wrote: >>> On 2010-7-22, at =A4W=A4=C81:04, Stefan Weil wrote: >>> >>>> Am 21.07.2010 09:03, schrieb Chen Yufei: >>>>> On 2010-7-21, at =A4W=A4=C85:43, Blue Swirl wrote: >>>>> >>>>> >>>>>> On Sat, Jul 17, 2010 at 10:27 AM, Chen Yufei wr= ote: >>>>>> >>>>>>> We are pleased to announce COREMU, which is a "multicore-on-multico= re" full-system emulator built on Qemu. (Simply speaking, we made Qemu para= llel.) >>>>>>> >>>>>>> The project web page is located at: >>>>>>> http://ppi.fudan.edu.cn/coremu >>>>>>> >>>>>>> You can also download the source code, images for playing on source= forge >>>>>>> http://sf.net/p/coremu >>>>>>> >>>>>>> COREMU is composed of >>>>>>> 1. a parallel emulation library >>>>>>> 2. a set of patches to qemu >>>>>>> (We worked on the master branch, commit 54d7cf136f040713095cbc064f6= 2d753bff6f9d2) >>>>>>> >>>>>>> It currently supports full-system emulation of x64 and ARM MPcore p= latforms. >>>>>>> >>>>>>> By leveraging the underlying multicore resources, it can emulate up= to 255 cores running commodity operating systems (even on a 4-core machine= ). >>>>>>> >>>>>>> Enjoy, >>>>>>> >>>>>> Nice work. Do you plan to submit the improvements back to upstream Q= EMU? >>>>>> >>>>> It would be great if we can submit our code to QEMU, but we do not kn= ow the process. >>>>> Would you please give us some instructions? >>>>> >>>>> -- >>>>> Best regards, >>>>> Chen Yufei >>>>> >>>> Some hints can be found here: >>>> http://wiki.qemu.org/Contribute/StartHere >>>> >>>> Kind regards, >>>> Stefan Weil >>> The patch is in the attachment, produced with command >>> git diff 54d7cf136f040713095cbc064f62d753bff6f9d2 >>> >>> In order to separate what need to be done to make QEMU parallel, we cre= ated a separate library, and the patched QEMU need to be compiled and linke= d with that library. To submit our enhancement to QEMU, maybe we need to in= corporate this library into QEMU. I don't know what would be the best solut= ion. >>> >>> Our approach to make QEMU parallel can be found at http://ppi.fudan.edu= .cn/coremu >>> >>> I will give a short summary here: >>> >>> 1. Each emulated core thread runs a separate binary translator engine a= nd has private code cache. We marked some variables in TCG as thread local.= We also modified the TB invalidation mechanism. >>> >>> 2. Each core has a queue holding pending interrupts. The COREMU library= provides this queue, and interrupt notification is done by sending realtim= e signals to the emulated core thread. >>> >>> 3. Atomic instruction emulation has to be modified for parallel emulati= on. We use lightweight memory transaction which requires only compare-and-s= wap instruction to emulate atomic instruction. >>> >>> 4. Some code in the original QEMU may cause data race bug after we make= it parallel. We fixed these problems. >>> >>> >>> >>> >>> -- >>> Best regards, >>> Chen Yufei >> >> Looking at the patch it seems there is a global lock for hardware >> access via coremu_spin_lock(&cm_hw_lock). How many cores have you >> tried running and do you have lock contention data for cm_hw_lock? > > BTW, this kind of lock is called qemu_global_mutex in QEMU, thus it is a > sleepy lock here which is likely better for the code paths protected by > it in upstream. Are they shorter in COREMU? > >> Have you thought about making hardware emulation concurrent? >> >> These are issues that qemu-kvm faces today since it executes vcpu >> threads in parallel. Both qemu-kvm and the COREMU patches could >> benefit from a solution for concurrent hardware access. > > While we are all looking forward to see more scalable hardware models > :), I think it is a topic that can be addressed widely independent of > parallelizing TCG VCPUs. The latter can benefit from the former, for > sure, but it first of all has to solve its own issues. Right, but it's worth discussing with people who have worked on parallel vcpus from a different angle. > Note that --enable-io-thread provides truly parallel KVM VCPUs also in > upstream these days. Just for TCG, we need that sightly suboptimal CPU > scheduling inside single-threaded tcg_cpu_exec (was renamed to > cpu_exec_all today). > > Jan > > -- > Siemens AG, Corporate Technology, CT T DE IT 1 > Corporate Competence Center Embedded Linux >