From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:57758) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SiCg8-00024f-Eq for qemu-devel@nongnu.org; Fri, 22 Jun 2012 18:50:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SiCg6-00034L-Bg for qemu-devel@nongnu.org; Fri, 22 Jun 2012 18:50:12 -0400 Received: from mout.web.de ([212.227.17.12]:55411) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SiCg6-000343-0t for qemu-devel@nongnu.org; Fri, 22 Jun 2012 18:50:10 -0400 Message-ID: <4FE4F56D.1020201@web.de> Date: Sat, 23 Jun 2012 00:45:01 +0200 From: Jan Kiszka MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig867E145E7C510F6A3F19997A" Subject: [Qemu-devel] [PATCH] kvm: First step to push iothread lock out of inner run loop List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel Cc: Liu Ping Fan , Alexander Graf , Anthony Liguori This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig867E145E7C510F6A3F19997A Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable This sketches a possible path to get rid of the iothread lock on vmexits in KVM mode. On x86, the the in-kernel irqchips has to be used because we otherwise need to synchronize APIC and other per-cpu state accesses that could be changed concurrently. Not yet fully analyzed is the NMI injection path in the absence of an APIC. s390x should be fine without specific locking as their pre/post-run callbacks are empty. Power requires locking for the pre-run callback. This patch is untested, but a similar version was successfully used in a x86 setup with a network I/O path that needed no central iothread locking anymore (required special MMIO exit handling). --- kvm-all.c | 18 ++++++++++++++++-- target-i386/kvm.c | 7 +++++++ target-ppc/kvm.c | 4 ++++ 3 files changed, 27 insertions(+), 2 deletions(-) diff --git a/kvm-all.c b/kvm-all.c index f8e4328..9c3e26f 100644 --- a/kvm-all.c +++ b/kvm-all.c @@ -1460,6 +1460,8 @@ int kvm_cpu_exec(CPUArchState *env) return EXCP_HLT; } =20 + qemu_mutex_unlock_iothread(); + do { if (env->kvm_vcpu_dirty) { kvm_arch_put_registers(env, KVM_PUT_RUNTIME_STATE); @@ -1476,14 +1478,16 @@ int kvm_cpu_exec(CPUArchState *env) */ qemu_cpu_kick_self(); } - qemu_mutex_unlock_iothread(); =20 run_ret =3D kvm_vcpu_ioctl(env, KVM_RUN, 0); =20 - qemu_mutex_lock_iothread(); kvm_arch_post_run(env, run); =20 + /* TODO: push coalesced mmio flushing to the point where we acce= ss + * devices that are using it (currently VGA and E1000). */ + qemu_mutex_lock_iothread(); kvm_flush_coalesced_mmio_buffer(); + qemu_mutex_unlock_iothread(); =20 if (run_ret < 0) { if (run_ret =3D=3D -EINTR || run_ret =3D=3D -EAGAIN) { @@ -1499,19 +1503,23 @@ int kvm_cpu_exec(CPUArchState *env) switch (run->exit_reason) { case KVM_EXIT_IO: DPRINTF("handle_io\n"); + qemu_mutex_lock_iothread(); kvm_handle_io(run->io.port, (uint8_t *)run + run->io.data_offset, run->io.direction, run->io.size, run->io.count); + qemu_mutex_unlock_iothread(); ret =3D 0; break; case KVM_EXIT_MMIO: DPRINTF("handle_mmio\n"); + qemu_mutex_lock_iothread(); cpu_physical_memory_rw(run->mmio.phys_addr, run->mmio.data, run->mmio.len, run->mmio.is_write); + qemu_mutex_unlock_iothread(); ret =3D 0; break; case KVM_EXIT_IRQ_WINDOW_OPEN: @@ -1520,7 +1528,9 @@ int kvm_cpu_exec(CPUArchState *env) break; case KVM_EXIT_SHUTDOWN: DPRINTF("shutdown\n"); + qemu_mutex_lock_iothread(); qemu_system_reset_request(); + qemu_mutex_unlock_iothread(); ret =3D EXCP_INTERRUPT; break; case KVM_EXIT_UNKNOWN: @@ -1533,11 +1543,15 @@ int kvm_cpu_exec(CPUArchState *env) break; default: DPRINTF("kvm_arch_handle_exit\n"); + qemu_mutex_lock_iothread(); ret =3D kvm_arch_handle_exit(env, run); + qemu_mutex_unlock_iothread(); break; } } while (ret =3D=3D 0); =20 + qemu_mutex_lock_iothread(); + if (ret < 0) { cpu_dump_state(env, stderr, fprintf, CPU_DUMP_CODE); vm_stop(RUN_STATE_INTERNAL_ERROR); diff --git a/target-i386/kvm.c b/target-i386/kvm.c index 0d0d8f6..0ad64d1 100644 --- a/target-i386/kvm.c +++ b/target-i386/kvm.c @@ -1631,7 +1631,10 @@ void kvm_arch_pre_run(CPUX86State *env, struct kvm= _run *run) =20 /* Inject NMI */ if (env->interrupt_request & CPU_INTERRUPT_NMI) { + qemu_mutex_lock_iothread(); env->interrupt_request &=3D ~CPU_INTERRUPT_NMI; + qemu_mutex_unlock_iothread(); + DPRINTF("injected NMI\n"); ret =3D kvm_vcpu_ioctl(env, KVM_NMI); if (ret < 0) { @@ -1641,6 +1644,8 @@ void kvm_arch_pre_run(CPUX86State *env, struct kvm_= run *run) } =20 if (!kvm_irqchip_in_kernel()) { + qemu_mutex_lock_iothread(); + /* Force the VCPU out of its inner loop to process any INIT requ= ests * or pending TPR access reports. */ if (env->interrupt_request & @@ -1682,6 +1687,8 @@ void kvm_arch_pre_run(CPUX86State *env, struct kvm_= run *run) =20 DPRINTF("setting tpr\n"); run->cr8 =3D cpu_get_apic_tpr(env->apic_state); + + qemu_mutex_unlock_iothread(); } } =20 diff --git a/target-ppc/kvm.c b/target-ppc/kvm.c index c09cc39..60d91a5 100644 --- a/target-ppc/kvm.c +++ b/target-ppc/kvm.c @@ -471,6 +471,8 @@ void kvm_arch_pre_run(CPUPPCState *env, struct kvm_ru= n *run) int r; unsigned irq; =20 + qemu_mutex_lock_iothread(); + /* PowerPC QEMU tracks the various core input pins (interrupt, criti= cal * interrupt, reset, etc) in PPC-specific env->irq_input_state. */ if (!cap_interrupt_level && @@ -497,6 +499,8 @@ void kvm_arch_pre_run(CPUPPCState *env, struct kvm_ru= n *run) /* We don't know if there are more interrupts pending after this. Ho= wever, * the guest will return to userspace in the course of handling this= one * anyways, so we will get a chance to deliver the rest. */ + + qemu_mutex_unlock_iothread(); } =20 void kvm_arch_post_run(CPUPPCState *env, struct kvm_run *run) --=20 1.7.3.4 --------------enig867E145E7C510F6A3F19997A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/k9W0ACgkQitSsb3rl5xT8JgCfcODIaNp4lCQxx55Zpu0Vrs30 UuoAoObxw04Hwknx129G3bcVQMXYu8VE =biyH -----END PGP SIGNATURE----- --------------enig867E145E7C510F6A3F19997A--