[Qemu-devel] [PATCH v3 0/3] io-thread optimizations

[Qemu-devel] [PATCH v3 0/3] io-thread optimizations

[Jan Kiszka]

Rebased as requested, also fixing a compiler warning about last_io being
write-only on !CONFIG_IOTHREAD. That "__attribute__ ((unused))" in patch
2 can be removed again when CONFIG_IOTHREAD becomes mandatory.

Jan Kiszka (3):
  Do not drop global mutex for polled main loop runs
  Poll main loop after I/O events were received
  Do not kick vcpus in TCG mode

 cpus.c   |    2 +-
 sysemu.h |    2 +-
 vl.c     |   22 ++++++++++++++++------
 3 files changed, 18 insertions(+), 8 deletions(-)

-- 
1.7.3.4

[Qemu-devel] [PATCH v3 1/3] Do not drop global mutex for polled main loop runs

[Jan Kiszka]

If we call select without a timeout, it's more efficient to keep the
global mutex locked as we may otherwise just play ping pong with a
vcpu thread contending for it. This is particularly important for TCG
mode where we run in lock-step with the vcpu thread.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 vl.c |   10 ++++++++--
 1 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/vl.c b/vl.c
index 06a6f80..0e8b6a4 100644
--- a/vl.c
+++ b/vl.c
@@ -1349,9 +1349,15 @@ void main_loop_wait(int nonblocking)
     qemu_iohandler_fill(&nfds, &rfds, &wfds, &xfds);
     slirp_select_fill(&nfds, &rfds, &wfds, &xfds);
 
-    qemu_mutex_unlock_iothread();
+    if (timeout > 0) {
+        qemu_mutex_unlock_iothread();
+    }
+
     ret = select(nfds + 1, &rfds, &wfds, &xfds, &tv);
-    qemu_mutex_lock_iothread();
+
+    if (timeout > 0) {
+        qemu_mutex_lock_iothread();
+    }
 
     qemu_iohandler_poll(&rfds, &wfds, &xfds, ret);
     slirp_select_poll(&rfds, &wfds, &xfds, (ret < 0));
-- 
1.7.3.4

[Qemu-devel] [PATCH v3 2/3] Poll main loop after I/O events were received

[Jan Kiszka]

Polling until select returns empty fdsets helps to reduce the switches
between iothread and vcpus. The benefit of this patch is best visible
when running an SMP guest on an SMP host in emulation mode.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 sysemu.h |    2 +-
 vl.c     |   12 ++++++++----
 2 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/sysemu.h b/sysemu.h
index bd830e5..9090457 100644
--- a/sysemu.h
+++ b/sysemu.h
@@ -67,7 +67,7 @@ void do_info_snapshots(Monitor *mon);
 
 void qemu_announce_self(void);
 
-void main_loop_wait(int nonblocking);
+int main_loop_wait(int nonblocking);
 
 bool qemu_savevm_state_blocked(Monitor *mon);
 int qemu_savevm_state_begin(Monitor *mon, QEMUFile *f, int blk_enable,
diff --git a/vl.c b/vl.c
index 0e8b6a4..d7e46a9 100644
--- a/vl.c
+++ b/vl.c
@@ -1321,7 +1321,7 @@ void qemu_system_vmstop_request(int reason)
     qemu_notify_event();
 }
 
-void main_loop_wait(int nonblocking)
+int main_loop_wait(int nonblocking)
 {
     fd_set rfds, wfds, xfds;
     int ret, nfds;
@@ -1368,6 +1368,7 @@ void main_loop_wait(int nonblocking)
        them.  */
     qemu_bh_poll();
 
+    return ret;
 }
 
 #ifndef CONFIG_IOTHREAD
@@ -1385,7 +1386,8 @@ qemu_irq qemu_system_powerdown;
 
 static void main_loop(void)
 {
-    bool nonblocking = false;
+    bool nonblocking;
+    int last_io __attribute__ ((unused)) = 0;
 #ifdef CONFIG_PROFILER
     int64_t ti;
 #endif
@@ -1394,7 +1396,9 @@ static void main_loop(void)
     qemu_main_loop_start();
 
     for (;;) {
-#ifndef CONFIG_IOTHREAD
+#ifdef CONFIG_IOTHREAD
+        nonblocking = !kvm_enabled() && last_io > 0;
+#else
         nonblocking = cpu_exec_all();
         if (vm_request_pending()) {
             nonblocking = true;
@@ -1403,7 +1407,7 @@ static void main_loop(void)
 #ifdef CONFIG_PROFILER
         ti = profile_getclock();
 #endif
-        main_loop_wait(nonblocking);
+        last_io = main_loop_wait(nonblocking);
 #ifdef CONFIG_PROFILER
         dev_time += profile_getclock() - ti;
 #endif
-- 
1.7.3.4

[Qemu-devel] [PATCH v3 3/3] Do not kick vcpus in TCG mode

[Jan Kiszka]

In TCG mode, iothread and vcpus run in lock-step. So it's pointless to
send a signal from qemu_cpu_kick to the vcpu thread - if we got here,
the receiver already left the vcpu loop.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 cpus.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/cpus.c b/cpus.c
index c996ac5..b60410c 100644
--- a/cpus.c
+++ b/cpus.c
@@ -869,7 +869,7 @@ void qemu_cpu_kick(void *_env)
     CPUState *env = _env;
 
     qemu_cond_broadcast(env->halt_cond);
-    if (!env->thread_kicked) {
+    if (kvm_enabled() && !env->thread_kicked) {
         qemu_cpu_kick_thread(env);
         env->thread_kicked = true;
     }
-- 
1.7.3.4

Re: [Qemu-devel] [PATCH v3 0/3] io-thread optimizations

[Edgar E. Iglesias]

On Mon, Aug 22, 2011 at 05:46:00PM +0200, Jan Kiszka wrote:
> Rebased as requested, also fixing a compiler warning about last_io being
> write-only on !CONFIG_IOTHREAD. That "__attribute__ ((unused))" in patch
> 2 can be removed again when CONFIG_IOTHREAD becomes mandatory.
> 
> Jan Kiszka (3):
>   Do not drop global mutex for polled main loop runs
>   Poll main loop after I/O events were received
>   Do not kick vcpus in TCG mode
> 
>  cpus.c   |    2 +-
>  sysemu.h |    2 +-
>  vl.c     |   22 ++++++++++++++++------
>  3 files changed, 18 insertions(+), 8 deletions(-)


I tested with SMP guests on SMP hosts and can confirm that this
brings a huge speed up. Last time I tested these patches I only
ran UP, and saw no difference.

Acked-by: Edgar E. Iglesias <edgar.iglesias@gmail.com>

Re: [Qemu-devel] [PATCH v3 0/3] io-thread optimizations

[Anthony Liguori]

On 08/22/2011 10:46 AM, Jan Kiszka wrote:
> Rebased as requested, also fixing a compiler warning about last_io being
> write-only on !CONFIG_IOTHREAD. That "__attribute__ ((unused))" in patch
> 2 can be removed again when CONFIG_IOTHREAD becomes mandatory.

Applied all.  Thanks.

Regards,

Anthony Liguori

>
> Jan Kiszka (3):
>    Do not drop global mutex for polled main loop runs
>    Poll main loop after I/O events were received
>    Do not kick vcpus in TCG mode
>
>   cpus.c   |    2 +-
>   sysemu.h |    2 +-
>   vl.c     |   22 ++++++++++++++++------
>   3 files changed, 18 insertions(+), 8 deletions(-)
>