From: Marcus Granado <marcus.granado@citrix.com>
To: xen-devel <xen-devel@lists.xensource.com>
Subject: Prioritising dom0 vcpus
Date: Fri, 31 May 2013 18:18:11 +0100 [thread overview]
Message-ID: <51A8DB53.6010301@citrix.com> (raw)
As an experiment trying to reduce the latency when scheduling dom0
vcpus, I applied the following patch to __runq_insert() to xen 4.2:
diff -r 8643ca19d356 -r 91b13479c1a2 xen/common/sched_credit.c
--- a/xen/common/sched_credit.c
+++ b/xen/common/sched_credit.c
@@ -205,6 +205,15 @@
BUG_ON( __vcpu_on_runq(svc) );
BUG_ON( cpu != svc->vcpu->processor );
+ /* if svc is a dom0 vcpu, put it always before all the other vcpus
in the runq,
+ * so that dom0 vcpus always have priority
+ */
+ if (svc->vcpu->domain->domain_id == 0) {
+ svc->pri = CSCHED_PRI_TS_BOOST; /* make sure no vcpu goes in
front of this one until this vcpu is scheduled */
+ list_add(&svc->runq_elem, (struct list_head *)runq);
+ return;
+ }
+
list_for_each( iter, runq )
{
const struct csched_vcpu * const iter_svc = __runq_elem(iter);
However, this patch seems to have had the opposite effect, and I would
like to understand why. A win7 guest now takes hours to start up, and I
believe this is due to dom0 taking an order of 10ms to serve each vm i/o
request, even though the dom0 vcpus and the guest vcpu are in different
pcpus.
xenalyze-a.out: http://pastelink.me/getfile.php?key=390a25
xentrace-D-T5.out: http://pastelink.me/getfile.php?key=b3d584
Any ideas why this is the case?
thanks,
Marcus
--
xenalyze-a.out head:
--
0.006977926 ------ x d32768v23 runstate_change d4v0
blocked->runnable
Creating domain 4
Creating vcpu 0 for dom 4
] 0.006979023 ------ x d32768v23 28004(2:8:4) 2 [ 4 0 ]
] 0.006980999 ------ x d32768v23 2800e(2:8:e) 2 [
7fff edd9df ]
] 0.006981126 ------ x d32768v23 2800f(2:8:f) 3 [ 4
e82 1c9c380 ]
] 0.006981403 ------ x d32768v23 2800a(2:8:a) 4 [
7fff 17 4 0 ]
0.006981687 ------ x d32768v23 runstate_change
d32767v23 running->runnable
Creating vcpu 23 for dom 32767
Using first_tsc for d32767v23 (9024 cycles)
0.006982783 ------ x d?v? runstate_change d4v0
runnable->running
] 0.006996466 ------ x d4v0 28006(2:8:6) 2 [ 4 0 ]
] 0.006997600 ------ x d4v0 2800e(2:8:e) 2 [ 4 4d19 ]
] 0.006997726 ------ x d4v0 2800f(2:8:f) 3 [ 7fff
4d19 ffffffff ]
] 0.006997881 ------ x d4v0 2800a(2:8:a) 4 [ 4 0 7fff
17 ]
0.006998070 ------ x d4v0 runstate_change d4v0
running->blocked
0.006998242 ------ x d?v? runstate_change d32767v23
runnable->running
0.014874949 ----x- - d32767v4 runstate_change d0v4
blocked->runnable
] 0.014879473 ----x- - d32767v4 28004(2:8:4) 2 [ 0 4 ]
0.014880331 -x---- - d32767v1 runstate_change d0v1
blocked->runnable
] 0.014884417 ----x- - d32767v4 2800e(2:8:e) 2 [ 7fff
97fc06 ]
] 0.014884544 ----x- - d32767v4 2800f(2:8:f) 3 [ 0
1978 1c9c380 ]
] 0.014884916 ----x- - d32767v4 2800a(2:8:a) 4 [ 7fff
4 0 4 ]
] 0.014885022 -x---- - d32767v1 28004(2:8:4) 2 [ 0 1 ]
0.014885134 ----x- - d32767v4 runstate_change
d32767v4 running->runnable
0.014885251 --x- - - d32767v2 runstate_change d0v2
blocked->runnable
] 0.014889526 -x-- - - d32767v1 2800e(2:8:e) 2 [ 7fff
97cdd8 ]
] 0.014889731 -x-- - - d32767v1 2800f(2:8:f) 3 [ 0
1b68 1c9c380 ]
] 0.014889949 -x-- - - d32767v1 2800a(2:8:a) 4 [ 7fff
1 0 1 ]
0.014890084 ----x- - d?v? runstate_change d0v4
runnable->running
0.014890176 -x--|- - d32767v1 runstate_change
d32767v1 running->runnable
] 0.014890291 - x-|- - d32767v2 28004(2:8:4) 2 [ 0 2 ]
0.014890374 - -x|- - d32767v3 runstate_change d0v3
blocked->runnable
0.014891134 -x--|- - d?v? runstate_change d0v1
runnable->running
] 0.014891811 -|x-|- - d32767v2 2800e(2:8:e) 2 [ 7fff
96f8a4 ]
] 0.014891905 -|-x|- - d32767v3 28004(2:8:4) 2 [ 0 3 ]
] 0.014891936 -|x-|- - d32767v2 2800f(2:8:f) 3 [ 0
1c23 1c9c380 ]
] 0.014892155 -|x-|- - d32767v2 2800a(2:8:a) 4 [ 7fff
2 0 2 ]
0.014892362 -|--|x - d32767v5 runstate_change d0v5
blocked->runnable
0.014892395 -|x-|- - d32767v2 runstate_change
d32767v2 running->runnable
] 0.014893226 -| x|- - d32767v3 2800e(2:8:e) 2 [ 7fff
982ddb ]
] 0.014893343 -| x|- - d32767v3 2800f(2:8:f) 3 [ 0
c64 1c9c380 ]
0.014893386 -|x-|- - d?v? runstate_change d0v2
runnable->running
] 0.014893556 -||x|- - d32767v3 2800a(2:8:a) 4 [ 7fff
3 0 3 ]
] 0.014893778 -||-|x - d32767v5 28004(2:8:4) 2 [ 0 5 ]
0.014893867 -||x|- - d32767v3 runstate_change
d32767v3 running->runnable
0.014894811 -||x|- - d?v? runstate_change d0v3
runnable->running
] 0.014895067 -||||x - d32767v5 2800e(2:8:e) 2 [ 7fff
982654 ]
] 0.014895192 -||||x - d32767v5 2800f(2:8:f) 3 [ 0
c3c 1c9c380 ]
] 0.014895439 -||||x - d32767v5 2800a(2:8:a) 4 [ 7fff
5 0 5 ]
0.014895815 -||||x - d32767v5 runstate_change
d32767v5 running->runnable
0.014896751 -||||x - d?v? runstate_change d0v5
runnable->running
] 0.014908155 -|||x| - d0v4 28006(2:8:6) 2 [ 0 4 ]
] 0.014908228 -||||x - d0v5 28006(2:8:6) 2 [ 0 5 ]
] 0.014908405 -x|||| - d0v1 28006(2:8:6) 2 [ 0 1 ]
] 0.014909231 -||x|| - d0v3 28006(2:8:6) 2 [ 0 3 ]
] 0.014910265 -|||x| - d0v4 2800e(2:8:e) 2 [ 0 7f14 ]
] 0.014910384 -|||x| - d0v4 2800f(2:8:f) 3 [ 7fff
7f14 ffffffff ]
] 0.014910550 -|||x| - d0v4 2800a(2:8:a) 4 [ 0 4 7fff 4 ]
] 0.014910566 -x|||| - d0v1 2800e(2:8:e) 2 [ 0 6743 ]
] 0.014910679 -x|||| - d0v1 2800f(2:8:f) 3 [ 7fff
6743 ffffffff ]
] 0.014910707 -||||x - d0v5 2800e(2:8:e) 2 [ 0 3f80 ]
0.014910783 -|||x| - d0v4 runstate_change d0v4
running->blocked
] 0.014910803 -x|| | - d0v1 2800a(2:8:a) 4 [ 0 1 7fff 1 ]
] 0.014910819 -||| x - d0v5 2800f(2:8:f) 3 [ 7fff
3f80 ffffffff ]
] 0.014910944 -||| x - d0v5 2800a(2:8:a) 4 [ 0 5 7fff 5 ]
0.014911030 -x|| | - d0v1 runstate_change d0v1
running->blocked
0.014911109 - || x - d0v5 runstate_change d0v5
running->blocked
] 0.014911307 - |x - d0v3 2800e(2:8:e) 2 [ 0 4c74 ]
0.014911367 - || x - d?v? runstate_change d32767v5
runnable->running
] 0.014911417 - |x - - d0v3 2800f(2:8:f) 3 [ 7fff
4c74 ffffffff ]
0.014911471 - ||x- - d?v? runstate_change d32767v4
runnable->running
0.014911512 -x||-- - d?v? runstate_change d32767v1
runnable->running
] 0.014911530 --|x-- - d0v3 2800a(2:8:a) 4 [ 0 3 7fff 3 ]
0.014911687 --|x-- - d0v3 runstate_change d0v3
running->blocked
0.014912276 --|x-- - d?v? runstate_change d32767v3
runnable->running
] 0.015036914 --x--- - d0v2 28006(2:8:6) 2 [ 0 2 ]
] 0.015038191 --x--- - d0v2 2800e(2:8:e) 2 [ 0 28d83 ]
] 0.015038313 --x--- - d0v2 2800f(2:8:f) 3 [ 7fff
28d83 ffffffff ]
] 0.015038445 --x--- - d0v2 2800a(2:8:a) 4 [ 0 2 7fff 2 ]
0.015038617 --x--- - d0v2 runstate_change d0v2
running->blocked
0.015039232 --x--- - d?v? runstate_change d32767v2
runnable->running
0.020630385 ------ x d32767v23 runstate_change d4v0
blocked->runnable
] 0.020631491 ------ x d32767v23 28004(2:8:4) 2 [ 4 0 ]
] 0.020633401 ------ x d32767v23 2800e(2:8:e) 2 [
7fff edb796 ]
] 0.020633555 ------ x d32767v23 2800f(2:8:f) 3 [ 4
d97 1c9c380 ]
] 0.020633813 ------ x d32767v23 2800a(2:8:a) 4 [
7fff 17 4 0 ]
0.020634086 ------ x d32767v23 runstate_change
d32767v23 running->runnable
0.020635147 ------ x d?v? runstate_change d4v0
runnable->running
] 0.020650487 ------ x d4v0 28006(2:8:6) 2 [ 4 0 ]
] 0.020651616 ------ x d4v0 2800e(2:8:e) 2 [ 4 5400 ]
] 0.020651739 ------ x d4v0 2800f(2:8:f) 3 [ 7fff
5400 ffffffff ]
] 0.020651876 ------ x d4v0 2800a(2:8:a) 4 [ 4 0 7fff
17 ]
0.020652054 ------ x d4v0 runstate_change d4v0
running->blocked
next reply other threads:[~2013-05-31 17:18 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-31 17:18 Marcus Granado [this message]
2013-06-03 16:47 ` Prioritising dom0 vcpus Dario Faggioli
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=51A8DB53.6010301@citrix.com \
--to=marcus.granado@citrix.com \
--cc=xen-devel@lists.xensource.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.