another scheduler problem, on x86_64 only?

another scheduler problem, on x86_64 only?

[Andrew Theurer]

I Noticed that dom0 is only getting about 50% cpu after seeing about 1/2 
the performance on some benchmarks, xen0 vs baremetal.  I just ran a 
quick cpu bound script:

while [ 1 ]; do x=1; done&

..and I get this cpu util:

	total   domains
	------- ----------
 cpu0:  [050.7] d0-0[050.7]
 
 cpu0:  [049.8] d0-0[049.8]
 
 cpu0:  [049.8] d0-0[049.8]
 
 cpu0:  [049.8] d0-0[049.8]


Anyone else seen this on x86_64 dom0 with BVT? 

I do not see this in i386 with BVT.

[root@e326-1 src]# xm dmesg | grep sched
(XEN) Using scheduler: Borrowed Virtual Time (bvt)

[root@e326-1 src]# xm dmesg | grep -i changeset
 Latest ChangeSet: Tue Jul 12 08:19:10 2005 
9f6057761c8f485b1948fdd3308ae10e3ea66ad5


-Andrew

RE: another scheduler problem, on x86_64 only?

[Nakajima, Jun]

Andrew Theurer wrote:
> I Noticed that dom0 is only getting about 50% cpu after seeing about
> 1/2 the performance on some benchmarks, xen0 vs baremetal.  I just
> ran a quick cpu bound script:
> 
> while [ 1 ]; do x=1; done&
> 
> ..and I get this cpu util:
> 
> 	total   domains
> 	------- ----------
>  cpu0:  [050.7] d0-0[050.7]
> 
>  cpu0:  [049.8] d0-0[049.8]
> 
>  cpu0:  [049.8] d0-0[049.8]
> 
>  cpu0:  [049.8] d0-0[049.8]
> 
> 
> Anyone else seen this on x86_64 dom0 with BVT?

I also see this problem. For example, if I run lmbench on dom0, I used
to see almost 100% CPU load, but it's clearly limited to <= 50% (and
performance is bad). This stared to happen when the default scheduler is
switched to sedf. I also tried various parameters with sedf, the results
were basically same, i.e. never got > 50%. To me it looks like the
scheduler is always giving 50% CPU cycles to the idle domain(?)
regardless of the scheduling policy.

> 
> I do not see this in i386 with BVT.

So maybe it assumes that the guest kernels run in ring1.

> 
> [root@e326-1 src]# xm dmesg | grep sched
> (XEN) Using scheduler: Borrowed Virtual Time (bvt)
> 
> [root@e326-1 src]# xm dmesg | grep -i changeset
>  Latest ChangeSet: Tue Jul 12 08:19:10 2005
> 9f6057761c8f485b1948fdd3308ae10e3ea66ad5
> 
> 
> -Andrew

Jun
---
Intel Open Source Technology Center

Re: another scheduler problem, on x86_64 only?

[Stephan Diestelhorst]

Just for clarification:
   Andrew: Which scheduler did you use? sedf? or bvt?
   Jun: did you try the bvt scheduler too?

Stephan
> Andrew Theurer wrote:
> > I Noticed that dom0 is only getting about 50% cpu after seeing about
> > 1/2 the performance on some benchmarks, xen0 vs baremetal.  I just
> > ran a quick cpu bound script:
> >
> > while [ 1 ]; do x=1; done&
> >
> > ..and I get this cpu util:
> >
> > 	total   domains
> > 	------- ----------
> >  cpu0:  [050.7] d0-0[050.7]
> >
> >  cpu0:  [049.8] d0-0[049.8]
> >
> >  cpu0:  [049.8] d0-0[049.8]
> >
> >  cpu0:  [049.8] d0-0[049.8]
> >
> >
> > Anyone else seen this on x86_64 dom0 with BVT?
>
> I also see this problem. For example, if I run lmbench on dom0, I used
> to see almost 100% CPU load, but it's clearly limited to <= 50% (and
> performance is bad). This stared to happen when the default scheduler is
> switched to sedf. I also tried various parameters with sedf, the results
> were basically same, i.e. never got > 50%. To me it looks like the
> scheduler is always giving 50% CPU cycles to the idle domain(?)
> regardless of the scheduling policy.
>
> > I do not see this in i386 with BVT.
>
> So maybe it assumes that the guest kernels run in ring1.
>
> > [root@e326-1 src]# xm dmesg | grep sched
> > (XEN) Using scheduler: Borrowed Virtual Time (bvt)
> >
> > [root@e326-1 src]# xm dmesg | grep -i changeset
> >  Latest ChangeSet: Tue Jul 12 08:19:10 2005
> > 9f6057761c8f485b1948fdd3308ae10e3ea66ad5
> >
> >
> > -Andrew
>
> Jun
> ---
> Intel Open Source Technology Center
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: another scheduler problem, on x86_64 only?

[Nakajima, Jun]

Stephan Diestelhorst wrote:
> Just for clarification:
>    Andrew: Which scheduler did you use? sedf? or bvt?
>    Jun: did you try the bvt scheduler too?
> 
Yes, I tried to both (either by rebuilding xen or sched=<> boot
parameter), confirming like: 
(XEN) Using scheduler: Borrowed Virtual Time (bvt)

I see the problem no matter which scheduler I choose. I think it should
be a schduling issue on x86_64 xenlinux because the cutoff over CPU
utilization is very obvious; 50% max in the top command, and top is not
confused because benchmarks run slower. 

Jun
---
Intel Open Source Technology Center

Re: another scheduler problem, on x86_64 only?

[Andrew Theurer]

Stephan Diestelhorst wrote:

>Just for clarification:
>   Andrew: Which scheduler did you use? sedf? or bvt?
>  
>
BVT.

I'll try sedf with "xm sedf 0 0 0 0 1 0"

-Andrew