Re: (Further timing queries) Re: Strange behavior of simple loop programs in Xen guests.

[Anthony Liguori <aliguori@us.ibm.com>]

Arjun wrote:
> Anthony,
>
> Thanks for that. Are there any specific documents or links I can check 
> out regarding this issue ?

Try googling.

> I have another query along these lines :
>
> I have a simple java "Echo" client and server. In the client, I send a 
> message of specific size to the server which echoes it back to the 
> client and time the response. I use the java " 
> System.currentTimeMillis()" function before and after the echo to 
> measure the round-trip-time (RTT). Question is : if both my client and 
> server programs are running inside Xen 3.0 guests, are the timing 
> results trustworthy ?  (Note that I'm measuring on a millisecond scale 
> and am using the jdk1.5)

In this case, you're measuring wall clock time.  Since you're only 
interested in knowing the length of the period, your program will work 
just fine under Xen.

Let me clarify things a little bit more.  Let's say that your kernel 
keeps track of process CPU usage in the following manner.

When a process starts to run, you check a high resolution timer.  When 
the process stops running (due to preemption or IO blocking), you check 
the timer again and compute a delta and store that delta in the 
processes "total run time".  You also add this delta to "total system 
run time".

You can then calculate the percent CPU from dividing the processes 
"total run time" by the "total system run time".  The problem here in 
virtualization is that while that process is running, it may be 
pre-empted to allow another guest to run.  This means the delta that we 
calculate to measure the length that the process ran also includes how 
long the other guest had run.  This extra time is typically referred to 
as "stolen time".

As I mentioned before, to get proper CPU usage information in a guest, 
the kernel has to take "stolen time" into account.  This is something 
that I believe z/Linux does and hopefully we can also do in xenolinux in 
the not to distant future.  For now though, you cannot trust anything 
that relies on knowing how long a process has been using the CPU.

Any other sort of time measurement will work as expected though.

Regards,

Anthony Liguori

> Thanks and regards
> Arjun
>
>
> On 4/15/06, *Anthony Liguori* <aliguori@us.ibm.com 
> <mailto:aliguori@us.ibm.com>> wrote:
>
>     You're experience a well known problem with just about any piece of
>     virtualization software.
>
>     Tools that run in the guest (like sar and top) do not realize that the
>     guest isn't always running which means they don't take this into
>     account
>     when calculating percentages.  This makes the output from them
>     pretty much
>     useless in a virtualized environment.
>
>     The IBM zSeries Linux port has some special modifications to take this
>     trait into account within the kernel so tools like top work like you'd
>     expect.  I believe someone was looking into doing something like
>     this for
>     Xen.
>
>     Regards,
>
>     Anthony Liguori
>
>     On Sat, 15 Apr 2006 02:57:26 -0400, Arjun wrote:
>
>     > Hi,
>     >
>     > I'm running some simple C programs in a Xen 3.0 guest. I have a
>     program
>     > testprog.c containing simple nested loops (see code below). I
>     run this
>     > program and check "sar" and "top" output inside the guest and
>     also xentop
>     > on the host. It shows that the running program takes up almost
>     all the CPU
>     > time as expected.
>     >
>     > Now I introduce a sleep(1) statement inside my outer loop. On
>     running the
>     > modified program, both "sar" and "top" in the guest VM show 0%
>     and xentop
>     > on the host shows about 0.5%
>     > Next I start about a 100 instances of my loop program (started
>     at skewed
>     > times) inside the guest VM. A check on sar and top still shows
>     0% CPU,
>     > however a check on xentop on the host shows an increase in CPU
>     usage for
>     > the guest. This changes with the number of instances of the loop
>     program I
>     > start (more or less linearly).
>     >
>     > I don't understand why this is happening. Why are sar and top
>     showing
>     > incorrect output in the guest VM. Can anyone explain ?
>     >
>     > P.S: code for the programs is posted below.
>     >
>     > Thanks and regards
>     > Arjun
>     >
>
>
>
>     _______________________________________________
>     Xen-devel mailing list
>     Xen-devel@lists.xensource.com <mailto:Xen-devel@lists.xensource.com>
>     http://lists.xensource.com/xen-devel
>
>