RE: Xen automated testing framework

RE: Xen automated testing framework

[Ian Pratt]

> We're starting to work on an automated build and test tool for Xen,
> mainly for the purpose of conveniently allowing automated nightly
> testing so that bugs can be found and fixed quickly.

One thing we've found very useful is to periodically run a series of
benchmarks and record the results. We've found this to be really useful
in quickly tracking down a number of very subtle performance bugs that
have were accidently introduced that we would probably never have even
noticed without the benchmark data showing us things had got a couple of
percent slower.

Particularly useful benchmarks for this purpose are lmbench, ttcp and
dd.

It would be great if someone with spare machines could run these
benchmarks on the nightly snapshots and record the results. Running
various milestones since the 2.0 (or even 1.2) release would be good too
for comparison. 

Thanks,
Ian


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

RE: Xen automated testing framework

[Paul Larson]

[-- Attachment #1: Type: text/plain, Size: 763 bytes --]

On Fri, 2005-01-21 at 22:58 +0000, Ian Pratt wrote:
> It would be great if someone with spare machines could run these
> benchmarks on the nightly snapshots and record the results. Running
> various milestones since the 2.0 (or even 1.2) release would be good too
> for comparison. 
We'll have some machines doing that once we get a workable tool in
place.  The tool is the first priority so that hopefully even more
people could do this if interested.

We'd also like to add tests to this as development goes forward.  So, if
anyone has xen-specific tests, feel free to send them my way!  I'd like
to start putting together a testsuite for this at some point.

-- 
Thanks,
Paul Larson
plars@linuxtestproject.org
http://www.linuxtestproject.org

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Xen automated testing framework

[Ian Pratt]

> On Fri, 2005-01-21 at 22:58 +0000, Ian Pratt wrote:
> > It would be great if someone with spare machines could run these
> > benchmarks on the nightly snapshots and record the results. Running
> > various milestones since the 2.0 (or even 1.2) release would be good too
> > for comparison. 
> We'll have some machines doing that once we get a workable tool in
> place.  The tool is the first priority so that hopefully even more
> people could do this if interested.
> 
> We'd also like to add tests to this as development goes forward.  So, if
> anyone has xen-specific tests, feel free to send them my way!  I'd like
> to start putting together a testsuite for this at some point.

One area where it would be useful to have specific tests is to
exercise the various mechanisms for accessing thread local
storage that have been used by different versions of
libc. Ideally, these tests would be run simultaneously in two
domains running on the same CPU, to ensure everything gets
context switched OK. I've appended a little test written by Keir
that exercises fs/gs with segments from an LDT. We should
probably augment it with tests that use set_thread_area.

Has LTP already got tests that exercise NPTL? Running these in
two domains on the same CPU may be sufficient. 

Another area where it would be worth writing Xen specific tests
is to test time by comparing the progress of gettimeofday against
the cycle counter to ensure that it progresses monotonically and
'smoothly' when the domain is running, and to compare it against
the RTC to check the rate. I've previously posted a test that
does part of this.

Ian

--

#include <stdio.h>
#include <stdlib.h>


/* Structure passed on `modify_ldt' call.  */
struct modify_ldt_ldt_s
{
    unsigned int entry_number;
    unsigned long int base_addr;
    unsigned int limit;
    unsigned int seg_32bit:1;
    unsigned int contents:2;
    unsigned int read_exec_only:1;
    unsigned int limit_in_pages:1;
    unsigned int seg_not_present:1;
    unsigned int useable:1;
    unsigned int empty:25;
};

static void DO_CLEAR_LDT(unsigned int nr)
{
    int rc;
    struct modify_ldt_ldt_s ldt_entry = { 
        nr, (unsigned long)0, 0,
        0, 0, 0, 0, 0, 0, 0 
    };

    __asm__ __volatile__ (
        "int $0x80" : "=&a" (rc) : "0" (123), "b" (1), 
        "c" (&ldt_entry), "d" (sizeof(ldt_entry)));

    if (rc != 0)
    {
        printf("Urk! %d\n", rc);
        exit(0);
    }
}

static void DO_MODIFY_LDT(void *descr, unsigned int nr)
{
    int rc;
    struct modify_ldt_ldt_s ldt_entry = { 
        nr, (unsigned long)descr, 0xfffff /* 4GB in pages */,
        1, 0, 0, 1, 0, 1, 0 
    };

    printf("Setting LDT entry...\n");

    __asm__ __volatile__ (
        "int $0x80" : "=&a" (rc) : "0" (123), "b" (1), 
        "c" (&ldt_entry), "d" (sizeof(ldt_entry)));

    if (rc != 0)
    {
        printf("Urk! %d\n", rc);
        exit(0);
    }

    printf("LDT entry set.\nLoading FS...\n");

    __asm__ __volatile__ ("movw %w0, %%fs" : : "q" (nr * 8 + 7));

    printf("FS loaded.\n");
}

unsigned long segment [8192] = { 0xaaaaaaaa };

int main(int argc, char **argv)
{
    int i = 0, nr;
    unsigned long foo, fs, gs;

    memset(segment, 0xaa, sizeof(segment));

    if ( argc != 2 )
    {
        printf("Usage: %s <entry number>\n", argv[0]);
        return 0;
    }

    nr = atoi(argv[1]);
    printf("Going for desc %d\n", nr);

    DO_MODIFY_LDT(segment, nr); 
    __asm__ __volatile__ ("movl %%fs:(%0),%0" : "=&a" (foo) : "0" (0) );
    printf("Found %08lx\n", foo);

    while ( 1 )
    {
        usleep(10000);

        __asm__ __volatile__ (
            "pushl %%fs; pushl %%gs; popl %%eax; popl %%ebx"
            : "=a" (gs), "=b" (fs) );

//        if ( ((fs&0xffff) != (nr*8+7)) )
            printf("Urkle! %08lx %08lx\n", fs, gs);

        DO_CLEAR_LDT(nr);
        
        if ( i++ == 100 )
        {
            i = 0;
            printf("."); fflush(stdout);
        }
    }

    return 0;
}



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: Xen automated testing framework

[David Hopwood]

Ian Pratt wrote:
> Has LTP already got tests that exercise NPTL?

<http://nptl.bullopensource.org/home.php>

-- 
David Hopwood <david.nospam.hopwood@blueyonder.co.uk>



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Xen automated testing framework

[Paul Larson]

[-- Attachment #1.1: Type: text/plain, Size: 715 bytes --]

We're starting to work on an automated build and test tool for Xen,
mainly for the purpose of conveniently allowing automated nightly
testing so that bugs can be found and fixed quickly.

Right now it's not terribly useful and doesn't do error checking, but is
more of a proof of concept.  If anyone has any ideas or suggestions
about this, please let me know.  I'd like to get some feedback before
going too far, so that it can be made useful for more people.  It should
eventually contain init scripts to kick off the tests, support more
domains, and gather the results via something like scp or a shared
mount.

-- 
Thanks,
Paul Larson
plars@linuxtestproject.org
http://www.linuxtestproject.org

[-- Attachment #1.2: xentest.tgz --]
[-- Type: application/x-compressed-tar, Size: 2621 bytes --]

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Xen automated testing framework

[Ronald G. Minnich]

would you actually test booting domains other than dom0? This is a pretty 
neat idea.

ron



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: Xen automated testing framework

[Paul Larson]

[-- Attachment #1: Type: text/plain, Size: 281 bytes --]

On Fri, 2005-01-21 at 15:21 -0700, Ronald G. Minnich wrote:
> would you actually test booting domains other than dom0? This is a pretty 
> neat idea.
Yes, and run tests in those domains.

-- 
Thanks,
Paul Larson
plars@linuxtestproject.org
http://www.linuxtestproject.org

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Xen automated testing framework

[Ronald G. Minnich]

On Fri, 21 Jan 2005, Paul Larson wrote:

> On Fri, 2005-01-21 at 15:21 -0700, Ronald G. Minnich wrote:
> > would you actually test booting domains other than dom0? This is a pretty 
> > neat idea.
> Yes, and run tests in those domains.

I guess I'll need to work with you on making plan 9 testable then.

ron


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: Xen automated testing framework

[Li Ge]

[-- Attachment #1.1.1: Type: text/plain, Size: 2357 bytes --]






Paul,
      The tar file contains the old xentest.cfg which does not have those
tests sections. Well, this is no big deal cause I highly suspect people
will actually run those scripts now.

Thanks,

Li Ge
Linux Technology Center
Tie Line 678-0047, External Line (512)838-0047
lge@us.ibm.com



                                                                       
             Paul Larson                                               
             <plars@linuxtestp                                         
             roject.org>                                                To
                                       xen-devel                       
             01/21/2005 04:17          <xen-devel@lists.sourceforge.net>
             PM                                                         cc
                                       Li Ge/Austin/IBM@IBMUS, David F 
                                       Barrera/Austin/IBM@IBMUS        
                                                                   Subject
                                       Xen automated testing framework 
                                                                       
                                                                       
                                                                       
                                                                       
                                                                       
                                                                       




We're starting to work on an automated build and test tool for Xen,
mainly for the purpose of conveniently allowing automated nightly
testing so that bugs can be found and fixed quickly.

Right now it's not terribly useful and doesn't do error checking, but is
more of a proof of concept.  If anyone has any ideas or suggestions
about this, please let me know.  I'd like to get some feedback before
going too far, so that it can be made useful for more people.  It should
eventually contain init scripts to kick off the tests, support more
domains, and gather the results via something like scp or a shared
mount.

--
Thanks,
Paul Larson
plars@linuxtestproject.org
http://www.linuxtestproject.org
(See attached file: xentest.tgz)(See attached file: signature.asc)

[-- Attachment #1.1.2: Type: text/html, Size: 3661 bytes --]

[-- Attachment #1.2: graycol.gif --]
[-- Type: image/gif, Size: 105 bytes --]

[-- Attachment #1.3: pic03074.gif --]
[-- Type: image/gif, Size: 1255 bytes --]

[-- Attachment #1.4: ecblank.gif --]
[-- Type: image/gif, Size: 45 bytes --]

[-- Attachment #2: xentest.tgz --]
[-- Type: application/octet-stream, Size: 2621 bytes --]

[-- Attachment #3: signature.asc --]
[-- Type: application/octet-stream, Size: 196 bytes --]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQBB8X9tbkpggQiFDqcRAhF0AJ9KV9DqcnnGyZ6g4Q1AMyN+rRsnDACbBtwI
XM8vFd3546jeETZDa5g/ut0=
=36kk
-----END PGP SIGNATURE-----