Re: [kvm-unit-tests PATCH] kvm-unit-tests: Gtests Framework and Example Tests

Re: [kvm-unit-tests PATCH] kvm-unit-tests: Gtests Framework and Example Tests

[Radim Krčmář]

2018-01-25 10:48-0800, Ken Hofsass:
> Test framework for KVM and the Linux kernel that enables standalone
> tests to exercise host, guest, and nested guest functionality without
> requiring a separate .flat file. The framework and tests are written
> in C and assembly.
> 
> Signed-off-by: Ken Hofsass <hofsass@google.com>
> ---
>  configure                          |    4 +
>  x86/Makefile.common                |   31 +-
>  gtests/include/kvm_util.h          |  346 ++++
>  gtests/include/test_sparsebit.h    |   82 +
>  gtests/include/test_util.h         |  602 +++++++
>  gtests/include/vmx.h               |  674 +++++++
>  gtests/include/x86.h               |  306 ++++
>  gtests/lib/kvm_util.c              | 3003 +++++++++++++++++++++++++++++++
>  gtests/lib/test_sparsebit.c        | 2793 +++++++++++++++++++++++++++++
>  gtests/lib/test_util.c             | 3411 ++++++++++++++++++++++++++++++++++++
>  gtests/tests/set_sregs_test.c      |   66 +
>  gtests/tests/vmx_tsc_adjust_test.c |  215 +++
>  gtests/.gitignore                  |    2 +
>  13 files changed, 11532 insertions(+), 3 deletions(-)
>  create mode 100644 gtests/include/kvm_util.h
>  create mode 100644 gtests/include/test_sparsebit.h
>  create mode 100644 gtests/include/test_util.h
>  create mode 100644 gtests/include/vmx.h
>  create mode 100644 gtests/include/x86.h
>  create mode 100644 gtests/lib/kvm_util.c
>  create mode 100644 gtests/lib/test_sparsebit.c
>  create mode 100644 gtests/lib/test_util.c
>  create mode 100644 gtests/tests/set_sregs_test.c
>  create mode 100644 gtests/tests/vmx_tsc_adjust_test.c
>  create mode 100644 gtests/.gitignore

This email didn't make it to the kvm list.  Might be because it is too
big, please see what the server said and potentially resend after
splitting into parts, thanks.

(Also, doesn't compile here due to missing #include <asm/msr-index.h>
 and I didn't look further.)

Re: [kvm-unit-tests PATCH] kvm-unit-tests: Gtests Framework and Example Tests

[Ken Hofsass]

Thanks for looking at this Radim,

On Tue, Jan 30, 2018 at 8:09 AM, Radim Krčmář <rkrcmar@redhat.com> wrote:
> 2018-01-25 10:48-0800, Ken Hofsass:
>> Test framework for KVM and the Linux kernel that enables standalone
> ....
> This email didn't make it to the kvm list.  Might be because it is too
> big, please see what the server said and potentially resend after
> splitting into parts, thanks.

OK. The gtests framework code doesn't readily break into smaller
functionally independent patches. I will break it into patches that
separately add the large files and then have the last patch update the
Makefiles/etc, so the project still builds after each patch. (If
that's not acceptable, please recommend an alternative.)

> (Also, doesn't compile here due to missing #include <asm/msr-index.h>
>  and I didn't look further.)

My apologies. Please try "KERNEL=<kernel-srcdir> make"

The gtests code needs the actual KVM/kernel headers for the API/ABI
details. I should have added a check for $KERNEL before building
gtests. If you have a different recommendation for referencing the
kernel sources from the kvm-unit-test directory, let me know.

thanks again,
Ken

Re: [kvm-unit-tests PATCH] kvm-unit-tests: Gtests Framework and Example Tests

[Andrew Jones]

On Tue, Jan 30, 2018 at 10:28:34AM -0800, Ken Hofsass wrote:
> Thanks for looking at this Radim,
> 
> On Tue, Jan 30, 2018 at 8:09 AM, Radim Krčmář <rkrcmar@redhat.com> wrote:
> > 2018-01-25 10:48-0800, Ken Hofsass:
> >> Test framework for KVM and the Linux kernel that enables standalone
> > ....
> > This email didn't make it to the kvm list.  Might be because it is too
> > big, please see what the server said and potentially resend after
> > splitting into parts, thanks.
> 
> OK. The gtests framework code doesn't readily break into smaller
> functionally independent patches. I will break it into patches that
> separately add the large files and then have the last patch update the
> Makefiles/etc, so the project still builds after each patch. (If
> that's not acceptable, please recommend an alternative.)
> 
> > (Also, doesn't compile here due to missing #include <asm/msr-index.h>
> >  and I didn't look further.)
> 
> My apologies. Please try "KERNEL=<kernel-srcdir> make"
> 
> The gtests code needs the actual KVM/kernel headers for the API/ABI
> details. I should have added a check for $KERNEL before building
> gtests. If you have a different recommendation for referencing the
> kernel sources from the kvm-unit-test directory, let me know.
>

We've been copying the kernel files we need (see lib/linux/), or even
just cherry-picking the defines we need from linux headers and dropping
them into their respective kvm-unit-tests' lib/$ARCH/asm/ headers. Is
there some reason you've opted not to do that?

On a historical note, kvm-unit-tests used to look in kernel sources for
headers, but we decided it would be more useful for the test suite to
remove the dependency. That was pre 'make standalone' though. We could
start a discussion about how much we need kvm-unit-tests to _build_
without the dependency now, as one can always build standalone tests
on a build machine and then easily copy them to test machines.

Thanks,
drew

Re: [kvm-unit-tests PATCH] kvm-unit-tests: Gtests Framework and Example Tests

[Radim Krčmář]

2018-01-31 09:56+0100, Andrew Jones:
> On Tue, Jan 30, 2018 at 10:28:34AM -0800, Ken Hofsass wrote:
> > Thanks for looking at this Radim,
> > 
> > On Tue, Jan 30, 2018 at 8:09 AM, Radim Krčmář <rkrcmar@redhat.com> wrote:
> > > 2018-01-25 10:48-0800, Ken Hofsass:
> > >> Test framework for KVM and the Linux kernel that enables standalone
> > > ....
> > > This email didn't make it to the kvm list.  Might be because it is too
> > > big, please see what the server said and potentially resend after
> > > splitting into parts, thanks.
> > 
> > OK. The gtests framework code doesn't readily break into smaller
> > functionally independent patches. I will break it into patches that
> > separately add the large files and then have the last patch update the
> > Makefiles/etc, so the project still builds after each patch. (If
> > that's not acceptable, please recommend an alternative.)

Sounds good, thanks.

> > > (Also, doesn't compile here due to missing #include <asm/msr-index.h>
> > >  and I didn't look further.)
> > 
> > My apologies. Please try "KERNEL=<kernel-srcdir> make"
> > 
> > The gtests code needs the actual KVM/kernel headers for the API/ABI
> > details. I should have added a check for $KERNEL before building
> > gtests. If you have a different recommendation for referencing the
> > kernel sources from the kvm-unit-test directory, let me know.
> >
> 
> We've been copying the kernel files we need (see lib/linux/), or even
> just cherry-picking the defines we need from linux headers and dropping
> them into their respective kvm-unit-tests' lib/$ARCH/asm/ headers. Is
> there some reason you've opted not to do that?
> 
> On a historical note, kvm-unit-tests used to look in kernel sources for
> headers, but we decided it would be more useful for the test suite to
> remove the dependency. That was pre 'make standalone' though. We could
> start a discussion about how much we need kvm-unit-tests to _build_
> without the dependency now, as one can always build standalone tests
> on a build machine and then easily copy them to test machines.

Yeah, I'd just copy the file(s) for now.

Re: [kvm-unit-tests PATCH] kvm-unit-tests: Gtests Framework and Example Tests

[Paolo Bonzini]

On 31/01/2018 03:56, Andrew Jones wrote:
>>> (Also, doesn't compile here due to missing #include <asm/msr-index.h>
>>>  and I didn't look further.)
>>
>> My apologies. Please try "KERNEL=<kernel-srcdir> make"
>>
>> The gtests code needs the actual KVM/kernel headers for the API/ABI
>> details. I should have added a check for $KERNEL before building
>> gtests. If you have a different recommendation for referencing the
>> kernel sources from the kvm-unit-test directory, let me know.
>>
> 
> We've been copying the kernel files we need (see lib/linux/), or even
> just cherry-picking the defines we need from linux headers and dropping
> them into their respective kvm-unit-tests' lib/$ARCH/asm/ headers. Is
> there some reason you've opted not to do that?
> 
> On a historical note, kvm-unit-tests used to look in kernel sources for
> headers, but we decided it would be more useful for the test suite to
> remove the dependency. That was pre 'make standalone' though. We could
> start a discussion about how much we need kvm-unit-tests to _build_
> without the dependency now, as one can always build standalone tests
> on a build machine and then easily copy them to test machines.

There are two things to consider in this.

The first thing is separating the kvm-unit-tests "mini guest" tests from
the API tests that Ken is adding.  For the "mini guest" tests, having no
Linux dependency is a feature.  While we occasionally copy some code
because we're lazy, the code is completely freestanding and not
necessarily related to KVM actually.  It could even be compiled on
Windows or OS X if somebody cared.  API tests instead are Linux- and
KVM-specific.

The second thing is separating UAPI vs. kernel development headers for
API tests.  Here, the API tests certainly can use the installed UAPI
headers from /usr/include/linux (we _can_ copy them, but don't
necessarily _have to_), but they shouldn't use the kernel development
headers because those are reserved to kernel modules.

Paolo