* [kvm-unit-tests PATCH] Makefile: add portable mode @ 2025-01-05 17:57 Jon Kohler 2025-01-13 13:07 ` Alexandru Elisei 0 siblings, 1 reply; 6+ messages in thread From: Jon Kohler @ 2025-01-05 17:57 UTC (permalink / raw) To: Jon Kohler, Andrew Jones, Thomas Huth, Nicholas Piggin, Eric Auger, Nina Schoetterl-Glausch, kvm Add a 'portable' mode that packages all relevant flat files and helper scripts into a tarball named 'kut-portable.tar.gz'. This mode is useful for compiling tests on one machine and running them on another without needing to clone the entire repository. It allows the runner scripts and unit test configurations to remain local to the machine under test. Signed-off-by: Jon Kohler <jon@nutanix.com> --- .gitignore | 2 ++ Makefile | 17 +++++++++++++++++ 2 files changed, 19 insertions(+) diff --git a/.gitignore b/.gitignore index 2168e013..643220f8 100644 --- a/.gitignore +++ b/.gitignore @@ -18,6 +18,8 @@ cscope.* /lib/config.h /config.mak /*-run +/kut-portable +kut-portable.tar.gz /msr.out /tests /build-head diff --git a/Makefile b/Makefile index 7471f728..c6333c1a 100644 --- a/Makefile +++ b/Makefile @@ -125,6 +125,23 @@ all: directories $(shell (cd $(SRCDIR) && git rev-parse --verify --short=8 HEAD) standalone: all @scripts/mkstandalone.sh +portable: all + rm -f kut-portable.tar.gz + rm -rf kut-portable + mkdir -p kut-portable/scripts/s390x + mkdir -p kut-portable/$(TEST_DIR) + cp build-head kut-portable + cp errata.txt kut-portable + cp config.mak kut-portable + sed -i '/^ERRATATXT/cERRATATXT=errata.txt' kut-portable/config.mak + cp run_tests.sh kut-portable + cp -r scripts/* kut-portable/scripts + cp $(TEST_DIR)-run kut-portable + cp $(TEST_DIR)/*.flat kut-portable/$(TEST_DIR) + cp $(TEST_DIR)/unittests.cfg kut-portable/$(TEST_DIR) + cp $(TEST_DIR)/run kut-portable/$(TEST_DIR) + tar -czf kut-portable.tar.gz kut-portable + install: standalone mkdir -p $(DESTDIR) install tests/* $(DESTDIR) -- 2.43.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [kvm-unit-tests PATCH] Makefile: add portable mode 2025-01-05 17:57 [kvm-unit-tests PATCH] Makefile: add portable mode Jon Kohler @ 2025-01-13 13:07 ` Alexandru Elisei 2025-01-13 14:49 ` Jon Kohler 0 siblings, 1 reply; 6+ messages in thread From: Alexandru Elisei @ 2025-01-13 13:07 UTC (permalink / raw) To: Jon Kohler Cc: Andrew Jones, Thomas Huth, Nicholas Piggin, Eric Auger, Nina Schoetterl-Glausch, kvm Hi, On Sun, Jan 05, 2025 at 10:57:23AM -0700, Jon Kohler wrote: > Add a 'portable' mode that packages all relevant flat files and helper > scripts into a tarball named 'kut-portable.tar.gz'. > > This mode is useful for compiling tests on one machine and running them > on another without needing to clone the entire repository. It allows > the runner scripts and unit test configurations to remain local to the > machine under test. Have you tried make standalone? You can then copy the tests directory, or even a particular test. Thanks, Alex > > Signed-off-by: Jon Kohler <jon@nutanix.com> > > --- > .gitignore | 2 ++ > Makefile | 17 +++++++++++++++++ > 2 files changed, 19 insertions(+) > > diff --git a/.gitignore b/.gitignore > index 2168e013..643220f8 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -18,6 +18,8 @@ cscope.* > /lib/config.h > /config.mak > /*-run > +/kut-portable > +kut-portable.tar.gz > /msr.out > /tests > /build-head > diff --git a/Makefile b/Makefile > index 7471f728..c6333c1a 100644 > --- a/Makefile > +++ b/Makefile > @@ -125,6 +125,23 @@ all: directories $(shell (cd $(SRCDIR) && git rev-parse --verify --short=8 HEAD) > standalone: all > @scripts/mkstandalone.sh > > +portable: all > + rm -f kut-portable.tar.gz > + rm -rf kut-portable > + mkdir -p kut-portable/scripts/s390x > + mkdir -p kut-portable/$(TEST_DIR) > + cp build-head kut-portable > + cp errata.txt kut-portable > + cp config.mak kut-portable > + sed -i '/^ERRATATXT/cERRATATXT=errata.txt' kut-portable/config.mak > + cp run_tests.sh kut-portable > + cp -r scripts/* kut-portable/scripts > + cp $(TEST_DIR)-run kut-portable > + cp $(TEST_DIR)/*.flat kut-portable/$(TEST_DIR) > + cp $(TEST_DIR)/unittests.cfg kut-portable/$(TEST_DIR) > + cp $(TEST_DIR)/run kut-portable/$(TEST_DIR) > + tar -czf kut-portable.tar.gz kut-portable > + > install: standalone > mkdir -p $(DESTDIR) > install tests/* $(DESTDIR) > -- > 2.43.0 > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [kvm-unit-tests PATCH] Makefile: add portable mode 2025-01-13 13:07 ` Alexandru Elisei @ 2025-01-13 14:49 ` Jon Kohler 2025-01-13 15:36 ` Andrew Jones 0 siblings, 1 reply; 6+ messages in thread From: Jon Kohler @ 2025-01-13 14:49 UTC (permalink / raw) To: Alexandru Elisei Cc: Andrew Jones, Thomas Huth, Nicholas Piggin, Eric Auger, Nina Schoetterl-Glausch, kvm@vger.kernel.org > On Jan 13, 2025, at 8:07 AM, Alexandru Elisei <alexandru.elisei@arm.com> wrote: > > !-------------------------------------------------------------------| > CAUTION: External Email > > |-------------------------------------------------------------------! > > Hi, > > On Sun, Jan 05, 2025 at 10:57:23AM -0700, Jon Kohler wrote: >> Add a 'portable' mode that packages all relevant flat files and helper >> scripts into a tarball named 'kut-portable.tar.gz'. >> >> This mode is useful for compiling tests on one machine and running them >> on another without needing to clone the entire repository. It allows >> the runner scripts and unit test configurations to remain local to the >> machine under test. > > Have you tried make standalone? You can then copy the tests directory, or even a > particular test. Yes, standalone does not work when copying tests from one host to another. The use case for portable mode is to be able to compile within one environment and test in completely separate environment. I was not able to accomplish that with standalone mode by itself. > > Thanks, > Alex > >> >> Signed-off-by: Jon Kohler <jon@nutanix.com> >> >> --- >> .gitignore | 2 ++ >> Makefile | 17 +++++++++++++++++ >> 2 files changed, 19 insertions(+) >> >> diff --git a/.gitignore b/.gitignore >> index 2168e013..643220f8 100644 >> --- a/.gitignore >> +++ b/.gitignore >> @@ -18,6 +18,8 @@ cscope.* >> /lib/config.h >> /config.mak >> /*-run >> +/kut-portable >> +kut-portable.tar.gz >> /msr.out >> /tests >> /build-head >> diff --git a/Makefile b/Makefile >> index 7471f728..c6333c1a 100644 >> --- a/Makefile >> +++ b/Makefile >> @@ -125,6 +125,23 @@ all: directories $(shell (cd $(SRCDIR) && git rev-parse --verify --short=8 HEAD) >> standalone: all >> @scripts/mkstandalone.sh >> >> +portable: all >> + rm -f kut-portable.tar.gz >> + rm -rf kut-portable >> + mkdir -p kut-portable/scripts/s390x >> + mkdir -p kut-portable/$(TEST_DIR) >> + cp build-head kut-portable >> + cp errata.txt kut-portable >> + cp config.mak kut-portable >> + sed -i '/^ERRATATXT/cERRATATXT=errata.txt' kut-portable/config.mak >> + cp run_tests.sh kut-portable >> + cp -r scripts/* kut-portable/scripts >> + cp $(TEST_DIR)-run kut-portable >> + cp $(TEST_DIR)/*.flat kut-portable/$(TEST_DIR) >> + cp $(TEST_DIR)/unittests.cfg kut-portable/$(TEST_DIR) >> + cp $(TEST_DIR)/run kut-portable/$(TEST_DIR) >> + tar -czf kut-portable.tar.gz kut-portable >> + >> install: standalone >> mkdir -p $(DESTDIR) >> install tests/* $(DESTDIR) >> -- >> 2.43.0 >> >> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [kvm-unit-tests PATCH] Makefile: add portable mode 2025-01-13 14:49 ` Jon Kohler @ 2025-01-13 15:36 ` Andrew Jones 2025-01-13 20:53 ` Jon Kohler 0 siblings, 1 reply; 6+ messages in thread From: Andrew Jones @ 2025-01-13 15:36 UTC (permalink / raw) To: Jon Kohler Cc: Alexandru Elisei, Thomas Huth, Nicholas Piggin, Eric Auger, Nina Schoetterl-Glausch, kvm@vger.kernel.org On Mon, Jan 13, 2025 at 02:49:11PM +0000, Jon Kohler wrote: > > > > On Jan 13, 2025, at 8:07 AM, Alexandru Elisei <alexandru.elisei@arm.com> wrote: > > > > !-------------------------------------------------------------------| > > CAUTION: External Email > > > > |-------------------------------------------------------------------! > > > > Hi, > > > > On Sun, Jan 05, 2025 at 10:57:23AM -0700, Jon Kohler wrote: > >> Add a 'portable' mode that packages all relevant flat files and helper > >> scripts into a tarball named 'kut-portable.tar.gz'. > >> > >> This mode is useful for compiling tests on one machine and running them > >> on another without needing to clone the entire repository. It allows > >> the runner scripts and unit test configurations to remain local to the > >> machine under test. > > > > Have you tried make standalone? You can then copy the tests directory, or even a > > particular test. > > Yes, standalone does not work when copying tests from one host to another. The > use case for portable mode is to be able to compile within one environment and > test in completely separate environment. I was not able to accomplish that with > standalone mode by itself. > standalone scripts should be portable. If they're missing something, then we should fix that. Also 'make install' should include everything necessary, otherwise it should be fixed. Then, we could consider adding another target like 'make package' which would do 'make install' to a temporary directory and tar/gzip or whatever the installation into a package. Thanks, drew ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [kvm-unit-tests PATCH] Makefile: add portable mode 2025-01-13 15:36 ` Andrew Jones @ 2025-01-13 20:53 ` Jon Kohler 2025-01-14 9:20 ` Andrew Jones 0 siblings, 1 reply; 6+ messages in thread From: Jon Kohler @ 2025-01-13 20:53 UTC (permalink / raw) To: Andrew Jones Cc: Alexandru Elisei, Thomas Huth, Nicholas Piggin, Eric Auger, Nina Schoetterl-Glausch, kvm@vger.kernel.org > On Jan 13, 2025, at 10:36 AM, Andrew Jones <andrew.jones@linux.dev> wrote: > > !-------------------------------------------------------------------| > CAUTION: External Email > > |-------------------------------------------------------------------! > > On Mon, Jan 13, 2025 at 02:49:11PM +0000, Jon Kohler wrote: >> >> >>> On Jan 13, 2025, at 8:07 AM, Alexandru Elisei <alexandru.elisei@arm.com> wrote: >>> >>> !-------------------------------------------------------------------| >>> CAUTION: External Email >>> >>> |-------------------------------------------------------------------! >>> >>> Hi, >>> >>> On Sun, Jan 05, 2025 at 10:57:23AM -0700, Jon Kohler wrote: >>>> Add a 'portable' mode that packages all relevant flat files and helper >>>> scripts into a tarball named 'kut-portable.tar.gz'. >>>> >>>> This mode is useful for compiling tests on one machine and running them >>>> on another without needing to clone the entire repository. It allows >>>> the runner scripts and unit test configurations to remain local to the >>>> machine under test. >>> >>> Have you tried make standalone? You can then copy the tests directory, or even a >>> particular test. >> >> Yes, standalone does not work when copying tests from one host to another. The >> use case for portable mode is to be able to compile within one environment and >> test in completely separate environment. I was not able to accomplish that with >> standalone mode by itself. >> > > standalone scripts should be portable. If they're missing something, then > we should fix that. Also 'make install' should include everything > necessary, otherwise it should be fixed. Then, we could consider adding > another target like 'make package' which would do 'make install' to a > temporary directory and tar/gzip or whatever the installation into a > package. Thanks, Drew. The standalone scripts are not portable in my experience, as in I can not just pick them up, copy them as is, and put them on another host with different directory layouts, etc (and importantly, no kvm-unit-tests repo whatsoever). The make package idea is effectively what I’m doing here, but it happens to use the word portable instead of package (and not using make install to do the data movement). That bit is important: The biggest tidbit is that things like the errata path is hard coded, so in my “make portable” it fixes those things to be not hard coded, and all is well after files are tar’d up and then untar'd later. Also, in standalone mode, you’d be missing the runner scripts, etc, so it makes it harder to tweak and iterate locally on the system under test. Happy to take guidance if I’m missing something painfully obvious with the existing standalone setup (I felt this way when I started looking at this!) as it doesn’t do what I’d think it would do. Thanks again, Jon > Thanks, > drew ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [kvm-unit-tests PATCH] Makefile: add portable mode 2025-01-13 20:53 ` Jon Kohler @ 2025-01-14 9:20 ` Andrew Jones 0 siblings, 0 replies; 6+ messages in thread From: Andrew Jones @ 2025-01-14 9:20 UTC (permalink / raw) To: Jon Kohler Cc: Alexandru Elisei, Thomas Huth, Nicholas Piggin, Eric Auger, Nina Schoetterl-Glausch, kvm@vger.kernel.org On Mon, Jan 13, 2025 at 08:53:40PM +0000, Jon Kohler wrote: > > > > On Jan 13, 2025, at 10:36 AM, Andrew Jones <andrew.jones@linux.dev> wrote: > > > > !-------------------------------------------------------------------| > > CAUTION: External Email > > > > |-------------------------------------------------------------------! > > > > On Mon, Jan 13, 2025 at 02:49:11PM +0000, Jon Kohler wrote: > >> > >> > >>> On Jan 13, 2025, at 8:07 AM, Alexandru Elisei <alexandru.elisei@arm.com> wrote: > >>> > >>> !-------------------------------------------------------------------| > >>> CAUTION: External Email > >>> > >>> |-------------------------------------------------------------------! > >>> > >>> Hi, > >>> > >>> On Sun, Jan 05, 2025 at 10:57:23AM -0700, Jon Kohler wrote: > >>>> Add a 'portable' mode that packages all relevant flat files and helper > >>>> scripts into a tarball named 'kut-portable.tar.gz'. > >>>> > >>>> This mode is useful for compiling tests on one machine and running them > >>>> on another without needing to clone the entire repository. It allows > >>>> the runner scripts and unit test configurations to remain local to the > >>>> machine under test. > >>> > >>> Have you tried make standalone? You can then copy the tests directory, or even a > >>> particular test. > >> > >> Yes, standalone does not work when copying tests from one host to another. The > >> use case for portable mode is to be able to compile within one environment and > >> test in completely separate environment. I was not able to accomplish that with > >> standalone mode by itself. > >> > > > > standalone scripts should be portable. If they're missing something, then > > we should fix that. Also 'make install' should include everything > > necessary, otherwise it should be fixed. Then, we could consider adding > > another target like 'make package' which would do 'make install' to a > > temporary directory and tar/gzip or whatever the installation into a > > package. > > Thanks, Drew. The standalone scripts are not portable in my experience, as > in I can not just pick them up, copy them as is, and put them on another host > with different directory layouts, etc (and importantly, no kvm-unit-tests repo > whatsoever). They should work without a kvm-unit-tests repo (they used to anyway). If they're not working when moved to a host that doesn't have anything except bash and qemu, then we should fix them. Which architecture are you mainly testing? > > The make package idea is effectively what I’m doing here, but it happens to > use the word portable instead of package (and not using make install to do > the data movement). I know, but I'm suggesting that we do use 'make install' rather than introduce an independent target which should be 'make install' plus packaging. We don't want to maintain multiple independent targets which do almost the same thing. > > That bit is important: > The biggest tidbit is that things like the errata path is hard coded, so > in my “make portable” it fixes those things to be not hard coded, and all is well > after files are tar’d up and then untar'd later. So let's fix 'make install' to make sure nothing installed has hard coded paths (which standalone should do, since it should only depend on /tmp) > > Also, in standalone mode, you’d be missing the runner scripts, etc, so it > makes it harder to tweak and iterate locally on the system under test. standalone scripts don't need the run_tests.sh since they include everything from unittests.cfg already. It probably would make sense to provide a script that allows running all or a subset of tests at once, which would just be a for-loop, e.g. run_subset() { for t in ${tests}/${1} ${tests}/${1}_*; do [ -e $t ] && $t; done; } Running tests with different parameters than what are specified in the unittests.cfg file isn't currently possible, unless they're also configurable with environment variables like MACHINE, ACCEL, TIMEOUT, and MAX_SMP. If more configuration changes are necessary, then a make target for packaging $TEST_DIR/run and the .flat files is probably the right way to go, but it still isn't clear to me if that's what you're trying to do. > > Happy to take guidance if I’m missing something painfully obvious with the > existing standalone setup (I felt this way when I started looking at this!) as it > doesn’t do what I’d think it would do. There's a good chance that standalone hasn't been maintained along with the rest of the scripts, since we don't currently have a CI test for it. But, unless you need to modify test parameters, then we should try to fix it (and add CI for it). Thanks, drew ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-01-14 9:21 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-01-05 17:57 [kvm-unit-tests PATCH] Makefile: add portable mode Jon Kohler 2025-01-13 13:07 ` Alexandru Elisei 2025-01-13 14:49 ` Jon Kohler 2025-01-13 15:36 ` Andrew Jones 2025-01-13 20:53 ` Jon Kohler 2025-01-14 9:20 ` Andrew Jones
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox