* [PATCH 1/1] scripts/ci: add configuration for a Vagrant virtual machine @ 2020-11-24 8:09 Nicolas Iooss 2020-12-02 19:20 ` Petr Lautrbach 0 siblings, 1 reply; 4+ messages in thread From: Nicolas Iooss @ 2020-11-24 8:09 UTC (permalink / raw) To: selinux Using Vagrant makes reproducing and debugging CI issues easier: after "vagrant up", a test virtual machine is up and running, and ready to run "fedora-test-runner.sh". In order to make using this VM even easier, a helper script, "run-selinux-test.sh" is created inside and instructions on how to use it are documented at the beginning of Vagrantfile. Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org> --- scripts/ci/.gitignore | 1 + scripts/ci/Vagrantfile | 55 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+) create mode 100644 scripts/ci/.gitignore create mode 100644 scripts/ci/Vagrantfile diff --git a/scripts/ci/.gitignore b/scripts/ci/.gitignore new file mode 100644 index 000000000000..a977916f6583 --- /dev/null +++ b/scripts/ci/.gitignore @@ -0,0 +1 @@ +.vagrant/ diff --git a/scripts/ci/Vagrantfile b/scripts/ci/Vagrantfile new file mode 100644 index 000000000000..68a2414a8429 --- /dev/null +++ b/scripts/ci/Vagrantfile @@ -0,0 +1,55 @@ +# -*- mode: ruby -*- +# vi: set ft=ruby : +# Vagrant configuration file which creates a virtual machine that can run the +# test suite using fedora-test-runner.sh, in an environment similar to the one +# used for automated continuous integration tests (Travis-CI) +# +# To create a new virtual machine: +# +# vagrant up --provision +# +# To launch tests (for example after modifications to libsepol, libselinux... are made): +# +# vagrant rsync && echo ./run-selinux-test.sh | vagrant ssh +# +# To destroy the virtual machine (for example to start again from a clean environment): +# +# vagrant destroy + +# Create a helper script in the VM to run the testsuite as root from a clean environment +$script = <<SCRIPT +cat > /home/vagrant/run-selinux-test.sh << EOF +#/bin/sh +set -e -v + +# Clean-up a previous run. +# "make" is not installed when the machine is first bootstrapped +if command -v make > /dev/null 2>&1 ; then + sudo make -C /root/selinux clean distclean +fi +sudo rm -rf /root/selinux-testsuite + +# Run the tests +sudo /root/selinux/scripts/ci/fedora-test-runner.sh +echo 'All tests passed :)' +EOF +chmod +x /home/vagrant/run-selinux-test.sh +SCRIPT + +# All Vagrant configuration is done below. The "2" in Vagrant.configure +# configures the configuration version (we support older styles for +# backwards compatibility). Please don't change it unless you know what +# you're doing. +Vagrant.configure("2") do |config| + config.vm.box = "fedora/33-cloud-base" + config.vm.synced_folder "../..", "/root/selinux" + + config.vm.provider "virtualbox" do |v| + v.memory = 4096 + end + config.vm.provider "libvirt" do |v| + v.memory = 4096 + end + + config.vm.provision :shell, inline: $script +end -- 2.29.2 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] scripts/ci: add configuration for a Vagrant virtual machine 2020-11-24 8:09 [PATCH 1/1] scripts/ci: add configuration for a Vagrant virtual machine Nicolas Iooss @ 2020-12-02 19:20 ` Petr Lautrbach 2020-12-02 19:35 ` William Roberts 0 siblings, 1 reply; 4+ messages in thread From: Petr Lautrbach @ 2020-12-02 19:20 UTC (permalink / raw) To: Nicolas Iooss, selinux Nicolas Iooss <nicolas.iooss@m4x.org> writes: > Using Vagrant makes reproducing and debugging CI issues easier: after > "vagrant up", a test virtual machine is up and running, and ready to run > "fedora-test-runner.sh". In order to make using this VM even easier, a > helper script, "run-selinux-test.sh" is created inside and instructions > on how to use it are documented at the beginning of Vagrantfile. > > Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org> > --- > scripts/ci/.gitignore | 1 + > scripts/ci/Vagrantfile | 55 ++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 56 insertions(+) > create mode 100644 scripts/ci/.gitignore > create mode 100644 scripts/ci/Vagrantfile > > diff --git a/scripts/ci/.gitignore b/scripts/ci/.gitignore > new file mode 100644 > index 000000000000..a977916f6583 > --- /dev/null > +++ b/scripts/ci/.gitignore > @@ -0,0 +1 @@ > +.vagrant/ > diff --git a/scripts/ci/Vagrantfile b/scripts/ci/Vagrantfile > new file mode 100644 > index 000000000000..68a2414a8429 > --- /dev/null > +++ b/scripts/ci/Vagrantfile > @@ -0,0 +1,55 @@ > +# -*- mode: ruby -*- > +# vi: set ft=ruby : > +# Vagrant configuration file which creates a virtual machine that can run the > +# test suite using fedora-test-runner.sh, in an environment similar to the one > +# used for automated continuous integration tests (Travis-CI) > +# > +# To create a new virtual machine: > +# > +# vagrant up --provision > +# > +# To launch tests (for example after modifications to libsepol, libselinux... are made): > +# > +# vagrant rsync && echo ./run-selinux-test.sh | vagrant ssh > +# > +# To destroy the virtual machine (for example to start again from a clean environment): > +# > +# vagrant destroy > + > +# Create a helper script in the VM to run the testsuite as root from a clean environment > +$script = <<SCRIPT > +cat > /home/vagrant/run-selinux-test.sh << EOF > +#/bin/sh > +set -e -v > + > +# Clean-up a previous run. > +# "make" is not installed when the machine is first bootstrapped > +if command -v make > /dev/null 2>&1 ; then > + sudo make -C /root/selinux clean distclean > +fi > +sudo rm -rf /root/selinux-testsuite Would it make sense to move clean-up directly to fedora-test-runner.sh so that Vagrantfile would not need to know about test runner internals? --- a/scripts/ci/fedora-test-runner.sh +++ b/scripts/ci/fedora-test-runner.sh @@ -75,6 +75,7 @@ git log --oneline -1 # # Build and replace userspace components # +make clean distclean make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install-pywrap make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 relabel @@ -84,6 +85,7 @@ make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 relabel # first. # cd "$HOME" +rm -rf selinux-testsuite git clone --depth=1 https://github.com/SELinuxProject/selinux-testsuite.git cd selinux-testsuite > +# Run the tests > +sudo /root/selinux/scripts/ci/fedora-test-runner.sh > +echo 'All tests passed :)' > +EOF > +chmod +x /home/vagrant/run-selinux-test.sh > +SCRIPT > + > +# All Vagrant configuration is done below. The "2" in Vagrant.configure > +# configures the configuration version (we support older styles for > +# backwards compatibility). Please don't change it unless you know what > +# you're doing. > +Vagrant.configure("2") do |config| > + config.vm.box = "fedora/33-cloud-base" > + config.vm.synced_folder "../..", "/root/selinux" > + > + config.vm.provider "virtualbox" do |v| > + v.memory = 4096 > + end > + config.vm.provider "libvirt" do |v| > + v.memory = 4096 > + end > + > + config.vm.provision :shell, inline: $script > +end > -- > 2.29.2 ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] scripts/ci: add configuration for a Vagrant virtual machine 2020-12-02 19:20 ` Petr Lautrbach @ 2020-12-02 19:35 ` William Roberts 2020-12-03 8:14 ` Nicolas Iooss 0 siblings, 1 reply; 4+ messages in thread From: William Roberts @ 2020-12-02 19:35 UTC (permalink / raw) To: Petr Lautrbach; +Cc: Nicolas Iooss, SElinux list On Wed, Dec 2, 2020 at 1:27 PM Petr Lautrbach <plautrba@redhat.com> wrote: > > Nicolas Iooss <nicolas.iooss@m4x.org> writes: > > > Using Vagrant makes reproducing and debugging CI issues easier: after > > "vagrant up", a test virtual machine is up and running, and ready to run > > "fedora-test-runner.sh". In order to make using this VM even easier, a > > helper script, "run-selinux-test.sh" is created inside and instructions > > on how to use it are documented at the beginning of Vagrantfile. > > > > Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org> > > --- > > scripts/ci/.gitignore | 1 + > > scripts/ci/Vagrantfile | 55 ++++++++++++++++++++++++++++++++++++++++++ > > 2 files changed, 56 insertions(+) > > create mode 100644 scripts/ci/.gitignore > > create mode 100644 scripts/ci/Vagrantfile > > > > diff --git a/scripts/ci/.gitignore b/scripts/ci/.gitignore > > new file mode 100644 > > index 000000000000..a977916f6583 > > --- /dev/null > > +++ b/scripts/ci/.gitignore > > @@ -0,0 +1 @@ > > +.vagrant/ > > diff --git a/scripts/ci/Vagrantfile b/scripts/ci/Vagrantfile > > new file mode 100644 > > index 000000000000..68a2414a8429 > > --- /dev/null > > +++ b/scripts/ci/Vagrantfile > > @@ -0,0 +1,55 @@ > > +# -*- mode: ruby -*- > > +# vi: set ft=ruby : > > +# Vagrant configuration file which creates a virtual machine that can run the > > +# test suite using fedora-test-runner.sh, in an environment similar to the one > > +# used for automated continuous integration tests (Travis-CI) > > +# > > +# To create a new virtual machine: > > +# > > +# vagrant up --provision > > +# > > +# To launch tests (for example after modifications to libsepol, libselinux... are made): > > +# > > +# vagrant rsync && echo ./run-selinux-test.sh | vagrant ssh > > +# > > +# To destroy the virtual machine (for example to start again from a clean environment): > > +# > > +# vagrant destroy > > + > > +# Create a helper script in the VM to run the testsuite as root from a clean environment > > +$script = <<SCRIPT > > +cat > /home/vagrant/run-selinux-test.sh << EOF > > +#/bin/sh > > +set -e -v > > + > > +# Clean-up a previous run. > > +# "make" is not installed when the machine is first bootstrapped > > +if command -v make > /dev/null 2>&1 ; then > > + sudo make -C /root/selinux clean distclean > > +fi > > +sudo rm -rf /root/selinux-testsuite > > Would it make sense to move clean-up directly to fedora-test-runner.sh so > that Vagrantfile would not need to know about test runner internals? > > > --- a/scripts/ci/fedora-test-runner.sh > +++ b/scripts/ci/fedora-test-runner.sh > @@ -75,6 +75,7 @@ git log --oneline -1 > # > # Build and replace userspace components > # > +make clean distclean > make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install > make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install-pywrap > make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 relabel > @@ -84,6 +85,7 @@ make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 relabel > # first. > # > cd "$HOME" > +rm -rf selinux-testsuite > git clone --depth=1 https://github.com/SELinuxProject/selinux-testsuite.git > cd selinux-testsuite > > > > > > +# Run the tests > > +sudo /root/selinux/scripts/ci/fedora-test-runner.sh > > +echo 'All tests passed :)' > > +EOF > > +chmod +x /home/vagrant/run-selinux-test.sh > > +SCRIPT > > + > > +# All Vagrant configuration is done below. The "2" in Vagrant.configure > > +# configures the configuration version (we support older styles for > > +# backwards compatibility). Please don't change it unless you know what > > +# you're doing. > > +Vagrant.configure("2") do |config| > > + config.vm.box = "fedora/33-cloud-base" > > + config.vm.synced_folder "../..", "/root/selinux" > > + > > + config.vm.provider "virtualbox" do |v| > > + v.memory = 4096 > > + end > > + config.vm.provider "libvirt" do |v| > > + v.memory = 4096 > > + end > > + > > + config.vm.provision :shell, inline: $script > > +end > > -- > > 2.29.2 > That's pretty sweet looking. AFAICT vagrant just makes VM's easier to manipulate, is that right? Does this change any of the infrastructure we run on as we migrate away from Travis? ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] scripts/ci: add configuration for a Vagrant virtual machine 2020-12-02 19:35 ` William Roberts @ 2020-12-03 8:14 ` Nicolas Iooss 0 siblings, 0 replies; 4+ messages in thread From: Nicolas Iooss @ 2020-12-03 8:14 UTC (permalink / raw) To: William Roberts, Petr Lautrbach; +Cc: SElinux list On Wed, Dec 2, 2020 at 8:35 PM William Roberts <bill.c.roberts@gmail.com> wrote: > > On Wed, Dec 2, 2020 at 1:27 PM Petr Lautrbach <plautrba@redhat.com> wrote: > > > > Nicolas Iooss <nicolas.iooss@m4x.org> writes: > > > > > Using Vagrant makes reproducing and debugging CI issues easier: after > > > "vagrant up", a test virtual machine is up and running, and ready to run > > > "fedora-test-runner.sh". In order to make using this VM even easier, a > > > helper script, "run-selinux-test.sh" is created inside and instructions > > > on how to use it are documented at the beginning of Vagrantfile. > > > > > > Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org> > > > --- > > > scripts/ci/.gitignore | 1 + > > > scripts/ci/Vagrantfile | 55 ++++++++++++++++++++++++++++++++++++++++++ > > > 2 files changed, 56 insertions(+) > > > create mode 100644 scripts/ci/.gitignore > > > create mode 100644 scripts/ci/Vagrantfile > > > > > > diff --git a/scripts/ci/.gitignore b/scripts/ci/.gitignore > > > new file mode 100644 > > > index 000000000000..a977916f6583 > > > --- /dev/null > > > +++ b/scripts/ci/.gitignore > > > @@ -0,0 +1 @@ > > > +.vagrant/ > > > diff --git a/scripts/ci/Vagrantfile b/scripts/ci/Vagrantfile > > > new file mode 100644 > > > index 000000000000..68a2414a8429 > > > --- /dev/null > > > +++ b/scripts/ci/Vagrantfile > > > @@ -0,0 +1,55 @@ > > > +# -*- mode: ruby -*- > > > +# vi: set ft=ruby : > > > +# Vagrant configuration file which creates a virtual machine that can run the > > > +# test suite using fedora-test-runner.sh, in an environment similar to the one > > > +# used for automated continuous integration tests (Travis-CI) > > > +# > > > +# To create a new virtual machine: > > > +# > > > +# vagrant up --provision > > > +# > > > +# To launch tests (for example after modifications to libsepol, libselinux... are made): > > > +# > > > +# vagrant rsync && echo ./run-selinux-test.sh | vagrant ssh > > > +# > > > +# To destroy the virtual machine (for example to start again from a clean environment): > > > +# > > > +# vagrant destroy > > > + > > > +# Create a helper script in the VM to run the testsuite as root from a clean environment > > > +$script = <<SCRIPT > > > +cat > /home/vagrant/run-selinux-test.sh << EOF > > > +#/bin/sh > > > +set -e -v > > > + > > > +# Clean-up a previous run. > > > +# "make" is not installed when the machine is first bootstrapped > > > +if command -v make > /dev/null 2>&1 ; then > > > + sudo make -C /root/selinux clean distclean > > > +fi > > > +sudo rm -rf /root/selinux-testsuite > > > > Would it make sense to move clean-up directly to fedora-test-runner.sh so > > that Vagrantfile would not need to know about test runner internals? Yes, I will send a v2 with this. Thanks for this suggestion! > > > > --- a/scripts/ci/fedora-test-runner.sh > > +++ b/scripts/ci/fedora-test-runner.sh > > @@ -75,6 +75,7 @@ git log --oneline -1 > > # > > # Build and replace userspace components > > # > > +make clean distclean > > make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install > > make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install-pywrap > > make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 relabel > > @@ -84,6 +85,7 @@ make -j"$(nproc)" LIBDIR=/usr/lib64 SHLIBDIR=/lib64 relabel > > # first. > > # > > cd "$HOME" > > +rm -rf selinux-testsuite > > git clone --depth=1 https://github.com/SELinuxProject/selinux-testsuite.git > > cd selinux-testsuite > > > > > > > > > > > +# Run the tests > > > +sudo /root/selinux/scripts/ci/fedora-test-runner.sh > > > +echo 'All tests passed :)' > > > +EOF > > > +chmod +x /home/vagrant/run-selinux-test.sh > > > +SCRIPT > > > + > > > +# All Vagrant configuration is done below. The "2" in Vagrant.configure > > > +# configures the configuration version (we support older styles for > > > +# backwards compatibility). Please don't change it unless you know what > > > +# you're doing. > > > +Vagrant.configure("2") do |config| > > > + config.vm.box = "fedora/33-cloud-base" > > > + config.vm.synced_folder "../..", "/root/selinux" > > > + > > > + config.vm.provider "virtualbox" do |v| > > > + v.memory = 4096 > > > + end > > > + config.vm.provider "libvirt" do |v| > > > + v.memory = 4096 > > > + end > > > + > > > + config.vm.provision :shell, inline: $script > > > +end > > > -- > > > 2.29.2 > > > > That's pretty sweet looking. AFAICT vagrant just makes VM's easier to > manipulate, is that right? > Does this change any of the infrastructure we run on as we migrate > away from Travis? Yes, Vagrant is a tool that makes using virtual machines on a local computer much easier. It is a powerful tool, which is only used here in a very simple configuration with a single VM running a system as close as possible as the CI. My aim here was to "make reproducing and debugging CI issues easier", because when something goes wrong in the CI, it is much easier to find out what went wrong on a local computer than trying to tweak things and submit new CI jobs. This does not replace Travis, because this is not run for every push to the git repository, but this shows that fedora-test-runner.sh is generic enough to be run in a generic Fedora VM that could be provided by other CI services (though I have not searched if any of them exists for free). Nicolas ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-12-03 8:15 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-11-24 8:09 [PATCH 1/1] scripts/ci: add configuration for a Vagrant virtual machine Nicolas Iooss 2020-12-02 19:20 ` Petr Lautrbach 2020-12-02 19:35 ` William Roberts 2020-12-03 8:14 ` Nicolas Iooss
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.