From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1CCF6149C6F for ; Mon, 28 Jul 2025 01:14:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753665281; cv=none; b=kmJU9kJii3c15ycLpVTVG+fCxrca0Vt2BKdbSyg1xL5NlOKyqBtpRDmNfFCgEuedT+T2iccRo8a7acc2QPWYi0m4TSdVD/vPc/GBH4Zr1aRAepgJi6r6riwViz/dpQCzyw7cT87YgtG9Io2CZF/J4nQ9LmlYXyJVcPZyln7CH9g= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753665281; c=relaxed/simple; bh=ZMpX1nYNcCbf46+M6dkQNYI7rzUmqJXFTaD0hVr3Z8Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VHsmnHStVMd0QVzi5QwYHpkUiS9OThHR314zrfHyq4UFJDOLdvgb3Gaq/Z+IWK7hpKnm0dOlL1SBecteyfdIGQ/x8TzoGdj7zwBMGtdakzquwjJCohZqok9Opoza4YTEA7llx3tlceL/15ejic9rQb6231lHyRAGPa7GsRp15Dg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=u4rfYNKl; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="u4rfYNKl" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=EVVutVTAfRHk6AdmcxQq9spipxm16kbGBhVJLXx9YQU=; b=u4rfYNKlq5p4hFOjKJuDHM4Ewb TrkUPrtV3EncHTkmnMMBjwHNjzaENqET4WkweoIIBFge55S06PS5JUJ+oSI7MqGvW6kynkbtPWNmD f6RAEK6dHEmbxaiHodwIrLA8tHDRmjvjES7zbycRm7aq/yhepUsx67kGtWaaRJc2z5iNw/Hxt4/aF OZFpy60G6nUHNiMcQKnrg/hrkVO5nYZ8WSEEKdkyejz+lUcInevyE49ud+3LeVoAouuW37sMWoceG sk4Spdax2643dT9PLiFVjFUA7dg7dsYQ1oBw8/mqwpsXN791BO8p9NzfoPlhclNcDsSlvO4I5qE9d FoPSLfXg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1ugCRn-0000000DPj3-2uv2; Mon, 28 Jul 2025 01:14:35 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: Luis Chamberlain Subject: [PATCH v2 06/33] docs: remove Vagrant-specific documentation files Date: Sun, 27 Jul 2025 18:14:06 -0700 Message-ID: <20250728011434.3197091-7-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250728011434.3197091-1-mcgrof@kernel.org> References: <20250728011434.3197091-1-mcgrof@kernel.org> Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain Remove all Vagrant-specific documentation as Vagrant support has been removed from kdevops. This includes: - kdevops-vagrant.md - custom-vagrant-boxes.md - why-vagrant.md - the-gen-nodes-ansible-role-vagrant.md Generated-by: Claude AI Suggested-by: Chuck Lever Signed-off-by: Luis Chamberlain --- docs/custom-vagrant-boxes.md | 581 --------------------- docs/kdevops-vagrant.md | 130 ----- docs/the-gen-nodes-ansible-role-vagrant.md | 78 --- docs/why-vagrant.md | 66 --- 4 files changed, 855 deletions(-) delete mode 100644 docs/custom-vagrant-boxes.md delete mode 100644 docs/kdevops-vagrant.md delete mode 100644 docs/the-gen-nodes-ansible-role-vagrant.md delete mode 100644 docs/why-vagrant.md diff --git a/docs/custom-vagrant-boxes.md b/docs/custom-vagrant-boxes.md deleted file mode 100644 index aa73a426..00000000 --- a/docs/custom-vagrant-boxes.md +++ /dev/null @@ -1,581 +0,0 @@ -# Creating your own custom Vagrant boxes or qcow2 virtual images - -You can create your own custom Vagrant boxes if the publicly available -Vagrant boxes do not suit your needs. We document how to do this here. -One reason might be you are an Enterprise Linux distribution and don't -have public Vagrant boxes for older releases you might still support -but want the benefit of having Vagrant boxes to work with kdevops -kernel-ci. Another reason might be you just cannot legally share you -images, for one reason or another. Another reason is you may have some -new technology which is not yet easily available on distribution kernels -and want to enable folks to test technology on some development subsystem -or linux-nxt. - -## Re-using an existing box for development - -If you are doing Linux kernel development you may want to just enable -a QA or other developers to quickly test a built kernel for you. -If you don't want to do a full fresh install of a distribution you can -opt to re-use a distribution Vagrant box and just augment it with a -custom kernel build. This section documents how to do that with a demo -of a successful box built using this technique. - -This is the lazy developer approach to customizing a Vagrant box for Linux -kernel development. This involves four steps: - - * 1) One is getting your kernel binary and modules - * 2) The GRUB configuration stuff right. - * 3) Edit the Vagrantfile to remove stupid stuff - * 4) Building the tarball - -We break this down below. - -### Getting your kernel over - -From a libvirt perspective Vagrant boxes are just compressed tarballs -with a qcow2 file. So to hack on one first download the box. So for -example if we visit the [debian/testing64](https://app.vagrantup.com/debian/boxes/testing64) -page there you will see a [libvirt download URL](https://app.vagrantup.com/debian/boxes/testing64/versions/20220626.1/providers/libvirt.box) -with the box file. - -So we do: - -```bash -wget https://app.vagrantup.com/debian/boxes/testing64/versions/20220626.1/providers/libvirt.box - -sha1sum libvirt.box -06b07c0d0b78df5369d9ed35eaf39098c1ec7846 libvirt.box - -file libvirt.box -libvirt.box: gzip compressed data, from Unix, original size modulo 2^32 1197363200 -``` - -The file can be decompressed as a regular tarball. Since it has no -directory in it you want to copy the file to another directory and uncompress -there, otherwise it will clutter your current directory: - -```bash -mkdir box-dev -cp libvirt.box box-dev -cd box-dev -tar zxvf libvirt.box -``` - -So you will see 3 files, one just a qcow2 file the other expresses -how big of a drive was used to create this qcow2 file along with a -provider, and the Vagrantfile defines how to initialize this qcow2 file -with libvirt: - -```bash -ls -1 - -box.img -metadata.json -Vagrantfile - -file box.img -box.img: QEMU QCOW2 Image (v3), 21474836480 bytes - -cat metadata.json -{ - "provider" : "libvirt", - "format" : "qcow2", - "virtual_size" : 20 -} - -cat Vagrantfile -# -*- mode: ruby -*- -# vi: set ft=ruby : - -Vagrant.configure("2") do |config| - - config.vm.post_up_message = "Vanilla Debian box. See https://app.vagrantup.com/debian for help and bug reports" - - # workaround for #837992 - # use nfsv4 mode by default since rpcbind is not available on startup - # we need to force tcp because udp is not available for nfsv4 - config.vm.synced_folder ".", "/vagrant", type: "nfs", nfs_version: "4", nfs_udp: false - - # Options for libvirt vagrant provider. - config.vm.provider :libvirt do |libvirt| - - # A hypervisor name to access. Different drivers can be specified, but - # this version of provider creates KVM machines only. Some examples of - # drivers are kvm (QEMU hardware accelerated), qemu (QEMU emulated), - # xen (Xen hypervisor), lxc (Linux Containers), - # esx (VMware ESX), vmwarews (VMware Workstation) and more. Refer to - # documentation for available drivers (http://libvirt.org/drivers.html). - libvirt.driver = "kvm" - - # The name of the server, where libvirtd is running. - # libvirt.host = "localhost" - - # If use SSH tunnel to connect to Libvirt. - libvirt.connect_via_ssh = false - - # The username and password to access Libvirt. Password is not used when - # connecting via SSH. - libvirt.username = "root" - #libvirt.password = "secret" - - # Libvirt storage pool name, where box image and instance snapshots will - # be stored. - libvirt.storage_pool_name = "default" - - # Set a prefix for the machines that's different than the project dir name. - #libvirt.default_prefix = '' - end -end -``` - -And so to hack on the qcow file we just use nbd: - -```bash -sudo qemu-nbd --connect=/dev/nbd0 box.img -mkdir debian-testing-root-vagrant -sudo mount /dev/nbd0p1 ./vanilla-debian-zns/debian-testing-root-vagrant -``` - -When you finish just do: - -```bash -sudo modprobe nbd max_part=8 -sudo umount debian-testing-root-vagrant -sudo qemu-nbd --disconnect /dev/nbd0 -``` - -It is important to run the disconnect command before copying the box file as -backups or using it for anything. - -What I do then is use a kdevops linux-next guest to compile linux-next or -whatever I want, and then I use a two step process. One to scp the modules -directory locally, and the respective `/boot/*$(uname -r)*` files over to -a new directory and then install these on the target. Then you need to configure -the GRUB console so `sudo virsh console ` works and you also want to -update the GRUB menu. That is covered in the next subsection. - -### Getting your kernel over - -OK now you just need to update the `/etc/default/grub` file and also the -`/boot/grub/grub.cfg` file. Editing `/etc/default/grub` is easy you can -just run your editor on the mounted partition for the file and ensure -you have these entries: - -``` -GRUB_DEFAULT=0 -GRUB_TIMEOUT=5 -GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` -GRUB_CMDLINE_LINUX_DEFAULT="net.ifnames=0 biosdevname=0" -GRUB_CMDLINE_LINUX="console=tty0 console=tty1 console=ttyS0,115200n8" - -GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --parity=no --stop=1" -GRUB_TERMINAL="console serial" -GRUB_DISABLE_SUBMENU=y -``` - -The last part of this file helps ensure you can get to pick a different -kernel at boot through the GRUB prompt using the serial console -(`virsh console`), however sadly it seems the GRUB version on the Debian -testing as of today doesn't work with this yet, so this step could be enhanced -to enable more flexibility to the user from the start. Until this is fixed then -developers have to do the work manually to perhaps update GRUB to get this -fixed. - -The last step is then to update the `/boot/grub/grub.cfg` file. -To do this, I just have two guests running: - - * dev: some development system where I compile and install some kernel - * baseline: a fresh Debian testing guest just brought up with Vagrant - -And then I scp to it the kernel / modules from dev over, run update-grub -and copy its grub.cfg file over. Something like the following: - -```bash -mkdir -p tmp/boot -scp -r dev:/lib/modules/5.19.0-rc4-next-20220629/ tmp/ -scp -r dev:/boot/*5.19.0-rc4-next-20220629* tmp/ - -scp -r tmp/boot/* baseline:/boot/ -scp -r tmp-provision-dir/5.19.0-rc4-next-20220629 baseline:/lib/modules/ -ssh baseline sudo update-grub -scp baseline:/boot/grub/grub.cfg tmp -``` - -OK so finally we can copy that grub.cfg to the mounted nbd partition and -hope that works. - -### Editing your Vagrantfile to remove stupid stuff - -By default Vagrant boxes enable sharing your directory to the guest -through NFS. From a Linux kernel development perspective this is just -lunacy. And so I like to disable it. By default then Debian uses this -for its sync thing - -``` - config.vm.synced_folder ".", "/vagrant", type: "nfs", nfs_version: "4", nfs_udp: false -``` - -Replace this with the more sane which disables this: - -``` - config.vm.synced_folder './', '/vagrant', type: '9p', disabled: true, accessmode: "mapped", mount: false -``` - -You may also want to edit the `config.vm.post_up_message` with whatever. - -``` - config.vm.post_up_message = "this is kernel build for send bug reports to ignore@ignore.org" -``` - -### Example network name resolution - -Most distros use udev for consistent names for networking interfaces. If -you just want to get a box out which will work fast the best way is to just -append to your GRUB kernel parameter: - -``` -GRUB_CMDLINE_LINUX_DEFAULT="net.ifnames=0 biosdevname=0" -``` - -### Example network DHCP fix - -On a basic debian console install, so where no Network Manager is installed, -you want to just have something simply like this, after testing that the -default network name that comes up is eth0: - -``` -# cat /etc/network/interfaces -# interfaces(5) file used by ifup(8) and ifdown(8) -# Include files from /etc/network/interfaces.d: -source-directory /etc/network/interfaces.d - -# The loopback network interface -auto lo -iface lo inet loopback - -# The primary network interface -allow-hotplug eth0 -iface eth0 inet dhcp -``` - -### Ensuring ssh works - -You will want to ensure the user ~vagrant has a .ssh/ directory with -chmod 700 permissions and the vagrant ssh key installed by default. -When vagrant detects this a new one random one is replaced. - -https://github.com/hashicorp/vagrant/tree/main/keys - -So just do: - -```bash -mkdir .ssh -chmod 700 .ssh -echo ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key >> authorized_keys -``` - -### Creating the Vagrant new box file tarball - -To create the box file you just tar it up. Assuming you want maximum -compression: - -```bash -tar -cvf ../mcgrof-20220629.box box.img metadata.json Vagrantfile -I "gzip --best" -``` - -You can now upload this box on the Vagrant cloud and use it with the -nice shorthands provided. - -## A fresh install - -To try to save the most amount of space you want to do a fresh install. -This section documents how to do that. - -First you'd install a guest using virt-install, an example script which -you can extend to your own needs is the -[virt-install-demo.sh](docs/virt-install-demo.sh). -There are a a series of adjustments that needed to be made for guests -to work for older kernel releases, should you need that. This is all -documented below. - -But below is an example simple script - - -```bash -virt-install --virt-type kvm \ - --name lio \ - --arch x86_64 \ - --cpu host \ - --location /home/mcgrof/debian-testing-amd64-netinst.iso \ - --os-variant debiantesting \ - --memory 8192 --vcpus 8 \ - --disk pool=data3-xfs,size=20,bus=virtio,format=qcow2 \ - --graphics none \ - --network network=vagrant-libvirt-private,model=virtio \ - --console pty,target_type=serial \ - --debug \ - --extra-args "console=ttyS0" -``` - -## Extending the Vagrant box definition - -A Vagrant box is essentially a tarball (gzip, xc are both supported) with a -qcow2 image and a small metadata file explaining how large the drive for the -guest is. The Vagrant box also ensures that the guest brought up will also -work on any new system, and so a few things need to be done to ensure for -instance that the network interface will get a DHCP address successfully, and -that you can ssh into the system. So a way to deal with moving guests around -the `cloud` are needed, an example is avoiding -`/etc/udev/rules.d/70-persistent-net.rules` upon first boot on some SLE -systems. - -Standard userspace development needs are typically met with the above -requirements. Kernel development however, have more more needs. Extensions -can be made by you so that the custom Vagrant boxes you build extend -what is promised by a Vagrant box, with other things which kernel developers -would typically prefer to have set on the system. This can save time -on bringup. Kdevops does a slew of things to help with this for you, like -setting up the serial console, but if Vagrant boxes already have these things -done on them that is a step which could be skipped. - -Below we provide a recipe of items which can be done to help on a fresh install -of a Linux distribution on a qcow2 image so that it is easier to test new -kernels and debug them. The standard Vagrant box definition *only* requires a -guest to come up, and to be able to ssh into them with the Vagrant user. - - * 1) root/vagrant user password is vagrant - * 2) vagrant user on /etc/sudoers does not need a password to gain root - * 3) vagrant user has an insecure SSH key installed to enable adding a new random one - * 4) Disable the firewall and AppArmor - * 5) Deal with persistent net rules - * 6) Ensure DHCP will work on the first network interface - * 7) Ensure the console is allowed - * 8) Ensure the correct disk size is used on the metadata json file - * 9) Try to use disk partitions by UUID on /etc/fstab - * 10) GRUB disk setup with UUID - * 11) GRUB console setup - * 12) Address lack of virtio - * 13) Address changes in sudo for old systems - -These all deserve their own attention so a section is provided for them below. - -### root / vagrant password - -Although no password is needed, just to ensure one is set the `vagrant` password -is set, just in case you need to use it. - -### vagrant sudoers - -The goal is to never have to use root directly since the boxes are for development -purposes, and so the following entry is expected: - -``` -vagrant ALL=(ALL) NOPASSWD: ALL -``` - -### vagrant user has an insecure SSH key installed - -Vagrant publishes an arbitrary public static SSH public and private key, so -that if it is detected a random SSH key is instead generated and used and -installed. - -Using a random SSH key for each host is a better idea due to possible -risks of guests being left `spawned` and then `pawned` by motivated eager -beavers for a key the entire internet has access to. - -You can surely use your own key as well, however, when sharing Vagrant boxes -you likely don't want to be sharing SSH keys. - -### Disable the firewall and AppArmor - -You are not trying to secure a bank when running a guest for -kernel development, so running a firewall and apparmor is just -typically noise. This is unless of course if you are testing apparmor -or firewall changes to the kernel. - -The firewall and AppArmor can be enabled after initialization, however, -if you really need it. - -### Deal with persistent net rules - -Each Linux distribution has dealt with a way to keep MAC addresses mapped -to a specific interface name. This is to allow a network card to always get -the same IP address, in case it changes the bus it uses on a system. - -Spawning a guest from a Vagrant box means we *want* a different MAC -address for each new guest. This also means we want the first interface -to take the first possible interface name, which will be used for DHCP -purposes after bootup. - -### Ensure DHCP will work on the first network interface - -We expect at least one ethernet interface to come up so that it can get an IP -address and so that we can then SSH into it. Using a network interface just to -communicate to guests is rather archaic, however, it is the current norm. - -A Vagrant box *expects* a random interface to be spawned with it, and we -want to just ensure it, whatever it is, will ask for an IP address via -DHCP. We want to use a mechanism that will work, if possible, for older -systems as well. - -The following recipe works from SLES12-SP3 down to SLES10-SP3 as -an example: - -```bash -if [ -d ] /etc/sysconfig ]; then - cat << FIXIFCGHETH0 > /etc/sysconfig/network/ifcfg-eth0 -DEVICE=eth0 -BOOTPROTO=dhcp -ONBOOT=yes -STARTMODE='auto' - -FIXIFCGHETH0 -``` - -### Ensure the correct disk size is used on the metadata json file - -Vagrant boxes have a metadata json file. If you create a qcow2 image -as follows: - -``` -qemu-img create -f qcow2 foo.qcow2 50G -``` - -Then ensure you have 50 as your size for your metadata.json file: - -``` -{ - "provider" : "libvirt", - "format" : "qcow2", - "virtual_size" : 50 -} -``` - -Note that the Vagrant libvirt provider does not seem to provide support -yet for versioning. - -### Try to use disk partitions by UUID on /etc/fstab - -Some releases do not use /dev/disk/by-uuid labels on /etc/fstab. This -poses a risk when trying to move a guest qcow2 image file from one -system to another. We want something static, and the UUID assigned to -partitions addresses this problem. - -This however means that if you are creating your own custom Vagrant -box or installing your own new fresh kernel you may have to take steps -to ensure the UUID is used instead of the raw /dev/vda or alternative -device disk name. - -### GRUB disk setup with UUID - -As an example, GRUB 0.97 is used on SLE10-SP3, SLE11-SP1, and SLE11-SP4. On -these releases you must ensure that the same /dev/disk/by-uuid is used so that -that a Vagrant box can function on new systems. You do this by editing -/boot/grub/menu.lst and ensuring the appropriate full path /dev/disk/by-uuid/ is -used for the partition in question. - -On SLES10-SP3 swap partitions lacked a respective UUID, and so the -resume entry must be removed. - -SLES12-SP1 and newer use GRUB2, and started using UUID=id as a shortcut on -/etc/fstab and /etc/default/grub, and so no modifications are needed there. - -### GRUB console setup - -As a kernel developer you want to be able to pick what kernel boots -easily. Today, we do this via the console. So we must ensure the console -works on the guest. We support GRUB2 (GRUB) and GRUB 0.97 (GRUB Legacy). - -#### GRUB 0.97 console setup - -On GRUB 0.97 based systems you must add the following to the top of the -/boot/grub/menu.lst file (when in doubt check one of the custom Vagrant -boxes): - -``` -serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1 -terminal --timeout=4 serial console -``` - -Replace `gfxmenu` with `menu` on /boot/grub/menu.lst. - -Below are the base releases of GRUB 0.97 used on our older SLE releases: - - * sles10sp3: grub-0.97-16.25.9 - * sles11sp1: grub-0.97-162.9.1 - * sles11sp4: grub-0.97-162.172.1 - -Ensure any `quiet` entry is removed from the kernel command lines. - -#### GRUB2 console setup - -If you install an ISO on a guest using the console the installer has the sanity -to set up *most* of the right setting for you. However, on SLE12-SP1 you must -make the following change on /etc/default/grub: - -``` --GRUB_TERMINAL=console -+GRUB_TERMINAL="console serial" -``` - -Also ensure GRUB_CMDLINE_LINUX_DEFAULT has the entry `quiet` removed. - -### Address lack of virtio - -Certain modern Linux releases releases support virtio drivers for networking, -or storage. Since Vagrant uses KVM and KVM relies on QEMU, when virtio is not -available emulation is needed. Vagrant is controlled via its Vagrantfile, and -users of a Vagrant box can specify one. However sensible defaults must also be -set and provided on a base Vagrantfile for a Vagrant box. The lack of a virtio -driver then is codified into the Vagrantfile used to build the first Vagrant -box. A Vagrant box is nothing more than a tarball (gzip or xz, use xz, it has -better compression) with the guest qcow2 image, the metadata.json file shown -above, and an initial Vagrantfile. - -Below we document the currently needed modifications needed to the Vagrantfile -for our older Linux distributions, we use older SUSE Linux releases as -example which were targeted to create Vagrant boxes for kernel development -and testing. - -#### Disable default /vagrant nfs sharing for all releases - -Vagrant boxes by default share folders under the current directory via NFS -to the guests you create. This doesn't work so well for all hosts, and sharing -via NFS isn't the best really. We disable NFS sharing as the default then. - -This applies to SLE12-SP3 and older releases. - -#### Disable virtio drives - -SLE10-SP3 doesn't have virtio block driver, and so the root drive uses scsi as the -emulated backend for the drive. Note that currently we still use IDE drives for -the other alternative drives, given using scsi doesn't work at the moment. This -discrepancy should be resolved. - -This is done as follows on the respective Vagrantfile: - -``` - libvirt.disk_bus = "scsi" -``` - -#### Disable virtio networking - -SLE10-SP3 doesn't have a virtio network driver, and so the ne2k_pci network driver -is emulated. - -This is done as follows on the respective Vagrantfile: - -``` - libvirt.nic_model_type = "ne2k_pci" -``` - -### Address changes in sudo for old systems - -SLE10-SP3 has an old version of sudo which lacks the `-E` argument. And since -the default in Vagrant is to use -E, we have to disable this for SLE10-SP3. - -This is done as follows on the respective Vagrantfile: - -``` - config.ssh.sudo_command = "sudo -H %c" -``` diff --git a/docs/kdevops-vagrant.md b/docs/kdevops-vagrant.md deleted file mode 100644 index 1e214d09..00000000 --- a/docs/kdevops-vagrant.md +++ /dev/null @@ -1,130 +0,0 @@ -# Vagrant support - localized VMs - -Vagrant is used to easily deploy non-cloud virtual machines. Below are -the list of providers supported: - - * VirtualBox - * libvirt (KVM) - -The following Operating Systems are supported: - - * OS X - * Linux - -## Running libvirt as a regular user - -kdevops is designed be used without requiring root privileges, however your -user must be allowed to run sudo without a password, and for the regular -user to also be able to run libvirt commands as a regular user. We have an -Ansible role which takes care of dealing with this for you. You'd only use -libvirt if using Linux. - -## Node configuration - -The term `host` is often used to describe `localhost`, and so to help -distinguish `localhost` from your target hosts you'd use for development we -refer to target hosts for development as `nodes`. - -We use a yml file to let you describe your project's nodes and how to configure -them. You configure your node target deployment on the -``vagrant/${PROJECT}_nodes.yaml`` file by default. Since this file is committed -into git, if you want to override the defaults and keep that file outside of -git you can use use the file: - - * ``vagrant/${PROJECT}_nodes_override.yaml`` - -If you prefer a different override file, you can use the environment variable -``KDEVOPS_VAGRANT_NODE_CONFIG`` to define the vagrant host description file -used. - -## Provisioning with vagrant - -If on Linux we'll assume you are using KVM / libvirt. If on OS X we'll assume -you are using VirtualBox. If these assumptions are incorrect you can override -on the configuration file for your node provisioning. For instance, for this -demo you'd use `vagrant/kdevops_nodes.yaml` and set the `force_provider` variable -to either "libvirt" or "kvm". You can also use environment variables to -override the provider: - - * KDEVOPS_VAGRANT_PROVIDER - -You are responsible for having a pretty recent system with some fresh -libvirt, or VirtualBox installed. You are encouraged to use the latest release -for your OS and preferably a rolling Linux distribution release. A VirtualBox -which supports NVMe is required. - -To ramp up your guests with vagrant: - -```bash -make -make bringup -``` - -The last step in the above `make bringup` is to run optional Ansible roles -which can enable direct ssh access to nodes and also run a bit of basic -provisioning. Although vagrant has direct support for running Ansible we do -not make use of this mechanism as it has proven to be fragile before. If -`CONFIG_KDEVOPS_SSH_CONFIG_UPDATE` is enabled your ssh configuration -is updated to enable Ansible to connect to the nodes which have come up. If -`CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK` is enabled then the Ansible -role configured in `CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK` is the first -Ansible to run against nodes to do initial provisioning which by default is the -`devconfig` Ansible role. This is all done as part of the `bringup_vagrant` -target on the file `scripts/vagrant.Makefile`. The roles used are: - - * [update_ssh_config_vagrant](playbooks/roles/update_ssh_config_vagrant/README.md) - * [devconfig](playbooks/roles/devconfig/README.md) - -At this point basic initial provisioning is complete. - -### Code changes for update_ssh_config_vagrant - -The Ansible role `update_ssh_config_vagrant` is used to help update your -ssh configuration when using vagrant. The actual code used is a python -script which is also shared for kdevop's support of Terraform for cloud -provisioning support. The `update_ssh_config_vagrant` Ansible role in -kdevops has the for for the python script locally by using a git subtree. -Updates to actual python code used should be made atomically so that these -changes get pushed back upstream. For more details refer to the following -documentation: - - * [update_ssh_config shared code documentation](playbooks/roles/update_ssh_config_vagrant/update_ssh_config/README.md) - -## Destroying provisioned nodes with vagrant - -You can just use the helper: - -```bash -make destroy -``` - -Or you can either destroy directly with vagrant: - -```bash -cd vagrant/ -vagrant destroy -f -rm -rf .vagrant -``` - -Or you can just use virsh directly, if using KVM: - -```bash -sudo virsh list --all -sudo virsh destroy name-of-guest -sudo virsh undefine name-of-guest -``` - -## Limiting vagrant's number of boxes - -By default using vagrant will try to create *all* the nodes specified on -your configuration file. By default this is `vagrant/kdevops_nodes.yaml` for -this project, and there are currently 2 nodes there. If you are going to just -test this framework you can limit this initially using environment variables: - -```bash -export KDEVOPS_VAGRANT_LIMIT_BOXES="yes" -export KDEVOPS_VAGRANT_LIMIT_NUM_BOXES=1 -``` - -This will ensure only the first host, for example, would be created and -provisioned. diff --git a/docs/the-gen-nodes-ansible-role-vagrant.md b/docs/the-gen-nodes-ansible-role-vagrant.md deleted file mode 100644 index 0bf90700..00000000 --- a/docs/the-gen-nodes-ansible-role-vagrant.md +++ /dev/null @@ -1,78 +0,0 @@ -# Vagrant gen_nodes work - -The `gen_nodes` ansible role is used to let us generate certain target files we -need upon bringup. The amount of files it generates will depend on the features -you have enabled. For local virtualization we support vagrant and the files -generated are listed below. - - * The `KDEVOPS_NODES` file: defines which nodes to create - * The `vagrant/Vagrantfile`: a dynamic Vagrantfile - -## KDEVOPS_NODES and KDEVOPS_NODES_TEMPLATE for vagrant - -If you have local virtualization enabled vagrant then we default these: - - * `KDEVOPS_NODES` as `vagrant/kdevops_nodes.yaml` - * `KDEVOPS_NODES_TEMPLATE` is set to the jinja2 template `$(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/kdevops_nodes_split_start.j2.yaml`. - -Your workflow can override this to something different if needed. - -### Dynamic vagrant/kdevops_nodes.yaml file - -Although the top level ansible hosts file is already dynamic -(see [dynamic kdevops ansible hosts file)](docs/the-gen-hosts-ansible-role.md)) -we currently also generate another yaml file which is used by both local -virtualizaition and cloud environment for node definitions. This file -is a bit more complex given we also extend it with a set of libvirt options -for dynamic features to hosts, to do this jinja2 routines are used. - - * [kdevops dynamic ansible hosts file](playbooks/roles/gen_nodes/templates/kdevops_nodes_split_start.j2.yaml) - * [kdevops dynamic ansible hosts file](playbooks/roles/gen_nodes/templates/hosts.j2) - -As you will see, the `kdevops_nodes_split_start.j2.yaml` starts off with -a set of variables. Most of these were defined and used before we started -embracing Kconfig, so one future work item is to try to see which ones -make sense to move to Kconfig and then `extra_vars.yaml` so we don't have them -in this file. - -The last part of the file is as follows - -``` -vagrant_boxes: -{% include './templates/hosts.j2' %} -``` - -This just calls the jinja2 helper `gen_nodes_list()` with a series of arguments. -This in turn just processes the loop of `gen_node()` for each node in the -array nodes_list. That routine will augment the series of requirements which -are node specific. For example if `passthrough_enable` is `True` it will augment -the node with PCIe passthrough options. - -#### Hyper dynamic qemu feature - -Support for [PCIe passthrough support docs](docs/libvirt-pcie-passthrough.md) is -accomplished by scraping your host system and generating dynamic Kconfig files. -These files are unique to your host and because of this kdevops is -`hyper dynamic`. kdevops uses this support to let you pick which target -nodes will get what features and extends qemu arguments for each guest. - -This demonstrate both how we both: - - * static Kconfig options for qemu per guest - * hyper dynamic Kconfig options for qemu per guest - -#### Static dynamic qemu features per guest - -Given dynamic Kconfig features are possible, it should be easy to then extend -vagrant support to also support different features which don't depend on your -host system, but rather are defined statically on everyone's Kconfig options. -This might be useful, for example, to extend CXL topologies statically. -However, considerations should be also taken to evaluate if a dynamic set -of CXL topologies *could* be inferred based on programming, how would we -define them? Evaluation for this should be done. - -If you want you can just define a custom `KDEVOPS_NODES_TEMPLATE` as well. - -## Related files - - * [Vagrant top level Makefile](scripts/vagrant.Makefile) diff --git a/docs/why-vagrant.md b/docs/why-vagrant.md deleted file mode 100644 index ac075bfe..00000000 --- a/docs/why-vagrant.md +++ /dev/null @@ -1,66 +0,0 @@ -# Vagrant deprecation - -vagrant is not actively maintained so we have moved away from it for -new development / features, [libguestfs](https://libguestfs.org/) is the -focus of support for new features involving local virtualization. - -# Why Vagrant is used for virtualization - -There are a few reasons why kdevops has embraced Vagrant. Below we go through -each of them. - -## What is a Vagrant box anyway? - -First let's understand what a Vagrant box is. A Vagrant box is essentially a -tarball (gzip, xc are both supported) with a qcow2 image and a small metadata -file explaining how large the drive for the guest is. - -## Reducing bring up speed - -Hackers just want to get a guest up fast, slap a kernel on it as fast as -possible, and hope it comes up, and then run some tests or hack something up -on the kernel. The faster we can do this the better. - -The typical approach to using guests is to install a guest using an ISO, and -that takes time. Another way is to just keep copies around of some qcow2 images -and just copy them over when you want to do something new. This can become a -bit of a managing nightmare if you are doing this manually though. Vagrant -abstracts this for you, and essentially does just that. - -## Vagrant has sensible developer friendly settings - -If you are installing a guest with an ISO, you likely will then start doing -a few custom things as a developer. The first thing most developers do is -add a sudoers for your username. A Vagrant box already comes with these things -preset. Also, sensible defaults for the username and root password are set, -because we don't care about security when doing quick bringups for quick -development / testing. - -Another example setup which Vagrant takes care of is bringing up SSH and -letting you SSH to the guest right away. Below is a list of things Vagrant -boxes already come set up with, so you as a developer don't have to do this: - - * 1) root/vagrant user password is vagrant - * 2) Vagrant user on /etc/sudoers does not need a password to gain root - * 3) Vagrant can create a random SSH key for each guest - * 4) Deal with persistent net rules - * 5) Ensure DHCP will work on the first network interface - * 6) Ensure the console is allowed - * 7) Try to use disk partitions by UUID on /etc/fstab - * 8) GRUB disk setup with UUID - -If you'd like to learn more about this read the -[making custom vagrant boxes documentation](docs/custom-vagrant-boxes.md) - -## Supporting different virtualization technologies - -There are different software solutions available which can take -advantage of your architecture's virtualization features and control -guests. Examples are libvirt, VirtualBox. Vagrant abstracts the -virtualization solution used as a "provider" and each provider then -can support its own way to bring up guests. - -## Supporting different Operating Systems - -If we only supported libvirt it would mean you cannot use kdevops on Mac OS X. -OS X users can make user of virtualbox for kdevops. -- 2.47.2