[PATCH v2 0/4] raisin: introduce tests

[PATCH v2 0/4] raisin: introduce tests

[Stefano Stabellini]

Hi all,

this patch series introduces a framework to execute simple unit and
functional tests in raisin. It can be used by developers to validate
their changes before submitting a patch series to xen-devel. It can also
be used by OSSTest to test for regressions on one particular
functionality. This patch series only introduces two tests: a PV guest
creation test and an HVM guest creation test. They are both based on
busybox. More tests will follow.


Changes in v2:

- use found as a boolean
- print error to stderr
- use "$@" instead of $* in error_echo and verbose_echo
- small stlye improvement in for_each_component


Stefano Stabellini (4):
      raisin: introduce tests
      raisin: add an hvm test
      raisin: improve output
      raisin: small stlye improvement in for_each_component

 README                      |   14 ++++
 components/grub             |    4 +-
 components/libvirt          |    4 +-
 components/linux            |    2 +-
 components/ovmf             |    6 +-
 components/qemu             |    2 +-
 components/qemu_traditional |    2 +-
 components/seabios          |    6 +-
 components/xen              |    8 +-
 defconfig                   |    5 ++
 lib/commands.sh             |   22 +++---
 lib/common-functions.sh     |  144 +++++++++++++++++++++++++++++------
 lib/common-tests.sh         |  174 +++++++++++++++++++++++++++++++++++++++++++
 raise                       |    8 +-
 scripts/lopartsetup         |   67 +++++++++++++++++
 tests/busybox-hvm           |   44 +++++++++++
 tests/busybox-pv            |   38 ++++++++++
 tests/series                |    2 +
 18 files changed, 502 insertions(+), 50 deletions(-)
 create mode 100644 lib/common-tests.sh
 create mode 100755 scripts/lopartsetup
 create mode 100755 tests/busybox-hvm
 create mode 100755 tests/busybox-pv
 create mode 100644 tests/series


Cheers,

Stefano

[PATCH v2 1/4] raisin: introduce tests

[Stefano Stabellini]

Introduce a new command to run functional tests and unit tests.
Introduce a generic infrastrucutre to run tests on the local machine.
Add a library of common functions that can be used by the test scripts
to setup guest VMs.

Add a simple test script that boots a single busybox based PV guest.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

---

Changes in v2:

- use found as a boolean
- print error to stderr
---
 README                  |   14 ++++++
 defconfig               |    5 +++
 lib/commands.sh         |    4 ++
 lib/common-functions.sh |   74 +++++++++++++++++++++++++++++++
 lib/common-tests.sh     |  112 +++++++++++++++++++++++++++++++++++++++++++++++
 raise                   |    8 ++--
 tests/busybox-pv        |   38 ++++++++++++++++
 tests/series            |    1 +
 8 files changed, 253 insertions(+), 3 deletions(-)
 create mode 100644 lib/common-tests.sh
 create mode 100755 tests/busybox-pv
 create mode 100644 tests/series

diff --git a/README b/README
index b7832da..42c0f4d 100644
--- a/README
+++ b/README
@@ -102,3 +102,17 @@ check-package
 
 If your component comes with additional data, maybe a config script or
 anything else, place it under "data".
+
+
+= Testing =
+
+Raisin can also be used for testing. Make sure to have Xen already up
+and running (raise build, raise install and host reboot).
+Ask Raisin to run tests like this:
+
+./raise test
+
+You can specify a subset of tests to run with ENABLED_TESTS in the
+config file, or the TESTS environmental variable:
+
+TESTS="busybox-pv" ./raise test
diff --git a/defconfig b/defconfig
index b4ed94d..e88f3d3 100644
--- a/defconfig
+++ b/defconfig
@@ -39,3 +39,8 @@ GRUB_REVISION="master"
 LIBVIRT_REVISION="master"
 OVMF_REVISION="master"
 LINUX_REVISION="master"
+
+# Tests
+## All tests: busybox-pv
+## ENABLED_TESTS is the list of test run by raise test
+ENABLED_TESTS="busybox-pv"
diff --git a/lib/commands.sh b/lib/commands.sh
index 801341b..ffbadb4 100755
--- a/lib/commands.sh
+++ b/lib/commands.sh
@@ -103,3 +103,7 @@ function configure() {
     for_each_component configure
 }
 
+function test() {
+    init_tests
+    run_tests
+}
diff --git a/lib/common-functions.sh b/lib/common-functions.sh
index d38788b..5b82c4c 100644
--- a/lib/common-functions.sh
+++ b/lib/common-functions.sh
@@ -39,6 +39,7 @@ function common_init() {
     get_distro
     get_arch
     get_components
+    get_tests
 
     verbose_echo "Distro: $DISTRO"
     verbose_echo "Arch: $RAISIN_ARCH"
@@ -73,6 +74,24 @@ function get_components() {
     export COMPONENTS
 }
 
+function get_tests() {
+    if [[ -z "$TESTS" ]]
+    then
+        TESTS="$ENABLED_TESTS"
+    fi
+
+    if [[ -z "$TESTS" ]] 
+    then
+        local t
+        for t in `cat "$BASEDIR"/tests/series`
+        do
+            TESTS="$TESTS $t"
+            verbose_echo "Found test $t"
+        done
+    fi
+    export TESTS
+}
+
 function get_distro() {
     if [[ -x "`which lsb_release 2>/dev/null`" ]]
     then
@@ -278,6 +297,61 @@ function for_each_component () {
     done
 }
 
+function run_tests() {
+    local t
+    local enabled
+    local found
+
+    for t in `cat "$BASEDIR"/tests/series`
+    do
+        found=false
+        for enabled in $TESTS
+        do
+            if [[ $enabled = $t ]]
+            then
+                found=true
+                break
+            fi
+        done
+        if ! $found
+        then
+            verbose_echo "$t" is disabled
+            continue
+        fi
+
+        verbose_echo running test "$t"
+        "$BASEDIR"/tests/$t
+        verbose_echo "test "$t" done"
+    done
+}
+
+function init_tests() {
+    local -a missing
+
+    check-package bridge-utils
+    if [[ $DISTRO = "Debian" ]]
+    then
+        check-package busybox-static
+    elif [[ $DISTRO = "Fedora" ]]
+    then
+        check-package busybox grub2 which
+    else
+        echo "I don't know distro $DISTRO. It might be missing packages."
+    fi
+    
+    if [[ -n "${missing[@]}" ]]
+    then
+        verbose_echo "Installing ${missing[@]}"
+        install-package "${missing[@]}"
+    fi
+
+    if ! ifconfig xenbr1 &>/dev/null
+    then
+        $SUDO brctl addbr xenbr1
+        $SUDO ifconfig xenbr1 169.254.0.1 up
+    fi
+}
+
 function _build_package_deb() {
     fakeroot bash ./scripts/mkdeb "$1"
 }
diff --git a/lib/common-tests.sh b/lib/common-tests.sh
new file mode 100644
index 0000000..8d2ee6b
--- /dev/null
+++ b/lib/common-tests.sh
@@ -0,0 +1,112 @@
+#!/usr/bin/env bash
+
+source ${RAISIN_PATH}/common-functions.sh
+
+# $1 disk name
+# $2 disk size
+function allocate_disk() {
+    local disk
+    local size
+
+    disk=$1
+    size=$2
+
+    size=$((size+511))
+    size=$((size/512))
+
+    dd if=/dev/zero of=$disk bs=512 count=$size
+    sync
+}
+
+# $1 disk name
+# print loop device name
+function create_loop() {
+    local disk
+    local loop
+
+    disk=`readlink -f $1`
+
+    $SUDO losetup -f $disk
+    loop=`$SUDO losetup -a | grep $disk | cut -d : -f 1`
+    echo $loop
+}
+
+# $1 dev name
+function busybox_rootfs() {
+    local dev
+    local tmpdir
+
+    dev=$1
+
+    $SUDO mkfs.ext3 $dev
+
+    tmpdir=`mktemp -d`
+    $SUDO mount $dev $tmpdir
+    mkdir -p $tmpdir/bin
+    mkdir -p $tmpdir/sbin
+    mkdir -p $tmpdir/dev
+    mkdir -p $tmpdir/proc
+    mkdir -p $tmpdir/sys
+    mkdir -p $tmpdir/lib
+    mkdir -p $tmpdir/var
+    cp `which busybox` $tmpdir/bin
+    $tmpdir/bin/busybox --install $tmpdir/bin
+
+    $SUDO umount $tmpdir
+    rmdir $tmpdir
+}
+
+function busybox_network_init() {
+    local dev
+    local tmpdir
+
+    dev=$1
+    tmpdir=`mktemp -d`
+
+    $SUDO mount $dev $tmpdir
+    rm -f $tmpdir/bin/init
+    cat >$tmpdir/bin/init <<EOF
+#!/bin/sh
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+ifconfig eth0 169.254.0.2 up
+/bin/sh
+EOF
+    chmod +x $tmpdir/bin/init
+
+    $SUDO umount $tmpdir
+    rmdir $tmpdir
+}
+
+function check_guest_alive() {
+    local i
+    i=0
+    while ! ping -c 1 169.254.0.2 &> /dev/null
+    do
+        sleep 1
+        i=$((i+1))
+        if [[ $i -gt 60 ]]
+        then
+            echo Timeout connecting to guest
+            return 1
+        fi
+    done
+    return 0
+}
+
+function get_host_kernel() {
+    echo "/boot/vmlinuz-`uname -r`"
+}
+
+function get_host_initrd() {
+    if [[ $DISTRO = "Debian" ]]
+    then
+        echo "/boot/initrd.img-`uname -r`"
+    elif [[ $DISTRO = "Fedora" ]]
+    then
+        echo "/boot/initramfs-`uname -r`".img
+    else
+        echo "I don't know how to find the initrd" >&2
+        exit 1
+    fi
+}
diff --git a/raise b/raise
index 68dbfd8..dd275ad 100755
--- a/raise
+++ b/raise
@@ -3,7 +3,7 @@
 set -e
 
 _help() {
-    echo "Usage: ./build.sh <options> <command>"
+    echo "Usage: ./raise <options> <command>"
     echo "where options are:"
     echo "    -v | --verbose       Verbose"
     echo "    -y | --yes           Do not ask questions and continue"
@@ -14,6 +14,7 @@ _help() {
     echo "    install              Install binaries under /  (requires sudo)"
     echo "    configure            Configure the system  (requires sudo)"
     echo "    unraise              Uninstall and unconfigure the system  (requires sudo)"
+    echo "    test                 Runs tests on the system (requires sudo, Xen must be running)"
 }
 
 # Include your defaults
@@ -25,10 +26,11 @@ fi
 source ./config
 
 # To use this as a library, set RAISIN_PATH appropriately
-[[ -z "$RAISIN_PATH" ]] && RAISIN_PATH="$PWD/lib"
+[[ -z "$RAISIN_PATH" ]] && export RAISIN_PATH="$PWD/lib"
 
 # Then as many as the sub-libraries as you need
 source ${RAISIN_PATH}/common-functions.sh
+source ${RAISIN_PATH}/common-tests.sh
 source ${RAISIN_PATH}/git-checkout.sh
 source ${RAISIN_PATH}/commands.sh
 
@@ -59,7 +61,7 @@ do
 done
 
 case "$1" in
-    "install-builddep" | "build" | "install" | "configure" | "unraise" )
+    "install-builddep" | "build" | "install" | "configure" | "unraise" | "test" )
         COMMAND=$1
         ;;
     *)
diff --git a/tests/busybox-pv b/tests/busybox-pv
new file mode 100755
index 0000000..941c739
--- /dev/null
+++ b/tests/busybox-pv
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+set -e
+
+source $BASEDIR/lib/common-tests.sh
+
+function cleanup() {
+    $SUDO xl destroy raisin-test || true
+    umount $LOOP || true
+    cd "$BASEDIR"
+    $SUDO losetup -d $LOOP
+    rm -rf $TMPDIR
+}
+
+trap cleanup EXIT
+TMPDIR=`mktemp -d`
+cd $TMPDIR
+
+allocate_disk busybox-vm-disk $((20*1024*1024))
+LOOP=`create_loop busybox-vm-disk`
+busybox_rootfs $LOOP
+busybox_network_init $LOOP
+
+cat >busybox-pv <<EOF
+kernel = "`get_host_kernel`"
+ramdisk = "`get_host_initrd`"
+extra = "root=/dev/xvda console=hvc0"
+memory = 512
+name = "raisin-test"
+vcpus = 2
+disk = [ '$LOOP,raw,xvda,w' ]
+serial="pty"
+boot="c"
+vif=['bridge=xenbr1']
+EOF
+
+$SUDO xl create busybox-pv
+check_guest_alive
diff --git a/tests/series b/tests/series
new file mode 100644
index 0000000..a5ec626
--- /dev/null
+++ b/tests/series
@@ -0,0 +1 @@
+busybox-pv
-- 
1.7.10.4

[PATCH v2 2/4] raisin: add an hvm test

[Stefano Stabellini]

Add a few functions to create partitions, install and configure grub in
the VM disk. Introduce a script to loopmount a partition within a VM
disk.

Add a new test that creates a local HVM guest, boots it and check the
network.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 defconfig           |    4 +--
 lib/common-tests.sh |   62 +++++++++++++++++++++++++++++++++++++++++++++++
 scripts/lopartsetup |   67 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/busybox-hvm   |   44 +++++++++++++++++++++++++++++++++
 tests/series        |    1 +
 5 files changed, 176 insertions(+), 2 deletions(-)
 create mode 100755 scripts/lopartsetup
 create mode 100755 tests/busybox-hvm

diff --git a/defconfig b/defconfig
index e88f3d3..664aee6 100644
--- a/defconfig
+++ b/defconfig
@@ -41,6 +41,6 @@ OVMF_REVISION="master"
 LINUX_REVISION="master"
 
 # Tests
-## All tests: busybox-pv
+## All tests: busybox-pv busybox-hvm
 ## ENABLED_TESTS is the list of test run by raise test
-ENABLED_TESTS="busybox-pv"
+ENABLED_TESTS="busybox-pv busybox-hvm"
diff --git a/lib/common-tests.sh b/lib/common-tests.sh
index 8d2ee6b..abbc91a 100644
--- a/lib/common-tests.sh
+++ b/lib/common-tests.sh
@@ -31,6 +31,18 @@ function create_loop() {
     echo $loop
 }
 
+# $1 disk name
+# print loop device name of the partition
+function create_one_partition() {
+    local disk
+    local dev
+
+    disk=$1
+    echo -e "o\nn\np\n1\n\n\nw" | $SUDO fdisk $disk &>/dev/null
+    dev=`$SUDO $BASEDIR/scripts/lopartsetup $disk | head -1 | cut -d ":" -f 1`
+    echo $dev
+}
+
 # $1 dev name
 function busybox_rootfs() {
     local dev
@@ -78,6 +90,56 @@ EOF
     rmdir $tmpdir
 }
 
+function bootloader_init() {
+    local dev
+    local devp
+    local tmpdir
+
+    dev=$1
+    devp=$2
+    tmpdir=`mktemp -d`
+
+    $SUDO mount $devp $tmpdir
+    mkdir -p $tmpdir/boot/grub
+    cp "`get_host_kernel`" $tmpdir/boot
+    cp "`get_host_initrd`" $tmpdir/boot || true
+    cat >$tmpdir/boot/grub/grub.cfg <<EOF
+set default="0"
+set timeout=0
+
+menuentry 'Xen Guest' {
+ set root=hd0,1
+ linux `get_host_kernel` root=/dev/xvda1 console=ttyS0
+EOF
+    if [[ -e `get_host_initrd` ]]
+    then
+        echo "initrd `get_host_initrd`" >> $tmpdir/boot/grub/grub.cfg
+    fi
+    echo "}" >> $tmpdir/boot/grub/grub.cfg
+
+    cat >$tmpdir/boot/grub/device.map <<EOF
+(hd0)   $dev
+(hd0,1) $devp
+EOF
+
+    if [[ $DISTRO = "Debian" ]]
+    then
+        $SUDO grub-install --no-floppy \
+            --grub-mkdevicemap=$tmpdir/boot/grub/device.map \
+            --root-directory=$tmpdir $dev
+    elif [[ $DISTRO = "Fedora" ]]
+    then
+        $SUDO grub2-install --no-floppy \
+            --grub-mkdevicemap=$tmpdir/boot/grub/device.map \
+            --root-directory=$tmpdir $dev
+    else
+        echo "I don't know how to install grub on $DISTRO"
+    fi
+
+    $SUDO umount $tmpdir
+    rmdir $tmpdir
+}
+
 function check_guest_alive() {
     local i
     i=0
diff --git a/scripts/lopartsetup b/scripts/lopartsetup
new file mode 100755
index 0000000..bf33a28
--- /dev/null
+++ b/scripts/lopartsetup
@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+#
+# Need root privileges
+#
+# Create loop devices corresponding to partitions within an existing
+# file.
+
+set -e
+
+function _help() {
+    echo "Usage: lopartsetup file"
+}
+
+function _create_loop_device() {
+    local devnode
+    local filename
+    local offset
+    local index
+
+    filename="$1"
+    index="$2"
+    offset="$3"
+
+    devnode="`losetup -f 2>/dev/null`"
+    if [[ -z "$devnode" ]]
+    then
+        echo "no loop devices available"
+        exit 1
+    fi
+
+    echo "$devnode: partition $index of $filename"
+    losetup "$devnode" "$filename" -o "$offset"
+}
+
+if [[ $# -lt 1 ]]
+then
+    _help
+    exit 1
+fi
+
+if [[ -f "$1" && -r "$1" ]]
+then
+    filename="$1"
+    shift
+else
+    echo invalid image file
+    exit 1
+fi
+
+if [[ ! "`file -b $filename`" = *"boot sector"* ]]
+then
+    echo "$filename does not have a partition table"
+    exit 1
+fi
+
+unit="`fdisk -lu $filename 2>/dev/null | grep -e "^Units = " | cut -d " " -f 9`"
+index=0
+for i in "`fdisk -lu $filename 2>/dev/null | grep -e "^$filename"`"
+do
+    index=$((index+1))
+    offset=`echo $i | tr -s " " | cut -d " " -f 2`
+    offset=$((unit*offset))
+
+    _create_loop_device "$filename" "$index" "$offset"
+done
+
+exit 0
diff --git a/tests/busybox-hvm b/tests/busybox-hvm
new file mode 100755
index 0000000..b97ae34
--- /dev/null
+++ b/tests/busybox-hvm
@@ -0,0 +1,44 @@
+#!/usr/bin/env bash
+
+set -e
+
+source $BASEDIR/lib/common-tests.sh
+
+function cleanup() {
+    $SUDO xl destroy raisin-test || true
+    umount $LOOP_P0 || true
+    cd "$BASEDIR"
+    $SUDO losetup -d $LOOP_P0 $LOOP
+    rm -rf $TMPDIR
+}
+
+if [[ $RAISIN_ARCH != "x86_64" && $RAISIN_ARCH != "x86_32" ]]
+then
+    echo busybox hvm test only valid on x86
+    exit 0
+fi
+
+trap cleanup EXIT
+TMPDIR=`mktemp -d`
+cd $TMPDIR
+
+allocate_disk busybox-vm-disk $((20*1024*1024))
+LOOP=`create_loop busybox-vm-disk`
+LOOP_P0=`create_one_partition busybox-vm-disk`
+busybox_rootfs $LOOP_P0
+busybox_network_init $LOOP_P0
+bootloader_init $LOOP $LOOP_P0
+
+cat >busybox-hvm <<EOF
+builder = "hvm"
+memory = 512
+name = "raisin-test"
+vcpus = 2
+disk = [ '$LOOP,raw,hda,w' ]
+serial="pty"
+boot="c"
+vif=['bridge=xenbr1']
+EOF
+
+$SUDO xl create busybox-hvm
+check_guest_alive
diff --git a/tests/series b/tests/series
index a5ec626..1f5f5c6 100644
--- a/tests/series
+++ b/tests/series
@@ -1 +1,2 @@
 busybox-pv
+busybox-hvm
-- 
1.7.10.4

[PATCH v2 3/4] raisin: improve output

[Stefano Stabellini]

Introduce an error_echo function that prints to stderr.
Call error_echo or verbose_echo, instead of echo, when possible and
appropriate.
Redirect build and tests output to /dev/null unless VERBOSE==1.
Redirect apt-get and yum output to /dev/null.
Fix echo arguments in check-package-deb.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

---

Changes in v2:

- use "$@" instead of $* in error_echo and verbose_echo
---
 components/grub             |    4 +--
 components/libvirt          |    4 +--
 components/linux            |    2 +-
 components/ovmf             |    6 ++--
 components/qemu             |    2 +-
 components/qemu_traditional |    2 +-
 components/seabios          |    6 ++--
 components/xen              |    8 ++---
 lib/commands.sh             |   18 +++++------
 lib/common-functions.sh     |   72 ++++++++++++++++++++++++++++++-------------
 lib/common-tests.sh         |    6 ++--
 tests/busybox-hvm           |    2 +-
 12 files changed, 80 insertions(+), 52 deletions(-)

diff --git a/components/grub b/components/grub
index fa72b99..3bb5254 100644
--- a/components/grub
+++ b/components/grub
@@ -19,10 +19,10 @@ function grub_check_package() {
 
     if [[ $RAISIN_ARCH != "x86_64" && $RAISIN_ARCH != "x86_32" ]]
     then
-        echo grub is only supported on x86_32 and x86_64
+        verbose_echo grub is only supported on x86_32 and x86_64
         return
     fi
-    echo Checking Grub dependencies
+    verbose_echo Checking Grub dependencies
     eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
diff --git a/components/libvirt b/components/libvirt
index a554643..20405ae 100644
--- a/components/libvirt
+++ b/components/libvirt
@@ -22,7 +22,7 @@ function libvirt_check_package() {
     local DEP_CentOS_x86_32="$DEP_Fedora_x86_32"
     local DEP_CentOS_x86_64="$DEP_Fedora_x86_64"
 
-    echo Checking Libvirt dependencies
+    verbose_echo Checking Libvirt dependencies
     eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
@@ -52,7 +52,7 @@ function libvirt_build() {
         cp daemon/libvirtd.init "$INST_DIR"/etc/rc.d/init.d/libvirtd
         chmod +x "$INST_DIR"/etc/rc.d/init.d/libvirtd
     else
-        echo "I don't know how write an init script for Libvirt on $DISTRO"
+        error_echo "I don't know how write an init script for Libvirt on $DISTRO"
     fi
     cd ..
 }
diff --git a/components/linux b/components/linux
index f90a894..9ae6955 100644
--- a/components/linux
+++ b/components/linux
@@ -15,7 +15,7 @@ function linux_check_package() {
     local DEP_CentOS_x86_32="$DEP_Fedora_x86_32"
     local DEP_CentOS_x86_64="$DEP_Fedora_x86_64"
 
-    echo Checking Linux dependencies
+    verbose_echo Checking Linux dependencies
     eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
diff --git a/components/ovmf b/components/ovmf
index a59a771..387044a 100644
--- a/components/ovmf
+++ b/components/ovmf
@@ -14,10 +14,10 @@ function ovmf_check_package() {
 
     if [[ $RAISIN_ARCH != "x86_64" ]]
     then
-        echo ovmf is only supported on x86_64
+        verbose_echo ovmf is only supported on x86_64
         return
     fi
-    echo Checking OVMF dependencies
+    verbose_echo Checking OVMF dependencies
     eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
@@ -25,7 +25,7 @@ function ovmf_check_package() {
 function ovmf_build() {
     if [[ $RAISIN_ARCH != "x86_64" ]]
     then
-        echo ovmf is only supported on x86_64
+        verbose_echo ovmf is only supported on x86_64
         return
     fi
 
diff --git a/components/qemu b/components/qemu
index 72cfec1..dce4ce0 100644
--- a/components/qemu
+++ b/components/qemu
@@ -11,7 +11,7 @@ function qemu_check_package() {
     local DEP_Fedora_x86_32="$DEP_Fedora_common"
     local DEP_Fedora_x86_64="$DEP_Fedora_common"
 
-    echo Checking QEMU dependencies
+    verbose_echo Checking QEMU dependencies
     eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
diff --git a/components/qemu_traditional b/components/qemu_traditional
index b338007..a9609b1 100644
--- a/components/qemu_traditional
+++ b/components/qemu_traditional
@@ -12,7 +12,7 @@ function qemu_traditional_check_package() {
     local DEP_Fedora_x86_32="$DEP_Fedora_common"
     local DEP_Fedora_x86_64="$DEP_Fedora_common"
 
-    echo Checking QEMU dependencies
+    verbose_echo Checking QEMU dependencies
     eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
diff --git a/components/seabios b/components/seabios
index ed2c7d2..8fea193 100644
--- a/components/seabios
+++ b/components/seabios
@@ -14,10 +14,10 @@ function seabios_check_package() {
 
     if [[ $RAISIN_ARCH != "x86_64" && $RAISIN_ARCH != "x86_32" ]]
     then
-        echo seabios is only supported on x86_32 and x86_64
+        verbose_echo seabios is only supported on x86_32 and x86_64
         return
     fi
-    echo Checking SeaBIOS dependencies
+    verbose_echo Checking SeaBIOS dependencies
     eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
@@ -25,7 +25,7 @@ function seabios_check_package() {
 function seabios_build() {
     if [[ $RAISIN_ARCH != "x86_64" && $RAISIN_ARCH != "x86_32" ]]
     then
-        echo seabios is only supported on x86_32 and x86_64
+        verbose_echo seabios is only supported on x86_32 and x86_64
         return
     fi
 
diff --git a/components/xen b/components/xen
index add8136..6b700e5 100644
--- a/components/xen
+++ b/components/xen
@@ -19,7 +19,7 @@ function xen_check_package() {
     local DEP_CentOS_x86_32="$DEP_CentOS_common dev86 texinfo iasl"
     local DEP_CentOS_x86_64="$DEP_CentOS_x86_32 glibc-devel.i686"
 
-    echo Checking Xen dependencies
+    verbose_echo Checking Xen dependencies
     eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
@@ -62,13 +62,13 @@ function _xen_create_bridge_Debian() {
 
     if [[ -z "$IFACE" ]]
     then
-        echo "Please refer to the following page to setup networking:"
-        echo "http://wiki.xenproject.org/wiki/Network_Configuration_Examples_(Xen_4.1%2B)"
+        error_echo "Please refer to the following page to setup networking:"
+        error_echo "http://wiki.xenproject.org/wiki/Network_Configuration_Examples_(Xen_4.1%2B)"
         return 1
     fi
     if [[ "`grep $BRIDGE /etc/network/interfaces`" ]]
     then
-        echo "a network bridge seems to be already setup"
+        verbose_echo "a network bridge seems to be already setup"
         return 0
     fi
 
diff --git a/lib/commands.sh b/lib/commands.sh
index ffbadb4..a79611e 100755
--- a/lib/commands.sh
+++ b/lib/commands.sh
@@ -17,29 +17,29 @@ function check-builddep() {
 
     if [[ -n "${missing[@]}" ]]
     then
-        echo "Missing packages: ${missing[@]}"
+        echo "$PREPEND Missing packages: ${missing[@]}"
         if [[ "$YES" = "n" ]]
         then
             return
         elif [[ "$YES" != "y" ]]
         then
-            echo "Do you want Raisin to automatically install them for you? (y/n)"
+            echo "$PREPEND Do you want Raisin to automatically install them for you? (y/n)"
             while read answer
             do
                 if [[ "$answer" = "n" ]]
                 then
-                    echo "Please install, or run ./raise install-builddep"
+                    echo "$PREPEND Please install, or run ./raise install-builddep"
                     exit 1
                 elif [[ "$answer" = "y" ]]
                 then
                     break
                 else
-                    echo "Reply y or n"
+                    echo "$PREPEND Reply y or n"
                 fi
             done
         fi
 
-        echo "Installing..."
+        echo "$PREPEND Installing..."
         install-package "${missing[@]}"
     fi
 }
@@ -72,7 +72,7 @@ function install() {
     # need single braces for filename matching expansion
     if [ ! -f xen-sytem*rpm ] && [ ! -f xen-system*deb ]
     then
-        echo You need to raise build first.
+        error_echo You need to raise build first.
         exit 1
     fi
     install_package xen-system
@@ -84,8 +84,8 @@ function configure() {
         return
     elif [[ "$YES" != "y" ]]
     then
-        echo "Proceeding we'll make changes to the running system,"
-        echo "are you sure that you want to continue? (y/n)"
+        echo "$PREPEND Proceeding we'll make changes to the running system,"
+        echo "$PREPEND are you sure that you want to continue? (y/n)"
         while read answer
         do
             if [[ "$answer" = "n" ]]
@@ -95,7 +95,7 @@ function configure() {
             then
                 break
             else
-                echo "Reply y or n"
+                echo "$PREPEND Reply y or n"
             fi
         done
     fi
diff --git a/lib/common-functions.sh b/lib/common-functions.sh
index 5b82c4c..cbb658e 100644
--- a/lib/common-functions.sh
+++ b/lib/common-functions.sh
@@ -3,10 +3,14 @@
 function verbose_echo() {
     if [[ $VERBOSE -eq 1 ]]
     then
-        echo $*
+        echo "$PREPEND" "$@"
     fi
 }
 
+function error_echo() {
+    echo "$PREPEND" "$@" >&2
+}
+
 # Executed once at the beginning of the script
 function common_init() {
     export BASEDIR=`pwd`
@@ -15,6 +19,7 @@ function common_init() {
     export RAISIN_MAKE=${MAKE-make}
     export PREFIX=${PREFIX-/usr}
     export INST_DIR=${DESTDIR-dist}
+    export PREPEND="[raisin]"
     
     INST_DIR=`readlink -f $INST_DIR`
     
@@ -24,15 +29,15 @@ function common_init() {
         export SUDO=""
     elif [[ ! -f `which sudo 2>/dev/null` ]]
     then
-        echo "Raisin requires sudo to install build dependencies for you."
-        echo "You can only build without it."
+        error_echo "Raisin requires sudo to install build dependencies for you."
+        error_echo "You can only build without it."
         export SUDO=""
     fi
 
     if [[ -z "$BASH_VERSINFO" || ${BASH_VERSINFO[0]} -lt 3 ||
         (${BASH_VERSINFO[0]} -eq 3 && ${BASH_VERSINFO[1]} -lt 2) ]]
     then
-        echo "Raisin requires BASH 3.2 or newer."
+        error_echo "Raisin requires BASH 3.2 or newer."
         exit 1
     fi
 
@@ -186,7 +191,7 @@ function get_arch() {
 }
 
 function _check-package-deb() {
-    verbose_echo "Checking for package ${args[0]}"
+    verbose_echo "Checking for package $1"
 
     if dpkg -s "$1" 2>/dev/null | grep -q "Status:.*installed"
     then
@@ -197,7 +202,7 @@ function _check-package-deb() {
 }
 
 function _install-package-deb() {
-    $SUDO apt-get install -y $*
+    $SUDO apt-get install -y $* > /dev/null
 }
 
 function _check-package-rpm() {
@@ -212,7 +217,7 @@ function _check-package-rpm() {
 }
 
 function _install-package-rpm() {
-    $SUDO yum install -y $*
+    $SUDO yum install -y $* > /dev/null
 }
 
 # Modifies inherited variable "missing"
@@ -236,13 +241,13 @@ function start_initscripts() {
     do
         case $DISTRO in
             "Debian" )
-            $SUDO update-rc.d $1 defaults || echo "Couldn't set $1 to start"
+            $SUDO update-rc.d $1 defaults || error_echo "Couldn't set $1 to start"
             ;;
             "Fedora" )
-            $SUDO chkconfig --add $1 || echo "Couldn't set $1 to start"
+            $SUDO chkconfig --add $1 || error_echo "Couldn't set $1 to start"
             ;;
             * )
-            echo "I don't know how to start initscripts on $DISTRO"
+            error_echo "I don't know how to start initscripts on $DISTRO"
             return 1
             ;;
         esac
@@ -255,13 +260,13 @@ function stop_initscripts() {
     do
         case $DISTRO in
             "Debian" )
-            $SUDO update-rc.d $1 remove || echo "Couldn't remove $1 from init"
+            $SUDO update-rc.d $1 remove || error_echo "Couldn't remove $1 from init"
             ;;
             "Fedora" )
-            $SUDO chkconfig --del $1 || echo "Couldn't remove $1 from init"
+            $SUDO chkconfig --del $1 || error_echo "Couldn't remove $1 from init"
             ;;
             * )
-            echo "I don't know how to start initscripts on $DISTRO"
+            error_echo "I don't know how to start initscripts on $DISTRO"
             return 1
             ;;
         esac
@@ -291,9 +296,14 @@ function for_each_component () {
             continue
         fi
 
-        verbose_echo calling "$component"_"$1"
-        "$component"_"$1"
-        verbose_echo "$component"_"$1" done
+        echo "$PREPEND" calling "$component"_"$1"
+        if [[ $VERBOSE -eq 0 ]]
+        then
+            "$component"_"$1" &> /dev/null
+        else
+            "$component"_"$1"
+        fi
+        echo "$PREPEND" "$component"_"$1" done
     done
 }
 
@@ -301,6 +311,7 @@ function run_tests() {
     local t
     local enabled
     local found
+    local ret
 
     for t in `cat "$BASEDIR"/tests/series`
     do
@@ -319,9 +330,26 @@ function run_tests() {
             continue
         fi
 
-        verbose_echo running test "$t"
-        "$BASEDIR"/tests/$t
-        verbose_echo "test "$t" done"
+        ret=0
+        if [[ $VERBOSE -eq 0 ]]
+        then
+            echo -n "$PREPEND test $t: "
+            "$BASEDIR"/tests/$t &>/dev/null || ret=1
+            if [[ $ret -eq 0 ]]
+            then
+                echo "success"
+            else
+                echo "fail"
+            fi
+        else
+            "$BASEDIR"/tests/$t || ret=1
+            if [[ $ret -eq 0 ]]
+            then
+                echo "$PREPEND test $t: success"
+            else
+                echo "$PREPEND test $t: fail"
+            fi
+        fi
     done
 }
 
@@ -336,7 +364,7 @@ function init_tests() {
     then
         check-package busybox grub2 which
     else
-        echo "I don't know distro $DISTRO. It might be missing packages."
+        error_echo "I don't know distro $DISTRO. It might be missing packages."
     fi
     
     if [[ -n "${missing[@]}" ]]
@@ -372,7 +400,7 @@ function install_package() {
     then
         $SUDO rpm -i --force "$1"-`git show --oneline | head -1 | cut -d " " -f 1`-0.$RAISIN_ARCH.rpm
     else
-        echo "Don't know how to install packages on $DISTRO"
+        error_echo "Don't know how to install packages on $DISTRO"
     fi
 }
 
@@ -384,6 +412,6 @@ function uninstall_package() {
     then
         $SUDO rpm -e "$1"
     else
-        echo "Don't know how to uninstall packages on $DISTRO"
+        error_echo "Don't know how to uninstall packages on $DISTRO"
     fi
 }
diff --git a/lib/common-tests.sh b/lib/common-tests.sh
index abbc91a..0e51539 100644
--- a/lib/common-tests.sh
+++ b/lib/common-tests.sh
@@ -133,7 +133,7 @@ EOF
             --grub-mkdevicemap=$tmpdir/boot/grub/device.map \
             --root-directory=$tmpdir $dev
     else
-        echo "I don't know how to install grub on $DISTRO"
+        echo "$PREPEND I don't know how to install grub on $DISTRO"
     fi
 
     $SUDO umount $tmpdir
@@ -149,7 +149,7 @@ function check_guest_alive() {
         i=$((i+1))
         if [[ $i -gt 60 ]]
         then
-            echo Timeout connecting to guest
+            echo $PREPEND Timeout connecting to guest
             return 1
         fi
     done
@@ -168,7 +168,7 @@ function get_host_initrd() {
     then
         echo "/boot/initramfs-`uname -r`".img
     else
-        echo "I don't know how to find the initrd" >&2
+        echo "$PREPEND I don't know how to find the initrd" >&2
         exit 1
     fi
 }
diff --git a/tests/busybox-hvm b/tests/busybox-hvm
index b97ae34..c8024e5 100755
--- a/tests/busybox-hvm
+++ b/tests/busybox-hvm
@@ -14,7 +14,7 @@ function cleanup() {
 
 if [[ $RAISIN_ARCH != "x86_64" && $RAISIN_ARCH != "x86_32" ]]
 then
-    echo busybox hvm test only valid on x86
+    echo $PREPEND busybox hvm test only valid on x86
     exit 0
 fi
 
-- 
1.7.10.4

[PATCH v2 4/4] raisin: small stlye improvement in for_each_component

[Stefano Stabellini]

Use the found variable as a boolean.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 lib/common-functions.sh |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/common-functions.sh b/lib/common-functions.sh
index cbb658e..5ee4816 100644
--- a/lib/common-functions.sh
+++ b/lib/common-functions.sh
@@ -281,16 +281,16 @@ function for_each_component () {
 
     for component in `cat "$BASEDIR"/components/series`
     do
-        found=0
+        found=false
         for enabled in $COMPONENTS
         do
             if [[ $enabled = $component ]]
             then
-                found=1
+                found=true
                 break
             fi
         done
-        if [[ $found -eq 0 ]]
+        if ! $found
         then
             verbose_echo "$component" is disabled
             continue
-- 
1.7.10.4

Re: [PATCH v2 0/4] raisin: introduce tests

[Ian Campbell]

On Fri, 2015-05-01 at 16:48 +0100, Stefano Stabellini wrote:
> Hi all,
> 
> this patch series introduces a framework to execute simple unit and
> functional tests in raisin. It can be used by developers to validate
> their changes before submitting a patch series to xen-devel. It can also
> be used by OSSTest to test for regressions on one particular
> functionality. This patch series only introduces two tests: a PV guest
> creation test and an HVM guest creation test. They are both based on
> busybox. More tests will follow.

What is the intended scope of these tests? e.g. when should a test be
added here rather than to osstest? Do you eventually plan to have an
osstest job which executes the raisin tests?

Ian.

Re: [PATCH v2 0/4] raisin: introduce tests

[Stefano Stabellini]

On Tue, 5 May 2015, Ian Campbell wrote:
> On Fri, 2015-05-01 at 16:48 +0100, Stefano Stabellini wrote:
> > Hi all,
> > 
> > this patch series introduces a framework to execute simple unit and
> > functional tests in raisin. It can be used by developers to validate
> > their changes before submitting a patch series to xen-devel. It can also
> > be used by OSSTest to test for regressions on one particular
> > functionality. This patch series only introduces two tests: a PV guest
> > creation test and an HVM guest creation test. They are both based on
> > busybox. More tests will follow.
> 
> What is the intended scope of these tests? e.g. when should a test be
> added here rather than to osstest?

Small functional tests that can be easily run on a single host, without
requiring a specific host or guest operating system. Tests that every
developer should run on their test machine before submitting a patch
series.


> Do you eventually plan to have an osstest job which executes the
> raisin tests?

Yes, that is the goal. We should be able to get the bisector running and
all the usual osstest goodness.

Re: [PATCH v2 0/4] raisin: introduce tests

[George Dunlap]

On Tue, May 5, 2015 at 2:55 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Tue, 5 May 2015, Ian Campbell wrote:
>> On Fri, 2015-05-01 at 16:48 +0100, Stefano Stabellini wrote:
>> > Hi all,
>> >
>> > this patch series introduces a framework to execute simple unit and
>> > functional tests in raisin. It can be used by developers to validate
>> > their changes before submitting a patch series to xen-devel. It can also
>> > be used by OSSTest to test for regressions on one particular
>> > functionality. This patch series only introduces two tests: a PV guest
>> > creation test and an HVM guest creation test. They are both based on
>> > busybox. More tests will follow.
>>
>> What is the intended scope of these tests? e.g. when should a test be
>> added here rather than to osstest?
>
> Small functional tests that can be easily run on a single host, without
> requiring a specific host or guest operating system. Tests that every
> developer should run on their test machine before submitting a patch
> series.

So a sort of BVT (build verification test), such that that we could be
reasonably annoyed at if an experienced developer submitted a patch
that broke said functionality?

 -George

Re: [PATCH v2 0/4] raisin: introduce tests

[Stefano Stabellini]

On Wed, 6 May 2015, George Dunlap wrote:
> On Tue, May 5, 2015 at 2:55 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > On Tue, 5 May 2015, Ian Campbell wrote:
> >> On Fri, 2015-05-01 at 16:48 +0100, Stefano Stabellini wrote:
> >> > Hi all,
> >> >
> >> > this patch series introduces a framework to execute simple unit and
> >> > functional tests in raisin. It can be used by developers to validate
> >> > their changes before submitting a patch series to xen-devel. It can also
> >> > be used by OSSTest to test for regressions on one particular
> >> > functionality. This patch series only introduces two tests: a PV guest
> >> > creation test and an HVM guest creation test. They are both based on
> >> > busybox. More tests will follow.
> >>
> >> What is the intended scope of these tests? e.g. when should a test be
> >> added here rather than to osstest?
> >
> > Small functional tests that can be easily run on a single host, without
> > requiring a specific host or guest operating system. Tests that every
> > developer should run on their test machine before submitting a patch
> > series.
> 
> So a sort of BVT (build verification test), such that that we could be
> reasonably annoyed at if an experienced developer submitted a patch
> that broke said functionality?

That's right.

In general any functionality tests, that can be added without
introducing too much complexity, should be in raisin. osstest will be
able to run these tests via raisin.

More complex tests, involving more than one host, or specific guest/host
OS combinations, should only be in osstest.

Re: [PATCH v2 0/4] raisin: introduce tests

[Ian Campbell]

On Wed, 2015-05-06 at 15:39 +0100, Stefano Stabellini wrote:
> On Wed, 6 May 2015, George Dunlap wrote:
> > On Tue, May 5, 2015 at 2:55 PM, Stefano Stabellini
> > <stefano.stabellini@eu.citrix.com> wrote:
> > > On Tue, 5 May 2015, Ian Campbell wrote:
> > >> On Fri, 2015-05-01 at 16:48 +0100, Stefano Stabellini wrote:
> > >> > Hi all,
> > >> >
> > >> > this patch series introduces a framework to execute simple unit and
> > >> > functional tests in raisin. It can be used by developers to validate
> > >> > their changes before submitting a patch series to xen-devel. It can also
> > >> > be used by OSSTest to test for regressions on one particular
> > >> > functionality. This patch series only introduces two tests: a PV guest
> > >> > creation test and an HVM guest creation test. They are both based on
> > >> > busybox. More tests will follow.
> > >>
> > >> What is the intended scope of these tests? e.g. when should a test be
> > >> added here rather than to osstest?
> > >
> > > Small functional tests that can be easily run on a single host, without
> > > requiring a specific host or guest operating system. Tests that every
> > > developer should run on their test machine before submitting a patch
> > > series.
> > 
> > So a sort of BVT (build verification test), such that that we could be
> > reasonably annoyed at if an experienced developer submitted a patch
> > that broke said functionality?
> 
> That's right.
> 
> In general any functionality tests, that can be added without
> introducing too much complexity, should be in raisin. osstest will be
> able to run these tests via raisin.

I think if it is intended as a BVT which I'll get shouted out for not
having run then it needs to be time bound as much as functionality
bound.

Ian.

> More complex tests, involving more than one host, or specific guest/host
> OS combinations, should only be in osstest.

Re: [PATCH v2 0/4] raisin: introduce tests

[George Dunlap]

On 05/06/2015 03:46 PM, Ian Campbell wrote:
> On Wed, 2015-05-06 at 15:39 +0100, Stefano Stabellini wrote:
>> On Wed, 6 May 2015, George Dunlap wrote:
>>> On Tue, May 5, 2015 at 2:55 PM, Stefano Stabellini
>>> <stefano.stabellini@eu.citrix.com> wrote:
>>>> On Tue, 5 May 2015, Ian Campbell wrote:
>>>>> On Fri, 2015-05-01 at 16:48 +0100, Stefano Stabellini wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> this patch series introduces a framework to execute simple unit and
>>>>>> functional tests in raisin. It can be used by developers to validate
>>>>>> their changes before submitting a patch series to xen-devel. It can also
>>>>>> be used by OSSTest to test for regressions on one particular
>>>>>> functionality. This patch series only introduces two tests: a PV guest
>>>>>> creation test and an HVM guest creation test. They are both based on
>>>>>> busybox. More tests will follow.
>>>>>
>>>>> What is the intended scope of these tests? e.g. when should a test be
>>>>> added here rather than to osstest?
>>>>
>>>> Small functional tests that can be easily run on a single host, without
>>>> requiring a specific host or guest operating system. Tests that every
>>>> developer should run on their test machine before submitting a patch
>>>> series.
>>>
>>> So a sort of BVT (build verification test), such that that we could be
>>> reasonably annoyed at if an experienced developer submitted a patch
>>> that broke said functionality?
>>
>> That's right.
>>
>> In general any functionality tests, that can be added without
>> introducing too much complexity, should be in raisin. osstest will be
>> able to run these tests via raisin.
> 
> I think if it is intended as a BVT which I'll get shouted out for not
> having run then it needs to be time bound as much as functionality
> bound.

If we start to use raisin as the repository for a lot of the actual
basic functionality tests which are now in osstest, then yeah, there
should be a smallish subset that we might expect people to run.

OTOH, do we really have a problem with people breaking things
accidentally at the moment?  Does it make sense to impose the cultural
expectation of running the BVT before every submission, when in the vast
majority of cases developers are able to decide for themselves what
testing needs to be done?

Maybe reviewers / maintainers might decide require assurance of having
run a certain subset of the tests, based on the particular complexity of
the patch / code being modified?

 -George

Re: [PATCH v2 0/4] raisin: introduce tests

[Ian Campbell]

On Wed, 2015-05-06 at 16:11 +0100, George Dunlap wrote:
> On 05/06/2015 03:46 PM, Ian Campbell wrote:
> > On Wed, 2015-05-06 at 15:39 +0100, Stefano Stabellini wrote:
> >> On Wed, 6 May 2015, George Dunlap wrote:
> >>> On Tue, May 5, 2015 at 2:55 PM, Stefano Stabellini
> >>> <stefano.stabellini@eu.citrix.com> wrote:
> >>>> On Tue, 5 May 2015, Ian Campbell wrote:
> >>>>> On Fri, 2015-05-01 at 16:48 +0100, Stefano Stabellini wrote:
> >>>>>> Hi all,
> >>>>>>
> >>>>>> this patch series introduces a framework to execute simple unit and
> >>>>>> functional tests in raisin. It can be used by developers to validate
> >>>>>> their changes before submitting a patch series to xen-devel. It can also
> >>>>>> be used by OSSTest to test for regressions on one particular
> >>>>>> functionality. This patch series only introduces two tests: a PV guest
> >>>>>> creation test and an HVM guest creation test. They are both based on
> >>>>>> busybox. More tests will follow.
> >>>>>
> >>>>> What is the intended scope of these tests? e.g. when should a test be
> >>>>> added here rather than to osstest?
> >>>>
> >>>> Small functional tests that can be easily run on a single host, without
> >>>> requiring a specific host or guest operating system. Tests that every
> >>>> developer should run on their test machine before submitting a patch
> >>>> series.
> >>>
> >>> So a sort of BVT (build verification test), such that that we could be
> >>> reasonably annoyed at if an experienced developer submitted a patch
> >>> that broke said functionality?
> >>
> >> That's right.
> >>
> >> In general any functionality tests, that can be added without
> >> introducing too much complexity, should be in raisin. osstest will be
> >> able to run these tests via raisin.
> > 
> > I think if it is intended as a BVT which I'll get shouted out for not
> > having run then it needs to be time bound as much as functionality
> > bound.
> 
> If we start to use raisin as the repository for a lot of the actual
> basic functionality tests which are now in osstest, then yeah, there
> should be a smallish subset that we might expect people to run.
> 
> OTOH, do we really have a problem with people breaking things
> accidentally at the moment?

I don't think we do really.

>   Does it make sense to impose the cultural
> expectation of running the BVT before every submission, when in the vast
> majority of cases developers are able to decide for themselves what
> testing needs to be done?
> 
> Maybe reviewers / maintainers might decide require assurance of having
> run a certain subset of the tests, based on the particular complexity of
> the patch / code being modified?
> 
>  -George