Re: [RFC PATCH 30/33] vfio: selftests: Add a script to help with running VFIO selftests

[Alex Williamson <alex.williamson@redhat.com>]

On Fri, 23 May 2025 23:30:15 +0000
David Matlack <dmatlack@google.com> wrote:

> Introduce run.sh, a script to help with running VFIO selftests. The
> script is intended to be used for both humans manually running VFIO
> selftests, and to incorporate into test automation where VFIO selftests
> may run alongside other tests. As such the script aims to be hermetic,
> returning the system to the state it was before the test started.
> 
> The script takes as input the BDF of a device to use and a command to
> run (typically the command would be a VFIO selftest). e.g.
> 
>   $ ./run.sh -d 0000:6a:01.0 ./vfio_pci_device_test
> 
>  or
> 
>   $ ./run.sh -d 0000:6a:01.0 -- ./vfio_pci_device_test
> 
> The script then handles unbinding device 0000:6a:01.0 from its current
> driver, binding it to vfio-pci, running the test, unbinding from
> vfio-pci, and binding back to the original driver.
> 
> When run.sh runs the provided test, it does so by appending the BDF as
> the last parameter. For example:
> 
>   $ ./run.sh -d 0000:6a:01.0 -- echo hello
> 
> Results in the following being printed to stdout:
> 
>   hello 0000:6a:01.0
> 
> The script also supports a mode where it can break out into a shell so
> that multiple tests can be run manually.
> 
>   $ ./run.sh -d 0000:6a:01.0 -s
>   ... bind to vfio-pci and launch $SHELL ...
>   $ echo $BDF
>   0000:6a:01.0
>   $ ./vfio_pci_device_test $BDF
>   ...
>   $ exit
>   ... unbind from vfio-pci ...
>   $
> 
> Choosing which device to use is up to the user.
> 
> In the future this script should be extensible to tests that want to use
> multiple devices. The script can support accepting -d BDF multiple times
> and parse them into an array, setup all the devices, pass the list of
> BDFs to the test, and then cleanup all the devices.
> 
> Signed-off-by: David Matlack <dmatlack@google.com>
> ---
>  tools/testing/selftests/vfio/Makefile |   1 +
>  tools/testing/selftests/vfio/run.sh   | 110 ++++++++++++++++++++++++++
>  2 files changed, 111 insertions(+)
>  create mode 100755 tools/testing/selftests/vfio/run.sh
> 
> diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile
> index 21fb1809035e..2ab86bd930b0 100644
> --- a/tools/testing/selftests/vfio/Makefile
> +++ b/tools/testing/selftests/vfio/Makefile
> @@ -3,6 +3,7 @@ TEST_GEN_PROGS_EXTENDED += vfio_dma_mapping_test
>  TEST_GEN_PROGS_EXTENDED += vfio_iommufd_setup_test
>  TEST_GEN_PROGS_EXTENDED += vfio_pci_device_test
>  TEST_GEN_PROGS_EXTENDED += vfio_pci_driver_test
> +TEST_PROGS_EXTENDED := run.sh
>  include ../lib.mk
>  include lib/libvfio.mk
>  
> diff --git a/tools/testing/selftests/vfio/run.sh b/tools/testing/selftests/vfio/run.sh
> new file mode 100755
> index 000000000000..b461cc1b2f11
> --- /dev/null
> +++ b/tools/testing/selftests/vfio/run.sh
> @@ -0,0 +1,110 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +readonly VFIO_PCI_DRIVER=/sys/bus/pci/drivers/vfio-pci
> +
> +function bind() {
> +	echo "Binding ${1} to ${2}"
> +	echo "${1}" > "${2}/bind"
> +}
> +
> +function unbind() {
> +	echo "Unbinding ${1} from ${2}"
> +	echo "${1}" > "${2}/unbind"
> +}
> +
> +function set_sriov_numvfs() {
> +	echo "Setting ${1} sriov_numvfs to ${2}"
> +	echo ${2} > /sys/bus/pci/devices/${1}/sriov_numvfs
> +}
> +
> +function add_id() {
> +	if echo $(echo ${1} | tr : ' ') > ${2}/new_id 2> /dev/null; then
> +		echo "Added ${1} to ${2}"
> +		return 0
> +	fi
> +
> +	return 1
> +}

I'd suggest using the newer driver_override mechanism rather than
new_id/remove_id.

I appreciate the work here, I think this could be really useful.  As
Jason notes a few times, I'd also like to see this automatically
iterate through all the combinations of options.  Maybe we could also
think about whether we can define some build environment options to
specify devices to use and this could all be run automatically from
'make kselftest'.  Thanks,

Alex

> +
> +function remove_id() {
> +	echo "Removing ${1} from ${2}"
> +	echo $(echo ${1} | tr : ' ') > ${2}/remove_id
> +}
> +
> +function cleanup() {
> +	if [ "${new_driver}" ]; then unbind ${bdf} ${new_driver} ; fi
> +	if [ "${new_id}"     ]; then remove_id ${device_id} ${VFIO_PCI_DRIVER} ; fi
> +	if [ "${old_driver}" ]; then bind ${bdf} ${old_driver} ; fi
> +	if [ "${old_numvfs}" ]; then set_sriov_numvfs ${bdf} ${old_numvfs} ; fi
> +}
> +
> +function usage() {
> +	echo "usage: $0 [-d segment:bus:device.function] [-s] [-h] [cmd ...]" >&2
> +	echo >&2
> +	echo "  -d: The BDF of the device to use for the test (required)" >&2
> +	echo "  -h: Show this help message" >&2
> +	echo "  -s: Drop into a shell rather than running a command" >&2
> +	echo >&2
> +	echo "   cmd: The command to run and arguments to pass to it." >&2
> +	echo "        Required when not using -s. The SBDF will be " >&2
> +	echo "        appended to the argument list." >&2
> +	exit 1
> +}
> +
> +function main() {
> +	while getopts "d:hs" opt; do
> +		case $opt in
> +			d) bdf="$OPTARG" ;;
> +			s) shell=true ;;
> +			*) usage ;;
> +		esac
> +	done
> +
> +	# Shift past all optional arguments.
> +	shift $((OPTIND - 1))
> +
> +	# Check that the user passed in the command to run.
> +	[ ! "${shell}" ] && [ $# = 0 ] && usage
> +
> +	# Check that the user passed in a BDF.
> +	[ "${bdf}" ] || usage
> +
> +	trap cleanup EXIT
> +	set -e
> +
> +	test -d /sys/bus/pci/devices/${bdf}
> +
> +	device_id=$(lspci -s ${bdf} -n | cut -d' ' -f3)
> +
> +	if [ -f /sys/bus/pci/devices/${bdf}/sriov_numvfs ]; then
> +		old_numvfs=$(cat /sys/bus/pci/devices/${bdf}/sriov_numvfs)
> +		set_sriov_numvfs ${bdf} 0
> +	fi
> +
> +	if [ -L /sys/bus/pci/devices/${bdf}/driver ]; then
> +		old_driver=$(readlink -m /sys/bus/pci/devices/${bdf}/driver)
> +		unbind ${bdf} ${old_driver}
> +	fi
> +
> +	# Add the device ID to vfio-pci. If it hasn't already been added, this will
> +	# succeed and bind the device to vfio-pci. If it has already been added, this
> +	# will fail and we have to manually bind the device.
> +	if add_id ${device_id} ${VFIO_PCI_DRIVER}; then
> +		new_id=true
> +	else
> +		bind ${bdf} ${VFIO_PCI_DRIVER}
> +	fi
> +
> +	new_driver=${VFIO_PCI_DRIVER}
> +
> +	echo
> +	if [ "${shell}" ]; then
> +		echo "Dropping into ${SHELL} with BDF=${bdf}"
> +		BDF=${bdf} ${SHELL}
> +	else
> +		"$@" ${bdf}
> +	fi
> +	echo
> +}
> +
> +main "$@"