* [PATCH BlueZ v6 0/6] Functional/integration testing
@ 2026-06-13 15:48 Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 1/6] doc: add functional/integration testing documentation Pauli Virtanen
` (5 more replies)
0 siblings, 6 replies; 8+ messages in thread
From: Pauli Virtanen @ 2026-06-13 15:48 UTC (permalink / raw)
To: linux-bluetooth; +Cc: Pauli Virtanen
Add framework for writing tests simulating "real" environments where
BlueZ and other parts of the stack run on different virtual machine
hosts that communicate with each other.
*** v6 ***
* Fix minor issues in documentation and shellcheck warnings.
* Bump to latest pytest-bluezenv==0.1.6
* NOTE: hci_uart is partly broken in current bluetooth-next / 7.1 kernel,
which causes some of the tests here fail sporadically. Needs the following patch:
https://lore.kernel.org/linux-bluetooth/6888691461070a011d31632e6dcbfd73016dcc6e.1781364475.git.pav@iki.fi/
*** v5 ***
https://github.com/pv/bluez/compare/func-test-v4-r..func-test-v5
* Factor out the pytest-bluezenv plugin, to be maintained separately.
https://pypi.org/project/pytest-bluezenv/
It could in principle be moved under the BlueZ organization, but
there's no particular reason why it should be in bluez repository.
Generally, it's better to have the pytest plugin separate so it's
easier to reuse and can have its own version cycle.
* Pipewire tests are moved to pipewire repository where they probably
belong to.
They can be run easily vs. given BlueZ build dir.
We are currently running them in Pipewire CI, but at frozen
kernel/BlueZ version, so it is not testing BlueZ/kernel upstream
development.
* No changes in the emulator/test-runner patches since v4
They are stand-alone bug fixes / improvements, and make sense
also separately from the rest.
*** v4 ***
https://github.com/pv/bluez/compare/func-test-v3-r..func-test-v4
* Use virtconsole for simpler HCI forwarding to the vm
* Fix typoed vm_module -> vm_once
* Skip tests for some pipewire versions
*** v3 ***
https://github.com/pv/bluez/compare/func-test-v2-r..func-test-v3
* fix configure.ac openpty() detection to match TOOLS conditional,
to fix make distcheck
* properly retry virtio RPC connection if it fails initially
* properly restart VM if previous test hangs
* allow custom parent host side proxy objects, use them for pexpect
* improve --list with out-of-tree test files
* fix missing bus.set_exit_on_disconnect(False) for obex tests
* have --vm-timeout etc. change values also on VM host side
* use larger-memory VM instances for Pipewire, in case ASAN enabled
* set reasonable inside-VM ASAN_OPTION default values
* don't run btvirt under stdbuf, since not compatible with ASAN
*** v2 ***
https://github.com/pv/bluez/compare/func-test-v1-r..func-test-v2
* move unit/func_test -> test/functional & test/pytest_bluez
The pytest_bluez plugin is in principle reusable for other projects,
so we can eg. have more complete Pipewire integration tests that can
live in Pipewire repository.
* openpty() is in -lutil on some platforms, detect this in autoconf
* more emulator adjustments:
- fix SCO data packet support in btvirt
- more complete Reset command
* improve logging: get timestamps from kernel, and reorder logs
to timestamp order, so that lines from different hosts, btmon,
and parent tester appear in right order regardless of whether
VM console / btmon is lagging
- this requires accurate clock sync in the VM, so enable KVM PTP in
config and run chronyd inside the VMs
- use virtio port instead of qemu console to export logs, since the
console has fixed baud rate and is too slow
* add --btmon & export btsnoop dumps from VM hosts
* fix compatibility with older Python versions
* add parametrized_host_config()
* split Pipewire test to A2DP/BAP/HFP and really stream audio.
These catch the 5.86 regression fixed in 066a164a524e498 and
the 5.84 one in 6b0a08776a
* add support for tests that reuse tester environment, so they can run
faster without needing Bluetoothd teardown/setup in between
* add HostPlugin.presetup (mainly for test skipping)
* deal with RPC virtio port buffer possibly containing unflushed
commands from previous failed test
* add some Agent1 interface tests
* add basic Obex file transfer tests
* add support for logging in to a running test instance (for gdb etc)
* export any core dumps out from test environ
Some bells & whistles:
* add --kernel-build for kernel image build
* test suite Python code formatting checks
***
Implements:
- RPC communication with tester instances running each of the VM hosts.
Tests run on parent host, which instructs VM hosts what to do.
- Extensible way to add stateful test-specific code inside the VM
instances
- Logging control: output from different processes running inside the VM
are separated and can be filtered.
- Test runner framework with Pytest (more convenient than Python/unittest)
- Automatic grouping of tests to minimize VM reboots
- Redirecting USB controllers to use for testing in addition to btvirt
- Fairly straightforward, ~1600 sloc for the framework
There is no requirement that the tests spawn VM instances, the test
runner can be used for any tests written in Python.
See doc/test-functional.rst for various examples.
Also test/functional/test_bluetoothctl_vm.py has some simple cases, and
test/functional/test_pipewire.py for a more complicated setup
host0(qemu): Pipewire <-> BlueZ <-> kernel
<-> btvirt
host1(qemu): kernel <-> BlueZ <-> Pipewire
The framework allows easily passing any data and code between the parent
and VM hosts, so writing tests is straightforward.
***
Some examples:
$ test/test-functional --list
test/functional/lib/tests/test_rpc.py::test_basic
test/functional/test_bluetoothctl_vm.py::test_bluetoothctl_pair[hosts0-vm2]
test/functional/test_bluetoothctl_vm.py::test_bluetoothctl_script_show[hosts1-vm1]
test/functional/test_btmgmt_vm.py::test_btmgmt_info[hosts2-vm1]
test/functional/test_pipewire.py::test_pipewire[hosts3-vm2]
$ test/test-functional --kernel=../linux
============================= test session starts ==============================
platform linux -- Python 3.14.3, pytest-8.3.5, pluggy-1.6.0
rootdir: /home/pauli/prj/external/bluez/unit
configfile: pytest.ini
plugins: cov-5.0.0, forked-1.6.0, rerunfailures-15.0, timeout-2.4.0, xdist-3.7.0, hypothesis-6.123.0, flaky-3.8.1, anyio-4.12.1
collected 5 items
test/functional/lib/tests/test_rpc.py . [ 20%]
test/functional/test_bluetoothctl_vm.py . [ 40%]
test/functional/test_btmgmt_vm.py . [ 60%]
test/functional/test_bluetoothctl_vm.py . [ 80%]
test/functional/test_pipewire.py . [100%]
============================== 5 passed in 41.92s ==============================
$ test/test-functional --kernel=../linux -k test_btmgmt
============================= test session starts ==============================
platform linux -- Python 3.14.3, pytest-8.3.5, pluggy-1.6.0
rootdir: /home/pauli/prj/external/bluez/unit
configfile: pytest.ini
plugins: cov-5.0.0, forked-1.6.0, rerunfailures-15.0, timeout-2.4.0, xdist-3.7.0, hypothesis-6.123.0, flaky-3.8.1, anyio-4.12.1
collected 5 items / 4 deselected / 1 selected
test/functional/test_btmgmt_vm.py . [100%]
======================= 1 passed, 4 deselected in 9.15s ========================
$ grep btmgmt test-functional.log
13:15:42 INFO rpc.host.0.0 : client: call_plugin ('call', '__call__', <function run at 0x7f27b81ce140>, ['/home/pauli/prj/external/bluez/build/tools/btmgmt', '--index', '0', 'info']) {'stdout': -1, 'stdin': -3, 'encoding': 'utf-8'}
13:15:42 INFO host.0.0.rpc : server: call_plugin ('call', '__call__', <function run at 0x7fd5e35a1010>, ['/home/pauli/prj/external/bluez/build/tools/btmgmt', '--index', '0', 'info']) {'stdout': -1, 'stdin': -3, 'encoding': 'utf-8'}
13:15:42 INFO host.0.0.run : $ /home/pauli/prj/external/bluez/build/tools/btmgmt --index 0 info
$ test/test-functional --kernel=../linux -k test_btmgmt --log-cli-level=0
============================= test session starts ==============================
platform linux -- Python 3.14.3, pytest-8.3.5, pluggy-1.6.0
rootdir: /home/pauli/prj/external/bluez/unit
configfile: pytest.ini
plugins: cov-5.0.0, forked-1.6.0, rerunfailures-15.0, timeout-2.4.0, xdist-3.7.0, hypothesis-6.123.0, flaky-3.8.1, anyio-4.12.1
collected 5 items / 4 deselected / 1 selected
test/functional/test_btmgmt_vm.py::test_btmgmt_info[hosts2-vm1]
-------------------------------- live log setup --------------------------------
13:00:31 INFO func_test.lib.env : Starting btvirt: /usr/bin/stdbuf -o L -e L /home/pauli/prj/external/bluez/build/emulator/btvirt --server=/tmp/bluez-func-test-8t6ychy8
13:00:31 OUT btvirt : Bluetooth emulator ver 5.86
13:00:31 INFO func_test.lib.env : Starting host: /home/pauli/prj/external/bluez/build/tools/test-runner --kernel=../linux/arch/x86/boot/bzImage -u/tmp/bluez-func-test-8t6ychy8/bt-server-bredrle -o -chardev -o socket,id=ser0,path=/tmp/bluez-func-test-8t6ychy8/bluez-func-test-rpc-0,server=on,wait=off -o -device -o virtio-serial -o -device -o virtserialport,chardev=ser0,name=bluez-func-test-rpc -H -- /usr/bin/python3 -P /home/pauli/prj/external/bluez/test/functional/lib/runner.py /dev/ttyS2
13:00:31 OUT btvirt : Request for /tmp/bluez-func-test-8t6ychy8/bt-server-bredrle
13:00:32 OUT host.0.0 : early console in extract_kernel
13:00:32 OUT host.0.0 : input_data: 0x000000000425c2c4
...
13:00:39 INFO rpc.host.0.0 : client: call_plugin ('call', '__call__', <function run at 0x7f7547472140>, ['/home/pauli/prj/external/bluez/build/tools/btmgmt', '--index', '0', 'info']) {'stdout': -1, 'stdin': -3, 'encoding': 'utf-8'}
13:00:39 DEBUG host.0.0.rpc : server: done
13:00:39 INFO host.0.0.rpc : server: call_plugin ('call', '__call__', <function run at 0x7f77dcc81010>, ['/home/pauli/prj/external/bluez/build/tools/btmgmt', '--index', '0', 'info']) {'stdout': -1, 'stdin': -3, 'encoding': 'utf-8'}
13:00:39 INFO host.0.0.run : $ /home/pauli/prj/external/bluez/build/tools/btmgmt --index 0 info
13:00:40 OUT host.0.0.run.out : hci0: Primary controller
13:00:40 OUT host.0.0.run.out : addr 00:AA:01:00:00:42 version 11 manufacturer 1521 class 0x000000
13:00:40 OUT host.0.0.run.out : supported settings: powered connectable fast-connectable discoverable bondable link-security ssp br/edr le advertising secure-conn debug-keys privacy static-addr phy-configuration cis-central cis-peripheral iso-broadcaster sync-receiver ll-privacy past-sender past-receiver
13:00:40 OUT host.0.0.run.out : current settings: br/edr
13:00:40 OUT host.0.0.run.out : name
13:00:40 OUT host.0.0.run.out : short name
13:00:40 INFO host.0.0.run : (return code 0)
13:00:40 DEBUG rpc.host.0.0 : client-reply
PASSED [100%]
13:00:40 OUT host.0.0 : qemu-system-x86_64: terminating on signal 15 from pid 149047 (python3)
======================= 1 passed, 4 deselected in 8.84s ========================
$ test/test-functional --kernel=../linux -k test_bluetoothctl_pair --log-cli-level=0 --log-filter=*.bluetoothctl,rpc.* --force-usb
============================= test session starts ==============================
platform linux -- Python 3.14.3, pytest-8.3.5, pluggy-1.6.0
rootdir: /home/pauli/prj/external/bluez/unit
configfile: pytest.ini
plugins: cov-5.0.0, forked-1.6.0, rerunfailures-15.0, timeout-2.4.0, xdist-3.7.0, hypothesis-6.123.0, flaky-3.8.1, anyio-4.12.1
collected 5 items / 4 deselected / 1 selected
test/functional/test_bluetoothctl_vm.py::test_bluetoothctl_pair[hosts0-vm2]
-------------------------------- live log setup --------------------------------
13:03:20 INFO rpc.host.0.0 : client: start_load (<func_test.lib.host_plugins.Bdaddr object at 0x7f268712d160>,) {}
13:03:20 INFO rpc.host.0.0 : client: start_load (<func_test.lib.host_plugins.Call object at 0x7f268712d2b0>,) {}
13:03:20 INFO rpc.host.0.0 : client: start_load (<func_test.lib.host_plugins.DbusSystem object at 0x7f2687aa30e0>,) {}
13:03:20 INFO rpc.host.0.0 : client: start_load (<func_test.lib.host_plugins.Bluetoothd object at 0x7f2687aa3230>,) {}
13:03:20 INFO rpc.host.0.0 : client: start_load (<func_test.lib.host_plugins.Bluetoothctl object at 0x7f268712d010>,) {}
13:03:20 INFO rpc.host.0.1 : client: start_load (<func_test.lib.host_plugins.Bdaddr object at 0x7f26871542d0>,) {}
13:03:20 INFO rpc.host.0.1 : client: start_load (<func_test.lib.host_plugins.Call object at 0x7f2687154410>,) {}
13:03:20 INFO rpc.host.0.1 : client: start_load (<func_test.lib.host_plugins.DbusSystem object at 0x7f2687aa30e0>,) {}
13:03:20 INFO rpc.host.0.1 : client: start_load (<func_test.lib.host_plugins.Bluetoothd object at 0x7f2687aa3230>,) {}
13:03:20 INFO rpc.host.0.1 : client: start_load (<func_test.lib.host_plugins.Bluetoothctl object at 0x7f2687154190>,) {}
13:03:20 INFO rpc.host.0.0 : client: wait_load () {}
13:03:21 DEBUG rpc.host.0.0 : client-reply
13:03:21 INFO rpc.host.0.1 : client: wait_load () {}
13:03:21 DEBUG rpc.host.0.1 : client-reply
-------------------------------- live log call ---------------------------------
13:03:21 INFO rpc.host.0.0 : client: call_plugin ('bluetoothctl', 'send', 'show\n') {}
13:03:21 DEBUG rpc.host.0.0 : client-reply
13:03:21 INFO rpc.host.0.0 : client: call_plugin ('bluetoothctl', 'expect', 'Powered: yes') {}
...
13:03:23 INFO rpc.host.0.0 : client: call_plugin ('bluetoothctl', 'send', 'pair 70:1a:b8:73:99:bb\n') {}
13:03:23 OUT host.0.0.bluetoothctl: pair 70:1a:b8:73:99:bb
13:03:23 DEBUG rpc.host.0.0 : client-reply
13:03:23 INFO rpc.host.0.0 : client: call_plugin ('bluetoothctl', 'expect', 'Confirm passkey (\\d+).*:') {}
13:03:23 OUT host.0.0.bluetoothctl: [bluetoothctl]> pair 70:1a:b8:73:99:bb
13:03:23 OUT host.0.0.bluetoothctl: Attempting to pair with 70:1A:B8:73:99:BB
13:03:23 OUT host.0.0.bluetoothctl: [bluetoothctl]> hci0 device_flags_changed: 70:1A:B8:73:99:BB (BR/EDR)
13:03:23 OUT host.0.0.bluetoothctl: [bluetoothctl]> supp: 0x00000007 curr: 0x00000000
13:03:23 OUT host.0.0.bluetoothctl: [bluetoothctl]> hci0 type 7 discovering off
13:03:25 OUT host.0.0.bluetoothctl: [bluetoothctl]> hci0 70:1A:B8:73:99:BB type BR/EDR connected eir_len 12
13:03:25 OUT host.0.0.bluetoothctl: [bluetoothctl]> [BlueZ 5.86]> [CHG] Device 70:1A:B8:73:99:BB Connected: yes
13:03:25 OUT host.0.0.bluetoothctl: [BlueZ 5.86]> Request confirmation
13:03:25 DEBUG rpc.host.0.0 : client-reply
13:03:25 INFO rpc.host.0.1 : client: call_plugin ('bluetoothctl', 'expect', 'Confirm passkey 237345') {}
13:03:25 OUT host.0.1.bluetoothctl: [bluetoothctl]> hci0 84:5C:F3:77:31:19 type BR/EDR connected eir_len 12
13:03:25 OUT host.0.1.bluetoothctl: [bluetoothctl]> [NEW] Device 84:5C:F3:77:31:19 BlueZ 5.86
13:03:25 DEBUG rpc.host.0.1 : client-reply
13:03:25 INFO rpc.host.0.0 : client: call_plugin ('bluetoothctl', 'send', 'yes\n') {}
13:03:25 OUT host.0.1.bluetoothctl: [bluetoothctl]> [BlueZ 5.86]> Request confirmation
13:03:25 OUT host.0.0.bluetoothctl: [BlueZ 5.86]> [agent] Confirm passkey 237345 (yes/no): yes
13:03:25 DEBUG rpc.host.0.0 : client-reply
13:03:25 INFO rpc.host.0.1 : client: call_plugin ('bluetoothctl', 'send', 'yes\n') {}
13:03:25 OUT host.0.1.bluetoothctl: [BlueZ 5.86]> [agent] Confirm passkey 237345 (yes/no): yes
13:03:25 DEBUG rpc.host.0.1 : client-reply
13:03:25 INFO rpc.host.0.0 : client: call_plugin ('bluetoothctl', 'expect', 'Pairing successful') {}
13:03:25 OUT host.0.0.bluetoothctl: yes
13:03:25 OUT host.0.0.bluetoothctl: [BlueZ 5.86]> hci0 new_link_key 70:1A:B8:73:99:BB type 0x08 pin_len 0 store_hint 1
13:03:25 OUT host.0.0.bluetoothctl: [BlueZ 5.86]> [CHG] Device 70:1A:B8:73:99:BB Bonded: yes
13:03:26 OUT host.0.0.bluetoothctl: [BlueZ 5.86]> [CHG] Device 70:1A:B8:73:99:BB AddressType: public
13:03:26 OUT host.0.0.bluetoothctl: [BlueZ 5.86]> [CHG] Device 70:1A:B8:73:99:BB UUIDs: 0000110c-0000-1000-8000-00805f9b34fb
13:03:26 OUT host.0.0.bluetoothctl: [BlueZ 5.86]> [CHG] Device 70:1A:B8:73:99:BB UUIDs: 0000110e-0000-1000-8000-00805f9b34fb
13:03:26 DEBUG rpc.host.0.0 : client-reply
PASSED [100%]
------------------------------ live log teardown -------------------------------
13:03:26 OUT host.0.0.bluetoothctl: [BlueZ 5.86]> [CHG] Device 70:1A:B8:98:FF:qemu-system-x86_64: terminating on signal 15 from pid 149357 (python3)
======================= 1 passed, 4 deselected in 13.22s =======================
$ test/test-functional -k test_btmgmt --kernel=../linux --trace
============================= test session starts ==============================
platform linux -- Python 3.14.3, pytest-8.3.5, pluggy-1.6.0
rootdir: /home/pauli/prj/external/bluez/unit
configfile: pytest.ini
plugins: cov-5.0.0, forked-1.6.0, rerunfailures-15.0, timeout-2.4.0, xdist-3.7.0, hypothesis-6.123.0, flaky-3.8.1, anyio-4.12.1
collected 5 items / 4 deselected / 1 selected
test/functional/test_btmgmt_vm.py
>>>>>>>>>>>>>>>>>>>> PDB runcall (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>
> /home/pauli/prj/external/bluez/test/functional/test_btmgmt_vm.py(19)test_btmgmt_info()
-> (host,) = hosts
(Pdb) n
> /home/pauli/prj/external/bluez/test/functional/test_btmgmt_vm.py(21)test_btmgmt_info()
-> result = host.call(
(Pdb) p host.bdaddr
'00:aa:01:00:00:42'
(Pdb) n
> /home/pauli/prj/external/bluez/test/functional/test_btmgmt_vm.py(22)test_btmgmt_info()
-> run,
(Pdb) n
> /home/pauli/prj/external/bluez/test/functional/test_btmgmt_vm.py(23)test_btmgmt_info()
-> [btmgmt, "--index", "0", "info"],
(Pdb) n
> /home/pauli/prj/external/bluez/test/functional/test_btmgmt_vm.py(24)test_btmgmt_info()
-> stdout=subprocess.PIPE,
(Pdb) n
> /home/pauli/prj/external/bluez/test/functional/test_btmgmt_vm.py(25)test_btmgmt_info()
-> stdin=subprocess.DEVNULL,
(Pdb) n
> /home/pauli/prj/external/bluez/test/functional/test_btmgmt_vm.py(26)test_btmgmt_info()
-> encoding="utf-8",
(Pdb) n
> /home/pauli/prj/external/bluez/test/functional/test_btmgmt_vm.py(21)test_btmgmt_info()
-> result = host.call(
(Pdb) n
> /home/pauli/prj/external/bluez/test/functional/test_btmgmt_vm.py(28)test_btmgmt_info()
-> assert result.returncode == 0
(Pdb) p result
CompletedProcess(args=['/home/pauli/prj/external/bluez/build/tools/btmgmt', '--index', '0', 'info'], returncode=0, stdout='hci0:\tPrimary controller\n\taddr 00:AA:01:00:00:42 version 11 manufacturer 1521 class 0x000000\n\tsupported settings: powered connectable fast-connectable discoverable bondable link-security ssp br/edr le advertising secure-conn debug-keys privacy static-addr phy-configuration cis-central cis-peripheral iso-broadcaster sync-receiver ll-privacy past-sender past-receiver \n\tcurrent settings: br/edr \n\tname \n\tshort name \n')
(Pdb) print(result.stdout)
hci0: Primary controller
addr 00:AA:01:00:00:42 version 11 manufacturer 1521 class 0x000000
supported settings: powered connectable fast-connectable discoverable bondable link-security ssp br/edr le advertising secure-conn debug-keys privacy static-addr phy-configuration cis-central cis-peripheral iso-broadcaster sync-receiver ll-privacy past-sender past-receiver
current settings: br/edr
name
short name
(Pdb) q
!!!!!!!!!!!!!!!!!!! _pytest.outcomes.Exit: Quitting debugger !!!!!!!!!!!!!!!!!!!
======================= 4 deselected in 75.91s (0:01:15) =======================
Pauli Virtanen (6):
doc: add functional/integration testing documentation
test: add functional/integration testing framework
build: add functional testing target
test: functional: impose Python code formatting
test: functional: add some Agent1 interface tests
test: functional: add basic obex file transfer tests
Makefile.am | 10 +
configure.ac | 22 ++
doc/test-functional.rst | 313 ++++++++++++++++++++++++
test/functional/__init__.py | 2 +
test/functional/conftest.py | 48 ++++
test/functional/requirements.txt | 2 +
test/functional/test_agent.py | 46 ++++
test/functional/test_bluetoothctl_vm.py | 152 ++++++++++++
test/functional/test_btmgmt_vm.py | 30 +++
test/functional/test_obex.py | 285 +++++++++++++++++++++
test/functional/test_tests.py | 23 ++
test/pytest.ini | 17 ++
test/test-functional | 21 ++
test/test-functional-attach | 7 +
14 files changed, 978 insertions(+)
create mode 100644 doc/test-functional.rst
create mode 100644 test/functional/__init__.py
create mode 100644 test/functional/conftest.py
create mode 100644 test/functional/requirements.txt
create mode 100644 test/functional/test_agent.py
create mode 100644 test/functional/test_bluetoothctl_vm.py
create mode 100644 test/functional/test_btmgmt_vm.py
create mode 100644 test/functional/test_obex.py
create mode 100644 test/functional/test_tests.py
create mode 100644 test/pytest.ini
create mode 100755 test/test-functional
create mode 100755 test/test-functional-attach
--
2.54.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH BlueZ v6 1/6] doc: add functional/integration testing documentation
2026-06-13 15:48 [PATCH BlueZ v6 0/6] Functional/integration testing Pauli Virtanen
@ 2026-06-13 15:48 ` Pauli Virtanen
2026-06-13 17:09 ` Functional/integration testing bluez.test.bot
2026-06-13 15:48 ` [PATCH BlueZ v6 2/6] test: add functional/integration testing framework Pauli Virtanen
` (4 subsequent siblings)
5 siblings, 1 reply; 8+ messages in thread
From: Pauli Virtanen @ 2026-06-13 15:48 UTC (permalink / raw)
To: linux-bluetooth; +Cc: Pauli Virtanen
Add documentation for functional/integration test suite.
---
doc/test-functional.rst | 313 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 313 insertions(+)
create mode 100644 doc/test-functional.rst
diff --git a/doc/test-functional.rst b/doc/test-functional.rst
new file mode 100644
index 000000000..35245da72
--- /dev/null
+++ b/doc/test-functional.rst
@@ -0,0 +1,313 @@
+===============
+test-functional
+===============
+
+**test-functional** [*OPTIONS*]
+
+DESCRIPTION
+===========
+
+**test-functional(1)** is used for functional testing of BlueZ and
+kernel using multiple virtual machine environments, connected by real
+or virtual controllers.
+
+It uses https://pypi.org/project/pytest-bluezenv as VM-based test
+framework. For details, see its documentation.
+
+QUICK EXAMPLE
+=============
+
+Install `qemu-system-x86_64` first. Then,
+
+.. code-block::
+
+ $ ./configure --enable-functional-testing --enable-testing --enable-tools
+ $ make -j8
+ $ python3 -mpip install -r test/functional/requirements.txt
+ $ test/test-functional --kernel-build -v
+
+OPTIONS
+=======
+
+The `test-functional` script simply runs `Pytest
+<https://pytest.org>`__ which can take the following options:
+https://docs.pytest.org/en/stable/reference/reference.html#command-line-flags
+
+The following additional options apply:
+
+``--list``
+ Output brief lists of existing tests.
+
+``--kernel=<image>``
+ Kernel image (or built Linux source tree root) to
+ use. See **test-runner(1)** and `tester.config` for required
+ kernel config.
+
+ If not provided, value from `FUNCTIONAL_TESTING_KERNEL`
+ environment variable is used. If none, no image is used.
+
+``--usb=hci0,hci1``
+ USB controllers to use in tests that require use of
+ real controllers.
+
+ If not provided, value from `FUNCTIONAL_TESTING_CONTROLLERS`
+ environment variable is used. If none, all USB controllers
+ with suitable permissions are considered.
+
+``--force-usb``
+ Force tests to use USB controllers instead of `btvirt`.
+
+``--bluez-build-dir=<path>``
+ Path to build directory where to search for BlueZ
+ executables.
+
+``--bluez-src-dir=<path>``
+ Path to build BlueZ source directory.
+
+``--log-filter=[+-]<pattern>,[+-]<pattern>,...``
+ Allow/deny lists
+ for filtering logging output. The pattern is a shell glob matching
+ to the logger names.
+
+``--no-log-reorder``
+ Don't reorder logs to timestamp order.
+
+``--vm-timeout=<seconds>``
+ Specify timeout for communication with VM hosts.
+
+``--btmon``
+ Launch btmon on all hosts to log events, and dump traffic to
+ test-bluezenv-\*.btsnoop
+
+``--kernel-build=no/use/auto/force``
+ Build a suitable kernel image from source.
+
+``--kernel-upstream=<GIT_URL>``
+ URL for Git clone of kernel sources.
+
+``--kernel-branch=<GIT_BRANCH>``
+ Git branch to build from.
+
+
+Tests that require kernel image or USB controllers are skipped if none
+are available. Normally, tests use `btvirt`.
+
+VM instances share a directory ``/run/shared`` with host machine,
+located on host usually in ``/tmp/bluez-func-test-*/shared-*``. Core
+dumps etc. are copied out from it before test instance is shut down.
+
+REQUIREMENTS
+============
+
+General
+-------
+
+The following are needed:
+
+- QEmu (x86_64)
+- ``dbus-daemon`` available
+
+Recommended:
+
+- KVM-enabled x86_64 host system
+- Preferably built BlueZ source tree
+- ``chronyd`` available
+- ``util-linux`` tools available
+- ``agetty`` available
+
+Python
+------
+
+The following Python packages are required:
+
+.. code-block::
+
+ pytest>=8
+ pytest-bluezenv==0.1.6
+
+To install them via pip::
+
+ python3 -m pip install -r test/functional/requirements.txt
+
+On Fedora / RHEL, the dependencies aside from `pytest-bluezenv` can be
+installed via::
+
+ sudo dnf install python3-pytest python3-pexpect python3-dbus
+
+Kernel
+------
+
+The **test-functional(1)** tool requires a kernel image with similar
+config as **test-runner(1)**. If given `--kernel-build` option, a
+suitable image is built from sources downloaded under
+`test/.pytest_cache`.
+
+Simplest setup is
+
+.. code-block::
+
+ cp ../bluez/doc/tester.config .config
+ make olddefconfig
+ make -j8
+
+To get log timestamps right, the kernel should have the following
+configuration enabled:
+
+.. code-block::
+
+ CONFIG_HYPERVISOR_GUEST=y
+ CONFIG_PARAVIRT=y
+ CONFIG_KVM_GUEST=y
+
+ CONFIG_PTP_1588_CLOCK=y
+ CONFIG_PTP_1588_CLOCK_KVM=y
+ CONFIG_PTP_1588_CLOCK_VMCLOCK=y
+
+USB
+---
+
+Some tests may require a hardware controller instead of the virtual `btvirt` one.
+
+EXAMPLES
+========
+
+Run all tests
+-------------
+
+.. code-block::
+
+ $ test/test-functional --kernel=/pathto/bzImage
+
+ $ export FUNCTIONAL_TESTING_KERNEL=/pathto/bzImage
+ $ test/test-functional
+
+Test output is logged in ``test-functional.log``.
+
+Show output during run
+----------------------
+
+.. code-block::
+
+ $ test/test-functional --log-cli-level=0
+
+Show only specific loggers:
+
+.. code-block::
+
+ $ test/test-functional --log-cli-level=0 --log-filter=rpc,host
+
+ $ test/test-functional --log-cli-level=0 --log-filter=*.bluetoothctl
+
+Filter out loggers:
+
+.. code-block::
+
+ $ test/test-functional --log-cli-level=0 --log-filter=-host
+
+ $ test/test-functional --log-cli-level=0 --log-filter=host,-host.*.1
+
+Run selected tests
+------------------
+
+.. code-block::
+
+ $ test/test-functional test/functional/test_cli_simple.py::test_bluetoothctl_script_show
+
+ $ test/test-functional -k test_bluetoothctl_script_show
+
+ $ test/test-functional -k 'test_btmgmt or test_bluetoothctl'
+
+Don't run tests with a given marker:
+
+.. code-block::
+
+ $ test/test-functional -m "not pipewire"
+
+Don't run known-failing tests:
+
+.. code-block::
+
+ $ test/test-functional -m "not xfail"
+
+Note that otherwise known-failing tests would be run, but with
+failures suppressed.
+
+Run previously failed and stop on failure
+-----------------------------------------
+
+.. code-block::
+
+ $ test/test-functional -x --ff
+
+List all tests
+--------------
+
+.. code-block::
+
+ $ test/test-functional --list
+
+Show errors from know-failing test
+----------------------------------
+
+.. code-block::
+
+ $ test/test-functional --runxfail -k test_btmgmt_info
+
+Redirect USB devices
+--------------------
+
+.. code-block::
+
+ $ test/test-functional --usb=hci0,hci1
+
+ $ export FUNCTIONAL_TESTING_CONTROLLERS=hci0,hci1
+ $ test/test-functional -vv
+
+This does not require running as root. Changing device permissions is
+sufficient. In verbose mode (``-vv``) some instructions are printed.
+
+Run all tests using the USB controllers:
+
+.. code-block::
+
+ $ test/test-functional --usb=hci0,hci1 --force-usb
+
+Run tests in parallel
+---------------------
+
+pytest-xdist is required for parallel execution. To run:
+
+.. code-block::
+
+ $ test/test-functional -n auto --dist loadgroup
+
+Logging in to a test VM instance
+--------------------------------
+
+While test is running:
+
+.. code-block::
+
+ $ test/test-functional-attach
+
+For this to be useful, usually, you need to pause the test
+e.g. by running with ``--trace`` option.
+
+To do it manually, when starting the tester will log a line like::
+
+ TTY: socat /tmp/bluez-func-test-q658swgi/bluez-func-test-tty-0 STDIO,rawer
+
+with the location of the socket where the serial is connected to.
+
+WRITING TESTS
+=============
+
+The functional tests are written in files (test modules) names
+`test/functional/test_*.py`. They are written using standard Pytest
+style. See https://docs.pytest.org/en/stable/getting-started.html
+
+See https://pypi.org/project/pytest-bluezenv/ for documentation of
+how to write VM-using tests.
+
+Use `Black <https://black.readthedocs.io/en/stable/>`__ to autoformat
+Python test code.
--
2.54.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH BlueZ v6 2/6] test: add functional/integration testing framework
2026-06-13 15:48 [PATCH BlueZ v6 0/6] Functional/integration testing Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 1/6] doc: add functional/integration testing documentation Pauli Virtanen
@ 2026-06-13 15:48 ` Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 3/6] build: add functional testing target Pauli Virtanen
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Pauli Virtanen @ 2026-06-13 15:48 UTC (permalink / raw)
To: linux-bluetooth; +Cc: Pauli Virtanen
Add framework for writing tests simulating "real" environments where
BlueZ and other parts of the stack run on different virtual machine
hosts that communicate with each other.
Add some smoke tests for bluetoothctl and btmgmt.
The implementation for the VM setup is maintained separately in the
pytest-bluezenv plugin, https://pypi.org/project/pytest-bluezenv
Implements:
- RPC communication with tester instances running each of the VM hosts,
so that tests can be written on the parent host which coordinates the
execution.
- Extensible way to add stateful test-specific code inside the VM
instances
- Logging control: output from different processes running inside the VM
are separated and can be filtered.
- Test runner framework with Pytest, factored into a pytest plugin
- Grouping tests to minimize VM reboots
- Redirecting USB controllers to use for testing
There is no requirement that the tests spawn VM instances.
---
test/functional/__init__.py | 2 +
test/functional/conftest.py | 48 ++++++++
test/functional/requirements.txt | 2 +
test/functional/test_bluetoothctl_vm.py | 152 ++++++++++++++++++++++++
test/functional/test_btmgmt_vm.py | 30 +++++
test/pytest.ini | 17 +++
test/test-functional | 21 ++++
test/test-functional-attach | 7 ++
8 files changed, 279 insertions(+)
create mode 100644 test/functional/__init__.py
create mode 100644 test/functional/conftest.py
create mode 100644 test/functional/requirements.txt
create mode 100644 test/functional/test_bluetoothctl_vm.py
create mode 100644 test/functional/test_btmgmt_vm.py
create mode 100644 test/pytest.ini
create mode 100755 test/test-functional
create mode 100755 test/test-functional-attach
diff --git a/test/functional/__init__.py b/test/functional/__init__.py
new file mode 100644
index 000000000..fe1c85178
--- /dev/null
+++ b/test/functional/__init__.py
@@ -0,0 +1,2 @@
+# -*- coding: utf-8; mode: python; eval: (blacken-mode); -*-
+# SPDX-License-Identifier: GPL-2.0-or-later
diff --git a/test/functional/conftest.py b/test/functional/conftest.py
new file mode 100644
index 000000000..196afa08d
--- /dev/null
+++ b/test/functional/conftest.py
@@ -0,0 +1,48 @@
+# -*- coding: utf-8; mode: python; eval: (blacken-mode); -*-
+# SPDX-License-Identifier: GPL-2.0-or-later
+import os
+import re
+from pathlib import Path
+
+
+def pytest_addoption(parser):
+ parser.addoption(
+ "--list",
+ action="store_true",
+ default=None,
+ help=("List tests"),
+ )
+
+
+def pytest_configure(config):
+ if config.option.list:
+ config.option.reportchars = "A"
+ config.option.no_header = True
+ config.option.verbose = -2
+
+
+COLLECT_ERRORS = []
+
+
+def pytest_collectreport(report):
+ if report.outcome != "passed":
+ COLLECT_ERRORS.append((report.outcome, report.fspath))
+
+
+def pytest_collection_finish(session):
+ if session.config.option.list:
+ cwd = Path(".").resolve()
+ root = session.config.rootpath.absolute()
+
+ regex = re.compile(r"\[.*")
+ names = set(
+ (root.joinpath(item.location[0]), regex.sub("", item.location[2]))
+ for item in session.items
+ )
+
+ for path, name in sorted(names):
+ print(f"{path.resolve().relative_to(cwd, walk_up=True)}::{name}")
+ for outcome, name in COLLECT_ERRORS:
+ print(f"{outcome.upper()} {name}")
+ print()
+ os._exit(0)
diff --git a/test/functional/requirements.txt b/test/functional/requirements.txt
new file mode 100644
index 000000000..cd66e74a8
--- /dev/null
+++ b/test/functional/requirements.txt
@@ -0,0 +1,2 @@
+pytest>=8
+pytest-bluezenv==0.1.6
diff --git a/test/functional/test_bluetoothctl_vm.py b/test/functional/test_bluetoothctl_vm.py
new file mode 100644
index 000000000..0ba75e9a9
--- /dev/null
+++ b/test/functional/test_bluetoothctl_vm.py
@@ -0,0 +1,152 @@
+# -*- coding: utf-8; mode: python; eval: (blacken-mode); -*-
+# SPDX-License-Identifier: GPL-2.0-or-later
+"""
+Tests for bluetoothctl using VM instances
+"""
+import sys
+import re
+import pytest
+import subprocess
+import tempfile
+import warnings
+
+import time
+import logging
+
+
+from pytest_bluezenv import host_config, find_exe, run, Bluetoothd, Bluetoothctl
+
+pytestmark = [pytest.mark.vm]
+
+bluetoothctl = find_exe("client", "bluetoothctl")
+
+bluetoothd_reuse_config = host_config([Bluetoothd()], reuse=True)
+
+
+@host_config(
+ [Bluetoothctl()],
+ [Bluetoothctl()],
+)
+def test_bluetoothctl_pair_bredr(hosts):
+ host0, host1 = hosts
+
+ host0.bluetoothctl.send("scan on\n")
+ host0.bluetoothctl.expect(f"Controller {host0.bdaddr.upper()} Discovering: yes")
+
+ host1.bluetoothctl.send("pairable on\n")
+ host1.bluetoothctl.expect("Changing pairable on succeeded")
+ host1.bluetoothctl.send("discoverable on\n")
+ host1.bluetoothctl.expect(f"Controller {host1.bdaddr.upper()} Discoverable: yes")
+
+ host0.bluetoothctl.expect(f"Device {host1.bdaddr.upper()}")
+ host0.bluetoothctl.send(f"pair {host1.bdaddr}\n")
+
+ idx, m = host0.bluetoothctl.expect(r"Confirm passkey (\d+).*:")
+ key = m[0].decode("utf-8")
+
+ host1.bluetoothctl.expect(f"Confirm passkey {key}")
+
+ host0.bluetoothctl.send("yes\n")
+ host1.bluetoothctl.send("yes\n")
+
+ host0.bluetoothctl.expect("Pairing successful")
+
+
+@host_config(
+ [Bluetoothd(conf="[General]\nControllerMode = le\n"), Bluetoothctl()],
+ [Bluetoothd(conf="[General]\nControllerMode = le\n"), Bluetoothctl()],
+)
+def test_bluetoothctl_pair_le(hosts):
+ host0, host1 = hosts
+
+ host0.bluetoothctl.send("scan on\n")
+ host0.bluetoothctl.expect(f"Controller {host0.bdaddr.upper()} Discovering: yes")
+
+ host1.bluetoothctl.send("advertise on\n")
+ host1.bluetoothctl.expect("Advertising object registered")
+
+ host0.bluetoothctl.expect(f"Device {host1.bdaddr.upper()}")
+ host0.bluetoothctl.send(f"pair {host1.bdaddr.upper()}\n")
+
+ # BUG!: if controller is power cycled off/on at boot (before bluetoothd)
+ # BUG!: which is what the tester here does,
+ # BUG!: bluetoothd MGMT command to enable Secure Connections Host Support
+ # BUG!: fails and we are left with legacy passkey. It seems we get randomly
+ # BUG!: one of these depending on what state controller/kernel were before
+ # BUG!: btmgmt power off/on
+
+ idx, m = host0.bluetoothctl.expect(
+ [r"\[agent\].*Passkey:.*m(\d+)", r"Confirm passkey (\d+).*:"]
+ )
+ key = m[0].decode("utf-8")
+
+ if idx == 0:
+ warnings.warn(
+ "BUG: we got passkey authentication, bluetoothd/kernel should be fixed"
+ )
+ host1.bluetoothctl.expect(r"\[agent\] Enter passkey \(number in 0-999999\):")
+ host1.bluetoothctl.send(f"{key}\n")
+ else:
+ host1.bluetoothctl.expect(f"Confirm passkey {key}")
+
+ host0.bluetoothctl.send("yes\n")
+ host1.bluetoothctl.send("yes\n")
+
+ host0.bluetoothctl.expect("Pairing successful")
+
+
+def run_bluetoothctl(*args):
+ return run(
+ [bluetoothctl] + list(args),
+ stdout=subprocess.PIPE,
+ stdin=subprocess.DEVNULL,
+ encoding="utf-8",
+ )
+
+
+def run_bluetoothctl_script(script):
+ with tempfile.NamedTemporaryFile(mode="w", encoding="utf-8") as f:
+ f.write(script)
+ f.write("\nquit")
+ f.flush()
+ return run_bluetoothctl("--init-script", f.name)
+
+
+@bluetoothd_reuse_config
+def test_bluetoothctl_show(hosts):
+ (host,) = hosts
+
+ result = host.call(run_bluetoothctl, f"show")
+ assert result.returncode == 0
+ assert f"Controller {host.bdaddr.upper()}" in result.stdout
+ assert "Powered: " in result.stdout
+ assert "Discoverable: no" in result.stdout
+
+
+@bluetoothd_reuse_config
+def test_bluetoothctl_list(hosts):
+ (host,) = hosts
+
+ result = host.call(run_bluetoothctl, "list")
+ assert result.returncode == 0
+ assert re.search(rf"{host.bdaddr.upper()}.*\[default\]", result.stdout)
+
+
+@bluetoothd_reuse_config
+def test_bluetoothctl_script_show(hosts):
+ (host,) = hosts
+
+ result = host.call(run_bluetoothctl_script, f"show")
+ assert result.returncode == 0
+ assert f"Controller {host.bdaddr.upper()}" in result.stdout
+ assert "Powered: " in result.stdout
+ assert "Discoverable: no" in result.stdout
+
+
+@bluetoothd_reuse_config
+def test_bluetoothctl_script_list(hosts):
+ (host,) = hosts
+
+ result = host.call(run_bluetoothctl_script, f"list")
+ assert result.returncode == 0
+ assert re.search(rf"{host.bdaddr.upper()}.*\[default\]", result.stdout)
diff --git a/test/functional/test_btmgmt_vm.py b/test/functional/test_btmgmt_vm.py
new file mode 100644
index 000000000..3d11d7616
--- /dev/null
+++ b/test/functional/test_btmgmt_vm.py
@@ -0,0 +1,30 @@
+# -*- coding: utf-8; mode: python; eval: (blacken-mode); -*-
+# SPDX-License-Identifier: GPL-2.0-or-later
+"""
+Tests for btmgmt using VM instances
+"""
+import sys
+import pytest
+import subprocess
+import tempfile
+
+from pytest_bluezenv import host_config, find_exe, run
+
+pytestmark = [pytest.mark.vm]
+
+btmgmt = find_exe("tools", "btmgmt")
+
+
+@host_config([])
+def test_btmgmt_info(hosts):
+ (host,) = hosts
+
+ result = host.call(
+ run,
+ [btmgmt, "--index", "0", "info"],
+ stdout=subprocess.PIPE,
+ stdin=subprocess.DEVNULL,
+ encoding="utf-8",
+ )
+ assert result.returncode == 0
+ assert f"addr {host.bdaddr.upper()}" in result.stdout
diff --git a/test/pytest.ini b/test/pytest.ini
new file mode 100644
index 000000000..0d4e48d69
--- /dev/null
+++ b/test/pytest.ini
@@ -0,0 +1,17 @@
+[pytest]
+log_format = %(asctime)s %(levelname)-6s %(name)-20s: %(message)s
+log_date_format = %Y-%m-%d %H:%M:%S.%f
+log_level = 0
+log_file = test-functional.log
+markers =
+ vm: tests requiring VM image
+
+addopts =
+ -p pytest_bluezenv
+
+# Default timeout
+vm_timeout = 30
+
+# Default sources for kernel-build when requested
+kernel_upstream = https://git.kernel.org/pub/scm/linux/kernel/git/bluetooth/bluetooth-next.git/
+kernel_branch = master
diff --git a/test/test-functional b/test/test-functional
new file mode 100755
index 000000000..95f5c57e9
--- /dev/null
+++ b/test/test-functional
@@ -0,0 +1,21 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# See doc/test-functional.rst
+#
+TESTDIR="$(dirname "$0")"
+SRCDIR="$TESTDIR/.."
+
+BUILDDIR=
+for d in "$SRCDIR" "$SRCDIR/build" "$SRCDIR/builddir"; do
+ if [ -f "$d/src/bluetoothd" ]; then
+ BUILDDIR="$d"
+ break
+ fi
+done
+
+if [ -n "$BUILDDIR" ] && [ -d "$BUILDDIR" ]; then
+ exec python3 -m pytest "$TESTDIR/functional" --bluez-src-dir "$SRCDIR" --bluez-build-dir "$BUILDDIR" "$@"
+else
+ exec python3 -m pytest "$TESTDIR/functional" --bluez-src-dir "$SRCDIR" "$@"
+fi
diff --git a/test/test-functional-attach b/test/test-functional-attach
new file mode 100755
index 000000000..6e65464f7
--- /dev/null
+++ b/test/test-functional-attach
@@ -0,0 +1,7 @@
+#!/bin/sh
+#
+# test-functional-attach
+#
+# Start Tmux and connect to active test-functional VM hosts.
+#
+exec python3 -mpytest_bluezenv attach "$@"
--
2.54.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH BlueZ v6 3/6] build: add functional testing target
2026-06-13 15:48 [PATCH BlueZ v6 0/6] Functional/integration testing Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 1/6] doc: add functional/integration testing documentation Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 2/6] test: add functional/integration testing framework Pauli Virtanen
@ 2026-06-13 15:48 ` Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 4/6] test: functional: impose Python code formatting Pauli Virtanen
` (2 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Pauli Virtanen @ 2026-06-13 15:48 UTC (permalink / raw)
To: linux-bluetooth; +Cc: Pauli Virtanen
This adds check-functional: target that runs the functional test suite.
Also add a --enable-functional-testing=<kernel-image> argument for
configure that can be used to include it in the check: make target,
possibly with a predefined kernel image.
---
Makefile.am | 10 ++++++++++
configure.ac | 22 ++++++++++++++++++++++
2 files changed, 32 insertions(+)
diff --git a/Makefile.am b/Makefile.am
index 76c4ab5d4..7920cae68 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -812,6 +812,16 @@ endif
TESTS = $(unit_tests)
AM_TESTS_ENVIRONMENT = MALLOC_CHECK_=3 MALLOC_PERTURB_=69
+check-functional: all
+ python3 -m pytest "$(srcdir)/test/functional" -v \
+ --kernel="$(FUNCTIONAL_TESTING_KERNEL)" \
+ --bluez-build-dir="$(top_builddir)" \
+ --bluez-src-dir="$(srcdir)"
+
+if FUNCTIONAL_TESTING
+check: check-functional
+endif
+
if DBUS_RUN_SESSION
AM_TESTS_ENVIRONMENT += dbus-run-session --
endif
diff --git a/configure.ac b/configure.ac
index 1cdd551f6..f50d8c9b3 100644
--- a/configure.ac
+++ b/configure.ac
@@ -407,6 +407,28 @@ if (test "${enable_testing}" = "yes"); then
#include <linux/net_tstamp.h>]])
fi
+AC_ARG_ENABLE(functional-testing, AS_HELP_STRING([--enable-functional-testing],
+ [enable functional testing tools]),
+ [enable_functional_testing=yes; functional_testing_kernel=${enableval}],
+ [enable_functional_testing=no])
+AM_CONDITIONAL(FUNCTIONAL_TESTING, test "${enable_functional_testing}" = "yes")
+AC_ARG_VAR(FUNCTIONAL_TESTING_KERNEL, [vmlinux image to use for functional testing])
+FUNCTIONAL_TESTING_KERNEL=${functional_testing_kernel}
+
+if (test "${enable_functional_testing}" = "yes"); then
+ if (test "${enable_client}" = "no" || \
+ test "${enable_tools}" != "yes" || \
+ test "${enable_testing}" != "yes"); then
+ AC_MSG_ERROR([--enable-functional-testing requires --enable-client --enable-tools --enable-testing])
+ fi
+ AC_MSG_CHECKING([pytest and dependencies])
+ python3 -m pip install --dry-run --no-index -r "${srcdir}/test/functional/requirements.txt" >/dev/null
+ if (test "$?" != "0"); then
+ AC_MSG_ERROR([pytest or dependencies missing])
+ fi
+ AC_MSG_RESULT([ok])
+fi
+
AC_ARG_ENABLE(experimental, AS_HELP_STRING([--enable-experimental],
[enable experimental tools]),
[enable_experimental=${enableval}])
--
2.54.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH BlueZ v6 4/6] test: functional: impose Python code formatting
2026-06-13 15:48 [PATCH BlueZ v6 0/6] Functional/integration testing Pauli Virtanen
` (2 preceding siblings ...)
2026-06-13 15:48 ` [PATCH BlueZ v6 3/6] build: add functional testing target Pauli Virtanen
@ 2026-06-13 15:48 ` Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 5/6] test: functional: add some Agent1 interface tests Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 6/6] test: functional: add basic obex file transfer tests Pauli Virtanen
5 siblings, 0 replies; 8+ messages in thread
From: Pauli Virtanen @ 2026-06-13 15:48 UTC (permalink / raw)
To: linux-bluetooth; +Cc: Pauli Virtanen
Check Python code formatting of the functional test suite.
---
test/functional/test_tests.py | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
create mode 100644 test/functional/test_tests.py
diff --git a/test/functional/test_tests.py b/test/functional/test_tests.py
new file mode 100644
index 000000000..561b04703
--- /dev/null
+++ b/test/functional/test_tests.py
@@ -0,0 +1,23 @@
+# -*- coding: utf-8; mode: python; eval: (blacken-mode); -*-
+# SPDX-License-Identifier: GPL-2.0-or-later
+"""
+Tests for the test suite itself
+"""
+import sys
+import subprocess
+import warnings
+from pathlib import Path
+
+import pytest
+
+
+def test_formatting():
+ pytest.importorskip("black")
+
+ result = subprocess.run(
+ [sys.executable, "-mblack", "--check", "--diff", "-q", Path(__file__).parent],
+ stdout=subprocess.PIPE,
+ encoding="utf-8",
+ )
+ if result.returncode != 0:
+ warnings.warn(f"Formatting incorrect:\n{result.stdout}")
--
2.54.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH BlueZ v6 5/6] test: functional: add some Agent1 interface tests
2026-06-13 15:48 [PATCH BlueZ v6 0/6] Functional/integration testing Pauli Virtanen
` (3 preceding siblings ...)
2026-06-13 15:48 ` [PATCH BlueZ v6 4/6] test: functional: impose Python code formatting Pauli Virtanen
@ 2026-06-13 15:48 ` Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 6/6] test: functional: add basic obex file transfer tests Pauli Virtanen
5 siblings, 0 replies; 8+ messages in thread
From: Pauli Virtanen @ 2026-06-13 15:48 UTC (permalink / raw)
To: linux-bluetooth; +Cc: Pauli Virtanen
Add test
test/functional/test_agent.py::test_agent_pair_bredr
---
test/functional/test_agent.py | 46 +++++++++++++++++++++++++++++++++++
1 file changed, 46 insertions(+)
create mode 100644 test/functional/test_agent.py
diff --git a/test/functional/test_agent.py b/test/functional/test_agent.py
new file mode 100644
index 000000000..24593090b
--- /dev/null
+++ b/test/functional/test_agent.py
@@ -0,0 +1,46 @@
+# -*- coding: utf-8; mode: python; eval: (blacken-mode); -*-
+# SPDX-License-Identifier: GPL-2.0-or-later
+"""
+Tests for bluetoothctl using VM instances
+"""
+import sys
+import re
+import pytest
+import subprocess
+import tempfile
+
+import time
+import logging
+
+
+from pytest_bluezenv import host_config, Agent, wait_until
+
+pytestmark = [pytest.mark.vm]
+
+
+@host_config([Agent()], [Agent()])
+@pytest.mark.parametrize("success", [True, False], ids=["accept", "reject"])
+def test_agent_pair_bredr(hosts, success):
+ host0, host1 = hosts
+
+ host0.agent.adapter_method("StartDiscovery")
+ host0.agent.expect("org.bluez.Adapter1.StartDiscovery:reply")
+
+ host1.agent.adapter_set("Pairable", True)
+ host1.agent.adapter_set("Discoverable", True)
+
+ wait_until(host0.agent.has_device, host1.bdaddr)
+
+ host0.agent.device_method(host1.bdaddr, "Pair")
+
+ confirm_0 = host0.agent.expect("org.bluez.Agent1.RequestConfirmation")
+ confirm_1 = host1.agent.expect("org.bluez.Agent1.RequestConfirmation")
+ assert confirm_0.passkey == confirm_1.passkey
+ host0.agent.reply()
+
+ if success:
+ host1.agent.reply()
+ host0.agent.expect("org.bluez.Device1.Pair:reply")
+ else:
+ host1.agent.reply_error()
+ host0.agent.expect("org.bluez.Device1.Pair:error")
--
2.54.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH BlueZ v6 6/6] test: functional: add basic obex file transfer tests
2026-06-13 15:48 [PATCH BlueZ v6 0/6] Functional/integration testing Pauli Virtanen
` (4 preceding siblings ...)
2026-06-13 15:48 ` [PATCH BlueZ v6 5/6] test: functional: add some Agent1 interface tests Pauli Virtanen
@ 2026-06-13 15:48 ` Pauli Virtanen
5 siblings, 0 replies; 8+ messages in thread
From: Pauli Virtanen @ 2026-06-13 15:48 UTC (permalink / raw)
To: linux-bluetooth; +Cc: Pauli Virtanen
Add tests for Obex DBus API and obexctl
test/functional/test_obex.py::test_obex_ftp_get
test/functional/test_obex.py::test_obex_ftp_list
test/functional/test_obex.py::test_obexctl_list
---
test/functional/test_obex.py | 285 +++++++++++++++++++++++++++++++++++
1 file changed, 285 insertions(+)
create mode 100644 test/functional/test_obex.py
diff --git a/test/functional/test_obex.py b/test/functional/test_obex.py
new file mode 100644
index 000000000..fcb9105e5
--- /dev/null
+++ b/test/functional/test_obex.py
@@ -0,0 +1,285 @@
+# -*- coding: utf-8; mode: python; eval: (blacken-mode); -*-
+# SPDX-License-Identifier: LGPL-2.1-or-later
+"""
+Tests for Obex
+"""
+import sys
+import os
+import re
+import pytest
+import subprocess
+import tempfile
+import time
+import logging
+import json
+import dbus
+import threading
+from pathlib import Path
+
+import pytest
+
+from pytest_bluezenv import (
+ HostPlugin,
+ Agent,
+ host_config,
+ find_exe,
+ Bluetoothd,
+ Bluetoothctl,
+ Obexd,
+ LogStream,
+ wait_until,
+ mainloop_wrap,
+ mainloop_assert,
+ Event,
+ EventPluginMixin,
+ dbus_service_event_method,
+ Pexpect,
+ utils,
+)
+
+pytestmark = [pytest.mark.vm]
+
+log = logging.getLogger(__name__)
+
+
+BUS_NAME = "org.bluez.obex"
+PATH = "/org/bluez/obex"
+AGENT_MANAGER_INTERFACE = "org.bluez.obex.AgentManager1"
+AGENT_INTERFACE = "org.bluez.obex.Agent1"
+CLIENT_INTERFACE = "org.bluez.obex.Client1"
+SESSION_INTERFACE = "org.bluez.obex.Session1"
+FILE_TRANSFER_INTERFACE = "org.bluez.obex.FileTransfer1"
+TRANSFER_INTERFACE = "org.bluez.obex.Transfer1"
+
+FTP_UUID = "00001106-0000-1000-8000-00805f9b34fb"
+
+
+class ObexAgent(HostPlugin, EventPluginMixin):
+ depends = [Bluetoothd()]
+ name = "obex_agent"
+
+ def __init__(self, path="/obexagent"):
+ self.path = path
+
+ @mainloop_wrap
+ def setup(self, impl):
+ EventPluginMixin.setup(self, impl)
+
+ self.bus = dbus.SessionBus()
+ self.bus.set_exit_on_disconnect(False)
+
+ self.agent = ObexAgentObject(self.bus, self.path, self.events)
+
+ bluez = self.bus.get_object(BUS_NAME, PATH)
+ self.manager = dbus.Interface(bluez, AGENT_MANAGER_INTERFACE)
+ self.manager.RegisterAgent(self.path)
+
+ log.info("Obex agent registered")
+
+ def cleanup(self):
+ path = Path("/run/obex")
+ for f in path.iterdir():
+ f.unlink()
+
+
+def agent_method(*a, **kw):
+ return dbus_service_event_method(AGENT_INTERFACE, *a, **kw)
+
+
+class ObexAgentObject(dbus.service.Object):
+ @mainloop_assert
+ def __init__(self, bus, path, events):
+ self.events = events
+ super().__init__(bus, path)
+
+ AuthorizePush = agent_method("AuthorizePush", ("path",), "o", "s", sync=False)
+ Cancel = agent_method("Cancel")
+
+
+def write_obex_file(name, content):
+ with open(f"/run/obex/{name}", "w") as f:
+ f.write(content)
+
+
+def read_file(name):
+ with open(name, "r") as f:
+ return f.read()
+
+
+#
+# Direct Obex Python client API tests
+#
+
+
+class ObexClient(HostPlugin, EventPluginMixin):
+ name = "obex"
+
+ @mainloop_wrap
+ def setup(self, impl):
+ EventPluginMixin.setup(self, impl)
+
+ self.transferred = 0
+ self.transfer_path = None
+ self.transfer_size = 0
+
+ self.bus = dbus.SessionBus()
+ self.bus.set_exit_on_disconnect(False)
+ self.log = logging.getLogger(self.name)
+ self.client = dbus.Interface(
+ self.bus.get_object(BUS_NAME, PATH), CLIENT_INTERFACE
+ )
+
+ self.bus.add_signal_receiver(
+ self.properties_changed,
+ dbus_interface="org.freedesktop.DBus.Properties",
+ signal_name="PropertiesChanged",
+ path_keyword="path",
+ )
+
+ @mainloop_wrap
+ def connect(self, bdaddr):
+ def reply(path):
+ obj = self.bus.get_object(BUS_NAME, path)
+ self.session = dbus.Interface(obj, SESSION_INTERFACE)
+ self.ftp = dbus.Interface(obj, FILE_TRANSFER_INTERFACE)
+
+ self._object_method(
+ self.client, "CreateSession", bdaddr, {"Target": "ftp"}, reply_handler=reply
+ )
+
+ @mainloop_assert
+ def properties_changed(self, interface, properties, invalidated, path):
+ if path != self.transfer_path:
+ return
+
+ if "Status" in properties and (
+ properties["Status"] == "complete" or properties["Status"] == "error"
+ ):
+ self.events.put(
+ Event(
+ f"{FILE_TRANSFER_INTERFACE}:{properties['Status']}",
+ properties=properties,
+ )
+ )
+ self.log.debug(f"Transfer {properties['Status']}")
+
+ if "Transferred" not in properties:
+ return
+
+ value = properties["Transferred"]
+ speed = (value - self.transferred) / 1000
+ self.log.debug(
+ f"Transfer progress {value}/{self.transfer_size} at {speed} kBps"
+ )
+ self.transferred = value
+
+ @mainloop_wrap
+ def ftp_list_folder(self):
+ return self.ftp.ListFolder()
+
+ @mainloop_wrap
+ def ftp_get_file(self, dst, src):
+ path, properties = self.ftp.GetFile(dst, src)
+ self.transfer_path = path
+ self.transfer_size = properties["Size"]
+ return properties["Filename"]
+
+
+@pytest.fixture
+def paired_hosts(hosts):
+ from .test_agent import test_agent_pair_bredr
+
+ if hosts[0].agent.has_device(hosts[1].bdaddr):
+ return hosts
+
+ test_agent_pair_bredr(hosts, True)
+ return hosts
+
+
+obex_host_config = host_config(
+ [Agent(), Obexd(), ObexClient(), Pexpect()],
+ [Agent(), Obexd(), ObexAgent()],
+ reuse=True,
+)
+
+
+@pytest.fixture
+def obex_hosts(paired_hosts):
+ host0, host1 = paired_hosts
+
+ if hasattr(host0, "session"):
+ return paired_hosts
+
+ host0.obex.connect(host1.bdaddr)
+
+ service = host1.agent.expect("org.bluez.Agent1.AuthorizeService")
+ assert service.uuid == FTP_UUID
+ host1.agent.reply()
+
+ host0.obex.expect("org.bluez.obex.Client1.CreateSession:reply")
+
+ yield paired_hosts
+
+ host1.obex_agent.cleanup()
+
+
+@obex_host_config
+def test_obex_ftp_list(obex_hosts):
+ host0, host1 = obex_hosts
+
+ host1.call(write_obex_file, "test", "1234")
+
+ (item,) = host0.obex.ftp_list_folder()
+ assert item["Type"] == "file"
+ assert item["Name"] == "test"
+ assert item["Size"] == 4
+
+
+@obex_host_config
+def test_obex_ftp_get(obex_hosts):
+ host0, host1 = obex_hosts
+
+ host1.call(write_obex_file, "test", "1234")
+
+ filename = host0.obex.ftp_get_file("", "test")
+ host0.obex.expect("org.bluez.obex.FileTransfer1:complete")
+ assert host0.call(read_file, filename) == "1234"
+
+
+#
+# obexctl tests
+#
+
+
+@pytest.fixture
+def obexctl(obex_hosts):
+ host0, host1 = obex_hosts
+
+ exe = find_exe("tools", "obexctl")
+ obexctl = host0.pexpect.spawn([exe])
+
+ obexctl.expect("Client /org/bluez/obex")
+ obexctl.send(f"connect {host1.bdaddr} {FTP_UUID}\n")
+
+ service = host1.agent.expect("org.bluez.Agent1.AuthorizeService")
+ assert service.uuid == FTP_UUID
+ host1.agent.reply()
+
+ obexctl.expect("Connection successful")
+ obexctl.send(f"select /org/bluez/obex/client/session1\n")
+
+ yield obexctl
+
+ obexctl.close()
+
+
+@obex_host_config
+def test_obexctl_list(obex_hosts, obexctl):
+ host0, host1 = obex_hosts
+
+ host1.call(write_obex_file, "test", "1234")
+
+ obexctl.send(f"ls\n")
+ obexctl.expect(f"Type: file")
+ obexctl.expect(f"Name: test")
+ obexctl.expect(f"Size: 4")
--
2.54.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* RE: Functional/integration testing
2026-06-13 15:48 ` [PATCH BlueZ v6 1/6] doc: add functional/integration testing documentation Pauli Virtanen
@ 2026-06-13 17:09 ` bluez.test.bot
0 siblings, 0 replies; 8+ messages in thread
From: bluez.test.bot @ 2026-06-13 17:09 UTC (permalink / raw)
To: linux-bluetooth, pav
[-- Attachment #1: Type: text/plain, Size: 1990 bytes --]
This is automated email and please do not reply to this email!
Dear submitter,
Thank you for submitting the patches to the linux bluetooth mailing list.
This is a CI test results with your patch series:
PW Link:https://patchwork.kernel.org/project/bluetooth/list/?series=1111076
---Test result---
Test Summary:
CheckPatch FAIL 2.09 seconds
GitLint PASS 1.46 seconds
BuildEll PASS 21.37 seconds
BluezMake PASS 624.17 seconds
MakeCheck PASS 19.31 seconds
MakeDistcheck PASS 238.73 seconds
CheckValgrind PASS 276.72 seconds
CheckSmatch PASS 328.01 seconds
bluezmakeextell PASS 167.45 seconds
IncrementalBuild PASS 644.71 seconds
ScanBuild PASS 973.07 seconds
Details
##############################
Test: CheckPatch - FAIL
Desc: Run checkpatch.pl script
Output:
[BlueZ,v6,2/6] test: add functional/integration testing framework
ERROR:EXECUTE_PERMISSIONS: do not set execute permissions for source files
#422: FILE: test/test-functional
ERROR:EXECUTE_PERMISSIONS: do not set execute permissions for source files
#449: FILE: test/test-functional-attach
/github/workspace/src/patch/14627534.patch total: 2 errors, 0 warnings, 279 lines checked
NOTE: For some of the reported defects, checkpatch may be able to
mechanically convert to the typical style using --fix or --fix-inplace.
/github/workspace/src/patch/14627534.patch has style problems, please review.
NOTE: Ignored message types: COMMIT_MESSAGE COMPLEX_MACRO CONST_STRUCT FILE_PATH_CHANGES MISSING_SIGN_OFF PREFER_PACKED SPDX_LICENSE_TAG SPLIT_STRING SSCANF_TO_KSTRTO
NOTE: If any of the errors are false positives, please report
them to the maintainer, see CHECKPATCH in MAINTAINERS.
https://github.com/bluez/bluez/pull/2228
---
Regards,
Linux Bluetooth
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-06-13 17:09 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-06-13 15:48 [PATCH BlueZ v6 0/6] Functional/integration testing Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 1/6] doc: add functional/integration testing documentation Pauli Virtanen
2026-06-13 17:09 ` Functional/integration testing bluez.test.bot
2026-06-13 15:48 ` [PATCH BlueZ v6 2/6] test: add functional/integration testing framework Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 3/6] build: add functional testing target Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 4/6] test: functional: impose Python code formatting Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 5/6] test: functional: add some Agent1 interface tests Pauli Virtanen
2026-06-13 15:48 ` [PATCH BlueZ v6 6/6] test: functional: add basic obex file transfer tests Pauli Virtanen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox