Re: Integration testing for BlueZ

[Bastien Nocera <hadess@hadess.net>]

On Mon, 2026-03-30 at 12:59 +0000, Pauli Virtanen wrote:
> > Do you still have a copy of the above file somewhere? I wanted to
> > finish porting them into my own integration tests which I'm hoping
> > to
> > add to the regression/unit tests that are usually run for every
> > commit/MR.
> 
> https://github.com/pv/bluez/blob/func-test-v3/test/functional/test_bluetoothctl_vm.py
> 
> and test_btmgmt_vm.py
> 
> Although these don't currently contain much, but could be extended.
> 
> FWIW I'd maybe consider using pytest also for Python regression &
> unit tests, at least if you can make them run without root.
> 
> There's interesting bluetoothd mock dbus library by the bluez-alsa
> developer, which looks like it could be useful for bluetoothctl/etc
> utility testing
> 
> <https://github.com/Samsung/bluezoo>

I'm pretty familiar with python-dbusmock which includes a bluetoothd
mock server which can be used as non-root:
https://github.com/martinpitt/python-dbusmock

It's used extensively in the gnome-bluetooth test suite:
https://gitlab.gnome.org/GNOME/gnome-bluetooth/-/blob/master/tests/integration-test.py?ref_type=heads
along with its mock upower implementation.

bluezoo looks more like a complete replacement including some behaviour
when running certain commands. The python-dbusmock implementation is
voluntarily simpler.

The questions at the end of the day boils down to *what* we want to
test:

1) Just like the gnome-bluetooth test suite, we could only care for our
own code, bluetoothd and upower are mocked, no specific hardware or
permissions is required.

This usually requires tweaking the mock implementation to better match
the behaviour of the real thing.

2) As I've implemented in this integration test:
https://github.com/hadess/bluez/blob/wip/hadess/add-meson-ell-wrap/unit/integration-test.py
we test command-line tools against the real bluetoothd which we
compile, it requires either root (to run btvirt if needed, and run
bluetoothd), or a machine with a real/mocked hardware and bluetoothd
already running. Ideally, with root access, you're testing the
"integration" of the command-line tools and bluetoothd.
Do we want to run those against a mocked bluetoothd that could get out
of sync with reality?

This wouldn't allow us to test bluetoothd itself as I've done in one
test, but maybe that's good to allow us to run those tests in more
circumstances?

My goal was being able to run "meson test" on my local development
machine without worrying about a layer of possibly outdated mocking
code.

I'd be fine running those tests that could be against a mocked
bluetoothd, if that means more tests being run.

I'd expect both those first options to be run for every merge request
or patch set, possibly even for every commit if we run into errors.

3) We have your variant of the test suite that does what 2) does but
with many more moving parts, and pretty high requirements in terms of
setup (and possibly resources). The only thing that's mocked is the
Bluetooth adapters, and not even for all tests.


In short, I think I might, before sending a patch out, re-focus the
basic tools tests to run against a mocked dbusmock bluez, so they
always run, whatever the environment.

Next, we can try to figure out whether some tests like the bluetoothd
startup warning one can be shared between the "run on every merge
request" (option 2) vs. "run once a week or once a day" heavier duty
tests (option 3).