[PATCH blktests] bcache: add bcache/001

[PATCH blktests] bcache: add bcache/001

[Daniel Wagner]

So far we are missing tests for bcache. Besides a relative simple
setup/teardown tests add also the corresponding infrastructure. More
tests are to be expected to depend on this.

_create_bcache/_remove_bcache are tracking the resources and if anything
is missing it will complain.

Signed-off-by: Daniel Wagner <dwagner@suse.de>
---
The initial test is very simple but it alreayd shows some troubles.

After running make-bcache, sometime there is immediatly some kernel logs and
this is the good case. User land finds all devices and thus is happy.

Though when there is not activities (for seconds), user space complains about
missing devices. I have added some debugging stuff in this verison so there is
some more info to help figure out what's going on.

"""
bcache/001 => nvme0n1 vdb vdc (test bcache setup and teardown) [failed]
    runtime  67.538s  ...  67.479s
    --- tests/bcache/001.out    2026-01-20 12:57:37.151032990 +0000
    +++ /tmp/blktests/nvme0n1_vdb_vdc/bcache/001.out.bad        2026-01-20 13:00:11.129159120 +0000
    @@ -1 +1,32 @@
     Running bcache/001
    +WARNING: Could not find device node for UUID c23b55bf-0619-4825-ada6-8d26c57210bb after 60s
    +$ make-bcache --bucket 64k --block 4k --cache /dev/nvme0n1 --bdev /dev/vdb
    +UUID:                      1b8ed2a8-4bed-4d55-a9de-1da3d5d60922
    +Set UUID:          c9fee0e5-b330-443a-8d8a-66750c1d4b86
    +version:           0
    +nbuckets:          65536
    ...
    (Run 'diff -u tests/bcache/001.out /tmp/blktests/nvme0n1_vdb_vdc/bcache/001.out.bad' to see the entire diff)

[ 2882.431357] run blktests bcache/001 at 2026-01-20 12:59:10
[...] make-bcache runs shortly after the test starts though we start to see logs way later:
[ 2946.570731] bcache: run_cache_set() invalidating existing data
[ 2946.576450] bcache: register_cache() registered cache device nvme0n1
[ 2946.582442] bcache: register_bdev() registered backing device vdb
[ 2946.590678] bcache: bch_cached_dev_attach() Caching vdb as bcache0 on set ed5c9c6d-a4f8-41fe-b6a2-73433b561796
[ 2946.599839] bcache: register_bdev() registered backing device vdc
[ 2946.609148] bcache: bch_cached_dev_attach() Caching vdc as bcache1 on set ed5c9c6d-a4f8-41fe-b6a2-73433b561796
[ 2946.774259] bcache: bcache_device_free() bcache0 stopped
[ 2946.818099] bcache: bcache_device_free() bcache1 stopped
[ 2947.181586] bcache: cache_set_free() Cache set ed5c9c6d-a4f8-41fe-b6a2-73433b561796 unregistered

Running bcache/001
WARNING: Could not find device node for UUID c23b55bf-0619-4825-ada6-8d26c57210bb after 60s
$ make-bcache --bucket 64k --block 4k --cache /dev/nvme0n1 --bdev /dev/vdb
UUID:                   1b8ed2a8-4bed-4d55-a9de-1da3d5d60922
Set UUID:               c9fee0e5-b330-443a-8d8a-66750c1d4b86
version:                0
nbuckets:               65536
block_size:             8
bucket_size:            128
nr_in_set:              1
nr_this_dev:            0
first_bucket:           1
UUID:                   c23b55bf-0619-4825-ada6-8d26c57210bb
Set UUID:               c9fee0e5-b330-443a-8d8a-66750c1d4b86
version:                1
block_size:             8
data_offset:            16

/sys/fs/bcache*:
total 0
--w------- 1 root root 4096 Jan 20 12:12 pendings_cleanup
--w------- 1 root root 4096 Jan 20 12:11 register
--w------- 1 root root 4096 Jan 20 12:12 register_quiet

/sys/block/bcache*:
ls: cannot access '/sys/block/bcache*': No such file or directory

/dev/bcache*:
ls: cannot access '/dev/bcache*': No such file or directory

/dev/bcache/by-uuid/:
ls: cannot access '/dev/bcache/by-uuid/': No such file or directory
"""

With one cache and one bdev device this happens pretty much everytime. I had
some more luck using one cache and two bdev devices which has a successrate of
roughly 50%.

Note: to run this tests blktests TEST_CASE_DEV_ARRAY needs to be set:

  # cat config
  TEST_CASE_DEV_ARRAY[bcache/*]="/dev/nvme0n1 /dev/vdb /dev/vdc"

# ./check -c config bcache
 
---
 tests/bcache/001     |  32 ++++++++
 tests/bcache/001.out |   1 +
 tests/bcache/rc      | 225 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 258 insertions(+)

diff --git a/tests/bcache/001 b/tests/bcache/001
new file mode 100644
index 000000000000..a723f14f80bf
--- /dev/null
+++ b/tests/bcache/001
@@ -0,0 +1,32 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2026 Daniel Wagner, SUSE Labs
+#
+# Test bcache setup and teardown
+
+. tests/bcache/rc
+
+DESCRIPTION="test bcache setup and teardown"
+
+requires() {
+		_bcache_requires
+}
+
+test_device_array() {
+	echo "Running ${TEST_NAME}"
+
+	if [[ ${#TEST_DEV_ARRAY[@]} -lt 3 ]]; then
+		SKIP_REASONS+=("requires at least 3 devices")
+		return 1
+	fi
+
+	 _create_bcache \
+		--cache "${TEST_DEV_ARRAY[0]##*/}" \
+		--bdev "${TEST_DEV_ARRAY[1]##*/}"
+     _remove_bcache
+
+	 _create_bcache \
+	 	--cache "${TEST_DEV_ARRAY[0]##*/}" \
+	 	--bdev "${TEST_DEV_ARRAY[1]##*/}" "${TEST_DEV_ARRAY[2]##*/}"
+     _remove_bcache
+}
diff --git a/tests/bcache/001.out b/tests/bcache/001.out
new file mode 100644
index 000000000000..f890aed2736c
--- /dev/null
+++ b/tests/bcache/001.out
@@ -0,0 +1 @@
+Running bcache/001
diff --git a/tests/bcache/rc b/tests/bcache/rc
new file mode 100644
index 000000000000..cd71edf1042c
--- /dev/null
+++ b/tests/bcache/rc
@@ -0,0 +1,225 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2026 Daniel Wagner, SUSE Labs
+
+. common/rc
+
+declare -a BCACHE_DEVS=()
+declare -a BCACHE_BDEVS=()
+declare -a BCACHE_CSETS=()
+
+BCACHE_MAX_RETRIES=60
+
+_bcache_requires() {
+	_have_kernel_options MD BCACHE BCACHE_DEBUG AUTOFS_FS
+	_have_program make-bcache
+	_have_crypto_algorithm crc32c
+}
+
+_create_bcache() {
+    local -a cdevs=()
+    local -a bdevs=()
+    local bucket_size="64k"
+    local block_size="4k"
+
+	_register_test_cleanup _cleanup_bcache
+
+	while [[ $# -gt 0 ]]; do
+        case $1 in
+            --cache)
+                shift
+                # Collect arguments until the next flag or end of input
+                while [[ $# -gt 0 && ! $1 =~ ^-- ]]; do
+                    cdevs+=("$1")
+                    shift
+                done
+                ;;
+            --bdev)
+                shift
+                # Collect arguments until the next flag or end of input
+                while [[ $# -gt 0 && ! $1 =~ ^-- ]]; do
+                    bdevs+=("$1")
+                    shift
+                done
+                ;;
+            --bucket-size)
+                bucket_size="$2"
+                shift 2
+                ;;
+            --block-size)
+                block_size="$2"
+                shift 2
+                ;;
+            --writeback)
+				ARGS+=(--writeback)
+                shift 1
+                ;;
+			--discard)
+				ARGS+=(--discard)
+				shift 1
+				;;
+            *)
+                echo "WARNING: unknown argument: $1"
+                shift
+                ;;
+        esac
+    done
+
+	# add /dev prefix to device names
+	cdevs=( "${cdevs[@]/#/\/dev\/}" )
+	bdevs=( "${bdevs[@]/#/\/dev\/}" )
+
+	# make-bcache expects empty/cleared devices
+	local dev
+    for dev in "${cdevs[@]}" "${bdevs[@]}"; do
+    	wipefs --all --quiet "${dev}"
+		BCACHE_DEVS+=("${dev}")
+    done
+
+    local -a cdevs_args=()
+    for dev in "${cdevs[@]}"; do
+        cdevs_args+=("--cache" "${dev}")
+    done
+
+	local -a bdevs_args=()
+    for dev in "${bdevs[@]}"; do
+        bdevs_args+=("--bdev" "${dev}")
+    done
+
+	local output cmd
+	cmd=(make-bcache \
+		--bucket "${bucket_size}" \
+		--block "${block_size}" \
+		"${cdevs_args[@]}" \
+        "${bdevs_args[@]}" \
+		"${ARGS[@]}")
+
+	output=$("${cmd[@]}" 2>&1)
+	local rc=$?
+    if [[ "${rc}" -ne 0 ]]; then
+        echo "ERROR: make-bcache failed:" >&2
+        echo "$output" >&2
+        return 1
+    fi
+
+	local cset_uuid
+	cset_uuid=$(echo "$output" | awk '/Set UUID:/ {print $3}' | head -n 1)
+	if [[ -z "${cset_uuid}" ]]; then
+        echo "ERROR: Could not extract cset UUID from make-bcache output" >&2
+        return 1
+    fi
+	BCACHE_CSETS+=("${cset_uuid}")
+
+	local -a bdev_uuids
+	mapfile -t bdev_uuids < <(echo "$output" | awk '
+  	  $1 == "UUID:" { last_uuid = $2 }
+      $1 == "version:" && $2 == "1" { print last_uuid}
+	')
+
+	udevadm settle
+
+	for uuid in "${bdev_uuids[@]}"; do
+    	local link found attempt
+
+		link=/dev/bcache/by-uuid/"${uuid}"
+		found=false
+		attempt=0
+
+	    while (( attempt < BCACHE_MAX_RETRIES )); do
+    	    if [[ -L "$link" ]]; then
+        	    BCACHE_BDEVS+=("${uuid}")
+            	found=true
+				break
+			fi
+
+		    (( attempt++ ))
+			sleep 1
+    	done
+
+		if [[ "$found" == "false" ]]; then
+    	   echo "bcache: ${uuid} not found" > /dev/kmsg
+		   echo "WARNING: Could not find device node for UUID ${uuid} after ${BCACHE_MAX_RETRIES}s" >&2
+		   echo "\$ ${cmd[*]}" >&2
+		   echo "${output}" >&2
+		   echo "" >&2
+		   echo "/sys/fs/bcache*:" >&2
+		   ls -l /sys/fs/bcache* >&2
+		   echo "" >&2
+		   echo "/sys/block/bcache*:" >&2
+		   ls -l /sys/block/bcache* >&2
+		   echo "" >&2
+		   echo "/dev/bcache*:" >&2
+		   ls -l /dev/bcache* >&2
+		   echo "" >&2
+		   echo "/dev/bcache/by-uuid/:" >&2
+		   ls -l /dev/bcache/by-uuid/ >&2
+		fi
+	done
+}
+
+_remove_bcache() {
+    local uuid
+
+	for uuid in "${BCACHE_BDEVS[@]}"; do
+        local link full_path dev
+
+		link=/dev/disk/by-uuid/"${uuid}"
+  		full_path=$(readlink -f "${link}")
+	    dev="${full_path##*/}"
+
+		if [ -f /sys/block/"${dev}"/bcache/stop ] ; then
+		   echo 1 > /sys/block/"${dev}"/bcache/stop
+		fi
+	done
+
+	for uuid in "${BCACHE_CSETS[@]}"; do
+		if [ -f /sys/fs/bcache/"${uuid}"/unregister ] ; then
+		   echo 1 > /sys/fs/bcache/"${uuid}"/unregister
+		fi
+    done
+
+	udevadm settle
+	# XXX sic!
+	sleep 3
+
+	for dev in "${BCACHE_DEVS[@]}"; do
+        wipefs --all --quiet "${dev}"
+    done
+
+	BCACHE_CSETS=()
+	BCACHE_BDEVS=()
+}
+
+_cleanup_bcache() {
+    local cset dev
+
+	shopt -s nullglob
+    for dev in /sys/block/bcache* ; do
+        [ -e "${dev}" ] || continue
+
+        dev=$(basename "${dev}")
+        echo "WARNING: bcache device ${dev} found"
+
+        if [[ -f /sys/block/"${dev}"/bcache/stop ]]; then
+            echo 1 > /sys/block/"${dev}"/bcache/stop 2>/dev/null || true
+        fi
+    done
+
+	for cset in /sys/fs/bcache/*-*-*-*-*; do
+        if [[ -d "${cset}" ]]; then
+			echo "WARNING: Unregistering cset $(basename "${cset}")"
+            echo 1 > "${cset}"/unregister 2>/dev/null || true
+        fi
+    done
+    shopt -u nullglob
+
+    udevadm settle
+
+    for dev in "${BCACHE_DEVS[@]}"; do
+        if ! wipefs --all --force "${dev}"; then
+           echo "ERROR: Failed to wipe ${dev}. It may still be in use." >&2
+        fi
+    done
+
+	BCACHE_DEVS=()
+}

---
base-commit: e387a7e0169cc012eb6a7140a0561d2901c92a76
change-id: 20260120-bcache-35ec7368c8f4

Best regards,
-- 
Daniel Wagner <dwagner@suse.de>

Re: [PATCH blktests] bcache: add bcache/001

[Christoph Hellwig]

On Tue, Jan 20, 2026 at 02:28:18PM +0100, Daniel Wagner wrote:
> So far we are missing tests for bcache. Besides a relative simple
> setup/teardown tests add also the corresponding infrastructure. More
> tests are to be expected to depend on this.
> 
> _create_bcache/_remove_bcache are tracking the resources and if anything
> is missing it will complain.

Not really an expert on bcache, and not that much in blktests either,
but having some testing for bcache readabily available is fantastic!

> Note: to run this tests blktests TEST_CASE_DEV_ARRAY needs to be set:
> 
>   # cat config
>   TEST_CASE_DEV_ARRAY[bcache/*]="/dev/nvme0n1 /dev/vdb /dev/vdc"

Can you add a blurb to Documentation/running-tests.md about this?

I also wonder if we could have a generic TEST_CASE_DEV_ARRAY that
would just work for all tests needing multiple devices?

Re: [PATCH blktests] bcache: add bcache/001

[Johannes Thumshirn]

On 1/20/26 3:01 PM, Daniel Wagner wrote:
> +
> +_cleanup_bcache() {
> +    local cset dev
> +
> +	shopt -s nullglob
> +    for dev in /sys/block/bcache* ; do
> +        [ -e "${dev}" ] || continue
> +
> +        dev=$(basename "${dev}")
> +        echo "WARNING: bcache device ${dev} found"
> +
> +        if [[ -f/sys/block/"${dev}"/bcache/stop ]]; then
> +            echo 1 >/sys/block/"${dev}"/bcache/stop 2>/dev/null || true
> +        fi
> +    done
> +
> +	for cset in/sys/fs/bcache/*-*-*-*-*; do
> +        if [[ -d "${cset}" ]]; then
> +			echo "WARNING: Unregistering cset $(basename "${cset}")"
> +            echo 1 > "${cset}"/unregister 2>/dev/null || true
> +        fi
> +    done
> +    shopt -u nullglob

It's either my MUA or there's some mix of space and tabs here.

Re: [PATCH blktests] bcache: add bcache/001

[Daniel Wagner]

On Wed, Jan 21, 2026 at 08:19:08AM +0000, Johannes Thumshirn wrote:
> It's either my MUA or there's some mix of space and tabs here.

I had a mix of tabs and spaces and didn't realize it before sending it
out. I've converted it all to spaces. I'll sent an new version, so it's
actually readable. Sorry!

Re: [PATCH blktests] bcache: add bcache/001

[Daniel Wagner]

On Tue, Jan 20, 2026 at 11:56:33PM -0800, Christoph Hellwig wrote:
> On Tue, Jan 20, 2026 at 02:28:18PM +0100, Daniel Wagner wrote:
> > So far we are missing tests for bcache. Besides a relative simple
> > setup/teardown tests add also the corresponding infrastructure. More
> > tests are to be expected to depend on this.
> > 
> > _create_bcache/_remove_bcache are tracking the resources and if anything
> > is missing it will complain.
> 
> Not really an expert on bcache, and not that much in blktests either,
> but having some testing for bcache readabily available is fantastic!

I think so too. Anyway, I'll hope, I figure out to make it more
stable/consistent. 
 
> > Note: to run this tests blktests TEST_CASE_DEV_ARRAY needs to be set:
> > 
> >   # cat config
> >   TEST_CASE_DEV_ARRAY[bcache/*]="/dev/nvme0n1 /dev/vdb /dev/vdc"
> 
> Can you add a blurb to Documentation/running-tests.md about this?
> 
> I also wonder if we could have a generic TEST_CASE_DEV_ARRAY that
> would just work for all tests needing multiple devices?

Sure, I'll update the documentation.

On this note, It took me a while to understand that using
TEST_DEVS="/dev/nvme0n1 /dev/vdb /dev/vdc" is not populating the
TEST_DEV_ARRAY array.

Commit 653ace845911 ("check, new: introduce test_device_array()")
explains why:

    As to the test target devices defined in TEST_DEVS variable, blktests
    assumes that each test case with test_device() function is run for each
    single device defined in TEST_DEVS. On the other hand, it is suggested
    to support a test case for not a single device but multiple devices.

Maybe we could add a default config with all options listed and
documented but commented out.

Re: [PATCH blktests] bcache: add bcache/001

[Christoph Hellwig]

On Wed, Jan 21, 2026 at 01:48:27PM +0100, Daniel Wagner wrote:
> Sure, I'll update the documentation.
> 
> On this note, It took me a while to understand that using
> TEST_DEVS="/dev/nvme0n1 /dev/vdb /dev/vdc" is not populating the
> TEST_DEV_ARRAY array.
> 
> Commit 653ace845911 ("check, new: introduce test_device_array()")
> explains why:
> 
>     As to the test target devices defined in TEST_DEVS variable, blktests
>     assumes that each test case with test_device() function is run for each
>     single device defined in TEST_DEVS. On the other hand, it is suggested
>     to support a test case for not a single device but multiple devices.
> 
> Maybe we could add a default config with all options listed and
> documented but commented out.

The default config would be useful for sure.

But I also thing the TEST_DEVS vs TEST_DEV_ARRAY thing is weird, and the
fact that you need to declare the array for multiple tests doesn't help.

IMHO having a TEST_DEV_ARRAY should imply that normal single device tests
pick the first one from it if not explicit TEST_DEVS is set, and tests
using multiple devices can grab as many as they support from it.  That
would mirror what SCRATCH_DEV_POOL does in xfstests, which works very
well.

I'd love to help with this, but I'm not sure my bash abilities are
enough for this :(

Re: [PATCH blktests] bcache: add bcache/001

[Shinichiro Kawasaki]

On Jan 21, 2026 / 22:35, Christoph Hellwig wrote:
> On Wed, Jan 21, 2026 at 01:48:27PM +0100, Daniel Wagner wrote:
> > Sure, I'll update the documentation.
> > 
> > On this note, It took me a while to understand that using
> > TEST_DEVS="/dev/nvme0n1 /dev/vdb /dev/vdc" is not populating the
> > TEST_DEV_ARRAY array.
> > 
> > Commit 653ace845911 ("check, new: introduce test_device_array()")
> > explains why:
> > 
> >     As to the test target devices defined in TEST_DEVS variable, blktests
> >     assumes that each test case with test_device() function is run for each
> >     single device defined in TEST_DEVS. On the other hand, it is suggested
> >     to support a test case for not a single device but multiple devices.
> > 
> > Maybe we could add a default config with all options listed and
> > documented but commented out.
> 
> The default config would be useful for sure.

Agreed. It can be the file named "config.example".

> But I also thing the TEST_DEVS vs TEST_DEV_ARRAY thing is weird, and the
> fact that you need to declare the array for multiple tests doesn't help.
> 
> IMHO having a TEST_DEV_ARRAY should imply that normal single device tests
> pick the first one from it if not explicit TEST_DEVS is set, and tests
> using multiple devices can grab as many as they support from it.  That
> would mirror what SCRATCH_DEV_POOL does in xfstests, which works very
> well.
> 
> I'd love to help with this, but I'm not sure my bash abilities are
> enough for this :(

When I introduced TEST_DEV_ARRAY, the motivation was the test case md/003. It
was the first test case that required multiple devices for testing. It tests
"NMVe Atomic Writes with MD devices". My assumption was that users wants to
specify test target devices unique to md/003 which are different devices from
other test cases (special NVMe with atomic write capability). Based on this
understanding, I introduced the associative array config option
TEST_CASE_DEV_ARRAY[X/0??]="/dev/Y /dev/Z", which can specify test target
devices unique to each test case (or multiple test cases specified with regular
expression). I think this TEST_CASE_DEV_ARRAY fits the use case of md/003.

On the other hand, reading the comment above, I find there is another different
expectation for testing with multiple devices. As to bcache/* test cases, the
test target devices can be common to other test groups. I can understand that
something similar as SCRATCH_DEV_POOL of fstests is desired to reduce the number
of config options to specify. We maybe able to introduce the new config option
"TEST_DEV_POOL", which will work as described above: for most of the test
cases with test_device() function, blktests can pick up only the first device
from TEST_DEV_POOL. If the test case has test_device_array(), all devices in
TEST_DEV_POOL can be provided to test_device_array() through TEST_DEV_ARRAY.
The user can specify only TEST_DEV_POOL, to cover both test cases with
test_device() and those with test_device_array().

I will try to find my time to prototype TEST_DEV_POOL to see if the expectation
can be fulfilled.