Re: [PATCH 1/3] fio-tests: add multi-filesystem testing support

[Daniel Gomez <da.gomez@kernel.org>]

On 20/11/2025 04.15, Luis Chamberlain wrote:
> This merges the long-pending fio-tests filesystem support patch that adds
> comprehensive filesystem-specific performance testing capabilities to
> kdevops. The implementation allows testing filesystem optimizations,
> block size configurations, and I/O patterns against actual mounted
> filesystems rather than just raw block devices.
> 
> The implementation follows the proven mmtests architecture patterns with
> modular Kconfig files and tag-based ansible task organization, avoiding
> the proliferation of separate playbook files that would make maintenance
> more complex.
> 
> Key filesystem testing features include XFS support with configurable
> block sizes from 4K to 64K with various sector sizes and modern features
> like reflink and rmapbt. The ext4 support provides both standard and
> bigalloc configurations with different cluster sizes. For btrfs, modern
> features including no-holes, free-space-tree, and compression options
> are available.
> 
> The multi-filesystem section-based testing enables comprehensive
> performance comparison across different filesystem configurations by
> creating separate VMs for each configuration. This includes support for
> XFS block size comparisons, comprehensive XFS block size analysis, and
> cross-filesystem comparisons between XFS, ext4, and btrfs.
> 
> Node generation for multi-filesystem testing uses dynamic detection
> based on enabled sections, creating separate VM nodes for each enabled
> section with proper Ansible groups for each filesystem configuration.
> A/B testing support is included across all configurations.
> 
> Results collection and analysis is handled through specialized tooling
> with performance overview across filesystems, block size performance
> heatmaps, IO depth scaling analysis, and statistical summaries with CSV
> exports.
> 
> The patch has been updated to work with the current codebase which now
> uses workflow-specific template includes for host file generation rather
> than embedding all workflow templates in a single hosts.j2 file. The
> fio-tests specific template has been enhanced with multi-filesystem
> support while maintaining backward compatibility with single filesystem
> testing.
> 
> Generated-by: Claude AI
> Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
> ---
>  .github/workflows/fio-tests.yml               |  98 +++
>  CLAUDE.md                                     | 401 ++++++++++++
>  PROMPTS.md                                    | 344 ++++++++++
>  defconfigs/fio-tests-fs-btrfs-zstd            |  25 +
>  defconfigs/fio-tests-fs-ext4-bigalloc         |  24 +
>  defconfigs/fio-tests-fs-ranges                |  24 +
>  defconfigs/fio-tests-fs-xfs                   |  74 +++
>  defconfigs/fio-tests-fs-xfs-4k-vs-16k         |  57 ++
>  defconfigs/fio-tests-fs-xfs-all-blocksizes    |  63 ++
>  defconfigs/fio-tests-fs-xfs-all-fsbs          |  57 ++
>  defconfigs/fio-tests-fs-xfs-vs-ext4-vs-btrfs  |  57 ++
>  defconfigs/fio-tests-quick                    |  74 +++
>  playbooks/fio-tests-graph-host.yml            |  76 +++
>  playbooks/fio-tests-graph.yml                 | 168 +++--
>  playbooks/fio-tests-multi-fs-compare.yml      | 140 ++++
>  .../fio-tests/fio-multi-fs-compare.py         | 434 +++++++++++++
>  .../tasks/install-deps/debian/main.yml        |   1 +
>  .../tasks/install-deps/redhat/main.yml        |   1 +
>  .../tasks/install-deps/suse/main.yml          |   1 +
>  playbooks/roles/fio-tests/tasks/main.yaml     | 430 ++++++++++---
>  .../roles/fio-tests/templates/fio-job.ini.j2  |  31 +-
>  playbooks/roles/gen_hosts/tasks/main.yml      |  60 ++
>  .../templates/workflows/fio-tests.j2          |  66 ++
>  playbooks/roles/gen_nodes/tasks/main.yml      | 100 ++-
>  workflows/fio-tests/Kconfig                   | 370 ++++++++---
>  workflows/fio-tests/Kconfig.btrfs             |  87 +++
>  workflows/fio-tests/Kconfig.ext4              | 114 ++++
>  workflows/fio-tests/Kconfig.fs                |  75 +++
>  workflows/fio-tests/Kconfig.xfs               | 170 +++++
>  workflows/fio-tests/Makefile                  |  65 +-
>  .../scripts/generate_comparison_graphs.py     | 605 ++++++++++++++++++
>  .../generate_comprehensive_analysis.py        | 297 +++++++++
>  workflows/fio-tests/sections.conf             |  47 ++
>  33 files changed, 4350 insertions(+), 286 deletions(-)
>  create mode 100644 .github/workflows/fio-tests.yml
>  create mode 100644 defconfigs/fio-tests-fs-btrfs-zstd
>  create mode 100644 defconfigs/fio-tests-fs-ext4-bigalloc
>  create mode 100644 defconfigs/fio-tests-fs-ranges
>  create mode 100644 defconfigs/fio-tests-fs-xfs
>  create mode 100644 defconfigs/fio-tests-fs-xfs-4k-vs-16k
>  create mode 100644 defconfigs/fio-tests-fs-xfs-all-blocksizes
>  create mode 100644 defconfigs/fio-tests-fs-xfs-all-fsbs
>  create mode 100644 defconfigs/fio-tests-fs-xfs-vs-ext4-vs-btrfs
>  create mode 100644 defconfigs/fio-tests-quick
>  create mode 100644 playbooks/fio-tests-graph-host.yml
>  create mode 100644 playbooks/fio-tests-multi-fs-compare.yml
>  create mode 100644 playbooks/python/workflows/fio-tests/fio-multi-fs-compare.py
>  create mode 100644 workflows/fio-tests/Kconfig.btrfs
>  create mode 100644 workflows/fio-tests/Kconfig.ext4
>  create mode 100644 workflows/fio-tests/Kconfig.fs
>  create mode 100644 workflows/fio-tests/Kconfig.xfs
>  create mode 100755 workflows/fio-tests/scripts/generate_comparison_graphs.py
>  create mode 100644 workflows/fio-tests/scripts/generate_comprehensive_analysis.py
>  create mode 100644 workflows/fio-tests/sections.conf
> 
> diff --git a/.github/workflows/fio-tests.yml b/.github/workflows/fio-tests.yml
> new file mode 100644
> index 00000000..0a7c0234
> --- /dev/null
> +++ b/.github/workflows/fio-tests.yml

This is not following the current CI modular approach.
It should be way easier to support a new workflow.

tree .github/workflows
.github/workflows
├── config-tests.yml
└── kdevops.yml

1 directory, 2 files

Check kdevops.yml, it has this modular and reusable design. Steps are split
in actions:

tree .github/actions
.github/actions
├── archive
│   └── action.yml
├── bringup
│   └── action.yml
├── build-test
│   └── action.yml
├── cleanup
│   └── action.yml
├── configure
│   └── action.yml
├── linux
│   └── action.yml
└── test
    └── action.yml

8 directories, 7 files

I know the documentation is still lacking, but Claude should be able to parse
the directory structure and understand what's needed.

In short: add the new defconfig option to the ci_workflow list in kdevops.yml.
Make sure the workflow is implemented in .github/actions/test/action.yml,
right now we support blktests, fstests, and selftests. Also ensure the matching
defconfig and .ci/ mappings exist so CI can resolve them correctly.

Clarification: we have 2 CI modes: the quick one for kdevops workflow validation
called "kdevops-ci". And the one used for linux kernel validation called
"linux-ci". That's why the test/action.yml needs to declare TESTS= so it goes
quickly for kdevops-ci test mode. If you also want this to run daily (linux-next
and Linus' latest tag) you can also augment the ci-matrix: strategy: matrix:
ci_workflow on .github/workflows/kdevops.yml. We currently only have support
for blktests.

Hopefully we can use this thread and example for docs and PROMPT. Can you split
the patch so, CI support is added atomically and after the workflow support
is added?

> @@ -0,0 +1,98 @@
> +name: Run fio-tests on self-hosted runner
> +
> +on:
> +  push:
> +    branches:
> +      - '**'
> +  pull_request:
> +    branches:
> +      - '**'
> +  workflow_dispatch:  # Add this for manual triggering of the workflow