* [PATCH] workflows: add fio-tests performance tests
@ 2025-07-26 3:40 Luis Chamberlain
0 siblings, 0 replies; only message in thread
From: Luis Chamberlain @ 2025-07-26 3:40 UTC (permalink / raw)
To: Chuck Lever, Daniel Gomez, Viacheslav Dubeyko, Vincent Fu,
Kundan Kumar, Anuj Gupta, shoaibasif, kdevops
Cc: Luis Chamberlain
I had written fio-tests [0] a long time ago but although it used Kconfig,
at that point I had lacked the for-sight of leveraging jinja2 / ansible
to make things more declarative. It's been on my back log to try to get
that ported over to kdevops but now with Claude Code, it just required
a series of prompts. And its even better now. *This* is how we scale.
I've added a demo tree which just has the graphs for those just itching
to see what this produces [1]. It's just a demo comparing 6.15.0 vs
6.16.0-rc7 so don't get too excited as its just a silly guest with
virtio drives. However this should hopefully show you how easily and
quickly with the new A/B testing feature I just posted (not yet merged,
but that this series depends on) we can do full performance analysis.
Key features:
- Configurable test matrix: block sizes (4K-128K), IO depths (1-64), job counts
- Multiple workload patterns: random/sequential read/write, mixed workloads
- Advanced configuration: IO engines, direct IO, fsync options
- Performance logging: bandwidth, IOPS, and latency metrics
- Baseline management and results analysis
- FIO_TESTS_ENABLE_GRAPHING: Enable/disable graphing capabilities
- Graph format, DPI, and theme configuration options
- Updated CI defconfig with graphing support (150 DPI for faster CI)
Key graphing features support:
- Performance analysis: bandwidth heatmaps, IOPS scaling, latency distributions
- A/B comparison: baseline vs development configuration analysis
- Trend analysis: block size scaling, IO depth optimization, correlation
matrices
- Configurable output: PNG/SVG/PDF formats, DPI settings, matplotlib themes
Documentation:
- docs/fio-tests.md: Comprehensive workflow documentation covering:
* Origin story and relationship to upstream fio-tests project
* Quick start and configuration examples
* Test matrix configuration and device setup
* A/B testing and baseline management
* Graphing and visualization capabilities
* CI integration and troubleshooting guides
* Best practices and performance considerations
A minimal CI configuration (defconfig-fio-tests-ci) enables automated testing
in GitHub Actions using /dev/null as the target device with a reduced test
matrix for fast execution.
We extend PROMPTS.md with prompts used for all this.
Usage:
make defconfig-fio-tests-ci # Simple CI testing
make menuconfig # Interactive configuration
make fio-tests # Run performance tests
make fio-tests-baseline # Establish baseline
make fio-tests-results # Collect results
make fio-tests-graph # Generate performance graphs
make fio-tests-compare # Compare baseline vs dev results
make fio-tests-trend-analysis # Analyze performance trends
Generated-by: Claude AI
Link: https://github.com/mcgrof/fio-tests # [0]
Link: https://github.com/mcgrof/fio-tests-graphs-on-kdevops # [1]
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
If you want to take this for a spin:
https://github.com/linux-kdevops/kdevops/tree/fio-tests-v1
.github/workflows/docker-tests.yml | 6 +
PROMPTS.md | 108 +++++
README.md | 10 +
defconfigs/fio-tests-ci | 75 ++++
docs/fio-tests.md | 377 ++++++++++++++++++
kconfigs/workflows/Kconfig | 27 ++
playbooks/fio-tests-baseline.yml | 29 ++
playbooks/fio-tests-compare.yml | 33 ++
playbooks/fio-tests-graph.yml | 43 ++
playbooks/fio-tests-results.yml | 87 ++++
playbooks/fio-tests-trend-analysis.yml | 30 ++
playbooks/fio-tests.yml | 9 +
.../python/workflows/fio-tests/fio-compare.py | 289 ++++++++++++++
.../python/workflows/fio-tests/fio-plot.py | 276 +++++++++++++
.../workflows/fio-tests/fio-trend-analysis.py | 350 ++++++++++++++++
playbooks/roles/fio-tests/defaults/main.yml | 48 +++
.../tasks/install-deps/debian/main.yml | 20 +
.../fio-tests/tasks/install-deps/main.yml | 3 +
.../tasks/install-deps/redhat/main.yml | 20 +
.../tasks/install-deps/suse/main.yml | 20 +
playbooks/roles/fio-tests/tasks/main.yaml | 92 +++++
.../roles/fio-tests/templates/fio-job.ini.j2 | 18 +
playbooks/roles/gen_hosts/tasks/main.yml | 13 +
playbooks/roles/gen_hosts/templates/hosts.j2 | 34 ++
playbooks/roles/gen_nodes/tasks/main.yml | 32 ++
workflows/Makefile | 4 +
workflows/fio-tests/Kconfig | 359 +++++++++++++++++
workflows/fio-tests/Makefile | 53 +++
28 files changed, 2465 insertions(+)
create mode 100644 defconfigs/fio-tests-ci
create mode 100644 docs/fio-tests.md
create mode 100644 playbooks/fio-tests-baseline.yml
create mode 100644 playbooks/fio-tests-compare.yml
create mode 100644 playbooks/fio-tests-graph.yml
create mode 100644 playbooks/fio-tests-results.yml
create mode 100644 playbooks/fio-tests-trend-analysis.yml
create mode 100644 playbooks/fio-tests.yml
create mode 100755 playbooks/python/workflows/fio-tests/fio-compare.py
create mode 100755 playbooks/python/workflows/fio-tests/fio-plot.py
create mode 100755 playbooks/python/workflows/fio-tests/fio-trend-analysis.py
create mode 100644 playbooks/roles/fio-tests/defaults/main.yml
create mode 100644 playbooks/roles/fio-tests/tasks/install-deps/debian/main.yml
create mode 100644 playbooks/roles/fio-tests/tasks/install-deps/main.yml
create mode 100644 playbooks/roles/fio-tests/tasks/install-deps/redhat/main.yml
create mode 100644 playbooks/roles/fio-tests/tasks/install-deps/suse/main.yml
create mode 100644 playbooks/roles/fio-tests/tasks/main.yaml
create mode 100644 playbooks/roles/fio-tests/templates/fio-job.ini.j2
create mode 100644 workflows/fio-tests/Kconfig
create mode 100644 workflows/fio-tests/Makefile
diff --git a/.github/workflows/docker-tests.yml b/.github/workflows/docker-tests.yml
index c0e0d03d..adea1182 100644
--- a/.github/workflows/docker-tests.yml
+++ b/.github/workflows/docker-tests.yml
@@ -53,3 +53,9 @@ jobs:
echo "Running simple make targets on ${{ matrix.distro_container }} environment"
make mrproper
+ - name: Test fio-tests defconfig
+ run: |
+ echo "Testing fio-tests CI configuration"
+ make defconfig-fio-tests-ci
+ make
+ echo "Configuration test passed for fio-tests"
diff --git a/PROMPTS.md b/PROMPTS.md
index 554215f9..fbe79dd7 100644
--- a/PROMPTS.md
+++ b/PROMPTS.md
@@ -124,6 +124,114 @@ source "workflows/mmtests/Kconfig.fs"
This separation is preferred as it helps us scale.
+## Port an external project into kdevops
+
+The fio-tests was an older external project however its more suitably placed
+into kdevops as jinja2 lets us easily scale this project. The projects also
+were authored by the same person and the same license was used. The porting
+took a few separate prompts as described below.
+
+### Initial implementation of fio-tests workflow on kdevops
+
+**Prompt:**
+Now that we merged steady state to kdevops -- now let's add specific target
+workflow support for different target different simple workflows. Learn from
+how sysbench added two guests so we can do A/B testing in two separate guests.
+The workflows you will focus on will be the workflows from
+https://github.com/mcgrof/fio-tests. We already took steady state and
+pre-conditioning from there so no need to do that. All we need to do is just
+now target the different other workflows. Leverage the Kconfig documentation we
+used on that project and adapt it to leverage output yaml on kdevops. Then also
+to help test things we can simply add a basic test so that
+.github/workflows/docker-tests.yml can run some tests using /dev/null as a
+target block device for just one simple workflow.
+
+**AI:** Claude Code
+**Commit:** TDB
+**Result:** Excellent implementation with comprehensive workflow structure.
+**Grading:** 90%
+
+**Notes:**
+
+The implementation successfully:
+- Added complete fio-tests workflow with A/B testing support following sysbench patterns
+- Created comprehensive Kconfig structure with output yaml support for all options
+- Implemented configurable test matrices (block sizes, IO depths, job counts, patterns)
+- Added ansible role with template-based job generation
+- Integrated with main kdevops workflow system and makefiles
+- Created CI-optimized defconfig using /dev/null target device
+- Updated GitHub Actions workflow for automated testing
+
+Minor areas for improvement:
+- Could have included more detailed help text in some Kconfig options
+- Template generation could be more dynamic for complex configurations
+- Didn't add documentation, which means we should extend CLAUDE.md to
+ add documentation when adding a new workflow.
+- Did not pick up on the trend to prefer to have 'make foo-results' to always
+ copy results locally.
+- Not sure why it didn't pick up on the trend that we can simplify grammatically
+ by using tags to limits scope on ansible, instead of using all these odd
+ small playbooks.
+
+### Extend fio-tests with graphing support
+
+**Prompt:**
+The fio-tests project had support for graphing. Bring that over and add that to
+kdevops. I am the author of fio-tests so I own all the code. Be sure to use
+SPDX for my top header files with the copyleft-next license as is done with
+tons of code on kdevops.
+
+**AI:** Claude Code
+**Commit:** TDB
+**Result:** Comprehensive graphing implementation with proper licensing.
+**Grading:** 95%
+
+**Notes:**
+
+Outstanding implementation that:
+- Improved upon the graphs I had originally had on fio-tests and actually
+ innovated on some! Also took the initiative to do A/B performance analysis!
+- Created three comprehensive Python scripts with proper SPDX copyleft-next-0.3.1 headers
+- Implemented advanced graphing: performance analysis, A/B comparison, trend analysis
+- Added configurable graphing options through Kconfig (format, DPI, themes)
+- Included conditional dependency installation across distributions
+- Created ansible playbooks for automated graph generation
+- Added make targets for different types of analysis
+- Updated CI configuration with graphing support
+
+The implementation perfectly followed kdevops patterns and demonstrated
+excellent understanding of the codebase structure. The graphing capabilities
+are comprehensive and production-ready.
+
+### Add the fio-tests documentation
+
+**Prompt:**
+Now add documentation for fio-tests on kdevops. Extend README.md with a small
+section and point to its own documentation file. You can use the upstream
+fio-tests https://github.com/mcgrof/fio-tests page for inspiration, but
+obviously we want to port this to how you've implemented support on kdevops.
+You can point back to the old https://github.com/mcgrof/fio-tests page as an
+origin story. Also extend PROMPTS.md with the few prompts I've given you to
+help add support for fio-tests and graphing support.
+
+**AI:** Claude Code
+**Commit:** TDB
+**Result:** Comprehensive documentation with examples and troubleshooting.
+**Grading:** 90%
+
+**Notes:**
+
+The documentation implementation includes:
+- Updated README.md with fio-tests section linking to detailed documentation
+- Created comprehensive docs/fio-tests.md with full workflow coverage
+- Included origin story referencing original fio-tests framework
+- Added detailed configuration examples and troubleshooting guides
+- Documented all graphing capabilities with usage examples
+- Extended PROMPTS.md with the implementation prompts for future AI reference
+
+This demonstrates the complete lifecycle of implementing a complex workflow in
+kdevops from initial implementation through comprehensive documentation.
+
## Kernel development and A/B testing support
### Adding A/B kernel testing support for different kernel versions
diff --git a/README.md b/README.md
index 15bb7119..741a0d3b 100644
--- a/README.md
+++ b/README.md
@@ -12,6 +12,7 @@ Table of Contents
* [Runs some kernel selftests in a parallel manner](#runs-some-kernel-selftests-in-a-parallel-manner)
* [CXL](#cxl)
* [sysbench](#sysbench)
+ * [fio-tests](#fio-tests)
* [kdevops chats](#kdevops-chats)
* [kdevops on discord](#kdevops-on-discord)
* [kdevops IRC](#kdevops-irc)
@@ -229,6 +230,15 @@ kdevops supports automation of sysbench tests on VMs with or without
providers. For details refer to the
[kdevops sysbench documentation](docs/sysbench/sysbench.md).
+### fio-tests
+
+kdevops includes comprehensive storage performance testing through the fio-tests
+workflow, adapted from the original [fio-tests framework](https://github.com/mcgrof/fio-tests).
+This workflow provides flexible I/O benchmarking with configurable test matrices,
+A/B testing capabilities, and advanced graphing and visualization support. For
+detailed configuration and usage information, refer to the
+[kdevops fio-tests documentation](docs/fio-tests.md).
+
## kdevops chats
We use discord and IRC. Right now we have more folks on discord than on IRC.
diff --git a/defconfigs/fio-tests-ci b/defconfigs/fio-tests-ci
new file mode 100644
index 00000000..403fb34d
--- /dev/null
+++ b/defconfigs/fio-tests-ci
@@ -0,0 +1,75 @@
+# Minimal fio-tests configuration for CI testing
+CONFIG_KDEVOPS_FIRST_RUN=n
+CONFIG_LIBVIRT=y
+CONFIG_LIBVIRT_URI="qemu:///system"
+CONFIG_LIBVIRT_HOST_PASSTHROUGH=y
+CONFIG_LIBVIRT_MACHINE_TYPE_DEFAULT=y
+CONFIG_LIBVIRT_CPU_MODEL_PASSTHROUGH=y
+CONFIG_LIBVIRT_VCPUS=2
+CONFIG_LIBVIRT_RAM=2048
+CONFIG_LIBVIRT_OS_VARIANT="generic"
+CONFIG_LIBVIRT_STORAGE_POOL_PATH_CUSTOM=n
+CONFIG_LIBVIRT_STORAGE_POOL_CREATE=y
+CONFIG_LIBVIRT_EXTRA_STORAGE_DRIVE_NVME=n
+CONFIG_LIBVIRT_EXTRA_STORAGE_DRIVE_VIRTIO=n
+CONFIG_LIBVIRT_EXTRA_STORAGE_DRIVE_IDE=n
+CONFIG_LIBVIRT_EXTRA_STORAGE_DRIVE_SCSI=n
+
+# Network configuration
+CONFIG_KDEVOPS_NETWORK_TYPE_NATUAL_BRIDGE=y
+
+# Workflow configuration
+CONFIG_WORKFLOWS=y
+CONFIG_WORKFLOWS_TESTS=y
+CONFIG_WORKFLOWS_LINUX_TESTS=y
+CONFIG_WORKFLOWS_DEDICATED_WORKFLOW=y
+CONFIG_KDEVOPS_WORKFLOW_DEDICATE_FIO_TESTS=y
+
+# fio-tests specific config for CI
+CONFIG_FIO_TESTS_PERFORMANCE_ANALYSIS=y
+CONFIG_FIO_TESTS_DEVICE="/dev/null"
+CONFIG_FIO_TESTS_RUNTIME="10"
+CONFIG_FIO_TESTS_RAMP_TIME="2"
+
+# Minimal test matrix for CI
+CONFIG_FIO_TESTS_BS_4K=y
+CONFIG_FIO_TESTS_BS_8K=n
+CONFIG_FIO_TESTS_BS_16K=n
+CONFIG_FIO_TESTS_BS_32K=n
+CONFIG_FIO_TESTS_BS_64K=n
+CONFIG_FIO_TESTS_BS_128K=n
+
+CONFIG_FIO_TESTS_IODEPTH_1=y
+CONFIG_FIO_TESTS_IODEPTH_4=n
+CONFIG_FIO_TESTS_IODEPTH_8=n
+CONFIG_FIO_TESTS_IODEPTH_16=n
+CONFIG_FIO_TESTS_IODEPTH_32=n
+CONFIG_FIO_TESTS_IODEPTH_64=n
+
+CONFIG_FIO_TESTS_NUMJOBS_1=y
+CONFIG_FIO_TESTS_NUMJOBS_2=n
+CONFIG_FIO_TESTS_NUMJOBS_4=n
+CONFIG_FIO_TESTS_NUMJOBS_8=n
+CONFIG_FIO_TESTS_NUMJOBS_16=n
+
+CONFIG_FIO_TESTS_PATTERN_RAND_READ=y
+CONFIG_FIO_TESTS_PATTERN_RAND_WRITE=n
+CONFIG_FIO_TESTS_PATTERN_SEQ_READ=n
+CONFIG_FIO_TESTS_PATTERN_SEQ_WRITE=n
+CONFIG_FIO_TESTS_PATTERN_MIXED_75_25=n
+CONFIG_FIO_TESTS_PATTERN_MIXED_50_50=n
+
+CONFIG_FIO_TESTS_IOENGINE="io_uring"
+CONFIG_FIO_TESTS_DIRECT=y
+CONFIG_FIO_TESTS_FSYNC_ON_CLOSE=y
+CONFIG_FIO_TESTS_RESULTS_DIR="/data/fio-tests"
+CONFIG_FIO_TESTS_LOG_AVG_MSEC=1000
+
+# Graphing configuration
+CONFIG_FIO_TESTS_ENABLE_GRAPHING=y
+CONFIG_FIO_TESTS_GRAPH_FORMAT="png"
+CONFIG_FIO_TESTS_GRAPH_DPI=150
+CONFIG_FIO_TESTS_GRAPH_THEME="default"
+
+# Baseline/dev testing setup
+CONFIG_KDEVOPS_BASELINE_AND_DEV=y
\ No newline at end of file
diff --git a/docs/fio-tests.md b/docs/fio-tests.md
new file mode 100644
index 00000000..3383d81a
--- /dev/null
+++ b/docs/fio-tests.md
@@ -0,0 +1,377 @@
+# kdevops fio-tests workflow
+
+kdevops includes comprehensive storage performance testing through the fio-tests
+workflow, providing flexible I/O benchmarking with configurable test matrices,
+A/B testing capabilities, and advanced graphing and visualization support.
+
+## Origin and inspiration
+
+The fio-tests workflow in kdevops is adapted from the original
+[fio-tests framework](https://github.com/mcgrof/fio-tests), which was designed
+to provide systematic storage performance testing with dynamic test generation
+and comprehensive analysis capabilities. The kdevops implementation brings
+these capabilities into the kdevops ecosystem with seamless integration to
+support virtualization, cloud providers, and bare metal testing.
+
+## Overview
+
+The fio-tests workflow enables comprehensive storage device performance testing
+by generating configurable test matrices across multiple dimensions:
+
+- **Block sizes**: 4K, 8K, 16K, 32K, 64K, 128K
+- **I/O depths**: 1, 4, 8, 16, 32, 64
+- **Job counts**: 1, 2, 4, 8, 16 concurrent fio jobs
+- **Workload patterns**: Random/sequential read/write, mixed workloads
+- **A/B testing**: Baseline vs development configuration comparison
+
+## Quick start
+
+### Basic configuration
+
+Configure fio-tests for quick testing:
+
+```bash
+make defconfig-fio-tests-ci # Use minimal CI configuration
+make menuconfig # Or configure interactively
+make bringup # Provision test environment
+make fio-tests # Run performance tests
+```
+
+### Comprehensive testing
+
+For full performance analysis:
+
+```bash
+make menuconfig # Select fio-tests dedicated workflow
+# Configure test matrix, block sizes, IO depths, patterns
+make bringup # Provision baseline and dev nodes
+make fio-tests # Run comprehensive test suite
+make fio-tests-graph # Generate performance graphs
+make fio-tests-compare # Compare baseline vs dev results
+```
+
+## Configuration options
+
+### Test types
+
+The workflow supports multiple test types optimized for different analysis goals:
+
+- **Performance analysis**: Comprehensive testing across all configured parameters
+- **Latency analysis**: Focus on latency characteristics and tail latency
+- **Throughput scaling**: Optimize for maximum throughput analysis
+- **Mixed workloads**: Real-world application pattern simulation
+
+### Test matrix configuration
+
+Configure the test matrix through menuconfig:
+
+```
+Block size configuration →
+ [*] 4K block size tests
+ [*] 8K block size tests
+ [*] 16K block size tests
+ [ ] 32K block size tests
+ [ ] 64K block size tests
+ [ ] 128K block size tests
+
+IO depth configuration →
+ [*] IO depth 1
+ [*] IO depth 4
+ [*] IO depth 8
+ [*] IO depth 16
+ [ ] IO depth 32
+ [ ] IO depth 64
+
+Thread/job configuration →
+ [*] Single job
+ [*] 2 jobs
+ [*] 4 jobs
+ [ ] 8 jobs
+ [ ] 16 jobs
+
+Workload patterns →
+ [*] Random read
+ [*] Random write
+ [*] Sequential read
+ [*] Sequential write
+ [ ] Mixed 75% read / 25% write
+ [ ] Mixed 50% read / 50% write
+```
+
+### Advanced configuration
+
+Advanced settings for fine-tuning:
+
+- **I/O engine**: io_uring (recommended), libaio, psync, sync
+- **Direct I/O**: Bypass page cache for accurate device testing
+- **Test duration**: Runtime per test job (default: 60 seconds)
+- **Ramp time**: Warm-up period before measurements (default: 10 seconds)
+- **Results directory**: Storage location for test results and logs
+
+## Device configuration
+
+The workflow automatically selects appropriate storage devices based on your
+infrastructure configuration:
+
+### Virtualization (libvirt)
+- NVMe: `/dev/disk/by-id/nvme-QEMU_NVMe_Ctrl_kdevops1`
+- VirtIO: `/dev/disk/by-id/virtio-kdevops1`
+- IDE: `/dev/disk/by-id/ata-QEMU_HARDDISK_kdevops1`
+- SCSI: `/dev/sdc`
+
+### Cloud providers
+- AWS: `/dev/nvme2n1` (instance store)
+- GCE: `/dev/nvme1n1`
+- Azure: `/dev/sdd`
+- OCI: Configurable sparse volume device
+
+### Testing/CI
+- `/dev/null`: For configuration validation and CI testing
+
+## A/B testing
+
+The fio-tests workflow supports comprehensive A/B testing through the
+`KDEVOPS_BASELINE_AND_DEV` configuration, which provisions separate
+nodes for baseline and development testing.
+
+### Baseline establishment
+
+```bash
+make fio-tests # Run tests on both baseline and dev
+make fio-tests-baseline # Save current results as baseline
+```
+
+### Comparison analysis
+
+```bash
+make fio-tests-compare # Generate A/B comparison analysis
+```
+
+This creates comprehensive comparison reports including:
+- Side-by-side performance metrics
+- Percentage improvement/regression analysis
+- Statistical summaries
+- Visual comparison charts
+
+## Graphing and visualization
+
+The fio-tests workflow includes comprehensive graphing capabilities through
+Python scripts with matplotlib, pandas, and seaborn.
+
+### Enable graphing
+
+```bash
+# In menuconfig:
+Advanced configuration →
+ [*] Enable graphing and visualization
+ Graph output format (png) --->
+ (300) Graph resolution (DPI)
+ (default) Matplotlib theme
+```
+
+### Available visualizations
+
+#### Performance analysis graphs
+```bash
+make fio-tests-graph
+```
+
+Generates:
+- **Bandwidth heatmaps**: Performance across block sizes and I/O depths
+- **IOPS scaling**: Scaling behavior with increasing I/O depth
+- **Latency distributions**: Read/write latency characteristics
+- **Pattern comparisons**: Performance across different workload patterns
+
+#### A/B comparison analysis
+```bash
+make fio-tests-compare
+```
+
+Creates:
+- **Comparison bar charts**: Side-by-side baseline vs development
+- **Performance delta analysis**: Percentage improvements across metrics
+- **Summary reports**: Detailed statistical analysis
+
+#### Trend analysis
+```bash
+make fio-tests-trend-analysis
+```
+
+Provides:
+- **Block size trends**: Performance scaling with block size
+- **I/O depth scaling**: Efficiency analysis across patterns
+- **Latency percentiles**: P95, P99 latency analysis
+- **Correlation matrices**: Relationships between test parameters
+
+### Graph customization
+
+Configure graph output through Kconfig:
+
+- **Format**: PNG (default), SVG, PDF, JPG
+- **Resolution**: 150 DPI (CI), 300 DPI (standard), 600 DPI (high quality)
+- **Theme**: default, seaborn, dark_background, ggplot, bmh
+
+## Workflow targets
+
+The fio-tests workflow provides several make targets:
+
+### Core testing
+- `make fio-tests`: Run the configured test matrix
+- `make fio-tests-baseline`: Establish performance baseline
+- `make fio-tests-results`: Collect and summarize test results
+
+### Analysis and visualization
+- `make fio-tests-graph`: Generate performance graphs
+- `make fio-tests-compare`: Compare baseline vs development results
+- `make fio-tests-trend-analysis`: Analyze performance trends
+
+### Help
+- `make fio-tests-help-menu`: Display available fio-tests targets
+
+## Results and output
+
+### Test results structure
+
+Results are organized in the configured results directory (default: `/data/fio-tests`):
+
+```
+/data/fio-tests/
+├── jobs/ # Generated fio job files
+│ ├── randread_bs4k_iodepth1_jobs1.ini
+│ └── ...
+├── results_*.json # JSON format results
+├── results_*.txt # Human-readable results
+├── bw_*, iops_*, lat_* # Performance logs
+├── graphs/ # Generated visualizations
+│ ├── performance_bandwidth_heatmap.png
+│ ├── performance_iops_scaling.png
+│ └── ...
+├── analysis/ # Trend analysis
+│ ├── block_size_trends.png
+│ └── correlation_heatmap.png
+└── baseline/ # Baseline results
+ └── baseline_*.txt
+```
+
+### Result interpretation
+
+#### JSON output structure
+Each test produces detailed JSON output with:
+- Bandwidth metrics (KB/s)
+- IOPS measurements
+- Latency statistics (mean, stddev, percentiles)
+- Job-specific performance data
+
+#### Performance logs
+Detailed time-series logs for:
+- Bandwidth over time
+- IOPS over time
+- Latency over time
+
+## CI integration
+
+The fio-tests workflow includes CI-optimized configuration:
+
+```bash
+make defconfig-fio-tests-ci
+```
+
+CI-specific optimizations:
+- Uses `/dev/null` as target device
+- Minimal test matrix (4K block size, IO depth 1, single job)
+- Short test duration (10 seconds) and ramp time (2 seconds)
+- Lower DPI (150) for faster graph generation
+- Essential workload patterns only (random read)
+
+## Troubleshooting
+
+### Common issues
+
+#### Missing dependencies
+```bash
+# Ensure graphing dependencies are installed
+# This is handled automatically when FIO_TESTS_ENABLE_GRAPHING=y
+```
+
+#### No test results
+- Verify device permissions and accessibility
+- Check fio installation: `fio --version`
+- Examine fio job files in results directory
+
+#### Graph generation failures
+- Verify Python dependencies: matplotlib, pandas, seaborn
+- Check results directory contains JSON output files
+- Ensure sufficient disk space for graph files
+
+### Debug information
+
+Enable verbose output:
+```bash
+make V=1 fio-tests # Verbose build output
+make AV=2 fio-tests # Ansible verbose output
+```
+
+## Performance considerations
+
+### Test duration vs coverage
+- **Short tests** (10-60 seconds): Quick validation, less accurate
+- **Medium tests** (5-10 minutes): Balanced accuracy and time
+- **Long tests** (30+ minutes): High accuracy, comprehensive analysis
+
+### Resource requirements
+- **CPU**: Scales with job count and I/O depth
+- **Memory**: Minimal for fio, moderate for graphing (pandas/matplotlib)
+- **Storage**: Depends on test duration and logging configuration
+- **Network**: Minimal except for result collection
+
+### Optimization tips
+- Use dedicated storage for results directory
+- Enable direct I/O for accurate device testing
+- Configure appropriate test matrix for your analysis goals
+- Use A/B testing for meaningful performance comparisons
+
+## Integration with other workflows
+
+The fio-tests workflow integrates seamlessly with other kdevops workflows:
+
+### Combined testing
+- Run fio-tests alongside fstests for comprehensive filesystem analysis
+- Use with sysbench for database vs raw storage performance comparison
+- Combine with blktests for block layer and device-level testing
+
+### Steady state preparation
+- Use `KDEVOPS_WORKFLOW_ENABLE_SSD_STEADY_STATE` for SSD conditioning
+- Run steady state before fio-tests for consistent results
+
+## Best practices
+
+### Configuration
+1. Start with CI configuration for validation
+2. Gradually expand test matrix based on analysis needs
+3. Use A/B testing for meaningful comparisons
+4. Enable graphing for visual analysis
+
+### Testing methodology
+1. Establish baseline before configuration changes
+2. Run multiple iterations for statistical significance
+3. Use appropriate test duration for your workload
+4. Document test conditions and configuration
+
+### Result analysis
+1. Focus on relevant metrics for your use case
+2. Use trend analysis to identify optimal configurations
+3. Compare against baseline for regression detection
+4. Share graphs and summaries for team collaboration
+
+## Contributing
+
+The fio-tests workflow follows kdevops development practices:
+
+- Use atomic commits with DCO sign-off
+- Include "Generated-by: Claude AI" for AI-assisted contributions
+- Test changes with CI configuration
+- Update documentation for new features
+- Follow existing code style and patterns
+
+For more information about contributing to kdevops, see the main project
+documentation and CLAUDE.md for AI development guidelines.
diff --git a/kconfigs/workflows/Kconfig b/kconfigs/workflows/Kconfig
index a1d2c331..6835d4f5 100644
--- a/kconfigs/workflows/Kconfig
+++ b/kconfigs/workflows/Kconfig
@@ -189,6 +189,13 @@ config KDEVOPS_WORKFLOW_DEDICATE_MMTESTS
This will dedicate your configuration to running only the
mmtests workflow for memory fragmentation testing.
+config KDEVOPS_WORKFLOW_DEDICATE_FIO_TESTS
+ bool "fio-tests"
+ select KDEVOPS_WORKFLOW_ENABLE_FIO_TESTS
+ help
+ This will dedicate your configuration to running only the
+ fio-tests workflow for comprehensive storage performance testing.
+
endchoice
config KDEVOPS_WORKFLOW_NAME
@@ -203,6 +210,7 @@ config KDEVOPS_WORKFLOW_NAME
default "nfstest" if KDEVOPS_WORKFLOW_DEDICATE_NFSTEST
default "sysbench" if KDEVOPS_WORKFLOW_DEDICATE_SYSBENCH
default "mmtests" if KDEVOPS_WORKFLOW_DEDICATE_MMTESTS
+ default "fio-tests" if KDEVOPS_WORKFLOW_DEDICATE_FIO_TESTS
endif
@@ -304,6 +312,14 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_MMTESTS
Select this option if you want to provision mmtests on a
single target node for by-hand testing.
+config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_FIO_TESTS
+ bool "fio-tests"
+ select KDEVOPS_WORKFLOW_ENABLE_FIO_TESTS
+ depends on LIBVIRT || TERRAFORM_PRIVATE_NET
+ help
+ Select this option if you want to provision fio-tests on a
+ single target node for by-hand testing.
+
endif # !WORKFLOWS_DEDICATED_WORKFLOW
config KDEVOPS_WORKFLOW_ENABLE_FSTESTS
@@ -417,6 +433,17 @@ source "workflows/mmtests/Kconfig"
endmenu
endif # KDEVOPS_WORKFLOW_ENABLE_MMTESTS
+config KDEVOPS_WORKFLOW_ENABLE_FIO_TESTS
+ bool
+ output yaml
+ default y if KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_FIO_TESTS || KDEVOPS_WORKFLOW_DEDICATE_FIO_TESTS
+
+if KDEVOPS_WORKFLOW_ENABLE_FIO_TESTS
+menu "Configure and run fio-tests"
+source "workflows/fio-tests/Kconfig"
+endmenu
+endif # KDEVOPS_WORKFLOW_ENABLE_FIO_TESTS
+
config KDEVOPS_WORKFLOW_ENABLE_SSD_STEADY_STATE
bool "Attain SSD steady state prior to tests"
output yaml
diff --git a/playbooks/fio-tests-baseline.yml b/playbooks/fio-tests-baseline.yml
new file mode 100644
index 00000000..061e3e9e
--- /dev/null
+++ b/playbooks/fio-tests-baseline.yml
@@ -0,0 +1,29 @@
+---
+- hosts:
+ - baseline
+ - dev
+ become: no
+ vars:
+ ansible_ssh_pipelining: True
+ tasks:
+ - name: Create baseline directory structure
+ file:
+ path: "{{ fio_tests_results_dir }}/baseline"
+ state: directory
+ mode: '0755'
+ become: yes
+
+ - name: Save current test configuration as baseline
+ copy:
+ src: "{{ fio_tests_results_dir }}/results_{{ item }}.txt"
+ dest: "{{ fio_tests_results_dir }}/baseline/baseline_{{ item }}.txt"
+ remote_src: yes
+ backup: yes
+ with_fileglob:
+ - "{{ fio_tests_results_dir }}/results_*.txt"
+ become: yes
+ ignore_errors: yes
+
+ - name: Create baseline timestamp
+ shell: date > "{{ fio_tests_results_dir }}/baseline/baseline_timestamp.txt"
+ become: yes
\ No newline at end of file
diff --git a/playbooks/fio-tests-compare.yml b/playbooks/fio-tests-compare.yml
new file mode 100644
index 00000000..15fcebcc
--- /dev/null
+++ b/playbooks/fio-tests-compare.yml
@@ -0,0 +1,33 @@
+---
+- hosts: localhost
+ become: no
+ vars:
+ ansible_ssh_pipelining: True
+ tasks:
+ - name: Check if baseline and dev hosts exist in inventory
+ fail:
+ msg: "Both baseline and dev hosts must exist for comparison"
+ when: "'baseline' not in groups or 'dev' not in groups"
+
+ - name: Create local graph results comparison directory
+ file:
+ path: "{{ topdir_path }}/workflows/fio-tests/results/graphs"
+ state: directory
+ mode: '0755'
+
+ - name: Generate comparison graphs
+ shell: |
+ python3 {{ topdir_path }}/playbooks/python/workflows/fio-tests/fio-compare.py \
+ {{ topdir_path }}/workflows/fio-tests/results/{{ groups['baseline'][0] }} \
+ {{ topdir_path }}/workflows/fio-tests/results/{{ groups['dev'][0] }} \
+ --output-dir {{ topdir_path }}/workflows/fio-tests/results/graphs \
+ --baseline-label "Baseline" \
+ --dev-label "Development"
+
+ - name: List comparison results
+ shell: ls -la {{ topdir_path }}/workflows/fio-tests/results/graphs
+ register: comparison_list
+
+ - name: Display comparison results
+ debug:
+ msg: "{{ comparison_list.stdout_lines }}"
diff --git a/playbooks/fio-tests-graph.yml b/playbooks/fio-tests-graph.yml
new file mode 100644
index 00000000..f94b8e2b
--- /dev/null
+++ b/playbooks/fio-tests-graph.yml
@@ -0,0 +1,43 @@
+---
+- hosts:
+ - baseline
+ - dev
+ become: no
+ vars:
+ ansible_ssh_pipelining: True
+ tasks:
+ - include_role:
+ name: create_data_partition
+ tags: [ 'oscheck', 'data_partition' ]
+
+ - name: git clone kdevops
+ environment:
+ GIT_SSL_NO_VERIFY: true
+ git:
+ repo: "{{ kdevops_git }}"
+ dest: "{{ kdevops_data }}"
+ version: "{{ kdevops_git_version }}"
+ retries: 3
+ delay: 5
+ register: result
+ until: not result.failed
+ tags: [ 'oscheck', 'install', 'git']
+
+ - name: Generate fio performance graphs
+ shell: |
+ cd {{ fio_tests_results_dir }}
+ mkdir -p graphs
+ python3 {{ kdevops_data }}/playbooks/python/workflows/fio-tests/fio-plot.py \
+ . --output-dir graphs --prefix "{{ inventory_hostname }}_performance"
+ args:
+ creates: "{{ fio_tests_results_dir }}/graphs/{{ inventory_hostname }}_performance_bandwidth_heatmap.png"
+ become: yes
+
+ - name: List generated graphs
+ shell: ls -la {{ fio_tests_results_dir }}/graphs/
+ become: yes
+ register: graph_list
+
+ - name: Display generated graphs
+ debug:
+ msg: "{{ graph_list.stdout_lines }}"
diff --git a/playbooks/fio-tests-results.yml b/playbooks/fio-tests-results.yml
new file mode 100644
index 00000000..e34f908f
--- /dev/null
+++ b/playbooks/fio-tests-results.yml
@@ -0,0 +1,87 @@
+---
+- hosts:
+ - baseline
+ - dev
+ become: no
+ vars:
+ ansible_ssh_pipelining: True
+ tasks:
+ - name: Generate performance summary
+ shell: |
+ cd {{ fio_tests_results_dir }}
+ echo "=== FIO Test Results Summary ===" > summary.txt
+ echo "Generated on: $(date)" >> summary.txt
+ echo "" >> summary.txt
+ for result_file in results_*.txt; do
+ if [[ -f "$result_file" ]]; then
+ echo "=== $(basename $result_file .txt) ===" >> summary.txt
+ # Extract key metrics from fio output
+ grep -E "(read:|write:|READ:|WRITE:)" "$result_file" | head -4 >> summary.txt || true
+ echo "" >> summary.txt
+ fi
+ done
+ become: yes
+
+ - name: Show summary
+ shell: cat {{ fio_tests_results_dir }}/summary.txt
+ become: yes
+ register: fio_summary
+
+ - name: Display results
+ debug:
+ msg: "{{ fio_summary.stdout_lines }}"
+
+ - name: Create local results directory
+ delegate_to: localhost
+ ansible.builtin.file:
+ path: "{{ topdir_path }}/workflows/fio-tests/results/{{ inventory_hostname }}/"
+ state: directory
+ mode: '0755'
+ run_once: false
+ tags: ['results']
+
+ - name: Ensure old fio-tests results archive is removed if it exists
+ become: yes
+ become_method: sudo
+ ansible.builtin.file:
+ path: "{{ fio_tests_results_dir }}/../{{ inventory_hostname }}.tar.gz"
+ state: absent
+ tags: [ 'results' ]
+
+ - name: Archive fio-tests results directory on remote host
+ become: yes
+ become_method: sudo
+ command: >
+ tar czf {{ fio_tests_results_dir }}/../{{ inventory_hostname }}.tar.gz -C {{ fio_tests_results_dir }} .
+ args:
+ creates: "{{ fio_tests_results_dir }}/../{{ inventory_hostname }}.tar.gz"
+ tags: [ 'results' ]
+
+ - name: Copy fio-tests results locally
+ tags: [ 'results' ]
+ become: yes
+ become_method: sudo
+ ansible.builtin.fetch:
+ src: "{{ fio_tests_results_dir }}/../{{ inventory_hostname }}.tar.gz"
+ dest: "{{ topdir_path }}/workflows/fio-tests/results/{{ inventory_hostname }}.tar.gz"
+ flat: yes
+
+ - name: Extract fio-tests results archive locally
+ become: no
+ delegate_to: localhost
+ ansible.builtin.unarchive:
+ src: "{{ topdir_path }}/workflows/fio-tests/results/{{ inventory_hostname }}.tar.gz"
+ dest: "{{ topdir_path }}/workflows/fio-tests/results/{{ inventory_hostname }}"
+ remote_src: no
+ tags: [ 'results' ]
+
+ - name: Remove fio-tests archive on localhost
+ tags: [ 'clean' ]
+ become: yes
+ become_method: sudo
+ delegate_to: localhost
+ ansible.builtin.file:
+ path: "{{ item }}"
+ state: absent
+ with_items:
+ - "{{ topdir_path }}/workflows/fio-tests/results/{{ inventory_hostname }}.tar.gz"
diff --git a/playbooks/fio-tests-trend-analysis.yml b/playbooks/fio-tests-trend-analysis.yml
new file mode 100644
index 00000000..e94a1a1f
--- /dev/null
+++ b/playbooks/fio-tests-trend-analysis.yml
@@ -0,0 +1,30 @@
+---
+- hosts:
+ - baseline
+ - dev
+ become: no
+ vars:
+ ansible_ssh_pipelining: True
+ tasks:
+ - include_role:
+ name: create_data_partition
+ tags: [ 'oscheck', 'data_partition' ]
+
+ - name: Generate fio trend analysis
+ shell: |
+ cd {{ fio_tests_results_dir }}
+ mkdir -p analysis
+ python3 {{ kdevops_data }}/playbooks/python/workflows/fio-tests/fio-trend-analysis.py \
+ . --output-dir analysis
+ args:
+ creates: "{{ fio_tests_results_dir }}/analysis/block_size_trends.png"
+ become: yes
+
+ - name: List generated analysis files
+ shell: ls -la {{ fio_tests_results_dir }}/analysis/
+ become: yes
+ register: analysis_list
+
+ - name: Display generated analysis files
+ debug:
+ msg: "{{ analysis_list.stdout_lines }}"
diff --git a/playbooks/fio-tests.yml b/playbooks/fio-tests.yml
new file mode 100644
index 00000000..084f4d57
--- /dev/null
+++ b/playbooks/fio-tests.yml
@@ -0,0 +1,9 @@
+---
+- hosts:
+ - baseline
+ - dev
+ become: no
+ vars:
+ ansible_ssh_pipelining: True
+ roles:
+ - role: fio-tests
\ No newline at end of file
diff --git a/playbooks/python/workflows/fio-tests/fio-compare.py b/playbooks/python/workflows/fio-tests/fio-compare.py
new file mode 100755
index 00000000..2a7aa223
--- /dev/null
+++ b/playbooks/python/workflows/fio-tests/fio-compare.py
@@ -0,0 +1,289 @@
+#!/usr/bin/python3
+# SPDX-License-Identifier: copyleft-next-0.3.1
+
+# Compare fio test results between baseline and dev configurations for A/B testing
+
+import pandas as pd
+import matplotlib.pyplot as plt
+import json
+import argparse
+import os
+import sys
+from pathlib import Path
+
+def parse_fio_json(file_path):
+ """Parse fio JSON output and extract key metrics"""
+ try:
+ with open(file_path, 'r') as f:
+ data = json.load(f)
+
+ if 'jobs' not in data:
+ return None
+
+ job = data['jobs'][0] # Use first job
+
+ # Extract read metrics
+ read_stats = job.get('read', {})
+ read_bw = read_stats.get('bw', 0) / 1024 # Convert to MB/s
+ read_iops = read_stats.get('iops', 0)
+ read_lat_mean = read_stats.get('lat_ns', {}).get('mean', 0) / 1000000 # Convert to ms
+
+ # Extract write metrics
+ write_stats = job.get('write', {})
+ write_bw = write_stats.get('bw', 0) / 1024 # Convert to MB/s
+ write_iops = write_stats.get('iops', 0)
+ write_lat_mean = write_stats.get('lat_ns', {}).get('mean', 0) / 1000000 # Convert to ms
+
+ return {
+ 'read_bw': read_bw,
+ 'read_iops': read_iops,
+ 'read_lat': read_lat_mean,
+ 'write_bw': write_bw,
+ 'write_iops': write_iops,
+ 'write_lat': write_lat_mean,
+ 'total_bw': read_bw + write_bw,
+ 'total_iops': read_iops + write_iops
+ }
+ except (json.JSONDecodeError, FileNotFoundError, KeyError) as e:
+ print(f"Error parsing {file_path}: {e}")
+ return None
+
+def extract_test_params(filename):
+ """Extract test parameters from filename"""
+ parts = filename.replace('.json', '').replace('results_', '').split('_')
+
+ params = {}
+ for part in parts:
+ if part.startswith('bs'):
+ params['block_size'] = part[2:]
+ elif part.startswith('iodepth'):
+ params['io_depth'] = int(part[7:])
+ elif part.startswith('jobs'):
+ params['num_jobs'] = int(part[4:])
+ elif part in ['randread', 'randwrite', 'seqread', 'seqwrite', 'mixed_75_25', 'mixed_50_50']:
+ params['pattern'] = part
+
+ return params
+
+def load_results(results_dir, config_name):
+ """Load all fio results from a directory"""
+ results = []
+
+ json_files = list(Path(results_dir).glob('results_*.json'))
+ if not json_files:
+ json_files = list(Path(results_dir).glob('results_*.txt'))
+
+ for file_path in json_files:
+ if file_path.name.endswith('.json'):
+ metrics = parse_fio_json(file_path)
+ else:
+ continue
+
+ if metrics:
+ params = extract_test_params(file_path.name)
+ result = {**params, **metrics, 'config': config_name}
+ results.append(result)
+
+ return pd.DataFrame(results) if results else None
+
+def plot_comparison_bar_chart(baseline_df, dev_df, metric, output_file, title, ylabel):
+ """Create side-by-side bar chart comparison"""
+ if baseline_df.empty or dev_df.empty:
+ return
+
+ # Group by test configuration and calculate means
+ baseline_grouped = baseline_df.groupby(['pattern', 'block_size', 'io_depth'])[metric].mean()
+ dev_grouped = dev_df.groupby(['pattern', 'block_size', 'io_depth'])[metric].mean()
+
+ # Find common test configurations
+ common_configs = baseline_grouped.index.intersection(dev_grouped.index)
+
+ if len(common_configs) == 0:
+ return
+
+ baseline_values = [baseline_grouped[config] for config in common_configs]
+ dev_values = [dev_grouped[config] for config in common_configs]
+
+ # Create labels from config tuples
+ labels = [f"{pattern}\n{bs}@{depth}" for pattern, bs, depth in common_configs]
+
+ x = range(len(labels))
+ width = 0.35
+
+ plt.figure(figsize=(16, 8))
+
+ plt.bar([i - width/2 for i in x], baseline_values, width,
+ label='Baseline', color='skyblue', edgecolor='navy')
+ plt.bar([i + width/2 for i in x], dev_values, width,
+ label='Development', color='lightcoral', edgecolor='darkred')
+
+ # Add percentage improvement annotations
+ for i, (baseline_val, dev_val) in enumerate(zip(baseline_values, dev_values)):
+ if baseline_val > 0:
+ improvement = ((dev_val - baseline_val) / baseline_val) * 100
+ y_pos = max(baseline_val, dev_val) * 1.05
+ color = 'green' if improvement > 0 else 'red'
+ plt.text(i, y_pos, f'{improvement:+.1f}%',
+ ha='center', va='bottom', color=color, fontweight='bold')
+
+ plt.xlabel('Test Configuration (Pattern Block_Size@IO_Depth)')
+ plt.ylabel(ylabel)
+ plt.title(title)
+ plt.xticks(x, labels, rotation=45, ha='right')
+ plt.legend()
+ plt.grid(True, alpha=0.3, axis='y')
+ plt.tight_layout()
+ plt.savefig(output_file, dpi=300, bbox_inches='tight')
+ plt.close()
+
+def plot_performance_delta(baseline_df, dev_df, output_file):
+ """Plot performance delta (percentage improvement) across metrics"""
+ if baseline_df.empty or dev_df.empty:
+ return
+
+ metrics = ['total_bw', 'total_iops', 'read_lat', 'write_lat']
+ metric_names = ['Bandwidth', 'IOPS', 'Read Latency', 'Write Latency']
+
+ fig, axes = plt.subplots(2, 2, figsize=(16, 12))
+ axes = axes.flatten()
+
+ for idx, (metric, name) in enumerate(zip(metrics, metric_names)):
+ baseline_grouped = baseline_df.groupby(['pattern', 'block_size', 'io_depth'])[metric].mean()
+ dev_grouped = dev_df.groupby(['pattern', 'block_size', 'io_depth'])[metric].mean()
+
+ common_configs = baseline_grouped.index.intersection(dev_grouped.index)
+
+ if len(common_configs) == 0:
+ continue
+
+ # Calculate percentage changes
+ percent_changes = []
+ config_labels = []
+
+ for config in common_configs:
+ baseline_val = baseline_grouped[config]
+ dev_val = dev_grouped[config]
+
+ if baseline_val > 0:
+ # For latency, lower is better, so invert the calculation
+ if 'lat' in metric:
+ change = ((baseline_val - dev_val) / baseline_val) * 100
+ else:
+ change = ((dev_val - baseline_val) / baseline_val) * 100
+
+ percent_changes.append(change)
+ pattern, bs, depth = config
+ config_labels.append(f"{pattern}\n{bs}@{depth}")
+
+ if percent_changes:
+ colors = ['green' if x > 0 else 'red' for x in percent_changes]
+ bars = axes[idx].bar(range(len(percent_changes)), percent_changes, color=colors)
+
+ # Add value labels on bars
+ for bar, value in zip(bars, percent_changes):
+ height = bar.get_height()
+ axes[idx].text(bar.get_x() + bar.get_width()/2., height,
+ f'{value:.1f}%', ha='center',
+ va='bottom' if height > 0 else 'top')
+
+ axes[idx].set_title(f'{name} Performance Change')
+ axes[idx].set_ylabel('Percentage Change (%)')
+ axes[idx].set_xticks(range(len(config_labels)))
+ axes[idx].set_xticklabels(config_labels, rotation=45, ha='right')
+ axes[idx].axhline(y=0, color='black', linestyle='-', alpha=0.3)
+ axes[idx].grid(True, alpha=0.3, axis='y')
+
+ plt.tight_layout()
+ plt.savefig(output_file, dpi=300, bbox_inches='tight')
+ plt.close()
+
+def generate_summary_report(baseline_df, dev_df, output_file):
+ """Generate a text summary report of the comparison"""
+ with open(output_file, 'w') as f:
+ f.write("FIO Performance Comparison Report\n")
+ f.write("=" * 40 + "\n\n")
+
+ f.write(f"Baseline tests: {len(baseline_df)} configurations\n")
+ f.write(f"Development tests: {len(dev_df)} configurations\n\n")
+
+ metrics = ['total_bw', 'total_iops', 'read_lat', 'write_lat']
+ metric_names = ['Total Bandwidth (MB/s)', 'Total IOPS', 'Read Latency (ms)', 'Write Latency (ms)']
+
+ for metric, name in zip(metrics, metric_names):
+ f.write(f"{name}:\n")
+ f.write("-" * len(name) + "\n")
+
+ baseline_mean = baseline_df[metric].mean()
+ dev_mean = dev_df[metric].mean()
+
+ if baseline_mean > 0:
+ if 'lat' in metric:
+ improvement = ((baseline_mean - dev_mean) / baseline_mean) * 100
+ direction = "reduction" if improvement > 0 else "increase"
+ else:
+ improvement = ((dev_mean - baseline_mean) / baseline_mean) * 100
+ direction = "improvement" if improvement > 0 else "regression"
+
+ f.write(f" Baseline average: {baseline_mean:.2f}\n")
+ f.write(f" Development average: {dev_mean:.2f}\n")
+ f.write(f" Change: {improvement:+.1f}% {direction}\n\n")
+ else:
+ f.write(f" No data available\n\n")
+
+def main():
+ parser = argparse.ArgumentParser(description='Compare fio performance between baseline and development configurations')
+ parser.add_argument('baseline_dir', type=str, help='Directory containing baseline results')
+ parser.add_argument('dev_dir', type=str, help='Directory containing development results')
+ parser.add_argument('--output-dir', type=str, default='.', help='Output directory for comparison graphs')
+ parser.add_argument('--prefix', type=str, default='fio_comparison', help='Prefix for output files')
+ parser.add_argument('--baseline-label', type=str, default='Baseline', help='Label for baseline configuration')
+ parser.add_argument('--dev-label', type=str, default='Development', help='Label for development configuration')
+
+ args = parser.parse_args()
+
+ if not os.path.exists(args.baseline_dir):
+ print(f"Error: Baseline directory '{args.baseline_dir}' not found.")
+ sys.exit(1)
+
+ if not os.path.exists(args.dev_dir):
+ print(f"Error: Development directory '{args.dev_dir}' not found.")
+ sys.exit(1)
+
+ os.makedirs(args.output_dir, exist_ok=True)
+
+ print("Loading baseline results...")
+ baseline_df = load_results(args.baseline_dir, args.baseline_label)
+
+ print("Loading development results...")
+ dev_df = load_results(args.dev_dir, args.dev_label)
+
+ if baseline_df is None or baseline_df.empty:
+ print("No baseline results found.")
+ sys.exit(1)
+
+ if dev_df is None or dev_df.empty:
+ print("No development results found.")
+ sys.exit(1)
+
+ print(f"Comparing {len(baseline_df)} baseline vs {len(dev_df)} development results...")
+
+ # Generate comparison charts
+ plot_comparison_bar_chart(baseline_df, dev_df, 'total_bw',
+ os.path.join(args.output_dir, f'{args.prefix}_bandwidth_comparison.png'),
+ 'Bandwidth Comparison', 'Bandwidth (MB/s)')
+
+ plot_comparison_bar_chart(baseline_df, dev_df, 'total_iops',
+ os.path.join(args.output_dir, f'{args.prefix}_iops_comparison.png'),
+ 'IOPS Comparison', 'IOPS')
+
+ plot_performance_delta(baseline_df, dev_df,
+ os.path.join(args.output_dir, f'{args.prefix}_performance_delta.png'))
+
+ # Generate summary report
+ generate_summary_report(baseline_df, dev_df,
+ os.path.join(args.output_dir, f'{args.prefix}_summary.txt'))
+
+ print(f"Comparison results saved to {args.output_dir}")
+
+if __name__ == '__main__':
+ main()
diff --git a/playbooks/python/workflows/fio-tests/fio-plot.py b/playbooks/python/workflows/fio-tests/fio-plot.py
new file mode 100755
index 00000000..5cfc7ce6
--- /dev/null
+++ b/playbooks/python/workflows/fio-tests/fio-plot.py
@@ -0,0 +1,276 @@
+#!/usr/bin/python3
+# SPDX-License-Identifier: copyleft-next-0.3.1
+
+# Accepts fio output and provides comprehensive plots for performance analysis
+
+import pandas as pd
+import matplotlib.pyplot as plt
+import json
+import argparse
+import os
+import sys
+from pathlib import Path
+
+def parse_fio_json(file_path):
+ """Parse fio JSON output and extract key metrics"""
+ try:
+ with open(file_path, 'r') as f:
+ data = json.load(f)
+
+ if 'jobs' not in data:
+ return None
+
+ job = data['jobs'][0] # Use first job
+
+ # Extract read metrics
+ read_stats = job.get('read', {})
+ read_bw = read_stats.get('bw', 0) / 1024 # Convert to MB/s
+ read_iops = read_stats.get('iops', 0)
+ read_lat_mean = read_stats.get('lat_ns', {}).get('mean', 0) / 1000000 # Convert to ms
+
+ # Extract write metrics
+ write_stats = job.get('write', {})
+ write_bw = write_stats.get('bw', 0) / 1024 # Convert to MB/s
+ write_iops = write_stats.get('iops', 0)
+ write_lat_mean = write_stats.get('lat_ns', {}).get('mean', 0) / 1000000 # Convert to ms
+
+ return {
+ 'read_bw': read_bw,
+ 'read_iops': read_iops,
+ 'read_lat': read_lat_mean,
+ 'write_bw': write_bw,
+ 'write_iops': write_iops,
+ 'write_lat': write_lat_mean,
+ 'total_bw': read_bw + write_bw,
+ 'total_iops': read_iops + write_iops
+ }
+ except (json.JSONDecodeError, FileNotFoundError, KeyError) as e:
+ print(f"Error parsing {file_path}: {e}")
+ return None
+
+def extract_test_params(filename):
+ """Extract test parameters from filename"""
+ # Expected format: pattern_bs4k_iodepth1_jobs1.json
+ parts = filename.replace('.json', '').replace('results_', '').split('_')
+
+ params = {}
+ for part in parts:
+ if part.startswith('bs'):
+ params['block_size'] = part[2:]
+ elif part.startswith('iodepth'):
+ params['io_depth'] = int(part[7:])
+ elif part.startswith('jobs'):
+ params['num_jobs'] = int(part[4:])
+ elif part in ['randread', 'randwrite', 'seqread', 'seqwrite', 'mixed_75_25', 'mixed_50_50']:
+ params['pattern'] = part
+
+ return params
+
+def create_performance_matrix(results_dir):
+ """Create performance matrix from all test results"""
+ results = []
+
+ # Look for JSON result files
+ json_files = list(Path(results_dir).glob('results_*.json'))
+ if not json_files:
+ # Fallback to text files if JSON not available
+ json_files = list(Path(results_dir).glob('results_*.txt'))
+
+ for file_path in json_files:
+ if file_path.name.endswith('.json'):
+ metrics = parse_fio_json(file_path)
+ else:
+ continue # Skip text files for now, could add text parsing later
+
+ if metrics:
+ params = extract_test_params(file_path.name)
+ result = {**params, **metrics}
+ results.append(result)
+
+ return pd.DataFrame(results) if results else None
+
+def plot_bandwidth_heatmap(df, output_file):
+ """Create bandwidth heatmap across block sizes and IO depths"""
+ if df.empty or 'block_size' not in df.columns or 'io_depth' not in df.columns:
+ return
+
+ # Create pivot table for heatmap
+ pivot_data = df.pivot_table(values='total_bw',
+ index='io_depth',
+ columns='block_size',
+ aggfunc='mean')
+
+ plt.figure(figsize=(12, 8))
+ im = plt.imshow(pivot_data.values, cmap='viridis', aspect='auto')
+
+ # Add colorbar
+ plt.colorbar(im, label='Bandwidth (MB/s)')
+
+ # Set ticks and labels
+ plt.xticks(range(len(pivot_data.columns)), pivot_data.columns)
+ plt.yticks(range(len(pivot_data.index)), pivot_data.index)
+
+ plt.xlabel('Block Size')
+ plt.ylabel('IO Depth')
+ plt.title('Bandwidth Performance Matrix')
+
+ # Add text annotations
+ for i in range(len(pivot_data.index)):
+ for j in range(len(pivot_data.columns)):
+ if not pd.isna(pivot_data.iloc[i, j]):
+ plt.text(j, i, f'{pivot_data.iloc[i, j]:.0f}',
+ ha='center', va='center', color='white', fontweight='bold')
+
+ plt.tight_layout()
+ plt.savefig(output_file, dpi=300, bbox_inches='tight')
+ plt.close()
+
+def plot_iops_scaling(df, output_file):
+ """Plot IOPS scaling with IO depth"""
+ if df.empty or 'io_depth' not in df.columns:
+ return
+
+ plt.figure(figsize=(12, 8))
+
+ # Group by pattern and plot separately
+ patterns = df['pattern'].unique() if 'pattern' in df.columns else ['all']
+
+ for pattern in patterns:
+ if pattern != 'all':
+ pattern_df = df[df['pattern'] == pattern]
+ else:
+ pattern_df = df
+
+ # Group by IO depth and calculate mean IOPS
+ iops_by_depth = pattern_df.groupby('io_depth')['total_iops'].mean()
+
+ plt.plot(iops_by_depth.index, iops_by_depth.values,
+ marker='o', linewidth=2, markersize=6, label=pattern)
+
+ plt.xlabel('IO Depth')
+ plt.ylabel('IOPS')
+ plt.title('IOPS Scaling with IO Depth')
+ plt.grid(True, alpha=0.3)
+ plt.legend()
+ plt.tight_layout()
+ plt.savefig(output_file, dpi=300, bbox_inches='tight')
+ plt.close()
+
+def plot_latency_distribution(df, output_file):
+ """Plot latency distribution across different configurations"""
+ if df.empty:
+ return
+
+ fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(16, 6))
+
+ # Read latency
+ if 'read_lat' in df.columns:
+ read_lat_data = df[df['read_lat'] > 0]['read_lat']
+ if not read_lat_data.empty:
+ ax1.hist(read_lat_data, bins=20, alpha=0.7, color='blue', edgecolor='black')
+ ax1.set_xlabel('Read Latency (ms)')
+ ax1.set_ylabel('Frequency')
+ ax1.set_title('Read Latency Distribution')
+ ax1.grid(True, alpha=0.3)
+
+ # Write latency
+ if 'write_lat' in df.columns:
+ write_lat_data = df[df['write_lat'] > 0]['write_lat']
+ if not write_lat_data.empty:
+ ax2.hist(write_lat_data, bins=20, alpha=0.7, color='red', edgecolor='black')
+ ax2.set_xlabel('Write Latency (ms)')
+ ax2.set_ylabel('Frequency')
+ ax2.set_title('Write Latency Distribution')
+ ax2.grid(True, alpha=0.3)
+
+ plt.tight_layout()
+ plt.savefig(output_file, dpi=300, bbox_inches='tight')
+ plt.close()
+
+def plot_pattern_comparison(df, output_file):
+ """Compare performance across different workload patterns"""
+ if df.empty or 'pattern' not in df.columns:
+ return
+
+ patterns = df['pattern'].unique()
+ if len(patterns) <= 1:
+ return
+
+ # Calculate mean metrics for each pattern
+ pattern_stats = df.groupby('pattern').agg({
+ 'total_bw': 'mean',
+ 'total_iops': 'mean',
+ 'read_lat': 'mean',
+ 'write_lat': 'mean'
+ }).reset_index()
+
+ fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2, figsize=(16, 12))
+
+ # Bandwidth comparison
+ ax1.bar(pattern_stats['pattern'], pattern_stats['total_bw'], color='skyblue', edgecolor='navy')
+ ax1.set_ylabel('Bandwidth (MB/s)')
+ ax1.set_title('Bandwidth by Workload Pattern')
+ ax1.tick_params(axis='x', rotation=45)
+
+ # IOPS comparison
+ ax2.bar(pattern_stats['pattern'], pattern_stats['total_iops'], color='lightgreen', edgecolor='darkgreen')
+ ax2.set_ylabel('IOPS')
+ ax2.set_title('IOPS by Workload Pattern')
+ ax2.tick_params(axis='x', rotation=45)
+
+ # Read latency comparison
+ read_lat_data = pattern_stats[pattern_stats['read_lat'] > 0]
+ if not read_lat_data.empty:
+ ax3.bar(read_lat_data['pattern'], read_lat_data['read_lat'], color='orange', edgecolor='darkorange')
+ ax3.set_ylabel('Read Latency (ms)')
+ ax3.set_title('Read Latency by Workload Pattern')
+ ax3.tick_params(axis='x', rotation=45)
+
+ # Write latency comparison
+ write_lat_data = pattern_stats[pattern_stats['write_lat'] > 0]
+ if not write_lat_data.empty:
+ ax4.bar(write_lat_data['pattern'], write_lat_data['write_lat'], color='salmon', edgecolor='darkred')
+ ax4.set_ylabel('Write Latency (ms)')
+ ax4.set_title('Write Latency by Workload Pattern')
+ ax4.tick_params(axis='x', rotation=45)
+
+ plt.tight_layout()
+ plt.savefig(output_file, dpi=300, bbox_inches='tight')
+ plt.close()
+
+def main():
+ parser = argparse.ArgumentParser(description='Generate comprehensive performance graphs from fio test results')
+ parser.add_argument('results_dir', type=str, help='Directory containing fio test results')
+ parser.add_argument('--output-dir', type=str, default='.', help='Output directory for graphs')
+ parser.add_argument('--prefix', type=str, default='fio_performance', help='Prefix for output files')
+
+ args = parser.parse_args()
+
+ if not os.path.exists(args.results_dir):
+ print(f"Error: Results directory '{args.results_dir}' not found.")
+ sys.exit(1)
+
+ # Create output directory if it doesn't exist
+ os.makedirs(args.output_dir, exist_ok=True)
+
+ # Load and process results
+ print("Loading fio test results...")
+ df = create_performance_matrix(args.results_dir)
+
+ if df is None or df.empty:
+ print("No valid fio results found.")
+ sys.exit(1)
+
+ print(f"Found {len(df)} test results")
+ print("Generating graphs...")
+
+ # Generate different types of graphs
+ plot_bandwidth_heatmap(df, os.path.join(args.output_dir, f'{args.prefix}_bandwidth_heatmap.png'))
+ plot_iops_scaling(df, os.path.join(args.output_dir, f'{args.prefix}_iops_scaling.png'))
+ plot_latency_distribution(df, os.path.join(args.output_dir, f'{args.prefix}_latency_distribution.png'))
+ plot_pattern_comparison(df, os.path.join(args.output_dir, f'{args.prefix}_pattern_comparison.png'))
+
+ print(f"Graphs saved to {args.output_dir}")
+
+if __name__ == '__main__':
+ main()
\ No newline at end of file
diff --git a/playbooks/python/workflows/fio-tests/fio-trend-analysis.py b/playbooks/python/workflows/fio-tests/fio-trend-analysis.py
new file mode 100755
index 00000000..b42158cd
--- /dev/null
+++ b/playbooks/python/workflows/fio-tests/fio-trend-analysis.py
@@ -0,0 +1,350 @@
+#!/usr/bin/python3
+# SPDX-License-Identifier: copyleft-next-0.3.1
+
+# Analyze fio performance trends across different test parameters
+
+import pandas as pd
+import matplotlib.pyplot as plt
+import seaborn as sns
+import numpy as np
+import json
+import argparse
+import os
+import sys
+from pathlib import Path
+
+def parse_fio_json(file_path):
+ """Parse fio JSON output and extract detailed metrics"""
+ try:
+ with open(file_path, 'r') as f:
+ data = json.load(f)
+
+ if 'jobs' not in data:
+ return None
+
+ job = data['jobs'][0] # Use first job
+
+ # Extract read metrics
+ read_stats = job.get('read', {})
+ read_bw = read_stats.get('bw', 0) / 1024 # Convert to MB/s
+ read_iops = read_stats.get('iops', 0)
+ read_lat = read_stats.get('lat_ns', {})
+ read_lat_mean = read_lat.get('mean', 0) / 1000000 # Convert to ms
+ read_lat_stddev = read_lat.get('stddev', 0) / 1000000
+ read_lat_p95 = read_lat.get('percentile', {}).get('95.000000', 0) / 1000000
+ read_lat_p99 = read_lat.get('percentile', {}).get('99.000000', 0) / 1000000
+
+ # Extract write metrics
+ write_stats = job.get('write', {})
+ write_bw = write_stats.get('bw', 0) / 1024 # Convert to MB/s
+ write_iops = write_stats.get('iops', 0)
+ write_lat = write_stats.get('lat_ns', {})
+ write_lat_mean = write_lat.get('mean', 0) / 1000000 # Convert to ms
+ write_lat_stddev = write_lat.get('stddev', 0) / 1000000
+ write_lat_p95 = write_lat.get('percentile', {}).get('95.000000', 0) / 1000000
+ write_lat_p99 = write_lat.get('percentile', {}).get('99.000000', 0) / 1000000
+
+ return {
+ 'read_bw': read_bw,
+ 'read_iops': read_iops,
+ 'read_lat_mean': read_lat_mean,
+ 'read_lat_stddev': read_lat_stddev,
+ 'read_lat_p95': read_lat_p95,
+ 'read_lat_p99': read_lat_p99,
+ 'write_bw': write_bw,
+ 'write_iops': write_iops,
+ 'write_lat_mean': write_lat_mean,
+ 'write_lat_stddev': write_lat_stddev,
+ 'write_lat_p95': write_lat_p95,
+ 'write_lat_p99': write_lat_p99,
+ 'total_bw': read_bw + write_bw,
+ 'total_iops': read_iops + write_iops
+ }
+ except (json.JSONDecodeError, FileNotFoundError, KeyError) as e:
+ print(f"Error parsing {file_path}: {e}")
+ return None
+
+def extract_test_params(filename):
+ """Extract test parameters from filename"""
+ parts = filename.replace('.json', '').replace('results_', '').split('_')
+
+ params = {}
+ for part in parts:
+ if part.startswith('bs'):
+ # Convert block size to numeric KB for sorting
+ bs_str = part[2:]
+ if bs_str.endswith('k'):
+ params['block_size_kb'] = int(bs_str[:-1])
+ params['block_size'] = bs_str
+ else:
+ params['block_size_kb'] = int(bs_str)
+ params['block_size'] = bs_str
+ elif part.startswith('iodepth'):
+ params['io_depth'] = int(part[7:])
+ elif part.startswith('jobs'):
+ params['num_jobs'] = int(part[4:])
+ elif part in ['randread', 'randwrite', 'seqread', 'seqwrite', 'mixed_75_25', 'mixed_50_50']:
+ params['pattern'] = part
+
+ return params
+
+def load_all_results(results_dir):
+ """Load all fio results from directory"""
+ results = []
+
+ json_files = list(Path(results_dir).glob('results_*.json'))
+ if not json_files:
+ json_files = list(Path(results_dir).glob('results_*.txt'))
+
+ for file_path in json_files:
+ if file_path.name.endswith('.json'):
+ metrics = parse_fio_json(file_path)
+ else:
+ continue
+
+ if metrics:
+ params = extract_test_params(file_path.name)
+ result = {**params, **metrics}
+ results.append(result)
+
+ return pd.DataFrame(results) if results else None
+
+def plot_block_size_trends(df, output_dir):
+ """Plot performance trends across block sizes"""
+ if df.empty or 'block_size_kb' not in df.columns:
+ return
+
+ # Group by block size and calculate means
+ bs_trends = df.groupby('block_size_kb').agg({
+ 'total_bw': 'mean',
+ 'total_iops': 'mean',
+ 'read_lat_mean': 'mean',
+ 'write_lat_mean': 'mean'
+ }).reset_index()
+
+ bs_trends = bs_trends.sort_values('block_size_kb')
+
+ fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2, figsize=(16, 12))
+
+ # Bandwidth trend
+ ax1.plot(bs_trends['block_size_kb'], bs_trends['total_bw'],
+ marker='o', linewidth=2, markersize=8, color='blue')
+ ax1.set_xlabel('Block Size (KB)')
+ ax1.set_ylabel('Bandwidth (MB/s)')
+ ax1.set_title('Bandwidth vs Block Size')
+ ax1.grid(True, alpha=0.3)
+
+ # IOPS trend
+ ax2.plot(bs_trends['block_size_kb'], bs_trends['total_iops'],
+ marker='s', linewidth=2, markersize=8, color='green')
+ ax2.set_xlabel('Block Size (KB)')
+ ax2.set_ylabel('IOPS')
+ ax2.set_title('IOPS vs Block Size')
+ ax2.grid(True, alpha=0.3)
+
+ # Read latency trend
+ read_lat_data = bs_trends[bs_trends['read_lat_mean'] > 0]
+ if not read_lat_data.empty:
+ ax3.plot(read_lat_data['block_size_kb'], read_lat_data['read_lat_mean'],
+ marker='^', linewidth=2, markersize=8, color='orange')
+ ax3.set_xlabel('Block Size (KB)')
+ ax3.set_ylabel('Read Latency (ms)')
+ ax3.set_title('Read Latency vs Block Size')
+ ax3.grid(True, alpha=0.3)
+
+ # Write latency trend
+ write_lat_data = bs_trends[bs_trends['write_lat_mean'] > 0]
+ if not write_lat_data.empty:
+ ax4.plot(write_lat_data['block_size_kb'], write_lat_data['write_lat_mean'],
+ marker='v', linewidth=2, markersize=8, color='red')
+ ax4.set_xlabel('Block Size (KB)')
+ ax4.set_ylabel('Write Latency (ms)')
+ ax4.set_title('Write Latency vs Block Size')
+ ax4.grid(True, alpha=0.3)
+
+ plt.tight_layout()
+ plt.savefig(os.path.join(output_dir, 'block_size_trends.png'), dpi=300, bbox_inches='tight')
+ plt.close()
+
+def plot_io_depth_scaling(df, output_dir):
+ """Plot performance scaling with IO depth"""
+ if df.empty or 'io_depth' not in df.columns:
+ return
+
+ patterns = df['pattern'].unique() if 'pattern' in df.columns else [None]
+
+ fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2, figsize=(16, 12))
+
+ colors = plt.cm.tab10(np.linspace(0, 1, len(patterns)))
+
+ for pattern, color in zip(patterns, colors):
+ if pattern is not None:
+ pattern_df = df[df['pattern'] == pattern]
+ label = pattern
+ else:
+ pattern_df = df
+ label = 'All'
+
+ if pattern_df.empty:
+ continue
+
+ depth_trends = pattern_df.groupby('io_depth').agg({
+ 'total_bw': 'mean',
+ 'total_iops': 'mean',
+ 'read_lat_mean': 'mean',
+ 'write_lat_mean': 'mean'
+ }).reset_index()
+
+ depth_trends = depth_trends.sort_values('io_depth')
+
+ # Bandwidth scaling
+ ax1.plot(depth_trends['io_depth'], depth_trends['total_bw'],
+ marker='o', linewidth=2, markersize=6, label=label, color=color)
+
+ # IOPS scaling
+ ax2.plot(depth_trends['io_depth'], depth_trends['total_iops'],
+ marker='s', linewidth=2, markersize=6, label=label, color=color)
+
+ # Read latency scaling
+ read_lat_data = depth_trends[depth_trends['read_lat_mean'] > 0]
+ if not read_lat_data.empty:
+ ax3.plot(read_lat_data['io_depth'], read_lat_data['read_lat_mean'],
+ marker='^', linewidth=2, markersize=6, label=label, color=color)
+
+ # Write latency scaling
+ write_lat_data = depth_trends[depth_trends['write_lat_mean'] > 0]
+ if not write_lat_data.empty:
+ ax4.plot(write_lat_data['io_depth'], write_lat_data['write_lat_mean'],
+ marker='v', linewidth=2, markersize=6, label=label, color=color)
+
+ ax1.set_xlabel('IO Depth')
+ ax1.set_ylabel('Bandwidth (MB/s)')
+ ax1.set_title('Bandwidth Scaling with IO Depth')
+ ax1.grid(True, alpha=0.3)
+ ax1.legend()
+
+ ax2.set_xlabel('IO Depth')
+ ax2.set_ylabel('IOPS')
+ ax2.set_title('IOPS Scaling with IO Depth')
+ ax2.grid(True, alpha=0.3)
+ ax2.legend()
+
+ ax3.set_xlabel('IO Depth')
+ ax3.set_ylabel('Read Latency (ms)')
+ ax3.set_title('Read Latency vs IO Depth')
+ ax3.grid(True, alpha=0.3)
+ ax3.legend()
+
+ ax4.set_xlabel('IO Depth')
+ ax4.set_ylabel('Write Latency (ms)')
+ ax4.set_title('Write Latency vs IO Depth')
+ ax4.grid(True, alpha=0.3)
+ ax4.legend()
+
+ plt.tight_layout()
+ plt.savefig(os.path.join(output_dir, 'io_depth_scaling.png'), dpi=300, bbox_inches='tight')
+ plt.close()
+
+def plot_latency_percentiles(df, output_dir):
+ """Plot latency percentile analysis"""
+ if df.empty:
+ return
+
+ latency_cols = ['read_lat_mean', 'read_lat_p95', 'read_lat_p99',
+ 'write_lat_mean', 'write_lat_p95', 'write_lat_p99']
+
+ # Filter out zero latencies
+ lat_df = df[latency_cols]
+ lat_df = lat_df[(lat_df > 0).any(axis=1)]
+
+ if lat_df.empty:
+ return
+
+ fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(16, 6))
+
+ # Read latency percentiles
+ read_cols = [col for col in latency_cols if col.startswith('read_')]
+ if any(col in lat_df.columns for col in read_cols):
+ read_data = lat_df[read_cols].dropna()
+ if not read_data.empty:
+ bp1 = ax1.boxplot([read_data[col] for col in read_cols],
+ labels=['Mean', 'P95', 'P99'], patch_artist=True)
+ for patch in bp1['boxes']:
+ patch.set_facecolor('lightblue')
+ ax1.set_ylabel('Latency (ms)')
+ ax1.set_title('Read Latency Distribution')
+ ax1.grid(True, alpha=0.3)
+
+ # Write latency percentiles
+ write_cols = [col for col in latency_cols if col.startswith('write_')]
+ if any(col in lat_df.columns for col in write_cols):
+ write_data = lat_df[write_cols].dropna()
+ if not write_data.empty:
+ bp2 = ax2.boxplot([write_data[col] for col in write_cols],
+ labels=['Mean', 'P95', 'P99'], patch_artist=True)
+ for patch in bp2['boxes']:
+ patch.set_facecolor('lightcoral')
+ ax2.set_ylabel('Latency (ms)')
+ ax2.set_title('Write Latency Distribution')
+ ax2.grid(True, alpha=0.3)
+
+ plt.tight_layout()
+ plt.savefig(os.path.join(output_dir, 'latency_percentiles.png'), dpi=300, bbox_inches='tight')
+ plt.close()
+
+def create_correlation_heatmap(df, output_dir):
+ """Create correlation heatmap of performance metrics"""
+ if df.empty:
+ return
+
+ # Select numeric columns for correlation
+ numeric_cols = ['block_size_kb', 'io_depth', 'num_jobs', 'total_bw', 'total_iops',
+ 'read_lat_mean', 'write_lat_mean']
+
+ corr_df = df[numeric_cols].dropna()
+
+ if corr_df.empty:
+ return
+
+ correlation_matrix = corr_df.corr()
+
+ plt.figure(figsize=(10, 8))
+ sns.heatmap(correlation_matrix, annot=True, cmap='coolwarm', center=0,
+ square=True, linewidths=0.5, cbar_kws={"shrink": .8})
+ plt.title('Performance Metrics Correlation Matrix')
+ plt.tight_layout()
+ plt.savefig(os.path.join(output_dir, 'correlation_heatmap.png'), dpi=300, bbox_inches='tight')
+ plt.close()
+
+def main():
+ parser = argparse.ArgumentParser(description='Analyze fio performance trends and patterns')
+ parser.add_argument('results_dir', type=str, help='Directory containing fio test results')
+ parser.add_argument('--output-dir', type=str, default='.', help='Output directory for analysis graphs')
+
+ args = parser.parse_args()
+
+ if not os.path.exists(args.results_dir):
+ print(f"Error: Results directory '{args.results_dir}' not found.")
+ sys.exit(1)
+
+ os.makedirs(args.output_dir, exist_ok=True)
+
+ print("Loading fio test results...")
+ df = load_all_results(args.results_dir)
+
+ if df is None or df.empty:
+ print("No valid fio results found.")
+ sys.exit(1)
+
+ print(f"Analyzing {len(df)} test results...")
+
+ # Generate trend analysis
+ plot_block_size_trends(df, args.output_dir)
+ plot_io_depth_scaling(df, args.output_dir)
+ plot_latency_percentiles(df, args.output_dir)
+ create_correlation_heatmap(df, args.output_dir)
+
+ print(f"Trend analysis saved to {args.output_dir}")
+
+if __name__ == '__main__':
+ main()
\ No newline at end of file
diff --git a/playbooks/roles/fio-tests/defaults/main.yml b/playbooks/roles/fio-tests/defaults/main.yml
new file mode 100644
index 00000000..bbf7f5e9
--- /dev/null
+++ b/playbooks/roles/fio-tests/defaults/main.yml
@@ -0,0 +1,48 @@
+---
+# fio-tests role defaults
+
+fio_tests_results_dir: "/data/fio-tests"
+fio_tests_binary: "/usr/bin/fio"
+
+# These variables are populated from kconfig via extra_vars.yaml
+# fio_tests_device: ""
+# fio_tests_runtime: ""
+# fio_tests_ramp_time: ""
+# fio_tests_ioengine: ""
+# fio_tests_direct: ""
+# fio_tests_fsync_on_close: ""
+# fio_tests_log_avg_msec: ""
+
+# Test configuration booleans (populated from kconfig)
+# fio_tests_bs_4k: false
+# fio_tests_bs_8k: false
+# fio_tests_bs_16k: false
+# fio_tests_bs_32k: false
+# fio_tests_bs_64k: false
+# fio_tests_bs_128k: false
+
+# fio_tests_iodepth_1: false
+# fio_tests_iodepth_4: false
+# fio_tests_iodepth_8: false
+# fio_tests_iodepth_16: false
+# fio_tests_iodepth_32: false
+# fio_tests_iodepth_64: false
+
+# fio_tests_numjobs_1: false
+# fio_tests_numjobs_2: false
+# fio_tests_numjobs_4: false
+# fio_tests_numjobs_8: false
+# fio_tests_numjobs_16: false
+
+# fio_tests_pattern_rand_read: false
+# fio_tests_pattern_rand_write: false
+# fio_tests_pattern_seq_read: false
+# fio_tests_pattern_seq_write: false
+# fio_tests_pattern_mixed_75_25: false
+# fio_tests_pattern_mixed_50_50: false
+
+# Derived configuration lists
+fio_tests_block_sizes: []
+fio_tests_io_depths: []
+fio_tests_num_jobs: []
+fio_tests_patterns: []
\ No newline at end of file
diff --git a/playbooks/roles/fio-tests/tasks/install-deps/debian/main.yml b/playbooks/roles/fio-tests/tasks/install-deps/debian/main.yml
new file mode 100644
index 00000000..5b42eaba
--- /dev/null
+++ b/playbooks/roles/fio-tests/tasks/install-deps/debian/main.yml
@@ -0,0 +1,20 @@
+---
+- name: Install fio for Debian/Ubuntu
+ package:
+ name:
+ - fio
+ - python3
+ state: present
+ become: yes
+
+- name: Install graphing dependencies for Debian/Ubuntu
+ package:
+ name:
+ - python3-pip
+ - python3-pandas
+ - python3-matplotlib
+ - python3-seaborn
+ - python3-numpy
+ state: present
+ become: yes
+ when: fio_tests_enable_graphing is defined and fio_tests_enable_graphing
\ No newline at end of file
diff --git a/playbooks/roles/fio-tests/tasks/install-deps/main.yml b/playbooks/roles/fio-tests/tasks/install-deps/main.yml
new file mode 100644
index 00000000..26594c2a
--- /dev/null
+++ b/playbooks/roles/fio-tests/tasks/install-deps/main.yml
@@ -0,0 +1,3 @@
+---
+- name: Include distribution-specific installation tasks
+ include_tasks: "{{ ansible_os_family | lower }}/main.yml"
\ No newline at end of file
diff --git a/playbooks/roles/fio-tests/tasks/install-deps/redhat/main.yml b/playbooks/roles/fio-tests/tasks/install-deps/redhat/main.yml
new file mode 100644
index 00000000..e7fc8d0f
--- /dev/null
+++ b/playbooks/roles/fio-tests/tasks/install-deps/redhat/main.yml
@@ -0,0 +1,20 @@
+---
+- name: Install fio for RHEL/CentOS/Fedora
+ package:
+ name:
+ - fio
+ - python3
+ state: present
+ become: yes
+
+- name: Install graphing dependencies for RHEL/CentOS/Fedora
+ package:
+ name:
+ - python3-pip
+ - python3-pandas
+ - python3-matplotlib
+ - python3-seaborn
+ - python3-numpy
+ state: present
+ become: yes
+ when: fio_tests_enable_graphing is defined and fio_tests_enable_graphing
\ No newline at end of file
diff --git a/playbooks/roles/fio-tests/tasks/install-deps/suse/main.yml b/playbooks/roles/fio-tests/tasks/install-deps/suse/main.yml
new file mode 100644
index 00000000..a4581ef4
--- /dev/null
+++ b/playbooks/roles/fio-tests/tasks/install-deps/suse/main.yml
@@ -0,0 +1,20 @@
+---
+- name: Install fio for SUSE
+ package:
+ name:
+ - fio
+ - python3
+ state: present
+ become: yes
+
+- name: Install graphing dependencies for SUSE
+ package:
+ name:
+ - python3-pip
+ - python3-pandas
+ - python3-matplotlib
+ - python3-seaborn
+ - python3-numpy
+ state: present
+ become: yes
+ when: fio_tests_enable_graphing is defined and fio_tests_enable_graphing
\ No newline at end of file
diff --git a/playbooks/roles/fio-tests/tasks/main.yaml b/playbooks/roles/fio-tests/tasks/main.yaml
new file mode 100644
index 00000000..66a3cace
--- /dev/null
+++ b/playbooks/roles/fio-tests/tasks/main.yaml
@@ -0,0 +1,92 @@
+---
+- name: Set derived configuration variables
+ set_fact:
+ fio_tests_block_sizes: >-
+ {{
+ (['4k'] if fio_tests_bs_4k else []) +
+ (['8k'] if fio_tests_bs_8k else []) +
+ (['16k'] if fio_tests_bs_16k else []) +
+ (['32k'] if fio_tests_bs_32k else []) +
+ (['64k'] if fio_tests_bs_64k else []) +
+ (['128k'] if fio_tests_bs_128k else [])
+ }}
+ fio_tests_io_depths: >-
+ {{
+ ([1] if fio_tests_iodepth_1 else []) +
+ ([4] if fio_tests_iodepth_4 else []) +
+ ([8] if fio_tests_iodepth_8 else []) +
+ ([16] if fio_tests_iodepth_16 else []) +
+ ([32] if fio_tests_iodepth_32 else []) +
+ ([64] if fio_tests_iodepth_64 else [])
+ }}
+ fio_tests_num_jobs: >-
+ {{
+ ([1] if fio_tests_numjobs_1 else []) +
+ ([2] if fio_tests_numjobs_2 else []) +
+ ([4] if fio_tests_numjobs_4 else []) +
+ ([8] if fio_tests_numjobs_8 else []) +
+ ([16] if fio_tests_numjobs_16 else [])
+ }}
+ fio_tests_patterns: >-
+ {{
+ ([{'name': 'randread', 'rw': 'randread', 'rwmixread': 100}] if fio_tests_pattern_rand_read else []) +
+ ([{'name': 'randwrite', 'rw': 'randwrite', 'rwmixread': 0}] if fio_tests_pattern_rand_write else []) +
+ ([{'name': 'seqread', 'rw': 'read', 'rwmixread': 100}] if fio_tests_pattern_seq_read else []) +
+ ([{'name': 'seqwrite', 'rw': 'write', 'rwmixread': 0}] if fio_tests_pattern_seq_write else []) +
+ ([{'name': 'mixed_75_25', 'rw': 'randrw', 'rwmixread': 75}] if fio_tests_pattern_mixed_75_25 else []) +
+ ([{'name': 'mixed_50_50', 'rw': 'randrw', 'rwmixread': 50}] if fio_tests_pattern_mixed_50_50 else [])
+ }}
+
+- name: Install fio and dependencies
+ include_tasks: install-deps/main.yml
+
+- name: Create results directory
+ file:
+ path: "{{ fio_tests_results_dir }}"
+ state: directory
+ mode: '0755'
+ become: yes
+
+- name: Create fio job files directory
+ file:
+ path: "{{ fio_tests_results_dir }}/jobs"
+ state: directory
+ mode: '0755'
+ become: yes
+
+- name: Generate fio job files
+ template:
+ src: fio-job.ini.j2
+ dest: "{{ fio_tests_results_dir }}/jobs/{{ item.0.name }}_bs{{ item.1 }}_iodepth{{ item.2 }}_jobs{{ item.3 }}.ini"
+ mode: '0644'
+ vars:
+ pattern: "{{ item.0 }}"
+ block_size: "{{ item.1 }}"
+ io_depth: "{{ item.2 }}"
+ num_jobs: "{{ item.3 }}"
+ with_nested:
+ - "{{ fio_tests_patterns }}"
+ - "{{ fio_tests_block_sizes }}"
+ - "{{ fio_tests_io_depths }}"
+ - "{{ fio_tests_num_jobs }}"
+ become: yes
+
+- name: Run fio tests
+ shell: |
+ cd {{ fio_tests_results_dir }}/jobs
+ for job_file in *.ini; do
+ echo "Running test: $job_file"
+ # Run test with both JSON and normal output
+ fio "$job_file" --output="{{ fio_tests_results_dir }}/results_${job_file%.ini}.json" \
+ --output-format=json \
+ --write_bw_log="{{ fio_tests_results_dir }}/bw_${job_file%.ini}" \
+ --write_iops_log="{{ fio_tests_results_dir }}/iops_${job_file%.ini}" \
+ --write_lat_log="{{ fio_tests_results_dir }}/lat_${job_file%.ini}" \
+ --log_avg_msec={{ fio_tests_log_avg_msec }}
+ # Also create text output for compatibility
+ fio "$job_file" --output="{{ fio_tests_results_dir }}/results_${job_file%.ini}.txt" \
+ --output-format=normal
+ done
+ become: yes
+ async: 7200 # 2 hours timeout
+ poll: 30
\ No newline at end of file
diff --git a/playbooks/roles/fio-tests/templates/fio-job.ini.j2 b/playbooks/roles/fio-tests/templates/fio-job.ini.j2
new file mode 100644
index 00000000..d5c83356
--- /dev/null
+++ b/playbooks/roles/fio-tests/templates/fio-job.ini.j2
@@ -0,0 +1,18 @@
+[global]
+ioengine={{ fio_tests_ioengine }}
+direct={{ fio_tests_direct | int }}
+fsync_on_close={{ fio_tests_fsync_on_close | int }}
+group_reporting=1
+time_based=1
+runtime={{ fio_tests_runtime }}
+ramp_time={{ fio_tests_ramp_time }}
+
+[{{ pattern.name }}_bs{{ block_size }}_iodepth{{ io_depth }}_jobs{{ num_jobs }}]
+filename={{ fio_tests_device }}
+rw={{ pattern.rw }}
+{% if pattern.rwmixread is defined and pattern.rw in ['randrw', 'rw'] %}
+rwmixread={{ pattern.rwmixread }}
+{% endif %}
+bs={{ block_size }}
+iodepth={{ io_depth }}
+numjobs={{ num_jobs }}
\ No newline at end of file
diff --git a/playbooks/roles/gen_hosts/tasks/main.yml b/playbooks/roles/gen_hosts/tasks/main.yml
index c10abc5a..3ee599d7 100644
--- a/playbooks/roles/gen_hosts/tasks/main.yml
+++ b/playbooks/roles/gen_hosts/tasks/main.yml
@@ -312,6 +312,19 @@
- kdevops_workflow_enable_sysbench
- ansible_hosts_template.stat.exists
+- name: Generate the Ansible hosts file for a dedicated fio-tests setup
+ tags: [ 'hosts' ]
+ template:
+ src: "{{ kdevops_hosts_template }}"
+ dest: "{{ topdir_path }}/{{ kdevops_hosts }}"
+ force: yes
+ trim_blocks: True
+ lstrip_blocks: True
+ when:
+ - kdevops_workflows_dedicated_workflow
+ - kdevops_workflow_enable_fio_tests
+ - ansible_hosts_template.stat.exists
+
- name: Infer enabled mmtests test types
set_fact:
mmtests_enabled_test_types: >-
diff --git a/playbooks/roles/gen_hosts/templates/hosts.j2 b/playbooks/roles/gen_hosts/templates/hosts.j2
index 5032f592..7321025f 100644
--- a/playbooks/roles/gen_hosts/templates/hosts.j2
+++ b/playbooks/roles/gen_hosts/templates/hosts.j2
@@ -6,8 +6,42 @@ Each workflow which has its own custom ansible host file generated should use
its own jinja2 template file and define its own ansible task for its generation.
#}
{% if kdevops_workflows_dedicated_workflow %}
+{% if kdevops_workflow_enable_fio_tests %}
+[all]
+{{ kdevops_host_prefix }}-fio-tests
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-fio-tests-dev
+{% endif %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{{ kdevops_host_prefix }}-fio-tests
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{{ kdevops_host_prefix }}-fio-tests-dev
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[fio_tests]
+{{ kdevops_host_prefix }}-fio-tests
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-fio-tests-dev
+{% endif %}
+
+[fio_tests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% else %}
[all]
write-your-own-template-for-your-workflow-and-task
+{% endif %}
{% else %}
[all]
{{ kdevops_hosts_prefix }}
diff --git a/playbooks/roles/gen_nodes/tasks/main.yml b/playbooks/roles/gen_nodes/tasks/main.yml
index cb8459a0..f6e369cc 100644
--- a/playbooks/roles/gen_nodes/tasks/main.yml
+++ b/playbooks/roles/gen_nodes/tasks/main.yml
@@ -486,6 +486,38 @@
- kdevops_workflow_enable_sysbench
- ansible_nodes_template.stat.exists
+- name: Generate the fio-tests kdevops nodes file using {{ kdevops_nodes_template }} as jinja2 source template
+ tags: [ 'hosts' ]
+ vars:
+ node_template: "{{ kdevops_nodes_template | basename }}"
+ nodes: "{{ [kdevops_host_prefix + '-fio-tests'] }}"
+ all_generic_nodes: "{{ [kdevops_host_prefix + '-fio-tests'] }}"
+ template:
+ src: "{{ node_template }}"
+ dest: "{{ topdir_path }}/{{ kdevops_nodes }}"
+ force: yes
+ when:
+ - kdevops_workflows_dedicated_workflow
+ - kdevops_workflow_enable_fio_tests
+ - ansible_nodes_template.stat.exists
+ - not kdevops_baseline_and_dev
+
+- name: Generate the fio-tests kdevops nodes file with dev hosts using {{ kdevops_nodes_template }} as jinja2 source template
+ tags: [ 'hosts' ]
+ vars:
+ node_template: "{{ kdevops_nodes_template | basename }}"
+ nodes: "{{ [kdevops_host_prefix + '-fio-tests', kdevops_host_prefix + '-fio-tests-dev'] }}"
+ all_generic_nodes: "{{ [kdevops_host_prefix + '-fio-tests', kdevops_host_prefix + '-fio-tests-dev'] }}"
+ template:
+ src: "{{ node_template }}"
+ dest: "{{ topdir_path }}/{{ kdevops_nodes }}"
+ force: yes
+ when:
+ - kdevops_workflows_dedicated_workflow
+ - kdevops_workflow_enable_fio_tests
+ - ansible_nodes_template.stat.exists
+ - kdevops_baseline_and_dev
+
- name: Infer enabled mmtests test section types
set_fact:
mmtests_enabled_test_types: >-
diff --git a/workflows/Makefile b/workflows/Makefile
index ef3cc2d1..b5f54ff5 100644
--- a/workflows/Makefile
+++ b/workflows/Makefile
@@ -62,6 +62,10 @@ ifeq (y,$(CONFIG_KDEVOPS_WORKFLOW_ENABLE_SSD_STEADY_STATE))
include workflows/steady_state/Makefile
endif # CONFIG_KDEVOPS_WORKFLOW_ENABLE_SSD_STEADY_STATE == y
+ifeq (y,$(CONFIG_KDEVOPS_WORKFLOW_ENABLE_FIO_TESTS))
+include workflows/fio-tests/Makefile
+endif # CONFIG_KDEVOPS_WORKFLOW_ENABLE_FIO_TESTS == y
+
ANSIBLE_EXTRA_ARGS += $(WORKFLOW_ARGS)
ANSIBLE_EXTRA_ARGS_SEPARATED += $(WORKFLOW_ARGS_SEPARATED)
ANSIBLE_EXTRA_ARGS_DIRECT += $(WORKFLOW_ARGS_DIRECT)
diff --git a/workflows/fio-tests/Kconfig b/workflows/fio-tests/Kconfig
new file mode 100644
index 00000000..c4f69f26
--- /dev/null
+++ b/workflows/fio-tests/Kconfig
@@ -0,0 +1,359 @@
+choice
+ prompt "What type of fio testing do you want to run?"
+ default FIO_TESTS_PERFORMANCE_ANALYSIS
+
+config FIO_TESTS_PERFORMANCE_ANALYSIS
+ bool "Performance analysis tests"
+ select KDEVOPS_BASELINE_AND_DEV
+ output yaml
+ help
+ Run comprehensive performance analysis tests across different
+ configurations to understand storage device characteristics.
+ This includes testing various block sizes, IO depths, and
+ thread counts to generate performance profiles.
+
+ A/B testing is enabled to compare performance across different
+ configurations using baseline and development nodes.
+
+config FIO_TESTS_LATENCY_ANALYSIS
+ bool "Latency analysis tests"
+ select KDEVOPS_BASELINE_AND_DEV
+ output yaml
+ help
+ Focus on latency characteristics and tail latency analysis
+ across different workload patterns. This helps identify
+ performance outliers and latency distribution patterns.
+
+config FIO_TESTS_THROUGHPUT_SCALING
+ bool "Throughput scaling tests"
+ select KDEVOPS_BASELINE_AND_DEV
+ output yaml
+ help
+ Test how throughput scales with increasing IO depth and
+ thread count. Useful for understanding the optimal
+ configuration for maximum throughput.
+
+config FIO_TESTS_MIXED_WORKLOADS
+ bool "Mixed workload tests"
+ select KDEVOPS_BASELINE_AND_DEV
+ output yaml
+ help
+ Test mixed read/write workloads with various ratios to
+ simulate real-world application patterns.
+
+endchoice
+
+config FIO_TESTS_DEVICE
+ string "Device to use for fio testing"
+ output yaml
+ default "/dev/disk/by-id/nvme-QEMU_NVMe_Ctrl_kdevops1" if LIBVIRT && LIBVIRT_EXTRA_STORAGE_DRIVE_NVME
+ default "/dev/disk/by-id/virtio-kdevops1" if LIBVIRT && LIBVIRT_EXTRA_STORAGE_DRIVE_VIRTIO
+ default "/dev/disk/by-id/ata-QEMU_HARDDISK_kdevops1" if LIBVIRT && LIBVIRT_EXTRA_STORAGE_DRIVE_IDE
+ default "/dev/sdc" if LIBVIRT && LIBVIRT_EXTRA_STORAGE_DRIVE_SCSI
+ default "/dev/nvme2n1" if TERRAFORM_AWS_INSTANCE_M5AD_2XLARGE
+ default "/dev/nvme2n1" if TERRAFORM_AWS_INSTANCE_M5AD_4XLARGE
+ default "/dev/nvme1n1" if TERRAFORM_GCE
+ default "/dev/sdd" if TERRAFORM_AZURE
+ default TERRAFORM_OCI_SPARSE_VOLUME_DEVICE_FILE_NAME if TERRAFORM_OCI
+ default "/dev/null"
+ help
+ The block device to use for fio testing. For CI/testing
+ purposes, /dev/null can be used as a simple target.
+
+config FIO_TESTS_RUNTIME
+ string "Test runtime per job"
+ output yaml
+ default "60"
+ help
+ Runtime in seconds for each fio job. Default is 60 seconds
+ for reasonable test duration while allowing comprehensive
+ testing.
+
+config FIO_TESTS_RAMP_TIME
+ string "Ramp time before measurements"
+ output yaml
+ default "10"
+ help
+ Time in seconds to ramp up before starting measurements.
+ This allows the workload to stabilize before collecting
+ performance data.
+
+menu "Block size configuration"
+
+config FIO_TESTS_BS_4K
+ bool "4K block size tests"
+ output yaml
+ default y
+ help
+ Enable 4K block size testing. This is the most common
+ block size for many applications.
+
+config FIO_TESTS_BS_8K
+ bool "8K block size tests"
+ output yaml
+ default y
+ help
+ Enable 8K block size testing.
+
+config FIO_TESTS_BS_16K
+ bool "16K block size tests"
+ output yaml
+ default y
+ help
+ Enable 16K block size testing.
+
+config FIO_TESTS_BS_32K
+ bool "32K block size tests"
+ output yaml
+ default n
+ help
+ Enable 32K block size testing.
+
+config FIO_TESTS_BS_64K
+ bool "64K block size tests"
+ output yaml
+ default n
+ help
+ Enable 64K block size testing.
+
+config FIO_TESTS_BS_128K
+ bool "128K block size tests"
+ output yaml
+ default n
+ help
+ Enable 128K block size testing.
+
+endmenu
+
+menu "IO depth configuration"
+
+config FIO_TESTS_IODEPTH_1
+ bool "IO depth 1"
+ output yaml
+ default y
+ help
+ Test with IO depth of 1 (synchronous IO).
+
+config FIO_TESTS_IODEPTH_4
+ bool "IO depth 4"
+ output yaml
+ default y
+ help
+ Test with IO depth of 4.
+
+config FIO_TESTS_IODEPTH_8
+ bool "IO depth 8"
+ output yaml
+ default y
+ help
+ Test with IO depth of 8.
+
+config FIO_TESTS_IODEPTH_16
+ bool "IO depth 16"
+ output yaml
+ default y
+ help
+ Test with IO depth of 16.
+
+config FIO_TESTS_IODEPTH_32
+ bool "IO depth 32"
+ output yaml
+ default n
+ help
+ Test with IO depth of 32.
+
+config FIO_TESTS_IODEPTH_64
+ bool "IO depth 64"
+ output yaml
+ default n
+ help
+ Test with IO depth of 64.
+
+endmenu
+
+menu "Thread/job configuration"
+
+config FIO_TESTS_NUMJOBS_1
+ bool "Single job"
+ output yaml
+ default y
+ help
+ Test with a single fio job.
+
+config FIO_TESTS_NUMJOBS_2
+ bool "2 jobs"
+ output yaml
+ default y
+ help
+ Test with 2 concurrent fio jobs.
+
+config FIO_TESTS_NUMJOBS_4
+ bool "4 jobs"
+ output yaml
+ default y
+ help
+ Test with 4 concurrent fio jobs.
+
+config FIO_TESTS_NUMJOBS_8
+ bool "8 jobs"
+ output yaml
+ default n
+ help
+ Test with 8 concurrent fio jobs.
+
+config FIO_TESTS_NUMJOBS_16
+ bool "16 jobs"
+ output yaml
+ default n
+ help
+ Test with 16 concurrent fio jobs.
+
+endmenu
+
+menu "Workload patterns"
+
+config FIO_TESTS_PATTERN_RAND_READ
+ bool "Random read"
+ output yaml
+ default y
+ help
+ Enable random read workload testing.
+
+config FIO_TESTS_PATTERN_RAND_WRITE
+ bool "Random write"
+ output yaml
+ default y
+ help
+ Enable random write workload testing.
+
+config FIO_TESTS_PATTERN_SEQ_READ
+ bool "Sequential read"
+ output yaml
+ default y
+ help
+ Enable sequential read workload testing.
+
+config FIO_TESTS_PATTERN_SEQ_WRITE
+ bool "Sequential write"
+ output yaml
+ default y
+ help
+ Enable sequential write workload testing.
+
+config FIO_TESTS_PATTERN_MIXED_75_25
+ bool "Mixed 75% read / 25% write"
+ output yaml
+ default n
+ help
+ Enable mixed workload with 75% reads and 25% writes.
+
+config FIO_TESTS_PATTERN_MIXED_50_50
+ bool "Mixed 50% read / 50% write"
+ output yaml
+ default n
+ help
+ Enable mixed workload with 50% reads and 50% writes.
+
+endmenu
+
+menu "Advanced configuration"
+
+config FIO_TESTS_IOENGINE
+ string "IO engine to use"
+ output yaml
+ default "io_uring"
+ help
+ The fio IO engine to use. Options include:
+ - io_uring: Linux native async IO (recommended)
+ - libaio: Linux native async IO (legacy)
+ - psync: POSIX sync IO
+ - sync: Basic sync IO
+
+config FIO_TESTS_DIRECT
+ bool "Use direct IO"
+ output yaml
+ default y
+ help
+ Enable direct IO to bypass the page cache. This provides
+ more accurate storage device performance measurements.
+
+config FIO_TESTS_FSYNC_ON_CLOSE
+ bool "Fsync on close"
+ output yaml
+ default y
+ help
+ Call fsync() before closing files to ensure data is
+ written to storage.
+
+config FIO_TESTS_RESULTS_DIR
+ string "Results directory"
+ output yaml
+ default "/data/fio-tests"
+ help
+ Directory where test results and logs will be stored.
+ This should be on a different filesystem than the test
+ target to avoid interference.
+
+config FIO_TESTS_LOG_AVG_MSEC
+ int "Log averaging interval (msec)"
+ output yaml
+ default 1000
+ help
+ Interval in milliseconds for averaging performance logs.
+ Lower values provide more granular data but larger log files.
+
+config FIO_TESTS_ENABLE_GRAPHING
+ bool "Enable graphing and visualization"
+ output yaml
+ default y
+ help
+ Enable comprehensive graphing and visualization capabilities
+ for fio test results. This installs Python dependencies
+ including matplotlib, pandas, and seaborn for generating
+ performance analysis graphs.
+
+ Graphing features include:
+ - Performance heatmaps across block sizes and IO depths
+ - IOPS scaling analysis
+ - Latency distribution charts
+ - Workload pattern comparisons
+ - Baseline vs development A/B comparisons
+ - Trend analysis and correlation matrices
+
+if FIO_TESTS_ENABLE_GRAPHING
+
+config FIO_TESTS_GRAPH_FORMAT
+ string "Graph output format"
+ output yaml
+ default "png"
+ help
+ Output format for generated graphs. Common formats include:
+ - png: Portable Network Graphics (recommended)
+ - svg: Scalable Vector Graphics
+ - pdf: Portable Document Format
+ - jpg: JPEG format
+
+config FIO_TESTS_GRAPH_DPI
+ int "Graph resolution (DPI)"
+ output yaml
+ default 300
+ help
+ Resolution for generated graphs in dots per inch.
+ Higher values produce better quality but larger files.
+ Common values: 150 (screen), 300 (print), 600 (high quality).
+
+config FIO_TESTS_GRAPH_THEME
+ string "Matplotlib theme"
+ output yaml
+ default "default"
+ help
+ Matplotlib style theme for graphs. Options include:
+ - default: Default matplotlib style
+ - seaborn: Clean seaborn style
+ - dark_background: Dark theme
+ - ggplot: R ggplot2 style
+ - bmh: Bayesian Methods for Hackers style
+
+endif # FIO_TESTS_ENABLE_GRAPHING
+
+endmenu
diff --git a/workflows/fio-tests/Makefile b/workflows/fio-tests/Makefile
new file mode 100644
index 00000000..ea5bc2ce
--- /dev/null
+++ b/workflows/fio-tests/Makefile
@@ -0,0 +1,53 @@
+fio-tests:
+ $(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+ --connection=ssh \
+ --inventory $(KDEVOPS_HOSTFILE) \
+ --extra-vars=@$(KDEVOPS_EXTRA_VARS) \
+ playbooks/fio-tests.yml
+
+fio-tests-baseline:
+ $(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+ --connection=ssh \
+ --inventory $(KDEVOPS_HOSTFILE) \
+ --extra-vars=@$(KDEVOPS_EXTRA_VARS) \
+ playbooks/fio-tests-baseline.yml
+
+fio-tests-results:
+ $(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+ --connection=ssh \
+ --inventory $(KDEVOPS_HOSTFILE) \
+ --extra-vars=@$(KDEVOPS_EXTRA_VARS) \
+ playbooks/fio-tests-results.yml
+
+fio-tests-graph:
+ $(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+ --connection=ssh \
+ --inventory $(KDEVOPS_HOSTFILE) \
+ --extra-vars=@$(KDEVOPS_EXTRA_VARS) \
+ playbooks/fio-tests-graph.yml
+
+fio-tests-compare:
+ $(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+ --connection=ssh \
+ --inventory $(KDEVOPS_HOSTFILE) \
+ --extra-vars=@$(KDEVOPS_EXTRA_VARS) \
+ playbooks/fio-tests-compare.yml
+
+fio-tests-trend-analysis:
+ $(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+ --connection=ssh \
+ --inventory $(KDEVOPS_HOSTFILE) \
+ --extra-vars=@$(KDEVOPS_EXTRA_VARS) \
+ playbooks/fio-tests-trend-analysis.yml
+
+fio-tests-help-menu:
+ @echo "fio-tests options:"
+ @echo "fio-tests - run fio performance tests"
+ @echo "fio-tests-baseline - establish baseline results"
+ @echo "fio-tests-results - collect and analyze results"
+ @echo "fio-tests-graph - generate performance graphs"
+ @echo "fio-tests-compare - compare baseline vs dev results"
+ @echo "fio-tests-trend-analysis - analyze performance trends"
+ @echo ""
+
+HELP_TARGETS += fio-tests-help-menu
--
2.47.2
^ permalink raw reply related [flat|nested] only message in thread
only message in thread, other threads:[~2025-07-26 3:40 UTC | newest]
Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-07-26 3:40 [PATCH] workflows: add fio-tests performance tests Luis Chamberlain
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox