From: Luis Chamberlain <mcgrof@kernel.org>
To: Chuck Lever <cel@kernel.org>, Daniel Gomez <da.gomez@kruces.com>,
hui81.qi@samsung.com, kundan.kumar@samsung.com,
kdevops@lists.linux.dev
Cc: Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH 0/2] kdevops: add milvus with minio support
Date: Wed, 27 Aug 2025 02:31:59 -0700 [thread overview]
Message-ID: <20250827093202.3539990-1-mcgrof@kernel.org> (raw)
This adds the ability to test milvus on minio with different filesystem
configuration targets. There's a basic configuration you can run which
will just support one default filesystem, which will be used where you
place your docker image and also where we place the minio instance. Then
there is multifs support where just as with fstests support on kdevops
you can select a slew of different filesystem targets to try to test.
Recommendation is to stick to 40 iterations at 1,000,000 tests unless
you have more than 100 GiB per guest to spare. If you have space to
spare then you know how to ballpark it.
On High Capacity SSDs, the world is our oyster.
You can see a demo of results here:
https://github.com/mcgrof/demo-milvus-kdevops-results
These are just demos. On guests. Nothing really useful.
I should point out this has AB testing automated as well so we can
leverage this to test for instance ... parallel writeback in an
automated way ;)
If you want to test this you can also use this branch on kdevops:
https://github.com/linux-kdevops/kdevops/tree/mcgrof/20250827-milvus
I am in hopes someone will just prompt an AI for bare metal support
while I sleep. It should be... easy. Just create the partitions already,
use one host and ask the prompt to not mkfs for you. So don't use
multi-fs support at first. Just use the option to create the storage
partition where you place docker. In fact you can copy and paste this
prompt the the AI, and I think it will know what to do. You just skip
some steps as the filesystems can be created and mounted for you. You
just need the host file created by you manually for the target node.
That and infer user and group id support (WORKFLOW_INFER_USER_AND_GROUP).
Luis Chamberlain (2):
ai: add Milvus vector database benchmarking support
ai: add multi-filesystem testing support for Milvus benchmarks
.github/workflows/docker-tests.yml | 6 +
.gitignore | 3 +-
Makefile | 2 +-
README.md | 18 +
defconfigs/ai-milvus-docker | 113 ++
defconfigs/ai-milvus-docker-ci | 51 +
defconfigs/ai-milvus-multifs | 67 +
defconfigs/ai-milvus-multifs-distro | 109 ++
defconfigs/ai-milvus-multifs-extended | 108 ++
docs/ai/README.md | 108 ++
docs/ai/vector-databases/README.md | 75 +
docs/ai/vector-databases/milvus.md | 264 +++
kconfigs/workflows/Kconfig | 27 +
playbooks/ai.yml | 11 +
playbooks/ai_benchmark.yml | 8 +
playbooks/ai_destroy.yml | 24 +
playbooks/ai_install.yml | 14 +
playbooks/ai_multifs.yml | 24 +
playbooks/ai_results.yml | 6 +
playbooks/ai_setup.yml | 6 +
playbooks/ai_tests.yml | 31 +
playbooks/ai_uninstall.yml | 6 +
.../debian13-ai-btrfs-default-dev.yml | 8 +
.../host_vars/debian13-ai-btrfs-default.yml | 8 +
.../debian13-ai-ext4-16k-bigalloc-dev.yml | 8 +
.../debian13-ai-ext4-16k-bigalloc.yml | 8 +
.../host_vars/debian13-ai-ext4-4k-dev.yml | 8 +
playbooks/host_vars/debian13-ai-ext4-4k.yml | 8 +
.../host_vars/debian13-ai-xfs-16k-4ks-dev.yml | 10 +
.../host_vars/debian13-ai-xfs-16k-4ks.yml | 10 +
.../host_vars/debian13-ai-xfs-32k-4ks-dev.yml | 10 +
.../host_vars/debian13-ai-xfs-32k-4ks.yml | 10 +
.../host_vars/debian13-ai-xfs-4k-4ks-dev.yml | 10 +
.../host_vars/debian13-ai-xfs-64k-4ks-dev.yml | 10 +
.../host_vars/debian13-ai-xfs-64k-4ks.yml | 10 +
.../files/analyze_results.py | 1701 +++++++++++++++++
.../files/generate_better_graphs.py | 550 ++++++
.../files/generate_graphs.py | 362 ++++
.../files/generate_html_report.py | 610 ++++++
.../roles/ai_collect_results/tasks/main.yml | 202 ++
.../templates/analysis_config.json.j2 | 6 +
playbooks/roles/ai_destroy/tasks/main.yml | 63 +
.../roles/ai_docker_storage/tasks/main.yml | 123 ++
playbooks/roles/ai_install/tasks/main.yml | 90 +
.../roles/ai_milvus_storage/tasks/main.yml | 161 ++
.../tasks/generate_comparison.yml | 279 +++
playbooks/roles/ai_multifs_run/tasks/main.yml | 23 +
.../tasks/run_single_filesystem.yml | 104 +
.../templates/milvus_config.json.j2 | 42 +
.../roles/ai_multifs_setup/defaults/main.yml | 49 +
.../roles/ai_multifs_setup/tasks/main.yml | 70 +
playbooks/roles/ai_results/tasks/main.yml | 22 +
.../files/milvus_benchmark.py | 556 ++++++
.../roles/ai_run_benchmarks/tasks/main.yml | 181 ++
.../templates/benchmark_config.json.j2 | 24 +
playbooks/roles/ai_setup/tasks/main.yml | 115 ++
playbooks/roles/ai_uninstall/tasks/main.yml | 62 +
playbooks/roles/gen_hosts/tasks/main.yml | 33 +
.../roles/gen_hosts/templates/fstests.j2 | 2 +
playbooks/roles/gen_hosts/templates/gitr.j2 | 2 +
playbooks/roles/gen_hosts/templates/hosts.j2 | 99 +
.../roles/gen_hosts/templates/nfstest.j2 | 2 +
playbooks/roles/gen_hosts/templates/pynfs.j2 | 2 +
playbooks/roles/gen_nodes/tasks/main.yml | 124 ++
.../roles/guestfs/tasks/bringup/main.yml | 15 +
playbooks/roles/milvus/README.md | 181 ++
playbooks/roles/milvus/defaults/main.yml | 74 +
.../roles/milvus/files/milvus_benchmark.py | 348 ++++
playbooks/roles/milvus/files/milvus_utils.py | 134 ++
playbooks/roles/milvus/meta/main.yml | 30 +
playbooks/roles/milvus/tasks/benchmark.yml | 61 +
.../roles/milvus/tasks/benchmark_setup.yml | 58 +
.../roles/milvus/tasks/install_docker.yml | 97 +
playbooks/roles/milvus/tasks/main.yml | 52 +
playbooks/roles/milvus/tasks/setup.yml | 107 ++
.../milvus/templates/benchmark_config.json.j2 | 25 +
.../templates/docker-compose.override.yml.j2 | 24 +
.../milvus/templates/docker-compose.yml.j2 | 64 +
.../roles/milvus/templates/milvus.yaml.j2 | 30 +
.../milvus/templates/test_connection.py.j2 | 25 +
scripts/guestfs.Makefile | 2 +-
workflows/Makefile | 4 +
workflows/ai/Kconfig | 177 ++
workflows/ai/Kconfig.docker | 172 ++
workflows/ai/Kconfig.docker-storage | 201 ++
workflows/ai/Kconfig.fs | 118 ++
workflows/ai/Kconfig.multifs | 184 ++
workflows/ai/Kconfig.native | 184 ++
workflows/ai/Makefile | 160 ++
workflows/ai/scripts/analysis_config.json | 6 +
workflows/ai/scripts/analyze_results.py | 1701 +++++++++++++++++
workflows/ai/scripts/generate_graphs.py | 362 ++++
workflows/ai/scripts/generate_html_report.py | 610 ++++++
93 files changed, 12061 insertions(+), 3 deletions(-)
create mode 100644 defconfigs/ai-milvus-docker
create mode 100644 defconfigs/ai-milvus-docker-ci
create mode 100644 defconfigs/ai-milvus-multifs
create mode 100644 defconfigs/ai-milvus-multifs-distro
create mode 100644 defconfigs/ai-milvus-multifs-extended
create mode 100644 docs/ai/README.md
create mode 100644 docs/ai/vector-databases/README.md
create mode 100644 docs/ai/vector-databases/milvus.md
create mode 100644 playbooks/ai.yml
create mode 100644 playbooks/ai_benchmark.yml
create mode 100644 playbooks/ai_destroy.yml
create mode 100644 playbooks/ai_install.yml
create mode 100644 playbooks/ai_multifs.yml
create mode 100644 playbooks/ai_results.yml
create mode 100644 playbooks/ai_setup.yml
create mode 100644 playbooks/ai_tests.yml
create mode 100644 playbooks/ai_uninstall.yml
create mode 100644 playbooks/host_vars/debian13-ai-btrfs-default-dev.yml
create mode 100644 playbooks/host_vars/debian13-ai-btrfs-default.yml
create mode 100644 playbooks/host_vars/debian13-ai-ext4-16k-bigalloc-dev.yml
create mode 100644 playbooks/host_vars/debian13-ai-ext4-16k-bigalloc.yml
create mode 100644 playbooks/host_vars/debian13-ai-ext4-4k-dev.yml
create mode 100644 playbooks/host_vars/debian13-ai-ext4-4k.yml
create mode 100644 playbooks/host_vars/debian13-ai-xfs-16k-4ks-dev.yml
create mode 100644 playbooks/host_vars/debian13-ai-xfs-16k-4ks.yml
create mode 100644 playbooks/host_vars/debian13-ai-xfs-32k-4ks-dev.yml
create mode 100644 playbooks/host_vars/debian13-ai-xfs-32k-4ks.yml
create mode 100644 playbooks/host_vars/debian13-ai-xfs-4k-4ks-dev.yml
create mode 100644 playbooks/host_vars/debian13-ai-xfs-64k-4ks-dev.yml
create mode 100644 playbooks/host_vars/debian13-ai-xfs-64k-4ks.yml
create mode 100755 playbooks/roles/ai_collect_results/files/analyze_results.py
create mode 100755 playbooks/roles/ai_collect_results/files/generate_better_graphs.py
create mode 100755 playbooks/roles/ai_collect_results/files/generate_graphs.py
create mode 100755 playbooks/roles/ai_collect_results/files/generate_html_report.py
create mode 100644 playbooks/roles/ai_collect_results/tasks/main.yml
create mode 100644 playbooks/roles/ai_collect_results/templates/analysis_config.json.j2
create mode 100644 playbooks/roles/ai_destroy/tasks/main.yml
create mode 100644 playbooks/roles/ai_docker_storage/tasks/main.yml
create mode 100644 playbooks/roles/ai_install/tasks/main.yml
create mode 100644 playbooks/roles/ai_milvus_storage/tasks/main.yml
create mode 100644 playbooks/roles/ai_multifs_run/tasks/generate_comparison.yml
create mode 100644 playbooks/roles/ai_multifs_run/tasks/main.yml
create mode 100644 playbooks/roles/ai_multifs_run/tasks/run_single_filesystem.yml
create mode 100644 playbooks/roles/ai_multifs_run/templates/milvus_config.json.j2
create mode 100644 playbooks/roles/ai_multifs_setup/defaults/main.yml
create mode 100644 playbooks/roles/ai_multifs_setup/tasks/main.yml
create mode 100644 playbooks/roles/ai_results/tasks/main.yml
create mode 100644 playbooks/roles/ai_run_benchmarks/files/milvus_benchmark.py
create mode 100644 playbooks/roles/ai_run_benchmarks/tasks/main.yml
create mode 100644 playbooks/roles/ai_run_benchmarks/templates/benchmark_config.json.j2
create mode 100644 playbooks/roles/ai_setup/tasks/main.yml
create mode 100644 playbooks/roles/ai_uninstall/tasks/main.yml
create mode 100644 playbooks/roles/milvus/README.md
create mode 100644 playbooks/roles/milvus/defaults/main.yml
create mode 100644 playbooks/roles/milvus/files/milvus_benchmark.py
create mode 100644 playbooks/roles/milvus/files/milvus_utils.py
create mode 100644 playbooks/roles/milvus/meta/main.yml
create mode 100644 playbooks/roles/milvus/tasks/benchmark.yml
create mode 100644 playbooks/roles/milvus/tasks/benchmark_setup.yml
create mode 100644 playbooks/roles/milvus/tasks/install_docker.yml
create mode 100644 playbooks/roles/milvus/tasks/main.yml
create mode 100644 playbooks/roles/milvus/tasks/setup.yml
create mode 100644 playbooks/roles/milvus/templates/benchmark_config.json.j2
create mode 100644 playbooks/roles/milvus/templates/docker-compose.override.yml.j2
create mode 100644 playbooks/roles/milvus/templates/docker-compose.yml.j2
create mode 100644 playbooks/roles/milvus/templates/milvus.yaml.j2
create mode 100644 playbooks/roles/milvus/templates/test_connection.py.j2
create mode 100644 workflows/ai/Kconfig
create mode 100644 workflows/ai/Kconfig.docker
create mode 100644 workflows/ai/Kconfig.docker-storage
create mode 100644 workflows/ai/Kconfig.fs
create mode 100644 workflows/ai/Kconfig.multifs
create mode 100644 workflows/ai/Kconfig.native
create mode 100644 workflows/ai/Makefile
create mode 100644 workflows/ai/scripts/analysis_config.json
create mode 100755 workflows/ai/scripts/analyze_results.py
create mode 100755 workflows/ai/scripts/generate_graphs.py
create mode 100755 workflows/ai/scripts/generate_html_report.py
--
2.50.1
next reply other threads:[~2025-08-27 9:32 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-08-27 9:31 Luis Chamberlain [this message]
2025-08-27 9:32 ` [PATCH 1/2] ai: add Milvus vector database benchmarking support Luis Chamberlain
2025-08-27 9:32 ` [PATCH 2/2] ai: add multi-filesystem testing support for Milvus benchmarks Luis Chamberlain
2025-08-27 14:47 ` Chuck Lever
2025-08-27 19:24 ` Luis Chamberlain
2025-09-01 20:11 ` Daniel Gomez
2025-09-01 20:27 ` Luis Chamberlain
2025-08-29 2:05 ` [PATCH 0/2] kdevops: add milvus with minio support Luis Chamberlain
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250827093202.3539990-1-mcgrof@kernel.org \
--to=mcgrof@kernel.org \
--cc=cel@kernel.org \
--cc=da.gomez@kruces.com \
--cc=hui81.qi@samsung.com \
--cc=kdevops@lists.linux.dev \
--cc=kundan.kumar@samsung.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox