From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60B8418A6AD for ; Wed, 27 Aug 2025 09:32:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756287127; cv=none; b=k86fuwbQ8oI5a79BjaneSNBmGbrTVf+6EJVikzLv2PRLiiETYVfIufFBJg8AuwJd6YEmJJ0e7IKBoN41ysKRxJyYlAPuRMtiKptYx11Om3hak3YmtKnvxTPaa2NvxMaCuUpKWCg708rwToOk5J5akmD1tWRUPi6dQ8g4okf9/bE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756287127; c=relaxed/simple; bh=N9wwoKTdPQaI6hYtsovRDEOFDJgqhufizPET2m2RHtY=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=NoNX3P7LDiqpe57vYnIM3nhiV5tKitUoosx5CnrVz5pj41B4I6xvhNg/QGkQolWU3oV3e0wb6ll9hO1y3q661BZM1Q/D9wWwbiKmoqpTzXlQx4e5C67iWk8EXh3y+DzI/B9FDvgAJ0Bpb7PEQCwzGOhTbwOCJAb8t+vwgnXESDo= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=SxT4NQ0o; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="SxT4NQ0o" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=c8m4gjhDPT0gHW6qgRo5xov2QCbAE798mO4BhfdfIWo=; b=SxT4NQ0oFJp0kZTymGQqPRaPvK GUs+XuuypKDQU/o/7dTSyAv6RYTPZV4MPloIX5huvkFQW02Bmzeyhu9r5ww8geof2R9/S5fztLnz5 iXammiLtgzJmhaKdyx0aurSDkyDzUtrnaYiFhpgYFbjpQwo/X3s/csfjPv2SarhdKaXhl2/auYnu0 5ZO6NKLwVnc8EXuZC7GmV9Yhw0DuzKeo/PL1gK0cLrZoS/HZbFISa1jUpUYUgNb4ybKrweLUNPBvF u2vTMz2vB6LYzyDPZhanYU7LxJ8vWIr8reN3SyLTl/LXrgglaPdbKaBBeElEV/kVUuSjr8u1BECkK nOZPJyBA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1urCVg-0000000Equk-1cHW; Wed, 27 Aug 2025 09:32:04 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , hui81.qi@samsung.com, kundan.kumar@samsung.com, kdevops@lists.linux.dev Cc: Luis Chamberlain Subject: [PATCH 0/2] kdevops: add milvus with minio support Date: Wed, 27 Aug 2025 02:31:59 -0700 Message-ID: <20250827093202.3539990-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain This adds the ability to test milvus on minio with different filesystem configuration targets. There's a basic configuration you can run which will just support one default filesystem, which will be used where you place your docker image and also where we place the minio instance. Then there is multifs support where just as with fstests support on kdevops you can select a slew of different filesystem targets to try to test. Recommendation is to stick to 40 iterations at 1,000,000 tests unless you have more than 100 GiB per guest to spare. If you have space to spare then you know how to ballpark it. On High Capacity SSDs, the world is our oyster. You can see a demo of results here: https://github.com/mcgrof/demo-milvus-kdevops-results These are just demos. On guests. Nothing really useful. I should point out this has AB testing automated as well so we can leverage this to test for instance ... parallel writeback in an automated way ;) If you want to test this you can also use this branch on kdevops: https://github.com/linux-kdevops/kdevops/tree/mcgrof/20250827-milvus I am in hopes someone will just prompt an AI for bare metal support while I sleep. It should be... easy. Just create the partitions already, use one host and ask the prompt to not mkfs for you. So don't use multi-fs support at first. Just use the option to create the storage partition where you place docker. In fact you can copy and paste this prompt the the AI, and I think it will know what to do. You just skip some steps as the filesystems can be created and mounted for you. You just need the host file created by you manually for the target node. That and infer user and group id support (WORKFLOW_INFER_USER_AND_GROUP). Luis Chamberlain (2): ai: add Milvus vector database benchmarking support ai: add multi-filesystem testing support for Milvus benchmarks .github/workflows/docker-tests.yml | 6 + .gitignore | 3 +- Makefile | 2 +- README.md | 18 + defconfigs/ai-milvus-docker | 113 ++ defconfigs/ai-milvus-docker-ci | 51 + defconfigs/ai-milvus-multifs | 67 + defconfigs/ai-milvus-multifs-distro | 109 ++ defconfigs/ai-milvus-multifs-extended | 108 ++ docs/ai/README.md | 108 ++ docs/ai/vector-databases/README.md | 75 + docs/ai/vector-databases/milvus.md | 264 +++ kconfigs/workflows/Kconfig | 27 + playbooks/ai.yml | 11 + playbooks/ai_benchmark.yml | 8 + playbooks/ai_destroy.yml | 24 + playbooks/ai_install.yml | 14 + playbooks/ai_multifs.yml | 24 + playbooks/ai_results.yml | 6 + playbooks/ai_setup.yml | 6 + playbooks/ai_tests.yml | 31 + playbooks/ai_uninstall.yml | 6 + .../debian13-ai-btrfs-default-dev.yml | 8 + .../host_vars/debian13-ai-btrfs-default.yml | 8 + .../debian13-ai-ext4-16k-bigalloc-dev.yml | 8 + .../debian13-ai-ext4-16k-bigalloc.yml | 8 + .../host_vars/debian13-ai-ext4-4k-dev.yml | 8 + playbooks/host_vars/debian13-ai-ext4-4k.yml | 8 + .../host_vars/debian13-ai-xfs-16k-4ks-dev.yml | 10 + .../host_vars/debian13-ai-xfs-16k-4ks.yml | 10 + .../host_vars/debian13-ai-xfs-32k-4ks-dev.yml | 10 + .../host_vars/debian13-ai-xfs-32k-4ks.yml | 10 + .../host_vars/debian13-ai-xfs-4k-4ks-dev.yml | 10 + .../host_vars/debian13-ai-xfs-64k-4ks-dev.yml | 10 + .../host_vars/debian13-ai-xfs-64k-4ks.yml | 10 + .../files/analyze_results.py | 1701 +++++++++++++++++ .../files/generate_better_graphs.py | 550 ++++++ .../files/generate_graphs.py | 362 ++++ .../files/generate_html_report.py | 610 ++++++ .../roles/ai_collect_results/tasks/main.yml | 202 ++ .../templates/analysis_config.json.j2 | 6 + playbooks/roles/ai_destroy/tasks/main.yml | 63 + .../roles/ai_docker_storage/tasks/main.yml | 123 ++ playbooks/roles/ai_install/tasks/main.yml | 90 + .../roles/ai_milvus_storage/tasks/main.yml | 161 ++ .../tasks/generate_comparison.yml | 279 +++ playbooks/roles/ai_multifs_run/tasks/main.yml | 23 + .../tasks/run_single_filesystem.yml | 104 + .../templates/milvus_config.json.j2 | 42 + .../roles/ai_multifs_setup/defaults/main.yml | 49 + .../roles/ai_multifs_setup/tasks/main.yml | 70 + playbooks/roles/ai_results/tasks/main.yml | 22 + .../files/milvus_benchmark.py | 556 ++++++ .../roles/ai_run_benchmarks/tasks/main.yml | 181 ++ .../templates/benchmark_config.json.j2 | 24 + playbooks/roles/ai_setup/tasks/main.yml | 115 ++ playbooks/roles/ai_uninstall/tasks/main.yml | 62 + playbooks/roles/gen_hosts/tasks/main.yml | 33 + .../roles/gen_hosts/templates/fstests.j2 | 2 + playbooks/roles/gen_hosts/templates/gitr.j2 | 2 + playbooks/roles/gen_hosts/templates/hosts.j2 | 99 + .../roles/gen_hosts/templates/nfstest.j2 | 2 + playbooks/roles/gen_hosts/templates/pynfs.j2 | 2 + playbooks/roles/gen_nodes/tasks/main.yml | 124 ++ .../roles/guestfs/tasks/bringup/main.yml | 15 + playbooks/roles/milvus/README.md | 181 ++ playbooks/roles/milvus/defaults/main.yml | 74 + .../roles/milvus/files/milvus_benchmark.py | 348 ++++ playbooks/roles/milvus/files/milvus_utils.py | 134 ++ playbooks/roles/milvus/meta/main.yml | 30 + playbooks/roles/milvus/tasks/benchmark.yml | 61 + .../roles/milvus/tasks/benchmark_setup.yml | 58 + .../roles/milvus/tasks/install_docker.yml | 97 + playbooks/roles/milvus/tasks/main.yml | 52 + playbooks/roles/milvus/tasks/setup.yml | 107 ++ .../milvus/templates/benchmark_config.json.j2 | 25 + .../templates/docker-compose.override.yml.j2 | 24 + .../milvus/templates/docker-compose.yml.j2 | 64 + .../roles/milvus/templates/milvus.yaml.j2 | 30 + .../milvus/templates/test_connection.py.j2 | 25 + scripts/guestfs.Makefile | 2 +- workflows/Makefile | 4 + workflows/ai/Kconfig | 177 ++ workflows/ai/Kconfig.docker | 172 ++ workflows/ai/Kconfig.docker-storage | 201 ++ workflows/ai/Kconfig.fs | 118 ++ workflows/ai/Kconfig.multifs | 184 ++ workflows/ai/Kconfig.native | 184 ++ workflows/ai/Makefile | 160 ++ workflows/ai/scripts/analysis_config.json | 6 + workflows/ai/scripts/analyze_results.py | 1701 +++++++++++++++++ workflows/ai/scripts/generate_graphs.py | 362 ++++ workflows/ai/scripts/generate_html_report.py | 610 ++++++ 93 files changed, 12061 insertions(+), 3 deletions(-) create mode 100644 defconfigs/ai-milvus-docker create mode 100644 defconfigs/ai-milvus-docker-ci create mode 100644 defconfigs/ai-milvus-multifs create mode 100644 defconfigs/ai-milvus-multifs-distro create mode 100644 defconfigs/ai-milvus-multifs-extended create mode 100644 docs/ai/README.md create mode 100644 docs/ai/vector-databases/README.md create mode 100644 docs/ai/vector-databases/milvus.md create mode 100644 playbooks/ai.yml create mode 100644 playbooks/ai_benchmark.yml create mode 100644 playbooks/ai_destroy.yml create mode 100644 playbooks/ai_install.yml create mode 100644 playbooks/ai_multifs.yml create mode 100644 playbooks/ai_results.yml create mode 100644 playbooks/ai_setup.yml create mode 100644 playbooks/ai_tests.yml create mode 100644 playbooks/ai_uninstall.yml create mode 100644 playbooks/host_vars/debian13-ai-btrfs-default-dev.yml create mode 100644 playbooks/host_vars/debian13-ai-btrfs-default.yml create mode 100644 playbooks/host_vars/debian13-ai-ext4-16k-bigalloc-dev.yml create mode 100644 playbooks/host_vars/debian13-ai-ext4-16k-bigalloc.yml create mode 100644 playbooks/host_vars/debian13-ai-ext4-4k-dev.yml create mode 100644 playbooks/host_vars/debian13-ai-ext4-4k.yml create mode 100644 playbooks/host_vars/debian13-ai-xfs-16k-4ks-dev.yml create mode 100644 playbooks/host_vars/debian13-ai-xfs-16k-4ks.yml create mode 100644 playbooks/host_vars/debian13-ai-xfs-32k-4ks-dev.yml create mode 100644 playbooks/host_vars/debian13-ai-xfs-32k-4ks.yml create mode 100644 playbooks/host_vars/debian13-ai-xfs-4k-4ks-dev.yml create mode 100644 playbooks/host_vars/debian13-ai-xfs-64k-4ks-dev.yml create mode 100644 playbooks/host_vars/debian13-ai-xfs-64k-4ks.yml create mode 100755 playbooks/roles/ai_collect_results/files/analyze_results.py create mode 100755 playbooks/roles/ai_collect_results/files/generate_better_graphs.py create mode 100755 playbooks/roles/ai_collect_results/files/generate_graphs.py create mode 100755 playbooks/roles/ai_collect_results/files/generate_html_report.py create mode 100644 playbooks/roles/ai_collect_results/tasks/main.yml create mode 100644 playbooks/roles/ai_collect_results/templates/analysis_config.json.j2 create mode 100644 playbooks/roles/ai_destroy/tasks/main.yml create mode 100644 playbooks/roles/ai_docker_storage/tasks/main.yml create mode 100644 playbooks/roles/ai_install/tasks/main.yml create mode 100644 playbooks/roles/ai_milvus_storage/tasks/main.yml create mode 100644 playbooks/roles/ai_multifs_run/tasks/generate_comparison.yml create mode 100644 playbooks/roles/ai_multifs_run/tasks/main.yml create mode 100644 playbooks/roles/ai_multifs_run/tasks/run_single_filesystem.yml create mode 100644 playbooks/roles/ai_multifs_run/templates/milvus_config.json.j2 create mode 100644 playbooks/roles/ai_multifs_setup/defaults/main.yml create mode 100644 playbooks/roles/ai_multifs_setup/tasks/main.yml create mode 100644 playbooks/roles/ai_results/tasks/main.yml create mode 100644 playbooks/roles/ai_run_benchmarks/files/milvus_benchmark.py create mode 100644 playbooks/roles/ai_run_benchmarks/tasks/main.yml create mode 100644 playbooks/roles/ai_run_benchmarks/templates/benchmark_config.json.j2 create mode 100644 playbooks/roles/ai_setup/tasks/main.yml create mode 100644 playbooks/roles/ai_uninstall/tasks/main.yml create mode 100644 playbooks/roles/milvus/README.md create mode 100644 playbooks/roles/milvus/defaults/main.yml create mode 100644 playbooks/roles/milvus/files/milvus_benchmark.py create mode 100644 playbooks/roles/milvus/files/milvus_utils.py create mode 100644 playbooks/roles/milvus/meta/main.yml create mode 100644 playbooks/roles/milvus/tasks/benchmark.yml create mode 100644 playbooks/roles/milvus/tasks/benchmark_setup.yml create mode 100644 playbooks/roles/milvus/tasks/install_docker.yml create mode 100644 playbooks/roles/milvus/tasks/main.yml create mode 100644 playbooks/roles/milvus/tasks/setup.yml create mode 100644 playbooks/roles/milvus/templates/benchmark_config.json.j2 create mode 100644 playbooks/roles/milvus/templates/docker-compose.override.yml.j2 create mode 100644 playbooks/roles/milvus/templates/docker-compose.yml.j2 create mode 100644 playbooks/roles/milvus/templates/milvus.yaml.j2 create mode 100644 playbooks/roles/milvus/templates/test_connection.py.j2 create mode 100644 workflows/ai/Kconfig create mode 100644 workflows/ai/Kconfig.docker create mode 100644 workflows/ai/Kconfig.docker-storage create mode 100644 workflows/ai/Kconfig.fs create mode 100644 workflows/ai/Kconfig.multifs create mode 100644 workflows/ai/Kconfig.native create mode 100644 workflows/ai/Makefile create mode 100644 workflows/ai/scripts/analysis_config.json create mode 100755 workflows/ai/scripts/analyze_results.py create mode 100755 workflows/ai/scripts/generate_graphs.py create mode 100755 workflows/ai/scripts/generate_html_report.py -- 2.50.1