Re: kdevops-ng: graduating kdevops beyond Ansible

Re: kdevops-ng: graduating kdevops beyond Ansible

[Daniel Gomez]

On 2026-06-18T08:31:51-04:00, Jeff Layton <jlayton@kernel.org> wrote:
> On Thu, 2026-06-18 at 11:30 +0200, Daniel Gomez wrote:
> > kdevops is a framework for Linux kernel development and test automation.
> > Its core features, namely workflow reproducibility, variability, and
> > scalability, are delivered through Kconfig, the variability language,
> > and Ansible, which provides host and guest idempotency along with
> > workflow orchestration at scale, whether on baremetal, local VMs, or
> > the cloud.
> > 
> > kdevops supports rolling distributions such as Debian testing, Fedora,
> > and openSUSE. Recently we extended Nix support, which raised the
> > question: how do we drive Nix's declarative language from Ansible? We
> > answered by wiring Nix in under Ansible and its templates, as one more
> > way to declare host and guest environments. But that was the wrong
> > framing: we had bolted Nix onto today's toolkit instead of rethinking
> > it. Reproducibility and idempotency now come from Nix by construction,
> > so I think Ansible's original reason for being in kdevops falls away.
> > The better question is: how do we keep kdevops's core principles, lean
> > on Nix, and drop Ansible?
> > 
> > What remains once you do is not the configuration management plane. It
> > is development workflow orchestration: build QEMU, build the kernel,
> > build a guest rootfs/closure, boot it, run a test, collect results,
> > diff against a baseline. That work is imperative and sequenced, work
> > for a workflow engine, which is where tools like Windmill [1] come in.
> > Windmill calls itself as "the fastest workflow engine" and an
> > "open-source developer platform to power your entire infra and turn
> > scripts into webhooks, workflows and UIs." Choosing to move kdevops
> > onto Windmill would keep what made kdevops kdevops, namely workflows,
> > quick bring-ups, baselines, and A/B regression detection, while trading
> > Kconfig, Make, Ansible, and host-distro provisioning for typed
> > run-forms, flows as code, and a worker queue. Nix supplies the
> > environment, much like a container or venv/poetry, along with the guest
> > OS system closure: declarative and portable. Windmill orchestrates the
> > whole pipeline end to end, graduating kdevops into a fully reproducible,
> > scalable, and configurable kernel-development framework, with both a UI
> > and a CLI, that runs locally or in the cloud. Defined as code and driven
> > by schedules and triggers, the same flows also make it a continuous
> > integration pipeline. Because steps can be written in any language
> > Windmill supports, including Ansible, Bash, Go, Python, and Rust,
> > developers can not only use kdevops but extend it with their own
> > scripts, turning it into a workflow hub. Note that choosing this path
> > does not mean NixOS is required on the controller node; Nix is simply a
> > runtime dependency that can be installed alongside your distro of
> > choice.
> > 
> > It'd be good to know what folks think about the possibility of evolving
> > kdevops in this direction, deprecating Ansible along with Kconfig and
> > Makefiles in favor of the new approach. To that end, I suggest a demo
> > day where I can show why I think this is the next step worth taking, and
> > whether it's a tradeoff users and maintainers are willing to make.
> >
> > If this is of interest and you'd like a look, I've ported equivalents of
> > bootlinux (direct boot), qemu-build, and the systemd/QEMU bringup (QSU),
> > plus an fstests run for XFS in the proof-of-concept demo project [2].
> > You can also find some screenshots in [3].
> > 
> > A note on licensing. Windmill's engine is AGPLv3; its OpenFlow flow
> > format and client libraries are Apache-2.0. kdevops-ng runs Windmill
> > unmodified and self-hosted as a separate service, and the flows and
> > scripts are kdevops-ng's own copyleft-next-0.3.1 code, executed by
> > Windmill rather than derived from it, so there shouldn't be any
> > licensing concerns.
> > 
> > [1] https://windmill.dev
> > [2] https://github.com/dagomez137/kdevops-ng
> > [3] https://github.com/dagomez137/kdevops-ng/tree/main/screenshots
>
> Some thoughts:
>
> I'd be interested to see the demo.  It's a little hard to make a
> judgment about moving it in this direction without knowing specifically
> what it would look like.  I took a quick look at the git repo and the
> windmill site, but I don't really "get it" yet.

+cc xfs folks.

I have published a documentation website here:
https://kdevops-ng.readthedocs.io/en/latest/

It covers the project's concepts, terminology, deployment, roadmap and more.
I've also included a demo video to showcasing how to build the kernel
and run the fstest suite for XFS and review the results (all in Windmill UI):

https://kdevops-ng.readthedocs.io/en/latest/getting-started/demos.html

I'd also like to suggest holding an office hours session toward the end of 
this month (in about four weeks), so we can discuss about this, questions, etc.

kdevops-ng: graduating kdevops beyond Ansible

[Daniel Gomez]

kdevops is a framework for Linux kernel development and test automation.
Its core features, namely workflow reproducibility, variability, and
scalability, are delivered through Kconfig, the variability language,
and Ansible, which provides host and guest idempotency along with
workflow orchestration at scale, whether on baremetal, local VMs, or
the cloud.

kdevops supports rolling distributions such as Debian testing, Fedora,
and openSUSE. Recently we extended Nix support, which raised the
question: how do we drive Nix's declarative language from Ansible? We
answered by wiring Nix in under Ansible and its templates, as one more
way to declare host and guest environments. But that was the wrong
framing: we had bolted Nix onto today's toolkit instead of rethinking
it. Reproducibility and idempotency now come from Nix by construction,
so I think Ansible's original reason for being in kdevops falls away.
The better question is: how do we keep kdevops's core principles, lean
on Nix, and drop Ansible?

What remains once you do is not the configuration management plane. It
is development workflow orchestration: build QEMU, build the kernel,
build a guest rootfs/closure, boot it, run a test, collect results,
diff against a baseline. That work is imperative and sequenced, work
for a workflow engine, which is where tools like Windmill [1] come in.
Windmill calls itself as "the fastest workflow engine" and an
"open-source developer platform to power your entire infra and turn
scripts into webhooks, workflows and UIs." Choosing to move kdevops
onto Windmill would keep what made kdevops kdevops, namely workflows,
quick bring-ups, baselines, and A/B regression detection, while trading
Kconfig, Make, Ansible, and host-distro provisioning for typed
run-forms, flows as code, and a worker queue. Nix supplies the
environment, much like a container or venv/poetry, along with the guest
OS system closure: declarative and portable. Windmill orchestrates the
whole pipeline end to end, graduating kdevops into a fully reproducible,
scalable, and configurable kernel-development framework, with both a UI
and a CLI, that runs locally or in the cloud. Defined as code and driven
by schedules and triggers, the same flows also make it a continuous
integration pipeline. Because steps can be written in any language
Windmill supports, including Ansible, Bash, Go, Python, and Rust,
developers can not only use kdevops but extend it with their own
scripts, turning it into a workflow hub. Note that choosing this path
does not mean NixOS is required on the controller node; Nix is simply a
runtime dependency that can be installed alongside your distro of
choice.

It'd be good to know what folks think about the possibility of evolving
kdevops in this direction, deprecating Ansible along with Kconfig and
Makefiles in favor of the new approach. To that end, I suggest a demo
day where I can show why I think this is the next step worth taking, and
whether it's a tradeoff users and maintainers are willing to make.

If this is of interest and you'd like a look, I've ported equivalents of
bootlinux (direct boot), qemu-build, and the systemd/QEMU bringup (QSU),
plus an fstests run for XFS in the proof-of-concept demo project [2].
You can also find some screenshots in [3].

A note on licensing. Windmill's engine is AGPLv3; its OpenFlow flow
format and client libraries are Apache-2.0. kdevops-ng runs Windmill
unmodified and self-hosted as a separate service, and the flows and
scripts are kdevops-ng's own copyleft-next-0.3.1 code, executed by
Windmill rather than derived from it, so there shouldn't be any
licensing concerns.

[1] https://windmill.dev
[2] https://github.com/dagomez137/kdevops-ng
[3] https://github.com/dagomez137/kdevops-ng/tree/main/screenshots

Re: kdevops-ng: graduating kdevops beyond Ansible

[Jeff Layton]

On Thu, 2026-06-18 at 11:30 +0200, Daniel Gomez wrote:
> kdevops is a framework for Linux kernel development and test automation.
> Its core features, namely workflow reproducibility, variability, and
> scalability, are delivered through Kconfig, the variability language,
> and Ansible, which provides host and guest idempotency along with
> workflow orchestration at scale, whether on baremetal, local VMs, or
> the cloud.
> 
> kdevops supports rolling distributions such as Debian testing, Fedora,
> and openSUSE. Recently we extended Nix support, which raised the
> question: how do we drive Nix's declarative language from Ansible? We
> answered by wiring Nix in under Ansible and its templates, as one more
> way to declare host and guest environments. But that was the wrong
> framing: we had bolted Nix onto today's toolkit instead of rethinking
> it. Reproducibility and idempotency now come from Nix by construction,
> so I think Ansible's original reason for being in kdevops falls away.
> The better question is: how do we keep kdevops's core principles, lean
> on Nix, and drop Ansible?
> 
> What remains once you do is not the configuration management plane. It
> is development workflow orchestration: build QEMU, build the kernel,
> build a guest rootfs/closure, boot it, run a test, collect results,
> diff against a baseline. That work is imperative and sequenced, work
> for a workflow engine, which is where tools like Windmill [1] come in.
> Windmill calls itself as "the fastest workflow engine" and an
> "open-source developer platform to power your entire infra and turn
> scripts into webhooks, workflows and UIs." Choosing to move kdevops
> onto Windmill would keep what made kdevops kdevops, namely workflows,
> quick bring-ups, baselines, and A/B regression detection, while trading
> Kconfig, Make, Ansible, and host-distro provisioning for typed
> run-forms, flows as code, and a worker queue. Nix supplies the
> environment, much like a container or venv/poetry, along with the guest
> OS system closure: declarative and portable. Windmill orchestrates the
> whole pipeline end to end, graduating kdevops into a fully reproducible,
> scalable, and configurable kernel-development framework, with both a UI
> and a CLI, that runs locally or in the cloud. Defined as code and driven
> by schedules and triggers, the same flows also make it a continuous
> integration pipeline. Because steps can be written in any language
> Windmill supports, including Ansible, Bash, Go, Python, and Rust,
> developers can not only use kdevops but extend it with their own
> scripts, turning it into a workflow hub. Note that choosing this path
> does not mean NixOS is required on the controller node; Nix is simply a
> runtime dependency that can be installed alongside your distro of
> choice.
> 
> It'd be good to know what folks think about the possibility of evolving
> kdevops in this direction, deprecating Ansible along with Kconfig and
> Makefiles in favor of the new approach. To that end, I suggest a demo
> day where I can show why I think this is the next step worth taking, and
> whether it's a tradeoff users and maintainers are willing to make.
>
> If this is of interest and you'd like a look, I've ported equivalents of
> bootlinux (direct boot), qemu-build, and the systemd/QEMU bringup (QSU),
> plus an fstests run for XFS in the proof-of-concept demo project [2].
> You can also find some screenshots in [3].
> 
> A note on licensing. Windmill's engine is AGPLv3; its OpenFlow flow
> format and client libraries are Apache-2.0. kdevops-ng runs Windmill
> unmodified and self-hosted as a separate service, and the flows and
> scripts are kdevops-ng's own copyleft-next-0.3.1 code, executed by
> Windmill rather than derived from it, so there shouldn't be any
> licensing concerns.
> 
> [1] https://windmill.dev
> [2] https://github.com/dagomez137/kdevops-ng
> [3] https://github.com/dagomez137/kdevops-ng/tree/main/screenshots

Some thoughts:

I'd be interested to see the demo.  It's a little hard to make a
judgment about moving it in this direction without knowing specifically
what it would look like.  I took a quick look at the git repo and the
windmill site, but I don't really "get it" yet.

I do agree that kconfig/makefiles are not really suited to this task.
We've made it work, but it's a bit of a square peg in a round hole.

One of the things I liked is that kdevops spawns a normal (familiar)
distro, and that makes it easy to get in and troubleshoot when things
are broken. If I have to learn how to operate in yet another new
distro, I suppose I can, but it doesn't excite me.

OTOH, the goal here is kernel testing, so userland really doesn't
matter too much.
-- 
Jeff Layton <jlayton@kernel.org>

Re: kdevops-ng: graduating kdevops beyond Ansible

[Daniel Gomez]

On 18/06/2026 14.31, Jeff Layton wrote:
> On Thu, 2026-06-18 at 11:30 +0200, Daniel Gomez wrote:
> Some thoughts:
> 
> I'd be interested to see the demo.  

Great! I will send an invite if this thread moves forward.

> It's a little hard to make a
> judgment about moving it in this direction without knowing specifically
> what it would look like.  I took a quick look at the git repo and the
> windmill site, but I don't really "get it" yet.

100%.

AFAIK, even Windmill itself struggles a bit to position what it is in
the market. But I think they get some obviously useful things right for
developer workflow orchestration that nobody else does. BTW, before
landing on this project I also looked at others: cijoe [1], Kestra,
and, at a different level, task.dev and just.systems. And these are the
alternatives Windmill compares itself against [2], in case any of them
are familiar to you (none were to me).

For a quick sense of what Windmill is, here's their architecture
diagram: https://www.windmill.dev/docs/misc/architecture

The way I think of it: a GitLab runner plus the YAML pipeline in
.gitlab-ci.yml (or the GitHub Actions equivalent), but for your
development workflows, not your project's CI. Though it can obviously be
used for project's CI too. All in one package, with far more pipeline
("flow", in Windmill terms) features [3]. That last part is one that I'm
also interested in: we could turn kdevops into a kernel regression
workflow engine with git-bisect and generate reports we can submit
directly to the mailing list and maintainers (similar to 0-day). Yes, we
can already do this today with Luis's patches from last year (I tried
them and they work), but it's exactly the kind of workflow where you can
see Ansible isn't the right tool.

Link: https://cijoe.readthedocs.io/en/latest/# [1]
Link: https://www.windmill.dev/docs/compared_to/peers [2]
Link: https://www.windmill.dev/docs/core_concepts#flow-specific-features [3]

> 
> I do agree that kconfig/makefiles are not really suited to this task.
> We've made it work, but it's a bit of a square peg in a round hole.
> 
> One of the things I liked is that kdevops spawns a normal (familiar)
> distro, and that makes it easy to get in and troubleshoot when things
> are broken. If I have to learn how to operate in yet another new
> distro, I suppose I can, but it doesn't excite me.

I understand, and I'm 100% with you: Nix (+ Windmill) is simply too
many new things at once.

The real benefit here is that the guest and the controller run the exact
same environment. You could compare it to virtme-ng with overlayfs,
where, AFAIK, you boot a guest off the host's rootfs (read-only, with
an overlay for writes), so the guest mirrors the host. The difference is
that with virtme-ng that shared environment is still whatever Debian or
Fedora you happen to be running, so it isn't pinned or reproducible;
with Nix it's the same shared environment (same /nix/store).

A way forward could be to use Windmill as the Ansible, Makefile, Kconfig
replacement or the Nix-based paths, and keep the current toolkit for the
libvirt/libguestfs/Terraform paths. Both can coexist.

> 
> OTOH, the goal here is kernel testing, so userland really doesn't
> matter too much.

But we also need changes on userland test suites so I think it matters
too. I use Nix for exactly that, to build and pin custom versions
of the test suites and tools (e.g. fstests, fio, bcc, etc.) that I can
run either baremetal or in a VM. The benefit is that the host and guest
share the same /nix/store (the guest mounts it over virtiofs), so a tool
you build on the host is the exact same build the guest runs.

Re: kdevops-ng: graduating kdevops beyond Ansible

[Chuck Lever]

On Thu, Jun 18, 2026, at 5:30 AM, Daniel Gomez wrote:
> kdevops is a framework for Linux kernel development and test automation.
> Its core features, namely workflow reproducibility, variability, and
> scalability, are delivered through Kconfig, the variability language,
> and Ansible, which provides host and guest idempotency along with
> workflow orchestration at scale, whether on baremetal, local VMs, or
> the cloud.
>
> kdevops supports rolling distributions such as Debian testing, Fedora,
> and openSUSE. Recently we extended Nix support, which raised the
> question: how do we drive Nix's declarative language from Ansible? We
> answered by wiring Nix in under Ansible and its templates, as one more
> way to declare host and guest environments. But that was the wrong
> framing: we had bolted Nix onto today's toolkit instead of rethinking
> it. Reproducibility and idempotency now come from Nix by construction,
> so I think Ansible's original reason for being in kdevops falls away.
> The better question is: how do we keep kdevops's core principles, lean
> on Nix, and drop Ansible?
>
> What remains once you do is not the configuration management plane. It
> is development workflow orchestration: build QEMU, build the kernel,
> build a guest rootfs/closure, boot it, run a test, collect results,
> diff against a baseline. That work is imperative and sequenced, work
> for a workflow engine, which is where tools like Windmill [1] come in.
> Windmill calls itself as "the fastest workflow engine" and an
> "open-source developer platform to power your entire infra and turn
> scripts into webhooks, workflows and UIs." Choosing to move kdevops
> onto Windmill would keep what made kdevops kdevops, namely workflows,
> quick bring-ups, baselines, and A/B regression detection, while trading
> Kconfig, Make, Ansible, and host-distro provisioning for typed
> run-forms, flows as code, and a worker queue. Nix supplies the
> environment, much like a container or venv/poetry, along with the guest
> OS system closure: declarative and portable. Windmill orchestrates the
> whole pipeline end to end, graduating kdevops into a fully reproducible,
> scalable, and configurable kernel-development framework, with both a UI
> and a CLI, that runs locally or in the cloud. Defined as code and driven
> by schedules and triggers, the same flows also make it a continuous
> integration pipeline. Because steps can be written in any language
> Windmill supports, including Ansible, Bash, Go, Python, and Rust,
> developers can not only use kdevops but extend it with their own
> scripts, turning it into a workflow hub. Note that choosing this path
> does not mean NixOS is required on the controller node; Nix is simply a
> runtime dependency that can be installed alongside your distro of
> choice.
>
> It'd be good to know what folks think about the possibility of evolving
> kdevops in this direction, deprecating Ansible along with Kconfig and
> Makefiles in favor of the new approach. To that end, I suggest a demo
> day where I can show why I think this is the next step worth taking, and
> whether it's a tradeoff users and maintainers are willing to make.
>
> If this is of interest and you'd like a look, I've ported equivalents of
> bootlinux (direct boot), qemu-build, and the systemd/QEMU bringup (QSU),
> plus an fstests run for XFS in the proof-of-concept demo project [2].
> You can also find some screenshots in [3].
>
> A note on licensing. Windmill's engine is AGPLv3; its OpenFlow flow
> format and client libraries are Apache-2.0. kdevops-ng runs Windmill
> unmodified and self-hosted as a separate service, and the flows and
> scripts are kdevops-ng's own copyleft-next-0.3.1 code, executed by
> Windmill rather than derived from it, so there shouldn't be any
> licensing concerns.
>
> [1] https://windmill.dev
> [2] https://github.com/dagomez137/kdevops-ng
> [3] https://github.com/dagomez137/kdevops-ng/tree/main/screenshots

Agreed that Ansible is evolving faster than is convenient for us
and kdevops' use of it seems to be laden with technical debt.

I don't object at all to creative forward thinking, and I am not
outright objecting to the proposal, but my concerns are:

+ How much development effort is this? I have nearly zero time to
  work on kdevops these days. Even with AI assistance, this seems
  like an enormous task.

+ Also as a user of kdevops, this means I would have to spend a
  lot of time I don't have learning the new configuration shapes
  and figuring out how to migrate my current CI to them. That's
  not inviting.

+ What would cloud support look like in a post-Ansible world? I
  don't think Nix has impact on the cloud aspect of kdevops, and this
  proposal seems to shift Nix into the role of the primary kdevops
  virtualization method. Should we simply split kdevops into an "all
  local virtualization" project and an "all cloud virtualization"
  project that go their separate ways? Tying cloud and local together
  seems to be a frequent source of chafing.

+ I'd rather see a focus on addressing the technical debt of
  continuing to support distributions that no one uses any more,
  and moving away from requiring root to run local virtualization
  by installing kdevops from a package. Those are narrower work
  items that offer a bigger bang-for-buck. (You might argue that
  Nix solves the latter problem already).

-- 
Chuck Lever

Re: kdevops-ng: graduating kdevops beyond Ansible

[Jeff Layton]

On Thu, 2026-06-18 at 09:22 -0400, Chuck Lever wrote:
> On Thu, Jun 18, 2026, at 5:30 AM, Daniel Gomez wrote:
> > kdevops is a framework for Linux kernel development and test automation.
> > Its core features, namely workflow reproducibility, variability, and
> > scalability, are delivered through Kconfig, the variability language,
> > and Ansible, which provides host and guest idempotency along with
> > workflow orchestration at scale, whether on baremetal, local VMs, or
> > the cloud.
> > 
> > kdevops supports rolling distributions such as Debian testing, Fedora,
> > and openSUSE. Recently we extended Nix support, which raised the
> > question: how do we drive Nix's declarative language from Ansible? We
> > answered by wiring Nix in under Ansible and its templates, as one more
> > way to declare host and guest environments. But that was the wrong
> > framing: we had bolted Nix onto today's toolkit instead of rethinking
> > it. Reproducibility and idempotency now come from Nix by construction,
> > so I think Ansible's original reason for being in kdevops falls away.
> > The better question is: how do we keep kdevops's core principles, lean
> > on Nix, and drop Ansible?
> > 
> > What remains once you do is not the configuration management plane. It
> > is development workflow orchestration: build QEMU, build the kernel,
> > build a guest rootfs/closure, boot it, run a test, collect results,
> > diff against a baseline. That work is imperative and sequenced, work
> > for a workflow engine, which is where tools like Windmill [1] come in.
> > Windmill calls itself as "the fastest workflow engine" and an
> > "open-source developer platform to power your entire infra and turn
> > scripts into webhooks, workflows and UIs." Choosing to move kdevops
> > onto Windmill would keep what made kdevops kdevops, namely workflows,
> > quick bring-ups, baselines, and A/B regression detection, while trading
> > Kconfig, Make, Ansible, and host-distro provisioning for typed
> > run-forms, flows as code, and a worker queue. Nix supplies the
> > environment, much like a container or venv/poetry, along with the guest
> > OS system closure: declarative and portable. Windmill orchestrates the
> > whole pipeline end to end, graduating kdevops into a fully reproducible,
> > scalable, and configurable kernel-development framework, with both a UI
> > and a CLI, that runs locally or in the cloud. Defined as code and driven
> > by schedules and triggers, the same flows also make it a continuous
> > integration pipeline. Because steps can be written in any language
> > Windmill supports, including Ansible, Bash, Go, Python, and Rust,
> > developers can not only use kdevops but extend it with their own
> > scripts, turning it into a workflow hub. Note that choosing this path
> > does not mean NixOS is required on the controller node; Nix is simply a
> > runtime dependency that can be installed alongside your distro of
> > choice.
> > 
> > It'd be good to know what folks think about the possibility of evolving
> > kdevops in this direction, deprecating Ansible along with Kconfig and
> > Makefiles in favor of the new approach. To that end, I suggest a demo
> > day where I can show why I think this is the next step worth taking, and
> > whether it's a tradeoff users and maintainers are willing to make.
> > 
> > If this is of interest and you'd like a look, I've ported equivalents of
> > bootlinux (direct boot), qemu-build, and the systemd/QEMU bringup (QSU),
> > plus an fstests run for XFS in the proof-of-concept demo project [2].
> > You can also find some screenshots in [3].
> > 
> > A note on licensing. Windmill's engine is AGPLv3; its OpenFlow flow
> > format and client libraries are Apache-2.0. kdevops-ng runs Windmill
> > unmodified and self-hosted as a separate service, and the flows and
> > scripts are kdevops-ng's own copyleft-next-0.3.1 code, executed by
> > Windmill rather than derived from it, so there shouldn't be any
> > licensing concerns.
> > 
> > [1] https://windmill.dev
> > [2] https://github.com/dagomez137/kdevops-ng
> > [3] https://github.com/dagomez137/kdevops-ng/tree/main/screenshots
> 
> Agreed that Ansible is evolving faster than is convenient for us
> and kdevops' use of it seems to be laden with technical debt.
> 
> I don't object at all to creative forward thinking, and I am not
> outright objecting to the proposal, but my concerns are:
> 
> + How much development effort is this? I have nearly zero time to
>   work on kdevops these days. Even with AI assistance, this seems
>   like an enormous task.
> 

Ditto. I'm mostly a user of kdevops these days and don't have as much
time to work on it.

> + Also as a user of kdevops, this means I would have to spend a
>   lot of time I don't have learning the new configuration shapes
>   and figuring out how to migrate my current CI to them. That's
>   not inviting.
> 
> + What would cloud support look like in a post-Ansible world? I
>   don't think Nix has impact on the cloud aspect of kdevops, and this
>   proposal seems to shift Nix into the role of the primary kdevops
>   virtualization method. Should we simply split kdevops into an "all
>   local virtualization" project and an "all cloud virtualization"
>   project that go their separate ways? Tying cloud and local together
>   seems to be a frequent source of chafing.
> 

It does.

OTOH, some parts of kdevops (the test running bits, in particular) are
mostly agnostic to the backend where it's being run. I don't think we
want to have to replicate that part.

Maybe we need to split the whole thing into two -- one part that does
the bringup and setup, and one part that does the test running?

> + I'd rather see a focus on addressing the technical debt of
>   continuing to support distributions that no one uses any more,
>   and moving away from requiring root to run local virtualization
>   by installing kdevops from a package. Those are narrower work
>   items that offer a bigger bang-for-buck. (You might argue that
>   Nix solves the latter problem already).

I already don't run as root for local virtualization. I thought
everyone did? Some of the initial setup requires root, but that's sort
of a separate problem.
-- 
Jeff Layton <jlayton@kernel.org>

Re: kdevops-ng: graduating kdevops beyond Ansible

[Daniel Gomez]

On 18/06/2026 15.22, Chuck Lever wrote:
> 
> On Thu, Jun 18, 2026, at 5:30 AM, Daniel Gomez wrote:
> 
> + How much development effort is this? I have nearly zero time to
>   work on kdevops these days. Even with AI assistance, this seems
>   like an enormous task.

You're right that it's large, but not all of it needs to migrate. I
wouldn't move libvirt or libguestfs unless people actually want it; the
main thing that path buys you is the auto-generated UI [1].

Workflows could be migrated as-is, since Ansible is one of the languages
Windmill runs, but I think most of them are entangled with libvirt and
9p. If we're not migrating libvirt/libguestfs, porting the workflows
verbatim doesn't make sense either; the effort is better spent rewriting
them in whichever Windmill language fits (Bash, Python, Rust, ...). The
ones that are already decoupled should convert fairly straightforwardly.

One thing I should have made clearer in the cover: I lean on systemd as
the process manager, not just for QSU (QEMU in systemd), but also to run
the test suites themselves, e.g. the fstests check script as a systemd
unit. That keeps the "scripts" simple while still being powerful.

I do think the rewrite into Nix/Windmill (Bash/Python/Rust/whatever)
plus systemd is worth it. Until then, both worlds can coexist.

Link: https://www.windmill.dev/docs/script_editor/customize_ui [1]

> 
> + Also as a user of kdevops, this means I would have to spend a
>   lot of time I don't have learning the new configuration shapes
>   and figuring out how to migrate my current CI to them. That's
>   not inviting.

Agreed, the learning curve is real. The upside is that migrating also
gives you CI essentially for free: the same flows, run on Windmill's
workers and schedules, are the CI.

> 
> + What would cloud support look like in a post-Ansible world? I>   don't think Nix has impact on the cloud aspect of kdevops, and this

Correct.


>   proposal seems to shift Nix into the role of the primary kdevops
>   virtualization method. 

Correct.

> Should we simply split kdevops into an "all
>   local virtualization" project and an "all cloud virtualization"
>   project that go their separate ways? Tying cloud and local together
>   seems to be a frequent source of chafing.

Good question. The same tension arguably applies to the Nix/QSU world
versus the Debian/Fedora/openSUSE + libvirt/libguestfs world. But I
don't think kdevops is really about which tools you use underneath; it's
a broader framework and idea, the same principles applied as a
kernel-development framework whatever the technology beneath. So rather
than splitting by tooling, I'd keep the framework and let the backends
differ.

> 
> + I'd rather see a focus on addressing the technical debt of
>   continuing to support distributions that no one uses any more,
>   and moving away from requiring root to run local virtualization
>   by installing kdevops from a package. 

That's exactly why QSU was merged, and I'm very happy with the
transition. Guests now run as systemd user services with virtiofsd
instead of 9p, with all the QEMU knobs I need and KVM acceleration, and
no libvirt permissions or qemu:///session vs qemu:///system headaches.

That said, fully unwiring the controller setup and the sudo escalations
is tedious, significant work for relatively little reward. I do reach
this paths even with QSU.


> Those are narrower work
>   items that offer a bigger bang-for-buck. (You might argue that
>   Nix solves the latter problem already).

Roughly, yes: Nix solves the reproducibility part, on both the
controller and guest sides, and QSU sidesteps the libvirt permission and
SELinux/apparmor sudo configuration steps.

BTW, on the cloud side with Nix I came across terranix [2], which lets
you describe Terraform/Opentofu infrastructure in the Nix language. And
either way, once a cloud guest is up you can just install Nix on it and
run the same paths as in local virtualization, so the workflow layer is
identical regardless of which deployment users choose. Only the bringup
differs.

Link: https://terranix.org/docs/what-is-terranix/ [2]