public inbox for kdevops@lists.linux.dev
 help / color / mirror / Atom feed
* [PATCH v4 0/2] Lucid: A custom Ansible stdout callback for kdevops
@ 2025-12-01 23:11 Daniel Gomez
  2025-12-01 23:11 ` [PATCH v4 1/2] ansible: use native verbosity config instead of custom AV variable Daniel Gomez
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Daniel Gomez @ 2025-12-01 23:11 UTC (permalink / raw)
  To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez

The Ansible stdout callback ecosystem is quite wide, supporting multiple
output formats and options. However, after testing a variety of them
with kdevops workflows, none fit well for our use cases. Most existing
callbacks provide verbose output that makes it difficult for users to
understand what's actually happening during playbook executions.

kdevops relies on Ansible's idempotency principle [1], meaning playbooks
are designed to reach the desired state regardless of the current state.
Users shouldn't need to worry about which tasks are running or being
skipped, the output should simply guide them to the expected result.
With this principle in mind and inspired by other projects [2] like
bitbake (from OpenEmbedded/Yocto), we can focus the default output
on what actually matters: the currently running task, a short list of
recent tasks, and elapsed time for the current operation.

This is the core principle behind lucid, a new Ansible stdout callback
plugin tailored for kdevops where less is more. Errors are always
displayed in human-readable format on stdout, while comprehensive logs
remain available for detailed inspection of playbook execution.

Lucid is designed to be flexible and evolve with community feedback.
Users who prefer different output styles can easily configure the plugin
through  kdevops' Kconfig system. For example, if the default dynamic
mode with live updates doesn't suit a particular workflow, users can
switch to static scrollable output. The goal is to provide sensible
defaults that work for most kernel and kdevops testing scenarios while
allowing individual customization when needed.

Here's a linux workflow output demo using lucid. The video is long, but
the first few seconds already give you a sense of how the plugin works.
Jump to 3:36 to see the output logs. The video will be deleted after
7 days:

https://asciinema.org/a/fCqZ8S8F0vNBfjkA8FLDGUue5

[1] https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_intro.html#desired-state-and-idempotency
[2] https://docs.yoctoproject.org/bitbake/

Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
Changes in v4:
- Replace custom AV makefile variable with Ansible's native verbosity
setting in ansible.cfg, controlled via Kconfig, with ANSIBLE_VERBOSITY=N
environment variable for runtime override
- Link to v3: https://lore.kernel.org/r/20251201-lucid-v3-0-78b205691be0@samsung.com

Changes in v3:
- Fix dynamic display flickering by preloading play hosts at task start
- Add delegation info display (host -> delegate) matching Ansible default
- Use fixed-width columns to prevent display jumping
- Always show spinner for running tasks
- Add longer time string support to duration formatter
- Update documentation for dynamic mode and delegation display
- New demo (linux-modules-kpd): https://asciinema.org/a/759342
- Link to v2: https://lore.kernel.org/r/20251201-lucid-v2-0-f3c924fb5586@samsung.com

Changes in v2:
- Drop lucid stdout callback as default (let's discuss this separately)
- Add retry support when tasks are displayed
- Fix for small terminal width in dynamic mode that prevented tasks to
be cleared properly
- Change task status symbols to be width consistent
- Drop "unused" lucid options and simplify Kconfig. It's unclear the value
of them so, let's add them when needed.
- Add a new Ansible config to configure Ansible verbosity levels via Kconfig
- Add lucid-ci.config fragment that leverages the new kconfig Ansible
verbosity configuration to configure lucid in static mode with the
required verbosity needed in CI scenarios.
- Other minor plugin fixes
- Link to v1: https://lore.kernel.org/r/20251121-lucid-v1-1-4981e36f71f4@samsung.com

---
Daniel Gomez (2):
      ansible: use native verbosity config instead of custom AV variable
      ansible: add lucid callback plugin for clean output

 .gitignore                                         |   4 +
 CLAUDE.md                                          |  16 +-
 Makefile                                           |   6 +-
 Makefile.btrfs_progs                               |   2 +-
 Makefile.build_qemu                                |   8 +-
 Makefile.docker-mirror                             |   4 +-
 Makefile.hypervisor-tunings                        |   2 +-
 Makefile.kdevops                                   |  10 +-
 Makefile.postfix                                   |   2 +-
 callback_plugins/__init__.py                       |   0
 callback_plugins/lucid.py                          | 879 +++++++++++++++++++++
 defconfigs/configs/lucid-ci.config                 |   3 +
 defconfigs/configs/lucid.config                    |   2 +
 docs/ansible-callbacks.md                          | 294 +++++++
 docs/fio-tests.md                                  |   2 +-
 docs/monitoring.md                                 |   2 +-
 docs/reboot-limit.md                               |   2 +-
 kconfigs/Kconfig.ansible_cfg                       |  94 +++
 .../roles/ansible_cfg/templates/ansible.cfg.j2     |  13 +-
 scripts/ansible.Makefile                           |   3 -
 scripts/archive.Makefile                           |   2 +-
 scripts/devconfig.Makefile                         |   4 +-
 scripts/dynamic-pci-kconfig.Makefile               |   2 +-
 scripts/firstconfig.Makefile                       |   2 +-
 scripts/guestfs.Makefile                           |  18 +-
 scripts/install-menuconfig-deps.Makefile           |   2 +-
 scripts/install-rcloud-deps.Makefile               |   2 +-
 scripts/iscsi.Makefile                             |   2 +-
 scripts/journal-server.Makefile                    |  10 +-
 scripts/kconfig/Makefile                           |   2 +-
 scripts/kotd.Makefile                              |   6 +-
 scripts/krb5.Makefile                              |   4 +-
 scripts/ktls.Makefile                              |   2 +-
 scripts/nfsd.Makefile                              |   2 +-
 scripts/nixos.Makefile                             |  10 +-
 scripts/provision.Makefile                         |   2 +-
 scripts/rdma.Makefile                              |   4 +-
 scripts/smbd.Makefile                              |   2 +-
 scripts/systemd-timesync.Makefile                  |   4 +-
 scripts/terraform.Makefile                         |  14 +-
 scripts/update_etc_hosts.Makefile                  |   2 +-
 scripts/validate_av.py                             |  28 -
 workflows/ai/Makefile                              |  20 +-
 workflows/blktests/Makefile                        |  16 +-
 workflows/build-linux/Makefile                     |   8 +-
 workflows/common/Makefile                          |   2 +-
 workflows/cxl/Makefile                             |  14 +-
 workflows/demos/reboot-limit/Makefile              |  14 +-
 workflows/fio-tests/Makefile                       |  12 +-
 workflows/fstests/Makefile                         |  28 +-
 workflows/gitr/Makefile                            |   8 +-
 workflows/linux/Makefile                           |  28 +-
 workflows/ltp/Makefile                             |   8 +-
 workflows/minio/Makefile                           |  10 +-
 workflows/mmtests/Makefile                         |  16 +-
 workflows/nfstest/Makefile                         |   8 +-
 workflows/pynfs/Makefile                           |  12 +-
 workflows/rcloud/Makefile                          |   4 +-
 workflows/selftests/Makefile                       |   8 +-
 workflows/steady_state/Makefile                    |   8 +-
 workflows/sysbench/Makefile                        |  16 +-
 workflows/vllm/Makefile                            |  20 +-
 62 files changed, 1500 insertions(+), 234 deletions(-)
---
base-commit: c7daa8cf23b3c7bfb8579b161f71c7520a1d088f
change-id: 20251121-lucid-adeb42a71d42

Best regards,
--  
Daniel Gomez <da.gomez@samsung.com>


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2025-12-03 21:11 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-12-01 23:11 [PATCH v4 0/2] Lucid: A custom Ansible stdout callback for kdevops Daniel Gomez
2025-12-01 23:11 ` [PATCH v4 1/2] ansible: use native verbosity config instead of custom AV variable Daniel Gomez
2025-12-03 20:26   ` Chuck Lever
2025-12-01 23:11 ` [PATCH v4 2/2] ansible: add lucid callback plugin for clean output Daniel Gomez
2025-12-03 21:11   ` Chuck Lever
2025-12-01 23:14 ` [PATCH v4 0/2] Lucid: A custom Ansible stdout callback for kdevops Daniel Gomez

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox