public inbox for kdevops@lists.linux.dev
 help / color / mirror / Atom feed
* [PATCH 0/5] kdevops: embrace kconfig selective yaml output
@ 2024-09-04 23:30 Luis Chamberlain
  2024-09-04 23:30 ` [PATCH 1/5] Makefile: add support for selective yamlconfig Luis Chamberlain
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: Luis Chamberlain @ 2024-09-04 23:30 UTC (permalink / raw)
  To: kdevops, chuck.lever, da.gomez, jlayton; +Cc: mcgrof

13 days ago I posted an RFC for Linux upstream to support selective yaml
output on kconfig [0]. What this does is it lets you annnotate "output yaml"
on kconfig symbols and if you do that you also get an additional yaml output of
only those symbols. Long ago I theorized we could do something like this so to
avoid having to do Makefile edits like this to propagate variables to ansible:

ifeq (y,$(CONFIG_GUESTFS_REQUIRES_UEFI))
GUESTFS_ARGS += guestfs_requires_uefi=True
endif  

Alternatives to this are to do full config postprocessing and converting all
symbols to yaml entries, or even having an inclusion file to determine which
variables should be converted.

The kconfig selective yaml output support streamlines the need by easily
associating which symbols you want to be converted over to yaml directly
on the kconfig file itself:

 config GUESTFS_REQUIRES_UEFI
        bool
+       output yaml 

Since there are no upstream users of this in the kernel it is obvious that this
likely wouldn't have been desirable to be maintained in the kernel, and Masahiro
Yamada confirmed so. So this is not upstreamable to Linux. However, we can just
carry the delta forward as its rather small. What this would mean for us since
we are using a git subtree for kconfig is that we embrace a yamlconfig branch
on kconfig and keep that with the small delta. It can sync with the master
branch when new updates for new linux-next tags are updated.

I believe that considering the alternatives, carrying the delta forward on a
branch on kconfig for this is a much more straight forward approach but I would
like to hear other's opinions on this now that we have the work done and a proof
of concept completed. This approach lets us clearly annotate the inclusion
process of conversion of kconfig symbols to ansible variables, while also
allowing the extra_vars.yaml file to remain slim. This makes a deterministic
association between kconfig symbol and ansible variables names clear. The
value in this we remove ambiguation between kconfig symbol and ansible
variables, while also allowing us to keep our Makefiles for workflows pretty
small, removing the complexities in variable conversion.

The work left is then:

 a) We need to see how a delta in our kconfig could be maintained
 b) What the code changes for actual selective yaml output would look like
 c) What the cleanups actually look like as a demonstration

For a) it's real simple: we keep a yamlconfig branch  on the kconfig tree [1].
Likewise a demonstration project which uses this would be needed, for that I
have created the yamlconfig branch on init-kconfig [2]. The previous patches
lets us upgrade the kconfig use din kdevops up to the kconfig as of linux-next
tag next-20240830. Now to merge the changes required for yamlconfig all we need
to do is run:

git subtree --squash pull --prefix=scripts/kconfig/  kconfig yamlconfig

The patches in this series then make use of this optional feature and
demonstrate how it fits into kdevops. The last patch is really what I was
hoping for to demonstrate. The third patch in this series titled
"scripts/ssh.Makefile: remove Makefile appending of KDEVOPS_HOSTS_PREFIX"
demonstrates how easy it is for us to loose ourselves with variables and
havne't even realized it. It implicates a long term cleanup we should strive
to where essentially we won't have to expect users of the old variables so
we remove the pesky old variables.

What this does is, it let's us *selectively* move forward with the conversion
proceess workflow by workflow, while allowing *new* workflows to come in
squeeky clean from the start because proper determinism on variable names
would be used from the start.

The real meat and bones and rationale for this effort is demonstrated on
the last patch. Thoughts?

If you want to take this for a spin:

https://github.com/mcgrof/kdevops/tree/yamlconfig

PS. nconfig doesn't yet have yamlconfig support, I can add it easily
if this approach is agreeable.

[0] https://lkml.kernel.org/r/20240823034455.3593819-1-mcgrof@kernel.org
[1] https://github.com/linux-kdevops/kconfig/tree/yamlconfig
[2] https://github.com/linux-kdevops/init-kconfig/tree/yamlconfig

Luis Chamberlain (5):
  Makefile: add support for selective yamlconfig
  scripts/guestfs.Makefile: use output yaml for GUESTFS_REQUIRES_UEFI
  scripts/ssh.Makefile: remove Makefile appending of
    KDEVOPS_HOSTS_PREFIX
  workflows/Makefile: replace direct named entries with output yaml
  fstests: use selective yaml output on a slew of different variables

 Makefile                   |  3 ++-
 Makefile.extra_vars        | 11 +++++----
 kconfigs/Kconfig.guestfs   |  1 +
 kconfigs/Kconfig.kdevops   |  7 ++++++
 kconfigs/workflows/Kconfig |  8 +++++++
 scripts/gen-hosts.Makefile |  1 -
 scripts/guestfs.Makefile   |  4 ----
 scripts/ssh.Makefile       |  3 ---
 workflows/Makefile         |  8 -------
 workflows/fstests/Kconfig  | 11 +++++++++
 workflows/fstests/Makefile | 46 --------------------------------------
 11 files changed, 34 insertions(+), 69 deletions(-)

-- 
2.43.0


^ permalink raw reply	[flat|nested] 7+ messages in thread
end of thread, other threads:[~2024-09-05  8:03 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-09-04 23:30 [PATCH 0/5] kdevops: embrace kconfig selective yaml output Luis Chamberlain
2024-09-04 23:30 ` [PATCH 1/5] Makefile: add support for selective yamlconfig Luis Chamberlain
2024-09-04 23:30 ` [PATCH 2/5] scripts/guestfs.Makefile: use output yaml for GUESTFS_REQUIRES_UEFI Luis Chamberlain
2024-09-04 23:30 ` [PATCH 3/5] scripts/ssh.Makefile: remove Makefile appending of KDEVOPS_HOSTS_PREFIX Luis Chamberlain
2024-09-04 23:30 ` [PATCH 4/5] workflows/Makefile: replace direct named entries with output yaml Luis Chamberlain
2024-09-04 23:30 ` [PATCH 5/5] fstests: use selective yaml output on a slew of different variables Luis Chamberlain
2024-09-05  8:03 ` [PATCH 0/5] kdevops: embrace kconfig selective yaml output Luis Chamberlain

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox