From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0751C1779A5 for ; Wed, 4 Sep 2024 23:30:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725492643; cv=none; b=gdnhCUQYBImfjETGk3dE0A3gRBDkVBlz+KG6IWX1tooU8MbSIsvNoDc2hU3WS8Gne7Amp9Z5xpmf/2FviZPPCXhLhCtfc59TxKWKNzr+ZAljYVYrLuusi3PleoCa1Ze96PWRSpg/58NjfWOGF7RKVZS7nttcoZ86VCGfoulyTio= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725492643; c=relaxed/simple; bh=iKKMAingHL0g1ZJYTFIZ94woE5v5FTz6rJPVetfxNCo=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=TZGRisaQXUMKGziIGsQVi9z2Uc4B+zb94j3BCZoRIK0terbYT2Ixsrop4mqhm2X4WHV7vnOzs+Vz05ovX3VUBsyh2hMcNSTrJptQRMUUdFIXMP6XHVmKacOgSDMLep/FyfsaPUXn1XP7Fss6pSyXNTfnP48hhPOcfpkDJPE4e9g= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=Ox9mHZHo; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Ox9mHZHo" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=O8VxwM9Shx7N4U1z2bfbv+2aQawsZn5E/Cp6nYqEQbs=; b=Ox9mHZHoXkdYJV/qahZAibpKKY HpCh0pa71/OEkSGVgD/psZ8yoLC8MKwWWUIeLdWYaeYD7lCGIUvpI81ESSIk1Y75uisQZxvRdP9rw 8f2+QfAajz7MZlzhYiRnDdD8A+g5QkfhjWwWgB0FMYo6soJlSTdoE0OiUIIGKeSsuDn1q87aTkBa8 xSb0Yijgiq27Uvo8TzwxmCOESRYd/QwIYoVIFJs0PXHb27zMWovPxStQcctlatKKo8PGbCZy2Noo2 SSBvnAcxX9yipphyGVNxR9S3O0jTOnnv3/r6CclMqVZq1LetsocYEupFYJsrrCXAZaVFdoHmKHehn 5NrNL2xA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1slzSS-00000006MRp-3zrP; Wed, 04 Sep 2024 23:30:40 +0000 From: Luis Chamberlain To: kdevops@lists.linux.dev, chuck.lever@oracle.com, da.gomez@samsung.com, jlayton@kernel.org Cc: mcgrof@kernel.org Subject: [PATCH 0/5] kdevops: embrace kconfig selective yaml output Date: Wed, 4 Sep 2024 16:30:35 -0700 Message-ID: <20240904233040.1516250-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.44.0 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain 13 days ago I posted an RFC for Linux upstream to support selective yaml output on kconfig [0]. What this does is it lets you annnotate "output yaml" on kconfig symbols and if you do that you also get an additional yaml output of only those symbols. Long ago I theorized we could do something like this so to avoid having to do Makefile edits like this to propagate variables to ansible: ifeq (y,$(CONFIG_GUESTFS_REQUIRES_UEFI)) GUESTFS_ARGS += guestfs_requires_uefi=True endif Alternatives to this are to do full config postprocessing and converting all symbols to yaml entries, or even having an inclusion file to determine which variables should be converted. The kconfig selective yaml output support streamlines the need by easily associating which symbols you want to be converted over to yaml directly on the kconfig file itself: config GUESTFS_REQUIRES_UEFI bool + output yaml Since there are no upstream users of this in the kernel it is obvious that this likely wouldn't have been desirable to be maintained in the kernel, and Masahiro Yamada confirmed so. So this is not upstreamable to Linux. However, we can just carry the delta forward as its rather small. What this would mean for us since we are using a git subtree for kconfig is that we embrace a yamlconfig branch on kconfig and keep that with the small delta. It can sync with the master branch when new updates for new linux-next tags are updated. I believe that considering the alternatives, carrying the delta forward on a branch on kconfig for this is a much more straight forward approach but I would like to hear other's opinions on this now that we have the work done and a proof of concept completed. This approach lets us clearly annotate the inclusion process of conversion of kconfig symbols to ansible variables, while also allowing the extra_vars.yaml file to remain slim. This makes a deterministic association between kconfig symbol and ansible variables names clear. The value in this we remove ambiguation between kconfig symbol and ansible variables, while also allowing us to keep our Makefiles for workflows pretty small, removing the complexities in variable conversion. The work left is then: a) We need to see how a delta in our kconfig could be maintained b) What the code changes for actual selective yaml output would look like c) What the cleanups actually look like as a demonstration For a) it's real simple: we keep a yamlconfig branch on the kconfig tree [1]. Likewise a demonstration project which uses this would be needed, for that I have created the yamlconfig branch on init-kconfig [2]. The previous patches lets us upgrade the kconfig use din kdevops up to the kconfig as of linux-next tag next-20240830. Now to merge the changes required for yamlconfig all we need to do is run: git subtree --squash pull --prefix=scripts/kconfig/ kconfig yamlconfig The patches in this series then make use of this optional feature and demonstrate how it fits into kdevops. The last patch is really what I was hoping for to demonstrate. The third patch in this series titled "scripts/ssh.Makefile: remove Makefile appending of KDEVOPS_HOSTS_PREFIX" demonstrates how easy it is for us to loose ourselves with variables and havne't even realized it. It implicates a long term cleanup we should strive to where essentially we won't have to expect users of the old variables so we remove the pesky old variables. What this does is, it let's us *selectively* move forward with the conversion proceess workflow by workflow, while allowing *new* workflows to come in squeeky clean from the start because proper determinism on variable names would be used from the start. The real meat and bones and rationale for this effort is demonstrated on the last patch. Thoughts? If you want to take this for a spin: https://github.com/mcgrof/kdevops/tree/yamlconfig PS. nconfig doesn't yet have yamlconfig support, I can add it easily if this approach is agreeable. [0] https://lkml.kernel.org/r/20240823034455.3593819-1-mcgrof@kernel.org [1] https://github.com/linux-kdevops/kconfig/tree/yamlconfig [2] https://github.com/linux-kdevops/init-kconfig/tree/yamlconfig Luis Chamberlain (5): Makefile: add support for selective yamlconfig scripts/guestfs.Makefile: use output yaml for GUESTFS_REQUIRES_UEFI scripts/ssh.Makefile: remove Makefile appending of KDEVOPS_HOSTS_PREFIX workflows/Makefile: replace direct named entries with output yaml fstests: use selective yaml output on a slew of different variables Makefile | 3 ++- Makefile.extra_vars | 11 +++++---- kconfigs/Kconfig.guestfs | 1 + kconfigs/Kconfig.kdevops | 7 ++++++ kconfigs/workflows/Kconfig | 8 +++++++ scripts/gen-hosts.Makefile | 1 - scripts/guestfs.Makefile | 4 ---- scripts/ssh.Makefile | 3 --- workflows/Makefile | 8 ------- workflows/fstests/Kconfig | 11 +++++++++ workflows/fstests/Makefile | 46 -------------------------------------- 11 files changed, 34 insertions(+), 69 deletions(-) -- 2.43.0