[PATCH net-next 0/4] tools: ynl: fixes for the page-pool sample and the generation process

[PATCH net-next 0/4] tools: ynl: fixes for the page-pool sample and the generation process

[Jakub Kicinski]

Minor fixes to the new sample and the Makefiles.

Jakub Kicinski (4):
  tools: ynl: fix build of the page-pool sample
  tools: ynl: make sure we use local headers for page-pool
  tools: ynl: order building samples after generated code
  tools: ynl: don't skip regeneration from make targets

 tools/net/ynl/Makefile            |  2 ++
 tools/net/ynl/samples/Makefile    |  2 ++
 tools/net/ynl/samples/page-pool.c |  2 +-
 tools/net/ynl/ynl-gen-c.py        | 12 ++++++++----
 tools/net/ynl/ynl-regen.sh        |  4 ++--
 5 files changed, 15 insertions(+), 7 deletions(-)

-- 
2.43.0

[PATCH net-next 1/4] tools: ynl: fix build of the page-pool sample

[Jakub Kicinski]

The name of the "destroyed" field in the reply was not changed
in the sample after we started calling it "detach_time".

page-pool.c: In function ‘main’:
page-pool.c:84:33: error: ‘struct <anonymous>’ has no member named ‘destroyed’
   84 |                 if (pp->_present.destroyed)
      |                                 ^

Fixes: 637567e4a3ef ("tools: ynl: add sample for getting page-pool information")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
CC: hawk@kernel.org
---
 tools/net/ynl/samples/page-pool.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/net/ynl/samples/page-pool.c b/tools/net/ynl/samples/page-pool.c
index 18d359713469..098b5190d0e5 100644
--- a/tools/net/ynl/samples/page-pool.c
+++ b/tools/net/ynl/samples/page-pool.c
@@ -81,7 +81,7 @@ int main(int argc, char **argv)
 		struct stat *s = find_ifc(&a, pp->ifindex);
 
 		count(s, 1, pp);
-		if (pp->_present.destroyed)
+		if (pp->_present.detach_time)
 			count(s, 0, pp);
 	}
 	netdev_page_pool_get_list_free(pools);
-- 
2.43.0

[PATCH net-next 2/4] tools: ynl: make sure we use local headers for page-pool

[Jakub Kicinski]

Building samples generates the following warning:

  In file included from page-pool.c:11:
  generated/netdev-user.h:21:45: warning: ‘enum netdev_xdp_rx_metadata’ declared inside parameter list will not be visible outside of this definition or declaration
   21 | const char *netdev_xdp_rx_metadata_str(enum netdev_xdp_rx_metadata value);
      |                                             ^~~~~~~~~~~~~~~~~~~~~~

Our magic way of including uAPI headers assumes the sample
name matches the family name. We need to copy the flags over.

Fixes: 637567e4a3ef ("tools: ynl: add sample for getting page-pool information")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
CC: hawk@kernel.org
CC: john.fastabend@gmail.com
CC: willemb@google.com
CC: sdf@google.com
---
 tools/net/ynl/samples/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/net/ynl/samples/Makefile b/tools/net/ynl/samples/Makefile
index 1afefc266b7a..28bdb1557a54 100644
--- a/tools/net/ynl/samples/Makefile
+++ b/tools/net/ynl/samples/Makefile
@@ -18,6 +18,8 @@ include $(wildcard *.d)
 
 all: $(BINS)
 
+CFLAGS_page-pool=$(CFLAGS_netdev)
+
 $(BINS): ../lib/ynl.a ../generated/protos.a $(SRCS)
 	@echo -e '\tCC sample $@'
 	@$(COMPILE.c) $(CFLAGS_$@) $@.c -o $@.o
-- 
2.43.0

[PATCH net-next 3/4] tools: ynl: order building samples after generated code

[Jakub Kicinski]

Parallel builds of ynl:

  make -C tools/net/ynl/ -j 4

don't work correctly right now. samples get handled before
generated, so build of samples does not notice that protos.a
has changed. Order samples to be last.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
CC: sdf@google.com
CC: willemb@google.com
---
 tools/net/ynl/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/net/ynl/Makefile b/tools/net/ynl/Makefile
index d664b36deb5b..da1aa10bbcc3 100644
--- a/tools/net/ynl/Makefile
+++ b/tools/net/ynl/Makefile
@@ -4,6 +4,8 @@ SUBDIRS = lib generated samples
 
 all: $(SUBDIRS)
 
+samples: | lib generated
+
 $(SUBDIRS):
 	@if [ -f "$@/Makefile" ] ; then \
 		$(MAKE) -C $@ ; \
-- 
2.43.0

[PATCH net-next 4/4] tools: ynl: don't skip regeneration from make targets

[Jakub Kicinski]

Commit 2b7ac0c87d98 ("tools: ynl-gen: don't touch the output file if
content is the same") is working too well. It was added so that
ynl-regen -f doesn't make us rebuild half of the kernel, if there
are no actual changes in any generated code.

When ynl-gen-c is called by make, however, we're better off trusting
make's tracking and overwrite the file. Otherwise if output is identical
we won't update file timestamps and make will retry code gen on every
invocation.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
CC: sdf@google.com
---
 tools/net/ynl/ynl-gen-c.py | 12 ++++++++----
 tools/net/ynl/ynl-regen.sh |  4 ++--
 2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/tools/net/ynl/ynl-gen-c.py b/tools/net/ynl/ynl-gen-c.py
index cbbda276f6d1..ba1c3611c6fa 100755
--- a/tools/net/ynl/ynl-gen-c.py
+++ b/tools/net/ynl/ynl-gen-c.py
@@ -1164,8 +1164,9 @@ from lib import SpecFamily, SpecAttrSet, SpecAttr, SpecOperation, SpecEnumSet, S
 
 
 class CodeWriter:
-    def __init__(self, nlib, out_file=None):
+    def __init__(self, nlib, out_file=None, overwrite=True):
         self.nlib = nlib
+        self._overwrite = overwrite
 
         self._nl = False
         self._block_end = False
@@ -1186,8 +1187,9 @@ from lib import SpecFamily, SpecAttrSet, SpecAttr, SpecOperation, SpecEnumSet, S
             return
         # Avoid modifying the file if contents didn't change
         self._out.flush()
-        if os.path.isfile(self._out_file) and filecmp.cmp(self._out.name, self._out_file, shallow=False):
-            return
+        if not self._overwrite and os.path.isfile(self._out_file):
+            if filecmp.cmp(self._out.name, self._out_file, shallow=False):
+                return
         with open(self._out_file, 'w+') as out_file:
             self._out.seek(0)
             shutil.copyfileobj(self._out, out_file)
@@ -2516,6 +2518,8 @@ _C_KW = {
     parser.add_argument('--header', dest='header', action='store_true', default=None)
     parser.add_argument('--source', dest='header', action='store_false')
     parser.add_argument('--user-header', nargs='+', default=[])
+    parser.add_argument('--cmp-out', action='store_true', default=None,
+                        help='Do not overwrite the output file if the new output is identical to the old')
     parser.add_argument('--exclude-op', action='append', default=[])
     parser.add_argument('-o', dest='out_file', type=str, default=None)
     args = parser.parse_args()
@@ -2543,7 +2547,7 @@ _C_KW = {
         print(f'Message enum-model {parsed.msg_id_model} not supported for {args.mode} generation')
         os.sys.exit(1)
 
-    cw = CodeWriter(BaseNlLib(), args.out_file)
+    cw = CodeWriter(BaseNlLib(), args.out_file, overwrite=(not args.cmp_out))
 
     _, spec_kernel = find_kernel_root(args.spec)
     if args.mode == 'uapi' or args.header:
diff --git a/tools/net/ynl/ynl-regen.sh b/tools/net/ynl/ynl-regen.sh
index bdba24066cf1..a37304dcc88e 100755
--- a/tools/net/ynl/ynl-regen.sh
+++ b/tools/net/ynl/ynl-regen.sh
@@ -30,8 +30,8 @@ for f in $files; do
     fi
 
     echo -e "\tGEN ${params[2]}\t$f"
-    $TOOL --mode ${params[2]} --${params[3]} --spec $KDIR/${params[0]} \
-	  $args -o $f
+    $TOOL --cmp-out --mode ${params[2]} --${params[3]} \
+	  --spec $KDIR/${params[0]} $args -o $f
 done
 
 popd >>/dev/null
-- 
2.43.0

Re: [PATCH net-next 0/4] tools: ynl: fixes for the page-pool sample and the generation process

[Jakub Kicinski]

On Wed, 29 Nov 2023 11:36:18 -0800 Jakub Kicinski wrote:
> Minor fixes to the new sample and the Makefiles.

I'll apply this already, sorry for the rush.
I have stupidly activated a test in patchwork to catch breaking
the build again, without waiting for the fix to land.

Re: [PATCH net-next 0/4] tools: ynl: fixes for the page-pool sample and the generation process

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Wed, 29 Nov 2023 11:36:18 -0800 you wrote:
> Minor fixes to the new sample and the Makefiles.
> 
> Jakub Kicinski (4):
>   tools: ynl: fix build of the page-pool sample
>   tools: ynl: make sure we use local headers for page-pool
>   tools: ynl: order building samples after generated code
>   tools: ynl: don't skip regeneration from make targets
> 
> [...]

Here is the summary with links:
  - [net-next,1/4] tools: ynl: fix build of the page-pool sample
    https://git.kernel.org/netdev/net-next/c/ee1eb9de81db
  - [net-next,2/4] tools: ynl: make sure we use local headers for page-pool
    https://git.kernel.org/netdev/net-next/c/929003723f6d
  - [net-next,3/4] tools: ynl: order building samples after generated code
    https://git.kernel.org/netdev/net-next/c/9cf9b5708241
  - [net-next,4/4] tools: ynl: don't skip regeneration from make targets
    https://git.kernel.org/netdev/net-next/c/a115b9279f48

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html