From: Jeff Layton <jlayton@kernel.org>
To: cel@kernel.org, linux-nfs@vger.kernel.org
Cc: Chuck Lever <chuck.lever@oracle.com>
Subject: Re: [RFC PATCH 0/2] xdrgen - machine-generated XDR functions
Date: Wed, 21 Aug 2024 10:03:58 -0400 [thread overview]
Message-ID: <d78c9d50dcdefb13c96f1fac6f708781aad0bfe3.camel@kernel.org> (raw)
In-Reply-To: <20240820144600.189744-1-cel@kernel.org>
On Tue, 2024-08-20 at 10:45 -0400, cel@kernel.org wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
>
> For a description of this work-in-progress, see the new README file.
> For unfinished work items, ditto.
>
> The first patch adds the tool.
>
> The second patch is an nfs4_1.x file that I created to confirm that
> the tool indeed works for Jeff's purposes. Actual generated header
> and source are in fs/nfsd/nfs4xdr_gen.? .
>
> Feel free to ignore 2/2. It is included here only to demonstrate the
> tool's new "pragma" directives that I think can avoid the need to
> hand-edit generated source, as discussed in an earlier thread. Grep
> the new README or nfs4_1.x for "pragma".
>
> These patches apply to v6.11-rc4, but can be rebased on nfsd-next if
> they are found merge-worthy. See also:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/cel/linux.git/log/?h=lkxdrgen
>
> For review, note that the numerous .j2 files are all tiny and can be
> skimmed in favor of the .py files, where the tool's logic is.
>
> All Python-related advice and guidance is gratefully accepted.
>
> --
>
> Chuck Lever (2):
> tools: Add xdrgen
> NFSD: Create an initial nfs4_1.x file
>
> fs/nfsd/nfs4_1.x | 164 +++++++
> fs/nfsd/nfs4xdr_gen.c | 236 ++++++++++
> fs/nfsd/nfs4xdr_gen.h | 113 +++++
> include/linux/sunrpc/xdrgen-builtins.h | 253 ++++++++++
> tools/net/sunrpc/xdrgen/.gitignore | 2 +
> tools/net/sunrpc/xdrgen/README | 231 +++++++++
> tools/net/sunrpc/xdrgen/__init__.py | 0
> tools/net/sunrpc/xdrgen/common.py | 21 +
> .../net/sunrpc/xdrgen/generators/__init__.py | 0
> .../sunrpc/xdrgen/generators/boilerplate.py | 66 +++
> .../net/sunrpc/xdrgen/generators/constant.py | 38 ++
> tools/net/sunrpc/xdrgen/generators/enum.py | 67 +++
> tools/net/sunrpc/xdrgen/generators/program.py | 96 ++++
> tools/net/sunrpc/xdrgen/generators/struct.py | 435 +++++++++++++++++
> tools/net/sunrpc/xdrgen/generators/typedef.py | 282 +++++++++++
> tools/net/sunrpc/xdrgen/generators/union.py | 297 ++++++++++++
> tools/net/sunrpc/xdrgen/subcmds/__init__.py | 0
> tools/net/sunrpc/xdrgen/subcmds/header.py | 62 +++
> tools/net/sunrpc/xdrgen/subcmds/lint.py | 28 ++
> tools/net/sunrpc/xdrgen/subcmds/source.py | 82 ++++
> .../templates/C/boilerplate/header_bottom.j2 | 3 +
> .../templates/C/boilerplate/header_top.j2 | 11 +
> .../templates/C/boilerplate/source_top.j2 | 5 +
> .../templates/C/constants/declaration.j2 | 5 +
> .../templates/C/enum/declaration/close.j2 | 7 +
> .../C/enum/declaration/enumerator.j2 | 2 +
> .../templates/C/enum/declaration/open.j2 | 3 +
> .../xdrgen/templates/C/enum/decoder/enum.j2 | 19 +
> .../xdrgen/templates/C/enum/encoder/enum.j2 | 14 +
> .../C/program/declaration/argument.j2 | 2 +
> .../templates/C/program/declaration/result.j2 | 2 +
> .../templates/C/program/decoder/argument.j2 | 21 +
> .../templates/C/program/encoder/result.j2 | 21 +
> .../templates/C/struct/declaration/basic.j2 | 5 +
> .../templates/C/struct/declaration/close.j2 | 7 +
> .../struct/declaration/fixed_length_array.j2 | 5 +
> .../struct/declaration/fixed_length_opaque.j2 | 5 +
> .../templates/C/struct/declaration/open.j2 | 6 +
> .../C/struct/declaration/optional_data.j2 | 5 +
> .../C/struct/declaration/pointer-open.j2 | 6 +
> .../declaration/variable_length_array.j2 | 8 +
> .../declaration/variable_length_opaque.j2 | 5 +
> .../declaration/variable_length_string.j2 | 5 +
> .../templates/C/struct/decoder/basic.j2 | 6 +
> .../templates/C/struct/decoder/close.j2 | 3 +
> .../C/struct/decoder/fixed_length_array.j2 | 8 +
> .../C/struct/decoder/fixed_length_opaque.j2 | 6 +
> .../xdrgen/templates/C/struct/decoder/open.j2 | 12 +
> .../C/struct/decoder/optional_data.j2 | 6 +
> .../C/struct/decoder/pointer-open.j2 | 22 +
> .../C/struct/decoder/variable_length_array.j2 | 13 +
> .../struct/decoder/variable_length_opaque.j2 | 6 +
> .../struct/decoder/variable_length_string.j2 | 6 +
> .../templates/C/struct/encoder/basic.j2 | 10 +
> .../templates/C/struct/encoder/close.j2 | 3 +
> .../C/struct/encoder/fixed_length_array.j2 | 12 +
> .../C/struct/encoder/fixed_length_opaque.j2 | 6 +
> .../xdrgen/templates/C/struct/encoder/open.j2 | 12 +
> .../C/struct/encoder/optional_data.j2 | 6 +
> .../C/struct/encoder/pointer-open.j2 | 20 +
> .../C/struct/encoder/variable_length_array.j2 | 15 +
> .../struct/encoder/variable_length_opaque.j2 | 8 +
> .../struct/encoder/variable_length_string.j2 | 8 +
> .../templates/C/typedef/declaration/basic.j2 | 11 +
> .../typedef/declaration/fixed_length_array.j2 | 11 +
> .../declaration/fixed_length_opaque.j2 | 11 +
> .../declaration/variable_length_array.j2 | 14 +
> .../declaration/variable_length_opaque.j2 | 11 +
> .../declaration/variable_length_string.j2 | 11 +
> .../templates/C/typedef/decoder/basic.j2 | 17 +
> .../C/typedef/decoder/fixed_length_array.j2 | 25 +
> .../C/typedef/decoder/fixed_length_opaque.j2 | 17 +
> .../typedef/decoder/variable_length_array.j2 | 26 ++
> .../typedef/decoder/variable_length_opaque.j2 | 17 +
> .../typedef/decoder/variable_length_string.j2 | 17 +
> .../templates/C/typedef/encoder/basic.j2 | 21 +
> .../C/typedef/encoder/fixed_length_array.j2 | 25 +
> .../C/typedef/encoder/fixed_length_opaque.j2 | 17 +
> .../typedef/encoder/variable_length_array.j2 | 30 ++
> .../typedef/encoder/variable_length_opaque.j2 | 17 +
> .../typedef/encoder/variable_length_string.j2 | 17 +
> .../C/union/declaration/case_spec.j2 | 2 +
> .../templates/C/union/declaration/close.j2 | 8 +
> .../C/union/declaration/default_spec.j2 | 2 +
> .../templates/C/union/declaration/open.j2 | 6 +
> .../C/union/declaration/switch_spec.j2 | 3 +
> .../xdrgen/templates/C/union/decoder/basic.j2 | 6 +
> .../xdrgen/templates/C/union/decoder/break.j2 | 2 +
> .../templates/C/union/decoder/case_spec.j2 | 2 +
> .../xdrgen/templates/C/union/decoder/close.j2 | 4 +
> .../templates/C/union/decoder/default_spec.j2 | 2 +
> .../xdrgen/templates/C/union/decoder/open.j2 | 12 +
> .../C/union/decoder/optional_data.j2 | 6 +
> .../templates/C/union/decoder/switch_spec.j2 | 7 +
> .../C/union/decoder/variable_length_array.j2 | 13 +
> .../C/union/decoder/variable_length_opaque.j2 | 6 +
> .../C/union/decoder/variable_length_string.j2 | 6 +
> .../xdrgen/templates/C/union/decoder/void.j2 | 3 +
> .../xdrgen/templates/C/union/encoder/basic.j2 | 10 +
> .../xdrgen/templates/C/union/encoder/break.j2 | 2 +
> .../templates/C/union/encoder/case_spec.j2 | 2 +
> .../xdrgen/templates/C/union/encoder/close.j2 | 4 +
> .../templates/C/union/encoder/default_spec.j2 | 2 +
> .../xdrgen/templates/C/union/encoder/open.j2 | 12 +
> .../templates/C/union/encoder/switch_spec.j2 | 7 +
> .../xdrgen/templates/C/union/encoder/void.j2 | 3 +
> tools/net/sunrpc/xdrgen/tests/test.x | 36 ++
> tools/net/sunrpc/xdrgen/xdr | 1 +
> tools/net/sunrpc/xdrgen/xdr.ebnf | 117 +++++
> tools/net/sunrpc/xdrgen/xdr_ast.py | 437 ++++++++++++++++++
> tools/net/sunrpc/xdrgen/xdr_parse.py | 20 +
> tools/net/sunrpc/xdrgen/xdrgen | 95 ++++
> 112 files changed, 3986 insertions(+)
> create mode 100644 fs/nfsd/nfs4_1.x
> create mode 100644 fs/nfsd/nfs4xdr_gen.c
> create mode 100644 fs/nfsd/nfs4xdr_gen.h
> create mode 100644 include/linux/sunrpc/xdrgen-builtins.h
> create mode 100644 tools/net/sunrpc/xdrgen/.gitignore
> create mode 100644 tools/net/sunrpc/xdrgen/README
> create mode 100644 tools/net/sunrpc/xdrgen/__init__.py
> create mode 100644 tools/net/sunrpc/xdrgen/common.py
> create mode 100644 tools/net/sunrpc/xdrgen/generators/__init__.py
> create mode 100644 tools/net/sunrpc/xdrgen/generators/boilerplate.py
> create mode 100644 tools/net/sunrpc/xdrgen/generators/constant.py
> create mode 100644 tools/net/sunrpc/xdrgen/generators/enum.py
> create mode 100644 tools/net/sunrpc/xdrgen/generators/program.py
> create mode 100644 tools/net/sunrpc/xdrgen/generators/struct.py
> create mode 100644 tools/net/sunrpc/xdrgen/generators/typedef.py
> create mode 100644 tools/net/sunrpc/xdrgen/generators/union.py
> create mode 100644 tools/net/sunrpc/xdrgen/subcmds/__init__.py
> create mode 100755 tools/net/sunrpc/xdrgen/subcmds/header.py
> create mode 100755 tools/net/sunrpc/xdrgen/subcmds/lint.py
> create mode 100755 tools/net/sunrpc/xdrgen/subcmds/source.py
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/boilerplate/header_bottom.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/boilerplate/header_top.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/boilerplate/source_top.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/constants/declaration.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/enum/declaration/close.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/enum/declaration/enumerator.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/enum/declaration/open.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/program/declaration/argument.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/program/declaration/result.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/program/decoder/argument.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/program/encoder/result.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/basic.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/close.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/fixed_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/fixed_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/open.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/optional_data.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/pointer-open.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/variable_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/variable_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/variable_length_string.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/basic.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/close.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/fixed_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/fixed_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/open.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/optional_data.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/pointer-open.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/variable_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/variable_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/variable_length_string.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/basic.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/close.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/fixed_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/fixed_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/open.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/optional_data.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/pointer-open.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_string.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/basic.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/fixed_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/fixed_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/variable_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/variable_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/variable_length_string.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/basic.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/fixed_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/fixed_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/variable_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/variable_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/variable_length_string.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/basic.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/fixed_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/fixed_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/variable_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/variable_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/variable_length_string.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/declaration/case_spec.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/declaration/close.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/declaration/default_spec.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/declaration/open.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/declaration/switch_spec.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/basic.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/break.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/case_spec.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/close.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/default_spec.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/open.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/optional_data.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/switch_spec.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/variable_length_array.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/variable_length_opaque.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/variable_length_string.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/decoder/void.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/encoder/basic.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/encoder/break.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/encoder/case_spec.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/encoder/close.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/encoder/default_spec.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/encoder/open.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/encoder/switch_spec.j2
> create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/encoder/void.j2
> create mode 100644 tools/net/sunrpc/xdrgen/tests/test.x
> create mode 120000 tools/net/sunrpc/xdrgen/xdr
> create mode 100644 tools/net/sunrpc/xdrgen/xdr.ebnf
> create mode 100644 tools/net/sunrpc/xdrgen/xdr_ast.py
> create mode 100644 tools/net/sunrpc/xdrgen/xdr_parse.py
> create mode 100755 tools/net/sunrpc/xdrgen/xdrgen
>
This all looks great to me. I really support autogenerating our XDR
boilerplate, particularly as I've been looking at implementing
CB_NOTIFY (which would be quite tedious to roll by hand). This would
make it a lot simpler and less error prone.
Do you have these patches in a branch in your tree that I could pick
from?
--
Jeff Layton <jlayton@kernel.org>
next prev parent reply other threads:[~2024-08-21 14:04 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-08-20 14:45 [RFC PATCH 0/2] xdrgen - machine-generated XDR functions cel
2024-08-20 14:45 ` [RFC PATCH 1/2] tools: Add xdrgen cel
2024-08-20 14:46 ` [RFC PATCH 2/2] NFSD: Create an initial nfs4_1.x file cel
2024-08-21 14:22 ` Jeff Layton
2024-08-21 14:38 ` Chuck Lever
2024-08-21 16:51 ` Jeff Layton
2024-08-21 17:03 ` Chuck Lever
2024-08-21 17:34 ` Jeff Layton
2024-08-21 15:00 ` Christoph Hellwig
2024-08-21 15:59 ` Jeff Layton
2024-08-21 19:03 ` Chuck Lever
2024-08-21 21:14 ` Chuck Lever
2024-08-22 16:34 ` Jeff Layton
2024-08-22 17:47 ` Chuck Lever III
2024-08-21 14:03 ` Jeff Layton [this message]
2024-08-21 14:06 ` [RFC PATCH 0/2] xdrgen - machine-generated XDR functions Chuck Lever III
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d78c9d50dcdefb13c96f1fac6f708781aad0bfe3.camel@kernel.org \
--to=jlayton@kernel.org \
--cc=cel@kernel.org \
--cc=chuck.lever@oracle.com \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).