* [PATCH v1 00/10] New pragmas for the xdrgen tool
@ 2026-09-03 15:03 Chuck Lever
2026-09-03 15:03 ` [PATCH v1 01/10] SUNRPC: Carry a generated-codec context pointer in struct xdr_stream Chuck Lever
` (9 more replies)
0 siblings, 10 replies; 11+ messages in thread
From: Chuck Lever @ 2026-09-03 15:03 UTC (permalink / raw)
To: NeilBrown, Jeff Layton, Olga Kornievskaia, Dai Ngo, Tom Talpey; +Cc: linux-nfs
The NFSv2, NFSv3, and NFS_ACL conversions to xdrgen-generated codecs
need two important new features in the xdrgen tool to deal with a
few of the more complex structures that XDR can express.
The first is a payload that resides in the Linux page cache.
Currently the generated codec for a variable-length opaque copies
through a data pointer, but a READ result already sits in the pages
of rq_res and a WRITE payload arrives in the pages of the Receive
buffer. Marshaling and unmarshaling this type of payload needs to
avoid copying the payload's content.
The second is an array of non-scalar objects. Examples include the
NFS_ACL aclent<> list, which is the on-the-wire form of the kernel's
posix_acl, and the READDIR entry list, which is produced from a VFS
iteration.
No in-tree XDR specification applies either directive yet, so the
generated output remains unchanged across the whole series.
Chuck Lever (10):
SUNRPC: Carry a generated-codec context pointer in struct xdr_stream
SUNRPC: Bind the svc_rqst to its XDR streams
SUNRPC: Add svcxdr_encode_opaque_payload()
xdrgen: Pass the containing struct name to member codec emitters
xdrgen: Add a "pragma pages" directive
SUNRPC: Add svcxdr_decode_opaque_payload()
xdrgen: Extend the pages directive to page-resident arguments
xdrgen: Add hook-driven aggregate codec for variable-length arrays
xdrgen: Extend the aggregate codec to optional-data list members
xdrgen: Stream optional-data aggregate lists during encode
fs/nfsd/nfs4xdr.c | 1 +
include/linux/sunrpc/svc.h | 77 +++++
include/linux/sunrpc/xdr.h | 1 +
include/linux/sunrpc/xdrgen/_defs.h | 32 ++
net/sunrpc/xdr.c | 3 +
tools/net/sunrpc/xdrgen/README | 136 +++++++-
tools/net/sunrpc/xdrgen/generators/pointer.py | 68 +++-
tools/net/sunrpc/xdrgen/generators/struct.py | 197 +++++++++++-
tools/net/sunrpc/xdrgen/generators/union.py | 69 +++-
tools/net/sunrpc/xdrgen/grammars/xdr.lark | 2 +
.../C/pointer/decoder/pages_opaque.j2 | 6 +
.../C/pointer/definition/pages_opaque.j2 | 5 +
.../C/pointer/encoder/pages_opaque.j2 | 10 +
.../C/struct/declaration/aggregate_hooks.j2 | 14 +
.../C/struct/decoder/aggregate_array.j2 | 35 ++
.../C/struct/decoder/aggregate_optional.j2 | 31 ++
.../C/struct/decoder/pages_opaque.j2 | 6 +
.../C/struct/definition/pages_opaque.j2 | 5 +
.../C/struct/encoder/aggregate_array.j2 | 39 +++
.../C/struct/encoder/aggregate_optional.j2 | 29 ++
.../C/struct/encoder/pages_opaque.j2 | 10 +
.../templates/C/union/encoder/pages_opaque.j2 | 10 +
tools/net/sunrpc/xdrgen/xdr_ast.py | 300 ++++++++++++++++++
23 files changed, 1043 insertions(+), 43 deletions(-)
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/pages_opaque.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/pointer/definition/pages_opaque.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/pages_opaque.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/aggregate_hooks.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/aggregate_array.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/aggregate_optional.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/pages_opaque.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/definition/pages_opaque.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_array.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_optional.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/pages_opaque.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/encoder/pages_opaque.j2
--
2.55.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v1 01/10] SUNRPC: Carry a generated-codec context pointer in struct xdr_stream
2026-09-03 15:03 [PATCH v1 00/10] New pragmas for the xdrgen tool Chuck Lever
@ 2026-09-03 15:03 ` Chuck Lever
2026-09-03 15:03 ` [PATCH v1 02/10] SUNRPC: Bind the svc_rqst to its XDR streams Chuck Lever
` (8 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Chuck Lever @ 2026-09-03 15:03 UTC (permalink / raw)
To: NeilBrown, Jeff Layton, Olga Kornievskaia, Dai Ngo, Tom Talpey; +Cc: linux-nfs
A generated XDR codec function receives only an xdr_stream, yet
some codecs need state that the stream does not carry. Passing
that state as an additional argument changes the signature of
every function on the path from the program entry point down to
the one that consumes it, and for generated code it makes each
function's argument list depend on an analysis of the entire
specification.
Add xdrgen_ctx to struct xdr_stream, a pointer that the layer
initializing the stream sets aside for generated codecs to find,
following the precedent of seq_file->private and
netlink_callback->data. Every xdr_stream initializer clears it, so
a codec never reads a residual value left in a stream that was
allocated on the stack or taken from a transport's free pool.
Signed-off-by: Chuck Lever <cel@kernel.org>
---
fs/nfsd/nfs4xdr.c | 1 +
include/linux/sunrpc/svc.h | 1 +
include/linux/sunrpc/xdr.h | 1 +
net/sunrpc/xdr.c | 3 +++
4 files changed, 6 insertions(+)
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 00ddaac499c6..6ea6cb0632f8 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -4609,6 +4609,7 @@ static void svcxdr_init_encode_from_buffer(struct xdr_stream *xdr,
xdr->p = p;
xdr->end = (void *)p + bytes;
buf->buflen = bytes;
+ xdr->xdrgen_ctx = NULL;
}
__be32 nfsd4_encode_fattr_to_buf(__be32 **p, int words,
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index 5fa9417e034d..b80161b849ce 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -567,6 +567,7 @@ static inline void svcxdr_init_encode(struct svc_rqst *rqstp)
xdr->page_ptr = buf->pages - 1;
buf->buflen = PAGE_SIZE * (rqstp->rq_page_end - buf->pages);
xdr->rqst = NULL;
+ xdr->xdrgen_ctx = NULL;
}
/**
diff --git a/include/linux/sunrpc/xdr.h b/include/linux/sunrpc/xdr.h
index b102b4f21e6b..2b33fc64be17 100644
--- a/include/linux/sunrpc/xdr.h
+++ b/include/linux/sunrpc/xdr.h
@@ -244,6 +244,7 @@ struct xdr_stream {
unsigned int nwords; /* Remaining decode buffer length */
struct rpc_rqst *rqst; /* For debugging */
+ void *xdrgen_ctx; /* Context for generated codecs */
};
/*
diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c
index cb2ef428651f..d95e80d85a89 100644
--- a/net/sunrpc/xdr.c
+++ b/net/sunrpc/xdr.c
@@ -1162,6 +1162,7 @@ void xdr_init_encode(struct xdr_stream *xdr, struct xdr_buf *buf, __be32 *p,
iov->iov_len += len;
}
xdr->rqst = rqst;
+ xdr->xdrgen_ctx = NULL;
}
EXPORT_SYMBOL_GPL(xdr_init_encode);
@@ -1181,6 +1182,7 @@ void xdr_init_encode_pages(struct xdr_stream *xdr, struct xdr_buf *buf)
xdr->p = page_address(*xdr->page_ptr);
xdr->end = (void *)xdr->p + min_t(u32, buf->buflen, PAGE_SIZE);
xdr->rqst = NULL;
+ xdr->xdrgen_ctx = NULL;
}
EXPORT_SYMBOL_GPL(xdr_init_encode_pages);
@@ -1619,6 +1621,7 @@ void xdr_init_decode(struct xdr_stream *xdr, struct xdr_buf *buf, __be32 *p,
xdr->p = p;
}
xdr->rqst = rqst;
+ xdr->xdrgen_ctx = NULL;
}
EXPORT_SYMBOL_GPL(xdr_init_decode);
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v1 02/10] SUNRPC: Bind the svc_rqst to its XDR streams
2026-09-03 15:03 [PATCH v1 00/10] New pragmas for the xdrgen tool Chuck Lever
2026-09-03 15:03 ` [PATCH v1 01/10] SUNRPC: Carry a generated-codec context pointer in struct xdr_stream Chuck Lever
@ 2026-09-03 15:03 ` Chuck Lever
2026-09-03 15:03 ` [PATCH v1 03/10] SUNRPC: Add svcxdr_encode_opaque_payload() Chuck Lever
` (7 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Chuck Lever @ 2026-09-03 15:03 UTC (permalink / raw)
To: NeilBrown, Jeff Layton, Olga Kornievskaia, Dai Ngo, Tom Talpey; +Cc: linux-nfs
A generated XDR codec function receives only an xdr_stream, but
some codecs need the RPC transaction the stream belongs to: to
reach the procedure's argument and result structures, or to mark
a page-resident byte range as a result payload. A client stream
already records its transaction in the rqst field; a server
stream records nothing, so each generated codec that needs the
svc_rqst would have to store it in xdrgen_ctx itself, and every
such site would decide on its own what the pointer means.
Store the svc_rqst in xdrgen_ctx from svcxdr_init_decode() and
svcxdr_init_encode() instead, and read it back through
svcxdr_rqst() so that no codec open-codes the cast from void *.
A codec running on a stream either of those initialized reaches
the transaction, and through it rq_argp and rq_resp. Every other
initializer leaves xdrgen_ctx NULL, so svcxdr_rqst() returns NULL
there and the guarantee is scoped to those two entry points; a
codec that needs the svc_rqst names them in its Context: line.
Signed-off-by: Chuck Lever <cel@kernel.org>
---
include/linux/sunrpc/svc.h | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index b80161b849ce..e69aeb864a8d 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -532,6 +532,8 @@ static inline void svc_reserve_auth(struct svc_rqst *rqstp, int space)
* svcxdr_init_decode - Prepare an xdr_stream for Call decoding
* @rqstp: controlling server RPC transaction context
*
+ * The stream records @rqstp, so a codec running on it reaches the
+ * transaction with svcxdr_rqst().
*/
static inline void svcxdr_init_decode(struct svc_rqst *rqstp)
{
@@ -544,12 +546,15 @@ static inline void svcxdr_init_decode(struct svc_rqst *rqstp)
xdr_init_decode(xdr, buf, argv->iov_base, NULL);
xdr_set_scratch_folio(xdr, rqstp->rq_scratch_folio);
+ xdr->xdrgen_ctx = rqstp;
}
/**
* svcxdr_init_encode - Prepare an xdr_stream for svc Reply encoding
* @rqstp: controlling server RPC transaction context
*
+ * The stream records @rqstp, so a codec running on it reaches the
+ * transaction with svcxdr_rqst().
*/
static inline void svcxdr_init_encode(struct svc_rqst *rqstp)
{
@@ -567,7 +572,19 @@ static inline void svcxdr_init_encode(struct svc_rqst *rqstp)
xdr->page_ptr = buf->pages - 1;
buf->buflen = PAGE_SIZE * (rqstp->rq_page_end - buf->pages);
xdr->rqst = NULL;
- xdr->xdrgen_ctx = NULL;
+ xdr->xdrgen_ctx = rqstp;
+}
+
+/**
+ * svcxdr_rqst - Retrieve the transaction bound to an xdr_stream
+ * @xdr: stream to query
+ *
+ * Return: the controlling svc_rqst when @xdr was initialized by
+ * svcxdr_init_decode() or svcxdr_init_encode(), otherwise NULL.
+ */
+static inline struct svc_rqst *svcxdr_rqst(struct xdr_stream *xdr)
+{
+ return xdr->xdrgen_ctx;
}
/**
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v1 03/10] SUNRPC: Add svcxdr_encode_opaque_payload()
2026-09-03 15:03 [PATCH v1 00/10] New pragmas for the xdrgen tool Chuck Lever
2026-09-03 15:03 ` [PATCH v1 01/10] SUNRPC: Carry a generated-codec context pointer in struct xdr_stream Chuck Lever
2026-09-03 15:03 ` [PATCH v1 02/10] SUNRPC: Bind the svc_rqst to its XDR streams Chuck Lever
@ 2026-09-03 15:03 ` Chuck Lever
2026-09-03 15:03 ` [PATCH v1 04/10] xdrgen: Pass the containing struct name to member codec emitters Chuck Lever
` (6 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Chuck Lever @ 2026-09-03 15:03 UTC (permalink / raw)
To: NeilBrown, Jeff Layton, Olga Kornievskaia, Dai Ngo, Tom Talpey; +Cc: linux-nfs
The encoder functions that xdrgen emits handle a variable-length
opaque with xdr_stream_encode_opaque(), which copies the opaque
content into the RPC buffer. On the server, an NFS READ payload
already resides in the pages of rqstp->rq_res, placed there by an
underlying file system read. Copying that payload into the RPC
Reply buffer adds a data copy on every READ and prevents a
transport capable of direct data placement from conveying the
payload separately from the rest of the Reply.
Add a helper that encodes a page-resident opaque by reference: it
encodes the length, inserts the payload pages into the XDR stream,
then marks the byte range as a result payload.
Generated code calls this helper, so its parameters are limited to
the xdr_stream and the payload length. It recovers the svc_rqst
with svcxdr_rqst(), and takes the reply's page array from the
stream's buffer.
Signed-off-by: Chuck Lever <cel@kernel.org>
---
include/linux/sunrpc/svc.h | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index e69aeb864a8d..93b980439159 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -608,6 +608,33 @@ static inline void svcxdr_encode_opaque_pages(struct svc_rqst *rqstp,
xdr->page_ptr = rqstp->rq_next_page - 1;
}
+/**
+ * svcxdr_encode_opaque_payload - Encode a page-resident opaque data item
+ * @xdr: xdr_stream to be updated
+ * @len: number of octets of content in the data item
+ *
+ * Context: Process context. @xdr must have been initialized by
+ * svcxdr_init_encode() and still be positioned in the reply
+ * head.
+ *
+ * Return:
+ * %true: Success
+ * %false: The length prefix would overrun the buffer, or the
+ * transport could not accommodate the result payload
+ */
+static inline bool svcxdr_encode_opaque_payload(struct xdr_stream *xdr, u32 len)
+{
+ struct svc_rqst *rqstp = svcxdr_rqst(xdr);
+ struct xdr_buf *buf = xdr->buf;
+
+ if (xdr_stream_encode_u32(xdr, len) < 0)
+ return false;
+ svcxdr_encode_opaque_pages(rqstp, xdr, buf->pages, buf->page_base, len);
+ if (svc_encode_result_payload(rqstp, buf->head->iov_len, len) < 0)
+ return false;
+ return true;
+}
+
/**
* svcxdr_set_auth_slack -
* @rqstp: RPC transaction
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v1 04/10] xdrgen: Pass the containing struct name to member codec emitters
2026-09-03 15:03 [PATCH v1 00/10] New pragmas for the xdrgen tool Chuck Lever
` (2 preceding siblings ...)
2026-09-03 15:03 ` [PATCH v1 03/10] SUNRPC: Add svcxdr_encode_opaque_payload() Chuck Lever
@ 2026-09-03 15:03 ` Chuck Lever
2026-09-03 15:03 ` [PATCH v1 05/10] xdrgen: Add a "pragma pages" directive Chuck Lever
` (5 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Chuck Lever @ 2026-09-03 15:03 UTC (permalink / raw)
To: NeilBrown, Jeff Layton, Olga Kornievskaia, Dai Ngo, Tom Talpey; +Cc: linux-nfs
A per-member code generation directive keyed on the (struct, member)
pair needs the name of the containing struct, but
emit_struct_member_encoder() and emit_struct_member_decoder()
receive only the member declaration.
Pass the containing struct's name to emit_struct_member_encoder() and
emit_struct_member_decoder(), and to the matching emitters in
pointer.py, as a required argument. A struct whose last field refers
to its own type is an _XdrPointer rather than an _XdrStruct, so
pointer.py takes the same argument. No emitter renders the new value
yet, so generated output is byte-identical. A subsequent patch keys
the "pragma pages" encoder on it.
Signed-off-by: Chuck Lever <cel@kernel.org>
---
tools/net/sunrpc/xdrgen/generators/pointer.py | 12 ++++++++----
tools/net/sunrpc/xdrgen/generators/struct.py | 12 ++++++++----
2 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/tools/net/sunrpc/xdrgen/generators/pointer.py b/tools/net/sunrpc/xdrgen/generators/pointer.py
index 6dbda60ad2db..9a540e125786 100644
--- a/tools/net/sunrpc/xdrgen/generators/pointer.py
+++ b/tools/net/sunrpc/xdrgen/generators/pointer.py
@@ -91,7 +91,9 @@ def emit_pointer_definition(environment: Environment, node: _XdrPointer) -> None
def emit_pointer_member_decoder(
- environment: Environment, field: _XdrDeclaration
+ environment: Environment,
+ field: _XdrDeclaration,
+ struct_name: str,
) -> None:
"""Emit a decoder for one field in an XDR pointer"""
if isinstance(field, _XdrBasic):
@@ -164,14 +166,16 @@ def emit_pointer_decoder(environment: Environment, node: _XdrPointer) -> None:
print(template.render(name=node.name))
for field in node.fields[0:-1]:
- emit_pointer_member_decoder(environment, field)
+ emit_pointer_member_decoder(environment, field, node.name)
template = get_jinja2_template(environment, "decoder", "close")
print(template.render())
def emit_pointer_member_encoder(
- environment: Environment, field: _XdrDeclaration
+ environment: Environment,
+ field: _XdrDeclaration,
+ struct_name: str,
) -> None:
"""Emit an encoder for one field in a XDR pointer"""
if isinstance(field, _XdrBasic):
@@ -241,7 +245,7 @@ def emit_pointer_encoder(environment: Environment, node: _XdrPointer) -> None:
print(template.render(name=node.name))
for field in node.fields[0:-1]:
- emit_pointer_member_encoder(environment, field)
+ emit_pointer_member_encoder(environment, field, node.name)
template = get_jinja2_template(environment, "encoder", "close")
print(template.render())
diff --git a/tools/net/sunrpc/xdrgen/generators/struct.py b/tools/net/sunrpc/xdrgen/generators/struct.py
index 64911de46f62..91da3863c166 100644
--- a/tools/net/sunrpc/xdrgen/generators/struct.py
+++ b/tools/net/sunrpc/xdrgen/generators/struct.py
@@ -91,7 +91,9 @@ def emit_struct_definition(environment: Environment, node: _XdrStruct) -> None:
def emit_struct_member_decoder(
- environment: Environment, field: _XdrDeclaration
+ environment: Environment,
+ field: _XdrDeclaration,
+ struct_name: str,
) -> None:
"""Emit a decoder for one field in an XDR struct"""
if isinstance(field, _XdrBasic):
@@ -164,14 +166,16 @@ def emit_struct_decoder(environment: Environment, node: _XdrStruct) -> None:
print(template.render(name=node.name))
for field in node.fields:
- emit_struct_member_decoder(environment, field)
+ emit_struct_member_decoder(environment, field, node.name)
template = get_jinja2_template(environment, "decoder", "close")
print(template.render())
def emit_struct_member_encoder(
- environment: Environment, field: _XdrDeclaration
+ environment: Environment,
+ field: _XdrDeclaration,
+ struct_name: str,
) -> None:
"""Emit an encoder for one field in an XDR struct"""
if isinstance(field, _XdrBasic):
@@ -241,7 +245,7 @@ def emit_struct_encoder(environment: Environment, node: _XdrStruct) -> None:
print(template.render(name=node.name))
for field in node.fields:
- emit_struct_member_encoder(environment, field)
+ emit_struct_member_encoder(environment, field, node.name)
template = get_jinja2_template(environment, "encoder", "close")
print(template.render())
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v1 05/10] xdrgen: Add a "pragma pages" directive
2026-09-03 15:03 [PATCH v1 00/10] New pragmas for the xdrgen tool Chuck Lever
` (3 preceding siblings ...)
2026-09-03 15:03 ` [PATCH v1 04/10] xdrgen: Pass the containing struct name to member codec emitters Chuck Lever
@ 2026-09-03 15:03 ` Chuck Lever
2026-09-03 15:03 ` [PATCH v1 06/10] SUNRPC: Add svcxdr_decode_opaque_payload() Chuck Lever
` (4 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Chuck Lever @ 2026-09-03 15:03 UTC (permalink / raw)
To: NeilBrown, Jeff Layton, Olga Kornievskaia, Dai Ngo, Tom Talpey; +Cc: linux-nfs
The generated encoder for a variable-length opaque or string
member copies its content into the Reply buffer via
xdr_stream_encode_opaque(). A bulk payload such as an NFS READ
result never resides at the member's data pointer: the server
deposits it directly in the pages of the Reply buffer. Until now
the containing procedure encoder had to be hand-rolled.
Add a per-member directive that makes the emitted encoder call
svcxdr_encode_opaque_payload() for the marked member. The README
documents its syntax and the members it may name. Beyond the
struct and union cases described there, the directive also applies
to a member of a self-referential struct, which xdrgen generates
as an XDR pointer type.
A directive that binds to nothing would degrade silently to the
copying encoder, so the front end rejects one that names an
undeclared type or member, or a member that cannot carry a
payload. It also rejects a second marked member of the same type:
svcxdr_encode_opaque_payload() consumes the whole page vector and
moves the stream into the tail, so one encoder carries at most one
payload.
Generated code is unchanged until a specification applies the
directive.
Signed-off-by: Chuck Lever <cel@kernel.org>
---
tools/net/sunrpc/xdrgen/README | 42 +++++-
tools/net/sunrpc/xdrgen/generators/pointer.py | 25 +++-
tools/net/sunrpc/xdrgen/generators/struct.py | 23 ++-
tools/net/sunrpc/xdrgen/generators/union.py | 69 +++++++--
.../C/pointer/encoder/pages_opaque.j2 | 10 ++
.../C/struct/encoder/pages_opaque.j2 | 10 ++
.../templates/C/union/encoder/pages_opaque.j2 | 10 ++
tools/net/sunrpc/xdrgen/xdr_ast.py | 131 ++++++++++++++++++
8 files changed, 298 insertions(+), 22 deletions(-)
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/pages_opaque.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/pages_opaque.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/union/encoder/pages_opaque.j2
diff --git a/tools/net/sunrpc/xdrgen/README b/tools/net/sunrpc/xdrgen/README
index 2cf05d1e4cd9..5af9cf506637 100644
--- a/tools/net/sunrpc/xdrgen/README
+++ b/tools/net/sunrpc/xdrgen/README
@@ -230,6 +230,43 @@ Adds these prototypes in the generated header:
And, in the generated source code, both of these functions appear
without the "static __maybe_unused" modifiers.
+Pragma pages
+------ -----
+
+ pragma pages <type> <member> ;
+
+A variable-length opaque or string member is normally encoded by
+copying its content into the Reply buffer. When the member carries
+a bulk payload that a server-side read operation has already
+deposited in the pages of the Reply buffer, that copy is wasted
+work, and it prevents a transport capable of direct data placement
+from conveying the payload separately from the rest of the Reply.
+This directive marks such a member so that the emitted encoder
+inserts the payload pages into the encoding stream by reference and
+marks the byte range as a result payload.
+
+The marked member may be a member of a struct or an arm of a union,
+and its type may be named directly or reached through a typedef. A
+union arm declared directly as a string is the one exception: xdrgen
+generates such an arm as a char *, which carries no length field, so
+the directive is rejected there and the arm has to be declared
+through a typedef. For example:
+
+ pragma pages READ3resok data;
+ pragma pages readlinkres data;
+
+The first marks the opaque data member of struct READ3resok; the
+second marks the path arm of union readlinkres, where path is
+"typedef string path<NFS_MAXPATHLEN>".
+
+The emitted encoder reads only the marked member's length field to
+determine how many octets of page content to insert; the member's
+data pointer is not read. The server and client manage payload
+pages differently, so this directive applies only to server-side
+code generation, and so far only to the encoding direction.
+Decode-side handling of a page-resident argument (for example, the
+payload of an NFS WRITE) remains hand-coded.
+
Future Work
-----------
@@ -242,11 +279,12 @@ Expand the README into a user guide similar to rpcgen(1)
Add more pragma directives:
- * @pages -- use xdr_read/write_pages() for the specified opaque
- field
* @skip -- do not decode, but rather skip, the specified argument
field
+Extend pragma pages to the decoding direction and to
+client-side code generation
+
Enable something like a #include to dynamically insert the content
of other specification files
diff --git a/tools/net/sunrpc/xdrgen/generators/pointer.py b/tools/net/sunrpc/xdrgen/generators/pointer.py
index 9a540e125786..448ebd684212 100644
--- a/tools/net/sunrpc/xdrgen/generators/pointer.py
+++ b/tools/net/sunrpc/xdrgen/generators/pointer.py
@@ -13,6 +13,7 @@ from xdr_ast import _XdrFixedLengthOpaque, _XdrVariableLengthOpaque
from xdr_ast import _XdrFixedLengthArray, _XdrVariableLengthArray
from xdr_ast import _XdrOptionalData, _XdrPointer, _XdrDeclaration
from xdr_ast import public_apis, get_header_name
+from xdr_ast import pages_members, pages_member_maxsize
def emit_pointer_declaration(environment: Environment, node: _XdrPointer) -> None:
@@ -176,8 +177,24 @@ def emit_pointer_member_encoder(
environment: Environment,
field: _XdrDeclaration,
struct_name: str,
+ peer: str,
) -> None:
"""Emit an encoder for one field in a XDR pointer"""
+ if (struct_name, field.name) in pages_members:
+ if peer != "server":
+ raise NotImplementedError(
+ "pragma pages is server-side encode-only; "
+ + peer
+ + " generation is not yet supported"
+ )
+ template = get_jinja2_template(environment, "encoder", "pages_opaque")
+ print(
+ template.render(
+ name=field.name,
+ maxsize=pages_member_maxsize(field),
+ )
+ )
+ return
if isinstance(field, _XdrBasic):
template = get_jinja2_template(environment, "encoder", field.template)
print(
@@ -239,13 +256,15 @@ def emit_pointer_member_encoder(
)
-def emit_pointer_encoder(environment: Environment, node: _XdrPointer) -> None:
+def emit_pointer_encoder(
+ environment: Environment, node: _XdrPointer, peer: str
+) -> None:
"""Emit one encoder function for an XDR pointer type"""
template = get_jinja2_template(environment, "encoder", "open")
print(template.render(name=node.name))
for field in node.fields[0:-1]:
- emit_pointer_member_encoder(environment, field, node.name)
+ emit_pointer_member_encoder(environment, field, node.name, peer)
template = get_jinja2_template(environment, "encoder", "close")
print(template.render())
@@ -285,7 +304,7 @@ class XdrPointerGenerator(SourceGenerator):
def emit_encoder(self, node: _XdrPointer) -> None:
"""Emit one encoder function for an XDR pointer type"""
- emit_pointer_encoder(self.environment, node)
+ emit_pointer_encoder(self.environment, node, self.peer)
def emit_maxsize(self, node: _XdrPointer) -> None:
"""Emit one maxsize macro for an XDR pointer type"""
diff --git a/tools/net/sunrpc/xdrgen/generators/struct.py b/tools/net/sunrpc/xdrgen/generators/struct.py
index 91da3863c166..76fd3afbeea0 100644
--- a/tools/net/sunrpc/xdrgen/generators/struct.py
+++ b/tools/net/sunrpc/xdrgen/generators/struct.py
@@ -13,6 +13,7 @@ from xdr_ast import _XdrFixedLengthOpaque, _XdrVariableLengthOpaque
from xdr_ast import _XdrFixedLengthArray, _XdrVariableLengthArray
from xdr_ast import _XdrOptionalData, _XdrStruct, _XdrDeclaration
from xdr_ast import public_apis, get_header_name
+from xdr_ast import pages_members, pages_member_maxsize
def emit_struct_declaration(environment: Environment, node: _XdrStruct) -> None:
@@ -176,8 +177,24 @@ def emit_struct_member_encoder(
environment: Environment,
field: _XdrDeclaration,
struct_name: str,
+ peer: str,
) -> None:
"""Emit an encoder for one field in an XDR struct"""
+ if (struct_name, field.name) in pages_members:
+ if peer != "server":
+ raise NotImplementedError(
+ "pragma pages is server-side encode-only; "
+ + peer
+ + " generation is not yet supported"
+ )
+ template = get_jinja2_template(environment, "encoder", "pages_opaque")
+ print(
+ template.render(
+ name=field.name,
+ maxsize=pages_member_maxsize(field),
+ )
+ )
+ return
if isinstance(field, _XdrBasic):
template = get_jinja2_template(environment, "encoder", field.template)
print(
@@ -239,13 +256,13 @@ def emit_struct_member_encoder(
)
-def emit_struct_encoder(environment: Environment, node: _XdrStruct) -> None:
+def emit_struct_encoder(environment: Environment, node: _XdrStruct, peer: str) -> None:
"""Emit one encoder function for an XDR struct type"""
template = get_jinja2_template(environment, "encoder", "open")
print(template.render(name=node.name))
for field in node.fields:
- emit_struct_member_encoder(environment, field, node.name)
+ emit_struct_member_encoder(environment, field, node.name, peer)
template = get_jinja2_template(environment, "encoder", "close")
print(template.render())
@@ -285,7 +302,7 @@ class XdrStructGenerator(SourceGenerator):
def emit_encoder(self, node: _XdrStruct) -> None:
"""Emit one encoder function for an XDR struct type"""
- emit_struct_encoder(self.environment, node)
+ emit_struct_encoder(self.environment, node, self.peer)
def emit_maxsize(self, node: _XdrStruct) -> None:
"""Emit one maxsize macro for an XDR struct type"""
diff --git a/tools/net/sunrpc/xdrgen/generators/union.py b/tools/net/sunrpc/xdrgen/generators/union.py
index d15837dae651..47d63236f88f 100644
--- a/tools/net/sunrpc/xdrgen/generators/union.py
+++ b/tools/net/sunrpc/xdrgen/generators/union.py
@@ -9,7 +9,8 @@ from generators import SourceGenerator
from generators import create_jinja2_environment, get_jinja2_template
from xdr_ast import _XdrBasic, _XdrUnion, _XdrVoid, _XdrString, get_header_name
-from xdr_ast import _XdrDeclaration, _XdrCaseSpec, public_apis, big_endian
+from xdr_ast import _XdrDeclaration, _XdrCaseSpec
+from xdr_ast import public_apis, big_endian, pages_members, pages_member_maxsize
def emit_union_declaration(environment: Environment, node: _XdrUnion) -> None:
@@ -214,13 +215,41 @@ def emit_union_switch_spec_encoder(
print(template.render(name=node.name, type=node.spec.type_name))
+def emit_union_pages_arm_encoder(
+ environment: Environment, union_name: str, arm: _XdrDeclaration, peer: str
+) -> bool:
+ """Emit a page-resident encoder for a "pragma pages" union arm
+
+ Return True when the arm is pages-marked and was emitted here, so
+ the caller skips the ordinary arm encoder.
+ """
+ if (union_name, arm.name) not in pages_members:
+ return False
+ if peer != "server":
+ raise NotImplementedError(
+ "pragma pages is server-side encode-only; "
+ + peer
+ + " generation is not yet supported"
+ )
+ template = get_jinja2_template(environment, "encoder", "pages_opaque")
+ print(
+ template.render(
+ name=arm.name,
+ maxsize=pages_member_maxsize(arm),
+ )
+ )
+ return True
+
+
def emit_union_arm_encoder(
- environment: Environment, node: _XdrCaseSpec
+ environment: Environment, node: _XdrCaseSpec, union_name: str, peer: str
) -> None:
"""Emit encoder for an XDR union's arm (data only, no case/break)"""
if isinstance(node.arm, _XdrVoid):
return
+ if emit_union_pages_arm_encoder(environment, union_name, node.arm, peer):
+ return
if isinstance(node.arm, _XdrString):
type_name = "char *"
else:
@@ -237,7 +266,11 @@ def emit_union_arm_encoder(
def emit_union_case_spec_encoder(
- environment: Environment, node: _XdrCaseSpec, big_endian_discriminant: bool
+ environment: Environment,
+ node: _XdrCaseSpec,
+ big_endian_discriminant: bool,
+ union_name: str,
+ peer: str,
) -> None:
"""Emit encoder functions for an XDR union's case arm"""
@@ -254,19 +287,22 @@ def emit_union_case_spec_encoder(
for case in node.values:
print(template.render(case=case))
- template = get_jinja2_template(environment, "encoder", node.arm.template)
- print(
- template.render(
- name=node.arm.name,
- type=type_name,
+ if not emit_union_pages_arm_encoder(environment, union_name, node.arm, peer):
+ template = get_jinja2_template(environment, "encoder", node.arm.template)
+ print(
+ template.render(
+ name=node.arm.name,
+ type=type_name,
+ )
)
- )
template = get_jinja2_template(environment, "encoder", "break")
print(template.render())
-def emit_union_default_spec_encoder(environment: Environment, node: _XdrUnion) -> None:
+def emit_union_default_spec_encoder(
+ environment: Environment, node: _XdrUnion, peer: str
+) -> None:
"""Emit an encoder function for an XDR union's default arm"""
default_case = node.default
@@ -282,6 +318,9 @@ def emit_union_default_spec_encoder(environment: Environment, node: _XdrUnion) -
print(template.render())
return
+ if emit_union_pages_arm_encoder(environment, node.name, default_case.arm, peer):
+ return
+
template = get_jinja2_template(environment, "encoder", default_case.arm.template)
print(
template.render(
@@ -291,7 +330,7 @@ def emit_union_default_spec_encoder(environment: Environment, node: _XdrUnion) -
)
-def emit_union_encoder(environment, node: _XdrUnion) -> None:
+def emit_union_encoder(environment, node: _XdrUnion, peer: str) -> None:
"""Emit one XDR union encoder"""
template = get_jinja2_template(environment, "encoder", "open")
print(template.render(name=node.name))
@@ -304,7 +343,7 @@ def emit_union_encoder(environment, node: _XdrUnion) -> None:
# Find and emit the TRUE case
for case in node.cases:
if case.values and case.values[0] == "TRUE":
- emit_union_arm_encoder(environment, case)
+ emit_union_arm_encoder(environment, case, node.name, peer)
break
template = get_jinja2_template(environment, "encoder", "close")
@@ -317,9 +356,11 @@ def emit_union_encoder(environment, node: _XdrUnion) -> None:
environment,
case,
node.discriminant.spec.type_name in big_endian,
+ node.name,
+ peer,
)
- emit_union_default_spec_encoder(environment, node)
+ emit_union_default_spec_encoder(environment, node, peer)
template = get_jinja2_template(environment, "encoder", "close")
print(template.render())
@@ -359,7 +400,7 @@ class XdrUnionGenerator(SourceGenerator):
def emit_encoder(self, node: _XdrUnion) -> None:
"""Emit one encoder function for an XDR union"""
- emit_union_encoder(self.environment, node)
+ emit_union_encoder(self.environment, node, self.peer)
def emit_maxsize(self, node: _XdrUnion) -> None:
"""Emit one maxsize macro for an XDR union"""
diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/pages_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/pages_opaque.j2
new file mode 100644
index 000000000000..e868220a59f6
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/pages_opaque.j2
@@ -0,0 +1,10 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+{% if annotate %}
+ /* member {{ name }} (page-resident variable-length opaque) */
+{% endif %}
+{% if maxsize != "0" %}
+ if (value->{{ name }}.len > {{ maxsize }})
+ return false;
+{% endif %}
+ if (!svcxdr_encode_opaque_payload(xdr, value->{{ name }}.len))
+ return false;
diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/pages_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/pages_opaque.j2
new file mode 100644
index 000000000000..e868220a59f6
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/pages_opaque.j2
@@ -0,0 +1,10 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+{% if annotate %}
+ /* member {{ name }} (page-resident variable-length opaque) */
+{% endif %}
+{% if maxsize != "0" %}
+ if (value->{{ name }}.len > {{ maxsize }})
+ return false;
+{% endif %}
+ if (!svcxdr_encode_opaque_payload(xdr, value->{{ name }}.len))
+ return false;
diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/pages_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/pages_opaque.j2
new file mode 100644
index 000000000000..7fe3108a44c0
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/pages_opaque.j2
@@ -0,0 +1,10 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+{% if annotate %}
+ /* member {{ name }} (page-resident variable-length opaque) */
+{% endif %}
+{% if maxsize != "0" %}
+ if (ptr->u.{{ name }}.len > {{ maxsize }})
+ return false;
+{% endif %}
+ if (!svcxdr_encode_opaque_payload(xdr, ptr->u.{{ name }}.len))
+ return false;
diff --git a/tools/net/sunrpc/xdrgen/xdr_ast.py b/tools/net/sunrpc/xdrgen/xdr_ast.py
index 9dab8bc545b0..81b40cef839c 100644
--- a/tools/net/sunrpc/xdrgen/xdr_ast.py
+++ b/tools/net/sunrpc/xdrgen/xdr_ast.py
@@ -19,6 +19,21 @@ public_apis = []
structs = set()
pass_by_reference = set()
+# (type_name, member_name) pairs marked "pragma pages": the member's
+# content resides in the pages of the Reply buffer, so the emitted
+# encoder inserts those pages by reference instead of copying.
+pages_members = set()
+
+# The same pairs, keyed to the directive's identifier node so a
+# diagnostic can point at the directive rather than at the type.
+pages_member_meta = {}
+
+# Typedefs of a variable-length opaque or string, mapped to the length
+# bound. A "pragma pages" member declared through one ("path data;"
+# with "typedef string path<NFS_MAXPATHLEN>") reaches the AST as a
+# basic type reference, so the bound must be recovered from here.
+varlen_object_typedefs = {}
+
constants = {}
@@ -427,15 +442,34 @@ class _XdrTypedef(_XdrAst):
return self.declaration.symbolic_width()
def __post_init__(self):
+ if isinstance(self.declaration, (_XdrVariableLengthOpaque, _XdrString)):
+ varlen_object_typedefs[self.declaration.name] = self.declaration.maxsize
if isinstance(self.declaration, _XdrBasic):
new_type = self.declaration
if isinstance(new_type.spec, _XdrDefinedType):
if new_type.spec.type_name in pass_by_reference:
pass_by_reference.add(new_type.name)
+ if new_type.spec.type_name in varlen_object_typedefs:
+ varlen_object_typedefs[new_type.name] = varlen_object_typedefs[
+ new_type.spec.type_name
+ ]
max_widths[new_type.name] = self.max_width()
symbolic_widths[new_type.name] = self.symbolic_width()
+def pages_member_maxsize(field: _XdrDeclaration):
+ """Return the length bound of a "pragma pages" member.
+
+ Return None when FIELD is neither a variable-length opaque nor a
+ string, whether declared inline or through a typedef.
+ """
+ if isinstance(field, (_XdrVariableLengthOpaque, _XdrString)):
+ return field.maxsize
+ if isinstance(field, _XdrBasic) and field.spec.type_name in varlen_object_typedefs:
+ return varlen_object_typedefs[field.spec.type_name]
+ return None
+
+
@dataclass
class _XdrCaseSpec(_XdrAst):
"""One case in an XDR union"""
@@ -791,6 +825,15 @@ class ParseToAst(Transformer):
header_name = children[1].symbol
case "public_directive":
public_apis.append(children[1].symbol)
+ case "pages_directive":
+ if children[2] is None:
+ raise XdrSemanticError(
+ "pragma pages requires a type name and a member name",
+ children[1],
+ )
+ marked = (children[1].symbol, children[2].symbol)
+ pages_members.add(marked)
+ pages_member_meta[marked] = children[2]
case _:
raise NotImplementedError("Directive not supported")
return _Pragma()
@@ -990,12 +1033,100 @@ def check_rpc_number_range(root: "Specification") -> None:
)
+def _pages_candidate_members(value):
+ """Yield (member_name, declaration, container) for a type's members.
+
+ CONTAINER is the AST node holding the declaration; it decides how
+ the member is rendered in the target language.
+ """
+ if isinstance(value, _XdrStruct):
+ for field in value.fields:
+ yield field.name, field, value
+ elif isinstance(value, _XdrPointer):
+ # The trailing field is the self-reference that makes this an
+ # XDR pointer type; the emitter does not encode it.
+ for field in value.fields[0:-1]:
+ yield field.name, field, value
+ elif isinstance(value, _XdrUnion):
+ cases = list(value.cases)
+ if value.default is not None:
+ cases.append(value.default)
+ for case in cases:
+ if not isinstance(case.arm, _XdrVoid):
+ yield case.arm.name, case.arm, value
+
+
+def check_pages_directives(root: "Specification") -> None:
+ """Reject a "pragma pages" directive that cannot be honored.
+
+ Run in the front end so the diagnostic carries the directive's own
+ source position rather than surfacing as a traceback from whichever
+ emitter reaches the member. A directive that binds to nothing would
+ otherwise degrade silently to the copying encoder.
+ """
+ payloads = {}
+ resolved = set()
+ for definition in root.definitions:
+ value = definition.value
+ type_name = getattr(value, "name", None)
+ members = dict(
+ (name, (field, container))
+ for name, field, container in _pages_candidate_members(value)
+ )
+ for marked in sorted(pages_members):
+ if marked[0] != type_name:
+ continue
+ meta = pages_member_meta.get(marked)
+ if marked[1] not in members:
+ raise XdrSemanticError(
+ f"type '{type_name}' has no member '{marked[1]}'",
+ meta,
+ )
+ field, container = members[marked[1]]
+ # A union arm declared directly as a string is generated as
+ # a char *, with no length field for the page encoder to
+ # read; inside a struct it becomes a { len, data } object.
+ if isinstance(container, _XdrUnion) and isinstance(field, _XdrString):
+ raise XdrSemanticError(
+ f"union arm '{type_name}.{marked[1]}' is declared"
+ " directly as a string and carries no length field;"
+ " declare it through a typedef instead",
+ meta,
+ )
+ if pages_member_maxsize(field) is None:
+ raise XdrSemanticError(
+ f"'{type_name}.{marked[1]}' is neither a"
+ " variable-length opaque nor a string",
+ meta,
+ )
+ # svcxdr_encode_opaque_payload() consumes the whole page
+ # vector and moves the stream into the tail; a second
+ # payload would overwrite the first one's framing.
+ first = payloads.get(type_name)
+ if first is not None:
+ raise XdrSemanticError(
+ f"'{type_name}' already marks member '{first}' as"
+ " page-resident; an encoder emits at most one"
+ " page-resident payload",
+ meta,
+ )
+ payloads[type_name] = marked[1]
+ resolved.add(marked)
+
+ for marked in sorted(pages_members - resolved):
+ raise XdrSemanticError(
+ f"pragma pages names unknown type '{marked[0]}'",
+ pages_member_meta.get(marked),
+ )
+
+
def transform_parse_tree(parse_tree):
"""Transform productions into an abstract syntax tree"""
ast = transformer.transform(parse_tree)
ast.definitions = _merge_consecutive_passthru(ast.definitions)
check_duplicate_definitions(ast)
check_rpc_number_range(ast)
+ check_pages_directives(ast)
return ast
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v1 06/10] SUNRPC: Add svcxdr_decode_opaque_payload()
2026-09-03 15:03 [PATCH v1 00/10] New pragmas for the xdrgen tool Chuck Lever
` (4 preceding siblings ...)
2026-09-03 15:03 ` [PATCH v1 05/10] xdrgen: Add a "pragma pages" directive Chuck Lever
@ 2026-09-03 15:03 ` Chuck Lever
2026-09-03 15:03 ` [PATCH v1 07/10] xdrgen: Extend the pages directive to page-resident arguments Chuck Lever
` (3 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Chuck Lever @ 2026-09-03 15:03 UTC (permalink / raw)
To: NeilBrown, Jeff Layton, Olga Kornievskaia, Dai Ngo, Tom Talpey; +Cc: linux-nfs
A bulk argument payload such as the content of an NFS WRITE request
arrives in the pages of the server's Receive buffer. Copying it into
a contiguous data pointer wastes memory bandwidth and defeats a
transport capable of direct data placement, so a server decodes such
an item in place: it reads the length prefix and captures the payload
octets by reference in an xdr_buf that later stages hand to the VFS.
svcxdr_encode_opaque_payload() already provides the encode-side
counterpart for a page-resident result payload. Add the decode-side
helper so a generated decoder can express the same in-place handling
for a page-resident argument.
Signed-off-by: Chuck Lever <cel@kernel.org>
---
include/linux/sunrpc/svc.h | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index 93b980439159..7bdb0b5fd3e3 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -635,6 +635,38 @@ static inline bool svcxdr_encode_opaque_payload(struct xdr_stream *xdr, u32 len)
return true;
}
+/**
+ * svcxdr_decode_opaque_payload - Decode a page-resident opaque data item
+ * @xdr: xdr_stream to be decoded
+ * @payload: on success, describes the octets of the data item's content
+ * @maxlen: largest data item length the caller will accept, or zero for
+ * no limit
+ *
+ * A bulk payload such as the content of an NFS WRITE request resides in
+ * the pages of the Receive buffer. Rather than copy it, set @payload to
+ * describe the item's content in place.
+ *
+ * Context: Process context. @xdr must have been initialized by
+ * svcxdr_init_decode().
+ *
+ * Return:
+ * %true: @payload has been initialized and @xdr advanced past the item
+ * %false: a bounds error occurred, or the length prefix exceeds
+ * @maxlen; @payload is undefined
+ */
+static inline bool svcxdr_decode_opaque_payload(struct xdr_stream *xdr,
+ struct xdr_buf *payload,
+ u32 maxlen)
+{
+ u32 len;
+
+ if (xdr_stream_decode_u32(xdr, &len) < 0)
+ return false;
+ if (maxlen && len > maxlen)
+ return false;
+ return xdr_stream_subsegment(xdr, payload, len);
+}
+
/**
* svcxdr_set_auth_slack -
* @rqstp: RPC transaction
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v1 07/10] xdrgen: Extend the pages directive to page-resident arguments
2026-09-03 15:03 [PATCH v1 00/10] New pragmas for the xdrgen tool Chuck Lever
` (5 preceding siblings ...)
2026-09-03 15:03 ` [PATCH v1 06/10] SUNRPC: Add svcxdr_decode_opaque_payload() Chuck Lever
@ 2026-09-03 15:03 ` Chuck Lever
2026-09-03 15:03 ` [PATCH v1 08/10] xdrgen: Add hook-driven aggregate codec for variable-length arrays Chuck Lever
` (2 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Chuck Lever @ 2026-09-03 15:03 UTC (permalink / raw)
To: NeilBrown, Jeff Layton, Olga Kornievskaia, Dai Ngo, Tom Talpey; +Cc: linux-nfs
The pages directive so far drives only the encoder. A bulk argument
such as the payload of an NFS WRITE arrives in the pages of the
server's Receive buffer, but the generated opaque decoder copies it
into a contiguous data pointer. The copy wastes memory bandwidth and
forces the containing procedure decoder to be hand-rolled.
Let the directive drive the decoder as well. Whether a marked member
is decoded in place or encoded by reference follows from whether its
containing type is reachable from an RPC procedure argument. Record
that set during parsing, then close it over nested aggregates and
typedef aliases before the directives are checked. A decoded
page-resident member is represented as a struct xdr_buf so that the
payload can be handed on without a copy. A marked union arm of an
argument type is rejected, since the union generator still emits
only the copying decoder.
Signed-off-by: Chuck Lever <cel@kernel.org>
---
include/linux/sunrpc/xdrgen/_defs.h | 6 ++
tools/net/sunrpc/xdrgen/README | 32 +++++--
tools/net/sunrpc/xdrgen/generators/pointer.py | 39 ++++++--
tools/net/sunrpc/xdrgen/generators/struct.py | 39 ++++++--
tools/net/sunrpc/xdrgen/generators/union.py | 2 +-
.../C/pointer/decoder/pages_opaque.j2 | 6 ++
.../C/pointer/definition/pages_opaque.j2 | 5 +
.../C/struct/decoder/pages_opaque.j2 | 6 ++
.../C/struct/definition/pages_opaque.j2 | 5 +
tools/net/sunrpc/xdrgen/xdr_ast.py | 93 ++++++++++++++++++-
10 files changed, 204 insertions(+), 29 deletions(-)
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/pages_opaque.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/pointer/definition/pages_opaque.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/pages_opaque.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/definition/pages_opaque.j2
diff --git a/include/linux/sunrpc/xdrgen/_defs.h b/include/linux/sunrpc/xdrgen/_defs.h
index 8f3776ef3229..1e183b9d1c1d 100644
--- a/include/linux/sunrpc/xdrgen/_defs.h
+++ b/include/linux/sunrpc/xdrgen/_defs.h
@@ -5,11 +5,17 @@
* This header defines XDR data type primitives specified in
* Section 4 of RFC 4506, used by RPC programs implemented
* in the Linux kernel.
+ *
+ * A generated definition can also embed a struct xdr_buf, which
+ * carries a page-resident payload captured by reference, so this
+ * header pulls in the kernel's XDR types as well.
*/
#ifndef _SUNRPC_XDRGEN__DEFS_H_
#define _SUNRPC_XDRGEN__DEFS_H_
+#include <linux/sunrpc/xdr.h>
+
#define TRUE (true)
#define FALSE (false)
diff --git a/tools/net/sunrpc/xdrgen/README b/tools/net/sunrpc/xdrgen/README
index 5af9cf506637..40bfa1c27f04 100644
--- a/tools/net/sunrpc/xdrgen/README
+++ b/tools/net/sunrpc/xdrgen/README
@@ -259,13 +259,31 @@ The first marks the opaque data member of struct READ3resok; the
second marks the path arm of union readlinkres, where path is
"typedef string path<NFS_MAXPATHLEN>".
+Whether the directive drives the encoder or the decoder follows
+from how the containing type is used. A server decodes arguments
+and encodes results, so the directive on a member of a result type
+(such as READ3resok) marks a payload to encode by reference, while
+the directive on a member of an argument type (such as the data
+member of an NFS WRITE argument) marks a payload to decode in place.
+
+An argument type here is the struct named as an RPC procedure
+argument, any aggregate reachable from it, and any typedef alias of
+either. The directive on a union arm of such a type is rejected:
+the union generator emits only the ordinary copying decoder, so a
+page-resident union arm needs generator support that does not yet
+exist.
+
The emitted encoder reads only the marked member's length field to
determine how many octets of page content to insert; the member's
-data pointer is not read. The server and client manage payload
-pages differently, so this directive applies only to server-side
-code generation, and so far only to the encoding direction.
-Decode-side handling of a page-resident argument (for example, the
-payload of an NFS WRITE) remains hand-coded.
+data pointer is not read. The emitted decoder reads the length
+prefix and captures the payload octets by reference, leaving them in
+the pages of the Receive buffer. A decoded page-resident member is
+represented as a struct xdr_buf so that a later stage can hand the
+payload to the server's file system without a copy.
+
+The server and client manage payload pages differently, so this
+directive applies only to server-side code generation. Client-side
+handling of a page-resident member remains future work.
Future Work
@@ -282,8 +300,8 @@ Add more pragma directives:
* @skip -- do not decode, but rather skip, the specified argument
field
-Extend pragma pages to the decoding direction and to
-client-side code generation
+Extend pragma pages to client-side code generation, and to union
+arms of RPC procedure arguments
Enable something like a #include to dynamically insert the content
of other specification files
diff --git a/tools/net/sunrpc/xdrgen/generators/pointer.py b/tools/net/sunrpc/xdrgen/generators/pointer.py
index 448ebd684212..aae78b42d717 100644
--- a/tools/net/sunrpc/xdrgen/generators/pointer.py
+++ b/tools/net/sunrpc/xdrgen/generators/pointer.py
@@ -13,7 +13,7 @@ from xdr_ast import _XdrFixedLengthOpaque, _XdrVariableLengthOpaque
from xdr_ast import _XdrFixedLengthArray, _XdrVariableLengthArray
from xdr_ast import _XdrOptionalData, _XdrPointer, _XdrDeclaration
from xdr_ast import public_apis, get_header_name
-from xdr_ast import pages_members, pages_member_maxsize
+from xdr_ast import pages_members, pages_member_maxsize, pages_member_is_decoded
def emit_pointer_declaration(environment: Environment, node: _XdrPointer) -> None:
@@ -24,9 +24,13 @@ def emit_pointer_declaration(environment: Environment, node: _XdrPointer) -> Non
def emit_pointer_member_definition(
- environment: Environment, field: _XdrDeclaration
+ environment: Environment, field: _XdrDeclaration, struct_name: str, peer: str
) -> None:
"""Emit a definition for one field in an XDR struct"""
+ if peer == "server" and pages_member_is_decoded(struct_name, field.name):
+ template = get_jinja2_template(environment, "definition", "pages_opaque")
+ print(template.render(name=field.name))
+ return
if isinstance(field, _XdrBasic):
template = get_jinja2_template(environment, "definition", field.template)
print(
@@ -79,13 +83,15 @@ def emit_pointer_member_definition(
)
-def emit_pointer_definition(environment: Environment, node: _XdrPointer) -> None:
+def emit_pointer_definition(
+ environment: Environment, node: _XdrPointer, peer: str
+) -> None:
"""Emit a definition for an XDR pointer type"""
template = get_jinja2_template(environment, "definition", "open")
print(template.render(name=node.name))
for field in node.fields[0:-1]:
- emit_pointer_member_definition(environment, field)
+ emit_pointer_member_definition(environment, field, node.name, peer)
template = get_jinja2_template(environment, "definition", "close")
print(template.render(name=node.name))
@@ -95,8 +101,18 @@ def emit_pointer_member_decoder(
environment: Environment,
field: _XdrDeclaration,
struct_name: str,
+ peer: str,
) -> None:
"""Emit a decoder for one field in an XDR pointer"""
+ if peer == "server" and pages_member_is_decoded(struct_name, field.name):
+ template = get_jinja2_template(environment, "decoder", "pages_opaque")
+ print(
+ template.render(
+ name=field.name,
+ maxsize=pages_member_maxsize(field),
+ )
+ )
+ return
if isinstance(field, _XdrBasic):
template = get_jinja2_template(environment, "decoder", field.template)
print(
@@ -161,13 +177,15 @@ def emit_pointer_member_decoder(
)
-def emit_pointer_decoder(environment: Environment, node: _XdrPointer) -> None:
+def emit_pointer_decoder(
+ environment: Environment, node: _XdrPointer, peer: str
+) -> None:
"""Emit one decoder function for an XDR pointer type"""
template = get_jinja2_template(environment, "decoder", "open")
print(template.render(name=node.name))
for field in node.fields[0:-1]:
- emit_pointer_member_decoder(environment, field, node.name)
+ emit_pointer_member_decoder(environment, field, node.name, peer)
template = get_jinja2_template(environment, "decoder", "close")
print(template.render())
@@ -183,10 +201,13 @@ def emit_pointer_member_encoder(
if (struct_name, field.name) in pages_members:
if peer != "server":
raise NotImplementedError(
- "pragma pages is server-side encode-only; "
+ "pragma pages is server-side only; "
+ peer
+ " generation is not yet supported"
)
+ # Both representations carry the length field, which is all the
+ # pages encoder reads, so pages_member_is_decoded() does not
+ # gate this site.
template = get_jinja2_template(environment, "encoder", "pages_opaque")
print(
template.render(
@@ -296,11 +317,11 @@ class XdrPointerGenerator(SourceGenerator):
def emit_definition(self, node: _XdrPointer) -> None:
"""Emit one declaration for an XDR pointer type"""
- emit_pointer_definition(self.environment, node)
+ emit_pointer_definition(self.environment, node, self.peer)
def emit_decoder(self, node: _XdrPointer) -> None:
"""Emit one decoder function for an XDR pointer type"""
- emit_pointer_decoder(self.environment, node)
+ emit_pointer_decoder(self.environment, node, self.peer)
def emit_encoder(self, node: _XdrPointer) -> None:
"""Emit one encoder function for an XDR pointer type"""
diff --git a/tools/net/sunrpc/xdrgen/generators/struct.py b/tools/net/sunrpc/xdrgen/generators/struct.py
index 76fd3afbeea0..e71f8126806d 100644
--- a/tools/net/sunrpc/xdrgen/generators/struct.py
+++ b/tools/net/sunrpc/xdrgen/generators/struct.py
@@ -13,7 +13,7 @@ from xdr_ast import _XdrFixedLengthOpaque, _XdrVariableLengthOpaque
from xdr_ast import _XdrFixedLengthArray, _XdrVariableLengthArray
from xdr_ast import _XdrOptionalData, _XdrStruct, _XdrDeclaration
from xdr_ast import public_apis, get_header_name
-from xdr_ast import pages_members, pages_member_maxsize
+from xdr_ast import pages_members, pages_member_maxsize, pages_member_is_decoded
def emit_struct_declaration(environment: Environment, node: _XdrStruct) -> None:
@@ -24,9 +24,13 @@ def emit_struct_declaration(environment: Environment, node: _XdrStruct) -> None:
def emit_struct_member_definition(
- environment: Environment, field: _XdrDeclaration
+ environment: Environment, field: _XdrDeclaration, struct_name: str, peer: str
) -> None:
"""Emit a definition for one field in an XDR struct"""
+ if peer == "server" and pages_member_is_decoded(struct_name, field.name):
+ template = get_jinja2_template(environment, "definition", "pages_opaque")
+ print(template.render(name=field.name))
+ return
if isinstance(field, _XdrBasic):
template = get_jinja2_template(environment, "definition", field.template)
print(
@@ -79,13 +83,15 @@ def emit_struct_member_definition(
)
-def emit_struct_definition(environment: Environment, node: _XdrStruct) -> None:
+def emit_struct_definition(
+ environment: Environment, node: _XdrStruct, peer: str
+) -> None:
"""Emit one definition for an XDR struct type"""
template = get_jinja2_template(environment, "definition", "open")
print(template.render(name=node.name))
for field in node.fields:
- emit_struct_member_definition(environment, field)
+ emit_struct_member_definition(environment, field, node.name, peer)
template = get_jinja2_template(environment, "definition", "close")
print(template.render(name=node.name))
@@ -95,8 +101,18 @@ def emit_struct_member_decoder(
environment: Environment,
field: _XdrDeclaration,
struct_name: str,
+ peer: str,
) -> None:
"""Emit a decoder for one field in an XDR struct"""
+ if peer == "server" and pages_member_is_decoded(struct_name, field.name):
+ template = get_jinja2_template(environment, "decoder", "pages_opaque")
+ print(
+ template.render(
+ name=field.name,
+ maxsize=pages_member_maxsize(field),
+ )
+ )
+ return
if isinstance(field, _XdrBasic):
template = get_jinja2_template(environment, "decoder", field.template)
print(
@@ -161,13 +177,15 @@ def emit_struct_member_decoder(
)
-def emit_struct_decoder(environment: Environment, node: _XdrStruct) -> None:
+def emit_struct_decoder(
+ environment: Environment, node: _XdrStruct, peer: str
+) -> None:
"""Emit one decoder function for an XDR struct type"""
template = get_jinja2_template(environment, "decoder", "open")
print(template.render(name=node.name))
for field in node.fields:
- emit_struct_member_decoder(environment, field, node.name)
+ emit_struct_member_decoder(environment, field, node.name, peer)
template = get_jinja2_template(environment, "decoder", "close")
print(template.render())
@@ -183,10 +201,13 @@ def emit_struct_member_encoder(
if (struct_name, field.name) in pages_members:
if peer != "server":
raise NotImplementedError(
- "pragma pages is server-side encode-only; "
+ "pragma pages is server-side only; "
+ peer
+ " generation is not yet supported"
)
+ # Both representations carry the length field, which is all the
+ # pages encoder reads, so pages_member_is_decoded() does not
+ # gate this site.
template = get_jinja2_template(environment, "encoder", "pages_opaque")
print(
template.render(
@@ -294,11 +315,11 @@ class XdrStructGenerator(SourceGenerator):
def emit_definition(self, node: _XdrStruct) -> None:
"""Emit one definition for an XDR struct type"""
- emit_struct_definition(self.environment, node)
+ emit_struct_definition(self.environment, node, self.peer)
def emit_decoder(self, node: _XdrStruct) -> None:
"""Emit one decoder function for an XDR struct type"""
- emit_struct_decoder(self.environment, node)
+ emit_struct_decoder(self.environment, node, self.peer)
def emit_encoder(self, node: _XdrStruct) -> None:
"""Emit one encoder function for an XDR struct type"""
diff --git a/tools/net/sunrpc/xdrgen/generators/union.py b/tools/net/sunrpc/xdrgen/generators/union.py
index 47d63236f88f..e1e7d38fdf44 100644
--- a/tools/net/sunrpc/xdrgen/generators/union.py
+++ b/tools/net/sunrpc/xdrgen/generators/union.py
@@ -227,7 +227,7 @@ def emit_union_pages_arm_encoder(
return False
if peer != "server":
raise NotImplementedError(
- "pragma pages is server-side encode-only; "
+ "pragma pages is server-side only; "
+ peer
+ " generation is not yet supported"
)
diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/pages_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/pages_opaque.j2
new file mode 100644
index 000000000000..3b5ca6a5d977
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/pages_opaque.j2
@@ -0,0 +1,6 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+{% if annotate %}
+ /* member {{ name }} (page-resident variable-length opaque) */
+{% endif %}
+ if (!svcxdr_decode_opaque_payload(xdr, &ptr->{{ name }}, {{ maxsize }}))
+ return false;
diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/pages_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/pages_opaque.j2
new file mode 100644
index 000000000000..6e1a344d6665
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/pages_opaque.j2
@@ -0,0 +1,5 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+{% if annotate %}
+ /* member {{ name }} (page-resident variable-length opaque) */
+{% endif %}
+ struct xdr_buf {{ name }};
diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/pages_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/pages_opaque.j2
new file mode 100644
index 000000000000..3b5ca6a5d977
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/pages_opaque.j2
@@ -0,0 +1,6 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+{% if annotate %}
+ /* member {{ name }} (page-resident variable-length opaque) */
+{% endif %}
+ if (!svcxdr_decode_opaque_payload(xdr, &ptr->{{ name }}, {{ maxsize }}))
+ return false;
diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/definition/pages_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/pages_opaque.j2
new file mode 100644
index 000000000000..6e1a344d6665
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/pages_opaque.j2
@@ -0,0 +1,5 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+{% if annotate %}
+ /* member {{ name }} (page-resident variable-length opaque) */
+{% endif %}
+ struct xdr_buf {{ name }};
diff --git a/tools/net/sunrpc/xdrgen/xdr_ast.py b/tools/net/sunrpc/xdrgen/xdr_ast.py
index 81b40cef839c..c50c738ea131 100644
--- a/tools/net/sunrpc/xdrgen/xdr_ast.py
+++ b/tools/net/sunrpc/xdrgen/xdr_ast.py
@@ -20,14 +20,23 @@ structs = set()
pass_by_reference = set()
# (type_name, member_name) pairs marked "pragma pages": the member's
-# content resides in the pages of the Reply buffer, so the emitted
-# encoder inserts those pages by reference instead of copying.
+# content resides in the pages of the Receive or Reply buffer, so
+# the emitted codec captures or inserts those pages by reference
+# instead of copying.
pages_members = set()
# The same pairs, keyed to the directive's identifier node so a
# diagnostic can point at the directive rather than at the type.
pages_member_meta = {}
+# Names of types reachable from an RPC procedure argument, closed over
+# the types their members reach. A "pragma pages" member of one is
+# decoded in place rather than encoded by reference, and the two need
+# different C representations; see pages_member_is_decoded(). A type
+# reached from both an argument and a result takes the argument
+# representation on both paths.
+argument_types = set()
+
# Typedefs of a variable-length opaque or string, mapped to the length
# bound. A "pragma pages" member declared through one ("path data;"
# with "typedef string path<NFS_MAXPATHLEN>") reaches the AST as a
@@ -470,6 +479,20 @@ def pages_member_maxsize(field: _XdrDeclaration):
return None
+def pages_member_is_decoded(struct_name: str, member_name: str) -> bool:
+ """Return True when a "pragma pages" member is decoded in place.
+
+ A server decodes arguments and encodes results. A page-resident
+ member of an argument type is decoded from the pages of the
+ Receive buffer into an xdr_buf that captures the content by
+ reference; one of a result type is encoded from the Reply buffer's
+ pages and keeps the ordinary opaque or string representation.
+ """
+ return (struct_name, member_name) in pages_members and (
+ struct_name in argument_types
+ )
+
+
@dataclass
class _XdrCaseSpec(_XdrAst):
"""One case in an XDR union"""
@@ -783,6 +806,7 @@ class ParseToAst(Transformer):
argument = children[2]
number = children[3].value
+ argument_types.add(argument.type_name)
return _RpcProcedure(
ident.symbol,
number,
@@ -1062,7 +1086,7 @@ def check_pages_directives(root: "Specification") -> None:
Run in the front end so the diagnostic carries the directive's own
source position rather than surfacing as a traceback from whichever
emitter reaches the member. A directive that binds to nothing would
- otherwise degrade silently to the copying encoder.
+ otherwise degrade silently to the copying codec.
"""
payloads = {}
resolved = set()
@@ -1093,6 +1117,17 @@ def check_pages_directives(root: "Specification") -> None:
" declare it through a typedef instead",
meta,
)
+ # The union generator does not consult
+ # pages_member_is_decoded(), so a marked arm would reach the
+ # copying decoder and pull the payload through the bounded
+ # scratch buffer.
+ if isinstance(container, _XdrUnion) and type_name in argument_types:
+ raise XdrSemanticError(
+ f"union arm '{type_name}.{marked[1]}' cannot be"
+ " decoded in place; declare a page-resident member"
+ " of an argument type in a struct instead",
+ meta,
+ )
if pages_member_maxsize(field) is None:
raise XdrSemanticError(
f"'{type_name}.{marked[1]}' is neither a"
@@ -1120,12 +1155,64 @@ def check_pages_directives(root: "Specification") -> None:
)
+def _referenced_type_names(value) -> set:
+ """Return the type names an aggregate references through its
+ members."""
+ if isinstance(value, (_XdrStruct, _XdrPointer)):
+ fields = value.fields
+ elif isinstance(value, _XdrUnion):
+ fields = [case.arm for case in value.cases]
+ if value.default:
+ fields.append(value.default.arm)
+ else:
+ return set()
+ names = set()
+ for field in fields:
+ spec = getattr(field, "spec", None)
+ if spec is not None:
+ names.add(spec.type_name)
+ return names
+
+
+def _expand_argument_types(root: "Specification") -> None:
+ """Close argument_types over the types reachable from an RPC
+ argument.
+
+ A "pragma pages" member decoded in place can sit in a struct nested
+ within an argument type (symlinkdata3 within SYMLINK3args), so the
+ top-level argument types alone do not suffice.
+ """
+ references = {}
+ for definition in root.definitions:
+ value = definition.value
+ name = getattr(value, "name", None)
+ if name is not None:
+ references[name] = _referenced_type_names(value)
+ elif isinstance(value, _XdrTypedef):
+ # An argument named through an alias reaches whatever
+ # aggregate the alias resolves to.
+ declaration = value.declaration
+ spec = getattr(declaration, "spec", None)
+ if spec is not None:
+ references[declaration.name] = {spec.type_name}
+
+ worklist = list(argument_types)
+ while worklist:
+ for referenced in references.get(worklist.pop(), ()):
+ if referenced in references and referenced not in argument_types:
+ argument_types.add(referenced)
+ worklist.append(referenced)
+
+
def transform_parse_tree(parse_tree):
"""Transform productions into an abstract syntax tree"""
ast = transformer.transform(parse_tree)
ast.definitions = _merge_consecutive_passthru(ast.definitions)
check_duplicate_definitions(ast)
check_rpc_number_range(ast)
+ # The pages checks consult argument_types, so close that set over
+ # the nested aggregates before the directives are validated.
+ _expand_argument_types(ast)
check_pages_directives(ast)
return ast
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v1 08/10] xdrgen: Add hook-driven aggregate codec for variable-length arrays
2026-09-03 15:03 [PATCH v1 00/10] New pragmas for the xdrgen tool Chuck Lever
` (6 preceding siblings ...)
2026-09-03 15:03 ` [PATCH v1 07/10] xdrgen: Extend the pages directive to page-resident arguments Chuck Lever
@ 2026-09-03 15:03 ` Chuck Lever
2026-09-03 15:03 ` [PATCH v1 09/10] xdrgen: Extend the aggregate codec to optional-data list members Chuck Lever
2026-09-03 15:03 ` [PATCH v1 10/10] xdrgen: Stream optional-data aggregate lists during encode Chuck Lever
9 siblings, 0 replies; 11+ messages in thread
From: Chuck Lever @ 2026-09-03 15:03 UTC (permalink / raw)
To: NeilBrown, Jeff Layton, Olga Kornievskaia, Dai Ngo, Tom Talpey; +Cc: linux-nfs
A generated variable-length array codec stages the wire array
as a C array and walks it. When the in-kernel object is not that
array, building that array is wasted memory traffic. The NFS_ACLv2
secattr aclent<> list is one such case: it is a Solaris wire form of
the kernel's posix_acl. The hand-rolled codec that performs that
transform forces its containing types to keep external linkage
(pragma public), and that linkage collides at link time with the
same generated types in a sibling program.
Add a per-member "pragma aggregate <struct> <member>" directive. For
a marked member the generated code owns only the wire framing and
calls hand-written begin, item, and end hooks that handle one
element at a time. Interior codec signatures do not change.
The framing-owned loop follows xdr_encode_array2(), whose only
remaining user is fs/nfs_common/nfsacl.c, and seq_file's
start/show/stop, where a guaranteed stop() releases what start()
took. Those APIs take an ops vtable because one loop serves
consumers chosen at run time. Here the consumer is known at
generation time, so the hooks bind by name at link time, as the
ASN.1 compiler's actions do. A missing hook is a link error, the
emitted prototypes enforce the element type, and no indirect call
is added per element.
The item hooks return only success or failure, with no early "no
more elements" return, so that the same hook contract can later
frame the RFC 4506 optional-data idiom, where the application
pushes entries into generated framing.
The front end resolves each marker against the specification and
reports one it cannot honor against its own source position, rather
than degrading silently to the array-walking codec or surfacing as a
traceback from whichever emitter reached it. Absent the marker the
generator emits byte-identical output.
Signed-off-by: Chuck Lever <cel@kernel.org>
---
include/linux/sunrpc/xdrgen/_defs.h | 19 ++++
tools/net/sunrpc/xdrgen/README | 60 ++++++++++++-
tools/net/sunrpc/xdrgen/generators/struct.py | 89 ++++++++++++++++++-
tools/net/sunrpc/xdrgen/grammars/xdr.lark | 2 +
.../C/struct/declaration/aggregate_hooks.j2 | 14 +++
.../C/struct/decoder/aggregate_array.j2 | 35 ++++++++
.../C/struct/encoder/aggregate_array.j2 | 39 ++++++++
tools/net/sunrpc/xdrgen/xdr_ast.py | 80 +++++++++++++++++
8 files changed, 335 insertions(+), 3 deletions(-)
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/declaration/aggregate_hooks.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/aggregate_array.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_array.j2
diff --git a/include/linux/sunrpc/xdrgen/_defs.h b/include/linux/sunrpc/xdrgen/_defs.h
index 1e183b9d1c1d..f4375d73c5ca 100644
--- a/include/linux/sunrpc/xdrgen/_defs.h
+++ b/include/linux/sunrpc/xdrgen/_defs.h
@@ -29,6 +29,25 @@ typedef struct {
u8 *data;
} opaque;
+/*
+ * Cursor a hook-driven aggregate codec hands to its application hooks,
+ * one element at a time, in place of a materialized C array. The
+ * generated framing owns it. @xdr is the RPC layer's stream; @ctx is
+ * that stream's xdrgen_ctx, the svc_rqst on the server. @index is
+ * the current element and @count the wire array length: a decoder
+ * fills @count from the wire before the begin hook runs, an
+ * encoder's begin hook sets it. @member_id selects among a type's
+ * marked members. The begin/item/end contract is under "Pragma
+ * aggregate" in tools/net/sunrpc/xdrgen/README.
+ */
+struct xdrgen_aggregate_cursor {
+ struct xdr_stream *xdr;
+ u32 index;
+ u32 count;
+ unsigned int member_id;
+ void *ctx;
+};
+
#define XDR_void (0)
#define XDR_bool (1)
#define XDR_short (1)
diff --git a/tools/net/sunrpc/xdrgen/README b/tools/net/sunrpc/xdrgen/README
index 40bfa1c27f04..92578671b5e9 100644
--- a/tools/net/sunrpc/xdrgen/README
+++ b/tools/net/sunrpc/xdrgen/README
@@ -147,8 +147,64 @@ Pragmas
-------
Pragma directives specify exceptions to the normal generation of
-encoding and decoding functions. Currently one directive is
-implemented: "public".
+encoding and decoding functions. The directives are described
+below.
+
+Pragma aggregate
+------ ---------
+
+ pragma aggregate <struct> <member> ;
+
+A variable-length array member is normally encoded and decoded by
+staging it in a C array and walking that array. When the application
+object is not that array -- for instance when the member is a wire
+projection of a different in-kernel structure -- building it is
+wasted work. This directive marks such a member so that xdrgen emits a
+codec that owns only the wire framing (the length prefix, the bound
+check, and the per-element codec) and drives the application
+through begin/item/end hooks, processing one element at a time with
+no staged array. The marked member must be a variable-length
+array member of a struct; xdrgen rejects a directive naming
+anything else, and generates the codec for the server side only.
+
+For example:
+
+ pragma aggregate secattr aclent;
+ pragma aggregate secattr dfaclent;
+
+The hooks are supplied by hand. Their names derive from the pragma
+header and the containing struct, so that two programs sharing a
+type name resolve to distinct symbols. All marked members of one
+struct share a hook set; the cursor's member_id field selects the
+member:
+
+ nfs_acl2_secattr_encode_begin()
+ nfs_acl2_secattr_encode()
+ nfs_acl2_secattr_encode_end()
+ nfs_acl2_secattr_decode_begin()
+ nfs_acl2_secattr_decode()
+ nfs_acl2_secattr_decode_end()
+
+A decoder fills the cursor's count from the wire before calling the
+begin hook; an encoder's begin hook sets that count, and the framing
+then bound-checks it and emits the length prefix. Once a begin hook
+has succeeded its end hook runs, so that it can release what begin
+took; the end hook receives the running success flag.
+
+For each marked member xdrgen emits an enumeration constant -- the
+pragma header, struct, and member joined by underscores and
+upper-cased -- in the generated header, and the generated framing
+initializes the cursor's member_id with it. The hooks compare
+member_id against these constants rather than bare integers:
+
+ NFS_ACL2_SECATTR_ACLENT
+ NFS_ACL2_SECATTR_DFACLENT
+
+The hooks reach their application state through the xdr_stream's
+xdrgen_ctx pointer: svcxdr_init_decode() and svcxdr_init_encode()
+bind it to the svc_rqst, and the generated framing copies it into
+the cursor's ctx field, from which a hook reaches rq_argp or
+rq_resp. xdrgen emits the hook prototypes in the generated header.
Pragma big_endian
------ ----------
diff --git a/tools/net/sunrpc/xdrgen/generators/struct.py b/tools/net/sunrpc/xdrgen/generators/struct.py
index e71f8126806d..41030403b759 100644
--- a/tools/net/sunrpc/xdrgen/generators/struct.py
+++ b/tools/net/sunrpc/xdrgen/generators/struct.py
@@ -12,15 +12,58 @@ from xdr_ast import _XdrBasic, _XdrString
from xdr_ast import _XdrFixedLengthOpaque, _XdrVariableLengthOpaque
from xdr_ast import _XdrFixedLengthArray, _XdrVariableLengthArray
from xdr_ast import _XdrOptionalData, _XdrStruct, _XdrDeclaration
-from xdr_ast import public_apis, get_header_name
+from xdr_ast import public_apis, get_header_name, aggregate_members
from xdr_ast import pages_members, pages_member_maxsize, pages_member_is_decoded
+def aggregate_hook_base(struct_name: str) -> str:
+ """Return the application hook base name for a struct's aggregate members.
+
+ The pragma header name prefixes it so two programs that share a
+ type name (nfs_acl2 vs nfs_acl3) derive distinct external hook
+ symbols. All marked members of one struct share a hook set; the
+ cursor's member_id tells them apart.
+ """
+ return "_".join((get_header_name(), struct_name))
+
+
+def aggregate_member_symbol(struct_name: str, member_name: str) -> str:
+ """Return the symbolic member id for one marked aggregate member.
+
+ The generated framing sets the cursor's member_id to this constant
+ and the hooks compare against it. The enum in aggregate_hooks.j2
+ assigns the values implicitly in field order, so a hook that
+ compared against the bare integer would silently bind to the
+ wrong member once the specification's members were reordered.
+ """
+ return "_".join((aggregate_hook_base(struct_name), member_name)).upper()
+
+
def emit_struct_declaration(environment: Environment, node: _XdrStruct) -> None:
"""Emit one declaration pair for an XDR struct type"""
if node.name in public_apis:
template = get_jinja2_template(environment, "declaration", "close")
print(template.render(name=node.name))
+ marked = [
+ field
+ for field in node.fields
+ if (node.name, field.name) in aggregate_members
+ ]
+ if marked:
+ template = get_jinja2_template(
+ environment, "declaration", "aggregate_hooks"
+ )
+ print(
+ template.render(
+ hook=aggregate_hook_base(node.name),
+ c_type=kernel_c_type(marked[0].spec),
+ classifier=marked[0].spec.c_classifier,
+ members=[
+ aggregate_member_symbol(node.name, field.name)
+ for field in marked
+ ],
+ )
+ )
def emit_struct_member_definition(
@@ -113,6 +156,28 @@ def emit_struct_member_decoder(
)
)
return
+ if isinstance(field, _XdrVariableLengthArray) and (
+ (struct_name, field.name) in aggregate_members
+ ):
+ if peer != "server":
+ raise NotImplementedError(
+ "pragma aggregate is server-side only; "
+ + peer
+ + " generation is not yet supported"
+ )
+ template = get_jinja2_template(environment, "decoder", "aggregate_array")
+ print(
+ template.render(
+ name=field.name,
+ type=field.spec.type_name,
+ c_type=kernel_c_type(field.spec),
+ classifier=field.spec.c_classifier,
+ maxsize=field.maxsize,
+ hook=aggregate_hook_base(struct_name),
+ member_sym=aggregate_member_symbol(struct_name, field.name),
+ )
+ )
+ return
if isinstance(field, _XdrBasic):
template = get_jinja2_template(environment, "decoder", field.template)
print(
@@ -198,6 +263,28 @@ def emit_struct_member_encoder(
peer: str,
) -> None:
"""Emit an encoder for one field in an XDR struct"""
+ if isinstance(field, _XdrVariableLengthArray) and (
+ (struct_name, field.name) in aggregate_members
+ ):
+ if peer != "server":
+ raise NotImplementedError(
+ "pragma aggregate is server-side only; "
+ + peer
+ + " generation is not yet supported"
+ )
+ template = get_jinja2_template(environment, "encoder", "aggregate_array")
+ print(
+ template.render(
+ name=field.name,
+ type=field.spec.type_name,
+ c_type=kernel_c_type(field.spec),
+ classifier=field.spec.c_classifier,
+ maxsize=field.maxsize,
+ hook=aggregate_hook_base(struct_name),
+ member_sym=aggregate_member_symbol(struct_name, field.name),
+ )
+ )
+ return
if (struct_name, field.name) in pages_members:
if peer != "server":
raise NotImplementedError(
diff --git a/tools/net/sunrpc/xdrgen/grammars/xdr.lark b/tools/net/sunrpc/xdrgen/grammars/xdr.lark
index 1d2afff98ac5..64cd73df412c 100644
--- a/tools/net/sunrpc/xdrgen/grammars/xdr.lark
+++ b/tools/net/sunrpc/xdrgen/grammars/xdr.lark
@@ -100,6 +100,7 @@ directive : big_endian_directive
| pages_directive
| public_directive
| skip_directive
+ | aggregate_directive
big_endian_directive : "big_endian"
exclude_directive : "exclude"
@@ -107,6 +108,7 @@ header_directive : "header"
pages_directive : "pages"
public_directive : "public"
skip_directive : "skip"
+aggregate_directive : "aggregate"
//
// XDR language primitives
diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/declaration/aggregate_hooks.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/declaration/aggregate_hooks.j2
new file mode 100644
index 000000000000..b8b2b0766f8a
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/struct/declaration/aggregate_hooks.j2
@@ -0,0 +1,14 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+
+enum {
+{% for member in members %}
+ {{ member }},
+{% endfor %}
+};
+
+bool {{ hook }}_encode_begin(struct xdrgen_aggregate_cursor *c);
+bool {{ hook }}_encode(struct xdrgen_aggregate_cursor *c, {{ classifier }}{{ c_type }} *out);
+bool {{ hook }}_encode_end(struct xdrgen_aggregate_cursor *c, bool ok);
+bool {{ hook }}_decode_begin(struct xdrgen_aggregate_cursor *c);
+bool {{ hook }}_decode(struct xdrgen_aggregate_cursor *c, const {{ classifier }}{{ c_type }} *in);
+bool {{ hook }}_decode_end(struct xdrgen_aggregate_cursor *c, bool ok);
diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/aggregate_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/aggregate_array.j2
new file mode 100644
index 000000000000..9550e9cef30d
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/aggregate_array.j2
@@ -0,0 +1,35 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+{% if annotate %}
+ /* member {{ name }} (aggregate array) */
+{% endif %}
+ {
+ struct xdrgen_aggregate_cursor cursor = {
+ .xdr = xdr,
+ .member_id = {{ member_sym }},
+ .ctx = xdr->xdrgen_ctx,
+ };
+ bool ok = true;
+
+ if (xdr_stream_decode_u32(xdr, &cursor.count) < 0)
+ return false;
+{% if maxsize != "0" %}
+ if (cursor.count > {{ maxsize }})
+ return false;
+{% endif %}
+ if (!{{ hook }}_decode_begin(&cursor))
+ return false;
+ for (cursor.index = 0; cursor.index < cursor.count; cursor.index++) {
+ {{ classifier }}{{ c_type }} element = {};
+
+ if (!xdrgen_decode_{{ type }}(xdr, &element)) {
+ ok = false;
+ break;
+ }
+ if (!{{ hook }}_decode(&cursor, &element)) {
+ ok = false;
+ break;
+ }
+ }
+ if (!{{ hook }}_decode_end(&cursor, ok) || !ok)
+ return false;
+ }
diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_array.j2
new file mode 100644
index 000000000000..1fccd1771bd3
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_array.j2
@@ -0,0 +1,39 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+{% if annotate %}
+ /* member {{ name }} (aggregate array) */
+{% endif %}
+ {
+ struct xdrgen_aggregate_cursor cursor = {
+ .xdr = xdr,
+ .member_id = {{ member_sym }},
+ .ctx = xdr->xdrgen_ctx,
+ };
+ bool ok = true;
+
+ if (!{{ hook }}_encode_begin(&cursor))
+ return false;
+{% if maxsize != "0" %}
+ if (cursor.count > {{ maxsize }})
+ ok = false;
+{% endif %}
+ if (ok && xdr_stream_encode_u32(xdr, cursor.count) != XDR_UNIT)
+ ok = false;
+ for (cursor.index = 0; ok && cursor.index < cursor.count; cursor.index++) {
+ {{ classifier }}{{ c_type }} element = {};
+
+ if (!{{ hook }}_encode(&cursor, &element)) {
+ ok = false;
+ break;
+ }
+{% if type in pass_by_reference %}
+ if (!xdrgen_encode_{{ type }}(xdr, &element)) {
+{% else %}
+ if (!xdrgen_encode_{{ type }}(xdr, element)) {
+{% endif %}
+ ok = false;
+ break;
+ }
+ }
+ if (!{{ hook }}_encode_end(&cursor, ok) || !ok)
+ return false;
+ }
diff --git a/tools/net/sunrpc/xdrgen/xdr_ast.py b/tools/net/sunrpc/xdrgen/xdr_ast.py
index c50c738ea131..680a87e3bb39 100644
--- a/tools/net/sunrpc/xdrgen/xdr_ast.py
+++ b/tools/net/sunrpc/xdrgen/xdr_ast.py
@@ -19,6 +19,15 @@ public_apis = []
structs = set()
pass_by_reference = set()
+# (type_name, member_name) pairs whose variable-length array member is
+# marked "pragma aggregate" -- codec emission streams the member through
+# application hooks instead of iterating a materialized C array.
+aggregate_members = set()
+
+# Source position of each "pragma aggregate" marker, so a directive
+# that cannot be honored is reported where it was written.
+aggregate_member_meta = {}
+
# (type_name, member_name) pairs marked "pragma pages": the member's
# content resides in the pages of the Receive or Reply buffer, so
# the emitted codec captures or inserts those pages by reference
@@ -849,6 +858,15 @@ class ParseToAst(Transformer):
header_name = children[1].symbol
case "public_directive":
public_apis.append(children[1].symbol)
+ case "aggregate_directive":
+ if children[2] is None:
+ raise XdrSemanticError(
+ "pragma aggregate requires a type name and a member name",
+ children[1],
+ )
+ marked = (children[1].symbol, children[2].symbol)
+ aggregate_members.add(marked)
+ aggregate_member_meta[marked] = children[2]
case "pages_directive":
if children[2] is None:
raise XdrSemanticError(
@@ -1155,6 +1173,67 @@ def check_pages_directives(root: "Specification") -> None:
)
+def check_aggregate_directives(root: "Specification") -> None:
+ """Reject a "pragma aggregate" directive that cannot be honored.
+
+ As with the pages checks, this runs in the front end so a directive
+ naming a missing type or member, or a member with no hook-driven
+ codec, is reported at its own source position. Left to the
+ emitters, an unbound directive would degrade silently to the
+ materializing codec, and a malformed one would surface as a
+ traceback from whichever emitter reached it.
+ """
+ if aggregate_members and header_name == "none":
+ raise XdrSemanticError(
+ "pragma aggregate derives its external hook symbols from the"
+ " pragma header name, which this specification does not set",
+ aggregate_member_meta.get(min(aggregate_members)),
+ )
+
+ resolved = set()
+ for definition in root.definitions:
+ value = definition.value
+ if not isinstance(value, _XdrStruct):
+ continue
+ fields = dict((field.name, field) for field in value.fields)
+ element_type = None
+ for marked in sorted(aggregate_members):
+ if marked[0] != value.name:
+ continue
+ meta = aggregate_member_meta.get(marked)
+ if marked[1] not in fields:
+ raise XdrSemanticError(
+ f"type '{value.name}' has no member '{marked[1]}'",
+ meta,
+ )
+ field = fields[marked[1]]
+ # Only the counted-array framing is generated, so any other
+ # member form would emit hook prototypes that nothing calls.
+ if not isinstance(field, _XdrVariableLengthArray):
+ raise XdrSemanticError(
+ f"'{value.name}.{marked[1]}' is not a variable-length"
+ " array",
+ meta,
+ )
+ if element_type is None:
+ element_type = field.spec.type_name
+ elif field.spec.type_name != element_type:
+ raise XdrSemanticError(
+ f"'{value.name}.{marked[1]}' has element type"
+ f" '{field.spec.type_name}', but '{value.name}' already"
+ f" marks a member of element type '{element_type}';"
+ " one hook set serves all of a type's marked members",
+ meta,
+ )
+ resolved.add(marked)
+
+ for marked in sorted(aggregate_members - resolved):
+ raise XdrSemanticError(
+ f"pragma aggregate names unknown struct '{marked[0]}'",
+ aggregate_member_meta.get(marked),
+ )
+
+
def _referenced_type_names(value) -> set:
"""Return the type names an aggregate references through its
members."""
@@ -1214,6 +1293,7 @@ def transform_parse_tree(parse_tree):
# the nested aggregates before the directives are validated.
_expand_argument_types(ast)
check_pages_directives(ast)
+ check_aggregate_directives(ast)
return ast
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v1 09/10] xdrgen: Extend the aggregate codec to optional-data list members
2026-09-03 15:03 [PATCH v1 00/10] New pragmas for the xdrgen tool Chuck Lever
` (7 preceding siblings ...)
2026-09-03 15:03 ` [PATCH v1 08/10] xdrgen: Add hook-driven aggregate codec for variable-length arrays Chuck Lever
@ 2026-09-03 15:03 ` Chuck Lever
2026-09-03 15:03 ` [PATCH v1 10/10] xdrgen: Stream optional-data aggregate lists during encode Chuck Lever
9 siblings, 0 replies; 11+ messages in thread
From: Chuck Lever @ 2026-09-03 15:03 UTC (permalink / raw)
To: NeilBrown, Jeff Layton, Olga Kornievskaia, Dai Ngo, Tom Talpey; +Cc: linux-nfs
The hook-driven aggregate codec emits only the counted-array wire
form. XDR list types built on the optional-data idiom ("type *name")
carry no count and cannot be expressed that way. NFSv2 READDIR's
entry list is one such type.
Accept an optional-data member under "pragma aggregate" and emit
value-follows framing in place of the counted array. The element
type's existing single-node encoder already writes its own
value-follows boolean, so the loop calls it once per element and
once more with a NULL element to write the terminator. The struct C
definitions are untouched; only the marked member's framing changes.
The generated decoder is present for linkage only. Decoding a
value-follows list needs a presence-reporting element decoder, which
this change does not add. NFSv2 READDIR only encodes its entry list,
so the decode path is never exercised.
Signed-off-by: Chuck Lever <cel@kernel.org>
---
tools/net/sunrpc/xdrgen/README | 23 +++++++---
tools/net/sunrpc/xdrgen/generators/struct.py | 42 +++++++++++++++++++
.../C/struct/decoder/aggregate_optional.j2 | 31 ++++++++++++++
.../C/struct/encoder/aggregate_optional.j2 | 31 ++++++++++++++
tools/net/sunrpc/xdrgen/xdr_ast.py | 12 +++---
5 files changed, 128 insertions(+), 11 deletions(-)
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/decoder/aggregate_optional.j2
create mode 100644 tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_optional.j2
diff --git a/tools/net/sunrpc/xdrgen/README b/tools/net/sunrpc/xdrgen/README
index 92578671b5e9..ae8a8050698b 100644
--- a/tools/net/sunrpc/xdrgen/README
+++ b/tools/net/sunrpc/xdrgen/README
@@ -163,9 +163,18 @@ wasted work. This directive marks such a member so that xdrgen emits a
codec that owns only the wire framing (the length prefix, the bound
check, and the per-element codec) and drives the application
through begin/item/end hooks, processing one element at a time with
-no staged array. The marked member must be a variable-length
-array member of a struct; xdrgen rejects a directive naming
-anything else, and generates the codec for the server side only.
+no staged array. The marked member must be a struct member
+declared either as a variable-length array or as an optional-data
+list ("type *name"); xdrgen rejects a directive naming anything
+else, and generates the codec for the server side only.
+
+The two forms differ only in framing. A variable-length array
+carries a u32 element count, so the framing writes that count and
+bound-checks it. An optional-data list carries no count: each
+element is prefixed by a value-follows TRUE and the sequence is
+closed by a FALSE, which the framing writes after the last
+element. For the optional-data form xdrgen generates a working
+encoder only; the decoder it emits does not decode the list.
For example:
@@ -185,9 +194,11 @@ member:
nfs_acl2_secattr_decode()
nfs_acl2_secattr_decode_end()
-A decoder fills the cursor's count from the wire before calling the
-begin hook; an encoder's begin hook sets that count, and the framing
-then bound-checks it and emits the length prefix. Once a begin hook
+For a variable-length array, a decoder fills the cursor's count from
+the wire before calling the begin hook, and an encoder's begin hook
+sets that count for the framing to bound-check and emit. An
+optional-data list's encoder begin hook sets the count as well; its
+decoder never fills it. Once a begin hook
has succeeded its end hook runs, so that it can release what begin
took; the end hook receives the running success flag.
diff --git a/tools/net/sunrpc/xdrgen/generators/struct.py b/tools/net/sunrpc/xdrgen/generators/struct.py
index 41030403b759..945b9c36a701 100644
--- a/tools/net/sunrpc/xdrgen/generators/struct.py
+++ b/tools/net/sunrpc/xdrgen/generators/struct.py
@@ -178,6 +178,27 @@ def emit_struct_member_decoder(
)
)
return
+ if isinstance(field, _XdrOptionalData) and (
+ (struct_name, field.name) in aggregate_members
+ ):
+ if peer != "server":
+ raise NotImplementedError(
+ "pragma aggregate is server-side only; "
+ + peer
+ + " generation is not yet supported"
+ )
+ template = get_jinja2_template(environment, "decoder", "aggregate_optional")
+ print(
+ template.render(
+ name=field.name,
+ type=field.spec.type_name,
+ c_type=kernel_c_type(field.spec),
+ classifier=field.spec.c_classifier,
+ hook=aggregate_hook_base(struct_name),
+ member_sym=aggregate_member_symbol(struct_name, field.name),
+ )
+ )
+ return
if isinstance(field, _XdrBasic):
template = get_jinja2_template(environment, "decoder", field.template)
print(
@@ -285,6 +306,27 @@ def emit_struct_member_encoder(
)
)
return
+ if isinstance(field, _XdrOptionalData) and (
+ (struct_name, field.name) in aggregate_members
+ ):
+ if peer != "server":
+ raise NotImplementedError(
+ "pragma aggregate is server-side only; "
+ + peer
+ + " generation is not yet supported"
+ )
+ template = get_jinja2_template(environment, "encoder", "aggregate_optional")
+ print(
+ template.render(
+ name=field.name,
+ type=field.spec.type_name,
+ c_type=kernel_c_type(field.spec),
+ classifier=field.spec.c_classifier,
+ hook=aggregate_hook_base(struct_name),
+ member_sym=aggregate_member_symbol(struct_name, field.name),
+ )
+ )
+ return
if (struct_name, field.name) in pages_members:
if peer != "server":
raise NotImplementedError(
diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/aggregate_optional.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/aggregate_optional.j2
new file mode 100644
index 000000000000..add4813aa803
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/aggregate_optional.j2
@@ -0,0 +1,31 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+{# Placeholder: cannot decode value-follows framing, because the
+ element decoder does not report presence. #}
+{% if annotate %}
+ /* member {{ name }} (aggregate list) */
+{% endif %}
+ {
+ struct xdrgen_aggregate_cursor cursor = {
+ .xdr = xdr,
+ .member_id = {{ member_sym }},
+ .ctx = xdr->xdrgen_ctx,
+ };
+ bool ok = true;
+
+ if (!{{ hook }}_decode_begin(&cursor))
+ return false;
+ for (cursor.index = 0; cursor.index < cursor.count; cursor.index++) {
+ {{ classifier }}{{ c_type }} element = {};
+
+ if (!xdrgen_decode_{{ type }}(xdr, &element)) {
+ ok = false;
+ break;
+ }
+ if (!{{ hook }}_decode(&cursor, &element)) {
+ ok = false;
+ break;
+ }
+ }
+ if (!{{ hook }}_decode_end(&cursor, ok) || !ok)
+ return false;
+ }
diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_optional.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_optional.j2
new file mode 100644
index 000000000000..d45b823225f7
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_optional.j2
@@ -0,0 +1,31 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+{% if annotate %}
+ /* member {{ name }} (aggregate list) */
+{% endif %}
+ {
+ struct xdrgen_aggregate_cursor cursor = {
+ .xdr = xdr,
+ .member_id = {{ member_sym }},
+ .ctx = xdr->xdrgen_ctx,
+ };
+ bool ok = true;
+
+ if (!{{ hook }}_encode_begin(&cursor))
+ return false;
+ for (cursor.index = 0; cursor.index < cursor.count; cursor.index++) {
+ {{ classifier }}{{ c_type }} element = {};
+
+ if (!{{ hook }}_encode(&cursor, &element)) {
+ ok = false;
+ break;
+ }
+ if (!xdrgen_encode_{{ type }}(xdr, &element)) {
+ ok = false;
+ break;
+ }
+ }
+ if (ok && !xdrgen_encode_{{ type }}(xdr, NULL))
+ ok = false;
+ if (!{{ hook }}_encode_end(&cursor, ok) || !ok)
+ return false;
+ }
diff --git a/tools/net/sunrpc/xdrgen/xdr_ast.py b/tools/net/sunrpc/xdrgen/xdr_ast.py
index 680a87e3bb39..1fbfae49e0cd 100644
--- a/tools/net/sunrpc/xdrgen/xdr_ast.py
+++ b/tools/net/sunrpc/xdrgen/xdr_ast.py
@@ -1207,12 +1207,14 @@ def check_aggregate_directives(root: "Specification") -> None:
meta,
)
field = fields[marked[1]]
- # Only the counted-array framing is generated, so any other
- # member form would emit hook prototypes that nothing calls.
- if not isinstance(field, _XdrVariableLengthArray):
+ # Any other member form would emit hook prototypes that
+ # nothing calls.
+ if not isinstance(
+ field, (_XdrVariableLengthArray, _XdrOptionalData)
+ ):
raise XdrSemanticError(
- f"'{value.name}.{marked[1]}' is not a variable-length"
- " array",
+ f"'{value.name}.{marked[1]}' is neither a"
+ " variable-length array nor an optional-data list",
meta,
)
if element_type is None:
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v1 10/10] xdrgen: Stream optional-data aggregate lists during encode
2026-09-03 15:03 [PATCH v1 00/10] New pragmas for the xdrgen tool Chuck Lever
` (8 preceding siblings ...)
2026-09-03 15:03 ` [PATCH v1 09/10] xdrgen: Extend the aggregate codec to optional-data list members Chuck Lever
@ 2026-09-03 15:03 ` Chuck Lever
9 siblings, 0 replies; 11+ messages in thread
From: Chuck Lever @ 2026-09-03 15:03 UTC (permalink / raw)
To: NeilBrown, Jeff Layton, Olga Kornievskaia, Dai Ngo, Tom Talpey; +Cc: linux-nfs
The optional-data aggregate encoder loops cursor.count times, so the
begin hook must report the element count before the first element is
written. A producer that reads its elements while the reply is
encoded -- NFSv2 READDIR streaming directory entries straight into the
send buffer -- does not know that count in advance, and would have to
materialize the whole list just to satisfy the loop bound.
An optional-data list carries no count on the wire, so nothing forces
one on the encoder. Drive it from the item hook instead: a false
return ends the list, and the framing writes the terminator.
The counted-array form and its hook contract are unchanged. No
in-tree spec marks an optional-data member yet, so no generated code
changes here.
Signed-off-by: Chuck Lever <cel@kernel.org>
---
include/linux/sunrpc/xdrgen/_defs.h | 25 ++++++++++++-------
tools/net/sunrpc/xdrgen/README | 11 +++++---
.../C/struct/encoder/aggregate_optional.j2 | 6 ++---
3 files changed, 26 insertions(+), 16 deletions(-)
diff --git a/include/linux/sunrpc/xdrgen/_defs.h b/include/linux/sunrpc/xdrgen/_defs.h
index f4375d73c5ca..be77e9c57296 100644
--- a/include/linux/sunrpc/xdrgen/_defs.h
+++ b/include/linux/sunrpc/xdrgen/_defs.h
@@ -30,15 +30,22 @@ typedef struct {
} opaque;
/*
- * Cursor a hook-driven aggregate codec hands to its application hooks,
- * one element at a time, in place of a materialized C array. The
- * generated framing owns it. @xdr is the RPC layer's stream; @ctx is
- * that stream's xdrgen_ctx, the svc_rqst on the server. @index is
- * the current element and @count the wire array length: a decoder
- * fills @count from the wire before the begin hook runs, an
- * encoder's begin hook sets it. @member_id selects among a type's
- * marked members. The begin/item/end contract is under "Pragma
- * aggregate" in tools/net/sunrpc/xdrgen/README.
+ * Cursor a hook-driven aggregate codec hands to its begin/item/end
+ * hooks, one element at a time, in place of a materialized C array.
+ * The generated framing owns it. @xdr is the RPC layer's stream and
+ * @ctx its xdrgen_ctx, the svc_rqst on the server. @index is the
+ * current element; @member_id selects among a type's marked members.
+ * For a counted array @count is the wire length: a decoder fills it
+ * before the begin hook runs, an encoder's begin hook sets it. The
+ * hook contract is under "Pragma aggregate" in
+ * tools/net/sunrpc/xdrgen/README.
+ *
+ * An optional-data list ("type *name") carries no count. Its encoder
+ * pulls elements until the item hook returns false. False ends the
+ * list rather than failing the encode: the framing writes the
+ * value-follows terminator, so a producer that stops early leaves a
+ * valid truncated list on the wire. A producer that stops for a
+ * mid-list error reports that error through @ctx.
*/
struct xdrgen_aggregate_cursor {
struct xdr_stream *xdr;
diff --git a/tools/net/sunrpc/xdrgen/README b/tools/net/sunrpc/xdrgen/README
index ae8a8050698b..958d068bcad0 100644
--- a/tools/net/sunrpc/xdrgen/README
+++ b/tools/net/sunrpc/xdrgen/README
@@ -173,8 +173,13 @@ carries a u32 element count, so the framing writes that count and
bound-checks it. An optional-data list carries no count: each
element is prefixed by a value-follows TRUE and the sequence is
closed by a FALSE, which the framing writes after the last
-element. For the optional-data form xdrgen generates a working
-encoder only; the decoder it emits does not decode the list.
+element. That form has no count to loop on, so its encoder pulls
+elements until the item hook returns false. False ends the list
+rather than failing the encode: a producer stops when its source
+is exhausted or when a reply budget fills. For the optional-data
+form xdrgen
+generates a working encoder only; the decoder it emits does not
+decode the list.
For example:
@@ -197,7 +202,7 @@ member:
For a variable-length array, a decoder fills the cursor's count from
the wire before calling the begin hook, and an encoder's begin hook
sets that count for the framing to bound-check and emit. An
-optional-data list's encoder begin hook sets the count as well; its
+optional-data list's encoder does not read the count and its
decoder never fills it. Once a begin hook
has succeeded its end hook runs, so that it can release what begin
took; the end hook receives the running success flag.
diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_optional.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_optional.j2
index d45b823225f7..4f35b6976109 100644
--- a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_optional.j2
+++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/aggregate_optional.j2
@@ -12,13 +12,11 @@
if (!{{ hook }}_encode_begin(&cursor))
return false;
- for (cursor.index = 0; cursor.index < cursor.count; cursor.index++) {
+ for (cursor.index = 0; ; cursor.index++) {
{{ classifier }}{{ c_type }} element = {};
- if (!{{ hook }}_encode(&cursor, &element)) {
- ok = false;
+ if (!{{ hook }}_encode(&cursor, &element))
break;
- }
if (!xdrgen_encode_{{ type }}(xdr, &element)) {
ok = false;
break;
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-09-03 15:04 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-03 15:03 [PATCH v1 00/10] New pragmas for the xdrgen tool Chuck Lever
2026-09-03 15:03 ` [PATCH v1 01/10] SUNRPC: Carry a generated-codec context pointer in struct xdr_stream Chuck Lever
2026-09-03 15:03 ` [PATCH v1 02/10] SUNRPC: Bind the svc_rqst to its XDR streams Chuck Lever
2026-09-03 15:03 ` [PATCH v1 03/10] SUNRPC: Add svcxdr_encode_opaque_payload() Chuck Lever
2026-09-03 15:03 ` [PATCH v1 04/10] xdrgen: Pass the containing struct name to member codec emitters Chuck Lever
2026-09-03 15:03 ` [PATCH v1 05/10] xdrgen: Add a "pragma pages" directive Chuck Lever
2026-09-03 15:03 ` [PATCH v1 06/10] SUNRPC: Add svcxdr_decode_opaque_payload() Chuck Lever
2026-09-03 15:03 ` [PATCH v1 07/10] xdrgen: Extend the pages directive to page-resident arguments Chuck Lever
2026-09-03 15:03 ` [PATCH v1 08/10] xdrgen: Add hook-driven aggregate codec for variable-length arrays Chuck Lever
2026-09-03 15:03 ` [PATCH v1 09/10] xdrgen: Extend the aggregate codec to optional-data list members Chuck Lever
2026-09-03 15:03 ` [PATCH v1 10/10] xdrgen: Stream optional-data aggregate lists during encode Chuck Lever
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox