[PATCH nft 1/2] parser_json: use stdin buffer if available

[PATCH nft 1/2] parser_json: use stdin buffer if available

[Pablo Neira Ayuso]

Since 5c2b2b0a2ba7 ("src: error reporting with -f and read from stdin")
stdin is stored in a buffer, update json support to use it instead of
reading from /dev/stdin.

Some systems do not provide /dev/stdin symlink to /proc/self/fd/0
according to reporter (that mentions Yocto Linux as example).

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/parser_json.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/src/parser_json.c b/src/parser_json.c
index ee4657ee8044..4912d3608b2b 100644
--- a/src/parser_json.c
+++ b/src/parser_json.c
@@ -4357,6 +4357,13 @@ int nft_parse_json_filename(struct nft_ctx *nft, const char *filename,
 	json_error_t err;
 	int ret;
 
+	if (nft->stdin_buf) {
+		json_indesc.type = INDESC_STDIN;
+		json_indesc.name = "/dev/stdin";
+
+		return nft_parse_json_buffer(nft, nft->stdin_buf, msgs, cmds);
+	}
+
 	json_indesc.type = INDESC_FILE;
 	json_indesc.name = filename;
 
-- 
2.30.2

Re: [PATCH nft 1/2] parser_json: use stdin buffer if available

[Phil Sutter]

Hi Pablo,

On Tue, Jul 09, 2024 at 04:59:52PM +0200, Pablo Neira Ayuso wrote:
> Since 5c2b2b0a2ba7 ("src: error reporting with -f and read from stdin")
> stdin is stored in a buffer, update json support to use it instead of
> reading from /dev/stdin.
> 
> Some systems do not provide /dev/stdin symlink to /proc/self/fd/0
> according to reporter (that mentions Yocto Linux as example).
> 
> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> ---
>  src/parser_json.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/src/parser_json.c b/src/parser_json.c
> index ee4657ee8044..4912d3608b2b 100644
> --- a/src/parser_json.c
> +++ b/src/parser_json.c
> @@ -4357,6 +4357,13 @@ int nft_parse_json_filename(struct nft_ctx *nft, const char *filename,
>  	json_error_t err;
>  	int ret;
>  
> +	if (nft->stdin_buf) {
> +		json_indesc.type = INDESC_STDIN;
> +		json_indesc.name = "/dev/stdin";
> +
> +		return nft_parse_json_buffer(nft, nft->stdin_buf, msgs, cmds);
> +	}

Is this sufficient? In nft_run_cmd_from_filename(), nft->stdin_buf is
populated conditionally:

| if (!strcmp(filename, "/dev/stdin") &&
|     !nft_output_json(&nft->output))
|         nft->stdin_buf = stdin_to_buffer();

Later (in the wrapped __nft_run_cmd_from_filename()), we try JSON parsing
conditionally:

| if (nft_output_json(&nft->output) || nft_input_json(&nft->input))
|         rc = nft_parse_json_filename(nft, filename, &msgs, &cmds);

Things got complicated by commit 2034d8c60ed91 ("src: add input flag
NFT_CTX_INPUT_JSON to enable JSON parsing") and my request to remain
compatible, i.e. '-j' flag which enables JSON output shall continue to
make JSON the assumed input format.

So long story short, I guess in order to cover all cases, we have to
enable nft->stdin_buf population also if nft_input_json(...) returns
true, i.e. cover for library users requesting JSON input (but standard
output). WDYT?

Cheers, Phil

Re: [PATCH nft 1/2] parser_json: use stdin buffer if available

[Phil Sutter]

On Wed, Jul 10, 2024 at 03:53:52PM +0200, Phil Sutter wrote:
> Hi Pablo,
> 
> On Tue, Jul 09, 2024 at 04:59:52PM +0200, Pablo Neira Ayuso wrote:
> > Since 5c2b2b0a2ba7 ("src: error reporting with -f and read from stdin")
> > stdin is stored in a buffer, update json support to use it instead of
> > reading from /dev/stdin.
> > 
> > Some systems do not provide /dev/stdin symlink to /proc/self/fd/0
> > according to reporter (that mentions Yocto Linux as example).
> > 
> > Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> > ---
> >  src/parser_json.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/src/parser_json.c b/src/parser_json.c
> > index ee4657ee8044..4912d3608b2b 100644
> > --- a/src/parser_json.c
> > +++ b/src/parser_json.c
> > @@ -4357,6 +4357,13 @@ int nft_parse_json_filename(struct nft_ctx *nft, const char *filename,
> >  	json_error_t err;
> >  	int ret;
> >  
> > +	if (nft->stdin_buf) {
> > +		json_indesc.type = INDESC_STDIN;
> > +		json_indesc.name = "/dev/stdin";
> > +
> > +		return nft_parse_json_buffer(nft, nft->stdin_buf, msgs, cmds);
> > +	}
> 
> Is this sufficient? In nft_run_cmd_from_filename(), nft->stdin_buf is
> populated conditionally:
> 
> | if (!strcmp(filename, "/dev/stdin") &&
> |     !nft_output_json(&nft->output))
> |         nft->stdin_buf = stdin_to_buffer();
> 
> Later (in the wrapped __nft_run_cmd_from_filename()), we try JSON parsing
> conditionally:
> 
> | if (nft_output_json(&nft->output) || nft_input_json(&nft->input))
> |         rc = nft_parse_json_filename(nft, filename, &msgs, &cmds);
> 
> Things got complicated by commit 2034d8c60ed91 ("src: add input flag
> NFT_CTX_INPUT_JSON to enable JSON parsing") and my request to remain
> compatible, i.e. '-j' flag which enables JSON output shall continue to
> make JSON the assumed input format.
> 
> So long story short, I guess in order to cover all cases, we have to
> enable nft->stdin_buf population also if nft_input_json(...) returns
> true, i.e. cover for library users requesting JSON input (but standard
> output). WDYT?

On second review, I think the right change is to make
nft_run_cmd_from_filename() *always* populate nft->stdin_buf if
'filename' is '/dev/stdin', i.e. drop the !nft_output_json(...) clause.

Sorry for the confusion.

Cheers, Phil

Re: [PATCH nft 1/2] parser_json: use stdin buffer if available

[Pablo Neira Ayuso]

[-- Attachment #1: Type: text/plain, Size: 2534 bytes --]

On Wed, Jul 10, 2024 at 04:01:19PM +0200, Phil Sutter wrote:
> On Wed, Jul 10, 2024 at 03:53:52PM +0200, Phil Sutter wrote:
> > Hi Pablo,
> > 
> > On Tue, Jul 09, 2024 at 04:59:52PM +0200, Pablo Neira Ayuso wrote:
> > > Since 5c2b2b0a2ba7 ("src: error reporting with -f and read from stdin")
> > > stdin is stored in a buffer, update json support to use it instead of
> > > reading from /dev/stdin.
> > > 
> > > Some systems do not provide /dev/stdin symlink to /proc/self/fd/0
> > > according to reporter (that mentions Yocto Linux as example).
> > > 
> > > Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> > > ---
> > >  src/parser_json.c | 7 +++++++
> > >  1 file changed, 7 insertions(+)
> > > 
> > > diff --git a/src/parser_json.c b/src/parser_json.c
> > > index ee4657ee8044..4912d3608b2b 100644
> > > --- a/src/parser_json.c
> > > +++ b/src/parser_json.c
> > > @@ -4357,6 +4357,13 @@ int nft_parse_json_filename(struct nft_ctx *nft, const char *filename,
> > >  	json_error_t err;
> > >  	int ret;
> > >  
> > > +	if (nft->stdin_buf) {
> > > +		json_indesc.type = INDESC_STDIN;
> > > +		json_indesc.name = "/dev/stdin";
> > > +
> > > +		return nft_parse_json_buffer(nft, nft->stdin_buf, msgs, cmds);
> > > +	}
> > 
> > Is this sufficient? In nft_run_cmd_from_filename(), nft->stdin_buf is
> > populated conditionally:
> > 
> > | if (!strcmp(filename, "/dev/stdin") &&
> > |     !nft_output_json(&nft->output))
> > |         nft->stdin_buf = stdin_to_buffer();
> > 
> > Later (in the wrapped __nft_run_cmd_from_filename()), we try JSON parsing
> > conditionally:
> > 
> > | if (nft_output_json(&nft->output) || nft_input_json(&nft->input))
> > |         rc = nft_parse_json_filename(nft, filename, &msgs, &cmds);
> > 
> > Things got complicated by commit 2034d8c60ed91 ("src: add input flag
> > NFT_CTX_INPUT_JSON to enable JSON parsing") and my request to remain
> > compatible, i.e. '-j' flag which enables JSON output shall continue to
> > make JSON the assumed input format.
> > 
> > So long story short, I guess in order to cover all cases, we have to
> > enable nft->stdin_buf population also if nft_input_json(...) returns
> > true, i.e. cover for library users requesting JSON input (but standard
> > output). WDYT?
> 
> On second review, I think the right change is to make
> nft_run_cmd_from_filename() *always* populate nft->stdin_buf if
> 'filename' is '/dev/stdin', i.e. drop the !nft_output_json(...) clause.
> 
> Sorry for the confusion.

I can squash this incremental fix to 1/2 send post a v2.

Thanks.

[-- Attachment #2: fix.patch --]
[-- Type: text/x-diff, Size: 476 bytes --]

diff --git a/src/libnftables.c b/src/libnftables.c
index af4734c05004..89317f9f6049 100644
--- a/src/libnftables.c
+++ b/src/libnftables.c
@@ -807,8 +807,7 @@ int nft_run_cmd_from_filename(struct nft_ctx *nft, const char *filename)
 	if (!strcmp(filename, "-"))
 		filename = "/dev/stdin";
 
-	if (!strcmp(filename, "/dev/stdin") &&
-	    !nft_output_json(&nft->output))
+	if (!strcmp(filename, "/dev/stdin"))
 		nft->stdin_buf = stdin_to_buffer();
 
 	if (!nft->stdin_buf &&

Re: [PATCH nft 1/2] parser_json: use stdin buffer if available

[Phil Sutter]

On Wed, Jul 10, 2024 at 04:04:25PM +0200, Pablo Neira Ayuso wrote:
> On Wed, Jul 10, 2024 at 04:01:19PM +0200, Phil Sutter wrote:
> > On Wed, Jul 10, 2024 at 03:53:52PM +0200, Phil Sutter wrote:
> > > Hi Pablo,
> > > 
> > > On Tue, Jul 09, 2024 at 04:59:52PM +0200, Pablo Neira Ayuso wrote:
> > > > Since 5c2b2b0a2ba7 ("src: error reporting with -f and read from stdin")
> > > > stdin is stored in a buffer, update json support to use it instead of
> > > > reading from /dev/stdin.
> > > > 
> > > > Some systems do not provide /dev/stdin symlink to /proc/self/fd/0
> > > > according to reporter (that mentions Yocto Linux as example).
> > > > 
> > > > Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> > > > ---
> > > >  src/parser_json.c | 7 +++++++
> > > >  1 file changed, 7 insertions(+)
> > > > 
> > > > diff --git a/src/parser_json.c b/src/parser_json.c
> > > > index ee4657ee8044..4912d3608b2b 100644
> > > > --- a/src/parser_json.c
> > > > +++ b/src/parser_json.c
> > > > @@ -4357,6 +4357,13 @@ int nft_parse_json_filename(struct nft_ctx *nft, const char *filename,
> > > >  	json_error_t err;
> > > >  	int ret;
> > > >  
> > > > +	if (nft->stdin_buf) {
> > > > +		json_indesc.type = INDESC_STDIN;
> > > > +		json_indesc.name = "/dev/stdin";
> > > > +
> > > > +		return nft_parse_json_buffer(nft, nft->stdin_buf, msgs, cmds);
> > > > +	}
> > > 
> > > Is this sufficient? In nft_run_cmd_from_filename(), nft->stdin_buf is
> > > populated conditionally:
> > > 
> > > | if (!strcmp(filename, "/dev/stdin") &&
> > > |     !nft_output_json(&nft->output))
> > > |         nft->stdin_buf = stdin_to_buffer();
> > > 
> > > Later (in the wrapped __nft_run_cmd_from_filename()), we try JSON parsing
> > > conditionally:
> > > 
> > > | if (nft_output_json(&nft->output) || nft_input_json(&nft->input))
> > > |         rc = nft_parse_json_filename(nft, filename, &msgs, &cmds);
> > > 
> > > Things got complicated by commit 2034d8c60ed91 ("src: add input flag
> > > NFT_CTX_INPUT_JSON to enable JSON parsing") and my request to remain
> > > compatible, i.e. '-j' flag which enables JSON output shall continue to
> > > make JSON the assumed input format.
> > > 
> > > So long story short, I guess in order to cover all cases, we have to
> > > enable nft->stdin_buf population also if nft_input_json(...) returns
> > > true, i.e. cover for library users requesting JSON input (but standard
> > > output). WDYT?
> > 
> > On second review, I think the right change is to make
> > nft_run_cmd_from_filename() *always* populate nft->stdin_buf if
> > 'filename' is '/dev/stdin', i.e. drop the !nft_output_json(...) clause.
> > 
> > Sorry for the confusion.
> 
> I can squash this incremental fix to 1/2 send post a v2.

Acked-by: Phil Sutter <phil@nwl.cc>

Thanks, Phil

[PATCH nft 1/2] parser_json: use stdin buffer if available

[Pablo Neira Ayuso]

Since 5c2b2b0a2ba7 ("src: error reporting with -f and read from stdin")
stdin is stored in a buffer, update json support to use it instead of
reading from /dev/stdin.

Some systems do not provide /dev/stdin symlink to /proc/self/fd/0
according to reporter (that mentions Yocto Linux as example).

Fixes: 935f82e7dd49 ("Support 'nft -f -' to read from stdin")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
v2: remove check for nft_output_json() in nft_run_cmd_from_filename()
    as suggested by Phil Sutter, so JSON support does not really use
    /dev/stdin.

 src/libnftables.c | 3 +--
 src/parser_json.c | 7 +++++++
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/src/libnftables.c b/src/libnftables.c
index af4734c05004..89317f9f6049 100644
--- a/src/libnftables.c
+++ b/src/libnftables.c
@@ -807,8 +807,7 @@ int nft_run_cmd_from_filename(struct nft_ctx *nft, const char *filename)
 	if (!strcmp(filename, "-"))
 		filename = "/dev/stdin";
 
-	if (!strcmp(filename, "/dev/stdin") &&
-	    !nft_output_json(&nft->output))
+	if (!strcmp(filename, "/dev/stdin"))
 		nft->stdin_buf = stdin_to_buffer();
 
 	if (!nft->stdin_buf &&
diff --git a/src/parser_json.c b/src/parser_json.c
index ee4657ee8044..4912d3608b2b 100644
--- a/src/parser_json.c
+++ b/src/parser_json.c
@@ -4357,6 +4357,13 @@ int nft_parse_json_filename(struct nft_ctx *nft, const char *filename,
 	json_error_t err;
 	int ret;
 
+	if (nft->stdin_buf) {
+		json_indesc.type = INDESC_STDIN;
+		json_indesc.name = "/dev/stdin";
+
+		return nft_parse_json_buffer(nft, nft->stdin_buf, msgs, cmds);
+	}
+
 	json_indesc.type = INDESC_FILE;
 	json_indesc.name = filename;
 
-- 
2.30.2

[PATCH nft 2/2] libnftables: skip useable checks for /dev/stdin

[Pablo Neira Ayuso]

/dev/stdin is a placeholder, read() from STDIN_FILENO is used to fetch
the standard input into a buffer.

Since 5c2b2b0a2ba7 ("src: error reporting with -f and read from stdin")
stdin is stored in a buffer to fix error reporting.

This patch requires: ("parser_json: use stdin buffer if available")

Fixes: 149b1c95d129 ("libnftables: refuse to open onput files other than named pipes or regular files")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
v2: no changes.

 src/libnftables.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/src/libnftables.c b/src/libnftables.c
index 89317f9f6049..36d6a854ff50 100644
--- a/src/libnftables.c
+++ b/src/libnftables.c
@@ -664,6 +664,7 @@ retry:
 
 /* need to use stat() to, fopen() will block for named fifos and
  * libjansson makes no checks before or after open either.
+ * /dev/stdin is *never* used, read() from STDIN_FILENO is used instead.
  */
 static struct error_record *filename_is_useable(struct nft_ctx *nft, const char *name)
 {
@@ -671,6 +672,9 @@ static struct error_record *filename_is_useable(struct nft_ctx *nft, const char
 	struct stat sb;
 	int err;
 
+	if (!strcmp(name, "/dev/stdin"))
+		return NULL;
+
 	err = stat(name, &sb);
 	if (err)
 		return error(&internal_location, "Could not open file \"%s\": %s\n",
@@ -681,9 +685,6 @@ static struct error_record *filename_is_useable(struct nft_ctx *nft, const char
 	if (type == S_IFREG || type == S_IFIFO)
 		return NULL;
 
-	if (type == S_IFCHR && 0 == strcmp(name, "/dev/stdin"))
-		return NULL;
-
 	return error(&internal_location, "Not a regular file: \"%s\"\n", name);
 }
 
-- 
2.30.2

Re: [PATCH nft 1/2] parser_json: use stdin buffer if available

[Phil Sutter]

On Wed, Jul 10, 2024 at 05:20:03PM +0200, Pablo Neira Ayuso wrote:
> Since 5c2b2b0a2ba7 ("src: error reporting with -f and read from stdin")
> stdin is stored in a buffer, update json support to use it instead of
> reading from /dev/stdin.
> 
> Some systems do not provide /dev/stdin symlink to /proc/self/fd/0
> according to reporter (that mentions Yocto Linux as example).
> 
> Fixes: 935f82e7dd49 ("Support 'nft -f -' to read from stdin")
> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> ---
> v2: remove check for nft_output_json() in nft_run_cmd_from_filename()
>     as suggested by Phil Sutter, so JSON support does not really use
>     /dev/stdin.

Series:
Acked-by: Phil Sutter <phil@nwl.cc>

Thanks, Phil

Re: [PATCH nft 1/2] parser_json: use stdin buffer if available

[Pablo Neira Ayuso]

On Wed, Jul 10, 2024 at 05:52:21PM +0200, Phil Sutter wrote:
> On Wed, Jul 10, 2024 at 05:20:03PM +0200, Pablo Neira Ayuso wrote:
> > Since 5c2b2b0a2ba7 ("src: error reporting with -f and read from stdin")
> > stdin is stored in a buffer, update json support to use it instead of
> > reading from /dev/stdin.
> > 
> > Some systems do not provide /dev/stdin symlink to /proc/self/fd/0
> > according to reporter (that mentions Yocto Linux as example).
> > 
> > Fixes: 935f82e7dd49 ("Support 'nft -f -' to read from stdin")
> > Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> > ---
> > v2: remove check for nft_output_json() in nft_run_cmd_from_filename()
> >     as suggested by Phil Sutter, so JSON support does not really use
> >     /dev/stdin.
> 
> Series:
> Acked-by: Phil Sutter <phil@nwl.cc>

Pushed out, thanks