Bug with automated processing of git status results

Bug with automated processing of git status results

[Сергей Шестаков]

Hi!

I am trying to make an automated processing of "git status" results.
I execute the command

git status -z -uno

I expect that it has stable output format. However, it still can print
warnings like

warning: CRLF will be replaced by LF in somefile.xml

I understand that we can turn off core.safecrlf, but it's
inconvinient. It would be better if "git status" command had an
optional parameter that disables any other output besides changed
files.

Thanks!
Sergey Shestakov
Playrix

Re: Bug with automated processing of git status results

[Konstantin Khomoutov]

On Fri, Jun 30, 2017 at 09:00:14AM +0300, Сергей Шестаков wrote:

> I am trying to make an automated processing of "git status" results.
> I execute the command
> 
> git status -z -uno
> 
> I expect that it has stable output format. However, it still can print
> warnings like
> 
> warning: CRLF will be replaced by LF in somefile.xml
> 
> I understand that we can turn off core.safecrlf, but it's
> inconvinient. It would be better if "git status" command had an
> optional parameter that disables any other output besides changed
> files.

The `git status` command supposedly writes their "regular" data to its
standard output while warnings go to its standard error stream.
Is this not the case?

Re: Bug with automated processing of git status results

[Matthieu Moy]

Сергей Шестаков <s_shestakov@playrix.com> writes:

> I understand that we can turn off core.safecrlf, but it's
> inconvinient.

Note that you can do that without actually changing the config file:

  git -c core.safecrlf=false status ...

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Bug with automated processing of git status results

[Torsten Bögershausen]

On 30/06/17 11:09, Matthieu Moy wrote:
> Сергей Шестаков <s_shestakov@playrix.com> writes:
> 
>> I understand that we can turn off core.safecrlf, but it's
>> inconvinient.
> 
> Note that you can do that without actually changing the config file:
> 
>    git -c core.safecrlf=false status ...
> 
Beside that, I would recommend to set up a .gitattributes file:

$ echo "*.xml text eol=lf" >>.gitattributes
$ git add .gitattributes
$ git commit -m "xml files are text with LF line endings"

[PATCH] status: suppress additional warning output in plumbing modes

[Stefan Beller]

When status is called with '--porcelain' (as implied by '-z'), we promise
to output only messages as described in the man page.

Suppress CRLF warnings.

Signed-off-by: Stefan Beller <sbeller@google.com>
---

Maybe something like this?

 builtin/commit.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/builtin/commit.c b/builtin/commit.c
index 00a01f07c3..3705d5ec6f 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1126,6 +1126,11 @@ static void finalize_deferred_config(struct wt_status *s)
 			die(_("--long and -z are incompatible"));
 	}
 
+	/* suppress all additional output in porcelain mode */
+	if (status_format == STATUS_FORMAT_PORCELAIN ||
+	    status_format == STATUS_FORMAT_PORCELAIN_V2)
+		safe_crlf = SAFE_CRLF_FALSE;
+
 	if (use_deferred_config && status_format == STATUS_FORMAT_UNSPECIFIED)
 		status_format = status_deferred_config.status_format;
 	if (status_format == STATUS_FORMAT_UNSPECIFIED)
-- 
2.13.0.31.g9b732c453e

Re: [PATCH] status: suppress additional warning output in plumbing modes

[Torsten Bögershausen]

 >On 30/06/17 18:28, Stefan Beller wrote:

The patch makes a lot of sense - thanks for the fast reply.
A question: does the header correspond to the patch ?

< [PATCH] status: suppress additional warning output in plumbing modes
 > [PATCH] status: suppress CRLF warnings in porcelain modes

(And may be the comment in the code:)

< / * suppress all additional output in porcelain mode */
 > / * suppress CRLF conversion warnings in porcelain mode */

> When status is called with '--porcelain' (as implied by '-z'), we promise
> to output only messages as described in the man page.
> 
> Suppress CRLF warnings.
> 
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
> 
> Maybe something like this?
> 
>   builtin/commit.c | 5 +++++
>   1 file changed, 5 insertions(+)
> 
> diff --git a/builtin/commit.c b/builtin/commit.c
> index 00a01f07c3..3705d5ec6f 100644
> --- a/builtin/commit.c
> +++ b/builtin/commit.c
> @@ -1126,6 +1126,11 @@ static void finalize_deferred_config(struct wt_status *s)
>   			die(_("--long and -z are incompatible"));
>   	}
>   
> +	/* suppress all additional output in porcelain mode */
> +	if (status_format == STATUS_FORMAT_PORCELAIN ||
> +	    status_format == STATUS_FORMAT_PORCELAIN_V2)
> +		safe_crlf = SAFE_CRLF_FALSE;
> +
>   	if (use_deferred_config && status_format == STATUS_FORMAT_UNSPECIFIED)
>   		status_format = status_deferred_config.status_format;
>   	if (status_format == STATUS_FORMAT_UNSPECIFIED)
> 

Re: [PATCH] status: suppress additional warning output in plumbing modes

[Ævar Arnfjörð Bjarmason]

On Fri, Jun 30 2017, Stefan Beller jotted:

> When status is called with '--porcelain' (as implied by '-z'), we promise
> to output only messages as described in the man page.
>
> Suppress CRLF warnings.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>
> Maybe something like this?

It looks sensibly implemented, but as for the approach I think we should
just document that you might get errors on stderr, but stable output on
stdout.

Many consumers of --porcelain, such as magit, will run arbitrary git
output and expect that if they get something on stderr they should be
showing it in some special buffer to the user as associated error
information.

I think it makes sense to do this & document it in the man page, if you
don't care about possible error messages 2>/dev/null is trivial, but
it's not trivial to discover in some non-expensive way (parsing the
non-porcelain output) that there *are* errors.

>  builtin/commit.c | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/builtin/commit.c b/builtin/commit.c
> index 00a01f07c3..3705d5ec6f 100644
> --- a/builtin/commit.c
> +++ b/builtin/commit.c
> @@ -1126,6 +1126,11 @@ static void finalize_deferred_config(struct wt_status *s)
>  			die(_("--long and -z are incompatible"));
>  	}
>
> +	/* suppress all additional output in porcelain mode */
> +	if (status_format == STATUS_FORMAT_PORCELAIN ||
> +	    status_format == STATUS_FORMAT_PORCELAIN_V2)
> +		safe_crlf = SAFE_CRLF_FALSE;
> +
>  	if (use_deferred_config && status_format == STATUS_FORMAT_UNSPECIFIED)
>  		status_format = status_deferred_config.status_format;
>  	if (status_format == STATUS_FORMAT_UNSPECIFIED)

Re: [PATCH] status: suppress additional warning output in plumbing modes

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

> When status is called with '--porcelain' (as implied by '-z'), we promise
> to output only messages as described in the man page.
>
> Suppress CRLF warnings.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>
> Maybe something like this?

This looks to me like a stimulus having enough time to go to the
spinal cord to induce a knee-jerk reaction, without giving a chance
to the brain to think things through.

Surely the reported symptom may have only been about CRLF, but who
says that would be the only kind of warning that would be seen
during "status --porcelain" codepath?

I tend to agree with Ævar's "output for the script can be read from
our standard output" should probably be our first response.

The patch _is_ a good start to document that we may want to do
something differently under _PORCELAIN output modes and one location
in the code that may be a good place to make that decision, but if
we are to squelch the warnings, we should make sure we do not give
any warning, not limited to squelching the safe-crlf warning, to the
standard error, but still diagnose errors and show error messages,
or something like that, I would think.

>
>  builtin/commit.c | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/builtin/commit.c b/builtin/commit.c
> index 00a01f07c3..3705d5ec6f 100644
> --- a/builtin/commit.c
> +++ b/builtin/commit.c
> @@ -1126,6 +1126,11 @@ static void finalize_deferred_config(struct wt_status *s)
>  			die(_("--long and -z are incompatible"));
>  	}
>  
> +	/* suppress all additional output in porcelain mode */
> +	if (status_format == STATUS_FORMAT_PORCELAIN ||
> +	    status_format == STATUS_FORMAT_PORCELAIN_V2)
> +		safe_crlf = SAFE_CRLF_FALSE;
> +
>  	if (use_deferred_config && status_format == STATUS_FORMAT_UNSPECIFIED)
>  		status_format = status_deferred_config.status_format;
>  	if (status_format == STATUS_FORMAT_UNSPECIFIED)

Re: [PATCH] status: suppress additional warning output in plumbing modes

[Stefan Beller]

On Sat, Jul 1, 2017 at 10:35 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> When status is called with '--porcelain' (as implied by '-z'), we promise
>> to output only messages as described in the man page.
>>
>> Suppress CRLF warnings.
>>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>>
>> Maybe something like this?
>
> This looks to me like a stimulus having enough time to go to the
> spinal cord to induce a knee-jerk reaction, without giving a chance
> to the brain to think things through.
>

sort of.

> Surely the reported symptom may have only been about CRLF, but who
> says that would be the only kind of warning that would be seen
> during "status --porcelain" codepath?

I was slightly worried about this, too.

>
> I tend to agree with Ævar's "output for the script can be read from
> our standard output" should probably be our first response.
>
> The patch _is_ a good start to document that we may want to do
> something differently under _PORCELAIN output modes and one location
> in the code that may be a good place to make that decision, but if
> we are to squelch the warnings, we should make sure we do not give
> any warning, not limited to squelching the safe-crlf warning, to the
> standard error, but still diagnose errors and show error messages,
> or something like that, I would think.

So for now we'd rather want to go with a documentation patch first
and then the refinement of the porcelain mode of potentially
suppressing more warnings?

Note that this patch was a one-off by me, so I no longer pursue
fixing the problem here, someone else is kindly asked to step up.

Thanks,
Stefan