FAILED: patch "[PATCH] scripts/tags.sh: fix incompatibility with PCRE2" failed to apply to 5.10-stable tree

FAILED: patch "[PATCH] scripts/tags.sh: fix incompatibility with PCRE2" failed to apply to 5.10-stable tree

[gregkh]

The patch below does not apply to the 5.10-stable tree.
If someone wants it applied there, or to any other stable or longterm
tree, then please email the backport, including the original git commit
id to <stable@vger.kernel.org>.

Possible dependencies:

6ec363fc6142 ("scripts/tags.sh: fix incompatibility with PCRE2")
7394d2ebb651 ("scripts/tags.sh: Invoke 'realpath' via 'xargs'")

thanks,

greg k-h

------------------ original commit in Linus's tree ------------------

From 6ec363fc6142226b9ab5a6528f65333d729d2b6b Mon Sep 17 00:00:00 2001
From: Carlos Llamas <cmllamas@google.com>
Date: Wed, 15 Feb 2023 18:38:50 +0000
Subject: [PATCH] scripts/tags.sh: fix incompatibility with PCRE2

Starting with release 10.38 PCRE2 drops default support for using \K in
lookaround patterns as described in [1]. Unfortunately, scripts/tags.sh
relies on such functionality to collect all_compiled_soures() leading to
the following error:

  $ make COMPILED_SOURCE=1 tags
    GEN     tags
  grep: \K is not allowed in lookarounds (but see PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK)

The usage of \K for this pattern was introduced in commit 4f491bb6ea2a
("scripts/tags.sh: collect compiled source precisely") which speeds up
the generation of tags significantly.

In order to fix this issue without compromising the performance we can
switch over to an equivalent sed expression. The same matching pattern
is preserved here except \K is replaced with a backreference \1.

[1] https://www.pcre.org/current/doc/html/pcre2syntax.html#SEC11

Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Jialu Xu <xujialu@vimux.org>
Cc: Vipin Sharma <vipinsh@google.com>
Cc: stable@vger.kernel.org
Fixes: 4f491bb6ea2a ("scripts/tags.sh: collect compiled source precisely")
Signed-off-by: Carlos Llamas <cmllamas@google.com>
Link: https://lore.kernel.org/r/20230215183850.3353198-1-cmllamas@google.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

diff --git a/scripts/tags.sh b/scripts/tags.sh
index 1ad45f17179a..6b9001853890 100755
--- a/scripts/tags.sh
+++ b/scripts/tags.sh
@@ -98,7 +98,7 @@ all_compiled_sources()
 	{
 		echo include/generated/autoconf.h
 		find $ignore -name "*.cmd" -exec \
-			grep -Poh '(?(?=^source_.* \K).*|(?=^  \K\S).*(?= \\))' {} \+ |
+			sed -n -E 's/^source_.* (.*)/\1/p; s/^  (\S.*) \\/\1/p' {} \+ |
 		awk '!a[$0]++'
 	} | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
 	sort -u

Re: FAILED: patch "[PATCH] scripts/tags.sh: fix incompatibility with PCRE2" failed to apply to 5.10-stable tree

[Carlos Llamas]

On Tue, Feb 28, 2023 at 07:49:37PM +0100, gregkh@linuxfoundation.org wrote:
> 
> The patch below does not apply to the 5.10-stable tree.
> If someone wants it applied there, or to any other stable or longterm
> tree, then please email the backport, including the original git commit
> id to <stable@vger.kernel.org>.
> 
> Possible dependencies:
> 
> 6ec363fc6142 ("scripts/tags.sh: fix incompatibility with PCRE2")
> 7394d2ebb651 ("scripts/tags.sh: Invoke 'realpath' via 'xargs'")
> 
> thanks,
> 
> greg k-h
> 
> ------------------ original commit in Linus's tree ------------------
> 
> From 6ec363fc6142226b9ab5a6528f65333d729d2b6b Mon Sep 17 00:00:00 2001
> From: Carlos Llamas <cmllamas@google.com>
> Date: Wed, 15 Feb 2023 18:38:50 +0000
> Subject: [PATCH] scripts/tags.sh: fix incompatibility with PCRE2
> 
> Starting with release 10.38 PCRE2 drops default support for using \K in
> lookaround patterns as described in [1]. Unfortunately, scripts/tags.sh
> relies on such functionality to collect all_compiled_soures() leading to
> the following error:
> 
>   $ make COMPILED_SOURCE=1 tags
>     GEN     tags
>   grep: \K is not allowed in lookarounds (but see PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK)
> 
> The usage of \K for this pattern was introduced in commit 4f491bb6ea2a
> ("scripts/tags.sh: collect compiled source precisely") which speeds up
> the generation of tags significantly.
> 
> In order to fix this issue without compromising the performance we can
> switch over to an equivalent sed expression. The same matching pattern
> is preserved here except \K is replaced with a backreference \1.
> 
> [1] https://www.pcre.org/current/doc/html/pcre2syntax.html#SEC11
> 
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Cc: Jialu Xu <xujialu@vimux.org>
> Cc: Vipin Sharma <vipinsh@google.com>
> Cc: stable@vger.kernel.org
> Fixes: 4f491bb6ea2a ("scripts/tags.sh: collect compiled source precisely")
> Signed-off-by: Carlos Llamas <cmllamas@google.com>
> Link: https://lore.kernel.org/r/20230215183850.3353198-1-cmllamas@google.com
> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> 
> diff --git a/scripts/tags.sh b/scripts/tags.sh
> index 1ad45f17179a..6b9001853890 100755
> --- a/scripts/tags.sh
> +++ b/scripts/tags.sh
> @@ -98,7 +98,7 @@ all_compiled_sources()
>  	{
>  		echo include/generated/autoconf.h
>  		find $ignore -name "*.cmd" -exec \
> -			grep -Poh '(?(?=^source_.* \K).*|(?=^  \K\S).*(?= \\))' {} \+ |
> +			sed -n -E 's/^source_.* (.*)/\1/p; s/^  (\S.*) \\/\1/p' {} \+ |
>  		awk '!a[$0]++'
>  	} | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
>  	sort -u
> 

Hi Greg,

This patch would be needed in 5.10+ branches. Instead of backporting it,
I think is better to pick the "xargs" fix too. So can you please take
the following two commits for 5.10 and 5.15? I've verified they both
apply and work as expected with new PCRE2.

7394d2ebb651 ("scripts/tags.sh: Invoke 'realpath' via 'xargs'")
6ec363fc6142 ("scripts/tags.sh: fix incompatibility with PCRE2")

Thanks,
Carlos Llamas

Re: FAILED: patch "[PATCH] scripts/tags.sh: fix incompatibility with PCRE2" failed to apply to 5.10-stable tree

[Greg KH]

On Tue, Feb 28, 2023 at 08:22:01PM +0000, Carlos Llamas wrote:
> On Tue, Feb 28, 2023 at 07:49:37PM +0100, gregkh@linuxfoundation.org wrote:
> > 
> > The patch below does not apply to the 5.10-stable tree.
> > If someone wants it applied there, or to any other stable or longterm
> > tree, then please email the backport, including the original git commit
> > id to <stable@vger.kernel.org>.
> > 
> > Possible dependencies:
> > 
> > 6ec363fc6142 ("scripts/tags.sh: fix incompatibility with PCRE2")
> > 7394d2ebb651 ("scripts/tags.sh: Invoke 'realpath' via 'xargs'")
> > 
> > thanks,
> > 
> > greg k-h
> > 
> > ------------------ original commit in Linus's tree ------------------
> > 
> > From 6ec363fc6142226b9ab5a6528f65333d729d2b6b Mon Sep 17 00:00:00 2001
> > From: Carlos Llamas <cmllamas@google.com>
> > Date: Wed, 15 Feb 2023 18:38:50 +0000
> > Subject: [PATCH] scripts/tags.sh: fix incompatibility with PCRE2
> > 
> > Starting with release 10.38 PCRE2 drops default support for using \K in
> > lookaround patterns as described in [1]. Unfortunately, scripts/tags.sh
> > relies on such functionality to collect all_compiled_soures() leading to
> > the following error:
> > 
> >   $ make COMPILED_SOURCE=1 tags
> >     GEN     tags
> >   grep: \K is not allowed in lookarounds (but see PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK)
> > 
> > The usage of \K for this pattern was introduced in commit 4f491bb6ea2a
> > ("scripts/tags.sh: collect compiled source precisely") which speeds up
> > the generation of tags significantly.
> > 
> > In order to fix this issue without compromising the performance we can
> > switch over to an equivalent sed expression. The same matching pattern
> > is preserved here except \K is replaced with a backreference \1.
> > 
> > [1] https://www.pcre.org/current/doc/html/pcre2syntax.html#SEC11
> > 
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
> > Cc: Masahiro Yamada <masahiroy@kernel.org>
> > Cc: Jialu Xu <xujialu@vimux.org>
> > Cc: Vipin Sharma <vipinsh@google.com>
> > Cc: stable@vger.kernel.org
> > Fixes: 4f491bb6ea2a ("scripts/tags.sh: collect compiled source precisely")
> > Signed-off-by: Carlos Llamas <cmllamas@google.com>
> > Link: https://lore.kernel.org/r/20230215183850.3353198-1-cmllamas@google.com
> > Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > 
> > diff --git a/scripts/tags.sh b/scripts/tags.sh
> > index 1ad45f17179a..6b9001853890 100755
> > --- a/scripts/tags.sh
> > +++ b/scripts/tags.sh
> > @@ -98,7 +98,7 @@ all_compiled_sources()
> >  	{
> >  		echo include/generated/autoconf.h
> >  		find $ignore -name "*.cmd" -exec \
> > -			grep -Poh '(?(?=^source_.* \K).*|(?=^  \K\S).*(?= \\))' {} \+ |
> > +			sed -n -E 's/^source_.* (.*)/\1/p; s/^  (\S.*) \\/\1/p' {} \+ |
> >  		awk '!a[$0]++'
> >  	} | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
> >  	sort -u
> > 
> 
> Hi Greg,
> 
> This patch would be needed in 5.10+ branches. Instead of backporting it,
> I think is better to pick the "xargs" fix too. So can you please take
> the following two commits for 5.10 and 5.15? I've verified they both
> apply and work as expected with new PCRE2.
> 
> 7394d2ebb651 ("scripts/tags.sh: Invoke 'realpath' via 'xargs'")
> 6ec363fc6142 ("scripts/tags.sh: fix incompatibility with PCRE2")

Good idea, now queued up, thanks!

greg k-h