* Re: [PATCH] scripts/decode_stacktrace.sh: optionally use LLVM utilities
2023-09-29 3:48 [PATCH] scripts/decode_stacktrace.sh: optionally use LLVM utilities Carlos Llamas
@ 2023-09-29 15:28 ` Nick Desaulniers
2023-12-18 20:08 ` Elliot Berman
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Nick Desaulniers @ 2023-09-29 15:28 UTC (permalink / raw)
To: Carlos Llamas
Cc: Nathan Chancellor, Tom Rix, linux-kernel, kernel-team,
Will Deacon, John Stultz, Masahiro Yamada, Sami Tolvanen
On Thu, Sep 28, 2023 at 8:48 PM Carlos Llamas <cmllamas@google.com> wrote:
>
> GNU's addr2line can have problems parsing a vmlinux built with LLVM,
> particularly when LTO was used. In order to decode the traces correctly
> this patch adds the ability to switch to LLVM's utilities readelf and
> addr2line. The same approach is followed by Will in [1].
>
> Before:
> $ scripts/decode_stacktrace.sh vmlinux < kernel.log
> [17716.240635] Call trace:
> [17716.240646] skb_cow_data (??:?)
> [17716.240654] esp6_input (ld-temp.o:?)
> [17716.240666] xfrm_input (ld-temp.o:?)
> [17716.240674] xfrm6_rcv (??:?)
> [...]
>
> After:
> $ LLVM=1 scripts/decode_stacktrace.sh vmlinux < kernel.log
> [17716.240635] Call trace:
> [17716.240646] skb_cow_data (include/linux/skbuff.h:2172 net/core/skbuff.c:4503)
> [17716.240654] esp6_input (net/ipv6/esp6.c:977)
> [17716.240666] xfrm_input (net/xfrm/xfrm_input.c:659)
> [17716.240674] xfrm6_rcv (net/ipv6/xfrm6_input.c:172)
> [...]
>
> Note that one could set CROSS_COMPILE=llvm- instead to hack around this
> issue. However, doing so can break the decodecode routine as it will
> force the selection of other LLVM utilities down the line e.g. llvm-as.
>
> [1] https://lore.kernel.org/all/20230914131225.13415-3-will@kernel.org/
>
> Cc: Will Deacon <will@kernel.org>
> Cc: John Stultz <jstultz@google.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Carlos Llamas <cmllamas@google.com>
Thanks for the patch!
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> scripts/decode_stacktrace.sh | 19 +++++++++++++++++--
> 1 file changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/decode_stacktrace.sh b/scripts/decode_stacktrace.sh
> index 564c5632e1a2..bfe5a4082d8e 100755
> --- a/scripts/decode_stacktrace.sh
> +++ b/scripts/decode_stacktrace.sh
> @@ -16,6 +16,21 @@ elif type c++filt >/dev/null 2>&1 ; then
> cppfilt_opts=-i
> fi
>
> +UTIL_SUFFIX=
> +if [[ -z ${LLVM:-} ]]; then
> + UTIL_PREFIX=${CROSS_COMPILE:-}
> +else
> + UTIL_PREFIX=llvm-
> + if [[ ${LLVM} == */ ]]; then
> + UTIL_PREFIX=${LLVM}${UTIL_PREFIX}
> + elif [[ ${LLVM} == -* ]]; then
> + UTIL_SUFFIX=${LLVM}
> + fi
> +fi
> +
> +READELF=${UTIL_PREFIX}readelf${UTIL_SUFFIX}
> +ADDR2LINE=${UTIL_PREFIX}addr2line${UTIL_SUFFIX}
> +
> if [[ $1 == "-r" ]] ; then
> vmlinux=""
> basepath="auto"
> @@ -75,7 +90,7 @@ find_module() {
>
> if [[ "$modpath" != "" ]] ; then
> for fn in $(find "$modpath" -name "${module//_/[-_]}.ko*") ; do
> - if readelf -WS "$fn" | grep -qwF .debug_line ; then
> + if ${READELF} -WS "$fn" | grep -qwF .debug_line ; then
> echo $fn
> return
> fi
> @@ -169,7 +184,7 @@ parse_symbol() {
> if [[ $aarray_support == true && "${cache[$module,$address]+isset}" == "isset" ]]; then
> local code=${cache[$module,$address]}
> else
> - local code=$(${CROSS_COMPILE}addr2line -i -e "$objfile" "$address" 2>/dev/null)
> + local code=$(${ADDR2LINE} -i -e "$objfile" "$address" 2>/dev/null)
> if [[ $aarray_support == true ]]; then
> cache[$module,$address]=$code
> fi
> --
> 2.42.0.582.g8ccd20d70d-goog
>
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH] scripts/decode_stacktrace.sh: optionally use LLVM utilities
2023-09-29 3:48 [PATCH] scripts/decode_stacktrace.sh: optionally use LLVM utilities Carlos Llamas
2023-09-29 15:28 ` Nick Desaulniers
@ 2023-12-18 20:08 ` Elliot Berman
2023-12-18 23:59 ` Carlos Llamas
2023-12-19 0:23 ` Justin Stitt
2024-01-09 20:05 ` Carlos Llamas
3 siblings, 1 reply; 7+ messages in thread
From: Elliot Berman @ 2023-12-18 20:08 UTC (permalink / raw)
To: Carlos Llamas, Nathan Chancellor, Nick Desaulniers, Tom Rix
Cc: linux-kernel, kernel-team, Will Deacon, John Stultz,
Masahiro Yamada
On 9/28/2023 8:48 PM, Carlos Llamas wrote:
> GNU's addr2line can have problems parsing a vmlinux built with LLVM,
> particularly when LTO was used. In order to decode the traces correctly
> this patch adds the ability to switch to LLVM's utilities readelf and
> addr2line. The same approach is followed by Will in [1].
>
> Before:
> $ scripts/decode_stacktrace.sh vmlinux < kernel.log
> [17716.240635] Call trace:
> [17716.240646] skb_cow_data (??:?)
> [17716.240654] esp6_input (ld-temp.o:?)
> [17716.240666] xfrm_input (ld-temp.o:?)
> [17716.240674] xfrm6_rcv (??:?)
> [...]
>
> After:
> $ LLVM=1 scripts/decode_stacktrace.sh vmlinux < kernel.log
> [17716.240635] Call trace:
> [17716.240646] skb_cow_data (include/linux/skbuff.h:2172 net/core/skbuff.c:4503)
> [17716.240654] esp6_input (net/ipv6/esp6.c:977)
> [17716.240666] xfrm_input (net/xfrm/xfrm_input.c:659)
> [17716.240674] xfrm6_rcv (net/ipv6/xfrm6_input.c:172)
> [...]
>
> Note that one could set CROSS_COMPILE=llvm- instead to hack around this
> issue. However, doing so can break the decodecode routine as it will
> force the selection of other LLVM utilities down the line e.g. llvm-as.
>
> [1] https://lore.kernel.org/all/20230914131225.13415-3-will@kernel.org/
>
> Cc: Will Deacon <will@kernel.org>
> Cc: John Stultz <jstultz@google.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Carlos Llamas <cmllamas@google.com>
Reviewed-by: Elliot Berman <quic_eberman@quicinc.com>
> ---
> scripts/decode_stacktrace.sh | 19 +++++++++++++++++--
> 1 file changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/decode_stacktrace.sh b/scripts/decode_stacktrace.sh
> index 564c5632e1a2..bfe5a4082d8e 100755
> --- a/scripts/decode_stacktrace.sh
> +++ b/scripts/decode_stacktrace.sh
> @@ -16,6 +16,21 @@ elif type c++filt >/dev/null 2>&1 ; then
> cppfilt_opts=-i
> fi
>
> +UTIL_SUFFIX=
> +if [[ -z ${LLVM:-} ]]; then
> + UTIL_PREFIX=${CROSS_COMPILE:-}
> +else
> + UTIL_PREFIX=llvm-
> + if [[ ${LLVM} == */ ]]; then
> + UTIL_PREFIX=${LLVM}${UTIL_PREFIX}
> + elif [[ ${LLVM} == -* ]]; then
> + UTIL_SUFFIX=${LLVM}
> + fi
> +fi
> +
> +READELF=${UTIL_PREFIX}readelf${UTIL_SUFFIX}
> +ADDR2LINE=${UTIL_PREFIX}addr2line${UTIL_SUFFIX}
> +
> if [[ $1 == "-r" ]] ; then
> vmlinux=""
> basepath="auto"
> @@ -75,7 +90,7 @@ find_module() {
>
> if [[ "$modpath" != "" ]] ; then
> for fn in $(find "$modpath" -name "${module//_/[-_]}.ko*") ; do
> - if readelf -WS "$fn" | grep -qwF .debug_line ; then
> + if ${READELF} -WS "$fn" | grep -qwF .debug_line ; then
> echo $fn
> return
> fi
> @@ -169,7 +184,7 @@ parse_symbol() {
> if [[ $aarray_support == true && "${cache[$module,$address]+isset}" == "isset" ]]; then
> local code=${cache[$module,$address]}
> else
> - local code=$(${CROSS_COMPILE}addr2line -i -e "$objfile" "$address" 2>/dev/null)
> + local code=$(${ADDR2LINE} -i -e "$objfile" "$address" 2>/dev/null)
> if [[ $aarray_support == true ]]; then
> cache[$module,$address]=$code
> fi
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH] scripts/decode_stacktrace.sh: optionally use LLVM utilities
2023-12-18 20:08 ` Elliot Berman
@ 2023-12-18 23:59 ` Carlos Llamas
0 siblings, 0 replies; 7+ messages in thread
From: Carlos Llamas @ 2023-12-18 23:59 UTC (permalink / raw)
To: Elliot Berman
Cc: Nathan Chancellor, Nick Desaulniers, Tom Rix, linux-kernel,
kernel-team, Will Deacon, John Stultz, Masahiro Yamada
On Mon, Dec 18, 2023 at 12:08:14PM -0800, Elliot Berman wrote:
>
> Reviewed-by: Elliot Berman <quic_eberman@quicinc.com>
>
Oh, thanks Elliot for having a look. I had forgotten about this patch.
I'll RESEND if this doesn't get picked up next week.
--
Carlos Llamas
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] scripts/decode_stacktrace.sh: optionally use LLVM utilities
2023-09-29 3:48 [PATCH] scripts/decode_stacktrace.sh: optionally use LLVM utilities Carlos Llamas
2023-09-29 15:28 ` Nick Desaulniers
2023-12-18 20:08 ` Elliot Berman
@ 2023-12-19 0:23 ` Justin Stitt
2023-12-19 1:16 ` Carlos Llamas
2024-01-09 20:05 ` Carlos Llamas
3 siblings, 1 reply; 7+ messages in thread
From: Justin Stitt @ 2023-12-19 0:23 UTC (permalink / raw)
To: Carlos Llamas
Cc: Nathan Chancellor, Nick Desaulniers, Tom Rix, linux-kernel,
kernel-team, Will Deacon, John Stultz, Masahiro Yamada
Hi,
On Fri, Sep 29, 2023 at 03:48:17AM +0000, Carlos Llamas wrote:
> GNU's addr2line can have problems parsing a vmlinux built with LLVM,
> particularly when LTO was used. In order to decode the traces correctly
> this patch adds the ability to switch to LLVM's utilities readelf and
> addr2line. The same approach is followed by Will in [1].
>
> Before:
> $ scripts/decode_stacktrace.sh vmlinux < kernel.log
> [17716.240635] Call trace:
> [17716.240646] skb_cow_data (??:?)
> [17716.240654] esp6_input (ld-temp.o:?)
> [17716.240666] xfrm_input (ld-temp.o:?)
> [17716.240674] xfrm6_rcv (??:?)
> [...]
Perhaps it is due to where I am dumping the stack, but I am getting good
stack traces on mainline (with a LLVM-built kernel) without this patch.
With that being said, applying this patch still has good stack traces
and it makes sense if it helps others.
>
> After:
> $ LLVM=1 scripts/decode_stacktrace.sh vmlinux < kernel.log
> [17716.240635] Call trace:
> [17716.240646] skb_cow_data (include/linux/skbuff.h:2172 net/core/skbuff.c:4503)
> [17716.240654] esp6_input (net/ipv6/esp6.c:977)
> [17716.240666] xfrm_input (net/xfrm/xfrm_input.c:659)
> [17716.240674] xfrm6_rcv (net/ipv6/xfrm6_input.c:172)
> [...]
>
> Note that one could set CROSS_COMPILE=llvm- instead to hack around this
> issue. However, doing so can break the decodecode routine as it will
> force the selection of other LLVM utilities down the line e.g. llvm-as.
>
> [1] https://lore.kernel.org/all/20230914131225.13415-3-will@kernel.org/
>
> Cc: Will Deacon <will@kernel.org>
> Cc: John Stultz <jstultz@google.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Carlos Llamas <cmllamas@google.com>
Tested-by: Justin Stitt <justinstitt@google.com>
Note that this patch is similar: https://lore.kernel.org/all/20231215-llvm-decode-stacktrace-v1-1-201cb86f4879@quicinc.com/
> ---
> scripts/decode_stacktrace.sh | 19 +++++++++++++++++--
> 1 file changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/decode_stacktrace.sh b/scripts/decode_stacktrace.sh
> index 564c5632e1a2..bfe5a4082d8e 100755
> --- a/scripts/decode_stacktrace.sh
> +++ b/scripts/decode_stacktrace.sh
> @@ -16,6 +16,21 @@ elif type c++filt >/dev/null 2>&1 ; then
> cppfilt_opts=-i
> fi
>
> +UTIL_SUFFIX=
> +if [[ -z ${LLVM:-} ]]; then
> + UTIL_PREFIX=${CROSS_COMPILE:-}
> +else
> + UTIL_PREFIX=llvm-
> + if [[ ${LLVM} == */ ]]; then
> + UTIL_PREFIX=${LLVM}${UTIL_PREFIX}
> + elif [[ ${LLVM} == -* ]]; then
> + UTIL_SUFFIX=${LLVM}
> + fi
> +fi
> +
> +READELF=${UTIL_PREFIX}readelf${UTIL_SUFFIX}
> +ADDR2LINE=${UTIL_PREFIX}addr2line${UTIL_SUFFIX}
> +
> if [[ $1 == "-r" ]] ; then
> vmlinux=""
> basepath="auto"
> @@ -75,7 +90,7 @@ find_module() {
>
> if [[ "$modpath" != "" ]] ; then
> for fn in $(find "$modpath" -name "${module//_/[-_]}.ko*") ; do
> - if readelf -WS "$fn" | grep -qwF .debug_line ; then
> + if ${READELF} -WS "$fn" | grep -qwF .debug_line ; then
> echo $fn
> return
> fi
> @@ -169,7 +184,7 @@ parse_symbol() {
> if [[ $aarray_support == true && "${cache[$module,$address]+isset}" == "isset" ]]; then
> local code=${cache[$module,$address]}
> else
> - local code=$(${CROSS_COMPILE}addr2line -i -e "$objfile" "$address" 2>/dev/null)
> + local code=$(${ADDR2LINE} -i -e "$objfile" "$address" 2>/dev/null)
> if [[ $aarray_support == true ]]; then
> cache[$module,$address]=$code
> fi
> --
> 2.42.0.582.g8ccd20d70d-goog
>
Thanks
Justin
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH] scripts/decode_stacktrace.sh: optionally use LLVM utilities
2023-12-19 0:23 ` Justin Stitt
@ 2023-12-19 1:16 ` Carlos Llamas
0 siblings, 0 replies; 7+ messages in thread
From: Carlos Llamas @ 2023-12-19 1:16 UTC (permalink / raw)
To: Justin Stitt
Cc: Nathan Chancellor, Nick Desaulniers, Tom Rix, linux-kernel,
kernel-team, Will Deacon, John Stultz, Masahiro Yamada
On Tue, Dec 19, 2023 at 12:23:18AM +0000, Justin Stitt wrote:
> Hi,
>
> On Fri, Sep 29, 2023 at 03:48:17AM +0000, Carlos Llamas wrote:
> > GNU's addr2line can have problems parsing a vmlinux built with LLVM,
> > particularly when LTO was used. In order to decode the traces correctly
> > this patch adds the ability to switch to LLVM's utilities readelf and
> > addr2line. The same approach is followed by Will in [1].
> >
> > Before:
> > $ scripts/decode_stacktrace.sh vmlinux < kernel.log
> > [17716.240635] Call trace:
> > [17716.240646] skb_cow_data (??:?)
> > [17716.240654] esp6_input (ld-temp.o:?)
> > [17716.240666] xfrm_input (ld-temp.o:?)
> > [17716.240674] xfrm6_rcv (??:?)
> > [...]
>
> Perhaps it is due to where I am dumping the stack, but I am getting good
> stack traces on mainline (with a LLVM-built kernel) without this patch.
Maybe you have set CROSS_COMPILE=llvm- in your environment? This is an
alternative workaround. I write more about this in the commit log.
>
> With that being said, applying this patch still has good stack traces
> and it makes sense if it helps others.
>
> >
> > After:
> > $ LLVM=1 scripts/decode_stacktrace.sh vmlinux < kernel.log
> > [17716.240635] Call trace:
> > [17716.240646] skb_cow_data (include/linux/skbuff.h:2172 net/core/skbuff.c:4503)
> > [17716.240654] esp6_input (net/ipv6/esp6.c:977)
> > [17716.240666] xfrm_input (net/xfrm/xfrm_input.c:659)
> > [17716.240674] xfrm6_rcv (net/ipv6/xfrm6_input.c:172)
> > [...]
> >
> > Note that one could set CROSS_COMPILE=llvm- instead to hack around this
> > issue. However, doing so can break the decodecode routine as it will
> > force the selection of other LLVM utilities down the line e.g. llvm-as.
> >
> > [1] https://lore.kernel.org/all/20230914131225.13415-3-will@kernel.org/
> >
> > Cc: Will Deacon <will@kernel.org>
> > Cc: John Stultz <jstultz@google.com>
> > Cc: Masahiro Yamada <masahiroy@kernel.org>
> > Signed-off-by: Carlos Llamas <cmllamas@google.com>
>
> Tested-by: Justin Stitt <justinstitt@google.com>
>
> Note that this patch is similar: https://lore.kernel.org/all/20231215-llvm-decode-stacktrace-v1-1-201cb86f4879@quicinc.com/
Ha, I see. Elliot's patch help revive this thread. Cool!
--
Thanks,
Carlos Llamas
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] scripts/decode_stacktrace.sh: optionally use LLVM utilities
2023-09-29 3:48 [PATCH] scripts/decode_stacktrace.sh: optionally use LLVM utilities Carlos Llamas
` (2 preceding siblings ...)
2023-12-19 0:23 ` Justin Stitt
@ 2024-01-09 20:05 ` Carlos Llamas
3 siblings, 0 replies; 7+ messages in thread
From: Carlos Llamas @ 2024-01-09 20:05 UTC (permalink / raw)
To: Nathan Chancellor, Nick Desaulniers, Tom Rix, Andrew Morton
Cc: linux-kernel, kernel-team, Will Deacon, John Stultz,
Masahiro Yamada, Andrew Morton
On Fri, Sep 29, 2023 at 03:48:17AM +0000, Carlos Llamas wrote:
> GNU's addr2line can have problems parsing a vmlinux built with LLVM,
> particularly when LTO was used. In order to decode the traces correctly
> this patch adds the ability to switch to LLVM's utilities readelf and
> addr2line. The same approach is followed by Will in [1].
>
> Before:
> $ scripts/decode_stacktrace.sh vmlinux < kernel.log
> [17716.240635] Call trace:
> [17716.240646] skb_cow_data (??:?)
> [17716.240654] esp6_input (ld-temp.o:?)
> [17716.240666] xfrm_input (ld-temp.o:?)
> [17716.240674] xfrm6_rcv (??:?)
> [...]
>
> After:
> $ LLVM=1 scripts/decode_stacktrace.sh vmlinux < kernel.log
> [17716.240635] Call trace:
> [17716.240646] skb_cow_data (include/linux/skbuff.h:2172 net/core/skbuff.c:4503)
> [17716.240654] esp6_input (net/ipv6/esp6.c:977)
> [17716.240666] xfrm_input (net/xfrm/xfrm_input.c:659)
> [17716.240674] xfrm6_rcv (net/ipv6/xfrm6_input.c:172)
> [...]
>
> Note that one could set CROSS_COMPILE=llvm- instead to hack around this
> issue. However, doing so can break the decodecode routine as it will
> force the selection of other LLVM utilities down the line e.g. llvm-as.
>
> [1] https://lore.kernel.org/all/20230914131225.13415-3-will@kernel.org/
>
> Cc: Will Deacon <will@kernel.org>
> Cc: John Stultz <jstultz@google.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Carlos Llamas <cmllamas@google.com>
> ---
There is no clear maintainer for this script. However, I had a look at
git history and it seems this might need to be picked up by Andrew.
Cc: Andrew Morton <akpm@linux-foundation.org>
Andrew can you please pick this up?
--
Carlos Llamas
^ permalink raw reply [flat|nested] 7+ messages in thread