[PATCH] x86/tools: fix line number reported for malformed lines

[PATCH] x86/tools: fix line number reported for malformed lines

[Valentin Obst]

While debugging the `X86_DECODER_SELFTEST` failure first reported in [1],
we noticed that the line numbers reported by the `insn_decoder_test` tool
do not correspond to the line in the output of `objdump_reformat.awk` that
was causing the failure:

  # TEST    posttest
    llvm-objdump -d -j .text ./vmlinux | \
    awk -f ./arch/x86/tools/objdump_reformat.awk | \
    arch/x86/tools/insn_decoder_test -y -v
  arch/x86/tools/insn_decoder_test: error: malformed line 1657116:
  68db0

  $ llvm-objdump -d -j .text ./vmlinux | \
  awk -f ./arch/x86/tools/objdump_reformat.awk > objdump_reformat.txt
  $ head -n `echo 1657116+2 | bc` objdump_reformat.txt | tail -n 5
  ffffffff815430b1        41 8b 47 1c             movl
  ffffffff815430b5        89 c1                   movl
  ffffffff815430b7        81 c9 00 40 00 00       orl
  ffffffff815430bd        41 89 4e 18             movl
  ffffffff815430c1        a8 40                   testb

These lines are perfectly fine. The reason is that the line count reported
by the tool only includes instruction lines, i.e., it does not count symbol
lines.

Add a new variable to count the combined (insn+symbol) line count and
report this in the error message. With this patch, the line reported by the
tool is the line causing the failure (long line wrapped at 75 chars):

  # TEST    posttest
    llvm-objdump -d -j .text ./vmlinux | \
    awk -f ./arch/x86/tools/objdump_reformat.awk | \
    arch/x86/tools/insn_decoder_test -y -v
  arch/x86/tools/insn_decoder_test: error: malformed line 1699686:
  68db0

  $ head -n ` echo 1699686+2 | bc` objdump_reformat.txt | tail -n 5
  ffffffff81568dac        c3                      retq
  <_RNvXsP_NtCs7qddEHlz8fK_4core3fmtRINtNtNtNtB7_4iter8adapters5chain5Chain
  INtNtBA_7flatten7FlattenINtNtB7_6option8IntoIterNtNtB7_4char11EscapeDebug
  EEINtB1a_7FlatMapNtNtNtB7_3str4iter5CharsB1T_NtB2D_23CharEscapeDebugConti
  nueEENtB5_5Debug3fmtB7_>:ffffffff81568db0
  ffffffff81568dad        0f 1f 00                nopl
  ffffffff81568db0        f3 0f 1e fa             endbr64
  ffffffff81568db4        41 56                   pushq

[In this case the line causing the failure is interpreted as two lines by
the tool (due to its length, but this is fixed by [1, 2]), and the second
line is reported. Still the spatial closeness between the reported line and
the line causing the failure would have made debugging a lot easier.]

[1]: https://lore.kernel.org/lkml/Y9ES4UKl%2F+DtvAVS@gmail.com/T/
[2]: https://lore.kernel.org/rust-for-linux/20231119180145.157455-1-sergio.collado@gmail.com/

Signed-off-by: Valentin Obst <kernel@valentinobst.de>
---
 arch/x86/tools/insn_decoder_test.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/arch/x86/tools/insn_decoder_test.c b/arch/x86/tools/insn_decoder_test.c
index 472540aeabc2..727017a3c3c7 100644
--- a/arch/x86/tools/insn_decoder_test.c
+++ b/arch/x86/tools/insn_decoder_test.c
@@ -114,6 +114,7 @@ int main(int argc, char **argv)
 	unsigned char insn_buff[16];
 	struct insn insn;
 	int insns = 0;
+	int lines = 0;
 	int warnings = 0;

 	parse_args(argc, argv);
@@ -123,6 +124,8 @@ int main(int argc, char **argv)
 		int nb = 0, ret;
 		unsigned int b;

+		lines++;
+
 		if (line[0] == '<') {
 			/* Symbol line */
 			strcpy(sym, line);
@@ -134,12 +137,12 @@ int main(int argc, char **argv)
 		strcpy(copy, line);
 		tab1 = strchr(copy, '\t');
 		if (!tab1)
-			malformed_line(line, insns);
+			malformed_line(line, lines);
 		s = tab1 + 1;
 		s += strspn(s, " ");
 		tab2 = strchr(s, '\t');
 		if (!tab2)
-			malformed_line(line, insns);
+			malformed_line(line, lines);
 		*tab2 = '\0';	/* Characters beyond tab2 aren't examined */
 		while (s < tab2) {
 			if (sscanf(s, "%x", &b) == 1) {

---
base-commit: b401b621758e46812da61fa58a67c3fd8d91de0d
change-id: 20240221-x86-insn-decoder-line-fix-7b1f2e1732ff

Best regards,
--
Valentin Obst <kernel@valentinobst.de>

Re: [PATCH] x86/tools: fix line number reported for malformed lines

[Miguel Ojeda]

On Wed, Feb 21, 2024 at 10:00 AM Valentin Obst <kernel@valentinobst.de> wrote:
>
> While debugging the `X86_DECODER_SELFTEST` failure first reported in [1],
> [In this case the line causing the failure is interpreted as two lines by
> the tool (due to its length, but this is fixed by [1, 2]), and the second
> line is reported. Still the spatial closeness between the reported line and
> the line causing the failure would have made debugging a lot easier.]

Thanks Valentin, John et al. for digging into this (and the related
issue) -- very much appreciated.

It looks good to me:

Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: Miguel Ojeda <ojeda@kernel.org>

This should probably have a Fixes tag -- from a quick look, the
original test did not seem to have the problem because `insns` was
equivalent to the number of lines since there was no `if ... {
continue; }` for the symbol case. At some point that branch was added,
so that was not true anymore, thus that one should probably be the
Fixes tag, though please double-check:

    Fixes: 35039eb6b199 ("x86: Show symbol name if insn decoder test failed")

It is a minor issue for sure, so perhaps not worth backporting, but
nevertheless the hash is in a very old kernel, and thus the issue
applies to all stable kernels. So it does not hurt flagging it to the
stable team:

    Cc: stable@vger.kernel.org

In addition, John reported the original issue, but this one was found
due to that one, and I am not exactly sure who did what here. Please
consider whether one of the following (or similar) may be fair:

    Reported-by: John Baublitz <john.m.baublitz@gmail.com>
    Debugged-by: John Baublitz <john.m.baublitz@gmail.com>

An extra Link to the discussion in Zulip could be nice too:

    Link: https://rust-for-linux.zulipchat.com/#narrow/stream/291565-Help/topic/insn_decoder_test.20failure/near/421075039

Finally, a nit: links are typically written like the following -- you
can still use bracket references at the end:

    Link: https://lore.kernel.org/lkml/Y9ES4UKl%2F+DtvAVS@gmail.com/T/ [1]
    Link: https://lore.kernel.org/rust-for-linux/20231119180145.157455-1-sergio.collado@gmail.com/
[2]

Cheers,
Miguel

Re: [PATCH] x86/tools: fix line number reported for malformed lines

[Valentin Obst]

> On Wed, Feb 21, 2024 at 10:00 AM Valentin Obst <kernel@valentinobst.de> wrote:
> >
> > While debugging the `X86_DECODER_SELFTEST` failure first reported in [1],
> > [In this case the line causing the failure is interpreted as two lines by
> > the tool (due to its length, but this is fixed by [1, 2]), and the second
> > line is reported. Still the spatial closeness between the reported line and
> > the line causing the failure would have made debugging a lot easier.]
>
> Thanks Valentin, John et al. for digging into this (and the related
> issue) -- very much appreciated.
>
> It looks good to me:
>
> Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
> Tested-by: Miguel Ojeda <ojeda@kernel.org>

Thanks!

>
> This should probably have a Fixes tag -- from a quick look, the
> original test did not seem to have the problem because `insns` was
> equivalent to the number of lines since there was no `if ... {
> continue; }` for the symbol case. At some point that branch was added,
> so that was not true anymore, thus that one should probably be the
> Fixes tag, though please double-check:
>
>     Fixes: 35039eb6b199 ("x86: Show symbol name if insn decoder test failed")

Cross checked this as well, can confirm your assessment. Thanks for
bringing this up.

>
> It is a minor issue for sure, so perhaps not worth backporting, but
> nevertheless the hash is in a very old kernel, and thus the issue
> applies to all stable kernels. So it does not hurt flagging it to the
> stable team:
>
>     Cc: stable@vger.kernel.org
>
> In addition, John reported the original issue, but this one was found
> due to that one, and I am not exactly sure who did what here. Please
> consider whether one of the following (or similar) may be fair:
>
>     Reported-by: John Baublitz <john.m.baublitz@gmail.com>
>     Debugged-by: John Baublitz <john.m.baublitz@gmail.com>

Absolutely, without him reporting the test failure and narrowing down the
config I'd have never looked at this file. Adding him for **both** is fair.
(This particular fix was not discussed on Zulip though, its just something
I noticed along the way.)

>
> An extra Link to the discussion in Zulip could be nice too:
>
>     Link: https://rust-for-linux.zulipchat.com/#narrow/stream/291565-Help/topic/insn_decoder_test.20failure/near/421075039

Didn't add it because the discussion does not mention this particular
issue, but it might indeed be good for some context.

Will this need a v2, or are all of the 'Fixes', 'Reported-By',
'Debugged-By', 'Tested-By', 'Reviewed-By' and 'Link' tags something that
maintainers may add when merging?

    - Best Valentin

>
> Finally, a nit: links are typically written like the following -- you
> can still use bracket references at the end:
>
>     Link: https://lore.kernel.org/lkml/Y9ES4UKl%2F+DtvAVS@gmail.com/T/ [1]
>     Link: https://lore.kernel.org/rust-for-linux/20231119180145.157455-1-sergio.collado@gmail.com/
> [2]
>
> Cheers,
> Miguel

Re: [PATCH] x86/tools: fix line number reported for malformed lines

[Miguel Ojeda]

On Wed, Feb 21, 2024 at 10:51 PM Valentin Obst <kernel@valentinobst.de> wrote:
>
> Thanks!
>
> Cross checked this as well, can confirm your assessment. Thanks for
> bringing this up.

My pleasure!

> Absolutely, without him reporting the test failure and narrowing down the
> config I'd have never looked at this file. Adding him for **both** is fair.
> (This particular fix was not discussed on Zulip though, its just something
> I noticed along the way.)

In that case, up to you -- whatever you consider fair for this particular patch.

> Didn't add it because the discussion does not mention this particular
> issue, but it might indeed be good for some context.

Makes sense -- I saw the [1] reference and I thought it could be a
nice complement to it, but it is true that it may be not that useful,
so please feel free to leave it out.

> Will this need a v2, or are all of the 'Fixes', 'Reported-By',
> 'Debugged-By', 'Tested-By', 'Reviewed-By' and 'Link' tags something that
> maintainers may add when merging?

Typically, tags are picked up by maintainers when they apply the patch
(if it is the last version, otherwise you would already pick them up
in the next version you send).

However, in this case, since we have the Cc stable@ and you also have
the most context to decide on the tags (e.g. for the Reported-by
etc.), I would send a v2.

Thanks!

Cheers,
Miguel