Re: [PATCH bpf-next] bpftool: fix build failure

[Yonghong Song <yhs@fb.com>]

On 9/14/20 11:13 AM, Andrii Nakryiko wrote:
> On Mon, Sep 14, 2020 at 11:06 AM Yonghong Song <yhs@fb.com> wrote:
>>
>>
>>
>> On 9/14/20 10:55 AM, Andrii Nakryiko wrote:
>>> On Mon, Sep 14, 2020 at 10:46 AM Yonghong Song <yhs@fb.com> wrote:
>>>>
>>>>
>>>>
>>>> On 9/14/20 10:23 AM, Quentin Monnet wrote:
>>>>> On 14/09/2020 17:54, Yonghong Song wrote:
>>>>>>
>>>>>>
>>>>>> On 9/14/20 9:46 AM, Yonghong Song wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 9/14/20 1:16 AM, Quentin Monnet wrote:
>>>>>>>> On 14/09/2020 07:12, Yonghong Song wrote:
>>>>>>>>> When building bpf selftests like
>>>>>>>>>       make -C tools/testing/selftests/bpf -j20
>>>>>>>>> I hit the following errors:
>>>>>>>>>       ...
>>>>>>>>>       GEN
>>>>>>>>> /net-next/tools/testing/selftests/bpf/tools/build/bpftool/Documentation/bpftool-gen.8
>>>>>>>>>
>>>>>>>>>       <stdin>:75: (WARNING/2) Block quote ends without a blank line;
>>>>>>>>> unexpected unindent.
>>>>>>>>>       <stdin>:71: (WARNING/2) Literal block ends without a blank line;
>>>>>>>>> unexpected unindent.
>>>>>>>>>       <stdin>:85: (WARNING/2) Literal block ends without a blank line;
>>>>>>>>> unexpected unindent.
>>>>>>>>>       <stdin>:57: (WARNING/2) Block quote ends without a blank line;
>>>>>>>>> unexpected unindent.
>>>>>>>>>       <stdin>:66: (WARNING/2) Literal block ends without a blank line;
>>>>>>>>> unexpected unindent.
>>>>>>>>>       <stdin>:109: (WARNING/2) Literal block ends without a blank line;
>>>>>>>>> unexpected unindent.
>>>>>>>>>       <stdin>:175: (WARNING/2) Literal block ends without a blank line;
>>>>>>>>> unexpected unindent.
>>>>>>>>>       <stdin>:273: (WARNING/2) Literal block ends without a blank line;
>>>>>>>>> unexpected unindent.
>>>>>>>>>       make[1]: ***
>>>>>>>>> [/net-next/tools/testing/selftests/bpf/tools/build/bpftool/Documentation/bpftool-perf.8]
>>>>>>>>> Error 12
>>>>>>>>>       make[1]: *** Waiting for unfinished jobs....
>>>>>>>>>       make[1]: ***
>>>>>>>>> [/net-next/tools/testing/selftests/bpf/tools/build/bpftool/Documentation/bpftool-iter.8]
>>>>>>>>> Error 12
>>>>>>>>>       make[1]: ***
>>>>>>>>> [/net-next/tools/testing/selftests/bpf/tools/build/bpftool/Documentation/bpftool-struct_ops.8]
>>>>>>>>> Error 12
>>>>>>>>>       ...
>>>>>>>>>
>>>>>>>>> I am using:
>>>>>>>>>       -bash-4.4$ rst2man --version
>>>>>>>>>       rst2man (Docutils 0.11 [repository], Python 2.7.5, on linux2)
>>>>>>>>>       -bash-4.4$
>>>>>>>>>
>>>>>>>>> Looks like that particular version of rst2man prefers to have a
>>>>>>>>> blank line
>>>>>>>>> after literal blocks. This patch added block lines in related .rst
>>>>>>>>> files
>>>>>>>>> and compilation can then pass.
>>>>>>>>>
>>>>>>>>> Cc: Quentin Monnet <quentin@isovalent.com>
>>>>>>>>> Fixes: 18841da98100 ("tools: bpftool: Automate generation for "SEE
>>>>>>>>> ALSO" sections in man pages")
>>>>>>>>> Signed-off-by: Yonghong Song <yhs@fb.com>
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Yonghong, thanks for the fix! I didn't see those warnings on my
>>>>>>>> setup. For the record my rst2man version is:
>>>>>>>>
>>>>>>>>        rst2man (Docutils 0.16 [release], Python 3.8.2, on linux)
>>>>>>>>
>>>>>>>> Your patch looks good, but instead of having blank lines at the end of
>>>>>>>> most files, could you please check if the following works?
>>>>>>>
>>>>>>> Thanks for the tip! I looked at the generated output again. My above
>>>>>>> fix can silent the warning, but certainly not correct.
>>>>>>>
>>>>>>> With the following change, I captured the intermediate result of the
>>>>>>> .rst file.
>>>>>>>
>>>>>>>      ifndef RST2MAN_DEP
>>>>>>>             $(error "rst2man not found, but required to generate man pages")
>>>>>>>      endif
>>>>>>> -       $(QUIET_GEN)( cat $< ; echo -n $(call see_also,$<) ) | rst2man
>>>>>>> $(RST2MAN_OPTS) > $@
>>>>>>> +       $(QUIET_GEN)( cat $< ; echo -n $(call see_also,$<) ) | tee
>>>>>>> /tmp/tt | rst2man $(RST2MAN_OPTS) > $@
>>>>>>>
>>>>>>> With below command,
>>>>>>>        make clean && make bpftool-cgroup.8
>>>>>>> I can get the new .rst file for bpftool-cgroup.
>>>>>>>
>>>>>>> At the end of file /tmp/tt (changed bpftool-cgroup.rst), I see
>>>>>>>
>>>>>>>         ID       AttachType      AttachFlags     Name
>>>>>>> \n SEE ALSO\n========\n\t**bpf**\ (2),\n\t**bpf-helpers**\
>>>>>>> (7),\n\t**bpftool**\ (8),\n\t**bpftool-btf**\
>>>>>>> (8),\n\t**bpftool-feature**\ (8),\n\t**bpftool-gen**\
>>>>>>> (8),\n\t**bpftool-iter**\ (8),\n\t**bpftool-link**\
>>>>>>> (8),\n\t**bpftool-map**\ (8),\n\t**bpftool-net**\
>>>>>>> (8),\n\t**bpftool-perf**\ (8),\n\t**bpftool-prog**\
>>>>>>> (8),\n\t**bpftool-struct_ops**\ (8)\n
>>>>>>>
>>>>>>> This sounds correct if we rst2man can successfully transforms '\n'
>>>>>>> or '\t' to proper encoding in the man page.
>>>>>>>
>>>>>>> Unfortunately, with my version of rst2man, I got
>>>>>>>
>>>>>>> .IP "System Message: WARNING/2 (<stdin>:, line 146)"
>>>>>>> Literal block ends without a blank line; unexpected unindent.
>>>>>>> .sp
>>>>>>> n SEE
>>>>>>> ALSOn========nt**bpf**(2),nt**bpf\-helpers**(7),nt**bpftool**(8),nt**bpftool\-btf**(8),nt**bpftool\-feature**(8),nt**bpftool\-gen**(8),nt**bpftool\-iter**(8),nt**bpftool\-link**(8),nt**bpftool\-map**(8),nt**bpftool\-net**(8),nt**bpftool\-perf**(8),nt**bpftool\-prog**(8),nt**bpftool\-struct_ops**(8)n
>>>>>>>
>>>>>>> .\" Generated by docutils manpage writer.
>>>>>>>
>>>>>>> The rst2man simply considered \n as 'n'. The same for '\t' and
>>>>>>> this caused the issue.
>>>>>>>
>>>>>>> I did not find a way to fix the problem https://urldefense.proofpoint.com/v2/url?u=https-3A__www.google.com_url-3Fq-3Dhttps-3A__zoom.us_j_94864957378-3Fpwd-253DbXFRL1ZaRUxTbDVKcm9uRitpTXgyUT09-26sa-3DD-26source-3Dcalendar-26ust-3D1600532408208000-26usg-3DAOvVaw3SJ0i8oz4ZM-2DGRb7hYkrYlyet&d=DwIDaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=DA8e1B5r073vIqRrFz7MRA&m=kEK7ScPMF-y-i8dli-or8wWEGREW5V4qPB7UqHqDnkg&s=Br0g0MFXxL_pJuDVTOY5UrmvfD2ru_6Uf_X_8Nr2Rhk&e= .
>>>>>>
>>>>>> The following change works for me.
>>>>>>
>>>>>> @@ -44,7 +44,7 @@ $(OUTPUT)%.8: %.rst
>>>>>>     ifndef RST2MAN_DEP
>>>>>>            $(error "rst2man not found, but required to generate man pages")
>>>>>>     endif
>>>>>> -       $(QUIET_GEN)( cat $< ; echo -n $(call see_also,$<) ) | rst2man
>>>>>> $(RST2MAN_OPTS) > $@
>>>>>> +       $(QUIET_GEN)( cat $< ; echo -e $(call see_also,$<) ) | rst2man
>>>>>> $(RST2MAN_OPTS) > $@
>>>>>>
>>>>>>     clean: helpers-clean
>>>>>>            $(call QUIET_CLEAN, Documentation)
>>>>>> -bash-4.4$
>>>>>>
>>>>>> I will send revision 2 shortly.
>>>>>
>>>>> Thanks Yonghong, but this does not work on my setup :/. The version of
>>>>> echo which is called on my machine from the Makefile does not seem to
>>>>> interpret the "-e" option and writes something like "-e\nSEE ALSO",
>>>>> which causes rst2man to complain.
>>>>>
>>>>> I suspect the portable option here would be printf, even though Andrii
>>>>> had some concerns that we could pass a format specifier through the file
>>>>> names [0].
>>>>>
>>>>> Would this work on your setup?
>>>>>
>>>>>         $(QUIET_GEN)( cat $< ; printf $(call see_also,$<) ) | rst2man...
>>>>>
>>>>> Would that be acceptable?
>>>>
>>>> It works for me. Andrii originally suggested `echo -e`, but since `echo
>>>> -e` does not work in your environment let us use printf then. I will add
>>>> a comment about '%' in the bpftool man page name.
>>>
>>> It's amazing how incompatible echo can be. But that aside, I have
>>> nothing against printf itself, but:
>>>
>>> printf "%s" "whatever we want to print out"
>>>
>>> seems like the way to go, similarly how you'd do it in your C code, no?
>>
>> This won't really work :-(
>>
>> -bash-4.4$ printf "%s" "\n\n"
>> \n\n-bash-4.4$ printf "\n\n"
>>
>>
>> -bash-4.4$
>>
>> Looks like "\n" needs to be in format string to make a difference.
>>
> 
> I just learned that %b was added specifically for that case:
> 
> $ printf "%b" "\n\n"

Great! I learned some bash magic today... Will send v3.

> 
> 
> $
> 
>>
>>>
>>>>
>>>>>
>>>>> [0]
>>>>> https://lore.kernel.org/bpf/ca595fd6-e807-ac8e-f15f-68bfc7b7dbc4@isovalent.com/T/#m01bb298fd512121edd5e77a26ed5382c0c53939e
>>>>>
>>>>> Quentin
>>>>>