[LTP] New LTP documentation!

[LTP] New LTP documentation!

[Andrea Cervesato via ltp]

Hello everyone,

as already mentioned in the monthly LTP meeting, Linux Test Project 
lacks of a nice and clean documentation that can be easily accessed by 
users, developers and maintainers.
The current LTP documentation is also not matching with our expectancy 
towards the entire project, which is has been heavily refactored and it 
has changed in the past years, providing a higher quality code and new 
testing features.

For this reasons, we think it's time to move forward and to start 
working on documentation, helping people to use, to develop and to 
maintain LTP in an easier way, increasing quality of the overall project 
and to call more developers in the community.

I started to work on documentation refactoring, re-organizing the 
overall structure. The first prototype can be found here:

https://ltp-acerv.readthedocs.io/en/latest/index.html

The idea is to move documents from the current asciidoc format to RST 
format, following the current kernel docs guide lines [1], and to move 
API headers descriptions from regular C comments to Doxygen format.
By using the powerful readthedocs service [2], it's possible to deploy a 
documentation website with one simple setup, using Sphinx [3] as the 
main documentation framework.

For now, website prototype is showing a couple of pages, but the overall 
structure is there and ready to be filled.

The purpose of this email is to ask for feedback and ideas from the LTP 
community, so we can make documentation even better. Let me know what 
you think.


Have a good day,
Andrea Cervesato


[1] https://docs.kernel.org/doc-guide/sphinx.html#writing-documentation
[2] https://about.readthedocs.com/?ref=readthedocs.com
[3] https://www.sphinx-doc.org/en/master


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Petr Vorel]

Hi Andrea, all,

First, thank you for doing this POC.

> Hello everyone,

> as already mentioned in the monthly LTP meeting, Linux Test Project lacks of
> a nice and clean documentation that can be easily accessed by users,
> developers and maintainers.

This was from the start confusing for me, because we have docs in wiki and
metadata documentation. But the text below explains the main advantage :).

> The current LTP documentation is also not matching with our expectancy
> towards the entire project, which is has been heavily refactored and it has
> changed in the past years, providing a higher quality code and new testing
> features.

> For this reasons, we think it's time to move forward and to start working on
> documentation, helping people to use, to develop and to maintain LTP in an
> easier way, increasing quality of the overall project and to call more
> developers in the community.

> I started to work on documentation refactoring, re-organizing the overall
> structure. The first prototype can be found here:

> https://ltp-acerv.readthedocs.io/en/latest/index.html

+1, nice POC. I see from your github LTP fork it's generated via
.readthedocs.yml - +1 for docs sources staying in LTP main repository and
generated as a result of simple push.

I wonder what to make with our static page:

https://linux-test-project.github.io/

Could we somehow redirect it to our github URL or to docs on readthedocs.io?

> The idea is to move documents from the current asciidoc format to RST
> format, following the current kernel docs guide lines [1], and to move API
> headers descriptions from regular C comments to Doxygen format.

IMHO library docs generated with Doxygen is IMHO the main advantage.

> By using the powerful readthedocs service [2], it's possible to deploy a
> documentation website with one simple setup, using Sphinx [3] as the main
> documentation framework.

Maybe, we could later generate our metadata docs with Sphinx. To have single
source

Although markdown is supported elsewhere (if we one day want to migrate e.g. to
gitlab), using readthedocs.

> For now, website prototype is showing a couple of pages, but the overall
> structure is there and ready to be filled.

> The purpose of this email is to ask for feedback and ideas from the LTP
> community, so we can make documentation even better. Let me know what you
> think.

+1

I hope you plan to do the conversion and plan in the future to add Doxygen
generated docs (to actually add more content than what we have now with github
wiki).

Kind regards,
Petr

> Have a good day,
> Andrea Cervesato


> [1] https://docs.kernel.org/doc-guide/sphinx.html#writing-documentation
> [2] https://about.readthedocs.com/?ref=readthedocs.com
> [3] https://www.sphinx-doc.org/en/master

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Andrea Cervesato via ltp]

Hi Petr,

On 3/18/24 18:01, Petr Vorel wrote:
> Hi Andrea, all,
>
> First, thank you for doing this POC.
>
>> Hello everyone,
>> as already mentioned in the monthly LTP meeting, Linux Test Project lacks of
>> a nice and clean documentation that can be easily accessed by users,
>> developers and maintainers.
> This was from the start confusing for me, because we have docs in wiki and
> metadata documentation. But the text below explains the main advantage :).
>
>> The current LTP documentation is also not matching with our expectancy
>> towards the entire project, which is has been heavily refactored and it has
>> changed in the past years, providing a higher quality code and new testing
>> features.
>> For this reasons, we think it's time to move forward and to start working on
>> documentation, helping people to use, to develop and to maintain LTP in an
>> easier way, increasing quality of the overall project and to call more
>> developers in the community.
>> I started to work on documentation refactoring, re-organizing the overall
>> structure. The first prototype can be found here:
>> https://ltp-acerv.readthedocs.io/en/latest/index.html
> +1, nice POC. I see from your github LTP fork it's generated via
> .readthedocs.yml - +1 for docs sources staying in LTP main repository and
> generated as a result of simple push.
>
> I wonder what to make with our static page:
>
> https://linux-test-project.github.io/
>
> Could we somehow redirect it to our github URL or to docs on readthedocs.io?
I don't think it's possible. One it's meant to be github pages while 
readthedocs is hosted somewhere else in https://readthedocs.org/
>> The idea is to move documents from the current asciidoc format to RST
>> format, following the current kernel docs guide lines [1], and to move API
>> headers descriptions from regular C comments to Doxygen format.
> IMHO library docs generated with Doxygen is IMHO the main advantage.
>
>> By using the powerful readthedocs service [2], it's possible to deploy a
>> documentation website with one simple setup, using Sphinx [3] as the main
>> documentation framework.
> Maybe, we could later generate our metadata docs with Sphinx. To have single
> source
>
> Although markdown is supported elsewhere (if we one day want to migrate e.g. to
> gitlab), using readthedocs.
As far as I know, both github and gitlab support RST and the idea is to 
rewrite README as well, because it contains multiple errors and it will 
be replaced by index.rst
>> For now, website prototype is showing a couple of pages, but the overall
>> structure is there and ready to be filled.
>> The purpose of this email is to ask for feedback and ideas from the LTP
>> community, so we can make documentation even better. Let me know what you
>> think.
> +1
>
> I hope you plan to do the conversion and plan in the future to add Doxygen
> generated docs (to actually add more content than what we have now with github
> wiki).
Yes, we will use Doxygen. It will come in a second phase, after 
refactoring the whole documentation into RST.
>
> Kind regards,
> Petr
>
>> Have a good day,
>> Andrea Cervesato
>
>> [1] https://docs.kernel.org/doc-guide/sphinx.html#writing-documentation
>> [2] https://about.readthedocs.com/?ref=readthedocs.com
>> [3] https://www.sphinx-doc.org/en/master

I also forgot to mention that Sphinx is highly customizable due to the 
python scripting, so we can generate static pages at build time, 
introducing (for example) versions release letters and many other things.

Best regards,
Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Cyril Hrubis]

Hi!
> I started to work on documentation refactoring, re-organizing the 
> overall structure. The first prototype can be found here:
> 
> https://ltp-acerv.readthedocs.io/en/latest/index.html

Thanks for doing this! This looks way better than the current wiki we
have.

I'm also looking forward to the API documentation parsed from the
library headers.

A few minor comments:

"Install and run" -> "Installation and test execution"

"Patch reviewing" -> "Patch review"

"Debugging with gdb and strace" -> Just change that to "Debugging"

And we should mention the newly introduced -D test parameter in that
section as well.


-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Li Wang]

Hi Andrea,

On Mon, Mar 18, 2024 at 8:41 PM Andrea Cervesato via ltp <ltp@lists.linux.it>
wrote:

> Hello everyone,
>
> as already mentioned in the monthly LTP meeting, Linux Test Project
> lacks of a nice and clean documentation that can be easily accessed by
> users, developers and maintainers.
> The current LTP documentation is also not matching with our expectancy
> towards the entire project, which is has been heavily refactored and it
> has changed in the past years, providing a higher quality code and new
> testing features.
>
> For this reasons, we think it's time to move forward and to start
> working on documentation, helping people to use, to develop and to
> maintain LTP in an easier way, increasing quality of the overall project
> and to call more developers in the community.
>
> I started to work on documentation refactoring, re-organizing the
> overall structure. The first prototype can be found here:
>
> https://ltp-acerv.readthedocs.io/en/latest/index.html


Looks great! Thanks for bringing this.

And I just tried that the latest LTP compiled & executed succeed on
RHEL-7.9.
Can you update the "Oldest tested distributions" [1] part by adding:

  RHEL-7.9 Maipo (Distro)  3.10 (kernel)  4.8.5(gcc)  2.17(glibc)  -(clang)

[1] https://ltp-acerv.readthedocs.io/en/latest/users/supported_systems.html


-- 
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Cyril Hrubis]

Hi!
> And I just tried that the latest LTP compiled & executed succeed on
> RHEL-7.9.
> Can you update the "Oldest tested distributions" [1] part by adding:
> 
>   RHEL-7.9 Maipo (Distro)  3.10 (kernel)  4.8.5(gcc)  2.17(glibc)  -(clang)

Sure we can add this, however you will be responsible for making sure it
works and at least do pre-release testing.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Li Wang]

On Tue, Mar 19, 2024 at 10:41 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > And I just tried that the latest LTP compiled & executed succeed on
> > RHEL-7.9.
> > Can you update the "Oldest tested distributions" [1] part by adding:
> >
> >   RHEL-7.9 Maipo (Distro)  3.10 (kernel)  4.8.5(gcc)  2.17(glibc)
> -(clang)
>
> Sure we can add this, however you will be responsible for making sure it
> works and at least do pre-release testing.
>

Yes, of course. Thanks for reminding me.


-- 
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Andrea Cervesato via ltp]

Hi Li,

On 3/20/24 03:52, Li Wang wrote:
>
>
> On Tue, Mar 19, 2024 at 10:41 PM Cyril Hrubis <chrubis@suse.cz> wrote:
>
>     Hi!
>     > And I just tried that the latest LTP compiled & executed succeed on
>     > RHEL-7.9.
>     > Can you update the "Oldest tested distributions" [1] part by adding:
>     >
>     >   RHEL-7.9 Maipo (Distro)  3.10 (kernel)  4.8.5(gcc)
>     2.17(glibc)  -(clang)
>
>     Sure we can add this, however you will be responsible for making
>     sure it
>     works and at least do pre-release testing.
>
>
> Yes, of course. Thanks for reminding me.
>
If it's Ok for you, I will add RHEL to the "Oldest tested distribution" 
chapter as suggested.
>
> -- 
> Regards,
> Li Wang

Andrea

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Li Wang]

On Wed, Mar 20, 2024 at 3:45 PM Andrea Cervesato <andrea.cervesato@suse.com>
wrote:

> Hi Li,
>
> On 3/20/24 03:52, Li Wang wrote:
>
>
>
> On Tue, Mar 19, 2024 at 10:41 PM Cyril Hrubis <chrubis@suse.cz> wrote:
>
>> Hi!
>> > And I just tried that the latest LTP compiled & executed succeed on
>> > RHEL-7.9.
>> > Can you update the "Oldest tested distributions" [1] part by adding:
>> >
>> >   RHEL-7.9 Maipo (Distro)  3.10 (kernel)  4.8.5(gcc)  2.17(glibc)
>> -(clang)
>>
>> Sure we can add this, however you will be responsible for making sure it
>> works and at least do pre-release testing.
>>
>
> Yes, of course. Thanks for reminding me.
>
> If it's Ok for you, I will add RHEL to the "Oldest tested distribution"
> chapter as suggested.
>

Yes, please. I do that pre-release testing for each season.

Thank you~

-- 
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Petr Vorel]

Hi all,

> Hi Andrea,

> On Mon, Mar 18, 2024 at 8:41 PM Andrea Cervesato via ltp <ltp@lists.linux.it>
> wrote:

> > Hello everyone,

> > as already mentioned in the monthly LTP meeting, Linux Test Project
> > lacks of a nice and clean documentation that can be easily accessed by
> > users, developers and maintainers.
> > The current LTP documentation is also not matching with our expectancy
> > towards the entire project, which is has been heavily refactored and it
> > has changed in the past years, providing a higher quality code and new
> > testing features.

> > For this reasons, we think it's time to move forward and to start
> > working on documentation, helping people to use, to develop and to
> > maintain LTP in an easier way, increasing quality of the overall project
> > and to call more developers in the community.

> > I started to work on documentation refactoring, re-organizing the
> > overall structure. The first prototype can be found here:

> > https://ltp-acerv.readthedocs.io/en/latest/index.html


> Looks great! Thanks for bringing this.

> And I just tried that the latest LTP compiled & executed succeed on
> RHEL-7.9.
> Can you update the "Oldest tested distributions" [1] part by adding:

>   RHEL-7.9 Maipo (Distro)  3.10 (kernel)  4.8.5(gcc)  2.17(glibc)  -(clang)

Although in the original docs, the table was not under section "Build testing
with GitHub Actions", I assumed it's clear we talk about GitHub Actions testing.
That's why I put there only distros we actually have in .github/workflows/ci.yml
and not e.g. SLE12-SP2 which we test with latest LTP release.

I actually find useful to stand which distros we test privately, but this info
should be obvious (either add a column "CI", where would be "GitHub Actions" or
"RHEL private CI" or whatever.

@Li Andrea put there as new info: "Minimal supported kernel version is 3.10."
which is not in old docs. Do you really test RHEL-7.9 (3.10 based) with the
latest LTP release?

@Yang Xu I found you removed fallbacks up to kernel 3.10:

06ada0329 ("Remove old kernel version check when using min_kver under 3.10")
862d94c45 ("Remove old kernel version check when using tst_kvercmp under 3.10")
971828c00 ("shell: Remove old kernel version check below 3.10")

Hopefully I haven't overlook anything.

@Andrea: Also, I would prefer to put back paragraph "1. Build testing with
GitHub Actions". I'll send a patch, to force myself to learn building new docs.

Kind regards,
Petr

> [1] https://ltp-acerv.readthedocs.io/en/latest/users/supported_systems.html

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Li Wang]

Hi Petr,

On Wed, Apr 10, 2024 at 6:12 PM Petr Vorel <pvorel@suse.cz> wrote:

> Hi all,
>
> > Hi Andrea,
>
> > On Mon, Mar 18, 2024 at 8:41 PM Andrea Cervesato via ltp <
> ltp@lists.linux.it>
> > wrote:
>
> > > Hello everyone,
>
> > > as already mentioned in the monthly LTP meeting, Linux Test Project
> > > lacks of a nice and clean documentation that can be easily accessed by
> > > users, developers and maintainers.
> > > The current LTP documentation is also not matching with our expectancy
> > > towards the entire project, which is has been heavily refactored and it
> > > has changed in the past years, providing a higher quality code and new
> > > testing features.
>
> > > For this reasons, we think it's time to move forward and to start
> > > working on documentation, helping people to use, to develop and to
> > > maintain LTP in an easier way, increasing quality of the overall
> project
> > > and to call more developers in the community.
>
> > > I started to work on documentation refactoring, re-organizing the
> > > overall structure. The first prototype can be found here:
>
> > > https://ltp-acerv.readthedocs.io/en/latest/index.html
>
>
> > Looks great! Thanks for bringing this.
>
> > And I just tried that the latest LTP compiled & executed succeed on
> > RHEL-7.9.
> > Can you update the "Oldest tested distributions" [1] part by adding:
>
> >   RHEL-7.9 Maipo (Distro)  3.10 (kernel)  4.8.5(gcc)  2.17(glibc)
> -(clang)
>
> Although in the original docs, the table was not under section "Build
> testing
> with GitHub Actions", I assumed it's clear we talk about GitHub Actions
> testing.
> That's why I put there only distros we actually have in
> .github/workflows/ci.yml
> and not e.g. SLE12-SP2 which we test with latest LTP release.
>
> I actually find useful to stand which distros we test privately, but this
> info
> should be obvious (either add a column "CI", where would be "GitHub
> Actions" or
> "RHEL private CI" or whatever.
>


>
> @Li Andrea put there as new info: "Minimal supported kernel version is
> 3.10."
> which is not in old docs. Do you really test RHEL-7.9 (3.10 based) with the
> latest LTP release?
>

Yes, I did but only checked on RHEL-7.9 with x86_64 and s390x
(compile & execution well) last time.

After rebuilding the latest branch on all arches, I found the rest
arches have
problems in compiling (forgive me careless last time).

Now I am hesitant to fix the error or raise the baseline to RHEL8.3.
(I talked to colleagues and got to know they always chose to use fixed
versions of LTP when <=RHEL8.2).


---------------------
# 3.10.0-1136.el7.ppc64le

# glibc-2.17-316.el7.ppc64le

In file included from /usr/include/termios.h:40:0,
                 from /usr/include/bits/ioctl-types.h:5,
                 from /usr/include/sys/ioctl.h:29,
                 from ../../../../include/lapi/ioctl.h:11,
                 from ioctl02.c:31:
/usr/include/bits/termios.h:33:8: error: redefinition of ‘struct termios’
 struct termios {
        ^
In file included from ioctl02.c:29:0:
/usr/include/asm/termbits.h:22:8: note: originally defined here
 struct termios {
        ^




>
> @Yang Xu I found you removed fallbacks up to kernel 3.10:
>
> 06ada0329 ("Remove old kernel version check when using min_kver under
> 3.10")
> 862d94c45 ("Remove old kernel version check when using tst_kvercmp under
> 3.10")
> 971828c00 ("shell: Remove old kernel version check below 3.10")
>
> Hopefully I haven't overlook anything.
>
> @Andrea: Also, I would prefer to put back paragraph "1. Build testing with
> GitHub Actions". I'll send a patch, to force myself to learn building new
> docs.
>
> Kind regards,
> Petr
>
> > [1]
> https://ltp-acerv.readthedocs.io/en/latest/users/supported_systems.html
>
>

-- 
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Petr Vorel]

Hi Li,

> Hi Petr,

> On Wed, Apr 10, 2024 at 6:12 PM Petr Vorel <pvorel@suse.cz> wrote:

> > Hi all,

> > > Hi Andrea,

> > > On Mon, Mar 18, 2024 at 8:41 PM Andrea Cervesato via ltp <
> > ltp@lists.linux.it>
> > > wrote:

> > > > Hello everyone,

> > > > as already mentioned in the monthly LTP meeting, Linux Test Project
> > > > lacks of a nice and clean documentation that can be easily accessed by
> > > > users, developers and maintainers.
> > > > The current LTP documentation is also not matching with our expectancy
> > > > towards the entire project, which is has been heavily refactored and it
> > > > has changed in the past years, providing a higher quality code and new
> > > > testing features.

> > > > For this reasons, we think it's time to move forward and to start
> > > > working on documentation, helping people to use, to develop and to
> > > > maintain LTP in an easier way, increasing quality of the overall
> > project
> > > > and to call more developers in the community.

> > > > I started to work on documentation refactoring, re-organizing the
> > > > overall structure. The first prototype can be found here:

> > > > https://ltp-acerv.readthedocs.io/en/latest/index.html


> > > Looks great! Thanks for bringing this.

> > > And I just tried that the latest LTP compiled & executed succeed on
> > > RHEL-7.9.
> > > Can you update the "Oldest tested distributions" [1] part by adding:

> > >   RHEL-7.9 Maipo (Distro)  3.10 (kernel)  4.8.5(gcc)  2.17(glibc)
> > -(clang)

> > Although in the original docs, the table was not under section "Build
> > testing
> > with GitHub Actions", I assumed it's clear we talk about GitHub Actions
> > testing.
> > That's why I put there only distros we actually have in
> > .github/workflows/ci.yml
> > and not e.g. SLE12-SP2 which we test with latest LTP release.

> > I actually find useful to stand which distros we test privately, but this
> > info
> > should be obvious (either add a column "CI", where would be "GitHub
> > Actions" or
> > "RHEL private CI" or whatever.




> > @Li Andrea put there as new info: "Minimal supported kernel version is
> > 3.10."
> > which is not in old docs. Do you really test RHEL-7.9 (3.10 based) with the
> > latest LTP release?


> Yes, I did but only checked on RHEL-7.9 with x86_64 and s390x
> (compile & execution well) last time.

> After rebuilding the latest branch on all arches, I found the rest
> arches have
> problems in compiling (forgive me careless last time).

> Now I am hesitant to fix the error or raise the baseline to RHEL8.3.
> (I talked to colleagues and got to know they always chose to use fixed
> versions of LTP when <=RHEL8.2).

Thanks for info! Even you fix the compilation outdated info is why I would
delete RHEL 7.9 info and 3.10 based kernel from our docs. Currently I'm not
fixing test runtime issues even for SLE12-SP3 (kernel 4.4 based). Having "we
support kernel 3.10" is asking for random testing folks creating useless github
issues "test broken on kernel >= 3.10 < 4.19 (oldest mainline longterm kernel).

I would really state what we test in CI and did not give false expectation about
the actual test functionality. And if we really want to state what we build for,
I'd prefer mentioning that we are compile for that distro (i.e. tests still can
have bugs, it might be skipped due old config and bug not notice or test failure
but WONTFIX etc). Again, missed update when we change GitHub Actions setup is
easily to be checked, but claiming something is testing private CI is asking to
have outdated info sooner or later.

Kind regards,
Petr

> ---------------------
> # 3.10.0-1136.el7.ppc64le

> # glibc-2.17-316.el7.ppc64le

> In file included from /usr/include/termios.h:40:0,
>                  from /usr/include/bits/ioctl-types.h:5,
>                  from /usr/include/sys/ioctl.h:29,
>                  from ../../../../include/lapi/ioctl.h:11,
>                  from ioctl02.c:31:
> /usr/include/bits/termios.h:33:8: error: redefinition of ‘struct termios’
>  struct termios {
>         ^
> In file included from ioctl02.c:29:0:
> /usr/include/asm/termbits.h:22:8: note: originally defined here
>  struct termios {
>         ^


Hm, not in https://sourceware.org/glibc/wiki/Synchronizing_Headers.

I suspect:
697a06a82 ("ioctl02: Use correct termios structure")

That was required to fix runtime error on ppc64le, not sure if it's easy to be
fixed (m4 macro would be probably required).

If not, feel free to bisect, whether one of these or something else broke it:

8878fc683 ("ioctl,pty: Add fallback definition of struct termio")
fed63ce97 ("pty03: Add fallback definition for TIOCVHANGUP")

Kind regards,
Petr

> > @Yang Xu I found you removed fallbacks up to kernel 3.10:

> > 06ada0329 ("Remove old kernel version check when using min_kver under
> > 3.10")
> > 862d94c45 ("Remove old kernel version check when using tst_kvercmp under
> > 3.10")
> > 971828c00 ("shell: Remove old kernel version check below 3.10")

> > Hopefully I haven't overlook anything.

> > @Andrea: Also, I would prefer to put back paragraph "1. Build testing with
> > GitHub Actions". I'll send a patch, to force myself to learn building new
> > docs.

> > Kind regards,
> > Petr

> > > [1]
> > https://ltp-acerv.readthedocs.io/en/latest/users/supported_systems.html

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Li Wang]

On Thu, Apr 11, 2024 at 9:38 PM Petr Vorel <pvorel@suse.cz> wrote:

> Hi Li,
>
> > Hi Petr,
>
> > On Wed, Apr 10, 2024 at 6:12 PM Petr Vorel <pvorel@suse.cz> wrote:
>
> > > Hi all,
>
> > > > Hi Andrea,
>
> > > > On Mon, Mar 18, 2024 at 8:41 PM Andrea Cervesato via ltp <
> > > ltp@lists.linux.it>
> > > > wrote:
>
> > > > > Hello everyone,
>
> > > > > as already mentioned in the monthly LTP meeting, Linux Test Project
> > > > > lacks of a nice and clean documentation that can be easily
> accessed by
> > > > > users, developers and maintainers.
> > > > > The current LTP documentation is also not matching with our
> expectancy
> > > > > towards the entire project, which is has been heavily refactored
> and it
> > > > > has changed in the past years, providing a higher quality code and
> new
> > > > > testing features.
>
> > > > > For this reasons, we think it's time to move forward and to start
> > > > > working on documentation, helping people to use, to develop and to
> > > > > maintain LTP in an easier way, increasing quality of the overall
> > > project
> > > > > and to call more developers in the community.
>
> > > > > I started to work on documentation refactoring, re-organizing the
> > > > > overall structure. The first prototype can be found here:
>
> > > > > https://ltp-acerv.readthedocs.io/en/latest/index.html
>
>
> > > > Looks great! Thanks for bringing this.
>
> > > > And I just tried that the latest LTP compiled & executed succeed on
> > > > RHEL-7.9.
> > > > Can you update the "Oldest tested distributions" [1] part by adding:
>
> > > >   RHEL-7.9 Maipo (Distro)  3.10 (kernel)  4.8.5(gcc)  2.17(glibc)
> > > -(clang)
>
> > > Although in the original docs, the table was not under section "Build
> > > testing
> > > with GitHub Actions", I assumed it's clear we talk about GitHub Actions
> > > testing.
> > > That's why I put there only distros we actually have in
> > > .github/workflows/ci.yml
> > > and not e.g. SLE12-SP2 which we test with latest LTP release.
>
> > > I actually find useful to stand which distros we test privately, but
> this
> > > info
> > > should be obvious (either add a column "CI", where would be "GitHub
> > > Actions" or
> > > "RHEL private CI" or whatever.
>
>
>
>
> > > @Li Andrea put there as new info: "Minimal supported kernel version is
> > > 3.10."
> > > which is not in old docs. Do you really test RHEL-7.9 (3.10 based)
> with the
> > > latest LTP release?
>
>
> > Yes, I did but only checked on RHEL-7.9 with x86_64 and s390x
> > (compile & execution well) last time.
>
> > After rebuilding the latest branch on all arches, I found the rest
> > arches have
> > problems in compiling (forgive me careless last time).
>
> > Now I am hesitant to fix the error or raise the baseline to RHEL8.3.
> > (I talked to colleagues and got to know they always chose to use fixed
> > versions of LTP when <=RHEL8.2).
>
> Thanks for info! Even you fix the compilation outdated info is why I would
> delete RHEL 7.9 info and 3.10 based kernel from our docs. Currently I'm not
> fixing test runtime issues even for SLE12-SP3 (kernel 4.4 based). Having
> "we
> support kernel 3.10" is asking for random testing folks creating useless
> github
> issues "test broken on kernel >= 3.10 < 4.19 (oldest mainline longterm
> kernel).
>


Yes, Make sense!

Please feel free to delete that line about RHEL7.9, we shouldn't leave
holes open to risk between kernel 3.10 and 4.19 versions.

(or at least bump up the support to kernel-4.18 (rhel8), but consider the
CI only tests on fedora and Centos-stream9, so let's give up it)



>
> I would really state what we test in CI and did not give false expectation
> about
> the actual test functionality. And if we really want to state what we
> build for,
> I'd prefer mentioning that we are compile for that distro (i.e. tests
> still can
> have bugs, it might be skipped due old config and bug not notice or test
> failure
> but WONTFIX etc). Again, missed update when we change GitHub Actions setup
> is
> easily to be checked, but claiming something is testing private CI is
> asking to
> have outdated info sooner or later.
>

+1

Let's align the maintenance version (In Doc) with GitHub Actions.



>
> Kind regards,
> Petr
>
> > ---------------------
> > # 3.10.0-1136.el7.ppc64le
>
> > # glibc-2.17-316.el7.ppc64le
>
> > In file included from /usr/include/termios.h:40:0,
> >                  from /usr/include/bits/ioctl-types.h:5,
> >                  from /usr/include/sys/ioctl.h:29,
> >                  from ../../../../include/lapi/ioctl.h:11,
> >                  from ioctl02.c:31:
> > /usr/include/bits/termios.h:33:8: error: redefinition of ‘struct termios’
> >  struct termios {
> >         ^
> > In file included from ioctl02.c:29:0:
> > /usr/include/asm/termbits.h:22:8: note: originally defined here
> >  struct termios {
> >         ^
>
>
> Hm, not in https://sourceware.org/glibc/wiki/Synchronizing_Headers.
>
> I suspect:
> 697a06a82 ("ioctl02: Use correct termios structure")
>
> That was required to fix runtime error on ppc64le, not sure if it's easy
> to be
> fixed (m4 macro would be probably required).
>
> If not, feel free to bisect, whether one of these or something else broke
> it:
>
> 8878fc683 ("ioctl,pty: Add fallback definition of struct termio")
> fed63ce97 ("pty03: Add fallback definition for TIOCVHANGUP")
>


Thanks, I wouldn't send a patch to fix it, to avoid compilation
complications.
Let's look ahead :).

We suggest people pick up fixed version ltp-20210927[1] for older RHEL7
testing if people ask.

[1] This version has stood the test of time internally for the RHEL7 series.


-- 
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] New LTP documentation!

[Andrea Cervesato via ltp]

Hi,

On 4/11/24 11:00, Li Wang wrote:
> Hi Petr,
>
> On Wed, Apr 10, 2024 at 6:12 PM Petr Vorel <pvorel@suse.cz> wrote:
>
>     Hi all,
>
>     > Hi Andrea,
>
>     > On Mon, Mar 18, 2024 at 8:41 PM Andrea Cervesato via ltp
>     <ltp@lists.linux.it>
>     > wrote:
>
>     > > Hello everyone,
>
>     > > as already mentioned in the monthly LTP meeting, Linux Test
>     Project
>     > > lacks of a nice and clean documentation that can be easily
>     accessed by
>     > > users, developers and maintainers.
>     > > The current LTP documentation is also not matching with our
>     expectancy
>     > > towards the entire project, which is has been heavily
>     refactored and it
>     > > has changed in the past years, providing a higher quality code
>     and new
>     > > testing features.
>
>     > > For this reasons, we think it's time to move forward and to start
>     > > working on documentation, helping people to use, to develop and to
>     > > maintain LTP in an easier way, increasing quality of the
>     overall project
>     > > and to call more developers in the community.
>
>     > > I started to work on documentation refactoring, re-organizing the
>     > > overall structure. The first prototype can be found here:
>
>     > > https://ltp-acerv.readthedocs.io/en/latest/index.html
>
>
>     > Looks great! Thanks for bringing this.
>
>     > And I just tried that the latest LTP compiled & executed succeed on
>     > RHEL-7.9.
>     > Can you update the "Oldest tested distributions" [1] part by adding:
>
>     >   RHEL-7.9 Maipo (Distro)  3.10 (kernel)  4.8.5(gcc)
>     2.17(glibc)  -(clang)
>
>     Although in the original docs, the table was not under section
>     "Build testing
>     with GitHub Actions", I assumed it's clear we talk about GitHub
>     Actions testing.
>     That's why I put there only distros we actually have in
>     .github/workflows/ci.yml
>     and not e.g. SLE12-SP2 which we test with latest LTP release.
>
>     I actually find useful to stand which distros we test privately,
>     but this info
>     should be obvious (either add a column "CI", where would be
>     "GitHub Actions" or
>     "RHEL private CI" or whatever.
>
>
>
>     @Li Andrea put there as new info: "Minimal supported kernel
>     version is 3.10."
>     which is not in old docs. Do you really test RHEL-7.9 (3.10 based)
>     with the
>     latest LTP release?
>
>
> Yes, I did but only checked on RHEL-7.9 with x86_64 and s390x
> (compile & execution well) last time.
>
> After rebuilding the latest branch on all arches, I found the rest 
> arches have
> problems in compiling (forgive me careless last time).
>
> Now I am hesitant to fix the error or raise the baseline to RHEL8.3.
> (I talked to colleagues and got to know they always chose to use fixed 
> versions of LTP when <=RHEL8.2).
>
>
> ---------------------
> # 3.10.0-1136.el7.ppc64le
>
> # glibc-2.17-316.el7.ppc64le
>
> In file included from /usr/include/termios.h:40:0,
>                   from /usr/include/bits/ioctl-types.h:5,
>                   from /usr/include/sys/ioctl.h:29,
>                   from ../../../../include/lapi/ioctl.h:11,
>                   from ioctl02.c:31:
> /usr/include/bits/termios.h:33:8: error: redefinition of ‘struct termios’
>   struct termios {
>          ^
> In file included from ioctl02.c:29:0:
> /usr/include/asm/termbits.h:22:8: note: originally defined here
>   struct termios {
>          ^
>
>
>     @Yang Xu I found you removed fallbacks up to kernel 3.10:
>
>     06ada0329 ("Remove old kernel version check when using min_kver
>     under 3.10")
>     862d94c45 ("Remove old kernel version check when using tst_kvercmp
>     under 3.10")
>     971828c00 ("shell: Remove old kernel version check below 3.10")
>
>     Hopefully I haven't overlook anything.
>
>     @Andrea: Also, I would prefer to put back paragraph "1. Build
>     testing with
>     GitHub Actions". I'll send a patch, to force myself to learn
>     building new docs.
>
Feel free, I think it's a good idea. At the moment there's just a short 
comment about Github actions here:

https://linux-test-project.readthedocs.io/en/latest/developers/writing_tests.html#testing-builds-with-github-actions


>
>     Kind regards,
>     Petr
>
>     > [1]
>     https://ltp-acerv.readthedocs.io/en/latest/users/supported_systems.html
>
>
>
> -- 
> Regards,
> Li Wang


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp