* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 12:44 ` [LTP] [PATCH v2 2/2] doc: Add ground rules page Cyril Hrubis
@ 2025-12-15 13:32 ` Li Wang via ltp
2025-12-15 14:03 ` Li Wang via ltp
` (2 more replies)
2025-12-15 14:52 ` Andrea Cervesato via ltp
` (2 subsequent siblings)
3 siblings, 3 replies; 25+ messages in thread
From: Li Wang via ltp @ 2025-12-15 13:32 UTC (permalink / raw)
To: Cyril Hrubis; +Cc: ltp
On Mon, Dec 15, 2025 at 8:43 PM Cyril Hrubis <chrubis@suse.cz> wrote:
> This is a continued effort to write down the unwritten rules we have in
> the project. Feel free to suggest more topics for the page.
>
> Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
> ---
> doc/developers/ground_rules.rst | 91 +++++++++++++++++++++++++++++++++
> doc/index.rst | 1 +
> 2 files changed, 92 insertions(+)
> create mode 100644 doc/developers/ground_rules.rst
>
> Changes in v2:
>
> - added two more rules
> - fixes and typos as pointed out by Peter
>
> diff --git a/doc/developers/ground_rules.rst
> b/doc/developers/ground_rules.rst
> new file mode 100644
> index 000000000..2bef426aa
> --- /dev/null
> +++ b/doc/developers/ground_rules.rst
> @@ -0,0 +1,91 @@
> +.. SPDX-License-Identifier: GPL-2.0-or-later
> +
> +Ground Rules
> +============
> +
> +Do not work around kernel bugs
> +------------------------------
> +
> +We have decided what we will not work around bugs in upstream LTP
> sources. If a
> +test fails on your system for a good reason, e.g. patch wasn't backported
> and
> +the bug is present, work around for this will not be accepted upstream.
> The
> +main reason for this decision is that this masks the failure for everyone
> else.
> +
> +
> +Do not synchronize by sleep
> +---------------------------
> +
> +Why is sleep in tests bad then?
> +```````````````````````````````
> +
> +The first problem is that it will likely introduce very rare test
> failures,
> +that means somebody has to spend time looking into these, which is a
> wasted
> +effort. Nobody likes tests that will fail rarely for no good reason. Even
> more
> +so you cannot run such tests with a background load to ensure that
> everything
> +works correctly on a busy system, because that will increase the likehood
> of a
> +failure.
> +
> +The second problem is that this wastes resources and slows down a test
> run. If
> +you think that adding a sleep to a test is not a big deal, lets have a
> look at
> +the bigger perspective. There is about 1600 syscall tests in Linux Test
> +Project, if 7.5% of them would sleep just for one second, we would end up
> with
> +two minutes of wasted time per testrun. In practice most of the test, that
> +historically misused sleep for synchronization, waited for much longer
> just to
> +be sure that things will works even on slower hardware. With sleeps
> between 2
> +and 5 seconds that puts us somewhere between 4 and 10 minutes which is
> between
> +13% and 33% of the syscall runtime on an outdated thinkpad, where the run
> +finishes in a bit less than half an hour. It's even worse on newer
> hardware,
> +because this slowdown will not change no matter how fast your machine is,
> which
> +is maybe the reason why this was acceptable twenty years ago but it's not
> now.
> +
> +
> +What to do instead?
> +```````````````````
> +
> +Use proper synchronization.
> +
> +There are different problems and different solutions. Most often test
> needs to
> +synchronize between child and parent proces.
> +
> +The easiest case is that parent needs to wait for a child to finish, that
> can
> +be fixed just be adding a :man2:`waitpid` in the parent which ensures
> that child is
> +finished before parent runs.
> +
> +Commonly child has to execute certain piece of code before parent can
> continue.
> +For that LTP library implements checkpoints with simple
> +:c:func:`TST_CHECKPOINT_WAIT()` and :c:func:`TST_CHECKPOINT_WAKE()`
> functions based
> +on futexes on a piece of shared memory set up by the test library.
> +
> +Another common case is where child must sleep in a syscall before parent
> can
> +continue, for which we have a :c:func:`TST_PROCESS_STATE_WAIT()` helper
> that
> +polls `/proc/$PID/stat`.
> +
> +Less often test needs to wait for an action that is done asynchronously,
> or a
> +kernel resource deallocation is deferred to a later time. In such cases
> the
> +best we can do is to poll. In LTP we ended up with a macro that polls by
> +calling a piece of code in a loop with exponentially increasing sleeps
> between
> +retries until it succeeds. Which means that instead of sleeping for a
> maximal
> +time event can possibly take the sleep is capped by twice of the optimal
> +sleeping time while we avoid polling too aggressively.
> +
> +
> +Use runtime checks for kernel features
> +======================================
> +
> +What is and what isn't supported by kernel is determined by the version
> +and configuration of the kernel the systems is currently running on.
> +That especially means that any checks done during the compilation cannot
> +be used to assume features supported by the kernel the tests end up
> +running on. The compile time checks, done by configure script, are only
> +useful for enabling fallback kernel API definitions when missing, as we
> +do in lapi/ directory.
> +
> +
> +Kernel features and RCs
> +=======================
> +
> +LTP tests or fixes for kernel changes that were not released yet can be
> posted
> +to the LTP list for a review but will not be be accepted until respective
> +kernel changes are released. Review of such changes is also considered to
> be
> +lower priority than rest of the changes. This is because kernel changes
> +especially in the early RC phase are volatile and could be changed or
> reverted.
>
It's really good to write these rules down, especially since maintainers
can reuse them in patch reviews to comment on issues and avoid repeatedly
responding to the same questions. Below are what I can think of:
Don’t require root unless it’s essential
============================
If root/caps are needed, say why in the test output. Drop privileges for
the part that doesn’t need them (and avoid running the whole test as
root “because it’s easier”).
Always clean up, even on failure
==========================
Every test should leave the system as it found it: unmount, restore sysctls,
delete temp files/dirs, kill spawned processes, remove cgroups/namespaces,
detach loop devices, restore ulimits, etc. Cleanup must run on early-exit
paths too.
Respect LTP portability goals
===========================
Avoid nonstandard libc APIs when a portable equivalent exists; don’t assume
64-bit,
page size, endianness, or particular tool versions.
diff --git a/doc/index.rst b/doc/index.rst
> index 06b75616f..659549cc3 100644
> --- a/doc/index.rst
> +++ b/doc/index.rst
> @@ -19,6 +19,7 @@
> :hidden:
> :caption: For developers
>
> + developers/ground_rules
> developers/setup_mailinglist
> developers/writing_tests
> developers/test_case_tutorial
> --
> 2.51.2
>
>
> --
> Mailing list info: https://lists.linux.it/listinfo/ltp
>
>
--
Regards,
Li Wang
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 13:32 ` Li Wang via ltp
@ 2025-12-15 14:03 ` Li Wang via ltp
2025-12-15 14:25 ` Anrea Cervesato via ltp
2025-12-15 14:30 ` Petr Vorel
2025-12-16 11:23 ` Cyril Hrubis
2025-12-21 10:30 ` Petr Vorel
2 siblings, 2 replies; 25+ messages in thread
From: Li Wang via ltp @ 2025-12-15 14:03 UTC (permalink / raw)
To: Cyril Hrubis; +Cc: ltp
Another *important* rule concerns artificial intelligence. I've noticed
some beginners submitting LTP patches directly generated by AI tools.
This puts a significant burden on patch review, as AI can sometimes
introduce a weird/unreliable perspective into the code.
Be careful when using AI tools
========================
AI tools can be useful for executing, summarizing, or suggesting approaches,
but they can also be confidently wrong and give an illusion of correctness.
Treat AI output as untrusted: verify claims against the code, documentation,
and actual behavior on a reproducer.
Do not send AI-generated changes as raw patches. AI-generated diffs often
contain
irrelevant churn, incorrect assumptions, inconsistent style, or subtle
bugs, which
creates additional burden for maintainers to review and fix.
Best practice is to write your own patches and have them reviewed by AI
before
submitting them, which helps add beneficial improvements to your work.
--
Regards,
Li Wang
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 14:03 ` Li Wang via ltp
@ 2025-12-15 14:25 ` Anrea Cervesato via ltp
2025-12-15 14:30 ` Petr Vorel
1 sibling, 0 replies; 25+ messages in thread
From: Anrea Cervesato via ltp @ 2025-12-15 14:25 UTC (permalink / raw)
To: ltp
Hi!
On 12/15/25 3:03 PM, Li Wang via ltp wrote:
> Another *important* rule concerns artificial intelligence. I've noticed
> some beginners submitting LTP patches directly generated by AI tools.
> This puts a significant burden on patch review, as AI can sometimes
> introduce a weird/unreliable perspective into the code.
>
> Be careful when using AI tools
> ========================
> AI tools can be useful for executing, summarizing, or suggesting approaches,
> but they can also be confidently wrong and give an illusion of correctness.
> Treat AI output as untrusted: verify claims against the code, documentation,
> and actual behavior on a reproducer.
>
> Do not send AI-generated changes as raw patches. AI-generated diffs often
> contain
> irrelevant churn, incorrect assumptions, inconsistent style, or subtle
> bugs, which
> creates additional burden for maintainers to review and fix.
>
> Best practice is to write your own patches and have them reviewed by AI
> before
> submitting them, which helps add beneficial improvements to your work.
>
>
+1 for this.
- Andrea
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 14:03 ` Li Wang via ltp
2025-12-15 14:25 ` Anrea Cervesato via ltp
@ 2025-12-15 14:30 ` Petr Vorel
2025-12-15 15:00 ` Andrea Cervesato via ltp
2025-12-16 7:27 ` Li Wang via ltp
1 sibling, 2 replies; 25+ messages in thread
From: Petr Vorel @ 2025-12-15 14:30 UTC (permalink / raw)
To: Li Wang; +Cc: ltp
> Another *important* rule concerns artificial intelligence. I've noticed
> some beginners submitting LTP patches directly generated by AI tools.
> This puts a significant burden on patch review, as AI can sometimes
> introduce a weird/unreliable perspective into the code.
> Be careful when using AI tools
+1 I like this title.
> ========================
> AI tools can be useful for executing, summarizing, or suggesting approaches,
> but they can also be confidently wrong and give an illusion of correctness.
> Treat AI output as untrusted: verify claims against the code, documentation,
> and actual behavior on a reproducer.
> Do not send AI-generated changes as raw patches. AI-generated diffs often
> contain
> irrelevant churn, incorrect assumptions, inconsistent style, or subtle
> bugs, which
> creates additional burden for maintainers to review and fix.
> Best practice is to write your own patches and have them reviewed by AI
> before
> submitting them, which helps add beneficial improvements to your work.
Hopefully the last paragraph will be understand how it is meant. Because we
really don't want to encourage people to send something generated by AI they
don't really understand at all. I'd consider not suggesting any AI.
I remember briefly reading kernel folks discussing their policy [1]:
> We cannot keep complaining about maintainer overload and, at the same
> time, encourage people to bombard us with even more of that stuff.
And another one I can't find any more talking that it's about the trust. If
somebody sends wrong patches generated by AI he risks patches will be simply
ignored.
Kind regards,
Petr
[1] https://lore.kernel.org/all/1bd04ce1-87c0-4e23-b155-84f7235f6072@redhat.com/
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 14:30 ` Petr Vorel
@ 2025-12-15 15:00 ` Andrea Cervesato via ltp
2025-12-16 7:07 ` Li Wang via ltp
2025-12-16 7:27 ` Li Wang via ltp
1 sibling, 1 reply; 25+ messages in thread
From: Andrea Cervesato via ltp @ 2025-12-15 15:00 UTC (permalink / raw)
To: Petr Vorel, Li Wang; +Cc: ltp
Hi!
On Mon Dec 15, 2025 at 3:30 PM CET, Petr Vorel wrote:
> > Another *important* rule concerns artificial intelligence. I've noticed
> > some beginners submitting LTP patches directly generated by AI tools.
> > This puts a significant burden on patch review, as AI can sometimes
> > introduce a weird/unreliable perspective into the code.
>
> > Be careful when using AI tools
> +1 I like this title.
>
> > ========================
> > AI tools can be useful for executing, summarizing, or suggesting approaches,
> > but they can also be confidently wrong and give an illusion of correctness.
> > Treat AI output as untrusted: verify claims against the code, documentation,
> > and actual behavior on a reproducer.
>
> > Do not send AI-generated changes as raw patches. AI-generated diffs often
> > contain
> > irrelevant churn, incorrect assumptions, inconsistent style, or subtle
> > bugs, which
> > creates additional burden for maintainers to review and fix.
>
> > Best practice is to write your own patches and have them reviewed by AI
> > before
> > submitting them, which helps add beneficial improvements to your work.
>
> Hopefully the last paragraph will be understand how it is meant. Because we
> really don't want to encourage people to send something generated by AI they
> don't really understand at all. I'd consider not suggesting any AI.
>
> I remember briefly reading kernel folks discussing their policy [1]:
>
There's nothing wrong with AI usage nowadays, since it's proven that
they can shine on certain specific tasks. In general, code generation
works bad, especially inside the kernel development. And in LTP,
obviously.
But when it comes to correct commit messages, learning what a certain
code is doing or understanding compile errors, they can be useful.
Said so, I like the Li approach, because it gives to AI the right place,
without expanding its boundaries which are well defined and well known.
> > We cannot keep complaining about maintainer overload and, at the same
> > time, encourage people to bombard us with even more of that stuff.
>
> And another one I can't find any more talking that it's about the trust. If
> somebody sends wrong patches generated by AI he risks patches will be simply
> ignored.
>
> Kind regards,
> Petr
>
> [1] https://lore.kernel.org/all/1bd04ce1-87c0-4e23-b155-84f7235f6072@redhat.com/
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
andrea.cervesato@suse.com
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 15:00 ` Andrea Cervesato via ltp
@ 2025-12-16 7:07 ` Li Wang via ltp
0 siblings, 0 replies; 25+ messages in thread
From: Li Wang via ltp @ 2025-12-16 7:07 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
On Mon, Dec 15, 2025 at 11:01 PM Andrea Cervesato <andrea.cervesato@suse.com>
wrote:
> Hi!
>
> On Mon Dec 15, 2025 at 3:30 PM CET, Petr Vorel wrote:
> > > Another *important* rule concerns artificial intelligence. I've noticed
> > > some beginners submitting LTP patches directly generated by AI tools.
> > > This puts a significant burden on patch review, as AI can sometimes
> > > introduce a weird/unreliable perspective into the code.
> >
> > > Be careful when using AI tools
> > +1 I like this title.
> >
> > > ========================
> > > AI tools can be useful for executing, summarizing, or suggesting
> approaches,
> > > but they can also be confidently wrong and give an illusion of
> correctness.
> > > Treat AI output as untrusted: verify claims against the code,
> documentation,
> > > and actual behavior on a reproducer.
> >
> > > Do not send AI-generated changes as raw patches. AI-generated diffs
> often
> > > contain
> > > irrelevant churn, incorrect assumptions, inconsistent style, or subtle
> > > bugs, which
> > > creates additional burden for maintainers to review and fix.
> >
> > > Best practice is to write your own patches and have them reviewed by AI
> > > before
> > > submitting them, which helps add beneficial improvements to your work.
> >
> > Hopefully the last paragraph will be understand how it is meant. Because
> we
> > really don't want to encourage people to send something generated by AI
> they
> > don't really understand at all. I'd consider not suggesting any AI.
> >
> > I remember briefly reading kernel folks discussing their policy [1]:
> >
>
> There's nothing wrong with AI usage nowadays, since it's proven that
> they can shine on certain specific tasks. In general, code generation
> works bad, especially inside the kernel development. And in LTP,
> obviously.
>
> But when it comes to correct commit messages, learning what a certain
> code is doing or understanding compile errors, they can be useful.
>
Exactly!
Using AI wisely can speed up debugging work, but user experience is
ultimately needed to determine its correctness.
> Said so, I like the Li approach, because it gives to AI the right place,
> without expanding its boundaries which are well defined and well known.
>
Thanks!
--
Regards,
Li Wang
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 14:30 ` Petr Vorel
2025-12-15 15:00 ` Andrea Cervesato via ltp
@ 2025-12-16 7:27 ` Li Wang via ltp
2025-12-16 10:11 ` Petr Vorel
1 sibling, 1 reply; 25+ messages in thread
From: Li Wang via ltp @ 2025-12-16 7:27 UTC (permalink / raw)
To: Petr Vorel; +Cc: ltp
On Mon, Dec 15, 2025 at 10:31 PM Petr Vorel <pvorel@suse.cz> wrote:
> > Another *important* rule concerns artificial intelligence. I've noticed
> > some beginners submitting LTP patches directly generated by AI tools.
> > This puts a significant burden on patch review, as AI can sometimes
> > introduce a weird/unreliable perspective into the code.
>
> > Be careful when using AI tools
> +1 I like this title.
>
> > ========================
> > AI tools can be useful for executing, summarizing, or suggesting
> approaches,
> > but they can also be confidently wrong and give an illusion of
> correctness.
> > Treat AI output as untrusted: verify claims against the code,
> documentation,
> > and actual behavior on a reproducer.
>
> > Do not send AI-generated changes as raw patches. AI-generated diffs often
> > contain
> > irrelevant churn, incorrect assumptions, inconsistent style, or subtle
> > bugs, which
> > creates additional burden for maintainers to review and fix.
>
> > Best practice is to write your own patches and have them reviewed by AI
> > before
> > submitting them, which helps add beneficial improvements to your work.
>
> Hopefully the last paragraph will be understand how it is meant. Because we
> really don't want to encourage people to send something generated by AI
> they
> don't really understand at all. I'd consider not suggesting any AI.
>
> I remember briefly reading kernel folks discussing their policy [1]:
>
> > We cannot keep complaining about maintainer overload and, at the same
> > time, encourage people to bombard us with even more of that stuff.
>
> And another one I can't find any more talking that it's about the trust. If
> somebody sends wrong patches generated by AI he risks patches will be
> simply
> ignored.
>
> Kind regards,
> Petr
>
>
> [1]
> https://lore.kernel.org/all/1bd04ce1-87c0-4e23-b155-84f7235f6072@redhat.com/
What a coincidence! I just spoke face-to-face with David Hildenbrand
at LPC last Friday. He expressed concerns about the increasing amount
of AI-generated code being sent to the LKML, pointing out that it does
indeed place a considerable review burden on maintainers. Moreover,
he himself rarely uses AI.
Furthermore, some experts at Huawei told me that they only allow patch
senders to review patches before sending them, but do not permit the
use of AI-generated code.
And, of course, clearly flagging content as AI-generated code in the patch
might help.
--
Regards,
Li Wang
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-16 7:27 ` Li Wang via ltp
@ 2025-12-16 10:11 ` Petr Vorel
2025-12-16 10:42 ` Li Wang via ltp
0 siblings, 1 reply; 25+ messages in thread
From: Petr Vorel @ 2025-12-16 10:11 UTC (permalink / raw)
To: Li Wang; +Cc: ltp
...
> > [1]
> > https://lore.kernel.org/all/1bd04ce1-87c0-4e23-b155-84f7235f6072@redhat.com/
> What a coincidence! I just spoke face-to-face with David Hildenbrand
> at LPC last Friday. He expressed concerns about the increasing amount
> of AI-generated code being sent to the LKML, pointing out that it does
> indeed place a considerable review burden on maintainers. Moreover,
> he himself rarely uses AI.
Nice :).
"DoS kernel maintainers with AI slop" is a real danger.
> Furthermore, some experts at Huawei told me that they only allow patch
> senders to review patches before sending them, but do not permit the
> use of AI-generated code.
+1. Is that due "AI slop" or legal issues?
While I also share the fear of "AI slop", I was surprised that QEMU really
strictly bans AI generated code due legal issues. I wonder if kernel endup the
same as well.
https://www.qemu.org/docs/master/devel/code-provenance.html#use-of-ai-generated-content
> And, of course, clearly flagging content as AI-generated code in the patch
> might help.
+1.
IMHO using AI for help with manual work, e.g. converting docs from asciidoc
to RST format is ok. For me the question is whether it can be used for
generating a test code (new tests from scratch, rewrite into into new API which
is often clearer write from scratch).
Anyway, because we already had some AI generated patches it'd be good to have an
AI policy. But I would like not to block this patchset it (it'd be nice to get
the patchset merged before Christmas and solve AI policy afterwards).
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-16 10:11 ` Petr Vorel
@ 2025-12-16 10:42 ` Li Wang via ltp
2025-12-16 11:08 ` Andrea Cervesato via ltp
0 siblings, 1 reply; 25+ messages in thread
From: Li Wang via ltp @ 2025-12-16 10:42 UTC (permalink / raw)
To: Petr Vorel; +Cc: ltp
On Tue, Dec 16, 2025 at 6:11 PM Petr Vorel <pvorel@suse.cz> wrote:
> ...
> > > [1]
> > >
> https://lore.kernel.org/all/1bd04ce1-87c0-4e23-b155-84f7235f6072@redhat.com/
>
>
> > What a coincidence! I just spoke face-to-face with David Hildenbrand
> > at LPC last Friday. He expressed concerns about the increasing amount
> > of AI-generated code being sent to the LKML, pointing out that it does
> > indeed place a considerable review burden on maintainers. Moreover,
> > he himself rarely uses AI.
>
> Nice :).
>
> "DoS kernel maintainers with AI slop" is a real danger.
>
> > Furthermore, some experts at Huawei told me that they only allow patch
> > senders to review patches before sending them, but do not permit the
> > use of AI-generated code.
>
> +1. Is that due "AI slop" or legal issues?
>
Due "AI slop", cuz mostly nowadays AI is unable to perfectly generate
suitable
code for complex projects such as kernels. They only used it in the first
round
of patch review.
> While I also share the fear of "AI slop", I was surprised that QEMU really
> strictly bans AI generated code due legal issues. I wonder if kernel endup
> the
> same as well.
>
>
> https://www.qemu.org/docs/master/devel/code-provenance.html#use-of-ai-generated-content
>
> > And, of course, clearly flagging content as AI-generated code in the
> patch
> > might help.
>
> +1.
>
> IMHO using AI for help with manual work, e.g. converting docs from asciidoc
>
Yes.
to RST format is ok. For me the question is whether it can be used for
> generating a test code (new tests from scratch, rewrite into into new API
> which
> is often clearer write from scratch).
>
I don't think AI can write better code than mankind for the whole test,
it might offer suggestions for a piece of function.
Just like Ying Huang (From Alibaba) spoke to me:
"I found that AI performs better only when I break the work down into
atomic units."
Therefore, it's especially important to rely on engineers' experience to
clearly understand what problems are hindering them and how to
communicate with AI.
>
> Anyway, because we already had some AI generated patches it'd be good to
> have an
> AI policy. But I would like not to block this patchset it (it'd be nice
> to get
> the patchset merged before Christmas and solve AI policy afterwards).
>
+1
--
Regards,
Li Wang
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-16 10:42 ` Li Wang via ltp
@ 2025-12-16 11:08 ` Andrea Cervesato via ltp
2025-12-16 11:23 ` Li Wang via ltp
0 siblings, 1 reply; 25+ messages in thread
From: Andrea Cervesato via ltp @ 2025-12-16 11:08 UTC (permalink / raw)
To: Li Wang, Petr Vorel; +Cc: ltp
>
> Just like Ying Huang (From Alibaba) spoke to me:
> "I found that AI performs better only when I break the work down into
> atomic units."
The problem I have with this approach is that there's a risk of "dumb
effect", where an engineer will rely on AI so much to forget how to
write 100 lines of clean code.
> Therefore, it's especially important to rely on engineers' experience to
> clearly understand what problems are hindering them and how to
> communicate with AI.
>
I agree with this. Boring, repetitive tasks is why we created automation
and AI is a part of it. Let's give it the right space and we won't end
working double because of AI madness :-)
This part of documentation is needed and it would be nice to have it
already by the end of the year. WDYT?
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
andrea.cervesato@suse.com
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-16 11:08 ` Andrea Cervesato via ltp
@ 2025-12-16 11:23 ` Li Wang via ltp
0 siblings, 0 replies; 25+ messages in thread
From: Li Wang via ltp @ 2025-12-16 11:23 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
On Tue, Dec 16, 2025 at 7:09 PM Andrea Cervesato <andrea.cervesato@suse.com>
wrote:
> >
> > Just like Ying Huang (From Alibaba) spoke to me:
> > "I found that AI performs better only when I break the work down into
> > atomic units."
>
> The problem I have with this approach is that there's a risk of "dumb
> effect", where an engineer will rely on AI so much to forget how to
> write 100 lines of clean code.
>
That's absolutely right.
>
> > Therefore, it's especially important to rely on engineers' experience to
> > clearly understand what problems are hindering them and how to
> > communicate with AI.
> >
>
> I agree with this. Boring, repetitive tasks is why we created automation
> and AI is a part of it. Let's give it the right space and we won't end
> working double because of AI madness :-)
>
> This part of documentation is needed and it would be nice to have it
> already by the end of the year. WDYT?
>
I agree. In conclusion, we need a specific AI rule for LTP community.
--
Regards,
Li Wang
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 13:32 ` Li Wang via ltp
2025-12-15 14:03 ` Li Wang via ltp
@ 2025-12-16 11:23 ` Cyril Hrubis
2025-12-16 11:24 ` Li Wang via ltp
2025-12-21 10:30 ` Petr Vorel
2 siblings, 1 reply; 25+ messages in thread
From: Cyril Hrubis @ 2025-12-16 11:23 UTC (permalink / raw)
To: Li Wang; +Cc: ltp
Hi!
> It's really good to write these rules down, especially since maintainers
> can reuse them in patch reviews to comment on issues and avoid repeatedly
> responding to the same questions. Below are what I can think of:
>
> Don’t require root unless it’s essential
> ============================
> If root/caps are needed, say why in the test output. Drop privileges for
> the part that doesn’t need them (and avoid running the whole test as
> root “because it’s easier”).
>
>
> Always clean up, even on failure
> ==========================
> Every test should leave the system as it found it: unmount, restore sysctls,
> delete temp files/dirs, kill spawned processes, remove cgroups/namespaces,
> detach loop devices, restore ulimits, etc. Cleanup must run on early-exit
> paths too.
>
>
> Respect LTP portability goals
> ===========================
> Avoid nonstandard libc APIs when a portable equivalent exists; don’t assume
> 64-bit,
> page size, endianness, or particular tool versions.
Shall I send a V3 with these included, or do you want to send a patch on
the top of the one I send?
--
Cyril Hrubis
chrubis@suse.cz
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-16 11:23 ` Cyril Hrubis
@ 2025-12-16 11:24 ` Li Wang via ltp
0 siblings, 0 replies; 25+ messages in thread
From: Li Wang via ltp @ 2025-12-16 11:24 UTC (permalink / raw)
To: Cyril Hrubis; +Cc: ltp
On Tue, Dec 16, 2025 at 7:22 PM Cyril Hrubis <chrubis@suse.cz> wrote:
> Hi!
> > It's really good to write these rules down, especially since maintainers
> > can reuse them in patch reviews to comment on issues and avoid repeatedly
> > responding to the same questions. Below are what I can think of:
> >
> > Don’t require root unless it’s essential
> > ============================
> > If root/caps are needed, say why in the test output. Drop privileges for
> > the part that doesn’t need them (and avoid running the whole test as
> > root “because it’s easier”).
> >
> >
> > Always clean up, even on failure
> > ==========================
> > Every test should leave the system as it found it: unmount, restore
> sysctls,
> > delete temp files/dirs, kill spawned processes, remove
> cgroups/namespaces,
> > detach loop devices, restore ulimits, etc. Cleanup must run on early-exit
> > paths too.
> >
> >
> > Respect LTP portability goals
> > ===========================
> > Avoid nonstandard libc APIs when a portable equivalent exists; don’t
> assume
> > 64-bit,
> > page size, endianness, or particular tool versions.
>
> Shall I send a V3 with these included, or do you want to send a patch on
> the top of the one I send?
>
Cyril, you can add those topics we discussed into your patch v3 directly.
--
Regards,
Li Wang
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 13:32 ` Li Wang via ltp
2025-12-15 14:03 ` Li Wang via ltp
2025-12-16 11:23 ` Cyril Hrubis
@ 2025-12-21 10:30 ` Petr Vorel
2 siblings, 0 replies; 25+ messages in thread
From: Petr Vorel @ 2025-12-21 10:30 UTC (permalink / raw)
To: Li Wang; +Cc: ltp
Hi all,
> It's really good to write these rules down, especially since maintainers
> can reuse them in patch reviews to comment on issues and avoid repeatedly
> responding to the same questions. Below are what I can think of:
+1
> Don’t require root unless it’s essential
> ============================
> If root/caps are needed, say why in the test output. Drop privileges for
> the part that doesn’t need them (and avoid running the whole test as
> root “because it’s easier”).
> Always clean up, even on failure
> ==========================
> Every test should leave the system as it found it: unmount, restore sysctls,
> delete temp files/dirs, kill spawned processes, remove cgroups/namespaces,
> detach loop devices, restore ulimits, etc. Cleanup must run on early-exit
> paths too.
> Respect LTP portability goals
> ===========================
> Avoid nonstandard libc APIs when a portable equivalent exists; don’t assume
> 64-bit,
> page size, endianness, or particular tool versions.
+1.
I would also mention POSIX shell compatibility.
Maybe also mention 'make check'.
"No regressions"
================
Each commit needs to compile. None commit can actually deliberately break tests.
As the document is getting quite big, I'd put AI policy into separate page.
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 12:44 ` [LTP] [PATCH v2 2/2] doc: Add ground rules page Cyril Hrubis
2025-12-15 13:32 ` Li Wang via ltp
@ 2025-12-15 14:52 ` Andrea Cervesato via ltp
2025-12-16 10:54 ` Petr Vorel
2025-12-16 11:01 ` Petr Vorel
3 siblings, 0 replies; 25+ messages in thread
From: Andrea Cervesato via ltp @ 2025-12-15 14:52 UTC (permalink / raw)
To: Cyril Hrubis, ltp
Hi!
In general I like the idea, but many sentences can be shorten. I didn't
use AI for this review, but usually they work pretty well on spotting
errors and english mistakes and it might be a good idea to review it
first with that.
Some comments below.
On Mon Dec 15, 2025 at 1:44 PM CET, Cyril Hrubis wrote:
> This is a continued effort to write down the unwritten rules we have in
> the project. Feel free to suggest more topics for the page.
>
> Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
> ---
> doc/developers/ground_rules.rst | 91 +++++++++++++++++++++++++++++++++
> doc/index.rst | 1 +
> 2 files changed, 92 insertions(+)
> create mode 100644 doc/developers/ground_rules.rst
>
> Changes in v2:
>
> - added two more rules
> - fixes and typos as pointed out by Peter
>
> diff --git a/doc/developers/ground_rules.rst b/doc/developers/ground_rules.rst
> new file mode 100644
> index 000000000..2bef426aa
> --- /dev/null
> +++ b/doc/developers/ground_rules.rst
> @@ -0,0 +1,91 @@
> +.. SPDX-License-Identifier: GPL-2.0-or-later
> +
> +Ground Rules
> +============
> +
> +Do not work around kernel bugs
> +------------------------------
> +
> +We have decided what we will not work around bugs in upstream LTP sources. If a
> +test fails on your system for a good reason, e.g. patch wasn't backported and
> +the bug is present, work around for this will not be accepted upstream. The
> +main reason for this decision is that this masks the failure for everyone else.
> +
> +
> +Do not synchronize by sleep
> +---------------------------
> +
> +Why is sleep in tests bad then?
> +```````````````````````````````
> +
> +The first problem is that it will likely introduce very rare test failures,
> +that means somebody has to spend time looking into these, which is a wasted
> +effort. Nobody likes tests that will fail rarely for no good reason. Even more
> +so you cannot run such tests with a background load to ensure that everything
> +works correctly on a busy system, because that will increase the likehood of a
> +failure.
> +
> +The second problem is that this wastes resources and slows down a test run. If
> +you think that adding a sleep to a test is not a big deal, lets have a look at
> +the bigger perspective. There is about 1600 syscall tests in Linux Test
> +Project, if 7.5% of them would sleep just for one second, we would end up with
> +two minutes of wasted time per testrun. In practice most of the test, that
> +historically misused sleep for synchronization, waited for much longer just to
> +be sure that things will works even on slower hardware. With sleeps between 2
> +and 5 seconds that puts us somewhere between 4 and 10 minutes which is between
> +13% and 33% of the syscall runtime on an outdated thinkpad, where the run
> +finishes in a bit less than half an hour. It's even worse on newer hardware,
> +because this slowdown will not change no matter how fast your machine is, which
> +is maybe the reason why this was acceptable twenty years ago but it's not now.
> +
> +
> +What to do instead?
> +```````````````````
> +
> +Use proper synchronization.
> +
> +There are different problems and different solutions. Most often test needs to
> +synchronize between child and parent proces.
> +
> +The easiest case is that parent needs to wait for a child to finish, that can
> +be fixed just be adding a :man2:`waitpid` in the parent which ensures that child is
^ by ^ . This ensures
> +finished before parent runs.
> +
> +Commonly child has to execute certain piece of code before parent can continue.
^ ,
> +For that LTP library implements checkpoints with simple
In these cases, LTP library implements..
> +:c:func:`TST_CHECKPOINT_WAIT()` and :c:func:`TST_CHECKPOINT_WAKE()` functions based
> +on futexes on a piece of shared memory set up by the test library.
> +
> +Another common case is where child must sleep in a syscall before parent can
> +continue, for which we have a :c:func:`TST_PROCESS_STATE_WAIT()` helper that
^ . In this case we have..
> +polls `/proc/$PID/stat`.
> +
> +Less often test needs to wait for an action that is done asynchronously, or a
> +kernel resource deallocation is deferred to a later time. In such cases the
> +best we can do is to poll. In LTP we ended up with a macro that polls by
> +calling a piece of code in a loop with exponentially increasing sleeps between
> +retries until it succeeds. Which means that instead of sleeping for a maximal
> +time event can possibly take the sleep is capped by twice of the optimal
> +sleeping time while we avoid polling too aggressively.
Less often, tests need to wait for an action that is done asynchronously
or they need to wait for a kernel resource deallocation. In such cases,
the best we can do is using polling mechanism. In LTP we have a macro
that loops over a piece of code, exponentially increasing sleeps between
retries until succeed.
-> it would be nice to add the link to it.
----
For the entire section I would put a bullet points list.
> +
> +
> +Use runtime checks for kernel features
> +======================================
> +
> +What is and what isn't supported by kernel is determined by the version
> +and configuration of the kernel the systems is currently running on.
Supported kernel features are determined by version and configuration
of the kernel running on the system.
> +That especially means that any checks done during the compilation cannot
> +be used to assume features supported by the kernel the tests end up
> +running on. The compile time checks, done by configure script, are only
> +useful for enabling fallback kernel API definitions when missing, as we
> +do in lapi/ directory.
This means that kernel features can't be always checked at compile-time.
Please keep compile-time checks only for fallback kernel API definitions,
when needed. -> (reference to lapi?)
> +
> +
> +Kernel features and RCs
> +=======================
> +
> +LTP tests or fixes for kernel changes that were not released yet can be posted
> +to the LTP list for a review but will not be be accepted until respective
> +kernel changes are released. Review of such changes is also considered to be
> +lower priority than rest of the changes. This is because kernel changes
> +especially in the early RC phase are volatile and could be changed or reverted.
LTP tests for new kernel features inside the mainline can be sent to the
mailing list, but they will get a low priority in the review process due
to the highly volatile nature of Linux kernel before release.
> diff --git a/doc/index.rst b/doc/index.rst
> index 06b75616f..659549cc3 100644
> --- a/doc/index.rst
> +++ b/doc/index.rst
> @@ -19,6 +19,7 @@
> :hidden:
> :caption: For developers
>
> + developers/ground_rules
> developers/setup_mailinglist
> developers/writing_tests
> developers/test_case_tutorial
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
andrea.cervesato@suse.com
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 12:44 ` [LTP] [PATCH v2 2/2] doc: Add ground rules page Cyril Hrubis
2025-12-15 13:32 ` Li Wang via ltp
2025-12-15 14:52 ` Andrea Cervesato via ltp
@ 2025-12-16 10:54 ` Petr Vorel
2025-12-16 11:01 ` Petr Vorel
3 siblings, 0 replies; 25+ messages in thread
From: Petr Vorel @ 2025-12-16 10:54 UTC (permalink / raw)
To: Cyril Hrubis; +Cc: ltp
Hi Cyril,
Thanks, very nice doc!
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Few more style/typo/grammar notes below. Found by chatgpt, I picked just few of
them which looked to me significant (obviously AI would improve the style, but
lets ignore that). Feel free to ignore, otherwise just amend before merge.
Labels "typo:" or "grammar:" should probably be fixed.
AI also suggests to use often articles, e.g. for "parent" and "child" (mostly "the",
sometimes "a"), but let's ignore that.
> +We have decided what we will not work around bugs in upstream LTP sources. If a
decided what => decided that
...
> +Why is sleep in tests bad then?
Missing "using", also maybe more formal (it's not a blog post but official doc):
Why is using sleep in tests bad?
> +```````````````````````````````
...
> +The second problem is that this wastes resources and slows down a test run. If
> +you think that adding a sleep to a test is not a big deal, lets have a look at
> +the bigger perspective. There is about 1600 syscall tests in Linux Test
grammar (plural): There is about 1600 => There are about 1600
(You have it wrong also in your blog post.)
> +Project, if 7.5% of them would sleep just for one second, we would end up with
> +two minutes of wasted time per testrun. In practice most of the test, that
> +historically misused sleep for synchronization, waited for much longer just to
> +be sure that things will works even on slower hardware. With sleeps between 2
> +and 5 seconds that puts us somewhere between 4 and 10 minutes which is between
And here "sleeps" is plural, maybe: puts => put.
Or maybe just use "sleep"?
> +13% and 33% of the syscall runtime on an outdated thinkpad, where the run
> +finishes in a bit less than half an hour. It's even worse on newer hardware,
> +because this slowdown will not change no matter how fast your machine is, which
> +is maybe the reason why this was acceptable twenty years ago but it's not now.
> +
> +
> +What to do instead?
> +```````````````````
> +
> +Use proper synchronization.
> +
> +There are different problems and different solutions. Most often test needs to
> +synchronize between child and parent proces.
typo: proces => process
> +
> +The easiest case is that parent needs to wait for a child to finish, that can
that parent => when parent
> +be fixed just be adding a :man2:`waitpid` in the parent which ensures that child is
> +finished before parent runs.
"child is finished" => "child finished" (or "child has finished")
> +
> +Commonly child has to execute certain piece of code before parent can continue.
> +For that LTP library implements checkpoints with simple
> +:c:func:`TST_CHECKPOINT_WAIT()` and :c:func:`TST_CHECKPOINT_WAKE()` functions based
> +on futexes on a piece of shared memory set up by the test library.
> +
> +Another common case is where child must sleep in a syscall before parent can
"where child" => "when child"
> +continue, for which we have a :c:func:`TST_PROCESS_STATE_WAIT()` helper that
> +polls `/proc/$PID/stat`.
> +
> +Less often test needs to wait for an action that is done asynchronously, or a
> +kernel resource deallocation is deferred to a later time. In such cases the
"a kernel resource deallocation is deferred to a later time"
Maybe:
"for kernel resource deallocation that is deferred to a later time"
> +best we can do is to poll. In LTP we ended up with a macro that polls by
> +calling a piece of code in a loop with exponentially increasing sleeps between
> +retries until it succeeds. Which means that instead of sleeping for a maximal
> +time event can possibly take the sleep is capped by twice of the optimal
> +sleeping time while we avoid polling too aggressively.
> +
> +
> +Use runtime checks for kernel features
> +======================================
> +
> +What is and what isn't supported by kernel is determined by the version
> +and configuration of the kernel the systems is currently running on.
grammar (singular): "systems is ... running on." => "system is ... running."
> +That especially means that any checks done during the compilation cannot
> +be used to assume features supported by the kernel the tests end up
> +running on. The compile time checks, done by configure script, are only
> +useful for enabling fallback kernel API definitions when missing, as we
> +do in lapi/ directory.
> +
> +
> +Kernel features and RCs
> +=======================
> +
> +LTP tests or fixes for kernel changes that were not released yet can be posted
grammar (present perfect): "were not released yet can" => "have not yet been released may"
> +to the LTP list for a review but will not be be accepted until respective
"review but will" => "review but they will"
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [LTP] [PATCH v2 2/2] doc: Add ground rules page
2025-12-15 12:44 ` [LTP] [PATCH v2 2/2] doc: Add ground rules page Cyril Hrubis
` (2 preceding siblings ...)
2025-12-16 10:54 ` Petr Vorel
@ 2025-12-16 11:01 ` Petr Vorel
3 siblings, 0 replies; 25+ messages in thread
From: Petr Vorel @ 2025-12-16 11:01 UTC (permalink / raw)
To: Cyril Hrubis; +Cc: ltp
Hi Cyril,
Few more notes.
...
> +Use runtime checks for kernel features
> +======================================
> +
> +What is and what isn't supported by kernel is determined by the version
> +and configuration of the kernel the systems is currently running on.
> +That especially means that any checks done during the compilation cannot
> +be used to assume features supported by the kernel the tests end up
> +running on. The compile time checks, done by configure script, are only
Also please use link to the script:
-running on. The compile time checks, done by configure script, are only
+running on. The compile time checks, done by :master:`configure.ac`, are only
> +useful for enabling fallback kernel API definitions when missing, as we
> +do in lapi/ directory.
And to the lapi directory:
-do in lapi/ directory.
+do in headers in :master:`include/lapi/`.
(I'm not sure whether you just overlooked it or you don't see it useful.)
> +Kernel features and RCs
> +=======================
> +
> +LTP tests or fixes for kernel changes that were not released yet can be posted
> +to the LTP list for a review but will not be be accepted until respective
> +kernel changes are released. Review of such changes is also considered to be
> +lower priority than rest of the changes. This is because kernel changes
> +especially in the early RC phase are volatile and could be changed or reverted.
I would personally mention staging runtest file, but maybe I overlook your
:master:`runtest/staging`
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 25+ messages in thread