[LTP] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[LTP] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Petr Vorel]

LTP_REPRODUCIBLE_OUTPUT=1 environment variable was added in 5894abc5f to
help valgrind to use LTP for testing. It discards the actual content of
the messages printed by the test (removes everything after printing
TINFO/TPASS/TFAIL/...  flag).

Later LTP_QUIET=1 was added which suppresses printing TCONF, TWARN,
TINFO, and TDEBUG messages. Therefore LTP_REPRODUCIBLE_OUTPUT=1 is not
needed any more.

Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
Hi Dan,

is it ok to remove LTP_REPRODUCIBLE_OUTPUT=1 from your tooling? Feel
free shout if it's needed/useful. If you delete it, I'd then send patch
to delete it from LTP.

Kind regards,
Petr

 auxprogs/ltp-tester.sh | 1 -
 1 file changed, 1 deletion(-)

diff --git a/auxprogs/ltp-tester.sh b/auxprogs/ltp-tester.sh
index a95c603c5..077304967 100755
--- a/auxprogs/ltp-tester.sh
+++ b/auxprogs/ltp-tester.sh
@@ -20,7 +20,6 @@ PARALLEL_JOBS=${PARALLEL_JOBS:-$(nproc)}
 # Configure the LTP testsuite behavior per
 # https://lore.kernel.org/ltp/20250505195003.GB137650@pevik/T/#t
 export LTP_COLORIZE_OUTPUT=0
-export LTP_REPRODUCIBLE_OUTPUT=1
 export LTP_QUIET=1
 
 # Initialize LOGDIR for bunsen upload (https://sourceware.org/bunsen/)
-- 
2.53.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Cyril Hrubis]

Hi!
> LTP_REPRODUCIBLE_OUTPUT=1 environment variable was added in 5894abc5f to
> help valgrind to use LTP for testing. It discards the actual content of
> the messages printed by the test (removes everything after printing
> TINFO/TPASS/TFAIL/...  flag).
> 
> Later LTP_QUIET=1 was added which suppresses printing TCONF, TWARN,
> TINFO, and TDEBUG messages. Therefore LTP_REPRODUCIBLE_OUTPUT=1 is not
> needed any more.

I do not think so. We still have things like pids and timestamps in
TPASS/TFAIL messages that needs to be removed in order for the output to
be reproducible.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [Valgrind-developers] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Martin Cermak via ltp]

On  Tue  2026-03-24  14:18 , Cyril Hrubis wrote:
> Hi!
> > LTP_REPRODUCIBLE_OUTPUT=1 environment variable was added in 5894abc5f to
> > help valgrind to use LTP for testing. It discards the actual content of
> > the messages printed by the test (removes everything after printing
> > TINFO/TPASS/TFAIL/...  flag).
> > 
> > Later LTP_QUIET=1 was added which suppresses printing TCONF, TWARN,
> > TINFO, and TDEBUG messages. Therefore LTP_REPRODUCIBLE_OUTPUT=1 is not
> > needed any more.
> 
> I do not think so. We still have things like pids and timestamps in
> TPASS/TFAIL messages that needs to be removed in order for the output to
> be reproducible.

From Valgrind persp, seems like we could drop it at the first
glance.  But let me confirm that based on a thorough check.

Thanks,
Martin


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [Valgrind-developers] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Martin Cermak via ltp]

On  Tue  2026-03-24  15:09 , Martin Cermak wrote:
> On  Tue  2026-03-24  14:18 , Cyril Hrubis wrote:
> > Hi!
> > > LTP_REPRODUCIBLE_OUTPUT=1 environment variable was added in 5894abc5f to
> > > help valgrind to use LTP for testing. It discards the actual content of
> > > the messages printed by the test (removes everything after printing
> > > TINFO/TPASS/TFAIL/...  flag).
> > > 
> > > Later LTP_QUIET=1 was added which suppresses printing TCONF, TWARN,
> > > TINFO, and TDEBUG messages. Therefore LTP_REPRODUCIBLE_OUTPUT=1 is not
> > > needed any more.
> > 
> > I do not think so. We still have things like pids and timestamps in
> > TPASS/TFAIL messages that needs to be removed in order for the output to
> > be reproducible.
> 
> From Valgrind persp, seems like we could drop it at the first
> glance.  But let me confirm that based on a thorough check.

My quick guess above was wrong.  We do need
LTP_REPRODUCIBLE_OUTPUT for Valgrind testing.
Please keep it.

Thank-you,
Martin



-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [Valgrind-developers] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Petr Vorel]

> On  Tue  2026-03-24  15:09 , Martin Cermak wrote:
> > On  Tue  2026-03-24  14:18 , Cyril Hrubis wrote:
> > > Hi!
> > > > LTP_REPRODUCIBLE_OUTPUT=1 environment variable was added in 5894abc5f to
> > > > help valgrind to use LTP for testing. It discards the actual content of
> > > > the messages printed by the test (removes everything after printing
> > > > TINFO/TPASS/TFAIL/...  flag).

> > > > Later LTP_QUIET=1 was added which suppresses printing TCONF, TWARN,
> > > > TINFO, and TDEBUG messages. Therefore LTP_REPRODUCIBLE_OUTPUT=1 is not
> > > > needed any more.

> > > I do not think so. We still have things like pids and timestamps in
> > > TPASS/TFAIL messages that needs to be removed in order for the output to
> > > be reproducible.

> > From Valgrind persp, seems like we could drop it at the first
> > glance.  But let me confirm that based on a thorough check.

> My quick guess above was wrong.  We do need
> LTP_REPRODUCIBLE_OUTPUT for Valgrind testing.
> Please keep it.

Thanks for info, Martin. Sure, please drop this patch.

I still feel these 2 variables somehow overlap, wouldn't it make sense to merge
the functionality into a single variable?

Kind regards,
Petr

> Thank-you,
> Martin



-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [Valgrind-developers] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Martin Cermak via ltp]

On  Wed  2026-03-25  10:10 , Petr Vorel wrote:
> > On  Tue  2026-03-24  15:09 , Martin Cermak wrote:
> > > On  Tue  2026-03-24  14:18 , Cyril Hrubis wrote:
> > > > Hi!
> > > > > LTP_REPRODUCIBLE_OUTPUT=1 environment variable was added in 5894abc5f to
> > > > > help valgrind to use LTP for testing. It discards the actual content of
> > > > > the messages printed by the test (removes everything after printing
> > > > > TINFO/TPASS/TFAIL/...  flag).
> 
> > > > > Later LTP_QUIET=1 was added which suppresses printing TCONF, TWARN,
> > > > > TINFO, and TDEBUG messages. Therefore LTP_REPRODUCIBLE_OUTPUT=1 is not
> > > > > needed any more.
> 
> > > > I do not think so. We still have things like pids and timestamps in
> > > > TPASS/TFAIL messages that needs to be removed in order for the output to
> > > > be reproducible.
> 
> > > From Valgrind persp, seems like we could drop it at the first
> > > glance.  But let me confirm that based on a thorough check.
> 
> > My quick guess above was wrong.  We do need
> > LTP_REPRODUCIBLE_OUTPUT for Valgrind testing.
> > Please keep it.
> 
> Thanks for info, Martin. Sure, please drop this patch.
> 
> I still feel these 2 variables somehow overlap, wouldn't it make sense to merge
> the functionality into a single variable?

That sounds reasonable to me.  Which one would you prefer to
keep?  I'm inclined to suggest keeping LTP_REPRODUCIBLE_OUTPUT
and dropping LTP_QUIET but no really strong preference on my
side.  Thoughts?

Martin


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [Valgrind-developers] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Cyril Hrubis]

Hi!
> > I still feel these 2 variables somehow overlap, wouldn't it make sense to merge
> > the functionality into a single variable?
> 
> That sounds reasonable to me.  Which one would you prefer to
> keep?  I'm inclined to suggest keeping LTP_REPRODUCIBLE_OUTPUT
> and dropping LTP_QUIET but no really strong preference on my
> side.  Thoughts?

Maybe we can finally implement proper debug levels in LTP. E.g. hide
TINFO and TWARN messages with debug level 0 and default to debug level
1. With debug level 2 we would show TDEBUG as well and with debug level
3 we would show TDEBUG messages for library too.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [Valgrind-developers] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Martin Cermak via ltp]

On  Wed  2026-03-25  10:37 , Cyril Hrubis wrote:
> Hi!
> > > I still feel these 2 variables somehow overlap, wouldn't it make sense to merge
> > > the functionality into a single variable?
> > 
> > That sounds reasonable to me.  Which one would you prefer to
> > keep?  I'm inclined to suggest keeping LTP_REPRODUCIBLE_OUTPUT
> > and dropping LTP_QUIET but no really strong preference on my
> > side.  Thoughts?
> 
> Maybe we can finally implement proper debug levels in LTP. E.g. hide
> TINFO and TWARN messages with debug level 0 and default to debug level
> 1. With debug level 2 we would show TDEBUG as well and with debug level
> 3 we would show TDEBUG messages for library too.

That would require running valgrind tests with something like
debug level -1 ;) because what LTP_REPRODUCIBLE_OUTPUT actually
does is that it hides testrun specific output like e.g. testrun
specific addresses, temporary file names etc.

But I think a regular human user of LTP expects these to be
printed out.  So that should be the default imho.

Martin


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [Valgrind-developers] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Cyril Hrubis]

Hi!
> > > That sounds reasonable to me.  Which one would you prefer to
> > > keep?  I'm inclined to suggest keeping LTP_REPRODUCIBLE_OUTPUT
> > > and dropping LTP_QUIET but no really strong preference on my
> > > side.  Thoughts?
> > 
> > Maybe we can finally implement proper debug levels in LTP. E.g. hide
> > TINFO and TWARN messages with debug level 0 and default to debug level
> > 1. With debug level 2 we would show TDEBUG as well and with debug level
> > 3 we would show TDEBUG messages for library too.
> 
> That would require running valgrind tests with something like
> debug level -1 ;) because what LTP_REPRODUCIBLE_OUTPUT actually
> does is that it hides testrun specific output like e.g. testrun
> specific addresses, temporary file names etc.

The LTP_REPRODUCIBLE_OUTPUT would have to stay, but the LTP_QUITE would
have been replaced by different debug levels.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [Valgrind-developers] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Martin Cermak via ltp]

On  Wed  2026-03-25  10:57 , Cyril Hrubis wrote:
> Hi!
> > > > That sounds reasonable to me.  Which one would you prefer to
> > > > keep?  I'm inclined to suggest keeping LTP_REPRODUCIBLE_OUTPUT
> > > > and dropping LTP_QUIET but no really strong preference on my
> > > > side.  Thoughts?
> > > 
> > > Maybe we can finally implement proper debug levels in LTP. E.g. hide
> > > TINFO and TWARN messages with debug level 0 and default to debug level
> > > 1. With debug level 2 we would show TDEBUG as well and with debug level
> > > 3 we would show TDEBUG messages for library too.
> > 
> > That would require running valgrind tests with something like
> > debug level -1 ;) because what LTP_REPRODUCIBLE_OUTPUT actually
> > does is that it hides testrun specific output like e.g. testrun
> > specific addresses, temporary file names etc.
> 
> The LTP_REPRODUCIBLE_OUTPUT would have to stay, but the LTP_QUITE would
> have been replaced by different debug levels.

That sounds like a good approach to me, yup.

m.


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [Valgrind-developers] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Petr Vorel]

> Hi!
> > > > That sounds reasonable to me.  Which one would you prefer to
> > > > keep?  I'm inclined to suggest keeping LTP_REPRODUCIBLE_OUTPUT
> > > > and dropping LTP_QUIET but no really strong preference on my
> > > > side.  Thoughts?

> > > Maybe we can finally implement proper debug levels in LTP. E.g. hide
> > > TINFO and TWARN messages with debug level 0 and default to debug level
> > > 1. With debug level 2 we would show TDEBUG as well and with debug level
> > > 3 we would show TDEBUG messages for library too.

> > That would require running valgrind tests with something like
> > debug level -1 ;) because what LTP_REPRODUCIBLE_OUTPUT actually
> > does is that it hides testrun specific output like e.g. testrun
> > specific addresses, temporary file names etc.

> The LTP_REPRODUCIBLE_OUTPUT would have to stay, but the LTP_QUITE would
> have been replaced by different debug levels.

I thought just remove LTP_QUITE and control everything with
LTP_REPRODUCIBLE_OUTPUT.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [Valgrind-developers] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Martin Cermak via ltp]

On  Wed  2026-03-25  14:57 , Petr Vorel wrote:
> > Hi!
> > > > > That sounds reasonable to me.  Which one would you prefer to
> > > > > keep?  I'm inclined to suggest keeping LTP_REPRODUCIBLE_OUTPUT
> > > > > and dropping LTP_QUIET but no really strong preference on my
> > > > > side.  Thoughts?
> 
> > > > Maybe we can finally implement proper debug levels in LTP. E.g. hide
> > > > TINFO and TWARN messages with debug level 0 and default to debug level
> > > > 1. With debug level 2 we would show TDEBUG as well and with debug level
> > > > 3 we would show TDEBUG messages for library too.
> 
> > > That would require running valgrind tests with something like
> > > debug level -1 ;) because what LTP_REPRODUCIBLE_OUTPUT actually
> > > does is that it hides testrun specific output like e.g. testrun
> > > specific addresses, temporary file names etc.
> 
> > The LTP_REPRODUCIBLE_OUTPUT would have to stay, but the LTP_QUITE would
> > have been replaced by different debug levels.
> 
> I thought just remove LTP_QUITE and control everything with
> LTP_REPRODUCIBLE_OUTPUT.

Sounds fine, Petr.  For Valgrind testing, the functionality of
both LTP_REPRODUCIBLE_OUTPUT and LTP_QUIET is useful.  But
there's no need to control the two features independently.
Merging the functionality under one controlling env var works
perfectly fine from the Valgrind testsuite perspective.

Thanks,
Martin


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [Valgrind-developers] [RFC PATCH 1/1] LTP tests: Run without LTP_REPRODUCIBLE_OUTPUT=1

[Petr Vorel]

> On  Wed  2026-03-25  14:57 , Petr Vorel wrote:
> > > Hi!
> > > > > > That sounds reasonable to me.  Which one would you prefer to
> > > > > > keep?  I'm inclined to suggest keeping LTP_REPRODUCIBLE_OUTPUT
> > > > > > and dropping LTP_QUIET but no really strong preference on my
> > > > > > side.  Thoughts?

> > > > > Maybe we can finally implement proper debug levels in LTP. E.g. hide
> > > > > TINFO and TWARN messages with debug level 0 and default to debug level
> > > > > 1. With debug level 2 we would show TDEBUG as well and with debug level
> > > > > 3 we would show TDEBUG messages for library too.

> > > > That would require running valgrind tests with something like
> > > > debug level -1 ;) because what LTP_REPRODUCIBLE_OUTPUT actually
> > > > does is that it hides testrun specific output like e.g. testrun
> > > > specific addresses, temporary file names etc.

> > > The LTP_REPRODUCIBLE_OUTPUT would have to stay, but the LTP_QUITE would
> > > have been replaced by different debug levels.

> > I thought just remove LTP_QUITE and control everything with
> > LTP_REPRODUCIBLE_OUTPUT.

> Sounds fine, Petr.  For Valgrind testing, the functionality of
> both LTP_REPRODUCIBLE_OUTPUT and LTP_QUIET is useful.  But
> there's no need to control the two features independently.
> Merging the functionality under one controlling env var works
> perfectly fine from the Valgrind testsuite perspective.

I prefer to keep LTP_REPRODUCIBLE_OUTPUT as variable name (with merged
functionality of both). 1) We have LTP_ENABLE_DEBUG, so one would expect
LTP_ENABLE_DEBUG and LTP_QUIET are mutually exclusive.
2) IMHO it's more about the reproducibility of the results.

Kind regards,
Petr

> Thanks,
> Martin

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp