From: Thomas Gleixner <tglx@linutronix.de>
To: Mateusz Guzik <mjguzik@gmail.com>
Cc: kernel test robot <oliver.sang@intel.com>,
oe-lkp@lists.linux.dev, lkp@intel.com,
linux-kernel@vger.kernel.org, x86@kernel.org,
Eric Dumazet <edumazet@google.com>,
Benjamin Segall <bsegall@google.com>,
Frederic Weisbecker <frederic@kernel.org>
Subject: Re: [tip:timers/core] [posix] 1535cb8028: stress-ng.epoll.ops_per_sec 36.2% regression
Date: Wed, 26 Mar 2025 22:43:32 +0100 [thread overview]
Message-ID: <87cye3zcvv.ffs@tglx> (raw)
In-Reply-To: <6sn76aya225pqikijue5uv5h3lyqk262hc6ru3vemn7xofdftd@sw7gith52xh7>
On Wed, Mar 26 2025 at 22:11, Mateusz Guzik wrote:
> On Wed, Mar 26, 2025 at 09:07:51AM +0100, Thomas Gleixner wrote:
>> How on earth can this commit result in both a 36% regression and a 25%
>> improvement with the same test?
>>
>> Unfortunately I can't reproduce any of it. I checked the epoll test
>> source and it uses a posix timer, but that commit makes the hash less
>> contended so there is zero explanation.
>>
>
> The short summary is:
> 1. your change is fine
> 2. stress-ng is doing seriously weird stuff here resulting in the above
> 3. there may or may not be something the scheduler can do to help
>
> for the regression stats are saying:
> feb864ee99a2d8a2 1535cb80286e6fbc834f075039f
> ---------------- ---------------------------
> %stddev %change %stddev
> \ | \
> 5.97 ± 56% +35.8 41.74 ± 24% mpstat.cpu.all.idle%
> 0.86 ± 3% -0.3 0.51 ± 11% mpstat.cpu.all.irq%
> 0.10 ± 3% +2.0 2.11 ± 13% mpstat.cpu.all.soft%
> 92.01 ± 3% -37.7 54.27 ± 18% mpstat.cpu.all.sys%
> 1.06 ± 3% +0.3 1.37 ± 8% mpstat.cpu.all.usr%
> 27.83 ± 38% -84.4% 4.33 ± 31% mpstat.max_utilization.seconds
>
> As in system time went down and idle went up.
>
> Your patch must have a side effect where it messes with some of the
> timings between workers.
It does as it removes the global lock and the potential contention on
it.
> The testcase is doing a lot of weird stuff, including calling yield()
> for every loop iteration. On top of that if the other worker does not
> win the race there is also a sleep of 0.1s thrown in. I commented these
> suckers out and weird anomalies persisted.
>
> All that said, I'm not going to further look into it. Was curious wtf
> though hence the write up.
Thak you for taking the time and looking into this. The analysis of this
"benchmark" is a fun read and I agree that it matches my impression of
looking into the source of this thing that it does weird stuff, which
does not make any sense at all.
Thanks,
tglx
next prev parent reply other threads:[~2025-03-26 21:43 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-03-24 6:39 [tip:timers/core] [posix] 1535cb8028: stress-ng.epoll.ops_per_sec 36.2% regression kernel test robot
2025-03-26 8:07 ` Thomas Gleixner
2025-03-26 21:11 ` Mateusz Guzik
2025-03-26 21:43 ` Thomas Gleixner [this message]
2025-03-27 6:21 ` Eric Dumazet
2025-03-27 8:10 ` Thomas Gleixner
2025-03-27 8:26 ` Eric Dumazet
2025-03-27 9:11 ` Eric Dumazet
2025-03-27 10:50 ` Thomas Gleixner
2025-03-27 11:37 ` Eric Dumazet
2025-03-27 13:14 ` Thomas Gleixner
2025-03-27 13:17 ` Eric Dumazet
2025-03-27 13:43 ` Mateusz Guzik
2025-03-27 13:44 ` Eric Dumazet
2025-03-27 13:48 ` Mateusz Guzik
2025-03-27 20:45 ` David Laight
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87cye3zcvv.ffs@tglx \
--to=tglx@linutronix.de \
--cc=bsegall@google.com \
--cc=edumazet@google.com \
--cc=frederic@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lkp@intel.com \
--cc=mjguzik@gmail.com \
--cc=oe-lkp@lists.linux.dev \
--cc=oliver.sang@intel.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.