Re: [linux-next:master] [drivers/char/mem] 1b057bd800: stress-ng.splice.ops_per_sec -99.8% regression

[Greg Kroah-Hartman <gregkh@linuxfoundation.org>]

On Wed, Oct 18, 2023 at 03:07:20PM +0800, Oliver Sang wrote:
> hi, Greg Kroah-Hartman,
> 
> On Tue, Oct 17, 2023 at 06:56:56PM +0200, Greg Kroah-Hartman wrote:
> > On Tue, Oct 17, 2023 at 11:06:42PM +0800, kernel test robot wrote:
> > > 
> > > 
> > > Hello,
> > > 
> > > kernel test robot noticed a -99.8% regression of stress-ng.splice.ops_per_sec on:
> > > 
> > > 
> > > commit: 1b057bd800c3ea0c926191d7950cd2365eddc9bb ("drivers/char/mem: implement splice() for /dev/zero, /dev/full")
> > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
> > > 
> > > testcase: stress-ng
> > > test machine: 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory
> > > parameters:
> > > 
> > > 	nr_threads: 100%
> > > 	testtime: 60s
> > > 	class: pipe
> > > 	test: splice
> > > 	cpufreq_governor: performance
> > > 
> > > 
> > > In addition to that, the commit also has significant impact on the following tests:
> > > 
> > > +------------------+-------------------------------------------------------------------------------------------------+
> > > | testcase: change | stress-ng: stress-ng.splice.ops_per_sec 38.9% improvement                                       |
> > 
> > So everything now goes faster, right?  -99.8% regression means 99.8%
> > faster?
> 
> let me clarify.
> 
> our auto bisect captured this commit as 'first bad commit' in two tests.
> 
> Test 1:
> 
> it found a (very big) regression comparing to parent commit.
> 
> 19e3e6cdfdc73400 1b057bd800c3ea0c926191d7950 
> ---------------- --------------------------- 
>          %stddev     %change         %stddev
>              \          |                \  
>   12433266           -99.8%      22893 ±  3%  stress-ng.splice.ops_per_sec
> 
> the detail data for parent in multi-runs:
>   "stress-ng.splice.ops_per_sec": [

stress-ng is a performance test?

>     12444442.19,
>     12599010.87,
>     12416009.38,
>     12494132.89,
>     12286766.76,
>     12359235.82
>   ],
> 
> for 1b057bd800:
>   "stress-ng.splice.ops_per_sec": [
>     24055.57,
>     23235.46,
>     22142.13,
>     23782.13,
>     21732.13,
>     22415.46
>   ],
> 
> so this is much slower.

That's odd given that as was pointed out, this test does not even touch
the code paths that this patch changed.

confused,

greg k-h