Re: [PATCH v7 3/3] shmem: stable directory offsets

[Philip Li <philip.li@intel.com>]

On Tue, Jul 25, 2023 at 05:59:22PM +0200, Christian Brauner wrote:
> On Tue, Jul 25, 2023 at 11:54:26PM +0800, Philip Li wrote:
> > On Tue, Jul 25, 2023 at 03:12:22PM +0000, Chuck Lever III wrote:
> > > 
> > > 
> > > > On Jul 22, 2023, at 4:33 PM, Chuck Lever III <chuck.lever@oracle.com> wrote:
> > > > 
> > > > 
> > > > 
> > > >> On Jul 17, 2023, at 2:46 AM, kernel test robot <oliver.sang@intel.com> wrote:
> > > >> 
> > > >> 
> > > >> hi, Chuck Lever,
> > > >> 
> > > >> we reported a 3.0% improvement of stress-ng.handle.ops_per_sec for this commit
> > > >> on
> > > >> https://lore.kernel.org/oe-lkp/202307132153.a52cdb2d-oliver.sang@intel.com/
> > > >> 
> > > >> but now we noticed a regression, detail as below, FYI
> > > >> 
> > > >> Hello,
> > > >> 
> > > >> kernel test robot noticed a -15.5% regression of will-it-scale.per_thread_ops on:
> > > >> 
> > > >> 
> > > >> commit: a1a690e009744e4526526b2f838beec5ef9233cc ("[PATCH v7 3/3] shmem: stable directory offsets")
> > > >> url: https://github.com/intel-lab-lkp/linux/commits/Chuck-Lever/libfs-Add-directory-operations-for-stable-offsets/20230701-014925
> > > >> base: https://git.kernel.org/cgit/linux/kernel/git/akpm/mm.git mm-everything
> > > >> patch link: https://lore.kernel.org/all/168814734331.530310.3911190551060453102.stgit@manet.1015granger.net/
> > > >> patch subject: [PATCH v7 3/3] shmem: stable directory offsets
> > > >> 
> > > >> testcase: will-it-scale
> > > >> test machine: 104 threads 2 sockets (Skylake) with 192G memory
> > > >> parameters:
> > > >> 
> > > >> nr_task: 16
> > > >> mode: thread
> > > >> test: unlink2
> > > >> cpufreq_governor: performance
> > > >> 
> > > >> 
> > > >> In addition to that, the commit also has significant impact on the following tests:
> > > >> 
> > > >> +------------------+-------------------------------------------------------------------------------------------------+
> > > >> | testcase: change | will-it-scale: will-it-scale.per_thread_ops -40.0% regression                                   |
> > > >> | test machine     | 36 threads 1 sockets Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz (Cascade Lake) with 128G memory |
> > > >> | test parameters  | cpufreq_governor=performance                                                                    |
> > > >> |                  | mode=thread                                                                                     |
> > > >> |                  | nr_task=16                                                                                      |
> > > >> |                  | test=unlink2                                                                                    |
> > > >> +------------------+-------------------------------------------------------------------------------------------------+
> > > >> | testcase: change | stress-ng: stress-ng.handle.ops_per_sec 3.0% improvement                                        |
> > > >> | test machine     | 36 threads 1 sockets Intel(R) Core(TM) i9-9980XE CPU @ 3.00GHz (Skylake) with 32G memory        |
> > > >> | test parameters  | class=filesystem                                                                                |
> > > >> |                  | cpufreq_governor=performance                                                                    |
> > > >> |                  | disk=1SSD                                                                                       |
> > > >> |                  | fs=ext4                                                                                         |
> > > >> |                  | nr_threads=10%                                                                                  |
> > > >> |                  | test=handle                                                                                     |
> > > >> |                  | testtime=60s                                                                                    |
> > > >> +------------------+-------------------------------------------------------------------------------------------------+
> > > >> 
> > > >> 
> > > >> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> > > >> the same patch/commit), kindly add following tags
> > > >> | Reported-by: kernel test robot <oliver.sang@intel.com>
> > > >> | Closes: https://lore.kernel.org/oe-lkp/202307171436.29248fcf-oliver.sang@intel.com
> > > >> 
> > > >> 
> > > >> Details are as below:
> > > >> -------------------------------------------------------------------------------------------------->
> > > >> 
> > > >> 
> > > >> To reproduce:
> > > >> 
> > > >>       git clone https://github.com/intel/lkp-tests.git
> > > >>       cd lkp-tests
> > > >>       sudo bin/lkp install job.yaml           # job file is attached in this email
> > > 
> > > Has anyone from the lkp or ltp teams had a chance to look at this?
> > > I'm stuck without this reproducer.
> > 
> > Sorry about this that fedora is not fully supported now [1]. A possible way
> > is to run the test inside docker [2]. But we haven't fully tested the
> > reproduce steps in docker yet, which is in our TODO list. Also a concern is
> > that docker environment probably can't reproduce the performance regression.
> > 
> > For now, not sure whether it is convenient for you to have a ubuntu or debian
> > environment to give a try? Another alternative is if you have new patch, we
> > can assist to verify it on our machines.
> 
> So while we have your attention here. I've asked this a while ago in
> another mail: It would be really really helpful if there was a way for
> us to ask/trigger a perf test run for specific branches/patches we
> suspect of being performance sensitive.
> 
> It's a bit of a shame that we have no simple way of submitting a custom
> job and get performance results reported. I know that resources for this
> are probably scarce but some way to at least request it would be really
> really nice.

Apologize for this limitation. We have some mid-term TODO list to allow
the verification of reported issue (start from build report) for fix patch.

We will consider runtime side as well per this input to provide a better
experience. And we can start with a controlled scope like to queue the
test in the report, so test suite/parameters/platform is in a manageable
manner.

Thanks for the input.