* [linus:master] [btrfs] b7fdfd29a1: postmark.transactions 9.5% regression
@ 2025-10-27 7:41 kernel test robot
2025-10-27 7:49 ` Qu Wenruo
0 siblings, 1 reply; 4+ messages in thread
From: kernel test robot @ 2025-10-27 7:41 UTC (permalink / raw)
To: Qu Wenruo
Cc: oe-lkp, lkp, linux-kernel, David Sterba, HAN Yuwei, linux-btrfs,
oliver.sang
Hello,
kernel test robot noticed a 9.5% regression of postmark.transactions on:
commit: b7fdfd29a136a17c5c8ad9e9bbf89c48919c3d19 ("btrfs: only set the device specific options after devices are opened")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
we are in fact not sure what's the connection between this change and the
postmark.transactions performance. still report out due to below checks.
[still regression on linus/master 4bb1f7e19c4a1d6eeb52b80acff5ac63edd1b91d]
[regression chould be solved by reverting this commit on linus/master head]
[still regression on linux-next/master 72fb0170ef1f45addf726319c52a0562b6913707]
testcase: postmark
config: x86_64-rhel-9.4
compiler: gcc-14
test machine: 224 threads 4 sockets Intel(R) Xeon(R) Platinum 8380H CPU @ 2.90GHz (Cooper Lake) with 192G memory
parameters:
disk: 1HDD
fs: btrfs
fs1: nfsv4
number: 4000n
trans: 10000s
subdirs: 100d
cpufreq_governor: performance
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <oliver.sang@intel.com>
| Closes: https://lore.kernel.org/oe-lkp/202510271449.efa21738-lkp@intel.com
Details are as below:
-------------------------------------------------------------------------------------------------->
The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20251027/202510271449.efa21738-lkp@intel.com
=========================================================================================
compiler/cpufreq_governor/disk/fs1/fs/kconfig/number/rootfs/subdirs/tbox_group/testcase/trans:
gcc-14/performance/1HDD/nfsv4/btrfs/x86_64-rhel-9.4/4000n/debian-13-x86_64-20250902.cgz/100d/lkp-cpl-4sp2/postmark/10000s
commit:
53a4acbfc1 ("btrfs: fix memory leak on duplicated memory in the qgroup assign ioctl")
b7fdfd29a1 ("btrfs: only set the device specific options after devices are opened")
53a4acbfc1de85fa b7fdfd29a136a17c5c8ad9e9bbf
---------------- ---------------------------
%stddev %change %stddev
\ | \
2010 +10.5% 2222 nfsstat.Client.nfs.v4.open_noat
97983 ± 11% +18.5% 116101 numa-numastat.node1.other_node
97983 ± 11% +18.5% 116101 numa-vmstat.node1.numa_other
16001 ± 5% -7.5% 14797 ± 5% sched_debug.cfs_rq:/.load.avg
1474354 ± 3% +9.9% 1620281 ± 4% sched_debug.cpu.avg_idle.avg
756151 ± 3% +10.0% 831539 ± 4% sched_debug.cpu.max_idle_balance_cost.avg
3585 -2.6% 3490 perf-stat.i.context-switches
6141 ± 2% +4.0% 6385 perf-stat.i.cycles-between-cache-misses
5796 ± 2% +3.9% 6024 perf-stat.overall.cycles-between-cache-misses
3580 -2.6% 3486 perf-stat.ps.context-switches
9.548e+11 +7.3% 1.025e+12 perf-stat.total.instructions
136494 +4.3% 142419 proc-vmstat.nr_inactive_file
136494 +4.3% 142419 proc-vmstat.nr_zone_inactive_file
2784208 +4.8% 2917180 proc-vmstat.numa_hit
2435763 +5.5% 2568673 proc-vmstat.numa_local
3042276 +5.1% 3196281 proc-vmstat.pgalloc_normal
2627503 +6.9% 2808220 proc-vmstat.pgfault
2754381 +5.4% 2903034 proc-vmstat.pgfree
97857 +6.3% 104058 proc-vmstat.pgreuse
9.80 -9.4% 8.88 postmark.creation_mixed_trans
112312 -7.0% 104473 postmark.data_read
203502 -7.0% 189298 postmark.data_written
9.80 -9.4% 8.88 postmark.deletion_mixed_trans
9.73 -9.5% 8.80 postmark.files_appended
12.59 -7.0% 11.70 postmark.files_created
12.59 -7.0% 11.70 postmark.files_deleted
9.87 -9.5% 8.93 postmark.files_read
715.35 +7.5% 768.93 postmark.time.elapsed_time
715.35 +7.5% 768.93 postmark.time.elapsed_time.max
51508 -1.6% 50690 postmark.time.voluntary_context_switches
19.61 -9.5% 17.75 postmark.transactions
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [linus:master] [btrfs] b7fdfd29a1: postmark.transactions 9.5% regression
2025-10-27 7:41 [linus:master] [btrfs] b7fdfd29a1: postmark.transactions 9.5% regression kernel test robot
@ 2025-10-27 7:49 ` Qu Wenruo
2025-10-28 5:14 ` Oliver Sang
2025-10-31 19:20 ` David Sterba
0 siblings, 2 replies; 4+ messages in thread
From: Qu Wenruo @ 2025-10-27 7:49 UTC (permalink / raw)
To: kernel test robot, Qu Wenruo
Cc: oe-lkp, lkp, linux-kernel, David Sterba, HAN Yuwei, linux-btrfs
在 2025/10/27 18:11, kernel test robot 写道:
>
>
> Hello,
>
> kernel test robot noticed a 9.5% regression of postmark.transactions on:
>
>
> commit: b7fdfd29a136a17c5c8ad9e9bbf89c48919c3d19 ("btrfs: only set the device specific options after devices are opened")
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>
>
> we are in fact not sure what's the connection between this change and the
> postmark.transactions performance. still report out due to below checks.
>
> [still regression on linus/master 4bb1f7e19c4a1d6eeb52b80acff5ac63edd1b91d]
> [regression chould be solved by reverting this commit on linus/master head]
> [still regression on linux-next/master 72fb0170ef1f45addf726319c52a0562b6913707]
>
> testcase: postmark
> config: x86_64-rhel-9.4
> compiler: gcc-14
> test machine: 224 threads 4 sockets Intel(R) Xeon(R) Platinum 8380H CPU @ 2.90GHz (Cooper Lake) with 192G memory
> parameters:
>
> disk: 1HDD
> fs: btrfs
> fs1: nfsv4
> number: 4000n
> trans: 10000s
> subdirs: 100d
> cpufreq_governor: performance
>
>
>
>
> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> the same patch/commit), kindly add following tags
> | Reported-by: kernel test robot <oliver.sang@intel.com>
> | Closes: https://lore.kernel.org/oe-lkp/202510271449.efa21738-lkp@intel.com
>
>
> Details are as below:
> -------------------------------------------------------------------------------------------------->
>
>
> The kernel config and materials to reproduce are available at:
> https://download.01.org/0day-ci/archive/20251027/202510271449.efa21738-lkp@intel.com
>
> =========================================================================================
> compiler/cpufreq_governor/disk/fs1/fs/kconfig/number/rootfs/subdirs/tbox_group/testcase/trans:
> gcc-14/performance/1HDD/nfsv4/btrfs/x86_64-rhel-9.4/4000n/debian-13-x86_64-20250902.cgz/100d/lkp-cpl-4sp2/postmark/10000s
>
> commit:
> 53a4acbfc1 ("btrfs: fix memory leak on duplicated memory in the qgroup assign ioctl")
This is definitely not related.
> b7fdfd29a1 ("btrfs: only set the device specific options after devices are opened")
But this may affect performance, because without this fix, btrfs always
falls back to `ssd` mount option
Now it will properly detect rotating devices, and won't set `ssd` mount
option by default.
But if this is causing performance drop, we should really consider if
`ssd` should be the only mode we support.
Thanks,
Qu
>
> 53a4acbfc1de85fa b7fdfd29a136a17c5c8ad9e9bbf
> ---------------- ---------------------------
> %stddev %change %stddev
> \ | \
> 2010 +10.5% 2222 nfsstat.Client.nfs.v4.open_noat
> 97983 ± 11% +18.5% 116101 numa-numastat.node1.other_node
> 97983 ± 11% +18.5% 116101 numa-vmstat.node1.numa_other
> 16001 ± 5% -7.5% 14797 ± 5% sched_debug.cfs_rq:/.load.avg
> 1474354 ± 3% +9.9% 1620281 ± 4% sched_debug.cpu.avg_idle.avg
> 756151 ± 3% +10.0% 831539 ± 4% sched_debug.cpu.max_idle_balance_cost.avg
> 3585 -2.6% 3490 perf-stat.i.context-switches
> 6141 ± 2% +4.0% 6385 perf-stat.i.cycles-between-cache-misses
> 5796 ± 2% +3.9% 6024 perf-stat.overall.cycles-between-cache-misses
> 3580 -2.6% 3486 perf-stat.ps.context-switches
> 9.548e+11 +7.3% 1.025e+12 perf-stat.total.instructions
> 136494 +4.3% 142419 proc-vmstat.nr_inactive_file
> 136494 +4.3% 142419 proc-vmstat.nr_zone_inactive_file
> 2784208 +4.8% 2917180 proc-vmstat.numa_hit
> 2435763 +5.5% 2568673 proc-vmstat.numa_local
> 3042276 +5.1% 3196281 proc-vmstat.pgalloc_normal
> 2627503 +6.9% 2808220 proc-vmstat.pgfault
> 2754381 +5.4% 2903034 proc-vmstat.pgfree
> 97857 +6.3% 104058 proc-vmstat.pgreuse
> 9.80 -9.4% 8.88 postmark.creation_mixed_trans
> 112312 -7.0% 104473 postmark.data_read
> 203502 -7.0% 189298 postmark.data_written
> 9.80 -9.4% 8.88 postmark.deletion_mixed_trans
> 9.73 -9.5% 8.80 postmark.files_appended
> 12.59 -7.0% 11.70 postmark.files_created
> 12.59 -7.0% 11.70 postmark.files_deleted
> 9.87 -9.5% 8.93 postmark.files_read
> 715.35 +7.5% 768.93 postmark.time.elapsed_time
> 715.35 +7.5% 768.93 postmark.time.elapsed_time.max
> 51508 -1.6% 50690 postmark.time.voluntary_context_switches
> 19.61 -9.5% 17.75 postmark.transactions
>
>
>
>
> Disclaimer:
> Results have been estimated based on internal Intel analysis and are provided
> for informational purposes only. Any difference in system hardware or software
> design or configuration may affect actual performance.
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [linus:master] [btrfs] b7fdfd29a1: postmark.transactions 9.5% regression
2025-10-27 7:49 ` Qu Wenruo
@ 2025-10-28 5:14 ` Oliver Sang
2025-10-31 19:20 ` David Sterba
1 sibling, 0 replies; 4+ messages in thread
From: Oliver Sang @ 2025-10-28 5:14 UTC (permalink / raw)
To: Qu Wenruo
Cc: Qu Wenruo, oe-lkp, lkp, linux-kernel, David Sterba, HAN Yuwei,
linux-btrfs, oliver.sang
hi, Qu,
On Mon, Oct 27, 2025 at 06:19:59PM +1030, Qu Wenruo wrote:
>
>
> 在 2025/10/27 18:11, kernel test robot 写道:
> >
> >
> > Hello,
> >
> > kernel test robot noticed a 9.5% regression of postmark.transactions on:
> >
> >
> > commit: b7fdfd29a136a17c5c8ad9e9bbf89c48919c3d19 ("btrfs: only set the device specific options after devices are opened")
> > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> >
> >
> > we are in fact not sure what's the connection between this change and the
> > postmark.transactions performance. still report out due to below checks.
> >
> > [still regression on linus/master 4bb1f7e19c4a1d6eeb52b80acff5ac63edd1b91d]
> > [regression chould be solved by reverting this commit on linus/master head]
> > [still regression on linux-next/master 72fb0170ef1f45addf726319c52a0562b6913707]
> >
> > testcase: postmark
> > config: x86_64-rhel-9.4
> > compiler: gcc-14
> > test machine: 224 threads 4 sockets Intel(R) Xeon(R) Platinum 8380H CPU @ 2.90GHz (Cooper Lake) with 192G memory
> > parameters:
> >
> > disk: 1HDD
> > fs: btrfs
> > fs1: nfsv4
> > number: 4000n
> > trans: 10000s
> > subdirs: 100d
> > cpufreq_governor: performance
> >
> >
> >
> >
> > If you fix the issue in a separate patch/commit (i.e. not just a new version of
> > the same patch/commit), kindly add following tags
> > | Reported-by: kernel test robot <oliver.sang@intel.com>
> > | Closes: https://lore.kernel.org/oe-lkp/202510271449.efa21738-lkp@intel.com
> >
> >
> > Details are as below:
> > -------------------------------------------------------------------------------------------------->
> >
> >
> > The kernel config and materials to reproduce are available at:
> > https://download.01.org/0day-ci/archive/20251027/202510271449.efa21738-lkp@intel.com
> >
> > =========================================================================================
> > compiler/cpufreq_governor/disk/fs1/fs/kconfig/number/rootfs/subdirs/tbox_group/testcase/trans:
> > gcc-14/performance/1HDD/nfsv4/btrfs/x86_64-rhel-9.4/4000n/debian-13-x86_64-20250902.cgz/100d/lkp-cpl-4sp2/postmark/10000s
> >
> > commit:
> > 53a4acbfc1 ("btrfs: fix memory leak on duplicated memory in the qgroup assign ioctl")
>
> This is definitely not related.
just want to make a clarification here. we report regression based on bisect
results. this report is upon b7fdfd29a1. 53a4acbfc1 is the parent of b7fdfd29a1,
it's a 'good' commit in bisect.
below table gives out the comparison between 53a4acbfc1 and b7fdfd29a1, such
like
19.61 -9.5% 17.75 postmark.transactions
which means we run same tests upon 53a4acbfc1 and b7fdfd29a1.
the score of postmark.transactions for b7fdfd29a1 is 17.75 (average of at least
6 runs)
the score for 53a4acbfc1 is 19.61.
and this is the reason we report as in title
"b7fdfd29a1: postmark.transactions 9.5% regression"
>
> > b7fdfd29a1 ("btrfs: only set the device specific options after devices are opened")
>
> But this may affect performance, because without this fix, btrfs always
> falls back to `ssd` mount option
>
> Now it will properly detect rotating devices, and won't set `ssd` mount
> option by default.
>
> But if this is causing performance drop, we should really consider if `ssd`
> should be the only mode we support.
thanks a lot for information!
>
> Thanks,
> Qu
>
> >
> > 53a4acbfc1de85fa b7fdfd29a136a17c5c8ad9e9bbf
> > ---------------- ---------------------------
> > %stddev %change %stddev
> > \ | \
> > 2010 +10.5% 2222 nfsstat.Client.nfs.v4.open_noat
> > 97983 ± 11% +18.5% 116101 numa-numastat.node1.other_node
> > 97983 ± 11% +18.5% 116101 numa-vmstat.node1.numa_other
> > 16001 ± 5% -7.5% 14797 ± 5% sched_debug.cfs_rq:/.load.avg
> > 1474354 ± 3% +9.9% 1620281 ± 4% sched_debug.cpu.avg_idle.avg
> > 756151 ± 3% +10.0% 831539 ± 4% sched_debug.cpu.max_idle_balance_cost.avg
> > 3585 -2.6% 3490 perf-stat.i.context-switches
> > 6141 ± 2% +4.0% 6385 perf-stat.i.cycles-between-cache-misses
> > 5796 ± 2% +3.9% 6024 perf-stat.overall.cycles-between-cache-misses
> > 3580 -2.6% 3486 perf-stat.ps.context-switches
> > 9.548e+11 +7.3% 1.025e+12 perf-stat.total.instructions
> > 136494 +4.3% 142419 proc-vmstat.nr_inactive_file
> > 136494 +4.3% 142419 proc-vmstat.nr_zone_inactive_file
> > 2784208 +4.8% 2917180 proc-vmstat.numa_hit
> > 2435763 +5.5% 2568673 proc-vmstat.numa_local
> > 3042276 +5.1% 3196281 proc-vmstat.pgalloc_normal
> > 2627503 +6.9% 2808220 proc-vmstat.pgfault
> > 2754381 +5.4% 2903034 proc-vmstat.pgfree
> > 97857 +6.3% 104058 proc-vmstat.pgreuse
> > 9.80 -9.4% 8.88 postmark.creation_mixed_trans
> > 112312 -7.0% 104473 postmark.data_read
> > 203502 -7.0% 189298 postmark.data_written
> > 9.80 -9.4% 8.88 postmark.deletion_mixed_trans
> > 9.73 -9.5% 8.80 postmark.files_appended
> > 12.59 -7.0% 11.70 postmark.files_created
> > 12.59 -7.0% 11.70 postmark.files_deleted
> > 9.87 -9.5% 8.93 postmark.files_read
> > 715.35 +7.5% 768.93 postmark.time.elapsed_time
> > 715.35 +7.5% 768.93 postmark.time.elapsed_time.max
> > 51508 -1.6% 50690 postmark.time.voluntary_context_switches
> > 19.61 -9.5% 17.75 postmark.transactions
> >
> >
> >
> >
> > Disclaimer:
> > Results have been estimated based on internal Intel analysis and are provided
> > for informational purposes only. Any difference in system hardware or software
> > design or configuration may affect actual performance.
> >
> >
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [linus:master] [btrfs] b7fdfd29a1: postmark.transactions 9.5% regression
2025-10-27 7:49 ` Qu Wenruo
2025-10-28 5:14 ` Oliver Sang
@ 2025-10-31 19:20 ` David Sterba
1 sibling, 0 replies; 4+ messages in thread
From: David Sterba @ 2025-10-31 19:20 UTC (permalink / raw)
To: Qu Wenruo
Cc: kernel test robot, Qu Wenruo, oe-lkp, lkp, linux-kernel,
David Sterba, HAN Yuwei, linux-btrfs
On Mon, Oct 27, 2025 at 06:19:59PM +1030, Qu Wenruo wrote:
> 在 2025/10/27 18:11, kernel test robot 写道:
> > Hello,
> >
> > kernel test robot noticed a 9.5% regression of postmark.transactions on:
> >
> >
> > commit: b7fdfd29a136a17c5c8ad9e9bbf89c48919c3d19 ("btrfs: only set the device specific options after devices are opened")
> > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> >
> >
> > we are in fact not sure what's the connection between this change and the
> > postmark.transactions performance. still report out due to below checks.
> >
> > [still regression on linus/master 4bb1f7e19c4a1d6eeb52b80acff5ac63edd1b91d]
> > [regression chould be solved by reverting this commit on linus/master head]
> > [still regression on linux-next/master 72fb0170ef1f45addf726319c52a0562b6913707]
> >
> > testcase: postmark
> > config: x86_64-rhel-9.4
> > compiler: gcc-14
> > test machine: 224 threads 4 sockets Intel(R) Xeon(R) Platinum 8380H CPU @ 2.90GHz (Cooper Lake) with 192G memory
> > parameters:
> >
> > disk: 1HDD
> > fs: btrfs
> > fs1: nfsv4
> > number: 4000n
> > trans: 10000s
> > subdirs: 100d
> > cpufreq_governor: performance
> >
> >
> >
> >
> > If you fix the issue in a separate patch/commit (i.e. not just a new version of
> > the same patch/commit), kindly add following tags
> > | Reported-by: kernel test robot <oliver.sang@intel.com>
> > | Closes: https://lore.kernel.org/oe-lkp/202510271449.efa21738-lkp@intel.com
> >
> >
> > Details are as below:
> > -------------------------------------------------------------------------------------------------->
> >
> >
> > The kernel config and materials to reproduce are available at:
> > https://download.01.org/0day-ci/archive/20251027/202510271449.efa21738-lkp@intel.com
> >
> > =========================================================================================
> > compiler/cpufreq_governor/disk/fs1/fs/kconfig/number/rootfs/subdirs/tbox_group/testcase/trans:
> > gcc-14/performance/1HDD/nfsv4/btrfs/x86_64-rhel-9.4/4000n/debian-13-x86_64-20250902.cgz/100d/lkp-cpl-4sp2/postmark/10000s
> >
> > commit:
> > 53a4acbfc1 ("btrfs: fix memory leak on duplicated memory in the qgroup assign ioctl")
>
> This is definitely not related.
>
> > b7fdfd29a1 ("btrfs: only set the device specific options after devices are opened")
>
> But this may affect performance, because without this fix, btrfs always
> falls back to `ssd` mount option
>
> Now it will properly detect rotating devices, and won't set `ssd` mount
> option by default.
>
> But if this is causing performance drop, we should really consider if
> `ssd` should be the only mode we support.
I think it's a good idea, though this can have also negative performance
impact.
The SSD optimizations are now only the cluster size selection
(fetch_cluster_info) and waiting for log writers (btrfs_sync_log(),
added in 2011 cd354ad613a39). IIRC there used to be selection for the
async io. On rotational devices the extent placement can have impact on
seeking, but this becomes upredictable on aged filesystems.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2025-10-31 19:20 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-27 7:41 [linus:master] [btrfs] b7fdfd29a1: postmark.transactions 9.5% regression kernel test robot
2025-10-27 7:49 ` Qu Wenruo
2025-10-28 5:14 ` Oliver Sang
2025-10-31 19:20 ` David Sterba
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox