All of lore.kernel.org
 help / color / mirror / Atom feed
From: Namhyung Kim <namhyung@kernel.org>
To: Harry Yoo <harry@kernel.org>
Cc: Tim Menninger <tmenninger@everpuredata.com>,
	linux-mm@kvack.org, Chuck Lever <cel@kernel.org>,
	linux-nfs@vger.kernel.org, Jon Curley <jcurley@everpuredata.com>,
	Eric Badger <ebadger@everpuredata.com>,
	Vlastimil Babka <vbabka@kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Hao Li <hao.li@linux.dev>, Christoph Lameter <cl@gentwo.org>,
	David Rientjes <rientjes@google.com>,
	Roman Gushchin <roman.gushchin@linux.dev>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
	Adrian Hunter <adrian.hunter@intel.com>,
	James Clark <james.clark@linaro.org>,
	linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [SLUB] nfs_page cmpxchg_double_fail and perf lock perturbation on dual-socket NFS/RDMA
Date: Fri, 18 Sep 2026 00:14:02 -0700	[thread overview]
Message-ID: <aqzkugwB_I5X2JYx@z2> (raw)
In-Reply-To: <aqvy8NqRRRmUlxjR@thinkstation>

On Thu, Sep 17, 2026 at 03:10:47PM +0100, Harry Yoo wrote:
> ... now I realize we need to add PERFORMANCE EVENTS SUBSYSTEM folks 
> as well ;-)
> 
> Hmm, sounds like perf lock is somehow triggering slab allocations
> and interfering the workload.
> 
> It could be because SLUB is merging nfs_page cache with some other
> cache that perf uses.
> 
> Could you please check if it reproduces with slab_nomerge kernel
> parameter?
> 
> On Thu, Sep 17, 2026 at 02:46:24PM +0100, Harry Yoo wrote:
> > Hi Tim and Chuck, thanks for reporting this to linux-mm.
> > Will take a look at this but let me Cc SLAB ALLOCATOR folks here first.
> > 
> > -- 
> > Cheers,
> > Harry / Hyeonggon
> > 
[SNIP]
> > On Wed, Sep 16, 2026 at 11:22:27PM +0000, Tim Menninger wrote:
> > > My questions are:
> > > 
> > > 1. Do the uninstrumented cmpxchg_double_fail rates above look abnormal
> > >    for this workload/topology, or are they within the range one would
> > >    expect from this degree of concurrency and NUMA placement?
> > > 
> > > 2. Is there a less invasive way you would recommend measuring the
> > >    nfs_page list_lock/freelist contention? I would like to distinguish
> > >    the steady-state behavior from what is observed during the perf-lock
> > >    capture.

You can use the BPF version of perf lock to check lock contention like
below.  (it only work with 'contention' subcommand.)  It won't create
a data file and just shows the result.

  $ sudo perf lock con -ab sleep 10

or

  $ sudo perf lock con -ab -E 5 sleep 10

to see top-5 locks only.

Thanks,
Namhyung


      parent reply	other threads:[~2026-09-18  7:14 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-16 23:22 [SLUB] nfs_page cmpxchg_double_fail and perf lock perturbation on dual-socket NFS/RDMA Tim Menninger
2026-09-17 13:46 ` Harry Yoo
2026-09-17 14:10   ` Harry Yoo
2026-09-17 15:29     ` Peter Zijlstra
2026-09-18  7:04       ` Namhyung Kim
2026-09-17 16:25     ` Tim Menninger
2026-09-18  7:08       ` Vlastimil Babka (SUSE)
2026-09-21 15:31         ` Harry Yoo
2026-09-21 23:27           ` Tim Menninger
2026-09-18  7:14     ` Namhyung Kim [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aqzkugwB_I5X2JYx@z2 \
    --to=namhyung@kernel.org \
    --cc=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=cel@kernel.org \
    --cc=cl@gentwo.org \
    --cc=ebadger@everpuredata.com \
    --cc=hao.li@linux.dev \
    --cc=harry@kernel.org \
    --cc=irogers@google.com \
    --cc=james.clark@linaro.org \
    --cc=jcurley@everpuredata.com \
    --cc=jolsa@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rientjes@google.com \
    --cc=roman.gushchin@linux.dev \
    --cc=tmenninger@everpuredata.com \
    --cc=vbabka@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.