All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jan Glauber <jglauber@marvell.com>
To: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>,
	Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
	Jayachandran Chandrasekharan Nair <jnair@marvell.com>,
	Jan Glauber <jglauber@cavium.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Linux ARM <linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH] lockref: Limit number of cmpxchg loop retries
Date: Thu, 6 Jun 2019 10:28:12 +0000	[thread overview]
Message-ID: <20190606102803.GA15499@hc> (raw)
In-Reply-To: <20190606094154.GB6795@fuggles.cambridge.arm.com>

On Thu, Jun 06, 2019 at 10:41:54AM +0100, Will Deacon wrote:
> On Thu, Jun 06, 2019 at 08:03:27AM +0000, Jan Glauber wrote:
> > On Wed, Jun 05, 2019 at 01:16:46PM -0700, Linus Torvalds wrote:
> > > On Wed, Jun 5, 2019 at 6:49 AM Jan Glauber <jglauber@cavium.com> wrote:
> > > >
> > > > Add an upper bound to the loop to force the fallback to spinlocks
> > > > after some time. A retry value of 100 should not impact any hardware
> > > > that does not have this issue.
> > > >
> > > > With the retry limit the performance of an open-close testcase
> > > > improved between 60-70% on ThunderX2.
> > > 
> > > Btw, did you do any kind of performance analysis across different
> > > retry limit values?
> > 
> > I tried 15/50/100/200/500, results were largely identical up to 100.
> > For SMT=4 a higher retry value might be better, but unless we can add a
> > sysctl value 100 looked like a good compromise to me.
> 
> Perhaps I'm just getting confused pre-morning-coffee, but I thought the
> original complaint (and the reason for this patch even existing) was that
> when many CPUs were hammering the lockref then performance tanked? In which
> case, increasing the threshold as the number of CPUs increases seems
> counter-intuitive to me because it suggests that the larger the system,
> the harder we should try to make the cmpxchg work.

For SMT=4 the top hit I see is queued_spin_lock_slowpath(). Maybe this is more
costly with more threads, so trying harder to use lockref-cmpxchg makes
the microbenchmark faster in that case?

--Jan

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

WARNING: multiple messages have this Message-ID (diff)
From: Jan Glauber <jglauber@marvell.com>
To: Will Deacon <will.deacon@arm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	Jan Glauber <jglauber@cavium.com>,
	Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Linux ARM <linux-arm-kernel@lists.infradead.org>,
	"Jayachandran Chandrasekharan Nair" <jnair@marvell.com>
Subject: Re: [PATCH] lockref: Limit number of cmpxchg loop retries
Date: Thu, 6 Jun 2019 10:28:12 +0000	[thread overview]
Message-ID: <20190606102803.GA15499@hc> (raw)
In-Reply-To: <20190606094154.GB6795@fuggles.cambridge.arm.com>

On Thu, Jun 06, 2019 at 10:41:54AM +0100, Will Deacon wrote:
> On Thu, Jun 06, 2019 at 08:03:27AM +0000, Jan Glauber wrote:
> > On Wed, Jun 05, 2019 at 01:16:46PM -0700, Linus Torvalds wrote:
> > > On Wed, Jun 5, 2019 at 6:49 AM Jan Glauber <jglauber@cavium.com> wrote:
> > > >
> > > > Add an upper bound to the loop to force the fallback to spinlocks
> > > > after some time. A retry value of 100 should not impact any hardware
> > > > that does not have this issue.
> > > >
> > > > With the retry limit the performance of an open-close testcase
> > > > improved between 60-70% on ThunderX2.
> > > 
> > > Btw, did you do any kind of performance analysis across different
> > > retry limit values?
> > 
> > I tried 15/50/100/200/500, results were largely identical up to 100.
> > For SMT=4 a higher retry value might be better, but unless we can add a
> > sysctl value 100 looked like a good compromise to me.
> 
> Perhaps I'm just getting confused pre-morning-coffee, but I thought the
> original complaint (and the reason for this patch even existing) was that
> when many CPUs were hammering the lockref then performance tanked? In which
> case, increasing the threshold as the number of CPUs increases seems
> counter-intuitive to me because it suggests that the larger the system,
> the harder we should try to make the cmpxchg work.

For SMT=4 the top hit I see is queued_spin_lock_slowpath(). Maybe this is more
costly with more threads, so trying harder to use lockref-cmpxchg makes
the microbenchmark faster in that case?

--Jan

  reply	other threads:[~2019-06-06 10:28 UTC|newest]

Thread overview: 74+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-29 14:52 [RFC] Disable lockref on arm64 Jan Glauber
2019-04-29 14:52 ` Jan Glauber
2019-05-01 16:01 ` Will Deacon
2019-05-01 16:01   ` Will Deacon
2019-05-02  8:38   ` Jan Glauber
2019-05-02  8:38     ` Jan Glauber
2019-05-01 16:41 ` Linus Torvalds
2019-05-01 16:41   ` Linus Torvalds
2019-05-02  8:27   ` Jan Glauber
2019-05-02  8:27     ` Jan Glauber
2019-05-02 16:12     ` Linus Torvalds
2019-05-02 16:12       ` Linus Torvalds
2019-05-02 23:19       ` Jayachandran Chandrasekharan Nair
2019-05-02 23:19         ` Jayachandran Chandrasekharan Nair
2019-05-03 19:40         ` Linus Torvalds
2019-05-03 19:40           ` Linus Torvalds
2019-05-06  6:13           ` [EXT] " Jayachandran Chandrasekharan Nair
2019-05-06  6:13             ` Jayachandran Chandrasekharan Nair
2019-05-06 17:13             ` Linus Torvalds
2019-05-06 17:13               ` Linus Torvalds
2019-05-06 18:10             ` Will Deacon
2019-05-06 18:10               ` Will Deacon
2019-05-18  4:24               ` Jayachandran Chandrasekharan Nair
2019-05-18  4:24                 ` Jayachandran Chandrasekharan Nair
2019-05-18 10:00                 ` Ard Biesheuvel
2019-05-18 10:00                   ` Ard Biesheuvel
2019-05-22 16:04                   ` Will Deacon
2019-05-22 16:04                     ` Will Deacon
2019-06-12  4:10                     ` Jayachandran Chandrasekharan Nair
2019-06-12  4:10                       ` Jayachandran Chandrasekharan Nair
2019-06-12  9:31                       ` Will Deacon
2019-06-12  9:31                         ` Will Deacon
2019-06-14  7:09                         ` Jayachandran Chandrasekharan Nair
2019-06-14  7:09                           ` Jayachandran Chandrasekharan Nair
2019-06-14  9:58                           ` Will Deacon
2019-06-14  9:58                             ` Will Deacon
2019-06-14 10:24                             ` Ard Biesheuvel
2019-06-14 10:24                               ` Ard Biesheuvel
2019-06-14 10:38                               ` Will Deacon
2019-06-14 10:38                                 ` Will Deacon
2019-06-15  4:21                                 ` Kees Cook
2019-06-15  4:21                                   ` Kees Cook
2019-06-15  8:47                                   ` Ard Biesheuvel
2019-06-15  8:47                                     ` Ard Biesheuvel
2019-06-15 13:59                                     ` Kees Cook
2019-06-15 13:59                                       ` Kees Cook
2019-06-15 14:18                                       ` Ard Biesheuvel
2019-06-15 14:18                                         ` Ard Biesheuvel
2019-06-16 21:31                                         ` Kees Cook
2019-06-16 21:31                                           ` Kees Cook
2019-06-17 11:33                                           ` Ard Biesheuvel
2019-06-17 11:33                                             ` Ard Biesheuvel
2019-06-17 17:26                                             ` Will Deacon
2019-06-17 17:26                                               ` Will Deacon
2019-06-17 20:07                                               ` Jayachandran Chandrasekharan Nair
2019-06-17 20:07                                                 ` Jayachandran Chandrasekharan Nair
2019-06-18  5:41                                               ` Kees Cook
2019-06-18  5:41                                                 ` Kees Cook
2019-06-13  9:53                       ` Hanjun Guo
2019-06-13  9:53                         ` Hanjun Guo
2019-06-05 13:48   ` [PATCH] lockref: Limit number of cmpxchg loop retries Jan Glauber
2019-06-05 13:48     ` Jan Glauber
2019-06-05 20:16     ` Linus Torvalds
2019-06-05 20:16       ` Linus Torvalds
2019-06-06  8:03       ` Jan Glauber
2019-06-06  8:03         ` Jan Glauber
2019-06-06  9:41         ` Will Deacon
2019-06-06  9:41           ` Will Deacon
2019-06-06 10:28           ` Jan Glauber [this message]
2019-06-06 10:28             ` Jan Glauber
2019-06-07  7:27             ` Jan Glauber
2019-06-07  7:27               ` Jan Glauber
2019-06-07 20:14               ` Linus Torvalds
2019-06-07 20:14                 ` Linus Torvalds

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190606102803.GA15499@hc \
    --to=jglauber@marvell.com \
    --cc=catalin.marinas@arm.com \
    --cc=jglauber@cavium.com \
    --cc=jnair@marvell.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=will.deacon@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.