From: Jason Gunthorpe <jgg@nvidia.com>
To: Daniel Jordan <daniel.m.jordan@oracle.com>
Cc: Alexander Duyck <alexanderduyck@fb.com>,
Alex Williamson <alex.williamson@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
Ben Segall <bsegall@google.com>,
Cornelia Huck <cohuck@redhat.com>,
Dan Williams <dan.j.williams@intel.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Herbert Xu <herbert@gondor.apana.org.au>,
Ingo Molnar <mingo@redhat.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Josh Triplett <josh@joshtriplett.org>,
Michal Hocko <mhocko@suse.com>, Nico Pache <npache@redhat.com>,
Pasha Tatashin <pasha.tatashin@soleen.com>,
Peter Zijlstra <peterz@infradead.org>,
Steffen Klassert <steffen.klassert@secunet.com>,
Steve Sistare <steven.sistare@oracle.com>,
Tejun Heo <tj@kernel.org>, Tim Chen <tim.c.chen@linux.intel.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
linux-mm@kvack.org, kvm@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org
Subject: Re: [RFC 08/16] vfio/type1: Cache locked_vm to ease mmap_lock contention
Date: Wed, 5 Jan 2022 20:53:39 -0400 [thread overview]
Message-ID: <20220106005339.GX2328285@nvidia.com> (raw)
In-Reply-To: <20220106004656.126790-9-daniel.m.jordan@oracle.com>
On Wed, Jan 05, 2022 at 07:46:48PM -0500, Daniel Jordan wrote:
> padata threads hold mmap_lock as reader for the majority of their
> runtime in order to call pin_user_pages_remote(), but they also
> periodically take mmap_lock as writer for short periods to adjust
> mm->locked_vm, hurting parallelism.
>
> Alleviate the write-side contention with a per-thread cache of locked_vm
> which allows taking mmap_lock as writer far less frequently.
>
> Failure to refill the cache due to insufficient locked_vm will not cause
> the entire pinning operation to error out. This avoids spurious failure
> in case some pinned pages aren't accounted to locked_vm.
>
> Cache size is limited to provide some protection in the unlikely event
> of a concurrent locked_vm accounting operation in the same address space
> needlessly failing in case the cache takes more locked_vm than it needs.
Why not just do the pinned page accounting once at the start? Why does
it have to be done incrementally?
Jason
next prev parent reply other threads:[~2022-01-06 0:53 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-06 0:46 [RFC 00/16] padata, vfio, sched: Multithreaded VFIO page pinning Daniel Jordan
2022-01-06 0:46 ` [RFC 01/16] padata: Remove __init from multithreading functions Daniel Jordan
2022-01-06 0:46 ` [RFC 02/16] padata: Return first error from a job Daniel Jordan
2022-01-06 0:46 ` [RFC 03/16] padata: Add undo support Daniel Jordan
2022-01-06 0:46 ` [RFC 04/16] padata: Detect deadlocks between main and helper threads Daniel Jordan
2022-01-06 0:46 ` [RFC 05/16] vfio/type1: Pass mm to vfio_pin_pages_remote() Daniel Jordan
2022-01-06 0:46 ` [RFC 06/16] vfio/type1: Refactor dma map removal Daniel Jordan
2022-01-06 0:46 ` [RFC 07/16] vfio/type1: Parallelize vfio_pin_map_dma() Daniel Jordan
2022-01-06 0:46 ` [RFC 08/16] vfio/type1: Cache locked_vm to ease mmap_lock contention Daniel Jordan
2022-01-06 0:53 ` Jason Gunthorpe [this message]
2022-01-06 1:17 ` Daniel Jordan
2022-01-06 12:34 ` Jason Gunthorpe
2022-01-06 21:05 ` Alex Williamson
2022-01-07 0:19 ` Jason Gunthorpe
2022-01-07 3:06 ` Daniel Jordan
2022-01-07 15:18 ` Jason Gunthorpe
2022-01-07 16:39 ` Daniel Jordan
2022-01-06 0:46 ` [RFC 09/16] padata: Use kthreads in do_multithreaded Daniel Jordan
2022-01-06 0:46 ` [RFC 10/16] padata: Helpers should respect main thread's CPU affinity Daniel Jordan
2022-01-06 0:46 ` [RFC 11/16] padata: Cap helpers started to online CPUs Daniel Jordan
2022-01-06 0:46 ` [RFC 12/16] sched, padata: Bound max threads with max_cfs_bandwidth_cpus() Daniel Jordan
2022-01-06 0:46 ` [RFC 13/16] padata: Run helper threads at MAX_NICE Daniel Jordan
2022-01-06 0:46 ` [RFC 14/16] padata: Nice helper threads one by one to prevent starvation Daniel Jordan
2022-01-06 0:46 ` [RFC 15/16] sched/fair: Account kthread runtime debt for CFS bandwidth Daniel Jordan
2022-01-11 11:58 ` Peter Zijlstra
2022-01-11 16:29 ` Daniel Jordan
2022-01-12 20:18 ` Tejun Heo
2022-01-13 21:08 ` Daniel Jordan
2022-01-13 21:11 ` Daniel Jordan
2022-01-14 9:31 ` Peter Zijlstra
2022-01-14 9:40 ` Peter Zijlstra
2022-01-14 16:38 ` Tejun Heo
2022-01-18 17:40 ` Daniel Jordan
2022-01-14 16:30 ` Tejun Heo
2022-01-18 17:32 ` Daniel Jordan
2022-01-06 0:46 ` [RFC 16/16] sched/fair: Consider kthread debt in cputime Daniel Jordan
2022-01-06 1:13 ` [RFC 00/16] padata, vfio, sched: Multithreaded VFIO page pinning Jason Gunthorpe
2022-01-07 3:03 ` Daniel Jordan
2022-01-07 17:12 ` Jason Gunthorpe
2022-01-10 22:27 ` Daniel Jordan
2022-01-11 0:17 ` Jason Gunthorpe
2022-01-11 16:20 ` Daniel Jordan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220106005339.GX2328285@nvidia.com \
--to=jgg@nvidia.com \
--cc=akpm@linux-foundation.org \
--cc=alex.williamson@redhat.com \
--cc=alexanderduyck@fb.com \
--cc=bsegall@google.com \
--cc=cohuck@redhat.com \
--cc=dan.j.williams@intel.com \
--cc=daniel.m.jordan@oracle.com \
--cc=dave.hansen@linux.intel.com \
--cc=dietmar.eggemann@arm.com \
--cc=hannes@cmpxchg.org \
--cc=herbert@gondor.apana.org.au \
--cc=josh@joshtriplett.org \
--cc=kvm@vger.kernel.org \
--cc=linux-crypto@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=mingo@redhat.com \
--cc=npache@redhat.com \
--cc=pasha.tatashin@soleen.com \
--cc=peterz@infradead.org \
--cc=steffen.klassert@secunet.com \
--cc=steven.sistare@oracle.com \
--cc=tim.c.chen@linux.intel.com \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).