From: Jason Gunthorpe <jgg@ziepe.ca>
To: Christoph Hellwig <hch@infradead.org>
Cc: Andrea Arcangeli <aarcange@redhat.com>,
"Yang, Philip" <Philip.Yang@amd.com>,
Ralph Campbell <rcampbell@nvidia.com>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
John Hubbard <jhubbard@nvidia.com>,
"Kuehling, Felix" <Felix.Kuehling@amd.com>,
"dri-devel@lists.freedesktop.org"
<dri-devel@lists.freedesktop.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
Jerome Glisse <jglisse@redhat.com>,
"amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>,
Ben Skeggs <bskeggs@redhat.com>
Subject: Re: [PATCH v3 hmm 11/12] mm/hmm: Remove confusing comment and logic from hmm_release
Date: Wed, 19 Jun 2019 08:56:32 -0300 [thread overview]
Message-ID: <20190619115632.GC9360@ziepe.ca> (raw)
In-Reply-To: <20190619080705.GA5164@infradead.org>
On Wed, Jun 19, 2019 at 01:07:05AM -0700, Christoph Hellwig wrote:
> On Wed, Jun 19, 2019 at 12:53:55AM +0000, Kuehling, Felix wrote:
> > This code is derived from our old MMU notifier code. Before HMM we used
> > to register a single MMU notifier per mm_struct and look up virtual
> > address ranges that had been registered for mirroring via driver API
> > calls. The idea was to reuse a single MMU notifier for the life time of
> > the process. It would remain registered until we got a notifier_release.
> >
> > hmm_mirror took the place of that when we converted the code to HMM.
> >
> > I suppose we could destroy the mirror earlier, when we have no more
> > registered virtual address ranges, and create a new one if needed later.
>
> I didn't write the code, but if you look at hmm_mirror it already is
> a multiplexer over the mmu notifier, and the intent clearly seems that
> you register one per range that you want to mirror, and not multiplex
> it once again. In other words - I think each amdgpu_mn_node should
> probably have its own hmm_mirror. And while the amdgpu_mn_node objects
> are currently stored in an interval tree it seems like they are only
> linearly iterated anyway, so a list actually seems pretty suitable. If
> not we need to improve the core data structures instead of working
> around them.
This looks a lot like the ODP code (amdgpu_mn_node == ib_umem_odp)
The interval tree is to quickly find the driver object(s) that have
the virtual pages during invalidation:
static int amdgpu_mn_sync_pagetables_gfx(struct hmm_mirror *mirror,
const struct hmm_update *update)
{
it = interval_tree_iter_first(&amn->objects, start, end);
while (it) {
[..]
amdgpu_mn_invalidate_node(node, start, end);
And following the ODP model there should be a single hmm_mirror per-mm
(user can fork and stuff, this is something I want to have core code
help with).
The hmm_mirror can either exist so long as objects exist, or it can
exist until the chardev is closed - but never longer than the
chardev's lifetime.
Maybe we should be considering providing a mmu notifier & interval
tree & lock abstraction since ODP & AMD are very similar here..
Jason
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
WARNING: multiple messages have this Message-ID (diff)
From: Jason Gunthorpe <jgg@ziepe.ca>
To: Christoph Hellwig <hch@infradead.org>
Cc: "Kuehling, Felix" <Felix.Kuehling@amd.com>,
Jerome Glisse <jglisse@redhat.com>,
Ralph Campbell <rcampbell@nvidia.com>,
John Hubbard <jhubbard@nvidia.com>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
Andrea Arcangeli <aarcange@redhat.com>,
"dri-devel@lists.freedesktop.org"
<dri-devel@lists.freedesktop.org>,
"amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>,
Ben Skeggs <bskeggs@redhat.com>,
"Yang, Philip" <Philip.Yang@amd.com>
Subject: Re: [PATCH v3 hmm 11/12] mm/hmm: Remove confusing comment and logic from hmm_release
Date: Wed, 19 Jun 2019 08:56:32 -0300 [thread overview]
Message-ID: <20190619115632.GC9360@ziepe.ca> (raw)
In-Reply-To: <20190619080705.GA5164@infradead.org>
On Wed, Jun 19, 2019 at 01:07:05AM -0700, Christoph Hellwig wrote:
> On Wed, Jun 19, 2019 at 12:53:55AM +0000, Kuehling, Felix wrote:
> > This code is derived from our old MMU notifier code. Before HMM we used
> > to register a single MMU notifier per mm_struct and look up virtual
> > address ranges that had been registered for mirroring via driver API
> > calls. The idea was to reuse a single MMU notifier for the life time of
> > the process. It would remain registered until we got a notifier_release.
> >
> > hmm_mirror took the place of that when we converted the code to HMM.
> >
> > I suppose we could destroy the mirror earlier, when we have no more
> > registered virtual address ranges, and create a new one if needed later.
>
> I didn't write the code, but if you look at hmm_mirror it already is
> a multiplexer over the mmu notifier, and the intent clearly seems that
> you register one per range that you want to mirror, and not multiplex
> it once again. In other words - I think each amdgpu_mn_node should
> probably have its own hmm_mirror. And while the amdgpu_mn_node objects
> are currently stored in an interval tree it seems like they are only
> linearly iterated anyway, so a list actually seems pretty suitable. If
> not we need to improve the core data structures instead of working
> around them.
This looks a lot like the ODP code (amdgpu_mn_node == ib_umem_odp)
The interval tree is to quickly find the driver object(s) that have
the virtual pages during invalidation:
static int amdgpu_mn_sync_pagetables_gfx(struct hmm_mirror *mirror,
const struct hmm_update *update)
{
it = interval_tree_iter_first(&amn->objects, start, end);
while (it) {
[..]
amdgpu_mn_invalidate_node(node, start, end);
And following the ODP model there should be a single hmm_mirror per-mm
(user can fork and stuff, this is something I want to have core code
help with).
The hmm_mirror can either exist so long as objects exist, or it can
exist until the chardev is closed - but never longer than the
chardev's lifetime.
Maybe we should be considering providing a mmu notifier & interval
tree & lock abstraction since ODP & AMD are very similar here..
Jason
next prev parent reply other threads:[~2019-06-19 11:56 UTC|newest]
Thread overview: 90+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-14 0:44 [PATCH v3 hmm 00/12] mm/hmm: Various revisions from a locking/code review Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
2019-06-14 0:44 ` [PATCH v3 hmm 01/12] mm/hmm: fix use after free with struct hmm in the mmu notifiers Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-2-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 13:56 ` Christoph Hellwig
2019-06-15 13:56 ` Christoph Hellwig
2019-06-14 0:44 ` [PATCH v3 hmm 02/12] mm/hmm: Use hmm_mirror not mm as an argument for hmm_range_register Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-3-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 13:59 ` Christoph Hellwig
2019-06-15 13:59 ` Christoph Hellwig
2019-06-18 13:05 ` Jason Gunthorpe
2019-06-18 13:05 ` Jason Gunthorpe
[not found] ` <20190618130544.GC6961-uk2M96/98Pc@public.gmane.org>
2019-06-19 8:14 ` Christoph Hellwig
2019-06-19 8:14 ` Christoph Hellwig
2019-06-14 0:44 ` [PATCH v3 hmm 03/12] mm/hmm: Hold a mmgrab from hmm to mm Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-4-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 13:59 ` Christoph Hellwig
2019-06-15 13:59 ` Christoph Hellwig
2019-06-14 0:44 ` [PATCH v3 hmm 04/12] mm/hmm: Simplify hmm_get_or_create and make it reliable Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-5-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 14:12 ` Christoph Hellwig
2019-06-15 14:12 ` Christoph Hellwig
2019-06-18 0:36 ` Jason Gunthorpe
2019-06-18 0:36 ` Jason Gunthorpe
2019-06-18 18:55 ` Jason Gunthorpe
2019-06-18 18:55 ` Jason Gunthorpe
2019-06-14 0:44 ` [PATCH v3 hmm 05/12] mm/hmm: Remove duplicate condition test before wait_event_timeout Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-6-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 14:12 ` Christoph Hellwig
2019-06-15 14:12 ` Christoph Hellwig
2019-06-14 0:44 ` [PATCH v3 hmm 06/12] mm/hmm: Hold on to the mmget for the lifetime of the range Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-7-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 14:14 ` Christoph Hellwig
2019-06-15 14:14 ` Christoph Hellwig
2019-06-18 15:11 ` Jason Gunthorpe
2019-06-18 15:11 ` Jason Gunthorpe
[not found] ` <20190618151100.GI6961-uk2M96/98Pc@public.gmane.org>
2019-06-19 8:18 ` Christoph Hellwig
2019-06-19 8:18 ` Christoph Hellwig
2019-06-19 11:34 ` Jason Gunthorpe
2019-06-19 11:34 ` Jason Gunthorpe
[not found] ` <20190619113452.GB9360-uk2M96/98Pc@public.gmane.org>
2019-06-19 11:54 ` Christoph Hellwig
2019-06-19 11:54 ` Christoph Hellwig
2019-06-14 0:44 ` [PATCH v3 hmm 07/12] mm/hmm: Use lockdep instead of comments Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-8-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 14:14 ` Christoph Hellwig
2019-06-15 14:14 ` Christoph Hellwig
2019-06-14 0:44 ` [PATCH v3 hmm 08/12] mm/hmm: Remove racy protection against double-unregistration Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-9-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 14:16 ` Christoph Hellwig
2019-06-15 14:16 ` Christoph Hellwig
2019-06-18 13:13 ` Jason Gunthorpe
2019-06-18 13:13 ` Jason Gunthorpe
[not found] ` <20190618131324.GF6961-uk2M96/98Pc@public.gmane.org>
2019-06-18 13:27 ` Christoph Hellwig
2019-06-18 13:27 ` Christoph Hellwig
2019-06-18 18:57 ` Jason Gunthorpe
2019-06-18 18:57 ` Jason Gunthorpe
[not found] ` <20190618185757.GP6961-uk2M96/98Pc@public.gmane.org>
2019-06-19 8:19 ` Christoph Hellwig
2019-06-19 8:19 ` Christoph Hellwig
2019-06-14 0:44 ` [PATCH v3 hmm 09/12] mm/hmm: Poison hmm_range during unregister Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-10-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 14:17 ` Christoph Hellwig
2019-06-15 14:17 ` Christoph Hellwig
2019-06-18 18:04 ` Jason Gunthorpe
2019-06-18 18:04 ` Jason Gunthorpe
2019-06-14 0:44 ` [PATCH v3 hmm 10/12] mm/hmm: Do not use list*_rcu() for hmm->ranges Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-11-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 14:18 ` Christoph Hellwig
2019-06-15 14:18 ` Christoph Hellwig
2019-06-18 0:38 ` Jason Gunthorpe
2019-06-18 0:38 ` Jason Gunthorpe
2019-06-14 0:44 ` [PATCH v3 hmm 11/12] mm/hmm: Remove confusing comment and logic from hmm_release Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-12-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 14:21 ` Christoph Hellwig
2019-06-15 14:21 ` Christoph Hellwig
2019-06-18 0:45 ` Jason Gunthorpe
2019-06-18 0:45 ` Jason Gunthorpe
[not found] ` <20190618004509.GE30762-uk2M96/98Pc@public.gmane.org>
2019-06-18 5:37 ` Christoph Hellwig
2019-06-18 5:37 ` Christoph Hellwig
[not found] ` <20190618053733.GA25048-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2019-06-19 0:53 ` Kuehling, Felix
2019-06-19 0:53 ` Kuehling, Felix
[not found] ` <be4f8573-6284-04a6-7862-23bb357bfe3c-5C7GfCeVMHo@public.gmane.org>
2019-06-19 8:07 ` Christoph Hellwig
2019-06-19 8:07 ` Christoph Hellwig
2019-06-19 11:56 ` Jason Gunthorpe [this message]
2019-06-19 11:56 ` Jason Gunthorpe
[not found] ` <20190619115632.GC9360-uk2M96/98Pc@public.gmane.org>
2019-06-19 12:03 ` Christoph Hellwig
2019-06-19 12:03 ` Christoph Hellwig
2019-06-14 0:44 ` [PATCH v3 hmm 12/12] mm/hmm: Fix error flows in hmm_invalidate_range_start Jason Gunthorpe
2019-06-14 0:44 ` Jason Gunthorpe
[not found] ` <20190614004450.20252-13-jgg-uk2M96/98Pc@public.gmane.org>
2019-06-15 14:25 ` Christoph Hellwig
2019-06-15 14:25 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190619115632.GC9360@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=Felix.Kuehling@amd.com \
--cc=Philip.Yang@amd.com \
--cc=aarcange@redhat.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=bskeggs@redhat.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=hch@infradead.org \
--cc=jglisse@redhat.com \
--cc=jhubbard@nvidia.com \
--cc=linux-mm@kvack.org \
--cc=linux-rdma@vger.kernel.org \
--cc=rcampbell@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.