* Re: [PATCH v2 hmm 02/11] mm/hmm: Use hmm_mirror not mm as an argument for hmm_range_register
From: Ira Weiny @ 2019-06-07 22:33 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Andrea Arcangeli, Ralph Campbell, linux-rdma, John Hubbard,
Felix.Kuehling, dri-devel, linux-mm, Jerome Glisse,
Jason Gunthorpe, amd-gfx
In-Reply-To: <20190606184438.31646-3-jgg@ziepe.ca>
On Thu, Jun 06, 2019 at 03:44:29PM -0300, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> Ralph observes that hmm_range_register() can only be called by a driver
> while a mirror is registered. Make this clear in the API by passing in the
> mirror structure as a parameter.
>
> This also simplifies understanding the lifetime model for struct hmm, as
> the hmm pointer must be valid as part of a registered mirror so all we
> need in hmm_register_range() is a simple kref_get.
>
> Suggested-by: Ralph Campbell <rcampbell@nvidia.com>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> ---
> v2
> - Include the oneline patch to nouveau_svm.c
> ---
> drivers/gpu/drm/nouveau/nouveau_svm.c | 2 +-
> include/linux/hmm.h | 7 ++++---
> mm/hmm.c | 15 ++++++---------
> 3 files changed, 11 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouveau/nouveau_svm.c
> index 93ed43c413f0bb..8c92374afcf227 100644
> --- a/drivers/gpu/drm/nouveau/nouveau_svm.c
> +++ b/drivers/gpu/drm/nouveau/nouveau_svm.c
> @@ -649,7 +649,7 @@ nouveau_svm_fault(struct nvif_notify *notify)
> range.values = nouveau_svm_pfn_values;
> range.pfn_shift = NVIF_VMM_PFNMAP_V0_ADDR_SHIFT;
> again:
> - ret = hmm_vma_fault(&range, true);
> + ret = hmm_vma_fault(&svmm->mirror, &range, true);
> if (ret == 0) {
> mutex_lock(&svmm->mutex);
> if (!hmm_vma_range_done(&range)) {
> diff --git a/include/linux/hmm.h b/include/linux/hmm.h
> index 688c5ca7068795..2d519797cb134a 100644
> --- a/include/linux/hmm.h
> +++ b/include/linux/hmm.h
> @@ -505,7 +505,7 @@ static inline bool hmm_mirror_mm_is_alive(struct hmm_mirror *mirror)
> * Please see Documentation/vm/hmm.rst for how to use the range API.
> */
> int hmm_range_register(struct hmm_range *range,
> - struct mm_struct *mm,
> + struct hmm_mirror *mirror,
> unsigned long start,
> unsigned long end,
> unsigned page_shift);
> @@ -541,7 +541,8 @@ static inline bool hmm_vma_range_done(struct hmm_range *range)
> }
>
> /* This is a temporary helper to avoid merge conflict between trees. */
> -static inline int hmm_vma_fault(struct hmm_range *range, bool block)
> +static inline int hmm_vma_fault(struct hmm_mirror *mirror,
> + struct hmm_range *range, bool block)
> {
> long ret;
>
> @@ -554,7 +555,7 @@ static inline int hmm_vma_fault(struct hmm_range *range, bool block)
> range->default_flags = 0;
> range->pfn_flags_mask = -1UL;
>
> - ret = hmm_range_register(range, range->vma->vm_mm,
> + ret = hmm_range_register(range, mirror,
> range->start, range->end,
> PAGE_SHIFT);
> if (ret)
> diff --git a/mm/hmm.c b/mm/hmm.c
> index 547002f56a163d..8796447299023c 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -925,13 +925,13 @@ static void hmm_pfns_clear(struct hmm_range *range,
> * Track updates to the CPU page table see include/linux/hmm.h
> */
> int hmm_range_register(struct hmm_range *range,
> - struct mm_struct *mm,
> + struct hmm_mirror *mirror,
> unsigned long start,
> unsigned long end,
> unsigned page_shift)
> {
> unsigned long mask = ((1UL << page_shift) - 1UL);
> - struct hmm *hmm;
> + struct hmm *hmm = mirror->hmm;
>
> range->valid = false;
> range->hmm = NULL;
> @@ -945,15 +945,12 @@ int hmm_range_register(struct hmm_range *range,
> range->start = start;
> range->end = end;
>
> - hmm = hmm_get_or_create(mm);
> - if (!hmm)
> - return -EFAULT;
> -
> /* Check if hmm_mm_destroy() was call. */
> - if (hmm->mm == NULL || hmm->dead) {
> - hmm_put(hmm);
> + if (hmm->mm == NULL || hmm->dead)
> return -EFAULT;
> - }
> +
> + range->hmm = hmm;
I don't think you need this assignment here. In the code below (right after
the mutext_lock()) it is set already. And looks like it remains that way after
the end of the series.
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
> + kref_get(&hmm->kref);
>
> /* Initialize range to track CPU page table updates. */
> mutex_lock(&hmm->lock);
> --
> 2.21.0
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [PATCH v2 hmm 07/11] mm/hmm: Use lockdep instead of comments
From: Souptick Joarder @ 2019-06-07 22:16 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Andrea Arcangeli, Ralph Campbell, linux-rdma, John Hubbard,
Felix.Kuehling, dri-devel, Linux-MM, Jerome Glisse,
Jason Gunthorpe, amd-gfx
In-Reply-To: <20190606184438.31646-8-jgg@ziepe.ca>
On Fri, Jun 7, 2019 at 12:15 AM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> So we can check locking at runtime.
Little more descriptive change log would be helpful.
Acked-by: Souptick Joarder <jrdr.linux@gmail.com>
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
> ---
> v2
> - Fix missing & in lockdeps (Jason)
> ---
> mm/hmm.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/mm/hmm.c b/mm/hmm.c
> index f67ba32983d9f1..c702cd72651b53 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -254,11 +254,11 @@ static const struct mmu_notifier_ops hmm_mmu_notifier_ops = {
> *
> * To start mirroring a process address space, the device driver must register
> * an HMM mirror struct.
> - *
> - * THE mm->mmap_sem MUST BE HELD IN WRITE MODE !
> */
> int hmm_mirror_register(struct hmm_mirror *mirror, struct mm_struct *mm)
> {
> + lockdep_assert_held_exclusive(&mm->mmap_sem);
> +
> /* Sanity check */
> if (!mm || !mirror || !mirror->ops)
> return -EINVAL;
> --
> 2.21.0
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [PATCH v2 hmm 05/11] mm/hmm: Remove duplicate condition test before wait_event_timeout
From: Ralph Campbell @ 2019-06-07 22:13 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Andrea Arcangeli, linux-rdma, John Hubbard, Felix.Kuehling,
dri-devel, linux-mm, Jerome Glisse, amd-gfx
In-Reply-To: <20190607204427.GU14802@ziepe.ca>
On 6/7/19 1:44 PM, Jason Gunthorpe wrote:
> On Fri, Jun 07, 2019 at 01:21:12PM -0700, Ralph Campbell wrote:
>
>>> What I want to get to is a pattern like this:
>>>
>>> pagefault():
>>>
>>> hmm_range_register(&range);
>>> again:
>>> /* On the slow path, if we appear to be live locked then we get
>>> the write side of mmap_sem which will break the live lock,
>>> otherwise this gets the read lock */
>>> if (hmm_range_start_and_lock(&range))
>>> goto err;
>>>
>>> lockdep_assert_held(range->mm->mmap_sem);
>>>
>>> // Optional: Avoid useless expensive work
>>> if (hmm_range_needs_retry(&range))
>>> goto again;
>>> hmm_range_(touch vmas)
>>>
>>> take_lock(driver->update);
>>> if (hmm_range_end(&range) {
>>> release_lock(driver->update);
>>> goto again;
>>> }
>>> // Finish driver updates
>>> release_lock(driver->update);
>>>
>>> // Releases mmap_sem
>>> hmm_range_unregister_and_unlock(&range);
>>>
>>> What do you think?
>>>
>>> Is it clear?
>>>
>>> Jason
>>>
>>
>> Are you talking about acquiring mmap_sem in hmm_range_start_and_lock()?
>> Usually, the fault code has to lock mmap_sem for read in order to
>> call find_vma() so it can set range.vma.
>
>> If HMM drops mmap_sem - which I don't think it should, just return an
>> error to tell the caller to drop mmap_sem and retry - the find_vma()
>> will need to be repeated as well.
>
> Overall I don't think it makes a lot of sense to sleep for retry in
> hmm_range_start_and_lock() while holding mmap_sem. It would be better
> to drop that lock, sleep, then re-acquire it as part of the hmm logic.
>
> The find_vma should be done inside the critical section created by
> hmm_range_start_and_lock(), not before it. If we are retrying then we
> already slept and the additional CPU cost to repeat the find_vma is
> immaterial, IMHO?
>
> Do you see a reason why the find_vma() ever needs to be before the
> 'again' in my above example? range.vma does not need to be set for
> range_register.
Yes, for the GPU case, there can be many faults in an event queue
and the goal is to try to handle more than one page at a time.
The vma is needed to limit the amount of coalescing and checking
for pages that could be speculatively migrated or mapped.
>> I'm also not sure about acquiring the mmap_sem for write as way to
>> mitigate thrashing. It seems to me that if a device and a CPU are
>> both faulting on the same page,
>
> One of the reasons to prefer this approach is that it means we don't
> need to keep track of which ranges we are faulting, and if there is a
> lot of *unrelated* fault activity (unlikely?) we can resolve it using
> mmap sem instead of this elaborate ranges scheme and related
> locking.
>
> This would reduce the overall work in the page fault and
> invalidate_start/end paths for the common uncontended cases.
>
>> some sort of backoff delay is needed to let one side or the other
>> make some progress.
>
> What the write side of the mmap_sem would do is force the CPU and
> device to cleanly take turns. Once the device pages are registered
> under the write side the CPU will have to wait in invalidate_start for
> the driver to complete a shootdown, then the whole thing starts all
> over again.
>
> It is certainly imaginable something could have a 'min life' timer for
> a device mapping and hold mm invalidate_start, and device pagefault
> for that min time to promote better sharing.
>
> But, if we don't use the mmap_sem then we can livelock and the device
> will see an unrecoverable error from the timeout which means we have
> risk that under load the system will simply obscurely fail. This seems
> unacceptable to me..
>
> Particularly since for the ODP use case the issue is not trashing
> migration as a GPU might have, but simple system stability under swap
> load. We do not want the ODP pagefault to permanently fail due to
> timeout if the VMA is still valid..
>
> Jason
>
OK, I understand.
If you come up with a set of changes, I can try testing them.
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [PATCH v2 hmm 10/11] mm/hmm: Do not use list*_rcu() for hmm->ranges
From: Souptick Joarder @ 2019-06-07 22:11 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Andrea Arcangeli, Ralph Campbell, linux-rdma, John Hubbard,
Felix.Kuehling, dri-devel, Linux-MM, Jerome Glisse,
Jason Gunthorpe, amd-gfx
In-Reply-To: <20190606184438.31646-11-jgg@ziepe.ca>
On Fri, Jun 7, 2019 at 12:15 AM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> This list is always read and written while holding hmm->lock so there is
> no need for the confusing _rcu annotations.
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Acked-by: Souptick Joarder <jrdr.linux@gmail.com>
> ---
> mm/hmm.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/mm/hmm.c b/mm/hmm.c
> index c2fecb3ecb11e1..709d138dd49027 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -911,7 +911,7 @@ int hmm_range_register(struct hmm_range *range,
> mutex_lock(&hmm->lock);
>
> range->hmm = hmm;
> - list_add_rcu(&range->list, &hmm->ranges);
> + list_add(&range->list, &hmm->ranges);
>
> /*
> * If there are any concurrent notifiers we have to wait for them for
> @@ -941,7 +941,7 @@ void hmm_range_unregister(struct hmm_range *range)
> return;
>
> mutex_lock(&hmm->lock);
> - list_del_rcu(&range->list);
> + list_del(&range->list);
> mutex_unlock(&hmm->lock);
>
> /* Drop reference taken by hmm_range_register() */
> --
> 2.21.0
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [PATCH v2 hmm 11/11] mm/hmm: Remove confusing comment and logic from hmm_release
From: Ralph Campbell @ 2019-06-07 21:37 UTC (permalink / raw)
To: Jason Gunthorpe, Jerome Glisse, John Hubbard, Felix.Kuehling
Cc: Andrea Arcangeli, linux-rdma, amd-gfx, linux-mm, Jason Gunthorpe,
dri-devel
In-Reply-To: <20190606184438.31646-12-jgg@ziepe.ca>
On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> hmm_release() is called exactly once per hmm. ops->release() cannot
> accidentally trigger any action that would recurse back onto
> hmm->mirrors_sem.
>
> This fixes a use after-free race of the form:
>
> CPU0 CPU1
> hmm_release()
> up_write(&hmm->mirrors_sem);
> hmm_mirror_unregister(mirror)
> down_write(&hmm->mirrors_sem);
> up_write(&hmm->mirrors_sem);
> kfree(mirror)
> mirror->ops->release(mirror)
>
> The only user we have today for ops->release is an empty function, so this
> is unambiguously safe.
>
> As a consequence of plugging this race drivers are not allowed to
> register/unregister mirrors from within a release op.
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
I agree with the analysis above but I'm not sure that release() will
always be an empty function. It might be more efficient to write back
all data migrated to a device "in one pass" instead of relying
on unmap_vmas() calling hmm_start_range_invalidate() per VMA.
I think the bigger issue is potential deadlocks while calling
sync_cpu_device_pagetables() and tasks calling hmm_mirror_unregister():
Say you have three threads:
- Thread A is in try_to_unmap(), either without holding mmap_sem or with
mmap_sem held for read.
- Thread B has some unrelated driver calling hmm_mirror_unregister().
This doesn't require mmap_sem.
- Thread C is about to call migrate_vma().
Thread A Thread B Thread C
try_to_unmap hmm_mirror_unregister migrate_vma
---------------------- ----------------------- ----------------------
hmm_invalidate_range_start
down_read(mirrors_sem)
down_write(mirrors_sem)
// Blocked on A
device_lock
device_lock
// Blocked on C
migrate_vma()
hmm_invalidate_range_s
down_read(mirrors_sem)
// Blocked on B
// Deadlock
Perhaps we should consider using SRCU for walking the mirror->list?
> ---
> mm/hmm.c | 28 +++++++++-------------------
> 1 file changed, 9 insertions(+), 19 deletions(-)
>
> diff --git a/mm/hmm.c b/mm/hmm.c
> index 709d138dd49027..3a45dd3d778248 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -136,26 +136,16 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm)
> WARN_ON(!list_empty(&hmm->ranges));
> mutex_unlock(&hmm->lock);
>
> - down_write(&hmm->mirrors_sem);
> - mirror = list_first_entry_or_null(&hmm->mirrors, struct hmm_mirror,
> - list);
> - while (mirror) {
> - list_del_init(&mirror->list);
> - if (mirror->ops->release) {
> - /*
> - * Drop mirrors_sem so the release callback can wait
> - * on any pending work that might itself trigger a
> - * mmu_notifier callback and thus would deadlock with
> - * us.
> - */
> - up_write(&hmm->mirrors_sem);
> + down_read(&hmm->mirrors_sem);
> + list_for_each_entry(mirror, &hmm->mirrors, list) {
> + /*
> + * Note: The driver is not allowed to trigger
> + * hmm_mirror_unregister() from this thread.
> + */
> + if (mirror->ops->release)
> mirror->ops->release(mirror);
> - down_write(&hmm->mirrors_sem);
> - }
> - mirror = list_first_entry_or_null(&hmm->mirrors,
> - struct hmm_mirror, list);
> }
> - up_write(&hmm->mirrors_sem);
> + up_read(&hmm->mirrors_sem);
>
> hmm_put(hmm);
> }
> @@ -287,7 +277,7 @@ void hmm_mirror_unregister(struct hmm_mirror *mirror)
> struct hmm *hmm = mirror->hmm;
>
> down_write(&hmm->mirrors_sem);
> - list_del_init(&mirror->list);
> + list_del(&mirror->list);
> up_write(&hmm->mirrors_sem);
> hmm_put(hmm);
> memset(&mirror->hmm, POISON_INUSE, sizeof(mirror->hmm));
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [PATCH v2 hmm 09/11] mm/hmm: Poison hmm_range during unregister
From: Jason Gunthorpe @ 2019-06-07 20:49 UTC (permalink / raw)
To: Ralph Campbell
Cc: Andrea Arcangeli, linux-rdma, John Hubbard, Felix.Kuehling,
dri-devel, linux-mm, Jerome Glisse, amd-gfx
In-Reply-To: <96a2739f-6902-05be-7143-289b41c4304d@nvidia.com>
On Fri, Jun 07, 2019 at 01:46:30PM -0700, Ralph Campbell wrote:
>
> On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> > From: Jason Gunthorpe <jgg@mellanox.com>
> >
> > Trying to misuse a range outside its lifetime is a kernel bug. Use WARN_ON
> > and poison bytes to detect this condition.
> >
> > Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> > Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
>
> Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
>
> > v2
> > - Keep range start/end valid after unregistration (Jerome)
> > mm/hmm.c | 7 +++++--
> > 1 file changed, 5 insertions(+), 2 deletions(-)
> >
> > diff --git a/mm/hmm.c b/mm/hmm.c
> > index 6802de7080d172..c2fecb3ecb11e1 100644
> > +++ b/mm/hmm.c
> > @@ -937,7 +937,7 @@ void hmm_range_unregister(struct hmm_range *range)
> > struct hmm *hmm = range->hmm;
> > /* Sanity check this really should not happen. */
> > - if (hmm == NULL || range->end <= range->start)
> > + if (WARN_ON(range->end <= range->start))
> > return;
>
> WARN_ON() is definitely better than silent return but I wonder how
> useful it is since the caller shouldn't be modifying the hmm_range
> once it is registered. Other fields could be changed too...
I deleted the warn on (see the other thread), but I'm confused by your
"shouldn't be modified" statement.
The only thing that needs to be set and remain unchanged for register
is the virtual start/end address. Everything else should be done once
it is clear to proceed based on the collision-retry locking scheme
this uses.
Basically the range register only setups a 'detector' for colliding
invalidations. The other stuff in the struct is just random temporary
storage for the API.
AFAICS at least..
Jason
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [PATCH v2 hmm 10/11] mm/hmm: Do not use list*_rcu() for hmm->ranges
From: Ralph Campbell @ 2019-06-07 20:49 UTC (permalink / raw)
To: Jason Gunthorpe, Jerome Glisse, John Hubbard, Felix.Kuehling
Cc: Andrea Arcangeli, linux-rdma, amd-gfx, linux-mm, Jason Gunthorpe,
dri-devel
In-Reply-To: <20190606184438.31646-11-jgg@ziepe.ca>
On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> This list is always read and written while holding hmm->lock so there is
> no need for the confusing _rcu annotations.
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
> ---
> mm/hmm.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/mm/hmm.c b/mm/hmm.c
> index c2fecb3ecb11e1..709d138dd49027 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -911,7 +911,7 @@ int hmm_range_register(struct hmm_range *range,
> mutex_lock(&hmm->lock);
>
> range->hmm = hmm;
> - list_add_rcu(&range->list, &hmm->ranges);
> + list_add(&range->list, &hmm->ranges);
>
> /*
> * If there are any concurrent notifiers we have to wait for them for
> @@ -941,7 +941,7 @@ void hmm_range_unregister(struct hmm_range *range)
> return;
>
> mutex_lock(&hmm->lock);
> - list_del_rcu(&range->list);
> + list_del(&range->list);
> mutex_unlock(&hmm->lock);
>
> /* Drop reference taken by hmm_range_register() */
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [PATCH v2 hmm 09/11] mm/hmm: Poison hmm_range during unregister
From: Ralph Campbell @ 2019-06-07 20:46 UTC (permalink / raw)
To: Jason Gunthorpe, Jerome Glisse, John Hubbard, Felix.Kuehling
Cc: Andrea Arcangeli, linux-rdma, amd-gfx, linux-mm, Jason Gunthorpe,
dri-devel
In-Reply-To: <20190606184438.31646-10-jgg@ziepe.ca>
On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> Trying to misuse a range outside its lifetime is a kernel bug. Use WARN_ON
> and poison bytes to detect this condition.
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
> ---
> v2
> - Keep range start/end valid after unregistration (Jerome)
> ---
> mm/hmm.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/mm/hmm.c b/mm/hmm.c
> index 6802de7080d172..c2fecb3ecb11e1 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -937,7 +937,7 @@ void hmm_range_unregister(struct hmm_range *range)
> struct hmm *hmm = range->hmm;
>
> /* Sanity check this really should not happen. */
> - if (hmm == NULL || range->end <= range->start)
> + if (WARN_ON(range->end <= range->start))
> return;
WARN_ON() is definitely better than silent return but I wonder how
useful it is since the caller shouldn't be modifying the hmm_range
once it is registered. Other fields could be changed too...
> mutex_lock(&hmm->lock);
> @@ -948,7 +948,10 @@ void hmm_range_unregister(struct hmm_range *range)
> range->valid = false;
> mmput(hmm->mm);
> hmm_put(hmm);
> - range->hmm = NULL;
> +
> + /* The range is now invalid, leave it poisoned. */
> + range->valid = false;
> + memset(&range->hmm, POISON_INUSE, sizeof(range->hmm));
> }
> EXPORT_SYMBOL(hmm_range_unregister);
>
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [PATCH v2 hmm 05/11] mm/hmm: Remove duplicate condition test before wait_event_timeout
From: Jason Gunthorpe @ 2019-06-07 20:44 UTC (permalink / raw)
To: Ralph Campbell
Cc: Andrea Arcangeli, linux-rdma, John Hubbard, Felix.Kuehling,
dri-devel, linux-mm, Jerome Glisse, amd-gfx
In-Reply-To: <e17aa8c5-790c-d977-2eb8-c18cdaa4cbb3@nvidia.com>
On Fri, Jun 07, 2019 at 01:21:12PM -0700, Ralph Campbell wrote:
> > What I want to get to is a pattern like this:
> >
> > pagefault():
> >
> > hmm_range_register(&range);
> > again:
> > /* On the slow path, if we appear to be live locked then we get
> > the write side of mmap_sem which will break the live lock,
> > otherwise this gets the read lock */
> > if (hmm_range_start_and_lock(&range))
> > goto err;
> >
> > lockdep_assert_held(range->mm->mmap_sem);
> >
> > // Optional: Avoid useless expensive work
> > if (hmm_range_needs_retry(&range))
> > goto again;
> > hmm_range_(touch vmas)
> >
> > take_lock(driver->update);
> > if (hmm_range_end(&range) {
> > release_lock(driver->update);
> > goto again;
> > }
> > // Finish driver updates
> > release_lock(driver->update);
> >
> > // Releases mmap_sem
> > hmm_range_unregister_and_unlock(&range);
> >
> > What do you think?
> >
> > Is it clear?
> >
> > Jason
> >
>
> Are you talking about acquiring mmap_sem in hmm_range_start_and_lock()?
> Usually, the fault code has to lock mmap_sem for read in order to
> call find_vma() so it can set range.vma.
> If HMM drops mmap_sem - which I don't think it should, just return an
> error to tell the caller to drop mmap_sem and retry - the find_vma()
> will need to be repeated as well.
Overall I don't think it makes a lot of sense to sleep for retry in
hmm_range_start_and_lock() while holding mmap_sem. It would be better
to drop that lock, sleep, then re-acquire it as part of the hmm logic.
The find_vma should be done inside the critical section created by
hmm_range_start_and_lock(), not before it. If we are retrying then we
already slept and the additional CPU cost to repeat the find_vma is
immaterial, IMHO?
Do you see a reason why the find_vma() ever needs to be before the
'again' in my above example? range.vma does not need to be set for
range_register.
> I'm also not sure about acquiring the mmap_sem for write as way to
> mitigate thrashing. It seems to me that if a device and a CPU are
> both faulting on the same page,
One of the reasons to prefer this approach is that it means we don't
need to keep track of which ranges we are faulting, and if there is a
lot of *unrelated* fault activity (unlikely?) we can resolve it using
mmap sem instead of this elaborate ranges scheme and related
locking.
This would reduce the overall work in the page fault and
invalidate_start/end paths for the common uncontended cases.
> some sort of backoff delay is needed to let one side or the other
> make some progress.
What the write side of the mmap_sem would do is force the CPU and
device to cleanly take turns. Once the device pages are registered
under the write side the CPU will have to wait in invalidate_start for
the driver to complete a shootdown, then the whole thing starts all
over again.
It is certainly imaginable something could have a 'min life' timer for
a device mapping and hold mm invalidate_start, and device pagefault
for that min time to promote better sharing.
But, if we don't use the mmap_sem then we can livelock and the device
will see an unrecoverable error from the timeout which means we have
risk that under load the system will simply obscurely fail. This seems
unacceptable to me..
Particularly since for the ODP use case the issue is not trashing
migration as a GPU might have, but simple system stability under swap
load. We do not want the ODP pagefault to permanently fail due to
timeout if the VMA is still valid..
Jason
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [PATCH v2 hmm 08/11] mm/hmm: Remove racy protection against double-unregistration
From: Ralph Campbell @ 2019-06-07 20:33 UTC (permalink / raw)
To: Jason Gunthorpe, Jerome Glisse, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
In-Reply-To: <20190606184438.31646-9-jgg-uk2M96/98Pc@public.gmane.org>
On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> No other register/unregister kernel API attempts to provide this kind of
> protection as it is inherently racy, so just drop it.
>
> Callers should provide their own protection, it appears nouveau already
> does, but just in case drop a debugging POISON.
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
> ---
> mm/hmm.c | 9 ++-------
> 1 file changed, 2 insertions(+), 7 deletions(-)
>
> diff --git a/mm/hmm.c b/mm/hmm.c
> index c702cd72651b53..6802de7080d172 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -284,18 +284,13 @@ EXPORT_SYMBOL(hmm_mirror_register);
> */
> void hmm_mirror_unregister(struct hmm_mirror *mirror)
> {
> - struct hmm *hmm = READ_ONCE(mirror->hmm);
> -
> - if (hmm == NULL)
> - return;
> + struct hmm *hmm = mirror->hmm;
>
> down_write(&hmm->mirrors_sem);
> list_del_init(&mirror->list);
> - /* To protect us against double unregister ... */
> - mirror->hmm = NULL;
> up_write(&hmm->mirrors_sem);
> -
> hmm_put(hmm);
> + memset(&mirror->hmm, POISON_INUSE, sizeof(mirror->hmm));
> }
> EXPORT_SYMBOL(hmm_mirror_unregister);
>
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH v2 hmm 07/11] mm/hmm: Use lockdep instead of comments
From: Ralph Campbell @ 2019-06-07 20:31 UTC (permalink / raw)
To: Jason Gunthorpe, Jerome Glisse, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
In-Reply-To: <20190606184438.31646-8-jgg-uk2M96/98Pc@public.gmane.org>
On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> So we can check locking at runtime.
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
> ---
> v2
> - Fix missing & in lockdeps (Jason)
> ---
> mm/hmm.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/mm/hmm.c b/mm/hmm.c
> index f67ba32983d9f1..c702cd72651b53 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -254,11 +254,11 @@ static const struct mmu_notifier_ops hmm_mmu_notifier_ops = {
> *
> * To start mirroring a process address space, the device driver must register
> * an HMM mirror struct.
> - *
> - * THE mm->mmap_sem MUST BE HELD IN WRITE MODE !
> */
> int hmm_mirror_register(struct hmm_mirror *mirror, struct mm_struct *mm)
> {
> + lockdep_assert_held_exclusive(&mm->mmap_sem);
> +
> /* Sanity check */
> if (!mm || !mirror || !mirror->ops)
> return -EINVAL;
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH v2 hmm 06/11] mm/hmm: Hold on to the mmget for the lifetime of the range
From: Ralph Campbell @ 2019-06-07 20:29 UTC (permalink / raw)
To: Jason Gunthorpe, Jerome Glisse, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
In-Reply-To: <20190606184438.31646-7-jgg-uk2M96/98Pc@public.gmane.org>
On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> Range functions like hmm_range_snapshot() and hmm_range_fault() call
> find_vma, which requires hodling the mmget() and the mmap_sem for the mm.
>
> Make this simpler for the callers by holding the mmget() inside the range
> for the lifetime of the range. Other functions that accept a range should
> only be called if the range is registered.
>
> This has the side effect of directly preventing hmm_release() from
> happening while a range is registered. That means range->dead cannot be
> false during the lifetime of the range, so remove dead and
> hmm_mirror_mm_is_alive() entirely.
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Looks good to me.
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
> ---
> v2:
> - Use Jerome's idea of just holding the mmget() for the range lifetime,
> rework the patch to use that as as simplification to remove dead in
> one step
> ---
> include/linux/hmm.h | 26 --------------------------
> mm/hmm.c | 28 ++++++++++------------------
> 2 files changed, 10 insertions(+), 44 deletions(-)
>
> diff --git a/include/linux/hmm.h b/include/linux/hmm.h
> index 2ab35b40992b24..0e20566802967a 100644
> --- a/include/linux/hmm.h
> +++ b/include/linux/hmm.h
> @@ -91,7 +91,6 @@
> * @mirrors_sem: read/write semaphore protecting the mirrors list
> * @wq: wait queue for user waiting on a range invalidation
> * @notifiers: count of active mmu notifiers
> - * @dead: is the mm dead ?
> */
> struct hmm {
> struct mm_struct *mm;
> @@ -104,7 +103,6 @@ struct hmm {
> wait_queue_head_t wq;
> struct rcu_head rcu;
> long notifiers;
> - bool dead;
> };
>
> /*
> @@ -469,30 +467,6 @@ struct hmm_mirror {
> int hmm_mirror_register(struct hmm_mirror *mirror, struct mm_struct *mm);
> void hmm_mirror_unregister(struct hmm_mirror *mirror);
>
> -/*
> - * hmm_mirror_mm_is_alive() - test if mm is still alive
> - * @mirror: the HMM mm mirror for which we want to lock the mmap_sem
> - * Return: false if the mm is dead, true otherwise
> - *
> - * This is an optimization, it will not always accurately return false if the
> - * mm is dead; i.e., there can be false negatives (process is being killed but
> - * HMM is not yet informed of that). It is only intended to be used to optimize
> - * out cases where the driver is about to do something time consuming and it
> - * would be better to skip it if the mm is dead.
> - */
> -static inline bool hmm_mirror_mm_is_alive(struct hmm_mirror *mirror)
> -{
> - struct mm_struct *mm;
> -
> - if (!mirror || !mirror->hmm)
> - return false;
> - mm = READ_ONCE(mirror->hmm->mm);
> - if (mirror->hmm->dead || !mm)
> - return false;
> -
> - return true;
> -}
> -
> /*
> * Please see Documentation/vm/hmm.rst for how to use the range API.
> */
> diff --git a/mm/hmm.c b/mm/hmm.c
> index dc30edad9a8a02..f67ba32983d9f1 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -80,7 +80,6 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm)
> mutex_init(&hmm->lock);
> kref_init(&hmm->kref);
> hmm->notifiers = 0;
> - hmm->dead = false;
> hmm->mm = mm;
>
> hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops;
> @@ -124,20 +123,17 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm)
> {
> struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier);
> struct hmm_mirror *mirror;
> - struct hmm_range *range;
>
> /* hmm is in progress to free */
> if (!kref_get_unless_zero(&hmm->kref))
> return;
>
> - /* Report this HMM as dying. */
> - hmm->dead = true;
> -
> - /* Wake-up everyone waiting on any range. */
> mutex_lock(&hmm->lock);
> - list_for_each_entry(range, &hmm->ranges, list)
> - range->valid = false;
> - wake_up_all(&hmm->wq);
> + /*
> + * Since hmm_range_register() holds the mmget() lock hmm_release() is
> + * prevented as long as a range exists.
> + */
> + WARN_ON(!list_empty(&hmm->ranges));
> mutex_unlock(&hmm->lock);
>
> down_write(&hmm->mirrors_sem);
> @@ -909,8 +905,8 @@ int hmm_range_register(struct hmm_range *range,
> range->start = start;
> range->end = end;
>
> - /* Check if hmm_mm_destroy() was call. */
> - if (hmm->mm == NULL || hmm->dead)
> + /* Prevent hmm_release() from running while the range is valid */
> + if (!mmget_not_zero(hmm->mm))
> return -EFAULT;
>
> range->hmm = hmm;
> @@ -955,6 +951,7 @@ void hmm_range_unregister(struct hmm_range *range)
>
> /* Drop reference taken by hmm_range_register() */
> range->valid = false;
> + mmput(hmm->mm);
> hmm_put(hmm);
> range->hmm = NULL;
> }
> @@ -982,10 +979,7 @@ long hmm_range_snapshot(struct hmm_range *range)
> struct vm_area_struct *vma;
> struct mm_walk mm_walk;
>
> - /* Check if hmm_mm_destroy() was call. */
> - if (hmm->mm == NULL || hmm->dead)
> - return -EFAULT;
> -
> + lockdep_assert_held(&hmm->mm->mmap_sem);
> do {
> /* If range is no longer valid force retry. */
> if (!range->valid)
> @@ -1080,9 +1074,7 @@ long hmm_range_fault(struct hmm_range *range, bool block)
> struct mm_walk mm_walk;
> int ret;
>
> - /* Check if hmm_mm_destroy() was call. */
> - if (hmm->mm == NULL || hmm->dead)
> - return -EFAULT;
> + lockdep_assert_held(&hmm->mm->mmap_sem);
>
> do {
> /* If range is no longer valid force retry. */
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH v2 hmm 05/11] mm/hmm: Remove duplicate condition test before wait_event_timeout
From: Ralph Campbell @ 2019-06-07 20:21 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Andrea Arcangeli, linux-rdma-u79uwXL29TY76Z2rM5mHXA, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jerome Glisse,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
In-Reply-To: <20190607191302.GR14802-uk2M96/98Pc@public.gmane.org>
On 6/7/19 12:13 PM, Jason Gunthorpe wrote:
> On Fri, Jun 07, 2019 at 12:01:45PM -0700, Ralph Campbell wrote:
>>
>> On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
>>> From: Jason Gunthorpe <jgg@mellanox.com>
>>>
>>> The wait_event_timeout macro already tests the condition as its first
>>> action, so there is no reason to open code another version of this, all
>>> that does is skip the might_sleep() debugging in common cases, which is
>>> not helpful.
>>>
>>> Further, based on prior patches, we can no simplify the required condition
>>> test:
>>> - If range is valid memory then so is range->hmm
>>> - If hmm_release() has run then range->valid is set to false
>>> at the same time as dead, so no reason to check both.
>>> - A valid hmm has a valid hmm->mm.
>>>
>>> Also, add the READ_ONCE for range->valid as there is no lock held here.
>>>
>>> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
>>> Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
>>> include/linux/hmm.h | 12 ++----------
>>> 1 file changed, 2 insertions(+), 10 deletions(-)
>>>
>>> diff --git a/include/linux/hmm.h b/include/linux/hmm.h
>>> index 4ee3acabe5ed22..2ab35b40992b24 100644
>>> +++ b/include/linux/hmm.h
>>> @@ -218,17 +218,9 @@ static inline unsigned long hmm_range_page_size(const struct hmm_range *range)
>>> static inline bool hmm_range_wait_until_valid(struct hmm_range *range,
>>> unsigned long timeout)
>>> {
>>> - /* Check if mm is dead ? */
>>> - if (range->hmm == NULL || range->hmm->dead || range->hmm->mm == NULL) {
>>> - range->valid = false;
>>> - return false;
>>> - }
>>> - if (range->valid)
>>> - return true;
>>> - wait_event_timeout(range->hmm->wq, range->valid || range->hmm->dead,
>>> + wait_event_timeout(range->hmm->wq, range->valid,
>>> msecs_to_jiffies(timeout));
>>> - /* Return current valid status just in case we get lucky */
>>> - return range->valid;
>>> + return READ_ONCE(range->valid);
>>> }
>>> /*
>>>
>>
>> Since we are simplifying things, perhaps we should consider merging
>> hmm_range_wait_until_valid() info hmm_range_register() and
>> removing hmm_range_wait_until_valid() since the pattern
>> is to always call the two together.
>
> ? the hmm.rst shows the hmm_range_wait_until_valid being called in the
> (ret == -EAGAIN) path. It is confusing because it should really just
> have the again label moved up above hmm_range_wait_until_valid() as
> even if we get the driver lock it could still be a long wait for the
> colliding invalidation to clear.
>
> What I want to get to is a pattern like this:
>
> pagefault():
>
> hmm_range_register(&range);
> again:
> /* On the slow path, if we appear to be live locked then we get
> the write side of mmap_sem which will break the live lock,
> otherwise this gets the read lock */
> if (hmm_range_start_and_lock(&range))
> goto err;
>
> lockdep_assert_held(range->mm->mmap_sem);
>
> // Optional: Avoid useless expensive work
> if (hmm_range_needs_retry(&range))
> goto again;
> hmm_range_(touch vmas)
>
> take_lock(driver->update);
> if (hmm_range_end(&range) {
> release_lock(driver->update);
> goto again;
> }
> // Finish driver updates
> release_lock(driver->update);
>
> // Releases mmap_sem
> hmm_range_unregister_and_unlock(&range);
>
> What do you think?
>
> Is it clear?
>
> Jason
>
Are you talking about acquiring mmap_sem in hmm_range_start_and_lock()?
Usually, the fault code has to lock mmap_sem for read in order to
call find_vma() so it can set range.vma.
If HMM drops mmap_sem - which I don't think it should, just return an
error to tell the caller to drop mmap_sem and retry - the find_vma()
will need to be repeated as well.
I'm also not sure about acquiring the mmap_sem for write as way to
mitigate thrashing. It seems to me that if a device and a CPU are
both faulting on the same page, some sort of backoff delay is needed
to let one side or the other make some progress.
Thrashing mitigation and how migrate_vma() plays in this is a
deep topic for thought.
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH v2 hmm 05/11] mm/hmm: Remove duplicate condition test before wait_event_timeout
From: Jason Gunthorpe @ 2019-06-07 19:13 UTC (permalink / raw)
To: Ralph Campbell
Cc: Andrea Arcangeli, linux-rdma-u79uwXL29TY76Z2rM5mHXA, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jerome Glisse,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
In-Reply-To: <6833be96-12a3-1a1c-1514-c148ba2dd87b-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
On Fri, Jun 07, 2019 at 12:01:45PM -0700, Ralph Campbell wrote:
>
> On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> > From: Jason Gunthorpe <jgg@mellanox.com>
> >
> > The wait_event_timeout macro already tests the condition as its first
> > action, so there is no reason to open code another version of this, all
> > that does is skip the might_sleep() debugging in common cases, which is
> > not helpful.
> >
> > Further, based on prior patches, we can no simplify the required condition
> > test:
> > - If range is valid memory then so is range->hmm
> > - If hmm_release() has run then range->valid is set to false
> > at the same time as dead, so no reason to check both.
> > - A valid hmm has a valid hmm->mm.
> >
> > Also, add the READ_ONCE for range->valid as there is no lock held here.
> >
> > Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> > Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
> > include/linux/hmm.h | 12 ++----------
> > 1 file changed, 2 insertions(+), 10 deletions(-)
> >
> > diff --git a/include/linux/hmm.h b/include/linux/hmm.h
> > index 4ee3acabe5ed22..2ab35b40992b24 100644
> > +++ b/include/linux/hmm.h
> > @@ -218,17 +218,9 @@ static inline unsigned long hmm_range_page_size(const struct hmm_range *range)
> > static inline bool hmm_range_wait_until_valid(struct hmm_range *range,
> > unsigned long timeout)
> > {
> > - /* Check if mm is dead ? */
> > - if (range->hmm == NULL || range->hmm->dead || range->hmm->mm == NULL) {
> > - range->valid = false;
> > - return false;
> > - }
> > - if (range->valid)
> > - return true;
> > - wait_event_timeout(range->hmm->wq, range->valid || range->hmm->dead,
> > + wait_event_timeout(range->hmm->wq, range->valid,
> > msecs_to_jiffies(timeout));
> > - /* Return current valid status just in case we get lucky */
> > - return range->valid;
> > + return READ_ONCE(range->valid);
> > }
> > /*
> >
>
> Since we are simplifying things, perhaps we should consider merging
> hmm_range_wait_until_valid() info hmm_range_register() and
> removing hmm_range_wait_until_valid() since the pattern
> is to always call the two together.
? the hmm.rst shows the hmm_range_wait_until_valid being called in the
(ret == -EAGAIN) path. It is confusing because it should really just
have the again label moved up above hmm_range_wait_until_valid() as
even if we get the driver lock it could still be a long wait for the
colliding invalidation to clear.
What I want to get to is a pattern like this:
pagefault():
hmm_range_register(&range);
again:
/* On the slow path, if we appear to be live locked then we get
the write side of mmap_sem which will break the live lock,
otherwise this gets the read lock */
if (hmm_range_start_and_lock(&range))
goto err;
lockdep_assert_held(range->mm->mmap_sem);
// Optional: Avoid useless expensive work
if (hmm_range_needs_retry(&range))
goto again;
hmm_range_(touch vmas)
take_lock(driver->update);
if (hmm_range_end(&range) {
release_lock(driver->update);
goto again;
}
// Finish driver updates
release_lock(driver->update);
// Releases mmap_sem
hmm_range_unregister_and_unlock(&range);
What do you think?
Is it clear?
Jason
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH v2 hmm 05/11] mm/hmm: Remove duplicate condition test before wait_event_timeout
From: Ralph Campbell @ 2019-06-07 19:01 UTC (permalink / raw)
To: Jason Gunthorpe, Jerome Glisse, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
In-Reply-To: <20190606184438.31646-6-jgg-uk2M96/98Pc@public.gmane.org>
On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> The wait_event_timeout macro already tests the condition as its first
> action, so there is no reason to open code another version of this, all
> that does is skip the might_sleep() debugging in common cases, which is
> not helpful.
>
> Further, based on prior patches, we can no simplify the required condition
> test:
> - If range is valid memory then so is range->hmm
> - If hmm_release() has run then range->valid is set to false
> at the same time as dead, so no reason to check both.
> - A valid hmm has a valid hmm->mm.
>
> Also, add the READ_ONCE for range->valid as there is no lock held here.
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
> ---
> include/linux/hmm.h | 12 ++----------
> 1 file changed, 2 insertions(+), 10 deletions(-)
>
> diff --git a/include/linux/hmm.h b/include/linux/hmm.h
> index 4ee3acabe5ed22..2ab35b40992b24 100644
> --- a/include/linux/hmm.h
> +++ b/include/linux/hmm.h
> @@ -218,17 +218,9 @@ static inline unsigned long hmm_range_page_size(const struct hmm_range *range)
> static inline bool hmm_range_wait_until_valid(struct hmm_range *range,
> unsigned long timeout)
> {
> - /* Check if mm is dead ? */
> - if (range->hmm == NULL || range->hmm->dead || range->hmm->mm == NULL) {
> - range->valid = false;
> - return false;
> - }
> - if (range->valid)
> - return true;
> - wait_event_timeout(range->hmm->wq, range->valid || range->hmm->dead,
> + wait_event_timeout(range->hmm->wq, range->valid,
> msecs_to_jiffies(timeout));
> - /* Return current valid status just in case we get lucky */
> - return range->valid;
> + return READ_ONCE(range->valid);
> }
>
> /*
>
Since we are simplifying things, perhaps we should consider merging
hmm_range_wait_until_valid() info hmm_range_register() and
removing hmm_range_wait_until_valid() since the pattern
is to always call the two together.
In any case, this looks OK to me so you can add
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [pull request][for-next 0/9] Generic DIM lib for netdev and RDMA
From: Jason Gunthorpe @ 2019-06-07 18:57 UTC (permalink / raw)
To: Saeed Mahameed
Cc: Max Gurtovoy, linux-rdma@vger.kernel.org, davem@davemloft.net,
andy@greyhouse.net, Tal Gilboa, michael.chan@broadcom.com,
netdev@vger.kernel.org, leon@kernel.org, dledford@redhat.com
In-Reply-To: <9faeadac971aaf481b1066b1dde0fc9e77e893a5.camel@mellanox.com>
On Fri, Jun 07, 2019 at 06:14:11PM +0000, Saeed Mahameed wrote:
> On Thu, 2019-06-06 at 13:07 +0000, Jason Gunthorpe wrote:
> > On Thu, Jun 06, 2019 at 10:19:41AM +0300, Max Gurtovoy wrote:
> > > > > Solution:
> > > > > - Common logic is declared in include/linux/dim.h and
> > > > > implemented in
> > > > > lib/dim/dim.c
> > > > > - Net DIM (existing) logic is declared in
> > > > > include/linux/net_dim.h and
> > > > > implemented in lib/dim/net_dim.c, which uses the common
> > > > > logic from dim.h
> > > > > - Any new DIM logic will be declared in
> > > > > "/include/linux/new_dim.h" and
> > > > > implemented in "lib/dim/new_dim.c".
> > > > > - This new implementation will expose modified versions of
> > > > > profiles,
> > > > > dim_step() and dim_decision().
> > > > >
> > > > > Pros for this solution are:
> > > > > - Zero impact on existing net_dim implementation and usage
> > > > > - Relatively more code reuse (compared to two separate
> > > > > solutions)
> > > > > - Increased extensibility
> > > > >
> > > > > Tal Gilboa (6):
> > > > > linux/dim: Move logic to dim.h
> > > > > linux/dim: Remove "net" prefix from internal DIM members
> > > > > linux/dim: Rename externally exposed macros
> > > > > linux/dim: Rename net_dim_sample() to
> > > > > net_dim_update_sample()
> > > > > linux/dim: Rename externally used net_dim members
> > > > > linux/dim: Move implementation to .c files
> > > > >
> > > > > Yamin Friedman (3):
> > > > > linux/dim: Add completions count to dim_sample
> > > > > linux/dim: Implement rdma_dim
> > > > > RDMA/core: Provide RDMA DIM support for ULPs
> > > > Saeed,
> > > >
> > > > No, for the RDMA patches.
> > > > We need to see usage of those APIs before merging.
> > >
> > > I've asked Yamin to prepare patches for NVMeoF initiator and target
> > > for
> > > review, so I guess he has it on his plate (this is how he tested
> > > it..).
> > >
> > > It might cause conflict with NVMe/blk branch maintained by Sagi,
> > > Christoph
> > > and Jens.
> >
> > It looks like nvme could pull this series + the RDMA patches into the
> > nvme tree via PR? I'm not familiar with how that tree works.
> >
> > But we need to get the patches posted right away..
> >
>
> What do you suggest here ?
> I think the netdev community also deserve to see the rdma patches, at
> least with an external link, I can drop the last patch (or two ) ? but
> i need an external rdma link for people who are going to review this
> series.
Yes, all the patches need to be posted. We should have a 'double
branch' where you send the linux/dim & net stuff to net and then we
add the RDMA stuff on top and send to nvme & rdma with the ULP
patches
Assuming nvme takes pull requests.
But the whole thing should be posted as a single series on the list to
get acks before the PRs are generated.
Similar to how we've run the mlx5 shared branch
Jason
^ permalink raw reply
* Re: [PATCH v2 hmm 04/11] mm/hmm: Simplify hmm_get_or_create and make it reliable
From: Ralph Campbell @ 2019-06-07 18:52 UTC (permalink / raw)
To: Jason Gunthorpe, Jerome Glisse, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
In-Reply-To: <20190606184438.31646-5-jgg-uk2M96/98Pc@public.gmane.org>
On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> As coded this function can false-fail in various racy situations. Make it
> reliable by running only under the write side of the mmap_sem and avoiding
> the false-failing compare/exchange pattern.
>
> Also make the locking very easy to understand by only ever reading or
> writing mm->hmm while holding the write side of the mmap_sem.
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
> ---
> v2:
> - Fix error unwind of mmgrab (Jerome)
> - Use hmm local instead of 2nd container_of (Jerome)
> ---
> mm/hmm.c | 80 ++++++++++++++++++++------------------------------------
> 1 file changed, 29 insertions(+), 51 deletions(-)
>
> diff --git a/mm/hmm.c b/mm/hmm.c
> index cc7c26fda3300e..dc30edad9a8a02 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -40,16 +40,6 @@
> #if IS_ENABLED(CONFIG_HMM_MIRROR)
> static const struct mmu_notifier_ops hmm_mmu_notifier_ops;
>
> -static inline struct hmm *mm_get_hmm(struct mm_struct *mm)
> -{
> - struct hmm *hmm = READ_ONCE(mm->hmm);
> -
> - if (hmm && kref_get_unless_zero(&hmm->kref))
> - return hmm;
> -
> - return NULL;
> -}
> -
> /**
> * hmm_get_or_create - register HMM against an mm (HMM internal)
> *
> @@ -64,11 +54,20 @@ static inline struct hmm *mm_get_hmm(struct mm_struct *mm)
> */
> static struct hmm *hmm_get_or_create(struct mm_struct *mm)
> {
> - struct hmm *hmm = mm_get_hmm(mm);
> - bool cleanup = false;
> + struct hmm *hmm;
>
> - if (hmm)
> - return hmm;
> + lockdep_assert_held_exclusive(&mm->mmap_sem);
> +
> + if (mm->hmm) {
> + if (kref_get_unless_zero(&mm->hmm->kref))
> + return mm->hmm;
> + /*
> + * The hmm is being freed by some other CPU and is pending a
> + * RCU grace period, but this CPU can NULL now it since we
> + * have the mmap_sem.
> + */
> + mm->hmm = NULL;
> + }
>
> hmm = kmalloc(sizeof(*hmm), GFP_KERNEL);
> if (!hmm)
> @@ -83,57 +82,36 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm)
> hmm->notifiers = 0;
> hmm->dead = false;
> hmm->mm = mm;
> - mmgrab(hmm->mm);
> -
> - spin_lock(&mm->page_table_lock);
> - if (!mm->hmm)
> - mm->hmm = hmm;
> - else
> - cleanup = true;
> - spin_unlock(&mm->page_table_lock);
>
> - if (cleanup)
> - goto error;
> -
> - /*
> - * We should only get here if hold the mmap_sem in write mode ie on
> - * registration of first mirror through hmm_mirror_register()
> - */
> hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops;
> - if (__mmu_notifier_register(&hmm->mmu_notifier, mm))
> - goto error_mm;
> + if (__mmu_notifier_register(&hmm->mmu_notifier, mm)) {
> + kfree(hmm);
> + return NULL;
> + }
>
> + mmgrab(hmm->mm);
> + mm->hmm = hmm;
> return hmm;
> -
> -error_mm:
> - spin_lock(&mm->page_table_lock);
> - if (mm->hmm == hmm)
> - mm->hmm = NULL;
> - spin_unlock(&mm->page_table_lock);
> -error:
> - mmdrop(hmm->mm);
> - kfree(hmm);
> - return NULL;
> }
>
> static void hmm_free_rcu(struct rcu_head *rcu)
> {
> - kfree(container_of(rcu, struct hmm, rcu));
> + struct hmm *hmm = container_of(rcu, struct hmm, rcu);
> +
> + down_write(&hmm->mm->mmap_sem);
> + if (hmm->mm->hmm == hmm)
> + hmm->mm->hmm = NULL;
> + up_write(&hmm->mm->mmap_sem);
> + mmdrop(hmm->mm);
> +
> + kfree(hmm);
> }
>
> static void hmm_free(struct kref *kref)
> {
> struct hmm *hmm = container_of(kref, struct hmm, kref);
> - struct mm_struct *mm = hmm->mm;
> -
> - mmu_notifier_unregister_no_release(&hmm->mmu_notifier, mm);
>
> - spin_lock(&mm->page_table_lock);
> - if (mm->hmm == hmm)
> - mm->hmm = NULL;
> - spin_unlock(&mm->page_table_lock);
> -
> - mmdrop(hmm->mm);
> + mmu_notifier_unregister_no_release(&hmm->mmu_notifier, hmm->mm);
> mmu_notifier_call_srcu(&hmm->rcu, hmm_free_rcu);
> }
>
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH v2 hmm 03/11] mm/hmm: Hold a mmgrab from hmm to mm
From: Jason Gunthorpe @ 2019-06-07 18:51 UTC (permalink / raw)
To: Ralph Campbell
Cc: Andrea Arcangeli,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, Jerome Glisse,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
In-Reply-To: <605172dc-5c66-123f-61a3-8e6880678aef-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
On Fri, Jun 07, 2019 at 11:41:20AM -0700, Ralph Campbell wrote:
>
> On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> > From: Jason Gunthorpe <jgg@mellanox.com>
> >
> > So long a a struct hmm pointer exists, so should the struct mm it is
>
> s/a a/as a/
Got it, thanks
Jason
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH RFC 00/10] RDMA/FS DAX truncate proposal
From: Jason Gunthorpe @ 2019-06-07 18:50 UTC (permalink / raw)
To: Ira Weiny
Cc: Jan Kara, Dan Williams, Theodore Ts'o, Jeff Layton,
Dave Chinner, Matthew Wilcox, linux-xfs, Andrew Morton,
John Hubbard, Jérôme Glisse, linux-fsdevel,
linux-kernel, linux-nvdimm, linux-ext4, linux-mm, linux-rdma
In-Reply-To: <20190607182534.GC14559@iweiny-DESK2.sc.intel.com>
On Fri, Jun 07, 2019 at 11:25:35AM -0700, Ira Weiny wrote:
> And I think this is related to what Christoph Hellwig is doing with bio_vec and
> dma. Really we want drivers out of the page processing business.
At least for RDMA, and a few other places I've noticed, I'd really
like to get totally out of the handling struct pages game.
We are DMA based and really only want DMA addresses for the target
device. I know other places need CPU pages or more complicated
things.. But I also know there are other drivers like RDMA..
So I think it would be very helpful to have a driver API something
like:
int get_user_mem_for_dma(struct device *dma_device,
void __user *mem, size_t length,
struct gup_handle *res,
struct 'bio dma list' *dma_list,
const struct dma_params *params);
void put_user_mem_for_dma(struct gup_handle *res,
struct 'bio dma list' *dma_list);
And we could hope to put in there all the specialty logic we want to
have for this flow:
- The weird HMM stuff in hmm_range_dma_map()
- Interaction with DAX
- Interaction with DMA BUF
- Holding file leases
- PCI peer 2 peer features
- Optimizations for huge pages
- Handling page dirtying from DMA
- etc
I think Matthew was suggesting something like this at LS/MM, so +1
from here..
When Christoph sends his BIO dma work I was thinking of investigating
this avenue, as we already have something quite similiar in RDMA that
could perhaps be hoisted out for re-use into mm/
Jason
^ permalink raw reply
* Re: [PATCH v2 hmm 03/11] mm/hmm: Hold a mmgrab from hmm to mm
From: Ralph Campbell @ 2019-06-07 18:41 UTC (permalink / raw)
To: Jason Gunthorpe, Jerome Glisse, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
In-Reply-To: <20190606184438.31646-4-jgg-uk2M96/98Pc@public.gmane.org>
On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> So long a a struct hmm pointer exists, so should the struct mm it is
s/a a/as a/
> linked too. Hold the mmgrab() as soon as a hmm is created, and mmdrop() it
> once the hmm refcount goes to zero.
>
> Since mmdrop() (ie a 0 kref on struct mm) is now impossible with a !NULL
> mm->hmm delete the hmm_hmm_destroy().
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
> ---
> v2:
> - Fix error unwind paths in hmm_get_or_create (Jerome/Jason)
> ---
> include/linux/hmm.h | 3 ---
> kernel/fork.c | 1 -
> mm/hmm.c | 22 ++++------------------
> 3 files changed, 4 insertions(+), 22 deletions(-)
>
> diff --git a/include/linux/hmm.h b/include/linux/hmm.h
> index 2d519797cb134a..4ee3acabe5ed22 100644
> --- a/include/linux/hmm.h
> +++ b/include/linux/hmm.h
> @@ -586,14 +586,11 @@ static inline int hmm_vma_fault(struct hmm_mirror *mirror,
> }
>
> /* Below are for HMM internal use only! Not to be used by device driver! */
> -void hmm_mm_destroy(struct mm_struct *mm);
> -
> static inline void hmm_mm_init(struct mm_struct *mm)
> {
> mm->hmm = NULL;
> }
> #else /* IS_ENABLED(CONFIG_HMM_MIRROR) */
> -static inline void hmm_mm_destroy(struct mm_struct *mm) {}
> static inline void hmm_mm_init(struct mm_struct *mm) {}
> #endif /* IS_ENABLED(CONFIG_HMM_MIRROR) */
>
> diff --git a/kernel/fork.c b/kernel/fork.c
> index b2b87d450b80b5..588c768ae72451 100644
> --- a/kernel/fork.c
> +++ b/kernel/fork.c
> @@ -673,7 +673,6 @@ void __mmdrop(struct mm_struct *mm)
> WARN_ON_ONCE(mm == current->active_mm);
> mm_free_pgd(mm);
> destroy_context(mm);
> - hmm_mm_destroy(mm);
> mmu_notifier_mm_destroy(mm);
> check_mm(mm);
> put_user_ns(mm->user_ns);
> diff --git a/mm/hmm.c b/mm/hmm.c
> index 8796447299023c..cc7c26fda3300e 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -29,6 +29,7 @@
> #include <linux/swapops.h>
> #include <linux/hugetlb.h>
> #include <linux/memremap.h>
> +#include <linux/sched/mm.h>
> #include <linux/jump_label.h>
> #include <linux/dma-mapping.h>
> #include <linux/mmu_notifier.h>
> @@ -82,6 +83,7 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm)
> hmm->notifiers = 0;
> hmm->dead = false;
> hmm->mm = mm;
> + mmgrab(hmm->mm);
>
> spin_lock(&mm->page_table_lock);
> if (!mm->hmm)
> @@ -109,6 +111,7 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm)
> mm->hmm = NULL;
> spin_unlock(&mm->page_table_lock);
> error:
> + mmdrop(hmm->mm);
> kfree(hmm);
> return NULL;
> }
> @@ -130,6 +133,7 @@ static void hmm_free(struct kref *kref)
> mm->hmm = NULL;
> spin_unlock(&mm->page_table_lock);
>
> + mmdrop(hmm->mm);
> mmu_notifier_call_srcu(&hmm->rcu, hmm_free_rcu);
> }
>
> @@ -138,24 +142,6 @@ static inline void hmm_put(struct hmm *hmm)
> kref_put(&hmm->kref, hmm_free);
> }
>
> -void hmm_mm_destroy(struct mm_struct *mm)
> -{
> - struct hmm *hmm;
> -
> - spin_lock(&mm->page_table_lock);
> - hmm = mm_get_hmm(mm);
> - mm->hmm = NULL;
> - if (hmm) {
> - hmm->mm = NULL;
> - hmm->dead = true;
> - spin_unlock(&mm->page_table_lock);
> - hmm_put(hmm);
> - return;
> - }
> -
> - spin_unlock(&mm->page_table_lock);
> -}
> -
> static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm)
> {
> struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier);
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH RFC 00/10] RDMA/FS DAX truncate proposal
From: Ira Weiny @ 2019-06-07 18:25 UTC (permalink / raw)
To: Jan Kara
Cc: Jason Gunthorpe, Theodore Ts'o,
linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, Dave Chinner, Jeff Layton,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Matthew Wilcox,
linux-xfs-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
Jérôme Glisse, John Hubbard,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-ext4-u79uwXL29TY76Z2rM5mHXA, Andrew Morton
In-Reply-To: <20190607110426.GB12765-4I4JzKEfoa/jFM9bn6wA6Q@public.gmane.org>
On Fri, Jun 07, 2019 at 01:04:26PM +0200, Jan Kara wrote:
> On Thu 06-06-19 15:03:30, Ira Weiny wrote:
> > On Thu, Jun 06, 2019 at 12:42:03PM +0200, Jan Kara wrote:
> > > On Wed 05-06-19 18:45:33, ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org wrote:
> > > > From: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> > >
> > > So I'd like to actually mandate that you *must* hold the file lease until
> > > you unpin all pages in the given range (not just that you have an option to
> > > hold a lease). And I believe the kernel should actually enforce this. That
> > > way we maintain a sane state that if someone uses a physical location of
> > > logical file offset on disk, he has a layout lease. Also once this is done,
> > > sysadmin has a reasonably easy way to discover run-away RDMA application
> > > and kill it if he wishes so.
> >
> > Fair enough.
> >
> > I was kind of heading that direction but had not thought this far forward. I
> > was exploring how to have a lease remain on the file even after a "lease
> > break". But that is incompatible with the current semantics of a "layout"
> > lease (as currently defined in the kernel). [In the end I wanted to get an RFC
> > out to see what people think of this idea so I did not look at keeping the
> > lease.]
> >
> > Also hitch is that currently a lease is forcefully broken after
> > <sysfs>/lease-break-time. To do what you suggest I think we would need a new
> > lease type with the semantics you describe.
>
> I'd do what Dave suggested - add flag to mark lease as unbreakable by
> truncate and teach file locking core to handle that. There actually is
> support for locks that are not broken after given timeout so there
> shouldn't be too many changes need.
>
> > Previously I had thought this would be a good idea (for other reasons). But
> > what does everyone think about using a "longterm lease" similar to [1] which
> > has the semantics you proppose? In [1] I was not sure "longterm" was a good
> > name but with your proposal I think it makes more sense.
>
> As I wrote elsewhere in this thread I think FL_LAYOUT name still makes
> sense and I'd add there FL_UNBREAKABLE to mark unusal behavior with
> truncate.
Ok I want to make sure I understand what you and Dave are suggesting.
Are you suggesting that we have something like this from user space?
fcntl(fd, F_SETLEASE, F_LAYOUT | F_UNBREAKABLE);
>
> > > - probably I'd just transition all gup_longterm()
> > > users to a saner API similar to the one we have in mm/frame_vector.c where
> > > we don't hand out page pointers but an encapsulating structure that does
> > > all the necessary tracking.
> >
> > I'll take a look at that code. But that seems like a pretty big change.
>
> I was looking into that yesterday before proposing this and there aren't
> than many gup_longterm() users and most of them anyway just stick pages
> array into their tracking structure and then release them once done. So it
> shouldn't be that complex to convert to a new convention (and you have to
> touch all gup_longterm() users anyway to teach them track leases etc.).
I think in the direction we are heading this becomes more attractive for sure.
For me though it will take some time.
Should we convert the frame_vector over to this new mechanism? (Or more
accurately perhaps, add to frame_vector and use it?) It seems bad to have "yet
another object" returned from the pin pages interface...
And I think this is related to what Christoph Hellwig is doing with bio_vec and
dma. Really we want drivers out of the page processing business.
So for now I'm going to move forward with the idea of handing "some object" to
the GUP callers and figure out the lsof stuff, and let bigger questions like
this play out a bit more before I try and work with that code. Fair?
>
> > > Removing a lease would need to block until all
> > > pins are released - this is probably the most hairy part since we need to
> > > handle a case if application just closes the file descriptor which would
> > > release the lease but OTOH we need to make sure task exit does not deadlock.
> > > Maybe we could block only on explicit lease unlock and just drop the layout
> > > lease on file close and if there are still pinned pages, send SIGKILL to an
> > > application as a reminder it did something stupid...
> >
> > As presented at LSFmm I'm not opposed to killing a process which does not
> > "follow the rules". But I'm concerned about how to handle this across a fork.
> >
> > Limiting the open()/LEASE/GUP/close()/SIGKILL to a specific pid "leak"'s pins
> > to a child through the RDMA context. This was the major issue Jason had with
> > the SIGBUS proposal.
> >
> > Always sending a SIGKILL would prevent an RDMA process from doing something
> > like system("ls") (would kill the child unnecessarily). Are we ok with that?
>
> I answered this in another email but system("ls") won't kill anybody.
> fork(2) just creates new file descriptor for the same file and possibly
> then closes it but since there is still another file descriptor for the
> same struct file, the "close" code won't trigger.
Agreed. I was wrong. Sorry.
But if we can keep track of who has the pins in lsof can we agree no process
needs to be SIGKILL'ed? Admins can do this on their own "killing" if they
really need to stop the use of these files, right?
Ira
^ permalink raw reply
* Re: [PATCH v2 hmm 02/11] mm/hmm: Use hmm_mirror not mm as an argument for hmm_range_register
From: Ralph Campbell @ 2019-06-07 18:24 UTC (permalink / raw)
To: Jason Gunthorpe, Jerome Glisse, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
In-Reply-To: <20190606184438.31646-3-jgg-uk2M96/98Pc@public.gmane.org>
On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> Ralph observes that hmm_range_register() can only be called by a driver
> while a mirror is registered. Make this clear in the API by passing in the
> mirror structure as a parameter.
>
> This also simplifies understanding the lifetime model for struct hmm, as
> the hmm pointer must be valid as part of a registered mirror so all we
> need in hmm_register_range() is a simple kref_get.
>
> Suggested-by: Ralph Campbell <rcampbell@nvidia.com>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
You might CC Ben for the nouveau part.
CC: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
> ---
> v2
> - Include the oneline patch to nouveau_svm.c
> ---
> drivers/gpu/drm/nouveau/nouveau_svm.c | 2 +-
> include/linux/hmm.h | 7 ++++---
> mm/hmm.c | 15 ++++++---------
> 3 files changed, 11 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouveau/nouveau_svm.c
> index 93ed43c413f0bb..8c92374afcf227 100644
> --- a/drivers/gpu/drm/nouveau/nouveau_svm.c
> +++ b/drivers/gpu/drm/nouveau/nouveau_svm.c
> @@ -649,7 +649,7 @@ nouveau_svm_fault(struct nvif_notify *notify)
> range.values = nouveau_svm_pfn_values;
> range.pfn_shift = NVIF_VMM_PFNMAP_V0_ADDR_SHIFT;
> again:
> - ret = hmm_vma_fault(&range, true);
> + ret = hmm_vma_fault(&svmm->mirror, &range, true);
> if (ret == 0) {
> mutex_lock(&svmm->mutex);
> if (!hmm_vma_range_done(&range)) {
> diff --git a/include/linux/hmm.h b/include/linux/hmm.h
> index 688c5ca7068795..2d519797cb134a 100644
> --- a/include/linux/hmm.h
> +++ b/include/linux/hmm.h
> @@ -505,7 +505,7 @@ static inline bool hmm_mirror_mm_is_alive(struct hmm_mirror *mirror)
> * Please see Documentation/vm/hmm.rst for how to use the range API.
> */
> int hmm_range_register(struct hmm_range *range,
> - struct mm_struct *mm,
> + struct hmm_mirror *mirror,
> unsigned long start,
> unsigned long end,
> unsigned page_shift);
> @@ -541,7 +541,8 @@ static inline bool hmm_vma_range_done(struct hmm_range *range)
> }
>
> /* This is a temporary helper to avoid merge conflict between trees. */
> -static inline int hmm_vma_fault(struct hmm_range *range, bool block)
> +static inline int hmm_vma_fault(struct hmm_mirror *mirror,
> + struct hmm_range *range, bool block)
> {
> long ret;
>
> @@ -554,7 +555,7 @@ static inline int hmm_vma_fault(struct hmm_range *range, bool block)
> range->default_flags = 0;
> range->pfn_flags_mask = -1UL;
>
> - ret = hmm_range_register(range, range->vma->vm_mm,
> + ret = hmm_range_register(range, mirror,
> range->start, range->end,
> PAGE_SHIFT);
> if (ret)
> diff --git a/mm/hmm.c b/mm/hmm.c
> index 547002f56a163d..8796447299023c 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -925,13 +925,13 @@ static void hmm_pfns_clear(struct hmm_range *range,
> * Track updates to the CPU page table see include/linux/hmm.h
> */
> int hmm_range_register(struct hmm_range *range,
> - struct mm_struct *mm,
> + struct hmm_mirror *mirror,
> unsigned long start,
> unsigned long end,
> unsigned page_shift)
> {
> unsigned long mask = ((1UL << page_shift) - 1UL);
> - struct hmm *hmm;
> + struct hmm *hmm = mirror->hmm;
>
> range->valid = false;
> range->hmm = NULL;
> @@ -945,15 +945,12 @@ int hmm_range_register(struct hmm_range *range,
> range->start = start;
> range->end = end;
>
> - hmm = hmm_get_or_create(mm);
> - if (!hmm)
> - return -EFAULT;
> -
> /* Check if hmm_mm_destroy() was call. */
> - if (hmm->mm == NULL || hmm->dead) {
> - hmm_put(hmm);
> + if (hmm->mm == NULL || hmm->dead)
> return -EFAULT;
> - }
> +
> + range->hmm = hmm;
> + kref_get(&hmm->kref);
>
> /* Initialize range to track CPU page table updates. */
> mutex_lock(&hmm->lock);
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [pull request][for-next 0/9] Generic DIM lib for netdev and RDMA
From: Saeed Mahameed @ 2019-06-07 18:14 UTC (permalink / raw)
To: Jason Gunthorpe, Max Gurtovoy
Cc: linux-rdma@vger.kernel.org, davem@davemloft.net,
andy@greyhouse.net, Tal Gilboa, michael.chan@broadcom.com,
netdev@vger.kernel.org, leon@kernel.org, dledford@redhat.com
In-Reply-To: <20190606130713.GC17392@mellanox.com>
On Thu, 2019-06-06 at 13:07 +0000, Jason Gunthorpe wrote:
> On Thu, Jun 06, 2019 at 10:19:41AM +0300, Max Gurtovoy wrote:
> > > > Solution:
> > > > - Common logic is declared in include/linux/dim.h and
> > > > implemented in
> > > > lib/dim/dim.c
> > > > - Net DIM (existing) logic is declared in
> > > > include/linux/net_dim.h and
> > > > implemented in lib/dim/net_dim.c, which uses the common
> > > > logic from dim.h
> > > > - Any new DIM logic will be declared in
> > > > "/include/linux/new_dim.h" and
> > > > implemented in "lib/dim/new_dim.c".
> > > > - This new implementation will expose modified versions of
> > > > profiles,
> > > > dim_step() and dim_decision().
> > > >
> > > > Pros for this solution are:
> > > > - Zero impact on existing net_dim implementation and usage
> > > > - Relatively more code reuse (compared to two separate
> > > > solutions)
> > > > - Increased extensibility
> > > >
> > > > Tal Gilboa (6):
> > > > linux/dim: Move logic to dim.h
> > > > linux/dim: Remove "net" prefix from internal DIM members
> > > > linux/dim: Rename externally exposed macros
> > > > linux/dim: Rename net_dim_sample() to
> > > > net_dim_update_sample()
> > > > linux/dim: Rename externally used net_dim members
> > > > linux/dim: Move implementation to .c files
> > > >
> > > > Yamin Friedman (3):
> > > > linux/dim: Add completions count to dim_sample
> > > > linux/dim: Implement rdma_dim
> > > > RDMA/core: Provide RDMA DIM support for ULPs
> > > Saeed,
> > >
> > > No, for the RDMA patches.
> > > We need to see usage of those APIs before merging.
> >
> > I've asked Yamin to prepare patches for NVMeoF initiator and target
> > for
> > review, so I guess he has it on his plate (this is how he tested
> > it..).
> >
> > It might cause conflict with NVMe/blk branch maintained by Sagi,
> > Christoph
> > and Jens.
>
> It looks like nvme could pull this series + the RDMA patches into the
> nvme tree via PR? I'm not familiar with how that tree works.
>
> But we need to get the patches posted right away..
>
What do you suggest here ?
I think the netdev community also deserve to see the rdma patches, at
least with an external link, I can drop the last patch (or two ) ? but
i need an external rdma link for people who are going to review this
series.
> Jason
^ permalink raw reply
* Re: [PATCH v2 hmm 01/11] mm/hmm: fix use after free with struct hmm in the mmu notifiers
From: Ralph Campbell @ 2019-06-07 18:12 UTC (permalink / raw)
To: Jason Gunthorpe, Jerome Glisse, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
In-Reply-To: <20190606184438.31646-2-jgg-uk2M96/98Pc@public.gmane.org>
On 6/6/19 11:44 AM, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
>
> mmu_notifier_unregister_no_release() is not a fence and the mmu_notifier
> system will continue to reference hmm->mn until the srcu grace period
> expires.
>
> Resulting in use after free races like this:
>
> CPU0 CPU1
> __mmu_notifier_invalidate_range_start()
> srcu_read_lock
> hlist_for_each ()
> // mn == hmm->mn
> hmm_mirror_unregister()
> hmm_put()
> hmm_free()
> mmu_notifier_unregister_no_release()
> hlist_del_init_rcu(hmm-mn->list)
> mn->ops->invalidate_range_start(mn, range);
> mm_get_hmm()
> mm->hmm = NULL;
> kfree(hmm)
> mutex_lock(&hmm->lock);
>
> Use SRCU to kfree the hmm memory so that the notifiers can rely on hmm
> existing. Get the now-safe hmm struct through container_of and directly
> check kref_get_unless_zero to lock it against free.
>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
You can add
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
> ---
> v2:
> - Spell 'free' properly (Jerome/Ralph)
> ---
> include/linux/hmm.h | 1 +
> mm/hmm.c | 25 +++++++++++++++++++------
> 2 files changed, 20 insertions(+), 6 deletions(-)
>
> diff --git a/include/linux/hmm.h b/include/linux/hmm.h
> index 092f0234bfe917..688c5ca7068795 100644
> --- a/include/linux/hmm.h
> +++ b/include/linux/hmm.h
> @@ -102,6 +102,7 @@ struct hmm {
> struct mmu_notifier mmu_notifier;
> struct rw_semaphore mirrors_sem;
> wait_queue_head_t wq;
> + struct rcu_head rcu;
> long notifiers;
> bool dead;
> };
> diff --git a/mm/hmm.c b/mm/hmm.c
> index 8e7403f081f44a..547002f56a163d 100644
> --- a/mm/hmm.c
> +++ b/mm/hmm.c
> @@ -113,6 +113,11 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm)
> return NULL;
> }
>
> +static void hmm_free_rcu(struct rcu_head *rcu)
> +{
> + kfree(container_of(rcu, struct hmm, rcu));
> +}
> +
> static void hmm_free(struct kref *kref)
> {
> struct hmm *hmm = container_of(kref, struct hmm, kref);
> @@ -125,7 +130,7 @@ static void hmm_free(struct kref *kref)
> mm->hmm = NULL;
> spin_unlock(&mm->page_table_lock);
>
> - kfree(hmm);
> + mmu_notifier_call_srcu(&hmm->rcu, hmm_free_rcu);
> }
>
> static inline void hmm_put(struct hmm *hmm)
> @@ -153,10 +158,14 @@ void hmm_mm_destroy(struct mm_struct *mm)
>
> static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm)
> {
> - struct hmm *hmm = mm_get_hmm(mm);
> + struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier);
> struct hmm_mirror *mirror;
> struct hmm_range *range;
>
> + /* hmm is in progress to free */
> + if (!kref_get_unless_zero(&hmm->kref))
> + return;
> +
> /* Report this HMM as dying. */
> hmm->dead = true;
>
> @@ -194,13 +203,15 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm)
> static int hmm_invalidate_range_start(struct mmu_notifier *mn,
> const struct mmu_notifier_range *nrange)
> {
> - struct hmm *hmm = mm_get_hmm(nrange->mm);
> + struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier);
> struct hmm_mirror *mirror;
> struct hmm_update update;
> struct hmm_range *range;
> int ret = 0;
>
> - VM_BUG_ON(!hmm);
> + /* hmm is in progress to free */
> + if (!kref_get_unless_zero(&hmm->kref))
> + return 0;
>
> update.start = nrange->start;
> update.end = nrange->end;
> @@ -245,9 +256,11 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn,
> static void hmm_invalidate_range_end(struct mmu_notifier *mn,
> const struct mmu_notifier_range *nrange)
> {
> - struct hmm *hmm = mm_get_hmm(nrange->mm);
> + struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier);
>
> - VM_BUG_ON(!hmm);
> + /* hmm is in progress to free */
> + if (!kref_get_unless_zero(&hmm->kref))
> + return;
>
> mutex_lock(&hmm->lock);
> hmm->notifiers--;
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH] RDMA/ucma: Use struct_size() helper
From: Jason Gunthorpe @ 2019-06-07 18:05 UTC (permalink / raw)
To: Gustavo A. R. Silva; +Cc: Doug Ledford, linux-rdma, linux-kernel
In-Reply-To: <20190604154222.GA8938@embeddedor>
On Tue, Jun 04, 2019 at 10:42:22AM -0500, Gustavo A. R. Silva wrote:
> Make use of the struct_size() helper instead of an open-coded version
> in order to avoid any potential type mistakes, in particular in the
> context in which this code is being used.
>
> So, replace the following form:
>
> sizeof(*resp) + (i * sizeof(struct ib_path_rec_data))
>
> with:
>
> struct_size(resp, path_data, i)
>
> This code was detected with the help of Coccinelle.
>
> Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com>
> ---
> drivers/infiniband/core/ucma.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
Applied to for-next, thanks
Jason
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox