* Re: [PATCH v4 hmm 11/12] mm/hmm: Remove confusing comment and logic from hmm_release
From: Christoph Hellwig @ 2019-06-25 7:03 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Andrea Arcangeli, Philip Yang, Ralph Campbell,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jerome Glisse, Jason Gunthorpe,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-12-jgg-uk2M96/98Pc@public.gmane.org>
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH v4 hmm 04/12] mm/hmm: Simplify hmm_get_or_create and make it reliable
From: Christoph Hellwig @ 2019-06-25 7:02 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Andrea Arcangeli, Philip Yang, Ralph Campbell,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jerome Glisse, Jason Gunthorpe,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-5-jgg-uk2M96/98Pc@public.gmane.org>
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH v4 0/3] [v4.9.y] coredump: fix race condition between mmget_not_zero()/get_task_mm() and core dumping
From: Ajay Kaher @ 2019-06-25 6:10 UTC (permalink / raw)
To: Sasha Levin
Cc: aarcange@redhat.com, jannh@google.com, oleg@redhat.com,
peterx@redhat.com, rppt@linux.ibm.com, jgg@mellanox.com,
mhocko@suse.com, jglisse@redhat.com, akpm@linux-foundation.org,
mike.kravetz@oracle.com, viro@zeniv.linux.org.uk,
riandrews@android.com, arve@android.com, yishaih@mellanox.com,
dledford@redhat.com, sean.hefty@intel.com,
hal.rosenstock@gmail.com
In-Reply-To: <20190624202150.GC3881@sasha-vm>
On 25/06/19, 1:51 AM, "Sasha Levin" <sashal@kernel.org> wrote:
> On Tue, Jun 25, 2019 at 02:33:06AM +0530, Ajay Kaher wrote:
> > coredump: fix race condition between mmget_not_zero()/get_task_mm()
> > and core dumping
> >
> > [PATCH v4 1/3]:
> > Backporting of commit 04f5866e41fb70690e28397487d8bd8eea7d712a upstream.
> >
> > [PATCH v4 2/3]:
> > Extension of commit 04f5866e41fb to fix the race condition between
> > get_task_mm() and core dumping for IB->mlx4 and IB->mlx5 drivers.
> >
> > [PATCH v4 3/3]
> > Backporting of commit 59ea6d06cfa9247b586a695c21f94afa7183af74 upstream.
> >
> > [diff from v3]:
> > - added [PATCH v4 3/3]
> Why do all the patches have the same subject line?
Thanks for catching this. I will correct in next version of these patches,
along with review comments if any.
> I guess it's correct for the first one, but can you explain what's up
> with #2 and #3?
>
> If the second one isn't upstream, please explain in detail why not and
> how 4.9 differs from upstream so that it requires a custom backport.
#2 applied to 4.14.y:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git/tree/queue-4.14/infiniband-fix-race-condition-between-infiniband-mlx4-mlx5-driver-and-core-dumping.patch?id=e4041a3f6b569140549fe7b41ed527c5c1e38ec9
And then to 4.9.y (some part as requires).
4.18 and onwards doesn't have mmap_sem locking in mlx4 and mlx5,
so no need of #2 in 4.18 and onwards.
> The third one just looks like a different patch altogether with a wrong
> subject line?
#3 was in discussion here (during v1), so added here.
> --
> Thanks,
> Sasha
^ permalink raw reply
* Reminder: 11 open syzbot bugs in RDMA subsystem
From: Eric Biggers @ 2019-06-25 5:48 UTC (permalink / raw)
To: linux-rdma, Doug Ledford, Jason Gunthorpe, Yishai Hadas
Cc: linux-kernel, syzkaller-bugs
[This email was generated by a script. Let me know if you have any suggestions
to make it better.]
Of the currently open syzbot reports against the upstream kernel, I've manually
marked 11 of them as possibly being bugs in the RDMA subsystem. I've listed
these reports below, sorted by an algorithm that tries to list first the reports
most likely to be still valid, important, and actionable.
Of these 11 bugs, 1 was seen in mainline in the last week.
Of these 11 bugs, 1 was bisected to a commit from the following person:
Yishai Hadas <yishaih@mellanox.com>
If you believe a bug is no longer valid, please close the syzbot report by
sending a '#syz fix', '#syz dup', or '#syz invalid' command in reply to the
original thread, as explained at https://goo.gl/tpsmEJ#status
If you believe I misattributed a bug to the RDMA subsystem, please let me know,
and if possible forward the report to the correct people or mailing list.
Here are the bugs:
--------------------------------------------------------------------------------
Title: KASAN: use-after-free Read in rdma_listen (2)
Last occurred: 0 days ago
Reported: 86 days ago
Branches: Mainline and others
Dashboard link: https://syzkaller.appspot.com/bug?id=8dc0bcd9dd6ec915ba10b3354740eb420884acaa
Original thread: https://lkml.kernel.org/lkml/00000000000012a4cd05854a1d0a@google.com/T/#u
This bug has a syzkaller reproducer only.
No one has replied to the original thread for this bug yet.
If you fix this bug, please add the following tag to the commit:
Reported-by: syzbot+adb15cf8c2798e4e0db4@syzkaller.appspotmail.com
If you send any email or patch for this bug, please consider replying to the
original thread. For the git send-email command to use, or tips on how to reply
if the thread isn't in your mailbox, see the "Reply instructions" at
https://lkml.kernel.org/r/00000000000012a4cd05854a1d0a@google.com
--------------------------------------------------------------------------------
Title: WARNING: bad unlock balance in ucma_event_handler
Last occurred: 14 days ago
Reported: 376 days ago
Branches: Mainline and others
Dashboard link: https://syzkaller.appspot.com/bug?id=d5222b3e1659e0aea19df562c79f216515740daa
Original thread: https://lkml.kernel.org/lkml/000000000000af6530056e863794@google.com/T/#u
This bug has a C reproducer.
The original thread for this bug received 6 replies; the last was 13 days ago.
If you fix this bug, please add the following tag to the commit:
Reported-by: syzbot+e5579222b6a3edd96522@syzkaller.appspotmail.com
If you send any email or patch for this bug, please reply to the original
thread, which had activity only 13 days ago. For the git send-email command to
use, or tips on how to reply if the thread isn't in your mailbox, see the "Reply
instructions" at https://lkml.kernel.org/r/000000000000af6530056e863794@google.com
--------------------------------------------------------------------------------
Title: WARNING: bad unlock balance in ucma_destroy_id
Last occurred: 93 days ago
Reported: 300 days ago
Branches: Mainline and others
Dashboard link: https://syzkaller.appspot.com/bug?id=c600e111223ce0a20e5f2fb4e9a4ebdff54d7fa6
Original thread: https://lkml.kernel.org/lkml/0000000000003b9c4b0574806070@google.com/T/#u
This bug has a syzkaller reproducer only.
This bug was bisected to:
commit e1c30298ccab87151a0c4241fc5985c591598361
Author: Yishai Hadas <yishaih@mellanox.com>
Date: Thu Aug 13 15:32:07 2015 +0000
IB/ucma: HW Device hot-removal support
No one replied to the original thread for this bug.
If you fix this bug, please add the following tag to the commit:
Reported-by: syzbot+4b628fcc748474003457@syzkaller.appspotmail.com
If you send any email or patch for this bug, please consider replying to the
original thread. For the git send-email command to use, or tips on how to reply
if the thread isn't in your mailbox, see the "Reply instructions" at
https://lkml.kernel.org/r/0000000000003b9c4b0574806070@google.com
--------------------------------------------------------------------------------
Title: KASAN: use-after-free Read in __list_del_entry_valid (4)
Last occurred: 449 days ago
Reported: 456 days ago
Branches: Mainline
Dashboard link: https://syzkaller.appspot.com/bug?id=56b60fb3340c5995373fe5b8eae9e8722a012fc4
Original thread: https://lkml.kernel.org/lkml/001a1141551246502d056845782e@google.com/T/#u
This bug has a C reproducer.
The original thread for this bug received 6 replies; the last was 305 days ago.
If you fix this bug, please add the following tag to the commit:
Reported-by: syzbot+29ee8f76017ce6cf03da@syzkaller.appspotmail.com
If you send any email or patch for this bug, please consider replying to the
original thread. For the git send-email command to use, or tips on how to reply
if the thread isn't in your mailbox, see the "Reply instructions" at
https://lkml.kernel.org/r/001a1141551246502d056845782e@google.com
--------------------------------------------------------------------------------
Title: KASAN: use-after-free Read in cma_cancel_operation
Last occurred: 107 days ago
Reported: 448 days ago
Branches: Mainline and others
Dashboard link: https://syzkaller.appspot.com/bug?id=95f89b8fb9fdc42e28ad586e657fea074e4e719b
Original thread: https://lkml.kernel.org/lkml/94eb2c054604ad40010568e8ea21@google.com/T/#u
This bug has a C reproducer.
The original thread for this bug received 4 replies; the last was 124 days ago.
If you fix this bug, please add the following tag to the commit:
Reported-by: syzbot+6956235342b7317ec564@syzkaller.appspotmail.com
If you send any email or patch for this bug, please consider replying to the
original thread. For the git send-email command to use, or tips on how to reply
if the thread isn't in your mailbox, see the "Reply instructions" at
https://lkml.kernel.org/r/94eb2c054604ad40010568e8ea21@google.com
--------------------------------------------------------------------------------
Title: KASAN: use-after-free Read in addr_handler
Last occurred: 125 days ago
Reported: 193 days ago
Branches: Mainline and others
Dashboard link: https://syzkaller.appspot.com/bug?id=a9796acbdecc1b2ba927578917755899c63c48af
Original thread: https://lkml.kernel.org/lkml/00000000000055ee31057ce8f277@google.com/T/#u
This bug has a syzkaller reproducer only.
syzbot has bisected this bug, but I think the bisection result is incorrect.
The original thread for this bug received 2 replies; the last was 90 days ago.
If you fix this bug, please add the following tag to the commit:
Reported-by: syzbot+b358909d8d01556b790b@syzkaller.appspotmail.com
If you send any email or patch for this bug, please consider replying to the
original thread. For the git send-email command to use, or tips on how to reply
if the thread isn't in your mailbox, see the "Reply instructions" at
https://lkml.kernel.org/r/00000000000055ee31057ce8f277@google.com
--------------------------------------------------------------------------------
Title: general protection fault in rdma_listen (2)
Last occurred: 81 days ago
Reported: 229 days ago
Branches: Mainline and others
Dashboard link: https://syzkaller.appspot.com/bug?id=38d36d1b26b4299bf964d50af4d79688d39ab960
Original thread: https://lkml.kernel.org/lkml/000000000000396c09057a17b6fd@google.com/T/#u
This bug has a syzkaller reproducer only.
The original thread for this bug received 2 replies; the last was 76 days ago.
If you fix this bug, please add the following tag to the commit:
Reported-by: syzbot+6b46b135602a3f3ac99e@syzkaller.appspotmail.com
If you send any email or patch for this bug, please consider replying to the
original thread. For the git send-email command to use, or tips on how to reply
if the thread isn't in your mailbox, see the "Reply instructions" at
https://lkml.kernel.org/r/000000000000396c09057a17b6fd@google.com
--------------------------------------------------------------------------------
Title: BUG: corrupted list in rdma_listen
Last occurred: 105 days ago
Reported: 431 days ago
Branches: Mainline and others
Dashboard link: https://syzkaller.appspot.com/bug?id=25e00dd59f31783f233185cb60064b0ab645310f
Original thread: https://lkml.kernel.org/lkml/000000000000a366e2056a35c6fd@google.com/T/#u
This bug has a C reproducer.
No one replied to the original thread for this bug.
If you fix this bug, please add the following tag to the commit:
Reported-by: syzbot+8458d13b13562abf6b77@syzkaller.appspotmail.com
If you send any email or patch for this bug, please consider replying to the
original thread. For the git send-email command to use, or tips on how to reply
if the thread isn't in your mailbox, see the "Reply instructions" at
https://lkml.kernel.org/r/000000000000a366e2056a35c6fd@google.com
--------------------------------------------------------------------------------
Title: KASAN: use-after-free Read in addr_resolve
Last occurred: 95 days ago
Reported: 94 days ago
Branches: Mainline
Dashboard link: https://syzkaller.appspot.com/bug?id=07328fd3299fadf7005c46651d2ff50c1cd4e1dd
Original thread: https://lkml.kernel.org/lkml/0000000000006d637a0584aa6520@google.com/T/#u
Unfortunately, this bug does not have a reproducer.
No one replied to the original thread for this bug.
If you fix this bug, please add the following tag to the commit:
Reported-by: syzbot+bd034f3fdc0402e942ed@syzkaller.appspotmail.com
If you send any email or patch for this bug, please consider replying to the
original thread. For the git send-email command to use, or tips on how to reply
if the thread isn't in your mailbox, see the "Reply instructions" at
https://lkml.kernel.org/r/0000000000006d637a0584aa6520@google.com
--------------------------------------------------------------------------------
Title: KASAN: slab-out-of-bounds Read in rdma_listen
Last occurred: 118 days ago
Reported: 308 days ago
Branches: Mainline and others
Dashboard link: https://syzkaller.appspot.com/bug?id=fc5df2d4d88353572496fcf9caf8a9c7bdc034c3
Original thread: https://lkml.kernel.org/lkml/0000000000001de4b70573d62017@google.com/T/#u
Unfortunately, this bug does not have a reproducer.
No one replied to the original thread for this bug.
If you fix this bug, please add the following tag to the commit:
Reported-by: syzbot+c92378b32760a4eef756@syzkaller.appspotmail.com
If you send any email or patch for this bug, please consider replying to the
original thread. For the git send-email command to use, or tips on how to reply
if the thread isn't in your mailbox, see the "Reply instructions" at
https://lkml.kernel.org/r/0000000000001de4b70573d62017@google.com
--------------------------------------------------------------------------------
Title: KASAN: use-after-free Read in rdma_bind_addr
Last occurred: 82 days ago
Reported: 82 days ago
Branches: linux-next
Dashboard link: https://syzkaller.appspot.com/bug?id=ecb19d20c6748a78058dac77ad17468c4e6733c4
Original thread: https://lkml.kernel.org/lkml/000000000000ebb6bc05859ac2cf@google.com/T/#u
Unfortunately, this bug does not have a reproducer.
No one has replied to the original thread for this bug yet.
If you fix this bug, please add the following tag to the commit:
Reported-by: syzbot+68b44a1597636e0b342c@syzkaller.appspotmail.com
If you send any email or patch for this bug, please consider replying to the
original thread. For the git send-email command to use, or tips on how to reply
if the thread isn't in your mailbox, see the "Reply instructions" at
https://lkml.kernel.org/r/000000000000ebb6bc05859ac2cf@google.com
^ permalink raw reply
* Re: [PATCH 6/8] IB/srp: set virt_boundary_mask in the scsi host
From: Max Gurtovoy @ 2019-06-24 22:33 UTC (permalink / raw)
To: Christoph Hellwig, Martin K . Petersen
Cc: Sagi Grimberg, Bart Van Assche, linux-rdma, linux-scsi,
megaraidlinux.pdl, MPT-FusionLinux.pdl, linux-hyperv,
linux-kernel
In-Reply-To: <20190617122000.22181-7-hch@lst.de>
On 6/17/2019 3:19 PM, Christoph Hellwig wrote:
> This ensures all proper DMA layer handling is taken care of by the
> SCSI midlayer.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
Looks good,
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
^ permalink raw reply
* Re: [PATCH 5/8] IB/iser: set virt_boundary_mask in the scsi host
From: Max Gurtovoy @ 2019-06-24 22:32 UTC (permalink / raw)
To: Christoph Hellwig, Martin K . Petersen
Cc: Sagi Grimberg, Bart Van Assche, linux-rdma, linux-scsi,
megaraidlinux.pdl, MPT-FusionLinux.pdl, linux-hyperv,
linux-kernel
In-Reply-To: <20190617122000.22181-6-hch@lst.de>
On 6/17/2019 3:19 PM, Christoph Hellwig wrote:
> This ensures all proper DMA layer handling is taken care of by the
> SCSI midlayer.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
Looks good,
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
^ permalink raw reply
* Re: [PATCH mlx5-next] net/mlx5: Convert mkey_table to XArray
From: Saeed Mahameed @ 2019-06-24 22:05 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Jason Gunthorpe, willy@infradead.org, netdev@vger.kernel.org,
linux-rdma@vger.kernel.org
In-Reply-To: <20190620070305.31632-1-saeedm@mellanox.com>
On Thu, 2019-06-20 at 07:03 +0000, Saeed Mahameed wrote:
> From: Matthew Wilcox <willy@infradead.org>
>
> The lock protecting the data structure does not need to be an
> rwlock. The
> only read access to the lock is in an error path, and if that's
> limiting
> your scalability, you have bigger performance problems.
>
> Eliminate mlx5_mkey_table in favour of using the xarray directly.
> reg_mr_callback must use GFP_ATOMIC for allocating XArray nodes as it
> may
> be called in interrupt context.
>
> This also fixes a minor bug where SRCU locking was being used on the
> radix
> tree read side, when RCU was needed too.
>
> Change-Id: Ifeedbbc8f2c856cc044094d03167aae9f9162400
> Signed-off-by: Matthew Wilcox <willy@infradead.org>
> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
> Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
>
Applied to mlx5-next.
Thanks,
Saeed.
^ permalink raw reply
* Re: [PATCH rdma-next v1 00/12] DEVX asynchronous events
From: Saeed Mahameed @ 2019-06-24 21:57 UTC (permalink / raw)
To: leon@kernel.org
Cc: Jason Gunthorpe, Yishai Hadas, netdev@vger.kernel.org,
linux-rdma@vger.kernel.org, dledford@redhat.com
In-Reply-To: <20190619044557.GA11611@mtr-leonro.mtl.com>
On Wed, 2019-06-19 at 07:45 +0300, Leon Romanovsky wrote:
> On Tue, Jun 18, 2019 at 06:51:45PM +0000, Saeed Mahameed wrote:
> > On Tue, 2019-06-18 at 20:15 +0300, Leon Romanovsky wrote:
> > > From: Leon Romanovsky <leonro@mellanox.com>
> > >
> > > Changelog:
> > > v0 -> v1:
> >
> > Normally 1st submission is V1 and 2nd is V2.
> > so this should have been v1->v2.
>
> "Normally" depends on the language you are using. In C, everything
> starts from 0, including version of patches :).
>
You are wrong:
quoting: https://kernelnewbies.org/PatchTipsAndTricks
"For example, if you're sending the second revision of a patch, you
should use [PATCH v2]."
now don't tell me that second revision is actually 3rd revision or 1st
is 2nd :)..
> > For mlx5-next patches:
> >
> > Acked-by: Saeed Mahameed <saeedm@mellanox.com>
>
> Thanks
^ permalink raw reply
* [PATCH][next] RDMA/hns: fix potential integer overflow on left shift
From: Colin King @ 2019-06-24 21:46 UTC (permalink / raw)
To: Lijun Ou, Wei Hu, Doug Ledford, Jason Gunthorpe, linux-rdma
Cc: kernel-janitors, linux-kernel
From: Colin Ian King <colin.king@canonical.com>
There is a potential integer overflow when int i is left shifted
as this is evaluated using 32 bit arithmetic but is being used in
a context that expects an expression of type dma_addr_t. Fix this
by casting integer i to dma_addr_t before shifting to avoid the
overflow.
Addresses-Coverity: ("Unintentional integer overflow")
Fixes: 2ac0bc5e725e ("RDMA/hns: Add a group interfaces for optimizing buffers getting flow")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
---
drivers/infiniband/hw/hns/hns_roce_alloc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_alloc.c b/drivers/infiniband/hw/hns/hns_roce_alloc.c
index 14fcc359599c..2c8defa94107 100644
--- a/drivers/infiniband/hw/hns/hns_roce_alloc.c
+++ b/drivers/infiniband/hw/hns/hns_roce_alloc.c
@@ -257,7 +257,7 @@ int hns_roce_get_kmem_bufs(struct hns_roce_dev *hr_dev, dma_addr_t *bufs,
for (i = start; i < end; i++)
if (buf->nbufs == 1)
bufs[total++] = buf->direct.map +
- (i << buf->page_shift);
+ ((dma_addr_t)i << buf->page_shift);
else
bufs[total++] = buf->page_list[i].map;
--
2.20.1
^ permalink raw reply related
* [PATCH v4 0/3] [v4.9.y] coredump: fix race condition between mmget_not_zero()/get_task_mm() and core dumping
From: Ajay Kaher @ 2019-06-24 21:03 UTC (permalink / raw)
To: aarcange, jannh, oleg, peterx, rppt, jgg, mhocko
Cc: jglisse, akpm, mike.kravetz, viro, riandrews, arve, yishaih,
dledford, sean.hefty, hal.rosenstock, matanb, leonro,
linux-fsdevel, linux-mm, devel, linux-rdma, linux-kernel, stable,
akaher, srivatsab, amakhalov
In-Reply-To: <1561410186-3919-1-git-send-email-akaher@vmware.com>
coredump: fix race condition between mmget_not_zero()/get_task_mm()
and core dumping
[PATCH v4 1/3]:
Backporting of commit 04f5866e41fb70690e28397487d8bd8eea7d712a upstream.
[PATCH v4 2/3]:
Extension of commit 04f5866e41fb to fix the race condition between
get_task_mm() and core dumping for IB->mlx4 and IB->mlx5 drivers.
[PATCH v4 3/3]
Backporting of commit 59ea6d06cfa9247b586a695c21f94afa7183af74 upstream.
[diff from v3]:
- added [PATCH v4 3/3]
^ permalink raw reply
* [PATCH v4 3/3] [v4.9.y] coredump: fix race condition between mmget_not_zero()/get_task_mm() and core dumping
From: Ajay Kaher @ 2019-06-24 21:03 UTC (permalink / raw)
To: aarcange, jannh, oleg, peterx, rppt, jgg, mhocko
Cc: jglisse, akpm, mike.kravetz, viro, riandrews, arve, yishaih,
dledford, sean.hefty, hal.rosenstock, matanb, leonro,
linux-fsdevel, linux-mm, devel, linux-rdma, linux-kernel, stable,
akaher, srivatsab, amakhalov, Hugh Dickins, Mike Rapoport,
Linus Torvalds, Greg Kroah-Hartman
In-Reply-To: <1561410186-3919-1-git-send-email-akaher@vmware.com>
From: Andrea Arcangeli <aarcange@redhat.com>
coredump: fix race condition between collapse_huge_page() and core dumping
commit 59ea6d06cfa9247b586a695c21f94afa7183af74 upstream.
When fixing the race conditions between the coredump and the mmap_sem
holders outside the context of the process, we focused on
mmget_not_zero()/get_task_mm() callers in 04f5866e41fb70 ("coredump: fix
race condition between mmget_not_zero()/get_task_mm() and core
dumping"), but those aren't the only cases where the mmap_sem can be
taken outside of the context of the process as Michal Hocko noticed
while backporting that commit to older -stable kernels.
If mmgrab() is called in the context of the process, but then the
mm_count reference is transferred outside the context of the process,
that can also be a problem if the mmap_sem has to be taken for writing
through that mm_count reference.
khugepaged registration calls mmgrab() in the context of the process,
but the mmap_sem for writing is taken later in the context of the
khugepaged kernel thread.
collapse_huge_page() after taking the mmap_sem for writing doesn't
modify any vma, so it's not obvious that it could cause a problem to the
coredump, but it happens to modify the pmd in a way that breaks an
invariant that pmd_trans_huge_lock() relies upon. collapse_huge_page()
needs the mmap_sem for writing just to block concurrent page faults that
call pmd_trans_huge_lock().
Specifically the invariant that "!pmd_trans_huge()" cannot become a
"pmd_trans_huge()" doesn't hold while collapse_huge_page() runs.
The coredump will call __get_user_pages() without mmap_sem for reading,
which eventually can invoke a lockless page fault which will need a
functional pmd_trans_huge_lock().
So collapse_huge_page() needs to use mmget_still_valid() to check it's
not running concurrently with the coredump... as long as the coredump
can invoke page faults without holding the mmap_sem for reading.
This has "Fixes: khugepaged" to facilitate backporting, but in my view
it's more a bug in the coredump code that will eventually have to be
rewritten to stop invoking page faults without the mmap_sem for reading.
So the long term plan is still to drop all mmget_still_valid().
Link: http://lkml.kernel.org/r/20190607161558.32104-1-aarcange@redhat.com
Fixes: ba76149f47d8 ("thp: khugepaged")
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
Reported-by: Michal Hocko <mhocko@suse.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Jann Horn <jannh@google.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Jason Gunthorpe <jgg@mellanox.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
[Ajay: Just adjusted to apply on v4.9]
Signed-off-by: Ajay Kaher <akaher@vmware.com>
---
include/linux/mm.h | 4 ++++
mm/khugepaged.c | 3 +++
2 files changed, 7 insertions(+)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index c239984..8852158 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1194,6 +1194,10 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma,
* followed by taking the mmap_sem for writing before modifying the
* vmas or anything the coredump pretends not to change from under it.
*
+ * It also has to be called when mmgrab() is used in the context of
+ * the process, but then the mm_count refcount is transferred outside
+ * the context of the process to run down_write() on that pinned mm.
+ *
* NOTE: find_extend_vma() called from GUP context is the only place
* that can modify the "mm" (notably the vm_start/end) under mmap_sem
* for reading and outside the context of the process, so it is also
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index e0cfc3a..8217ee5 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1004,6 +1004,9 @@ static void collapse_huge_page(struct mm_struct *mm,
* handled by the anon_vma lock + PG_lock.
*/
down_write(&mm->mmap_sem);
+ result = SCAN_ANY_PROCESS;
+ if (!mmget_still_valid(mm))
+ goto out;
result = hugepage_vma_revalidate(mm, address, &vma);
if (result)
goto out;
--
2.7.4
^ permalink raw reply related
* [PATCH v4 2/3][v4.9.y] coredump: fix race condition between mmget_not_zero()/get_task_mm() and core dumping
From: Ajay Kaher @ 2019-06-24 21:03 UTC (permalink / raw)
To: aarcange, jannh, oleg, peterx, rppt, jgg, mhocko
Cc: jglisse, akpm, mike.kravetz, viro, riandrews, arve, yishaih,
dledford, sean.hefty, hal.rosenstock, matanb, leonro,
linux-fsdevel, linux-mm, devel, linux-rdma, linux-kernel, stable,
akaher, srivatsab, amakhalov
In-Reply-To: <1561410186-3919-1-git-send-email-akaher@vmware.com>
This patch is the extension of following upstream commit to fix
the race condition between get_task_mm() and core dumping
for IB->mlx4 and IB->mlx5 drivers:
commit 04f5866e41fb ("coredump: fix race condition between
mmget_not_zero()/get_task_mm() and core dumping")'
Thanks to Jason for pointing this.
Signed-off-by: Ajay Kaher <akaher@vmware.com>
---
drivers/infiniband/hw/mlx4/main.c | 4 +++-
drivers/infiniband/hw/mlx5/main.c | 3 +++
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c
index 8d59a59..7ccf722 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -1172,6 +1172,8 @@ static void mlx4_ib_disassociate_ucontext(struct ib_ucontext *ibcontext)
* mlx4_ib_vma_close().
*/
down_write(&owning_mm->mmap_sem);
+ if (!mmget_still_valid(owning_mm))
+ goto skip_mm;
for (i = 0; i < HW_BAR_COUNT; i++) {
vma = context->hw_bar_info[i].vma;
if (!vma)
@@ -1190,7 +1192,7 @@ static void mlx4_ib_disassociate_ucontext(struct ib_ucontext *ibcontext)
/* context going to be destroyed, should not access ops any more */
context->hw_bar_info[i].vma->vm_ops = NULL;
}
-
+skip_mm:
up_write(&owning_mm->mmap_sem);
mmput(owning_mm);
put_task_struct(owning_process);
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index b1daf5c..f94df0e 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -1307,6 +1307,8 @@ static void mlx5_ib_disassociate_ucontext(struct ib_ucontext *ibcontext)
* mlx5_ib_vma_close.
*/
down_write(&owning_mm->mmap_sem);
+ if (!mmget_still_valid(owning_mm))
+ goto skip_mm;
list_for_each_entry_safe(vma_private, n, &context->vma_private_list,
list) {
vma = vma_private->vma;
@@ -1321,6 +1323,7 @@ static void mlx5_ib_disassociate_ucontext(struct ib_ucontext *ibcontext)
list_del(&vma_private->list);
kfree(vma_private);
}
+skip_mm:
up_write(&owning_mm->mmap_sem);
mmput(owning_mm);
put_task_struct(owning_process);
--
2.7.4
^ permalink raw reply related
* [PATCH v4 1/3] [v4.9.y] coredump: fix race condition between mmget_not_zero()/get_task_mm() and core dumping
From: Ajay Kaher @ 2019-06-24 21:03 UTC (permalink / raw)
To: aarcange, jannh, oleg, peterx, rppt, jgg, mhocko
Cc: jglisse, akpm, mike.kravetz, viro, riandrews, arve, yishaih,
dledford, sean.hefty, hal.rosenstock, matanb, leonro,
linux-fsdevel, linux-mm, devel, linux-rdma, linux-kernel, stable,
akaher, srivatsab, amakhalov, Linus Torvalds, Greg Kroah-Hartman
From: Andrea Arcangeli <aarcange@redhat.com>
commit 04f5866e41fb70690e28397487d8bd8eea7d712a upstream.
The core dumping code has always run without holding the mmap_sem for
writing, despite that is the only way to ensure that the entire vma
layout will not change from under it. Only using some signal
serialization on the processes belonging to the mm is not nearly enough.
This was pointed out earlier. For example in Hugh's post from Jul 2017:
https://lkml.kernel.org/r/alpine.LSU.2.11.1707191716030.2055@eggly.anvils
"Not strictly relevant here, but a related note: I was very surprised
to discover, only quite recently, how handle_mm_fault() may be called
without down_read(mmap_sem) - when core dumping. That seems a
misguided optimization to me, which would also be nice to correct"
In particular because the growsdown and growsup can move the
vm_start/vm_end the various loops the core dump does around the vma will
not be consistent if page faults can happen concurrently.
Pretty much all users calling mmget_not_zero()/get_task_mm() and then
taking the mmap_sem had the potential to introduce unexpected side
effects in the core dumping code.
Adding mmap_sem for writing around the ->core_dump invocation is a
viable long term fix, but it requires removing all copy user and page
faults and to replace them with get_dump_page() for all binary formats
which is not suitable as a short term fix.
For the time being this solution manually covers the places that can
confuse the core dump either by altering the vma layout or the vma flags
while it runs. Once ->core_dump runs under mmap_sem for writing the
function mmget_still_valid() can be dropped.
Allowing mmap_sem protected sections to run in parallel with the
coredump provides some minor parallelism advantage to the swapoff code
(which seems to be safe enough by never mangling any vma field and can
keep doing swapins in parallel to the core dumping) and to some other
corner case.
In order to facilitate the backporting I added "Fixes: 86039bd3b4e6"
however the side effect of this same race condition in /proc/pid/mem
should be reproducible since before 2.6.12-rc2 so I couldn't add any
other "Fixes:" because there's no hash beyond the git genesis commit.
Because find_extend_vma() is the only location outside of the process
context that could modify the "mm" structures under mmap_sem for
reading, by adding the mmget_still_valid() check to it, all other cases
that take the mmap_sem for reading don't need the new check after
mmget_not_zero()/get_task_mm(). The expand_stack() in page fault
context also doesn't need the new check, because all tasks under core
dumping are frozen.
Link: http://lkml.kernel.org/r/20190325224949.11068-1-aarcange@redhat.com
Fixes: 86039bd3b4e6 ("userfaultfd: add new syscall to provide memory externalization")
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
Reported-by: Jann Horn <jannh@google.com>
Suggested-by: Oleg Nesterov <oleg@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Jann Horn <jannh@google.com>
Acked-by: Jason Gunthorpe <jgg@mellanox.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
[akaher@vmware.com: stable 4.9 backport
- handle binder_update_page_range - mhocko@suse.com]
Signed-off-by: Ajay Kaher <akaher@vmware.com>
---
drivers/android/binder.c | 6 ++++++
fs/proc/task_mmu.c | 18 ++++++++++++++++++
fs/userfaultfd.c | 9 +++++++++
include/linux/mm.h | 21 +++++++++++++++++++++
mm/mmap.c | 6 +++++-
5 files changed, 59 insertions(+), 1 deletion(-)
diff --git a/drivers/android/binder.c b/drivers/android/binder.c
index 80499f4..f05ab8f 100644
--- a/drivers/android/binder.c
+++ b/drivers/android/binder.c
@@ -581,6 +581,12 @@ static int binder_update_page_range(struct binder_proc *proc, int allocate,
if (mm) {
down_write(&mm->mmap_sem);
+ if (!mmget_still_valid(mm)) {
+ if (allocate == 0)
+ goto free_range;
+ goto err_no_vma;
+ }
+
vma = proc->vma;
if (vma && mm != proc->vma_vm_mm) {
pr_err("%d: vma mm and task mm mismatch\n",
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 5138e78..4b207b1 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -1057,6 +1057,24 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf,
count = -EINTR;
goto out_mm;
}
+ /*
+ * Avoid to modify vma->vm_flags
+ * without locked ops while the
+ * coredump reads the vm_flags.
+ */
+ if (!mmget_still_valid(mm)) {
+ /*
+ * Silently return "count"
+ * like if get_task_mm()
+ * failed. FIXME: should this
+ * function have returned
+ * -ESRCH if get_task_mm()
+ * failed like if
+ * get_proc_task() fails?
+ */
+ up_write(&mm->mmap_sem);
+ goto out_mm;
+ }
for (vma = mm->mmap; vma; vma = vma->vm_next) {
vma->vm_flags &= ~VM_SOFTDIRTY;
vma_set_page_prot(vma);
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index 784d667..8bf425a 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -479,6 +479,8 @@ static int userfaultfd_release(struct inode *inode, struct file *file)
* taking the mmap_sem for writing.
*/
down_write(&mm->mmap_sem);
+ if (!mmget_still_valid(mm))
+ goto skip_mm;
prev = NULL;
for (vma = mm->mmap; vma; vma = vma->vm_next) {
cond_resched();
@@ -501,6 +503,7 @@ static int userfaultfd_release(struct inode *inode, struct file *file)
vma->vm_flags = new_flags;
vma->vm_userfaultfd_ctx = NULL_VM_UFFD_CTX;
}
+skip_mm:
up_write(&mm->mmap_sem);
mmput(mm);
wakeup:
@@ -802,6 +805,9 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
goto out;
down_write(&mm->mmap_sem);
+ if (!mmget_still_valid(mm))
+ goto out_unlock;
+
vma = find_vma_prev(mm, start, &prev);
if (!vma)
goto out_unlock;
@@ -947,6 +953,9 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx,
goto out;
down_write(&mm->mmap_sem);
+ if (!mmget_still_valid(mm))
+ goto out_unlock;
+
vma = find_vma_prev(mm, start, &prev);
if (!vma)
goto out_unlock;
diff --git a/include/linux/mm.h b/include/linux/mm.h
index e3c8d40..c239984 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1189,6 +1189,27 @@ void zap_page_range(struct vm_area_struct *vma, unsigned long address,
void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma,
unsigned long start, unsigned long end);
+/*
+ * This has to be called after a get_task_mm()/mmget_not_zero()
+ * followed by taking the mmap_sem for writing before modifying the
+ * vmas or anything the coredump pretends not to change from under it.
+ *
+ * NOTE: find_extend_vma() called from GUP context is the only place
+ * that can modify the "mm" (notably the vm_start/end) under mmap_sem
+ * for reading and outside the context of the process, so it is also
+ * the only case that holds the mmap_sem for reading that must call
+ * this function. Generally if the mmap_sem is hold for reading
+ * there's no need of this check after get_task_mm()/mmget_not_zero().
+ *
+ * This function can be obsoleted and the check can be removed, after
+ * the coredump code will hold the mmap_sem for writing before
+ * invoking the ->core_dump methods.
+ */
+static inline bool mmget_still_valid(struct mm_struct *mm)
+{
+ return likely(!mm->core_state);
+}
+
/**
* mm_walk - callbacks for walk_page_range
* @pmd_entry: if set, called for each non-empty PMD (3rd-level) entry
diff --git a/mm/mmap.c b/mm/mmap.c
index 3f2314a..19368fb 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -2448,7 +2448,8 @@ find_extend_vma(struct mm_struct *mm, unsigned long addr)
vma = find_vma_prev(mm, addr, &prev);
if (vma && (vma->vm_start <= addr))
return vma;
- if (!prev || expand_stack(prev, addr))
+ /* don't alter vm_end if the coredump is running */
+ if (!prev || !mmget_still_valid(mm) || expand_stack(prev, addr))
return NULL;
if (prev->vm_flags & VM_LOCKED)
populate_vma_page_range(prev, addr, prev->vm_end, NULL);
@@ -2474,6 +2475,9 @@ find_extend_vma(struct mm_struct *mm, unsigned long addr)
return vma;
if (!(vma->vm_flags & VM_GROWSDOWN))
return NULL;
+ /* don't alter vm_start if the coredump is running */
+ if (!mmget_still_valid(mm))
+ return NULL;
start = vma->vm_start;
if (expand_stack(vma, addr))
return NULL;
--
2.7.4
^ permalink raw reply related
* [PATCH v4 hmm 12/12] mm/hmm: Fix error flows in hmm_invalidate_range_start
From: Jason Gunthorpe @ 2019-06-24 21:01 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
If the trylock on the hmm->mirrors_sem fails the function will return
without decrementing the notifiers that were previously incremented. Since
the caller will not call invalidate_range_end() on EAGAIN this will result
in notifiers becoming permanently incremented and deadlock.
If the sync_cpu_device_pagetables() required blocking the function will
not return EAGAIN even though the device continues to touch the
pages. This is a violation of the mmu notifier contract.
Switch, and rename, the ranges_lock to a spin lock so we can reliably
obtain it without blocking during error unwind.
The error unwind is necessary since the notifiers count must be held
incremented across the call to sync_cpu_device_pagetables() as we cannot
allow the range to become marked valid by a parallel
invalidate_start/end() pair while doing sync_cpu_device_pagetables().
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
include/linux/hmm.h | 2 +-
mm/hmm.c | 72 +++++++++++++++++++++++++++------------------
2 files changed, 45 insertions(+), 29 deletions(-)
diff --git a/include/linux/hmm.h b/include/linux/hmm.h
index bf013e96525771..0fa8ea34ccef6d 100644
--- a/include/linux/hmm.h
+++ b/include/linux/hmm.h
@@ -86,7 +86,7 @@
struct hmm {
struct mm_struct *mm;
struct kref kref;
- struct mutex lock;
+ spinlock_t ranges_lock;
struct list_head ranges;
struct list_head mirrors;
struct mmu_notifier mmu_notifier;
diff --git a/mm/hmm.c b/mm/hmm.c
index b224ea635a7716..89549eac03d506 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -64,7 +64,7 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm)
init_rwsem(&hmm->mirrors_sem);
hmm->mmu_notifier.ops = NULL;
INIT_LIST_HEAD(&hmm->ranges);
- mutex_init(&hmm->lock);
+ spin_lock_init(&hmm->ranges_lock);
kref_init(&hmm->kref);
hmm->notifiers = 0;
hmm->mm = mm;
@@ -144,6 +144,23 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm)
hmm_put(hmm);
}
+static void notifiers_decrement(struct hmm *hmm)
+{
+ lockdep_assert_held(&hmm->ranges_lock);
+
+ hmm->notifiers--;
+ if (!hmm->notifiers) {
+ struct hmm_range *range;
+
+ list_for_each_entry(range, &hmm->ranges, list) {
+ if (range->valid)
+ continue;
+ range->valid = true;
+ }
+ wake_up_all(&hmm->wq);
+ }
+}
+
static int hmm_invalidate_range_start(struct mmu_notifier *mn,
const struct mmu_notifier_range *nrange)
{
@@ -151,6 +168,7 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn,
struct hmm_mirror *mirror;
struct hmm_update update;
struct hmm_range *range;
+ unsigned long flags;
int ret = 0;
if (!kref_get_unless_zero(&hmm->kref))
@@ -161,12 +179,7 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn,
update.event = HMM_UPDATE_INVALIDATE;
update.blockable = mmu_notifier_range_blockable(nrange);
- if (mmu_notifier_range_blockable(nrange))
- mutex_lock(&hmm->lock);
- else if (!mutex_trylock(&hmm->lock)) {
- ret = -EAGAIN;
- goto out;
- }
+ spin_lock_irqsave(&hmm->ranges_lock, flags);
hmm->notifiers++;
list_for_each_entry(range, &hmm->ranges, list) {
if (update.end < range->start || update.start >= range->end)
@@ -174,7 +187,7 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn,
range->valid = false;
}
- mutex_unlock(&hmm->lock);
+ spin_unlock_irqrestore(&hmm->ranges_lock, flags);
if (mmu_notifier_range_blockable(nrange))
down_read(&hmm->mirrors_sem);
@@ -182,16 +195,26 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn,
ret = -EAGAIN;
goto out;
}
+
list_for_each_entry(mirror, &hmm->mirrors, list) {
- int ret;
+ int rc;
- ret = mirror->ops->sync_cpu_device_pagetables(mirror, &update);
- if (!update.blockable && ret == -EAGAIN)
+ rc = mirror->ops->sync_cpu_device_pagetables(mirror, &update);
+ if (rc) {
+ if (WARN_ON(update.blockable || rc != -EAGAIN))
+ continue;
+ ret = -EAGAIN;
break;
+ }
}
up_read(&hmm->mirrors_sem);
out:
+ if (ret) {
+ spin_lock_irqsave(&hmm->ranges_lock, flags);
+ notifiers_decrement(hmm);
+ spin_unlock_irqrestore(&hmm->ranges_lock, flags);
+ }
hmm_put(hmm);
return ret;
}
@@ -200,23 +223,14 @@ static void hmm_invalidate_range_end(struct mmu_notifier *mn,
const struct mmu_notifier_range *nrange)
{
struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier);
+ unsigned long flags;
if (!kref_get_unless_zero(&hmm->kref))
return;
- mutex_lock(&hmm->lock);
- hmm->notifiers--;
- if (!hmm->notifiers) {
- struct hmm_range *range;
-
- list_for_each_entry(range, &hmm->ranges, list) {
- if (range->valid)
- continue;
- range->valid = true;
- }
- wake_up_all(&hmm->wq);
- }
- mutex_unlock(&hmm->lock);
+ spin_lock_irqsave(&hmm->ranges_lock, flags);
+ notifiers_decrement(hmm);
+ spin_unlock_irqrestore(&hmm->ranges_lock, flags);
hmm_put(hmm);
}
@@ -868,6 +882,7 @@ int hmm_range_register(struct hmm_range *range,
{
unsigned long mask = ((1UL << page_shift) - 1UL);
struct hmm *hmm = mirror->hmm;
+ unsigned long flags;
range->valid = false;
range->hmm = NULL;
@@ -886,7 +901,7 @@ int hmm_range_register(struct hmm_range *range,
return -EFAULT;
/* Initialize range to track CPU page table updates. */
- mutex_lock(&hmm->lock);
+ spin_lock_irqsave(&hmm->ranges_lock, flags);
range->hmm = hmm;
kref_get(&hmm->kref);
@@ -898,7 +913,7 @@ int hmm_range_register(struct hmm_range *range,
*/
if (!hmm->notifiers)
range->valid = true;
- mutex_unlock(&hmm->lock);
+ spin_unlock_irqrestore(&hmm->ranges_lock, flags);
return 0;
}
@@ -914,10 +929,11 @@ EXPORT_SYMBOL(hmm_range_register);
void hmm_range_unregister(struct hmm_range *range)
{
struct hmm *hmm = range->hmm;
+ unsigned long flags;
- mutex_lock(&hmm->lock);
+ spin_lock_irqsave(&hmm->ranges_lock, flags);
list_del_init(&range->list);
- mutex_unlock(&hmm->lock);
+ spin_unlock_irqrestore(&hmm->ranges_lock, flags);
/* Drop reference taken by hmm_range_register() */
mmput(hmm->mm);
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
* [PATCH v4 hmm 11/12] mm/hmm: Remove confusing comment and logic from hmm_release
From: Jason Gunthorpe @ 2019-06-24 21:01 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
hmm_release() is called exactly once per hmm. ops->release() cannot
accidentally trigger any action that would recurse back onto
hmm->mirrors_sem.
This fixes a use after-free race of the form:
CPU0 CPU1
hmm_release()
up_write(&hmm->mirrors_sem);
hmm_mirror_unregister(mirror)
down_write(&hmm->mirrors_sem);
up_write(&hmm->mirrors_sem);
kfree(mirror)
mirror->ops->release(mirror)
The only user we have today for ops->release is an empty function, so this
is unambiguously safe.
As a consequence of plugging this race drivers are not allowed to
register/unregister mirrors from within a release op.
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
mm/hmm.c | 28 +++++++++-------------------
1 file changed, 9 insertions(+), 19 deletions(-)
diff --git a/mm/hmm.c b/mm/hmm.c
index c30aa9403dbe4d..b224ea635a7716 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -130,26 +130,16 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm)
*/
WARN_ON(!list_empty_careful(&hmm->ranges));
- down_write(&hmm->mirrors_sem);
- mirror = list_first_entry_or_null(&hmm->mirrors, struct hmm_mirror,
- list);
- while (mirror) {
- list_del_init(&mirror->list);
- if (mirror->ops->release) {
- /*
- * Drop mirrors_sem so the release callback can wait
- * on any pending work that might itself trigger a
- * mmu_notifier callback and thus would deadlock with
- * us.
- */
- up_write(&hmm->mirrors_sem);
+ down_read(&hmm->mirrors_sem);
+ list_for_each_entry(mirror, &hmm->mirrors, list) {
+ /*
+ * Note: The driver is not allowed to trigger
+ * hmm_mirror_unregister() from this thread.
+ */
+ if (mirror->ops->release)
mirror->ops->release(mirror);
- down_write(&hmm->mirrors_sem);
- }
- mirror = list_first_entry_or_null(&hmm->mirrors,
- struct hmm_mirror, list);
}
- up_write(&hmm->mirrors_sem);
+ up_read(&hmm->mirrors_sem);
hmm_put(hmm);
}
@@ -279,7 +269,7 @@ void hmm_mirror_unregister(struct hmm_mirror *mirror)
struct hmm *hmm = mirror->hmm;
down_write(&hmm->mirrors_sem);
- list_del_init(&mirror->list);
+ list_del(&mirror->list);
up_write(&hmm->mirrors_sem);
hmm_put(hmm);
}
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
* [PATCH v4 hmm 10/12] mm/hmm: Poison hmm_range during unregister
From: Jason Gunthorpe @ 2019-06-24 21:01 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Souptick Joarder,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
Trying to misuse a range outside its lifetime is a kernel bug. Use poison
bytes to help detect this condition. Double unregister will reliably crash.
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Souptick Joarder <jrdr.linux@gmail.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
v2
- Keep range start/end valid after unregistration (Jerome)
v3
- Revise some comments (John)
- Remove start/end WARN_ON (Souptick)
v4
- Fix tabs vs spaces in comment (Christoph)
---
mm/hmm.c | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/mm/hmm.c b/mm/hmm.c
index 2ef14b2b5505f6..c30aa9403dbe4d 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -925,19 +925,21 @@ void hmm_range_unregister(struct hmm_range *range)
{
struct hmm *hmm = range->hmm;
- /* Sanity check this really should not happen. */
- if (hmm == NULL || range->end <= range->start)
- return;
-
mutex_lock(&hmm->lock);
list_del_init(&range->list);
mutex_unlock(&hmm->lock);
/* Drop reference taken by hmm_range_register() */
- range->valid = false;
mmput(hmm->mm);
hmm_put(hmm);
- range->hmm = NULL;
+
+ /*
+ * The range is now invalid and the ref on the hmm is dropped, so
+ * poison the pointer. Leave other fields in place, for the caller's
+ * use.
+ */
+ range->valid = false;
+ memset(&range->hmm, POISON_INUSE, sizeof(range->hmm));
}
EXPORT_SYMBOL(hmm_range_unregister);
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
* [PATCH v4 hmm 09/12] mm/hmm: Remove racy protection against double-unregistration
From: Jason Gunthorpe @ 2019-06-24 21:01 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
No other register/unregister kernel API attempts to provide this kind of
protection as it is inherently racy, so just drop it.
Callers should provide their own protection, and it appears nouveau
already does.
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
v3
- Drop poison, looks like there are no new patches that will use this
wrong (Christoph)
---
mm/hmm.c | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/mm/hmm.c b/mm/hmm.c
index 6f5dc6d568feb1..2ef14b2b5505f6 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -276,17 +276,11 @@ EXPORT_SYMBOL(hmm_mirror_register);
*/
void hmm_mirror_unregister(struct hmm_mirror *mirror)
{
- struct hmm *hmm = READ_ONCE(mirror->hmm);
-
- if (hmm == NULL)
- return;
+ struct hmm *hmm = mirror->hmm;
down_write(&hmm->mirrors_sem);
list_del_init(&mirror->list);
- /* To protect us against double unregister ... */
- mirror->hmm = NULL;
up_write(&hmm->mirrors_sem);
-
hmm_put(hmm);
}
EXPORT_SYMBOL(hmm_mirror_unregister);
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
* [PATCH v4 hmm 08/12] mm/hmm: Use lockdep instead of comments
From: Jason Gunthorpe @ 2019-06-24 21:01 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Souptick Joarder,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
So we can check locking at runtime.
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
Acked-by: Souptick Joarder <jrdr.linux@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
v2
- Fix missing & in lockdeps (Jason)
---
mm/hmm.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/mm/hmm.c b/mm/hmm.c
index 1eddda45cefae7..6f5dc6d568feb1 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -246,11 +246,11 @@ static const struct mmu_notifier_ops hmm_mmu_notifier_ops = {
*
* To start mirroring a process address space, the device driver must register
* an HMM mirror struct.
- *
- * THE mm->mmap_sem MUST BE HELD IN WRITE MODE !
*/
int hmm_mirror_register(struct hmm_mirror *mirror, struct mm_struct *mm)
{
+ lockdep_assert_held_exclusive(&mm->mmap_sem);
+
/* Sanity check */
if (!mm || !mirror || !mirror->ops)
return -EINVAL;
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
* [PATCH v4 hmm 07/12] mm/hmm: Hold on to the mmget for the lifetime of the range
From: Jason Gunthorpe @ 2019-06-24 21:01 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
Range functions like hmm_range_snapshot() and hmm_range_fault() call
find_vma, which requires hodling the mmget() and the mmap_sem for the mm.
Make this simpler for the callers by holding the mmget() inside the range
for the lifetime of the range. Other functions that accept a range should
only be called if the range is registered.
This has the side effect of directly preventing hmm_release() from
happening while a range is registered. That means range->dead cannot be
false during the lifetime of the range, so remove dead and
hmm_mirror_mm_is_alive() entirely.
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
v2:
- Use Jerome's idea of just holding the mmget() for the range lifetime,
rework the patch to use that as as simplification to remove dead in
one step
v3:
- Use list_del_careful (Christoph)
---
include/linux/hmm.h | 26 --------------------------
mm/hmm.c | 32 +++++++++++---------------------
2 files changed, 11 insertions(+), 47 deletions(-)
diff --git a/include/linux/hmm.h b/include/linux/hmm.h
index 26e7c477490c4e..bf013e96525771 100644
--- a/include/linux/hmm.h
+++ b/include/linux/hmm.h
@@ -82,7 +82,6 @@
* @mirrors_sem: read/write semaphore protecting the mirrors list
* @wq: wait queue for user waiting on a range invalidation
* @notifiers: count of active mmu notifiers
- * @dead: is the mm dead ?
*/
struct hmm {
struct mm_struct *mm;
@@ -95,7 +94,6 @@ struct hmm {
wait_queue_head_t wq;
struct rcu_head rcu;
long notifiers;
- bool dead;
};
/*
@@ -459,30 +457,6 @@ struct hmm_mirror {
int hmm_mirror_register(struct hmm_mirror *mirror, struct mm_struct *mm);
void hmm_mirror_unregister(struct hmm_mirror *mirror);
-/*
- * hmm_mirror_mm_is_alive() - test if mm is still alive
- * @mirror: the HMM mm mirror for which we want to lock the mmap_sem
- * Return: false if the mm is dead, true otherwise
- *
- * This is an optimization, it will not always accurately return false if the
- * mm is dead; i.e., there can be false negatives (process is being killed but
- * HMM is not yet informed of that). It is only intended to be used to optimize
- * out cases where the driver is about to do something time consuming and it
- * would be better to skip it if the mm is dead.
- */
-static inline bool hmm_mirror_mm_is_alive(struct hmm_mirror *mirror)
-{
- struct mm_struct *mm;
-
- if (!mirror || !mirror->hmm)
- return false;
- mm = READ_ONCE(mirror->hmm->mm);
- if (mirror->hmm->dead || !mm)
- return false;
-
- return true;
-}
-
/*
* Please see Documentation/vm/hmm.rst for how to use the range API.
*/
diff --git a/mm/hmm.c b/mm/hmm.c
index 73c8af4827fe87..1eddda45cefae7 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -67,7 +67,6 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm)
mutex_init(&hmm->lock);
kref_init(&hmm->kref);
hmm->notifiers = 0;
- hmm->dead = false;
hmm->mm = mm;
hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops;
@@ -120,21 +119,16 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm)
{
struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier);
struct hmm_mirror *mirror;
- struct hmm_range *range;
/* Bail out if hmm is in the process of being freed */
if (!kref_get_unless_zero(&hmm->kref))
return;
- /* Report this HMM as dying. */
- hmm->dead = true;
-
- /* Wake-up everyone waiting on any range. */
- mutex_lock(&hmm->lock);
- list_for_each_entry(range, &hmm->ranges, list)
- range->valid = false;
- wake_up_all(&hmm->wq);
- mutex_unlock(&hmm->lock);
+ /*
+ * Since hmm_range_register() holds the mmget() lock hmm_release() is
+ * prevented as long as a range exists.
+ */
+ WARN_ON(!list_empty_careful(&hmm->ranges));
down_write(&hmm->mirrors_sem);
mirror = list_first_entry_or_null(&hmm->mirrors, struct hmm_mirror,
@@ -903,8 +897,8 @@ int hmm_range_register(struct hmm_range *range,
range->start = start;
range->end = end;
- /* Check if hmm_mm_destroy() was call. */
- if (hmm->mm == NULL || hmm->dead)
+ /* Prevent hmm_release() from running while the range is valid */
+ if (!mmget_not_zero(hmm->mm))
return -EFAULT;
/* Initialize range to track CPU page table updates. */
@@ -942,11 +936,12 @@ void hmm_range_unregister(struct hmm_range *range)
return;
mutex_lock(&hmm->lock);
- list_del(&range->list);
+ list_del_init(&range->list);
mutex_unlock(&hmm->lock);
/* Drop reference taken by hmm_range_register() */
range->valid = false;
+ mmput(hmm->mm);
hmm_put(hmm);
range->hmm = NULL;
}
@@ -974,10 +969,7 @@ long hmm_range_snapshot(struct hmm_range *range)
struct vm_area_struct *vma;
struct mm_walk mm_walk;
- /* Check if hmm_mm_destroy() was call. */
- if (hmm->mm == NULL || hmm->dead)
- return -EFAULT;
-
+ lockdep_assert_held(&hmm->mm->mmap_sem);
do {
/* If range is no longer valid force retry. */
if (!range->valid)
@@ -1072,9 +1064,7 @@ long hmm_range_fault(struct hmm_range *range, bool block)
struct mm_walk mm_walk;
int ret;
- /* Check if hmm_mm_destroy() was call. */
- if (hmm->mm == NULL || hmm->dead)
- return -EFAULT;
+ lockdep_assert_held(&hmm->mm->mmap_sem);
do {
/* If range is no longer valid force retry. */
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
* [PATCH v4 hmm 06/12] mm/hmm: Do not use list*_rcu() for hmm->ranges
From: Jason Gunthorpe @ 2019-06-24 21:01 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
Ira Weiny, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
Souptick Joarder, linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
Jason Gunthorpe, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
Ira Weiny, Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
This list is always read and written while holding hmm->lock so there is
no need for the confusing _rcu annotations.
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Souptick Joarder <jrdr.linux@gmail.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: Ira Weiny <iweiny@intel.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
mm/hmm.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/mm/hmm.c b/mm/hmm.c
index 0423f4ca3a7e09..73c8af4827fe87 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -912,7 +912,7 @@ int hmm_range_register(struct hmm_range *range,
range->hmm = hmm;
kref_get(&hmm->kref);
- list_add_rcu(&range->list, &hmm->ranges);
+ list_add(&range->list, &hmm->ranges);
/*
* If there are any concurrent notifiers we have to wait for them for
@@ -942,7 +942,7 @@ void hmm_range_unregister(struct hmm_range *range)
return;
mutex_lock(&hmm->lock);
- list_del_rcu(&range->list);
+ list_del(&range->list);
mutex_unlock(&hmm->lock);
/* Drop reference taken by hmm_range_register() */
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
* [PATCH v4 hmm 05/12] mm/hmm: Remove duplicate condition test before wait_event_timeout
From: Jason Gunthorpe @ 2019-06-24 21:01 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
The wait_event_timeout macro already tests the condition as its first
action, so there is no reason to open code another version of this, all
that does is skip the might_sleep() debugging in common cases, which is
not helpful.
Further, based on prior patches, we can now simplify the required condition
test:
- If range is valid memory then so is range->hmm
- If hmm_release() has run then range->valid is set to false
at the same time as dead, so no reason to check both.
- A valid hmm has a valid hmm->mm.
Allowing the return value of wait_event_timeout() (along with its internal
barriers) to compute the result of the function.
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
v3
- Simplify the wait_event_timeout to not check valid
---
include/linux/hmm.h | 13 ++-----------
1 file changed, 2 insertions(+), 11 deletions(-)
diff --git a/include/linux/hmm.h b/include/linux/hmm.h
index 1d97b6d62c5bcf..26e7c477490c4e 100644
--- a/include/linux/hmm.h
+++ b/include/linux/hmm.h
@@ -209,17 +209,8 @@ static inline unsigned long hmm_range_page_size(const struct hmm_range *range)
static inline bool hmm_range_wait_until_valid(struct hmm_range *range,
unsigned long timeout)
{
- /* Check if mm is dead ? */
- if (range->hmm == NULL || range->hmm->dead || range->hmm->mm == NULL) {
- range->valid = false;
- return false;
- }
- if (range->valid)
- return true;
- wait_event_timeout(range->hmm->wq, range->valid || range->hmm->dead,
- msecs_to_jiffies(timeout));
- /* Return current valid status just in case we get lucky */
- return range->valid;
+ return wait_event_timeout(range->hmm->wq, range->valid,
+ msecs_to_jiffies(timeout)) != 0;
}
/*
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
* [PATCH v4 hmm 04/12] mm/hmm: Simplify hmm_get_or_create and make it reliable
From: Jason Gunthorpe @ 2019-06-24 21:01 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
As coded this function can false-fail in various racy situations. Make it
reliable and simpler by running under the write side of the mmap_sem and
avoiding the false-failing compare/exchange pattern. Due to the mmap_sem
this no longer has to avoid racing with a 2nd parallel
hmm_get_or_create().
Unfortunately this still has to use the page_table_lock as the
non-sleeping lock protecting mm->hmm, since the contexts where we free the
hmm are incompatible with mmap_sem.
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
v2:
- Fix error unwind of mmgrab (Jerome)
- Use hmm local instead of 2nd container_of (Jerome)
v3:
- Can't use mmap_sem in the SRCU callback, keep using the
page_table_lock (Philip)
v4:
- Put the mm->hmm = NULL in the kref release, reduce LOC
in hmm_get_or_create() (Christoph)
---
mm/hmm.c | 77 ++++++++++++++++++++++----------------------------------
1 file changed, 30 insertions(+), 47 deletions(-)
diff --git a/mm/hmm.c b/mm/hmm.c
index 080b17a2e87e2d..0423f4ca3a7e09 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -31,16 +31,6 @@
#if IS_ENABLED(CONFIG_HMM_MIRROR)
static const struct mmu_notifier_ops hmm_mmu_notifier_ops;
-static inline struct hmm *mm_get_hmm(struct mm_struct *mm)
-{
- struct hmm *hmm = READ_ONCE(mm->hmm);
-
- if (hmm && kref_get_unless_zero(&hmm->kref))
- return hmm;
-
- return NULL;
-}
-
/**
* hmm_get_or_create - register HMM against an mm (HMM internal)
*
@@ -55,11 +45,16 @@ static inline struct hmm *mm_get_hmm(struct mm_struct *mm)
*/
static struct hmm *hmm_get_or_create(struct mm_struct *mm)
{
- struct hmm *hmm = mm_get_hmm(mm);
- bool cleanup = false;
+ struct hmm *hmm;
+
+ lockdep_assert_held_exclusive(&mm->mmap_sem);
- if (hmm)
- return hmm;
+ /* Abuse the page_table_lock to also protect mm->hmm. */
+ spin_lock(&mm->page_table_lock);
+ hmm = mm->hmm;
+ if (mm->hmm && kref_get_unless_zero(&mm->hmm->kref))
+ goto out_unlock;
+ spin_unlock(&mm->page_table_lock);
hmm = kmalloc(sizeof(*hmm), GFP_KERNEL);
if (!hmm)
@@ -74,57 +69,45 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm)
hmm->notifiers = 0;
hmm->dead = false;
hmm->mm = mm;
- mmgrab(hmm->mm);
- spin_lock(&mm->page_table_lock);
- if (!mm->hmm)
- mm->hmm = hmm;
- else
- cleanup = true;
- spin_unlock(&mm->page_table_lock);
+ hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops;
+ if (__mmu_notifier_register(&hmm->mmu_notifier, mm)) {
+ kfree(hmm);
+ return NULL;
+ }
- if (cleanup)
- goto error;
+ mmgrab(hmm->mm);
/*
- * We should only get here if hold the mmap_sem in write mode ie on
- * registration of first mirror through hmm_mirror_register()
+ * We hold the exclusive mmap_sem here so we know that mm->hmm is
+ * still NULL or 0 kref, and is safe to update.
*/
- hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops;
- if (__mmu_notifier_register(&hmm->mmu_notifier, mm))
- goto error_mm;
-
- return hmm;
-
-error_mm:
spin_lock(&mm->page_table_lock);
- if (mm->hmm == hmm)
- mm->hmm = NULL;
+ mm->hmm = hmm;
+
+out_unlock:
spin_unlock(&mm->page_table_lock);
-error:
- mmdrop(hmm->mm);
- kfree(hmm);
- return NULL;
+ return hmm;
}
static void hmm_free_rcu(struct rcu_head *rcu)
{
- kfree(container_of(rcu, struct hmm, rcu));
+ struct hmm *hmm = container_of(rcu, struct hmm, rcu);
+
+ mmdrop(hmm->mm);
+ kfree(hmm);
}
static void hmm_free(struct kref *kref)
{
struct hmm *hmm = container_of(kref, struct hmm, kref);
- struct mm_struct *mm = hmm->mm;
- mmu_notifier_unregister_no_release(&hmm->mmu_notifier, mm);
+ spin_lock(&hmm->mm->page_table_lock);
+ if (hmm->mm->hmm == hmm)
+ hmm->mm->hmm = NULL;
+ spin_unlock(&hmm->mm->page_table_lock);
- spin_lock(&mm->page_table_lock);
- if (mm->hmm == hmm)
- mm->hmm = NULL;
- spin_unlock(&mm->page_table_lock);
-
- mmdrop(hmm->mm);
+ mmu_notifier_unregister_no_release(&hmm->mmu_notifier, hmm->mm);
mmu_notifier_call_srcu(&hmm->rcu, hmm_free_rcu);
}
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
* [PATCH v4 hmm 03/12] mm/hmm: Hold a mmgrab from hmm to mm
From: Jason Gunthorpe @ 2019-06-24 21:01 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
So long as a struct hmm pointer exists, so should the struct mm it is
linked too. Hold the mmgrab() as soon as a hmm is created, and mmdrop() it
once the hmm refcount goes to zero.
Since mmdrop() (ie a 0 kref on struct mm) is now impossible with a !NULL
mm->hmm delete the hmm_hmm_destroy().
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
v2:
- Fix error unwind paths in hmm_get_or_create (Jerome/Jason)
---
include/linux/hmm.h | 3 ---
kernel/fork.c | 1 -
mm/hmm.c | 22 ++++------------------
3 files changed, 4 insertions(+), 22 deletions(-)
diff --git a/include/linux/hmm.h b/include/linux/hmm.h
index 1fba6979adf460..1d97b6d62c5bcf 100644
--- a/include/linux/hmm.h
+++ b/include/linux/hmm.h
@@ -577,14 +577,11 @@ static inline int hmm_vma_fault(struct hmm_mirror *mirror,
}
/* Below are for HMM internal use only! Not to be used by device driver! */
-void hmm_mm_destroy(struct mm_struct *mm);
-
static inline void hmm_mm_init(struct mm_struct *mm)
{
mm->hmm = NULL;
}
#else /* IS_ENABLED(CONFIG_HMM_MIRROR) */
-static inline void hmm_mm_destroy(struct mm_struct *mm) {}
static inline void hmm_mm_init(struct mm_struct *mm) {}
#endif /* IS_ENABLED(CONFIG_HMM_MIRROR) */
diff --git a/kernel/fork.c b/kernel/fork.c
index 75675b9bf6dfd3..c704c3cedee78d 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -673,7 +673,6 @@ void __mmdrop(struct mm_struct *mm)
WARN_ON_ONCE(mm == current->active_mm);
mm_free_pgd(mm);
destroy_context(mm);
- hmm_mm_destroy(mm);
mmu_notifier_mm_destroy(mm);
check_mm(mm);
put_user_ns(mm->user_ns);
diff --git a/mm/hmm.c b/mm/hmm.c
index 22a97ada108b4e..080b17a2e87e2d 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -20,6 +20,7 @@
#include <linux/swapops.h>
#include <linux/hugetlb.h>
#include <linux/memremap.h>
+#include <linux/sched/mm.h>
#include <linux/jump_label.h>
#include <linux/dma-mapping.h>
#include <linux/mmu_notifier.h>
@@ -73,6 +74,7 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm)
hmm->notifiers = 0;
hmm->dead = false;
hmm->mm = mm;
+ mmgrab(hmm->mm);
spin_lock(&mm->page_table_lock);
if (!mm->hmm)
@@ -100,6 +102,7 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm)
mm->hmm = NULL;
spin_unlock(&mm->page_table_lock);
error:
+ mmdrop(hmm->mm);
kfree(hmm);
return NULL;
}
@@ -121,6 +124,7 @@ static void hmm_free(struct kref *kref)
mm->hmm = NULL;
spin_unlock(&mm->page_table_lock);
+ mmdrop(hmm->mm);
mmu_notifier_call_srcu(&hmm->rcu, hmm_free_rcu);
}
@@ -129,24 +133,6 @@ static inline void hmm_put(struct hmm *hmm)
kref_put(&hmm->kref, hmm_free);
}
-void hmm_mm_destroy(struct mm_struct *mm)
-{
- struct hmm *hmm;
-
- spin_lock(&mm->page_table_lock);
- hmm = mm_get_hmm(mm);
- mm->hmm = NULL;
- if (hmm) {
- hmm->mm = NULL;
- hmm->dead = true;
- spin_unlock(&mm->page_table_lock);
- hmm_put(hmm);
- return;
- }
-
- spin_unlock(&mm->page_table_lock);
-}
-
static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm)
{
struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier);
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
* [PATCH v4 hmm 02/12] mm/hmm: Use hmm_mirror not mm as an argument for hmm_range_register
From: Jason Gunthorpe @ 2019-06-24 21:01 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
Ralph observes that hmm_range_register() can only be called by a driver
while a mirror is registered. Make this clear in the API by passing in the
mirror structure as a parameter.
This also simplifies understanding the lifetime model for struct hmm, as
the hmm pointer must be valid as part of a registered mirror so all we
need in hmm_register_range() is a simple kref_get.
Suggested-by: Ralph Campbell <rcampbell@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
v2
- Include the oneline patch to nouveau_svm.c
---
drivers/gpu/drm/nouveau/nouveau_svm.c | 2 +-
include/linux/hmm.h | 7 ++++---
mm/hmm.c | 13 ++++---------
3 files changed, 9 insertions(+), 13 deletions(-)
diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouveau/nouveau_svm.c
index 93ed43c413f0bb..8c92374afcf227 100644
--- a/drivers/gpu/drm/nouveau/nouveau_svm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_svm.c
@@ -649,7 +649,7 @@ nouveau_svm_fault(struct nvif_notify *notify)
range.values = nouveau_svm_pfn_values;
range.pfn_shift = NVIF_VMM_PFNMAP_V0_ADDR_SHIFT;
again:
- ret = hmm_vma_fault(&range, true);
+ ret = hmm_vma_fault(&svmm->mirror, &range, true);
if (ret == 0) {
mutex_lock(&svmm->mutex);
if (!hmm_vma_range_done(&range)) {
diff --git a/include/linux/hmm.h b/include/linux/hmm.h
index cb01cf1fa3c08b..1fba6979adf460 100644
--- a/include/linux/hmm.h
+++ b/include/linux/hmm.h
@@ -496,7 +496,7 @@ static inline bool hmm_mirror_mm_is_alive(struct hmm_mirror *mirror)
* Please see Documentation/vm/hmm.rst for how to use the range API.
*/
int hmm_range_register(struct hmm_range *range,
- struct mm_struct *mm,
+ struct hmm_mirror *mirror,
unsigned long start,
unsigned long end,
unsigned page_shift);
@@ -532,7 +532,8 @@ static inline bool hmm_vma_range_done(struct hmm_range *range)
}
/* This is a temporary helper to avoid merge conflict between trees. */
-static inline int hmm_vma_fault(struct hmm_range *range, bool block)
+static inline int hmm_vma_fault(struct hmm_mirror *mirror,
+ struct hmm_range *range, bool block)
{
long ret;
@@ -545,7 +546,7 @@ static inline int hmm_vma_fault(struct hmm_range *range, bool block)
range->default_flags = 0;
range->pfn_flags_mask = -1UL;
- ret = hmm_range_register(range, range->vma->vm_mm,
+ ret = hmm_range_register(range, mirror,
range->start, range->end,
PAGE_SHIFT);
if (ret)
diff --git a/mm/hmm.c b/mm/hmm.c
index f6956d78e3cb25..22a97ada108b4e 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -914,13 +914,13 @@ static void hmm_pfns_clear(struct hmm_range *range,
* Track updates to the CPU page table see include/linux/hmm.h
*/
int hmm_range_register(struct hmm_range *range,
- struct mm_struct *mm,
+ struct hmm_mirror *mirror,
unsigned long start,
unsigned long end,
unsigned page_shift)
{
unsigned long mask = ((1UL << page_shift) - 1UL);
- struct hmm *hmm;
+ struct hmm *hmm = mirror->hmm;
range->valid = false;
range->hmm = NULL;
@@ -934,20 +934,15 @@ int hmm_range_register(struct hmm_range *range,
range->start = start;
range->end = end;
- hmm = hmm_get_or_create(mm);
- if (!hmm)
- return -EFAULT;
-
/* Check if hmm_mm_destroy() was call. */
- if (hmm->mm == NULL || hmm->dead) {
- hmm_put(hmm);
+ if (hmm->mm == NULL || hmm->dead)
return -EFAULT;
- }
/* Initialize range to track CPU page table updates. */
mutex_lock(&hmm->lock);
range->hmm = hmm;
+ kref_get(&hmm->kref);
list_add_rcu(&range->list, &hmm->ranges);
/*
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
* [PATCH v4 hmm 01/12] mm/hmm: fix use after free with struct hmm in the mmu notifiers
From: Jason Gunthorpe @ 2019-06-24 21:00 UTC (permalink / raw)
To: Jerome Glisse, Ralph Campbell, John Hubbard,
Felix.Kuehling-5C7GfCeVMHo
Cc: Andrea Arcangeli, Philip Yang, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jason Gunthorpe,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Ira Weiny,
Christoph Hellwig, Ben Skeggs
In-Reply-To: <20190624210110.5098-1-jgg-uk2M96/98Pc@public.gmane.org>
From: Jason Gunthorpe <jgg@mellanox.com>
mmu_notifier_unregister_no_release() is not a fence and the mmu_notifier
system will continue to reference hmm->mn until the srcu grace period
expires.
Resulting in use after free races like this:
CPU0 CPU1
__mmu_notifier_invalidate_range_start()
srcu_read_lock
hlist_for_each ()
// mn == hmm->mn
hmm_mirror_unregister()
hmm_put()
hmm_free()
mmu_notifier_unregister_no_release()
hlist_del_init_rcu(hmm-mn->list)
mn->ops->invalidate_range_start(mn, range);
mm_get_hmm()
mm->hmm = NULL;
kfree(hmm)
mutex_lock(&hmm->lock);
Use SRCU to kfree the hmm memory so that the notifiers can rely on hmm
existing. Get the now-safe hmm struct through container_of and directly
check kref_get_unless_zero to lock it against free.
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Philip Yang <Philip.Yang@amd.com>
---
v2:
- Spell 'free' properly (Jerome/Ralph)
v3:
- Have only one clearer comment about kref_get_unless_zero (John)
---
include/linux/hmm.h | 1 +
mm/hmm.c | 23 +++++++++++++++++------
2 files changed, 18 insertions(+), 6 deletions(-)
diff --git a/include/linux/hmm.h b/include/linux/hmm.h
index 7007123842ba76..cb01cf1fa3c08b 100644
--- a/include/linux/hmm.h
+++ b/include/linux/hmm.h
@@ -93,6 +93,7 @@ struct hmm {
struct mmu_notifier mmu_notifier;
struct rw_semaphore mirrors_sem;
wait_queue_head_t wq;
+ struct rcu_head rcu;
long notifiers;
bool dead;
};
diff --git a/mm/hmm.c b/mm/hmm.c
index 826816ab237799..f6956d78e3cb25 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -104,6 +104,11 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm)
return NULL;
}
+static void hmm_free_rcu(struct rcu_head *rcu)
+{
+ kfree(container_of(rcu, struct hmm, rcu));
+}
+
static void hmm_free(struct kref *kref)
{
struct hmm *hmm = container_of(kref, struct hmm, kref);
@@ -116,7 +121,7 @@ static void hmm_free(struct kref *kref)
mm->hmm = NULL;
spin_unlock(&mm->page_table_lock);
- kfree(hmm);
+ mmu_notifier_call_srcu(&hmm->rcu, hmm_free_rcu);
}
static inline void hmm_put(struct hmm *hmm)
@@ -144,10 +149,14 @@ void hmm_mm_destroy(struct mm_struct *mm)
static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm)
{
- struct hmm *hmm = mm_get_hmm(mm);
+ struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier);
struct hmm_mirror *mirror;
struct hmm_range *range;
+ /* Bail out if hmm is in the process of being freed */
+ if (!kref_get_unless_zero(&hmm->kref))
+ return;
+
/* Report this HMM as dying. */
hmm->dead = true;
@@ -185,13 +194,14 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm)
static int hmm_invalidate_range_start(struct mmu_notifier *mn,
const struct mmu_notifier_range *nrange)
{
- struct hmm *hmm = mm_get_hmm(nrange->mm);
+ struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier);
struct hmm_mirror *mirror;
struct hmm_update update;
struct hmm_range *range;
int ret = 0;
- VM_BUG_ON(!hmm);
+ if (!kref_get_unless_zero(&hmm->kref))
+ return 0;
update.start = nrange->start;
update.end = nrange->end;
@@ -236,9 +246,10 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn,
static void hmm_invalidate_range_end(struct mmu_notifier *mn,
const struct mmu_notifier_range *nrange)
{
- struct hmm *hmm = mm_get_hmm(nrange->mm);
+ struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier);
- VM_BUG_ON(!hmm);
+ if (!kref_get_unless_zero(&hmm->kref))
+ return;
mutex_lock(&hmm->lock);
hmm->notifiers--;
--
2.22.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox