From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.3 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4EFF4C33C9C for ; Mon, 25 Nov 2019 00:53:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 26B6B20815 for ; Mon, 25 Nov 2019 00:53:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="bf4wQj5h" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727029AbfKYAxs (ORCPT ); Sun, 24 Nov 2019 19:53:48 -0500 Received: from mail-qt1-f193.google.com ([209.85.160.193]:45100 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727072AbfKYAxs (ORCPT ); Sun, 24 Nov 2019 19:53:48 -0500 Received: by mail-qt1-f193.google.com with SMTP id 30so15288281qtz.12 for ; Sun, 24 Nov 2019 16:53:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=W5KvmIUsZXoQMVbzDoxYnt7yT+pQC9oHjmWUHowLIDE=; b=bf4wQj5hUYAUJ6J+IG0+hDMiHwo0y37jHBHT8UdhIIsQlzBQ4CxdtIHJrRj51DxlRK CKLiYZNarSFFj4FEgRzMNxy+kI0kMWkp2JP+w0At3xJ5awQQfKCflGDMEbLjgI1axJbX hAKULgjdId2ND6JafSG19IZ76B47iyKuc//kc4PjYIOLOtUHlwtDFurR95sHJjXvaqXY fGrur8LfkpaaQNAoNniODg3+dsAf2S2GrC+6cEhUlICPkA/CU1JiTx+UkO0BkI15CMLp 7s5O7Rwk6bHEYMLHf8Cgu9Ix5CmyXm+g4xBp1CqvpB7TPmrxsr5ItFDOP1iwxt6Ai6Gn z9sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=W5KvmIUsZXoQMVbzDoxYnt7yT+pQC9oHjmWUHowLIDE=; b=MtxoDbmtFLhwkDOdY3parb6F6XFU46Sm4orf1+oBypDJbJAWSima9Yw0a5EUtS6Yit +cg/WQRXzH9oJTPDAb23RKfwYwooYFs9AJXagTxagH1yh2UZ23PS3WRSYMymy3v1O8nN es8IChh9mP+SLUCKi5W6VoFRTRAaqM5v0ZEWiQOQTyzOj1P6uFuBv+hu4RhDMfhdTVP4 t7DTGbPflWBHzGz8nScVkDsPZdvOOZcg9KmEEkpGj9fjtRqD8uflh6ucDcaxQjxbMuTT AhTrtjFI+vssx9CQt6N9WiggYB7uMvGT4wNPnoKH0MqTyrdHbP9Qj5VvIfr45idFgBYK ullg== X-Gm-Message-State: APjAAAXEcN/C8zkpvS3sLphbTGrTQu/9pGitK1G2Pq+mejJQOwE/aRsx VvLjH6FSVqPewQsL4+8k5wBMig== X-Google-Smtp-Source: APXvYqyaLXo5MUyD+I9dO6HiLJFn2uWdH+8xt8jSkZ8bt7a48VQTUuqWsbmWyw/EGV6UxA2EsKJ0Ug== X-Received: by 2002:ac8:2209:: with SMTP id o9mr27065091qto.246.1574643226463; Sun, 24 Nov 2019 16:53:46 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-162-113-180.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.113.180]) by smtp.gmail.com with ESMTPSA id o124sm2535273qkf.66.2019.11.24.16.53.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 24 Nov 2019 16:53:45 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1iZ2dH-00020J-3s; Sun, 24 Nov 2019 20:53:39 -0400 Date: Sun, 24 Nov 2019 20:53:39 -0400 From: Jason Gunthorpe To: John Hubbard Cc: Leon Romanovsky , Christoph Hellwig , Andrew Morton , Al Viro , Alex Williamson , Benjamin Herrenschmidt , =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= , Dan Williams , Daniel Vetter , Dave Chinner , David Airlie , "David S . Miller" , Ira Weiny , Jan Kara , Jens Axboe , Jonathan Corbet , =?utf-8?B?SsOpcsO0bWU=?= Glisse , Magnus Karlsson , Mauro Carvalho Chehab , Michael Ellerman , Michal Hocko , Mike Kravetz , Paul Mackerras , Shuah Khan , Vlastimil Babka , bpf@vger.kernel.org, dri-devel@lists.freedesktop.org, kvm@vger.kernel.org, linux-block@vger.kernel.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-media@vger.kernel.org, linux-rdma@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, netdev@vger.kernel.org, linux-mm@kvack.org, LKML Subject: Re: [PATCH v7 07/24] IB/umem: use get_user_pages_fast() to pin DMA pages Message-ID: <20191125005339.GC5634@ziepe.ca> References: <20191121071354.456618-1-jhubbard@nvidia.com> <20191121071354.456618-8-jhubbard@nvidia.com> <20191121080746.GC30991@infradead.org> <20191121143643.GC7448@ziepe.ca> <20191124100724.GH136476@unreal> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On Sun, Nov 24, 2019 at 04:05:16PM -0800, John Hubbard wrote: > I looked into this, and I believe that the problem is in gup.c. There appears to > have been an oversight, in commit 817be129e6f2 ("mm: validate get_user_pages_fast > flags"), in filtering out FOLL_FORCE. There is nothing in the _fast() implementation > that requires that we avoid writing to the pages. I think it is too late to be doing these kinds of changes, I will revert the patch and this will miss this merge window. Jason >From ec6cb45292d21d1af9b9d95997b8cf204bbe854c Mon Sep 17 00:00:00 2001 From: Jason Gunthorpe Date: Sun, 24 Nov 2019 20:47:59 -0400 Subject: [PATCH] Revert "IB/umem: use get_user_pages_fast() to pin DMA pages" This reverts commit c9a7a2ed837c563f9f89743a6db732591cb4035b. This was merged before enough testing was done, and it triggers a WARN_ON() in get_user_pages_fast(): WARNING: CPU: 1 PID: 2557 at mm/gup.c:2404 get_user_pages_fast+0x115/0x180 Call Trace: ib_umem_get+0x298/0x550 [ib_uverbs] mlx5_ib_db_map_user+0xad/0x130 [mlx5_ib] mlx5_ib_create_cq+0x1e8/0xaa0 [mlx5_ib] create_cq+0x1c8/0x2d0 [ib_uverbs] ib_uverbs_create_cq+0x70/0xa0 [ib_uverbs] ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0xc2/0xf0 [ib_uverbs] ib_uverbs_cmd_verbs.isra.6+0x5be/0xbe0 [ib_uverbs] ? uverbs_disassociate_api+0xd0/0xd0 [ib_uverbs] ? kvm_clock_get_cycles+0xd/0x10 ? kmem_cache_alloc+0x176/0x1c0 ? filemap_map_pages+0x18c/0x350 ib_uverbs_ioctl+0xc0/0x120 [ib_uverbs] do_vfs_ioctl+0xa1/0x610 ksys_ioctl+0x70/0x80 __x64_sys_ioctl+0x16/0x20 do_syscall_64+0x42/0x110 entry_SYSCALL_64_after_hwframe+0x44/0xa9 2404 if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM))) 2405 return -EINVAL; While we think this WARN_ON is probably bogus, resolving this will have to wait. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/umem.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 214e87aa609d6e..7a3b99597eada1 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -266,13 +266,16 @@ struct ib_umem *ib_umem_get(struct ib_udata *udata, unsigned long addr, sg = umem->sg_head.sgl; while (npages) { - ret = get_user_pages_fast(cur_base, - min_t(unsigned long, npages, - PAGE_SIZE / - sizeof(struct page *)), - gup_flags | FOLL_LONGTERM, page_list); - if (ret < 0) + down_read(&mm->mmap_sem); + ret = get_user_pages(cur_base, + min_t(unsigned long, npages, + PAGE_SIZE / sizeof (struct page *)), + gup_flags | FOLL_LONGTERM, + page_list, NULL); + if (ret < 0) { + up_read(&mm->mmap_sem); goto umem_release; + } cur_base += ret * PAGE_SIZE; npages -= ret; @@ -280,6 +283,8 @@ struct ib_umem *ib_umem_get(struct ib_udata *udata, unsigned long addr, sg = ib_umem_add_sg_table(sg, page_list, ret, dma_get_max_seg_size(context->device->dma_device), &umem->sg_nents); + + up_read(&mm->mmap_sem); } sg_mark_end(sg); -- 2.24.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E0BFC432C0 for ; Mon, 25 Nov 2019 00:55:42 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A70C72073F for ; Mon, 25 Nov 2019 00:55:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="bf4wQj5h" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A70C72073F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 47LpVg2wTYzDqbl for ; Mon, 25 Nov 2019 11:55:39 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ziepe.ca (client-ip=2607:f8b0:4864:20::841; helo=mail-qt1-x841.google.com; envelope-from=jgg@ziepe.ca; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=ziepe.ca header.i=@ziepe.ca header.b="bf4wQj5h"; dkim-atps=neutral Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 47LpSb3JqyzDqZ2 for ; Mon, 25 Nov 2019 11:53:49 +1100 (AEDT) Received: by mail-qt1-x841.google.com with SMTP id y10so15317749qto.3 for ; Sun, 24 Nov 2019 16:53:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=W5KvmIUsZXoQMVbzDoxYnt7yT+pQC9oHjmWUHowLIDE=; b=bf4wQj5hUYAUJ6J+IG0+hDMiHwo0y37jHBHT8UdhIIsQlzBQ4CxdtIHJrRj51DxlRK CKLiYZNarSFFj4FEgRzMNxy+kI0kMWkp2JP+w0At3xJ5awQQfKCflGDMEbLjgI1axJbX hAKULgjdId2ND6JafSG19IZ76B47iyKuc//kc4PjYIOLOtUHlwtDFurR95sHJjXvaqXY fGrur8LfkpaaQNAoNniODg3+dsAf2S2GrC+6cEhUlICPkA/CU1JiTx+UkO0BkI15CMLp 7s5O7Rwk6bHEYMLHf8Cgu9Ix5CmyXm+g4xBp1CqvpB7TPmrxsr5ItFDOP1iwxt6Ai6Gn z9sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=W5KvmIUsZXoQMVbzDoxYnt7yT+pQC9oHjmWUHowLIDE=; b=aOiyySv5lB73WU8mN483pLMJlQBrnd6nbsCiFAK1baq+76h+ed42o1+r3LRjefQX6b G6++dCErGVdulAekC0JBjDRG7iKxIAVMhXWP9tzq3/TU/fuHNEZ2k3OrmOD7HQV490Gf 1Ysy8FV6CLGcGozzN9Qrx8fMASwP7BbzXP0Wg72vaSyiahxYDuy5squB5BQqlQwnp3iV oj5FvqrXDx6C1Ldop/CBxgXEeTcm99H4b6TbzeYM9bdsOfiPzQJfLaZ016TJb92ULyLb hsSdb1NGYPT8273AqfggQdmz0Gs/bPqav/ISNj9zYI/Aqv/oBoFDAlXgevSnHaHyJ4cL MUQg== X-Gm-Message-State: APjAAAW2WTmKylApdFiCgDwAe4Ycjqa5Z+vY2m8ZZZ9mUxa8FbPixLcD sZ0Ur/rEcT72gck9kUSicQT6fg== X-Google-Smtp-Source: APXvYqyaLXo5MUyD+I9dO6HiLJFn2uWdH+8xt8jSkZ8bt7a48VQTUuqWsbmWyw/EGV6UxA2EsKJ0Ug== X-Received: by 2002:ac8:2209:: with SMTP id o9mr27065091qto.246.1574643226463; Sun, 24 Nov 2019 16:53:46 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-162-113-180.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.113.180]) by smtp.gmail.com with ESMTPSA id o124sm2535273qkf.66.2019.11.24.16.53.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 24 Nov 2019 16:53:45 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1iZ2dH-00020J-3s; Sun, 24 Nov 2019 20:53:39 -0400 Date: Sun, 24 Nov 2019 20:53:39 -0400 From: Jason Gunthorpe To: John Hubbard Subject: Re: [PATCH v7 07/24] IB/umem: use get_user_pages_fast() to pin DMA pages Message-ID: <20191125005339.GC5634@ziepe.ca> References: <20191121071354.456618-1-jhubbard@nvidia.com> <20191121071354.456618-8-jhubbard@nvidia.com> <20191121080746.GC30991@infradead.org> <20191121143643.GC7448@ziepe.ca> <20191124100724.GH136476@unreal> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michal Hocko , Jan Kara , kvm@vger.kernel.org, linux-doc@vger.kernel.org, David Airlie , Dave Chinner , dri-devel@lists.freedesktop.org, LKML , linux-mm@kvack.org, Paul Mackerras , linux-kselftest@vger.kernel.org, Ira Weiny , Leon Romanovsky , Jonathan Corbet , linux-rdma@vger.kernel.org, Christoph Hellwig , Vlastimil Babka , =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= , linux-media@vger.kernel.org, Shuah Khan , linux-block@vger.kernel.org, =?utf-8?B?SsOpcsO0bWU=?= Glisse , Al Viro , Dan Williams , Mauro Carvalho Chehab , bpf@vger.kernel.org, Magnus Karlsson , Jens Axboe , netdev@vger.kernel.org, Alex Williamson , Daniel Vetter , linux-fsdevel@vger.kernel.org, Andrew Morton , linuxppc-dev@lists.ozlabs.org, "David S . Miller" , Mike Kravetz Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Sun, Nov 24, 2019 at 04:05:16PM -0800, John Hubbard wrote: > I looked into this, and I believe that the problem is in gup.c. There appears to > have been an oversight, in commit 817be129e6f2 ("mm: validate get_user_pages_fast > flags"), in filtering out FOLL_FORCE. There is nothing in the _fast() implementation > that requires that we avoid writing to the pages. I think it is too late to be doing these kinds of changes, I will revert the patch and this will miss this merge window. Jason >From ec6cb45292d21d1af9b9d95997b8cf204bbe854c Mon Sep 17 00:00:00 2001 From: Jason Gunthorpe Date: Sun, 24 Nov 2019 20:47:59 -0400 Subject: [PATCH] Revert "IB/umem: use get_user_pages_fast() to pin DMA pages" This reverts commit c9a7a2ed837c563f9f89743a6db732591cb4035b. This was merged before enough testing was done, and it triggers a WARN_ON() in get_user_pages_fast(): WARNING: CPU: 1 PID: 2557 at mm/gup.c:2404 get_user_pages_fast+0x115/0x180 Call Trace: ib_umem_get+0x298/0x550 [ib_uverbs] mlx5_ib_db_map_user+0xad/0x130 [mlx5_ib] mlx5_ib_create_cq+0x1e8/0xaa0 [mlx5_ib] create_cq+0x1c8/0x2d0 [ib_uverbs] ib_uverbs_create_cq+0x70/0xa0 [ib_uverbs] ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0xc2/0xf0 [ib_uverbs] ib_uverbs_cmd_verbs.isra.6+0x5be/0xbe0 [ib_uverbs] ? uverbs_disassociate_api+0xd0/0xd0 [ib_uverbs] ? kvm_clock_get_cycles+0xd/0x10 ? kmem_cache_alloc+0x176/0x1c0 ? filemap_map_pages+0x18c/0x350 ib_uverbs_ioctl+0xc0/0x120 [ib_uverbs] do_vfs_ioctl+0xa1/0x610 ksys_ioctl+0x70/0x80 __x64_sys_ioctl+0x16/0x20 do_syscall_64+0x42/0x110 entry_SYSCALL_64_after_hwframe+0x44/0xa9 2404 if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM))) 2405 return -EINVAL; While we think this WARN_ON is probably bogus, resolving this will have to wait. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/umem.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 214e87aa609d6e..7a3b99597eada1 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -266,13 +266,16 @@ struct ib_umem *ib_umem_get(struct ib_udata *udata, unsigned long addr, sg = umem->sg_head.sgl; while (npages) { - ret = get_user_pages_fast(cur_base, - min_t(unsigned long, npages, - PAGE_SIZE / - sizeof(struct page *)), - gup_flags | FOLL_LONGTERM, page_list); - if (ret < 0) + down_read(&mm->mmap_sem); + ret = get_user_pages(cur_base, + min_t(unsigned long, npages, + PAGE_SIZE / sizeof (struct page *)), + gup_flags | FOLL_LONGTERM, + page_list, NULL); + if (ret < 0) { + up_read(&mm->mmap_sem); goto umem_release; + } cur_base += ret * PAGE_SIZE; npages -= ret; @@ -280,6 +283,8 @@ struct ib_umem *ib_umem_get(struct ib_udata *udata, unsigned long addr, sg = ib_umem_add_sg_table(sg, page_list, ret, dma_get_max_seg_size(context->device->dma_device), &umem->sg_nents); + + up_read(&mm->mmap_sem); } sg_mark_end(sg); -- 2.24.0 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Gunthorpe Subject: Re: [PATCH v7 07/24] IB/umem: use get_user_pages_fast() to pin DMA pages Date: Sun, 24 Nov 2019 20:53:39 -0400 Message-ID: <20191125005339.GC5634@ziepe.ca> References: <20191121071354.456618-1-jhubbard@nvidia.com> <20191121071354.456618-8-jhubbard@nvidia.com> <20191121080746.GC30991@infradead.org> <20191121143643.GC7448@ziepe.ca> <20191124100724.GH136476@unreal> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: Sender: netdev-owner@vger.kernel.org To: John Hubbard Cc: Leon Romanovsky , Christoph Hellwig , Andrew Morton , Al Viro , Alex Williamson , Benjamin Herrenschmidt , =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= , Dan Williams , Daniel Vetter , Dave Chinner , David Airlie , "David S . Miller" , Ira Weiny , Jan Kara , Jens Axboe , Jonathan Corbet , =?utf-8?B?SsOpcsO0bWU=?= Glisse , Magnus Karlsson , Mauro Carvalho Chehab , Mic List-Id: dri-devel@lists.freedesktop.org On Sun, Nov 24, 2019 at 04:05:16PM -0800, John Hubbard wrote: > I looked into this, and I believe that the problem is in gup.c. There appears to > have been an oversight, in commit 817be129e6f2 ("mm: validate get_user_pages_fast > flags"), in filtering out FOLL_FORCE. There is nothing in the _fast() implementation > that requires that we avoid writing to the pages. I think it is too late to be doing these kinds of changes, I will revert the patch and this will miss this merge window. Jason >>From ec6cb45292d21d1af9b9d95997b8cf204bbe854c Mon Sep 17 00:00:00 2001 From: Jason Gunthorpe Date: Sun, 24 Nov 2019 20:47:59 -0400 Subject: [PATCH] Revert "IB/umem: use get_user_pages_fast() to pin DMA pages" This reverts commit c9a7a2ed837c563f9f89743a6db732591cb4035b. This was merged before enough testing was done, and it triggers a WARN_ON() in get_user_pages_fast(): WARNING: CPU: 1 PID: 2557 at mm/gup.c:2404 get_user_pages_fast+0x115/0x180 Call Trace: ib_umem_get+0x298/0x550 [ib_uverbs] mlx5_ib_db_map_user+0xad/0x130 [mlx5_ib] mlx5_ib_create_cq+0x1e8/0xaa0 [mlx5_ib] create_cq+0x1c8/0x2d0 [ib_uverbs] ib_uverbs_create_cq+0x70/0xa0 [ib_uverbs] ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0xc2/0xf0 [ib_uverbs] ib_uverbs_cmd_verbs.isra.6+0x5be/0xbe0 [ib_uverbs] ? uverbs_disassociate_api+0xd0/0xd0 [ib_uverbs] ? kvm_clock_get_cycles+0xd/0x10 ? kmem_cache_alloc+0x176/0x1c0 ? filemap_map_pages+0x18c/0x350 ib_uverbs_ioctl+0xc0/0x120 [ib_uverbs] do_vfs_ioctl+0xa1/0x610 ksys_ioctl+0x70/0x80 __x64_sys_ioctl+0x16/0x20 do_syscall_64+0x42/0x110 entry_SYSCALL_64_after_hwframe+0x44/0xa9 2404 if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM))) 2405 return -EINVAL; While we think this WARN_ON is probably bogus, resolving this will have to wait. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/umem.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 214e87aa609d6e..7a3b99597eada1 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -266,13 +266,16 @@ struct ib_umem *ib_umem_get(struct ib_udata *udata, unsigned long addr, sg = umem->sg_head.sgl; while (npages) { - ret = get_user_pages_fast(cur_base, - min_t(unsigned long, npages, - PAGE_SIZE / - sizeof(struct page *)), - gup_flags | FOLL_LONGTERM, page_list); - if (ret < 0) + down_read(&mm->mmap_sem); + ret = get_user_pages(cur_base, + min_t(unsigned long, npages, + PAGE_SIZE / sizeof (struct page *)), + gup_flags | FOLL_LONGTERM, + page_list, NULL); + if (ret < 0) { + up_read(&mm->mmap_sem); goto umem_release; + } cur_base += ret * PAGE_SIZE; npages -= ret; @@ -280,6 +283,8 @@ struct ib_umem *ib_umem_get(struct ib_udata *udata, unsigned long addr, sg = ib_umem_add_sg_table(sg, page_list, ret, dma_get_max_seg_size(context->device->dma_device), &umem->sg_nents); + + up_read(&mm->mmap_sem); } sg_mark_end(sg); -- 2.24.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 352C8C432C0 for ; Mon, 25 Nov 2019 08:08:34 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 123BC207FD for ; Mon, 25 Nov 2019 08:08:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 123BC207FD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CFE8F89F6B; Mon, 25 Nov 2019 08:08:18 +0000 (UTC) Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by gabe.freedesktop.org (Postfix) with ESMTPS id C719F89CB9 for ; Mon, 25 Nov 2019 00:53:47 +0000 (UTC) Received: by mail-qt1-x842.google.com with SMTP id 14so15320381qtf.5 for ; Sun, 24 Nov 2019 16:53:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=W5KvmIUsZXoQMVbzDoxYnt7yT+pQC9oHjmWUHowLIDE=; b=fcprTrxLsqZ/DDehbX3kNID5KKn0p3ukaHtlR3gM7KyXpoAgq6UobDZJH1+3nX2+H0 YuZL+NBZis2Sf/hJFrN1b/64ZF30NuwcAmfcP48VSnxVJezP9dgSo53QUChhLYO/b9fM RLspcroLXFFIOWHJapCIEdQ95EPUJmpNc2BdvnV2QwnmZedx4uQAIZNVkri1FKvAVJBe RWpQy3YIJpKzA8+vJA4TxKmhWsBaUoV9iF2bZg9WR+7N+2a/p57HhVZELfPOO2jyr9MT /lNb6DilXf6Z7/sbGHfJfFBoTBOL+Bc8zfJh1Yu+1aA3o391iFH8dEuY8L9pII+u6Q/y 4+XA== X-Gm-Message-State: APjAAAVdzlYkdhyczgmstN1z2mCuYSUQR4lqaZ6Q6wc+xw/dBAMyeLfb 6HeDIWe1ywzyQocBdmtGLLtX3w== X-Google-Smtp-Source: APXvYqyaLXo5MUyD+I9dO6HiLJFn2uWdH+8xt8jSkZ8bt7a48VQTUuqWsbmWyw/EGV6UxA2EsKJ0Ug== X-Received: by 2002:ac8:2209:: with SMTP id o9mr27065091qto.246.1574643226463; Sun, 24 Nov 2019 16:53:46 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-162-113-180.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.113.180]) by smtp.gmail.com with ESMTPSA id o124sm2535273qkf.66.2019.11.24.16.53.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 24 Nov 2019 16:53:45 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1iZ2dH-00020J-3s; Sun, 24 Nov 2019 20:53:39 -0400 Date: Sun, 24 Nov 2019 20:53:39 -0400 From: Jason Gunthorpe To: John Hubbard Subject: Re: [PATCH v7 07/24] IB/umem: use get_user_pages_fast() to pin DMA pages Message-ID: <20191125005339.GC5634@ziepe.ca> References: <20191121071354.456618-1-jhubbard@nvidia.com> <20191121071354.456618-8-jhubbard@nvidia.com> <20191121080746.GC30991@infradead.org> <20191121143643.GC7448@ziepe.ca> <20191124100724.GH136476@unreal> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Mailman-Approved-At: Mon, 25 Nov 2019 08:07:41 +0000 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=W5KvmIUsZXoQMVbzDoxYnt7yT+pQC9oHjmWUHowLIDE=; b=bf4wQj5hUYAUJ6J+IG0+hDMiHwo0y37jHBHT8UdhIIsQlzBQ4CxdtIHJrRj51DxlRK CKLiYZNarSFFj4FEgRzMNxy+kI0kMWkp2JP+w0At3xJ5awQQfKCflGDMEbLjgI1axJbX hAKULgjdId2ND6JafSG19IZ76B47iyKuc//kc4PjYIOLOtUHlwtDFurR95sHJjXvaqXY fGrur8LfkpaaQNAoNniODg3+dsAf2S2GrC+6cEhUlICPkA/CU1JiTx+UkO0BkI15CMLp 7s5O7Rwk6bHEYMLHf8Cgu9Ix5CmyXm+g4xBp1CqvpB7TPmrxsr5ItFDOP1iwxt6Ai6Gn z9sA== X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michal Hocko , Jan Kara , kvm@vger.kernel.org, linux-doc@vger.kernel.org, David Airlie , Dave Chinner , dri-devel@lists.freedesktop.org, LKML , linux-mm@kvack.org, Paul Mackerras , linux-kselftest@vger.kernel.org, Ira Weiny , Leon Romanovsky , Jonathan Corbet , linux-rdma@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Vlastimil Babka , =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= , linux-media@vger.kernel.org, Shuah Khan , linux-block@vger.kernel.org, =?utf-8?B?SsOpcsO0bWU=?= Glisse , Al Viro , Dan Williams , Mauro Carvalho Chehab , bpf@vger.kernel.org, Magnus Karlsson , Jens Axboe , netdev@vger.kernel.org, Alex Williamson , linux-fsdevel@vger.kernel.org, Andrew Morton , linuxppc-dev@lists.ozlabs.org, "David S . Miller" , Mike Kravetz Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Message-ID: <20191125005339.I5MlvG0ppS4WbmTbpsdcNtgVaQ__kwYiJTXqSSHTxuc@z> T24gU3VuLCBOb3YgMjQsIDIwMTkgYXQgMDQ6MDU6MTZQTSAtMDgwMCwgSm9obiBIdWJiYXJkIHdy b3RlOgogCj4gSSBsb29rZWQgaW50byB0aGlzLCBhbmQgSSBiZWxpZXZlIHRoYXQgdGhlIHByb2Js ZW0gaXMgaW4gZ3VwLmMuIFRoZXJlIGFwcGVhcnMgdG8KPiBoYXZlIGJlZW4gYW4gb3ZlcnNpZ2h0 LCBpbiBjb21taXQgODE3YmUxMjllNmYyICgibW06IHZhbGlkYXRlIGdldF91c2VyX3BhZ2VzX2Zh c3QKPiBmbGFncyIpLCBpbiBmaWx0ZXJpbmcgb3V0IEZPTExfRk9SQ0UuIFRoZXJlIGlzIG5vdGhp bmcgaW4gdGhlIF9mYXN0KCkgaW1wbGVtZW50YXRpb24KPiB0aGF0IHJlcXVpcmVzIHRoYXQgd2Ug YXZvaWQgd3JpdGluZyB0byB0aGUgcGFnZXMuCgpJIHRoaW5rIGl0IGlzIHRvbyBsYXRlIHRvIGJl IGRvaW5nIHRoZXNlIGtpbmRzIG9mIGNoYW5nZXMsIEkgd2lsbApyZXZlcnQgdGhlIHBhdGNoIGFu ZCB0aGlzIHdpbGwgbWlzcyB0aGlzIG1lcmdlIHdpbmRvdy4KCkphc29uCgpGcm9tIGVjNmNiNDUy OTJkMjFkMWFmOWI5ZDk1OTk3YjhjZjIwNGJiZTg1NGMgTW9uIFNlcCAxNyAwMDowMDowMCAyMDAx CkZyb206IEphc29uIEd1bnRob3JwZSA8amdnQG1lbGxhbm94LmNvbT4KRGF0ZTogU3VuLCAyNCBO b3YgMjAxOSAyMDo0Nzo1OSAtMDQwMApTdWJqZWN0OiBbUEFUQ0hdIFJldmVydCAiSUIvdW1lbTog dXNlIGdldF91c2VyX3BhZ2VzX2Zhc3QoKSB0byBwaW4gRE1BIHBhZ2VzIgoKVGhpcyByZXZlcnRz IGNvbW1pdCBjOWE3YTJlZDgzN2M1NjNmOWY4OTc0M2E2ZGI3MzI1OTFjYjQwMzViLgoKVGhpcyB3 YXMgbWVyZ2VkIGJlZm9yZSBlbm91Z2ggdGVzdGluZyB3YXMgZG9uZSwgYW5kIGl0IHRyaWdnZXJz IGEgV0FSTl9PTigpCmluIGdldF91c2VyX3BhZ2VzX2Zhc3QoKToKCiAgV0FSTklORzogQ1BVOiAx IFBJRDogMjU1NyBhdCBtbS9ndXAuYzoyNDA0IGdldF91c2VyX3BhZ2VzX2Zhc3QrMHgxMTUvMHgx ODAKICBDYWxsIFRyYWNlOgogICBpYl91bWVtX2dldCsweDI5OC8weDU1MCBbaWJfdXZlcmJzXQog ICBtbHg1X2liX2RiX21hcF91c2VyKzB4YWQvMHgxMzAgW21seDVfaWJdCiAgIG1seDVfaWJfY3Jl YXRlX2NxKzB4MWU4LzB4YWEwIFttbHg1X2liXQogICBjcmVhdGVfY3ErMHgxYzgvMHgyZDAgW2li X3V2ZXJic10KICAgaWJfdXZlcmJzX2NyZWF0ZV9jcSsweDcwLzB4YTAgW2liX3V2ZXJic10KICAg aWJfdXZlcmJzX2hhbmRsZXJfVVZFUkJTX01FVEhPRF9JTlZPS0VfV1JJVEUrMHhjMi8weGYwIFtp Yl91dmVyYnNdCiAgIGliX3V2ZXJic19jbWRfdmVyYnMuaXNyYS42KzB4NWJlLzB4YmUwIFtpYl91 dmVyYnNdCiAgID8gdXZlcmJzX2Rpc2Fzc29jaWF0ZV9hcGkrMHhkMC8weGQwIFtpYl91dmVyYnNd CiAgID8ga3ZtX2Nsb2NrX2dldF9jeWNsZXMrMHhkLzB4MTAKICAgPyBrbWVtX2NhY2hlX2FsbG9j KzB4MTc2LzB4MWMwCiAgID8gZmlsZW1hcF9tYXBfcGFnZXMrMHgxOGMvMHgzNTAKICAgaWJfdXZl cmJzX2lvY3RsKzB4YzAvMHgxMjAgW2liX3V2ZXJic10KICAgZG9fdmZzX2lvY3RsKzB4YTEvMHg2 MTAKICAga3N5c19pb2N0bCsweDcwLzB4ODAKICAgX194NjRfc3lzX2lvY3RsKzB4MTYvMHgyMAog ICBkb19zeXNjYWxsXzY0KzB4NDIvMHgxMTAKICAgZW50cnlfU1lTQ0FMTF82NF9hZnRlcl9od2Zy YW1lKzB4NDQvMHhhOQoKMjQwNCAgICAgICAgIGlmIChXQVJOX09OX09OQ0UoZ3VwX2ZsYWdzICYg fihGT0xMX1dSSVRFIHwgRk9MTF9MT05HVEVSTSkpKQoyNDA1ICAgICAgICAgICAgICAgICByZXR1 cm4gLUVJTlZBTDsKCldoaWxlIHdlIHRoaW5rIHRoaXMgV0FSTl9PTiBpcyBwcm9iYWJseSBib2d1 cywgcmVzb2x2aW5nIHRoaXMgd2lsbCBoYXZlIHRvCndhaXQuCgpTaWduZWQtb2ZmLWJ5OiBKYXNv biBHdW50aG9ycGUgPGpnZ0BtZWxsYW5veC5jb20+Ci0tLQogZHJpdmVycy9pbmZpbmliYW5kL2Nv cmUvdW1lbS5jIHwgMTcgKysrKysrKysrKystLS0tLS0KIDEgZmlsZSBjaGFuZ2VkLCAxMSBpbnNl cnRpb25zKCspLCA2IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2RyaXZlcnMvaW5maW5pYmFu ZC9jb3JlL3VtZW0uYyBiL2RyaXZlcnMvaW5maW5pYmFuZC9jb3JlL3VtZW0uYwppbmRleCAyMTRl ODdhYTYwOWQ2ZS4uN2EzYjk5NTk3ZWFkYTEgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvaW5maW5pYmFu ZC9jb3JlL3VtZW0uYworKysgYi9kcml2ZXJzL2luZmluaWJhbmQvY29yZS91bWVtLmMKQEAgLTI2 NiwxMyArMjY2LDE2IEBAIHN0cnVjdCBpYl91bWVtICppYl91bWVtX2dldChzdHJ1Y3QgaWJfdWRh dGEgKnVkYXRhLCB1bnNpZ25lZCBsb25nIGFkZHIsCiAJc2cgPSB1bWVtLT5zZ19oZWFkLnNnbDsK IAogCXdoaWxlIChucGFnZXMpIHsKLQkJcmV0ID0gZ2V0X3VzZXJfcGFnZXNfZmFzdChjdXJfYmFz ZSwKLQkJCQkJICBtaW5fdCh1bnNpZ25lZCBsb25nLCBucGFnZXMsCi0JCQkJCQlQQUdFX1NJWkUg LwotCQkJCQkJc2l6ZW9mKHN0cnVjdCBwYWdlICopKSwKLQkJCQkJICBndXBfZmxhZ3MgfCBGT0xM X0xPTkdURVJNLCBwYWdlX2xpc3QpOwotCQlpZiAocmV0IDwgMCkKKwkJZG93bl9yZWFkKCZtbS0+ bW1hcF9zZW0pOworCQlyZXQgPSBnZXRfdXNlcl9wYWdlcyhjdXJfYmFzZSwKKwkJCQkgICAgIG1p bl90KHVuc2lnbmVkIGxvbmcsIG5wYWdlcywKKwkJCQkJICAgUEFHRV9TSVpFIC8gc2l6ZW9mIChz dHJ1Y3QgcGFnZSAqKSksCisJCQkJICAgICBndXBfZmxhZ3MgfCBGT0xMX0xPTkdURVJNLAorCQkJ CSAgICAgcGFnZV9saXN0LCBOVUxMKTsKKwkJaWYgKHJldCA8IDApIHsKKwkJCXVwX3JlYWQoJm1t LT5tbWFwX3NlbSk7CiAJCQlnb3RvIHVtZW1fcmVsZWFzZTsKKwkJfQogCiAJCWN1cl9iYXNlICs9 IHJldCAqIFBBR0VfU0laRTsKIAkJbnBhZ2VzICAgLT0gcmV0OwpAQCAtMjgwLDYgKzI4Myw4IEBA IHN0cnVjdCBpYl91bWVtICppYl91bWVtX2dldChzdHJ1Y3QgaWJfdWRhdGEgKnVkYXRhLCB1bnNp Z25lZCBsb25nIGFkZHIsCiAJCXNnID0gaWJfdW1lbV9hZGRfc2dfdGFibGUoc2csIHBhZ2VfbGlz dCwgcmV0LAogCQkJZG1hX2dldF9tYXhfc2VnX3NpemUoY29udGV4dC0+ZGV2aWNlLT5kbWFfZGV2 aWNlKSwKIAkJCSZ1bWVtLT5zZ19uZW50cyk7CisKKwkJdXBfcmVhZCgmbW0tPm1tYXBfc2VtKTsK IAl9CiAKIAlzZ19tYXJrX2VuZChzZyk7Ci0tIAoyLjI0LjAKCl9fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fCmRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKZHJpLWRl dmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9t YWlsbWFuL2xpc3RpbmZvL2RyaS1kZXZlbA==