From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, USER_AGENT_MUTT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49BF1C169C4 for ; Mon, 11 Feb 2019 22:22:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0ED6F218AD for ; Mon, 11 Feb 2019 22:22:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="C3ya1lIk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727054AbfBKWWL (ORCPT ); Mon, 11 Feb 2019 17:22:11 -0500 Received: from mail-pl1-f196.google.com ([209.85.214.196]:39128 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726300AbfBKWWK (ORCPT ); Mon, 11 Feb 2019 17:22:10 -0500 Received: by mail-pl1-f196.google.com with SMTP id 101so228360pld.6 for ; Mon, 11 Feb 2019 14:22:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=xdkSFtkXI2U9rILZa0OLPibj9Ly6nXCAyBPxVjPqxVk=; b=C3ya1lIkET7B0U5gcbeCUe1y0Gh1eANZFR2GVVA06FR4dychps4f5xAXwwn+RUT3KS rCUYYspMMPl5cOI3Rj0TobbZYURtvSEHzR9DYTIHVqrDbrimv7ulWx9vjEaVi+k2WrAr UK14ouUMB4UNsY/c7qiTfBE/Wz42BgP9X6oZLVX/9pisMzhgp3lhsLXGWX8ikwzdtJn0 QWb4KFB1F4fMRDxqD8DjgaB/qW85iPXcp8oOXGNsd4NvBa6eLN3luBJQMmr8KVy0tO/+ 3hxHhSkliZu4xmI2bPusnBUX8HLKGjG63DBkCshUQIYGgALQHI1Ha7816zP8kP0qfizK 8Rzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=xdkSFtkXI2U9rILZa0OLPibj9Ly6nXCAyBPxVjPqxVk=; b=nh7+pwCQ1SgSbrJ10YT6/+SScDbSanwVgZJ2WzD7v94eRhdeo/F8hiCqismVXMKUvq sPXTK7Uf3VwT04k0N+08CA2/yoQxUeby/hUvihch1AHn4RTi8x+UJ3O0p3s/n6pClyuR RwCHD5ZbvZsnaORLepoDJ3GsFalhsqVCVOP4aFoxUwA8OM71P94n5vTaS2CmuzIa/LEL EaF2AgErPLpxXQS3xJMV0OZZ4mJ/vwzdfFcFKbeIZFPw6NDbSQPxG+K3X3dfmyAghh1R 6MunCKgCsi7W1YV+oMJ3Dot6JN1quCeBtp9JhCDb9lF5lH07okqFLnOPway6J8XGOafe fVKw== X-Gm-Message-State: AHQUAubP7lYXdSPiDbIti8wJnnnBYYwOKFz3+geLghv+HF04QOry37fl 8xmUY7lIoh5SpJq7Hc83mxgZ2A== X-Google-Smtp-Source: AHgI3IYw7BALaGSULNv+3bUMQP+ikh8t2IwpZG1CncFiHzfzaPM812aozbs4PkbsejgksaVI4K2aBQ== X-Received: by 2002:a17:902:a588:: with SMTP id az8mr536627plb.77.1549923730086; Mon, 11 Feb 2019 14:22:10 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id h15sm13896692pgl.43.2019.02.11.14.22.09 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 11 Feb 2019 14:22:09 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gtJxo-0003WM-EN; Mon, 11 Feb 2019 15:22:08 -0700 Date: Mon, 11 Feb 2019 15:22:08 -0700 From: Jason Gunthorpe To: Ira Weiny Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Daniel Borkmann , netdev@vger.kernel.org, Mike Marciniszyn , Dennis Dalessandro , Doug Ledford , Andrew Morton , "Kirill A. Shutemov" , Dan Williams Subject: Re: [PATCH 0/3] Add gup fast + longterm and use it in HFI1 Message-ID: <20190211222208.GJ24692@ziepe.ca> References: <20190211201643.7599-1-ira.weiny@intel.com> <20190211203417.a2c2kbmjai43flyz@linux-r8p5> <20190211204710.GE24692@ziepe.ca> <20190211214257.GA7891@iweiny-DESK2.sc.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190211214257.GA7891@iweiny-DESK2.sc.intel.com> User-Agent: Mutt/1.9.4 (2018-02-28) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Mon, Feb 11, 2019 at 01:42:57PM -0800, Ira Weiny wrote: > On Mon, Feb 11, 2019 at 01:47:10PM -0700, Jason Gunthorpe wrote: > > On Mon, Feb 11, 2019 at 12:34:17PM -0800, Davidlohr Bueso wrote: > > > On Mon, 11 Feb 2019, ira.weiny@intel.com wrote: > > > > Ira Weiny (3): > > > > mm/gup: Change "write" parameter to flags > > > > mm/gup: Introduce get_user_pages_fast_longterm() > > > > IB/HFI1: Use new get_user_pages_fast_longterm() > > > > > > Out of curiosity, are you planning on having all rdma drivers > > > use get_user_pages_fast_longterm()? Ie: > > > > > > hw/mthca/mthca_memfree.c: ret = get_user_pages_fast(uaddr & PAGE_MASK, 1, FOLL_WRITE, pages); > > > > This one is certainly a mistake - this should be done with a umem. > > It looks like this is mapping a page allocated by user space for a > doorbell?!?! Many drivers do this, the 'doorbell' is a PCI -> CPU thing of some sort > This does not seem to be allocating memory regions. Jason, do you > want a patch to just convert these calls and consider it legacy > code? It needs to use umem like all the other drivers on this path. Otherwise it doesn't get the page pinning logic right There is also something else rotten with these longterm callsites, they seem to have very different ideas how to handle RLIMIT_MEMLOCK. ie vfio doesn't even touch pinned_vm.. and rdma is applying RLIMIT_MEMLOCK to mm->pinned_vm, while vfio is using locked_vm.. No idea which is right, but they should be the same, and this pattern should probably be in core code someplace. Jason