From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7E69C43381 for ; Thu, 14 Feb 2019 20:26:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A1FD32080A for ; Thu, 14 Feb 2019 20:26:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2439940AbfBNU00 (ORCPT ); Thu, 14 Feb 2019 15:26:26 -0500 Received: from mx1.redhat.com ([209.132.183.28]:54860 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387975AbfBNU00 (ORCPT ); Thu, 14 Feb 2019 15:26:26 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 22DF332FBC; Thu, 14 Feb 2019 20:26:26 +0000 (UTC) Received: from redhat.com (unknown [10.20.6.236]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A588B600C1; Thu, 14 Feb 2019 20:26:24 +0000 (UTC) Date: Thu, 14 Feb 2019 15:26:22 -0500 From: Jerome Glisse To: Jason Gunthorpe Cc: Dan Williams , Jan Kara , Dave Chinner , Christopher Lameter , Doug Ledford , Matthew Wilcox , Ira Weiny , lsf-pc@lists.linux-foundation.org, linux-rdma , Linux MM , Linux Kernel Mailing List , John Hubbard , Michal Hocko Subject: Re: [LSF/MM TOPIC] Discuss least bad options for resolving longterm-GUP usage by RDMA Message-ID: <20190214202622.GB3420@redhat.com> References: <0c868bc615a60c44d618fb0183fcbe0c418c7c83.camel@redhat.com> <01000168c8e2de6b-9ab820ed-38ad-469c-b210-60fcff8ea81c-000000@email.amazonses.com> <20190208044302.GA20493@dastard> <20190208111028.GD6353@quack2.suse.cz> <20190211102402.GF19029@quack2.suse.cz> <20190211180654.GB24692@ziepe.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20190211180654.GB24692@ziepe.ca> User-Agent: Mutt/1.10.0 (2018-05-17) X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Thu, 14 Feb 2019 20:26:26 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Feb 11, 2019 at 11:06:54AM -0700, Jason Gunthorpe wrote: > On Mon, Feb 11, 2019 at 09:22:58AM -0800, Dan Williams wrote: > > > I honestly don't like the idea that random subsystems can pin down > > file blocks as a side effect of gup on the result of mmap. Recall that > > it's not just RDMA that wants this guarantee. It seems safer to have > > the file be in an explicit block-allocation-immutable-mode so that the > > fallocate man page can describe this error case. Otherwise how would > > you describe the scenarios under which FALLOC_FL_PUNCH_HOLE fails? > > I rather liked CL's version of this - ftruncate/etc is simply racing > with a parallel pwrite - and it doesn't fail. > > But it also doesnt' trucate/create a hole. Another thread wrote to it > right away and the 'hole' was essentially instantly reallocated. This > is an inherent, pre-existing, race in the ftrucate/etc APIs. So it is kind of a // point to this, but direct I/O do "truncate" pages or more exactly after a write direct I/O invalidate_inode_pages2_range() is call and it will try to unmap and remove from page cache all pages that have been written too. So we probably want to think about what we want to do here if a device like RDMA has also pin those pages. Do we want to abort the invalidate ? Which would mean that then the direct I/O write was just a pointless exercise. Do we want to not direct I/O but instead memcpy into page cache memory ? Then you are just ignoring the direct I/O property of the write. Or do we want to both direct I/O to the block and also do a memcopy to the page so that we preserve the direct I/O semantics ? I would probably go with the last one. In any cases we will need to update the direct I/O code to handle GUPed page cache page. Cheers, Jérôme