From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D37F5C43334 for ; Tue, 28 Jun 2022 08:20:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0A24F8E0002; Tue, 28 Jun 2022 04:20:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 051F58E0001; Tue, 28 Jun 2022 04:20:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E5C078E0002; Tue, 28 Jun 2022 04:20:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D67DE8E0001 for ; Tue, 28 Jun 2022 04:20:50 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A665D2054C for ; Tue, 28 Jun 2022 08:20:50 +0000 (UTC) X-FDA: 79626948660.18.F1CE5E6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 452C3A002B for ; Tue, 28 Jun 2022 08:20:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1656404449; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ncuimMgu6pl/bxccvatpC+AvPLsScqnjbUVdgB7ZYmM=; b=Q8q9mbDjRPVP5/nWXxy1hnO5w6HOZr0/BKUR0y7bxO/jO1w6vbMKFUg+cWPY1fRJTrE4Vz AjhgS3UhkhiJB5HwFHcOop24GT6ZAoq7vneMbO0S7pGkPuwIBEDEmmdWBIoQpSZ4KpHCEN CKlXMKCg1J33D50ysX0VnAcuwLpniaQ= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-101-_z2Zhf-_M36I-hB9SlsJ0Q-1; Tue, 28 Jun 2022 04:20:46 -0400 X-MC-Unique: _z2Zhf-_M36I-hB9SlsJ0Q-1 Received: by mail-wr1-f72.google.com with SMTP id w12-20020adf8bcc000000b0021d20a5b24fso184221wra.22 for ; Tue, 28 Jun 2022 01:20:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=ncuimMgu6pl/bxccvatpC+AvPLsScqnjbUVdgB7ZYmM=; b=0Ng1OZOu9ualMZzuY0ZK4z32XNGAM2rjq0REx8gjiuDWXzIR5ZivZkyc1oHLM8RzKm 3KdQf7grNjBpLwlS935KwSA0yJvqzQHQmZyPEyz3avuYN+jXHzLXKOS+eYa/DNtUi65O qnpsCtnZ6siiZlPHMactkaFslqkyGZm+74sLizexlyOmXNekZU0yKDT4e2iQv/FUvKEU 7j7/APr3qQLXGCldZyJpX1nMQvMNlEfTICtfiFLa8OOjj1BPRzD5MGiOQHtY2yswAD6B ezIYgQgFxaRR2buJ6PJKHCBacyvZdwnKpaeVA20PeBqRjN1XPLk5xyvX+ALUT6iqjVZm YUmQ== X-Gm-Message-State: AJIora/ci/EyKkgqXESKWrIa2mB1sfGbqQHKezKqEMT7e3rIWiIXjcE6 oWjg0jJ208KHTVXKHJ6WPJNA22EE0ViGLsmhFzX9q0aAvMbVVz+Slmqa+6Q275WA4+qLL2wbER3 7yVgZvMzlvPo= X-Received: by 2002:a05:6000:1251:b0:21a:efae:6cbe with SMTP id j17-20020a056000125100b0021aefae6cbemr15965715wrx.281.1656404445297; Tue, 28 Jun 2022 01:20:45 -0700 (PDT) X-Google-Smtp-Source: AGRyM1uFs5cqhURYlQN3b9jrNVSRJIonINM5X+1RgZ3OW45c0RMzsf1qhtcrAfdCV4vgV+au5tBFvA== X-Received: by 2002:a05:6000:1251:b0:21a:efae:6cbe with SMTP id j17-20020a056000125100b0021aefae6cbemr15965679wrx.281.1656404445020; Tue, 28 Jun 2022 01:20:45 -0700 (PDT) Received: from work-vm (cpc109025-salf6-2-0-cust480.10-2.cable.virginm.net. [82.30.61.225]) by smtp.gmail.com with ESMTPSA id f13-20020a05600c154d00b0039ee391a024sm22785542wmg.14.2022.06.28.01.20.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jun 2022 01:20:43 -0700 (PDT) Date: Tue, 28 Jun 2022 09:20:41 +0100 From: "Dr. David Alan Gilbert" To: James Houghton Cc: Matthew Wilcox , Mike Kravetz , Muchun Song , Peter Xu , David Hildenbrand , David Rientjes , Axel Rasmussen , Mina Almasry , Jue Wang , Manish Mishra , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nadav Amit Subject: Re: [RFC PATCH 00/26] hugetlb: Introduce HugeTLB high-granularity mapping Message-ID: References: <20220624173656.2033256-1-jthoughton@google.com> MIME-Version: 1.0 In-Reply-To: User-Agent: Mutt/2.2.6 (2022-06-05) X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656404450; a=rsa-sha256; cv=none; b=T5Sl+UVM3JZgHkMNjMgL0SBtlU99NdRGqam6/01CpOUtKpE1u6ubZ5d8OoJF85MMGpScXm E7QlBnahB8uVPAITvD6/gC1aSBIrE8cJ5sNnC00idB5syqX4E7G3p165RNmnP5/v6a7YGf l+/mr3OJ/az4Rtyrv5s1o7x/YzxvATI= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Q8q9mbDj; spf=none (imf25.hostedemail.com: domain of dgilbert@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=dgilbert@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656404450; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ncuimMgu6pl/bxccvatpC+AvPLsScqnjbUVdgB7ZYmM=; b=LCgHCIHMM74uKqyAZaolpt0UrNL8jCPw7XRypu9dUpE6VSm0Yadm4oBUVrwWNGa+7eLdai qVVT/tUO+8FaZZB2kpFAneLMWdSLc8mX8TCyztF1jFtInhvgJFXCe23hAh+j+T1217q7FO euZz7qM5qH5PbSE9vMa7eEHQ4l4Lz8c= X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 452C3A002B Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Q8q9mbDj; spf=none (imf25.hostedemail.com: domain of dgilbert@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=dgilbert@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspam-User: X-Stat-Signature: txbn17sst5suae8ayppnfi781knbpygq X-HE-Tag: 1656404450-347246 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: * James Houghton (jthoughton@google.com) wrote: > On Mon, Jun 27, 2022 at 10:56 AM Dr. David Alan Gilbert > wrote: > > > > * James Houghton (jthoughton@google.com) wrote: > > > On Fri, Jun 24, 2022 at 11:29 AM Matthew Wilcox wrote: > > > > > > > > On Fri, Jun 24, 2022 at 05:36:30PM +0000, James Houghton wrote: > > > > > [1] This used to be called HugeTLB double mapping, a bad and confusing > > > > > name. "High-granularity mapping" is not a great name either. I am open > > > > > to better names. > > > > > > > > Oh good, I was grinding my teeth every time I read it ;-) > > > > > > > > How does "Fine granularity" work for you? > > > > "sub-page mapping" might work too. > > > > > > "Granularity", as I've come to realize, is hard to say, so I think I > > > prefer sub-page mapping. :) So to recap the suggestions I have so far: > > > > > > 1. Sub-page mapping > > > 2. Granular mapping > > > 3. Flexible mapping > > > > > > I'll pick one of these (or maybe some other one that works better) for > > > the next version of this series. > > > > Just a name; SPM might work (although may confuse those > > architectures which had subprotection for normal pages), and at least > > we can mispronounce it. > > > > In 14/26 your commit message says: > > > > 1. Faults can be passed to handle_userfault. (Userspace will want to > > use UFFD_FEATURE_REAL_ADDRESS to get the real address to know which > > region they should be call UFFDIO_CONTINUE on later.) > > > > can you explain what that new UFFD_FEATURE does? > > +cc Nadav Amit to check me here. > > Sorry, this should be UFFD_FEATURE_EXACT_ADDRESS. It isn't a new > feature, and it actually isn't needed (I will correct the commit > message). Why it isn't needed is a little bit complicated, though. Let > me explain: > > Before UFFD_FEATURE_EXACT_ADDRESS was introduced, the address that > userfaultfd gave userspace for HugeTLB pages was rounded down to be > hstate-size-aligned. This would have had to change, because userspace, > to take advantage of HGM, needs to know which 4K piece to install. > > However, after UFFD_FEATURE_EXACT_ADDRESS was introduced[1], the > address was rounded down to be PAGE_SIZE-aligned instead, even if the > flag wasn't used. I think this was an unintended change. If the flag > is used, then the address isn't rounded at all -- that was the > intended purpose of this flag. Hope that makes sense. Oh that's 'fun'; right but the need for the less-rounded address makes sense. One other thing I thought of; you provide the modified 'CONTINUE' behaviour, which works for postcopy as long as you use two mappings in userspace; one protected by userfault, and one which you do the writes to, and then issue the CONTINUE into the protected mapping; that's fine, but it's not currently how we have our postcopy code wired up in qemu, we have one mapping and use UFFDIO_COPY to place the page. Requiring the two mappings is fine, but it's probably worth pointing out the need for it somewhere. Dave > The new userfaultfd feature, UFFD_FEATURE_MINOR_HUGETLBFS_HGM, informs > userspace that high-granularity CONTINUEs are available. > > [1] commit 824ddc601adc ("userfaultfd: provide unmasked address on page-fault") > > > > > > Dave > > > > -- > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK