From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D8678CD128A for ; Wed, 10 Apr 2024 19:59:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=61Htl/CnnLi2BcwD30jEtpKGmMKB0RP/uA2fVpSdm/E=; b=z20L3aBrGY4l3i R0xH8Q8a7Rj+Os9eSi2pI4LZXKfhRnrnbwh7yPqEaRbhKIANMnLwmXsFEAkiLY3EvX9egr/uEKqo/ iak4AJ1gLk1sooOOL3wanlfUx5B2tdPHtzclzLQDVv10a41Em/wQwUmTAtL8HMgaq9byV1y7wGL2E z6MMr4qnM8uKGMwdWnsrnPDRk6Tfzlz37mMhQVLDJKAc9dBSgqNjXNFSpQV8siBQ/DOim/2mcOZPH K6hen62yMNnHq0INQU+z7b1nQMjOODN8WFA5eJcVN/9aqnbLVK2JquEPqYTvkTp79WFbE1NrlFn/t R9c6rouckzyURo0pNL0g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rue60-00000008leT-2SnO; Wed, 10 Apr 2024 19:59:00 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rue5u-00000008lc8-3QbZ for linux-riscv@lists.infradead.org; Wed, 10 Apr 2024 19:58:58 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712779133; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wxs+2ga91aRZGeZs4inMwPcuYTjJzWrBvKW8/ZIuNzA=; b=D/pED3XPzuVAx3i/ucgy3FKgEoQczaorNR0kiIi5ViBSexiRTmvR3+mF898M0p2oxsCNTv 7uNrrnb58HrzP8Hyms9fs7+D7cgQnAMVA3nVIHfwzwt4ervIQ+E3adZkBXVOOY8fY23nfo UlB9/G10hGJ+WV2O0rkcY5Uk3t8XvqU= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-456-94B5mkXoN1CBV3SeEl-n1w-1; Wed, 10 Apr 2024 15:58:51 -0400 X-MC-Unique: 94B5mkXoN1CBV3SeEl-n1w-1 Received: by mail-qv1-f71.google.com with SMTP id 6a1803df08f44-69b147e856aso15026926d6.1 for ; Wed, 10 Apr 2024 12:58:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712779131; x=1713383931; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wxs+2ga91aRZGeZs4inMwPcuYTjJzWrBvKW8/ZIuNzA=; b=NGt3vH9q4j6NtdDNvwpMPbP9SwmJ4JyiqUbTNTZNDMMKhkFJx1podMovmxZyO9r+j7 U0lhdWE81qcFY86p7bnkAP7BIUfkneqGjHffFfNSWxLVSkyRAGXH9j8ZFnpN8BoVuoEC AALnoweT+BQG9HHCkzveDMGexZKMwWFv4n0QdlEqOKT+L79flo718sbqOxrk6m1B4KMX 0FogJAymjc8nZcIX9nPmBsbFc+2nfdbPuw2mr9GG9+zJppYVBcPq50sKMo/saSS0xj1a j4n0ArXdmXohyXX+YpWIRu7Q+yNp1JS8OWw0gDqIE5QeeXzgOZiW5UWMimSqA5wg+R11 3h5A== X-Forwarded-Encrypted: i=1; AJvYcCUcquS0Nm71keHyEeJ2B3z1Dm9SJW2lIEmS60CpgIXs6lklIHWdoUkPJNLsTtUFSPUJMPtVtekGQzz+VdCZsM9KQc2DuP03sx/5SgqTs4Vg X-Gm-Message-State: AOJu0YyfcnMIpTrII60zH/LpccpXCLdqPhuyIWVpX2P/I4Y+E5QikjQd lq7qXMAEOWFUUxWdVwXD8GBZhVkFtErksyUjYouLS5pKkgbvmFOjSWjhXInuY8uslVraWdBq7bI Sl6EfnYee680oOk7GqTGAVuLq+qjEr7IDrhILZiM7ma1s2PPhNH1aHnwgasS9kmdNRA== X-Received: by 2002:a05:620a:4bc9:b0:78d:7760:e797 with SMTP id sw9-20020a05620a4bc900b0078d7760e797mr2835615qkn.2.1712779130646; Wed, 10 Apr 2024 12:58:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEv1bdfgwWalbeb7W8lw1wQ8AyOawE9AJrFWRW0xu4CX1VzWDIy7HrETiPCNEoM1hdJNWYFiA== X-Received: by 2002:a05:620a:4bc9:b0:78d:7760:e797 with SMTP id sw9-20020a05620a4bc900b0078d7760e797mr2835572qkn.2.1712779129838; Wed, 10 Apr 2024 12:58:49 -0700 (PDT) Received: from x1n (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id vy23-20020a05620a491700b0078d66e2ac1bsm1043qkn.61.2024.04.10.12.58.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 12:58:49 -0700 (PDT) Date: Wed, 10 Apr 2024 15:58:47 -0400 From: Peter Xu To: Christophe Leroy Cc: Jason Gunthorpe , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , "linuxppc-dev@lists.ozlabs.org" , Michael Ellerman , Matthew Wilcox , Rik van Riel , Lorenzo Stoakes , Axel Rasmussen , Yang Shi , John Hubbard , "linux-arm-kernel@lists.infradead.org" , "Kirill A . Shutemov" , Andrew Jones , Vlastimil Babka , Mike Rapoport , Andrew Morton , Muchun Song , Christoph Hellwig , "linux-riscv@lists.infradead.org" , James Houghton , David Hildenbrand , Andrea Arcangeli , "Aneesh Kumar K . V" , Mike Kravetz Subject: Re: [PATCH v3 00/12] mm/gup: Unify hugetlb, part 2 Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240322161000.GJ159172@nvidia.com> <20240326140252.GH6245@nvidia.com> <20240405181633.GH5383@nvidia.com> <20240409234355.GJ5383@nvidia.com> <7b75c4db-9dbe-4ff1-b649-06a9218ae0aa@csgroup.eu> MIME-Version: 1.0 In-Reply-To: <7b75c4db-9dbe-4ff1-b649-06a9218ae0aa@csgroup.eu> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240410_125854_951443_B93404A6 X-CRM114-Status: GOOD ( 57.37 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gV2VkLCBBcHIgMTAsIDIwMjQgYXQgMDQ6MzA6NDFQTSArMDAwMCwgQ2hyaXN0b3BoZSBMZXJv eSB3cm90ZToKPiAKPiAKPiBMZSAxMC8wNC8yMDI0IMOgIDE3OjI4LCBQZXRlciBYdSBhIMOpY3Jp dMKgOgo+ID4gT24gVHVlLCBBcHIgMDksIDIwMjQgYXQgMDg6NDM6NTVQTSAtMDMwMCwgSmFzb24g R3VudGhvcnBlIHdyb3RlOgo+ID4+IE9uIEZyaSwgQXByIDA1LCAyMDI0IGF0IDA1OjQyOjQ0UE0g LTA0MDAsIFBldGVyIFh1IHdyb3RlOgo+ID4+PiBJbiBzaG9ydCwgaHVnZXRsYiBtYXBwaW5ncyBz aG91bGRuJ3QgYmUgc3BlY2lhbCBjb21wYXJpbmcgdG8gb3RoZXIgaHVnZSBwWGQKPiA+Pj4gYW5k IGxhcmdlIGZvbGlvIChjb250LXBYZCkgbWFwcGluZ3MgZm9yIG1vc3Qgb2YgdGhlIHdhbGtlcnMg aW4gbXkgbWluZCwgaWYKPiA+Pj4gbm90IGFsbC4gIEkgbmVlZCB0byBsb29rIGF0IGFsbCB0aGUg d2Fsa2VycyBhbmQgdGhlcmUgY2FuIGJlIHNvbWUgdHJpY2t5Cj4gPj4+IG9uZXMsIGJ1dCBJIGJl bGlldmUgdGhhdCBhcHBsaWVzIGluIGdlbmVyYWwuICBJdCdzIGFjdHVhbGx5IHNpbWlsYXIgdG8g d2hhdAo+ID4+PiBJIGRpZCB3aXRoIHNsb3cgZ3VwIGhlcmUuCj4gPj4KPiA+PiBJIHRoaW5rIHRo YXQgaXMgdGhlIGJpZyBxdWVzdGlvbiwgSSBhbHNvIGhhdmVuJ3QgZG9uZSB0aGUgcmVzZWFyY2gg dG8KPiA+PiBrbm93IHRoZSBhbnN3ZXIuCj4gPj4KPiA+PiBBdCB0aGlzIHBvaW50IGZvY3VzaW5n IG9uIG1vdmluZyB3aGF0IGlzIHJlYXNvbmFibGUgdG8gdGhlIHBYWF8qIEFQSQo+ID4+IG1ha2Vz IHNlbnNlIHRvIG1lLiBUaGVuIHJldmlld2luZyB3aGF0IHJlbWFpbnMgYW5kIG1ha2luZyBzb21l Cj4gPj4gZGVjaXNpb24uCj4gPj4KPiA+Pj4gTGlrZSB0aGlzIHNlcmllcywgZm9yIGNvbnQtcFhk IHdlJ2xsIG5lZWQgbXVsdGlwbGUgd2Fsa3MgY29tcGFyaW5nIHRvCj4gPj4+IGJlZm9yZSAod2hl biB3aXRoIGh1Z2V0bGJfZW50cnkoKSksIGJ1dCBmb3IgdGhhdCBwYXJ0IEknbGwgcHJvdmlkZSBz b21lCj4gPj4+IHBlcmZvcm1hbmNlIHRlc3RzIHRvbywgYW5kIHdlIGFsc28gaGF2ZSBhIGZhbGxi YWNrIHBsYW4sIHdoaWNoIGlzIHRvIGRldGVjdAo+ID4+PiBjb250LXBYZCBleGlzdGFuY2UsIHdo aWNoIHdpbGwgYWxzbyB3b3JrIGZvciBsYXJnZSBmb2xpb3MuCj4gPj4KPiA+PiBJIHRoaW5rIHdl IGNhbiBvcHRpbWl6ZSB0aGlzIHByZXR0eSBlYXN5Lgo+ID4+ICAgCj4gPj4+PiBJIHRoaW5rIGlm IHlvdSBkbyB0aGUgZWFzeSBwbGFjZXMgZm9yIHBYWCBjb252ZXJzaW9uIHlvdSB3aWxsIGhhdmUg YQo+ID4+Pj4gZ29vZCBpZGVhIGFib3V0IHdoYXQgaXMgbmVlZGVkIGZvciB0aGUgaGFyZCBwbGFj ZXMuCj4gPj4+Cj4gPj4+IEhlcmUgSU1ITyB3ZSBkb24ndCBuZWVkIHRvIHVuZGVyc3RhbmQgIndo YXQgaXMgdGhlIHNpemUgb2YgdGhpcyBodWdldGxiCj4gPj4+IHZtYSIKPiA+Pgo+ID4+IFllaCwg SSBuZXZlciByZWFsbHkgdW5kZXJzdG9vZCB3aHkgaHVnZXRsYiB3YXMgbGlua2VkIHRvIHRoZSBW TUEuLiBUaGUKPiA+PiBwYWdlIHRhYmxlIGlzIHNlbGYgZGVzY3JpYmluZywgb2J2aW91c2x5Lgo+ ID4gCj4gPiBBdHRhY2hpbmcgdG8gdm1hIHN0aWxsIG1ha2VzIHNlbnNlIHRvIG1lLCB3aGVyZSB3 ZSBzaG91bGQgZGVmaW5pdGVseSBhdm9pZAo+ID4gYSBtaXh0dXJlIG9mIGh1Z2V0bGIgYW5kICFo dWdldGxiIHBhZ2VzIGluIGEgc2luZ2xlIHZtYSAtIGh1Z2V0bGIgcGFnZXMgYXJlCj4gPiBhbGxv Y2F0ZWQsIG1hbmFnZWQsIC4uLiAgdG90YWxseSBkaWZmZXJlbnRseS4KPiA+IAo+ID4gQW5kIHNp bmNlIGh1Z2V0bGIgaXMgZGVzaWduZWQgYXMgZmlsZS1iYXNlZCAod2hpY2ggYWxzbyBtYWtlcyBz ZW5zZSB0byBtZSwKPiA+IGF0IGxlYXN0IGZvciBub3cpLCBpdCdzIGFsc28gbmF0dXJhbCB0aGF0 IGl0J3Mgdm1hLWF0dGFjaGVkLgo+ID4gCj4gPj4KPiA+Pj4gb3IgIndoaWNoIGxldmVsIG9mIHBn dGFibGUgZG9lcyB0aGlzIGh1Z2V0bGIgdm1hIHBhZ2VzIGxvY2F0ZSIsCj4gPj4KPiA+PiBEaXR0 bwo+ID4+Cj4gPj4+IGJlY2F1c2Ugd2UgbWF5IG5vdCBuZWVkIHRoYXQsIGUuZy4sIHdoZW4gd2Ug b25seSB3YW50IHRvIGNvbGxlY3Qgc29tZSBzbWFwcwo+ID4+PiBzdGF0aXN0aWNzLiAgIndoZXRo ZXIgaXQncyBodWdldGxiIiBtYXkgbWF0dGVyLCB0aG91Z2guIEUuZy4gaW4gdGhlIG1tCj4gPj4+ IHdhbGtlciB3ZSBzZWUgYSBodWdlIHBtZCwgaXQgY2FuIGJlIGEgdGhwLCBpdCBjYW4gYmUgYSBo dWdldGxiICh3aGVuCj4gPj4+IGh1Z2V0bGJfZW50cnkgcmVtb3ZlZCksIHdlIG1heSBuZWVkIGV4 dHJhIGNoZWNrIGxhdGVyIHRvIHB1dCB0aGluZ3MgaW50bwo+ID4+PiB0aGUgcmlnaHQgYnVja2V0 LCBidXQgZm9yIHRoZSB3YWxrZXIgaXRzZWxmIGl0IGRvZXNuJ3QgbmVjZXNzYXJpbHkgbmVlZAo+ ID4+PiBodWdldGxiX2VudHJ5KCkuCj4gPj4KPiA+PiBSaWdodCwgcGxhY2VzIG1heSBzdGlsbCBu ZWVkIHRvIGtub3cgaXQgaXMgcGFydCBvZiBhIGh1Z2UgVk1BIGJlY2F1c2Ugd2UKPiA+PiBoYXZl IHNwZWNpYWwgc3R1ZmYgbGlua2VkIHRvIHRoYXQuCj4gPj4KPiA+Pj4+IEJ1dCB0aGVuIGFnYWlu IHdlIGNvbWUgYmFjayB0byBwb3dlciBhbmQgaXRzIGJpZyBsaXN0IG9mIHBhZ2Ugc2l6ZXMKPiA+ Pj4+IGFuZCB2YXJpZXR5IDooIExvb2tzIGxpa2Ugc29tZSB0aGVyZSBoYXZlIGh1Z2Ugc2l6ZXMg YXQgdGhlIHBnZCBsZXZlbAo+ID4+Pj4gYXQgbGVhc3QuCj4gPj4+Cj4gPj4+IFllYWggdGhpcyBp cyBzb21ldGhpbmcgSSB3YW50IHRvIGJlIHN1cGVyIGNsZWFyLCBiZWNhdXNlIEkgbWF5IG1pc3MK PiA+Pj4gc29tZXRoaW5nOiB3ZSBkb24ndCBoYXZlIHJlYWwgcGdkIHBhZ2VzLCByaWdodD8gIFBv d2VycGMgZG9lc24ndCBldmVuCj4gPj4+IGRlZmluZSBwNGRfbGVhZigpLCBBRkFJQ1QuCj4gPj4K PiA+PiBBRkFJQ1QgaXQgaXMgYmVjYXVzZSBpdCBoaWRlcyBpdCBhbGwgaW4gaHVnZXBkLgo+ID4g Cj4gPiBJTUhPIG9uZSB0aGluZyB3ZSBjYW4gYmVuZWZpdCBmcm9tIHN1Y2ggaHVnZXBkIHJld29y ayBpcywgaWYgd2UgY2FuIHNxdWFzaAo+ID4gYWxsIHRoZSBodWdlcGRzIGxpa2Ugd2hhdCBDaHJp c3RvcGhlIGRvZXMsIHRoZW4gd2UgcHVzaCBpdCBvbmUgbW9yZSBsYXllcgo+ID4gZG93biwgYW5k IHdlIGhhdmUgYSBnb29kIGNoYW5jZSBhbGwgdGhpbmdzIHNob3VsZCBqdXN0IHdvcmsuCj4gPiAK PiA+IFNvIGFnYWluIG15IFBvd2VyIGJyYWluIGlzIGNsb3NlIHRvIHplcm8sIGJ1dCBub3cgSSdt IHJlZmVycmluZyB0byB3aGF0Cj4gPiBDaHJpc3RvcGhlIHNoYXJlZCBpbiB0aGUgb3RoZXIgdGhy ZWFkOgo+ID4gCj4gPiBodHRwczovL2dpdGh1Yi5jb20vbGludXhwcGMvd2lraS93aWtpL0h1Z2Ut cGFnZXMKPiA+IAo+ID4gVG9nZXRoZXIgd2l0aDoKPiA+IAo+ID4gaHR0cHM6Ly9sb3JlLmtlcm5l bC5vcmcvci8yODhmMjZmNDg3NjQ4ZDIxZmQ5NTkwZTQwYjM5MDkzNGVhYTVkMjRhLjE3MTEzNzcy MzAuZ2l0LmNocmlzdG9waGUubGVyb3lAY3Nncm91cC5ldQo+ID4gCj4gPiBXaGVyZSBpdCBoYXM6 Cj4gPiAKPiA+IC0tLSBhL2FyY2gvcG93ZXJwYy9wbGF0Zm9ybXMvS2NvbmZpZy5jcHV0eXBlCj4g PiArKysgYi9hcmNoL3Bvd2VycGMvcGxhdGZvcm1zL0tjb25maWcuY3B1dHlwZQo+ID4gQEAgLTk4 LDYgKzk4LDcgQEAgY29uZmlnIFBQQ19CT09LM1NfNjQKPiA+ICAgICAgICAgIHNlbGVjdCBBUkNI X0VOQUJMRV9IVUdFUEFHRV9NSUdSQVRJT04gaWYgSFVHRVRMQl9QQUdFICYmIE1JR1JBVElPTgo+ ID4gICAgICAgICAgc2VsZWN0IEFSQ0hfRU5BQkxFX1NQTElUX1BNRF9QVExPQ0sKPiA+ICAgICAg ICAgIHNlbGVjdCBBUkNIX0VOQUJMRV9USFBfTUlHUkFUSU9OIGlmIFRSQU5TUEFSRU5UX0hVR0VQ QUdFCj4gPiArICAgICAgIHNlbGVjdCBBUkNIX0hBU19IVUdFUEQgaWYgSFVHRVRMQl9QQUdFCj4g PiAgICAgICAgICBzZWxlY3QgQVJDSF9TVVBQT1JUU19IVUdFVExCRlMKPiA+ICAgICAgICAgIHNl bGVjdCBBUkNIX1NVUFBPUlRTX05VTUFfQkFMQU5DSU5HCj4gPiAgICAgICAgICBzZWxlY3QgSEFW RV9NT1ZFX1BNRAo+ID4gQEAgLTI5MCw2ICsyOTEsNyBAQCBjb25maWcgUFBDX0JPT0szUwo+ID4g ICBjb25maWcgUFBDX0U1MDAKPiA+ICAgICAgICAgIHNlbGVjdCBGU0xfRU1CX1BFUkZNT04KPiA+ ICAgICAgICAgIGJvb2wKPiA+ICsgICAgICAgc2VsZWN0IEFSQ0hfSEFTX0hVR0VQRCBpZiBIVUdF VExCX1BBR0UKPiA+ICAgICAgICAgIHNlbGVjdCBBUkNIX1NVUFBPUlRTX0hVR0VUTEJGUyBpZiBQ SFlTXzY0QklUIHx8IFBQQzY0Cj4gPiAgICAgICAgICBzZWxlY3QgUFBDX1NNUF9NVVhFRF9JUEkK PiA+ICAgICAgICAgIHNlbGVjdCBQUENfRE9PUkJFTEwKPiA+IAo+ID4gU28gSSB0aGluayBpdCBt ZWFucyB3ZSBoYXZlIHRocmVlIFBvd2VyUEMgc3lzdGVtcyB0aGF0IHN1cHBvcnRzIGh1Z2VwZAo+ ID4gcmlnaHQgbm93IChiZXNpZGVzIHRoZSA4eHggd2hpY2ggQ2hyaXN0b3BoZSBpcyB0cnlpbmcg dG8gZHJvcCBzdXBwb3J0Cj4gPiB0aGVyZSksIGJlc2lkZXMgOHh4IHdlIHN0aWxsIGhhdmUgYm9v azNzXzY0IGFuZCBFNTAwLgo+ID4gCj4gPiBMZXQncyBjaGVjayBvbmUgYnkgb25lOgo+ID4gCj4g PiAgICAtIGJvb2szc182NAo+ID4gCj4gPiAgICAgIC0gaGFzaAo+ID4gCj4gPiAgICAgICAgLSA2 NEs6IHA0ZCBpcyBub3QgdXNlZCwgbGFyZ2VzdCBwZ3NpemUgcGdkIDE2RyBAcHVkIGxldmVsLiAg SXQKPiA+ICAgICAgICAgIG1lYW5zIGFmdGVyIHNxdWFzaGluZyBpdCdsbCBiZSBhIGJ1bmNoIG9m IGNvbnQtcG1kLCBhbGwgZ29vZC4KPiA+IAo+ID4gICAgICAgIC0gNEs6IHA0ZCBhbHNvIG5vdCB1 c2VkLCBsYXJnZXN0IHBnc2l6ZSBwZ2QgMTI4RywgYWZ0ZXIgc3F1YXNoZWQKPiA+ICAgICAgICAg IGl0J2xsIGJlIGNvbnQtcHVkLiBhbGwgZ29vZC4KPiA+IAo+ID4gICAgICAtIHJhZGl4Cj4gPiAK PiA+ICAgICAgICAtIDY0SzogbGFyZ2VzdCAxRyBAcHVkLCB0aGVuIGNvbnQtcG1kIGFmdGVyIHNx dWFzaGVkLiBhbGwgZ29vZC4KPiA+IAo+ID4gICAgICAgIC0gNEs6IGxhcmdlc3QgMUcgQHB1ZCwg dGhlbiBjb250LXBtZCwgYWxsIGdvb2QuCj4gPiAKPiA+ICAgIC0gZTUwMCAmIDh4eAo+ID4gCj4g PiAgICAgIC0gYm90aCBvZiB0aGVtIHVzZSAyLWxldmVsIHBndGFibGVzIChwZ2QgKyBwdGUpLCBh ZnRlciBzcXVhc2hlZCBodWdlcGQKPiA+ICAgICAgICBAcGdkIGxldmVsIHRoZXkgYmVjb21lIGNv bnQtcHRlLiBhbGwgZ29vZC4KPiAKPiBlNTAwIGhhcyB0d28gbW9kZXM6IDMyIGJpdHMgYW5kIDY0 IGJpdHMuCj4gCj4gRm9yIDMyIGJpdHM6Cj4gCj4gOHh4IGlzIHRoZSBvbmx5IG9uZSBoYW5kbGlu ZyBpdCB0aHJvdWdoIEhXLWFzc2lzdGVkIHBhZ2V0YWJsZSB3YWxrIGhlbmNlIAo+IHJlcXVpcmlu ZyBhIDItbGV2ZWwgd2hhdGV2ZXIgdGhlIHBhZ2VzaXplIGlzLgoKSG1tIEkgdGhpbmsgbWF5YmUg ZmluYWxseSBJIGdldCBpdC4uCgpJIHRoaW5rIHRoZSBjb25mdXNpb24gY2FtZSBmcm9tIHdoZW4g SSBzYXcgdGhlcmUncyBhbHdheXMgc3VjaCBsZXZlbC0yCnRhYmxlIGRlc2NyaWJlZCBpbiBGaWd1 cmUgOC01IG9mIHRoZSBtYW51YWw6CgpodHRwczovL3d3dy5ueHAuY29tL2RvY3MvZW4vcmVmZXJl bmNlLW1hbnVhbC9NUEM4NjBVTS5wZGYKClNvIEkgc3VwcG9zZSB5b3UgbWVhbnQgZm9yIDhNLCB0 aGUgUG93ZXJQQyA4eHggc3lzdGVtIGhhcmR3YXJlIHdpbGwgYmUKYXdhcmUgb2Ygc3VjaCA4TSBw Z3RhYmxlIChmcm9tIGxldmVsLTEncyBlbnRyeSwgd2hlcmUgaXQgaGFzIGJpdCAyOC0yOSBzZXQK MDExYiksIHRoZW4gaXQgd29uJ3QgZXZlciByZWFkIGFueXRoaW5nIHN0YXJ0aW5nIGZyb20gIkxl dmVsLTIgRGVzY3JpcHRvcgoxIiAoYnV0IG9ubHkgcmVhZCB0aGUgb25seSBlbnRyeSAiTGV2ZWwt MiBEZXNjcmlwdG9yIDAiKSwgc28gZnVuZGFtZW50YWxseQpodWdlcGQgZm9ybWF0IG11c3QgbG9v ayBsaWtlIHN1Y2ggZm9yIDh4eD8KCkJ1dCB0aGVuIHBlcmhhcHMgaXQncyBzdGlsbCBjb21wYXRp YmxlIHdpdGggY29udC1wdGUgYmVjYXVzZSB0aGUgcmVzdAplbnRyaWVzIChwdGUgaW5kZXggMSsp IHdpbGwgc2ltcGx5IGJlIGlnbm9yZWQgYnkgdGhlIGhhcmR3YXJlPwoKPiAKPiBPbiBlNTAwIGl0 IGlzIGFsbCBzb2Z0d2FyZSBzbyBwYWdlcyAyTSBhbmQgbGFyZ2VyIHNob3VsZCBiZSBjb250LVBH RCAoYnkgCj4gdGhlIHdheSBJJ20gYSBiaXQgcHV6emxlZCB0aGF0IG9uIGFyY2hlcyB0aGF0IGhh dmUgb25seSAyIGxldmVscywgaWUgUEdEIAo+IGFuZCBQVEUsIHRoZSBQR0QgZW50cmllcyBhcmUg cG9wdWxhdGVkIGJ5IGEgZnVuY3Rpb24gY2FsbGVkIFBNRF9wb3B1bGF0ZSgpKS4KClllYWguLiBJ IGFtIGFsc28gd29uZGVyaW5nIHdoZXRoZXIgcGdkX3BvcHVsYXRlKCkgY291bGQgYWxzbyB3b3Jr IHRoZXJlCihwZXJoYXBzIHdpdGggc29tZSB0cml2aWFsIGNoYW5nZXMsIG9yIG1heWJlIG5vdCBl dmVuIG5lZWRlZC4uKSwgYXMgd2hlbgpwNGQvcHVkL3BtZCBsZXZlbHMgYXJlIG1pc3NpbmcsIGxp bnV4IHNob3VsZCBqdXN0IGRvIHNvbWV0aGluZyBsaWtlIGFuCmVuZm9yY2VkIGNhc3QgZnJvbSBw Z2RfdCogLT4gcG1kX3QqIGluIHRoaXMgY2FzZS4KCkkgdGhpbmsgY3VycmVudGx5IHRoZXkncmUg YWxyZWFkeSBub3QgcGdkLCBhcyBfX2ZpbmRfbGludXhfcHRlKCkgYWxyZWFkeQpza2lwcGVkIHBn ZCB1bmNvbmRpdGlvbmFsbHk6CgoJcGdkcCA9IHBnZGlyICsgcGdkX2luZGV4KGVhKTsKCXA0ZHAg PSBwNGRfb2Zmc2V0KHBnZHAsIGVhKTsKCj4gCj4gQ3VycmVudCBzaXR1YXRpb24gZm9yIDh4eCBp cyBpbGx1c3RyYXRlZCBoZXJlOiAKPiBodHRwczovL2dpdGh1Yi5jb20vbGludXhwcGMvd2lraS93 aWtpL0h1Z2UtcGFnZXMjOHh4Cj4gCj4gSSBhbHNvIHRyaWVkIHRvIGJldHRlciBpbGx1c3RyYXRl IGU1MDAvMzIgaGVyZTogCj4gaHR0cHM6Ly9naXRodWIuY29tL2xpbnV4cHBjL3dpa2kvd2lraS9I dWdlLXBhZ2VzI2U1MDAKPiAKPiBGb3IgNjQgYml0czoKPiBXZSBoYXZlIFBURS9QTUQvUFVEL1BH RCwgbm8gUDRECj4gCj4gU2VlIGFyY2gvcG93ZXJwYy9pbmNsdWRlL2FzbS9ub2hhc2gvNjQvcGd0 YWJsZS00ay5oCgpXZSBkb24ndCBoYXZlIGFueXRoaW5nIHRoYXQgaXMgYWJvdmUgcHVkIGluIHRo aXMgY2F0ZWdvcnksIHJpZ2h0PyAgVGhhdCdzCndoYXQgSSByZWFkIGZyb20geW91ciB3aWtpIChh bmQgdGhhbmtzIGZvciBwcm92aWRpbmcgdGhhdCBpbiB0aGUgZmlyc3QKcGxhY2U7IGhlbHBzIGEg bG90IGZvciBtZSB0byB1bmRlcnN0YW5kIGhvdyBpdCB3b3JrcyBvbiBQb3dlclBDKS4KCkkgd2Fu dCB0byBtYWtlIHN1cmUgaWYgSSBjYW4gbW92ZSBvbiB3aXRob3V0IGNhcmluZyBvbiBwNGQvcGdk IGxlYWZzIGxpa2UKd2hhdCB3ZSBkbyByaWdodCBub3csIGV2ZW4gYWZ0ZXIgaWYgd2UgY2FuIHJl bW92ZSBodWdlcGQgZm9yIGdvb2QsIGluIHRoaXMKY2FzZSBzaW5jZSBwNGQgYWx3YXlzIG1pc3Np bmcsIHRoZW4gaXQncyBhYm91dCB3aGV0aGVyICJwdWR8cG1kfHB0ZV9sZWFmKCkiCmNhbiBhbHNv IGNvdmVyIHRoZSBwZ2Qgb25lcyB3aGVuIHRoYXQgZGF5IGNvbWVzLCBpaXVjLgoKVGhhbmtzLAoK PiAKPiAKPiA+IAo+ID4gSSB0aGluayB0aGUgdHJpY2sgaGVyZSBpcyB0aGVyZSdsbCBiZSBubyBw Z2QgbGVhdmVzIGFmdGVyIGh1Z2VwZCBzcXVhc2hpbmcKPiA+IHRvIGxvd2VyIGxldmVscywgdGhl biBzaW5jZSBQb3dlclBDIHNlZW1zIHRvIG5ldmVyIGhhdmUgcDRkLCB0aGVuIGFsbAo+ID4gdGhp bmdzIGZhbGwgaW50byBwdWQgb3IgbG93ZXIuICBXZSBzZWVtIHRvIGJlIGFsbCBnb29kIHRoZXJl Pwo+ID4gCj4gPj4KPiA+PiBJZiB0aGUgZ29hbCBpcyB0byBwdXJnZSBodWdlcGQgdGhlbiBzb21l IG9mIHRoZSBvcHRpb25zIG1pZ2h0IHR1cm4gb3V0Cj4gPj4gdG8gY29udmVydCBodWdlcGQgaW50 byBodWdlIHA0ZC9wZ2QsIGFzIEkgdW5kZXJzdGFuZCBpdC4gSXQgd291bGQgYmUKPiA+PiBuaWNl IHRvIGhhdmUgY2VydGFpbnR5IG9uIHRoaXMgYXQgbGVhc3QuCj4gPiAKPiA+IFJpZ2h0LiAgSSBo b3BlIHRoZSBwbWQvcHVkIHBsYW4gSSBwcm9wb3NlZCBhYm92ZSBjYW4gYWxyZWFkeSB3b3JrIHRv byB3aXRoCj4gPiBzdWNoIGFtYmljaW91cyBnb2FsIHRvby4gIEJ1dCByZXZpZXcgdmVyeSB3ZWxj b21lZCBmcm9tIGVpdGhlciB5b3Ugb3IKPiA+IENocmlzdG9waGUuCj4gPiAKPiA+IFBTOiBJIHRo aW5rIEknbGwgYWxzbyBoYXZlIGEgY2xvc2VyIGxvb2sgYXQgQ2hyaXN0b3BoZSdzIHNlcmllcyB0 aGlzIHdlZWsKPiA+IG9yIG5leHQuCj4gPiAKPiA+Pgo+ID4+IFdlIGhhdmUgZWZmZWN0aXZlbHkg dGhyZWUgQVBJcyB0byBwYXJzZSBhIHNpbmdsZSBwYWdlIHRhYmxlIGFuZAo+ID4+IGN1cnJlbnRs eSBub25lIG9mIHRoZSBBUElzIGNhbiByZXR1cm4gMTAwJSBvZiB0aGUgZGF0YSBmb3IgcG93ZXIu Cj4gPiAKPiA+IFRoYW5rcywKPiA+IAoKLS0gClBldGVyIFh1CgoKX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtcmlzY3YgbWFpbGluZyBsaXN0Cmxp bnV4LXJpc2N2QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcv bWFpbG1hbi9saXN0aW5mby9saW51eC1yaXNjdgo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 92337CD11C2 for ; Wed, 10 Apr 2024 19:59:40 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=D/pED3XP; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=D/pED3XP; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4VFDF66QbLz3vZY for ; Thu, 11 Apr 2024 05:59:38 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=D/pED3XP; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=D/pED3XP; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=redhat.com (client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=peterx@redhat.com; receiver=lists.ozlabs.org) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4VFDDK4Nh9z3d2B for ; Thu, 11 Apr 2024 05:58:56 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712779133; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wxs+2ga91aRZGeZs4inMwPcuYTjJzWrBvKW8/ZIuNzA=; b=D/pED3XPzuVAx3i/ucgy3FKgEoQczaorNR0kiIi5ViBSexiRTmvR3+mF898M0p2oxsCNTv 7uNrrnb58HrzP8Hyms9fs7+D7cgQnAMVA3nVIHfwzwt4ervIQ+E3adZkBXVOOY8fY23nfo UlB9/G10hGJ+WV2O0rkcY5Uk3t8XvqU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712779133; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wxs+2ga91aRZGeZs4inMwPcuYTjJzWrBvKW8/ZIuNzA=; b=D/pED3XPzuVAx3i/ucgy3FKgEoQczaorNR0kiIi5ViBSexiRTmvR3+mF898M0p2oxsCNTv 7uNrrnb58HrzP8Hyms9fs7+D7cgQnAMVA3nVIHfwzwt4ervIQ+E3adZkBXVOOY8fY23nfo UlB9/G10hGJ+WV2O0rkcY5Uk3t8XvqU= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-141-OUu8LlFsOU-e0Zkprtl2-Q-1; Wed, 10 Apr 2024 15:58:51 -0400 X-MC-Unique: OUu8LlFsOU-e0Zkprtl2-Q-1 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-69b147e856aso15026916d6.1 for ; Wed, 10 Apr 2024 12:58:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712779131; x=1713383931; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wxs+2ga91aRZGeZs4inMwPcuYTjJzWrBvKW8/ZIuNzA=; b=MrpHuG1In7S4rqmwKbv48CvAFZamKb71F7sGPzdK2Hj4EA1a+iELoY9EohPmDofgSZ Xm0ViYl/pAa28qXfV7E7RdQfqRASsO3Dm9ytBaWOCJdS8PpcVJvd6G3/MTrIRmkjJH2A tHGAARUsQbmLnHReH+kOEpmB/tazrEULXQQeBNKAdf28Qvzq8HzxLOiTgXVz2iBesE1E UvE7q1/p0d+fn39MP8Psnu8bz8LA4FxuFDeIkK6HSkdNWsh3xECDja/MJ9WIg+pSayt5 UtKuHuimnBMu8wO8aK62I5dqSY8ZTB+ZwfduDRF/Zxs87sD3rpe06BKBFHJGaNR1TJjn Yy7A== X-Forwarded-Encrypted: i=1; AJvYcCV7C0r8So3alzy/XRwAOBMlDkS7uQscYWaVO2otRmJYdIb9+kLwTWdcU/c6TAIMzZALPE+f6PvLr26oyp9uUCYjC/S+2NR1rOKMS3rCig== X-Gm-Message-State: AOJu0Yy39qXkCv8exgQ6qGtSup+IBtFAMtHOyjpTY3OfsHE8OPLWevU+ wXfoYYJ2Uw2rDixHvI3gsR3vlAdzL/QFj9Kal3i0t/CBxuKnfmunywjHKhBNQw32fiMU3ZhBNca ETooq/s/+61Rf5xsbMKW5QORJSdi4M2KiWGTCODba98x6xTUksV847A4VmM9qGbU= X-Received: by 2002:a05:620a:4bc9:b0:78d:7760:e797 with SMTP id sw9-20020a05620a4bc900b0078d7760e797mr2835611qkn.2.1712779130643; Wed, 10 Apr 2024 12:58:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEv1bdfgwWalbeb7W8lw1wQ8AyOawE9AJrFWRW0xu4CX1VzWDIy7HrETiPCNEoM1hdJNWYFiA== X-Received: by 2002:a05:620a:4bc9:b0:78d:7760:e797 with SMTP id sw9-20020a05620a4bc900b0078d7760e797mr2835572qkn.2.1712779129838; Wed, 10 Apr 2024 12:58:49 -0700 (PDT) Received: from x1n (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id vy23-20020a05620a491700b0078d66e2ac1bsm1043qkn.61.2024.04.10.12.58.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 12:58:49 -0700 (PDT) Date: Wed, 10 Apr 2024 15:58:47 -0400 From: Peter Xu To: Christophe Leroy Subject: Re: [PATCH v3 00/12] mm/gup: Unify hugetlb, part 2 Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240322161000.GJ159172@nvidia.com> <20240326140252.GH6245@nvidia.com> <20240405181633.GH5383@nvidia.com> <20240409234355.GJ5383@nvidia.com> <7b75c4db-9dbe-4ff1-b649-06a9218ae0aa@csgroup.eu> MIME-Version: 1.0 In-Reply-To: <7b75c4db-9dbe-4ff1-b649-06a9218ae0aa@csgroup.eu> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Houghton , David Hildenbrand , Yang Shi , Andrew Jones , "linux-mm@kvack.org" , "linux-riscv@lists.infradead.org" , Andrea Arcangeli , "Aneesh Kumar K . V" , Matthew Wilcox , Christoph Hellwig , "linux-arm-kernel@lists.infradead.org" , Jason Gunthorpe , Axel Rasmussen , Rik van Riel , John Hubbard , "Kirill A . Shutemov" , Vlastimil Babka , Lorenzo Stoakes , Muchun Song , "linux-kernel@vger.kernel.org" , Andrew Morton , "linuxppc-dev@lists.ozlabs.org" , Mike Rapoport , Mike Kravetz Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Wed, Apr 10, 2024 at 04:30:41PM +0000, Christophe Leroy wrote: > > > Le 10/04/2024 à 17:28, Peter Xu a écrit : > > On Tue, Apr 09, 2024 at 08:43:55PM -0300, Jason Gunthorpe wrote: > >> On Fri, Apr 05, 2024 at 05:42:44PM -0400, Peter Xu wrote: > >>> In short, hugetlb mappings shouldn't be special comparing to other huge pXd > >>> and large folio (cont-pXd) mappings for most of the walkers in my mind, if > >>> not all. I need to look at all the walkers and there can be some tricky > >>> ones, but I believe that applies in general. It's actually similar to what > >>> I did with slow gup here. > >> > >> I think that is the big question, I also haven't done the research to > >> know the answer. > >> > >> At this point focusing on moving what is reasonable to the pXX_* API > >> makes sense to me. Then reviewing what remains and making some > >> decision. > >> > >>> Like this series, for cont-pXd we'll need multiple walks comparing to > >>> before (when with hugetlb_entry()), but for that part I'll provide some > >>> performance tests too, and we also have a fallback plan, which is to detect > >>> cont-pXd existance, which will also work for large folios. > >> > >> I think we can optimize this pretty easy. > >> > >>>> I think if you do the easy places for pXX conversion you will have a > >>>> good idea about what is needed for the hard places. > >>> > >>> Here IMHO we don't need to understand "what is the size of this hugetlb > >>> vma" > >> > >> Yeh, I never really understood why hugetlb was linked to the VMA.. The > >> page table is self describing, obviously. > > > > Attaching to vma still makes sense to me, where we should definitely avoid > > a mixture of hugetlb and !hugetlb pages in a single vma - hugetlb pages are > > allocated, managed, ... totally differently. > > > > And since hugetlb is designed as file-based (which also makes sense to me, > > at least for now), it's also natural that it's vma-attached. > > > >> > >>> or "which level of pgtable does this hugetlb vma pages locate", > >> > >> Ditto > >> > >>> because we may not need that, e.g., when we only want to collect some smaps > >>> statistics. "whether it's hugetlb" may matter, though. E.g. in the mm > >>> walker we see a huge pmd, it can be a thp, it can be a hugetlb (when > >>> hugetlb_entry removed), we may need extra check later to put things into > >>> the right bucket, but for the walker itself it doesn't necessarily need > >>> hugetlb_entry(). > >> > >> Right, places may still need to know it is part of a huge VMA because we > >> have special stuff linked to that. > >> > >>>> But then again we come back to power and its big list of page sizes > >>>> and variety :( Looks like some there have huge sizes at the pgd level > >>>> at least. > >>> > >>> Yeah this is something I want to be super clear, because I may miss > >>> something: we don't have real pgd pages, right? Powerpc doesn't even > >>> define p4d_leaf(), AFAICT. > >> > >> AFAICT it is because it hides it all in hugepd. > > > > IMHO one thing we can benefit from such hugepd rework is, if we can squash > > all the hugepds like what Christophe does, then we push it one more layer > > down, and we have a good chance all things should just work. > > > > So again my Power brain is close to zero, but now I'm referring to what > > Christophe shared in the other thread: > > > > https://github.com/linuxppc/wiki/wiki/Huge-pages > > > > Together with: > > > > https://lore.kernel.org/r/288f26f487648d21fd9590e40b390934eaa5d24a.1711377230.git.christophe.leroy@csgroup.eu > > > > Where it has: > > > > --- a/arch/powerpc/platforms/Kconfig.cputype > > +++ b/arch/powerpc/platforms/Kconfig.cputype > > @@ -98,6 +98,7 @@ config PPC_BOOK3S_64 > > select ARCH_ENABLE_HUGEPAGE_MIGRATION if HUGETLB_PAGE && MIGRATION > > select ARCH_ENABLE_SPLIT_PMD_PTLOCK > > select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE > > + select ARCH_HAS_HUGEPD if HUGETLB_PAGE > > select ARCH_SUPPORTS_HUGETLBFS > > select ARCH_SUPPORTS_NUMA_BALANCING > > select HAVE_MOVE_PMD > > @@ -290,6 +291,7 @@ config PPC_BOOK3S > > config PPC_E500 > > select FSL_EMB_PERFMON > > bool > > + select ARCH_HAS_HUGEPD if HUGETLB_PAGE > > select ARCH_SUPPORTS_HUGETLBFS if PHYS_64BIT || PPC64 > > select PPC_SMP_MUXED_IPI > > select PPC_DOORBELL > > > > So I think it means we have three PowerPC systems that supports hugepd > > right now (besides the 8xx which Christophe is trying to drop support > > there), besides 8xx we still have book3s_64 and E500. > > > > Let's check one by one: > > > > - book3s_64 > > > > - hash > > > > - 64K: p4d is not used, largest pgsize pgd 16G @pud level. It > > means after squashing it'll be a bunch of cont-pmd, all good. > > > > - 4K: p4d also not used, largest pgsize pgd 128G, after squashed > > it'll be cont-pud. all good. > > > > - radix > > > > - 64K: largest 1G @pud, then cont-pmd after squashed. all good. > > > > - 4K: largest 1G @pud, then cont-pmd, all good. > > > > - e500 & 8xx > > > > - both of them use 2-level pgtables (pgd + pte), after squashed hugepd > > @pgd level they become cont-pte. all good. > > e500 has two modes: 32 bits and 64 bits. > > For 32 bits: > > 8xx is the only one handling it through HW-assisted pagetable walk hence > requiring a 2-level whatever the pagesize is. Hmm I think maybe finally I get it.. I think the confusion came from when I saw there's always such level-2 table described in Figure 8-5 of the manual: https://www.nxp.com/docs/en/reference-manual/MPC860UM.pdf So I suppose you meant for 8M, the PowerPC 8xx system hardware will be aware of such 8M pgtable (from level-1's entry, where it has bit 28-29 set 011b), then it won't ever read anything starting from "Level-2 Descriptor 1" (but only read the only entry "Level-2 Descriptor 0"), so fundamentally hugepd format must look like such for 8xx? But then perhaps it's still compatible with cont-pte because the rest entries (pte index 1+) will simply be ignored by the hardware? > > On e500 it is all software so pages 2M and larger should be cont-PGD (by > the way I'm a bit puzzled that on arches that have only 2 levels, ie PGD > and PTE, the PGD entries are populated by a function called PMD_populate()). Yeah.. I am also wondering whether pgd_populate() could also work there (perhaps with some trivial changes, or maybe not even needed..), as when p4d/pud/pmd levels are missing, linux should just do something like an enforced cast from pgd_t* -> pmd_t* in this case. I think currently they're already not pgd, as __find_linux_pte() already skipped pgd unconditionally: pgdp = pgdir + pgd_index(ea); p4dp = p4d_offset(pgdp, ea); > > Current situation for 8xx is illustrated here: > https://github.com/linuxppc/wiki/wiki/Huge-pages#8xx > > I also tried to better illustrate e500/32 here: > https://github.com/linuxppc/wiki/wiki/Huge-pages#e500 > > For 64 bits: > We have PTE/PMD/PUD/PGD, no P4D > > See arch/powerpc/include/asm/nohash/64/pgtable-4k.h We don't have anything that is above pud in this category, right? That's what I read from your wiki (and thanks for providing that in the first place; helps a lot for me to understand how it works on PowerPC). I want to make sure if I can move on without caring on p4d/pgd leafs like what we do right now, even after if we can remove hugepd for good, in this case since p4d always missing, then it's about whether "pud|pmd|pte_leaf()" can also cover the pgd ones when that day comes, iiuc. Thanks, > > > > > > I think the trick here is there'll be no pgd leaves after hugepd squashing > > to lower levels, then since PowerPC seems to never have p4d, then all > > things fall into pud or lower. We seem to be all good there? > > > >> > >> If the goal is to purge hugepd then some of the options might turn out > >> to convert hugepd into huge p4d/pgd, as I understand it. It would be > >> nice to have certainty on this at least. > > > > Right. I hope the pmd/pud plan I proposed above can already work too with > > such ambicious goal too. But review very welcomed from either you or > > Christophe. > > > > PS: I think I'll also have a closer look at Christophe's series this week > > or next. > > > >> > >> We have effectively three APIs to parse a single page table and > >> currently none of the APIs can return 100% of the data for power. > > > > Thanks, > > -- Peter Xu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1F879CD128A for ; Wed, 10 Apr 2024 19:59:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=iQWSUphVwiKVz2ybuaNwF3u/v39Z/gYR0iF8HNTzJvA=; b=nA1HhnRcfqPrRy 4IajylmnMXJQQtmMJ1eo84mUEsBEns87XXGBIKi0R1iHGmwBFkphbUgFw0ChS40wzHpqto92H7Eqq O9tw31umHsO6Pdwiof4kcSlSCRe73Vn72Gcbzc4LG9DKD7LvsK7HMkTX2wPodplP0FSXpALTEe7zm tJMLhiS/zfjswPT8IjpLX2ycJSF9Y8R2QZUHAx68XAGpTOF8ptwq4hUIPr/vOsdPqCjPJz8RoSHuq il5yuM/AaPirxNje1h2MLy8OIrZ4gMzqMbKGgXSlu1HF9geJBhumBbRe0m4iydFhWmhBj3Xj5AX34 KxbkvSNJEbua4TOlBlPA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rue5y-00000008ldy-32Nt; Wed, 10 Apr 2024 19:58:58 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rue5v-00000008lcs-3D21 for linux-arm-kernel@lists.infradead.org; Wed, 10 Apr 2024 19:58:57 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712779134; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wxs+2ga91aRZGeZs4inMwPcuYTjJzWrBvKW8/ZIuNzA=; b=hzirOTW8zd1th8PhbUcghXKQxjePRXBFRmk70tpkTT+uPC0gHqAio+/gP/BbFpsnh/oFQi DRgKTQ+wbUx91c7mXS2IcZT6JFx1htDvHrZstWB0g6eSYrs8tFfYLYVFZIHO5fFfmtAycY VKLG2oh3tmwhtquAgSSKHfImmmfaJNo= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-643-qd4C7KpyM3yjGASffT144g-1; Wed, 10 Apr 2024 15:58:51 -0400 X-MC-Unique: qd4C7KpyM3yjGASffT144g-1 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-69a01201ca0so18635206d6.0 for ; Wed, 10 Apr 2024 12:58:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712779131; x=1713383931; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wxs+2ga91aRZGeZs4inMwPcuYTjJzWrBvKW8/ZIuNzA=; b=i6XP7WKDL+CdZDX2eUXBKDIo2jbLKYFlwjkJu21KZXALfpoTqQLG0jWnR4Vde0hLC3 +48LooWu+162Y2cD1q+bwRoyA7qDs4BRvlgM1qT9I2WBvMvnU/sfPPsi2TMr0FU4NUI2 l94ZociDuRlXsVNsDUx+HfcC1MNnD2XpBqt4t7axN6vqgdnX6RrKU6noo+FwSDZuLGEj ag8GV92cvYCnu+zvoTxD3Xn4uQlLZMPBNDu1BfEG6zBGf5o5S89x+Yp83lhm/ZXhhzrs B326zaAGYuNhupEX+TBjBo2F+HgLHkcxQwE/mm4M0du7E2d3lfNqfxCuL3XI54yrDHlt PzuA== X-Forwarded-Encrypted: i=1; AJvYcCUz9LNCMk0tIdv2YsgD8mIUUlTUqNEM/xo13Sf7tv1tB5Z9aX3sIMpTpjYCxLGGjk0KxcKNHYj+mnNTlsx4/Sj6Aiq3C3TyFAvySt9uCl7m05hLzoA= X-Gm-Message-State: AOJu0YxYop0byj1evLSSmCC85V94JC+8pHewi+eX9IwBI/x69obNhkOa K3rTGvFGHld3vt+kXqvxbNB3KghT8XiDwdoYi88ESdkXKRdH0WhQfiOp/sj+XOmcfGFMeA+jVlK lbCoL6RM0Ry9jyhrgNNPx9DdeRUSmz8fmQPlgWX5oIPX1uxNhJoa5EqLNlJUBt+QpGJqqJbAe X-Received: by 2002:a05:620a:4bc9:b0:78d:7760:e797 with SMTP id sw9-20020a05620a4bc900b0078d7760e797mr2835605qkn.2.1712779130642; Wed, 10 Apr 2024 12:58:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEv1bdfgwWalbeb7W8lw1wQ8AyOawE9AJrFWRW0xu4CX1VzWDIy7HrETiPCNEoM1hdJNWYFiA== X-Received: by 2002:a05:620a:4bc9:b0:78d:7760:e797 with SMTP id sw9-20020a05620a4bc900b0078d7760e797mr2835572qkn.2.1712779129838; Wed, 10 Apr 2024 12:58:49 -0700 (PDT) Received: from x1n (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id vy23-20020a05620a491700b0078d66e2ac1bsm1043qkn.61.2024.04.10.12.58.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 12:58:49 -0700 (PDT) Date: Wed, 10 Apr 2024 15:58:47 -0400 From: Peter Xu To: Christophe Leroy Cc: Jason Gunthorpe , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , "linuxppc-dev@lists.ozlabs.org" , Michael Ellerman , Matthew Wilcox , Rik van Riel , Lorenzo Stoakes , Axel Rasmussen , Yang Shi , John Hubbard , "linux-arm-kernel@lists.infradead.org" , "Kirill A . Shutemov" , Andrew Jones , Vlastimil Babka , Mike Rapoport , Andrew Morton , Muchun Song , Christoph Hellwig , "linux-riscv@lists.infradead.org" , James Houghton , David Hildenbrand , Andrea Arcangeli , "Aneesh Kumar K . V" , Mike Kravetz Subject: Re: [PATCH v3 00/12] mm/gup: Unify hugetlb, part 2 Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240322161000.GJ159172@nvidia.com> <20240326140252.GH6245@nvidia.com> <20240405181633.GH5383@nvidia.com> <20240409234355.GJ5383@nvidia.com> <7b75c4db-9dbe-4ff1-b649-06a9218ae0aa@csgroup.eu> MIME-Version: 1.0 In-Reply-To: <7b75c4db-9dbe-4ff1-b649-06a9218ae0aa@csgroup.eu> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240410_125855_898817_3D2E3814 X-CRM114-Status: GOOD ( 59.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gV2VkLCBBcHIgMTAsIDIwMjQgYXQgMDQ6MzA6NDFQTSArMDAwMCwgQ2hyaXN0b3BoZSBMZXJv eSB3cm90ZToKPiAKPiAKPiBMZSAxMC8wNC8yMDI0IMOgIDE3OjI4LCBQZXRlciBYdSBhIMOpY3Jp dMKgOgo+ID4gT24gVHVlLCBBcHIgMDksIDIwMjQgYXQgMDg6NDM6NTVQTSAtMDMwMCwgSmFzb24g R3VudGhvcnBlIHdyb3RlOgo+ID4+IE9uIEZyaSwgQXByIDA1LCAyMDI0IGF0IDA1OjQyOjQ0UE0g LTA0MDAsIFBldGVyIFh1IHdyb3RlOgo+ID4+PiBJbiBzaG9ydCwgaHVnZXRsYiBtYXBwaW5ncyBz aG91bGRuJ3QgYmUgc3BlY2lhbCBjb21wYXJpbmcgdG8gb3RoZXIgaHVnZSBwWGQKPiA+Pj4gYW5k IGxhcmdlIGZvbGlvIChjb250LXBYZCkgbWFwcGluZ3MgZm9yIG1vc3Qgb2YgdGhlIHdhbGtlcnMg aW4gbXkgbWluZCwgaWYKPiA+Pj4gbm90IGFsbC4gIEkgbmVlZCB0byBsb29rIGF0IGFsbCB0aGUg d2Fsa2VycyBhbmQgdGhlcmUgY2FuIGJlIHNvbWUgdHJpY2t5Cj4gPj4+IG9uZXMsIGJ1dCBJIGJl bGlldmUgdGhhdCBhcHBsaWVzIGluIGdlbmVyYWwuICBJdCdzIGFjdHVhbGx5IHNpbWlsYXIgdG8g d2hhdAo+ID4+PiBJIGRpZCB3aXRoIHNsb3cgZ3VwIGhlcmUuCj4gPj4KPiA+PiBJIHRoaW5rIHRo YXQgaXMgdGhlIGJpZyBxdWVzdGlvbiwgSSBhbHNvIGhhdmVuJ3QgZG9uZSB0aGUgcmVzZWFyY2gg dG8KPiA+PiBrbm93IHRoZSBhbnN3ZXIuCj4gPj4KPiA+PiBBdCB0aGlzIHBvaW50IGZvY3VzaW5n IG9uIG1vdmluZyB3aGF0IGlzIHJlYXNvbmFibGUgdG8gdGhlIHBYWF8qIEFQSQo+ID4+IG1ha2Vz IHNlbnNlIHRvIG1lLiBUaGVuIHJldmlld2luZyB3aGF0IHJlbWFpbnMgYW5kIG1ha2luZyBzb21l Cj4gPj4gZGVjaXNpb24uCj4gPj4KPiA+Pj4gTGlrZSB0aGlzIHNlcmllcywgZm9yIGNvbnQtcFhk IHdlJ2xsIG5lZWQgbXVsdGlwbGUgd2Fsa3MgY29tcGFyaW5nIHRvCj4gPj4+IGJlZm9yZSAod2hl biB3aXRoIGh1Z2V0bGJfZW50cnkoKSksIGJ1dCBmb3IgdGhhdCBwYXJ0IEknbGwgcHJvdmlkZSBz b21lCj4gPj4+IHBlcmZvcm1hbmNlIHRlc3RzIHRvbywgYW5kIHdlIGFsc28gaGF2ZSBhIGZhbGxi YWNrIHBsYW4sIHdoaWNoIGlzIHRvIGRldGVjdAo+ID4+PiBjb250LXBYZCBleGlzdGFuY2UsIHdo aWNoIHdpbGwgYWxzbyB3b3JrIGZvciBsYXJnZSBmb2xpb3MuCj4gPj4KPiA+PiBJIHRoaW5rIHdl IGNhbiBvcHRpbWl6ZSB0aGlzIHByZXR0eSBlYXN5Lgo+ID4+ICAgCj4gPj4+PiBJIHRoaW5rIGlm IHlvdSBkbyB0aGUgZWFzeSBwbGFjZXMgZm9yIHBYWCBjb252ZXJzaW9uIHlvdSB3aWxsIGhhdmUg YQo+ID4+Pj4gZ29vZCBpZGVhIGFib3V0IHdoYXQgaXMgbmVlZGVkIGZvciB0aGUgaGFyZCBwbGFj ZXMuCj4gPj4+Cj4gPj4+IEhlcmUgSU1ITyB3ZSBkb24ndCBuZWVkIHRvIHVuZGVyc3RhbmQgIndo YXQgaXMgdGhlIHNpemUgb2YgdGhpcyBodWdldGxiCj4gPj4+IHZtYSIKPiA+Pgo+ID4+IFllaCwg SSBuZXZlciByZWFsbHkgdW5kZXJzdG9vZCB3aHkgaHVnZXRsYiB3YXMgbGlua2VkIHRvIHRoZSBW TUEuLiBUaGUKPiA+PiBwYWdlIHRhYmxlIGlzIHNlbGYgZGVzY3JpYmluZywgb2J2aW91c2x5Lgo+ ID4gCj4gPiBBdHRhY2hpbmcgdG8gdm1hIHN0aWxsIG1ha2VzIHNlbnNlIHRvIG1lLCB3aGVyZSB3 ZSBzaG91bGQgZGVmaW5pdGVseSBhdm9pZAo+ID4gYSBtaXh0dXJlIG9mIGh1Z2V0bGIgYW5kICFo dWdldGxiIHBhZ2VzIGluIGEgc2luZ2xlIHZtYSAtIGh1Z2V0bGIgcGFnZXMgYXJlCj4gPiBhbGxv Y2F0ZWQsIG1hbmFnZWQsIC4uLiAgdG90YWxseSBkaWZmZXJlbnRseS4KPiA+IAo+ID4gQW5kIHNp bmNlIGh1Z2V0bGIgaXMgZGVzaWduZWQgYXMgZmlsZS1iYXNlZCAod2hpY2ggYWxzbyBtYWtlcyBz ZW5zZSB0byBtZSwKPiA+IGF0IGxlYXN0IGZvciBub3cpLCBpdCdzIGFsc28gbmF0dXJhbCB0aGF0 IGl0J3Mgdm1hLWF0dGFjaGVkLgo+ID4gCj4gPj4KPiA+Pj4gb3IgIndoaWNoIGxldmVsIG9mIHBn dGFibGUgZG9lcyB0aGlzIGh1Z2V0bGIgdm1hIHBhZ2VzIGxvY2F0ZSIsCj4gPj4KPiA+PiBEaXR0 bwo+ID4+Cj4gPj4+IGJlY2F1c2Ugd2UgbWF5IG5vdCBuZWVkIHRoYXQsIGUuZy4sIHdoZW4gd2Ug b25seSB3YW50IHRvIGNvbGxlY3Qgc29tZSBzbWFwcwo+ID4+PiBzdGF0aXN0aWNzLiAgIndoZXRo ZXIgaXQncyBodWdldGxiIiBtYXkgbWF0dGVyLCB0aG91Z2guIEUuZy4gaW4gdGhlIG1tCj4gPj4+ IHdhbGtlciB3ZSBzZWUgYSBodWdlIHBtZCwgaXQgY2FuIGJlIGEgdGhwLCBpdCBjYW4gYmUgYSBo dWdldGxiICh3aGVuCj4gPj4+IGh1Z2V0bGJfZW50cnkgcmVtb3ZlZCksIHdlIG1heSBuZWVkIGV4 dHJhIGNoZWNrIGxhdGVyIHRvIHB1dCB0aGluZ3MgaW50bwo+ID4+PiB0aGUgcmlnaHQgYnVja2V0 LCBidXQgZm9yIHRoZSB3YWxrZXIgaXRzZWxmIGl0IGRvZXNuJ3QgbmVjZXNzYXJpbHkgbmVlZAo+ ID4+PiBodWdldGxiX2VudHJ5KCkuCj4gPj4KPiA+PiBSaWdodCwgcGxhY2VzIG1heSBzdGlsbCBu ZWVkIHRvIGtub3cgaXQgaXMgcGFydCBvZiBhIGh1Z2UgVk1BIGJlY2F1c2Ugd2UKPiA+PiBoYXZl IHNwZWNpYWwgc3R1ZmYgbGlua2VkIHRvIHRoYXQuCj4gPj4KPiA+Pj4+IEJ1dCB0aGVuIGFnYWlu IHdlIGNvbWUgYmFjayB0byBwb3dlciBhbmQgaXRzIGJpZyBsaXN0IG9mIHBhZ2Ugc2l6ZXMKPiA+ Pj4+IGFuZCB2YXJpZXR5IDooIExvb2tzIGxpa2Ugc29tZSB0aGVyZSBoYXZlIGh1Z2Ugc2l6ZXMg YXQgdGhlIHBnZCBsZXZlbAo+ID4+Pj4gYXQgbGVhc3QuCj4gPj4+Cj4gPj4+IFllYWggdGhpcyBp cyBzb21ldGhpbmcgSSB3YW50IHRvIGJlIHN1cGVyIGNsZWFyLCBiZWNhdXNlIEkgbWF5IG1pc3MK PiA+Pj4gc29tZXRoaW5nOiB3ZSBkb24ndCBoYXZlIHJlYWwgcGdkIHBhZ2VzLCByaWdodD8gIFBv d2VycGMgZG9lc24ndCBldmVuCj4gPj4+IGRlZmluZSBwNGRfbGVhZigpLCBBRkFJQ1QuCj4gPj4K PiA+PiBBRkFJQ1QgaXQgaXMgYmVjYXVzZSBpdCBoaWRlcyBpdCBhbGwgaW4gaHVnZXBkLgo+ID4g Cj4gPiBJTUhPIG9uZSB0aGluZyB3ZSBjYW4gYmVuZWZpdCBmcm9tIHN1Y2ggaHVnZXBkIHJld29y ayBpcywgaWYgd2UgY2FuIHNxdWFzaAo+ID4gYWxsIHRoZSBodWdlcGRzIGxpa2Ugd2hhdCBDaHJp c3RvcGhlIGRvZXMsIHRoZW4gd2UgcHVzaCBpdCBvbmUgbW9yZSBsYXllcgo+ID4gZG93biwgYW5k IHdlIGhhdmUgYSBnb29kIGNoYW5jZSBhbGwgdGhpbmdzIHNob3VsZCBqdXN0IHdvcmsuCj4gPiAK PiA+IFNvIGFnYWluIG15IFBvd2VyIGJyYWluIGlzIGNsb3NlIHRvIHplcm8sIGJ1dCBub3cgSSdt IHJlZmVycmluZyB0byB3aGF0Cj4gPiBDaHJpc3RvcGhlIHNoYXJlZCBpbiB0aGUgb3RoZXIgdGhy ZWFkOgo+ID4gCj4gPiBodHRwczovL2dpdGh1Yi5jb20vbGludXhwcGMvd2lraS93aWtpL0h1Z2Ut cGFnZXMKPiA+IAo+ID4gVG9nZXRoZXIgd2l0aDoKPiA+IAo+ID4gaHR0cHM6Ly9sb3JlLmtlcm5l bC5vcmcvci8yODhmMjZmNDg3NjQ4ZDIxZmQ5NTkwZTQwYjM5MDkzNGVhYTVkMjRhLjE3MTEzNzcy MzAuZ2l0LmNocmlzdG9waGUubGVyb3lAY3Nncm91cC5ldQo+ID4gCj4gPiBXaGVyZSBpdCBoYXM6 Cj4gPiAKPiA+IC0tLSBhL2FyY2gvcG93ZXJwYy9wbGF0Zm9ybXMvS2NvbmZpZy5jcHV0eXBlCj4g PiArKysgYi9hcmNoL3Bvd2VycGMvcGxhdGZvcm1zL0tjb25maWcuY3B1dHlwZQo+ID4gQEAgLTk4 LDYgKzk4LDcgQEAgY29uZmlnIFBQQ19CT09LM1NfNjQKPiA+ICAgICAgICAgIHNlbGVjdCBBUkNI X0VOQUJMRV9IVUdFUEFHRV9NSUdSQVRJT04gaWYgSFVHRVRMQl9QQUdFICYmIE1JR1JBVElPTgo+ ID4gICAgICAgICAgc2VsZWN0IEFSQ0hfRU5BQkxFX1NQTElUX1BNRF9QVExPQ0sKPiA+ICAgICAg ICAgIHNlbGVjdCBBUkNIX0VOQUJMRV9USFBfTUlHUkFUSU9OIGlmIFRSQU5TUEFSRU5UX0hVR0VQ QUdFCj4gPiArICAgICAgIHNlbGVjdCBBUkNIX0hBU19IVUdFUEQgaWYgSFVHRVRMQl9QQUdFCj4g PiAgICAgICAgICBzZWxlY3QgQVJDSF9TVVBQT1JUU19IVUdFVExCRlMKPiA+ICAgICAgICAgIHNl bGVjdCBBUkNIX1NVUFBPUlRTX05VTUFfQkFMQU5DSU5HCj4gPiAgICAgICAgICBzZWxlY3QgSEFW RV9NT1ZFX1BNRAo+ID4gQEAgLTI5MCw2ICsyOTEsNyBAQCBjb25maWcgUFBDX0JPT0szUwo+ID4g ICBjb25maWcgUFBDX0U1MDAKPiA+ICAgICAgICAgIHNlbGVjdCBGU0xfRU1CX1BFUkZNT04KPiA+ ICAgICAgICAgIGJvb2wKPiA+ICsgICAgICAgc2VsZWN0IEFSQ0hfSEFTX0hVR0VQRCBpZiBIVUdF VExCX1BBR0UKPiA+ICAgICAgICAgIHNlbGVjdCBBUkNIX1NVUFBPUlRTX0hVR0VUTEJGUyBpZiBQ SFlTXzY0QklUIHx8IFBQQzY0Cj4gPiAgICAgICAgICBzZWxlY3QgUFBDX1NNUF9NVVhFRF9JUEkK PiA+ICAgICAgICAgIHNlbGVjdCBQUENfRE9PUkJFTEwKPiA+IAo+ID4gU28gSSB0aGluayBpdCBt ZWFucyB3ZSBoYXZlIHRocmVlIFBvd2VyUEMgc3lzdGVtcyB0aGF0IHN1cHBvcnRzIGh1Z2VwZAo+ ID4gcmlnaHQgbm93IChiZXNpZGVzIHRoZSA4eHggd2hpY2ggQ2hyaXN0b3BoZSBpcyB0cnlpbmcg dG8gZHJvcCBzdXBwb3J0Cj4gPiB0aGVyZSksIGJlc2lkZXMgOHh4IHdlIHN0aWxsIGhhdmUgYm9v azNzXzY0IGFuZCBFNTAwLgo+ID4gCj4gPiBMZXQncyBjaGVjayBvbmUgYnkgb25lOgo+ID4gCj4g PiAgICAtIGJvb2szc182NAo+ID4gCj4gPiAgICAgIC0gaGFzaAo+ID4gCj4gPiAgICAgICAgLSA2 NEs6IHA0ZCBpcyBub3QgdXNlZCwgbGFyZ2VzdCBwZ3NpemUgcGdkIDE2RyBAcHVkIGxldmVsLiAg SXQKPiA+ICAgICAgICAgIG1lYW5zIGFmdGVyIHNxdWFzaGluZyBpdCdsbCBiZSBhIGJ1bmNoIG9m IGNvbnQtcG1kLCBhbGwgZ29vZC4KPiA+IAo+ID4gICAgICAgIC0gNEs6IHA0ZCBhbHNvIG5vdCB1 c2VkLCBsYXJnZXN0IHBnc2l6ZSBwZ2QgMTI4RywgYWZ0ZXIgc3F1YXNoZWQKPiA+ICAgICAgICAg IGl0J2xsIGJlIGNvbnQtcHVkLiBhbGwgZ29vZC4KPiA+IAo+ID4gICAgICAtIHJhZGl4Cj4gPiAK PiA+ICAgICAgICAtIDY0SzogbGFyZ2VzdCAxRyBAcHVkLCB0aGVuIGNvbnQtcG1kIGFmdGVyIHNx dWFzaGVkLiBhbGwgZ29vZC4KPiA+IAo+ID4gICAgICAgIC0gNEs6IGxhcmdlc3QgMUcgQHB1ZCwg dGhlbiBjb250LXBtZCwgYWxsIGdvb2QuCj4gPiAKPiA+ICAgIC0gZTUwMCAmIDh4eAo+ID4gCj4g PiAgICAgIC0gYm90aCBvZiB0aGVtIHVzZSAyLWxldmVsIHBndGFibGVzIChwZ2QgKyBwdGUpLCBh ZnRlciBzcXVhc2hlZCBodWdlcGQKPiA+ICAgICAgICBAcGdkIGxldmVsIHRoZXkgYmVjb21lIGNv bnQtcHRlLiBhbGwgZ29vZC4KPiAKPiBlNTAwIGhhcyB0d28gbW9kZXM6IDMyIGJpdHMgYW5kIDY0 IGJpdHMuCj4gCj4gRm9yIDMyIGJpdHM6Cj4gCj4gOHh4IGlzIHRoZSBvbmx5IG9uZSBoYW5kbGlu ZyBpdCB0aHJvdWdoIEhXLWFzc2lzdGVkIHBhZ2V0YWJsZSB3YWxrIGhlbmNlIAo+IHJlcXVpcmlu ZyBhIDItbGV2ZWwgd2hhdGV2ZXIgdGhlIHBhZ2VzaXplIGlzLgoKSG1tIEkgdGhpbmsgbWF5YmUg ZmluYWxseSBJIGdldCBpdC4uCgpJIHRoaW5rIHRoZSBjb25mdXNpb24gY2FtZSBmcm9tIHdoZW4g SSBzYXcgdGhlcmUncyBhbHdheXMgc3VjaCBsZXZlbC0yCnRhYmxlIGRlc2NyaWJlZCBpbiBGaWd1 cmUgOC01IG9mIHRoZSBtYW51YWw6CgpodHRwczovL3d3dy5ueHAuY29tL2RvY3MvZW4vcmVmZXJl bmNlLW1hbnVhbC9NUEM4NjBVTS5wZGYKClNvIEkgc3VwcG9zZSB5b3UgbWVhbnQgZm9yIDhNLCB0 aGUgUG93ZXJQQyA4eHggc3lzdGVtIGhhcmR3YXJlIHdpbGwgYmUKYXdhcmUgb2Ygc3VjaCA4TSBw Z3RhYmxlIChmcm9tIGxldmVsLTEncyBlbnRyeSwgd2hlcmUgaXQgaGFzIGJpdCAyOC0yOSBzZXQK MDExYiksIHRoZW4gaXQgd29uJ3QgZXZlciByZWFkIGFueXRoaW5nIHN0YXJ0aW5nIGZyb20gIkxl dmVsLTIgRGVzY3JpcHRvcgoxIiAoYnV0IG9ubHkgcmVhZCB0aGUgb25seSBlbnRyeSAiTGV2ZWwt MiBEZXNjcmlwdG9yIDAiKSwgc28gZnVuZGFtZW50YWxseQpodWdlcGQgZm9ybWF0IG11c3QgbG9v ayBsaWtlIHN1Y2ggZm9yIDh4eD8KCkJ1dCB0aGVuIHBlcmhhcHMgaXQncyBzdGlsbCBjb21wYXRp YmxlIHdpdGggY29udC1wdGUgYmVjYXVzZSB0aGUgcmVzdAplbnRyaWVzIChwdGUgaW5kZXggMSsp IHdpbGwgc2ltcGx5IGJlIGlnbm9yZWQgYnkgdGhlIGhhcmR3YXJlPwoKPiAKPiBPbiBlNTAwIGl0 IGlzIGFsbCBzb2Z0d2FyZSBzbyBwYWdlcyAyTSBhbmQgbGFyZ2VyIHNob3VsZCBiZSBjb250LVBH RCAoYnkgCj4gdGhlIHdheSBJJ20gYSBiaXQgcHV6emxlZCB0aGF0IG9uIGFyY2hlcyB0aGF0IGhh dmUgb25seSAyIGxldmVscywgaWUgUEdEIAo+IGFuZCBQVEUsIHRoZSBQR0QgZW50cmllcyBhcmUg cG9wdWxhdGVkIGJ5IGEgZnVuY3Rpb24gY2FsbGVkIFBNRF9wb3B1bGF0ZSgpKS4KClllYWguLiBJ IGFtIGFsc28gd29uZGVyaW5nIHdoZXRoZXIgcGdkX3BvcHVsYXRlKCkgY291bGQgYWxzbyB3b3Jr IHRoZXJlCihwZXJoYXBzIHdpdGggc29tZSB0cml2aWFsIGNoYW5nZXMsIG9yIG1heWJlIG5vdCBl dmVuIG5lZWRlZC4uKSwgYXMgd2hlbgpwNGQvcHVkL3BtZCBsZXZlbHMgYXJlIG1pc3NpbmcsIGxp bnV4IHNob3VsZCBqdXN0IGRvIHNvbWV0aGluZyBsaWtlIGFuCmVuZm9yY2VkIGNhc3QgZnJvbSBw Z2RfdCogLT4gcG1kX3QqIGluIHRoaXMgY2FzZS4KCkkgdGhpbmsgY3VycmVudGx5IHRoZXkncmUg YWxyZWFkeSBub3QgcGdkLCBhcyBfX2ZpbmRfbGludXhfcHRlKCkgYWxyZWFkeQpza2lwcGVkIHBn ZCB1bmNvbmRpdGlvbmFsbHk6CgoJcGdkcCA9IHBnZGlyICsgcGdkX2luZGV4KGVhKTsKCXA0ZHAg PSBwNGRfb2Zmc2V0KHBnZHAsIGVhKTsKCj4gCj4gQ3VycmVudCBzaXR1YXRpb24gZm9yIDh4eCBp cyBpbGx1c3RyYXRlZCBoZXJlOiAKPiBodHRwczovL2dpdGh1Yi5jb20vbGludXhwcGMvd2lraS93 aWtpL0h1Z2UtcGFnZXMjOHh4Cj4gCj4gSSBhbHNvIHRyaWVkIHRvIGJldHRlciBpbGx1c3RyYXRl IGU1MDAvMzIgaGVyZTogCj4gaHR0cHM6Ly9naXRodWIuY29tL2xpbnV4cHBjL3dpa2kvd2lraS9I dWdlLXBhZ2VzI2U1MDAKPiAKPiBGb3IgNjQgYml0czoKPiBXZSBoYXZlIFBURS9QTUQvUFVEL1BH RCwgbm8gUDRECj4gCj4gU2VlIGFyY2gvcG93ZXJwYy9pbmNsdWRlL2FzbS9ub2hhc2gvNjQvcGd0 YWJsZS00ay5oCgpXZSBkb24ndCBoYXZlIGFueXRoaW5nIHRoYXQgaXMgYWJvdmUgcHVkIGluIHRo aXMgY2F0ZWdvcnksIHJpZ2h0PyAgVGhhdCdzCndoYXQgSSByZWFkIGZyb20geW91ciB3aWtpIChh bmQgdGhhbmtzIGZvciBwcm92aWRpbmcgdGhhdCBpbiB0aGUgZmlyc3QKcGxhY2U7IGhlbHBzIGEg bG90IGZvciBtZSB0byB1bmRlcnN0YW5kIGhvdyBpdCB3b3JrcyBvbiBQb3dlclBDKS4KCkkgd2Fu dCB0byBtYWtlIHN1cmUgaWYgSSBjYW4gbW92ZSBvbiB3aXRob3V0IGNhcmluZyBvbiBwNGQvcGdk IGxlYWZzIGxpa2UKd2hhdCB3ZSBkbyByaWdodCBub3csIGV2ZW4gYWZ0ZXIgaWYgd2UgY2FuIHJl bW92ZSBodWdlcGQgZm9yIGdvb2QsIGluIHRoaXMKY2FzZSBzaW5jZSBwNGQgYWx3YXlzIG1pc3Np bmcsIHRoZW4gaXQncyBhYm91dCB3aGV0aGVyICJwdWR8cG1kfHB0ZV9sZWFmKCkiCmNhbiBhbHNv IGNvdmVyIHRoZSBwZ2Qgb25lcyB3aGVuIHRoYXQgZGF5IGNvbWVzLCBpaXVjLgoKVGhhbmtzLAoK PiAKPiAKPiA+IAo+ID4gSSB0aGluayB0aGUgdHJpY2sgaGVyZSBpcyB0aGVyZSdsbCBiZSBubyBw Z2QgbGVhdmVzIGFmdGVyIGh1Z2VwZCBzcXVhc2hpbmcKPiA+IHRvIGxvd2VyIGxldmVscywgdGhl biBzaW5jZSBQb3dlclBDIHNlZW1zIHRvIG5ldmVyIGhhdmUgcDRkLCB0aGVuIGFsbAo+ID4gdGhp bmdzIGZhbGwgaW50byBwdWQgb3IgbG93ZXIuICBXZSBzZWVtIHRvIGJlIGFsbCBnb29kIHRoZXJl Pwo+ID4gCj4gPj4KPiA+PiBJZiB0aGUgZ29hbCBpcyB0byBwdXJnZSBodWdlcGQgdGhlbiBzb21l IG9mIHRoZSBvcHRpb25zIG1pZ2h0IHR1cm4gb3V0Cj4gPj4gdG8gY29udmVydCBodWdlcGQgaW50 byBodWdlIHA0ZC9wZ2QsIGFzIEkgdW5kZXJzdGFuZCBpdC4gSXQgd291bGQgYmUKPiA+PiBuaWNl IHRvIGhhdmUgY2VydGFpbnR5IG9uIHRoaXMgYXQgbGVhc3QuCj4gPiAKPiA+IFJpZ2h0LiAgSSBo b3BlIHRoZSBwbWQvcHVkIHBsYW4gSSBwcm9wb3NlZCBhYm92ZSBjYW4gYWxyZWFkeSB3b3JrIHRv byB3aXRoCj4gPiBzdWNoIGFtYmljaW91cyBnb2FsIHRvby4gIEJ1dCByZXZpZXcgdmVyeSB3ZWxj b21lZCBmcm9tIGVpdGhlciB5b3Ugb3IKPiA+IENocmlzdG9waGUuCj4gPiAKPiA+IFBTOiBJIHRo aW5rIEknbGwgYWxzbyBoYXZlIGEgY2xvc2VyIGxvb2sgYXQgQ2hyaXN0b3BoZSdzIHNlcmllcyB0 aGlzIHdlZWsKPiA+IG9yIG5leHQuCj4gPiAKPiA+Pgo+ID4+IFdlIGhhdmUgZWZmZWN0aXZlbHkg dGhyZWUgQVBJcyB0byBwYXJzZSBhIHNpbmdsZSBwYWdlIHRhYmxlIGFuZAo+ID4+IGN1cnJlbnRs eSBub25lIG9mIHRoZSBBUElzIGNhbiByZXR1cm4gMTAwJSBvZiB0aGUgZGF0YSBmb3IgcG93ZXIu Cj4gPiAKPiA+IFRoYW5rcywKPiA+IAoKLS0gClBldGVyIFh1CgoKX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxp c3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZy YWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B0B5CD128A for ; Wed, 10 Apr 2024 19:58:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 807886B0082; Wed, 10 Apr 2024 15:58:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 790FE6B0083; Wed, 10 Apr 2024 15:58:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E3386B0087; Wed, 10 Apr 2024 15:58:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3D6766B0082 for ; Wed, 10 Apr 2024 15:58:56 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B340B407FF for ; Wed, 10 Apr 2024 19:58:55 +0000 (UTC) X-FDA: 81994685430.26.E8D5A99 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf22.hostedemail.com (Postfix) with ESMTP id A846CC0007 for ; Wed, 10 Apr 2024 19:58:53 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="D/pED3XP"; spf=pass (imf22.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712779133; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wxs+2ga91aRZGeZs4inMwPcuYTjJzWrBvKW8/ZIuNzA=; b=g0Nsir7YsEXe6O3sMEylpImavRQmMUVOedeFBCIAFgZXbBAgCR9EHZVLiY7zoEdBVp8OjQ YgjYXWHytPKfF4r/VngUi3tr8m+R8Q1PviEI8O5hK/hgo94Jv2mGezZuqsdMZxgES0vSTa K4BsruU+FYceTXQGGO3EQ9dDpVk2/TM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712779133; a=rsa-sha256; cv=none; b=o9lUXpTQ/niY2ll0FAfFt4XvXThwkqYkdlyalHMwpLTE7/KEtisXIcxv18Rsdpp/wcO258 GDaWnVAF+NB8EnfKTyyLa9mWqZT0DjXgtnRgaGlkvx3SvMtvtZDjspXprPBH9JPlzskPyj TXbIjPsNbjFkrWwTN7IlGXAhqt1nOJ4= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="D/pED3XP"; spf=pass (imf22.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712779133; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wxs+2ga91aRZGeZs4inMwPcuYTjJzWrBvKW8/ZIuNzA=; b=D/pED3XPzuVAx3i/ucgy3FKgEoQczaorNR0kiIi5ViBSexiRTmvR3+mF898M0p2oxsCNTv 7uNrrnb58HrzP8Hyms9fs7+D7cgQnAMVA3nVIHfwzwt4ervIQ+E3adZkBXVOOY8fY23nfo UlB9/G10hGJ+WV2O0rkcY5Uk3t8XvqU= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-456-dniCzCzVOFma6kK2MVBOBQ-1; Wed, 10 Apr 2024 15:58:51 -0400 X-MC-Unique: dniCzCzVOFma6kK2MVBOBQ-1 Received: by mail-qv1-f71.google.com with SMTP id 6a1803df08f44-69b147e856aso15026966d6.1 for ; Wed, 10 Apr 2024 12:58:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712779131; x=1713383931; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wxs+2ga91aRZGeZs4inMwPcuYTjJzWrBvKW8/ZIuNzA=; b=bolhO7Nrgl9MEOGxR/K2r0wU2R9TXFo/fbpV2EYW1uXM9ENcEu97PmeG5f/LOhi/Vm WU8oa1gG9LqkOowj5uDzavlhYpUmLJo8FKQcBvxDZQEgnMwywzGQQ4f8dN5PUGzLkUWp +T4h3COjJWWZF/dGSEemX9lxZtAgJIfnwlJ8KeJnmsdxsPxuMgM1tPbWCMGT2WVkz6vd BShnl1MdV8/2KWff9Fn4w2eYzqtp+atCGyetXag8GcmD9po+bDKIGEtwxNXZnvAwP7I6 Cv4uspya1SDOKpTbAjHdMjEjJWYUiHoN64A622d7H3IVcoLni+AHdKDqqe1K/B35YdHi 2JZw== X-Forwarded-Encrypted: i=1; AJvYcCVGMsFxhIumhEK7D6qsTV1TaTbVX0bNvyUtlzZZz6KKd8N4E8jw42Z+FBPTZPFjO/dPnJs13+eSePhEucKQiQN61WQ= X-Gm-Message-State: AOJu0YyOf36LDXIFBOlhGFzlR4VSnHI2JYhfSLLfujP/4J+Qmkv/RuJu lHAee26TQXnRtp85XS9d/BBJslgKFJfLKccUhQdSzQQzfYD19BsWbjZNCEgTNugEMCPmY+2D8y2 AoipdOf0cObWXH/gAY8a5QgS9UDgDNlBsUBW8ksvqluajOBYl X-Received: by 2002:a05:620a:4bc9:b0:78d:7760:e797 with SMTP id sw9-20020a05620a4bc900b0078d7760e797mr2835620qkn.2.1712779130648; Wed, 10 Apr 2024 12:58:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEv1bdfgwWalbeb7W8lw1wQ8AyOawE9AJrFWRW0xu4CX1VzWDIy7HrETiPCNEoM1hdJNWYFiA== X-Received: by 2002:a05:620a:4bc9:b0:78d:7760:e797 with SMTP id sw9-20020a05620a4bc900b0078d7760e797mr2835572qkn.2.1712779129838; Wed, 10 Apr 2024 12:58:49 -0700 (PDT) Received: from x1n (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id vy23-20020a05620a491700b0078d66e2ac1bsm1043qkn.61.2024.04.10.12.58.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 12:58:49 -0700 (PDT) Date: Wed, 10 Apr 2024 15:58:47 -0400 From: Peter Xu To: Christophe Leroy Cc: Jason Gunthorpe , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , "linuxppc-dev@lists.ozlabs.org" , Michael Ellerman , Matthew Wilcox , Rik van Riel , Lorenzo Stoakes , Axel Rasmussen , Yang Shi , John Hubbard , "linux-arm-kernel@lists.infradead.org" , "Kirill A . Shutemov" , Andrew Jones , Vlastimil Babka , Mike Rapoport , Andrew Morton , Muchun Song , Christoph Hellwig , "linux-riscv@lists.infradead.org" , James Houghton , David Hildenbrand , Andrea Arcangeli , "Aneesh Kumar K . V" , Mike Kravetz Subject: Re: [PATCH v3 00/12] mm/gup: Unify hugetlb, part 2 Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240322161000.GJ159172@nvidia.com> <20240326140252.GH6245@nvidia.com> <20240405181633.GH5383@nvidia.com> <20240409234355.GJ5383@nvidia.com> <7b75c4db-9dbe-4ff1-b649-06a9218ae0aa@csgroup.eu> MIME-Version: 1.0 In-Reply-To: <7b75c4db-9dbe-4ff1-b649-06a9218ae0aa@csgroup.eu> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Stat-Signature: ho541quottiqskp338jq53fp848ghab3 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: A846CC0007 X-Rspam-User: X-HE-Tag: 1712779133-559735 X-HE-Meta: U2FsdGVkX19gDOfja+QVCYaNoWtn0KB/jNpQKRtzlOaQFEMRLY1swjnBkSvRFkJE2JjYBIl6UkMXBzPuI7oflwmkskEtokKxiHoGbknLMWUKZ1a5TxjYPfVp8xSevDkc9AyQW2CzNaytHSkjznyEQHLeH8kTKw18M3PaUOry1/qzFlrgWuGKH+vEdivfADPYmhV91UHlZx3bn2vpMKcxRGBaVxtd0eMxju00E1tpPY1Zms+yGFdX4dwrsF0xML3VEtqfdV8X9LzyBaHitbMIZzsxbC5OvIO1HqPnEiWAagnH0VA1iaQq3e86TgB5Bc6i9D8CTBasGBCuzcBtYfEsk75XtyRALjSqSfqqb4ccpPNwGTW6+yiktP9mZ/ICjw4vd8RvtM38hL+EIPWIcPnASXi/ZrmnYKPBevf4ZnHtJBPaByy4AwyVTs8HIxuqka+/r9PaMPBEYO/NYxHou4imWvPb/RaHCgP7XFlv3fEi/O1VMNrLp90tHAftc7oSTHQKsT1lK1HXG7FejcYXE7Hvh2GChYTFN1tPJoT5bwDJQiYoTTQhPBti3l2anAQQxXmYfqWjfRgEfAVbtK58qRrTFXXmh6FgecHjMSXQPIPQz9zopbqoE45pqcI/Ezl9NsyBvCK/+uEEs2CkrepmeNcTgwPMfl+unb8qe/eiF+9btpEJGUqnGxbmrD4Qda3iNvKdL+s431ts4B8g8lH00kcOtqEEj2+OIM/DN7/SY8nXkBEE326mr6ZhsGc47ow5RmaOwogMfB32k4ABGYoCfrntklHeLW2Wl682qQSpHVqBoqYeF4tyO1scvYzI5k98NCx+jkoUB4WBkZi8AFnpTvFVSTnMWm3LLpVtC+BgrIUtHIxNzvy0cn7LiDg3don+eed8NEr0eL0UwuaFliL/+ZISiBWxVkGXZZmnry72TsYSQDb3AqsMWWCWH9ScPAq/qg1rGZ39I8eKUP5NCzvckXs +aRpcu/d 3maetc41PYi2onlp9IUg32Zq3B8pWlabKy2DHU8Nph4JU0HNbIu8PTul6c0IoK4ixosbveOww9WDVS5shXzxhtID18t8s1bMKAZ2Ohl3VDMYlOcBwMzviQk6JNLwbNutalCav X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Apr 10, 2024 at 04:30:41PM +0000, Christophe Leroy wrote: > > > Le 10/04/2024 à 17:28, Peter Xu a écrit : > > On Tue, Apr 09, 2024 at 08:43:55PM -0300, Jason Gunthorpe wrote: > >> On Fri, Apr 05, 2024 at 05:42:44PM -0400, Peter Xu wrote: > >>> In short, hugetlb mappings shouldn't be special comparing to other huge pXd > >>> and large folio (cont-pXd) mappings for most of the walkers in my mind, if > >>> not all. I need to look at all the walkers and there can be some tricky > >>> ones, but I believe that applies in general. It's actually similar to what > >>> I did with slow gup here. > >> > >> I think that is the big question, I also haven't done the research to > >> know the answer. > >> > >> At this point focusing on moving what is reasonable to the pXX_* API > >> makes sense to me. Then reviewing what remains and making some > >> decision. > >> > >>> Like this series, for cont-pXd we'll need multiple walks comparing to > >>> before (when with hugetlb_entry()), but for that part I'll provide some > >>> performance tests too, and we also have a fallback plan, which is to detect > >>> cont-pXd existance, which will also work for large folios. > >> > >> I think we can optimize this pretty easy. > >> > >>>> I think if you do the easy places for pXX conversion you will have a > >>>> good idea about what is needed for the hard places. > >>> > >>> Here IMHO we don't need to understand "what is the size of this hugetlb > >>> vma" > >> > >> Yeh, I never really understood why hugetlb was linked to the VMA.. The > >> page table is self describing, obviously. > > > > Attaching to vma still makes sense to me, where we should definitely avoid > > a mixture of hugetlb and !hugetlb pages in a single vma - hugetlb pages are > > allocated, managed, ... totally differently. > > > > And since hugetlb is designed as file-based (which also makes sense to me, > > at least for now), it's also natural that it's vma-attached. > > > >> > >>> or "which level of pgtable does this hugetlb vma pages locate", > >> > >> Ditto > >> > >>> because we may not need that, e.g., when we only want to collect some smaps > >>> statistics. "whether it's hugetlb" may matter, though. E.g. in the mm > >>> walker we see a huge pmd, it can be a thp, it can be a hugetlb (when > >>> hugetlb_entry removed), we may need extra check later to put things into > >>> the right bucket, but for the walker itself it doesn't necessarily need > >>> hugetlb_entry(). > >> > >> Right, places may still need to know it is part of a huge VMA because we > >> have special stuff linked to that. > >> > >>>> But then again we come back to power and its big list of page sizes > >>>> and variety :( Looks like some there have huge sizes at the pgd level > >>>> at least. > >>> > >>> Yeah this is something I want to be super clear, because I may miss > >>> something: we don't have real pgd pages, right? Powerpc doesn't even > >>> define p4d_leaf(), AFAICT. > >> > >> AFAICT it is because it hides it all in hugepd. > > > > IMHO one thing we can benefit from such hugepd rework is, if we can squash > > all the hugepds like what Christophe does, then we push it one more layer > > down, and we have a good chance all things should just work. > > > > So again my Power brain is close to zero, but now I'm referring to what > > Christophe shared in the other thread: > > > > https://github.com/linuxppc/wiki/wiki/Huge-pages > > > > Together with: > > > > https://lore.kernel.org/r/288f26f487648d21fd9590e40b390934eaa5d24a.1711377230.git.christophe.leroy@csgroup.eu > > > > Where it has: > > > > --- a/arch/powerpc/platforms/Kconfig.cputype > > +++ b/arch/powerpc/platforms/Kconfig.cputype > > @@ -98,6 +98,7 @@ config PPC_BOOK3S_64 > > select ARCH_ENABLE_HUGEPAGE_MIGRATION if HUGETLB_PAGE && MIGRATION > > select ARCH_ENABLE_SPLIT_PMD_PTLOCK > > select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE > > + select ARCH_HAS_HUGEPD if HUGETLB_PAGE > > select ARCH_SUPPORTS_HUGETLBFS > > select ARCH_SUPPORTS_NUMA_BALANCING > > select HAVE_MOVE_PMD > > @@ -290,6 +291,7 @@ config PPC_BOOK3S > > config PPC_E500 > > select FSL_EMB_PERFMON > > bool > > + select ARCH_HAS_HUGEPD if HUGETLB_PAGE > > select ARCH_SUPPORTS_HUGETLBFS if PHYS_64BIT || PPC64 > > select PPC_SMP_MUXED_IPI > > select PPC_DOORBELL > > > > So I think it means we have three PowerPC systems that supports hugepd > > right now (besides the 8xx which Christophe is trying to drop support > > there), besides 8xx we still have book3s_64 and E500. > > > > Let's check one by one: > > > > - book3s_64 > > > > - hash > > > > - 64K: p4d is not used, largest pgsize pgd 16G @pud level. It > > means after squashing it'll be a bunch of cont-pmd, all good. > > > > - 4K: p4d also not used, largest pgsize pgd 128G, after squashed > > it'll be cont-pud. all good. > > > > - radix > > > > - 64K: largest 1G @pud, then cont-pmd after squashed. all good. > > > > - 4K: largest 1G @pud, then cont-pmd, all good. > > > > - e500 & 8xx > > > > - both of them use 2-level pgtables (pgd + pte), after squashed hugepd > > @pgd level they become cont-pte. all good. > > e500 has two modes: 32 bits and 64 bits. > > For 32 bits: > > 8xx is the only one handling it through HW-assisted pagetable walk hence > requiring a 2-level whatever the pagesize is. Hmm I think maybe finally I get it.. I think the confusion came from when I saw there's always such level-2 table described in Figure 8-5 of the manual: https://www.nxp.com/docs/en/reference-manual/MPC860UM.pdf So I suppose you meant for 8M, the PowerPC 8xx system hardware will be aware of such 8M pgtable (from level-1's entry, where it has bit 28-29 set 011b), then it won't ever read anything starting from "Level-2 Descriptor 1" (but only read the only entry "Level-2 Descriptor 0"), so fundamentally hugepd format must look like such for 8xx? But then perhaps it's still compatible with cont-pte because the rest entries (pte index 1+) will simply be ignored by the hardware? > > On e500 it is all software so pages 2M and larger should be cont-PGD (by > the way I'm a bit puzzled that on arches that have only 2 levels, ie PGD > and PTE, the PGD entries are populated by a function called PMD_populate()). Yeah.. I am also wondering whether pgd_populate() could also work there (perhaps with some trivial changes, or maybe not even needed..), as when p4d/pud/pmd levels are missing, linux should just do something like an enforced cast from pgd_t* -> pmd_t* in this case. I think currently they're already not pgd, as __find_linux_pte() already skipped pgd unconditionally: pgdp = pgdir + pgd_index(ea); p4dp = p4d_offset(pgdp, ea); > > Current situation for 8xx is illustrated here: > https://github.com/linuxppc/wiki/wiki/Huge-pages#8xx > > I also tried to better illustrate e500/32 here: > https://github.com/linuxppc/wiki/wiki/Huge-pages#e500 > > For 64 bits: > We have PTE/PMD/PUD/PGD, no P4D > > See arch/powerpc/include/asm/nohash/64/pgtable-4k.h We don't have anything that is above pud in this category, right? That's what I read from your wiki (and thanks for providing that in the first place; helps a lot for me to understand how it works on PowerPC). I want to make sure if I can move on without caring on p4d/pgd leafs like what we do right now, even after if we can remove hugepd for good, in this case since p4d always missing, then it's about whether "pud|pmd|pte_leaf()" can also cover the pgd ones when that day comes, iiuc. Thanks, > > > > > > I think the trick here is there'll be no pgd leaves after hugepd squashing > > to lower levels, then since PowerPC seems to never have p4d, then all > > things fall into pud or lower. We seem to be all good there? > > > >> > >> If the goal is to purge hugepd then some of the options might turn out > >> to convert hugepd into huge p4d/pgd, as I understand it. It would be > >> nice to have certainty on this at least. > > > > Right. I hope the pmd/pud plan I proposed above can already work too with > > such ambicious goal too. But review very welcomed from either you or > > Christophe. > > > > PS: I think I'll also have a closer look at Christophe's series this week > > or next. > > > >> > >> We have effectively three APIs to parse a single page table and > >> currently none of the APIs can return 100% of the data for power. > > > > Thanks, > > -- Peter Xu