From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B70DDC001DE for ; Fri, 4 Aug 2023 21:31:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=EgQRXgFfkHoQdJtaeHP4msadBCsQ0el9WdmMrbLQhM4=; b=VEarfdUhcMDY6s bv6hH+hthrZmyg70eOZvwSgesyMIKNKHb1BnVaJdeP1nMGu876QoIszlkPKo/x9fsH1tn7LdoIDOk 3I2K2IhrakRistIDldWQX/dFE72otpApDG0oF2qTDLkwA1kcypUmhDRX7uuDnS3R7+bAm7hZO6Vx2 WHgYpreKL1HnCZIXysnWk6599VJ+dKkG7hCchxYJym5g7B6yakwgEqQJfvUaqpA6NsI2R/THa2fA5 HN/903hVfwAXR5Rekai9Inld0n9UVd3/h0KkVeRm2SVzbhlvMdxy2ZPZu9tqz6i6ieEzyt89ZcWgr jCVxPImeer/MbzC25WLw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qS2OA-00DHx2-2h; Fri, 04 Aug 2023 21:31:14 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qS2O6-00DHwJ-2x for linux-arm-kernel@lists.infradead.org; Fri, 04 Aug 2023 21:31:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691184670; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6xHzxUYjJIdDtURU1S3vX2xZDjMKXI19w61gLbe1Ayo=; b=gGF9pAqBIU8zRuL4o7SYZTf5tA9e9YLsEkREjkDglrQWglyBbBaKp/Ml+6S0Ko3ljzipur MAja4516Pl1f663F9Lf8Eklgk56/Gkm1CkIAKfMOAvX482NnHRLnlKtG+W6hRly6mPwNMV qGaa3n5/Ho/nNyOZs3DRJD+cfXT9b3E= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-447-8QrLqZshPpygL-ThjvRWZQ-1; Fri, 04 Aug 2023 17:31:08 -0400 X-MC-Unique: 8QrLqZshPpygL-ThjvRWZQ-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-3fe182913c5so15091855e9.0 for ; Fri, 04 Aug 2023 14:31:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691184652; x=1691789452; h=content-transfer-encoding:in-reply-to:organization:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6xHzxUYjJIdDtURU1S3vX2xZDjMKXI19w61gLbe1Ayo=; b=bbrno2zLScOOrZYOJKMkO53OXzy3Z0T+5UaxhXhN7OReQmG0DYv5Hvc1PGGVVWve/N O0DahsJlemJujAMQfN49xZx+zAocWjrRrEgfLbl54DsSui1DaLPV2OGJJ1r9z5QuW97a NF3jfW+7zHcU4vyY/XGpHMu1Q5npFh30JrNQHiPyS3HQe8wY3/LUC0JgnW+LDRhKocWO 9tH61uilXu5nT2MzLz5cwouG47jlQMt8wwMBBGbuBJDu7ejX0XK1VQQwlGQZOfRNzjYq oEWd/sErpU+P4c+Z8F3SlGlZTvSko18tCOKgwNg8/XdQjkvxUH84D5pbkUFw3ocVSHXQ yo4g== X-Gm-Message-State: AOJu0YzGC86SAAKgEWrueva59AVSHKU1wO+9SH0Erx8w6CCxwkL5aJ+H ibNjJt5yY33N7UoN+85p4RHijkxL2pxfI1xhnR9Kp9+BRLY/AjvN3ElpJfoTYWfEdGu+lFxAN0w MK9PH0I82hc5wuPlGNteU6S1FoVEJSMCyOsQ= X-Received: by 2002:a7b:ce94:0:b0:3f8:2777:15e with SMTP id q20-20020a7bce94000000b003f82777015emr2164965wmj.31.1691184652193; Fri, 04 Aug 2023 14:30:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHg76fpCPdISu/CKKhttE0AnUkU9rMMWiHgz3qJGhEnE09KvQCL77IQ1NcO4miQKPYK6bEQ/A== X-Received: by 2002:a7b:ce94:0:b0:3f8:2777:15e with SMTP id q20-20020a7bce94000000b003f82777015emr2164947wmj.31.1691184651768; Fri, 04 Aug 2023 14:30:51 -0700 (PDT) Received: from ?IPV6:2003:d8:2f2d:8e00:a20e:59bc:3c13:4806? (p200300d82f2d8e00a20e59bc3c134806.dip0.t-ipconnect.de. [2003:d8:2f2d:8e00:a20e:59bc:3c13:4806]) by smtp.gmail.com with ESMTPSA id 25-20020a05600c021900b003fe4ca8decdsm1425819wmi.31.2023.08.04.14.30.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 04 Aug 2023 14:30:51 -0700 (PDT) Message-ID: <0d502268-ebdc-8462-d88c-e6a41578d9ae@redhat.com> Date: Fri, 4 Aug 2023 23:30:49 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: [PATCH v4 2/5] mm: LARGE_ANON_FOLIO for improved performance To: Yu Zhao Cc: Ryan Roberts , Andrew Morton , Matthew Wilcox , Yin Fengwei , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , "Huang, Ying" , Zi Yan , Luis Chamberlain , Itaru Kitayama , "Kirill A. Shutemov" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org References: <20230726095146.2826796-1-ryan.roberts@arm.com> <20230726095146.2826796-3-ryan.roberts@arm.com> <5e595904-3dca-0e15-0769-7ed10975fd0d@arm.com> <259ad8fc-c12b-69b9-ba16-adb9e3e6d672@redhat.com> From: David Hildenbrand Organization: Red Hat In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230804_143111_093858_7B937417 X-CRM114-Status: GOOD ( 40.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gMDQuMDguMjMgMjM6MjYsIFl1IFpoYW8gd3JvdGU6Cj4gT24gRnJpLCBBdWcgNCwgMjAyMyBh dCAzOjEz4oCvUE0gRGF2aWQgSGlsZGVuYnJhbmQgPGRhdmlkQHJlZGhhdC5jb20+IHdyb3RlOgo+ Pgo+PiBPbiAwNC4wOC4yMyAyMzowMCwgWXUgWmhhbyB3cm90ZToKPj4+IE9uIEZyaSwgQXVnIDQs IDIwMjMgYXQgMjoyM+KAr1BNIERhdmlkIEhpbGRlbmJyYW5kIDxkYXZpZEByZWRoYXQuY29tPiB3 cm90ZToKPj4+Pgo+Pj4+IE9uIDA0LjA4LjIzIDEwOjI3LCBSeWFuIFJvYmVydHMgd3JvdGU6Cj4+ Pj4+IE9uIDA0LzA4LzIwMjMgMDA6NTAsIFl1IFpoYW8gd3JvdGU6Cj4+Pj4+PiBPbiBUaHUsIEF1 ZyAzLCAyMDIzIGF0IDY6NDPigK9BTSBSeWFuIFJvYmVydHMgPHJ5YW4ucm9iZXJ0c0Bhcm0uY29t PiB3cm90ZToKPj4+Pj4+Pgo+Pj4+Pj4+ICsgS2lyaWxsCj4+Pj4+Pj4KPj4+Pj4+PiBPbiAyNi8w Ny8yMDIzIDEwOjUxLCBSeWFuIFJvYmVydHMgd3JvdGU6Cj4+Pj4+Pj4+IEludHJvZHVjZSBMQVJH RV9BTk9OX0ZPTElPIGZlYXR1cmUsIHdoaWNoIGFsbG93cyBhbm9ueW1vdXMgbWVtb3J5IHRvIGJl Cj4+Pj4+Pj4+IGFsbG9jYXRlZCBpbiBsYXJnZSBmb2xpb3Mgb2YgYSBkZXRlcm1pbmVkIG9yZGVy LiBBbGwgcGFnZXMgb2YgdGhlIGxhcmdlCj4+Pj4+Pj4+IGZvbGlvIGFyZSBwdGUtbWFwcGVkIGR1 cmluZyB0aGUgc2FtZSBwYWdlIGZhdWx0LCBzaWduaWZpY2FudGx5IHJlZHVjaW5nCj4+Pj4+Pj4+ IHRoZSBudW1iZXIgb2YgcGFnZSBmYXVsdHMuIFRoZSBudW1iZXIgb2YgcGVyLXBhZ2Ugb3BlcmF0 aW9ucyAoZS5nLiByZWYKPj4+Pj4+Pj4gY291bnRpbmcsIHJtYXAgbWFuYWdlbWVudCBscnUgbGlz dCBtYW5hZ2VtZW50KSBhcmUgYWxzbyBzaWduaWZpY2FudGx5Cj4+Pj4+Pj4+IHJlZHVjZWQgc2lu Y2UgdGhvc2Ugb3BzIG5vdyBiZWNvbWUgcGVyLWZvbGlvLgo+Pj4+Pj4+Pgo+Pj4+Pj4+PiBUaGUg bmV3IGJlaGF2aW91ciBpcyBoaWRkZW4gYmVoaW5kIHRoZSBuZXcgTEFSR0VfQU5PTl9GT0xJTyBL Y29uZmlnLAo+Pj4+Pj4+PiB3aGljaCBkZWZhdWx0cyB0byBkaXNhYmxlZCBmb3Igbm93OyBUaGUg bG9uZyB0ZXJtIGFpbSBpcyBmb3IgdGhpcyB0bwo+Pj4+Pj4+PiBkZWZhdXQgdG8gZW5hYmxlZCwg YnV0IHRoZXJlIGFyZSBzb21lIHJpc2tzIGFyb3VuZCBpbnRlcm5hbAo+Pj4+Pj4+PiBmcmFnbWVu dGF0aW9uIHRoYXQgbmVlZCB0byBiZSBiZXR0ZXIgdW5kZXJzdG9vZCBmaXJzdC4KPj4+Pj4+Pj4K Pj4+Pj4+Pj4gV2hlbiBlbmFibGVkLCB0aGUgZm9saW8gb3JkZXIgaXMgZGV0ZXJtaW5lZCBhcyBz dWNoOiBGb3IgYSB2bWEsIHByb2Nlc3MKPj4+Pj4+Pj4gb3Igc3lzdGVtIHRoYXQgaGFzIGV4cGxp Y2l0bHkgZGlzYWJsZWQgVEhQLCB3ZSBjb250aW51ZSB0byBhbGxvY2F0ZQo+Pj4+Pj4+PiBvcmRl ci0wLiBUSFAgaXMgbW9zdCBsaWtlbHkgZGlzYWJsZWQgdG8gYXZvaWQgYW55IHBvc3NpYmxlIGlu dGVybmFsCj4+Pj4+Pj4+IGZyYWdtZW50YXRpb24gc28gd2UgaG9ub3VyIHRoYXQgcmVxdWVzdC4K Pj4+Pj4+Pj4KPj4+Pj4+Pj4gT3RoZXJ3aXNlLCB0aGUgcmV0dXJuIHZhbHVlIG9mIGFyY2hfd2Fu dHNfcHRlX29yZGVyKCkgaXMgdXNlZC4gRm9yIHZtYXMKPj4+Pj4+Pj4gdGhhdCBoYXZlIG5vdCBl eHBsaWNpdGx5IG9wdGVkLWluIHRvIHVzZSB0cmFuc3BhcmVudCBodWdlcGFnZXMgKGUuZy4KPj4+ Pj4+Pj4gd2hlcmUgdGhwPW1hZHZpc2UgYW5kIHRoZSB2bWEgZG9lcyBub3QgaGF2ZSBNQURWX0hV R0VQQUdFKSwgdGhlbgo+Pj4+Pj4+PiBhcmNoX3dhbnRzX3B0ZV9vcmRlcigpIGlzIGxpbWl0ZWQg dG8gNjRLIChvciBQQUdFX1NJWkUsIHdoaWNoZXZlciBpcwo+Pj4+Pj4+PiBiaWdnZXIpLiBUaGlz IGFsbG93cyBmb3IgYSBwZXJmb3JtYW5jZSBib29zdCB3aXRob3V0IHJlcXVpcmluZyBhbnkKPj4+ Pj4+Pj4gZXhwbGljaXQgb3B0LWluIGZyb20gdGhlIHdvcmtsb2FkIHdoaWxlIGxpbWl0dGluZyBp bnRlcm5hbAo+Pj4+Pj4+PiBmcmFnbWVudGF0aW9uLgo+Pj4+Pj4+Pgo+Pj4+Pj4+PiBJZiB0aGUg cHJlZmVycmVkIG9yZGVyIGNhbid0IGJlIHVzZWQgKGUuZy4gYmVjYXVzZSB0aGUgZm9saW8gd291 bGQKPj4+Pj4+Pj4gYnJlYWNoIHRoZSBib3VuZHMgb2YgdGhlIHZtYSwgb3IgYmVjYXVzZSBwdGVz IGluIHRoZSByZWdpb24gYXJlIGFscmVhZHkKPj4+Pj4+Pj4gbWFwcGVkKSB0aGVuIHdlIGZhbGwg YmFjayB0byBhIHN1aXRhYmxlIGxvd2VyIG9yZGVyOyBmaXJzdAo+Pj4+Pj4+PiBQQUdFX0FMTE9D X0NPU1RMWV9PUkRFUiwgdGhlbiBvcmRlci0wLgo+Pj4+Pj4+Pgo+Pj4+Pj4+Cj4+Pj4+Pj4gLi4u Cj4+Pj4+Pj4KPj4+Pj4+Pj4gKyNkZWZpbmUgQU5PTl9GT0xJT19NQVhfT1JERVJfVU5ISU5URUQg XAo+Pj4+Pj4+PiArICAgICAgICAgICAgIChpbG9nMihtYXhfdCh1bnNpZ25lZCBsb25nLCBTWl82 NEssIFBBR0VfU0laRSkpIC0gUEFHRV9TSElGVCkKPj4+Pj4+Pj4gKwo+Pj4+Pj4+PiArc3RhdGlj IGludCBhbm9uX2ZvbGlvX29yZGVyKHN0cnVjdCB2bV9hcmVhX3N0cnVjdCAqdm1hKQo+Pj4+Pj4+ PiArewo+Pj4+Pj4+PiArICAgICBpbnQgb3JkZXI7Cj4+Pj4+Pj4+ICsKPj4+Pj4+Pj4gKyAgICAg LyoKPj4+Pj4+Pj4gKyAgICAgICogSWYgVEhQIGlzIGV4cGxpY2l0bHkgZGlzYWJsZWQgZm9yIGVp dGhlciB0aGUgdm1hLCB0aGUgcHJvY2VzcyBvciB0aGUKPj4+Pj4+Pj4gKyAgICAgICogc3lzdGVt LCB0aGVuIHRoaXMgaXMgdmVyeSBsaWtlbHkgaW50ZW5kZWQgdG8gbGltaXQgaW50ZXJuYWwKPj4+ Pj4+Pj4gKyAgICAgICogZnJhZ21lbnRhdGlvbjsgaW4gdGhpcyBjYXNlLCBkb24ndCBhdHRlbXB0 IHRvIGFsbG9jYXRlIGEgbGFyZ2UKPj4+Pj4+Pj4gKyAgICAgICogYW5vbnltb3VzIGZvbGlvLgo+ Pj4+Pj4+PiArICAgICAgKgo+Pj4+Pj4+PiArICAgICAgKiBFbHNlLCBpZiB0aGUgdm1hIGlzIGVs aWdpYmxlIGZvciB0aHAsIGFsbG9jYXRlIGEgbGFyZ2UgZm9saW8gb2YgdGhlCj4+Pj4+Pj4+ICsg ICAgICAqIHNpemUgcHJlZmVycmVkIGJ5IHRoZSBhcmNoLiBPciBpZiB0aGUgYXJjaCByZXF1ZXN0 ZWQgYSB2ZXJ5IHNtYWxsCj4+Pj4+Pj4+ICsgICAgICAqIHNpemUgb3IgZGlkbid0IHJlcXVlc3Qg YSBzaXplLCB0aGVuIHVzZSBQQUdFX0FMTE9DX0NPU1RMWV9PUkRFUiwKPj4+Pj4+Pj4gKyAgICAg ICogd2hpY2ggc3RpbGwgbWVldHMgdGhlIGFyY2gncyByZXF1aXJlbWVudHMgYnV0IG1lYW5zIHdl IHN0aWxsIHRha2UKPj4+Pj4+Pj4gKyAgICAgICogYWR2YW50YWdlIG9mIFNXIG9wdGltaXphdGlv bnMgKGUuZy4gZmV3ZXIgcGFnZSBmYXVsdHMpLgo+Pj4+Pj4+PiArICAgICAgKgo+Pj4+Pj4+PiAr ICAgICAgKiBGaW5hbGx5IGlmIHRocCBpcyBlbmFibGVkIGJ1dCB0aGUgdm1hIGlzbid0IGVsaWdp YmxlLCB0YWtlIHRoZQo+Pj4+Pj4+PiArICAgICAgKiBhcmNoLXByZWZlcnJlZCBzaXplIGFuZCBs aW1pdCBpdCB0byBBTk9OX0ZPTElPX01BWF9PUkRFUl9VTkhJTlRFRC4KPj4+Pj4+Pj4gKyAgICAg ICogVGhpcyBlbnN1cmVzIHdvcmtsb2FkcyB0aGF0IGhhdmUgbm90IGV4cGxpY2l0bHkgb3B0ZWQt aW4gdGFrZSBiZW5lZml0Cj4+Pj4+Pj4+ICsgICAgICAqIHdoaWxlIGNhcHBpbmcgdGhlIHBvdGVu dGlhbCBmb3IgaW50ZXJuYWwgZnJhZ21lbnRhdGlvbi4KPj4+Pj4+Pj4gKyAgICAgICovCj4+Pj4+ Pj4+ICsKPj4+Pj4+Pj4gKyAgICAgaWYgKCh2bWEtPnZtX2ZsYWdzICYgVk1fTk9IVUdFUEFHRSkg fHwKPj4+Pj4+Pj4gKyAgICAgICAgIHRlc3RfYml0KE1NRl9ESVNBQkxFX1RIUCwgJnZtYS0+dm1f bW0tPmZsYWdzKSB8fAo+Pj4+Pj4+PiArICAgICAgICAgIWh1Z2VwYWdlX2ZsYWdzX2VuYWJsZWQo KSkKPj4+Pj4+Pj4gKyAgICAgICAgICAgICBvcmRlciA9IDA7Cj4+Pj4+Pj4+ICsgICAgIGVsc2Ug ewo+Pj4+Pj4+PiArICAgICAgICAgICAgIG9yZGVyID0gbWF4KGFyY2hfd2FudHNfcHRlX29yZGVy KCksIFBBR0VfQUxMT0NfQ09TVExZX09SREVSKTsKPj4+Pj4+Pj4gKwo+Pj4+Pj4+PiArICAgICAg ICAgICAgIGlmICghaHVnZXBhZ2Vfdm1hX2NoZWNrKHZtYSwgdm1hLT52bV9mbGFncywgZmFsc2Us IHRydWUsIHRydWUpKQo+Pj4+Pj4+PiArICAgICAgICAgICAgICAgICAgICAgb3JkZXIgPSBtaW4o b3JkZXIsIEFOT05fRk9MSU9fTUFYX09SREVSX1VOSElOVEVEKTsKPj4+Pj4+Pj4gKyAgICAgfQo+ Pj4+Pj4+PiArCj4+Pj4+Pj4+ICsgICAgIHJldHVybiBvcmRlcjsKPj4+Pj4+Pj4gK30KPj4+Pj4+ Pgo+Pj4+Pj4+Cj4+Pj4+Pj4gSGkgQWxsLAo+Pj4+Pj4+Cj4+Pj4+Pj4gSSdtIHdyaXRpbmcgdXAg dGhlIGNvbmNsdXNpb25zIHRoYXQgd2UgYXJyaXZlZCBhdCBkdXJpbmcgZGlzY3Vzc2lvbiBpbiB0 aGUgVEhQCj4+Pj4+Pj4gbWVldGluZyB5ZXN0ZXJkYXksIHJlZ2FyZGluZyBsaW5rYWdlIHdpdGgg ZXhpdGluZyBUSFAgQUJJcy4gSXQgd291bGQgYmUgZ3JlYXQgaWYKPj4+Pj4+PiBJIGNhbiBnZXQg ZXhwbGljaXQgImFncmVlIiBvciBkaXNhZ3JlZSArIHJhdGlvbmFsZSBmcm9tIGF0IGxlYXN0IERh dmlkLCBZdSBhbmQKPj4+Pj4+PiBLaXJpbGwuCj4+Pj4+Pj4KPj4+Pj4+PiBJbiBzdW1tYXJ5OyBJ IHRoaW5rIHdlIGFyZSBjb252ZXJnaW5nIG9uIHRoZSBhcHByb2FjaCB0aGF0IGlzIGFscmVhZHkg Y29kZWQsIGJ1dAo+Pj4+Pj4+IEknZCBsaWtlIGNvbmZpcm1hdGlvbi4KPj4+Pj4+Pgo+Pj4+Pj4+ Cj4+Pj4+Pj4KPj4+Pj4+PiBUaGUgVEhQIHNpdHVhdGlvbiB0b2RheQo+Pj4+Pj4+IC0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tCj4+Pj4+Pj4KPj4+Pj4+PiAgICAgLSBBdCBzeXN0ZW0gbGV2ZWw6IFRI UCBjYW4gYmUgc2V0IHRvICJuZXZlciIsICJtYWR2aXNlIiBvciAiYWx3YXlzIgo+Pj4+Pj4+ICAg ICAtIEF0IHByb2Nlc3MgbGV2ZWw6IFRIUCBjYW4gYmUgIm5ldmVyIiBvciAiZGVmZXIgdG8gc3lz dGVtIHNldHRpbmciCj4+Pj4+Pj4gICAgIC0gQXQgVk1BIGxldmVsOiBuby1oaW50LCBNQURWX0hV R0VQQUdFLCBNQURWX05PSFVHRVBBR0UKPj4+Pj4+Pgo+Pj4+Pj4+IFRoYXQgZ2l2ZXMgdXMgdGhp cyB0YWJsZSB0byBkZXNjcmliZSBob3cgYSBwYWdlIGZhdWx0IGlzIGhhbmRsZWQsIGFjY29yZGlu ZyB0bwo+Pj4+Pj4+IHByb2Nlc3Mgc3RhdGUgKGNvbHVtbnMpIGFuZCB2bWEgZmxhZ3MgKHJvd3Mp Ogo+Pj4+Pj4+Cj4+Pj4+Pj4gICAgICAgICAgICAgICAgICAgIHwgbmV2ZXIgICAgIHwgbWFkdmlz ZSAgIHwgYWx3YXlzCj4+Pj4+Pj4gLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLXwtLS0tLS0t LS0tLXwtLS0tLS0tLS0tLQo+Pj4+Pj4+IG5vIGhpbnQgICAgICAgICB8IFMgICAgICAgICB8IFMg ICAgICAgICB8IFRIUD5TCj4+Pj4+Pj4gTUFEVl9IVUdFUEFHRSAgIHwgUyAgICAgICAgIHwgVEhQ PlMgICAgIHwgVEhQPlMKPj4+Pj4+PiBNQURWX05PSFVHRVBBR0UgfCBTICAgICAgICAgfCBTICAg ICAgICAgfCBTCj4+Pj4+Pj4KPj4+Pj4+PiBMZWdlbmQ6Cj4+Pj4+Pj4gUyAgICAgICBhbGxvY2F0 ZSBzaW5nbGUgcGFnZSAoUFRFLW1hcHBlZCkKPj4+Pj4+PiBMQUYgICAgIGFsbG9jYXRlIGxhZ2Ug YW5vbiBmb2xpbyAoUFRFLW1hcHBlZCkKPj4+Pj4+PiBUSFAgICAgIGFsbG9jYXRlIFRIUC1zaXpl ZCBmb2xpbyAoUE1ELW1hcHBlZCkKPj4+Pj4+Pj4gICAgICAgICAgZmFsbGJhY2sgKHVzdWFsbHkg YmVjYXVzZSB2bWEgc2l6ZS9hbGlnbm1lbnQgaW5zdWZmaWNpZW50IGZvciBmb2xpbykKPj4+Pj4+ Pgo+Pj4+Pj4+Cj4+Pj4+Pj4KPj4+Pj4+PiBQcmluY2lwbGVzIGZvciBMYXJnZSBBbm9uIEZvbGlv cyAoTEFGKQo+Pj4+Pj4+IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCj4+ Pj4+Pj4KPj4+Pj4+PiBEYXZpZCB0ZWxscyB1cyB0aGVyZSBhcmUgdXNlIGNhc2VzIHRvZGF5IChl LmcuIHFlbXUgbGl2ZSBtaWdyYXRpb24pIHdoaWNoIHVzZQo+Pj4+Pj4+IE1BRFZfTk9IVUdFUEFH RSB0byBtZWFuICJkb24ndCBmaWxsIGFueSBQVEVzIHRoYXQgYXJlIG5vdCBleHBsaWNpdGx5IGZh dWx0ZWQiCj4+Pj4+Pj4gYW5kIHRoZXNlIHVzZSBjYXNlcyB3aWxsIGJyZWFrIChpLmUuIGZ1bmN0 aW9uYWxseSBpbmNvcnJlY3QpIGlmIHRoaXMgcmVxdWVzdCBpcwo+Pj4+Pj4+IG5vdCBob25vdXJl ZC4KPj4+Pj4+Cj4+Pj4+PiBJIGRvbid0IHJlbWVtYmVyIERhdmlkIHNheWluZyB0aGlzLiBJIHRo aW5rIGhlIHdhcyByZWZlcnJpbmcgdG8gVUZGRCwKPj4+Pj4+IG5vdCBNQURWX05PSFVHRVBBR0Us IHdoZW4gZGlzY3Vzc2luZyB3aGF0IHdlIG5lZWQgdG8gYWJzb2x1dGVseQo+Pj4+Pj4gcmVzcGVj dC4KPj4+Pj4KPj4+Pj4gTXkgdW5kZXJzdGFuZGluZyB3YXMgdGhhdCBNQURWX05PSFVHRVBBR0Ug d2FzIGJlaW5nIGFwcGxpZWQgdG8gcmVnaW9ucyAqYmVmb3JlKgo+Pj4+PiBVRkZEIHdhcyBiZWlu ZyByZWdpc3RlcmVkLCBhbmQgdGhlIGFwcCByZWxpZWQgb24gTUFEVl9OT0hVR0VQQUdFIHRvIG5v dCBiYWNrIGFueQo+Pj4+PiB1bmZhdWx0ZWQgcGFnZXMuIEl0J3Mgbm90IGNvbXBsZXRlbHkgY2xl YXIgdG8gbWUgaG93IG5vdCBob25vdXJpbmcKPj4+Pj4gTUFEVl9OT0hVR0VQQUdFIHdvdWxkIGJy ZWFrIHRoaW5ncyB0aG91Z2guIERhdmlkPwo+Pj4+Cj4+Pj4gU29ycnksIEknbSBzdGlsbCBsYWdn aW5nIGJlaGluZCBvbiBzb21lIHRocmVhZHMuCj4+Pj4KPj4+PiBJbWFnaW5lIHRoZSBmb2xsb3dp bmcgZm9yIFZNIHBvc3Rjb3B5IGxpdmUgbWlncmF0aW9uOgo+Pj4+Cj4+Pj4gKDEpIFNldCBNQURW X05PSFVHRVBBR0Ugb24gZ3Vlc3QgbWVtb3J5IGFuZCBkaXNjYXJkIGFsbCBtZW1vcnkgKGUuZy4s Cj4+Pj4gICAgICAgIE1BRFZfRE9OVE5FRUQpLCB0byBzdGFydCB3aXRoIGEgY2xlYW4gc2xhdGUu Cj4+Pj4gKDIpIE1pZ3JhdGVzIHNvbWUgcGFnZXMgZHVyaW5nIHByZWNvcHkgZnJvbSB0aGUgc291 cmNlIGFuZCBzdG9yZXMgdGhlbQo+Pj4+ICAgICAgICBpbnRvIGd1ZXN0IG1lbW9yeSBvbiB0aGUg ZGVzdGluYXRpb24uIFNvbWUgb2YgdGhlIG1lbW9yeSBsb2NhdGlvbnMKPj4+PiAgICAgICAgd2ls bCBoYXZlIHBhZ2VzIHBvcHVsYXRlZC4KPj4+PiAoMykgQXQgc29tZSBwb2ludCwgZGVjaWRlIHRv IGVuYWJsZSBwb3N0Y29weTogZW5hYmxlIHVzZXJmYXVsdGZkIG9uCj4+Pj4gICAgICAgIGd1ZXN0 IG1lbW9yeS4KPj4+PiAoNCkgRGlzY2FyZCAqc2VsZWN0ZWQqIHBhZ2VzIGFnYWluIHRoYXQgaGF2 ZSBiZWVuIGRpcnRpZWQgaW4gdGhlCj4+Pj4gICAgICAgIG1lYW50aW1lIG9uIHRoZSBzb3VyY2Uu IFRoZXNlIGFyZSBwYWdlcyB0aGF0IGhhdmUgYmVlbiBtaWdyYXRlZAo+Pj4+ICAgICAgICBwcmV2 aW91c2x5Lgo+Pj4+ICg1KSBTdGFydCBydW5uaW5nIHRoZSBWTSBvbiB0aGUgZGVzdGluYXRpb24u Cj4+Pj4gKDYpIEFueXRoaW5nIHRoYXQncyBub3QgcG9wdWxhdGVkIHdpbGwgdHJpZ2dlciB1c2Vy ZmF1bHRmZCBtaXNzaW5nCj4+Pj4gICAgICAgIGZhdWx0cy4gVGhlbiwgeW91IGNhbiByZXF1ZXN0 IHRoZW0gZnJvbSB0aGUgc291cmNlIGFuZCBwbGFjZSB0aGVtLgo+Pj4+Cj4+Pj4gQXNzdW1lIHlv dSB3b3VsZCBwb3B1bGF0ZSBtb3JlIHRoYW4gcmVxdWlyZWQgZHVyaW5nIDIpLCB5b3UgY2FuIGVu ZCB1cAo+Pj4+IG5vdCBnZXR0aW5nIHVzZXJmYXVsdGZkIGZhdWx0cyBkdXJpbmcgNCkgYW5kIGNv cnJ1cHQgeW91ciBndWVzdCBzdGF0ZS4KPj4+PiBJdCB3b3JrcyBpZiBkdXJpbmcgKDIpIHlvdSBt aWdyYXRlZCBhbGwgZ3Vlc3QgbWVtb3J5LCBvciBpZiBkdXJpbmcgNCkKPj4+PiB5b3UgemFwIGV2 ZXJ5dGhpbmcgdGhhdCBzdGlsbCBuZWVkcyBtaWdyCj4+Pgo+Pj4gSSBzZWUgd2hhdCB5b3UgbWVh biBub3cuIFRoYW5rcy4KPj4+Cj4+PiBZZXMsIGluIHRoaXMgY2FzZSB3ZSBoYXZlIHRvIGludGVy cHJldCBNQURWX05PSFVHRVBBR0UgYXMgbm90aGluZyA+NEtCLgo+Pgo+PiBOb3RlIHRoYXQgaXQn cyBzdGlsbCBldmVuIHVuY2xlYXIgdG8gbWUgd2h5IHdlIHdhbnQgdG8gKm5vdCogY2FsbCB0aGVz ZQo+PiB0aGluZ3MgVEhQLiBJdCB3b3VsZCBjZXJ0YWlubHkgbWFrZSBldmVyeXRoaW5nIGxlc3Mg Y29uZnVzaW5nIGlmIHdlIGNhbGwKPj4gdGhlbSBUSFAsIGJ1dCB3aXRoIGFkZGl0aW9uYWwgYXR0 cmlidXRlcy4KPj4KPj4gSSB0aGluayB0aGF0IGlzIG9uZSBvZiB0aGUgZmlyc3QgdGhpbmdzIHdl IHNob3VsZCBmaWd1cmUgb3V0IGJlY2F1c2UgaXQKPj4gYWxzbyBpbmRpcmVjdGx5IHRlbGxzIHVz IHdoYXQgYWxsIHRoZXNlIHRvZ2dsZXMgbWVhbiBhbmQgaG93L2lmIHdlCj4+IHNob3VsZCByZWRl ZmluZSB0aGVtIChhbmQgaWYgdGhleSBldmVuIGFwcGx5KS4KPj4KPj4gQ3VycmVudGx5IFRIUCA9 PSBQTUQgc2l6ZQo+Pgo+PiBJbiAyMDE2LCBIdWdoIGFscmVhZHkgZW52aXNpb25lZCBQVUQvUEdE IFRIUCAoc2VlIDQ5OTIwZDI4NzgxZCAoIm1tOgo+PiBtYWtlIHRyYW5zcGFyZW50IGh1Z2VwYWdl IHNpemUgcHVibGljIikpIHdoZW4gaGUgZXhwbGljaXRseSBleHBvc2VkCj4+ICJocGFnZV9wbWRf c2l6ZSIuIE5vdCAiaHBhZ2Vfc2l6ZSIuCj4+Cj4+IEZvciBodWdldGxiIG9uIGFybTY0IHdlIGFs cmVhZHkgc3VwcG9ydCB2YXJpb3VzIHNpemVzIHRoYXQgYXJlIDwgUE1ECj4+IHNpemUgYW5kICpu b3QqIGNhbGwgdGhlbSBkaWZmZXJlbnRseS4gSXQncyBhIGh1Z2UodGxiKSBwYWdlLiBTb21ldGlt ZXMKPj4gd2UgcmVmZXIgdG8gdGhlbSBhcyBjb250LVBURSBodWdldGxiIHBhZ2VzLgo+Pgo+Pgo+ PiBTbywgbm93YWRheXMgd2UgZG8gaGF2ZSAiUE1ELXNpemVkIFRIUCIsIHNvbWVkYXkgd2UgbWln aHQgaGF2ZQo+PiAiUFVELXNpemVkIFRIUCIuIENhbid0IHdlIGNvbWUgdXAgd2l0aCBhIG5hbWUg dG8gZGVzY3JpYmUgc3ViLVBNRCBUSFA/Cj4+Cj4+IElzIGl0IHJlYWxseSBvZiB2YWx1ZSBpZiB3 ZSBpbnZlbnQgYSBuZXcgdGVybSBmb3IgdGhlbT8gWWVzLCBJIHdhcyBub3QKPj4gZW5qb3lpbmcg IkZsZXhpYmxlIFRIUCIuCj4+Cj4+Cj4+IE9uY2Ugd2UgZmlndXJlZCB0aGF0IG91dCwgd2Ugc2hv dWxkIGZpZ3VyZSBvdXQgaWYgTUFEVl9IVUdFUEFHRSBtZWFudAo+PiAib25seSBQTUQtc2l6ZWQg VEhQIiBvciBhbnl0aGluZyBlbHNlPwo+Pgo+PiBBbHNvLCB3ZSBjYW4gdGhlbiBmaWd1cmUgb3V0 IGlmIE1BRFZfTk9IVUdFUEFHRSBtZWFudCAib25seSBQTUQtc2l6ZWQKPj4gVEhQIiBvciBhbnl0 aGluZyBlbHNlPwo+Pgo+Pgo+PiBUaGUgc2ltcGxlc3QgYXBwcm9hY2ggdG8gbWUgd291bGQgYmUg InRoZXkgaW1wbHkgYW55IFRIUCwgYW5kIG9uY2Ugd2UKPj4gbmVlZCBtb3JlIHR1bmFibGVzIHdl IG1pZ2h0IGFkZCBzb21lIiwgc2ltaWxhciB0byB3aGF0IEtpcmlsbCBhbHNvIHJhaXNlZC4KPj4K Pj4KPj4gQWdhaW4sIGl0J3MgYWxsIHVuY2xlYXIgdG8gbWUgYXQgdGhpcyBwb2ludCBhbmQgSSdt IGhhcHB5IHRvIGhlYXIKPj4gb3BpbmlvbnMsIGJlY2F1c2UgSSByZWFsbHkgZG9uJ3Qga25vdy4K PiAKPiBJIGFncmVlIHRoZXNlIHBvaW50cyByZXF1aXJlIG1vcmUgZGlzY3Vzc2lvbi4gQnV0IEkg ZG9uJ3QgdGhpbmsgd2UKPiBuZWVkIHRvIGNvbmNsdWRlIHRoZW0gbm93LCB1bmxlc3MgdGhleSBj YXVzZSBjb3JyZWN0bmVzcyBpc3N1ZXMgbGlrZQo+IGlnbm9yaW5nIE1BRFZfTk9IVUdFUEFHRSB3 b3VsZC4gTXkgY29uY2VybiBpcyB0aGF0IGlmIHdlIGRlY2lkZSB0byBnbwo+IHdpdGggInRoZXkg aW1wbHkgYW55IFRIUCIgYW5kICpleHBvc2UgdGhpcyB0byB1c2Vyc3BhY2Ugbm93Kiwgd2UgbWln aHQKPiByZWdyZXQgbGF0ZXIuCgpJZiB3ZSBkb24ndCB0aGluayB0aGV5IGFyZSBUSFAsIHByb2Jh Ymx5IE1BRFZfTk9IVUdFUEFHRSBzaG91bGQgbm90IAphcHBseSBhbmQgd2Ugc2hvdWxkIGJlIHJl YWR5IHRvIGZpbmQgb3RoZXIgd2F5cyB0byBkZWFsIHdpdGggdGhlIG1lc3Mgd2UgCmV2ZW50dWFs bHkgY3JlYXRlLiBJZiB3ZSB3YW50IHRvIGdvIGRvd24gdGhhdCBwYXRoLCBzdXJlLgoKSWYgdGhl eSBhcmUgVEhQLCB0byBtZSB0aGVyZSBpcyBub3QgcmVhbGx5IGEgcXVlc3Rpb24gaWYgTUFEVl9O T0hVR0VQQUdFIAphcHBsaWVzIHRvIHRoZW0gb3Igbm90LiBVbmxlc3Mgd2Ugd2FudCB0byBidWls ZCBhIGNvbmZ1c2luZyBwaWVjZSBvZiAKc29mdHdhcmUgOykKCi0tIApDaGVlcnMsCgpEYXZpZCAv IGRoaWxkZW5iCgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5p bmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8v bGludXgtYXJtLWtlcm5lbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97F0BC00528 for ; Fri, 4 Aug 2023 21:35:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 082FF6B0071; Fri, 4 Aug 2023 17:35:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 032CF8D0001; Fri, 4 Aug 2023 17:35:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E15DF6B0074; Fri, 4 Aug 2023 17:35:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D30756B0071 for ; Fri, 4 Aug 2023 17:35:57 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 95D9B120792 for ; Fri, 4 Aug 2023 21:35:57 +0000 (UTC) X-FDA: 81087729954.27.B157154 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf22.hostedemail.com (Postfix) with ESMTP id 4DF8FC0005 for ; Fri, 4 Aug 2023 21:35:55 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=af7XUGYX; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf22.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691184955; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6xHzxUYjJIdDtURU1S3vX2xZDjMKXI19w61gLbe1Ayo=; b=PkZCYAg8zgGf9LI4rGC0xYG7ru9iNPv4HIxzhiUvx6oeB3udZALLLiuilL0AqNHaXNEAvf q9BU1LA7IPIIRHB5Pq6X4H7TW3D3oNysZ2t1kPV1CKw8cRsC9SGQeabfUqs47v3+qXnjnx SZseXQMcmoE6An+jVVLa+PJoy6zaJ4U= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=af7XUGYX; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf22.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691184955; a=rsa-sha256; cv=none; b=qps9/WcwB7fxl5eBSuM3MVr6Fty3Kzi0cZz9LCAsWsC15Ra5qF3SZ6ob5sKSFkmEL6ugGF lO+x2qfNqO4QBU5qG3wGt9/1GV5zvxzn2dEvSuRgNsCWdgfxNB05uVnnR1nkG+7J7KjRX9 M6zZ1kLIT72nFuFpWtjVX8SMabJg8hQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691184954; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6xHzxUYjJIdDtURU1S3vX2xZDjMKXI19w61gLbe1Ayo=; b=af7XUGYXWenSrA63T/3RAsXfFSlb0CCGFeirHby4SZqZGlSJH56yRy+DRZQEUhQkbvZF4b b9s86fOmpcC9NIj6wTXpKPQHyBUindFhbb5qzQ0QKn1I5p1jOhbEEJLTQSSg0mWFnDK40a 6aFhWi7RPVNp2il1pZPj4JHbJ57w3C8= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-138-z0OGlJhXMsu6mS5w92ktzA-1; Fri, 04 Aug 2023 17:35:53 -0400 X-MC-Unique: z0OGlJhXMsu6mS5w92ktzA-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3fe210c47acso14852875e9.3 for ; Fri, 04 Aug 2023 14:35:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691184652; x=1691789452; h=content-transfer-encoding:in-reply-to:organization:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6xHzxUYjJIdDtURU1S3vX2xZDjMKXI19w61gLbe1Ayo=; b=WS2biWNo85fyT1O9z8pNdS5WFxnrVowPu041p+s7dw3MQxC28g7DvUV0jZo6w0Hswq dGsUzYj9ixbua1UV2cy6t2EkM3lqmDTB2QeUmNANmiPdnpRlRh/bpI0JfwNCY5pDliFK oJ9TyVripcd13daXnXVALWKbvWBeS6ucnz2D91aSTEJC22cDAeMvbEnK780wXjk71CjB SB8J8jGAGoSEuFnl3yeyXQXg/+JkWVF4lCNCTfQiAErF4lyM3g0q+6vyFTXbXXVKeohm bafuUqo+fcBuYrSRFB2XqkkhJrRJS5A/EH0gn7HyfMyrlGKP9kH1sgThR/8RHknN0Zg7 6QUg== X-Gm-Message-State: AOJu0Yzthhcck489dk/K37d+9Be+kczUWcr2M9izG/nWbpYfwF7QESZo CLoA33Qs1EZ3ZtZA7mIlV77zgbviWCCgg2o684sxx0lhC1yL3M6uq6TgOKjhon85lcfSbPKvKj7 8Jli8qPDIw5k= X-Received: by 2002:a7b:ce94:0:b0:3f8:2777:15e with SMTP id q20-20020a7bce94000000b003f82777015emr2164958wmj.31.1691184652193; Fri, 04 Aug 2023 14:30:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHg76fpCPdISu/CKKhttE0AnUkU9rMMWiHgz3qJGhEnE09KvQCL77IQ1NcO4miQKPYK6bEQ/A== X-Received: by 2002:a7b:ce94:0:b0:3f8:2777:15e with SMTP id q20-20020a7bce94000000b003f82777015emr2164947wmj.31.1691184651768; Fri, 04 Aug 2023 14:30:51 -0700 (PDT) Received: from ?IPV6:2003:d8:2f2d:8e00:a20e:59bc:3c13:4806? (p200300d82f2d8e00a20e59bc3c134806.dip0.t-ipconnect.de. [2003:d8:2f2d:8e00:a20e:59bc:3c13:4806]) by smtp.gmail.com with ESMTPSA id 25-20020a05600c021900b003fe4ca8decdsm1425819wmi.31.2023.08.04.14.30.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 04 Aug 2023 14:30:51 -0700 (PDT) Message-ID: <0d502268-ebdc-8462-d88c-e6a41578d9ae@redhat.com> Date: Fri, 4 Aug 2023 23:30:49 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: [PATCH v4 2/5] mm: LARGE_ANON_FOLIO for improved performance To: Yu Zhao Cc: Ryan Roberts , Andrew Morton , Matthew Wilcox , Yin Fengwei , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , "Huang, Ying" , Zi Yan , Luis Chamberlain , Itaru Kitayama , "Kirill A. Shutemov" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org References: <20230726095146.2826796-1-ryan.roberts@arm.com> <20230726095146.2826796-3-ryan.roberts@arm.com> <5e595904-3dca-0e15-0769-7ed10975fd0d@arm.com> <259ad8fc-c12b-69b9-ba16-adb9e3e6d672@redhat.com> From: David Hildenbrand Organization: Red Hat In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4DF8FC0005 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: qf9q15cxfkcrd8nsq19kj4yyh8skntf3 X-HE-Tag: 1691184955-601105 X-HE-Meta: U2FsdGVkX1+AMATXJJ5ZOzCXWZQ1m7sYMF2KNrOxjtNF5lU9bNtLg7t6Db1GrfQNqkti0fuT5jzx69O+RrTEofGcAcFzGfBUeh5NismmX7yIsI3VaZSZ//kMpS3/qll1hW6J2MOabXfstQn/x/5Bfo7dwjRDcAnFgPqgPmBYgWQqFHfsAEoEBBYw5Wye+W4fKULwFOT1rtxzqxQLZW2jgDBWQrYcreo+2KVGYv2F1BXnUmYHidRLlgR5MDDlYvaj6RLaLIR8jQuYFPI3U6RyS1qujGg4emLJLtN+lTn4rZKZN6kdQwwXb0sOh8XJVKpw4grwQLz899WPSuZ17VFQDgCzws8AmJB4DFN/F5yqgEQUU1sBxfyg3HAz/lJ3BnxIjV7V3I4BHBikZTM8/+WtQ7n74Hj8ab4HC/kIx0N0TRCfVB3BHKQLCA49Pt3BSNa2mPEJVhcXXNL1SsGzH2RgwEGH+5mNEUHark2Lxk+504/8jSruu1/T36Uhu9j3Zk3eCF2YJW77wKUjsKDMqTw36zdzGZIiuzxuk6FQD/IbrBVFX1YwSiSxGZzysLzQ9ZxNFZh+uomIbznHJ85PFgJb+2tsZed81hg1Z+Q0N0yqb/gaM1yu8zOjOCsWnB4c1oCI5en4/hChfCGWdXlJrc8CpSbG6wv20vqCNJhMTCEGAkhIkof2+BToVYOMGj5qgVhik+3kUkp/dTV7DXIrmgiL81hRoPDBxAxrOWAWkb7bdtdN0cbmp/KCsv3QTA6EmNaqj5OndSWh18R+x0Nuk31i1axxaOKa8FFDWaKdOOoTq6mVvYx503r9kDkQ/4MljotEz2gIPqv/W0tJJDt8BvKVn953vgR1xMn+ZEu+NdQpG39xeeuLv22aXFplGaDMRSerIN3JyrlT2KbBleXLXLbTHOgn0Yl6R/aEztZrW+KpHA331y3V/oRHorNhZdUne1Fvf6TUk/5uNYdBUbqeFqp oqFW1uSc rhvlvdoQw0v2SzwRAsJby2OFZaYgVsovWxnh1IOX4BnzFloSQlLjxNmf+b3GGx/gAA9UdTLb0OkOcVc3vf6L3yG9u+t5II6WFljnuy93TLDKS+5eYjkk2rhsISlzP8nUq19F5jkCLv0GztrNqf1nTKCK7lSotrMNhLazTb1aclIjaLwyqBxsDBj6fwd1bmX4Pd2dYHYDF/MFCdJTB2JAYhphgawS8eVWqQOUc/kB0gLl4jlmKC/23U3DhdQ+RGXj65p2zE0+VUqQE8DGemvV/Tptk9jdbaJEPJdjyIrVs4udlOEVioQGmLJEvWV0lpTRzDCkrAHxC1osvioSY14fC+80qiol0txC+0MRlV6k3HbP5Yvy60SrcrVNvy+xSF+TITeS/lv9fUOat0zZtopoLJIKO+sdlTVCV6XLQ9v9VSngNfept+4okCt+hcEedwDVwNXEgX+ZSizjAE//ikVT/4yzdHNbaqLTkdPb27P0O+uVNAuc2fztCwQz2lQ8gto2eYeRpD05K3ES+xQ2rzjSSwUVVzL8m6vRNp66syniY8na1kEuRI6ERBL014OZN4CtXSZh7uVuKWFZSpBk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 04.08.23 23:26, Yu Zhao wrote: > On Fri, Aug 4, 2023 at 3:13 PM David Hildenbrand wrote: >> >> On 04.08.23 23:00, Yu Zhao wrote: >>> On Fri, Aug 4, 2023 at 2:23 PM David Hildenbrand wrote: >>>> >>>> On 04.08.23 10:27, Ryan Roberts wrote: >>>>> On 04/08/2023 00:50, Yu Zhao wrote: >>>>>> On Thu, Aug 3, 2023 at 6:43 AM Ryan Roberts wrote: >>>>>>> >>>>>>> + Kirill >>>>>>> >>>>>>> On 26/07/2023 10:51, Ryan Roberts wrote: >>>>>>>> Introduce LARGE_ANON_FOLIO feature, which allows anonymous memory to be >>>>>>>> allocated in large folios of a determined order. All pages of the large >>>>>>>> folio are pte-mapped during the same page fault, significantly reducing >>>>>>>> the number of page faults. The number of per-page operations (e.g. ref >>>>>>>> counting, rmap management lru list management) are also significantly >>>>>>>> reduced since those ops now become per-folio. >>>>>>>> >>>>>>>> The new behaviour is hidden behind the new LARGE_ANON_FOLIO Kconfig, >>>>>>>> which defaults to disabled for now; The long term aim is for this to >>>>>>>> defaut to enabled, but there are some risks around internal >>>>>>>> fragmentation that need to be better understood first. >>>>>>>> >>>>>>>> When enabled, the folio order is determined as such: For a vma, process >>>>>>>> or system that has explicitly disabled THP, we continue to allocate >>>>>>>> order-0. THP is most likely disabled to avoid any possible internal >>>>>>>> fragmentation so we honour that request. >>>>>>>> >>>>>>>> Otherwise, the return value of arch_wants_pte_order() is used. For vmas >>>>>>>> that have not explicitly opted-in to use transparent hugepages (e.g. >>>>>>>> where thp=madvise and the vma does not have MADV_HUGEPAGE), then >>>>>>>> arch_wants_pte_order() is limited to 64K (or PAGE_SIZE, whichever is >>>>>>>> bigger). This allows for a performance boost without requiring any >>>>>>>> explicit opt-in from the workload while limitting internal >>>>>>>> fragmentation. >>>>>>>> >>>>>>>> If the preferred order can't be used (e.g. because the folio would >>>>>>>> breach the bounds of the vma, or because ptes in the region are already >>>>>>>> mapped) then we fall back to a suitable lower order; first >>>>>>>> PAGE_ALLOC_COSTLY_ORDER, then order-0. >>>>>>>> >>>>>>> >>>>>>> ... >>>>>>> >>>>>>>> +#define ANON_FOLIO_MAX_ORDER_UNHINTED \ >>>>>>>> + (ilog2(max_t(unsigned long, SZ_64K, PAGE_SIZE)) - PAGE_SHIFT) >>>>>>>> + >>>>>>>> +static int anon_folio_order(struct vm_area_struct *vma) >>>>>>>> +{ >>>>>>>> + int order; >>>>>>>> + >>>>>>>> + /* >>>>>>>> + * If THP is explicitly disabled for either the vma, the process or the >>>>>>>> + * system, then this is very likely intended to limit internal >>>>>>>> + * fragmentation; in this case, don't attempt to allocate a large >>>>>>>> + * anonymous folio. >>>>>>>> + * >>>>>>>> + * Else, if the vma is eligible for thp, allocate a large folio of the >>>>>>>> + * size preferred by the arch. Or if the arch requested a very small >>>>>>>> + * size or didn't request a size, then use PAGE_ALLOC_COSTLY_ORDER, >>>>>>>> + * which still meets the arch's requirements but means we still take >>>>>>>> + * advantage of SW optimizations (e.g. fewer page faults). >>>>>>>> + * >>>>>>>> + * Finally if thp is enabled but the vma isn't eligible, take the >>>>>>>> + * arch-preferred size and limit it to ANON_FOLIO_MAX_ORDER_UNHINTED. >>>>>>>> + * This ensures workloads that have not explicitly opted-in take benefit >>>>>>>> + * while capping the potential for internal fragmentation. >>>>>>>> + */ >>>>>>>> + >>>>>>>> + if ((vma->vm_flags & VM_NOHUGEPAGE) || >>>>>>>> + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags) || >>>>>>>> + !hugepage_flags_enabled()) >>>>>>>> + order = 0; >>>>>>>> + else { >>>>>>>> + order = max(arch_wants_pte_order(), PAGE_ALLOC_COSTLY_ORDER); >>>>>>>> + >>>>>>>> + if (!hugepage_vma_check(vma, vma->vm_flags, false, true, true)) >>>>>>>> + order = min(order, ANON_FOLIO_MAX_ORDER_UNHINTED); >>>>>>>> + } >>>>>>>> + >>>>>>>> + return order; >>>>>>>> +} >>>>>>> >>>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> I'm writing up the conclusions that we arrived at during discussion in the THP >>>>>>> meeting yesterday, regarding linkage with exiting THP ABIs. It would be great if >>>>>>> I can get explicit "agree" or disagree + rationale from at least David, Yu and >>>>>>> Kirill. >>>>>>> >>>>>>> In summary; I think we are converging on the approach that is already coded, but >>>>>>> I'd like confirmation. >>>>>>> >>>>>>> >>>>>>> >>>>>>> The THP situation today >>>>>>> ----------------------- >>>>>>> >>>>>>> - At system level: THP can be set to "never", "madvise" or "always" >>>>>>> - At process level: THP can be "never" or "defer to system setting" >>>>>>> - At VMA level: no-hint, MADV_HUGEPAGE, MADV_NOHUGEPAGE >>>>>>> >>>>>>> That gives us this table to describe how a page fault is handled, according to >>>>>>> process state (columns) and vma flags (rows): >>>>>>> >>>>>>> | never | madvise | always >>>>>>> ----------------|-----------|-----------|----------- >>>>>>> no hint | S | S | THP>S >>>>>>> MADV_HUGEPAGE | S | THP>S | THP>S >>>>>>> MADV_NOHUGEPAGE | S | S | S >>>>>>> >>>>>>> Legend: >>>>>>> S allocate single page (PTE-mapped) >>>>>>> LAF allocate lage anon folio (PTE-mapped) >>>>>>> THP allocate THP-sized folio (PMD-mapped) >>>>>>>> fallback (usually because vma size/alignment insufficient for folio) >>>>>>> >>>>>>> >>>>>>> >>>>>>> Principles for Large Anon Folios (LAF) >>>>>>> -------------------------------------- >>>>>>> >>>>>>> David tells us there are use cases today (e.g. qemu live migration) which use >>>>>>> MADV_NOHUGEPAGE to mean "don't fill any PTEs that are not explicitly faulted" >>>>>>> and these use cases will break (i.e. functionally incorrect) if this request is >>>>>>> not honoured. >>>>>> >>>>>> I don't remember David saying this. I think he was referring to UFFD, >>>>>> not MADV_NOHUGEPAGE, when discussing what we need to absolutely >>>>>> respect. >>>>> >>>>> My understanding was that MADV_NOHUGEPAGE was being applied to regions *before* >>>>> UFFD was being registered, and the app relied on MADV_NOHUGEPAGE to not back any >>>>> unfaulted pages. It's not completely clear to me how not honouring >>>>> MADV_NOHUGEPAGE would break things though. David? >>>> >>>> Sorry, I'm still lagging behind on some threads. >>>> >>>> Imagine the following for VM postcopy live migration: >>>> >>>> (1) Set MADV_NOHUGEPAGE on guest memory and discard all memory (e.g., >>>> MADV_DONTNEED), to start with a clean slate. >>>> (2) Migrates some pages during precopy from the source and stores them >>>> into guest memory on the destination. Some of the memory locations >>>> will have pages populated. >>>> (3) At some point, decide to enable postcopy: enable userfaultfd on >>>> guest memory. >>>> (4) Discard *selected* pages again that have been dirtied in the >>>> meantime on the source. These are pages that have been migrated >>>> previously. >>>> (5) Start running the VM on the destination. >>>> (6) Anything that's not populated will trigger userfaultfd missing >>>> faults. Then, you can request them from the source and place them. >>>> >>>> Assume you would populate more than required during 2), you can end up >>>> not getting userfaultfd faults during 4) and corrupt your guest state. >>>> It works if during (2) you migrated all guest memory, or if during 4) >>>> you zap everything that still needs migr >>> >>> I see what you mean now. Thanks. >>> >>> Yes, in this case we have to interpret MADV_NOHUGEPAGE as nothing >4KB. >> >> Note that it's still even unclear to me why we want to *not* call these >> things THP. It would certainly make everything less confusing if we call >> them THP, but with additional attributes. >> >> I think that is one of the first things we should figure out because it >> also indirectly tells us what all these toggles mean and how/if we >> should redefine them (and if they even apply). >> >> Currently THP == PMD size >> >> In 2016, Hugh already envisioned PUD/PGD THP (see 49920d28781d ("mm: >> make transparent hugepage size public")) when he explicitly exposed >> "hpage_pmd_size". Not "hpage_size". >> >> For hugetlb on arm64 we already support various sizes that are < PMD >> size and *not* call them differently. It's a huge(tlb) page. Sometimes >> we refer to them as cont-PTE hugetlb pages. >> >> >> So, nowadays we do have "PMD-sized THP", someday we might have >> "PUD-sized THP". Can't we come up with a name to describe sub-PMD THP? >> >> Is it really of value if we invent a new term for them? Yes, I was not >> enjoying "Flexible THP". >> >> >> Once we figured that out, we should figure out if MADV_HUGEPAGE meant >> "only PMD-sized THP" or anything else? >> >> Also, we can then figure out if MADV_NOHUGEPAGE meant "only PMD-sized >> THP" or anything else? >> >> >> The simplest approach to me would be "they imply any THP, and once we >> need more tunables we might add some", similar to what Kirill also raised. >> >> >> Again, it's all unclear to me at this point and I'm happy to hear >> opinions, because I really don't know. > > I agree these points require more discussion. But I don't think we > need to conclude them now, unless they cause correctness issues like > ignoring MADV_NOHUGEPAGE would. My concern is that if we decide to go > with "they imply any THP" and *expose this to userspace now*, we might > regret later. If we don't think they are THP, probably MADV_NOHUGEPAGE should not apply and we should be ready to find other ways to deal with the mess we eventually create. If we want to go down that path, sure. If they are THP, to me there is not really a question if MADV_NOHUGEPAGE applies to them or not. Unless we want to build a confusing piece of software ;) -- Cheers, David / dhildenb