From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A70CC3ABC9 for ; Fri, 16 May 2025 17:45:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 16FED6B0249; Fri, 16 May 2025 13:45:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BF1266B024A; Fri, 16 May 2025 13:45:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 710946B024B; Fri, 16 May 2025 13:45:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 216216B0249 for ; Fri, 16 May 2025 13:45:10 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2422A5E252 for ; Fri, 16 May 2025 17:45:12 +0000 (UTC) X-FDA: 83449497264.08.9B71D7F Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf30.hostedemail.com (Postfix) with ESMTP id 387D380006 for ; Fri, 16 May 2025 17:45:10 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ILBV3Z6m; spf=pass (imf30.hostedemail.com: domain of 3pHknaAYKCH0tfbokdhpphmf.dpnmjovy-nnlwbdl.psh@flex--seanjc.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3pHknaAYKCH0tfbokdhpphmf.dpnmjovy-nnlwbdl.psh@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747417510; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RmpgPqdPibQ4HZg5sdzMeXt+/PpG65nXkSoggikM9ks=; b=50RvqH5tVxAV/uNQACEiMBhav5H16OJZbCcVkuZ8qenjxZhWuZn0xz2yjDjEkn9INzSlCX DMR377eMZ9VgmrYp5leL2ZLol6qTW3duYEmCoLL05xSjh7T07ImzZQ5Jw8uHT4Iox+NaxH YZKB6qloOc1UbJOIbmbsFR/7sXLB+Jc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747417510; a=rsa-sha256; cv=none; b=C3d3JH6Fcs7n+SaoyExDpsKFyxzVm74XLWBfOdzrBL/pXQsEK8oLWsCotHFkI/UdNr+8SV KkhtWle5ib8ZBFujcP5AF1uKHrAFNjcGuM6G9ARq3nYdFJSoFz0eN4GmZ2h9QOaiVJI8d8 VM0PHapylITwYSXP2ajSnbHg+QlFdLA= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ILBV3Z6m; spf=pass (imf30.hostedemail.com: domain of 3pHknaAYKCH0tfbokdhpphmf.dpnmjovy-nnlwbdl.psh@flex--seanjc.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3pHknaAYKCH0tfbokdhpphmf.dpnmjovy-nnlwbdl.psh@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-30c9b0aa4ccso2196465a91.3 for ; Fri, 16 May 2025 10:45:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747417509; x=1748022309; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=RmpgPqdPibQ4HZg5sdzMeXt+/PpG65nXkSoggikM9ks=; b=ILBV3Z6mcHOV8zp8Z4a/xnsQr0i5o93W95YrfDSADEAL26eGQI6gcjO9tycrOa9ODO j2IacJSz33E28Qd3fxkKSQeUe3FzeEjjSgy/mr7wCoASMrD9MD0wg2l4TkSzuSeYlLea LCQ23+04jendQ6OBaBCsKoIaY3XU0hjJ7huCPxm8M+1igOz3UVikMCnyV6RJB3qkfJZl JngQ9M2jsu6BZn5QUY6yXge9oA2gk+sjeKSlT9OBMWam1jABfiBQ+uhrfRCcYTz/NRkv s2p8jyMyYByC43GAhXqargQRmygANO4lnHu/djAKmdW3HeVUL81KoG9ccU7kzj/6e2KJ wCFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747417509; x=1748022309; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=RmpgPqdPibQ4HZg5sdzMeXt+/PpG65nXkSoggikM9ks=; b=ZcJUNadxf97ci/8R33oxKv6xWyhXaVrjPCobFCwoDZ1HEHmjNbEVIG/u2l8jddnqKO +0dhRdH5n4uauoOj8xlyH4aRbm03MXyoL6LeDpSBgc/eHhk3uNaQI7uK11L7Bi4NtrSV IEjWU6ES6JqjWBLwz+csJKtz43ImuT6ZAhVZ5AfdK/lVfM1AhotvxSikeJPMzyH34FQT PGOydp5PA0xO6LOoS4w2Gzwfaw4U62FRD7oNJ3eIoNLqIjAKCoB7Kx4p678LXf6k6p79 g0dBkGJH/mhbGf4HxPjVyEpiFfgh/hwVEkdN7XSyKt5Gv+s9m/YS3qpmaTFYXWBoOvpw uUtw== X-Forwarded-Encrypted: i=1; AJvYcCXJAUrePOXaf3bChMJ4K6bEQg187c+TRAYHjXQZiKDecCzEA6x6HL2VOPT+AY9itb6fSvxMeasm/w==@kvack.org X-Gm-Message-State: AOJu0YxdXrRsuxsjrNB50TdSS7+jlmLBjL6wRa0vw90WHK96YryR3XSN /OuEQMPOQ4Gxd+GC7ryTHSdfIIvWXctOHGzNIaqvKOeTRx3eGSzkUmCZrAhVfbQ/bb5BxGGmREG scW2xjQ== X-Google-Smtp-Source: AGHT+IE5q1d8Vhf45B58FtB+DPajsRh9WD6SiM5jWunemSpWwoZ702eFXBFahwA7/IbgQCUG1fK7U0brYiw= X-Received: from pjbrs11.prod.google.com ([2002:a17:90b:2b8b:b0:308:867e:1ced]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:e706:b0:2f9:c144:9d13 with SMTP id 98e67ed59e1d1-30e7d5a8af4mr6412464a91.24.1747417508805; Fri, 16 May 2025 10:45:08 -0700 (PDT) Date: Fri, 16 May 2025 10:45:07 -0700 In-Reply-To: Mime-Version: 1.0 References: <24e8ae7483d0fada8d5042f9cd5598573ca8f1c5.camel@intel.com> <7d3b391f3a31396bd9abe641259392fd94b5e72f.camel@intel.com> Message-ID: Subject: Re: [RFC PATCH v2 00/51] 1G page support for guest_memfd From: Sean Christopherson To: Vishal Annapurve Cc: Rick P Edgecombe , "pvorel@suse.cz" , "kvm@vger.kernel.org" , "catalin.marinas@arm.com" , Jun Miao , Kirill Shutemov , "pdurrant@amazon.co.uk" , "steven.price@arm.com" , "peterx@redhat.com" , "x86@kernel.org" , "amoorthy@google.com" , "tabba@google.com" , "quic_svaddagi@quicinc.com" , "maz@kernel.org" , "vkuznets@redhat.com" , "quic_eberman@quicinc.com" , "keirf@google.com" , "hughd@google.com" , "mail@maciej.szmigiero.name" , "palmer@dabbelt.com" , Maciej Wieczor-Retman , Yan Y Zhao , "ajones@ventanamicro.com" , "willy@infradead.org" , "jack@suse.cz" , "paul.walmsley@sifive.com" , "aik@amd.com" , "usama.arif@bytedance.com" , "quic_mnalajal@quicinc.com" , "fvdl@google.com" , "rppt@kernel.org" , "quic_cvanscha@quicinc.com" , "nsaenz@amazon.es" , "vbabka@suse.cz" , Fan Du , "anthony.yznaga@oracle.com" , "linux-kernel@vger.kernel.org" , "thomas.lendacky@amd.com" , "mic@digikod.net" , "oliver.upton@linux.dev" , "akpm@linux-foundation.org" , "bfoster@redhat.com" , "binbin.wu@linux.intel.com" , "muchun.song@linux.dev" , Zhiquan1 Li , "rientjes@google.com" , "mpe@ellerman.id.au" , Erdem Aktas , "david@redhat.com" , "jgg@ziepe.ca" , "jhubbard@nvidia.com" , Haibo1 Xu , "anup@brainfault.org" , Dave Hansen , Isaku Yamahata , "jthoughton@google.com" , Wei W Wang , "steven.sistare@oracle.com" , "jarkko@kernel.org" , "quic_pheragu@quicinc.com" , "chenhuacai@kernel.org" , Kai Huang , "shuah@kernel.org" , "dwmw@amazon.co.uk" , "pankaj.gupta@amd.com" , Chao P Peng , "nikunj@amd.com" , Alexander Graf , "viro@zeniv.linux.org.uk" , "pbonzini@redhat.com" , "yuzenghui@huawei.com" , "jroedel@suse.de" , "suzuki.poulose@arm.com" , "jgowans@amazon.com" , Yilun Xu , "liam.merwick@oracle.com" , "michael.roth@amd.com" , "quic_tsoni@quicinc.com" , "richard.weiyang@gmail.com" , Ira Weiny , "aou@eecs.berkeley.edu" , Xiaoyao Li , "qperret@google.com" , "kent.overstreet@linux.dev" , "dmatlack@google.com" , "james.morse@arm.com" , "brauner@kernel.org" , "ackerleytng@google.com" , "linux-fsdevel@vger.kernel.org" , "pgonda@google.com" , "quic_pderrin@quicinc.com" , "roypat@amazon.co.uk" , "linux-mm@kvack.org" , "will@kernel.org" , "hch@infradead.org" Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: o79f39oprtxdtx4hhnmnwptwe1qw9neo X-Rspamd-Queue-Id: 387D380006 X-Rspam-User: X-Rspamd-Server: rspam02 X-HE-Tag: 1747417510-409311 X-HE-Meta: U2FsdGVkX18gab/Z0FlKtIaptg51/KIQWfxNPHT8HuMjb1/1h2qgFJ7M0xfc1bY4jBP8Pl/i5jBLT6TQqzNXlnJ4oEszmKo7bheCU7dPpZ9XUnuh81wbiyb0+j5WiWU/C9RulTdtxnvxFlt6s74pxH5972jIVD0x2u716LIkh967Z7naG1hSGRCS9+BknJq3ml6g803ygAG6nZtVHQNOOthPQnDMqN4cVQKfJyVMkjFWBd4ExVl3n++lzbLGm1v2qRRzDFM7250924DfMzvlK40nojwJZGQT0MjrpE0gRsxw3Jt+lOufInj8jOED4nTXgKh+sRFiL2Z+cOeoDEzhXsb7CAL++q2hATNaWJkBIuzhgpNoDhvbpjFi1dtzT/5ixg20NW1BFN/ELGIJna2+kEHB6rzquxqg2vAF8rhqFJ/6Gb3or9Avayfv0/oFVHLsIk21qiLD44rKJ1iNjKBgvVddoW//3Z9pwX7YsqYbyBl3F1co/KIPsYGu/0Vj9xZPw1VUGO4tjw8y7y3wCYSa2ohisAOxsM6181DxAx30i0ewOseqUjTk4xNsHylQwAT6AWggw+Cj4aUn39lNRtTKAsRrURg3fXpphIu2IC5/YpSou6JHLeN4hmnIN4Q7mjlG6CQqTWvpbrB8tMGna5pAIUsk8foIFwR06kMa3nk9uyN9ShfSS+KDGYtQ43qtwQO7QsT6d0o6BVHzdzMOXhy4q6NPScL4c8060b3EmZ1AwlXXfRalqhd9aVFysne1g3hZ+BkAwpZZzpNTm5UadHmFrz/fgN+vrPVQDqPxAoHrju7f1/KEhdt1lDv7K1nH0TML6EqOxU5jBzWAczNntdxkoTq04Vd9BcFAPNi7MECCic1zH7lVchVB5qqfwyq2dCh2FNltDOpufC8lvTGFOnErUBpmfCeUhIW/Ef++IlXgxxYyhtqeYRnK0qnH1EiTu0Jq/OU66xMFZnEZQKPnORL oRM2APRp HQnwtC8xSY6I1sUEF0MtkLGuIMjSzlXRCuk3KOdrP2mFA511x11j7zXME0G3cBfc6lzYGXoC0ZiMSRF8/VfrwQPObG02/mIoMR2VB/QQGfu5zdoseqXhUpG8YcKmW/ojXWrvPtZjKN18MljYCcZJY5AjWf7kUsnb+OTu629X1uo5hffBFmkxCRZ678e4G0k5mhsg7wkJiLbSRu3dFgvnAoMP+OMrx3xO6HbcwR2jA2iAJrRKsle8W0jxGkJkjdPFl3nzfFbB5oX0UVCUFMlrsKu9LZIxY/aXbtYIJL9F67fzoaTw9a2n7xE3m35Qr4qEzaim1+FXH3XPovo/FictVQjElZv64lwE0XErPdSE/i5r2Fx5MhFGvfkQqqFB4ZFLmpHxY4r9kqzpNcVETyjOsVQwEcu+PEZjZv45k0WIH7GLL8fuvtoUgfo7sElB/S1zV723PmGbL30MTEd0j2QZ2WC3OHBUgS/AhOM+2k2EKo44nwQeO4akJiWjneOTZ5CmfSWCqmeva+lGz215SHo3Va05TyIc/j2iCOwW/9zHACZS7jlErx8Umjw4mXRZOlipUjFVi9pDeIv9Rn+RQtUb7KkannDh9GJh4I6kOiq/NP2EQxkiyUYJVNAQWddW0oqsI5FbR2gv8A+Ipg9zbao01clILkNheXADpvCi7ZNjhJw7hrShetb9ngaIREknuciWPUXZ8027yGZAlbxP+30CZPNKiujQqmAZsBwJxOJ8CiAQWCRxFNDrUvXv5LN9UB/OWTsLL4GRLvOWhx4xl/1eG7WvkbsiKahGaXPFufIb12TUIOySf10g2yk4VoPQg3jyibNdFPt27FN+kex2D/SqcvTZnWC7jnjjYpBmuCYefQaLC8Oz0OUCJHocmnVapmLwhIX2LSBOIfhOyUo5BhF0L3V9Rw+Jn0KraGf33 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, May 16, 2025, Vishal Annapurve wrote: > On Thu, May 15, 2025 at 7:12=E2=80=AFPM Edgecombe, Rick P wrote: > > On Thu, 2025-05-15 at 17:57 -0700, Sean Christopherson wrote: > > > You're conflating two different things. guest_memfd allocating and m= anaging > > > 1GiB physical pages, and KVM mapping memory into the guest at 1GiB/2M= iB > > > granularity. Allocating memory in 1GiB chunks is useful even if KVM = can only > > > map memory into the guest using 4KiB pages. > > > > I'm aware of the 1.6% vmemmap benefits from the LPC talk. Is there more= ? The > > list quoted there was more about guest performance. Or maybe the clever= page > > table walkers that find contiguous small mappings could benefit guest > > performance too? It's the kind of thing I'd like to see at least broadl= y called > > out. >=20 > The crux of this series really is hugetlb backing support for guest_memfd= and > handling CoCo VMs irrespective of the page size as I suggested earlier, s= o 2M > page sizes will need to handle similar complexity of in-place conversion. >=20 > Google internally uses 1G hugetlb pages to achieve high bandwidth IO, E.g. hitting target networking line rates is only possible with 1GiB mappin= gs, otherwise TLB pressure gets in the way. > lower memory footprint using HVO and lower MMU/IOMMU page table memory > footprint among other improvements. These percentages carry a substantial > impact when working at the scale of large fleets of hosts each carrying > significant memory capacity. Yeah, 1.6% might sound small, but over however many bytes of RAM there are = in the fleet, it's a huge (lol) amount of memory saved. > > > Yes, some of this is useful for TDX, but we (and others) want to us= e > > > guest_memfd for far more than just CoCo VMs. > > > > > > > And for non-CoCo VMs, 1GiB hugepages are mandatory for various workl= oads. > > I've heard this a lot. It must be true, but I've never seen the actual = numbers. > > For a long time people believed 1GB huge pages on the direct map were c= ritical, > > but then benchmarking on a contemporary CPU couldn't find much differen= ce > > between 2MB and 1GB. I'd expect TDP huge pages to be different than tha= t because > > the combined walks are huge, iTLB, etc, but I'd love to see a real numb= er. The direct map is very, very different than userspace and thus guest mappin= gs. Software (hopefully) isn't using the direct map to index multi-TiB database= s, or to transfer GiBs of data over the network. The amount of memory the ker= nel is regularly accessing is an order or two magnitude smaller than single pro= cess use cases. A few examples from a quick search: http://pvk.ca/Blog/2014/02/18/how-bad-can-1gb-pages-be https://www.percona.com/blog/benchmark-postgresql-with-linux-hugepages/