From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 22633C54E58 for ; Mon, 25 Mar 2024 18:59:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kzQ8Er4YQT/vh1UFHQ4AzKl8ETTd2g58iDMJ5+n0v8o=; b=BWzhBuKTB3lDq8 XpgciUay/k3a0EgFAs++6e0uTboul7Cx41Xs/9sTdAKqkLwCrmAspVSK7ACaPwN5FuI2v7VzfnfY8 dApLOE4EU9vSDedKmnLDyh2ISgeBJS4JTrk/1re6FCFilB/Yw5U57X+LJgp1KwQpu0l7S0RtGhV+K 7p6EV9kB1IZAmsQx7FF2ALszQdtlsRkfK7R0pFD01zlwsHV1j8MYGCLmof+Anp3rJIzoiMwhgHGZu kSVt5EqUm6/YFNsiOlYKZYdulk7buKqRks9w+21qHtTphmZJpitP1KRf3rgzS8nYgrhkSSTI19itS rfTo4qVal60KTKAVjgjw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1ropXA-00000001TMV-3UW6; Mon, 25 Mar 2024 18:59:01 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1ropX7-00000001TK9-12b2 for linux-riscv@lists.infradead.org; Mon, 25 Mar 2024 18:58:59 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711393134; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=lGdQaCGcBkbZ2z9bjtfUzgfe0ZmyyKivIRm0X4ayYz0=; b=aH1+Mc9CpYASsGoFVsnVCbCCgja9EgMjBlIKVAPyL9lf1nueTR2orBSGnKS67D39S/Om1v 2pZpVmHkWUPgnK5z781yAsTWTsWfa9NZyXIIJSSk7cusvsJwc5QM09geTqEywAhjfupoq3 2BCpIvlkmNiMOcuLkjfM5gjSg6Bhx7g= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-27-qRNDnc8ONja9dgaehOuTJg-1; Mon, 25 Mar 2024 14:58:53 -0400 X-MC-Unique: qRNDnc8ONja9dgaehOuTJg-1 Received: by mail-qv1-f71.google.com with SMTP id 6a1803df08f44-69152af7760so17265586d6.1 for ; Mon, 25 Mar 2024 11:58:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711393133; x=1711997933; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=lGdQaCGcBkbZ2z9bjtfUzgfe0ZmyyKivIRm0X4ayYz0=; b=BD04MciToNRVvHxLNVma4JXSApdzM2KDJVBKSnPZoaXb513KheaPUe7emgBuKx3zqA txDFtdWy+6zKcst39FrEJbj05ruZms+yBbH1/ZOCGCpGp2AxZMIM0B95g/+eKyQY59NN C4I+jcHuZ2mr9/CIQrFPAnWTGyoCn3FF6JUzGKpPtUXgcaKeQ21iF2zYAktUGeYJlpO7 Hbjjt59HA5TH7npbWEiQHJHVv/Oj6wkZYoFNzSucBRuu4wCyHQGp552mzeWcAI7R8zYR k4adAD3y6NudzNlBgcqN6e9fyr4QXUdfvcN8wzxVbsgtKoqwHZSoiTcv6fSknYlxhKb1 UWAg== X-Forwarded-Encrypted: i=1; AJvYcCWIbLR2wich5RWPJ0CQOhoSIhISD6ccKqlzibk2et0BikcFkWnBqoHnbB+dl84PrX7Z1svupjrKmCiUNgIltvfi9ZFy5+04A5AV1zJW02Y2 X-Gm-Message-State: AOJu0YwjVwKEDuiC30XujBHNzBzl9S66lDhCrhWzKgPyoGaPTzBWABN5 DWNhwUzOUaa7+cIC4FX7WYe1L/zJqBPz9CE6qK7MyO5K6JyzxILTyZLdUygM3ESZifaP9Ib0lKY DgVppthj3iHwR/0Agy/zZdRU6rKvxWizF2vL7Za/+S2o/uv4vGnu39UWqeiItpk/Unw== X-Received: by 2002:a05:6214:3d13:b0:696:7b12:3744 with SMTP id ol19-20020a0562143d1300b006967b123744mr8189341qvb.0.1711393132764; Mon, 25 Mar 2024 11:58:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGE4pfdS6Owjbbcw6cCZoeZVpTG6xXlLRqHsMW/sJVGfLOPBLf6CMPjo5ntIkMeQpg5wPfu7g== X-Received: by 2002:a05:6214:3d13:b0:696:7b12:3744 with SMTP id ol19-20020a0562143d1300b006967b123744mr8189311qvb.0.1711393132312; Mon, 25 Mar 2024 11:58:52 -0700 (PDT) Received: from x1n ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id kd26-20020a056214401a00b006968e1bed43sm1380686qvb.79.2024.03.25.11.58.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Mar 2024 11:58:51 -0700 (PDT) Date: Mon, 25 Mar 2024 14:58:48 -0400 From: Peter Xu To: Jason Gunthorpe Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Michael Ellerman , Christophe Leroy , Matthew Wilcox , Rik van Riel , Lorenzo Stoakes , Axel Rasmussen , Yang Shi , John Hubbard , linux-arm-kernel@lists.infradead.org, "Kirill A . Shutemov" , Andrew Jones , Vlastimil Babka , Mike Rapoport , Andrew Morton , Muchun Song , Christoph Hellwig , linux-riscv@lists.infradead.org, James Houghton , David Hildenbrand , Andrea Arcangeli , "Aneesh Kumar K . V" , Mike Kravetz Subject: Re: [PATCH v3 00/12] mm/gup: Unify hugetlb, part 2 Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240322161000.GJ159172@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20240322161000.GJ159172@nvidia.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240325_115857_415958_418D501C X-CRM114-Status: GOOD ( 47.70 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org On Fri, Mar 22, 2024 at 01:10:00PM -0300, Jason Gunthorpe wrote: > On Thu, Mar 21, 2024 at 06:07:50PM -0400, peterx@redhat.com wrote: > > From: Peter Xu > > > > v3: > > - Rebased to latest mm-unstalbe (a824831a082f, of March 21th) > > - Dropped patch to introduce pmd_thp_or_huge(), replace such uses (and also > > pXd_huge() users) with pXd_leaf() [Jason] > > - Add a comment for CONFIG_PGTABLE_HAS_HUGE_LEAVES [Jason] > > - Use IS_ENABLED() in follow_huge_pud() [Jason] > > - Remove redundant none pud check in follow_pud_mask() [Jason] > > > > rfc: https://lore.kernel.org/r/20231116012908.392077-1-peterx@redhat.com > > v1: https://lore.kernel.org/r/20231219075538.414708-1-peterx@redhat.com > > v2: https://lore.kernel.org/r/20240103091423.400294-1-peterx@redhat.com > > > > The series removes the hugetlb slow gup path after a previous refactor work > > [1], so that slow gup now uses the exact same path to process all kinds of > > memory including hugetlb. > > > > For the long term, we may want to remove most, if not all, call sites of > > huge_pte_offset(). It'll be ideal if that API can be completely dropped > > from arch hugetlb API. This series is one small step towards merging > > hugetlb specific codes into generic mm paths. From that POV, this series > > removes one reference to huge_pte_offset() out of many others. > > This remark would be a little easier to understand if you refer to > hugetlb_walk() not huge_pte_offset() - the latter is only used to > implement hugetlb_walk() and isn't removed by this series, while a > single hugetlb_walk() was removed. Right. Here huge_pte_offset() is the arch API that I hope we can remove, the hugetlb_walk() is simply the wrapper. > > Regardless, I think the point is spot on, the direction should be to > make the page table reading generic with minimal/no interaction with > hugetlbfs in the generic code. Yes, and I also like your terms on calling them "pgtable readers". It's a better way to describe the difference in that regard between huge_pte_offset() v.s. huge_pte_alloc(). Exactly that's my goal, that we should start with the "readers". The writters might change semantics when merge, and can be more challenging, I'll need to look into details of each one, like page fault handlers. Such work may need to be analyzed case by case, and this GUP part is definitely the low hanging fruit comparing to the rest. > > After this series I would suggest doing the pagewalk.c stuff as it is > very parallel to GUP slow (indeed it would be amazing to figure out a > way to make GUP slow and pagewalk.c use the same code without a > performance cost) Yes. I hope there shouldn't be much perf degrade, I can do some more measurements too when getting there. And btw, IIUC the major challenge of pagewalk.c is not the removal of walk_hugetlb_range() alone - that may not be that hard if that's the solo purpose. The better way to go is to remove mm_walk_ops.hugetlb_entry() altogether, which will cause a change in all callers; that's "the challenge".. pretty much labor works, not a major technical challenge it seems. Not sure if it's a good news or bad.. One thing I'll soon look into is to allow huge mappings for PFNMAP; there's one request from VFIO side for MMIO. Dropping mm_walk_ops.hugetlb_entry() seems to be good for all such purposes; well, I may need to bite the bullet here.. for either of the purposes to move on. > > Some of the other core mm callers don't look too bad either, getting > to the point where hugetlb_walk() is internal to hugetlb.c would be a > nice step that looks reasonable size. Agree. > > > One goal of such a route is that we can reconsider merging hugetlb features > > like High Granularity Mapping (HGM). It was not accepted in the past > > because it may add lots of hugetlb specific codes and make the mm code even > > harder to maintain. With a merged codeset, features like HGM can hopefully > > share some code with THP, legacy (PMD+) or modern (continuous PTEs). > > Yeah, if all the special hugetlb stuff is using generic arch code and > generic page walkers (maybe with that special vma locking hook) it is > much easier to swallow than adding yet another special class of code > to all the page walkers. > > > To make it work, the generic slow gup code will need to at least understand > > hugepd, which is already done like so in fast-gup. Due to the specialty of > > hugepd to be software-only solution (no hardware recognizes the hugepd > > format, so it's purely artificial structures), there's chance we can merge > > some or all hugepd formats with cont_pte in the future. That question is > > yet unsettled from Power side to have an acknowledgement. > > At a minimum, I think we have a concurrence that the reading of the > hugepd entries should be fine through the standard contig pte APIs and > so all the page walkers doing read side operations could stop having > special hugepd code. It is just an implementation of contig pte with > the new property that the size of a PTE can be larger than a PMD > entry. > > If, or how much, the hugepd write side remains special is the open > question, I think. It seems balls are rolling in that aspect, I haven't looked in depth there in Christophe's series but it's great to have started! > > > this series, I kept the hugepd handling because we may still need to do so > > before getting a clearer picture of the future of hugepd. The other reason > > is simply that we did it already for fast-gup and most codes are still > > around to be reused. It'll make more sense to keep slow/fast gup behave > > the same before a decision is made to remove hugepd. > > Yeah, I think that is right for this series. Lets get this done and > then try to remove hugepd read side. Thanks a bunch for the reviews. -- Peter Xu _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E2839C54E64 for ; Mon, 25 Mar 2024 18:59:45 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Hm65WFqY; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Hm65WFqY; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4V3MgN3pjnz3dX2 for ; Tue, 26 Mar 2024 05:59:44 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Hm65WFqY; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Hm65WFqY; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=redhat.com (client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=peterx@redhat.com; receiver=lists.ozlabs.org) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4V3MfW171mz2xX4 for ; Tue, 26 Mar 2024 05:58:58 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711393135; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=lGdQaCGcBkbZ2z9bjtfUzgfe0ZmyyKivIRm0X4ayYz0=; b=Hm65WFqYPhB3yfRhsgsOohyet6vCN+TWqhbvFYZCYTTDlDHUqDAa/xfzn0WNoSA49wdK53 vO9aoq7myZV1Bj+kyJO7fcio0j1IPtsJy4XiLlqx8TNZoAyZ/rKSHUehfsKgo8qFqBPXV7 s7hhtvYWYf7Ooz3+EpkPkX2fZCcDGyI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711393135; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=lGdQaCGcBkbZ2z9bjtfUzgfe0ZmyyKivIRm0X4ayYz0=; b=Hm65WFqYPhB3yfRhsgsOohyet6vCN+TWqhbvFYZCYTTDlDHUqDAa/xfzn0WNoSA49wdK53 vO9aoq7myZV1Bj+kyJO7fcio0j1IPtsJy4XiLlqx8TNZoAyZ/rKSHUehfsKgo8qFqBPXV7 s7hhtvYWYf7Ooz3+EpkPkX2fZCcDGyI= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-27-LpQKkT9dNQKPl69uag82Zg-1; Mon, 25 Mar 2024 14:58:53 -0400 X-MC-Unique: LpQKkT9dNQKPl69uag82Zg-1 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-6968eb0a7dfso5048256d6.0 for ; Mon, 25 Mar 2024 11:58:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711393133; x=1711997933; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=lGdQaCGcBkbZ2z9bjtfUzgfe0ZmyyKivIRm0X4ayYz0=; b=mp3cbmaShN5UIEuQbkkFR4WOdc4ekelDe7hybdJnl3dye0lL8NpwL3aRXtV6TodmNd n/9um8Z5amFdBvyMa+6wPLqOzT46Te/GXgFs9KuiU3Us0uxfLfF10db7dV9hM7mMO7pb xjw3qzjwqxD0AifHb3OIMiz0+JKuiwXhgRPSQFWlpZd5kIVARhRRFv+zPvvMdRdTkgsE M9Gx6U2+idpYIESLO1eHmEGe2euGpDqOqRe7L2f535ihgejFABpv+qgdFbV+9oi+zXLH YjDLQZqHwcixVXRysB0mOlfMrSwGUpSSpgWH2LiAp6qn5bT2K9KAA2/r54k6wPSy5P6L ih0Q== X-Forwarded-Encrypted: i=1; AJvYcCVxkqLsviKHdbjaBhg0h9V1zFsk5VLMHX2Gl0EDodA+tdKCAz5LMRRE4NqKoNfjjMtvg4dQJkWDMpDWL09PVwtsSIDnp1bP2/tpCgWrYw== X-Gm-Message-State: AOJu0YwrO60n3suAF6j//qpikiClHRmXUKgCNDY11lQ5sKWCljEM1URX VW+z7gPpQD8lH6ej12WiWVz5oM/C/wCFLbqqEroS6atQtSB+AcDxDrMRuxQ0p9T+1Z5q8F000xS EwSisZkceiqeYAJMM3/DLWjFD+H654G+8MuqRVb4qzcjp4W9Xmnj+o6lbT7QMwQ0= X-Received: by 2002:a05:6214:3d13:b0:696:7b12:3744 with SMTP id ol19-20020a0562143d1300b006967b123744mr8189332qvb.0.1711393132761; Mon, 25 Mar 2024 11:58:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGE4pfdS6Owjbbcw6cCZoeZVpTG6xXlLRqHsMW/sJVGfLOPBLf6CMPjo5ntIkMeQpg5wPfu7g== X-Received: by 2002:a05:6214:3d13:b0:696:7b12:3744 with SMTP id ol19-20020a0562143d1300b006967b123744mr8189311qvb.0.1711393132312; Mon, 25 Mar 2024 11:58:52 -0700 (PDT) Received: from x1n ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id kd26-20020a056214401a00b006968e1bed43sm1380686qvb.79.2024.03.25.11.58.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Mar 2024 11:58:51 -0700 (PDT) Date: Mon, 25 Mar 2024 14:58:48 -0400 From: Peter Xu To: Jason Gunthorpe Subject: Re: [PATCH v3 00/12] mm/gup: Unify hugetlb, part 2 Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240322161000.GJ159172@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20240322161000.GJ159172@nvidia.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Houghton , David Hildenbrand , Yang Shi , Andrew Jones , linux-mm@kvack.org, linux-riscv@lists.infradead.org, Andrea Arcangeli , "Aneesh Kumar K . V" , Matthew Wilcox , Christoph Hellwig , linux-arm-kernel@lists.infradead.org, Axel Rasmussen , Rik van Riel , John Hubbard , "Kirill A . Shutemov" , Vlastimil Babka , Lorenzo Stoakes , Muchun Song , linux-kernel@vger.kernel.org, Andrew Morton , linuxppc-dev@lists.ozlabs.org, Mike Rapoport , Mike Kravetz Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Fri, Mar 22, 2024 at 01:10:00PM -0300, Jason Gunthorpe wrote: > On Thu, Mar 21, 2024 at 06:07:50PM -0400, peterx@redhat.com wrote: > > From: Peter Xu > > > > v3: > > - Rebased to latest mm-unstalbe (a824831a082f, of March 21th) > > - Dropped patch to introduce pmd_thp_or_huge(), replace such uses (and also > > pXd_huge() users) with pXd_leaf() [Jason] > > - Add a comment for CONFIG_PGTABLE_HAS_HUGE_LEAVES [Jason] > > - Use IS_ENABLED() in follow_huge_pud() [Jason] > > - Remove redundant none pud check in follow_pud_mask() [Jason] > > > > rfc: https://lore.kernel.org/r/20231116012908.392077-1-peterx@redhat.com > > v1: https://lore.kernel.org/r/20231219075538.414708-1-peterx@redhat.com > > v2: https://lore.kernel.org/r/20240103091423.400294-1-peterx@redhat.com > > > > The series removes the hugetlb slow gup path after a previous refactor work > > [1], so that slow gup now uses the exact same path to process all kinds of > > memory including hugetlb. > > > > For the long term, we may want to remove most, if not all, call sites of > > huge_pte_offset(). It'll be ideal if that API can be completely dropped > > from arch hugetlb API. This series is one small step towards merging > > hugetlb specific codes into generic mm paths. From that POV, this series > > removes one reference to huge_pte_offset() out of many others. > > This remark would be a little easier to understand if you refer to > hugetlb_walk() not huge_pte_offset() - the latter is only used to > implement hugetlb_walk() and isn't removed by this series, while a > single hugetlb_walk() was removed. Right. Here huge_pte_offset() is the arch API that I hope we can remove, the hugetlb_walk() is simply the wrapper. > > Regardless, I think the point is spot on, the direction should be to > make the page table reading generic with minimal/no interaction with > hugetlbfs in the generic code. Yes, and I also like your terms on calling them "pgtable readers". It's a better way to describe the difference in that regard between huge_pte_offset() v.s. huge_pte_alloc(). Exactly that's my goal, that we should start with the "readers". The writters might change semantics when merge, and can be more challenging, I'll need to look into details of each one, like page fault handlers. Such work may need to be analyzed case by case, and this GUP part is definitely the low hanging fruit comparing to the rest. > > After this series I would suggest doing the pagewalk.c stuff as it is > very parallel to GUP slow (indeed it would be amazing to figure out a > way to make GUP slow and pagewalk.c use the same code without a > performance cost) Yes. I hope there shouldn't be much perf degrade, I can do some more measurements too when getting there. And btw, IIUC the major challenge of pagewalk.c is not the removal of walk_hugetlb_range() alone - that may not be that hard if that's the solo purpose. The better way to go is to remove mm_walk_ops.hugetlb_entry() altogether, which will cause a change in all callers; that's "the challenge".. pretty much labor works, not a major technical challenge it seems. Not sure if it's a good news or bad.. One thing I'll soon look into is to allow huge mappings for PFNMAP; there's one request from VFIO side for MMIO. Dropping mm_walk_ops.hugetlb_entry() seems to be good for all such purposes; well, I may need to bite the bullet here.. for either of the purposes to move on. > > Some of the other core mm callers don't look too bad either, getting > to the point where hugetlb_walk() is internal to hugetlb.c would be a > nice step that looks reasonable size. Agree. > > > One goal of such a route is that we can reconsider merging hugetlb features > > like High Granularity Mapping (HGM). It was not accepted in the past > > because it may add lots of hugetlb specific codes and make the mm code even > > harder to maintain. With a merged codeset, features like HGM can hopefully > > share some code with THP, legacy (PMD+) or modern (continuous PTEs). > > Yeah, if all the special hugetlb stuff is using generic arch code and > generic page walkers (maybe with that special vma locking hook) it is > much easier to swallow than adding yet another special class of code > to all the page walkers. > > > To make it work, the generic slow gup code will need to at least understand > > hugepd, which is already done like so in fast-gup. Due to the specialty of > > hugepd to be software-only solution (no hardware recognizes the hugepd > > format, so it's purely artificial structures), there's chance we can merge > > some or all hugepd formats with cont_pte in the future. That question is > > yet unsettled from Power side to have an acknowledgement. > > At a minimum, I think we have a concurrence that the reading of the > hugepd entries should be fine through the standard contig pte APIs and > so all the page walkers doing read side operations could stop having > special hugepd code. It is just an implementation of contig pte with > the new property that the size of a PTE can be larger than a PMD > entry. > > If, or how much, the hugepd write side remains special is the open > question, I think. It seems balls are rolling in that aspect, I haven't looked in depth there in Christophe's series but it's great to have started! > > > this series, I kept the hugepd handling because we may still need to do so > > before getting a clearer picture of the future of hugepd. The other reason > > is simply that we did it already for fast-gup and most codes are still > > around to be reused. It'll make more sense to keep slow/fast gup behave > > the same before a decision is made to remove hugepd. > > Yeah, I think that is right for this series. Lets get this done and > then try to remove hugepd read side. Thanks a bunch for the reviews. -- Peter Xu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F3790C54E64 for ; Mon, 25 Mar 2024 18:59:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LR4JJPRvQ6xYHePlXlmDgPrjypT5zRn/OJUTwGdKHUE=; b=Vy0X4a+4YIPyvo vuP+/yRJOIbMmak5gkqksfVYxLNLCDpvJ+AF+Rh4xQXbn/OJPOhKe7LleTZ6IbIVRHt7UoRAdlj0P KqinF/lWskrD3XJBfdfkBWN5/F/I25igi04BjJgMcM/g317RJjqFmgnClm7kAhTvL+A+UiwoF/Ylh PF4CRFO63BF2OEHAT6Tr4W7aRYZhIDv9KsmFUwxnhi/u0YSzADWvMCZxzy4dyGpiDKzvzeiOQaTUI kxNBmPESrjX6c+JKdqbteGHnCVEAlOBFg/sWMaF/KzEiLJo3CRt+4qRpN8UoE0s9g0TNeqT40LiPU 0IlpNJPNV9lJAJau5nNw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1ropXC-00000001TMq-2udO; Mon, 25 Mar 2024 18:59:03 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1ropX7-00000001TKB-1vlD for linux-arm-kernel@lists.infradead.org; Mon, 25 Mar 2024 18:59:00 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711393134; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=lGdQaCGcBkbZ2z9bjtfUzgfe0ZmyyKivIRm0X4ayYz0=; b=aH1+Mc9CpYASsGoFVsnVCbCCgja9EgMjBlIKVAPyL9lf1nueTR2orBSGnKS67D39S/Om1v 2pZpVmHkWUPgnK5z781yAsTWTsWfa9NZyXIIJSSk7cusvsJwc5QM09geTqEywAhjfupoq3 2BCpIvlkmNiMOcuLkjfM5gjSg6Bhx7g= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-544-vRRmmNS-NqGZd4Am6r1xqQ-1; Mon, 25 Mar 2024 14:58:53 -0400 X-MC-Unique: vRRmmNS-NqGZd4Am6r1xqQ-1 Received: by mail-qv1-f71.google.com with SMTP id 6a1803df08f44-69152af7760so17265646d6.1 for ; Mon, 25 Mar 2024 11:58:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711393133; x=1711997933; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=lGdQaCGcBkbZ2z9bjtfUzgfe0ZmyyKivIRm0X4ayYz0=; b=EvDYnGavZHq+2pZvnMZTOoidxuvhh9RCOHgvu84LCKovf2v/Zqxqsbev7rXCrAzsb4 mvpY3zmqLvJprUceI2jDrSbqoS/iRvs1vz1wd41SGIyrYc5ISbIZSnKs2zYP0nW0y6qw Fog84WG51hxD/gBBaIXmZEiUi52E4jNY31+K3XHG4kPTLUbNbAutuqtC5Mg7XyjmCoXv ClevpB3L2N2V/Lt/9qcA6ffx9ptZBTHH4JB5dKT9YJ6Oqp0ip0/2mBax5LxVU3hcOr8K fswfefuy/xHNZQOPoWrpNuAKZFNclO6tt2oKmVCEbfZi5l1Qai+4/U7sXa1xxitqoKO4 n26A== X-Forwarded-Encrypted: i=1; AJvYcCU1Dkuxfqyi7ytk4Yh5DS22h32qIGuWw+/mf+VNn1YTt6fhtCd7Y1ax2z7epb7KeBZzvxWiVvgcCvSMyqrM+Vq4iB8bNEowkQg8uXCNwNhejL4BqI4= X-Gm-Message-State: AOJu0YxK7YxY3MG/Uice9hNWpUfl+uD6Ptg0ro2Y+hxPibmAQsN8jNwa Nsy3K5VLIjmtEv7IVie+bULvXpgoAljG19BXekMeSv5Bt4FEIxxDvzShyIk2p/HN4XqUwp+e9YC +KKLAUrO4oPdbrX3tURKp3UG0E/mE+hvB/T2/dv+dVUpN1jmA2UAcfvKZxChrY03xGfVsuBN+ X-Received: by 2002:a05:6214:3d13:b0:696:7b12:3744 with SMTP id ol19-20020a0562143d1300b006967b123744mr8189330qvb.0.1711393132761; Mon, 25 Mar 2024 11:58:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGE4pfdS6Owjbbcw6cCZoeZVpTG6xXlLRqHsMW/sJVGfLOPBLf6CMPjo5ntIkMeQpg5wPfu7g== X-Received: by 2002:a05:6214:3d13:b0:696:7b12:3744 with SMTP id ol19-20020a0562143d1300b006967b123744mr8189311qvb.0.1711393132312; Mon, 25 Mar 2024 11:58:52 -0700 (PDT) Received: from x1n ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id kd26-20020a056214401a00b006968e1bed43sm1380686qvb.79.2024.03.25.11.58.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Mar 2024 11:58:51 -0700 (PDT) Date: Mon, 25 Mar 2024 14:58:48 -0400 From: Peter Xu To: Jason Gunthorpe Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Michael Ellerman , Christophe Leroy , Matthew Wilcox , Rik van Riel , Lorenzo Stoakes , Axel Rasmussen , Yang Shi , John Hubbard , linux-arm-kernel@lists.infradead.org, "Kirill A . Shutemov" , Andrew Jones , Vlastimil Babka , Mike Rapoport , Andrew Morton , Muchun Song , Christoph Hellwig , linux-riscv@lists.infradead.org, James Houghton , David Hildenbrand , Andrea Arcangeli , "Aneesh Kumar K . V" , Mike Kravetz Subject: Re: [PATCH v3 00/12] mm/gup: Unify hugetlb, part 2 Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240322161000.GJ159172@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20240322161000.GJ159172@nvidia.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240325_115857_590806_71056AD9 X-CRM114-Status: GOOD ( 49.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Mar 22, 2024 at 01:10:00PM -0300, Jason Gunthorpe wrote: > On Thu, Mar 21, 2024 at 06:07:50PM -0400, peterx@redhat.com wrote: > > From: Peter Xu > > > > v3: > > - Rebased to latest mm-unstalbe (a824831a082f, of March 21th) > > - Dropped patch to introduce pmd_thp_or_huge(), replace such uses (and also > > pXd_huge() users) with pXd_leaf() [Jason] > > - Add a comment for CONFIG_PGTABLE_HAS_HUGE_LEAVES [Jason] > > - Use IS_ENABLED() in follow_huge_pud() [Jason] > > - Remove redundant none pud check in follow_pud_mask() [Jason] > > > > rfc: https://lore.kernel.org/r/20231116012908.392077-1-peterx@redhat.com > > v1: https://lore.kernel.org/r/20231219075538.414708-1-peterx@redhat.com > > v2: https://lore.kernel.org/r/20240103091423.400294-1-peterx@redhat.com > > > > The series removes the hugetlb slow gup path after a previous refactor work > > [1], so that slow gup now uses the exact same path to process all kinds of > > memory including hugetlb. > > > > For the long term, we may want to remove most, if not all, call sites of > > huge_pte_offset(). It'll be ideal if that API can be completely dropped > > from arch hugetlb API. This series is one small step towards merging > > hugetlb specific codes into generic mm paths. From that POV, this series > > removes one reference to huge_pte_offset() out of many others. > > This remark would be a little easier to understand if you refer to > hugetlb_walk() not huge_pte_offset() - the latter is only used to > implement hugetlb_walk() and isn't removed by this series, while a > single hugetlb_walk() was removed. Right. Here huge_pte_offset() is the arch API that I hope we can remove, the hugetlb_walk() is simply the wrapper. > > Regardless, I think the point is spot on, the direction should be to > make the page table reading generic with minimal/no interaction with > hugetlbfs in the generic code. Yes, and I also like your terms on calling them "pgtable readers". It's a better way to describe the difference in that regard between huge_pte_offset() v.s. huge_pte_alloc(). Exactly that's my goal, that we should start with the "readers". The writters might change semantics when merge, and can be more challenging, I'll need to look into details of each one, like page fault handlers. Such work may need to be analyzed case by case, and this GUP part is definitely the low hanging fruit comparing to the rest. > > After this series I would suggest doing the pagewalk.c stuff as it is > very parallel to GUP slow (indeed it would be amazing to figure out a > way to make GUP slow and pagewalk.c use the same code without a > performance cost) Yes. I hope there shouldn't be much perf degrade, I can do some more measurements too when getting there. And btw, IIUC the major challenge of pagewalk.c is not the removal of walk_hugetlb_range() alone - that may not be that hard if that's the solo purpose. The better way to go is to remove mm_walk_ops.hugetlb_entry() altogether, which will cause a change in all callers; that's "the challenge".. pretty much labor works, not a major technical challenge it seems. Not sure if it's a good news or bad.. One thing I'll soon look into is to allow huge mappings for PFNMAP; there's one request from VFIO side for MMIO. Dropping mm_walk_ops.hugetlb_entry() seems to be good for all such purposes; well, I may need to bite the bullet here.. for either of the purposes to move on. > > Some of the other core mm callers don't look too bad either, getting > to the point where hugetlb_walk() is internal to hugetlb.c would be a > nice step that looks reasonable size. Agree. > > > One goal of such a route is that we can reconsider merging hugetlb features > > like High Granularity Mapping (HGM). It was not accepted in the past > > because it may add lots of hugetlb specific codes and make the mm code even > > harder to maintain. With a merged codeset, features like HGM can hopefully > > share some code with THP, legacy (PMD+) or modern (continuous PTEs). > > Yeah, if all the special hugetlb stuff is using generic arch code and > generic page walkers (maybe with that special vma locking hook) it is > much easier to swallow than adding yet another special class of code > to all the page walkers. > > > To make it work, the generic slow gup code will need to at least understand > > hugepd, which is already done like so in fast-gup. Due to the specialty of > > hugepd to be software-only solution (no hardware recognizes the hugepd > > format, so it's purely artificial structures), there's chance we can merge > > some or all hugepd formats with cont_pte in the future. That question is > > yet unsettled from Power side to have an acknowledgement. > > At a minimum, I think we have a concurrence that the reading of the > hugepd entries should be fine through the standard contig pte APIs and > so all the page walkers doing read side operations could stop having > special hugepd code. It is just an implementation of contig pte with > the new property that the size of a PTE can be larger than a PMD > entry. > > If, or how much, the hugepd write side remains special is the open > question, I think. It seems balls are rolling in that aspect, I haven't looked in depth there in Christophe's series but it's great to have started! > > > this series, I kept the hugepd handling because we may still need to do so > > before getting a clearer picture of the future of hugepd. The other reason > > is simply that we did it already for fast-gup and most codes are still > > around to be reused. It'll make more sense to keep slow/fast gup behave > > the same before a decision is made to remove hugepd. > > Yeah, I think that is right for this series. Lets get this done and > then try to remove hugepd read side. Thanks a bunch for the reviews. -- Peter Xu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FAA7C54E64 for ; Mon, 25 Mar 2024 18:58:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 308906B0085; Mon, 25 Mar 2024 14:58:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2B89A6B0087; Mon, 25 Mar 2024 14:58:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A7BB6B0088; Mon, 25 Mar 2024 14:58:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 0B30D6B0085 for ; Mon, 25 Mar 2024 14:58:58 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6F91CA0BAA for ; Mon, 25 Mar 2024 18:58:57 +0000 (UTC) X-FDA: 81936473514.01.5028869 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf26.hostedemail.com (Postfix) with ESMTP id 5A05B14000D for ; Mon, 25 Mar 2024 18:58:55 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=aH1+Mc9C; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf26.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711393135; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lGdQaCGcBkbZ2z9bjtfUzgfe0ZmyyKivIRm0X4ayYz0=; b=1FEnKkh4b/grYvvcgnM0zk5BLMwgiAYPE2p0rSYwJ61vEIV42nGjIou0EnncqR2bNn/Xqt mUx1PcW9ZxjT32YHHRgD90ZPl2qqcnpMMeHqf8d51SRhXs7xsd1oH1jVqGUk/nnPHYl3Op IKlSMQhr1ZT7t55/pAqN3rFBdftPafs= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=aH1+Mc9C; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf26.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711393135; a=rsa-sha256; cv=none; b=WOkp5F+IofgJVeOxCilldDX+uH0PUHzn4G73nXvQ6IoycaMYlOBSKroezWOxyu8LL9KmOV 1oaqx7OXnYpFMT2uzNF55eDXm+Zmq1rg3BffBYHSr7pMQ1QgCfoCoHcgzPo5/jHD4e3cfS YSKvtA9UkEhvDsCiRNC4PGSnJu7wESw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711393134; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=lGdQaCGcBkbZ2z9bjtfUzgfe0ZmyyKivIRm0X4ayYz0=; b=aH1+Mc9CpYASsGoFVsnVCbCCgja9EgMjBlIKVAPyL9lf1nueTR2orBSGnKS67D39S/Om1v 2pZpVmHkWUPgnK5z781yAsTWTsWfa9NZyXIIJSSk7cusvsJwc5QM09geTqEywAhjfupoq3 2BCpIvlkmNiMOcuLkjfM5gjSg6Bhx7g= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-354-VB8RBe2SOcuBSOLyFyAg1w-1; Mon, 25 Mar 2024 14:58:53 -0400 X-MC-Unique: VB8RBe2SOcuBSOLyFyAg1w-1 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-69152af7760so17265596d6.1 for ; Mon, 25 Mar 2024 11:58:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711393133; x=1711997933; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=lGdQaCGcBkbZ2z9bjtfUzgfe0ZmyyKivIRm0X4ayYz0=; b=SCJxHED8DUAXEuwxD2+HVSUzY4qV/nDIo6O2FwNDvIhpQ49rsotz7Olh1hZIGK0ccY ejcrfJR+SqxLU52FPFu4Dr7s4pgdMcFKDgOhKOIcsCWJP5L4DWKya5nqeOTB84sRIMAr x8qgUI7Hz/PL0bbZiQz4rzD3FwNLIzoWHISnbPwwUxfao88TEYVyiEiBsI8wG9pe7uvQ X+qBGUWoXE3Dox8CDsGZW/6v2F4tBpQtesjyySz6mOP8OpTqIMzo2/adlzseR2WmaKOR HifZl6Hn/3qnx1AYjKCkSXOrqixFlZiQT/tEL2U9uxC1y3NxfQwdtsPShKrhXphHU/Jv OhLA== X-Gm-Message-State: AOJu0YxlKGbaFAldIDCHLRAHJzTrMoQaV6bAaa1fbO4OFG80oEsiRypY SDQ+ql5DuamN3UGT0j/fKW2iUFIKuJGf0+7sLwYBu5LeL3ZRome95CkQWOXiRuvxa8r/sXxjoNJ 51duwGfR+uPPiADiGqHUfXVBy3dgWeZAxRlaUUkC2OrqMGLhY X-Received: by 2002:a05:6214:3d13:b0:696:7b12:3744 with SMTP id ol19-20020a0562143d1300b006967b123744mr8189329qvb.0.1711393132761; Mon, 25 Mar 2024 11:58:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGE4pfdS6Owjbbcw6cCZoeZVpTG6xXlLRqHsMW/sJVGfLOPBLf6CMPjo5ntIkMeQpg5wPfu7g== X-Received: by 2002:a05:6214:3d13:b0:696:7b12:3744 with SMTP id ol19-20020a0562143d1300b006967b123744mr8189311qvb.0.1711393132312; Mon, 25 Mar 2024 11:58:52 -0700 (PDT) Received: from x1n ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id kd26-20020a056214401a00b006968e1bed43sm1380686qvb.79.2024.03.25.11.58.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Mar 2024 11:58:51 -0700 (PDT) Date: Mon, 25 Mar 2024 14:58:48 -0400 From: Peter Xu To: Jason Gunthorpe Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Michael Ellerman , Christophe Leroy , Matthew Wilcox , Rik van Riel , Lorenzo Stoakes , Axel Rasmussen , Yang Shi , John Hubbard , linux-arm-kernel@lists.infradead.org, "Kirill A . Shutemov" , Andrew Jones , Vlastimil Babka , Mike Rapoport , Andrew Morton , Muchun Song , Christoph Hellwig , linux-riscv@lists.infradead.org, James Houghton , David Hildenbrand , Andrea Arcangeli , "Aneesh Kumar K . V" , Mike Kravetz Subject: Re: [PATCH v3 00/12] mm/gup: Unify hugetlb, part 2 Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240322161000.GJ159172@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20240322161000.GJ159172@nvidia.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Rspam-User: X-Stat-Signature: xeitgcwf7spe6z9oa6xkrkdxofyqhtya X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5A05B14000D X-HE-Tag: 1711393135-893172 X-HE-Meta: U2FsdGVkX19AWIpDSzMx9fNL7WaI6rYyGGPAXgyBee7XwAPXLlIVXObQHuv42wTXCt+i+oDczs9fbkLhu1HJRjN5QoM6HtIby4/5DgqF5vuzndax+vuz2xzhMLp0VDzHAz1dBfClQ05bfBMk/ekoz+PKPFWPgPkbZyCCZEAy+ypIm0TGDX0753wzJ/yWxwfeafszImFaQxLtWPEDdGU2T6urojM2TLN6AkFggPUT6x1yR++ebF2yvaZH1vEHq/wPMAXcu72XY55CDXtoNKU2RXirQA3661RMnN/f0Pkbp5eemBQKfuQY6kxWaCYnhuu3vEPxWkSrAOwsqGI2mNZUPp6uYsBC0soH17chpOaTUspf4JLRDi178WXghuIFcY99AlwTlM6D0vdwIFVeDrQcSB8fe9zMiAqobxiELH6Oa3jQLxwc+bxfDJkHv1Frwc2mQGHw2sP0gHuQoAPK+YgMqsu1D+ERKyn3pjnS/Qw07M/tQYrjdWt39hjSL3mxWGE5VLM56LvHzdCeJLTGgessPd0GB+bES+jogD2beBwNVHu2mvxyiiWe3c+3nJHmP8ii46YQ5fCZgigJKR81JkTUtNmdhDkM6eNsQWh3Zl62UA+a9jyMphaIr5IctGnV2DbSqZsgehDH9gy0ZaexWtHjGMYfLOyIbbmSEE3AzEaIIetXlQ2b2yHjORBsnKl99pD5quCq9AYFtmjvD8GV/aR9jxzCE/uf16hOFP8maivOsFCorysCHNth+RykcGY2nmLX297tgto4QaVVU8qG0m/VlUr5FLxReoqQ36f7F69FnI49wUCa7jDKzcPZTp9G3x67XwLJ3Sw27V8oZCJ2LO1UXobmzwQI1TuxtuGnzVUcFzNpOAtyMwDGH/shBgNaQPCHsY+iliTrxlRJxNIeY7dyEIqpA8meIwjqiIUbxWobYAwlMJImSGyuH3vnL/+gDcp/4lI5cjWnb7WsWRJNYW7 44BnwQ7n W93Lj9yhzlhEOU3D4dHoJiuRlx/lOKw0pyjGSGpY1KqSKmDqzSC1Kv9CSCcDEKxutSvpqp4SHTp0GlDXQijAtnUQXlZ3wZ/zevbSI/2ch/wi+JGEK0yHpW1hTxFSozFFtiKn6vSgwLE5L537DP5UIN5ahYh57Ei61irKxEWzJUaUiV/IaBxWvQvhks65isctYv68jx2G8ej/Eycs0oHhJ3TuaE/TbpW94S5Sh9r4c8IoyUnRyZqGX0iX/otsO7rhbgkv+tUO3gfxgLwKeCWuzfRM/4JMwy/uMclCfCAiSvURQciwzODBJs26LWCl86NE50yz783fP1US+79zHScV/d3dARpTmwAxvqIOlwUSfDtQaZ55PmbLwHkg2Min1fPE4LpQJnAQ5viNHLiZULXLZNQO5cOVEEs8SkaJZPKvfKc1DyHqYyZ0/gG7dUj1K2GVzc3VB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Mar 22, 2024 at 01:10:00PM -0300, Jason Gunthorpe wrote: > On Thu, Mar 21, 2024 at 06:07:50PM -0400, peterx@redhat.com wrote: > > From: Peter Xu > > > > v3: > > - Rebased to latest mm-unstalbe (a824831a082f, of March 21th) > > - Dropped patch to introduce pmd_thp_or_huge(), replace such uses (and also > > pXd_huge() users) with pXd_leaf() [Jason] > > - Add a comment for CONFIG_PGTABLE_HAS_HUGE_LEAVES [Jason] > > - Use IS_ENABLED() in follow_huge_pud() [Jason] > > - Remove redundant none pud check in follow_pud_mask() [Jason] > > > > rfc: https://lore.kernel.org/r/20231116012908.392077-1-peterx@redhat.com > > v1: https://lore.kernel.org/r/20231219075538.414708-1-peterx@redhat.com > > v2: https://lore.kernel.org/r/20240103091423.400294-1-peterx@redhat.com > > > > The series removes the hugetlb slow gup path after a previous refactor work > > [1], so that slow gup now uses the exact same path to process all kinds of > > memory including hugetlb. > > > > For the long term, we may want to remove most, if not all, call sites of > > huge_pte_offset(). It'll be ideal if that API can be completely dropped > > from arch hugetlb API. This series is one small step towards merging > > hugetlb specific codes into generic mm paths. From that POV, this series > > removes one reference to huge_pte_offset() out of many others. > > This remark would be a little easier to understand if you refer to > hugetlb_walk() not huge_pte_offset() - the latter is only used to > implement hugetlb_walk() and isn't removed by this series, while a > single hugetlb_walk() was removed. Right. Here huge_pte_offset() is the arch API that I hope we can remove, the hugetlb_walk() is simply the wrapper. > > Regardless, I think the point is spot on, the direction should be to > make the page table reading generic with minimal/no interaction with > hugetlbfs in the generic code. Yes, and I also like your terms on calling them "pgtable readers". It's a better way to describe the difference in that regard between huge_pte_offset() v.s. huge_pte_alloc(). Exactly that's my goal, that we should start with the "readers". The writters might change semantics when merge, and can be more challenging, I'll need to look into details of each one, like page fault handlers. Such work may need to be analyzed case by case, and this GUP part is definitely the low hanging fruit comparing to the rest. > > After this series I would suggest doing the pagewalk.c stuff as it is > very parallel to GUP slow (indeed it would be amazing to figure out a > way to make GUP slow and pagewalk.c use the same code without a > performance cost) Yes. I hope there shouldn't be much perf degrade, I can do some more measurements too when getting there. And btw, IIUC the major challenge of pagewalk.c is not the removal of walk_hugetlb_range() alone - that may not be that hard if that's the solo purpose. The better way to go is to remove mm_walk_ops.hugetlb_entry() altogether, which will cause a change in all callers; that's "the challenge".. pretty much labor works, not a major technical challenge it seems. Not sure if it's a good news or bad.. One thing I'll soon look into is to allow huge mappings for PFNMAP; there's one request from VFIO side for MMIO. Dropping mm_walk_ops.hugetlb_entry() seems to be good for all such purposes; well, I may need to bite the bullet here.. for either of the purposes to move on. > > Some of the other core mm callers don't look too bad either, getting > to the point where hugetlb_walk() is internal to hugetlb.c would be a > nice step that looks reasonable size. Agree. > > > One goal of such a route is that we can reconsider merging hugetlb features > > like High Granularity Mapping (HGM). It was not accepted in the past > > because it may add lots of hugetlb specific codes and make the mm code even > > harder to maintain. With a merged codeset, features like HGM can hopefully > > share some code with THP, legacy (PMD+) or modern (continuous PTEs). > > Yeah, if all the special hugetlb stuff is using generic arch code and > generic page walkers (maybe with that special vma locking hook) it is > much easier to swallow than adding yet another special class of code > to all the page walkers. > > > To make it work, the generic slow gup code will need to at least understand > > hugepd, which is already done like so in fast-gup. Due to the specialty of > > hugepd to be software-only solution (no hardware recognizes the hugepd > > format, so it's purely artificial structures), there's chance we can merge > > some or all hugepd formats with cont_pte in the future. That question is > > yet unsettled from Power side to have an acknowledgement. > > At a minimum, I think we have a concurrence that the reading of the > hugepd entries should be fine through the standard contig pte APIs and > so all the page walkers doing read side operations could stop having > special hugepd code. It is just an implementation of contig pte with > the new property that the size of a PTE can be larger than a PMD > entry. > > If, or how much, the hugepd write side remains special is the open > question, I think. It seems balls are rolling in that aspect, I haven't looked in depth there in Christophe's series but it's great to have started! > > > this series, I kept the hugepd handling because we may still need to do so > > before getting a clearer picture of the future of hugepd. The other reason > > is simply that we did it already for fast-gup and most codes are still > > around to be reused. It'll make more sense to keep slow/fast gup behave > > the same before a decision is made to remove hugepd. > > Yeah, I think that is right for this series. Lets get this done and > then try to remove hugepd read side. Thanks a bunch for the reviews. -- Peter Xu