From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BAF20C47DD9 for ; Fri, 22 Mar 2024 15:55:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Pb9PHh6puTH0IRK2HNQ+nOi31vKD03BWv1Yrv4dPq14=; b=loZUk866lgtGlt nBsauQ0vVN93+0JY+/7h0wkhsBshhYWyySV7eQNwq4n2qshLfpY2Yk+z7mVeQA2vkPw9me/+Cux9N i+1Z43IulF6e13DHOXqgoyOX8cbEWNXXExjjQEDXAvubBNc0B7LC6yAYDH/9Whr1r/7HAYYL05nPe J1L6yr8y4+xgA0iwKNolAm0GRuQ2sWJTd4+Bn4mU1Hw6mgHVvK+uY5CcnlVcVMvwUq+xX7q9TP8nq 861ZaSR1RDRk43i+KE+ofbjCytO0OWeEg4gWA+7nYu7p8qOu83ly63q4z1Hp+ZlbVQWY7UwY9uDAM TByTl0WiiBtMUQeNuRKg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnhEp-00000007ptE-3K5p; Fri, 22 Mar 2024 15:55:23 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnhEn-00000007ps2-0Bta for linux-riscv@lists.infradead.org; Fri, 22 Mar 2024 15:55:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711122920; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QJLFb75dlh9Ilo5izUjxoeGL/8+am1RJvB8MBnP+8/Q=; b=fWVURMiQ5dd5Ao5d0q5LExeFJMB0B0t6ZIcm6V4PNGgLub9IQ5KCfo1oqiniA9YePKSTrj hWCS7CFqi4hDuMOfnKlsEx/9zRFaotgq70f5jvPN72QhOrvGWYMoQ9tYTtJfIVBGQjctgo wwdLR4zAh51n45sR6lbKHPc8UwI88Pk= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-464-dPbukjC5Ple6j5fitMMYuw-1; Fri, 22 Mar 2024 11:55:16 -0400 X-MC-Unique: dPbukjC5Ple6j5fitMMYuw-1 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-69672754479so912106d6.1 for ; Fri, 22 Mar 2024 08:55:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711122916; x=1711727716; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=QJLFb75dlh9Ilo5izUjxoeGL/8+am1RJvB8MBnP+8/Q=; b=dnPYjJOTo/blTM8rD65XgXLnKcgXqVVg5CQEFgzWnbBIfNRQXgIgg2D+etyq6ZFqht p5KA+m0EjGKQStugyJvZ3rqw3YZDdgWGId1hIzh0CeAY6HMesfKSytbnZ9Y0kYltlOZN 0GnFlCi5AdN/ku4smtR3eQs9aWlWoWR7F6e9rqN0/a6ddpBNtHL0PRcfVDsfwxtYy5Cl Njf4IoD6+yDM6Cl3qnK69oronNblaQ3v20NCoRMKah2qduJ9baB1iNMr+xZ9ZA6xz/xX s0ZuMOpC1FL9UIwQZj3KLJMmBYBBkVVhFnmdjbyvzg6Ubl6A825NQcA+g8RNLYszGhtS l9xw== X-Forwarded-Encrypted: i=1; AJvYcCVxwODihPPKeuvIAiRO+StoShy5uCNCCnZPXWZQxjA3t70DcN7rOGz4PLi/mVi5lusrITM74wfbyMnUyHBTfDk4zgTMCvB0B9jrWwupcVXU X-Gm-Message-State: AOJu0YwCEocsnb9UFllTC8elE+DTEQd2wFB8A0hvKsedWHYdvgYtMeVQ 40HsQb7kHdlXd3qMDkjSIt+gwUmGtZTdMzVejB6OnKN8mFvxEN01xxprVIX+e3Bau0QnrXyzB0c Q5QNAvqOT5dutWd7zItGI9pTxeOTOg3k2crq9WWVxFLk6+VUtZwXbRwn/kZvKoVdMIg== X-Received: by 2002:a05:6214:2b86:b0:696:732f:41ce with SMTP id kr6-20020a0562142b8600b00696732f41cemr1030618qvb.0.1711122916052; Fri, 22 Mar 2024 08:55:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGcGgYl+6vk8GOgJHrnHh9Bk553GXwTOWFwJfwdxmayaEVIZH35b8wrrTSaV+/rgNR3L0gYLQ== X-Received: by 2002:a05:6214:2b86:b0:696:732f:41ce with SMTP id kr6-20020a0562142b8600b00696732f41cemr1030577qvb.0.1711122915578; Fri, 22 Mar 2024 08:55:15 -0700 (PDT) Received: from x1n ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id jx12-20020a0562142b0c00b0068ff8bda6c7sm1185452qvb.92.2024.03.22.08.55.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Mar 2024 08:55:14 -0700 (PDT) Date: Fri, 22 Mar 2024 11:55:11 -0400 From: Peter Xu To: Jason Gunthorpe Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Michael Ellerman , Christophe Leroy , Matthew Wilcox , Rik van Riel , Lorenzo Stoakes , Axel Rasmussen , Yang Shi , John Hubbard , linux-arm-kernel@lists.infradead.org, "Kirill A . Shutemov" , Andrew Jones , Vlastimil Babka , Mike Rapoport , Andrew Morton , Muchun Song , Christoph Hellwig , linux-riscv@lists.infradead.org, James Houghton , David Hildenbrand , Andrea Arcangeli , "Aneesh Kumar K . V" , Mike Kravetz Subject: Re: [PATCH v3 12/12] mm/gup: Handle hugetlb in the generic follow_page_mask code Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240321220802.679544-13-peterx@redhat.com> <20240322133012.GI159172@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20240322133012.GI159172@nvidia.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240322_085521_163796_8C18A76F X-CRM114-Status: GOOD ( 23.49 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Jason, On Fri, Mar 22, 2024 at 10:30:12AM -0300, Jason Gunthorpe wrote: > On Thu, Mar 21, 2024 at 06:08:02PM -0400, peterx@redhat.com wrote: > > > A quick performance test on an aarch64 VM on M1 chip shows 15% degrade over > > a tight loop of slow gup after the path switched. That shouldn't be a > > problem because slow-gup should not be a hot path for GUP in general: when > > page is commonly present, fast-gup will already succeed, while when the > > page is indeed missing and require a follow up page fault, the slow gup > > degrade will probably buried in the fault paths anyway. It also explains > > why slow gup for THP used to be very slow before 57edfcfd3419 ("mm/gup: > > accelerate thp gup even for "pages != NULL"") lands, the latter not part of > > a performance analysis but a side benefit. If the performance will be a > > concern, we can consider handle CONT_PTE in follow_page(). > > I think this is probably fine for the moment, at least for this > series, as CONT_PTE is still very new. > > But it will need to be optimized. "slow" GUP is the only GUP that is > used by FOLL_LONGTERM and it still needs to be optimized because you > can't assume a FOLL_LONGTERM user will be hitting the really slow > fault path. There are enough important cases where it is just reading > already populted page tables, and these days, often with large folios. Ah, I thought FOLL_LONGTERM should work in most cases for fast-gup, especially for hugetlb, but maybe I missed something? I do see that devmap skips fast-gup for LONGTERM, we also have that writeback issue but none of those that I can find applies to hugetlb. This might be a problem indeed if we have hugetlb cont_pte pages that will constantly fallback to slow gup. OTOH, I also agree with you that such batching would be nice to have for slow-gup, likely devmap or many fs (exclude shmem/hugetlb) file mappings can at least benefit from it due to above. But then that'll be a more generic issue to solve, IOW, we still don't do that for !hugetlb cont_pte large folios, before or after this series. > > Reviewed-by: Jason Gunthorpe Thanks! -- Peter Xu _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 32F49C47DD9 for ; Fri, 22 Mar 2024 15:56:11 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Toy4bsv3; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Toy4bsv3; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4V1Rkx59szz3vdv for ; Sat, 23 Mar 2024 02:56:09 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Toy4bsv3; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Toy4bsv3; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=redhat.com (client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=peterx@redhat.com; receiver=lists.ozlabs.org) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4V1Rk334h0z3cBN for ; Sat, 23 Mar 2024 02:55:21 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711122919; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QJLFb75dlh9Ilo5izUjxoeGL/8+am1RJvB8MBnP+8/Q=; b=Toy4bsv3Ad85EA+NvHhY+40Wru/FLAZfGe5Va6JZ2KGVavZpglqvuJLQ+1oTB7EtyfyHM2 ZydbIPdbAg0z6EHbiLi7IpXveKI5PQTW3tCfNf+/yoBiixUmpN3zvNLydlbKPjMaq58U0P fVP4gPz4L4phy7Szq+YaNPZgCJb93Rw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711122919; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QJLFb75dlh9Ilo5izUjxoeGL/8+am1RJvB8MBnP+8/Q=; b=Toy4bsv3Ad85EA+NvHhY+40Wru/FLAZfGe5Va6JZ2KGVavZpglqvuJLQ+1oTB7EtyfyHM2 ZydbIPdbAg0z6EHbiLi7IpXveKI5PQTW3tCfNf+/yoBiixUmpN3zvNLydlbKPjMaq58U0P fVP4gPz4L4phy7Szq+YaNPZgCJb93Rw= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-690-WVF4NXapPcqoDuvirRj74Q-1; Fri, 22 Mar 2024 11:55:16 -0400 X-MC-Unique: WVF4NXapPcqoDuvirRj74Q-1 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-690d3f5af86so5121046d6.0 for ; Fri, 22 Mar 2024 08:55:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711122916; x=1711727716; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=QJLFb75dlh9Ilo5izUjxoeGL/8+am1RJvB8MBnP+8/Q=; b=Wx/KvlPSc1pL5oL3+iprLtdw3aZpctv2J99GG3Uc07QwbHRT8w2ZVofOT8E9SynkhA Fj+ssNi1yI3+usFlwSaqDjty3x1V2UL50IqCBcGfRVmcNQF5coLq3wdMBHljY9vWPC/E MT3ysORtkkwisiKkmbrhObML57yGU9MSiyFPJb3+Pk4lK5TSkxqA3L73XdlE7eDpB2Z7 xyM2YIK+8/qFb13vNrkgf2x05OPg7bxOqjqyND1styLice/UzqYQumwaZUBJm7vyBHKc VtOvcgOdk4Cv28Ky20ndKZH2WMLlk7EMEwJIdLE2oDABiZiriL97jojNe1JdTL1PBQ+h Cb7g== X-Forwarded-Encrypted: i=1; AJvYcCWKCmq8Irk603/bOU5ETy/q0+Fimr7rmDITurokEBeA1LZUKQMxqM6Cbcvks7K0ENDTrVPVdDNnQFGZar91yMh7GHe44BQ1pTm4g7JtHA== X-Gm-Message-State: AOJu0YwGLNZ3C/wfnVbxqC15r+1hgJT5qJQZO2JxkZfb1xerSoDcyexJ EzCklZ6wsFaJq9FW9Z8IKRpF01SS3mbt9BIk5AG/70Mh0EQGMWdc4PovT6+++NcSiDUDMcVZQ2/ /H+wXNYhtM9v2ihgwoBrApuoTBlz60NiPsI2d+QIOPlk5M5b/e+Xw1ksuRPTaPd8= X-Received: by 2002:a05:6214:2b86:b0:696:732f:41ce with SMTP id kr6-20020a0562142b8600b00696732f41cemr1030605qvb.0.1711122916029; Fri, 22 Mar 2024 08:55:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGcGgYl+6vk8GOgJHrnHh9Bk553GXwTOWFwJfwdxmayaEVIZH35b8wrrTSaV+/rgNR3L0gYLQ== X-Received: by 2002:a05:6214:2b86:b0:696:732f:41ce with SMTP id kr6-20020a0562142b8600b00696732f41cemr1030577qvb.0.1711122915578; Fri, 22 Mar 2024 08:55:15 -0700 (PDT) Received: from x1n ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id jx12-20020a0562142b0c00b0068ff8bda6c7sm1185452qvb.92.2024.03.22.08.55.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Mar 2024 08:55:14 -0700 (PDT) Date: Fri, 22 Mar 2024 11:55:11 -0400 From: Peter Xu To: Jason Gunthorpe Subject: Re: [PATCH v3 12/12] mm/gup: Handle hugetlb in the generic follow_page_mask code Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240321220802.679544-13-peterx@redhat.com> <20240322133012.GI159172@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20240322133012.GI159172@nvidia.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Houghton , David Hildenbrand , Yang Shi , Andrew Jones , linux-mm@kvack.org, linux-riscv@lists.infradead.org, Andrea Arcangeli , "Aneesh Kumar K . V" , Matthew Wilcox , Christoph Hellwig , linux-arm-kernel@lists.infradead.org, Axel Rasmussen , Rik van Riel , John Hubbard , "Kirill A . Shutemov" , Vlastimil Babka , Lorenzo Stoakes , Muchun Song , linux-kernel@vger.kernel.org, Andrew Morton , linuxppc-dev@lists.ozlabs.org, Mike Rapoport , Mike Kravetz Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" Jason, On Fri, Mar 22, 2024 at 10:30:12AM -0300, Jason Gunthorpe wrote: > On Thu, Mar 21, 2024 at 06:08:02PM -0400, peterx@redhat.com wrote: > > > A quick performance test on an aarch64 VM on M1 chip shows 15% degrade over > > a tight loop of slow gup after the path switched. That shouldn't be a > > problem because slow-gup should not be a hot path for GUP in general: when > > page is commonly present, fast-gup will already succeed, while when the > > page is indeed missing and require a follow up page fault, the slow gup > > degrade will probably buried in the fault paths anyway. It also explains > > why slow gup for THP used to be very slow before 57edfcfd3419 ("mm/gup: > > accelerate thp gup even for "pages != NULL"") lands, the latter not part of > > a performance analysis but a side benefit. If the performance will be a > > concern, we can consider handle CONT_PTE in follow_page(). > > I think this is probably fine for the moment, at least for this > series, as CONT_PTE is still very new. > > But it will need to be optimized. "slow" GUP is the only GUP that is > used by FOLL_LONGTERM and it still needs to be optimized because you > can't assume a FOLL_LONGTERM user will be hitting the really slow > fault path. There are enough important cases where it is just reading > already populted page tables, and these days, often with large folios. Ah, I thought FOLL_LONGTERM should work in most cases for fast-gup, especially for hugetlb, but maybe I missed something? I do see that devmap skips fast-gup for LONGTERM, we also have that writeback issue but none of those that I can find applies to hugetlb. This might be a problem indeed if we have hugetlb cont_pte pages that will constantly fallback to slow gup. OTOH, I also agree with you that such batching would be nice to have for slow-gup, likely devmap or many fs (exclude shmem/hugetlb) file mappings can at least benefit from it due to above. But then that'll be a more generic issue to solve, IOW, we still don't do that for !hugetlb cont_pte large folios, before or after this series. > > Reviewed-by: Jason Gunthorpe Thanks! -- Peter Xu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2C678C54E71 for ; Fri, 22 Mar 2024 15:55:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ab/zZtKgIq4SajRvUXjJA+E2ob8+fGNTTNFCa13wlTc=; b=dMdNzL/mx+TL7T REN5xXGSnLFBOvd+pdLEjDESLxqObMi5+tOtGafnkb7jiLjMCN2aKP43T4blg89FaA0Nw0aQJd+7t d4FbytZCzYmHmD+0jcIufUtI9R+eOa6k9bFGp64f8CpAsP55BSB+uuhT25pHpM5CkUjT8jSHQbQfE icXn84LQ15E9DkmRXs/SOfSqntME/YvZO0IDTnaVfJBhoszLy3UJl5T6k9gWYd2pyIdjwF7T7A7kH JpPo54SPKR9Nwi4P/bcaVR4kLBWwXa9Oq6goZ8ooNZy8WJxaF8WtUWSqPD9SxWAIHR2Bad8EtkUYs 0JktQe91mh4+Xax4g3Hg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnhEp-00000007psp-0Phx; Fri, 22 Mar 2024 15:55:23 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnhEm-00000007prY-06MG for linux-arm-kernel@lists.infradead.org; Fri, 22 Mar 2024 15:55:21 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711122919; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QJLFb75dlh9Ilo5izUjxoeGL/8+am1RJvB8MBnP+8/Q=; b=Toy4bsv3Ad85EA+NvHhY+40Wru/FLAZfGe5Va6JZ2KGVavZpglqvuJLQ+1oTB7EtyfyHM2 ZydbIPdbAg0z6EHbiLi7IpXveKI5PQTW3tCfNf+/yoBiixUmpN3zvNLydlbKPjMaq58U0P fVP4gPz4L4phy7Szq+YaNPZgCJb93Rw= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-22-Xi_UVDE6MiipbAS-zMHoYw-1; Fri, 22 Mar 2024 11:55:16 -0400 X-MC-Unique: Xi_UVDE6MiipbAS-zMHoYw-1 Received: by mail-qv1-f72.google.com with SMTP id 6a1803df08f44-690d3f5af86so5121026d6.0 for ; Fri, 22 Mar 2024 08:55:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711122916; x=1711727716; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=QJLFb75dlh9Ilo5izUjxoeGL/8+am1RJvB8MBnP+8/Q=; b=W2Reiy3jgrKjpAMcMznBD9BaNZLK2vTFPn+JIVrn0ElhNWH+4HGKh59ZWci9D6b6Nu 6xbtxl81vuuC5uHAU2e67FqXqkXX6FrXxlbRLQvNsMpew327kxTzOjJW7THxXCsZlGif mV7dBwiiTI2mQW8yr+nx9aHv8PqDZKuSQrdNJx1MNN/cO6QHgJ600uIJFRaVe7+Kg8H+ MmwiZkY+7Duuz4ODS4SyJYocn0kdXdE/85l1mgqlxtgcMNwUseRBlKfip5T7t2tAh7Kh 0MvuAmFRk4PkrOIh1a3QqrDjOVD+sYmx7pejchy/CVh/L5ng53X+yEGFL7SvyjsapwLs b8LQ== X-Forwarded-Encrypted: i=1; AJvYcCVbFJecS6ENuuT1gXsACvym5/Z92gCd6Uj5RYNoeOxi7biKh3cW8sOOTZeUDVsZDD0Qumsyvt+xZN7dA/R9OMb71N6TCUMFC2hSMcTzJ2cWJMT0KK0= X-Gm-Message-State: AOJu0YxbIy6pa99ziVlme61f4elZPPgZfBa5j2nbxmT62hnxVY7pbIg8 7fVmCJExfKYpL0u3iNFpYuKyCiPXmNcFpLXFPx91ayUKENY2X66+csZPOXi8QngCCRGgcMGot09 7ZeWVnx4N33caMe1ME65PsMHleiTY/hA7jO/fJweC+75DSuvoUhfKx2bh44lp4e92oNePtiiM X-Received: by 2002:a05:6214:2b86:b0:696:732f:41ce with SMTP id kr6-20020a0562142b8600b00696732f41cemr1030609qvb.0.1711122916034; Fri, 22 Mar 2024 08:55:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGcGgYl+6vk8GOgJHrnHh9Bk553GXwTOWFwJfwdxmayaEVIZH35b8wrrTSaV+/rgNR3L0gYLQ== X-Received: by 2002:a05:6214:2b86:b0:696:732f:41ce with SMTP id kr6-20020a0562142b8600b00696732f41cemr1030577qvb.0.1711122915578; Fri, 22 Mar 2024 08:55:15 -0700 (PDT) Received: from x1n ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id jx12-20020a0562142b0c00b0068ff8bda6c7sm1185452qvb.92.2024.03.22.08.55.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Mar 2024 08:55:14 -0700 (PDT) Date: Fri, 22 Mar 2024 11:55:11 -0400 From: Peter Xu To: Jason Gunthorpe Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Michael Ellerman , Christophe Leroy , Matthew Wilcox , Rik van Riel , Lorenzo Stoakes , Axel Rasmussen , Yang Shi , John Hubbard , linux-arm-kernel@lists.infradead.org, "Kirill A . Shutemov" , Andrew Jones , Vlastimil Babka , Mike Rapoport , Andrew Morton , Muchun Song , Christoph Hellwig , linux-riscv@lists.infradead.org, James Houghton , David Hildenbrand , Andrea Arcangeli , "Aneesh Kumar K . V" , Mike Kravetz Subject: Re: [PATCH v3 12/12] mm/gup: Handle hugetlb in the generic follow_page_mask code Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240321220802.679544-13-peterx@redhat.com> <20240322133012.GI159172@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20240322133012.GI159172@nvidia.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240322_085520_331932_4CBC6407 X-CRM114-Status: GOOD ( 25.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Jason, On Fri, Mar 22, 2024 at 10:30:12AM -0300, Jason Gunthorpe wrote: > On Thu, Mar 21, 2024 at 06:08:02PM -0400, peterx@redhat.com wrote: > > > A quick performance test on an aarch64 VM on M1 chip shows 15% degrade over > > a tight loop of slow gup after the path switched. That shouldn't be a > > problem because slow-gup should not be a hot path for GUP in general: when > > page is commonly present, fast-gup will already succeed, while when the > > page is indeed missing and require a follow up page fault, the slow gup > > degrade will probably buried in the fault paths anyway. It also explains > > why slow gup for THP used to be very slow before 57edfcfd3419 ("mm/gup: > > accelerate thp gup even for "pages != NULL"") lands, the latter not part of > > a performance analysis but a side benefit. If the performance will be a > > concern, we can consider handle CONT_PTE in follow_page(). > > I think this is probably fine for the moment, at least for this > series, as CONT_PTE is still very new. > > But it will need to be optimized. "slow" GUP is the only GUP that is > used by FOLL_LONGTERM and it still needs to be optimized because you > can't assume a FOLL_LONGTERM user will be hitting the really slow > fault path. There are enough important cases where it is just reading > already populted page tables, and these days, often with large folios. Ah, I thought FOLL_LONGTERM should work in most cases for fast-gup, especially for hugetlb, but maybe I missed something? I do see that devmap skips fast-gup for LONGTERM, we also have that writeback issue but none of those that I can find applies to hugetlb. This might be a problem indeed if we have hugetlb cont_pte pages that will constantly fallback to slow gup. OTOH, I also agree with you that such batching would be nice to have for slow-gup, likely devmap or many fs (exclude shmem/hugetlb) file mappings can at least benefit from it due to above. But then that'll be a more generic issue to solve, IOW, we still don't do that for !hugetlb cont_pte large folios, before or after this series. > > Reviewed-by: Jason Gunthorpe Thanks! -- Peter Xu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5F11C54E71 for ; Fri, 22 Mar 2024 15:55:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2334F6B0089; Fri, 22 Mar 2024 11:55:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E3C06B008C; Fri, 22 Mar 2024 11:55:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0AB956B0095; Fri, 22 Mar 2024 11:55:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id ECC026B0089 for ; Fri, 22 Mar 2024 11:55:21 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A2B62C1178 for ; Fri, 22 Mar 2024 15:55:21 +0000 (UTC) X-FDA: 81925124442.30.887E9BB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 4491D1A0016 for ; Fri, 22 Mar 2024 15:55:19 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MY7ylr7p; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711122919; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QJLFb75dlh9Ilo5izUjxoeGL/8+am1RJvB8MBnP+8/Q=; b=62wvYNEdYuimE4M8EdXIEhGzF9LICrXZUKGcUGPbaDkPLgSZhapO///xmxb3M6emRVtTLZ xNicDjbZlwq3txKXhuaY9HruHDSBENPox6J8/tQDtgHse75fAOntIxCQtOcxmQSrjx2qys suKxiRJW1lEwWlOKx2RM/vmrvNgH/lI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711122919; a=rsa-sha256; cv=none; b=X5x4dM0IqIcoUGrVUv9I+LyxI9lnozE4yNMtVh/EilidHCCQDSMzw0OXiKgA0ZKQjEoAPI nFDZcK3nI4IVXOdrqaHGxUTpbyEmuGWKWVwVCTGIsZdYypMWkQgnbXTYLMlu5m81+OQ+HR 6hPsmuPmU13084IAM/N+BN/zqNj0jSE= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MY7ylr7p; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711122918; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QJLFb75dlh9Ilo5izUjxoeGL/8+am1RJvB8MBnP+8/Q=; b=MY7ylr7po+b1XrK0Q9tWQ8ggR80AH0obGLq6hNRTy/UGpQvSWjiodFdhJa5DsjbRt3q6Ni P3/Oh+qaqsHeySCSeM+DV0QpGqGARCDacTgm9JqxZNfmCBC075lwURV325Ql/Jsp7glUyn QjpxynzJzbb3If7VrUC8Ta0Z2X2pFBM= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-211-mWYzu-RdMWOwneCYx2RrRw-1; Fri, 22 Mar 2024 11:55:16 -0400 X-MC-Unique: mWYzu-RdMWOwneCYx2RrRw-1 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-69672754479so912346d6.1 for ; Fri, 22 Mar 2024 08:55:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711122916; x=1711727716; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=QJLFb75dlh9Ilo5izUjxoeGL/8+am1RJvB8MBnP+8/Q=; b=kBlLqxQsm/h9gtmicQDKb0MgYFhUiOeO1zCQbI1OWNtkI3npMTAb5mZ2X7TA7ACXrX tsO+7swB+F3Az+kHoTFQ0tuStgQdiXn0/+6I2OCVZFi9GgUQpGCE690+x0L3EyWMD4Sy B/6REVGujUd/UGLnd8+qL5WaTN38753coD5IQ0Vqiqsk7oan4l6pYDOYd9+mZjmVSkqq uOtrxxSRF+WOuVYmu6yr+0KpR8WvZW2qCpdgeraTa0MGAl/BHQazXEU6WSuc/Pqo/en8 aCR5Gsl1zjWfc27J0fBEzufilXdvUIrPA4/fjY+l6NAE9PBD4UjVC7WERaJkdcXFwYkJ +l+Q== X-Gm-Message-State: AOJu0YwkTxIKMoHO2W0K+qn+gD7zrbEMKVv4OuHQVv+62p+NQsQ0N6K7 3Cyhx83uEeNOmFS6mt/3UsqGht6jBQ+LlO6xVIwVUDZo3X1eN32I6XuYp5JdKtvM7zYlg+VDFiu GuZ5IfR0M2bjVQSm4DcC8vOZLqB902pJE+juuHbulf0EQP7qG X-Received: by 2002:a05:6214:2b86:b0:696:732f:41ce with SMTP id kr6-20020a0562142b8600b00696732f41cemr1030615qvb.0.1711122916043; Fri, 22 Mar 2024 08:55:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGcGgYl+6vk8GOgJHrnHh9Bk553GXwTOWFwJfwdxmayaEVIZH35b8wrrTSaV+/rgNR3L0gYLQ== X-Received: by 2002:a05:6214:2b86:b0:696:732f:41ce with SMTP id kr6-20020a0562142b8600b00696732f41cemr1030577qvb.0.1711122915578; Fri, 22 Mar 2024 08:55:15 -0700 (PDT) Received: from x1n ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id jx12-20020a0562142b0c00b0068ff8bda6c7sm1185452qvb.92.2024.03.22.08.55.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Mar 2024 08:55:14 -0700 (PDT) Date: Fri, 22 Mar 2024 11:55:11 -0400 From: Peter Xu To: Jason Gunthorpe Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Michael Ellerman , Christophe Leroy , Matthew Wilcox , Rik van Riel , Lorenzo Stoakes , Axel Rasmussen , Yang Shi , John Hubbard , linux-arm-kernel@lists.infradead.org, "Kirill A . Shutemov" , Andrew Jones , Vlastimil Babka , Mike Rapoport , Andrew Morton , Muchun Song , Christoph Hellwig , linux-riscv@lists.infradead.org, James Houghton , David Hildenbrand , Andrea Arcangeli , "Aneesh Kumar K . V" , Mike Kravetz Subject: Re: [PATCH v3 12/12] mm/gup: Handle hugetlb in the generic follow_page_mask code Message-ID: References: <20240321220802.679544-1-peterx@redhat.com> <20240321220802.679544-13-peterx@redhat.com> <20240322133012.GI159172@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20240322133012.GI159172@nvidia.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Rspamd-Queue-Id: 4491D1A0016 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: xki5mpgakbkwqfnym57j4mjgyqyjeykg X-HE-Tag: 1711122919-454561 X-HE-Meta: U2FsdGVkX1/rIszHhmxC3fEfT1s6tc01roZjqP3OoD+RYpMOYVm7hWbeuKzbRqb6KIfNHkzeYmAC1cFO0E72dFb/HcAhE5Fpa5eBoPldjC+RdvNFvk/K2pY2d3eRFvMVs5Bug3kH+wdP+4onCf1jEAyAnjREmNyA5jEODUH1JmOF7Or11YtPASQXCD3vycFB8rMud68SdBckWhvaSpdx9hWJlGreXYOm9XpFzxg+AZ+qqCpb7XGlgGCrn9GZP4CWps9zZ6D2lMIH1Ec5McYNnUHv+fJnQDTjEArumM4d+J5BrQaL6+Lp7UYbaDXB4EIx6briK1lmnNRzMwwhstxCieuTHZ8IM5G1AV3xoTFQGNboDwW6uWXrYd6ymbt5C+1NvD5S62PWFbYbB+KeK0ucvaYsS6DJt1X1nfFujQomwpPeRWWv9CHA1COAh9GdHOaAQAmUXd+jWPwS4MkIF40iIfKUZQ4nY7mRvQXwwS0mcQMBYE+WAVj5vvjLkqP+liPhGc9fi4/s/SoeDbBhD8LLIk/if2RSYOTHYcpCZrGwsx//mCtjbJ9Fa/3q0EMfB/HpR0Baim4zTQvcm24L3EFiuxV3twHLn+lXg1zB7w37nO92JHhDN7/TObm/WQEp6vlmTy+/Boo2GIWqsJ9GC86r+aMjv7NfooQ7TGGQTx9vXwiMQKBqltv1CsS6wKa20CxtC1q6HUitbKA/ma4LhQVgJE8mBh0c8xVXVrLmM9nM1G5LoNSZH1cuKTfPq1DB1XgaatKXr9O5irygUw43YKMyk1UH86kuZiHgc7bMsYWK1AB7BVb4HfaBFsg0PiJIqtn+de6Oln/VYSZJeumU02BgVk0YFoBrS+4HYwt9xNnUO+kOjkNIR9Ym0naTIXLM27tufV97uYE/X9nWGwEQef2cPcLgIogw/7MMMMu4RA/ZgfdPZ0wRlgZ4BBu+7Kr0DEyU8ebyT9Fsx2db84iN0cc +5kn/2A4 wdToawSgH7PH5pL9QiUuDk+zCZImE8gq39sSQl8rG3nmYNkN9M4QPGmEg0Bix5hskmm7oEta1/8wn04JLFLyzi/F8iIc9sC6t2/0wXCFJa0lUoRvHuIzlE+ZarDOyKhD4Tj94bRDgG7thhSoR/QhK55Jyw3QdQGx7o3F7bSRfEvd3aIuu/AekTzRoOgac2BzUtWVWbBvn+fbgW/ES6vaF2lrKmkwpYALZmjNp2FjDK0JHG5Ujp2flKYK/fMRaV6bbLmcSFt31VbUptDzfUPwD5QopVPNieUku95+PywdUNzhIxVfNAfSZ9YlJILfIt/YX5qek2R61O7sN2/SsdpxsqpWLqhbKP+uEEMIsSnrB3a75TyAHIgarzhqSLNcqCtcRmf6VJhIDLUOJajR2ISRbBDuWq/wYt+Ve/FbWCRuGIsqXqxpxFk09zok6cFEMWtSiiPNpntykbMo48hY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Jason, On Fri, Mar 22, 2024 at 10:30:12AM -0300, Jason Gunthorpe wrote: > On Thu, Mar 21, 2024 at 06:08:02PM -0400, peterx@redhat.com wrote: > > > A quick performance test on an aarch64 VM on M1 chip shows 15% degrade over > > a tight loop of slow gup after the path switched. That shouldn't be a > > problem because slow-gup should not be a hot path for GUP in general: when > > page is commonly present, fast-gup will already succeed, while when the > > page is indeed missing and require a follow up page fault, the slow gup > > degrade will probably buried in the fault paths anyway. It also explains > > why slow gup for THP used to be very slow before 57edfcfd3419 ("mm/gup: > > accelerate thp gup even for "pages != NULL"") lands, the latter not part of > > a performance analysis but a side benefit. If the performance will be a > > concern, we can consider handle CONT_PTE in follow_page(). > > I think this is probably fine for the moment, at least for this > series, as CONT_PTE is still very new. > > But it will need to be optimized. "slow" GUP is the only GUP that is > used by FOLL_LONGTERM and it still needs to be optimized because you > can't assume a FOLL_LONGTERM user will be hitting the really slow > fault path. There are enough important cases where it is just reading > already populted page tables, and these days, often with large folios. Ah, I thought FOLL_LONGTERM should work in most cases for fast-gup, especially for hugetlb, but maybe I missed something? I do see that devmap skips fast-gup for LONGTERM, we also have that writeback issue but none of those that I can find applies to hugetlb. This might be a problem indeed if we have hugetlb cont_pte pages that will constantly fallback to slow gup. OTOH, I also agree with you that such batching would be nice to have for slow-gup, likely devmap or many fs (exclude shmem/hugetlb) file mappings can at least benefit from it due to above. But then that'll be a more generic issue to solve, IOW, we still don't do that for !hugetlb cont_pte large folios, before or after this series. > > Reviewed-by: Jason Gunthorpe Thanks! -- Peter Xu