From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4F5CEE6446 for ; Fri, 15 Sep 2023 08:28:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 492386B0202; Fri, 15 Sep 2023 04:28:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4434E6B0203; Fri, 15 Sep 2023 04:28:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E3576B0204; Fri, 15 Sep 2023 04:28:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 19BA76B0202 for ; Fri, 15 Sep 2023 04:28:49 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D6F37140F18 for ; Fri, 15 Sep 2023 08:28:48 +0000 (UTC) X-FDA: 81238155936.29.311DD7F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf09.hostedemail.com (Postfix) with ESMTP id 8EE6E140007 for ; Fri, 15 Sep 2023 08:28:46 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IYrcxiSU; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of jbrouer@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=jbrouer@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694766526; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AcGHkHEkl2F+HrrCmBnGlpMxClskGfx3G9ZgUgAvCEo=; b=fzWMvs0ocCrm3XTs9+/hltr0XQuB54FWLAGRyemgPlXg3YBdZswRrnAorRN+6Jue42PzF8 Iap6ICMtU7VilfqJKLMmpXp1qfYqpMeGvjkYEUtsn6TUxeu9wmucKgOSH94QU2fUZKmA8T xbT5SCPnEOsBfUkBSHSpGanX9isuB9E= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IYrcxiSU; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of jbrouer@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=jbrouer@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694766526; a=rsa-sha256; cv=none; b=upY2vK0QxNPeHho0nfJKsQpCtNtA/A5aitRRUgM2Z+rKxFlpk+uD6whx84g6rjI/Z1cCXf bWRyh5XsgUNGOZ99r0OaWF/C28xwRiZgwIfp0m+IC8ouj/ThxZUbm4rlL8ApS5CRJkgnwK plAopgQ1lsjxDcKuf6QHQW/cgGC5opA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694766525; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AcGHkHEkl2F+HrrCmBnGlpMxClskGfx3G9ZgUgAvCEo=; b=IYrcxiSUPT4slav0QS1PxHTK2eAbqe62iSu+mjTk2BEwgCB3nti83wKPNoNtUCKh+GRDeW /lrQ/fvJ/G0U0quP0z5Pf6umBS2SbjWI724WRfRKkRWaRuJKj2SZbgPxM90OtU37x8/tLz MMED9Dhn9vK3/2oCgTPqV25thxNjCBw= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-584-_ZIGuDtPMtq739j9-xY_wQ-1; Fri, 15 Sep 2023 04:28:44 -0400 X-MC-Unique: _ZIGuDtPMtq739j9-xY_wQ-1 Received: by mail-lj1-f200.google.com with SMTP id 38308e7fff4ca-2bfbdcd728cso23135111fa.2 for ; Fri, 15 Sep 2023 01:28:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694766523; x=1695371323; h=content-transfer-encoding:in-reply-to:content-language:references :to:subject:cc:user-agent:mime-version:date:message-id:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AcGHkHEkl2F+HrrCmBnGlpMxClskGfx3G9ZgUgAvCEo=; b=H/L20pPWhSiJ7aJYauZ+RTmbCnNo3K7D3KOa853xWT9IUmv1XFqSpeJgzTd8g2uYK6 wdQP//wpKcUHtjML2x7NIfJWhO5cRRja7o1taDyc0wGQptVryt235bl3a14y7bRnpanO FZbpzQPUkxaotYRHZyYEFjDAkejUgGmrax9eOxGibPvtWowjvBYcg70WTKtMH+WR5CFf zGHDst73GtX/s9gLiHsB7SK/4Z2eT7kL/kL0R3Q7B0KJspfh40GHImdYZDpGQucGOkDd 1yL9qL1oQmKSPRinVQJxLeYFdxfYIvbyWrHtET5AqYmVyjvEKZUO+lTjXcnUdodA+fKm dZLQ== X-Gm-Message-State: AOJu0YynYAaWwPfQEelec2XLbsmNDt8fgHvwbOHkftX66jjsrbkyzM4Y HWoQy9qffd5hWG6mW61q+O1gK02rCYuJbhf9TvG8Ivea3lRcEpAIBzxrVVDJ6hYYBVPvLW10ZcR dkcrVxTrZ5kE= X-Received: by 2002:ac2:5e68:0:b0:502:a56b:65f7 with SMTP id a8-20020ac25e68000000b00502a56b65f7mr809376lfr.16.1694766522966; Fri, 15 Sep 2023 01:28:42 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEXe9WijGF+xd6sNUS8Btxyd7ZjXZH/hd49roTE/EzbjLuB9jzTKiTfy80O32sXsGz2tMVrcw== X-Received: by 2002:ac2:5e68:0:b0:502:a56b:65f7 with SMTP id a8-20020ac25e68000000b00502a56b65f7mr809354lfr.16.1694766522621; Fri, 15 Sep 2023 01:28:42 -0700 (PDT) Received: from [192.168.41.200] (83-90-141-187-cable.dk.customer.tdc.net. [83.90.141.187]) by smtp.gmail.com with ESMTPSA id c27-20020ac244bb000000b004f86d3e52c0sm557925lfm.4.2023.09.15.01.28.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 15 Sep 2023 01:28:41 -0700 (PDT) From: Jesper Dangaard Brouer X-Google-Original-From: Jesper Dangaard Brouer Message-ID: <84282e55-519c-0e17-30c5-b6de54d1001c@redhat.com> Date: Fri, 15 Sep 2023 10:28:40 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: brouer@redhat.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Lorenzo Bianconi , Alexander Duyck , Liang Chen , Alexander Lobakin , Guillaume Tucker , Jesper Dangaard Brouer , Ilias Apalodimas , Eric Dumazet , Linux-MM , Matthew Wilcox , Mel Gorman Subject: Re: [PATCH net-next v8 1/6] page_pool: frag API support for 32-bit arch with 64-bit DMA To: Yunsheng Lin , davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com References: <20230912083126.65484-1-linyunsheng@huawei.com> <20230912083126.65484-2-linyunsheng@huawei.com> In-Reply-To: <20230912083126.65484-2-linyunsheng@huawei.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspam-User: X-Stat-Signature: 45xtgraqnhr8366hb1ztzbyh1ptoetut X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8EE6E140007 X-HE-Tag: 1694766526-230348 X-HE-Meta: U2FsdGVkX19Khz/uXeZURI26TMHD3uKmweJ3CA5cJIMGMq/lrjMZ7QOFbgRypgWxGFa+8CiNAnEQquH/C4lrOzVgBNWeyL2C6Zf2UR8yS/gX9/SJ/+Rk8DNcZgX3f/hSDE5E3q20iBOBg5YA9uxeaJW3VXYhA9KKN0SsPZF/UNNy31lcJmWGGkRGCzjh/VBBaMq316xk8o2pAOeDnNGWEdwy25pcxKWs8r5ev5M1wNc2hNVDieGbobuFedYuEmZCE/6LU51vvLgWoMOeO+ZaANOEXRkEaGcwXuWARQB/GUa7posUcRhjRh0cMelfAfncU52geJOVEhc8jRomSGsZAaSFo5qc8FpCex4UMq9AKM2yxTC161gqTHU54bX8utjuBFXC9K+niM5dbNknftD+6wuxe1fTGU+Ei3e1ufNfjIEpeqkPdt+oJ9cx0IR3K2S8AOjDuRm3gt3UGN896RU3bEvJUfvjVcW00IhjXLMaaa0y0ApzjDmaS42pFBbO0LmbY07LgbI3hCiEqmMkAFhYAoSVBigP7sCsLKZyHwYAEe5hi3o7eqVTKwi7HiNt5jKLvS6W/q/7DQcXEKzbVYb1seGdWtgPHa3rB1XrE26w0Fy2SaJ/qAIRsksXdPCFi/WtZRJj7lz9VgYVaF4ayZNrM+y9gOXbP+xOspZCc4n+eiFSgB5TTl9FYg0rw9kLsFb6+MC221EpzYbfqI//NnySKXW1phbUC5qoRSeUu9FCslpaX+zZEPCGQv/hyzZcqMy7MvXswoOh0zb8vPp2UqSY3Rv55TJL9Q0AVoSB3RIHyKoQ4kP5MRT4mWDnaGVmSfVDrO9lFn1D/5j2PO9IYzGeXX3kARouOOCJM8zuOCm0JOwJTyZMzuGRAcve0zs3fR9lgp4M+CghNd/eGMG9UgUSdSEYdUGk1FdLWXv8Y4uSoM98FTX6vjxd+4C6gkmnLwzduTRiUVrJxcXHABgLwoB q1z3h63N Iv1KYv7GbZ5iscT+2nK8+qduRZQdTppOfC9XQs7n4QnrAr/1co7y0U8FdBSz9BoKMhkLByjppdp4F+MCu6N56iw9Vz9/SARIzCiYSQ1Xp80AJnHrLdmbwe3R6yr/rR+GoVCFC0PNDAEAHKT63ZEEg2O4clEpcChtWdi+xGNDAqJTOyJK8cHYlDQsAhGkGCEwxPCAODM1PrZoHepaXqt4okriYzrncYeevhV18olZE9HjTrFJ4UQsFRGxGB0ooay7ktwZ4xWz4YktgNejsQIXCHfm1DWb2WFBYIddcVwBMp6PhccZ163s8Wp2ZsNxaUS1fLrOzF38pJKw6gBXUGkHT3j+cwVS8OvHM23SZdYImh6KVK2NlwzjMqJPZQxuVOJkjZ8q/af633bdnTIhJr2ZkTcyXf3i9u7Fza4fs38FiVf7ZljlNRD23jOnlWzcL+5gmtJ92Cj6GaDWfRR6fdXMLLdRyOCDc8fQI/vtD6elvLLFcB3l1nrASC+0BblwxQ+nTb0IlX1hY/pSagw5OV1gh6oupmdtKNA9yhIqX2J4FsoYNR9orl1MiiCd17YFs7SBslmCPETajAaZw+ubQVGyO4jQqhLPo75vbeI6iIBkPyl32iPKM8yyIVMlB4c7B1Ko8HrxklAKvxOHY0jh7ydwL4s8hVqfSn1wxNvoI0N4MUGLqedBmR1rRPGOKR+/fZzaX5dP5U5VW4Qy/Vtk/YHUi9kwBSuyHtVYGrhtD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Lin, This looks reasonable, but given you are changing struct-page (include/linux/mm_types.h) we need to MM-list . Also Cc Wilcox. I think it was Ilias and Duyck that validated the assumptions, last time this patch was discussed. Thus I want to see their review before this is applied. -Jesper On 12/09/2023 10.31, Yunsheng Lin wrote: > Currently page_pool_alloc_frag() is not supported in 32-bit > arch with 64-bit DMA because of the overlap issue between > pp_frag_count and dma_addr_upper in 'struct page' for those > arches, which seems to be quite common, see [1], which means > driver may need to handle it when using frag API. > > It is assumed that the combination of the above arch with an > address space >16TB does not exist, as all those arches have > 64b equivalent, it seems logical to use the 64b version for a > system with a large address space. It is also assumed that dma > address is page aligned when we are dma mapping a page aliged > buffer, see [2]. > > That means we're storing 12 bits of 0 at the lower end for a > dma address, we can reuse those bits for the above arches to > support 32b+12b, which is 16TB of memory. > > If we make a wrong assumption, a warning is emitted so that > user can report to us. > > 1. https://lore.kernel.org/all/20211117075652.58299-1-linyunsheng@huawei.com/ > 2. https://lore.kernel.org/all/20230818145145.4b357c89@kernel.org/ > > Signed-off-by: Jakub Kicinski > Signed-off-by: Yunsheng Lin > CC: Lorenzo Bianconi > CC: Alexander Duyck > CC: Liang Chen > CC: Alexander Lobakin > CC: Guillaume Tucker > --- > include/linux/mm_types.h | 13 +------------ > include/net/page_pool/helpers.h | 20 ++++++++++++++------ > net/core/page_pool.c | 14 +++++++++----- > 3 files changed, 24 insertions(+), 23 deletions(-) > > diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h > index 36c5b43999e6..74b49c4c7a52 100644 > --- a/include/linux/mm_types.h > +++ b/include/linux/mm_types.h > @@ -125,18 +125,7 @@ struct page { > struct page_pool *pp; > unsigned long _pp_mapping_pad; > unsigned long dma_addr; > - union { > - /** > - * dma_addr_upper: might require a 64-bit > - * value on 32-bit architectures. > - */ > - unsigned long dma_addr_upper; > - /** > - * For frag page support, not supported in > - * 32-bit architectures with 64-bit DMA. > - */ > - atomic_long_t pp_frag_count; > - }; > + atomic_long_t pp_frag_count; > }; > struct { /* Tail pages of compound page */ > unsigned long compound_head; /* Bit zero is set */ > diff --git a/include/net/page_pool/helpers.h b/include/net/page_pool/helpers.h > index 94231533a369..8e1c85de4995 100644 > --- a/include/net/page_pool/helpers.h > +++ b/include/net/page_pool/helpers.h > @@ -197,7 +197,7 @@ static inline void page_pool_recycle_direct(struct page_pool *pool, > page_pool_put_full_page(pool, page, true); > } > > -#define PAGE_POOL_DMA_USE_PP_FRAG_COUNT \ > +#define PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA \ > (sizeof(dma_addr_t) > sizeof(unsigned long)) > > /** > @@ -211,17 +211,25 @@ static inline dma_addr_t page_pool_get_dma_addr(struct page *page) > { > dma_addr_t ret = page->dma_addr; > > - if (PAGE_POOL_DMA_USE_PP_FRAG_COUNT) > - ret |= (dma_addr_t)page->dma_addr_upper << 16 << 16; > + if (PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA) > + ret <<= PAGE_SHIFT; > > return ret; > } > > -static inline void page_pool_set_dma_addr(struct page *page, dma_addr_t addr) > +static inline bool page_pool_set_dma_addr(struct page *page, dma_addr_t addr) > { > + if (PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA) { > + page->dma_addr = addr >> PAGE_SHIFT; > + > + /* We assume page alignment to shave off bottom bits, > + * if this "compression" doesn't work we need to drop. > + */ > + return addr != (dma_addr_t)page->dma_addr << PAGE_SHIFT; > + } > + > page->dma_addr = addr; > - if (PAGE_POOL_DMA_USE_PP_FRAG_COUNT) > - page->dma_addr_upper = upper_32_bits(addr); > + return false; > } > > static inline bool page_pool_put(struct page_pool *pool) > diff --git a/net/core/page_pool.c b/net/core/page_pool.c > index 77cb75e63aca..8a9868ea5067 100644 > --- a/net/core/page_pool.c > +++ b/net/core/page_pool.c > @@ -211,10 +211,6 @@ static int page_pool_init(struct page_pool *pool, > */ > } > > - if (PAGE_POOL_DMA_USE_PP_FRAG_COUNT && > - pool->p.flags & PP_FLAG_PAGE_FRAG) > - return -EINVAL; > - > #ifdef CONFIG_PAGE_POOL_STATS > pool->recycle_stats = alloc_percpu(struct page_pool_recycle_stats); > if (!pool->recycle_stats) > @@ -359,12 +355,20 @@ static bool page_pool_dma_map(struct page_pool *pool, struct page *page) > if (dma_mapping_error(pool->p.dev, dma)) > return false; > > - page_pool_set_dma_addr(page, dma); > + if (page_pool_set_dma_addr(page, dma)) > + goto unmap_failed; > > if (pool->p.flags & PP_FLAG_DMA_SYNC_DEV) > page_pool_dma_sync_for_device(pool, page, pool->p.max_len); > > return true; > + > +unmap_failed: > + WARN_ON_ONCE("unexpected DMA address, please report to netdev@"); > + dma_unmap_page_attrs(pool->p.dev, dma, > + PAGE_SIZE << pool->p.order, pool->p.dma_dir, > + DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_WEAK_ORDERING); > + return false; > } > > static void page_pool_set_pp_info(struct page_pool *pool,