From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0119DC282CE for ; Tue, 12 Feb 2019 15:16:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C71CF217D9 for ; Tue, 12 Feb 2019 15:16:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="panJKHgc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729886AbfBLPP7 (ORCPT ); Tue, 12 Feb 2019 10:15:59 -0500 Received: from mail-yb1-f193.google.com ([209.85.219.193]:42221 "EHLO mail-yb1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728700AbfBLPP7 (ORCPT ); Tue, 12 Feb 2019 10:15:59 -0500 Received: by mail-yb1-f193.google.com with SMTP id j189so1146724ybj.9 for ; Tue, 12 Feb 2019 07:15:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=RqtMyuSranbSJ6NlUU6s0xosq9CQZC03joN1XI2VuCQ=; b=panJKHgcgOBhqJJPlBiMOZuPpZJzMk8Bngw9oHg3r6OueyqqakoAeebYaci/3edrMh 8fQmo7xAyAIBUQ80E4+CTSWybvrJ+Wj6CY4Zkt/46N0udIcNoAKcXILLmvD4iFafPJMr 7d6tF6I1ZvUKr463+PPLUhYrmCD0GmbH2YLoSSEd08Iqgx+bmBFxpAFVrt1J4nz9XHC6 LMhTqwX8tEb/btFO2mNwXEBD7cyb/gHpnUIsQ4/ggpS5u+Vj8HqPm5vhEHm+0JtJCvru yHHFxJBJ1Ho9gLKvPfpIVflOZi1pv+HA1BTRowQWeHb72rNLCkH301R854Hsbnubl9C6 6+/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=RqtMyuSranbSJ6NlUU6s0xosq9CQZC03joN1XI2VuCQ=; b=QsPjKsIh2dCkShWgsCX7biSqoUkNJhKGxm7tl5AIR2NMaN6n3G+FPKy+/CFya7tZhW KCB1X3EApNh2fISW0/v4Pvrp6zFh+/ANBFEg0gpnuMMZkICp/7n5k+s4X9cuD2cz489L kY+g0zSgXcr5G4wpjGvgl5Gd397LDlxv8r1N9w41gWHAGhEuYVICB24I0sI6KMhyc27C OgM2MP7VJMu/447cNW4/5tUltYpoOgTRO2dwuiCB8EsojHlLJFGbllq4/TAmPnA+PTeP viAE/c36Ttlg4VxCq/k8m0g2RZxbFvr9LjR5SkIwSNsHcwp4/Dl26EUadnvEV/+5pi+S BNTg== X-Gm-Message-State: AHQUAuZ+yX19BN6kXlgBsJ9P+C40KLvcdD6+jMfij/0L7aUXpLjdASVR BJ6uUrp6x4uptWsN0p1GwK0= X-Google-Smtp-Source: AHgI3IZfH0OCDGgp4potPiLlLuyWCzWBKkni/gD2JL9ai09luPJas2if4VY50FKbOZzF44s523/yhQ== X-Received: by 2002:a25:b98d:: with SMTP id r13mr3384874ybg.400.1549984558622; Tue, 12 Feb 2019 07:15:58 -0800 (PST) Received: from [192.168.86.235] (c-73-241-150-70.hsd1.ca.comcast.net. [73.241.150.70]) by smtp.gmail.com with ESMTPSA id g127sm4940451ywf.38.2019.02.12.07.15.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Feb 2019 07:15:57 -0800 (PST) Subject: Re: [RFC, PATCH] net: page_pool: Don't use page->private to store dma_addr_t To: Tariq Toukan , Eric Dumazet , Ilias Apalodimas , Matthew Wilcox , "brouer@redhat.com" Cc: David Miller , "toke@redhat.com" , "netdev@vger.kernel.org" , "mgorman@techsingularity.net" , "linux-mm@kvack.org" References: <1549550196-25581-1-git-send-email-ilias.apalodimas@linaro.org> <20190207150745.GW21860@bombadil.infradead.org> <20190207152034.GA3295@apalos> <20190207.132519.1698007650891404763.davem@davemloft.net> <20190207213400.GA21860@bombadil.infradead.org> <20190207214237.GA10676@Iliass-MBP.lan> <64f7af75-e6df-7abc-c4ce-82e6ca51fafe@gmail.com> <27e97aac-f25b-d46c-3e70-7d0d44f784b5@mellanox.com> From: Eric Dumazet Message-ID: Date: Tue, 12 Feb 2019 07:15:55 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <27e97aac-f25b-d46c-3e70-7d0d44f784b5@mellanox.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 02/12/2019 04:39 AM, Tariq Toukan wrote: > > > On 2/11/2019 7:14 PM, Eric Dumazet wrote: >> >> >> On 02/11/2019 12:53 AM, Tariq Toukan wrote: >>> >> >>> Hi, >>> >>> It's great to use the struct page to store its dma mapping, but I am >>> worried about extensibility. >>> page_pool is evolving, and it would need several more per-page fields. >>> One of them would be pageref_bias, a planned optimization to reduce the >>> number of the costly atomic pageref operations (and replace existing >>> code in several drivers). >>> >> >> But the point about pageref_bias is to place it in a different cache line than "struct page" >> >> The major cost is having a cache line bouncing between producer and consumer. >> > > pageref_bias is meant to be dirtied only by the page requester, i.e. the > NIC driver / page_pool. > All other components (basically, SKB release flow / put_page) should > continue working with the atomic page_refcnt, and not dirty the > pageref_bias. This is exactly my point. You suggested to put pageref_bias in struct page, which breaks this completely. pageref_bias is better kept in a driver structure, with appropriate prefetching since most NIC use a ring buffer for their queues. The dma address _can_ be put in the struct page, since the driver does not dirty it and does not even read it when page can be recycled.