From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1D8DC64EC4 for ; Fri, 3 Mar 2023 10:32:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230140AbjCCKc2 (ORCPT ); Fri, 3 Mar 2023 05:32:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230073AbjCCKc0 (ORCPT ); Fri, 3 Mar 2023 05:32:26 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D4AA1CF77 for ; Fri, 3 Mar 2023 02:31:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677839499; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7hFKulmrqZbTf9q6Sh0sTv9iLMhiQyVri/7U3fZos58=; b=XpeJBgJI5B4RYCIYnzi+nnFoTjWZkVIzuRTeH0vXkSvKIm5tL+Kue/C8powXR3OsXFHtAw jD/SWZd5atbh1czv8g0QhRjOMynqc/93Z/RuEp2uh2JuTxbmTid+qjbCrrGLSI5iii8Kre Rw1nmg2S/ak1nhTadxBy6vOXaJHNhjM= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-413-mRd9mHWGPACFAx0-BDGfyw-1; Fri, 03 Mar 2023 05:31:38 -0500 X-MC-Unique: mRd9mHWGPACFAx0-BDGfyw-1 Received: by mail-ed1-f69.google.com with SMTP id ck7-20020a0564021c0700b004a25d8d7593so3447668edb.0 for ; Fri, 03 Mar 2023 02:31:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:references:to :content-language:subject:cc:user-agent:mime-version:date:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7hFKulmrqZbTf9q6Sh0sTv9iLMhiQyVri/7U3fZos58=; b=Ul/5B0bYENmN/Pn2cVRB0OJo+xkoKUl6/Bqgi+zbcT21FWTL2/I9/D++GJGtVQDrkF WdaNKqVcwdn3djyVBl7/+cYCwE1mnmuVST6fal/rqLogKyAXRxv6FKgXaGx22tHqvdbm BDhuYNhOuDDIxeYd8ZTXehbbxRq8E0sOYw/1tW6J9iUx9yC+gpwOqPAqyCmrwA8KtApb QSP+oP7LSG68FHUK0XhbPoXJL/Plmgu6rTkKtJpbP16Dib/8KbYegcOO8/Rj5yglwnxi K97jEAiVNHM3Puibr4WuWVn0ybmYcm6HosnvGlGcmjPzsoPH6KqYBNanbAYghEfQWcAC yZIw== X-Gm-Message-State: AO0yUKVL83vaZ8Gk4wNyqIt76oO3iSx7Y8jGuAq5Pp8QsTXkVXF5cYnp HK7ntB2uldp0RXIC0fOEPsQ+YTx5HE3c1FJwxLoWuV7Qwp6pzigljZGEback/DHde+vNjayVCua JQWk8gA9ApwIb X-Received: by 2002:a17:906:a882:b0:8aa:a802:adcd with SMTP id ha2-20020a170906a88200b008aaa802adcdmr1126695ejb.30.1677839497577; Fri, 03 Mar 2023 02:31:37 -0800 (PST) X-Google-Smtp-Source: AK7set+dNeTha+dpDazmX7Y6KMFZSZy5QW/4mxFqD4WQL+q6eFTVdNlEwM6IxZz2Z/PPEAHrK/Ks/w== X-Received: by 2002:a17:906:a882:b0:8aa:a802:adcd with SMTP id ha2-20020a170906a88200b008aaa802adcdmr1126668ejb.30.1677839497257; Fri, 03 Mar 2023 02:31:37 -0800 (PST) Received: from [192.168.42.100] (nat-cgn9-185-107-15-52.static.kviknet.net. [185.107.15.52]) by smtp.gmail.com with ESMTPSA id ci25-20020a170906c35900b008b23e619960sm803620ejb.139.2023.03.03.02.31.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 03 Mar 2023 02:31:36 -0800 (PST) From: Jesper Dangaard Brouer X-Google-Original-From: Jesper Dangaard Brouer Message-ID: Date: Fri, 3 Mar 2023 11:31:35 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Cc: brouer@redhat.com, Maciej Fijalkowski , Larysa Zaremba , =?UTF-8?Q?Toke_H=c3=b8iland-J=c3=b8rgensen?= , Song Liu , Jesper Dangaard Brouer , Jakub Kicinski , bpf@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH bpf-next v1 1/2] xdp: recycle Page Pool backed skbs built from XDP frames Content-Language: en-US To: Yunsheng Lin , Alexander Lobakin , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau References: <20230301160315.1022488-1-aleksander.lobakin@intel.com> <20230301160315.1022488-2-aleksander.lobakin@intel.com> <36d42e20-b33f-5442-0db7-e9f5ef9d0941@huawei.com> In-Reply-To: <36d42e20-b33f-5442-0db7-e9f5ef9d0941@huawei.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On 02/03/2023 03.30, Yunsheng Lin wrote: > On 2023/3/2 0:03, Alexander Lobakin wrote: >> __xdp_build_skb_from_frame() state(d): >> >> /* Until page_pool get SKB return path, release DMA here */ >> >> Page Pool got skb pages recycling in April 2021, but missed this >> function. >> >> xdp_release_frame() is relevant only for Page Pool backed frames and it >> detaches the page from the corresponding Pool in order to make it >> freeable via page_frag_free(). It can instead just mark the output skb >> as eligible for recycling if the frame is backed by a PP. No change for >> other memory model types (the same condition check as before). >> cpumap redirect and veth on Page Pool drivers now become zero-alloc (or >> almost). >> >> Signed-off-by: Alexander Lobakin >> --- >> net/core/xdp.c | 4 ++-- >> 1 file changed, 2 insertions(+), 2 deletions(-) >> >> diff --git a/net/core/xdp.c b/net/core/xdp.c >> index 8c92fc553317..a2237cfca8e9 100644 >> --- a/net/core/xdp.c >> +++ b/net/core/xdp.c >> @@ -658,8 +658,8 @@ struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf, >> * - RX ring dev queue index (skb_record_rx_queue) >> */ >> >> - /* Until page_pool get SKB return path, release DMA here */ >> - xdp_release_frame(xdpf); >> + if (xdpf->mem.type == MEM_TYPE_PAGE_POOL) >> + skb_mark_for_recycle(skb); > > > We both rely on both skb->pp_recycle and page->pp_magic to decide > the page is really from page pool. So there was a few corner case > problem when we are sharing a page for different skb in the driver > level or calling skb_clone() or skb_try_coalesce(). > see: > https://github.com/torvalds/linux/commit/2cc3aeb5ecccec0d266813172fcd82b4b5fa5803 > https://lore.kernel.org/netdev/MW5PR15MB51214C0513DB08A3607FBC1FBDE19@MW5PR15MB5121.namprd15.prod.outlook.com/t/ > https://lore.kernel.org/netdev/167475990764.1934330.11960904198087757911.stgit@localhost.localdomain/ > > As the 'struct xdp_frame' also use 'struct skb_shared_info' which is > sharable, see xdp_get_shared_info_from_frame(). > > For now xdpf_clone() does not seems to handling frag page yet, > so it should be fine for now. > > IMHO we should find a way to use per-page marker, instead of both > per-skb and per-page markers, in order to avoid the above problem > for xdp if xdp has a similar processing as skb, as suggested by Eric. > Moving to a per-page marker can be *more* expensive if the struct-page memory isn't cache-hot. So, if struct-page is accessed anyhow then sure we can move it to a per-page marker. > https://lore.kernel.org/netdev/CANn89iKgZU4Q+THXupzZi4hETuKuCOvOB=iHpp5JzQTNv_Fg_A@mail.gmail.com/ > >> >> /* Allow SKB to reuse area used by xdp_frame */ >> xdp_scrub_frame(xdpf); >> >