From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B9B150291 for ; Tue, 23 Apr 2024 07:56:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713859001; cv=none; b=NdLvUFfFutNp7/zbocGHfZEOJ/g4wYZYyOslDsdeBI8hNMjzKx/8Zw0rKRTlWR0J0vzboJ+evrZm7E3esgcQhqKrVfjzHKA8uACmmj+P7l67SF9alT55fjw0QIX9Pvwute51lG59zXIXbOxT6HWDiszd6i7YAGWfUcqQ3fH4RI4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713859001; c=relaxed/simple; bh=1TlwN3xC88ayoJNrLD/eruVrZt0lEHJ+l9v4qt0so9s=; h=Message-ID:Subject:From:To:Date:In-Reply-To:References: Content-Type:MIME-Version; b=sYx7sBqt5I4OA5rDwjjd09FHqbt5AumS3M9yIHFXZ/b16jreHczSWGyzY95eZknW+d50rOf9LnETOuAjA/tNVpN/a1qJSjKld4gec40HKUeug9LegsSj+3OmSeWQ/ZGRULglzKlKM55yssbq2V8aGV9OfsVv7iiICSLaNjStEZ0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=DCpgKnms; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DCpgKnms" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713858999; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=1TlwN3xC88ayoJNrLD/eruVrZt0lEHJ+l9v4qt0so9s=; b=DCpgKnmscXEMFrR5mN4sljd5/QhaBFfjDVqTHpobI7dRH9dHjR389PsqIXN7+krcxCurZj 8Z7smPvcwi4HSIQFLSlNt2hupwGG8NJlMQtQn4G5VaSPaaDI6dZ+8Kgx8LEhNOolH0qJSU JlHVPKI4H0bZQWapSUjzHmbaBnAExSI= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-150-NAMgenYmMHeL6Oq9fd31sw-1; Tue, 23 Apr 2024 03:56:37 -0400 X-MC-Unique: NAMgenYmMHeL6Oq9fd31sw-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-4188cfbabe3so6566875e9.3 for ; Tue, 23 Apr 2024 00:56:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713858996; x=1714463796; h=mime-version:user-agent:content-transfer-encoding:autocrypt :references:in-reply-to:date:to:from:subject:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1TlwN3xC88ayoJNrLD/eruVrZt0lEHJ+l9v4qt0so9s=; b=anVKetmJa30xkoKvXPeFvls3A0HOcHBWvJWBfg3B3aK+RWxVZovuNGilXkEnOyz0Yu aD9cvurLcTnzv0A+enlnlvyfG8Myg1wenE5u4GDYMI8Kv9W7iiEITmyM8/CSmZAQBL4o KpFULLa7DxfG+T5ca+g+y3Yw1Ah8Jv/hI1f8B3S5KCJW9if9WG0GYgPAfkjVqAUtopMe gqIVWT0kGprJFVza2g+C5x2TswxD7ezBd9cYP4CNWxAaqVF99gj86r/8y+sx9oPjxqAX rtT5MVjlrwthlgsfYN7RHpBY+9zYAGgUWbN4BzKplx6QYJKnLElzI1qxIAuEwGpAHV0d q1xQ== X-Forwarded-Encrypted: i=1; AJvYcCUjcXdrAXBwuyJGD6C6Is7iGacp6XclfMnLuFhuf1gahsmge/iRj1mQYFh2PnvytU1dLWAAcLxS3DYb0QLkyW/ixbAUvqyn X-Gm-Message-State: AOJu0YzbFMWGmtOD3RunnwUhENxKu0850hpn4efbTJ6EaDkfrPLTgYL7 fLx0j+OJmQtxUee9Ma/oV7+Zu73MwTDA7UycKJR7mxu+4Kv8fjCbW+cNtqn2qPIVPpf2KsHQa6l hjlQya+fBK7JUnJ6uCO5cR2CtD0UhH3znC5vU9VQ4ApcoZxH/Ujkw0fHGtNCm4Q== X-Received: by 2002:a05:600c:1c9c:b0:418:ef65:4b5f with SMTP id k28-20020a05600c1c9c00b00418ef654b5fmr8515113wms.3.1713858995822; Tue, 23 Apr 2024 00:56:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGtvdpUVvfIE8HC9Y+s3Tq+/zaxNSctKzEy8pHTqosbTtivobUYHvfaAeLs5EJT7EP/75KXhA== X-Received: by 2002:a05:600c:1c9c:b0:418:ef65:4b5f with SMTP id k28-20020a05600c1c9c00b00418ef654b5fmr8515102wms.3.1713858995416; Tue, 23 Apr 2024 00:56:35 -0700 (PDT) Received: from gerbillo.redhat.com ([2a0d:3344:172c:4510::f71]) by smtp.gmail.com with ESMTPSA id o14-20020a05600c510e00b00419f572671dsm10859790wms.20.2024.04.23.00.56.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Apr 2024 00:56:34 -0700 (PDT) Message-ID: Subject: Re: [RFC PATCH] net: skb: Increasing allocation in __napi_alloc_skb() to 2k when needed. From: Paolo Abeni To: David J Wilder , netdev@vger.kernel.org Date: Tue, 23 Apr 2024 09:56:33 +0200 In-Reply-To: <20240419222328.3231075-1-dwilder@us.ibm.com> References: <20240419222328.3231075-1-dwilder@us.ibm.com> Autocrypt: addr=pabeni@redhat.com; prefer-encrypt=mutual; keydata=mQINBGISiDUBEAC5uMdJicjm3ZlWQJG4u2EU1EhWUSx8IZLUTmEE8zmjPJFSYDcjtfGcbzLPb63BvX7FADmTOkO7gwtDgm501XnQaZgBUnCOUT8qv5MkKsFH20h1XJyqjPeGM55YFAXc+a4WD0YyO5M0+KhDeRLoildeRna1ey944VlZ6Inf67zMYw9vfE5XozBtytFIrRyGEWkQwkjaYhr1cGM8ia24QQVQid3P7SPkR78kJmrT32sGk+TdR4YnZzBvVaojX4AroZrrAQVdOLQWR+w4w1mONfJvahNdjq73tKv51nIpu4SAC1Zmnm3x4u9r22mbMDr0uWqDqwhsvkanYmn4umDKc1ZkBnDIbbumd40x9CKgG6ogVlLYeJa9WyfVMOHDF6f0wRjFjxVoPO6p/ZDkuEa67KCpJnXNYipLJ3MYhdKWBZw0xc3LKiKc+nMfQlo76T/qHMDfRMaMhk+L8gWc3ZlRQFG0/Pd1pdQEiRuvfM5DUXDo/YOZLV0NfRFU9SmtIPhbdm9cV8Hf8mUwubihiJB/9zPvVq8xfiVbdT0sPzBtxW0fXwrbFxYAOFvT0UC2MjlIsukjmXOUJtdZqBE3v3Jf7VnjNVj9P58+MOx9iYo8jl3fNd7biyQWdPDfYk9ncK8km4skfZQIoUVqrWqGDJjHO1W9CQLAxkfOeHrmG29PK9tHIwARAQABtB9QYW9sbyBBYmVuaSA8cGFiZW5pQHJlZGhhdC5jb20+iQJSBBMBCAA8FiEEg1AjqC77wbdLX2LbKSR5jcyPE6QFAmISiDUCGwMFCwkIBwIDIgIBBhUKCQgLAgQWAgMBAh4HAheAAAoJECkkeY3MjxOkJSYQAJcc6MTsuFxYdYZkeWjW//zbD3ApRHzpNlHLVSuJqHr9/aDS+tyszgS8jj9MiqALzgq4iZbg 7ZxN9ZsDL38qVIuFkSpgMZCiUHdxBC11J8nbBSLlpnc924UAyr5XrGA99 6Wl5I4Km3128GY6iAkH54pZpOmpoUyBjcxbJWHstzmvyiXrjA2sMzYjt3Xkqp0cJfIEekOi75wnNPofEEJg28XPcFrpkMUFFvB4Aqrdc2yyR8Y36rbw18sIX3dJdomIP3dL7LoJi9mfUKOnr86Z0xltgcLPGYoCiUZMlXyWgB2IPmmcMP2jLJrusICjZxLYJJLofEjznAJSUEwB/3rlvFrSYvkKkVmfnfro5XEr5nStVTECxfy7RTtltwih85LlZEHP8eJWMUDj3P4Q9CWNgz2pWr1t68QuPHWaA+PrXyasDlcRpRXHZCOcvsKhAaCOG8TzCrutOZ5NxdfXTe3f1jVIEab7lNgr+7HiNVS+UPRzmvBc73DAyToKQBn9kC4jh9HoWyYTepjdcxnio0crmara+/HEyRZDQeOzSexf85I4dwxcdPKXv0fmLtxrN57Ae82bHuRlfeTuDG3x3vl/Bjx4O7Lb+oN2BLTmgpYq7V1WJPUwikZg8M+nvDNcsOoWGbU417PbHHn3N7yS0lLGoCCWyrK1OY0QM4EVsL3TjOfUtCNQYW9sbyBBYmVuaSA8cGFvbG8uYWJlbmlAZ21haWwuY29tPokCUgQTAQgAPBYhBINQI6gu+8G3S19i2ykkeY3MjxOkBQJiEoitAhsDBQsJCAcCAyICAQYVCgkICwIEFgIDAQIeBwIXgAAKCRApJHmNzI8TpBzHD/45pUctaCnhee1vkQnmStAYvHmwrWwIEH1lzDMDCpJQHTUQOOJWDAZOFnE/67bxSS81Wie0OKW2jvg1ylmpBA0gPpnzIExQmfP72cQ1TBoeVColVT6Io35BINn+ymM7c0Bn8RvngSEpr3jBtqvvWXjvtnJ5/HbOVQCg62NC6ewosoKJPWpGXMJ9SKsVIOUHsmoWK60spzeiJoSmAwm3zTJQnM5kRh2q iWjoCy8L35zPqR5TV+f5WR5hTVCqmLHSgm1jxwKhPg9L+GfuE4d0SWd84y GeOB3sSxlhWsuTj1K6K3MO9srD9hr0puqjO9sAizd0BJP8ucf/AACfrgmzIqZXCfVS7jJ/M+0ic+j1Si3yY8wYPEi3dvbVC0zsoGj9n1R7B7L9c3g1pZ4L9ui428vnPiMnDN3jh9OsdaXeWLvSvTylYvw9q0DEXVQTv4/OkcoMrfEkfbXbtZ3PRlAiddSZA5BDEkkm6P9KA2YAuooi1OD9d4MW8LFAeEicvHG+TPO6jtKTacdXDRe611EfRwTjBs19HmabSUfFcumL6BlVyceIoSqXFe5jOfGpbBevTZtg4kTSHqymGb6ra6sKs+/9aJiONs5NXY7iacZ55qG3Ib1cpQTps9bQILnqpwL2VTaH9TPGWwMY3Nc2VEc08zsLrXnA/yZKqZ1YzSY9MGXWYLkCDQRiEog1ARAAyXMKL+x1lDvLZVQjSUIVlaWswc0nV5y2EzBdbdZZCP3ysGC+s+n7xtq0o1wOvSvaG9h5q7sYZs+AKbuUbeZPu0bPWKoO02i00yVoSgWnEqDbyNeiSW+vI+VdiXITV83lG6pS+pAoTZlRROkpb5xo0gQ5ZeYok8MrkEmJbsPjdoKUJDBFTwrRnaDOfb+Qx1D22PlAZpdKiNtwbNZWiwEQFm6mHkIVSTUe2zSemoqYX4QQRvbmuMyPIbwbdNWlItukjHsffuPivLF/XsI1gDV67S1cVnQbBgrpFDxN62USwewXkNl+ndwa+15wgJFyq4Sd+RSMTPDzDQPFovyDfA/jxN2SK1Lizam6o+LBmvhIxwZOfdYH8bdYCoSpqcKLJVG3qVcTwbhGJr3kpRcBRz39Ml6iZhJyI3pEoX3bJTlR5Pr1Kjpx13qGydSMos94CIYWAKhegI06aTdvvuiigBwjngo/Rk5S+iEGR5KmTqGyp27o6YxZy6D4NIc6PKUzhIUxfvuHNvfu sD2W1U7eyLdm/jCgticGDsRtweytsgCSYfbz0gdgUuL3EBYN3JLbAU+UZpy v/fyD4cHDWaizNy/KmOI6FFjvVh4LRCpGTGDVPHsQXaqvzUybaMb7HSfmBBzZqqfVbq9n5FqPjAgD2lJ0rkzb9XnVXHgr6bmMRlaTlBMAEQEAAYkCNgQYAQgAIBYhBINQI6gu+8G3S19i2ykkeY3MjxOkBQJiEog1AhsMAAoJECkkeY3MjxOkY1YQAKdGjHyIdOWSjM8DPLdGJaPgJdugHZowaoyCxffilMGXqc8axBtmYjUIoXurpl+f+a7S0tQhXjGUt09zKlNXxGcebL5TEPFqgJTHN/77ayLslMTtZVYHE2FiIxkvW48yDjZUlefmphGpfpoXe4nRBNto1mMB9Pb9vR47EjNBZCtWWbwJTIEUwHP2Z5fV9nMx9Zw2BhwrfnODnzI8xRWVqk7/5R+FJvl7s3nY4F+svKGD9QHYmxfd8Gx42PZc/qkeCjUORaOf1fsYyChTtJI4iNm6iWbD9HK5LTMzwl0n0lL7CEsBsCJ97i2swm1DQiY1ZJ95G2Nz5PjNRSiymIw9/neTvUT8VJJhzRl3Nb/EmO/qeahfiG7zTpqSn2dEl+AwbcwQrbAhTPzuHIcoLZYV0xDWzAibUnn7pSrQKja+b8kHD9WF+m7dPlRVY7soqEYXylyCOXr5516upH8vVBmqweCIxXSWqPAhQq8d3hB/Ww2A0H0PBTN1REVw8pRLNApEA7C2nX6RW0XmA53PIQvAP0EAakWsqHoKZ5WdpeOcH9iVlUQhRgemQSkhfNaP9LqR1XKujlTuUTpoyT3xwAzkmSxN1nABoutHEO/N87fpIbpbZaIdinF7b9srwUvDOKsywfs5HMiUZhLKoZzCcU/AEFjQsPTATACGsWf3JYPnWxL9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On Fri, 2024-04-19 at 15:23 -0700, David J Wilder wrote: > When testing CONFIG_MAX_SKB_FRAGS=3D45 on ppc64le and x86_64 I ran into a > couple of issues. >=20 > __napi_alloc_skb() assumes its smallest fragment allocations will fit in > 1K. When CONFIG_MAX_SKB_FRAGS is increased this may no longer be true > resulting in __napi_alloc_skb() reverting to using page_frag_alloc(). > This results in the return of the bug fixed in: > Commit 3226b158e67c ("net: avoid 32 x truesize under-estimation for > tiny skbs") >=20 > That commit insured that "small skb head fragments are kmalloc backed, > so that other objects in the slab page can be reused instead of being hel= d > as long as skbs are sitting in socket queues." >=20 > On ppc64le the warning from napi_get_frags_check() is displayed when > CONFIG_MAX_SKB_FRAGS is set to 45. The purpose of the warning is to detec= t > when an increase of MAX_SKB_FRAGS has reintroduced the aforementioned bug= . > Unfortunately on x86_64 this warning is not seen, even though it should b= e. > I found the warning was disabled by: > commit dbae2b062824 ("net: skb: introduce and use a single page frag > cache") >=20 > This RFC patch to __napi_alloc_skb() determines if an skbuff allocation > with a head fragment of size GRO_MAX_HEAD will fit in a 1k allocation, > increasing the allocation to 2k if needed. >=20 > I have functionally tested this patch, performance testing is still neede= d. >=20 > TBD: Remove the limitation on 4k page size from the single page frag cach= e > allowing ppc64le (64K page size) to benefit from this change. >=20 > TBD: I have not address the warning in napi_get_frags_check() on x86_64. > Will the warning still be needed once the other changes are completed? Thanks for the detailed analysis. As mentioned by Eric in commit bf9f1baa279f0758dc2297080360c5a616843927, it should be now possible to revert dbae2b062824 without incurring in performance regressions for the relevant use-case. I had that on my todo list since a lot of time, but I was unable to allocate time for that. I think such revert would be preferable. Would you be able to evaluate such option? Thanks! Paolo