From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B824CCA479 for ; Mon, 20 Jun 2022 18:31:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6CD376B0071; Mon, 20 Jun 2022 14:31:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 67C196B0073; Mon, 20 Jun 2022 14:31:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56AB56B0074; Mon, 20 Jun 2022 14:31:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4976F6B0071 for ; Mon, 20 Jun 2022 14:31:49 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 110013018B for ; Mon, 20 Jun 2022 18:31:49 +0000 (UTC) X-FDA: 79599457938.28.22E9A75 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf28.hostedemail.com (Postfix) with ESMTP id 9AEAEC00B2 for ; Mon, 20 Jun 2022 18:31:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=ifeyTcl/AzgtaNQGujckbdAaI1u6Oc+W0eR30MKVMt0=; b=NeG2GYlsioxK7lRBX58s9G3eZ5 yRo26XRbGPTm2TRZ2ypskU8cAk6ATuCX9XeqIcL6q99nZISK4o4XKlgw3duF9qGzI4q//E4v9IyHM uI9GHFz1MTLikBAaaGAdcq+ppIHdEA40bVnCvB/PyU5mNHz7qzL67FSm+mkBhjnc7Ld677b3x+4iS QwDaEjsesc1GyR6RsQJ2TmelieatvC6ScmDqfRY3wkiGd2vDbnLpByP2enuIgEOUVaWUZxeP6soEN BKy2DScdJ4jusJezIR9VsYU15GAZlqJcX6Ym4UiN+Tiu3r3CK7euoC6F7tGBiQlaw/3yWiNh+mqi2 vrGmko9w==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1o3MBX-001rrj-Fc; Mon, 20 Jun 2022 18:31:39 +0000 Date: Mon, 20 Jun 2022 11:31:39 -0700 From: Luis Chamberlain To: Aaron Lu , Davidlohr Bueso Cc: Song Liu , linux-kernel@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, ast@kernel.org, daniel@iogearbox.net, peterz@infradead.org, torvalds@linux-foundation.org, rick.p.edgecombe@intel.com, kernel-team@fb.com Subject: Re: [PATCH v4 bpf-next 0/8] bpf_prog_pack followup Message-ID: References: <20220520235758.1858153-1-song@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655749908; a=rsa-sha256; cv=none; b=E7PzG0NM24UwzCZrwRHjHt2Qe+dAJzszBSed5NYGxFgyW6Fc9y6q0nWGW7gSCERfczOg1q rFnOS4xACaucNiBljfy70JYgKFjHEHStjOcqrcGuoHr/41n0VXPSAbngRuY+zFkdHhAYVH eCCr/6aQ1mGPLcHviqtN4mIavOAc8zY= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=NeG2GYls; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=none (imf28.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655749908; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ifeyTcl/AzgtaNQGujckbdAaI1u6Oc+W0eR30MKVMt0=; b=GDXYHQ7WAeGAX689mZk37SDZbRgZGKM+P2WKdh5T3XsxRPCAG+8LT5qfHwkuMQRp80RZNp iHde7tFzyzXFeGxyUCzy0tHZKLnsWK6R0alitCl36douujWxwi4nI7wC1etJH8F8MJEVBe axnojlOUXSaM6MfqK19lZsBt3pG5KmY= X-Rspamd-Queue-Id: 9AEAEC00B2 X-Rspam-User: Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=NeG2GYls; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=none (imf28.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org X-Rspamd-Server: rspam06 X-Stat-Signature: xdka4kquak88akxxtt6rawo5q9a9s9us X-HE-Tag: 1655749907-756222 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jun 20, 2022 at 07:11:45PM +0800, Aaron Lu wrote: > Hi Song, > > On Fri, May 20, 2022 at 04:57:50PM -0700, Song Liu wrote: > > ... ... > > > The primary goal of bpf_prog_pack is to reduce iTLB miss rate and reduce > > direct memory mapping fragmentation. This leads to non-trivial performance > > improvements. > > > > For our web service production benchmark, bpf_prog_pack on 4kB pages > > gives 0.5% to 0.7% more throughput than not using bpf_prog_pack. > > bpf_prog_pack on 2MB pages 0.6% to 0.9% more throughput than not using > > bpf_prog_pack. Note that 0.5% is a huge improvement for our fleet. I > > believe this is also significant for other companies with many thousand > > servers. > > > > I'm evaluationg performance impact due to direct memory mapping > fragmentation BTW how exactly are you doing this? Luis > and seeing the above, I wonder: is the performance improve > mostly due to prog pack and hugepage instead of less direct mapping > fragmentation? > > I can understand that when progs are packed together, iTLB miss rate will > be reduced and thus, performance can be improved. But I don't see > immediately how direct mapping fragmentation can impact performance since > the bpf code are running from the module alias addresses, not the direct > mapping addresses IIUC? > > I appreciate it if you can shed some light on performance impact direct > mapping fragmentation can cause, thanks.