From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51B52C2D0A3 for ; Wed, 4 Nov 2020 03:05:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0CF012236F for ; Wed, 4 Nov 2020 03:05:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729889AbgKDDFF (ORCPT ); Tue, 3 Nov 2020 22:05:05 -0500 Received: from vps0.lunn.ch ([185.16.172.187]:33924 "EHLO vps0.lunn.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728972AbgKDDFE (ORCPT ); Tue, 3 Nov 2020 22:05:04 -0500 Received: from andrew by vps0.lunn.ch with local (Exim 4.94) (envelope-from ) id 1ka96c-005915-En; Wed, 04 Nov 2020 04:05:02 +0100 Date: Wed, 4 Nov 2020 04:05:02 +0100 From: Andrew Lunn To: Vladimir Oltean Cc: netdev@vger.kernel.org Subject: Re: DSA and ptp_classify_raw: saving some CPU cycles causes worse throughput? Message-ID: <20201104030502.GT933237@lunn.ch> References: <20201104015834.mcn2eoibxf6j3ksw@skbuf> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201104015834.mcn2eoibxf6j3ksw@skbuf> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org > My untrained eye tells me that in the 'after patch' case (the worse > one), there are less branch misses, and less cache misses. So by all > perf metrics, the throughput should be better, but it isn't. What gives? Maybe the frame has been pushed out of the L1 cache. The classify code is pulling it back in. It suffers some cache misses to get what it needs, but also in the background some speculative cache loads also happen, which are 'free'. By the time the DSA tagger is called, which also needs the header in the frame, it is all in L1 and the taggers work is fast. Without the classify, the tagger is getting a cold cache. And it ends up waiting around longer since it cannot benefit from the speculative 'free' loads? In your little patch, rather than a plain return, try calling prefetch() on the skb data so it might be warm by the time the tagger needs to manipulate it. Andrew