From: Jesper Dangaard Brouer <brouer@redhat.com>
To: "Björn Töpel" <bjorn.topel@gmail.com>
Cc: bpf <bpf@vger.kernel.org>,
brouer@redhat.com, LKML <linux-kernel@vger.kernel.org>,
Tejun Heo <tj@kernel.org>, Christoph Lameter <cl@linux.com>,
Dennis Zhou <dennis@kernel.org>
Subject: Re: Percpu variables, benchmarking, and performance weirdness
Date: Fri, 20 Dec 2019 10:34:20 +0100 [thread overview]
Message-ID: <20191220103420.6f9304ab@carbon> (raw)
In-Reply-To: <CAJ+HfNgNAzvdBw7gBJTCDQsne-HnWm90H50zNvXBSp4izbwFTA@mail.gmail.com>
On Fri, 20 Dec 2019 09:25:43 +0100
Björn Töpel <bjorn.topel@gmail.com> wrote:
> I've been doing some benchmarking with AF_XDP, and more specific the
> bpf_xdp_redirect_map() helper and xdp_do_redirect(). One thing that
> puzzles me is that the percpu-variable accesses stands out.
>
> I did a horrible hack that just accesses a regular global variable,
> instead of the percpu struct bpf_redirect_info, and got a performance
> boost from 22.7 Mpps to 23.8 Mpps with the rxdrop scenario from
> xdpsock.
Yes, this an 2 ns overhead, which is annoying in XDP context.
(1/22.7-1/23.8)*1000 = 2 ns
> Have anyone else seen this?
Yes, I see it all the time...
> So, my question to the uarch/percpu folks out there: Why are percpu
> accesses (%gs segment register) more expensive than regular global
> variables in this scenario.
I'm also VERY interested in knowing the answer to above question!?
(Adding LKML to reach more people)
> One way around that is changing BPF_PROG_RUN, and BPF_CALL_x to pass a
> context (struct bpf_redirect_info) explicitly, and access that instead
> of doing percpu access. That would be a pretty churny patch, and
> before doing that it would be nice to understand why percpu stands out
> performance-wise.
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
next parent reply other threads:[~2019-12-20 9:34 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CAJ+HfNgNAzvdBw7gBJTCDQsne-HnWm90H50zNvXBSp4izbwFTA@mail.gmail.com>
2019-12-20 9:34 ` Jesper Dangaard Brouer [this message]
2019-12-20 15:12 ` Percpu variables, benchmarking, and performance weirdness Tejun Heo
2019-12-20 15:36 ` Christopher Lameter
2019-12-20 17:10 ` Dennis Zhou
2019-12-20 16:22 ` Eric Dumazet
2019-12-20 16:34 ` Tejun Heo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191220103420.6f9304ab@carbon \
--to=brouer@redhat.com \
--cc=bjorn.topel@gmail.com \
--cc=bpf@vger.kernel.org \
--cc=cl@linux.com \
--cc=dennis@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox