From: Bruce Richardson <bruce.richardson@intel.com>
To: Jerin Jacob <jerinjacobk@gmail.com>
Cc: Pravin Pathak <pravinpathakdev@gmail.com>,
<mattias.ronnblom@ericcson.com>,
"pravin.pathak.dev@gmail.com" <pravin.pathak.dev@gmail.com>,
"Sarkar, Tirthendu" <tirthendu.sarkar@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"jerinj@marvell.com" <jerinj@marvell.com>,
"mike.ximing.chen@intel.com" <mike.ximing.chen@intel.com>,
"thomas@monjalon.net" <thomas@monjalon.net>,
"david.marchand@redhat.com" <david.marchand@redhat.com>,
"nipun.gupta@amd.com" <nipun.gupta@amd.com>,
"chenbox@nvidia.com" <chenbox@nvidia.com>,
<mattias.ronnblom@ericsson.com>
Subject: Re: [PATCH v1 0/8] event/slb: introducing software load balancer eventdev PMD
Date: Tue, 17 Nov 2026 17:39:04 +0000 [thread overview]
Message-ID: <avyROPrxuK4BPY56@bricha3-mobl1.ger.corp.intel.com> (raw)
In-Reply-To: <CALBAE1M22ZcTG0ofhE1-_G6_SoukoHRGc03vP3Fm55AA122axg@mail.gmail.com>
On Tue, Nov 17, 2026 at 10:55:12PM +0530, Jerin Jacob wrote:
> On Tue, Nov 17, 2026 at 10:39 PM Pravin Pathak
> <pravinpathakdev@gmail.com> wrote:
> >
> > On Wed, Nov 11, 2026 at 9:22 AM Sarkar, Tirthendu
> > <tirthendu.sarkar@intel.com> wrote:
> > >
> > > On Mon, Nov 9, 2026 at 6:25 PM Jerin Jacob <jerinjacobk@gmail.com> wrote:
> > > > Lets get the performance numbers for SLB vs DSW based on same test
> > > > case so that we can decide.
> > > > Performance and scalability will be the aspect to decide the usage of
> > > > YET ANOTHER SW driver.
> > >
> > > Hi Pravin,
> > >
> > > A few points on top of Stephen's review, plus some early numbers since
> > > Jerin asked for SLB vs DSW data.
> > >
> > > +1 to Stephen's two Errors in patch 2 (reorder slot reuse when RELEASE
> > > returns credits before the slot drains, and the lost wakeup in
> > > slb_reorder_drain()) and to the credit-threshold issue in patch 1.
> > > Those look like the ones that can lose or strand events.
> > >
> > > Additional comments:
> > >
> > > 1. slb_event.c, slb_event_enqueue_burst_generic(): the return value of
> > > rte_event_ring_enqueue_burst() is ignored on every path and the
> > > function always returns events_len. Correctness then depends
> > > entirely on credits guaranteeing ring space. If that ever fails
> > > (e.g. the credit-threshold or reorder issues above, or application
> > > misuse), events are silently dropped and their credits leak. At
> > > minimum, check the return value and report a short enqueue.
> > > Because the burst is sorted by destination queue before enqueue,
> > > a clean prefix count isn't possible after a partial failure, so
> > > this probably needs a space check up front or an explicit design
> > > statement that rings can never fill.
> > >
> > > 2. RTE_VERIFY(num_forward + num_release <= cur_pending) in the
> > > enqueue fast path aborts the whole process on application misuse
> > > (forwarding more events than were dequeued on that port). Please
> > > fail the enqueue and set rte_errno instead.
> > >
> > > 3. slb_configure(): max_inflight = RTE_MAX(nb_events_limit,
> > > nb_ports * SLB_PORT_MAX_CREDITS), so the device can hold more
> > > events than the application configured via nb_events_limit. If
> > > that is intended (to avoid credit-cache deadlock), please document
> > > it in slb.rst.
> > >
> > > 4. slb_queue_setup() ignores queue_conf->priority; priority comes only
> > > from the link priority, yet dev_info reports
> > > max_event_queue_priority_levels = 2. Either honour queue priority or
> > > report 1 and document that priority is per link.
> > >
> > > Performance
> > > -----------
> > > Preliminary data from dpdk-test-eventdev perf_queue, series applied on
> > > v26.11-rc2, release build, Xeon 6505P (2x12C), physical cores only,
> > > turbo on, default device settings, worker_deq_depth 16, 1024 flows.
> > > Mpps, 1 synthetic producer, workers on socket 0:
> > >
> > > stlist driver 1w 2w 4w 8w
> > > p slb 79.7 71.5 44.5 34.0
> > > p dsw 50.1 46.3 45.1 41.9
> > > a slb 54.3 65.3 57.0 43.9
> > > a dsw 50.3 47.3 46.7 40.5
> > > a,a,a slb 20.8 16.9 22.9 26.7
> > > a,a,a dsw 20.8 18.4 20.1 18.2
> > > o slb 61.7 29.5 36.6 27.4
> > > o sw 7.7 10.0 11.7 13.7
> > > o,a slb 36.0 19.5 28.3 28.9
> > > o,a sw 3.8 5.7 7.5 7.2
> > >
> > > (event_dsw does not support ordered queues, so ordered rows are
> > > against event_sw.)
> > >
> > > Observations:
> > > - SLB is ahead at low worker counts but loses throughput faster as
> > > workers are added on single-stage parallel/atomic queues, which is
> > > consistent with all workers contending on one MP/MC ring per queue.
> > > - Ordered queues are well ahead of event_sw (the only other SW PMD with
> > > ordered support), but SLB ordered throughput halves going from 1 to
> > > 2 workers (61.7 -> 29.5 Mpps, reproduced), so the "close to
> > > parallel" claim only holds for a single worker (parallel: 79.7 ->
> > > 71.5). This looks like the serialized reorder drain.
> > > - With workers spanning both sockets (16/20 workers) all drivers drop
> > > sharply; SLB falls to 5-10 Mpps vs DSW 3-14 depending on stlist.
> > > - Note that with multiple producers perf_queue creates a separate
> > > queue set per producer, which splits SLB's per-queue contention; SLB
> > > scales very well in that mode (4 producers, 8 workers, atomic:
> > > ~235 Mpps vs ~158 for DSW at worker_deq_depth 16), but that does not
> > > reflect a pipeline where all traffic shares one atomic queue.
> > > - order_queue passes with up to 16 workers. In that test SLB slows down
> > > as workers are added: with 8 workers it sustained ~1.0 M events/s vs
> > > ~2.5 M events/s for event_sw (steady-state rate from the progress
> > > counter, 20M events); with 1-2 workers SLB was faster than event_sw
> > > (rough end-to-end timings, ~3.2 vs ~1.6-2.1 M events/s).
> > >
> > > These are untuned runs. In the 1-producer table the SLB/DSW atomic and
> > > a,a,a rows, DSW parallel and SLB ordered were run twice and agreed
> > > within ~5%; the other rows are single runs. Treat them as indicative.
> > > It would help if v2 included your own perf_queue/pipeline numbers with
> > > the exact command lines, ideally including a shared-queue case and
> > > workers on more than one socket.
> > >
> > > Thanks,
> > > Tirthendu
> >
> > Thanks Tirtha for the readings and comments. The main intention of introducing
> > SLB is to bridge the current gap where there is no distributed
> > software eventdev
> > supporting ordered queues and priorities. Current event_sw has a limitation of
> > single core scheduling limit. You can see it clearly from below
> > performance tests
> > where event_sw saturates early.
>
>
> In that case (And looking at the performance numbers), I am OK to have
> this new driver. @mattias.ronnblom@ericcson.com @Richardson, Bruce
> any objections.
>
> @pravin.pathak.dev@gmail.com If there is any common code between
> Event SW PMDs THEN please think about have some place for common code
> at driver/common/.../
>
No objections from me. Is this something that could completely replace the
event/sw driver in future, once it has stabilized?
/Bruce
next prev parent reply other threads:[~2026-11-17 17:39 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-07 1:30 [PATCH v1 0/8] event/slb: introducing software load balancer eventdev PMD Pravin Pathak
2026-11-07 1:30 ` [PATCH v1 1/8] event/slb: introduce software load balancer PMD Pravin Pathak
2026-11-07 1:30 ` [PATCH v1 2/8] event/slb: add support for ordered queues Pravin Pathak
2026-11-07 1:30 ` [PATCH v1 3/8] event/slb: add support for atomic queues Pravin Pathak
2026-11-07 1:30 ` [PATCH v1 4/8] event/slb: add dynamic flow migration " Pravin Pathak
2026-11-07 1:30 ` [PATCH v1 5/8] event/slb: add extended statistics and diagnostics Pravin Pathak
2026-11-07 1:30 ` [PATCH v1 6/8] event/slb: add self-test and documentation Pravin Pathak
2026-11-07 1:30 ` [PATCH v1 7/8] event/slb: update resets and credit defaults Pravin Pathak
2026-11-07 1:30 ` [PATCH v1 8/8] event/slb: improve port linking, config checks and atomic refill Pravin Pathak
2026-11-17 17:57 ` [PATCH v2 0/8] event/slb: introducing software load balancer eventdev PMD Pravin Pathak
2026-11-17 17:57 ` [PATCH v2 1/8] event/slb: introduce software load balancer PMD Pravin Pathak
2026-11-17 17:57 ` [PATCH v2 2/8] event/slb: add support for ordered queues Pravin Pathak
2026-11-17 17:57 ` [PATCH v2 3/8] event/slb: add support for atomic queues Pravin Pathak
2026-11-17 17:57 ` [PATCH v2 4/8] event/slb: add dynamic flow migration " Pravin Pathak
2026-11-17 17:57 ` [PATCH v2 5/8] event/slb: add extended statistics and diagnostics Pravin Pathak
2026-11-17 17:57 ` [PATCH v2 6/8] event/slb: add self-test and documentation Pravin Pathak
2026-11-17 17:57 ` [PATCH v2 7/8] event/slb: fine tuning and fixes Pravin Pathak
2026-11-17 17:57 ` [PATCH v2 8/8] event/slb: address ARM64 build and all code reviews Pravin Pathak
2026-11-17 18:46 ` [PATCH v2 0/8] event/slb: introducing software load balancer eventdev PMD Stephen Hemminger
2026-11-07 7:29 ` [PATCH v1 " Jerin Jacob
2026-11-08 17:10 ` Stephen Hemminger
2026-11-09 2:30 ` Pravin Pathak
2026-11-09 12:55 ` Jerin Jacob
2026-11-11 14:22 ` Sarkar, Tirthendu
2026-11-17 17:09 ` Pravin Pathak
2026-11-17 17:25 ` Jerin Jacob
2026-11-17 17:39 ` Bruce Richardson [this message]
2026-11-17 17:55 ` Pravin Pathak
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=avyROPrxuK4BPY56@bricha3-mobl1.ger.corp.intel.com \
--to=bruce.richardson@intel.com \
--cc=chenbox@nvidia.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=jerinj@marvell.com \
--cc=jerinjacobk@gmail.com \
--cc=mattias.ronnblom@ericcson.com \
--cc=mattias.ronnblom@ericsson.com \
--cc=mike.ximing.chen@intel.com \
--cc=nipun.gupta@amd.com \
--cc=pravin.pathak.dev@gmail.com \
--cc=pravinpathakdev@gmail.com \
--cc=thomas@monjalon.net \
--cc=tirthendu.sarkar@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox