DPDK-dev Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Bruce Richardson <bruce.richardson@intel.com>
To: Jerin Jacob <jerinjacobk@gmail.com>
Cc: Pravin Pathak <pravinpathakdev@gmail.com>,
	<mattias.ronnblom@ericcson.com>,
	"pravin.pathak.dev@gmail.com" <pravin.pathak.dev@gmail.com>,
	"Sarkar, Tirthendu" <tirthendu.sarkar@intel.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	"jerinj@marvell.com" <jerinj@marvell.com>,
	"mike.ximing.chen@intel.com" <mike.ximing.chen@intel.com>,
	"thomas@monjalon.net" <thomas@monjalon.net>,
	"david.marchand@redhat.com" <david.marchand@redhat.com>,
	"nipun.gupta@amd.com" <nipun.gupta@amd.com>,
	"chenbox@nvidia.com" <chenbox@nvidia.com>,
	<mattias.ronnblom@ericsson.com>
Subject: Re: [PATCH v1 0/8] event/slb: introducing software load balancer eventdev PMD
Date: Tue, 17 Nov 2026 17:39:04 +0000	[thread overview]
Message-ID: <avyROPrxuK4BPY56@bricha3-mobl1.ger.corp.intel.com> (raw)
In-Reply-To: <CALBAE1M22ZcTG0ofhE1-_G6_SoukoHRGc03vP3Fm55AA122axg@mail.gmail.com>

On Tue, Nov 17, 2026 at 10:55:12PM +0530, Jerin Jacob wrote:
> On Tue, Nov 17, 2026 at 10:39 PM Pravin Pathak
> <pravinpathakdev@gmail.com> wrote:
> >
> > On Wed, Nov 11, 2026 at 9:22 AM Sarkar, Tirthendu
> > <tirthendu.sarkar@intel.com> wrote:
> > >
> > > On Mon, Nov 9, 2026 at 6:25 PM Jerin Jacob <jerinjacobk@gmail.com> wrote:
> > > > Lets get the performance  numbers for SLB vs DSW based on same test
> > > > case so  that we can decide.
> > > > Performance and scalability will be the aspect to decide the usage of
> > > > YET ANOTHER SW driver.
> > >
> > > Hi Pravin,
> > >
> > > A few points on top of Stephen's review, plus some early numbers since
> > > Jerin asked for SLB vs DSW data.
> > >
> > > +1 to Stephen's two Errors in patch 2 (reorder slot reuse when RELEASE
> > > returns credits before the slot drains, and the lost wakeup in
> > > slb_reorder_drain()) and to the credit-threshold issue in patch 1.
> > > Those look like the ones that can lose or strand events.
> > >
> > > Additional comments:
> > >
> > > 1. slb_event.c, slb_event_enqueue_burst_generic(): the return value of
> > >    rte_event_ring_enqueue_burst() is ignored on every path and the
> > >    function always returns events_len. Correctness then depends
> > >    entirely on credits guaranteeing ring space. If that ever fails
> > >    (e.g. the credit-threshold or reorder issues above, or application
> > >    misuse), events are silently dropped and their credits leak. At
> > >    minimum, check the return value and report a short enqueue.
> > >    Because the burst is sorted by destination queue before enqueue,
> > >    a clean prefix count isn't possible after a partial failure, so
> > >    this probably needs a space check up front or an explicit design
> > >    statement that rings can never fill.
> > >
> > > 2. RTE_VERIFY(num_forward + num_release <= cur_pending) in the
> > >    enqueue fast path aborts the whole process on application misuse
> > >    (forwarding more events than were dequeued on that port). Please
> > >    fail the enqueue and set rte_errno instead.
> > >
> > > 3. slb_configure(): max_inflight = RTE_MAX(nb_events_limit,
> > >    nb_ports * SLB_PORT_MAX_CREDITS), so the device can hold more
> > >    events than the application configured via nb_events_limit. If
> > >    that is intended (to avoid credit-cache deadlock), please document
> > >    it in slb.rst.
> > >
> > > 4. slb_queue_setup() ignores queue_conf->priority; priority comes only
> > >    from the link priority, yet dev_info reports
> > >    max_event_queue_priority_levels = 2. Either honour queue priority or
> > >    report 1 and document that priority is per link.
> > >
> > > Performance
> > > -----------
> > > Preliminary data from dpdk-test-eventdev perf_queue, series applied on
> > > v26.11-rc2, release build, Xeon 6505P (2x12C), physical cores only,
> > > turbo on, default device settings, worker_deq_depth 16, 1024 flows.
> > > Mpps, 1 synthetic producer, workers on socket 0:
> > >
> > >   stlist   driver   1w     2w     4w     8w
> > >   p          slb       79.7   71.5   44.5   34.0
> > >   p          dsw     50.1   46.3   45.1   41.9
> > >   a          slb       54.3   65.3   57.0   43.9
> > >   a          dsw     50.3   47.3   46.7   40.5
> > >   a,a,a    slb      20.8    16.9   22.9   26.7
> > >   a,a,a    dsw    20.8    18.4   20.1   18.2
> > >   o          slb      61.7    29.5   36.6   27.4
> > >   o          sw        7.7    10.0   11.7   13.7
> > >   o,a       slb      36.0    19.5   28.3   28.9
> > >   o,a       sw       3.8       5.7     7.5      7.2
> > >
> > > (event_dsw does not support ordered queues, so ordered rows are
> > > against event_sw.)
> > >
> > > Observations:
> > > - SLB is ahead at low worker counts but loses throughput faster as
> > >   workers are added on single-stage parallel/atomic queues, which is
> > >   consistent with all workers contending on one MP/MC ring per queue.
> > > - Ordered queues are well ahead of event_sw (the only other SW PMD with
> > >   ordered support), but SLB ordered throughput halves going from 1 to
> > >   2 workers (61.7 -> 29.5 Mpps, reproduced), so the "close to
> > >   parallel" claim only holds for a single worker (parallel: 79.7 ->
> > >   71.5). This looks like the serialized reorder drain.
> > > - With workers spanning both sockets (16/20 workers) all drivers drop
> > >   sharply; SLB falls to 5-10 Mpps vs DSW 3-14 depending on stlist.
> > > - Note that with multiple producers perf_queue creates a separate
> > >   queue set per producer, which splits SLB's per-queue contention; SLB
> > >   scales very well in that mode (4 producers, 8 workers, atomic:
> > >   ~235 Mpps vs ~158 for DSW at worker_deq_depth 16), but that does not
> > >   reflect a pipeline where all traffic shares one atomic queue.
> > > - order_queue passes with up to 16 workers. In that test SLB slows down
> > >   as workers are added: with 8 workers it sustained ~1.0 M events/s vs
> > >   ~2.5 M events/s for event_sw (steady-state rate from the progress
> > >   counter, 20M events); with 1-2 workers SLB was faster than event_sw
> > >   (rough end-to-end timings, ~3.2 vs ~1.6-2.1 M events/s).
> > >
> > > These are untuned runs. In the 1-producer table the SLB/DSW atomic and
> > > a,a,a rows, DSW parallel and SLB ordered were run twice and agreed
> > > within ~5%; the other rows are single runs. Treat them as indicative.
> > > It would help if v2 included your own perf_queue/pipeline numbers with
> > > the exact command lines, ideally including a shared-queue case and
> > > workers on more than one socket.
> > >
> > > Thanks,
> > > Tirthendu
> >
> > Thanks Tirtha for the readings and comments. The main intention of introducing
> > SLB is to bridge the current gap where there is no distributed
> > software eventdev
> > supporting ordered queues and priorities. Current event_sw has a limitation of
> > single core scheduling limit. You can see it clearly from below
> > performance tests
> > where event_sw saturates early.
> 
> 
> In that case (And looking at the performance numbers), I am OK to have
> this new driver.  @mattias.ronnblom@ericcson.com  @Richardson, Bruce
> any objections.
> 
> @pravin.pathak.dev@gmail.com  If there is any common code between
> Event SW PMDs THEN please think about have some place for common code
> at driver/common/.../
> 
No objections from me. Is this something that could completely replace the
event/sw driver in future, once it has stabilized?

/Bruce

  reply	other threads:[~2026-11-17 17:39 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-07  1:30 [PATCH v1 0/8] event/slb: introducing software load balancer eventdev PMD Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 1/8] event/slb: introduce software load balancer PMD Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 2/8] event/slb: add support for ordered queues Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 3/8] event/slb: add support for atomic queues Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 4/8] event/slb: add dynamic flow migration " Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 5/8] event/slb: add extended statistics and diagnostics Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 6/8] event/slb: add self-test and documentation Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 7/8] event/slb: update resets and credit defaults Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 8/8] event/slb: improve port linking, config checks and atomic refill Pravin Pathak
2026-11-17 17:57   ` [PATCH v2 0/8] event/slb: introducing software load balancer eventdev PMD Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 1/8] event/slb: introduce software load balancer PMD Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 2/8] event/slb: add support for ordered queues Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 3/8] event/slb: add support for atomic queues Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 4/8] event/slb: add dynamic flow migration " Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 5/8] event/slb: add extended statistics and diagnostics Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 6/8] event/slb: add self-test and documentation Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 7/8] event/slb: fine tuning and fixes Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 8/8] event/slb: address ARM64 build and all code reviews Pravin Pathak
2026-11-17 18:46     ` [PATCH v2 0/8] event/slb: introducing software load balancer eventdev PMD Stephen Hemminger
2026-11-07  7:29 ` [PATCH v1 " Jerin Jacob
2026-11-08 17:10   ` Stephen Hemminger
2026-11-09  2:30   ` Pravin Pathak
2026-11-09 12:55     ` Jerin Jacob
2026-11-11 14:22       ` Sarkar, Tirthendu
2026-11-17 17:09         ` Pravin Pathak
2026-11-17 17:25           ` Jerin Jacob
2026-11-17 17:39             ` Bruce Richardson [this message]
2026-11-17 17:55               ` Pravin Pathak

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=avyROPrxuK4BPY56@bricha3-mobl1.ger.corp.intel.com \
    --to=bruce.richardson@intel.com \
    --cc=chenbox@nvidia.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=jerinj@marvell.com \
    --cc=jerinjacobk@gmail.com \
    --cc=mattias.ronnblom@ericcson.com \
    --cc=mattias.ronnblom@ericsson.com \
    --cc=mike.ximing.chen@intel.com \
    --cc=nipun.gupta@amd.com \
    --cc=pravin.pathak.dev@gmail.com \
    --cc=pravinpathakdev@gmail.com \
    --cc=thomas@monjalon.net \
    --cc=tirthendu.sarkar@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox