All of lore.kernel.org
 help / color / mirror / Atom feed
From: Bruce Richardson <bruce.richardson@intel.com>
To: Jerin Jacob <jerinjacobk@gmail.com>
Cc: Pravin Pathak <pravinpathakdev@gmail.com>,
	<mattias.ronnblom@ericcson.com>,
	"pravin.pathak.dev@gmail.com" <pravin.pathak.dev@gmail.com>,
	"Sarkar, Tirthendu" <tirthendu.sarkar@intel.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	"jerinj@marvell.com" <jerinj@marvell.com>,
	"mike.ximing.chen@intel.com" <mike.ximing.chen@intel.com>,
	"thomas@monjalon.net" <thomas@monjalon.net>,
	"david.marchand@redhat.com" <david.marchand@redhat.com>,
	"nipun.gupta@amd.com" <nipun.gupta@amd.com>,
	"chenbox@nvidia.com" <chenbox@nvidia.com>,
	<mattias.ronnblom@ericsson.com>
Subject: Re: [PATCH v1 0/8] event/slb: introducing software load balancer eventdev PMD
Date: Tue, 17 Nov 2026 17:39:04 +0000	[thread overview]
Message-ID: <avyROPrxuK4BPY56@bricha3-mobl1.ger.corp.intel.com> (raw)
In-Reply-To: <CALBAE1M22ZcTG0ofhE1-_G6_SoukoHRGc03vP3Fm55AA122axg@mail.gmail.com>

On Tue, Nov 17, 2026 at 10:55:12PM +0530, Jerin Jacob wrote:
> On Tue, Nov 17, 2026 at 10:39 PM Pravin Pathak
> <pravinpathakdev@gmail.com> wrote:
> >
> > On Wed, Nov 11, 2026 at 9:22 AM Sarkar, Tirthendu
> > <tirthendu.sarkar@intel.com> wrote:
> > >
> > > On Mon, Nov 9, 2026 at 6:25 PM Jerin Jacob <jerinjacobk@gmail.com> wrote:
> > > > Lets get the performance  numbers for SLB vs DSW based on same test
> > > > case so  that we can decide.
> > > > Performance and scalability will be the aspect to decide the usage of
> > > > YET ANOTHER SW driver.
> > >
> > > Hi Pravin,
> > >
> > > A few points on top of Stephen's review, plus some early numbers since
> > > Jerin asked for SLB vs DSW data.
> > >
> > > +1 to Stephen's two Errors in patch 2 (reorder slot reuse when RELEASE
> > > returns credits before the slot drains, and the lost wakeup in
> > > slb_reorder_drain()) and to the credit-threshold issue in patch 1.
> > > Those look like the ones that can lose or strand events.
> > >
> > > Additional comments:
> > >
> > > 1. slb_event.c, slb_event_enqueue_burst_generic(): the return value of
> > >    rte_event_ring_enqueue_burst() is ignored on every path and the
> > >    function always returns events_len. Correctness then depends
> > >    entirely on credits guaranteeing ring space. If that ever fails
> > >    (e.g. the credit-threshold or reorder issues above, or application
> > >    misuse), events are silently dropped and their credits leak. At
> > >    minimum, check the return value and report a short enqueue.
> > >    Because the burst is sorted by destination queue before enqueue,
> > >    a clean prefix count isn't possible after a partial failure, so
> > >    this probably needs a space check up front or an explicit design
> > >    statement that rings can never fill.
> > >
> > > 2. RTE_VERIFY(num_forward + num_release <= cur_pending) in the
> > >    enqueue fast path aborts the whole process on application misuse
> > >    (forwarding more events than were dequeued on that port). Please
> > >    fail the enqueue and set rte_errno instead.
> > >
> > > 3. slb_configure(): max_inflight = RTE_MAX(nb_events_limit,
> > >    nb_ports * SLB_PORT_MAX_CREDITS), so the device can hold more
> > >    events than the application configured via nb_events_limit. If
> > >    that is intended (to avoid credit-cache deadlock), please document
> > >    it in slb.rst.
> > >
> > > 4. slb_queue_setup() ignores queue_conf->priority; priority comes only
> > >    from the link priority, yet dev_info reports
> > >    max_event_queue_priority_levels = 2. Either honour queue priority or
> > >    report 1 and document that priority is per link.
> > >
> > > Performance
> > > -----------
> > > Preliminary data from dpdk-test-eventdev perf_queue, series applied on
> > > v26.11-rc2, release build, Xeon 6505P (2x12C), physical cores only,
> > > turbo on, default device settings, worker_deq_depth 16, 1024 flows.
> > > Mpps, 1 synthetic producer, workers on socket 0:
> > >
> > >   stlist   driver   1w     2w     4w     8w
> > >   p          slb       79.7   71.5   44.5   34.0
> > >   p          dsw     50.1   46.3   45.1   41.9
> > >   a          slb       54.3   65.3   57.0   43.9
> > >   a          dsw     50.3   47.3   46.7   40.5
> > >   a,a,a    slb      20.8    16.9   22.9   26.7
> > >   a,a,a    dsw    20.8    18.4   20.1   18.2
> > >   o          slb      61.7    29.5   36.6   27.4
> > >   o          sw        7.7    10.0   11.7   13.7
> > >   o,a       slb      36.0    19.5   28.3   28.9
> > >   o,a       sw       3.8       5.7     7.5      7.2
> > >
> > > (event_dsw does not support ordered queues, so ordered rows are
> > > against event_sw.)
> > >
> > > Observations:
> > > - SLB is ahead at low worker counts but loses throughput faster as
> > >   workers are added on single-stage parallel/atomic queues, which is
> > >   consistent with all workers contending on one MP/MC ring per queue.
> > > - Ordered queues are well ahead of event_sw (the only other SW PMD with
> > >   ordered support), but SLB ordered throughput halves going from 1 to
> > >   2 workers (61.7 -> 29.5 Mpps, reproduced), so the "close to
> > >   parallel" claim only holds for a single worker (parallel: 79.7 ->
> > >   71.5). This looks like the serialized reorder drain.
> > > - With workers spanning both sockets (16/20 workers) all drivers drop
> > >   sharply; SLB falls to 5-10 Mpps vs DSW 3-14 depending on stlist.
> > > - Note that with multiple producers perf_queue creates a separate
> > >   queue set per producer, which splits SLB's per-queue contention; SLB
> > >   scales very well in that mode (4 producers, 8 workers, atomic:
> > >   ~235 Mpps vs ~158 for DSW at worker_deq_depth 16), but that does not
> > >   reflect a pipeline where all traffic shares one atomic queue.
> > > - order_queue passes with up to 16 workers. In that test SLB slows down
> > >   as workers are added: with 8 workers it sustained ~1.0 M events/s vs
> > >   ~2.5 M events/s for event_sw (steady-state rate from the progress
> > >   counter, 20M events); with 1-2 workers SLB was faster than event_sw
> > >   (rough end-to-end timings, ~3.2 vs ~1.6-2.1 M events/s).
> > >
> > > These are untuned runs. In the 1-producer table the SLB/DSW atomic and
> > > a,a,a rows, DSW parallel and SLB ordered were run twice and agreed
> > > within ~5%; the other rows are single runs. Treat them as indicative.
> > > It would help if v2 included your own perf_queue/pipeline numbers with
> > > the exact command lines, ideally including a shared-queue case and
> > > workers on more than one socket.
> > >
> > > Thanks,
> > > Tirthendu
> >
> > Thanks Tirtha for the readings and comments. The main intention of introducing
> > SLB is to bridge the current gap where there is no distributed
> > software eventdev
> > supporting ordered queues and priorities. Current event_sw has a limitation of
> > single core scheduling limit. You can see it clearly from below
> > performance tests
> > where event_sw saturates early.
> 
> 
> In that case (And looking at the performance numbers), I am OK to have
> this new driver.  @mattias.ronnblom@ericcson.com  @Richardson, Bruce
> any objections.
> 
> @pravin.pathak.dev@gmail.com  If there is any common code between
> Event SW PMDs THEN please think about have some place for common code
> at driver/common/.../
> 
No objections from me. Is this something that could completely replace the
event/sw driver in future, once it has stabilized?

/Bruce

  reply	other threads:[~2026-11-17 17:39 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-07  1:30 [PATCH v1 0/8] event/slb: introducing software load balancer eventdev PMD Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 1/8] event/slb: introduce software load balancer PMD Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 2/8] event/slb: add support for ordered queues Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 3/8] event/slb: add support for atomic queues Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 4/8] event/slb: add dynamic flow migration " Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 5/8] event/slb: add extended statistics and diagnostics Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 6/8] event/slb: add self-test and documentation Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 7/8] event/slb: update resets and credit defaults Pravin Pathak
2026-11-07  1:30 ` [PATCH v1 8/8] event/slb: improve port linking, config checks and atomic refill Pravin Pathak
2026-11-17 17:57   ` [PATCH v2 0/8] event/slb: introducing software load balancer eventdev PMD Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 1/8] event/slb: introduce software load balancer PMD Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 2/8] event/slb: add support for ordered queues Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 3/8] event/slb: add support for atomic queues Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 4/8] event/slb: add dynamic flow migration " Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 5/8] event/slb: add extended statistics and diagnostics Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 6/8] event/slb: add self-test and documentation Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 7/8] event/slb: fine tuning and fixes Pravin Pathak
2026-11-17 17:57     ` [PATCH v2 8/8] event/slb: address ARM64 build and all code reviews Pravin Pathak
2026-11-17 18:46     ` [PATCH v2 0/8] event/slb: introducing software load balancer eventdev PMD Stephen Hemminger
2026-11-07  7:29 ` [PATCH v1 " Jerin Jacob
2026-11-08 17:10   ` Stephen Hemminger
2026-11-09  2:30   ` Pravin Pathak
2026-11-09 12:55     ` Jerin Jacob
2026-11-11 14:22       ` Sarkar, Tirthendu
2026-11-17 17:09         ` Pravin Pathak
2026-11-17 17:25           ` Jerin Jacob
2026-11-17 17:39             ` Bruce Richardson [this message]
2026-11-17 17:55               ` Pravin Pathak

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=avyROPrxuK4BPY56@bricha3-mobl1.ger.corp.intel.com \
    --to=bruce.richardson@intel.com \
    --cc=chenbox@nvidia.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=jerinj@marvell.com \
    --cc=jerinjacobk@gmail.com \
    --cc=mattias.ronnblom@ericcson.com \
    --cc=mattias.ronnblom@ericsson.com \
    --cc=mike.ximing.chen@intel.com \
    --cc=nipun.gupta@amd.com \
    --cc=pravin.pathak.dev@gmail.com \
    --cc=pravinpathakdev@gmail.com \
    --cc=thomas@monjalon.net \
    --cc=tirthendu.sarkar@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.