From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1AD1C47095 for ; Wed, 7 Oct 2020 14:16:34 +0000 (UTC) Received: from dpdk.org (dpdk.org [92.243.14.124]) by mail.kernel.org (Postfix) with ESMTP id 07DCB212CC for ; Wed, 7 Oct 2020 14:16:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 07DCB212CC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1D8631BBFD; Wed, 7 Oct 2020 16:16:32 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 675121BBD6 for ; Wed, 7 Oct 2020 16:16:30 +0200 (CEST) IronPort-SDR: ALZIcLw+ZPg5x2rQ2Zh6iOrz9mnCCotIlSa/ARQI/CpDfqk2rTcVuJ6j8MK7F+5907eF/jJSNV iDyKpi7us4Ig== X-IronPort-AV: E=McAfee;i="6000,8403,9767"; a="164186874" X-IronPort-AV: E=Sophos;i="5.77,347,1596524400"; d="scan'208";a="164186874" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Oct 2020 07:16:27 -0700 IronPort-SDR: ObPDd2y33vJ8oMz9zXHJxAPp+9gI16dmy2cfHvoILORVj09FTUIWzYSxMVJavgqEqrTlDasvA1 IvA2+awYaUbQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,347,1596524400"; d="scan'208";a="342893341" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by fmsmga004.fm.intel.com with ESMTP; 07 Oct 2020 07:16:27 -0700 From: Gage Eads To: dev@dpdk.org Cc: olivier.matz@6wind.com, arybchenko@solarflare.com, john.mcnamara@intel.com, marko.kovacevic@intel.com Date: Wed, 7 Oct 2020 09:18:32 -0500 Message-Id: <20201007141832.23716-1-gage.eads@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20200811211015.29704-1-gage.eads@intel.com> References: <20200811211015.29704-1-gage.eads@intel.com> Subject: [dpdk-dev] [PATCH v4] doc: add stack mempool guide X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This guide describes the two stack modes, their tradeoffs, and (via a reference to the mempool guide) how to enable them. Signed-off-by: Gage Eads --- v4: Expanded first paragraph in stack.rst per Olivier's feedback v3: Fixed "Title underline too short" warning v2: Added commit description doc/guides/mempool/index.rst | 1 + doc/guides/mempool/stack.rst | 43 +++++++++++++++++++++++++++++++++++ doc/guides/prog_guide/mempool_lib.rst | 2 ++ doc/guides/prog_guide/stack_lib.rst | 4 ++++ 4 files changed, 50 insertions(+) create mode 100644 doc/guides/mempool/stack.rst diff --git a/doc/guides/mempool/index.rst b/doc/guides/mempool/index.rst index bbd02fd81..a0e55467e 100644 --- a/doc/guides/mempool/index.rst +++ b/doc/guides/mempool/index.rst @@ -14,3 +14,4 @@ application through the mempool API. octeontx octeontx2 ring + stack diff --git a/doc/guides/mempool/stack.rst b/doc/guides/mempool/stack.rst new file mode 100644 index 000000000..ab71716bd --- /dev/null +++ b/doc/guides/mempool/stack.rst @@ -0,0 +1,43 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Intel Corporation. + +Stack Mempool Driver +==================== + +**rte_mempool_stack** is a pure software mempool driver based on the +``rte_stack`` DPDK library. For run-to-completion workloads with sufficiently +large per-lcore caches, the mbufs will likely stay in the per-lcore caches and +the mempool type (ring, stack, etc.) will have a negligible impact on +performance. However a stack-based mempool is often better suited to pipelined +packet-processing workloads (which allocate and free mbufs on different lcores) +than a ring-based mempool, since its LIFO behavior results in better temporal +locality and a minimal memory footprint even if the mempool is +over-provisioned. Users are encouraged to benchmark with multiple mempool types +to determine which works best for their specific application. + +The following modes of operation are available for the stack mempool driver and +can be selected as described in :ref:`Mempool_Handlers`: + +- ``stack`` + + The underlying **rte_stack** operates in standard (lock-based) mode. + For more information please refer to :ref:`Stack_Library_Std_Stack`. + +- ``lf_stack`` + + The underlying **rte_stack** operates in lock-free mode. For more + information please refer to :ref:`Stack_Library_LF_Stack`. + +The standard stack outperforms the lock-free stack on average, however the +standard stack is non-preemptive: if a mempool user is preempted while holding +the stack lock, that thread will block all other mempool accesses until it +returns and releases the lock. As a result, an application using the standard +stack whose threads can be preempted can suffer from brief, infrequent +performance hiccups. + +The lock-free stack, by design, is not susceptible to this problem; one thread can +be preempted at any point during a push or pop operation and will not impede +the progress of any other thread. + +For a more detailed description of the stack implementations, please refer to +:doc:`../prog_guide/stack_lib`. diff --git a/doc/guides/prog_guide/mempool_lib.rst b/doc/guides/prog_guide/mempool_lib.rst index f0bdcd3be..890535eb2 100644 --- a/doc/guides/prog_guide/mempool_lib.rst +++ b/doc/guides/prog_guide/mempool_lib.rst @@ -104,6 +104,8 @@ These user-owned caches can be explicitly passed to ``rte_mempool_generic_put()` The ``rte_mempool_default_cache()`` call returns the default internal cache if any. In contrast to the default caches, user-owned caches can be used by unregistered non-EAL threads too. +.. _Mempool_Handlers: + Mempool Handlers ------------------------ diff --git a/doc/guides/prog_guide/stack_lib.rst b/doc/guides/prog_guide/stack_lib.rst index 8fe8804e3..3097cab0c 100644 --- a/doc/guides/prog_guide/stack_lib.rst +++ b/doc/guides/prog_guide/stack_lib.rst @@ -28,6 +28,8 @@ Implementation The library supports two types of stacks: standard (lock-based) and lock-free. Both types use the same set of interfaces, but their implementations differ. +.. _Stack_Library_Std_Stack: + Lock-based Stack ---------------- @@ -35,6 +37,8 @@ The lock-based stack consists of a contiguous array of pointers, a current index, and a spinlock. Accesses to the stack are made multi-thread safe by the spinlock. +.. _Stack_Library_LF_Stack: + Lock-free Stack ------------------ -- 2.13.6