Re: [SPDK] SPDK Ring Enqueue/Dequeue Issue

[Walker, Benjamin <benjamin.walker at intel.com>]

[-- Attachment #1: Type: text/plain, Size: 2612 bytes --]

On Wed, 2018-09-19 at 19:07 +0000, Harris, James R wrote:
> 
> On 9/19/18, 11:37 AM, "SPDK on behalf of Andrey Kuzmin" <
> spdk-bounces(a)lists.01.org on behalf of andrey.v.kuzmin(a)gmail.com> wrote:
> 
>     On Wed, Sep 19, 2018 at 9:13 PM John Barnard <john.barnard(a)broadcom.com>
>     wrote:
>     
> <snip>
> 
>     >
>     > It seems that the spdk_ring_enqueue function calls
>     > rte_ring_mp_enqueue_bulk(), while spdk_ring_dequeue function calls
>     > rte_ring_sc_dequeue_burst().  When we change it to call
>     > ret_ring_mp_dequeue_burst(), the problem went away.  So my question is,
> why
>     > this difference in the rte calls made by the SPDK?  Is this a bug or on
>     > purpose?  Also, we noticed that there is no ring create flag (in
>     > include/spdk/env.h) for multi-producer, multi-consumer (i.e.
>     > SPDK_RING_TYPE_MP_MC), although it doesn't seem to matter if we pass the
>     > SPDK_RING_TYPE_MP_SC flag to  spdk_ring_create() (i.e. it's the call to
>     > rte_ring_mp_dequeue_burst that's critical).  Is there a reason this flag
>     > was left out?
>     >
>     
>     The two issues above are interrelated. SPDK seemingly does not care for
>     MP/MC rings, so it neither defines the flag to create the respective ring
>     type, nor appreciates this ring type under ring_dequeue (for enqueue, it
>     uses MP flavor which works for SP case as well).
>     
>     The definitive answer as to why belongs to the SPDK team, while my guess
> is
>     that rings in SPDK are primarily (and solely, to the best of my knowledge)
>     for passing messages between SPDK threads where  MP/MC scenario does not
>   apply.
> 
> Andrey is correct.  There hasn’t been a use case for MP/MC rings in SPDK
> yet.  Rings are primarily used today for passing messages to a specific
> reactor – each reactor has its own ring (so it’s single consumer) but any
> thread may post messages to it.
> 
> I see no reason to not add MP/MC support if it’s needed for NVMe-oF FC.

The best way to introduce this into SPDK is to add flags or function variants
for single/multi-consumer, so the appropriate DPDK function call can be used
underneath. Changing the ring from single consumer to multi-consumer introduces
an atomic instruction. That's why we avoid that, given our only use case today
is single consumer.

> 
> John – is this something you’d like to tackle?
> 
> -Jim
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk