Re: [SPDK] Rocksdb test case issue

[Harris, James R <james.r.harris at intel.com>]

[-- Attachment #1: Type: text/plain, Size: 2190 bytes --]

Hi Kunal,

We have come up with a slightly different approach than spdk_bdev_io_reserve() – this one is called spdk_bdev_queue_io_wait().  Please see https://review.gerrithub.io/#/c/spdk/spdk/+/415074/.

When an SPDK bdev I/O submission function is called (i.e. read, readv, read_blocks, write, unmap, etc.) and it fails with –ENOMEM, it means no spdk_bdev_io structure was available.  The caller can then request a callback using spdk_bdev_queue_io_wait() to be notified when an spdk_bdev_io is available on the same thread.  The caller is responsible for providing the storage for the spdk_bdev_io_wait_entry structure, and saving any state so that it knows where to continue when the spdk_bdev_io is available.  In your striping case, that might mean keeping track if the first child I/O is successfully submitted, but you need to wait for an spdk_bdev_io for subsequent child I/O.  Even the most basic case, where the striping translation results in only one child I/O – and that one fails with –ENOMEM – will need to be accounted for.  (All virtual bdev modules will need to make similar changes).

Note that other patches have been pushed this week to allow specifying the spdk_bdev_io pool size - specifically to help with developing and testing this new code and applications/drivers that will be using the new API.

The patch still needs a bit more work but I wanted to bring it to your attention so you could provide feedback.  The bdevperf utility has been updated in this patch to make use of the new API and served well as a test case (i.e. limit the spdk_bdev_io pool to 32 but run with a queue depth of 128).

Regards,

-Jim



On 6/11/18, 8:30 AM, "SPDK on behalf of Sablok, Kunal" <spdk-bounces(a)lists.01.org on behalf of kunal.sablok(a)intel.com> wrote:

    Thanks Ben, As I understand from one of your recent review comment, somebody is working on SPDK API spdk_bdev_io_reserve(). I will be using this API and then waitq and poller will be removed, also the RocksDB test case issue should go away with this. Please let me know when spdk_bdev_io_reserve() is up in SPDK, I will merge with this.
    
    Regards,
    Kunal
    
[…]