Re: [SPDK] how to use blobstore with pthread

[Zhengyu Zhang <freeman.zhang1992 at gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4411 bytes --]

Hi Cunyin,

Thanks for your reply! I do call spdk_bs_alloc_io_channle() before the
read/write. But since the function needs a `struct spdk_blob_store *bs`
parameter, we have to call it after spdk_bs_init/load, usually in
init/load callbacks. But spdk_bs_init/load is where segmentation fault
happens:

(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x0000000000430290 in spdk_poller_register (fn=fn(a)entry=0x4065f0
<ioat_poll>, arg=0x2629080, period_microseconds =
period_microseconds(a)entry=0)at io_channel.c:226
#2  0x00000000004065c1 in ioat_create_cb (io_device=<optimized out>,
ctx_buf=0x7f16ac00ecc0) at copy_engine_ioat.c:206
#3  0x000000000043074c in spdk_get_io_channel (io_device=0x67d020
<ioat_copy_engine>) at io_channel.c:453
#4  0x000000000042d11b in copy_create_cb (io_device=<optimized out>,
ctx_buf=0x7f16ac00ec70) at copy_engine.c:186
#5  0x000000000043074c in spdk_get_io_channel (io_device=0x67d9d0
<spdk_copy_module_list>) at io_channel.c:453
#6  0x0000000000429b69 in spdk_bdev_channel_create (io_device=<optimized
out>, ctx_buf=0x7f16ac00ebc0) at bdev.c:847
#7  0x000000000043074c in spdk_get_io_channel (io_device=0x7f18abbeb5c0)
at io_channel.c:453
#8  0x000000000041c426 in _spdk_bs_channel_create
(io_device=0x7f16ac000a00, ctx_buf=0x7f16ac000b40) at blobstore.c:1425
#9  0x000000000043074c in spdk_get_io_channel
(io_device=io_device(a)entry=0x7f16ac000a00) at io_channel.c:453
#10 0x000000000041c588 in spdk_bs_register_md_thread (bs=0x7f16ac000a00)
at blobstore.c:2649
#11 _spdk_bs_alloc (dev=dev(a)entry=0x7f16ac000940,
opts=opts(a)entry=0x7f18bd414c10) at blobstore.c:1551
#12 0x000000000041f1ee in spdk_bs_init (dev=0x7f16ac000940,
o=o(a)entry=0x0, cb_fn=cb_fn(a)entry=0x405e70 <bs_init_complete>,
cb_arg=cb_arg(a)entry=0x2622270)
    at blobstore.c:2279
#13 0x0000000000405dca in hello_work (hello_context=0x2622270) at
hello_blob.c:412
#14 0x00007f18be74adc5 in start_thread () from /lib64/libpthread.so.0


Thanks!
Zhengyu

On 1/16/18 9:00 AM, Chang, Cunyin wrote:
> Hi Zhengyu,
> 
>  
> 
> for new threads:
> 
> after you allocate thread, before you do the create, read, write, you
> need allocate io channel by calling:
> 
> spdk_bs_alloc_io_channel(),
> 
> the blobs read/write need one IO channel.
> 
>  
> 
> -Cunyin
> 
>  
> 
> *From:*SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhengyu Zhang
> *Sent:* Monday, January 15, 2018 11:20 PM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* [SPDK] how to use blobstore with pthread
> 
>  
> 
> Hi list,
> 
>  
> 
> Could someone tell me how to integrate the blobstore into multithread
> programs? I saw some examples using SPDK event framework to do
> multithreading, but how to do that with the pthread library or C++
> std::thread is still mysterious to me...
> 
>  
> 
> My work flow is shown below. It gives me segmentation fault when it
> init/load blobstore:
> 
>  
> 
> [main]: spdk_app_start -> get bdev -> pthread_create -> ...
> 
>  
> 
> [new threads]:  spdk_allocate_thread -> spdk_bdev_create_bs_dev ->
> spdk_bs_init/load[SEG FAULT] -> create/open/write/ blobs
> 
>  
> 
>  
> 
> Thanks!
> 
> Zhengyu
> 
>  
> 
>  
> 
>  
> 
>  
> 
> ---
> 
> My thread routine:
> 
>  
> 
> + 393 #define THREAD_NUM    (1)
> 
> + 394 
> 
> + 395 static void _spdk_send_msg(spdk_thread_fn fn, void *ctx, void
> *thread_ctx)
> 
> + 396 {
> 
> + 397     assert(false);
> 
> + 398 }
> 
> + 399 
> 
> + 400 int thread_routine(struct hello_context_t *hello_context)
> 
> + 401 {
> 
> + 402     spdk_allocate_thread(_spdk_send_msg, NULL, NULL, NULL, NULL);
> 
> + 403 
> 
> + 404     struct spdk_bs_dev *bs_dev = NULL;
> 
> + 405     bs_dev = spdk_bdev_create_bs_dev(hello_context->bdev, NULL, NULL);
> 
> + 406     if (bs_dev == NULL) {
> 
> + 407           SPDK_ERRLOG("Could not create blob bdev!!\n");
> 
> + 408           spdk_app_stop(-1);
> 
> + 409           return;
> 
> + 410     }
> 
> + 411 
> 
> + 412     spdk_bs_init(bs_dev, NULL, bs_init_complete, hello_context); 
> // SEG FAULT
> 
> + 413 }
> 
> 
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>