Re: [SPDK] BDEV Registration

[Kumaraparameshwaran Rathnavel <krath at cloudsimple.com>]

[-- Attachment #1: Type: text/plain, Size: 9790 bytes --]

These config lines were commented. I just enabled it. It fails even after that. Is it because of this error No free hugepages reported in hugepages-1048576kB

Thanking You,
Param

On 21-Oct-2016, at 12:11 PM, Yang, Ziye <ziye.yang(a)intel.com> wrote:
> 
> Do you have set malloc module in your configuration file? You need have the malloc configuration such as
>  
> [Malloc]
>   NumberOfLuns 4
>   LunSizeInMB  64
>  
>   <>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 2:37 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] BDEV Registration
>  
> My platform does not have RNIC but the bdev should get initialised right. Does it have any dependency on this transport type. Irrespective of this bdevs should get added right.
>  
> Thanking You,
> Param
> On 21-Oct-2016, at 11:59 AM, Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> wrote:
>  
> Virtual controller is supported.   From your output, it seems that you did not compile with RDMA, does your platform have RNIC?
>  
> “conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
> ”
>  
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 2:27 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] BDEV Registration
>  
> Is Virtual controller not supported in the current NVMe/F target implementation. Because in configuration file it is written as only direct mode is currently supported.  This is the output generated. and my config file.
>  
> Starting Intel(R) DPDK initialization ... 
> [ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
> EAL: Detected 4 lcore(s)
> EAL: Auto-detected process type: PRIMARY
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> done.
> Occupied cpu core mask is 0x1
> Occupied cpu socket mask is 0x1
> copy_engine_ioat.c: 288:copy_engine_ioat_init: ***ERROR*** Not enough IOAT channels for all cores
> Total cores available: 1
> Reactor started on core 0x0
>  
> *** NVMf Pool Creation ***
> conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
> conf.c: 516:spdk_nvmf_parse_subsystem: ***ERROR*** bdev is NULL
> nvmf_tgt.c: 254:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed
>  
> The subystem part of the config file
> [Subsystem1]
>   NQN nqn.2016-06.io.spdk:cnode2
>   Core 0
>   Mode Virtual
>   Listen RDMA 192.168.2.21:4420
>   Host nqn.2016-06.io.spdk:init
>   SN SPDK00000000000001
>   Namespace Malloc0
>   Namespace Malloc1
>  
> Thanking You,
> Param.
> On 21-Oct-2016, at 11:43 AM, Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> wrote:
>  
> #define SPDK_SUBSYSTEM_REGISTER(_name, _init, _fini, _config)                                    \
>             struct spdk_subsystem __spdk_subsystem_ ## _name = {                           \
>             .name = #_name,                                                                                            \
>             .init = _init,                                                                                         \
>             .fini = _fini,                                                                                        \
>             .config = _config,                                                                               \
>             };                                                                                                         \
>             __attribute__((constructor)) static void _name ## _register(void)            \
>             {                                                                                                          \
>                         spdk_add_subsystem(&__spdk_subsystem_ ## _name);                        \
>             }
> 
> The attribute is constructor, so the function will be directly called before the execution of main.  I think that for your case, you just need to change the configuration file.
> 
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 2:09 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] BDEV Registration
> 
> Thanks for the update.
> 
> I am still not clear who should call the function SPDK_SUBSYSTEM_REGISTER. I just wanted to add virtual mode subsystem as the NVMe/F target. In configuration the Virtual has two Malloc drives. So even before the main function I should create these devices and add. I find SPDK_SUBSYSTEM_REGISTER in lib/bdev/malloc/blocdev_malloc.c . So who should call this function? Should I call it by writing a separate program.
> 
> Thanking You,
> Param 
> 
> 
> 
> Hi Param,
> 
> In the iscsi_tgt's main:
> 
> spdk_app_init  -> spdk_subsystem_init   ->   subsystem->init()
> 
> (bdev is also a subsystem:   registered with  SPDK_SUBSYSTEM_REGISTER(bdev, spdk_bdev_initialize, spdk_bdev_finish, spdk_bdev_config_text))
> 
> SPDK_SUBSYSTEM_REGISTER  in bdev will be executed before main, and add the bdev subsystem.
> 
> It will finally call spdk_bdev_initialize   ->  spdk_bdev_module_initialize  ->  call  module_init.
> Thanks.
> 
> Best Regards,
> Ziye Yang
> 
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of 
> Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 12:09 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] BDEV Registration
> 
> Thanks Daniel.
> 
> When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
> 
> Thanking You,
> Param
> 
> 
> On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com <mailto:daniel.verkamp(a)intel.com>> wrote:
> 
> On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
> 
> 
> Hi All,
> 
> I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
> 
> There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
> 
> How is this function getting called? Who calls this function
> 
> Regards
> Param
> 
> Hi Param,
> 
> You can find the interface for writing new bdev modules in the header 
> lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined.  This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
> 
> Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register().  Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
> 
> When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
> 
> To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list.  This adds linker options to all of the applications in the SPDK tree that use bdev modules.
> 
> Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
> 
> Thanks,
> -- Daniel
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>  
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>  
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 41419 bytes --]