Re: [SPDK] BDEV Registration

[Kumaraparameshwaran Rathnavel <krath at cloudsimple.com>]

[-- Attachment #1: Type: text/plain, Size: 10352 bytes --]

Hi Everyone ,

Got the issue resolved by increasing the number huge pages.

Thanks for your Support will definitely need in future too.

Thanking You,
Param
> On 21-Oct-2016, at 12:23 PM, Kumaraparameshwaran Rathnavel <krath(a)cloudsimple.com> wrote:
> 
> These config lines were commented. I just enabled it. It fails even after that. Is it because of this error No free hugepages reported in hugepages-1048576kB
> 
> Thanking You,
> Param
> 
> On 21-Oct-2016, at 12:11 PM, Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> wrote:
>> 
>> Do you have set malloc module in your configuration file? You need have the malloc configuration such as
>>  
>> [Malloc]
>>   NumberOfLuns 4
>>   LunSizeInMB  64
>>  
>>   <>
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Kumaraparameshwaran Rathnavel
>> Sent: Friday, October 21, 2016 2:37 PM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
>> Subject: Re: [SPDK] BDEV Registration
>>  
>> My platform does not have RNIC but the bdev should get initialised right. Does it have any dependency on this transport type. Irrespective of this bdevs should get added right.
>>  
>> Thanking You,
>> Param
>> On 21-Oct-2016, at 11:59 AM, Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> wrote:
>>  
>> Virtual controller is supported.   From your output, it seems that you did not compile with RDMA, does your platform have RNIC?
>>  
>> “conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
>> ”
>>  
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Kumaraparameshwaran Rathnavel
>> Sent: Friday, October 21, 2016 2:27 PM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
>> Subject: Re: [SPDK] BDEV Registration
>>  
>> Is Virtual controller not supported in the current NVMe/F target implementation. Because in configuration file it is written as only direct mode is currently supported.  This is the output generated. and my config file.
>>  
>> Starting Intel(R) DPDK initialization ... 
>> [ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
>> EAL: Detected 4 lcore(s)
>> EAL: Auto-detected process type: PRIMARY
>> EAL: No free hugepages reported in hugepages-1048576kB
>> EAL: Probing VFIO support...
>> done.
>> Occupied cpu core mask is 0x1
>> Occupied cpu socket mask is 0x1
>> copy_engine_ioat.c: 288:copy_engine_ioat_init: ***ERROR*** Not enough IOAT channels for all cores
>> Total cores available: 1
>> Reactor started on core 0x0
>>  
>> *** NVMf Pool Creation ***
>> conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
>> conf.c: 516:spdk_nvmf_parse_subsystem: ***ERROR*** bdev is NULL
>> nvmf_tgt.c: 254:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed
>>  
>> The subystem part of the config file
>> [Subsystem1]
>>   NQN nqn.2016-06.io.spdk:cnode2
>>   Core 0
>>   Mode Virtual
>>   Listen RDMA 192.168.2.21:4420
>>   Host nqn.2016-06.io.spdk:init
>>   SN SPDK00000000000001
>>   Namespace Malloc0
>>   Namespace Malloc1
>>  
>> Thanking You,
>> Param.
>> On 21-Oct-2016, at 11:43 AM, Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> wrote:
>>  
>> #define SPDK_SUBSYSTEM_REGISTER(_name, _init, _fini, _config)                                    \
>>             struct spdk_subsystem __spdk_subsystem_ ## _name = {                           \
>>             .name = #_name,                                                                                            \
>>             .init = _init,                                                                                         \
>>             .fini = _fini,                                                                                        \
>>             .config = _config,                                                                               \
>>             };                                                                                                         \
>>             __attribute__((constructor)) static void _name ## _register(void)            \
>>             {                                                                                                          \
>>                         spdk_add_subsystem(&__spdk_subsystem_ ## _name);                        \
>>             }
>> 
>> The attribute is constructor, so the function will be directly called before the execution of main.  I think that for your case, you just need to change the configuration file.
>> 
>> -----Original Message-----
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Kumaraparameshwaran Rathnavel
>> Sent: Friday, October 21, 2016 2:09 PM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
>> Subject: Re: [SPDK] BDEV Registration
>> 
>> Thanks for the update.
>> 
>> I am still not clear who should call the function SPDK_SUBSYSTEM_REGISTER. I just wanted to add virtual mode subsystem as the NVMe/F target. In configuration the Virtual has two Malloc drives. So even before the main function I should create these devices and add. I find SPDK_SUBSYSTEM_REGISTER in lib/bdev/malloc/blocdev_malloc.c . So who should call this function? Should I call it by writing a separate program.
>> 
>> Thanking You,
>> Param 
>> 
>> 
>> 
>> Hi Param,
>> 
>> In the iscsi_tgt's main:
>> 
>> spdk_app_init  -> spdk_subsystem_init   ->   subsystem->init()
>> 
>> (bdev is also a subsystem:   registered with  SPDK_SUBSYSTEM_REGISTER(bdev, spdk_bdev_initialize, spdk_bdev_finish, spdk_bdev_config_text))
>> 
>> SPDK_SUBSYSTEM_REGISTER  in bdev will be executed before main, and add the bdev subsystem.
>> 
>> It will finally call spdk_bdev_initialize   ->  spdk_bdev_module_initialize  ->  call  module_init.
>> Thanks.
>> 
>> Best Regards,
>> Ziye Yang
>> 
>> -----Original Message-----
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of 
>> Kumaraparameshwaran Rathnavel
>> Sent: Friday, October 21, 2016 12:09 PM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
>> Subject: Re: [SPDK] BDEV Registration
>> 
>> Thanks Daniel.
>> 
>> When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
>> 
>> Thanking You,
>> Param
>> 
>> 
>> On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com <mailto:daniel.verkamp(a)intel.com>> wrote:
>> 
>> On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
>> 
>> 
>> Hi All,
>> 
>> I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
>> 
>> There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
>> 
>> How is this function getting called? Who calls this function
>> 
>> Regards
>> Param
>> 
>> Hi Param,
>> 
>> You can find the interface for writing new bdev modules in the header 
>> lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined.  This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
>> 
>> Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register().  Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
>> 
>> When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
>> 
>> To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list.  This adds linker options to all of the applications in the SPDK tree that use bdev modules.
>> 
>> Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
>> 
>> Thanks,
>> -- Daniel
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>> 
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>> 
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>>  
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>>  
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 42255 bytes --]