Re: [SPDK] Need help for iSCSI configuration for NVMe disc

[Kiran Dikshit <kdikshit at cloudsimple.com>]

[-- Attachment #1: Type: text/plain, Size: 5440 bytes --]

Hi Liu,

I did make the changes but still see the error.
Attached is the modified config file and the error log for the same.




-Kiran


> On 31-May-2017, at 8:32 AM, Liu, Changpeng <changpeng.liu(a)intel.com> wrote:
> 
> Hi Kiran,
>  
> Can you try set TransportID “trtype:PCIe traddr:0000.0a:00.0” Nvme3  in your configuration file, and also
> change your “LUN0 Nvme3” with “LUN0 Nvme3n1” ?
>  
> Nvme3 is not a block device name
>  
>   <>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Harris, James R
> Sent: Friday, May 26, 2017 11:42 PM
> To: Kiran Dikshit <kdikshit(a)cloudsimple.com>
> Cc: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] Need help for iSCSI configuration for NVMe disc
>  
> Hi Kiran,
>  
> Could you attach the following:
>  
> 1)      A log of “lspci –vvv”
> 2)      Your full SPDK iscsi.conf file
> 3)      A full log of the iscsi_tgt output.  Please enable debug (./configure --enable-debug) and add “-t all” to your command line to enable some extra debug logging
>  
> Thanks,
>  
> -Jim
>  
>  
> From: Kiran Dikshit <kdikshit(a)cloudsimple.com <mailto:kdikshit(a)cloudsimple.com>>
> Date: Thursday, May 25, 2017 at 7:07 PM
> To: James Harris <james.r.harris(a)intel.com <mailto:james.r.harris(a)intel.com>>
> Cc: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] Need help for iSCSI configuration for NVMe disc
>  
> Hi Harris, 
>  
> After the previous mail, i did check out the masters and DPDK version 17.05 and found the same issue. 
>  
> SPDK git commit ID : 5597e835c8981d6500d00c0f88fd10edae173621
>  
> Thank you
> Kiran
>  
> On 26-May-2017, at 12:38 AM, Harris, James R <james.r.harris(a)intel.com <mailto:james.r.harris(a)intel.com>> wrote:
>  
> Hi Kiran,
>  
> Can you confirm which SPDK git commit ID you are working from?  Since there have been changes in this transport ID format, I want to make sure I provide you the correct details based on the SPDK version you are using.
>  
> Thanks,
>  
> -Jim
>  
>  
> From: Kiran Dikshit <kdikshit(a)cloudsimple.com <mailto:kdikshit(a)cloudsimple.com>>
> Date: Wednesday, May 24, 2017 at 10:50 PM
> To: James Harris <james.r.harris(a)intel.com <mailto:james.r.harris(a)intel.com>>
> Cc: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] Need help for iSCSI configuration for NVMe disc
>  
> Hi Harris, 
>  
> I did try with the TransportID “trtype:PCIe traddr:0000.0a:00.0” Nvme3  option i am still seeing the same issue. 
>  
> Thank you
> Kiran
>  
> On 24-May-2017, at 9:50 PM, Harris, James R <james.r.harris(a)intel.com <mailto:james.r.harris(a)intel.com>> wrote:
>  
> 
> On May 24, 2017, at 4:10 AM, Kiran Dikshit <kdikshit(a)cloudsimple.com <mailto:kdikshit(a)cloudsimple.com>> wrote:
> 
> Hi All,
> 
> I am trying to configure iSCSI for a NVMe drive and getting the below error’s. I have unbound the NVMe drive from kernel using the setup.sh.
> 
> tgt_node.c: 718:spdk_iscsi_tgt_node_construct: ***ERROR*** Could not construct SCSI device
> tgt_node.c: 984:spdk_cf_add_iscsi_tgt_node: ***ERROR*** tgt_node1: add_iscsi_target_node error
> tgt_node.c:1010:spdk_iscsi_init_tgt_nodes: ***ERROR*** spdk_cf_add_iscsi_tgt_node() failed
> iscsi_subsystem.c: 965:spdk_iscsi_subsystem_init: ***ERROR*** spdk_iscsi_init_tgt_nodes() failed
> app.c: 404:spdk_app_init: ***ERROR*** spdk_subsystem_init() failed
> 
> 
> 
> Below is the section of my iscsi.conf.in <http://iscsi.conf.in/> file for reference.
> 
> # NVMe configuration options
> [Nvme]
> # NVMe Device Whitelist
> # Users may specify which NVMe devices to claim by their PCI
> # domain, bus, device, and function. The format is dddd:bb:dd.f, which is
> # the same format displayed by lspci or in /sys/bus/pci/devices. The second
> # argument is a "name" for the device that can be anything. The name
> # is referenced later in the Subsystem section.
> #
> # Alternatively, the user can specify ClaimAllDevices. All
> # NVMe devices will be claimed and named Nvme0, Nvme1, etc.
> # BDF 0000:00:00.0 Nvme0
>  BDF 0000:0a:00.0 Nvme3
> 
> Hi Kiran,
> 
> The format of the NVMe section changed in the last SPDK release, so that we could add support for remote NVMe-oF devices as well.
> 
> Can you try the following instead?
> 
> [Nvme]
>  TransportID “trtype:PCIe traddr:0000.0a:00.0” Nvme3
> 
> The comment for spdk_nvme_transport_id_parse() in include/spdk/nvme.h describes the format in more detail, including how to specify NVMe-oFtargets.
> 
> Thanks,
> 
> -Jim
> 
> 
> 
> 
> [TargetNode1]
>  TargetName disk2
>  TargetAlias "Data Disk2"
>  Mapping PortalGroup1 InitiatorGroup1
>  AuthMethod Auto
>  AuthGroup AuthGroup1
>  UseDigest Auto
>  LUN0 Nvme3
>  QueueDepth 64
> 
> 
> Thank you
> Kiran
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>  
>  


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 461 bytes --]

[-- Attachment #3: iscsi.conf.in --]
[-- Type: application/octet-stream, Size: 3915 bytes --]

# iSCSI target configuration file
#
# Please write all parameters using ASCII.
# The parameter must be quoted if it includes whitespace.
#
# Configuration syntax:
# Leading whitespace is ignored.
# Lines starting with '#' are comments.
# Lines ending with '\' are concatenated with the next line.
# Bracketed ([]) names define sections

[Global]
  # Shared Memory Group ID. SPDK applications with the same ID will share memory.
  # Default: <the process PID>
  #SharedMemoryID 0

  # Users can restrict work items to only run on certain cores by
  #  specifying a ReactorMask.  Default is to allow work items to run
  #  on core 0.
  #ReactorMask 0xFFFF

  # Tracepoint group mask for spdk trace buffers
  # Default: 0x0 (all tracepoint groups disabled)
  # Set to 0xFFFFFFFFFFFFFFFF to enable all tracepoint groups.
  #TpointGroupMask 0x0

  # syslog facility
  LogFacility "local7"

[iSCSI]
  # node name (not include optional part)
  # Users can optionally change this to fit their environment.
  NodeBase "iqn.2016-06.io.spdk"

  AuthFile /usr/local/etc/spdk/auth.conf

  MinConnectionsPerCore 4
  # Power saving related variable, this parameter defines how long an iSCSI
  # connection must be idle before moving it to a state where it will consume
  # less power. This variable is defined in terms of microseconds. We set default
  # value as 5ms.
  MinConnectionIdleInterval 5000

  # Socket I/O timeout sec. (0 is infinite)
  Timeout 30

  # authentication information for discovery session
  DiscoveryAuthMethod Auto

  #MaxSessions 128
  #MaxConnectionsPerSession 2

  # iSCSI initial parameters negotiate with initiators
  # NOTE: incorrect values might crash
  DefaultTime2Wait 2
  DefaultTime2Retain 60

  ImmediateData Yes
  ErrorRecoveryLevel 0

# Users must change the PortalGroup section(s) to match the IP addresses
#  for their environment.
# PortalGroup sections define which TCP ports the iSCSI server will use
#  to listen for incoming connections.  These are also used to determine
#  which targets are accessible over each portal group.
[PortalGroup1]
  Portal DA1 192.168.0.10:3260

# Users must change the InitiatorGroup section(s) to match the IP
#  addresses and initiator configuration in their environment.
# Netmask can be used to specify a single IP address or a range of IP addresses
#  Netmask 192.168.1.20   <== single IP address
#  Netmask 192.168.1.0/24 <== IP range 192.168.1.*
[InitiatorGroup1]
  InitiatorName ALL
   Netmask 192.168.0.0/24

# NVMe configuration options
[Nvme]
  # NVMe Device Whitelist
  # Users may specify which NVMe devices to claim by their transport id.
  # See spdk_nvme_transport_id_parse() in spdk/nvme.h for the correct format.
  # The second argument is the assigned name, which can be referenced from
  # other sections in the configuration file. For NVMe devices, a namespace
  # is automatically appended to each name in the format <YourName>nY, where
  # Y is the NSID (starts at 1).
  #TransportID "trtype:PCIe traddr:0000:00:00.0" Nvme0
  # BDF 0000:0a:00.0 Nvme3n1
  TransportID “trtype:PCIe traddr:0000.0a:00.0” Nvme3

  # The number of attempts per I/O when an I/O fails. Do not include
  # this key to get the default behavior.
  NvmeRetryCount 4
  # Registers the application to receive timeout callback and to reset the controller.
  ResetControllerOnTimeout Yes
  # Timeout value.
  NvmeTimeoutValue 30
  # Set how often the admin queue is polled for asynchronous events.
  # Units in microseconds.
  AdminPollRate 100000

# Users should change the TargetNode section(s) below to match the
# desired iSCSI target node configuration.
# TargetName, Mapping, LUN0 are minimum required
[TargetNode1]
  TargetName disk1
  TargetAlias "Data Disk1"
  Mapping PortalGroup1 InitiatorGroup1
  AuthMethod Auto
  AuthGroup AuthGroup1
  # Enable header and data digest
  # UseDigest Header Data
  UseDigest Auto
  LUN0 Nvme3n1
  QueueDepth 64 

[-- Attachment #4: attachment.html --]
[-- Type: text/html, Size: 215 bytes --]

[-- Attachment #5: iscsi_error_log.txt --]
[-- Type: text/plain, Size: 4016 bytes --]

EAL: Detected 20 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
Ioat Copy Engine Offload Enabled
NvmeRetryCount was renamed to RetryCount
Please update your configuration filenvme.c: 576:spdk_nvme_transport_id_parse: ***ERROR*** Unknown transport ID key '“trtype'
NvmeTimeoutValue was renamed to Timeout
Please update your configuration file
ResetControllerOnTimeout was renamed to ActionOnTimeout
Please update your configuration file
[bdev_nvme] blockdev_nvme.c: 523:probe_cb: Probing device 0000:0a:00.0
[bdev_nvme] blockdev_nvme.c: 533:probe_cb: Not claiming device at 0000:0a:00.0
[debug] iscsi_subsystem.c: 555:spdk_iscsi_app_read_parameters: spdk_iscsi_app_read_parameters
[debug] iscsi_subsystem.c: 577:spdk_iscsi_app_read_parameters: AuthFile /usr/local/etc/spdk/auth.conf
[debug] iscsi_subsystem.c: 593:spdk_iscsi_app_read_parameters: NodeBase iqn.2016-06.io.spdk
[debug] iscsi_subsystem.c: 604:spdk_iscsi_app_read_parameters: MaxSessions 128
[debug] iscsi_subsystem.c: 618:spdk_iscsi_app_read_parameters: MaxConnectionsPerSession 2
[debug] iscsi_subsystem.c: 639:spdk_iscsi_app_read_parameters: DefaultTime2Wait 2
[debug] iscsi_subsystem.c: 647:spdk_iscsi_app_read_parameters: DefaultTime2Retain 60
[debug] iscsi_subsystem.c: 665:spdk_iscsi_app_read_parameters: FirstBurstLength 8192
[debug] iscsi_subsystem.c: 669:spdk_iscsi_app_read_parameters: MaxBurstLength 1048576
[debug] iscsi_subsystem.c: 673:spdk_iscsi_app_read_parameters: MaxRecvDataSegmentLength 65536
[debug] iscsi_subsystem.c: 712:spdk_iscsi_app_read_parameters: InitialR2T Yes
[debug] iscsi_subsystem.c: 727:spdk_iscsi_app_read_parameters: ImmediateData Yes
[debug] iscsi_subsystem.c: 747:spdk_iscsi_app_read_parameters: DataPDUInOrder Yes
[debug] iscsi_subsystem.c: 766:spdk_iscsi_app_read_parameters: AllowDuplicateIsid No
[debug] iscsi_subsystem.c: 786:spdk_iscsi_app_read_parameters: DataSequenceInOrder Yes
[debug] iscsi_subsystem.c: 797:spdk_iscsi_app_read_parameters: ErrorRecoveryLevel 0
[debug] iscsi_subsystem.c: 805:spdk_iscsi_app_read_parameters: Timeout 30
[debug] iscsi_subsystem.c: 815:spdk_iscsi_app_read_parameters: FlushTimeout 18304
[debug] iscsi_subsystem.c: 829:spdk_iscsi_app_read_parameters: NopInInterval 30
[debug] iscsi_subsystem.c: 868:spdk_iscsi_app_read_parameters: DiscoveryAuthMethod Auto
[debug] iscsi_subsystem.c: 899:spdk_iscsi_app_read_parameters: DiscoveryAuthGroup None
[debug] portal_grp.c: 365:spdk_iscsi_portal_grp_create_from_configfile: add portal group (from config file) 1
[debug] portal_grp.c: 420:spdk_iscsi_portal_grp_create_from_configfile: RIndex=0, Host=192.168.0.10, Port=3260, Tag=1
[debug] init_grp.c:  59:spdk_iscsi_init_grp_create_from_configfile: add initiator group 1
[debug] init_grp.c: 108:spdk_iscsi_init_grp_create_from_configfile: InitiatorName ALL
[debug] init_grp.c: 129:spdk_iscsi_init_grp_create_from_configfile: Netmask 192.168.0.0/24
[debug] init_grp.c: 201:spdk_iscsi_init_grp_create_from_initiator_list: add initiator group (from initiator list) tag=1, #initiators=1, #masks=1
[debug] init_grp.c: 219:spdk_iscsi_init_grp_create_from_initiator_list: InitiatorName ALL
[debug] init_grp.c: 224:spdk_iscsi_init_grp_create_from_initiator_list: Netmask 192.168.0.0/24
[debug] tgt_node.c: 978:spdk_iscsi_init_tgt_nodes: spdk_iscsi_init_tgt_nodes
[debug] tgt_node.c: 781:spdk_cf_add_iscsi_tgt_node: add unit 1
[debug] tgt_node.c: 870:spdk_cf_add_iscsi_tgt_node: AuthMethod Auto
[debug] tgt_node.c: 900:spdk_cf_add_iscsi_tgt_node: AuthGroup AuthGroup1
[debug] tgt_node.c: 923:spdk_cf_add_iscsi_tgt_node: UseDigest Auto
tgt_node.c: 699:spdk_iscsi_tgt_node_construct: ***ERROR*** Could not construct SCSI device
tgt_node.c: 965:spdk_cf_add_iscsi_tgt_node: ***ERROR*** tgt_node1: add_iscsi_target_node error
tgt_node.c: 991:spdk_iscsi_init_tgt_nodes: ***ERROR*** spdk_cf_add_iscsi_tgt_node() failed
iscsi_subsystem.c: 971:spdk_iscsi_subsystem_init: ***ERROR*** spdk_iscsi_init_tgt_nodes() failed
app.c: 416:spdk_app_init: ***ERROR*** spdk_subsystem_init() failed

[-- Attachment #6: attachment.html --]
[-- Type: text/html, Size: 37059 bytes --]