Re: [PATCH 0/3] nvme fabrics polling fixes

[Sagi Grimberg <sagi@grimberg.me>]

> I couldn't test the existing tcp

It is very simple to do, for tcp you can just operate on the loopback
iface 127.0.0.1

See example:
$ ./target.sh 1 1
$ nvme connect-all -t tcp -a 127.0.0.1 -P 1

For rdma you can work on any interface you want, you just need
iproute with rdma utility (which is available with recent iproute
packages).

Setup Software RoCE:
$ rdma link add rxe0 type rxe netdev <iface>
Setup Software iWarp:
$ rdma link add siw0 type siw netdev <iface>

Then the same as before...
$ ./target.sh 1 1
$ nvme connect-all -t tcp -a <iface-addr> -P 1


That is essentially what blktest do.

target.sh
--
#!/bin/bash

# args: <num_subsys> <nr_ns-per-subsystem> <nvme|file>
# examples:
# - ./target.sh 1 1       # uses null_blk
# - ./target.sh 1 1 nvme
# - ./target.sh 1 1 file

CFGFS=/sys/kernel/config/nvmet
NQN=testnqn

mount -t configfs configfs /sys/kernel/config > /dev/null 2>&1

rm -f $CFGFS/ports/*/subsystems/*
rm -f $CFGFS/subsystems/*/allowed_hosts/* > /dev/null 2>&1
rmdir $CFGFS/subsystems/*/namespaces/* > /dev/null 2>&1
rmdir $CFGFS/subsystems/* > /dev/null 2>&1
rmdir $CFGFS/ports/* > /dev/null 2>&1
rmdir $CFGFS/hosts/* > /dev/null 2>&1

modprobe -r nvmet_rdma nvmet-tcp > /dev/null 2>&1
modprobe -r ib_umad rdma_ucm ib_uverbs rdma_cm > /dev/null 2>&1

modprobe -r null_blk nvme-tcp nvme-rdma
modprobe -r nvme-loop
rm -f /tmp/nsf*

if [[ $1 == "0" ]]; then
         exit
fi

modprobe null_blk nr_devices=$2 #queue_mode=0 irqmode=0
modprobe -a nvmet_tcp nvmet_rdma

mkdir $CFGFS/ports/1
echo 0.0.0.0 > $CFGFS/ports/1/addr_traddr
echo 8009 > $CFGFS/ports/1/addr_trsvcid
echo "tcp" >  $CFGFS/ports/1/addr_trtype
echo "ipv4" >  $CFGFS/ports/1/addr_adrfam

mkdir $CFGFS/ports/2
echo 0.0.0.0 > $CFGFS/ports/2/addr_traddr
echo 4420 > $CFGFS/ports/2/addr_trsvcid
echo "rdma" >  $CFGFS/ports/2/addr_trtype
echo "ipv4" >  $CFGFS/ports/2/addr_adrfam

mkdir $CFGFS/ports/3
echo "loop" >  $CFGFS/ports/3/addr_trtype

for j in `seq $1`
do
         mkdir $CFGFS/subsystems/$NQN$j
         echo 1 > $CFGFS/subsystems/$NQN$j/attr_allow_any_host
         k=0
         for i in `seq $2`
         do
                 mkdir $CFGFS/subsystems/$NQN$j/namespaces/$i
                 if [[ "$3" == "nvme" ]]; then
                         echo -n "/dev/nvme0n1" > 
$CFGFS/subsystems/$NQN$j/namespaces/$i/device_path
                 elif [[ "$3" == "file" ]]; then
                         ns="/tmp/nsf$k"
                         touch $ns
                         truncate --size=2G $ns
                         echo -n "$ns" > 
$CFGFS/subsystems/$NQN$j/namespaces/$i/device_path
                 else
                         echo -n "/dev/nullb$k" > 
$CFGFS/subsystems/$NQN$j/namespaces/$i/device_path
                 fi
                 echo 1 > $CFGFS/subsystems/$NQN$j/namespaces/$i/enable
                 let "k+=1"
         done
         ln -s $CFGFS/subsystems/$NQN$j $CFGFS/ports/1/subsystems/
         ln -s $CFGFS/subsystems/$NQN$j $CFGFS/ports/2/subsystems/
         ln -s $CFGFS/subsystems/$NQN$j $CFGFS/ports/3/subsystems/
done
--