Re: pvSCSI test

Re: pvSCSI test

[Juergen Gross]

On 02/03/2015 07:16 PM, Kristian Hagsted Rasmussen wrote:
>
> Hi Olaf and Juergen
>
>
> I am interested in testing pvSCSI as I have a system were it would be ideal.
> I have tried to apply this patch "http://marc.info/?l=xen-devel&m=139885599019457&w=2" called "libbxl: add support for pvscsi, iteration 1", to my xen-4.5 tree.
> I am using kernel 3.18.4 compiled with xen-scsiback and xen-scsifront compiled into the kernel, for both Dom0 and DomU.
>
>   My Ubuntu domU is running fine, however the scsi disk I want to pass through does not appear.
>
> in my config I have added the line:
> vscsi= ['3:0:0:0,0:0:0:0']
> I have also tried with:
> vscsi= ['/dev/sdb,0:0:0:0']
>
> but no matter which syntax I use I get the following error in dmesg log:
> xen-pvscsi: 3:0:0:0 doesn't exist
>
> I have no errors in the log files for the domU.
>
> Am I missing something in my configuration, perhaps some device hiding like for pci pass through?

The upstream pvscsi backend is using the target infrastructure. Until
the tools are aware of this you'll have to configure the device to
pass to a domain in Dom0 e.g. via a script:

#!/bin/bash
# usage:
# mkpvscsi device
# e.g. mkpvscsi /dev/sr0

DEV=$1

gen_uuid()
{
         cat /proc/sys/kernel/random/uuid | \
	awk '{print "naa.6001405" substr($1,1,8) substr($1,10,1);}'
}

TARG=`gen_uuid`
INIT=`gen_uuid`

NODE=`lsscsi | awk '$(NF) == "'$DEV'"
	{ print substr($1,2,length($1)-4); }'`
NAME=`echo $NODE | sed 's/:/_/g'`

modprobe configfs
mount -t configfs configfs /sys/kernel/config
modprobe xen-scsiback
modprobe target_core_mod
cd /sys/kernel/config/target

mkdir -p core/pscsi_0/$NAME
echo "$DEV" >core/pscsi_0/$NAME/udev_path

mkdir -p xen-pvscsi/$TARG/tpgt_0
echo "$NODE" >xen-pvscsi/$TARG/tpgt_0/param/alias
echo $INIT >xen-pvscsi/$TARG/tpgt_0/nexus
mkdir xen-pvscsi/$TARG/tpgt_0/lun/lun_0

cd xen-pvscsi/$TARG/tpgt_0/lun/lun_0
ln -s ../../../../../../target/core/pscsi_0/$NAME xen-pvscsi_port



-------------

After doing this you can use the xen tools to give the device to a domU.
Please note: this script is untested, I've used a simpler one which
fitted my needs (using targetcli, which isn't available everywhere).
I have tested all single steps of the script above, though.

Happy testing,

Juergen

P.S.: If you are feeling adventurous you can try other target backends
       than pscsi, e.g. iscsi or fileio.

pvSCSI test

[Kristian Hagsted Rasmussen]

On Wednesday, February 4, 2015 05:41, Juergen Gross <jgross@suse.com> wrote:
> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
> Subject: Re: pvSCSI test
> 
> On 02/03/2015 07:16 PM, Kristian Hagsted Rasmussen wrote:
>>
>> Hi Olaf and Juergen
>>
>>
>> I am interested in testing pvSCSI as I have a system were it would be ideal.
>> I have tried to apply this patch "http://marc.info/?l=xen-devel&m=139885599019457&w=2" called "libbxl: add support for pvscsi, iteration 1", to my xen-4.5 tree.
>> I am using kernel 3.18.4 compiled with xen-scsiback and xen-scsifront compiled into the kernel, for both Dom0 and DomU.
>>
>>   My Ubuntu domU is running fine, however the scsi disk I want to pass through does not appear.
>>
>> in my config I have added the line:
>> vscsi= ['3:0:0:0,0:0:0:0']
>> I have also tried with:
>> vscsi= ['/dev/sdb,0:0:0:0']
>>
>> but no matter which syntax I use I get the following error in dmesg log:
>> xen-pvscsi: 3:0:0:0 doesn't exist
>>
>> I have no errors in the log files for the domU.
>>
>> Am I missing something in my configuration, perhaps some device hiding like for pci pass through?
> 
> The upstream pvscsi backend is using the target infrastructure. Until
> the tools are aware of this you'll have to configure the device to
> pass to a domain in Dom0 e.g. via a script:
> 
> #!/bin/bash
> # usage:
> # mkpvscsi device
> # e.g. mkpvscsi /dev/sr0
> 
> DEV=$1
> 
> gen_uuid()
> {
>          cat /proc/sys/kernel/random/uuid | \
>         awk '{print "naa.6001405" substr($1,1,8) substr($1,10,1);}'
> }
> 
> TARG=`gen_uuid`
> INIT=`gen_uuid`
> 
> NODE=`lsscsi | awk '$(NF) == "'$DEV'"
>         { print substr($1,2,length($1)-4); }'`
> NAME=`echo $NODE | sed 's/:/_/g'`
> 

I used targetcli to create the pscsi target, so 
skipped from here

> modprobe configfs
> mount -t configfs configfs /sys/kernel/config
> modprobe xen-scsiback
> modprobe target_core_mod

to here 

> cd /sys/kernel/config/target
> 

and here to 

> mkdir -p core/pscsi_0/$NAME
> echo "$DEV" >core/pscsi_0/$NAME/udev_path
> 
to here

> mkdir -p xen-pvscsi/$TARG/tpgt_0
> echo "$NODE" >xen-pvscsi/$TARG/tpgt_0/param/alias
> echo $INIT >xen-pvscsi/$TARG/tpgt_0/nexus
> mkdir xen-pvscsi/$TARG/tpgt_0/lun/lun_0
> 
> cd xen-pvscsi/$TARG/tpgt_0/lun/lun_0
> ln -s ../../../../../../target/core/pscsi_0/$NAME xen-pvscsi_port
> 
> 
> 
> -------------
> 
> After doing this you can use the xen tools to give the device to a domU.
> Please note: this script is untested, I've used a simpler one which
> fitted my needs (using targetcli, which isn't available everywhere).
> I have tested all single steps of the script above, though.

Thanks for the script, I had some problems with the symlinking when
running the script, so tried to make the pscsi in targetcli, and then
shortening the script to skip all script lines concerning the core dir.
This made the script working for me, however I still get:
xen-pvscsi: 3:0:0:0 doesn't exist
error in dmesg log.

Are you using another patch for the xen-tree than the one I listed above?

Thanks Kristian

> Happy testing,
> 
> Juergen
> 
> P.S.: If you are feeling adventurous you can try other target backends
>        than pscsi, e.g. iscsi or fileio.

Re: pvSCSI test

[Juergen Gross]

On 02/06/2015 10:32 AM, Kristian Hagsted Rasmussen wrote:
> On Wednesday, February 4, 2015 05:41, Juergen Gross <jgross@suse.com> wrote:
>> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
>> Subject: Re: pvSCSI test
>>
>> On 02/03/2015 07:16 PM, Kristian Hagsted Rasmussen wrote:
>>>
>>> Hi Olaf and Juergen
>>>
>>>
>>> I am interested in testing pvSCSI as I have a system were it would be ideal.
>>> I have tried to apply this patch "http://marc.info/?l=xen-devel&m=139885599019457&w=2" called "libbxl: add support for pvscsi, iteration 1", to my xen-4.5 tree.
>>> I am using kernel 3.18.4 compiled with xen-scsiback and xen-scsifront compiled into the kernel, for both Dom0 and DomU.
>>>
>>>    My Ubuntu domU is running fine, however the scsi disk I want to pass through does not appear.
>>>
>>> in my config I have added the line:
>>> vscsi= ['3:0:0:0,0:0:0:0']
>>> I have also tried with:
>>> vscsi= ['/dev/sdb,0:0:0:0']
>>>
>>> but no matter which syntax I use I get the following error in dmesg log:
>>> xen-pvscsi: 3:0:0:0 doesn't exist
>>>
>>> I have no errors in the log files for the domU.
>>>
>>> Am I missing something in my configuration, perhaps some device hiding like for pci pass through?
>>
>> The upstream pvscsi backend is using the target infrastructure. Until
>> the tools are aware of this you'll have to configure the device to
>> pass to a domain in Dom0 e.g. via a script:
>>
>> #!/bin/bash
>> # usage:
>> # mkpvscsi device
>> # e.g. mkpvscsi /dev/sr0
>>
>> DEV=$1
>>
>> gen_uuid()
>> {
>>           cat /proc/sys/kernel/random/uuid | \
>>          awk '{print "naa.6001405" substr($1,1,8) substr($1,10,1);}'
>> }
>>
>> TARG=`gen_uuid`
>> INIT=`gen_uuid`
>>
>> NODE=`lsscsi | awk '$(NF) == "'$DEV'"
>>          { print substr($1,2,length($1)-4); }'`
>> NAME=`echo $NODE | sed 's/:/_/g'`
>>
>
> I used targetcli to create the pscsi target, so
> skipped from here
>
>> modprobe configfs
>> mount -t configfs configfs /sys/kernel/config
>> modprobe xen-scsiback
>> modprobe target_core_mod
>
> to here
>
>> cd /sys/kernel/config/target
>>
>
> and here to
>
>> mkdir -p core/pscsi_0/$NAME
>> echo "$DEV" >core/pscsi_0/$NAME/udev_path
>>
> to here
>
>> mkdir -p xen-pvscsi/$TARG/tpgt_0
>> echo "$NODE" >xen-pvscsi/$TARG/tpgt_0/param/alias
>> echo $INIT >xen-pvscsi/$TARG/tpgt_0/nexus
>> mkdir xen-pvscsi/$TARG/tpgt_0/lun/lun_0
>>
>> cd xen-pvscsi/$TARG/tpgt_0/lun/lun_0
>> ln -s ../../../../../../target/core/pscsi_0/$NAME xen-pvscsi_port
>>
>>
>>
>> -------------
>>
>> After doing this you can use the xen tools to give the device to a domU.
>> Please note: this script is untested, I've used a simpler one which
>> fitted my needs (using targetcli, which isn't available everywhere).
>> I have tested all single steps of the script above, though.
>
> Thanks for the script, I had some problems with the symlinking when
> running the script, so tried to make the pscsi in targetcli, and then
> shortening the script to skip all script lines concerning the core dir.
> This made the script working for me, however I still get:
> xen-pvscsi: 3:0:0:0 doesn't exist
> error in dmesg log.
>
> Are you using another patch for the xen-tree than the one I listed above?

I've used xm to get the drivers working, not xl. I wanted to have only
one changed component during tests in order to know which component is
failing. :-)

What are the contents of

/sys/kernel/config/target/xen-pvscsi/$TARG/tpgt_0/param/alias

($TARG replaced by the UUID generated above, of course)?

This should be "3:0:0:0" in your case. That's where the backend is
looking for the match from xenstore.

And the final symlink in the script is required. $NAME can be anything,
but has to match the pscsi name, of course.

Juergen

Re: pvSCSI test

[Kristian Hagsted Rasmussen]

On Friday, February 6, 2015 10:57, Juergen Gross <jgross@suse.com> wrote:
> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
> Subject: Re: pvSCSI test
> 
> On 02/06/2015 10:32 AM, Kristian Hagsted Rasmussen wrote:
>> On Wednesday, February 4, 2015 05:41, Juergen Gross <jgross@suse.com> wrote:
>>> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
>>> Subject: Re: pvSCSI test
>>>
>>> On 02/03/2015 07:16 PM, Kristian Hagsted Rasmussen wrote:
>>>>
>>>> Hi Olaf and Juergen
>>>>
>>>>
>>>> I am interested in testing pvSCSI as I have a system were it would be ideal.
>>>> I have tried to apply this patch "http://marc.info/?l=xen-devel&m=139885599019457&w=2" called "libbxl: add support for pvscsi, iteration 1", to my xen-4.5 tree.
>>>> I am using kernel 3.18.4 compiled with xen-scsiback and xen-scsifront compiled into the kernel, for both Dom0 and DomU.

I have by the way changed my kernel config, so that xen-scsiback, xen-scsifront, target_core_mod is compiled as modules.

snip 

> 
> I've used xm to get the drivers working, not xl. I wanted to have only
> one changed component during tests in order to know which component is
> failing. :-)
> 
> What are the contents of
> 
> /sys/kernel/config/target/xen-pvscsi/$TARG/tpgt_0/param/alias
> 
> ($TARG replaced by the UUID generated above, of course)?
> 
> This should be "3:0:0:0" in your case. That's where the backend is
> looking for the match from xenstore.
> 
> And the final symlink in the script is required. $NAME can be anything,
> but has to match the pscsi name, of course.

After some more fiddling around, I believe the configuration should be okay. My /etc/target/xen-pvscsi_start.sh look like this:

modprobe xen-scsiback
mkdir /sys/kernel/config/target/xen-pvscsi
mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0
echo naa.6001405708ab297e > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/nexus
#### xen-pvscsi Target Ports
mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0
ln -s /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0/../../../../../../target/core/iblock_0/3_0_0_0 /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0/xen-pvscsi_port
#### Attributes for xen-pvscsi Target Portal Group
#### Parameters for xen-pvscsi Target Portal Group
echo "3:0:0:0" > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/param/alias

And
cat /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/param/alias
returns 3:0:0:0 as expected.

If I understand this correctly, it is xen-scsiback that reads the target configuration from ConfigFS and hence the problem that xen-pvscsi cannot find the device has nothing to do with which toolstack is used?


If you are willing to part with the script you used together with targetcli, I would be more then happy to try that out.

Best Kristian

Re: pvSCSI test

[Juergen Gross]

On 02/06/2015 03:02 PM, Kristian Hagsted Rasmussen wrote:
>
> On Friday, February 6, 2015 10:57, Juergen Gross <jgross@suse.com> wrote:
>> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
>> Subject: Re: pvSCSI test
>>
>> On 02/06/2015 10:32 AM, Kristian Hagsted Rasmussen wrote:
>>> On Wednesday, February 4, 2015 05:41, Juergen Gross <jgross@suse.com> wrote:
>>>> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
>>>> Subject: Re: pvSCSI test
>>>>
>>>> On 02/03/2015 07:16 PM, Kristian Hagsted Rasmussen wrote:
>>>>>
>>>>> Hi Olaf and Juergen
>>>>>
>>>>>
>>>>> I am interested in testing pvSCSI as I have a system were it would be ideal.
>>>>> I have tried to apply this patch "http://marc.info/?l=xen-devel&m=139885599019457&w=2" called "libbxl: add support for pvscsi, iteration 1", to my xen-4.5 tree.
>>>>> I am using kernel 3.18.4 compiled with xen-scsiback and xen-scsifront compiled into the kernel, for both Dom0 and DomU.
>
> I have by the way changed my kernel config, so that xen-scsiback, xen-scsifront, target_core_mod is compiled as modules.
>
> snip
>
>>
>> I've used xm to get the drivers working, not xl. I wanted to have only
>> one changed component during tests in order to know which component is
>> failing. :-)
>>
>> What are the contents of
>>
>> /sys/kernel/config/target/xen-pvscsi/$TARG/tpgt_0/param/alias
>>
>> ($TARG replaced by the UUID generated above, of course)?
>>
>> This should be "3:0:0:0" in your case. That's where the backend is
>> looking for the match from xenstore.
>>
>> And the final symlink in the script is required. $NAME can be anything,
>> but has to match the pscsi name, of course.
>
> After some more fiddling around, I believe the configuration should be okay. My /etc/target/xen-pvscsi_start.sh look like this:
>
> modprobe xen-scsiback
> mkdir /sys/kernel/config/target/xen-pvscsi
> mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0
> echo naa.6001405708ab297e > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/nexus
> #### xen-pvscsi Target Ports
> mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0
> ln -s /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0/../../../../../../target/core/iblock_0/3_0_0_0 /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0/xen-pvscsi_port

iblock_0?

You are not using pscsi, but iblock. Is that on purpose? I have tested
pscsi and fileio only.

What does lsscsi tell you after adding the device via targetcli? I
suppose you see a new scsi target you should use instead of 3:0:0:0
(that's what I did in the fileio case).

> #### Attributes for xen-pvscsi Target Portal Group
> #### Parameters for xen-pvscsi Target Portal Group
> echo "3:0:0:0" > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/param/alias
>
> And
> cat /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/param/alias
> returns 3:0:0:0 as expected.
>
> If I understand this correctly, it is xen-scsiback that reads the target configuration from ConfigFS and hence the problem that xen-pvscsi cannot find the device has nothing to do with which toolstack is used?

Yes and no. In theory the backend would accept anything from xenstore
which it can find in configfs. The toolstack however will only write
values into the xenstore it believes are valid SCSI devices.

> If you are willing to part with the script you used together with targetcli, I would be more then happy to try that out.

I did my first tests with fileio on a machine I have no longer access
to. after I got it running I changed to my local test machine and pscsi.
Here I did the targetcli stuff manually and verified afterwards that
the single steps I put in my script were working. So the script I gave
you is basically the documentation of the manual setup I used.

Juergen

Re: pvSCSI test

[Kristian Hagsted Rasmussen]

On Friday, February 6, 2015 15:25, Juergen Gross <jgross@suse.com> wrote:
> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] pvSCSI test
> 

>> After some more fiddling around, I believe the configuration should be okay. My /etc/target/xen-pvscsi_start.sh look like this:
>>
>> modprobe xen-scsiback
>> mkdir /sys/kernel/config/target/xen-pvscsi
>> mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0
>> echo naa.6001405708ab297e > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/nexus
>> #### xen-pvscsi Target Ports
>> mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0
>> ln -s /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0/../../../../../../target/core/iblock_0/3_0_0_0 /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0/xen-pvscsi_port
> 
> iblock_0?
 
Sorry the result is the same if I use pscsi, I just tried iblock as it was a harddisk I tried to pass through and lio documentation advice against using pscsi for hard drives.

I just now tried to pass through a bluray drive on another machine, however I get the exact same error. What I noticed, however, was that the error only appears if the DomU kernel supports xen-scsifront. I hazzard a guess that this is because the backend only tries to connect to the device if the frontend makes a call for it.

In xenstore I have the following entry for vscsi under /local/domain/0/backend:
vscsi = ""
     3 = ""
      0 = ""
       frontend = "/local/domain/3/device/vscsi/0"
       frontend-id = "3"
       online = "1"
       state = "4"
       feature-host = "0"
       vscsi-devs = ""
        dev-0 = ""
         p-dev = "3:0:0:0"
         v-dev = "0:0:0:0"
         state = "6"
       feature-sg-grant = "128"

As I am no expert I am not sure about this, but shouldn't there be a path to the device, more then just the p-dev statement?

Do I have to manually add the device to xenstore?

If you do not feel for answering more of my questions please feel free to say so, I am just interested in this work and really look forward to its inclusion in xen.

/Kristian

> You are not using pscsi, but iblock. Is that on purpose? I have tested
> pscsi and fileio only.
> 
> What does lsscsi tell you after adding the device via targetcli? I
> suppose you see a new scsi target you should use instead of 3:0:0:0
> (that's what I did in the fileio case).
> 
>> #### Attributes for xen-pvscsi Target Portal Group
>> #### Parameters for xen-pvscsi Target Portal Group
>> echo "3:0:0:0" > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/param/alias
>>
>> And
>> cat /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/param/alias
>> returns 3:0:0:0 as expected.
>>
>> If I understand this correctly, it is xen-scsiback that reads the target configuration from ConfigFS and hence the problem that xen-pvscsi cannot find the device has nothing to do with which toolstack is used?
> 
> Yes and no. In theory the backend would accept anything from xenstore
> which it can find in configfs. The toolstack however will only write
> values into the xenstore it believes are valid SCSI devices.
> 
>> If you are willing to part with the script you used together with targetcli, I would be more then happy to try that out.
> 
> I did my first tests with fileio on a machine I have no longer access
> to. after I got it running I changed to my local test machine and pscsi.
> Here I did the targetcli stuff manually and verified afterwards that
> the single steps I put in my script were working. So the script I gave
> you is basically the documentation of the manual setup I used.
> 
> Juergen
> 

Re: pvSCSI test

[Juergen Gross]

On 02/06/2015 09:33 PM, Kristian Hagsted Rasmussen wrote:
>
> On Friday, February 6, 2015 15:25, Juergen Gross <jgross@suse.com> wrote:
>> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
>> Subject: Re: [Xen-devel] pvSCSI test
>>
>
>>> After some more fiddling around, I believe the configuration should be okay. My /etc/target/xen-pvscsi_start.sh look like this:
>>>
>>> modprobe xen-scsiback
>>> mkdir /sys/kernel/config/target/xen-pvscsi
>>> mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0
>>> echo naa.6001405708ab297e > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/nexus
>>> #### xen-pvscsi Target Ports
>>> mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0
>>> ln -s /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0/../../../../../../target/core/iblock_0/3_0_0_0 /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0/xen-pvscsi_port
>>
>> iblock_0?
>
> Sorry the result is the same if I use pscsi, I just tried iblock as it was a harddisk I tried to pass through and lio documentation advice against using pscsi for hard drives.
>
> I just now tried to pass through a bluray drive on another machine, however I get the exact same error. What I noticed, however, was that the error only appears if the DomU kernel supports xen-scsifront. I hazzard a guess that this is because the backend only tries to connect to the device if the frontend makes a call for it.
>
> In xenstore I have the following entry for vscsi under /local/domain/0/backend:
> vscsi = ""
>       3 = ""
>        0 = ""
>         frontend = "/local/domain/3/device/vscsi/0"
>         frontend-id = "3"
>         online = "1"
>         state = "4"
>         feature-host = "0"
>         vscsi-devs = ""
>          dev-0 = ""
>           p-dev = "3:0:0:0"
>           v-dev = "0:0:0:0"
>           state = "6"
>         feature-sg-grant = "128"
>
> As I am no expert I am not sure about this, but shouldn't there be a path to the device, more then just the p-dev statement?

No, that's okay. The connection between p-dev and the drive is done
via the target infrastructure.

Something seems to be wrong with your link in configfs: the target
seems not to be active. Could you please check the link to be correct?
Please check whether the pscsi (or iblock) entry is active. This can
be done via the "ls" command in targetcli for example.

When I tested the pscsi entry in configfs switched to "active" when I
linked the xen-pvscsi entry to it.

> Do I have to manually add the device to xenstore?

I never did it. :-)


Juergen

>
> If you do not feel for answering more of my questions please feel free to say so, I am just interested in this work and really look forward to its inclusion in xen.
>
> /Kristian
>
>> You are not using pscsi, but iblock. Is that on purpose? I have tested
>> pscsi and fileio only.
>>
>> What does lsscsi tell you after adding the device via targetcli? I
>> suppose you see a new scsi target you should use instead of 3:0:0:0
>> (that's what I did in the fileio case).
>>
>>> #### Attributes for xen-pvscsi Target Portal Group
>>> #### Parameters for xen-pvscsi Target Portal Group
>>> echo "3:0:0:0" > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/param/alias
>>>
>>> And
>>> cat /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/param/alias
>>> returns 3:0:0:0 as expected.
>>>
>>> If I understand this correctly, it is xen-scsiback that reads the target configuration from ConfigFS and hence the problem that xen-pvscsi cannot find the device has nothing to do with which toolstack is used?
>>
>> Yes and no. In theory the backend would accept anything from xenstore
>> which it can find in configfs. The toolstack however will only write
>> values into the xenstore it believes are valid SCSI devices.
>>
>>> If you are willing to part with the script you used together with targetcli, I would be more then happy to try that out.
>>
>> I did my first tests with fileio on a machine I have no longer access
>> to. after I got it running I changed to my local test machine and pscsi.
>> Here I did the targetcli stuff manually and verified afterwards that
>> the single steps I put in my script were working. So the script I gave
>> you is basically the documentation of the manual setup I used.
>>
>> Juergen
>>

Re: pvSCSI test

[Kristian Hagsted Rasmussen]

On Monday, February 9, 2015 07:02, Juergen Gross <jgross@suse.com> wrote:
> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] pvSCSI test

snip

> 
> No, that's okay. The connection between p-dev and the drive is done
> via the target infrastructure.
> 
> Something seems to be wrong with your link in configfs: the target
> seems not to be active. Could you please check the link to be correct?
> Please check whether the pscsi (or iblock) entry is active. This can
> be done via the "ls" command in targetcli for example.
> 

In targetcli, ls returns:

o- / ......................................................................................................................... [...]
  o- backstores .............................................................................................................. [...]
  | o- fileio ................................................................................................... [0 Storage Object]
  | o- iblock ................................................................................................... [0 Storage Object]
  | o- pscsi .................................................................................................... [1 Storage Object]
  | | o- 3:0:0:0 .............................................................................................. [/dev/sdb activated]
  | o- rd_dr .................................................................................................... [0 Storage Object]
  | o- rd_mcp ................................................................................................... [0 Storage Object]
  o- ib_srpt ........................................................................................................... [0 Targets]
  o- iscsi ............................................................................................................. [0 Targets]
  o- loopback .......................................................................................................... [0 Targets]
  o- qla2xxx ........................................................................................................... [0 Targets]
  o- tcm_fc ............................................................................................................ [0 Targets]

And my script for starting xen-pvscsi is this:

modprobe xen-scsiback
mkdir /sys/kernel/config/target/xen-pvscsi
mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0
echo naa.6001405708ab297e > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/nexus
#### pvscsi Target Ports
mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0
ln -s /sys/kernel/config/target/core/pscsi_0/3:0:0:0 /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0/xen-pvscsi_port
#### Attributes for pvscsi Target Portal Group
#### Parameters for pvscsi Target Portal Group
echo "3:0:0:0" > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/param/alias

I hope you can spot my error, as I am a little lost right now.

> When I tested the pscsi entry in configfs switched to "active" when I
> linked the xen-pvscsi entry to it.
> 
>> Do I have to manually add the device to xenstore?
> 
> I never did it. :-)
> 
> 
> Juergen
> 
>>
>> If you do not feel for answering more of my questions please feel free to say so, I am just interested in this work and really look forward to its inclusion in xen.
>>
>> /Kristian
>>
>>> You are not using pscsi, but iblock. Is that on purpose? I have tested
>>> pscsi and fileio only.
>>>
>>> What does lsscsi tell you after adding the device via targetcli? I
>>> suppose you see a new scsi target you should use instead of 3:0:0:0
>>> (that's what I did in the fileio case).
>>>

I do not see more devices with lsscsi when I add and iBlock devices, however I also tested with a fileIO device, this does also not show up in lsscsi. However I can get it to show up by making a loopback entry in targetcli, this however does not change the outcome of my domain creation.

Best regards Kristian

Re: pvSCSI test

[Juergen Gross]

On 02/12/2015 05:43 PM, Kristian Hagsted Rasmussen wrote:
> On Monday, February 9, 2015 07:02, Juergen Gross <jgross@suse.com> wrote:
>> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
>> Subject: Re: [Xen-devel] pvSCSI test
>
> snip
>
>>
>> No, that's okay. The connection between p-dev and the drive is done
>> via the target infrastructure.
>>
>> Something seems to be wrong with your link in configfs: the target
>> seems not to be active. Could you please check the link to be correct?
>> Please check whether the pscsi (or iblock) entry is active. This can
>> be done via the "ls" command in targetcli for example.
>>
>
> In targetcli, ls returns:
>
> o- / ......................................................................................................................... [...]
>    o- backstores .............................................................................................................. [...]
>    | o- fileio ................................................................................................... [0 Storage Object]
>    | o- iblock ................................................................................................... [0 Storage Object]
>    | o- pscsi .................................................................................................... [1 Storage Object]
>    | | o- 3:0:0:0 .............................................................................................. [/dev/sdb activated]
>    | o- rd_dr .................................................................................................... [0 Storage Object]
>    | o- rd_mcp ................................................................................................... [0 Storage Object]
>    o- ib_srpt ........................................................................................................... [0 Targets]
>    o- iscsi ............................................................................................................. [0 Targets]
>    o- loopback .......................................................................................................... [0 Targets]
>    o- qla2xxx ........................................................................................................... [0 Targets]
>    o- tcm_fc ............................................................................................................ [0 Targets]
>
> And my script for starting xen-pvscsi is this:
>
> modprobe xen-scsiback
> mkdir /sys/kernel/config/target/xen-pvscsi
> mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0
> echo naa.6001405708ab297e > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/nexus
> #### pvscsi Target Ports
> mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0
> ln -s /sys/kernel/config/target/core/pscsi_0/3:0:0:0 /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0/xen-pvscsi_port
> #### Attributes for pvscsi Target Portal Group
> #### Parameters for pvscsi Target Portal Group
> echo "3:0:0:0" > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/param/alias
>
> I hope you can spot my error, as I am a little lost right now.

At least I have spotted man error, but I think I am to blame:

I've told you to write "3:0:0:0" to alias. This was wrong. It should
be only "3:0:0". The LUN number is not part of the alias to use.

Sorry for that,

Juergen

>
>> When I tested the pscsi entry in configfs switched to "active" when I
>> linked the xen-pvscsi entry to it.
>>
>>> Do I have to manually add the device to xenstore?
>>
>> I never did it. :-)
>>
>>
>> Juergen
>>
>>>
>>> If you do not feel for answering more of my questions please feel free to say so, I am just interested in this work and really look forward to its inclusion in xen.
>>>
>>> /Kristian
>>>
>>>> You are not using pscsi, but iblock. Is that on purpose? I have tested
>>>> pscsi and fileio only.
>>>>
>>>> What does lsscsi tell you after adding the device via targetcli? I
>>>> suppose you see a new scsi target you should use instead of 3:0:0:0
>>>> (that's what I did in the fileio case).
>>>>
>
> I do not see more devices with lsscsi when I add and iBlock devices, however I also tested with a fileIO device, this does also not show up in lsscsi. However I can get it to show up by making a loopback entry in targetcli, this however does not change the outcome of my domain creation.
>
> Best regards Kristian
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

Re: pvSCSI test

[Kristian Hagsted Rasmussen]

On Monday, February 16, 2015 07:32, Juergen Gross <jgross@suse.com> wrote:
> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] pvSCSI test
> 
> On 02/12/2015 05:43 PM, Kristian Hagsted Rasmussen wrote:
>> On Monday, February 9, 2015 07:02, Juergen Gross <jgross@suse.com> wrote:
>>> To: Kristian Hagsted Rasmussen; Olaf Hering; xen-devel@lists.xensource.com
>>> Subject: Re: [Xen-devel] pvSCSI test
>>
>> snip
>>
>>>
>>> No, that's okay. The connection between p-dev and the drive is done
>>> via the target infrastructure.
>>>
>>> Something seems to be wrong with your link in configfs: the target
>>> seems not to be active. Could you please check the link to be correct?
>>> Please check whether the pscsi (or iblock) entry is active. This can
>>> be done via the "ls" command in targetcli for example.
>>>
>>
>> In targetcli, ls returns:
>>
>> o- / ......................................................................................................................... [...]
>>    o- backstores .............................................................................................................. [...]
>>    | o- fileio ................................................................................................... [0 Storage Object]
>>    | o- iblock ................................................................................................... [0 Storage Object]
>>    | o- pscsi .................................................................................................... [1 Storage Object]
>>    | | o- 3:0:0:0 .............................................................................................. [/dev/sdb activated]
>>    | o- rd_dr .................................................................................................... [0 Storage Object]
>>    | o- rd_mcp ................................................................................................... [0 Storage Object]
>>    o- ib_srpt ........................................................................................................... [0 Targets]
>>    o- iscsi ............................................................................................................. [0 Targets]
>>    o- loopback .......................................................................................................... [0 Targets]
>>    o- qla2xxx ........................................................................................................... [0 Targets]
>>    o- tcm_fc ............................................................................................................ [0 Targets]
>>
>> And my script for starting xen-pvscsi is this:
>>
>> modprobe xen-scsiback
>> mkdir /sys/kernel/config/target/xen-pvscsi
>> mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0
>> echo naa.6001405708ab297e > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/nexus
>> #### pvscsi Target Ports
>> mkdir -p /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0
>> ln -s /sys/kernel/config/target/core/pscsi_0/3:0:0:0 /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/lun/lun_0/xen-pvscsi_port
>> #### Attributes for pvscsi Target Portal Group
>> #### Parameters for pvscsi Target Portal Group
>> echo "3:0:0:0" > /sys/kernel/config/target/xen-pvscsi/naa.600140512a981c66/tpgt_0/param/alias
>>
>> I hope you can spot my error, as I am a little lost right now.
> 
> At least I have spotted man error, but I think I am to blame:
> 
> I've told you to write "3:0:0:0" to alias. This was wrong. It should
> be only "3:0:0". The LUN number is not part of the alias to use.
>
This was spot on, now pvscsi works. The error might also have occured when I changed your scripts to get them working on my systems.

Now my only problem is that my bluray drive cannot play DVD's in a linux domU, probably because it is initialized wrong by the kernel. Dom0 list it as follows:

scsi 2:0:0:0: CD-ROM            Optiarc  BD ROM BC-5600S  1.82 PQ: 0 ANSI: 5
sr 2:0:0:0: [sr0] scsi3-mmc drive: 31x/31x writer dvd-ram cd/rw xa/form2 cdda tray
sr 2:0:0:0: Attached scsi CD-ROM sr0
sr 2:0:0:0: Attached scsi generic sg2 type 5

while the domU list it as follows:

scsi 2:0:0:0: CD-ROM            Optiarc  BD ROM BC-5600S  1.82 PQ: 0 ANSI: 5
sr 2:0:0:0: [sr0] scsi-1 drive
sr 2:0:0:0: Attached scsi CD-ROM sr0
sr 2:0:0:0: Attached scsi generic sg0 type 5

This is the exact same thing I see when I use iSCSI on linux, however iSCSI works on windows. So this is more of a problem related to the linux scsi stack than something to do with xen-pvscsi.

Thanks so much for the help, and thumbs up for the good work.

Best regards Kristian Hagsted Rasmussen

> Sorry for that,
> 
> Juergen