NetApp vfiler example scripts

NetApp vfiler example scripts

[Stefan de Konink]

Hi,


For anyone who is interested; I created some basic scripts based on my
current iSCSI block script.

http://kinkrsoftware.nl/contrib/xen/block-netapp
http://kinkrsoftware.nl/contrib/xen/netapp-lun.py

Basically these two scripts allow you to start a vfiler with a qtree of
customers. You log in the NetApp upon boot, and can use:

netapp://customer1/disk1


It is work in progress... because I would prefer the vfiler IP to be in
that line too, but it seems the iSCSI portal IP != the NetApp shell IP.
Suggestions can be made on the list or as private reply. The helper
script is for persons that (like me) are not yet able to add a key to the
NetApp authorized_keys2 file.


Stefan

Re: NetApp vfiler example scripts

[Mark Williamson]

Cool idea.

Might be generally useful to people?  Is anybody else using this with a 
NetApp?  Any other features wanted in order to make this work?

If it's valuable it would be worth looking at adding it, or something like it, 
into the Xen tree as an additional block script.  We've still not exploited 
the ability to transparently support crazy networked block devices as much as 
we could :-)

Cheers,
Mark

On Monday 28 April 2008, Stefan de Konink wrote:
> Hi,
>
>
> For anyone who is interested; I created some basic scripts based on my
> current iSCSI block script.
>
> http://kinkrsoftware.nl/contrib/xen/block-netapp
> http://kinkrsoftware.nl/contrib/xen/netapp-lun.py
>
> Basically these two scripts allow you to start a vfiler with a qtree of
> customers. You log in the NetApp upon boot, and can use:
>
> netapp://customer1/disk1
>
>
> It is work in progress... because I would prefer the vfiler IP to be in
> that line too, but it seems the iSCSI portal IP != the NetApp shell IP.
> Suggestions can be made on the list or as private reply. The helper
> script is for persons that (like me) are not yet able to add a key to the
> NetApp authorized_keys2 file.
>
>
> Stefan
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel



-- 
Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)

Re: NetApp vfiler example scripts

[Stefan de Konink]

Mark Williamson schreef:
> Might be generally useful to people?  Is anybody else using this with a 
> NetApp?  Any other features wanted in order to make this work?

I have also created a patch for libvirt.

http://kinkrsoftware.nl/contrib/libvirt/pool.patch

Will most likely be included soon too. This allows to directly map the 
iscsi part:

<pool type="iscsi">
<name>netapp</name>
<source>
<host name="172.16.103.200"/>
<device 
path="iqn.1992-08.com.netapp:sn.118046347:vf.88fa4694-0ba6-11dd-b8a9-00a09807592f"/>
</source>
<target>
<path>/dev/disk/by-path</path>
</target>
</pool>


with:

<domain type='xen' >
   <name>Gentoo</name>
   <os>
     <type>linux</type>
     <kernel>/usr/lib/xen/boot/linux-2.6.20-xen-r6</kernel>
     <cmdline> root=/dev/xvda ro 
ip=85.17.131.4:1.2.3.4:85.17.131.253:255.255.255.0::eth0:off</cmdline>
   </os>
   <memory>131072</memory>
   <vcpu>1</vcpu>
   <devices>
     <disk type='pool'>
       <source pool='netapp' volume='lun-3' />
       <target dev='xvda'/>
     </disk>
     <interface type='bridge'>
       <source bridge='xenbr0'/>
       <mac address='aa:00:00:00:00:11'/>
       <script path='/etc/xen/scripts/vif-bridge'/>
     </interface>
   </devices>
</domain>


As you see this does not exploit the OnTap interface, but uses preknown 
lun numbers. More clean for Daniel and his crew.

But about NetApp; within Citrix, which persons do you talk to? The reply 
about Xen/NetApp that reached me didn't sound like exploiting the best 
of both worlds.


> If it's valuable it would be worth looking at adding it, or something like it, 
> into the Xen tree as an additional block script.  We've still not exploited 
> the ability to transparently support crazy networked block devices as much as 
> we could :-)

Please add my iscsi script too then, I have heard it is also in Suse ;) 
I've talked with the open-iscsi developer crew about making their code 
available in a shared library. This could eventually lead to a iscsi 
'program' instead of an iscsi script with heuristics.




Stefan

Re: NetApp vfiler example scripts

[Mark Williamson]

> As you see this does not exploit the OnTap interface, but uses preknown
> lun numbers. More clean for Daniel and his crew.

The OnTap interface is the SSH commandline interface you're using, right?  It 
does sound cleaner to avoid that; I wasn't 100% convinced by the use of that 
in the Xen script as it seemed like an "unexpected" behaviour for a low lever 
block script...  On the other hand, presumably it makes interacting with the 
filer a bit less friendly.

> But about NetApp; within Citrix, which persons do you talk to? The reply
> about Xen/NetApp that reached me didn't sound like exploiting the best
> of both worlds.

I don't know what the reply you've already seen is regarding but for the patch 
in question, it seems like it would be reasonable to just submit to the 
xen-devel list and request a merge.  I wouldn't be surprised if there's 
someone within Citrix working on NetApp integration (speculation on my part, 
since I'm not strictly a Citrix/XenSource employee and generally lack inside 
information) but equally well, they'd not necessarily be concerned with the 
OSS version of Xen.

> > If it's valuable it would be worth looking at adding it, or something
> > like it, into the Xen tree as an additional block script.  We've still
> > not exploited the ability to transparently support crazy networked block
> > devices as much as we could :-)
>
> Please add my iscsi script too then, I have heard it is also in Suse ;)
> I've talked with the open-iscsi developer crew about making their code
> available in a shared library. This could eventually lead to a iscsi
> 'program' instead of an iscsi script with heuristics.

Ah, I thought I'd seen an iSCSI patch whizz by on the mailing lists at some 
point - hadn't realised it was also you!

Have you done any further work on this or is the original patch posting on the 
2007-11-25 the most recent state of the patch?  Browsing the mailing archives 
I also see a script from Kurt Garloff mentioned.  Do you know if it is yours 
that is in SuSE or Kurt's?

It certainly seems like it'd be desirable to have these scripts in upstream 
Xen where possible, especially if distros are shipping them and people want 
to use them!

Cheers,
Mark

-- 
Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)

Re: [Xen-devel] NetApp vfiler example scripts

[Stefan de Konink]

Mark Williamson schreef:
>> As you see this does not exploit the OnTap interface, but uses preknown
>> lun numbers. More clean for Daniel and his crew.
> 
> The OnTap interface is the SSH commandline interface you're using, right?  It 
> does sound cleaner to avoid that; I wasn't 100% convinced by the use of that 
> in the Xen script as it seemed like an "unexpected" behaviour for a low lever 
> block script...  On the other hand, presumably it makes interacting with the 
> filer a bit less friendly.

Exactly it was the same compromise I have made for libvirt. I like the 
prototyping and freedom of the Xen scripts directory very much. But 
sometimes strictness in required.

>> But about NetApp; within Citrix, which persons do you talk to? The reply
>> about Xen/NetApp that reached me didn't sound like exploiting the best
>> of both worlds.
> 
> I don't know what the reply you've already seen is regarding but for the patch 
> in question, it seems like it would be reasonable to just submit to the 
> xen-devel list and request a merge.  I wouldn't be surprised if there's 
> someone within Citrix working on NetApp integration (speculation on my part, 
> since I'm not strictly a Citrix/XenSource employee and generally lack inside 
> information) but equally well, they'd not necessarily be concerned with the 
> OSS version of Xen.

 From what I understood of the forwarded mail was they 'implemented' 
Xen+iSCSI on their filers as 'local storage', in my opinion missing the 
point of the power of iSCSI in migration. Thus a Xen node gets one or 
two iSCSI disks, and thats it. Their limitation on LUNs (2048) is in my 
humble opinion a very bad thing, and a reason to advice something like 
OpenSolaris for the target job.

>>> If it's valuable it would be worth looking at adding it, or something
>>> like it, into the Xen tree as an additional block script.  We've still
>>> not exploited the ability to transparently support crazy networked block
>>> devices as much as we could :-)
>> Please add my iscsi script too then, I have heard it is also in Suse ;)
>> I've talked with the open-iscsi developer crew about making their code
>> available in a shared library. This could eventually lead to a iscsi
>> 'program' instead of an iscsi script with heuristics.
> 
> Ah, I thought I'd seen an iSCSI patch whizz by on the mailing lists at some 
> point - hadn't realised it was also you!

;)

> Have you done any further work on this or is the original patch posting on the 
> 2007-11-25 the most recent state of the patch?  Browsing the mailing archives 
> I also see a script from Kurt Garloff mentioned.  Do you know if it is yours 
> that is in SuSE or Kurt's?

The Suse guy that mailed me took my script as basis again, and changed 
the path of iscsiadm, trivial changes. Think if we can do an 
/usr/bin/env on it, it is distro safe.

http://kinkrsoftware.nl/contrib/xen/


> It certainly seems like it'd be desirable to have these scripts in upstream 
> Xen where possible, especially if distros are shipping them and people want 
> to use them!

I think stuff like iSCSI makes adoption more easy for people, but from 
my anti-xend feelings, I think it would be much better to invest time 
and resources to help Red Hat with their patches of Qemu+XEN, and only 
use libvirt as API.



Mark Johnson schreef:
 > I've been playing with adding extensions to phy: to handle dynamic
 > block devices. For iscsi, it's something like.
 >    phy:iscsi:[static]/<targetid>/<lun>/[serverIpAddr]
 >
 > e.g.
 >   disk = ['phy:/dev/zvol/dsk/tank/guests/snv89,0,w',
 > 
'phy:iscsi:static/iqn.1986-03.com.sun:02:17f34578-00a9-ef69-f3e9-b8a2896a4915/0/192.168.0.70,1,w']
 >
 > when the block hotplug script see's the phy:<ext>: it
 > looks for something like a block-<ext> script and runs
 > it if it can find it, else, defaults down normal path.

Isn't that what normally happens? Just type blabla: and it will go to 
block-blabla?

 > The iscsi hotplug script verifies/sets up the disk, then adds
 > a params-dynamic (equivalent to Stefan's node?) entry in
 > xenstore.  A small change to the disk backend looks for a
 > dynamic entry first, then defaults back to param if it can't find it.

I have based all my code on the shoulders of giants, the final code was 
one big mash up.



Stefan

Re: NetApp vfiler example scripts

[Mark Johnson]

On Mon, May 19, 2008 at 4:27 PM, Stefan de Konink <skinkie@xs4all.nl> wrote:
> Mark Williamson schreef:
> Mark Johnson schreef:
>> I've been playing with adding extensions to phy: to handle dynamic
>> block devices. For iscsi, it's something like.
>>    phy:iscsi:[static]/<targetid>/<lun>/[serverIpAddr]
>>
>> e.g.
>>   disk = ['phy:/dev/zvol/dsk/tank/guests/snv89,0,w',
>>
>> 'phy:iscsi:static/iqn.1986-03.com.sun:02:17f34578-00a9-ef69-f3e9-b8a2896a4915/0/192.168.0.70,1,w']
>>
>> when the block hotplug script see's the phy:<ext>: it
>> looks for something like a block-<ext> script and runs
>> it if it can find it, else, defaults down normal path.
>
> Isn't that what normally happens? Just type blabla: and it will go to
> block-blabla?

Not for Solaris.


>> The iscsi hotplug script verifies/sets up the disk, then adds
>> a params-dynamic (equivalent to Stefan's node?) entry in
>> xenstore.  A small change to the disk backend looks for a
>> dynamic entry first, then defaults back to param if it can't find it.
>
> I have based all my code on the shoulders of giants, the final code was one
> big mash up.

I assume you aren't using pygrub to boot off of a iscsi disk?



Thanks,

MRJ

Re: NetApp vfiler example scripts

[Stefan de Konink]

On Tue, 20 May 2008, Mark Johnson wrote:

> On Mon, May 19, 2008 at 4:27 PM, Stefan de Konink <skinkie@xs4all.nl> wrote:
> > Mark Williamson schreef:
> > Mark Johnson schreef:
> >> I've been playing with adding extensions to phy: to handle dynamic
> >> block devices. For iscsi, it's something like.
> >>    phy:iscsi:[static]/<targetid>/<lun>/[serverIpAddr]
> >>
> >> e.g.
> >>   disk = ['phy:/dev/zvol/dsk/tank/guests/snv89,0,w',
> >>
> >> 'phy:iscsi:static/iqn.1986-03.com.sun:02:17f34578-00a9-ef69-f3e9-b8a2896a4915/0/192.168.0.70,1,w']
> >>
> >> when the block hotplug script see's the phy:<ext>: it
> >> looks for something like a block-<ext> script and runs
> >> it if it can find it, else, defaults down normal path.
> >
> > Isn't that what normally happens? Just type blabla: and it will go to
> > block-blabla?
>
> Not for Solaris.

I guess the main problem is that the scripts are targeted for Linux with
sysfs.

> >> The iscsi hotplug script verifies/sets up the disk, then adds
> >> a params-dynamic (equivalent to Stefan's node?) entry in
> >> xenstore.  A small change to the disk backend looks for a
> >> dynamic entry first, then defaults back to param if it can't find it.
> >
> > I have based all my code on the shoulders of giants, the final code was one
> > big mash up.
>
> I assume you aren't using pygrub to boot off of a iscsi disk?

Exactly. I never understood the need for pygrub in the first place.


Stefan

Re: NetApp vfiler example scripts

[Mark Johnson]

On Mon, May 19, 2008 at 12:29 PM, Mark Williamson
<mark.williamson@cl.cam.ac.uk> wrote:
> Cool idea.
>
> Might be generally useful to people?  Is anybody else using this with a
> NetApp?  Any other features wanted in order to make this work?
>
> If it's valuable it would be worth looking at adding it, or something like it,
> into the Xen tree as an additional block script.  We've still not exploited
> the ability to transparently support crazy networked block devices as much as
> we could :-)

I've been playing with adding extensions to phy: to handle dynamic
block devices. For iscsi, it's something like.
   phy:iscsi:[static]/<targetid>/<lun>/[serverIpAddr]

e.g.
  disk = ['phy:/dev/zvol/dsk/tank/guests/snv89,0,w',
             'phy:iscsi:static/iqn.1986-03.com.sun:02:17f34578-00a9-ef69-f3e9-b8a2896a4915/0/192.168.0.70,1,w']

when the block hotplug script see's the phy:<ext>: it
looks for something like a block-<ext> script and runs
it if it can find it, else, defaults down normal path.

The iscsi hotplug script verifies/sets up the disk, then adds
a params-dynamic (equivalent to Stefan's node?) entry in
xenstore.  A small change to the disk backend looks for a
dynamic entry first, then defaults back to param if it can't find it.

I currently have a small change to xend to handle this for non
boot disks.  I still need to look at setting up the disks before pygrub
runs so you can boot off of it..

def _parse_uname(uname):
      fn = taptype = None
      if uname.find(":") != -1:
          (typ, fn) = uname.split(":", 1)
 -        if typ == "phy" and not fn.startswith("/"):
 -            fn = "/dev/%s" %(fn,)
 -        if typ == "tap":
+        if typ == "tap" or typ == "phy":
              (taptype, fn) = fn.split(":", 1)

The nice thing about this, is it's really easy to extend to other
block devices, etc.

e.g.
  phy:fc:...
  phy:npiv:...


MRJ
MRJ