best userspace iscsi target lib?

best userspace iscsi target lib?

[Jeff Garzik]

Can someone recommend an open source userspace library that enables one 
to implement a userspace iSCSI target?

Thanks,

	Jeff

Re: best userspace iscsi target lib?

[FUJITA Tomonori]

On Mon, 10 Nov 2008 17:29:06 -0500
Jeff Garzik <jeff@garzik.org> wrote:

> Can someone recommend an open source userspace library that enables one 
> to implement a userspace iSCSI target?

I'm not sure there is an open source userspace library enables you to
easily implement your own userspace iSCSI target but there are some
userspace iSCSI target implementations, which might useful.

Some Linux distributions use:

http://stgt.berlios.de/

OpenSolaris also has one:

http://opensolaris.org/os/project/iscsitgt/

*BSD also have userspace iSCSI target implementations.

Re: best userspace iscsi target lib?

[Boaz Harrosh]

FUJITA Tomonori wrote:
> On Mon, 10 Nov 2008 17:29:06 -0500
> Jeff Garzik <jeff@garzik.org> wrote:
> 
>> Can someone recommend an open source userspace library that enables one 
>> to implement a userspace iSCSI target?
> 
> I'm not sure there is an open source userspace library enables you to
> easily implement your own userspace iSCSI target but there are some
> userspace iSCSI target implementations, which might useful.
> 
> Some Linux distributions use:
> 
> http://stgt.berlios.de/
> 

Look in this patch how we added an external OSD scsi-implementor to stgt:
http://git.open-osd.org/gitweb.cgi?p=osc-osd/tgt/.git;a=commitdiff;h=5f3b25ef217ee417d8afe255d1a9bc5ed7f25f79
The implementation itself is in an external library.

I find stgt very easy to be extended.

<snip>

Boaz

Re: best userspace iscsi target lib?

[Vladislav Bolkhovitin]

Jeff Garzik wrote:
> Can someone recommend an open source userspace library that enables one 
> to implement a userspace iSCSI target?

Check SCST (http://scst.sf.net). Few weeks ago scst_local module added 
to it ability to create user space target drivers using regular sg/bsg 
interface. It's at the moment under active development, so still has a 
number of issues, but they shouldn't prevent you from evaluating it. You 
can find the code in the SVN using command:

$ svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk

But there is a plenty of open source iSCSI targets in the world, 
including one fully implemented in user space, so I wonder, what's wrong 
with them to create another one?

Vlad

Re: best userspace iscsi target lib?

[Jeff Garzik]

Vladislav Bolkhovitin wrote:
> Jeff Garzik wrote:
>> Can someone recommend an open source userspace library that enables 
>> one to implement a userspace iSCSI target?
> 
> Check SCST (http://scst.sf.net). Few weeks ago scst_local module added 
> to it ability to create user space target drivers using regular sg/bsg 
> interface. It's at the moment under active development, so still has a 
> number of issues, but they shouldn't prevent you from evaluating it. You 
> can find the code in the SVN using command:
> 
> $ svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk
> 
> But there is a plenty of open source iSCSI targets in the world, 
> including one fully implemented in user space, so I wonder, what's wrong 
> with them to create another one?

None of them are massively distributed OSD and SBC targets, which are 
part of my experiments in distributed computing :)

	Jeff

Re: best userspace iscsi target lib?

[Bart Van Assche]

On Tue, Nov 11, 2008 at 1:43 PM, Jeff Garzik <jeff@garzik.org> wrote:
> None of them are massively distributed OSD and SBC targets, which are part
> of my experiments in distributed computing :)

Apparently you are performing some very interesting experiments ?

Bart.

Re: best userspace iscsi target lib?

[Boaz Harrosh]

Jeff Garzik wrote:
> Vladislav Bolkhovitin wrote:
>> Jeff Garzik wrote:
>>> Can someone recommend an open source userspace library that enables 
>>> one to implement a userspace iSCSI target?
>> Check SCST (http://scst.sf.net). Few weeks ago scst_local module added 
>> to it ability to create user space target drivers using regular sg/bsg 
>> interface. It's at the moment under active development, so still has a 
>> number of issues, but they shouldn't prevent you from evaluating it. You 
>> can find the code in the SVN using command:
>>
>> $ svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk
>>
>> But there is a plenty of open source iSCSI targets in the world, 
>> including one fully implemented in user space, so I wonder, what's wrong 
>> with them to create another one?
> 
> None of them are massively distributed OSD and SBC targets, which are 
> part of my experiments in distributed computing :)
> 

OK now it sounds even more interesting. Do you have any public
information/documentation on these experiments.

There might be some interesting links on http://open-osd.org

also look up at: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.1.4604
It's a project from UCSC that implements a local object storage SW. It might
look interesting for embedded systems that don't want to invest in a complete
file system, and might give better performance in implementation of OSD protocol.

> 	Jeff
> 

Boaz

Re: best userspace iscsi target lib?

[Jeff Garzik]

Boaz Harrosh wrote:
> Jeff Garzik wrote:
>> Vladislav Bolkhovitin wrote:
>>> Jeff Garzik wrote:
>>>> Can someone recommend an open source userspace library that enables 
>>>> one to implement a userspace iSCSI target?
>>> Check SCST (http://scst.sf.net). Few weeks ago scst_local module added 
>>> to it ability to create user space target drivers using regular sg/bsg 
>>> interface. It's at the moment under active development, so still has a 
>>> number of issues, but they shouldn't prevent you from evaluating it. You 
>>> can find the code in the SVN using command:
>>>
>>> $ svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk
>>>
>>> But there is a plenty of open source iSCSI targets in the world, 
>>> including one fully implemented in user space, so I wonder, what's wrong 
>>> with them to create another one?
>> None of them are massively distributed OSD and SBC targets, which are 
>> part of my experiments in distributed computing :)
>>
> 
> OK now it sounds even more interesting. Do you have any public
> information/documentation on these experiments.

If I am at the point of looking for userspace target libraries, then you 
can be sure the SCSI piece does not yet exist...  :)  I'm working on 
various building blocks related to distributed computing in my spare 
time (gotta string together all those cheap SATA disks, ya know).

One of the experiments is replicated/shared, multi-node storage exported 
via an iSCSI interface.  Doing similar things by wedding NFS version 4 
(http://linux.yyz.us/projects/nfsv4.html) with distributed storage.

	Jeff

Re: best userspace iscsi target lib?

[Vladislav Bolkhovitin]

Jeff Garzik wrote:
> Vladislav Bolkhovitin wrote:
>> Jeff Garzik wrote:
>>> Can someone recommend an open source userspace library that enables 
>>> one to implement a userspace iSCSI target?
>> Check SCST (http://scst.sf.net). Few weeks ago scst_local module added 
>> to it ability to create user space target drivers using regular sg/bsg 
>> interface. It's at the moment under active development, so still has a 
>> number of issues, but they shouldn't prevent you from evaluating it. You 
>> can find the code in the SVN using command:
>>
>> $ svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk
>>
>> But there is a plenty of open source iSCSI targets in the world, 
>> including one fully implemented in user space, so I wonder, what's wrong 
>> with them to create another one?
> 
> None of them are massively distributed OSD and SBC targets, which are 
> part of my experiments in distributed computing :)

Yes, OSD is a good thing. I'd love to add an OSD backend in SCST.

> 	Jeff
> 
> 
> 
> 

Re: best userspace iscsi target lib?

[Jeff Garzik]

FUJITA Tomonori wrote:
> On Mon, 10 Nov 2008 17:29:06 -0500
> Jeff Garzik <jeff@garzik.org> wrote:
> 
>> Can someone recommend an open source userspace library that enables one 
>> to implement a userspace iSCSI target?
> 
> I'm not sure there is an open source userspace library enables you to
> easily implement your own userspace iSCSI target but there are some
> userspace iSCSI target implementations, which might useful.
> 
> Some Linux distributions use:
> 
> http://stgt.berlios.de/
> 
> OpenSolaris also has one:
> 
> http://opensolaris.org/os/project/iscsitgt/
> 
> *BSD also have userspace iSCSI target implementations.

Thanks much, I will take a look.  I have already found and read the code 
for NetBSD's iSCSI target.

I am mainly interested in 100% userspace solution, because in my case, 
the storage software (distributed goop) is also in userspace.  Something 
that exports through the kernel actually adds overhead, when a 
straight-to-sockets userspace implementation would be more efficient.

	Jeff

Re: best userspace iscsi target lib?

[FUJITA Tomonori]

On Tue, 11 Nov 2008 09:06:52 -0500
Jeff Garzik <jeff@garzik.org> wrote:

> FUJITA Tomonori wrote:
> > On Mon, 10 Nov 2008 17:29:06 -0500
> > Jeff Garzik <jeff@garzik.org> wrote:
> > 
> >> Can someone recommend an open source userspace library that enables one 
> >> to implement a userspace iSCSI target?
> > 
> > I'm not sure there is an open source userspace library enables you to
> > easily implement your own userspace iSCSI target but there are some
> > userspace iSCSI target implementations, which might useful.
> > 
> > Some Linux distributions use:
> > 
> > http://stgt.berlios.de/
> > 
> > OpenSolaris also has one:
> > 
> > http://opensolaris.org/os/project/iscsitgt/
> > 
> > *BSD also have userspace iSCSI target implementations.
> 
> Thanks much, I will take a look.  I have already found and read the code 
> for NetBSD's iSCSI target.
> 
> I am mainly interested in 100% userspace solution, because in my case, 
> the storage software (distributed goop) is also in userspace.  Something 
> that exports through the kernel actually adds overhead, when a 
> straight-to-sockets userspace implementation would be more efficient.

All the implementations that I referred to above are 100% userspace
solution (if I correctly remember *BSD's implementations).

You might think that stgt is a kernel space solution but it's 100%
userspace solution if you use only its iSCSI (or FCoE) support. As you
expect, it's just a userspace daemon. It doesn't require any fancy
kernel features.

About OSD support, I think that Pete and other osc.edu people
implemented iSCSI OSD target with stgt (though I've not merged their
OSD code to stgt or tried it yet):

http://mailman.open-osd.org/pipermail/osd-dev/2008-June/000047.html

Re: best userspace iscsi target lib?

[Boaz Harrosh]

FUJITA Tomonori wrote:
> On Tue, 11 Nov 2008 09:06:52 -0500
> Jeff Garzik <jeff@garzik.org> wrote:
> 
>> FUJITA Tomonori wrote:
>>> On Mon, 10 Nov 2008 17:29:06 -0500
>>> Jeff Garzik <jeff@garzik.org> wrote:
>>>
>>>> Can someone recommend an open source userspace library that enables one 
>>>> to implement a userspace iSCSI target?
>>> I'm not sure there is an open source userspace library enables you to
>>> easily implement your own userspace iSCSI target but there are some
>>> userspace iSCSI target implementations, which might useful.
>>>
>>> Some Linux distributions use:
>>>
>>> http://stgt.berlios.de/
>>>
>>> OpenSolaris also has one:
>>>
>>> http://opensolaris.org/os/project/iscsitgt/
>>>
>>> *BSD also have userspace iSCSI target implementations.
>> Thanks much, I will take a look.  I have already found and read the code 
>> for NetBSD's iSCSI target.
>>
>> I am mainly interested in 100% userspace solution, because in my case, 
>> the storage software (distributed goop) is also in userspace.  Something 
>> that exports through the kernel actually adds overhead, when a 
>> straight-to-sockets userspace implementation would be more efficient.
> 
> All the implementations that I referred to above are 100% userspace
> solution (if I correctly remember *BSD's implementations).
> 
> You might think that stgt is a kernel space solution but it's 100%
> userspace solution if you use only its iSCSI (or FCoE) support. As you
> expect, it's just a userspace daemon. It doesn't require any fancy
> kernel features.
> 
> About OSD support, I think that Pete and other osc.edu people
> implemented iSCSI OSD target with stgt (though I've not merged their
> OSD code to stgt or tried it yet):
> 

Do not use these URL's they are out of date. The OSC OSD project
is now hosted on open-osd.org git tree's.

Look here for latest git tree and instructions:
http://www.open-osd.org/bin/view/Main/OscOsdProject

We also have stgt + osd-patches hosted as a sub-module under
the OSC OSD target.

Boaz