[LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[Tony Asleson]

I would like to give a brief background on the storage management API
and library that I have been working on** and discuss the design and
implementation challenges that I have encountered thus far.

Most notable is the same topic Shyam Iyer brought up about proprietary
plug-in support (http://marc.info/?l=linux-scsi&m=132454179114038&w=2).
This very feature has influenced libStorageMgmt heavily and has raised a
number of discussions on design and licensing.

Discussion topics
* Scope (What a storage library should/should not do)
* Role (library position in management stack)
* Features (specific: must have to would be nice)
* Usage scenarios which highlight needed features
* Licensing (most appropriate open source license)
* Proprietary plug-in support


** http://sourceforge.net/apps/trac/libstoragemgmt

Regards,
Tony Asleson

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[Dan Williams]

On Fri, Jan 20, 2012 at 9:50 AM, Tony Asleson <tasleson@redhat.com> wrote:
> I would like to give a brief background on the storage management API
> and library that I have been working on** and discuss the design and
> implementation challenges that I have encountered thus far.
>
> Most notable is the same topic Shyam Iyer brought up about proprietary
> plug-in support (http://marc.info/?l=linux-scsi&m=132454179114038&w=2).
> This very feature has influenced libStorageMgmt heavily and has raised a
> number of discussions on design and licensing.
>
> Discussion topics
> * Scope (What a storage library should/should not do)
> * Role (library position in management stack)
> * Features (specific: must have to would be nice)
> * Usage scenarios which highlight needed features
> * Licensing (most appropriate open source license)
> * Proprietary plug-in support

Interesting, how does this relate, or not relate, to udisks in device-kit?

--
Dan

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[James Bottomley]

On Fri, 2012-01-20 at 11:50 -0600, Tony Asleson wrote:
> I would like to give a brief background on the storage management API
> and library that I have been working on** and discuss the design and
> implementation challenges that I have encountered thus far.
> 
> Most notable is the same topic Shyam Iyer brought up about proprietary
> plug-in support (http://marc.info/?l=linux-scsi&m=132454179114038&w=2).
> This very feature has influenced libStorageMgmt heavily and has raised a
> number of discussions on design and licensing.
> 
> Discussion topics
> * Scope (What a storage library should/should not do)
> * Role (library position in management stack)
> * Features (specific: must have to would be nice)
> * Usage scenarios which highlight needed features
> * Licensing (most appropriate open source license)
> * Proprietary plug-in support
> 
> 
> ** http://sourceforge.net/apps/trac/libstoragemgmt

Its a bit thin at 20 commits over 4 months.

It's also a bit insular ... the first thing you usually ask in open
source is what can I steal^Wborrow from other projects and how do I
recruit others to do the work for me.  Once you can actually answer
that, the list you have above mostly takes care of itself.

The first question is probably: is there anything we can liberate from
the Oracle storage API fisasco to help with this.  The second might be
what would it take to get vendors interested in doing the array plugin
glue.

James

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[Tony Asleson]

On 01/20/2012 05:28 PM, Dan Williams wrote:
> Interesting, how does this relate, or not relate, to udisks in device-kit?

The initial focus for libStorageMgmt is on managing external block level
storage arrays (LUN creation, deletion, re-sizing, replication and
mapping/un-mapping storage to initiators).  Once the logical unit is
available for the system, other libraries/tools will take over to get
the storage ready for use (Partitioning, FS creation etc.).

One could envision udisks as being a user of libStorageMgmt.  To
dynamically add/remove storage devices to a system and to leverage the
operations that can be completed on the array hardware vs. local system
(e.g. hardware array snapshot vs. lvm snapshot).

Regards,
Tony

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[Tony Asleson]

On 01/21/2012 08:57 AM, James Bottomley wrote:
> It's also a bit insular ... the first thing you usually ask in open
> source is what can I steal^Wborrow from other projects and how do I
> recruit others to do the work for me.

The project site is a little sparse on details about what we have looked
at and considered.  Ric Wheeler covered some of this during his
presentation in Prague, but I will place more information on the project
web site.

> The first question is probably: is there anything we can liberate
> from the Oracle storage API fisasco to help with this.

The Oracle storage connect library was evaluated and subsequently
rejected for the following reasons:

1. The Oracle storage connect library is dual licensed, GPL and a
proprietary Oracle license which allows proprietary use.  This allows
hardware vendors the ability to write proprietary plug-ins.  The design
of the library has the library user and plug-in executing in the same
address space.  Based on the information presented on plug-ins
(http://www.gnu.org/licenses/gpl-faq.html#GPLAndPlugins) this design
goes against the requirements of GPL license compliance.  Proprietary
plug-ins could be in compliance if they are implemented to execute in a
separate process.

2. At the time the Oracle storage connect library was being evaluated we
were unable to obtain a plug-in from a few different vendors.  Without
plug-ins the value of any library becomes greatly diminished.

> The second might be what would it take to get vendors interested in
> doing the array plugin glue.

By providing:
* Permissive license (LGPL)
* Easy to use out of process plug-in support to allow proprietary
plug-ins (IPC is abstracted)
* Language agnostic plug-in support (initial support is C and python)

We are hoping we can get hardware vendors interested in providing their
own plug-ins.  If anyone has additional ideas, we would certainly like
to discuss them.

Regards,
Tony

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[James Bottomley]

On Mon, 2012-01-23 at 10:21 -0600, Tony Asleson wrote:
> On 01/21/2012 08:57 AM, James Bottomley wrote:
> > It's also a bit insular ... the first thing you usually ask in open
> > source is what can I steal^Wborrow from other projects and how do I
> > recruit others to do the work for me.
> 
> The project site is a little sparse on details about what we have looked
> at and considered.  Ric Wheeler covered some of this during his
> presentation in Prague, but I will place more information on the project
> web site.

That would be good for those who didn't go to that presentation ...

> > The first question is probably: is there anything we can liberate
> > from the Oracle storage API fisasco to help with this.
> 
> The Oracle storage connect library was evaluated and subsequently
> rejected for the following reasons:
> 
> 1. The Oracle storage connect library is dual licensed, GPL and a
> proprietary Oracle license which allows proprietary use.  This allows
> hardware vendors the ability to write proprietary plug-ins.  The design
> of the library has the library user and plug-in executing in the same
> address space.  Based on the information presented on plug-ins
> (http://www.gnu.org/licenses/gpl-faq.html#GPLAndPlugins) this design
> goes against the requirements of GPL license compliance.  Proprietary
> plug-ins could be in compliance if they are implemented to execute in a
> separate process.

This is actually wrong thinking.  Almost every Open Source project has
an incompatible licence.  The way we work around this (once we've
determined there's something worth stealing) is to ask the project owner
for a compatible grant (in this case, some pieces under LGPL).  That's
how we share driver code between Linux and BSD for instance.

If Oracle is intent on pursuing an open core business model for this and
says "no" to the request, then you can say they're incompatible ... did
anyone ask them?

> 2. At the time the Oracle storage connect library was being evaluated we
> were unable to obtain a plug-in from a few different vendors.  Without
> plug-ins the value of any library becomes greatly diminished.

Why?  You have no plug ins for your project currently ... that doesn't
really diminish the value of it as an emerging project.

It sounds like you evaluated the Oracle project from an all or nothing
standpoint rather than a what can we learn/steal standpoint.

One of the big things we could co-opt from oracle looks to be vendor buy
in (they seem to be investing more in marketing than actual
engineering).  Vendors hate being confused or making choices, so one
easy way to bring them on board might be API compatibility with the
Oracle project.

> > The second might be what would it take to get vendors interested in
> > doing the array plugin glue.
> 
> By providing:
> * Permissive license (LGPL)
> * Easy to use out of process plug-in support to allow proprietary
> plug-ins (IPC is abstracted)
> * Language agnostic plug-in support (initial support is C and python)
> 
> We are hoping we can get hardware vendors interested in providing their
> own plug-ins.  If anyone has additional ideas, we would certainly like
> to discuss them.

So no actual vendors have provided any input yet?

James

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[Ric Wheeler]

On 01/24/2012 11:52 AM, James Bottomley wrote:
> On Mon, 2012-01-23 at 10:21 -0600, Tony Asleson wrote:
>> On 01/21/2012 08:57 AM, James Bottomley wrote:
>>> It's also a bit insular ... the first thing you usually ask in open
>>> source is what can I steal^Wborrow from other projects and how do I
>>> recruit others to do the work for me.
>> The project site is a little sparse on details about what we have looked
>> at and considered.  Ric Wheeler covered some of this during his
>> presentation in Prague, but I will place more information on the project
>> web site.
> That would be good for those who didn't go to that presentation ...
>
>>> The first question is probably: is there anything we can liberate
>>> from the Oracle storage API fisasco to help with this.
>> The Oracle storage connect library was evaluated and subsequently
>> rejected for the following reasons:
>>
>> 1. The Oracle storage connect library is dual licensed, GPL and a
>> proprietary Oracle license which allows proprietary use.  This allows
>> hardware vendors the ability to write proprietary plug-ins.  The design
>> of the library has the library user and plug-in executing in the same
>> address space.  Based on the information presented on plug-ins
>> (http://www.gnu.org/licenses/gpl-faq.html#GPLAndPlugins) this design
>> goes against the requirements of GPL license compliance.  Proprietary
>> plug-ins could be in compliance if they are implemented to execute in a
>> separate process.
> This is actually wrong thinking.  Almost every Open Source project has
> an incompatible licence.  The way we work around this (once we've
> determined there's something worth stealing) is to ask the project owner
> for a compatible grant (in this case, some pieces under LGPL).  That's
> how we share driver code between Linux and BSD for instance.
>
> If Oracle is intent on pursuing an open core business model for this and
> says "no" to the request, then you can say they're incompatible ... did
> anyone ask them?

I think that ignores the history behind the Oracle storage management project. 
We did look at the Oracle work, it certainly has been a long time coming and was 
not done in a community fashion as far as I can tell.

Putting the burden back on you, can you point to a single post about their 
project on a community list or name an individual (still employed by Oracle) 
that we can discuss with :) ?

We waited years for them to get to the dual license decision in the first place.

Having looked at their code, I think it is easier to write it - in a community 
way - the way we would have it done than to spend another year waiting on their 
legal team to change the license.

Clearly, using GPL for a library (not LGPL) is meant to push people into the 
commercial license....

Ric



>> 2. At the time the Oracle storage connect library was being evaluated we
>> were unable to obtain a plug-in from a few different vendors.  Without
>> plug-ins the value of any library becomes greatly diminished.
> Why?  You have no plug ins for your project currently ... that doesn't
> really diminish the value of it as an emerging project.
>
> It sounds like you evaluated the Oracle project from an all or nothing
> standpoint rather than a what can we learn/steal standpoint.
>
> One of the big things we could co-opt from oracle looks to be vendor buy
> in (they seem to be investing more in marketing than actual
> engineering).  Vendors hate being confused or making choices, so one
> easy way to bring them on board might be API compatibility with the
> Oracle project.
>
>>> The second might be what would it take to get vendors interested in
>>> doing the array plugin glue.
>> By providing:
>> * Permissive license (LGPL)
>> * Easy to use out of process plug-in support to allow proprietary
>> plug-ins (IPC is abstracted)
>> * Language agnostic plug-in support (initial support is C and python)
>>
>> We are hoping we can get hardware vendors interested in providing their
>> own plug-ins.  If anyone has additional ideas, we would certainly like
>> to discuss them.
> So no actual vendors have provided any input yet?
>
> James
>

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[James Bottomley]

On Tue, 2012-01-24 at 13:11 -0500, Ric Wheeler wrote:
> On 01/24/2012 11:52 AM, James Bottomley wrote:
> > On Mon, 2012-01-23 at 10:21 -0600, Tony Asleson wrote:
> >> On 01/21/2012 08:57 AM, James Bottomley wrote:
> >>> It's also a bit insular ... the first thing you usually ask in open
> >>> source is what can I steal^Wborrow from other projects and how do I
> >>> recruit others to do the work for me.
> >> The project site is a little sparse on details about what we have looked
> >> at and considered.  Ric Wheeler covered some of this during his
> >> presentation in Prague, but I will place more information on the project
> >> web site.
> > That would be good for those who didn't go to that presentation ...
> >
> >>> The first question is probably: is there anything we can liberate
> >>> from the Oracle storage API fisasco to help with this.
> >> The Oracle storage connect library was evaluated and subsequently
> >> rejected for the following reasons:
> >>
> >> 1. The Oracle storage connect library is dual licensed, GPL and a
> >> proprietary Oracle license which allows proprietary use.  This allows
> >> hardware vendors the ability to write proprietary plug-ins.  The design
> >> of the library has the library user and plug-in executing in the same
> >> address space.  Based on the information presented on plug-ins
> >> (http://www.gnu.org/licenses/gpl-faq.html#GPLAndPlugins) this design
> >> goes against the requirements of GPL license compliance.  Proprietary
> >> plug-ins could be in compliance if they are implemented to execute in a
> >> separate process.
> > This is actually wrong thinking.  Almost every Open Source project has
> > an incompatible licence.  The way we work around this (once we've
> > determined there's something worth stealing) is to ask the project owner
> > for a compatible grant (in this case, some pieces under LGPL).  That's
> > how we share driver code between Linux and BSD for instance.
> >
> > If Oracle is intent on pursuing an open core business model for this and
> > says "no" to the request, then you can say they're incompatible ... did
> > anyone ask them?
> 
> I think that ignores the history behind the Oracle storage management project. 
> We did look at the Oracle work, it certainly has been a long time coming and was 
> not done in a community fashion as far as I can tell.

I'm not denying it's been a royal screw up ... particularly with the
"sure we're releasing it any day now" message we got for two years.  But
that is partly due to the fact that the open source guy left the project
a while back.

> Putting the burden back on you, can you point to a single post about their 
> project on a community list or name an individual (still employed by Oracle) 
> that we can discuss with :) ?

Um, that question is rhetorical, right?  Or are you implying Martin
Petersen is no longer at oracle (which will come as news to him, I
think)?

> We waited years for them to get to the dual license decision in the first place.
> 
> Having looked at their code, I think it is easier to write it - in a community 
> way - the way we would have it done than to spend another year waiting on their 
> legal team to change the license.

However annoyed we might be with Oracle, reinventing an existing project
that already has committed vendor plugin contributors and thus forcing
vendors to choose which project to support isn't going to be hugely
helpful.  Assuming the worst about Oracle, what's the barriers to
reusing the plugins (the API is public and doesn't have a licensing
issue)?

> Clearly, using GPL for a library (not LGPL) is meant to push people into the 
> commercial license....

You're assuming a malign motive where there could be simple open source
ignorance.  I'll dig around and see if I can get a read on what they're
actually up to.

James

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[Hannes Reinecke]

On 01/23/2012 05:21 PM, Tony Asleson wrote:
> On 01/21/2012 08:57 AM, James Bottomley wrote:
[ .. ]
>> The second might be what would it take to get vendors interested in
>> doing the array plugin glue.
> 
> By providing:
> * Permissive license (LGPL)
> * Easy to use out of process plug-in support to allow proprietary
> plug-ins (IPC is abstracted)
> * Language agnostic plug-in support (initial support is C and python)
> 
> We are hoping we can get hardware vendors interested in providing their
> own plug-ins.  If anyone has additional ideas, we would certainly like
> to discuss them.
> 
The various hardware vendors already have the means of managing their
arrays, in most cases also providing a CLI.
I would be hoping we could draft the API support such that we can
use the existing tools, without the need of special plug-ins.

Cheers,

Hannes

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[Christoph Hellwig]

On Wed, Jan 25, 2012 at 02:09:35PM +0100, Hannes Reinecke wrote:
> The various hardware vendors already have the means of managing their
> arrays, in most cases also providing a CLI.
> I would be hoping we could draft the API support such that we can
> use the existing tools, without the need of special plug-ins.

Wrapping tools from libraries is more of a last resort than a proper
solution.  But if we can't get better APIs that's probably what we
will have to resort to if vendors don't cooperate.

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[Christoph Hellwig]

There's another project which is creating a very ad-hoc storage
management abstraction, the nova-volume code in openstack.  Right now
it supports iSCSI with a few vendor-specific extensions for management,
RBD and sheepdog, but it looks like the numbers of vendors that want
to be supported is growing.

See https://github.com/openstack/nova/blob/master/nova/volume/driver.py
and other files in that directory for the little bits that exist at the
moment.

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[Andy Grover]

On 01/25/2012 09:44 AM, Christoph Hellwig wrote:
> There's another project which is creating a very ad-hoc storage
> management abstraction, the nova-volume code in openstack.  Right now
> it supports iSCSI with a few vendor-specific extensions for management,
> RBD and sheepdog, but it looks like the numbers of vendors that want
> to be supported is growing.
> 
> See https://github.com/openstack/nova/blob/master/nova/volume/driver.py
> and other files in that directory for the little bits that exist at the
> moment.

<two minutes prior>
On 01/25/2012 09:42 AM, Christoph Hellwig wrote:
> Wrapping tools from libraries is more of a last resort than a proper
> solution.  But if we can't get better APIs that's probably what we
> will have to resort to if vendors don't cooperate.

Nova is wrapping cmdline tools, and all the vendors interested so far
are foss/cloudy.

This definitely shows others see a need for an API, but if we really
want big storage vendor buy-in, then a better (C?/sockets?) API and
working with non-foss code seems key.

Also, Nova code executes on the target machine, whereas libstoragemgmt
is running on the mgmt node and the plugins are responsible for
communication with the target node. (just wanted to point out another
difference.)

-- Andy

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[Christoph Hellwig]

On Thu, Jan 26, 2012 at 12:53:16PM -0800, Andy Grover wrote:
> Nova is wrapping cmdline tools, and all the vendors interested so far
> are foss/cloudy.

I'm defintively not advocating that code, quite contrary.  Just pointing
out that it exists.

> This definitely shows others see a need for an API, but if we really
> want big storage vendor buy-in, then a better (C?/sockets?) API and
> working with non-foss code seems key.
> 
> Also, Nova code executes on the target machine, whereas libstoragemgmt
> is running on the mgmt node and the plugins are responsible for
> communication with the target node. (just wanted to point out another
> difference.)

As said, I'm not fond of the nova code, and I'm defintively not fond
of placing a storage management abstraction into highlevel "cloud"
management code.  But this just shows where the lack of a low-level
abstraction gets us.

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[FUJITA Tomonori]

On Thu, 26 Jan 2012 12:53:16 -0800
Andy Grover <agrover@redhat.com> wrote:

> On 01/25/2012 09:44 AM, Christoph Hellwig wrote:
> > There's another project which is creating a very ad-hoc storage
> > management abstraction, the nova-volume code in openstack.  Right now
> > it supports iSCSI with a few vendor-specific extensions for management,
> > RBD and sheepdog, but it looks like the numbers of vendors that want
> > to be supported is growing.
> > 
> > See https://github.com/openstack/nova/blob/master/nova/volume/driver.py
> > and other files in that directory for the little bits that exist at the
> > moment.
> 
> <two minutes prior>
> On 01/25/2012 09:42 AM, Christoph Hellwig wrote:
> > Wrapping tools from libraries is more of a last resort than a proper
> > solution.  But if we can't get better APIs that's probably what we
> > will have to resort to if vendors don't cooperate.
> 
> Nova is wrapping cmdline tools

I think that some drivers are better a bit. For example, looks like
SolidFire uses the REST API to talk to their storage:

https://review.openstack.org/gitweb?p=openstack%2Fnova.git;a=commit;h=47a357bd2b46a2279a98ed726facd2b12fe0a0f5


, and all the vendors interested so far
> are foss/cloudy.

True (HP lefthand is supported though).


> This definitely shows others see a need for an API, but if we really
> want big storage vendor buy-in, then a better (C?/sockets?) API and
> working with non-foss code seems key.

Agreed.

Re: [Lsf-pc] [LSF/MM TOPIC] [ATTEND] Storage management (API & Library)

[Christoph Hellwig]

On Fri, Jan 27, 2012 at 06:12:34AM +0900, FUJITA Tomonori wrote:
> , and all the vendors interested so far
> > are foss/cloudy.
> 
> True (HP lefthand is supported though).

NetApp is adding one, too:
https://blueprints.launchpad.net/nova/+spec/netapp-volume-driver

Seems like cloud currently is the better buzzword than Linux, even if
this is all for Linux deployments, at least at the hypervisor level.