* Re: Slides from the pNFS BoF we delivered at LISA 11 - with attachment this time
2011-12-09 2:53 ` Rick Macklem
@ 2011-12-09 3:11 ` Sorin Faibish
2011-12-09 7:48 ` [nfsv4] " Tigran Mkrtchyan
2011-12-09 11:08 ` Boaz Harrosh
2 siblings, 0 replies; 11+ messages in thread
From: Sorin Faibish @ 2011-12-09 3:11 UTC (permalink / raw)
To: Rick Macklem; +Cc: honey, linux-nfs, nfsv4
On Thu, 08 Dec 2011 21:53:20 -0500, Rick Macklem <rmacklem@uoguelph.ca>
wrote:
> Thanks for the slides, Sorin.
>
> Just fyi, it was correct to not mention a FreeBSD client, since the
> 4.1 client code hasn't made it into any release yet. It is available
> as a patch against the FreeBSD9.0 kernel, in case anyone is interested:
> http://people.freebsd.org/~rmacklem/nfsv4.1-client/nfsv41.patch
>
> I am just starting to work on the logistics of getting this into the
> FreeBSD source tree, so it may be in a FreeBSD9.n release, where n > 0.
>
> Also, the above does not include any pNFS support. I am about ready to
> start coding pNFS client stuff, which brings me to...
> - It will be difficult for me to make much progress without any
> pNFS servers to test against. (The only NFSv4.1 server I have to
> test against is Fedora15 and I don't think it knows how to do pNFS?
> If there is a way to make Fedora15/16 (or any other freebie Linux
> distro) do pNFS, please let me know.
One suggestion is to try to use the pNFS for Lustre. I can send you the
patches or you can get them from Lustre tree. Let me know if this will be
good for you. We are already using it to test Linux file client as well as
Windows. Whamcloud demo it during SC11 and demo wire speed.
What I would also suggest is for you maybe to come to the Connectathon in
February and test the client against the other servers that you cannot have
as they are not products but prototypes.
>
> If others have suggestions on how I might gain access to pNFS servers
> for testing, I would appreciate hearing. (However, I am not funded to
> do any of this, so the suggestions can't include my spending $$$. I can
> make it to a Bakeathon when it happens to be close to where I am, so I
> can show up "on the cheap", like I did last June in Ann Arbor.)
Sure. The next BAT is at CITI so maybe you bring a client prototype
there.
/Sorin
>
> rick
>
> ----- Original Message -----
>> On Thu, 08 Dec 2011 19:40:19 -0500, Sorin Faibish <sfaibish@emc.com>
>> wrote:
>>
>> > Attached please find the slides that we presented yesterday at
>> > USENIX
>> > LISA 11
>> > conference in Boston. We would like to get more detailed input on
>> > other
>> > OS
>> > that will support pNFS server/client (this is for Rick as well) that
>> > were
>> > not mentioned in this slide set.
>> >
>> > /Sorin
>> >
>>
>>
>>
>> --
>> Best Regards
>> Sorin Faibish
>> Corporate Distinguished Engineer
>> Unified Storage Division
>>
>> EMC²
>> where information lives
>>
>> Phone: 508-435-1000 x 48545
>> Cellphone: 617-510-0422
>> Email : sfaibish@emc.com
>
>
--
Best Regards
Sorin Faibish
Corporate Distinguished Engineer
Unified Storage Division
EMC²
where information lives
Phone: 508-435-1000 x 48545
Cellphone: 617-510-0422
Email : sfaibish@emc.com
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [nfsv4] Slides from the pNFS BoF we delivered at LISA 11 - with attachment this time
2011-12-09 2:53 ` Rick Macklem
2011-12-09 3:11 ` Sorin Faibish
@ 2011-12-09 7:48 ` Tigran Mkrtchyan
2011-12-09 11:08 ` Boaz Harrosh
2 siblings, 0 replies; 11+ messages in thread
From: Tigran Mkrtchyan @ 2011-12-09 7:48 UTC (permalink / raw)
To: Rick Macklem; +Cc: Sorin Faibish, linux-nfs, honey, nfsv4
On Fri, Dec 9, 2011 at 3:53 AM, Rick Macklem <rmacklem@uoguelph.ca> wrote:
> Thanks for the slides, Sorin.
>
> Just fyi, it was correct to not mention a FreeBSD client, since the
> 4.1 client code hasn't made it into any release yet. It is available
> as a patch against the FreeBSD9.0 kernel, in case anyone is interested:
> http://people.freebsd.org/~rmacklem/nfsv4.1-client/nfsv41.patch
>
> I am just starting to work on the logistics of getting this into the
> FreeBSD source tree, so it may be in a FreeBSD9.n release, where n > 0.
>
> Also, the above does not include any pNFS support. I am about ready to
> start coding pNFS client stuff, which brings me to...
> - It will be difficult for me to make much progress without any
> pNFS servers to test against. (The only NFSv4.1 server I have to
> test against is Fedora15 and I don't think it knows how to do pNFS?
> If there is a way to make Fedora15/16 (or any other freebie Linux
> distro) do pNFS, please let me know.
>
> If others have suggestions on how I might gain access to pNFS servers
> for testing, I would appreciate hearing.
Hi Rick,
we have user space pNFS server written in java. It's part of dCache
project but can be used stand alone for testing. Requires JVM and disk
space to store data blocks:
1. checkout and build
$ hg clone http://www.dcache.org/~tigran/cgi-bin/hgwebdir.cgi/jpnfs dcache-nfs
$ cd dcache-nfs
$ mvn clean package
2. tweak nfs.properties:
put you host ip instead of 192.168.2.108
current setup will start mds and ds on the same host on ports 2049 and 2052.
File system is re-initialized on restart.
3. start it:
$ java -jar target/jpNFS-0.0.4-SNAPSHOT-jar-with-dependencies.jar oncrpcsvc.xml
4. test your client
Regards,
Tigran.
(However, I am not funded to
> do any of this, so the suggestions can't include my spending $$$. I can
> make it to a Bakeathon when it happens to be close to where I am, so I
> can show up "on the cheap", like I did last June in Ann Arbor.)
>
> rick
>
> ----- Original Message -----
>> On Thu, 08 Dec 2011 19:40:19 -0500, Sorin Faibish <sfaibish@emc.com>
>> wrote:
>>
>> > Attached please find the slides that we presented yesterday at
>> > USENIX
>> > LISA 11
>> > conference in Boston. We would like to get more detailed input on
>> > other
>> > OS
>> > that will support pNFS server/client (this is for Rick as well) that
>> > were
>> > not mentioned in this slide set.
>> >
>> > /Sorin
>> >
>>
>>
>>
>> --
>> Best Regards
>> Sorin Faibish
>> Corporate Distinguished Engineer
>> Unified Storage Division
>>
>> EMC²
>> where information lives
>>
>> Phone: 508-435-1000 x 48545
>> Cellphone: 617-510-0422
>> Email : sfaibish@emc.com
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [nfsv4] Slides from the pNFS BoF we delivered at LISA 11 - with attachment this time
2011-12-09 2:53 ` Rick Macklem
2011-12-09 3:11 ` Sorin Faibish
2011-12-09 7:48 ` [nfsv4] " Tigran Mkrtchyan
@ 2011-12-09 11:08 ` Boaz Harrosh
2011-12-12 13:08 ` Steve Dickson
2011-12-14 3:20 ` Rick Macklem
2 siblings, 2 replies; 11+ messages in thread
From: Boaz Harrosh @ 2011-12-09 11:08 UTC (permalink / raw)
To: Rick Macklem
Cc: Sorin Faibish, linux-nfs, honey, nfsv4, Steve Dickson,
Welch, Brent
On 12/09/2011 04:53 AM, Rick Macklem wrote:
> Thanks for the slides, Sorin.
>
> Just fyi, it was correct to not mention a FreeBSD client, since the
> 4.1 client code hasn't made it into any release yet. It is available
> as a patch against the FreeBSD9.0 kernel, in case anyone is interested:
> http://people.freebsd.org/~rmacklem/nfsv4.1-client/nfsv41.patch
>
> I am just starting to work on the logistics of getting this into the
> FreeBSD source tree, so it may be in a FreeBSD9.n release, where n > 0.
>
> Also, the above does not include any pNFS support. I am about ready to
> start coding pNFS client stuff, which brings me to...
> - It will be difficult for me to make much progress without any
> pNFS servers to test against. (The only NFSv4.1 server I have to
> test against is Fedora15 and I don't think it knows how to do pNFS?
> If there is a way to make Fedora15/16 (or any other freebie Linux
> distro) do pNFS, please let me know.
>
> If others have suggestions on how I might gain access to pNFS servers
> for testing, I would appreciate hearing. (However, I am not funded to
> do any of this, so the suggestions can't include my spending $$$. I can
> make it to a Bakeathon when it happens to be close to where I am, so I
> can show up "on the cheap", like I did last June in Ann Arbor.)
>
One way to go is the Linux pnfs project at linux-nfs.org start here:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree
It is an NFSD linux Kernel server with pNFS extensions destined to
be part of Linux at some future release.
You will need to compile your own Kernel, or alternatively this Kernel
is available in rpm form from Steve Dickson for some Fedoras:
http://steved.fedorapeople.org/repos/pnfs/
The easiest is to first use the what is called pnfs-exp which is an MDS
plus a single DS exporting any local file system.
Then you can also export a GFS2 read-only and an EXOFS which is a full
fixture pnfs-objects based cluster filesystem.
The Luster patch suggested to you is also based on above Benny's tree.
I have on my todo list a full osd (pnfs-objects) stack port to FreeBSD
But that's in the far future.
Meanwhile I'd suggest the pnfs-client-core to layout-driver API be closely
crafted to match that of the Linux LD API so it could be kept the same
code base. For example the pnfs-objects LD is mostly pure C code relying
on the ORE (Objects Raid Engine). The ORE is multy licensed library that
has ports to Many platforms including Linux-Kernel
Linux/FreeBSD/windows User-Mode and more of which only the Linux variants
actually run because of the need of the iscsi stack to support BiDi and
large CDB commands, Only supported by Linux.
Cheers
Boaz
> rick
>
> ----- Original Message -----
>> On Thu, 08 Dec 2011 19:40:19 -0500, Sorin Faibish <sfaibish@emc.com>
>> wrote:
>>
>>> Attached please find the slides that we presented yesterday at
>>> USENIX
>>> LISA 11
>>> conference in Boston. We would like to get more detailed input on
>>> other
>>> OS
>>> that will support pNFS server/client (this is for Rick as well) that
>>> were
>>> not mentioned in this slide set.
>>>
>>> /Sorin
>>>
>>
>>
>>
>> --
>> Best Regards
>> Sorin Faibish
>> Corporate Distinguished Engineer
>> Unified Storage Division
>>
>> EMC²
>> where information lives
>>
>> Phone: 508-435-1000 x 48545
>> Cellphone: 617-510-0422
>> Email : sfaibish@emc.com
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [nfsv4] Slides from the pNFS BoF we delivered at LISA 11 - with attachment this time
2011-12-09 11:08 ` Boaz Harrosh
@ 2011-12-12 13:08 ` Steve Dickson
2011-12-14 3:20 ` Rick Macklem
1 sibling, 0 replies; 11+ messages in thread
From: Steve Dickson @ 2011-12-12 13:08 UTC (permalink / raw)
To: Boaz Harrosh
Cc: Rick Macklem, Sorin Faibish, linux-nfs, honey, nfsv4,
Welch, Brent
On 12/09/2011 06:08 AM, Boaz Harrosh wrote:
> On 12/09/2011 04:53 AM, Rick Macklem wrote:
>> Thanks for the slides, Sorin.
>>
>> Just fyi, it was correct to not mention a FreeBSD client, since the
>> 4.1 client code hasn't made it into any release yet. It is available
>> as a patch against the FreeBSD9.0 kernel, in case anyone is interested:
>> http://people.freebsd.org/~rmacklem/nfsv4.1-client/nfsv41.patch
>>
>> I am just starting to work on the logistics of getting this into the
>> FreeBSD source tree, so it may be in a FreeBSD9.n release, where n > 0.
>>
>> Also, the above does not include any pNFS support. I am about ready to
>> start coding pNFS client stuff, which brings me to...
>> - It will be difficult for me to make much progress without any
>> pNFS servers to test against. (The only NFSv4.1 server I have to
>> test against is Fedora15 and I don't think it knows how to do pNFS?
>> If there is a way to make Fedora15/16 (or any other freebie Linux
>> distro) do pNFS, please let me know.
>>
>> If others have suggestions on how I might gain access to pNFS servers
>> for testing, I would appreciate hearing. (However, I am not funded to
>> do any of this, so the suggestions can't include my spending $$$. I can
>> make it to a Bakeathon when it happens to be close to where I am, so I
>> can show up "on the cheap", like I did last June in Ann Arbor.)
>>
>
> One way to go is the Linux pnfs project at linux-nfs.org start here:
> http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree
>
> It is an NFSD linux Kernel server with pNFS extensions destined to
> be part of Linux at some future release.
>
> You will need to compile your own Kernel, or alternatively this Kernel
> is available in rpm form from Steve Dickson for some Fedoras:
> http://steved.fedorapeople.org/repos/pnfs/
These are a bit crusty... Since pNFS is enabled in F16 kernels and
beyond, I would suggest for a recent kernel, either download one
from the rawhide repo or just grab one from here
http://koji.fedoraproject.org/koji/packageinfo?packageID=8
>
> The easiest is to first use the what is called pnfs-exp which is an MDS
> plus a single DS exporting any local file system.
Yes, this is by far the easiest... I've done some "hello world" testing
with this code and it seems fairly stable.. The code is on the
pnfsd-lexp branch in Benny's tree
git://linux-nfs.org/~bhalevy/linux-pnfs.git
Its a fairly simple process to port that code into a Fedora kernel rpm.
If that would help... just let me know...
steved.
>
> Then you can also export a GFS2 read-only and an EXOFS which is a full
> fixture pnfs-objects based cluster filesystem.
>
> The Luster patch suggested to you is also based on above Benny's tree.
>
> I have on my todo list a full osd (pnfs-objects) stack port to FreeBSD
> But that's in the far future.
> Meanwhile I'd suggest the pnfs-client-core to layout-driver API be closely
> crafted to match that of the Linux LD API so it could be kept the same
> code base. For example the pnfs-objects LD is mostly pure C code relying
> on the ORE (Objects Raid Engine). The ORE is multy licensed library that
> has ports to Many platforms including Linux-Kernel
> Linux/FreeBSD/windows User-Mode and more of which only the Linux variants
> actually run because of the need of the iscsi stack to support BiDi and
> large CDB commands, Only supported by Linux.
>
> Cheers
> Boaz
>
>> rick
>>
>> ----- Original Message -----
>>> On Thu, 08 Dec 2011 19:40:19 -0500, Sorin Faibish <sfaibish@emc.com>
>>> wrote:
>>>
>>>> Attached please find the slides that we presented yesterday at
>>>> USENIX
>>>> LISA 11
>>>> conference in Boston. We would like to get more detailed input on
>>>> other
>>>> OS
>>>> that will support pNFS server/client (this is for Rick as well) that
>>>> were
>>>> not mentioned in this slide set.
>>>>
>>>> /Sorin
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards
>>> Sorin Faibish
>>> Corporate Distinguished Engineer
>>> Unified Storage Division
>>>
>>> EMC²
>>> where information lives
>>>
>>> Phone: 508-435-1000 x 48545
>>> Cellphone: 617-510-0422
>>> Email : sfaibish@emc.com
>> _______________________________________________
>> nfsv4 mailing list
>> nfsv4@ietf.org
>> https://www.ietf.org/mailman/listinfo/nfsv4
>
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [nfsv4] Slides from the pNFS BoF we delivered at LISA 11 - with attachment this time
2011-12-09 11:08 ` Boaz Harrosh
2011-12-12 13:08 ` Steve Dickson
@ 2011-12-14 3:20 ` Rick Macklem
2011-12-15 15:42 ` Benny Halevy
1 sibling, 1 reply; 11+ messages in thread
From: Rick Macklem @ 2011-12-14 3:20 UTC (permalink / raw)
To: Boaz Harrosh; +Cc: linux-nfs, honey, Brent Welch, nfsv4, Steve Dickson
Boaz Harrosh wrote:
[good stuff snipped for brevity]
>
> I have on my todo list a full osd (pnfs-objects) stack port to FreeBSD
> But that's in the far future.
> Meanwhile I'd suggest the pnfs-client-core to layout-driver API be
> closely
> crafted to match that of the Linux LD API so it could be kept the same
> code base.
Am I correct to assume that fs/nfs/nfsfilelayout* implements this API
for the file layout case? If so, I'll try and build glue that can use
that code unchanged (or at least minimally changed).
And Peter (I see you're on the cc list), is it possible to get a copy
of these files without any GPL? It looks like they have the U of Michigan
copyright, which I think should be fine for the FreeBSD folks, but then
there is
MODULE_LICENSE("GPL");
in it, which I assume means it's GPL'd. (Basically, exactly the same file,
but released without this line, would make the FreeBSD folks happy, I think.)
(No rush on this, I can use GPL'd code, I just can't put it in the FreeBSD
source tree easily.) The same would apply to any other layout driver that
others, like Panasas or EMC might be willing to open source for FreeBSD.
And Boaz, thanks for the useful information. I'll probably start with
Fedora plus a patched kernel for a server to test against. (It sounds
like Steve is willing to help me out, since I haven't used "rpm" in
years and have never typed "git" in my life.:-)
Thanks everyone for your help with this, rick
> For example the pnfs-objects LD is mostly pure C code
> relying
> on the ORE (Objects Raid Engine). The ORE is multy licensed library
> that
> has ports to Many platforms including Linux-Kernel
> Linux/FreeBSD/windows User-Mode and more of which only the Linux
> variants
> actually run because of the need of the iscsi stack to support BiDi
> and
> large CDB commands, Only supported by Linux.
>
> Cheers
> Boaz
>
> > rick
> >
> > ----- Original Message -----
> >> On Thu, 08 Dec 2011 19:40:19 -0500, Sorin Faibish
> >> <sfaibish@emc.com>
> >> wrote:
> >>
> >>> Attached please find the slides that we presented yesterday at
> >>> USENIX
> >>> LISA 11
> >>> conference in Boston. We would like to get more detailed input on
> >>> other
> >>> OS
> >>> that will support pNFS server/client (this is for Rick as well)
> >>> that
> >>> were
> >>> not mentioned in this slide set.
> >>>
> >>> /Sorin
> >>>
> >>
> >>
> >>
> >> --
> >> Best Regards
> >> Sorin Faibish
> >> Corporate Distinguished Engineer
> >> Unified Storage Division
> >>
> >> EMC²
> >> where information lives
> >>
> >> Phone: 508-435-1000 x 48545
> >> Cellphone: 617-510-0422
> >> Email : sfaibish@emc.com
> > _______________________________________________
> > nfsv4 mailing list
> > nfsv4@ietf.org
> > https://www.ietf.org/mailman/listinfo/nfsv4
>
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [nfsv4] Slides from the pNFS BoF we delivered at LISA 11 - with attachment this time
2011-12-14 3:20 ` Rick Macklem
@ 2011-12-15 15:42 ` Benny Halevy
0 siblings, 0 replies; 11+ messages in thread
From: Benny Halevy @ 2011-12-15 15:42 UTC (permalink / raw)
To: Rick Macklem; +Cc: linux-nfs, Brent Welch, honey, Steve Dickson, nfsv4
[-- Attachment #1.1: Type: text/plain, Size: 3554 bytes --]
On Wed, Dec 14, 2011 at 5:20 AM, Rick Macklem <rmacklem@uoguelph.ca> wrote:
> Boaz Harrosh wrote:
> [good stuff snipped for brevity]
> >
> > I have on my todo list a full osd (pnfs-objects) stack port to FreeBSD
> > But that's in the far future.
> > Meanwhile I'd suggest the pnfs-client-core to layout-driver API be
> > closely
> > crafted to match that of the Linux LD API so it could be kept the same
> > code base.
>
> Am I correct to assume that fs/nfs/nfsfilelayout* implements this API
> for the file layout case?
True.
> If so, I'll try and build glue that can use
> that code unchanged (or at least minimally changed).
>
> And Peter (I see you're on the cc list), is it possible to get a copy
> of these files without any GPL? It looks like they have the U of Michigan
> copyright, which I think should be fine for the FreeBSD folks, but then
> there is
>
> MODULE_LICENSE("GPL");
>
> in it, which I assume means it's GPL'd. (Basically, exactly the same file,
> but released without this line, would make the FreeBSD folks happy, I
> think.)
> (No rush on this, I can use GPL'd code, I just can't put it in the FreeBSD
> source tree easily.) The same would apply to any other layout driver that
> others, like Panasas or EMC might be willing to open source for FreeBSD.
>
Not up to me, but it's more likely you'd be able to get the Windows code
over the university license more easily...
Benny
>
> And Boaz, thanks for the useful information. I'll probably start with
> Fedora plus a patched kernel for a server to test against. (It sounds
> like Steve is willing to help me out, since I haven't used "rpm" in
> years and have never typed "git" in my life.:-)
>
> Thanks everyone for your help with this, rick
>
>
> > For example the pnfs-objects LD is mostly pure C code
> > relying
> > on the ORE (Objects Raid Engine). The ORE is multy licensed library
> > that
> > has ports to Many platforms including Linux-Kernel
> > Linux/FreeBSD/windows User-Mode and more of which only the Linux
> > variants
> > actually run because of the need of the iscsi stack to support BiDi
> > and
> > large CDB commands, Only supported by Linux.
> >
> > Cheers
> > Boaz
> >
> > > rick
> > >
> > > ----- Original Message -----
> > >> On Thu, 08 Dec 2011 19:40:19 -0500, Sorin Faibish
> > >> <sfaibish@emc.com>
> > >> wrote:
> > >>
> > >>> Attached please find the slides that we presented yesterday at
> > >>> USENIX
> > >>> LISA 11
> > >>> conference in Boston. We would like to get more detailed input on
> > >>> other
> > >>> OS
> > >>> that will support pNFS server/client (this is for Rick as well)
> > >>> that
> > >>> were
> > >>> not mentioned in this slide set.
> > >>>
> > >>> /Sorin
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Best Regards
> > >> Sorin Faibish
> > >> Corporate Distinguished Engineer
> > >> Unified Storage Division
> > >>
> > >> EMC²
> > >> where information lives
> > >>
> > >> Phone: 508-435-1000 x 48545
> > >> Cellphone: 617-510-0422
> > >> Email : sfaibish@emc.com
> > > _______________________________________________
> > > nfsv4 mailing list
> > > nfsv4@ietf.org
> > > https://www.ietf.org/mailman/listinfo/nfsv4
> >
> > _______________________________________________
> > nfsv4 mailing list
> > nfsv4@ietf.org
> > https://www.ietf.org/mailman/listinfo/nfsv4
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4
>
[-- Attachment #1.2: Type: text/html, Size: 5506 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
nfsv4 mailing list
nfsv4@ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4
^ permalink raw reply [flat|nested] 11+ messages in thread