[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[Smith, Stan]

Hello,
  I'm currently unable to search the lustre-devel archives + 1st post to this list.

Having read lustre documentation I find only references to building lustre server for stock 'distro' kernels.

Does someone have suggestions on how to patch ldiskfs for a kernel.org kernel.....specifically weeding out RHEL/SLES/* distro FS enhancements from a kernel.org base?

I have written a Reliable Datagram LND which passes the LNet selftests (client lustre build) now entering the world of lustre server building.
Patching the kernel.org kernel for lustre server was fairly straightforward, creating the patch series for ldiskfs is another story.

Thanks,

Stan.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20151102/997dc499/attachment-0001.htm>

[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[Patrick Farrell]

Stan,

Why not work with the CentOS 7 kernel on the server anyway?  Is 
something your LND needs not available?  CentOS 7 is freely available 
and isn't far off in terms of kernel version from 3.12.

You may be hard pressed to get useful advice on changing the ldiskfs 
patch series, that has the potential to be a pretty big job.

Regards,
- Patrick Farrell
On 11/02/2015 01:05 PM, Smith, Stan wrote:
>
> Hello,
>
>   I'm currently unable to search the lustre-devel archives + 1^st post 
> to this list.
>
> Having read lustre documentation I find only references to building 
> lustre server for stock 'distro' kernels.
>
> Does someone have suggestions on how to patch ldiskfs for a kernel.org 
> kernel.....specifically weeding out RHEL/SLES/* distro FS enhancements 
> from a kernel.org base?
>
> I have written a Reliable Datagram LND which passes the LNet selftests 
> (client lustre build) now entering the world of lustre server building.
>
> Patching the kernel.org kernel for lustre server was fairly 
> straightforward, creating the patch series for ldiskfs is another story.
>
> Thanks,
>
> Stan.
>
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20151102/e9b4448b/attachment.htm>

[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[Ben Evans]

Using ZFS as a backing filesystem may be a simpler way to go here if you want to stay with kernel.org.  The lustre server + ZFS is patchless, and if you?re writing a new LND, you probably don?t care about the backing filesystem anyway.

-Ben Evans

From: lustre-devel <lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces@lists.lustre.org>> on behalf of Patrick Farrell <paf at cray.com<mailto:paf@cray.com>>
Date: Monday, November 2, 2015 at 3:13 PM
To: "lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>" <lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>>
Subject: Re: [lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

Stan,

Why not work with the CentOS 7 kernel on the server anyway?  Is something your LND needs not available?  CentOS 7 is freely available and isn't far off in terms of kernel version from 3.12.

You may be hard pressed to get useful advice on changing the ldiskfs patch series, that has the potential to be a pretty big job.

Regards,
- Patrick Farrell
On 11/02/2015 01:05 PM, Smith, Stan wrote:
Hello,
  I?m currently unable to search the lustre-devel archives + 1st post to this list.

Having read lustre documentation I find only references to building lustre server for stock ?distro? kernels.

Does someone have suggestions on how to patch ldiskfs for a kernel.org kernel?..specifically weeding out RHEL/SLES/* distro FS enhancements from a kernel.org base?

I have written a Reliable Datagram LND which passes the LNet selftests (client lustre build) now entering the world of lustre server building.
Patching the kernel.org kernel for lustre server was fairly straightforward, creating the patch series for ldiskfs is another story.

Thanks,

Stan.



_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20151102/89540b8b/attachment.htm>

[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[Smith, Stan]

From: lustre-devel [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Patrick Farrell
Sent: Monday, November 2, 2015 11:13 AM
To: lustre-devel at lists.lustre.org
Subject: Re: [lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

Stan,

Why not work with the CentOS 7 kernel on the server anyway?  Is something your LND needs not available?  CentOS 7 is freely available and isn't far off in terms of kernel version from 3.12.

Yes -I'm somewhat stuck on RHEL 6.x series as I use warewulf as a cluster provisioning system (diskless compute nodes), so far have not found a functional warewulf for RHEL7.
I did choose linux-3.12.39 as a kernel base believing it to be somewhat close to RHEL7; although I'm finding this to less so than hoped for....
Additionally the RD drivers I'm using have not been ported to RHEL7;
In examining lustre-2_7.62 I found support for sles12 based on 3.12.39 kernel.; thinking the lustre patches would be close...
So now I'm wedged between RHEL7 ldiskfs patches and SLES12 ldiskfs patches - figure out distro specific enhancements vs lustre enhancements.

You may be hard pressed to get useful advice on changing the ldiskfs patch series, that has the potential to be a pretty big job.

I'm starting to understand the magnitude of the ldiskfs task.......

I'm ignorant about zfs*, would zfs be a better (read that as path-of-least-resistance) approach ?  What are the zfs pitfalls?

Thanks,

Stan.


Regards,
- Patrick Farrell
On 11/02/2015 01:05 PM, Smith, Stan wrote:
Hello,
  I'm currently unable to search the lustre-devel archives + 1st post to this list.

Having read lustre documentation I find only references to building lustre server for stock 'distro' kernels.

Does someone have suggestions on how to patch ldiskfs for a kernel.org kernel.....specifically weeding out RHEL/SLES/* distro FS enhancements from a kernel.org base?

I have written a Reliable Datagram LND which passes the LNet selftests (client lustre build) now entering the world of lustre server building.
Patching the kernel.org kernel for lustre server was fairly straightforward, creating the patch series for ldiskfs is another story.

Thanks,

Stan.




_______________________________________________

lustre-devel mailing list

lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>

http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20151102/a13f51df/attachment-0001.htm>

[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[Patrick Farrell]

For ZFS:

The pitfall is primarily that it's less familiar than ldiskfs, which 
basically uses the standard Linux fs tools, like ext.  ZFS is its own 
world, which will probably have a learning curve for someone who has not 
used it before, and it must be built for the version of the kernel 
you're interested in.

But, Ben is right, that's likely to be easier than porting ldiskfs.

- Patrick

On 11/02/2015 01:35 PM, Smith, Stan wrote:
>
> *From:*lustre-devel [mailto:lustre-devel-bounces at lists.lustre.org] *On 
> Behalf Of *Patrick Farrell
> *Sent:* Monday, November 2, 2015 11:13 AM
> *To:* lustre-devel at lists.lustre.org
> *Subject:* Re: [lustre-devel] porting Lustre server to RHEL6.7 distro 
> running a kernel.org linux-3.12.39
>
> Stan,
>
> Why not work with the CentOS 7 kernel on the server anyway? Is 
> something your LND needs not available?  CentOS 7 is freely available 
> and isn't far off in terms of kernel version from 3.12.
>
> Yes --I'm somewhat stuck on RHEL 6.x series as I use warewulf as a 
> cluster provisioning system (diskless compute nodes), so far have not 
> found a functional warewulf for RHEL7.
>
> I did choose linux-3.12.39 as a kernel base believing it to be 
> somewhat close to RHEL7; although I'm finding this to less so than 
> hoped for....
>
> Additionally the RD drivers I'm using have not been ported to RHEL7;
>
> In examining lustre-2_7.62 I found support for sles12 based on 3.12.39 
> kernel.; thinking the lustre patches would be close...
>
> So now I'm wedged between RHEL7 ldiskfs patches and SLES12 ldiskfs 
> patches -- figure out distro specific enhancements vs lustre enhancements.
>
> You may be hard pressed to get useful advice on changing the ldiskfs 
> patch series, that has the potential to be a pretty big job.
>
> I'm starting to understand the magnitude of the ldiskfs task.......
>
> I'm ignorant about zfs*, would zfs be a better (read that as 
> path-of-least-resistance) approach ?  What are the zfs pitfalls?
>
> Thanks,
>
> Stan.
>
>
>
> Regards,
> - Patrick Farrell
>
> On 11/02/2015 01:05 PM, Smith, Stan wrote:
>
>     Hello,
>
>       I'm currently unable to search the lustre-devel archives + 1^st
>     post to this list.
>
>     Having read lustre documentation I find only references to
>     building lustre server for stock 'distro' kernels.
>
>     Does someone have suggestions on how to patch ldiskfs for a
>     kernel.org kernel.....specifically weeding out RHEL/SLES/* distro
>     FS enhancements from a kernel.org base?
>
>     I have written a Reliable Datagram LND which passes the LNet
>     selftests (client lustre build) now entering the world of lustre
>     server building.
>
>     Patching the kernel.org kernel for lustre server was fairly
>     straightforward, creating the patch series for ldiskfs is another
>     story.
>
>     Thanks,
>
>     Stan.
>
>
>
>
>     _______________________________________________
>
>     lustre-devel mailing list
>
>     lustre-devel at lists.lustre.org  <mailto:lustre-devel@lists.lustre.org>
>
>     http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20151102/33102ce4/attachment.htm>

[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[Rick Wagner]

We?ve run ZFS-backed Lustre servers using a patchless (i.e., without ldiskfs) Linux 3 kernel under CentOS 6. While I don?t recommend this for everyone I can say that the mechanics of building the kernel modules (ZFS and Lustre) and standing up the file system all worked well. We?re looking to move back to a stock CentOS 6 kernel, but are sticking with the patchless model. If you?re willing to try ZFS I suspect the build will be fine, and then you should test test test.

I?ve included the outline of commands we use to build our server RPMs. You?ll want to adjust the versions of SPL, ZFS, & Lustre, along with the --with-linux and --with-linux-config parameters. I apologize for any typos and subsequent frustration. What level of dependency hell you find yourself in is is largely dependent on your choice of distro.

?Rick

git clone https://github.com/zfsonlinux/spl.git
cd spl
git checkout spl-0.6.5.3
./autogen.sh
./configure
make pkg
rpm -ivh *x86_64.rpm

cd ../
git clone https://github.com/zfsonlinux/zfs.git
cd zfs
git checkout zfs-0.6.5.3
./autogen.sh
./configure
make pkg
rpm -ivh *x86_64.rpm

cd ../
git clone http://git.whamcloud.com/fs/lustre-release.git
cd lustre-release
git checkout 2.7.62
sh autogen.sh
./configure --with-zfs=/usr/src/zfs-0.6.5.3 --with-spl=/usr/src/spl-0.6.5.3 --with-linux=/usr/src/linux-3.10 --without-ldiskfs --with-linux-config=/boot/config-3.10.73-1.el6.elrepo.x86_64  --with-o2ib=no
make rpms


On Nov 2, 2015, at 11:45 AM, Patrick Farrell <paf at cray.com<mailto:paf@cray.com>> wrote:

For ZFS:

The pitfall is primarily that it's less familiar than ldiskfs, which basically uses the standard Linux fs tools, like ext.  ZFS is its own world, which will probably have a learning curve for someone who has not used it before, and it must be built for the version of the kernel you're interested in.

But, Ben is right, that's likely to be easier than porting ldiskfs.

- Patrick

On 11/02/2015 01:35 PM, Smith, Stan wrote:
From: lustre-devel [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Patrick Farrell
Sent: Monday, November 2, 2015 11:13 AM
To: lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
Subject: Re: [lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org<http://kernel.org> linux-3.12.39

Stan,

Why not work with the CentOS 7 kernel on the server anyway?  Is something your LND needs not available?  CentOS 7 is freely available and isn't far off in terms of kernel version from 3.12.

Yes ?I?m somewhat stuck on RHEL 6.x series as I use warewulf as a cluster provisioning system (diskless compute nodes), so far have not found a functional warewulf for RHEL7.
I did choose linux-3.12.39 as a kernel base believing it to be somewhat close to RHEL7; although I?m finding this to less so than hoped for?.
Additionally the RD drivers I?m using have not been ported to RHEL7;
In examining lustre-2_7.62 I found support for sles12 based on 3.12.39 kernel.; thinking the lustre patches would be close?
So now I?m wedged between RHEL7 ldiskfs patches and SLES12 ldiskfs patches ? figure out distro specific enhancements vs lustre enhancements.

You may be hard pressed to get useful advice on changing the ldiskfs patch series, that has the potential to be a pretty big job.

I?m starting to understand the magnitude of the ldiskfs task??.

I?m ignorant about zfs*, would zfs be a better (read that as path-of-least-resistance) approach ?  What are the zfs pitfalls?

Thanks,

Stan.


Regards,
- Patrick Farrell
On 11/02/2015 01:05 PM, Smith, Stan wrote:
Hello,
  I?m currently unable to search the lustre-devel archives + 1st post to this list.

Having read lustre documentation I find only references to building lustre server for stock ?distro? kernels.

Does someone have suggestions on how to patch ldiskfs for a kernel.org<http://kernel.org> kernel?..specifically weeding out RHEL/SLES/* distro FS enhancements from a kernel.org<http://kernel.org> base?

I have written a Reliable Datagram LND which passes the LNet selftests (client lustre build) now entering the world of lustre server building.
Patching the kernel.org<http://kernel.org> kernel for lustre server was fairly straightforward, creating the patch series for ldiskfs is another story.

Thanks,

Stan.




_______________________________________________

lustre-devel mailing list

lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>

http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org



_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20151102/45171c09/attachment-0001.htm>

[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[Smith, Stan]

From: lustre-devel [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Ben Evans
Sent: Monday, November 2, 2015 11:26 AM
To: Patrick Farrell <paf@cray.com>; lustre-devel at lists.lustre.org
Subject: Re: [lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

Using ZFS as a backing filesystem may be a simpler way to go here if you want to stay with kernel.org.  The lustre server + ZFS is patchless, and if you're writing a new LND, you probably don't care about the backing filesystem anyway.

You are 100% spot-on, at this juncture in the LND development, I don't really care so much about the backing filesystem; whatever builds/works such that I can continue LND testing in the client/server environment. I suspect a large step between LNet selftests and full-on LFS.

It's Monday after a time zone change, hence a small clarification on your statement 'The lustre server + ZFS is patchless'.
By patchless you saying the lustre source kernel patches are not required when using ZFS? Just to be crystal-clear.

Stan.


-Ben Evans

From: lustre-devel <lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces@lists.lustre.org>> on behalf of Patrick Farrell <paf at cray.com<mailto:paf@cray.com>>
Date: Monday, November 2, 2015 at 3:13 PM
To: "lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>" <lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>>
Subject: Re: [lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

Stan,

Why not work with the CentOS 7 kernel on the server anyway?  Is something your LND needs not available?  CentOS 7 is freely available and isn't far off in terms of kernel version from 3.12.

You may be hard pressed to get useful advice on changing the ldiskfs patch series, that has the potential to be a pretty big job.

Regards,
- Patrick Farrell
On 11/02/2015 01:05 PM, Smith, Stan wrote:
Hello,
  I'm currently unable to search the lustre-devel archives + 1st post to this list.

Having read lustre documentation I find only references to building lustre server for stock 'distro' kernels.

Does someone have suggestions on how to patch ldiskfs for a kernel.org kernel.....specifically weeding out RHEL/SLES/* distro FS enhancements from a kernel.org base?

I have written a Reliable Datagram LND which passes the LNet selftests (client lustre build) now entering the world of lustre server building.
Patching the kernel.org kernel for lustre server was fairly straightforward, creating the patch series for ldiskfs is another story.

Thanks,

Stan.




_______________________________________________

lustre-devel mailing list

lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20151102/745814fa/attachment.htm>

[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[Ben Evans]

By patchless, I mean that neither Lustre nor ZFS need to patch the Linux kernel in order to work.  With ldiskfs, some patching of the kernel needs to happen, which is where you?re running into trouble.

I?m not going to say that it will be pain-free, just that you?ll remove a potentially large pain point in your dev process.

-Ben Evans

From: "Smith, Stan" <stan.smith at intel.com<mailto:stan.smith@intel.com>>
Date: Monday, November 2, 2015 at 3:46 PM
To: Ben Evans <bevans at cray.com<mailto:bevans@cray.com>>, Patrick Farrell <paf at cray.com<mailto:paf@cray.com>>, "lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>" <lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>>
Subject: RE: [lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

From: lustre-devel [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Ben Evans
Sent: Monday, November 2, 2015 11:26 AM
To: Patrick Farrell <paf at cray.com<mailto:paf@cray.com>>; lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
Subject: Re: [lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

Using ZFS as a backing filesystem may be a simpler way to go here if you want to stay with kernel.org.  The lustre server + ZFS is patchless, and if you?re writing a new LND, you probably don?t care about the backing filesystem anyway.

You are 100% spot-on, at this juncture in the LND development, I don?t really care so much about the backing filesystem; whatever builds/works such that I can continue LND testing in the client/server environment. I suspect a large step between LNet selftests and full-on LFS.

It?s Monday after a time zone change, hence a small clarification on your statement ?The lustre server + ZFS is patchless?.
By patchless you saying the lustre source kernel patches are not required when using ZFS? Just to be crystal-clear.

Stan.


-Ben Evans

From: lustre-devel <lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces@lists.lustre.org>> on behalf of Patrick Farrell <paf at cray.com<mailto:paf@cray.com>>
Date: Monday, November 2, 2015 at 3:13 PM
To: "lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>" <lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>>
Subject: Re: [lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

Stan,

Why not work with the CentOS 7 kernel on the server anyway?  Is something your LND needs not available?  CentOS 7 is freely available and isn't far off in terms of kernel version from 3.12.

You may be hard pressed to get useful advice on changing the ldiskfs patch series, that has the potential to be a pretty big job.

Regards,
- Patrick Farrell
On 11/02/2015 01:05 PM, Smith, Stan wrote:
Hello,
  I?m currently unable to search the lustre-devel archives + 1st post to this list.

Having read lustre documentation I find only references to building lustre server for stock ?distro? kernels.

Does someone have suggestions on how to patch ldiskfs for a kernel.org kernel?..specifically weeding out RHEL/SLES/* distro FS enhancements from a kernel.org base?

I have written a Reliable Datagram LND which passes the LNet selftests (client lustre build) now entering the world of lustre server building.
Patching the kernel.org kernel for lustre server was fairly straightforward, creating the patch series for ldiskfs is another story.

Thanks,

Stan.




_______________________________________________

lustre-devel mailing list

lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20151102/3be92309/attachment-0001.htm>

[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[Smith, Stan]

From: Rick Wagner [mailto:rpwagner at sdsc.edu]
Sent: Monday, November 2, 2015 12:25 PM
To: Patrick Farrell <paf@cray.com>
Cc: Smith, Stan <stan.smith@intel.com>; lustre-devel at lists.lustre.org
Subject: Re: [lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

We've run ZFS-backed Lustre servers using a patchless (i.e., without ldiskfs) Linux 3 kernel under CentOS 6. While I don't recommend this for everyone I can say that the mechanics of building the kernel modules (ZFS and Lustre) and standing up the file system all worked well. We're looking to move back to a stock CentOS 6 kernel, but are sticking with the patchless model. If you're willing to try ZFS I suspect the build will be fine, and then you should test test test.

I've included the outline of commands we use to build our server RPMs. You'll want to adjust the versions of SPL, ZFS, & Lustre, along with the --with-linux and --with-linux-config parameters. I apologize for any typos and subsequent frustration. What level of dependency hell you find yourself in is is largely dependent on your choice of distro.

Wow -  A great assist!   Thank you much!!

And thank you Patrick, Ben & Rick for the timely sage advice on making forward progress in the land of lustre.

Let the ZFS adventure begin......will write if I find work...  ;-)

Stan.


-Rick

git clone https://github.com/zfsonlinux/spl.git
cd spl
git checkout spl-0.6.5.3
./autogen.sh
./configure
make pkg
rpm -ivh *x86_64.rpm

cd ../
git clone https://github.com/zfsonlinux/zfs.git
cd zfs
git checkout zfs-0.6.5.3
./autogen.sh
./configure
make pkg
rpm -ivh *x86_64.rpm

cd ../
git clone http://git.whamcloud.com/fs/lustre-release.git
cd lustre-release
git checkout 2.7.62
sh autogen.sh
./configure --with-zfs=/usr/src/zfs-0.6.5.3 --with-spl=/usr/src/spl-0.6.5.3 --with-linux=/usr/src/linux-3.10 --without-ldiskfs --with-linux-config=/boot/config-3.10.73-1.el6.elrepo.x86_64  --with-o2ib=no
make rpms


On Nov 2, 2015, at 11:45 AM, Patrick Farrell <paf at cray.com<mailto:paf@cray.com>> wrote:

For ZFS:

The pitfall is primarily that it's less familiar than ldiskfs, which basically uses the standard Linux fs tools, like ext.  ZFS is its own world, which will probably have a learning curve for someone who has not used it before, and it must be built for the version of the kernel you're interested in.

But, Ben is right, that's likely to be easier than porting ldiskfs.

- Patrick

On 11/02/2015 01:35 PM, Smith, Stan wrote:
From: lustre-devel [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Patrick Farrell
Sent: Monday, November 2, 2015 11:13 AM
To: lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
Subject: Re: [lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org<http://kernel.org> linux-3.12.39

Stan,

Why not work with the CentOS 7 kernel on the server anyway?  Is something your LND needs not available?  CentOS 7 is freely available and isn't far off in terms of kernel version from 3.12.

Yes -I'm somewhat stuck on RHEL 6.x series as I use warewulf as a cluster provisioning system (diskless compute nodes), so far have not found a functional warewulf for RHEL7.
I did choose linux-3.12.39 as a kernel base believing it to be somewhat close to RHEL7; although I'm finding this to less so than hoped for....
Additionally the RD drivers I'm using have not been ported to RHEL7;
In examining lustre-2_7.62 I found support for sles12 based on 3.12.39 kernel.; thinking the lustre patches would be close...
So now I'm wedged between RHEL7 ldiskfs patches and SLES12 ldiskfs patches - figure out distro specific enhancements vs lustre enhancements.

You may be hard pressed to get useful advice on changing the ldiskfs patch series, that has the potential to be a pretty big job.

I'm starting to understand the magnitude of the ldiskfs task.......

I'm ignorant about zfs*, would zfs be a better (read that as path-of-least-resistance) approach ?  What are the zfs pitfalls?

Thanks,

Stan.


Regards,
- Patrick Farrell
On 11/02/2015 01:05 PM, Smith, Stan wrote:
Hello,
  I'm currently unable to search the lustre-devel archives + 1st post to this list.

Having read lustre documentation I find only references to building lustre server for stock 'distro' kernels.

Does someone have suggestions on how to patch ldiskfs for a kernel.org<http://kernel.org> kernel.....specifically weeding out RHEL/SLES/* distro FS enhancements from a kernel.org<http://kernel.org> base?

I have written a Reliable Datagram LND which passes the LNet selftests (client lustre build) now entering the world of lustre server building.
Patching the kernel.org<http://kernel.org> kernel for lustre server was fairly straightforward, creating the patch series for ldiskfs is another story.

Thanks,

Stan.





_______________________________________________

lustre-devel mailing list

lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>

http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20151102/c8af57d4/attachment.htm>

[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[Colin Faber]

Hi Rick,

This is fantastic, if you haven't already would you mind adding this to the
wiki.lustre.org wiki site?

-cf

On Mon, Nov 2, 2015 at 1:25 PM, Rick Wagner <rpwagner@sdsc.edu> wrote:

> We?ve run ZFS-backed Lustre servers using a patchless (i.e., without
> ldiskfs) Linux 3 kernel under CentOS 6. While I don?t recommend this for
> everyone I can say that the mechanics of building the kernel modules (ZFS
> and Lustre) and standing up the file system all worked well. We?re looking
> to move back to a stock CentOS 6 kernel, but are sticking with the
> patchless model. If you?re willing to try ZFS I suspect the build will be
> fine, and then you should test test test.
>
> I?ve included the outline of commands we use to build our server RPMs.
> You?ll want to adjust the versions of SPL, ZFS, & Lustre, along with the
> --with-linux and --with-linux-config parameters. I apologize for any typos
> and subsequent frustration. What level of dependency hell you find yourself
> in is is largely dependent on your choice of distro.
>
> ?Rick
>
> git clone https://github.com/zfsonlinux/spl.git
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_zfsonlinux_spl.git&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=tHup8osKBWRBPj9rC-qu--lua4itTMknj0eiUjMPJMg&e=>
> cd spl
> git checkout spl-0.6.5.3
> ./autogen.sh
> ./configure
> make pkg
> rpm -ivh *x86_64.rpm
>
> cd ../
> git clone https://github.com/zfsonlinux/zfs.git
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_zfsonlinux_zfs.git&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=QijKf5IjRvVDun-yIB7EWmvWA7NX8KTdBzyCR5b2PsA&e=>
> cd zfs
> git checkout zfs-0.6.5.3
> ./autogen.sh
> ./configure
> make pkg
> rpm -ivh *x86_64.rpm
>
> cd ../
> git clone http://git.whamcloud.com/fs/lustre-release.git
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__git.whamcloud.com_fs_lustre-2Drelease.git&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=-Na1KZ7h6BNtaEVI_N0DRtTl1xhhwsqJ7YlqvXYxQNg&e=>
> cd lustre-release
> git checkout 2.7.62
> sh autogen.sh
> ./configure --with-zfs=/usr/src/zfs-0.6.5.3
> --with-spl=/usr/src/spl-0.6.5.3 --with-linux=/usr/src/linux-3.10
> --without-ldiskfs
> --with-linux-config=/boot/config-3.10.73-1.el6.elrepo.x86_64  --with-o2ib=no
> make rpms
>
>
> On Nov 2, 2015, at 11:45 AM, Patrick Farrell <paf@cray.com> wrote:
>
> For ZFS:
>
> The pitfall is primarily that it's less familiar than ldiskfs, which
> basically uses the standard Linux fs tools, like ext.  ZFS is its own
> world, which will probably have a learning curve for someone who has not
> used it before, and it must be built for the version of the kernel you're
> interested in.
>
> But, Ben is right, that's likely to be easier than porting ldiskfs.
>
> - Patrick
>
> On 11/02/2015 01:35 PM, Smith, Stan wrote:
>
> *From:* lustre-devel [mailto:lustre-devel-bounces at lists.lustre.org
> <lustre-devel-bounces@lists.lustre.org>] *On Behalf Of *Patrick Farrell
> *Sent:* Monday, November 2, 2015 11:13 AM
> *To:* lustre-devel at lists.lustre.org
> *Subject:* Re: [lustre-devel] porting Lustre server to RHEL6.7 distro
> running a kernel.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__kernel.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=JNDXqBVf-dJrXGvONRXY6Ow6ONNvfCJCHdkedQYE1WA&e=>
> linux-3.12.39
>
> Stan,
>
> Why not work with the CentOS 7 kernel on the server anyway?  Is something
> your LND needs not available?  CentOS 7 is freely available and isn't far
> off in terms of kernel version from 3.12.
>
> Yes ?I?m somewhat stuck on RHEL 6.x series as I use warewulf as a cluster
> provisioning system (diskless compute nodes), so far have not found a
> functional warewulf for RHEL7.
> I did choose linux-3.12.39 as a kernel base believing it to be somewhat
> close to RHEL7; although I?m finding this to less so than hoped for?.
> Additionally the RD drivers I?m using have not been ported to RHEL7;
> In examining lustre-2_7.62 I found support for sles12 based on 3.12.39
> kernel.; thinking the lustre patches would be close?
> So now I?m wedged between RHEL7 ldiskfs patches and SLES12 ldiskfs patches
> ? figure out distro specific enhancements vs lustre enhancements.
>
> You may be hard pressed to get useful advice on changing the ldiskfs patch
> series, that has the potential to be a pretty big job.
>
> I?m starting to understand the magnitude of the ldiskfs task??.
>
> I?m ignorant about zfs*, would zfs be a better (read that as
> path-of-least-resistance) approach ?  What are the zfs pitfalls?
>
> Thanks,
>
> Stan.
>
>
> Regards,
> - Patrick Farrell
> On 11/02/2015 01:05 PM, Smith, Stan wrote:
>
> Hello,
>   I?m currently unable to search the lustre-devel archives + 1st post to
> this list.
>
> Having read lustre documentation I find only references to building lustre
> server for stock ?distro? kernels.
>
> Does someone have suggestions on how to patch ldiskfs for a kernel.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__kernel.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=JNDXqBVf-dJrXGvONRXY6Ow6ONNvfCJCHdkedQYE1WA&e=>
> kernel?..specifically weeding out RHEL/SLES/* distro FS enhancements from a
> kernel.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__kernel.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=JNDXqBVf-dJrXGvONRXY6Ow6ONNvfCJCHdkedQYE1WA&e=>
> base?
>
> I have written a Reliable Datagram LND which passes the LNet selftests
> (client lustre build) now entering the world of lustre server building.
> Patching the kernel.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__kernel.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=JNDXqBVf-dJrXGvONRXY6Ow6ONNvfCJCHdkedQYE1WA&e=>
> kernel for lustre server was fairly straightforward, creating the patch
> series for ldiskfs is another story.
>
> Thanks,
>
> Stan.
>
>
>
> _______________________________________________
>
> lustre-devel mailing list
>
> lustre-devel at lists.lustre.org
>
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddevel-2Dlustre.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=A6GC4M4a2RZsUUrt-VAKwS0MmRRdPLYYgVDeJ5RDF7Q&e=>
>
>
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddevel-2Dlustre.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=A6GC4M4a2RZsUUrt-VAKwS0MmRRdPLYYgVDeJ5RDF7Q&e=>
>
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddevel-2Dlustre.org&d=CwICAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=A6GC4M4a2RZsUUrt-VAKwS0MmRRdPLYYgVDeJ5RDF7Q&e=
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20151102/d757171b/attachment-0001.htm>

[lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org linux-3.12.39

[Rick Wagner]

Thanks, Colin. Let me do a sanity check on the process and I?ll see about updating the Lustre with ZFS page [1].

?Rick

[1] http://wiki.lustre.org/Lustre_with_ZFS_Install

On Nov 2, 2015, at 1:07 PM, Colin Faber <colin.faber at seagate.com<mailto:colin.faber@seagate.com>> wrote:

Hi Rick,

This is fantastic, if you haven't already would you mind adding this to the wiki.lustre.org<http://wiki.lustre.org/> wiki site?

-cf

On Mon, Nov 2, 2015 at 1:25 PM, Rick Wagner <rpwagner at sdsc.edu<mailto:rpwagner@sdsc.edu>> wrote:
We?ve run ZFS-backed Lustre servers using a patchless (i.e., without ldiskfs) Linux 3 kernel under CentOS 6. While I don?t recommend this for everyone I can say that the mechanics of building the kernel modules (ZFS and Lustre) and standing up the file system all worked well. We?re looking to move back to a stock CentOS 6 kernel, but are sticking with the patchless model. If you?re willing to try ZFS I suspect the build will be fine, and then you should test test test.

I?ve included the outline of commands we use to build our server RPMs. You?ll want to adjust the versions of SPL, ZFS, & Lustre, along with the --with-linux and --with-linux-config parameters. I apologize for any typos and subsequent frustration. What level of dependency hell you find yourself in is is largely dependent on your choice of distro.

?Rick

git clone https://github.com/zfsonlinux/spl.git<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_zfsonlinux_spl.git&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=tHup8osKBWRBPj9rC-qu--lua4itTMknj0eiUjMPJMg&e=>
cd spl
git checkout spl-0.6.5.3
./autogen.sh
./configure
make pkg
rpm -ivh *x86_64.rpm

cd ../
git clone https://github.com/zfsonlinux/zfs.git<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_zfsonlinux_zfs.git&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=QijKf5IjRvVDun-yIB7EWmvWA7NX8KTdBzyCR5b2PsA&e=>
cd zfs
git checkout zfs-0.6.5.3
./autogen.sh
./configure
make pkg
rpm -ivh *x86_64.rpm

cd ../
git clone http://git.whamcloud.com/fs/lustre-release.git<https://urldefense.proofpoint.com/v2/url?u=http-3A__git.whamcloud.com_fs_lustre-2Drelease.git&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=-Na1KZ7h6BNtaEVI_N0DRtTl1xhhwsqJ7YlqvXYxQNg&e=>
cd lustre-release
git checkout 2.7.62
sh autogen.sh
./configure --with-zfs=/usr/src/zfs-0.6.5.3 --with-spl=/usr/src/spl-0.6.5.3 --with-linux=/usr/src/linux-3.10 --without-ldiskfs --with-linux-config=/boot/config-3.10.73-1.el6.elrepo.x86_64  --with-o2ib=no
make rpms


On Nov 2, 2015, at 11:45 AM, Patrick Farrell <paf at cray.com<mailto:paf@cray.com>> wrote:

For ZFS:

The pitfall is primarily that it's less familiar than ldiskfs, which basically uses the standard Linux fs tools, like ext.  ZFS is its own world, which will probably have a learning curve for someone who has not used it before, and it must be built for the version of the kernel you're interested in.

But, Ben is right, that's likely to be easier than porting ldiskfs.

- Patrick

On 11/02/2015 01:35 PM, Smith, Stan wrote:
From: lustre-devel [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Patrick Farrell
Sent: Monday, November 2, 2015 11:13 AM
To: lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
Subject: Re: [lustre-devel] porting Lustre server to RHEL6.7 distro running a kernel.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__kernel.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=JNDXqBVf-dJrXGvONRXY6Ow6ONNvfCJCHdkedQYE1WA&e=> linux-3.12.39

Stan,

Why not work with the CentOS 7 kernel on the server anyway?  Is something your LND needs not available?  CentOS 7 is freely available and isn't far off in terms of kernel version from 3.12.

Yes ?I?m somewhat stuck on RHEL 6.x series as I use warewulf as a cluster provisioning system (diskless compute nodes), so far have not found a functional warewulf for RHEL7.
I did choose linux-3.12.39 as a kernel base believing it to be somewhat close to RHEL7; although I?m finding this to less so than hoped for?.
Additionally the RD drivers I?m using have not been ported to RHEL7;
In examining lustre-2_7.62 I found support for sles12 based on 3.12.39 kernel.; thinking the lustre patches would be close?
So now I?m wedged between RHEL7 ldiskfs patches and SLES12 ldiskfs patches ? figure out distro specific enhancements vs lustre enhancements.

You may be hard pressed to get useful advice on changing the ldiskfs patch series, that has the potential to be a pretty big job.

I?m starting to understand the magnitude of the ldiskfs task??.

I?m ignorant about zfs*, would zfs be a better (read that as path-of-least-resistance) approach ?  What are the zfs pitfalls?

Thanks,

Stan.


Regards,
- Patrick Farrell
On 11/02/2015 01:05 PM, Smith, Stan wrote:
Hello,
  I?m currently unable to search the lustre-devel archives + 1st post to this list.

Having read lustre documentation I find only references to building lustre server for stock ?distro? kernels.

Does someone have suggestions on how to patch ldiskfs for a kernel.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__kernel.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=JNDXqBVf-dJrXGvONRXY6Ow6ONNvfCJCHdkedQYE1WA&e=> kernel?..specifically weeding out RHEL/SLES/* distro FS enhancements from a kernel.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__kernel.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=JNDXqBVf-dJrXGvONRXY6Ow6ONNvfCJCHdkedQYE1WA&e=> base?

I have written a Reliable Datagram LND which passes the LNet selftests (client lustre build) now entering the world of lustre server building.
Patching the kernel.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__kernel.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=JNDXqBVf-dJrXGvONRXY6Ow6ONNvfCJCHdkedQYE1WA&e=> kernel for lustre server was fairly straightforward, creating the patch series for ldiskfs is another story.

Thanks,

Stan.




_______________________________________________

lustre-devel mailing list

lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>

http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddevel-2Dlustre.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=A6GC4M4a2RZsUUrt-VAKwS0MmRRdPLYYgVDeJ5RDF7Q&e=>



_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddevel-2Dlustre.org&d=CwMF-g&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=A6GC4M4a2RZsUUrt-VAKwS0MmRRdPLYYgVDeJ5RDF7Q&e=>


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddevel-2Dlustre.org&d=CwICAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=CjVqTjggbFA1F-2WDW87aU9UmdOsvkSSdbrx6n_jVLg&s=A6GC4M4a2RZsUUrt-VAKwS0MmRRdPLYYgVDeJ5RDF7Q&e=



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20151102/8e6665ed/attachment-0001.htm>