compile ofed 1.5.3 against lustre kernel

compile ofed 1.5.3 against lustre kernel

[Michael Di Domenico]

is there a mechanism somewhere inside the kernel source rpm that
detects which specific RHEL kernel is running and then applies those
backport patches?

I'm trying to get OFED 1.5.3 to compile against RHEL5.6 with the lustre patches

I've already compiled OFED 1.5.3 against the stock RHEL5.6 kernel that
ships 2.6.18-238 and that worked fine

I've downloaded the kernel source rpm for 2.6.18-238 from redhat and
recompiled my kernel with the lustre 1.8.5 kernel patches

however, when i try to compile OFED against this new kernel version
2.6.18-lustre18-1 its failing with a bunch of random errors that look
like missing patches

the command I'm using

./install.pl --hpc -s /opt/lustre/kernel/rpmbuild/BUILD/kernel-2.6.18lustre18

When i diff the two OFED build logs i do see in the stock kernel build
patches being applied from the 2.6.18-EL5.6 directory, but don't see
that in the lustre build log file which heightens my suspicion of
missing patches.

Is there a parameter i can override somewhere tell rpmbuild that even
though the kernel version doesn't match 2.6.18-238 it really is a
2.6.18-238 kernel?

thanks
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: compile ofed 1.5.3 against lustre kernel

[Steve Wise]

On 06/14/2011 03:34 PM, Michael Di Domenico wrote:
> is there a mechanism somewhere inside the kernel source rpm that
> detects which specific RHEL kernel is running and then applies those
> backport patches?
>
> I'm trying to get OFED 1.5.3 to compile against RHEL5.6 with the lustre patches
>
> I've already compiled OFED 1.5.3 against the stock RHEL5.6 kernel that
> ships 2.6.18-238 and that worked fine
>
> I've downloaded the kernel source rpm for 2.6.18-238 from redhat and
> recompiled my kernel with the lustre 1.8.5 kernel patches
>
> however, when i try to compile OFED against this new kernel version
> 2.6.18-lustre18-1 its failing with a bunch of random errors that look
> like missing patches
>
> the command I'm using
>
> ./install.pl --hpc -s /opt/lustre/kernel/rpmbuild/BUILD/kernel-2.6.18lustre18
>
> When i diff the two OFED build logs i do see in the stock kernel build
> patches being applied from the 2.6.18-EL5.6 directory, but don't see
> that in the lustre build log file which heightens my suspicion of
> missing patches.

The ofa kernel configure/build scripts are not detecting that your kernel source is RHEL5.x.  If you just use the 
ofa_kernel tree to build/install the OFED modules, then you can specify explicitly which backport patches to use when 
you configure it.  Or if you correctly name your kernel tree and set the version to be something like 
2.6.18-238.el5.lustre18 it should correctly detect it.


> Is there a parameter i can override somewhere tell rpmbuild that even
> though the kernel version doesn't match 2.6.18-238 it really is a
> 2.6.18-238 kernel?
>
> thanks
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: compile ofed 1.5.3 against lustre kernel

[Christoph Lameter]

On Tue, 14 Jun 2011, Steve Wise wrote:

> The ofa kernel configure/build scripts are not detecting that your kernel
> source is RHEL5.x.  If you just use the ofa_kernel tree to build/install the
> OFED modules, then you can specify explicitly which backport patches to use
> when you configure it.  Or if you correctly name your kernel tree and set the
> version to be something like 2.6.18-238.el5.lustre18 it should correctly
> detect it.

I surely wish this constant nightmare would go away. Could we please have
OFED trees against each kernel version in use somewhere so that we can
just do a git pull to get these into the respective trees?

This OFED divining the kernel version and then the application of upgrade
/ downgrade patches makes a lot of things very hard.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: compile ofed 1.5.3 against lustre kernel

[Woodruff, Robert J]

Christoph wrote, 
>I surely wish this constant nightmare would go away. Could we please have
>OFED trees against each kernel version in use somewhere so that we can
>just do a git pull to get these into the respective trees?

Other people have also asked for this, i.e., a source tree for each kernel
that OFED supports that already has the backports applied, or an install.pl
option that will generate the tree for a given kernel. Perhaps we can
try to get this support in the OFED-1.5.4 release. I will bring it up
For discussion in the OFA EWG that manages the releases. 

Long term it would also be good if these lustre patches were in the upstream
kernel, which would allow them to eventually get into the Linux distros so that
people did not have to be applying patches and building custom kernels. 
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: compile ofed 1.5.3 against lustre kernel

[Michael Di Domenico]

On Wed, Jun 15, 2011 at 7:25 PM, Woodruff, Robert J
<robert.j.woodruff-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
> Christoph wrote,
>>I surely wish this constant nightmare would go away. Could we please have
>>OFED trees against each kernel version in use somewhere so that we can
>>just do a git pull to get these into the respective trees?
>
> Other people have also asked for this, i.e., a source tree for each kernel
> that OFED supports that already has the backports applied, or an install.pl
> option that will generate the tree for a given kernel. Perhaps we can
> try to get this support in the OFED-1.5.4 release. I will bring it up
> For discussion in the OFA EWG that manages the releases.

For those of us that are not hardcore kernel developers can you
explain what this change would actually do?  I'm not sure i understand
how having a source tree with the patches already applied is any
different then taking the base kernel and applying a patch tree.  But
then again i probably don't understand what your talking about at all
(me == admin not developer).

> Long term it would also be good if these lustre patches were in the upstream
> kernel, which would allow them to eventually get into the Linux distros so that
> people did not have to be applying patches and building custom kernels.

I think (but could be wrong) I recall some number of years ago a
skirmish between the lustre developers and the kernel maintainers over
their patches, which is why they never have been added.  But i think
some number of the patches have been applied or lustre code changes
over the years, i seem to recall in the lustre 1.6 days the patch tree
was monstrous, the one i applied for 2.6.18 was pretty short only
20-30 patches or so

what I'd really wish for is a completely patchless version of lustre,
I'd be happy with a 10% performance hit over the compilation
complexity.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: compile ofed 1.5.3 against lustre kernel

[Woodruff, Robert J]

Michael wrote,
>For those of us that are not hardcore kernel developers can you
>explain what this change would actually do?  I'm not sure i understand
>how having a source tree with the patches already applied is any
>different then taking the base kernel and applying a patch tree.  But
>then again i probably don't understand what your talking about at all
>(me == admin not developer).

Well what I would like to understand is what would you like to see ?

A tar ball or complete patch with the OFED drivers for each kernel that OFED supports
that can be applied to the kernel tree so that the kernel can be built using
the normal kernel build process ?

Or

A complete kernel tree with all of the OFED drivers already included that could be
downloaded via tar balls or git ?

Other ?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: compile ofed 1.5.3 against lustre kernel

[Michael Di Domenico]

On Wed, Jun 15, 2011 at 7:50 PM, Woodruff, Robert J
<robert.j.woodruff-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
> Michael wrote,
>>For those of us that are not hardcore kernel developers can you
>>explain what this change would actually do?  I'm not sure i understand
>>how having a source tree with the patches already applied is any
>>different then taking the base kernel and applying a patch tree.  But
>>then again i probably don't understand what your talking about at all
>>(me == admin not developer).
>
> Well what I would like to understand is what would you like to see ?
>
> A tar ball or complete patch with the OFED drivers for each kernel that OFED supports
> that can be applied to the kernel tree so that the kernel can be built using
> the normal kernel build process ?
>
> Or
>
> A complete kernel tree with all of the OFED drivers already included that could be
> downloaded via tar balls or git ?
>
> Other ?
>

I'm not sure I'm really qualified to answer such a question (nor
whether i can write what i want to say legibly), but I'll take a stab
at it.

First off you should note i can only speak to redhat and rpmbuild
processes, we don't use sles or any other distro where i work (so my
answer is definitely biased).

For me personally, the only time I rebuild the kernel is when I have
to build a lustre server, which is not all that often.

The steps i take (in brevity) are:

unpack kernel source from redhat
edit spec file to include lustre patches
rebuild kernel rpms using rpmbuild (redhat process)
install new kernel
reboot into new kernel
use ofed install.pl -s /<lustre kernel tree>
install ofed rpms
reboot
make lustre source using --with-linux=<lustre kernel tree>
--with-o2ib=/usr/src/ofa_kernel

Even though I'm not really answering your question, I'm not sure the
change really makes any difference to me.  I can't really foresee a
need where i would want a full kernel tree versus patches.  I'm sure
scenario's exist where it would be beneficial to have one method over
the other, but for me i can't think of one.  I would hazard a guess
that the majority of people are in the same boat as me with kernel
rebuilds

nope, even sitting here racking my brain some more, i can't see a need
for a full kernel tree for me.  even if i was rebuilding the kernel
and kernel-ib sections i'd still want to use the redhat rpmbuild
process, which will patch vanilla 2.6.18 to 2.6.18-238, and then i'd
add whatever patches i needed to the end of rhel's patch tree just
like i did with lustre and let the spec file do all the work

i guess if i was an IB developer though, having a git tree of 2.6.18
with all the RHEL and IB patches applied to it would probably be handy
though.  i can only guess that having full git trees might make
porting patches to other kernel versions easier (not a dev don't
really know).  how do ports get created today? does each developer
have to basically create a kernel dev tree and test their patch
against it and see what breaks?

dunno if i helped or wasted everyone's time, but that's my two cents...
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: compile ofed 1.5.3 against lustre kernel

[Ira Weiny]

On Wed, 15 Jun 2011 16:25:26 -0700
"Woodruff, Robert J" <robert.j.woodruff-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:

[snip]

> 
> Long term it would also be good if these lustre patches were in the upstream
> kernel, which would allow them to eventually get into the Linux distros so that
> people did not have to be applying patches and building custom kernels. 

FYI,

http://jira.whamcloud.com/browse/LU-20

Ira

> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
weiny2-i2BcT+NCU+M@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html