RE: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

RE: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Woodruff, Robert J]

I think that we tried to isolate a lot of these kernel calls into
one library, the component library, so that when the kernel APIs change,
which seems to happen every release, we only have to change the code 
in one spot. 

This actually helps porting to new kernels. For example, I think that
it only took a week to get the IBAL code to compile on 2.6 since all or
most 
of the kernel dependencies were isolated to one module. 

It also then allows most of the code to remain very readable, since we
don't have to put #ifdefs all over the place when an API or kernel data
structure changes. 

Are there any other examples of drivers that isolate kernel specific
calls
to one module or file of their code to ease portability between
different revisions
of the kernel ? If not, maybe they should look at what we have done,
it might save them some headaches in the future. 



-----Original Message-----
From: infiniband-general-admin@lists.sourceforge.net
[mailto:infiniband-general-admin@lists.sourceforge.net] On Behalf Of
King, Steven R
Sent: Thursday, February 05, 2004 11:39 AM
To: Greg KH; linux-kernel@vger.kernel.org
Cc: infiniband-general@lists.sourceforge.net
Subject: RE: [Infiniband-general] Getting an Infiniband access layer in
the linux kernel


We just use the kernel's spin_lock_irqsave(), so I don't know what
you're talking about.

-----Original Message-----
From: Greg KH [mailto:greg@kroah.com] 
Sent: Thursday, February 05, 2004 10:55 AM
To: King, Steven R; linux-kernel@vger.kernel.org
Cc: infiniband-general@lists.sourceforge.net
Subject: Re: [Infiniband-general] Getting an Infiniband access layer in
the linux kernel


On Thu, Feb 05, 2004 at 10:27:54AM -0800, King, Steven R wrote:
> Hi Greg,
> What exactly is wrong with spinlock?  Far as I know, it's been working

> bug-free on a variety of platforms for quite some time now.  The other

> abstractions such as atomic_t are for platform portability.

Again, compare them to the current kernel spinlocks and try to realize
why your implementation of spinlock_irqsave() will not work on all
platforms.

Come on, just use the kernel versions, there is no need to reinvent the
wheel all of the time, it just wastes everyones time (including mine...)

thanks,

greg k-h


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration See the
breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Infiniband-general mailing list Infiniband-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/infiniband-general

Re: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Greg KH]

On Thu, Feb 05, 2004 at 12:16:17PM -0800, Woodruff, Robert J wrote:
> I think that we tried to isolate a lot of these kernel calls into
> one library, the component library, so that when the kernel APIs change,
> which seems to happen every release, we only have to change the code 
> in one spot. 

Then get your code into the kernel tree, and you will not have to worry
about this "problem".  Remember, that is what we are talking about here.
If you want to keep your "compatibility" library in your out-of-tree
code, that's fine with me, I don't care.

It's when you try to push that into the main kernel tree that I start to
care.

> Are there any other examples of drivers that isolate kernel specific
> calls to one module or file of their code to ease portability between
> different revisions of the kernel ? If not, maybe they should look at
> what we have done, it might save them some headaches in the future. 

No, that is not how Linux kernel development is done.  Come on people,
do your research...

greg k-h

Re: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Jamie Lokier]

Greg KH wrote:
> > Are there any other examples of drivers that isolate kernel specific
> > calls to one module or file of their code to ease portability between
> > different revisions of the kernel ? If not, maybe they should look at
> > what we have done, it might save them some headaches in the future. 
> 
> No, that is not how Linux kernel development is done.  Come on people,
> do your research...

To clarify:

For out-of-tree 3rd party drivers, there are two or three
"compatibility" packages which are used to allow a single driver
source to work with many kernel versions.  (I wrote one, it is used
for quite a few private drivers.  This method works well.  See
also "kcompat").

For drivers in the kernel tree, or drivers that somone would like to
go into the tree, "compatibility" headers are never used.  API changes
are for good reasons; drivers need to be updated to follow them.

When the kernel APIs change, for minor changes and sometimes even
major ones, the author of the API change will often go through all the
in tree drivers making the necessary changes.

-- Jamie

Infiniband thread(s) and etiquette.

[David Woodhouse]

I had killed this thread because I was not particularly interested in
it. I have not looked at the code in question, and
QuiteFranklyIDontReallyWantToBecause I can imagine what kind of WORDs I
would end up using if I tried :)

However, people keep starting _new_ threads on the same subject.

Please, be polite. Do not top-post upside-down messages, and do not use
a mailer which is so broken it omits 'References:' and 'In-Reply-To:'
headers from replies, and hence breaks threading. It is simply not
acceptable. 

I think it's time the list started to automatically bounce messages with
'Re: ' in the subject line but lacking any form of references. There are
enough non-broken mail programs in the world; we don't have to let
inconsiderate people use broken ones just because they happen to like
it. The signal-to-noise ratio is low enough without that kind of
rudeness.

-- 
dwmw2

Re: Infiniband thread(s) and etiquette.

[Pete Zaitcev]

On Thu, 05 Feb 2004 22:58:00 +0000
David Woodhouse <dwmw2@infradead.org> wrote:

> However, people keep starting _new_ threads on the same subject.

Dude, I was subscribed to Infiniband-general until a week ago, and
I had to fish their messages out from +spam.html folder. This is quite
beyond repair. Just killfile with subject regex and be done with it.

> I think it's time the list started to automatically bounce messages with
> 'Re: ' in the subject line but lacking any form of references.

Might be a good idea, althoug you know, it's pretty tough to extract
references if Nate's gateway at post-office ate them. And I cannot
live without NNTP.

-- Pete

RE: [Infiniband-general] Getting an Infiniband access layer in the Linux kernel

[Tillier, Fabian]

Greg,

The component library (the abstraction layer used by IBAL) is in no way
tied to InfiniBand.  Whatever is in there can be used by any other
projects, and there's a lot of useful stuff in there that does provide
value.
 
One goal of IBAL is to get InfiniBand support for Linux.  As such, it is
a higher priority to get things working than to wait for changes to
appear in the Linux kernel before making forward progress.

Keep in mind also that InfiniBand is not a Linux-only technology.
Sharing code between different operating systems accelerates development
and reduces the cost of maintenance.

Lastly, there are things (like timers) that are blatantly missing from
user-mode in Linux, and having an abstraction here allows code to be
shared between kernel and user mode.

Keep in mind that we're not expecting you or the Linux community to
blindly take the code as is.  We're looking for constructive feedback to
make it so that everyone goes home happy.  It's disappointing that the
feedback we're getting from you is that any abstractions will be cause
for rejection.  What are the grounds for this policy?  What does it
accomplish?  Is portable code a problem for the Linux community in
general?  Removing the abstraction from the IBAL project would require
significant rework of the code that would add very little from a
functional perspective, and make the code base more complicated and
harder to maintain.

I think it would help us a lot to understand the motivation behind your
opinions so that we can try to meet your expectations.  If I've
misinterpreted your opinions, please clarify.

Thanks,

- Fab

-----Original Message-----
From: Greg KH [mailto:greg@kroah.com] 
Sent: Thursday, February 05, 2004 10:41 AM
To: Tillier, Fabian
Cc: Hefty, Sean; linux-kernel@vger.kernel.org; Troy Benjegerdes;
Woodruff, Robert J; Magro, Bill; Woodruff, Robert J;
infiniband-general@lists.sourceforge.net
Subject: Re: [Infiniband-general] Getting an Infiniband access layer in
the Linux kernel

On Thu, Feb 05, 2004 at 01:31:45PM -0500, Tillier, Fabian wrote:
> 
> Are you suggesting that if there is any abstraction, the code will
never
> be accepted?  Or rather that the abstraction better be correct?  I'm
> hoping for the latter, however please clarify.

The kernel has its own abstractions that seem to be working quite well
for all different types of platforms.  There is no need for you to
create your own, just for a driver subsystem.

If you have found any problems with the current locks please let the
entire kernel community benefit from your changes, and not relegate them
to a infiniband-only section of the kernel.

So yes, if you add your own versions of spinlocks and atomic_t types,
your code will be rejected, among other things :)

thanks,

greg k-h

Re: [Infiniband-general] Getting an Infiniband access layer in the Linux kernel

[Greg KH]

On Thu, Feb 05, 2004 at 02:44:07PM -0500, Tillier, Fabian wrote:
> 
> The component library (the abstraction layer used by IBAL) is in no way
> tied to InfiniBand.  Whatever is in there can be used by any other
> projects, and there's a lot of useful stuff in there that does provide
> value.

Great, if you think so, please provide patches to the main kernel tree
for all of us to see.  Just don't try to bury it in a driver subsystem.
That is my complaint.

> One goal of IBAL is to get InfiniBand support for Linux.  As such, it is
> a higher priority to get things working than to wait for changes to
> appear in the Linux kernel before making forward progress.

Isn't it a better idea to try to work with the community, instead of
ignoring them and thinking that you are progressing ahead of them?  :)

> Keep in mind also that InfiniBand is not a Linux-only technology.
> Sharing code between different operating systems accelerates development
> and reduces the cost of maintenance.

For your driver subsystem, possibly.  I know the whole history of ACPI,
and how that is developed too.  Also remember all of the man-hours that
were spent getting that acpi code into the proper shape for Linux...

> Lastly, there are things (like timers) that are blatantly missing from
> user-mode in Linux, and having an abstraction here allows code to be
> shared between kernel and user mode.

Again, patches are always appreciated to address anything that you feel
is missing in Linux.

> Keep in mind that we're not expecting you or the Linux community to
> blindly take the code as is.

That's good, as you have yet to even submit anything :)

> We're looking for constructive feedback to
> make it so that everyone goes home happy.  It's disappointing that the
> feedback we're getting from you is that any abstractions will be cause
> for rejection.  What are the grounds for this policy?

Um, "no one else does it as it is not necessary," is not enough?

> What does it accomplish?  Is portable code a problem for the Linux
> community in general?

Not at all.  Just look at the vast array of different hardware platforms
that Linux runs on!  And look at how almost all drivers work on all of
those platforms with no changes needed!  We are all for portability.

> Removing the abstraction from the IBAL project would require
> significant rework of the code that would add very little from a
> functional perspective, and make the code base more complicated and
> harder to maintain.

Harder for whom to maintain?  You, or the rest of the kernel community?
Remember, once your code is in the tree, everyone is free to make
changes to it, that's the power of a community effort.

That's enough of this, just submit a patch and we can work from there,
with real, technical answers for specific technical issues.

thanks,

greg k-h

RE: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[King, Steven R]

We just use the kernel's spin_lock_irqsave(), so I don't know what
you're talking about.

-----Original Message-----
From: Greg KH [mailto:greg@kroah.com] 
Sent: Thursday, February 05, 2004 10:55 AM
To: King, Steven R; linux-kernel@vger.kernel.org
Cc: infiniband-general@lists.sourceforge.net
Subject: Re: [Infiniband-general] Getting an Infiniband access layer in
the linux kernel


On Thu, Feb 05, 2004 at 10:27:54AM -0800, King, Steven R wrote:
> Hi Greg,
> What exactly is wrong with spinlock?  Far as I know, it's been working
> bug-free on a variety of platforms for quite some time now.  The other
> abstractions such as atomic_t are for platform portability.

Again, compare them to the current kernel spinlocks and try to realize
why your implementation of spinlock_irqsave() will not work on all
platforms.

Come on, just use the kernel versions, there is no need to reinvent the
wheel all of the time, it just wastes everyones time (including mine...)

thanks,

greg k-h

Re: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Greg KH]

On Thu, Feb 05, 2004 at 11:38:44AM -0800, King, Steven R wrote:
> We just use the kernel's spin_lock_irqsave(), so I don't know what
> you're talking about.

If you peruse the section, "The Fucked Up Sparc" in
Documentation/DocBook/kernel-locking you will see the problem.

thanks,

greg k-h

RE: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Raj, Ashok]

>linux kernel
>
>On Thu, Feb 05, 2004 at 10:27:54AM -0800, King, Steven R wrote:
>> Hi Greg,
>> What exactly is wrong with spinlock?  Far as I know, it's been
working
>> bug-free on a variety of platforms for quite some time now.  The
other
>> abstractions such as atomic_t are for platform portability.
>
>Again, compare them to the current kernel spinlocks and try to realize
>why your implementation of spinlock_irqsave() will not work on all
>platforms.

Humm... think the spin_lock macros used are the _ones_ that is defined
in the linux kernel, the other version (cl_spin_lock) is just a
wrapper... there is some precedence in the current linux code base that
does the same kind of wrapper thingies... , but iam sure no one is *so*
excited about *that* code anyway, so I will keep shut!


>
>Come on, just use the kernel versions, there is no need to reinvent the
>wheel all of the time, it just wastes everyones time (including
mine...)
>

Re: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Greg KH]

On Thu, Feb 05, 2004 at 10:27:54AM -0800, King, Steven R wrote:
> Hi Greg,
> What exactly is wrong with spinlock?  Far as I know, it's been working
> bug-free on a variety of platforms for quite some time now.  The other
> abstractions such as atomic_t are for platform portability.

Again, compare them to the current kernel spinlocks and try to realize
why your implementation of spinlock_irqsave() will not work on all
platforms.

Come on, just use the kernel versions, there is no need to reinvent the
wheel all of the time, it just wastes everyones time (including mine...)

thanks,

greg k-h

RE: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Raj, Ashok]

Hi Greg

I don't think re-implementing was the idea, but the motivation was that
this code was origininally developed for multiple operating environments
such as EFI for preboot, and also some embedded environment. This was
designed to just ease effort in porting.

Iam not sure what you point out that is not done correctly, please help
us understand.

Thanks
ashok

>On Thu, Feb 05, 2004 at 10:01:13AM -0800, Hefty, Sean wrote:
>> As an FYI, the code is available for download on bitkeeper at
>> http://infiniband.bkbits.net/iba.  We're still working on providing a
>> tarball and patch for 2.6, but if you would like to see the code now,
it
>> is available.
>
>Oh, I've seen that code, and still feel ill after looking at some of
>it...
>
>Come on, implementing your own spinlocks (and getting it wrong) and
>atomit_t?  Why in the world would you _ever_ want to do that.
>



>That code needs a _lot_ of cleanup to make it into the kernel tree.
>
>Good luck,
>
>greg k-h
>

RE: [Infiniband-general] Getting an Infiniband access layer in the Linux kernel

[Tillier, Fabian]

Greg,

The atomic operation abstraction is there because atomic support has
different prototypes between x86 and IA64 (at least it did when it was
written), with some of the x86 functions not returning values while the
IA64 ones did.  Further, comments in the x86 code base indicated that
only 24-bits are actually valid (probably from some i386 limitation that
is no longer relevant).  Thus, the implementation of atomic operations
for the x86 processor architecture fails a build (via #error) if you're
targeting an i386 processor, and provides the same semantics independent
of processor architecture.

To answer your broader question, the reason to have abstraction is to
facilitate portability.  I'm not just talking about portability between
different operating systems, but even portability between different
versions of Linux distributions and kernels.  Differences between Linux
distributions and kernel versions can be handled in a single place,
avoiding the need to pepper the rest of the code base with #ifdefs.
This results in more readable and maintainable code for the rest of the
project by concentrating platform specific issues to the abstraction
layer.

Are you suggesting that if there is any abstraction, the code will never
be accepted?  Or rather that the abstraction better be correct?  I'm
hoping for the latter, however please clarify.

- Fab

-----Original Message-----
From: Greg KH [mailto:greg@kroah.com] 
Sent: Thursday, February 05, 2004 10:10 AM
To: Hefty, Sean; linux-kernel@vger.kernel.org
Cc: Troy Benjegerdes; Woodruff, Robert J; Magro, Bill; Woodruff, Robert
J; infiniband-general@lists.sourceforge.net
Subject: Re: [Infiniband-general] Getting an Infiniband access layer in
the linux kernel

On Thu, Feb 05, 2004 at 10:01:13AM -0800, Hefty, Sean wrote:
> As an FYI, the code is available for download on bitkeeper at
> http://infiniband.bkbits.net/iba.  We're still working on providing a
> tarball and patch for 2.6, but if you would like to see the code now,
it
> is available.

Oh, I've seen that code, and still feel ill after looking at some of
it...

Come on, implementing your own spinlocks (and getting it wrong) and
atomit_t?  Why in the world would you _ever_ want to do that.

That code needs a _lot_ of cleanup to make it into the kernel tree.

Good luck,

greg k-h


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Infiniband-general mailing list
Infiniband-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/infiniband-general

Re: [Infiniband-general] Getting an Infiniband access layer in the Linux kernel

[Greg KH]

On Thu, Feb 05, 2004 at 01:31:45PM -0500, Tillier, Fabian wrote:
> 
> Are you suggesting that if there is any abstraction, the code will never
> be accepted?  Or rather that the abstraction better be correct?  I'm
> hoping for the latter, however please clarify.

The kernel has its own abstractions that seem to be working quite well
for all different types of platforms.  There is no need for you to
create your own, just for a driver subsystem.

If you have found any problems with the current locks please let the
entire kernel community benefit from your changes, and not relegate them
to a infiniband-only section of the kernel.

So yes, if you add your own versions of spinlocks and atomic_t types,
your code will be rejected, among other things :)

thanks,

greg k-h

Re: [Infiniband-general] Getting an Infiniband access layer in the Linux kernel

[Randy.Dunlap]

On Thu, 5 Feb 2004 13:31:45 -0500 "Tillier, Fabian" <ftillier@infiniconsys.com> wrote:

| Greg,
| 
| The atomic operation abstraction is there because atomic support has
| different prototypes between x86 and IA64 (at least it did when it was
| written), with some of the x86 functions not returning values while the
| IA64 ones did.  Further, comments in the x86 code base indicated that
| only 24-bits are actually valid (probably from some i386 limitation that
| is no longer relevant).  Thus, the implementation of atomic operations
| for the x86 processor architecture fails a build (via #error) if you're
| targeting an i386 processor, and provides the same semantics independent
| of processor architecture.
| 
| To answer your broader question, the reason to have abstraction is to
| facilitate portability.  I'm not just talking about portability between
| different operating systems, but even portability between different
| versions of Linux distributions and kernels.  Differences between Linux
| distributions and kernel versions can be handled in a single place,
| avoiding the need to pepper the rest of the code base with #ifdefs.
| This results in more readable and maintainable code for the rest of the
| project by concentrating platform specific issues to the abstraction
| layer.

Besides Greg's comments:

Can (will) you provide specific examples of these differences/problems?
They need to be quashed.

Thanks.

| Are you suggesting that if there is any abstraction, the code will never
| be accepted?  Or rather that the abstraction better be correct?  I'm
| hoping for the latter, however please clarify.
| 
| - Fab
| 
| -----Original Message-----
| From: Greg KH [mailto:greg@kroah.com] 
| Sent: Thursday, February 05, 2004 10:10 AM
| To: Hefty, Sean; linux-kernel@vger.kernel.org
| Cc: Troy Benjegerdes; Woodruff, Robert J; Magro, Bill; Woodruff, Robert
| J; infiniband-general@lists.sourceforge.net
| Subject: Re: [Infiniband-general] Getting an Infiniband access layer in
| the linux kernel
| 
| On Thu, Feb 05, 2004 at 10:01:13AM -0800, Hefty, Sean wrote:
| > As an FYI, the code is available for download on bitkeeper at
| > http://infiniband.bkbits.net/iba.  We're still working on providing a
| > tarball and patch for 2.6, but if you would like to see the code now,
| it
| > is available.
| 
| Oh, I've seen that code, and still feel ill after looking at some of
| it...
| 
| Come on, implementing your own spinlocks (and getting it wrong) and
| atomit_t?  Why in the world would you _ever_ want to do that.
| 
| That code needs a _lot_ of cleanup to make it into the kernel tree.
| 
| Good luck,
| 
| greg k-h


--
~Randy
kernel-janitors project:  http://janitor.kernelnewbies.org/

Re: [Infiniband-general] Getting an Infiniband access layer in the Linux kernel

[Jamie Lokier]

Tillier, Fabian wrote:
> Further, comments in the x86 code base indicated that only 24-bits
> are actually valid (probably from some i386 limitation that is no
> longer relevant).

It is because of the Sparc architecture.  I think Sparc doesn't (or
perhaps didn't) have atomic operations, so a cunning hack is used to
simulate them.

-- Jamie

Re: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Greg KH]

On Thu, Feb 05, 2004 at 10:01:13AM -0800, Hefty, Sean wrote:
> As an FYI, the code is available for download on bitkeeper at
> http://infiniband.bkbits.net/iba.  We're still working on providing a
> tarball and patch for 2.6, but if you would like to see the code now, it
> is available.

Oh, I've seen that code, and still feel ill after looking at some of
it...

Come on, implementing your own spinlocks (and getting it wrong) and
atomit_t?  Why in the world would you _ever_ want to do that.

That code needs a _lot_ of cleanup to make it into the kernel tree.

Good luck,

greg k-h

RE: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Woodruff, Robert J]

I agree we would not need to wait for a vendor driver to put something
together
that the community could start to review. I see what I can do to get
something
put together. 

-----Original Message-----
From: Troy Benjegerdes [mailto:hozer@hozed.org] 
Sent: Tuesday, February 03, 2004 8:09 PM
To: Woodruff, Robert J; Magro, Bill
Cc: Greg KH; Woodruff, Robert J;
infiniband-general@lists.sourceforge.net; linux-kernel@vger.kernel.org
Subject: Re: [Infiniband-general] Getting an Infiniband access layer in
the linux kernel


I don't think we need to wait for a vendor driver to put together a 2.6
IB access layer patch for review. I for one would like to see what
*just* the access layer, without anything else looks like.

Those of us with hardware know it works on 2.4 with a mellanox VPD.. the
real question is what the code looks like.

There are outstanding issues I've seen in performance testing related to
integration with the memory management subsystem.. if we integrate
directly into 2.6, we'll have the opportunity (but not the requirement) 
for more intelligent ways to deal locking and unlocking of memory. I
suspect that someone with more MMU-foo than myself will see some
intelligent way of useing the rmap changes.

In particular, I want someone paranoid about security to examine the
security model for allowing what processes get access to what IB card
resources.

On Tue, Feb 03, 2004 at 06:01:20PM -0800, Woodruff, Robert J wrote:
> Good I think the InfiniBand community would like to get some of the
> code into 2.6. 
> 
> The access layer is all loadable modules and does not require any 
> kernel changes. But we will need to look into integrating it into the 
> build environment (and provide a patch for that).
> 
> We will work with the InfiniBand HCA vendors, such as Mellanox and 
> Fujitsu, to get the 2.6 code tested on the hardware and put together a

> patch that can be submitted for review by the larger linux community. 
> If you would like to see the code before we
> run it on 2.6, we could possibly do that as well. 
> 
> 
> woody
> 
> 
> -----Original Message-----
> From: Greg KH [mailto:greg@kroah.com]
> Sent: Tuesday, February 03, 2004 5:03 PM
> To: Woodruff, Robert J
> Cc: Troy Benjegerdes; Woodruff, Robert J;
> infiniband-general@lists.sourceforge.net; linux-kernel@vger.kernel.org
> Subject: Re: [Infiniband-general] Getting an Infiniband access layer
in
> the linux kernel
> 
> 
> On Tue, Feb 03, 2004 at 04:17:36PM -0800, Woodruff, Robert J wrote:
> > 
> > I heard from a friend of mine that 2.6 was closed to new features. 
> > What sayith the community on allowing additional experimental 
> > drivers (like the infiniband access layer) into 2.6 ? Can we still 
> > submit something or do we have to wait till 2.7 ?
> 
> If it's a subsystem that does not touch any of the existing kernel 
> (with the exception of adding it to the build and Kconfig), then it 
> probably will not be a problem to add.
> 
> But then again, that all depends on what the code looks like, and none

> of us have seen that yet :)
> 
> thanks,
> 
> greg k-h

-- 
------------------------------------------------------------------------
--
Troy Benjegerdes                'da hozer'
hozer@hozed.org  

Somone asked my why I work on this free (http://www.fsf.org/philosophy/)
software stuff and not get a real job. Charles Shultz had the best
answer:

"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't. That's
why I draw cartoons. It's my life." -- Charles Shultz

RE: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Woodruff, Robert J]

Good I think the InfiniBand community would like to get some of the 
code into 2.6. 

The access layer is all loadable modules and does not require any kernel
changes. 
But we will need to look into integrating it into the build environment 
(and provide a patch for that). 

We will work with the InfiniBand HCA vendors, such as Mellanox and
Fujitsu, to 
get the 2.6 code tested on the hardware and put together a patch that
can be submitted for review by the larger linux community. If you would
like to see the code before we
run it on 2.6, we could possibly do that as well. 


woody


-----Original Message-----
From: Greg KH [mailto:greg@kroah.com] 
Sent: Tuesday, February 03, 2004 5:03 PM
To: Woodruff, Robert J
Cc: Troy Benjegerdes; Woodruff, Robert J;
infiniband-general@lists.sourceforge.net; linux-kernel@vger.kernel.org
Subject: Re: [Infiniband-general] Getting an Infiniband access layer in
the linux kernel


On Tue, Feb 03, 2004 at 04:17:36PM -0800, Woodruff, Robert J wrote:
> 
> I heard from a friend of mine that 2.6 was closed to new features.
> What sayith the community on allowing additional experimental drivers
> (like the infiniband access layer) into 2.6 ? Can we still submit
> something or do we have to wait till 2.7 ?

If it's a subsystem that does not touch any of the existing kernel (with
the exception of adding it to the build and Kconfig), then it probably
will not be a problem to add.

But then again, that all depends on what the code looks like, and none
of us have seen that yet :)

thanks,

greg k-h

Re: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Troy Benjegerdes]

I don't think we need to wait for a vendor driver to put together a 2.6
IB access layer patch for review. I for one would like to see what
*just* the access layer, without anything else looks like.

Those of us with hardware know it works on 2.4 with a mellanox VPD.. the
real question is what the code looks like.

There are outstanding issues I've seen in performance testing related to
integration with the memory management subsystem.. if we integrate
directly into 2.6, we'll have the opportunity (but not the requirement) 
for more intelligent ways to deal locking and unlocking of memory.
I suspect that someone with more MMU-foo than myself will see some
intelligent way of useing the rmap changes.

In particular, I want someone paranoid about security to examine the
security model for allowing what processes get access to what IB card
resources.

On Tue, Feb 03, 2004 at 06:01:20PM -0800, Woodruff, Robert J wrote:
> Good I think the InfiniBand community would like to get some of the 
> code into 2.6. 
> 
> The access layer is all loadable modules and does not require any kernel
> changes. 
> But we will need to look into integrating it into the build environment 
> (and provide a patch for that). 
> 
> We will work with the InfiniBand HCA vendors, such as Mellanox and
> Fujitsu, to 
> get the 2.6 code tested on the hardware and put together a patch that
> can be submitted for review by the larger linux community. If you would
> like to see the code before we
> run it on 2.6, we could possibly do that as well. 
> 
> 
> woody
> 
> 
> -----Original Message-----
> From: Greg KH [mailto:greg@kroah.com] 
> Sent: Tuesday, February 03, 2004 5:03 PM
> To: Woodruff, Robert J
> Cc: Troy Benjegerdes; Woodruff, Robert J;
> infiniband-general@lists.sourceforge.net; linux-kernel@vger.kernel.org
> Subject: Re: [Infiniband-general] Getting an Infiniband access layer in
> the linux kernel
> 
> 
> On Tue, Feb 03, 2004 at 04:17:36PM -0800, Woodruff, Robert J wrote:
> > 
> > I heard from a friend of mine that 2.6 was closed to new features.
> > What sayith the community on allowing additional experimental drivers
> > (like the infiniband access layer) into 2.6 ? Can we still submit
> > something or do we have to wait till 2.7 ?
> 
> If it's a subsystem that does not touch any of the existing kernel (with
> the exception of adding it to the build and Kconfig), then it probably
> will not be a problem to add.
> 
> But then again, that all depends on what the code looks like, and none
> of us have seen that yet :)
> 
> thanks,
> 
> greg k-h

-- 
--------------------------------------------------------------------------
Troy Benjegerdes                'da hozer'                hozer@hozed.org  

Somone asked my why I work on this free (http://www.fsf.org/philosophy/)
software stuff and not get a real job. Charles Shultz had the best answer:

"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't. That's why
I draw cartoons. It's my life." -- Charles Shultz

RE: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Woodruff, Robert J]

If the community would like to see a 2.6 patch for just the infiniband
access layer
for starters, I don't think it would be to hard to put something
together. 

I heard from a friend of mine that 2.6 was closed to new features. 
What sayith the community on allowing additional experimental drivers
(like the
infiniband access layer) into 2.6 ? Can we still submit something or do
we
have to wait till 2.7 ?

In any case, I am sure we can put together a patch and/or a tarball and
post it
to the sourceforge infiniband web site for those who want to play with
the
infiniband code on 2.6. 

-----Original Message-----
From: Troy Benjegerdes [mailto:hozer@hozed.org] 
Sent: Tuesday, February 03, 2004 2:37 PM
To: Woodruff, Robert J
Cc: infiniband-general@lists.sourceforge.net;
linux-kernel@vger.kernel.org
Subject: Re: [Infiniband-general] Getting an Infiniband access layer in
the linux kernel


So, for starters, can you or Jerrie post a patch containing just the 2.6
infiniband access layer, Or create a new BK tree for 2.6 infiniband
stuff that uses the new 2.6 kbuild system?




On Mon, Feb 02, 2004 at 03:58:56PM -0800, Woodruff, Robert J wrote:
> 
> We were waiting until we had some version of the InfiniBand code 
> ported to 2.6 before asking for it to be included in the 2.6 kernel 
> tree. Jerrie made the changes
> to the IB access layer to allow it to compile on 2.6, but it cannot
yet
> be tested 
> till we get a 2.6 driver from Mellanox. 
> 
> I'd also like to hear from the linux-kernel folks on what we would 
> need to do to get a basic InfiniBand access layer included in the 2.6 
> base.
> 
> We'd also like to hear from Mellanox if they have any plans to provide

> an open source VPD driver anytime soon.
> 
> woody
> 

Re: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Greg KH]

On Tue, Feb 03, 2004 at 04:17:36PM -0800, Woodruff, Robert J wrote:
> 
> I heard from a friend of mine that 2.6 was closed to new features. 
> What sayith the community on allowing additional experimental drivers
> (like the infiniband access layer) into 2.6 ? Can we still submit
> something or do we have to wait till 2.7 ?

If it's a subsystem that does not touch any of the existing kernel (with
the exception of adding it to the build and Kconfig), then it probably
will not be a problem to add.

But then again, that all depends on what the code looks like, and none
of us have seen that yet :)

thanks,

greg k-h

RE: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Woodruff, Robert J]

Hi Masonori, 

I think that Sean has already pushed the code changes for 2.6 for 
complib, IBAL, and SDP.  As I stated before, we have yet to test it on
2.6,
but it should now compile. Let us know if you have any issues.


-----Original Message-----
From: Masanori ITOH [mailto:marc@labs.fujitsu.com] 
Sent: Monday, February 02, 2004 5:39 PM
To: woody@jf.intel.com
Cc: hozer@hozed.org; infiniband-general@lists.sourceforge.net;
linux-kernel@vger.kernel.org
Subject: Re: [Infiniband-general] Getting an Infiniband access layer in
the linux kernel


Hello Folks,

Message-ID:
<F595A0622682C44DBBE0BBA91E56A5ED1C3662@orsmsx410.jf.intel.com>
> 
> We were waiting until we had some version of the InfiniBand code 
> ported to 2.6 before asking for it to be included in the 2.6 kernel 
> tree. Jerrie made the changes
> to the IB access layer to allow it to compile on 2.6, but it cannot
yet
> be tested 
> till we get a 2.6 driver from Mellanox. 

Now, I'm testing a VPD for Fujitsu InfiniBand HCA card.
The VPD requires no special SDK such as VAPI of Mellanox.

In order to do that, I made some changes locally on IBAL and COMPLIB
layers based on the change set 1.207. I still have some problems, but
opensm and ipoib are working on linux-2.6.1 environment.

Robert, if you could make Jarrie's modification public at bkbits.net, I
can test the modification for 2.6 kernel using Fujitsu HCA.

Meanwhile, Mellanox is not the only supplier of InfiniBand HCA. Fujitsu,
Japan, also has InfiniBand HCA card with an original LSI 
different from Tavor chip.

Thanks in advance,
Masanori

> I'd also like to hear from the linux-kernel folks on what we would 
> need to do to get a basic InfiniBand access layer included in the 2.6 
> base.
> 
> We'd also like to hear from Mellanox if they have any plans to provide

> an open source VPD driver anytime soon.
> 
> woody

---
Masanori ITOH  Grid Computing & Bioinformatics Lab., Fujitsu Labs. LTD.
               e-mail: marc@labs.fujitsu.com
               phone: +81-44-754-2628 (extension: 7112-6227)
               FingerPrint: 55AF C562 E415 FB1A 8A3A  35D1 AB40 8A9D
B8B1 99F8

Re: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Masanori ITOH]

Hi Robert, Sean,

Message-ID: <F595A0622682C44DBBE0BBA91E56A5ED1C3664@orsmsx410.jf.intel.com>
> Hi Masonori, 
> 
> I think that Sean has already pushed the code changes for 2.6 for 
> complib, IBAL, and SDP.  As I stated before, we have yet to test it on
> 2.6,
> but it should now compile. Let us know if you have any issues.

Currently, it doesn't seem to be available for 'bk pull', but
I will check it out as soon as it becomes public.

http://infiniband.bkbits.net:8080/iba/ChangeSet@-7d?nav=index.html
----
|Linux InfiniBand(tm) Project 
|-----------------------------------------------------------------------------
|ChangeSet Summaries 
|Age      Author   Rev   Comments 
|35 hours ardavis  1.221 878599 SDP fix for faster rejected connect requests.  
|6 days   jlcoffma 1.220 Fix for SourceForge bug number 880274 for build on the Linux 2.6kernel base. 

Thanks and stay tuned, :)
Masanori

---
Masanori ITOH  Grid Computing & Bioinformatics Lab., Fujitsu Labs. LTD.
               e-mail: marc@labs.fujitsu.com
               phone: +81-44-754-2628 (extension: 7112-6227)
               FingerPrint: 55AF C562 E415 FB1A 8A3A  35D1 AB40 8A9D B8B1 99F8

RE: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Woodruff, Robert J]

We were waiting until we had some version of the InfiniBand code ported
to 2.6 before
asking for it to be included in the 2.6 kernel tree. Jerrie made the
changes
to the IB access layer to allow it to compile on 2.6, but it cannot yet
be tested 
till we get a 2.6 driver from Mellanox. 

I'd also like to hear from the linux-kernel folks on what we would need
to
do to get a basic InfiniBand access layer included in the 2.6 base. 

We'd also like to hear from Mellanox if they have any plans to provide
an open
source VPD driver anytime soon. 

woody



-----Original Message-----
From: infiniband-general-admin@lists.sourceforge.net
[mailto:infiniband-general-admin@lists.sourceforge.net] On Behalf Of
Troy Benjegerdes
Sent: Monday, February 02, 2004 2:31 PM
To: infiniband-general@lists.sourceforge.net;
linux-kernel@vger.kernel.org
Subject: [Infiniband-general] Getting an Infiniband access layer in the
linux kernel


There are 10gigabit ethernet drivers in the kernel.org 2.6, but no
infiniband drivers. Infiniband cards are faster, AND cheaper than 10gig
hardware right now, so this makes no sense.

Has anyone gotten on linux-kernel and asked about what it would take to
get the infiniband access layer into the kernel? (Once that's there we
could *start* talking about IPOIB, and such)

The first, and biggest problem is the access layer is of little use
without a VPD, and since mellanox is the only game in town right now, we
need mellanox to release a VPD that's GPL-compatible. This is mellanox's
problem, so we can't really do anything but tell them to hurry up.

Second is extracting the code into smaller digestible chunks that people
aren't going to scream over. So, can someone tell me how to extract just
the access layer out of the infiniband.bkbits.net tree? I'd like just
enough to be able to run something like NetPIPE on the intel verbs
layer.

And can someone from lkml please respond with suggestions on what would
have a chance of getting blessed with the appropriate penguin-pee?



-- 
------------------------------------------------------------------------
--
Troy Benjegerdes                'da hozer'
hozer@hozed.org  

Somone asked my why I work on this free (http://www.fsf.org/philosophy/)
software stuff and not get a real job. Charles Shultz had the best
answer:

"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't. That's
why I draw cartoons. It's my life." -- Charles Shultz


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration See the
breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Infiniband-general mailing list Infiniband-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/infiniband-general

Re: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Greg KH]

On Mon, Feb 02, 2004 at 03:58:56PM -0800, Woodruff, Robert J wrote:
> 
> I'd also like to hear from the linux-kernel folks on what we would need
> to do to get a basic InfiniBand access layer included in the 2.6 base. 

How about submitting a patch to lkml that follows the guidelines in
Documentation/SubmittingPatches and Documentation/CodingStyle?  That
would be a good start...

thanks,

greg k-h

Re: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Masanori ITOH]

Hello Folks,

Message-ID: <F595A0622682C44DBBE0BBA91E56A5ED1C3662@orsmsx410.jf.intel.com>
> 
> We were waiting until we had some version of the InfiniBand code ported
> to 2.6 before
> asking for it to be included in the 2.6 kernel tree. Jerrie made the
> changes
> to the IB access layer to allow it to compile on 2.6, but it cannot yet
> be tested 
> till we get a 2.6 driver from Mellanox. 

Now, I'm testing a VPD for Fujitsu InfiniBand HCA card.
The VPD requires no special SDK such as VAPI of Mellanox.

In order to do that, I made some changes locally on IBAL and COMPLIB layers
based on the change set 1.207. I still have some problems, but opensm and
ipoib are working on linux-2.6.1 environment.

Robert, if you could make Jarrie's modification public at bkbits.net,
I can test the modification for 2.6 kernel using Fujitsu HCA.

Meanwhile, Mellanox is not the only supplier of InfiniBand HCA.
Fujitsu, Japan, also has InfiniBand HCA card with an original LSI 
different from Tavor chip.

Thanks in advance,
Masanori

> I'd also like to hear from the linux-kernel folks on what we would need
> to
> do to get a basic InfiniBand access layer included in the 2.6 base. 
> 
> We'd also like to hear from Mellanox if they have any plans to provide
> an open
> source VPD driver anytime soon. 
> 
> woody

---
Masanori ITOH  Grid Computing & Bioinformatics Lab., Fujitsu Labs. LTD.
               e-mail: marc@labs.fujitsu.com
               phone: +81-44-754-2628 (extension: 7112-6227)
               FingerPrint: 55AF C562 E415 FB1A 8A3A  35D1 AB40 8A9D B8B1 99F8

Re: [Infiniband-general] Getting an Infiniband access layer in the linux kernel

[Troy Benjegerdes]

So, for starters, can you or Jerrie post a patch containing just the 2.6
infiniband access layer, Or create a new BK tree for 2.6 infiniband stuff
that uses the new 2.6 kbuild system?




On Mon, Feb 02, 2004 at 03:58:56PM -0800, Woodruff, Robert J wrote:
> 
> We were waiting until we had some version of the InfiniBand code ported
> to 2.6 before
> asking for it to be included in the 2.6 kernel tree. Jerrie made the
> changes
> to the IB access layer to allow it to compile on 2.6, but it cannot yet
> be tested 
> till we get a 2.6 driver from Mellanox. 
> 
> I'd also like to hear from the linux-kernel folks on what we would need
> to
> do to get a basic InfiniBand access layer included in the 2.6 base. 
> 
> We'd also like to hear from Mellanox if they have any plans to provide
> an open
> source VPD driver anytime soon. 
> 
> woody
>