Hi Chet,

the trick is to check out the latest pkg-ofed source from debian SVN
(svn://svn.debian.org/svn/pkg-ofed/) and to update the upstream source
by merging the stuff by extracting the source RPMs or even better by
importing the source directly from the git repos of the OFED user space.
In the "debian" directory there are some patches e.g. which change some
stuff in shell scripts for the dash. These need to be adopted.

But you'll have to ensure that the kernel code matches the OFED user
space. The kernel stuff included in OFED doesn't support latest kernels
and is based on an older code base (e.g. OFED 1.5.4 kernel stuff is
based on 2.6.30). I hope that you don't need iSER. The open-iscsi kernel
stuff in there is also based on 2.6.30 which means that you would need
old open-iscsi user space.

This is why we've decided to follow what they call "upstream" in this
list. This means: Use the OFED kernel code from the matching vanilla
kernel from kernel.org.

Here a simple list of matching code:
OFED-1.5.4  ---> kernel 3.2.x
OFED-1.5.4.1 ---> kernel 3.3.x

I've attached the IB user space HOWTO from Or Gerlitz for the git repos.
Some of the git repos already have a debinan directory.

Do you know how to build Debian packages?

Cheers,
Sebastian


On 22/06/12 02:46, Chet Murthy wrote:
> 
> Hi,
> 
> A long while ago, I got OFED 1.5.2 working on Ubuntu 10.04 (Lucid) on
> Opterons with Mellanox DDR cards.  It was a little messy, getting the
> RPMs compiled, but it was pretty straightforward.  Basically, I (a)
> built a kernel with neither infiniband nor mellanox ethernet drivers,
> and (b) ran the OFED install.pl with some minor modifications to
> convert the RPMs into DEBs as they were built.  And everything worked,
> smooth as a whistle.
> 
> Today, I tried to do the same thing with OFED 1.5.4.1, and while the
> process of -building- was straightforward, once I get done, the card's
> state is all zeroes:
> 
> chet@memstore3:~$ sudo ibstatus
> Infiniband device 'mlx4_0' port 1 status:
>         default gid:     0000:0000:0000:0000:0000:0000:0000:0000
>         base lid:        0x0
>         sm lid:          0x0
>         state:           1: DOWN
>         phys state:      3: Disabled
>         rate:            2.5 Gb/sec (1X)
>         link_layer:      Ethernet
> 
> Infiniband device 'mlx4_0' port 2 status:
>         default gid:     0000:0000:0000:0000:0000:0000:0000:0000
>         base lid:        0x0
>         sm lid:          0x0
>         state:           1: DOWN
>         phys state:      3: Disabled
>         rate:            2.5 Gb/sec (1X)
>         link_layer:      Ethernet
> 
> The card's a modern ConnectX
> 
> 1f:00.0 Ethernet controller: Mellanox Technologies MT26448 [ConnectX EN 
> 10GigE, PCIe 2.0 5GT/s] (rev b0)
> 
> and on identical RedHat machines, the card's status is quite
> different:
> 
> 
> [root@memstore4 chet]# ibstatus
> Infiniband device 'mlx4_0' port 1 status:
>         default gid:     fe80:0000:0000:0000:0202:c9ff:fe4b:5890
>         base lid:        0x0
>         sm lid:          0x0
>         state:           1: DOWN
>         phys state:      3: Disabled
>         rate:            10 Gb/sec (1X QDR)
>         link_layer:      Ethernet
> 
> Infiniband device 'mlx4_0' port 2 status:
>         default gid:     fe80:0000:0000:0000:0202:c9ff:fe4b:5891
>         base lid:        0x0
>         sm lid:          0x0
>         state:           4: ACTIVE
>         phys state:      5: LinkUp
>         rate:            10 Gb/sec (1X QDR)
>         link_layer:      Ethernet
> 
> I'm not even sure how to go about debugging this.  Has anybody gotten
> OFED to work on Ubuntu with such modern cards?
> 
> Thanks,
> --chet--
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html