Old SDR IB HBA

Old SDR IB HBA

[Alexey Shvetsov]

Hi all!

Is it normal that kenrnel reports 8.5G speed on old SDR mellanox HBA?
c-2-1 1 # ibstatus
Infiniband device 'mthca0' port 1 status:
         default gid:     fe80:0000:0000:0000:0002:c902:0023:05a1
         base lid:        0x2f
         sm lid:          0xf
         state:           4: ACTIVE
         phys state:      5: LinkUp
         rate:            8.5 Gb/sec (4X)
         link_layer:      InfiniBand

c-2-1 1 # ibv_devinfo
hca_id: mthca0
         transport:                      InfiniBand (0)
         fw_ver:                         1.0.800
         node_guid:                      0002:c902:0023:05a0
         sys_image_guid:                 0002:c902:0023:05a3
         vendor_id:                      0x02c9
         vendor_part_id:                 25204
         hw_ver:                         0xA0
         board_id:                       MT_03B0120001
         phys_port_cnt:                  1
                 port:   1
                         state:                  PORT_ACTIVE (4)
                         max_mtu:                2048 (4)
                         active_mtu:             2048 (4)
                         sm_lid:                 15
                         port_lid:               47
                         port_lmc:               0x00
                         link_layer:             IB


-- 
Best Regards,
Alexey 'Alexxy' Shvetsov
Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, 
Gatchina, Russia
Department of Molecular and Radiation Biophysics
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexxyum-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
mailto:alexxy-aBrp7R+bbdUdnm+yROfE0A@public.gmane.org
mailto:alexxy-heEE5zn7HfaV8fSEjHp0FA@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Old SDR IB HBA

[Or Gerlitz]

On Tue, May 15, 2012 at 10:16 PM, Alexey Shvetsov <alexxy-aBrp7R+bbdUdnm+yROfE0A@public.gmane.org> wrote:

> Is it normal that kenrnel reports 8.5G speed on old SDR mellanox HBA?

yes, there used to be a bug that could cause that, what kernel
produced this output?

> c-2-1 1 # ibstatus
> Infiniband device 'mthca0' port 1 status:
>        default gid:     fe80:0000:0000:0000:0002:c902:0023:05a1
>        base lid:        0x2f
>        sm lid:          0xf
>        state:           4: ACTIVE
>        phys state:      5: LinkUp
>        rate:            8.5 Gb/sec (4X)
>        link_layer:      InfiniBand

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Old SDR IB HBA

[Alexey Shvetsov]

Affected kernel version 3.3

c-2-1 1 # uname -a
Linux c-2-1.local 3.3.4-gentoo #2 SMP Thu May 3 16:08:32 MSK 2012 
x86_64 Dual Core AMD Opteron(tm) Processor 280 AuthenticAMD GNU/Linux

PS It doesnt have patches related to ib so its like vanilla =D
btw seems like ibv_devinfo reports correct speed
c-2-1 1 # ibv_devinfo -v
hca_id: mthca0
         transport:                      InfiniBand (0)
         fw_ver:                         1.0.800
         node_guid:                      0002:c902:0023:05a0
         sys_image_guid:                 0002:c902:0023:05a3
         vendor_id:                      0x02c9
         vendor_part_id:                 25204
         hw_ver:                         0xA0
         board_id:                       MT_03B0120001
         phys_port_cnt:                  1
         max_mr_size:                    0xffffffffffffffff
         page_size_cap:                  0xfffff000
         max_qp:                         64512
         max_qp_wr:                      16384
         device_cap_flags:               0x00001c76
         max_sge:                        27
         max_sge_rd:                     0
         max_cq:                         65408
         max_cqe:                        131071
         max_mr:                         131056
         max_pd:                         32764
         max_qp_rd_atom:                 4
         max_ee_rd_atom:                 0
         max_res_rd_atom:                258048
         max_qp_init_rd_atom:            128
         max_ee_init_rd_atom:            0
         atomic_cap:                     ATOMIC_HCA (1)
         max_ee:                         0
         max_rdd:                        0
         max_mw:                         0
         max_raw_ipv6_qp:                0
         max_raw_ethy_qp:                0
         max_mcast_grp:                  8192
         max_mcast_qp_attach:            56
         max_total_mcast_qp_attach:      458752
         max_ah:                         0
         max_fmr:                        0
         max_srq:                        960
         max_srq_wr:                     16384
         max_srq_sge:                    27
         max_pkeys:                      64
         local_ca_ack_delay:             15
                 port:   1
                         state:                  PORT_ACTIVE (4)
                         max_mtu:                2048 (4)
                         active_mtu:             2048 (4)
                         sm_lid:                 15
                         port_lid:               47
                         port_lmc:               0x00
                         link_layer:             IB
                         max_msg_sz:             0x80000000
                         port_cap_flags:         0x02510a68
                         max_vl_num:             4 (3)
                         bad_pkey_cntr:          0x0
                         qkey_viol_cntr:         0x0
                         sm_sl:                  0
                         pkey_tbl_len:           64
                         gid_tbl_len:            32
                         subnet_timeout:         18
                         init_type_reply:        0
                         active_width:           4X (2)
                         active_speed:           2.5 Gbps (1)
                         phys_state:             LINK_UP (5)
                         GID[  0]:               
fe80:0000:0000:0000:0002:c902:0023:05a1


Or Gerlitz писал 2012-05-15 22:27:
> On Tue, May 15, 2012 at 10:16 PM, Alexey Shvetsov <alexxy-aBrp7R+bbdUdnm+yROfE0A@public.gmane.org> 
> wrote:
>
>> Is it normal that kenrnel reports 8.5G speed on old SDR mellanox 
>> HBA?
>
> yes, there used to be a bug that could cause that, what kernel
> produced this output?
>
>> c-2-1 1 # ibstatus
>> Infiniband device 'mthca0' port 1 status:
>>        default gid:     fe80:0000:0000:0000:0002:c902:0023:05a1
>>        base lid:        0x2f
>>        sm lid:          0xf
>>        state:           4: ACTIVE
>>        phys state:      5: LinkUp
>>        rate:            8.5 Gb/sec (4X)
>>        link_layer:      InfiniBand
>
> Or.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" 
> in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Best Regards,
Alexey 'Alexxy' Shvetsov
Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, 
Gatchina, Russia
Department of Molecular and Radiation Biophysics
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexxyum-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
mailto:alexxy-aBrp7R+bbdUdnm+yROfE0A@public.gmane.org
mailto:alexxy-heEE5zn7HfaV8fSEjHp0FA@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Old SDR IB HBA

[Or Gerlitz]

On 5/15/2012 10:31 PM, Alexey Shvetsov wrote:
> Affected kernel version 3.3
>

yep, there's a bug in the IB core sysfs code which is fixed in 3.4 that 
can cause that 8.5Gbs to be displayed...
The bug was introduced in 3.2 or 3.3 and fixed in 3.4, see commit 
e9319b0cb00d4d68792fdae37e81e316cb632cb9 "IB/core: Fix SDR rates in 
sysfs" and some later commits to that file

Or.

Or.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html