Re: dumping queue state

[Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 3224 bytes --]

On Wed, Dec 20, 2017 at 11:42:22AM -0600, Steve Wise wrote:
> > WIP: no manuals, no review yet and not massively tested - but works for me
> :
> >
> >
> > >
> > > Is there a way to extend commands with provider-specific data?
> > >
> >
> > In future yes, rdmatool provides "-d" option (detailed output) and IMHO
> > all vendor specific will got there.
>
> Cool.
>
> Looksie:
>
> [root@stevo1 iproute2]# ./rdma/rdma res
> 1: mlx4_0: curr/max: pd 4/32764 cq 17/65408 qp 3/163768
> 2: cxgb4_0: curr/max: pd 2/65536 cq 16/24576 qp 2/12288
> 2: cxgb4_0: curr/max: pd 2/65536 cq 16/24576 qp 2/12288
>
> [root@stevo1 iproute2]# ./rdma/rdma res show qp link cxgb4_0
> DEV/PORT  LQPN       TYPE  STATE  PID        COMM
> cxgb4_0/- 1028       RC    RTS    0          [rdma-cm]
> cxgb4_0/- 1026       RC    RTS    0          [rdma-cm]
>
> I'm not sure why the port is not specified on the above cxgb4 QPs which are
> connected.  They should have a port number.  But perhaps it isn't filled in
> correctly in the cm_id or something?

I'm not using CM and take this information from ib_query_qp
https://git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git/tree/drivers/infiniband/core/nldev.c?h=topic/restrack-rcu#n236
https://git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git/tree/drivers/infiniband/core/nldev.c?h=topic/restrack-rcu#n249

Most probably it returns port = 0 and it is not valid for the RDMAtool.

>
> [root@stevo1 iproute2]# ./rdma/rdma res show qp link mlx4_0
> DEV/PORT  LQPN       TYPE  STATE  PID        COMM
> mlx4_0/1  537        RC    RTS    0          [rdma-cm]
> mlx4_0/1  536        RC    RTS    0          [rdma-cm]
> mlx4_0/1  1          GSI   RTS    0          [rdma-mad]
> [root@stevo1 iproute2]#
>
> rdma should probably understand enough transport-specific details to not
> display ib-specific stuff on an iwarp qp.  Here PSNs and PATH-MIG are all
> IB-specific:

Right, it is bug.

>
> [root@stevo1 iproute2]# ./rdma/rdma res show qp link cxgb4_0 -d
> DEV/PORT  LQPN       RQPN       TYPE  STATE  PID        COMM
> SQ-PSN     RQ-PSN     PATH-MIG
> cxgb4_0/- 1028       0          RC    RTS    0          [rdma-cm]       0
> 0          MIGRATED
> cxgb4_0/- 1026       0          RC    RTS    0          [rdma-cm]       0
> 0          MIGRATED
>
> [root@stevo1 iproute2]# ./rdma/rdma res show qp link mlx4_0 -d
> DEV/PORT  LQPN       RQPN       TYPE  STATE  PID        COMM
> SQ-PSN     RQ-PSN     PATH-MIG
> mlx4_0/1  537        537        RC    RTS    0          [rdma-cm]
> 13145128   12441844   MIGRATED
> mlx4_0/1  536        536        RC    RTS    0          [rdma-cm]
> 14218681   1982803    MIGRATED
> mlx4_0/1  1          ---        GSI   RTS    0          [rdma-mad]      0
> ---        ---
>
> Looking good though!

Check filters options too, to filters by numbers and display columns.

>
> Why did you prefer netlink vs debugfs for this stuff?  (I'm sure it was
> discussed on the list but I missed it).

Three reasons:
1. We wanted rdmatool to be part of iproute2.
2. Debugfs requires root access and netlink doesn't
3. Better extensibility, if the device doesn't support the netlink won't
provide attribute and all neltink libraries know how to deal with it.

>
> Steve.
>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]