Re: Is it ok to use debugfs to dump some ucontext-level driver-defined info?

[liweihang <liweihang@huawei.com>]

On 2020/12/31 18:39, Leon Romanovsky wrote:
> On Thu, Dec 31, 2020 at 09:36:27AM +0000, liweihang wrote:
>> On 2020/12/31 15:59, Leon Romanovsky wrote:
>>> On Tue, Dec 29, 2020 at 01:31:39PM +0000, liweihang wrote:
>>>> Hi all,
>>>>
>>>> We want to dump some hns driver-defined information that belongs to a
>>>> process to keep track of current memory usage. For example, there is
>>>> a ucontext-level(process-level) memory pool to store WQE which is
>>>> shared by a lot of QPs, we want to record and query which QPs are using
>>>> this pool and how much space each QP is using.
>>>>
>>>> rdmatool don't have a ucontext-level resource tracking currently, is it
>>>> ok to achieve that through debugfs?
>>>>
>>>> This may looks like:
>>>>
>>>> $ echo 1 > <dbgfs_dir>/hns_roce/hns_0/<pid>/qp
>>>> QPN        Total(kB)  SQ(kB)     SGE(kB)    RQ(kB)
>>>> 110        6400       256        2048       4096
>>>> 118        6400       256        2048       0
>>>>
>>>> Or should it be achieved in rdmatool?
>>>
>>> I think so, because PID != ucontext. Why can't it be presented as QP
>>> attribute? Can you please send "rdmatool" example?
>>>
>>> Thanks
>>
>> Hi Leon,
>>
>> Thanks for your response. If we can achieve it in rdmatool, it may
>> looks like:
>>
>> 1) We want to get some information of a ucontext (assuming that each
>> ucontext has an ID), for example, the size of a memory pool that belongs
>> to a ucontext as I mentioned above:
>>
>> $ rdma res show ucontext
>> uctx_id	1 pid 20 qp_buf_sz 6400 sq_buf_sz 256 sge_buf_sz 2048 rq_buf_sz 4096
>> uctx_id	2 pid 20 qp_buf_sz 4800 sq_buf_sz 128 sge_buf_sz 2048 rq_buf_sz 2048
> 
> I have no problems to add "rdma res show ucontext" command, we just need
> to find what should be printed.
> 
>>
>> 2) We want to know which ucontext a QP belongs to:
>>
>> $ rdma res show qp
>> link hns_0/1 lqpn 1 type GSI ... uctx_id 1
>> link hns_0/1 lqpn 2 type RC ... uctx_id 1
>>
>> So the question is, we don't have a ucontext-level restrack currently, and
>> there in no 'id' for each ucontext.
> 
> We have IDs for every ucontext, it is called "ctxn" and because QP is
> not connected directly to ucontext, but through PDs, it is visible
> when you check PDs.
> 
> [leonro@vm ~]$ ibv_rc_pingpong &
> [leonro@vm ~]$ rdma res show
> 0: ibp0s9: pd 4 cq 4 qp 4 cm_id 0 mr 1 ctx 1
> [leonro@vm ~]$ rdma res show qp type RC
> link ibp0s9/1 lqpn 50 rqpn 0 type RC state INIT rq-psn 16777215 sq-psn 0 path-mig-state MIGRATED pdn 3 pid 479 comm ibv_rc_pingpong
> [leonro@vm ~]$ rdma res show pd pdn 3
> ifindex 0 ifname ibp0s9 pdn 3 users 2 ctxn 0 pid 479 comm ibv_rc_pingpong
>                                       ^^^^^^
> 
> Thanks
> 

OK, thank you, we will try to use rdmatool to achieve our goal.

By the way, can we use debugfs to add some trace functions?
Debugfs can quickly get a large amount of information at a time,
it's more convenient than netlink in such situation, especially
for some vendor-defined funtions.

Thanks
Weihang