* [PATCH for-next 0/2] RDMA/hns: Fix GMV entry management @ 2026-09-04 9:38 Junxian Huang 2026-09-04 9:38 ` [PATCH for-next 1/2] RDMA/hns: Limit gmv_entry_num to avoid memory waste Junxian Huang 2026-09-04 9:38 ` [PATCH for-next 2/2] RDMA/hns: Fix GID capacity loss in 64K system Junxian Huang 0 siblings, 2 replies; 5+ messages in thread From: Junxian Huang @ 2026-09-04 9:38 UTC (permalink / raw) To: jgg, leon, linux-rdma; +Cc: tangchengchang, huangjunxian6, linuxarm This patchset fixes two issues in the GMV entry management. Junxian Huang (2): RDMA/hns: Limit gmv_entry_num to avoid memory waste RDMA/hns: Fix GID capacity loss in 64K system drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 40 ++++++++++++++++------ 1 file changed, 30 insertions(+), 10 deletions(-) -- 2.33.0 ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH for-next 1/2] RDMA/hns: Limit gmv_entry_num to avoid memory waste 2026-09-04 9:38 [PATCH for-next 0/2] RDMA/hns: Fix GMV entry management Junxian Huang @ 2026-09-04 9:38 ` Junxian Huang 2026-09-06 9:12 ` Leon Romanovsky 2026-09-04 9:38 ` [PATCH for-next 2/2] RDMA/hns: Fix GID capacity loss in 64K system Junxian Huang 1 sibling, 1 reply; 5+ messages in thread From: Junxian Huang @ 2026-09-04 9:38 UTC (permalink / raw) To: jgg, leon, linux-rdma; +Cc: tangchengchang, huangjunxian6, linuxarm The GMV entry is a HW object corresponding to a GID. Since gid_table_len is already limited to a maximum of 256, there is no need to allocate memory for those extra GMV entries as they will never be touched. Fixes: 7243396aaf12 ("RDMA/hns: Add a max length of gid table") Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com> --- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index 27cc7df55ee7..3edceadbca76 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -2416,8 +2416,7 @@ static void apply_func_caps(struct hns_roce_dev *hr_dev) caps->gmv_bt_num * (HNS_HW_PAGE_SIZE / caps->gmv_entry_sz)); - caps->gmv_entry_num = caps->gmv_bt_num * (HNS_HW_PAGE_SIZE / - caps->gmv_entry_sz); + caps->gmv_entry_num = (u32)caps->gid_table_len[0]; } else { u32 func_num = max_t(u32, 1, hr_dev->func_num); -- 2.33.0 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH for-next 1/2] RDMA/hns: Limit gmv_entry_num to avoid memory waste 2026-09-04 9:38 ` [PATCH for-next 1/2] RDMA/hns: Limit gmv_entry_num to avoid memory waste Junxian Huang @ 2026-09-06 9:12 ` Leon Romanovsky 2026-09-07 2:21 ` Junxian Huang 0 siblings, 1 reply; 5+ messages in thread From: Leon Romanovsky @ 2026-09-06 9:12 UTC (permalink / raw) To: Junxian Huang; +Cc: jgg, linux-rdma, tangchengchang, linuxarm On Fri, Sep 04, 2026 at 05:38:36PM +0800, Junxian Huang wrote: > The GMV entry is a HW object corresponding to a GID. Since gid_table_len > is already limited to a maximum of 256, there is no need to allocate > memory for those extra GMV entries as they will never be touched. > > Fixes: 7243396aaf12 ("RDMA/hns: Add a max length of gid table") > Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com> > --- > drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c > index 27cc7df55ee7..3edceadbca76 100644 > --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c > +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c > @@ -2416,8 +2416,7 @@ static void apply_func_caps(struct hns_roce_dev *hr_dev) > caps->gmv_bt_num * > (HNS_HW_PAGE_SIZE / caps->gmv_entry_sz)); > > - caps->gmv_entry_num = caps->gmv_bt_num * (HNS_HW_PAGE_SIZE / > - caps->gmv_entry_sz); > + caps->gmv_entry_num = (u32)caps->gid_table_len[0]; Please fix the declaration of gid_table_len first. It should be u32 in the first place. Thanks > } else { > u32 func_num = max_t(u32, 1, hr_dev->func_num); > > -- > 2.33.0 > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH for-next 1/2] RDMA/hns: Limit gmv_entry_num to avoid memory waste 2026-09-06 9:12 ` Leon Romanovsky @ 2026-09-07 2:21 ` Junxian Huang 0 siblings, 0 replies; 5+ messages in thread From: Junxian Huang @ 2026-09-07 2:21 UTC (permalink / raw) To: Leon Romanovsky; +Cc: jgg, linux-rdma, tangchengchang, linuxarm On 2026/9/6 17:12, Leon Romanovsky wrote: > On Fri, Sep 04, 2026 at 05:38:36PM +0800, Junxian Huang wrote: >> The GMV entry is a HW object corresponding to a GID. Since gid_table_len >> is already limited to a maximum of 256, there is no need to allocate >> memory for those extra GMV entries as they will never be touched. >> >> Fixes: 7243396aaf12 ("RDMA/hns: Add a max length of gid table") >> Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com> >> --- >> drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 3 +-- >> 1 file changed, 1 insertion(+), 2 deletions(-) >> >> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c >> index 27cc7df55ee7..3edceadbca76 100644 >> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c >> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c >> @@ -2416,8 +2416,7 @@ static void apply_func_caps(struct hns_roce_dev *hr_dev) >> caps->gmv_bt_num * >> (HNS_HW_PAGE_SIZE / caps->gmv_entry_sz)); >> >> - caps->gmv_entry_num = caps->gmv_bt_num * (HNS_HW_PAGE_SIZE / >> - caps->gmv_entry_sz); >> + caps->gmv_entry_num = (u32)caps->gid_table_len[0]; > > Please fix the declaration of gid_table_len first. It should be u32 in the > first place. Will add a patch to fix it. Junxian > > Thanks > >> } else { >> u32 func_num = max_t(u32, 1, hr_dev->func_num); >> >> -- >> 2.33.0 >> ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH for-next 2/2] RDMA/hns: Fix GID capacity loss in 64K system 2026-09-04 9:38 [PATCH for-next 0/2] RDMA/hns: Fix GMV entry management Junxian Huang 2026-09-04 9:38 ` [PATCH for-next 1/2] RDMA/hns: Limit gmv_entry_num to avoid memory waste Junxian Huang @ 2026-09-04 9:38 ` Junxian Huang 1 sibling, 0 replies; 5+ messages in thread From: Junxian Huang @ 2026-09-04 9:38 UTC (permalink / raw) To: jgg, leon, linux-rdma; +Cc: tangchengchang, huangjunxian6, linuxarm Our HW always handles GMV BT pages with a fixed 4K size, but the driver calculates the needed GMV BT pages number with PAGE_SIZE, which is 64K in 64K system. Only the first 4K (GMV index 0-127) can be reached by HW, causing GID capacity loss and memory waste. Split a single 64K BT page into multiple 4K pages and register them to HW per 4K block so that HW can correctly reach all GMV entries. Fixes: 32053e584e4a ("RDMA/hns: Add support for filling GMV table") Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com> --- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 37 +++++++++++++++++----- 1 file changed, 29 insertions(+), 8 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index 3edceadbca76..05d6d3e6d17f 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -4352,18 +4352,39 @@ static int get_op_for_set_hem(struct hns_roce_dev *hr_dev, u32 type, static int config_gmv_ba_to_hw(struct hns_roce_dev *hr_dev, unsigned long obj, dma_addr_t base_addr) { + u32 obj_num_per_bt = HNS_HW_PAGE_SIZE / hr_dev->caps.gmv_entry_sz; + u32 chunk_size = 1 << (hr_dev->caps.gmv_buf_pg_sz + PAGE_SHIFT); + u32 bt_num_per_chunk = chunk_size / HNS_HW_PAGE_SIZE; + u32 first = obj / obj_num_per_bt; + u32 last = min(first + bt_num_per_chunk, hr_dev->caps.gmv_bt_num); struct hns_roce_cmq_desc desc; - struct hns_roce_cmq_req *req = (struct hns_roce_cmq_req *)desc.data; - u32 idx = obj / (HNS_HW_PAGE_SIZE / hr_dev->caps.gmv_entry_sz); - u64 addr = to_hr_hw_page_addr(base_addr); + struct hns_roce_cmq_req *req; + u64 addr; + int ret; + u32 i; - hns_roce_cmq_setup_basic_desc(&desc, HNS_ROCE_OPC_CFG_GMV_BT, false); + /* The GMV BT entry of hardware covers a fixed 4K region, so a buffer + * chunk larger than 4K must be registered to hardware with one BT + * entry per 4K block, otherwise the GMV entries beyond the first + * 4K of the chunk are unreachable. + */ + for (i = first; i < last; i++) { + hns_roce_cmq_setup_basic_desc(&desc, HNS_ROCE_OPC_CFG_GMV_BT, + false); + req = (struct hns_roce_cmq_req *)desc.data; - hr_reg_write(req, CFG_GMV_BT_BA_L, lower_32_bits(addr)); - hr_reg_write(req, CFG_GMV_BT_BA_H, upper_32_bits(addr)); - hr_reg_write(req, CFG_GMV_BT_IDX, idx); + addr = to_hr_hw_page_addr(base_addr + + (u64)(i - first) * HNS_HW_PAGE_SIZE); + hr_reg_write(req, CFG_GMV_BT_BA_L, lower_32_bits(addr)); + hr_reg_write(req, CFG_GMV_BT_BA_H, upper_32_bits(addr)); + hr_reg_write(req, CFG_GMV_BT_IDX, i); - return hns_roce_cmq_send(hr_dev, &desc, 1); + ret = hns_roce_cmq_send(hr_dev, &desc, 1); + if (ret) + return ret; + } + + return 0; } static int set_hem_to_hw(struct hns_roce_dev *hr_dev, int obj, -- 2.33.0 ^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-09-07 2:21 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-09-04 9:38 [PATCH for-next 0/2] RDMA/hns: Fix GMV entry management Junxian Huang 2026-09-04 9:38 ` [PATCH for-next 1/2] RDMA/hns: Limit gmv_entry_num to avoid memory waste Junxian Huang 2026-09-06 9:12 ` Leon Romanovsky 2026-09-07 2:21 ` Junxian Huang 2026-09-04 9:38 ` [PATCH for-next 2/2] RDMA/hns: Fix GID capacity loss in 64K system Junxian Huang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox