Re: [PATCH 1/1] RDMA/rxe: Fix blktests srp lead kernel panic with 64k page size

[Zhu Yanjun <yanjun.zhu@linux.dev>]

在 2023/10/26 19:42, Jason Gunthorpe 写道:
> On Thu, Oct 26, 2023 at 09:05:52AM +0000, Zhijian Li (Fujitsu) wrote:
>> The root cause is that
>>
>> rxe:rxe_set_page() gets wrong when mr.page_size != PAGE_SIZE where it only stores the *page to xarray.
>> So the offset will get lost.
>>
>> For example,
>> store process:
>> page_size = 0x1000;
>> PAGE_SIZE = 0x10000;
>> va0 = 0xffff000020651000;
>> page_offset = 0 = va & (page_size - 1);
>> page = va_to_page(va);
>> xa_store(&mr->page_list, mr->nbuf, page, GFP_KERNEL);
>>
>> load_process:
>> page = xa_load(&mr->page_list, index);
>> page_va = kmap_local_page(page) --> it must be a PAGE_SIZE align value, assume it as 0xffff000020650000
>> va1 = page_va + page_offset = 0xffff000020650000 + 0 = 0xffff000020650000;
>>
>> Obviously, *va0 != va1*, page_offset get lost.
>>
>>
>> How to fix:
>> - revert 325a7eb85199 ("RDMA/rxe: Cleanup page variables in rxe_mr.c")
>> - don't allow ulp registering mr.page_size != PAGE_SIZE ?
> 
> Lets do the second one please. Most devices only support PAGE_SIZE anyhow.

Normally page_size is PAGE_SIZE or the size of the whole compound page 
(in the latest kernel version, it is the size of folio). When compound 
page or folio is taken into account, the page_size is not equal to 
PAGE_SIZE.

If the ULP uses the compound page or folio, the similar problem will 
occur again. Since this problem is involved with RDMA and block, it will 
cost a lot of time and efforts to debug. If we fix it now, it will save 
a lot of time and efforts for the future ULP with folio.

Because folio has a lot of benefits, sooner or later, folio or other 
similar MMU technology will be popular in the kernel. At that time, this 
problem will appear again. We can do something for the future folio ^_^

Zhu Yanjun

> 
> Jason