答复: 答复: 答复: [PATCH] page_pool: mark unbound node page as reusable pages

["Li,Rongqing" <lirongqing@baidu.com>]

> -----邮件原件-----
> 发件人: Yunsheng Lin [mailto:linyunsheng@huawei.com]
> 发送时间: 2019年12月5日 10:06
> 收件人: Li,Rongqing <lirongqing@baidu.com>; netdev@vger.kernel.org;
> saeedm@mellanox.com
> 主题: Re: 答复: 答复: [PATCH] page_pool: mark unbound node page as
> reusable pages
> 
> On 2019/12/5 9:55, Li,Rongqing wrote:
> >
> >
> >> -----邮件原件-----
> >> 发件人: Yunsheng Lin [mailto:linyunsheng@huawei.com]
> >> 发送时间: 2019年12月5日 9:44
> >> 收件人: Li,Rongqing <lirongqing@baidu.com>; netdev@vger.kernel.org;
> >> saeedm@mellanox.com
> >> 主题: Re: 答复: [PATCH] page_pool: mark unbound node page as reusable
> >> pages
> >>
> >> On 2019/12/5 9:08, Li,Rongqing wrote:
> >>>
> >>>
> >>>> -----邮件原件-----
> >>>> 发件人: Yunsheng Lin [mailto:linyunsheng@huawei.com]
> >>>> 发送时间: 2019年12月5日 8:55
> >>>> 收件人: Li,Rongqing <lirongqing@baidu.com>; netdev@vger.kernel.org;
> >>>> saeedm@mellanox.com
> >>>> 主题: Re: [PATCH] page_pool: mark unbound node page as reusable
> pages
> >>>>
> >>>> On 2019/12/4 18:14, Li RongQing wrote:
> >>>>> some drivers uses page pool, but not require to allocate page from
> >>>>> bound node, so pool.p.nid is NUMA_NO_NODE, and this fixed patch
> >>>>> will block this kind of driver to recycle
> >>>>>
> >>>>> Fixes: d5394610b1ba ("page_pool: Don't recycle non-reusable
> >>>>> pages")
> >>>>> Signed-off-by: Li RongQing <lirongqing@baidu.com>
> >>>>> Cc: Saeed Mahameed <saeedm@mellanox.com>
> >>>>> ---
> >>>>>  net/core/page_pool.c | 4 +++-
> >>>>>  1 file changed, 3 insertions(+), 1 deletion(-)
> >>>>>
> >>>>> diff --git a/net/core/page_pool.c b/net/core/page_pool.c index
> >>>>> a6aefe989043..4054db683178 100644
> >>>>> --- a/net/core/page_pool.c
> >>>>> +++ b/net/core/page_pool.c
> >>>>> @@ -317,7 +317,9 @@ static bool __page_pool_recycle_direct(struct
> >>>>> page
> >>>> *page,
> >>>>>   */
> >>>>>  static bool pool_page_reusable(struct page_pool *pool, struct
> >>>>> page
> >>>>> *page)  {
> >>>>> -	return !page_is_pfmemalloc(page) && page_to_nid(page) ==
> >> pool->p.nid;
> >>>>> +	return !page_is_pfmemalloc(page) &&
> >>>>> +		(page_to_nid(page) == pool->p.nid ||
> >>>>> +		 pool->p.nid == NUMA_NO_NODE);
> >>>>
> >>>> If I understand it correctly, you are allowing recycling when
> >>>> pool->p.nid is NUMA_NO_NODE, which does not seems match the
> commit
> >>>> log: "this fixed patch will block this kind of driver to recycle".
> >>>>
> >>>> Maybe you mean "commit d5394610b1ba" by this fixed patch?
> >>>
> >>> yes
> >>>
> >>>>
> >>>> Also, maybe it is better to allow recycling if the below condition is
> matched:
> >>>>
> >>>> 	pool->p.nid == NUMA_NO_NODE && page_to_nid(page) ==
> >>>> numa_mem_id()
> >>>
> >>> If driver uses NUMA_NO_NODE, it does not care numa node, and maybe
> >>> its platform Only has a node, so not need to compare like
> >>> "page_to_nid(page) ==
> >> numa_mem_id()"
> >>
> >> Normally, driver does not care if the node of a device is
> >> NUMA_NO_NODE or not, it just uses the node that returns from
> dev_to_node().
> >>
> >> Even for multi node system, the node of a device may be NUMA_NO_NODE
> >> when BIOS/FW has not specified it through ACPI/DT, see [1].
> >>
> >>
> >> [1] https://lore.kernel.org/patchwork/patch/1141952/
> >>
> >
> > at this condition, page can be allocated from any node from driver
> > boot, why need to check "page_to_nid(page) == numa_mem_id()" at recycle?
> 
> For performance, the performance is better when the rx page is on the same
> node as the rx process is running.
> 
> We want the node of rx page is close to the node of device/cpu to achive better
> performance, since the node of device is unknown, maybe we choose the node
> of memory that is close to the cpu that is running to handle the rx cleaning.
> 

if the driver takes care about numa node, it should not assign NUMA_NO_NODE, it should
assign a detail numa node at starting step. Not depend on recycle to decide the numa
node

-RongQing


> >
> > -Li
> >
> >>>
> >>>
> >>> -RongQing
> >>>
> >>>
> >>>>
> >>>>>  }
> >>>>>
> >>>>>  void __page_pool_put_page(struct page_pool *pool, struct page
> >>>>> *page,
> >>>>>
> >>>
> >