Re: [Intel-wired-lan] Increased memory usage on NUMA nodes with ICE driver after upgrade to 6.13.y (regression in commit 492a044508ad)

[Jacob Keller <jacob.e.keller@intel.com>]

[-- Attachment #1.1: Type: text/plain, Size: 3099 bytes --]



On 6/30/2025 11:48 PM, Jaroslav Pulchart wrote:
>> On 6/30/2025 2:56 PM, Jacob Keller wrote:
>>> Unfortunately it looks like the fix I mentioned has landed in 6.14, so
>>> its not a fix for your issue (since you mentioned 6.14 has failed
>>> testing in your system)
>>>
>>> $ git describe --first-parent --contains --match=v* --exclude=*rc*
>>> 743bbd93cf29f653fae0e1416a31f03231689911
>>> v6.14~251^2~15^2~2
>>>
>>> I don't see any other relevant changes since v6.14. I can try to see if
>>> I see similar issues with CONFIG_MEM_ALLOC_PROFILING on some test
>>> systems here.
>>
>> On my system I see this at boot after loading the ice module from
>>
>> $ grep -F "/ice/" /proc/allocinfo | sort -g | tail | numfmt --to=iec>
>>       26K      230 drivers/net/ethernet/intel/ice/ice_irq.c:84 [ice]
>> func:ice_get_irq_res
>>>          48K        2 drivers/net/ethernet/intel/ice/ice_arfs.c:565 [ice] func:ice_init_arfs
>>>          57K      226 drivers/net/ethernet/intel/ice/ice_lib.c:397 [ice] func:ice_vsi_alloc_ring_stats
>>>          57K      226 drivers/net/ethernet/intel/ice/ice_lib.c:416 [ice] func:ice_vsi_alloc_ring_stats
>>>          85K      226 drivers/net/ethernet/intel/ice/ice_lib.c:1398 [ice] func:ice_vsi_alloc_rings
>>>         339K      226 drivers/net/ethernet/intel/ice/ice_lib.c:1422 [ice] func:ice_vsi_alloc_rings
>>>         678K      226 drivers/net/ethernet/intel/ice/ice_base.c:109 [ice] func:ice_vsi_alloc_q_vector
>>>         1.1M      257 drivers/net/ethernet/intel/ice/ice_fwlog.c:40 [ice] func:ice_fwlog_alloc_ring_buffs
>>>         7.2M      114 drivers/net/ethernet/intel/ice/ice_txrx.c:493 [ice] func:ice_setup_rx_ring
>>>         896M   229264 drivers/net/ethernet/intel/ice/ice_txrx.c:680 [ice] func:ice_alloc_mapped_page
>>
>> Its about 1GB for the mapped pages. I don't see any increase moment to
>> moment. I've started an iperf session to simulate some traffic, and I'll
>> leave this running to see if anything changes overnight.
>>
>> Is there anything else that you can share about the traffic setup or
>> otherwise that I could look into?  Your system seems to use ~2.5 x the
>> buffer size as mine, but that might just be a smaller number of CPUs.
>>
>> Hopefully I'll get some more results overnight.
> 
> The traffic is random production workloads from VMs, using standard
> Linux or OVS bridges. There is no specific pattern to it. I haven’t
> had any luck reproducing (or was not patient enough) this with iperf3
> myself. The two active (UP) interfaces are in an LACP bonding setup.
> Here are our ethtool settings for the two member ports (em1 and p3p1)
> 

I had iperf3 running overnight and the memory usage for
ice_alloc_mapped_pages is constant here. Mine was direct connections
without bridge or bonding. From your description I assume there's no XDP
happening either.

I guess the traffic patterns of an iperf session are too regular, or
something to do with bridge or bonding.. but I also struggle to see how
those could play a role in the buffer management in the ice driver...

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]