Re: Excessive memory usage when infiniband config is enabled

[Brian Baboch <brian.baboch@gmail.com>]

Hello Leon,

I feel that it's a bug because I don't understand why is this 
module/option allocating 6GB of RAM without any explicit configuration 
or usage from us.
It's also worth mentioning that we are using the default linux-image 
from Debian bookworm, and it took us a long time to understand the 
reason behind this memory increase by bisecting the kernel's config file.
Moreover the documentation of the module doesn't mention anything 
regarding additional memory usage, we're talking about an increase of 
6Gb which is huge since we're not using the option.
So is that an expected behavior, to have this much increase in the 
memory consumption, when activating the RDMA option even if we're not 
using it ? If that's the case, perhaps it would be good to mention this 
in the documentation.

Thank you


On 5/7/24 13:27, Leon Romanovsky wrote:
> On Mon, May 06, 2024 at 05:15:55PM +0200, Brian Baboch wrote:
>> Hello,
>>
>>
>> We discovered that the CONFIG_INFINIBAND_IRDMA configuration option in the
>> linux kernel is causing excessive memory usage on idle mode on specific
>> servers like the DELL VEP4600
>> (https://www.dell.com/en-us/shop/ipovw/virtual-edge-platform-4600.
>>
>> By default we were using Debian's linux-image-6.1.0-13-amd64 which is the
>> stable 6.1.55-1 amd64, we then compiled the kernel again with the same
>> config file from the stable 6.1.55 tag and had the same problem. We were
>> able to resolve the memory problem by removing the `CONFIG_INFINIBAND_IRDMA`
>> option from the kernel config.
>>
>> The tag used to reproduce the problem is v6.1.55.
>> adding the following config `CONFIG_INFINIBAND_IRDMA=m` causes the excessive
>> memory usage to go from 1.4Gb to 7Gb.
> 
> Hi Brian,
> 
> Why do you think that this is a bug?
> DELL VEP4600 supports RDMA, so by enabling CONFIG_INFINIBAND_IRDMA, you
> compiled RDMA support for Intel NIC.
> https://dl.dell.com/topicspdf/vep4600_tech_guide_en-us.pdf
> 
> You can unload irdma.ko module and restore memory footprint.
> 
> Thanks