Re: memory region limit at 32 GB?

[Yishai Hadas <yishaih-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>]

Hi Albert,
I have sent yesterday a patch that may enable working with 1TB of registered memory.
Its subject is: [PATCH] net/mlx4_core: Fix num mtt issues.

Can you test it and let us know whether your issue was solved ?
In the meanwhile I have found the root cause of the limit of log_num_mtt of 28.
Plan to send in coming days an extra patch that enables value of 31 which match to 8TB.
Yishai


On 8/2/2012 5:31 PM, Albert Strasheim wrote:
> Hello
>
> On Fri, Jan 20, 2012 at 8:00 PM, Roland Dreier<roland-BHEL68pLQRGGvPXPguhicg@public.gmane.org>  wrote:
>> On Fri, Jan 20, 2012 at 9:39 AM, Albert Strasheim<fullung-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>  wrote:
>>> Is there some kind of limit that would prevent me from registering
>>> more than 32 GiB worth of memory regions with ibv_reg_mr in
>>> libibverbs?
>> Yes, by default mlx4 allocates a limited amount of adapter resources
>> for tracking memory regions.  I forget the exact limits but 32GB looks
>> reasonable...
> I think we've run into this in a "real" application (i.e., not a test).
>
> Is this limit per-process or per-HCA or per-machine? It seems it might
> not be per-process.
>
> Any chance to get this limit increased upstream?
>
> Regards
>
> Albert
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html