Re: [ofa-general] Re: Demand paging for memory regions

Patrick Geoffray
2008-02-13T04:47:54Z

Jason Gunthorpe wrote:
> [mangled CC list trimmed]
Thanks, noticed that afterwards.

> This wasn't ment as a slight against Quadrics, only to point out that
> the specific wire protcols used by IB and iwarp are what cause this
> limitation, it would be easy to imagine that Quadrics has some
> additional twist that can make this easier..

The wire protocols are similar, nothing fancy. The specificity of
Quadrics (and many others) is that they can change the behavior of the
NIC in firmware, so they adapt to what the OS offers. They had the VM
notifier support in Tru64 back in the days, they just ported the
functionality to Linux.

> I ment that HPC users are unlikely to want to swap active RDMA pages
> if this causes a performance cost on normal operations. None of my

Swapping to disk is not a normal operations in HPC, it's going to be
slow anyway. The main problem for HPC users is not swapping, it's that
they do not know when a registered page is released to the OS through
free(), sbrk() or munmap(). Like swapping, they don't expect that it
will happen often, but they have to handle it gracefully.

Patrick
--
Patrick Geoffray
Myricom, Inc.