Re: [PATCH v3] ummunotify: Userspace support for MMU notifications

[Brice Goglin <Brice.Goglin@inria.fr>]

Roland Dreier wrote:
> As discussed in <http://article.gmane.org/gmane.linux.drivers.openib/61925>
> and follow-up messages, libraries using RDMA would like to track
> precisely when application code changes memory mapping via free(),
> munmap(), etc.  Current pure-userspace solutions using malloc hooks
> and other tricks are not robust, and the feeling among experts is that
> the issue is unfixable without kernel help.
>
> We solve this not by implementing the full API proposed in the email
> linked above but rather with a simpler and more generic interface,
> which may be useful in other contexts.  Specifically, we implement a
> new character device driver, ummunotify, that creates a /dev/ummunotify
> node.  A userspace process can open this node read-only and use the fd
> as follows:
>
>  1. ioctl() to register/unregister an address range to watch in the
>     kernel (cf struct ummunotify_register_ioctl in <linux/ummunotify.h>).
>
>  2. read() to retrieve events generated when a mapping in a watched
>     address range is invalidated (cf struct ummunotify_event in
>     <linux/ummunotify.h>).  select()/poll()/epoll() and SIGIO are
>     handled for this IO.
>   

Hello Roland,

I like the interface but I have a couple questions:

1) Why does userspace have to register these address ranges? I would
have just reported all invalidation evens and let user-space check which
ones are interesting. My feeling is that the number of invalidation
events will usually be lower than the number registered ranges, so
you'll report more events through the file descriptor, but userspace
will do a lot less ioctls.

2) What happens in case of fork? If father+child keep reading from the
previously-open /dev/ummunotify, each event will be delivered only to
the first reader, right? Fork is always a mess in HPC, but maybe there's
something to do here.

3) What's userspace supposed to do if 2 libraries need such events in
the same process? Should each of them open /dev/ummunotify separately?
Doesn't matter much for performance, just wondering.

thanks,
Brice