Re: [patch 2/2] cpusets: add interleave_over_allowed option

David Rientjes
2007-10-30T23:13:49Z

On Mon, 29 Oct 2007, Paul Jackson wrote:

> Blind siding users with a unilateral change like this will leave
> orphaned bits gasping in agony on the computer room floor. It can
> sometimes takes months of elapsed time and hundreds of hours of various
> peoples time across a dozen departments in three to five corporations
> to track down the root cause of such a problem, from the point of the
> initial failure, back to the desk of someone like you or me. And then
> it can take tens or hundreds more hours of human effort to deliver a
> fix. I refuse to knowingly go down that road.
>

If your argument is that most applications are written to implement
mempolicies without necessarily thinking too much about its cpuset
placement or interactions with cpusets, then the requirement of remapping
nodes when a cpuset changes for effected mempolicies isn't actually that
important. In other words, my Choice C with AND'd behavior as opposed to
remapping behavior could be introduced as a replacement for Choice A.

Those applications that currently rely on the remapping are going to be
broken anyway because they are unknowingly receiving different nodes than
they intended, this is the objection to remapping that Lee agreed with.
The remap doesn't take into account any notion of locality or affinity to
physical controllers and seems to be merely a convenience of not
invalidating the entire mempolicy in light of an ever-changing cpuset
policy.

> Not "AND". Fold - the n-th bit is set in a tasks mems_allowed iff
> there exists m such that (m % w) == n, and such that the m-th bit is
> set in the tasks mempolicy's remembered nodemask, where w is the weight
> (number of '1' bits) in the tasks current cpusets mems_allowed. See
> lib/bitmap.c:bitmap_remap(), and its wrapper nodes_remap() for the
> implementation.
>

Yes, I know, and my Choice C does _not_ want that folding behavior; it
wants the AND'd behavior because it fully respects the intent of the
application with regard to the actual nodes that it specified in its
memory policies. A node should only have one definition and policies that
are effected on a set of nodes, or one node in the preferred case, should
not change from beneath the application because it was not the intent of
the implementation. Doing so is dangerous, regardless of whether or not
it is currently the mempolicy behavior in HEAD.

David