First, rereading my e-mails I realize that there was some thinking behind my questions
which was only partially articulated. Going back to my earlier position which I will restate
here as:

Use of a mask should never corrupt the semantics defined by the Single Unix
Specification or its practical derivations and extensions, e.g., BSD and Linux.

That is, ideally if I created a standalone system that "looked" like the system the mask
provides then the process would behave equivalently. I emphasize ideally because to
actually make that the case would require something closer to a virtual machine---which
is clearly beyond the scope of this project.

So I am looking for potential semantic anomalies which could be introduced by a
mask over a reiser4 file system, i.e., border cases where some upfront awareness
and attention may save re-design and re-writing later on.

Hans Reiser wrote:

The issue is how the mask might change the semantics of sys_execve(). I have spent some time
starting to trace code around, but more work is required.

For the moment I think we are safe if the setuid and setgroup can not be altered by the mask.

Ok.

For the moment I will focus on how the uid (and assume the group id will be similar)
of a process and how the mask might lead to a different uid than would be expected
under standard semantics.

So we agree that the process has an associated user: actually there is the concept
of real and effective uid also.

I believe the statement above to be true if the process never makes a setuid() system's call of
one flavor or another because the uid is inherited from the process calling execev() or is determined
by the owner of the executable if the executable's file set-uid permission bit is turned on via chmod.

We only change the semantics if the real/effective uid or gid is different at some point in the
processes execution because of the mask---begging situations where a call to setuid happens
only if a certain file is not available.

Say more.

If the process has the code like

if (stat("/etc/foobar.conf", statbuf) < 0) {
     if (setuid(3) < 0) {   /* Try to become sys */
    }
    /* Do something here */
}

And the mask hides "/etc/foobar.conf" then in fact the uid
will change but not unexpectedly, IMO. The mask is
creating a changed environment but the behavior is
understandable from the source code.

Put this another way, the semantics still seem correct to me
because you are getting the behavior of "/etc/foobar.conf" actually
not being there.

Now that I think about it masking a named pipe or a lock file
is perhaps more likely to cause problems then the setuid bit.
More cogitation required... :-)