O_NOLINK for open()

O_NOLINK for open()

[Brent Casavant]

Bear with me, I present the problem I'm trying to solve first,
and then propose O_NOLINK as a solution.  Responses to either my
shared memory problem or the O_NOLINK idea would be most appreciated.

I've run into a rather unusual set of circumstances calling for
use of shared memory, but haven't found a bulletproof solution
which can be used under Linux.

I have a need to share memory between two unrelated processes,
but without the ability for any other process to attach to it,
for security reasons (e.g. encryption keys and plaintext will
be exchanged via this shared memory area).  These unrelated
processes will in all likelihood not be owned by the same user
(i.e. one of them is a daemon).

System V shmem is right out because the IPC key is publicly
visible and there is no combination of permissions which
will allow sharing the segment with just one other process
(or at least just one other user).  To my knowledge Linux's
implementation doesn't provide ACLs for SysV shmem.  SGI's
proposed XPMEM suffers from the same problems for my purposes.

I had a mistaken notion that multiple mmap's of /dev/zero using
a common file descriptor (which could be passed between the
processes via an AF_UNIX socket) would result in shared memory,
but apparently my understanding of /dev/zero mmap's was subtlely
wrong, so it appears this won't work.

I could mmap a temporary tmpfs file (tmpfs so that if there is a
machine crash no sensitive data persists) which is created with
permissions of 0, immediately unlink it, and pass the file
descriptor through an AF_UNIX socket.  This does open up a very
small window of vulnerability if another process is able to chmod
the file and open it before the unlink.

However, it occurs to me that this problem goes away if there were
a method create a file in an unlinked state to begin with.  However
there does not appear to be any such mechanism in Linux's open()
interface.  A bit of Googling indicates that Hurd has an O_NOLINK
flag which seems to accomplish what I'd need, but Linux doesn't
implement such a flag.  There was some discussion of this in various
lkml threads in the past, but none that went anywhere.  Perhaps
the best an explaining why other mechanisms (i.e. directories
with particular permissions aren't a solution) is:

	http://marc.info/?l=linux-kernel&m=93032806224160&w=2

Of course it is reasonable to take the stance that if root or the
daemon's user are malicious, all bets are off anyway.

Fully understanding this, it was suggested to me that I could fire
this problem off to lkml to see if anyone has a solution I haven't
already dismissed, or as a trial balloon for considering adding
O_NOLINK to open().

Thoughts?

Brent

-- 
Brent Casavant                          All music is folk music.  I ain't
bcasavan@sgi.com                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong

Re: O_NOLINK for open()

[H. Peter Anvin]

Brent Casavant wrote:
> 
> I could mmap a temporary tmpfs file (tmpfs so that if there is a
> machine crash no sensitive data persists) which is created with
> permissions of 0, immediately unlink it, and pass the file
> descriptor through an AF_UNIX socket.  This does open up a very
> small window of vulnerability if another process is able to chmod
> the file and open it before the unlink.
> 

To avoid this window, typically one creates a temporary directory first,
with 0700 permissions.  Make sure you verify that you actually created
the directory, and watch out for symlink attacks.  Then you create the
file in that directory.

This doesn't prevent another process owned by the same user (or root)
from attaching, but such a process can ptrace you or touch yoour /proc
spaec just as well, so you're screwed anyway (modulo SELinux-type policies.)

> However, it occurs to me that this problem goes away if there were
> a method create a file in an unlinked state to begin with.  However
> there does not appear to be any such mechanism in Linux's open()
> interface.  A bit of Googling indicates that Hurd has an O_NOLINK
> flag which seems to accomplish what I'd need, but Linux doesn't
> implement such a flag.  There was some discussion of this in various
> lkml threads in the past, but none that went anywhere.  Perhaps
> the best an explaining why other mechanisms (i.e. directories
> with particular permissions aren't a solution) is:
> 
> 	http://marc.info/?l=linux-kernel&m=93032806224160&w=2
> 

This link talks about file flags handling.  I don't see the relevance to
this problem at all.  However, this is a very long thread, so if there
is anything specific that you want to point to, then please elucidate.

> Of course it is reasonable to take the stance that if root or the
> daemon's user are malicious, all bets are off anyway.

Yup, see above.

	-hpa

Re: O_NOLINK for open()

[Brent Casavant]

On Wed, 12 Sep 2007, H. Peter Anvin wrote:

> Brent Casavant wrote:

> > 	http://marc.info/?l=linux-kernel&m=93032806224160&w=2
> 
> This link talks about file flags handling.  I don't see the relevance to
> this problem at all.  However, this is a very long thread, so if there
> is anything specific that you want to point to, then please elucidate.

Oops, my mistake -- I pasted the wrong URL.  I meant this thread, this
post in particular:

	http://marc.info/?l=linux-kernel&m=88937224115435&w=2

Still, O_NOLINK would seem to be a valuable addition, and greatly
simplify secure temporary file creation.

Brent

-- 
Brent Casavant                          All music is folk music.  I ain't
bcasavan@sgi.com                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong

Re: O_NOLINK for open()

[H. Peter Anvin]

Brent Casavant wrote:
> On Wed, 12 Sep 2007, H. Peter Anvin wrote:
> 
>> Brent Casavant wrote:
> 
>>> 	http://marc.info/?l=linux-kernel&m=93032806224160&w=2
>> This link talks about file flags handling.  I don't see the relevance to
>> this problem at all.  However, this is a very long thread, so if there
>> is anything specific that you want to point to, then please elucidate.
> 
> Oops, my mistake -- I pasted the wrong URL.  I meant this thread, this
> post in particular:
> 
> 	http://marc.info/?l=linux-kernel&m=88937224115435&w=2
> 
> Still, O_NOLINK would seem to be a valuable addition, and greatly
> simplify secure temporary file creation.
> 

Avoiding -- or at least detecting -- symlink racing with mkdir() is
relatively simple: run mkdir(), make sure you don't get EEXIST or
something like that, lstat() the resulting path -- it should be a
directory with all the right modes and ownerships.

I believe -- but I'm not certain -- that mkdtemp() in glibc will do all
this for you.  If not, I would consider that a glibc bug.

	-hpa

Re: O_NOLINK for open()

[Andreas Schwab]

Brent Casavant <bcasavan@sgi.com> writes:

> I could mmap a temporary tmpfs file (tmpfs so that if there is a
> machine crash no sensitive data persists) which is created with
> permissions of 0, immediately unlink it, and pass the file
> descriptor through an AF_UNIX socket.  This does open up a very
> small window of vulnerability if another process is able to chmod
> the file and open it before the unlink.

Only the owner can chmod a file, so why is that a vulnerability?

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: O_NOLINK for open()

[Brent Casavant]

On Wed, 12 Sep 2007, Andreas Schwab wrote:

> Brent Casavant <bcasavan@sgi.com> writes:
> 
> > I could mmap a temporary tmpfs file (tmpfs so that if there is a
> > machine crash no sensitive data persists) which is created with
> > permissions of 0, immediately unlink it, and pass the file
> > descriptor through an AF_UNIX socket.  This does open up a very
> > small window of vulnerability if another process is able to chmod
> > the file and open it before the unlink.
> 
> Only the owner can chmod a file, so why is that a vulnerability?

In this particular case because the user may not normally have direct
access to some of the data to be contained in that file.

Decryption keys in a key management system, in particular.  If the
keys are passed over secure network links such that they only ever
exist in system RAM, and are not reachable via the filesystem, these
keys can be protected from disclosure to the user (short of /proc/#/mem
type of tricks).  However, if there is even a brief window when the
user can gain access to the file, these keys are at risk of disclosure.

The problem can be addressed, in this case, by having the daemon half
of the design create these files, however it would provide a bit more
flexibility if the client side was also capable of creating them.  It's
not a make-or-break problem, by any means, but does somewhat motivate
an O_NOLINK flag for open().

Brent

P.S. By the way, there doesn't seem to be a way to remove /proc/#/mem
     files.  That might be an additional nicety -- programs worried about
     being snooped could unlink their own entry.  /dev/mem and /dev/kmem
     can simply be removed by the sysadmin of such a system.  If all of
     that were done you'd have to resort to attacking crash dumps, core
     dumps, or via something like kdb to extract "hidden" data.

-- 
Brent Casavant                          All music is folk music.  I ain't
bcasavan@sgi.com                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong

Re: O_NOLINK for open()

[Al Viro]

On Wed, Sep 12, 2007 at 05:44:30PM -0500, Brent Casavant wrote:

> P.S. By the way, there doesn't seem to be a way to remove /proc/#/mem
>      files.  That might be an additional nicety -- programs worried about
>      being snooped could unlink their own entry.  /dev/mem and /dev/kmem
>      can simply be removed by the sysadmin of such a system.  If all of
>      that were done you'd have to resort to attacking crash dumps, core
>      dumps, or via something like kdb to extract "hidden" data.

Give me a break.  And learn about ptrace(2).  This "unlinking" bullshit
buys you zero additional security, both for /proc/*/mem and for /dev/mem
(see mknod(2)).

Re: O_NOLINK for open()

[Brent Casavant]

On Wed, 12 Sep 2007, Al Viro wrote:

> On Wed, Sep 12, 2007 at 05:44:30PM -0500, Brent Casavant wrote:
> 
> > P.S. By the way, there doesn't seem to be a way to remove /proc/#/mem
> >      files.  That might be an additional nicety -- programs worried about
> >      being snooped could unlink their own entry.  /dev/mem and /dev/kmem
> >      can simply be removed by the sysadmin of such a system.  If all of
> >      that were done you'd have to resort to attacking crash dumps, core
> >      dumps, or via something like kdb to extract "hidden" data.
> 
> Give me a break.  And learn about ptrace(2).  This "unlinking" bullshit
> buys you zero additional security, both for /proc/*/mem and for /dev/mem
> (see mknod(2)).

Yes, I fully understand that mknod can recreate the nodes -- however
only the superuser can do so, and if the superuser is attacking a
process all bets are off anyway.  OK, so /dev/*mem isn't to worry
about, since it's already owned by root.  Still, /proc/#/mem is owned
by the user, not root, leaving it potentially open to inspection by
third party processes.

I'm thinking out loud.  Sorry to cause any grief.

My (limited) understanding of ptrace is that a parent-child
relationship is needed between the tracing process and the traced
process (at least that's what I gather from the man page).  This
does give cause for concern, and I might have to see what can be
done to alleviate this concern.  I fully realize that making this
design completely unassilable is a fools errand, but closing off
as many attack vectors as possible seems prudent.

-- 
Brent Casavant                          All music is folk music.  I ain't
bcasavan@sgi.com                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong

Re: O_NOLINK for open()

[Brent Casavant]

On Wed, 12 Sep 2007, Brent Casavant wrote:

> On Wed, 12 Sep 2007, Al Viro wrote:
> 
> > Give me a break.  And learn about ptrace(2).  This "unlinking" bullshit
> > buys you zero additional security, both for /proc/*/mem and for /dev/mem
> > (see mknod(2)).
> 
> My (limited) understanding of ptrace is that a parent-child
> relationship is needed between the tracing process and the traced
> process (at least that's what I gather from the man page).  This
> does give cause for concern, and I might have to see what can be
> done to alleviate this concern.  I fully realize that making this
> design completely unassilable is a fools errand, but closing off
> as many attack vectors as possible seems prudent.

Hmm.  The solution would appear to be as simple as making the
target program set-user-id.  As long as as the attacker isn't
the superuser (or has CAP_SYS_PTRACE) we should be OK.

Thanks for the heads-up,
Brent

-- 
Brent Casavant                          All music is folk music.  I ain't
bcasavan@sgi.com                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong

Re: O_NOLINK for open()

[Goswin von Brederlow]

Brent Casavant <bcasavan@sgi.com> writes:

> My (limited) understanding of ptrace is that a parent-child
> relationship is needed between the tracing process and the traced
> process (at least that's what I gather from the man page).  This
> does give cause for concern, and I might have to see what can be
> done to alleviate this concern.  I fully realize that making this
> design completely unassilable is a fools errand, but closing off
> as many attack vectors as possible seems prudent.

No relationship needed:

strace -p <pid>

MfG
        Goswin

Re: O_NOLINK for open()

[Gabor Gombas]

On Wed, Sep 12, 2007 at 03:37:44PM -0500, Brent Casavant wrote:

> System V shmem is right out because the IPC key is publicly
> visible and there is no combination of permissions which
> will allow sharing the segment with just one other process
> (or at least just one other user).  To my knowledge Linux's
> implementation doesn't provide ACLs for SysV shmem.  SGI's
> proposed XPMEM suffers from the same problems for my purposes.

SYSV shared memory has the concept of separate creator and owner ID's,
so you can share the shmem segment between exactly two users. Just use
IPC_SET and set shm_perm.uid to the user ID of the peer process.

> I could mmap a temporary tmpfs file (tmpfs so that if there is a
> machine crash no sensitive data persists) which is created with
> permissions of 0, immediately unlink it, and pass the file
> descriptor through an AF_UNIX socket.  This does open up a very
> small window of vulnerability if another process is able to chmod
> the file and open it before the unlink.

I think your worries about permissions has been cleared by the other
posts, but there is still a problem: the client may call ftruncate() on
the file descriptor, and then your daemon will get a nice SIGBUS when it
tries to access the shared memory. Handling that gracefully may not be
trivial esp. if your daemon is multi-threaded. SYSV shmem is _much_
nicer when you want shared memory between unrelated/untrusted processes.

Gabor

-- 
     ---------------------------------------------------------
     MTA SZTAKI Computer and Automation Research Institute
                Hungarian Academy of Sciences
     ---------------------------------------------------------

Re: O_NOLINK for open()

[Brent Casavant]

On Thu, 13 Sep 2007, Gabor Gombas wrote:

> On Wed, Sep 12, 2007 at 03:37:44PM -0500, Brent Casavant wrote:
> 
> > System V shmem is right out because the IPC key is publicly
> > visible and there is no combination of permissions which
> > will allow sharing the segment with just one other process
> > (or at least just one other user).  To my knowledge Linux's
> > implementation doesn't provide ACLs for SysV shmem.  SGI's
> > proposed XPMEM suffers from the same problems for my purposes.
> 
> SYSV shared memory has the concept of separate creator and owner ID's,
> so you can share the shmem segment between exactly two users. Just use
> IPC_SET and set shm_perm.uid to the user ID of the peer process.

Hmm.  This will work as long as the peer process is running setuid
to it's own unique user.  Excellent idea!  Since I need to make the
program setuid to avoid non-priveleged ptrace attacks, this is a
terrific solution.

> I think your worries about permissions has been cleared by the other
> posts, but there is still a problem: the client may call ftruncate() on
> the file descriptor, and then your daemon will get a nice SIGBUS when it
> tries to access the shared memory. Handling that gracefully may not be
> trivial esp. if your daemon is multi-threaded. SYSV shmem is _much_
> nicer when you want shared memory between unrelated/untrusted processes.

I'm actually not so concerned about the client -- that code will be
trusted as well.  The problem I'm trying to solve is preventing any
non-priveleged code except the server and client from gaining access
to their shared memory area.  With the feedback I've received from
this thread I think a solid design is emerging, some of which will
need to be solved by system configuration by the sysadmin.

Thanks,
Brent

-- 
Brent Casavant                          All music is folk music.  I ain't
bcasavan@sgi.com                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong

Re: O_NOLINK for open()

[Bodo Eggert]

Brent Casavant <bcasavan@sgi.com> wrote:

[...]
> I could mmap a temporary tmpfs file (tmpfs so that if there is a
> machine crash no sensitive data persists) which is created with
> permissions of 0, immediately unlink it, and pass the file
> descriptor through an AF_UNIX socket.  This does open up a very
> small window of vulnerability if another process is able to chmod
> the file and open it before the unlink.

If the process can chmod the file, it can ptrace the daemon, too.
Or, using CAP_DAC_OVERRIDE, it can patch the daemon.

Both will void any security.

> However, it occurs to me that this problem goes away if there were
> a method create a file in an unlinked state to begin with.  However
> there does not appear to be any such mechanism in Linux's open()
> interface.

Having no window for creating stale temp files is nice to have. We only
need a clever fool to implement it.-) But since it's hard to get killed
just in the right moment for having a stale temp file, there is very low
interest for this feature.
-- 
You know you're in trouble when packet floods are competing to flood you.
        -- grc.com

Friß, Spammer: dnLqD2P@t.7eggert.dyndns.org npkrx@imrx.fp6.7eggert.dyndns.org

Re: O_NOLINK for open()

[Jan Kara]

> > However, it occurs to me that this problem goes away if there were
> > a method create a file in an unlinked state to begin with.  However
> > there does not appear to be any such mechanism in Linux's open()
> > interface.
> 
> Having no window for creating stale temp files is nice to have. We only
> need a clever fool to implement it.-) But since it's hard to get killed
> just in the right moment for having a stale temp file, there is very low
> interest for this feature.
  I don't think this is a problem. The file is simply created with link
count 0. As soon as the process closes the file, it gets deleted. So
there would be no stale files... Or did you mean anything else?

								Honza
-- 
Jan Kara <jack@suse.cz>
SuSE CR Labs

Re: O_NOLINK for open()

[Bodo Eggert]

On Thu, 13 Sep 2007, Jan Kara wrote:

> > > However, it occurs to me that this problem goes away if there were
> > > a method create a file in an unlinked state to begin with.  However
> > > there does not appear to be any such mechanism in Linux's open()
> > > interface.
> > 
> > Having no window for creating stale temp files is nice to have. We only
> > need a clever fool to implement it.-) But since it's hard to get killed
> > just in the right moment for having a stale temp file, there is very low
> > interest for this feature.

>   I don't think this is a problem. The file is simply created with link
> count 0. As soon as the process closes the file, it gets deleted. So
> there would be no stale files... Or did you mean anything else?

This feature does, AFAIK, not yet exist. Therefore we'd need a code monkey.
-- 
Top 100 things you don't want the sysadmin to say:
42. Hey Fred, did you save that posting about restoring filesystems
    with vi and a toothpick?  More importantly, did you print it out?

Re: O_NOLINK for open()

[Bodo Eggert]

Brent Casavant <bcasavan@sgi.com> wrote:

[...]
> Hmm.  This will work as long as the peer process is running setuid
> to it's own unique user.  Excellent idea!  Since I need to make the
> program setuid to avoid non-priveleged ptrace attacks, this is a
> terrific solution.

Tried that:

~ > cd tmp
~/tmp > cp /bin/sleep .
~/tmp > chmod u+s sleep
~/tmp > ./sleep 2147483647 &
[1] 2823
~/tmp > strace -p 2823
Process 2823 attached - interrupt to quit
setup(

-- 
Top 100 things you don't want the sysadmin to say:
27. You can do this patch with the system up...

Friß, Spammer: 7TdJfPsslN@y4jaOy.7eggert.dyndns.org

Re: O_NOLINK for open()

[Andreas Schwab]

Bodo Eggert <7eggert@gmx.de> writes:

> ~/tmp > cp /bin/sleep .
> ~/tmp > chmod u+s sleep
> ~/tmp > ./sleep 2147483647 &
> [1] 2823
> ~/tmp > strace -p 2823
> Process 2823 attached - interrupt to quit
> setup(

You didn't change the owner, so this is not a setuid execution.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: O_NOLINK for open()

[Bodo Eggert]

On Fri, 14 Sep 2007, Andreas Schwab wrote:
> Bodo Eggert <7eggert@gmx.de> writes:

> > ~/tmp > cp /bin/sleep .
> > ~/tmp > chmod u+s sleep
> > ~/tmp > ./sleep 2147483647 &
> > [1] 2823
> > ~/tmp > strace -p 2823
> > Process 2823 attached - interrupt to quit
> > setup(
> 
> You didn't change the owner, so this is not a setuid execution.

I expected that, but I wanted to be sure before telling bull****.
Besides that, if the suid program was owned by the suid-to user,
that user could modify the binary in order to prepare a future attack.
-- 
Top 100 things you don't want the sysadmin to say:
16. find /usr2 -name nethack -exec rm -f {};