Makefile gcc -o /dev/null: the dissapearing of /dev/null

Makefile gcc -o /dev/null: the dissapearing of /dev/null

[proton]

I noticed this a long time ago in the Linux kernel
makefiles, I thought someone would have figured it
out by now tho.

The `gcc -o /dev/null' is a really neat way of
testing if gcc works or not.

Unfortunatly it doesnt work that well if you're root.

You see, gcc uses unlink(). /dev/null doesnt take
kindly to that kind of treatment...

Ofcourse, you cant unlink /dev/null unless you are root.

In any case, the `gcc -o /dev/null' test cases probably
need to go away.

I've seen this in linux/arch/i386/Makefile for 2.4.10,
it probably exists in all previous versions as well as
in other Makefiles.

Happy kernel hacking!

/proton
ps. any replies [read: flames] would do well being CC'd
to me since I dont subscribe to linux-kernel :)
[ http://www.energymech.net/users/proton/ ]

Re: Makefile gcc -o /dev/null: the dissapearing of /dev/null

[Jan-Benedict Glaw]

On Sat, 2001-09-29 09:55:35 +0200, proton <proton@energymech.net>
wrote in message <3BB57E77.4CDFF5D0@energymech.net>:

> Ofcourse, you cant unlink /dev/null unless you are root.

That's right and fine so far.

> In any case, the `gcc -o /dev/null' test cases probably
> need to go away.

No. Why? Well, the Linux kernel compiles just fine while
being an ordianary user. You don't have to be root to 
compile it. As it's just bad to do usual *work* as root,
you're the bug.

	*	Don't		*
	*	work		*
	*	with		*
	*	UID root	*
	*	if		*
	*	you		*
	*	don't		*
	*	need		*
	*	to		*
	*	! ! !		*

MfG, JBG

-- 
Jan-Benedict Glaw . jbglaw@lug-owl.de . +49-172-7608481

Re: Makefile gcc -o /dev/null: the dissapearing of /dev/null

[Jim Treadway]

On Sat, 29 Sep 2001, Jan-Benedict Glaw wrote:

> On Sat, 2001-09-29 09:55:35 +0200, proton <proton@energymech.net>
> wrote in message <3BB57E77.4CDFF5D0@energymech.net>:
>
> > Ofcourse, you cant unlink /dev/null unless you are root.
>
> That's right and fine so far.
>
> > In any case, the `gcc -o /dev/null' test cases probably
> > need to go away.
>
> No. Why? Well, the Linux kernel compiles just fine while
> being an ordianary user. You don't have to be root to
> compile it. As it's just bad to do usual *work* as root,
> you're the bug.

So then you can no longer 'make modules && make modules_install', or you
have to cp or chown /usr/src/linux on a fresh install to compile your
kernel?   Doesn't sound pleasant to me.

I think the "trick" is to redirect stdout and stderr to /dev/null as well,
so that /dev/null doesn't get removed from the file system since it is
held open by the shell.

Something like:

	gcc -o /dev/null -xc /dev/null /dev/null 2>&1

Perhaps someone just forgot the I/O redirection in one of the tests?

However, I just compiled (but did not install) 2.4.10, as root, and my
/dev/null still exists...

Re: Makefile gcc -o /dev/null: the dissapearing of /dev/null

[Jan-Benedict Glaw]

On Sat, 2001-09-29 06:10:52 -0700, Jim Treadway <jim@stardot-tech.com>
wrote in message <Pine.LNX.4.33.0109290535390.25966-100000@cerberus.stardot-tech.com>:
> On Sat, 29 Sep 2001, Jan-Benedict Glaw wrote:
> 
> > On Sat, 2001-09-29 09:55:35 +0200, proton <proton@energymech.net>
> > wrote in message <3BB57E77.4CDFF5D0@energymech.net>:
> >
> > > Ofcourse, you cant unlink /dev/null unless you are root.
> >
> > That's right and fine so far.
> >
> > > In any case, the `gcc -o /dev/null' test cases probably
> > > need to go away.
> >
> > No. Why? Well, the Linux kernel compiles just fine while
> > being an ordianary user. You don't have to be root to
> > compile it. As it's just bad to do usual *work* as root,
> > you're the bug.
> 
> So then you can no longer 'make modules && make modules_install', or you
> have to cp or chown /usr/src/linux on a fresh install to compile your
> kernel?   Doesn't sound pleasant to me.

You may do it this way:

make dep clean bzImage modules && su -c "make modules_install"

This is so far the minimal version for building as non-root user.

> I think the "trick" is to redirect stdout and stderr to /dev/null as well,
> so that /dev/null doesn't get removed from the file system since it is
> held open by the shell.
> 
> Something like:
> 
> 	gcc -o /dev/null -xc /dev/null /dev/null 2>&1

No-go. It's perfectly okay to remove an opened file. Test it yourself.
You may even replace a running (!) executable...

> Perhaps someone just forgot the I/O redirection in one of the tests?

No. It would be of no effect:-)

> However, I just compiled (but did not install) 2.4.10, as root, and my
> /dev/null still exists...

Fine:-) That's the way to go!

MfG, JBG

-- 
Jan-Benedict Glaw . jbglaw@lug-owl.de . +49-172-7608481

Re: Makefile gcc -o /dev/null: the dissapearing of /dev/null

[Jim Treadway]

On Sat, 29 Sep 2001, Jan-Benedict Glaw wrote:

> On Sat, 2001-09-29 06:10:52 -0700, Jim Treadway <jim@stardot-tech.com>
> wrote in message <Pine.LNX.4.33.0109290535390.25966-100000@cerberus.stardot-tech.com>:
> >
> > So then you can no longer 'make modules && make modules_install', or you
> > have to cp or chown /usr/src/linux on a fresh install to compile your
> > kernel?   Doesn't sound pleasant to me.
>
> You may do it this way:
>
> make dep clean bzImage modules && su -c "make modules_install"
>
> This is so far the minimal version for building as non-root user.

Except...

a) It doesn't work (yeah, I'm sure there are many, many, many other
work-arounds, but I can't believe I'm the only one compiling as root):

$ cd /usr/src/linux
$ make dep clean bzImage modules && su -c "make modules_install"
make[1]: Entering directory `/usr/src/linux/arch/i386/boot'
make[1]: Nothing to be done for `dep'.
make[1]: Leaving directory `/usr/src/linux/arch/i386/boot'
scripts/mkdep init/*.c > .depend
/bin/sh: .depend: Permission denied
make: *** [dep-files] Error 1

b) You are still evaluting most, if not all, of the lines in the Makefiles
that do "gcc -o /dev/null", as root.

> > I think the "trick" is to redirect stdout and stderr to /dev/null as well,
> > so that /dev/null doesn't get removed from the file system since it is
> > held open by the shell.
> >
> > Something like:
> >
> > 	gcc -o /dev/null -xc /dev/null /dev/null 2>&1
>
> No-go. It's perfectly okay to remove an opened file. Test it yourself.
> You may even replace a running (!) executable...

Which is exactly why I suggested keeping it open via I/O redirection
(except see below).

> > Perhaps someone just forgot the I/O redirection in one of the tests?
>
> No. It would be of no effect:-)

You are right here though...the I/O redirection doesn't have anything to
do with it, assuming you pass the -S flag to gcc, which I neglected, and
insert the missing '>' in my example above (oops).

But... this seems to be moot since in my case however /dev/null stays
around even when compiling as root. ;)

Re: Makefile gcc -o /dev/null: the dissapearing of /dev/null

[Rob Landley]

On Saturday 29 September 2001 14:21, Jan-Benedict Glaw wrote:

> No-go. It's perfectly okay to remove an opened file. Test it yourself.
> You may even replace a running (!) executable...

Which isn't actually too frightening since the filehandle of an open file 
counts as a link to the inode, so the inode won't actually be removed until 
that file handle is closed.  The associated disk space won't be freed until 
then either, etc.  And if you create a new file with the same name, it 
doesn't change which inode your executable is currently bound to.  It's like 
renaming your existing file to something else and renaming some other file to 
have that original name.  Who cares?  That's not the inode our file handle 
points to.  Deleting the last directory link just prevents new filehandles 
from being opened against it (except maybe by dup(2), or forking a child that 
inherits the parent's file handles...)

Slightly oversimplifying, running an executable opens the file and mmaps the 
sucker in to your process space and jumps to the code, page faulting stuff in 
as needed.  If the directory entry that was used to open it is then renamed 
or removed, the process doesn't care.  It already has the file open, it's 
happy.

This is also part of the magic (in linux/fs/exec.c) that makes shared 
libraries work.  mmaping in the same file twice can share physical pages 
without having to do anything special.  One cheap and easy (and ugly) way of 
doing writable shared memory is to create a writable temp file, mmap it in 
twice (once in each process), and then unlink it from the directory while 
keeping it open and mapped in in both processes.  It's ugly.  It's easy.  
It's portable.  It's really ugly.  It creates unnecessary disk access on a 
lot of systems, but then so does swapping...

Now opening an executable and WRITING to the file while it was running would 
be fairly unpleasant.  I believe changing its contents would change stuff in 
memory, because of the mmap.  (Correct me if I'm wrong.)  But I seriously 
doubt that's allowed.

The situation is slightly confused since so much stuff does an 
open/truncate/write instead of an unlink/create/write.  But if it didn't, 
stuff like "cp bootnet.img /dev/fd0" wouldn't work as expected, and "echo 
blah > /dev/null" would delete /dev/null and replace it with a short text 
file.  Not good.  Since truncate on block devices is ignored, the above two 
examples work as expected.  (It also automatically maintains ownership and 
permissions and such when editing other people's files that are world 
writable or being twiddled by root.  And there's no race condition between 
deleting the old file and creating the new one...)

Rob

Re: Makefile gcc -o /dev/null: the dissapearing of /dev/null

[J.H.M. Dassen (Ray)]

Jim Treadway <jim@stardot-tech.com> wrote:
> On Sat, 29 Sep 2001, Jan-Benedict Glaw wrote:
>> No. Why? Well, the Linux kernel compiles just fine while being an
>> ordianary user.

> So then you can no longer 'make modules && make modules_install', or you
> have to cp or chown /usr/src/linux on a fresh install to compile your
> kernel?   Doesn't sound pleasant to me.

I use that all the time, through the scripts in Debian's "kernel-package":
	cd /usr/local/src/linux
	cp /boot/config-some-recent-version .config
	make oldconfig
	fakeroot make-kpkg kernel_image modules_image
	dpkg -i ../kernel-image...deb ../alsa-modules-...deb

I find this quite comfortable: I always know where to find old configs, have
a matching System.map installed, can clean up old kernels and modules easily
etc.

Ray
-- 
What is this talk of software 'releases'? Klingons do not 'release'
software; our software ESCAPES, leaving a bloody trail of designers and
quality assurance people in its wake!

Re: Makefile gcc -o /dev/null: the dissapearing of /dev/null

[Olaf Titz]

> So then you can no longer 'make modules && make modules_install', or you
> have to cp or chown /usr/src/linux on a fresh install to compile your
> kernel?   Doesn't sound pleasant to me.

I just do "MakeDist -a" as user and untar the result as root. Script appended.

Olaf

#!/bin/sh
# MakeDist - build Linux kernel installation tarballs
# written by Olaf Titz, 1999-2000. Public domain.
#
# Call this as MakeDist [-a] [kernel-source-dir]
#
# This will create two tarballs in the kernel source dir:
# boot-x.y.z.v contains /boot and /lib/modules stuff
# inc-x.y.z.v  contains /usr/src/linux-x.y.z/include
# The "v" name component is the build number from .version.
#
# The "-a" argument does a "make dep bzImage modules" cycle
# before building tarballs.
#

set -e
#J=-j2
J=""
buildall=false
if [ "$1" = "-a" ]; then
    buildall=true
    shift
fi

[ "$1" -a -d "$1" ] && cd "$1"
[ -e kernel/panic.c -a -d include/linux ] || { echo "usage: $0 srcdir"; exit 1; }

if $buildall ; then
    make dep
    rm -f make.log.0
    [ -f make.log ] && mv make.log make.log.0
    time make $J bzImage modules 2>&1 | tee make.log
fi

mkdir tmp 2>/dev/null || true
rm -rf tmp/boot tmp/lib tmp/usr 2>/dev/null

va=`sed -n 's/^#define UTS_RELEASE "\(.*\)".*$/\1/p' include/linux/version.h`
vb=`cat .version`
vc="$va.$vb"

mkdir tmp/boot
cp -p arch/i386/boot/bzImage tmp/boot/vmlinuz-$va
cp -p System.map tmp/boot/System.map-$va
cp -p .config tmp/boot/config-$va
make modules_install INSTALL_MOD_PATH=`pwd`/tmp
rm -f tmp/lib/modules/$va/build || true

echo "Creating boot-$vc.tar.gz"
( cd tmp; tar chzf ../boot-$vc.tar.gz boot lib )

mkdir -p tmp/usr/src/linux-$va/include
ARCH=`uname -m | sed 's/i.86/i386/;s/sun4u/sparc64/;s/arm.*/arm/;s/sa110/arm/'`
GENINC=`find include -mindepth 1 -maxdepth 1 \! -name 'asm*'`

cp -pR $GENINC include/asm-$ARCH tmp/usr/src/linux-$va/include
( cd tmp/usr/src/linux-$va/include; ln -s asm-$ARCH asm )
echo "Creating inc-$vc.tar.gz"
( cd tmp; tar czf ../inc-$vc.tar.gz usr )

rm -rf tmp/boot tmp/lib tmp/usr

Re: Makefile gcc -o /dev/null: the dissapearing of /dev/null

[Jim Treadway]

On Sun, 30 Sep 2001, Olaf Titz wrote:

> > So then you can no longer 'make modules && make modules_install', or you
> > have to cp or chown /usr/src/linux on a fresh install to compile your
> > kernel?   Doesn't sound pleasant to me.
>
> I just do "MakeDist -a" as user and untar the result as root. Script
> appended.

[SNIPPED]

Thanks :) but my "complaint" was really that it was suggested that
compiling the kernel (or rather, evaluating certain Makefile variables)
as root was akin to crossing a busy freeway blindfolded or something.

This all goes back to a disappearing "/dev/null" which apparently may have
been an isolated incident.


Jim