pivot_root broken in 2.6.15-rc1-mm2

pivot_root broken in 2.6.15-rc1-mm2

[Neil Brown]

Pivot_root seems to be broken in 2.6.15-rc1-mm2.

I havea initramfs filesystem, mount a ext3 filesystem (which has /mnt)
at '/root' and

  cd /root
  pivot . mnt

and it says -EINVAL.

After putting in copious tracing printk, the offending test is:

	if (user_nd.mnt->mnt_parent == user_nd.mnt)
		goto out2; /* not attached */

If I remove this, it works (or seems to).
Presumably the initial root file system is 'not attached'.  But that
shouldn't be a problem, should it?

Could this be related to the new shared mounts stuff???

NeilBrown

Re: pivot_root broken in 2.6.15-rc1-mm2

[Al Viro]

On Wed, Nov 23, 2005 at 01:07:28PM +1100, Neil Brown wrote:
> After putting in copious tracing printk, the offending test is:
> 
> 	if (user_nd.mnt->mnt_parent == user_nd.mnt)
> 		goto out2; /* not attached */
> 
> If I remove this, it works (or seems to).
> Presumably the initial root file system is 'not attached'.  But that
> shouldn't be a problem, should it?

Initial root is root and will remain root, period.

> Could this be related to the new shared mounts stuff???

No.  And no, it's not going to change - any memory you win on killing
initramfs is not going to be worth the extra code needed to special-case
it.

pivot_root() does work in chroot jail (including the one we get after
chrooting to "final" root), but that's all it does.

Re: pivot_root broken in 2.6.15-rc1-mm2

[Johannes Stezenbach]

On Wed, Nov 23, 2005 at 02:15:45AM +0000, Al Viro wrote:
> On Wed, Nov 23, 2005 at 01:07:28PM +1100, Neil Brown wrote:
> > After putting in copious tracing printk, the offending test is:
> > 
> > 	if (user_nd.mnt->mnt_parent == user_nd.mnt)
> > 		goto out2; /* not attached */
> > 
> > If I remove this, it works (or seems to).
> > Presumably the initial root file system is 'not attached'.  But that
> > shouldn't be a problem, should it?
> 
> Initial root is root and will remain root, period.
> 
> > Could this be related to the new shared mounts stuff???
> 
> No.  And no, it's not going to change - any memory you win on killing
> initramfs is not going to be worth the extra code needed to special-case
> it.
> 
> pivot_root() does work in chroot jail (including the one we get after
> chrooting to "final" root), but that's all it does.

Just want to add that some nice person wrote
Documentation/filesystems/ramfs-rootfs-initramfs.txt.

http://lwn.net/Articles/156098/
http://kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;h=b3404a0325967de5fdce4a29e879258075ede500;hb=4b4a27dff4e2d4cc2eac1cde31aede834a966a48;f=Documentation/filesystems/ramfs-rootfs-initramfs.txt

HTH,
Johannes

Re: pivot_root broken in 2.6.15-rc1-mm2

[Neil Brown]

On Wednesday November 23, viro@ftp.linux.org.uk wrote:
> On Wed, Nov 23, 2005 at 01:07:28PM +1100, Neil Brown wrote:
> > After putting in copious tracing printk, the offending test is:
> > 
> > 	if (user_nd.mnt->mnt_parent == user_nd.mnt)
> > 		goto out2; /* not attached */
> > 
> > If I remove this, it works (or seems to).
> > Presumably the initial root file system is 'not attached'.  But that
> > shouldn't be a problem, should it?
> 
> Initial root is root and will remain root, period.
> 
> > Could this be related to the new shared mounts stuff???
> 
> No.  And no, it's not going to change - any memory you win on killing
> initramfs is not going to be worth the extra code needed to special-case
> it.

Ah, OK.
It's just that pivot_root works in this context in 2.6.11.9, so I
figured it was a breakage.

I'll go read ramfs-rootfs-initramfs.txt 

Thanks,
NeilBrown

Re: pivot_root broken in 2.6.15-rc1-mm2

[Rob Landley]

On Tuesday 22 November 2005 21:01, Neil Brown wrote:
> Ah, OK.
> It's just that pivot_root works in this context in 2.6.11.9, so I
> figured it was a breakage.

And if you then umount the ramfs you just pivoted, the kernel locks hard.

That was a bug.

What you're looking for is switch_root, which has variants buried in klib, or 
in the current CVS version of busybox, or glued into Red Hat's weird little 
multi-function ramdisk shell, and probably a few other places by now.

Rather than unmounting rootfs, it deletes everything out of it to free up the 
space.  (It basically does the functional equivalent of "find / -xdev | xargs 
rm -rf", overmounts the old root with the new root, does a chroot, and execs 
the new init out of the new root.  Actually getting it right's a bit tricky, 
of course.  I still need to test the busybox version a whole lot more.  It's 
on my to-do list...)

Rob

Re: pivot_root broken in 2.6.15-rc1-mm2

[Nix]

On 23 Nov 2005, Rob Landley gibbered uncontrollably:
> Rather than unmounting rootfs, it deletes everything out of it to free up the 
> space.  (It basically does the functional equivalent of "find / -xdev | xargs 
> rm -rf"

Er, find / -xdev | xargs rm -f, I hope.

(rm won't respect the -xdev you gave to find, and, well, if your new root is
mounted at all, you're dead :) )

-- 
`Y'know, London's nice at this time of year. If you like your cities
 freezing cold and full of surly gits.' --- David Damerell

Re: pivot_root broken in 2.6.15-rc1-mm2

[Rob Landley]

On Friday 25 November 2005 05:52, Nix wrote:
> On 23 Nov 2005, Rob Landley gibbered uncontrollably:
> > Rather than unmounting rootfs, it deletes everything out of it to free up
> > the space.  (It basically does the functional equivalent of "find / -xdev
> > | xargs rm -rf"
>
> Er, find / -xdev | xargs rm -f, I hope.

Yeah. :)

> (rm won't respect the -xdev you gave to find, and, well, if your new root
> is mounted at all, you're dead :) )

It's C code, not shell, so that was off the top of my head. :)

And that, in fact, is one of the big reasons that there's a utility for it: 
not accidentally deleting anything out of your root partition is kind of 
important.

(Another is that calling chroot and such after deleting their binaries out of 
initramfs but before the paths are adjusted so that the ones in the new root 
can find their shared libraries is a bit of a headache.  It's a lot easier to 
just have it all in one binary that's already loaded everything it needs and 
frees its memory on exec.)

Rob
-- 
Steve Ballmer: Innovation!  Inigo Montoya: You keep using that word.
I do not think it means what you think it means.

Re: pivot_root broken in 2.6.15-rc1-mm2

[Nix]

On Fri, 25 Nov 2005, Rob Landley prattled cheerily:
> (Another is that calling chroot and such after deleting their binaries out of 
> initramfs but before the paths are adjusted so that the ones in the new root 
> can find their shared libraries is a bit of a headache.  It's a lot easier to 
> just have it all in one binary that's already loaded everything it needs and 
> frees its memory on exec.)

Oh yes, agreed 100%: this is one of those rare places where doing shell-like
stuff in a tiny dedicated binary really does make sense.

-- 
`Y'know, London's nice at this time of year. If you like your cities
 freezing cold and full of surly gits.' --- David Damerell

Re: pivot_root broken in 2.6.15-rc1-mm2

[Rob Landley]

On Tuesday 22 November 2005 20:07, Neil Brown wrote:
> Pivot_root seems to be broken in 2.6.15-rc1-mm2.
>
> I havea initramfs filesystem, mount a ext3 filesystem (which has /mnt)
> at '/root' and
>
>   cd /root
>   pivot . mnt
>
> and it says -EINVAL.

You can't pivot_root initramfs because initramfs is rootfs.

I wrote Documentation/filesystems/ramfs-rootfs-initramfs.txt just for this 
occasion. :)

Rob

Re: pivot_root broken in 2.6.15-rc1-mm2

[Neil Brown]

On Wednesday November 23, rob@landley.net wrote:
> On Tuesday 22 November 2005 20:07, Neil Brown wrote:
> > Pivot_root seems to be broken in 2.6.15-rc1-mm2.
> >
> > I havea initramfs filesystem, mount a ext3 filesystem (which has /mnt)
> > at '/root' and
> >
> >   cd /root
> >   pivot . mnt
> >
> > and it says -EINVAL.
> 
> You can't pivot_root initramfs because initramfs is rootfs.
> 
> I wrote Documentation/filesystems/ramfs-rootfs-initramfs.txt just for this 
> occasion. :)

Unfortunately, 'man pivot_root' nor 'use the source, Luke' contain
pointers to this particular useful document.  They both list assorted
restrictions on pivot_root, but not this one.

How about the following?

Thanks,
NeilBrown

Signed-off-by: Neil Brown <neilb@suse.de>

### Diffstat output
 ./fs/namespace.c |    4 ++++
 1 file changed, 4 insertions(+)

diff ./fs/namespace.c~current~ ./fs/namespace.c
--- ./fs/namespace.c~current~	2005-11-23 14:24:59.000000000 +1100
+++ ./fs/namespace.c	2005-11-24 16:07:01.000000000 +1100
@@ -1526,6 +1526,10 @@ static void chroot_fs_refs(struct nameid
  * pointed to by put_old must yield the same directory as new_root. No other
  * file system may be mounted on put_old. After all, new_root is a mountpoint.
  *
+ * Also, the current root cannot be on the 'rootfs' (initial ramfs) filesystem.
+ * See Documentation/filesystems/ramfs-rootfs-initramfs.txt for alternatives
+ * in this situation.
+ *
  * Notes:
  *  - we don't move root/cwd if they are not at the root (reason: if something
  *    cared enough to change them, it's probably wrong to force them elsewhere)

Re: [PATCH] pivot_root broken in 2.6.15-rc1-mm2

[Rob Landley]

On Wednesday 23 November 2005 23:09, Neil Brown wrote:
> On Wednesday November 23, rob@landley.net wrote:
> > On Tuesday 22 November 2005 20:07, Neil Brown wrote:
> > > Pivot_root seems to be broken in 2.6.15-rc1-mm2.
> > >
> > > I havea initramfs filesystem, mount a ext3 filesystem (which has /mnt)
> > > at '/root' and
> > >
> > >   cd /root
> > >   pivot . mnt
> > >
> > > and it says -EINVAL.
> >
> > You can't pivot_root initramfs because initramfs is rootfs.
> >
> > I wrote Documentation/filesystems/ramfs-rootfs-initramfs.txt just for
> > this occasion. :)
>
> Unfortunately, 'man pivot_root' nor 'use the source, Luke' contain
> pointers to this particular useful document.  They both list assorted
> restrictions on pivot_root, but not this one.
>
> How about the following?
>
> Thanks,
> NeilBrown
>
> Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Rob Landley <rob@landley.net>

> ### Diffstat output
>  ./fs/namespace.c |    4 ++++
>  1 file changed, 4 insertions(+)
>
> diff ./fs/namespace.c~current~ ./fs/namespace.c
> --- ./fs/namespace.c~current~ 2005-11-23 14:24:59.000000000 +1100
> +++ ./fs/namespace.c 2005-11-24 16:07:01.000000000 +1100
> @@ -1526,6 +1526,10 @@ static void chroot_fs_refs(struct nameid
>   * pointed to by put_old must yield the same directory as new_root. No
> other * file system may be mounted on put_old. After all, new_root is a
> mountpoint. *
> + * Also, the current root cannot be on the 'rootfs' (initial ramfs)
> filesystem. + * See Documentation/filesystems/ramfs-rootfs-initramfs.txt
> for alternatives + * in this situation.
> + *
>   * Notes:
>   *  - we don't move root/cwd if they are not at the root (reason: if
> something *    cared enough to change them, it's probably wrong to force
> them elsewhere)

Re: pivot_root broken in 2.6.15-rc1-mm2

[Nix]

On 23 Nov 2005, Rob Landley said:
> I wrote Documentation/filesystems/ramfs-rootfs-initramfs.txt just for this 
> occasion. :)

And very damn helpful it was, too; thank you. I didn't really grasp
the fundamental difference between initramfs and initrd until now: I
thought it was just a matter of storage location (cpio-in-kernel versus
image-in-separate-file)...

-- 
`Y'know, London's nice at this time of year. If you like your cities
 freezing cold and full of surly gits.' --- David Damerell