Automounting on /. deleting entire FS!

Automounting on /. deleting entire FS!

[Martin Fick]

Yup,

  The #$@!! sys admin of my home machine, (that's me,)
decided to fool the automounter into using '/' to mount his
partiions on and lost his entire filesystem.  Pouff,
everything that was not a mounted partition gone!!!  Now I
realize that I shouldn' have been deceitfull to the poor
little autofs daemon, I told hime to use '/.' since he
balked at '/', but did I really deserve this?

So a few questions:

   1) Why does the automounter delete files (or entire filesystems)? 

   2) How can one go about automounting a cdrom on /cdrom?

   3) Should this be fixed? i.e. if it's bad to use slash, then
      should any other path that works out to be slash be checked
      also? :)

   4) Are my files really gone?  Is there anything I can do to
      recover? (it is an ext3 fs, not ext2)     


  Still weeping/laughing,
  
  -Martin

Re: Automounting on /. deleting entire FS!

[Jeff Moyer]

==> Regarding [autofs] Automounting on /. deleting entire FS!; Martin Fick <fick@fgm.com> adds:

fick> Yup, The #$@!! sys admin of my home machine, (that's me,) decided to
fick> fool the automounter into using '/' to mount his partiions on and
fick> lost his entire filesystem.  Pouff, everything that was not a mounted
fick> partition gone!!!  Now I realize that I shouldn' have been deceitfull
fick> to the poor little autofs daemon, I told hime to use '/.' since he
fick> balked at '/', but did I really deserve this?

fick> So a few questions:

fick>    1) Why does the automounter delete files (or entire filesystems)?

It doesn't, it simply mounted over your '/' mountpoint.  If you umount /,
you get what was there before (I'm guessing.  never really overmounted /
before).

fick>    2) How can one go about automounting a cdrom on /cdrom?

You could automount it in /misc/cdrom, and create a symlink.

# ln -sf /misc/cdrom /cdrom

fick>    3) Should this be fixed? i.e. if it's bad to use slash, then
fick> should any other path that works out to be slash be checked also? :)

Probably.

fick>    4) Are my files really gone?  Is there anything I can do to
fick> recover? (it is an ext3 fs, not ext2)

No, see the answer to 1) above.  If you can't simply umount /, then you can
boot in single user mode.  This shouldn't start the automounter.  Then, you
can edit your automount config files to not overmount /.  You should be
good to go after that.

fick>   Still weeping/laughing,

You gave me a chuckle.

-Jeff

Re: Automounting on /. deleting entire FS!

[Martin Fick]

Jeff,


> fick>    1) Why does the automounter delete files (or entire filesystems)?
> 
> It doesn't, it simply mounted over your '/' mountpoint.  If you umount /,
> you get what was there before (I'm guessing.  never really overmounted /
> before).

Actually, I have rebooted and that is not the case,
everything IS gone, I even rebooted using another (backup)
root filesystem  (I keep one around for this reason) and
mounted the old root and looked around.  Again, only the
old mount points for partitions that were mounted are
left.  

Are you saying that this not supposed to be the case?  Have
I uncovered a bug?





> fick>    2) How can one go about automounting a cdrom on /cdrom?
> 
> You could automount it in /misc/cdrom, and create a symlink.
> 
> # ln -sf /misc/cdrom /cdrom

Thanks.  I was hoping for a solution that did it for real,
thus the attempted hack triggering this conversation.




 
> fick>    4) Are my files really gone?  Is there anything I can do to
> fick> recover? (it is an ext3 fs, not ext2)
> 
> No, see the answer to 1) above.  If you can't simply umount /, then you can
> boot in single user mode.  This shouldn't start the automounter.  Then, you
> can edit your automount config files to not overmount /.  You should be
> good to go after that.



I didn't do single user, I guess I could check that too,
but I'm pretty sure that won't work either -> see above.





> fick>   Still weeping/laughing,
> 
> You gave me a chuckle.

If I can't share the misery, at least I can share the fun.:) 
Actually I think my wife is sharing the misery, :(



  -Martin

Re: Automounting on /. deleting entire FS!

[Jeff Moyer]

==> Regarding Re: [autofs] Automounting on /. deleting entire FS!; Martin Fick <fick@fgm.com> adds:

fick> Jeff, 1) Why does the automounter delete files (or entire
fick> filesystems)?
>> It doesn't, it simply mounted over your '/' mountpoint.  If you umount
>> /, you get what was there before (I'm guessing.  never really
>> overmounted / before).

fick> Actually, I have rebooted and that is not the case, everything IS
fick> gone, I even rebooted using another (backup) root filesystem (I keep
fick> one around for this reason) and mounted the old root and looked
fick> around.  Again, only the old mount points for partitions that were
fick> mounted are left.

fick> Are you saying that this not supposed to be the case?  Have I
fick> uncovered a bug?

Nothing is in /etc/ on your old root partition?  I don't see anything in
the autofs code that would remove an entire directory hierarchy plus its
contents.

-Jeff

Re: Automounting on /. deleting entire FS!

[Martin Fick]

> fick> Jeff, 1) Why does the automounter delete files (or entire
> fick> filesystems)?
> >> It doesn't, it simply mounted over your '/' mountpoint.  If you umount
> >> /, you get what was there before (I'm guessing.  never really
> >> overmounted / before).
> 
> fick> Actually, I have rebooted and that is not the case, everything IS
> fick> gone, I even rebooted using another (backup) root filesystem (I keep
> fick> one around for this reason) and mounted the old root and looked
> fick> around.  Again, only the old mount points for partitions that were
> fick> mounted are left.
> 
> fick> Are you saying that this not supposed to be the case?  Have I
> fick> uncovered a bug?
> 
> Nothing is in /etc/ on your old root partition?  I don't see anything in
> the autofs code that would remove an entire directory hierarchy plus its
> contents.


Well, I don't have acces to the system here, I am at work,
but I didn't see any files, just a few vacant directories
left such as:

	/bin (maybe)
	/dev
	/etc
	/home/fick (my home dir, empty!)
	/mnt
	/usr (was mounted externally)

I had assumed at first that it was something like what you
suggested and so I simply rebooted into the other partition
and was shocked to find that nothing came back!


  -Martin

Re: Automounting on /. deleting entire FS!

[Steffen Grunewald]

On Mon, Oct 11, 2004 at 12:29:27PM -0400, Martin Fick wrote:
> > Nothing is in /etc/ on your old root partition?  I don't see anything in
> > the autofs code that would remove an entire directory hierarchy plus its
> > contents.
> Well, I don't have acces to the system here, I am at work,
> but I didn't see any files, just a few vacant directories
> left such as:
> 
> 	/bin (maybe)
> 	/dev
> 	/etc
> 	/home/fick (my home dir, empty!)
> 	/mnt
> 	/usr (was mounted externally)

Looks like the contents of fs cache. 
You might try to find out what's there (from single user best, but
another root partition would do too) using df. Make sure to first mount
read-only, then (when you can see everything is somehow still alive)
remount rw. fsck'ing may be a Very Bad Idea. If you can see files it'd
be a good idea to do some kind of backup (if it works, you won't need it
afterwards in 99% of all cases) before going rw.

There's still hope.

Cheers,
 Steffen

(BTW, it's a quite common trap, almost as efficient as chown -R xyz .*)

> I had assumed at first that it was something like what you
> suggested and so I simply rebooted into the other partition
> and was shocked to find that nothing came back!

??? so you started another kernel and gave it the root=... arg?
Then of course automount would be running before you can do anything...
there's perhaps even more hope than you'd expect!

-- 
Steffen Grunewald * * * Merlin cluster admin (http://pandora.aei.mpg.de)
Albert-Einstein-Institut (MPI Gravitationsphysik, http://www.aei.mpg.de)
       Science Park Golm, Am Mühlenberg 1, 14476 Potsdam, Germany
e-mail: steffen.grunewald(*)aei.mpg.de * +49-331-567-{fon:7233,fax:7298}

Re: Automounting on /. deleting entire FS!

[Martin Fick]

On Mon, Oct 11, 2004 at 07:00:11PM +0200, Steffen Grunewald wrote:
> On Mon, Oct 11, 2004 at 12:29:27PM -0400, Martin Fick wrote:
> > > Nothing is in /etc/ on your old root partition?  I don't see anything in
> > > the autofs code that would remove an entire directory hierarchy plus its
> > > contents.
> > Well, I don't have acces to the system here, I am at work,
> > but I didn't see any files, just a few vacant directories
> > left such as:
> > 
> > 	/bin (maybe)
> > 	/dev
> > 	/etc
> > 	/home/fick (my home dir, empty!)
> > 	/mnt
> > 	/usr (was mounted externally)
> 
> Looks like the contents of fs cache. 
> You might try to find out what's there (from single user best, but
> another root partition would do too) using df. Make sure to first mount
> read-only, then (when you can see everything is somehow still alive)
> remount rw. fsck'ing may be a Very Bad Idea. If you can see files it'd
> be a good idea to do some kind of backup (if it works, you won't need it
> afterwards in 99% of all cases) before going rw.

  Hmm, I'n not familiar with the automounter design at all,
what do you mean by fs cache.  Chances are fsck has already
been run since the first time I rebooted it was into the
same partition.  I'm not really sure what you are
suggesting here since I already remounted it from another
partition and saw the above directories?




> There's still hope.

  thanks, :)




> (BTW, it's a quite common trap, almost as efficient as chown -R xyz .*)

  What is the common trap, automounting over slash?




> > I had assumed at first that it was something like what you
> > suggested and so I simply rebooted into the other partition
> > and was shocked to find that nothing came back!
> 
> ??? so you started another kernel and gave it the root=... arg?
> Then of course automount would be running before you can do anything...
> there's perhaps even more hope than you'd expect!


  No no no, I started another kernel and another root
partition, then mounted the old root on /mnt and looked
around.



  -Martin

Re: Automounting on /. deleting entire FS!

[Steffen Grunewald]

On Mon, Oct 11, 2004 at 01:16:12PM -0400, Martin Fick wrote:
> > Looks like the contents of fs cache. 
> > You might try to find out what's there (from single user best, but
> > another root partition would do too) using df. Make sure to first mount
> > read-only, then (when you can see everything is somehow still alive)
> > remount rw. fsck'ing may be a Very Bad Idea. If you can see files it'd
> > be a good idea to do some kind of backup (if it works, you won't need it
> > afterwards in 99% of all cases) before going rw.
> 
>   Hmm, I'n not familiar with the automounter design at all,
> what do you mean by fs cache.  Chances are fsck has already
> been run since the first time I rebooted it was into the
> same partition.  I'm not really sure what you are
> suggesting here since I already remounted it from another
> partition and saw the above directories?

Well, I'm a bit confused. You "rebooted into the same partition" ... and
"remounted it from another partition". At the same time, or were these
different attempts?
YOu should really "boot single" to be able to rename the auto.master
file. What about Knoppix? It's a perfect tool to repair damaged
installations :-)

Cheers,
 Steffen
 
> > (BTW, it's a quite common trap, almost as efficient as chown -R xyz .*)
>   What is the common trap, automounting over slash?

Yep. Did this ages ago, under SunOS (or was it already Solaris?)

> 
> > ??? so you started another kernel and gave it the root=... arg?
> > Then of course automount would be running before you can do anything...
> > there's perhaps even more hope than you'd expect!
> 
> 
>   No no no, I started another kernel and another root
> partition, then mounted the old root on /mnt and looked
> around.

Hmmm. What did 'df' tell you?

-- 
Steffen Grunewald * * * Merlin cluster admin (http://pandora.aei.mpg.de)
Albert-Einstein-Institut (MPI Gravitationsphysik, http://www.aei.mpg.de)
       Science Park Golm, Am Mühlenberg 1, 14476 Potsdam, Germany
e-mail: steffen.grunewald(*)aei.mpg.de * +49-331-567-{fon:7233,fax:7298}

Re: Automounting on /. deleting entire FS!

[Mike Waychison]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Steffen Grunewald wrote:
> On Mon, Oct 11, 2004 at 01:16:12PM -0400, Martin Fick wrote:

> 
>>>(BTW, it's a quite common trap, almost as efficient as chown -R xyz .*)
>>
>>  What is the common trap, automounting over slash?
> 
> 
> Yep. Did this ages ago, under SunOS (or was it already Solaris?)
> 

Heh, for another funny 'trap':   umount -l /

Don't do this unless you're ready for a reboot!


- --
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE:  The opinions expressed in this email are held by me,
and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBashXdQs4kOxk3/MRAiNYAJ9sObJ1deno7H7i/IvyKZAmPRCqDgCfQwBY
ibPElCHWeYDQ+Kzpg5/Qdx4=
=/Q3U
-----END PGP SIGNATURE-----

Re: Automounting on /. deleting entire FS!

[Martin Fick]

Steffen,


> > > Looks like the contents of fs cache.
> > > You might try to find out what's there (from single user best, but
> > > another root partition would do too) using df. Make sure to first mount
> > > read-only, then (when you can see everything is somehow still alive)
> > > remount rw. fsck'ing may be a Very Bad Idea. If you can see files it'd
> > > be a good idea to do some kind of backup (if it works, you won't need it
> > > afterwards in 99% of all cases) before going rw.
> >
> >   Hmm, I'n not familiar with the automounter design at all,
> > what do you mean by fs cache.  Chances are fsck has already
> > been run since the first time I rebooted it was into the
> > same partition.  I'm not really sure what you are
> > suggesting here since I already remounted it from another
> > partition and saw the above directories?
>
> Well, I'm a bit confused. You "rebooted into the same partition" ... and
> "remounted it from another partition".

  Yes


> At the same time, or were these different attempts?

  Separate attempts.  When I first noticed something wrong
(ls would no longer work), I rebooted into the same
partition figuring that if the automounter was somehow
covering up the / filesystem, that it would not do that on
reboot since I would not have accesed /cdrom yet. I assume
that with ext3, this would have fscked the partition.

  Once I rebooted and realized that (alomst) everything
was  still gone, I rebooted into my safe partition and
mounted the original (potentially trashed) partition on
/mnt so that I could peruse it, -> files still missing.
This mounting would probably not have run fsck (I did not
run it manually.)




> YOu should really "boot single" to be able to rename the auto.master
> file. What about Knoppix? It's a perfect tool to repair damaged
> installations :-)

  I will boot single tonight and look around even more.
When this first happened, I simply decided to cut my losses
and stop (you know, late at night).  I figured I simply did
something that would trash my filesystem, but I wasn't sure
and wanted more info before I decided to cut my losses and
rebuild.  It sounds like I am getting the signal that I
should not have actually lost anything, so I will need to
go home and take a deeper look.



> Hmmm. What did 'df' tell you?

  Again, mind you this was late Friday, but I think I
recall trying df and noticing that the available space was
very large and used space very low.  Before that my system
was pretty tight, at least 80% used.  This was another
indicator to me that I had goofed royally and that the
filesystem was not simply being masked.

  I'm sorry that I can't double check everyting I am saying
until tonight, I will though.  You now have me convinced
that my system should not have been deleted.  Is this
something that is valuable for this list to be sure about,
should I hold off rebuilding my machine until we know
exactly what happened?

  I am running debian sid if that makes a difference,


  -Martin