Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Patrick Mau]

Hallo all,

I'm using kernel 2.4.15-pre6 and I can see my journal file
on '/'. Should I worry ?


[root@tony] dmesg
...
ip_tables: (c)2000 Netfilter core team
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
...


[root@tony] ls -ali /
total 65720
   2 drwxr-xr-x   24 root     root         4096 Nov 20 00:26 .
   2 drwxr-xr-x   24 root     root         4096 Nov 20 00:26 ..
2930 -rw-------    1 root     root     67108864 Nov 18 19:56 .journal
					^^^^^^^ created as -J size=64


[root@tony] tune2fs -l /dev/sda1
tune2fs 1.25 (20-Sep-2001)
Filesystem volume name:   /
Last mounted on:          <not available>
Filesystem UUID:          b909b36d-8f16-4be1-9614-5049bad90e96
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal filetype needs_recovery sparse_super
                                               ^^^^^^^^^^^^^^ 
					       ??????????????

Journal inode:            2930    <--- like 'ls' said
Journal device:           0x0000


[root@tony] mount
/dev/sda1 on / type ext3 (rw)


lilo.conf sniplet:
	image   = /boot/vmlinuz-2.4.15-6
	label   = linux
	append  = "video=matrox:vesa:261,fv:80,font:VGA8x16 rootfstype=ext3"


Could someone please comment on this ?
I'm feeling kind of worried.

thanks,
Patrick

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Ryan Cumming]

On November 19, 2001 15:43, Patrick Mau wrote:
> Hallo all,
>
> I'm using kernel 2.4.15-pre6 and I can see my journal file
> on '/'. Should I worry ?
No, apparently the .journal file is visible if you created while the 
filesystem is mounted, but invisible if you create it when the filesystem is 
unmounted. Probably because you can't mess with the filesystem too much when 
its mounted without confusing the filesystem driver, so creating hidden files 
is out of the question. In either case, I created my journal with the 
filesystem mounted, and it created a visible, immutable .journal file in my 
root directory, and my filesystem has yet to explode.

> [root@tony] ls -ali /
> total 65720
>    2 drwxr-xr-x   24 root     root         4096 Nov 20 00:26 .
>    2 drwxr-xr-x   24 root     root         4096 Nov 20 00:26 ..
> 2930 -rw-------    1 root     root     67108864 Nov 18 19:56 .journal
> 					^^^^^^^ created as -J size=64
64 * (1024^2) = 67108864
Remember 1K = 1024, and 1M = 1024^2

>
> [root@tony] tune2fs -l /dev/sda1
> tune2fs 1.25 (20-Sep-2001)
> Filesystem volume name:   /
> Last mounted on:          <not available>
> Filesystem UUID:          b909b36d-8f16-4be1-9614-5049bad90e96
> Filesystem magic number:  0xEF53
> Filesystem revision #:    1 (dynamic)
> Filesystem features:      has_journal filetype needs_recovery sparse_super
>                                                ^^^^^^^^^^^^^^
> 					       ??????????????
The flag "filetype needs_recovery" is always set on a mounted filesystem, 
it's unset once you umount. In the event of power failure, the flag will be 
left set, and fsck knows the it has to recover the filesytem. Pretty clever, 
eh?

> Could someone please comment on this ?
> I'm feeling kind of worried.
Everything seems to be in order...

-Ryan

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Tommi Kyntola]

> > I'm using kernel 2.4.15-pre6 and I can see my journal file
> > on '/'. Should I worry ?
> No, apparently the .journal file is visible if you created while the 
> filesystem is mounted, but invisible if you create it when the filesystem is 
> unmounted.

Minor corrections though, atleast on my 2.4.15-pre6 with tune2fs 1.23
the .journal is visible even when created as unmounted.
And it is not immutable, the only ext2 file attribute set is the "d"
(which refers to making it not a candidate for backup for dump,
 immutable would be "i", the source of all those funny
 # whoami
 root
 # ls foobar
 foobar
 # rm foobar
 rm: cannot unlink `foobar': Operation not permitted
 # echo "wtf?"). 

I was a bit surprised when the 'rm .journal' succeeded and removed it but 
I take it that just hides it and the inode is still kept for the journal.

(This removal would've happened anyway when my /tmp sweeps would've erased 
 it had I not known it's there to begin with)

Is the 'rm .journal' the way preferred way to "hide" it?
Are there any side-effects caused by removal?
Could someone shed a little more light on this subject.
What would happen if root were to write to it?

--------------- 8< ----------------- 8< -----------------

root@behemoth / # grep /tmp /etc/fstab
/dev/hda2               /tmp                    auto    defaults        1 2
root@behemoth / # grep /tmp /etc/mtab
/dev/hda2 /tmp ext3 rw 0 0
root@behemoth / # umount /tmp
root@behemoth / # tune2fs -O ^has_journal /dev/hda2
tune2fs 1.23, 15-Aug-2001 for EXT2 FS 0.5b, 95/08/09
root@behemoth / # mount /tmp
root@behemoth / # grep /tmp /etc/mtab
/dev/hda2 /tmp ext2 rw 0 0
root@behemoth / # umount /tmp
root@behemoth / # tune2fs -j /dev/hda2
tune2fs 1.23, 15-Aug-2001 for EXT2 FS 0.5b, 95/08/09
Creating journal inode: done
This filesystem will be automatically checked every -1 mounts or
0 days, whichever comes first.  Use tune2fs -c or -i to override.
root@behemoth / # mount /tmp
kjournald starting.  Commit interval 5 seconds
EXT3 FS 2.4-0.9.15, 06 Nov 2001 on ide0(3,2), internal journal
EXT3-fs: mounted filesystem with ordered data mode.
root@behemoth / # ls -la /tmp | grep journal
-rw-------    1 root     root      8388608 Oct 23 17:10 .journal
root@behemoth / # lsattr /tmp/.journal
-----d------- /tmp/.journal
root@behemoth / # tune2fs
tune2fs 1.23, 15-Aug-2001 for EXT2 FS 0.5b, 95/08/09
...

--------------- 8< ----------------- 8< -----------------

root@behemoth / # cd /tmp
root@behemoth /tmp # rm .journal
root@behemoth /tmp # ls -la | grep journal
root@behemoth /tmp # cd ..
root@behemoth / # umount /tmp
root@behemoth / # tune2fs -j /dev/hda2
tune2fs 1.23, 15-Aug-2001 for EXT2 FS 0.5b, 95/08/09
The filesystem already has a journal.

-- 
  Tommi "Kynde" Kyntola      
     /* A man alone in the forest talking to himself and 
        no women around to hear him. Is he still wrong?  */

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Ryan Cumming]

On November 19, 2001 17:20, you wrote:
> > > I'm using kernel 2.4.15-pre6 and I can see my journal file
> > > on '/'. Should I worry ?
> >
> > No, apparently the .journal file is visible if you created while the
> > filesystem is mounted, but invisible if you create it when the filesystem
> > is unmounted.
>
> Minor corrections though, atleast on my 2.4.15-pre6 with tune2fs 1.23
> the .journal is visible even when created as unmounted.
That's why I said "apparently", I had only heard that the invisible journal 
was only expected to be created on unmounted filesystems, and I hadn't had an 
oppertunity to test it. Thanks for the info.

> And it is not immutable, the only ext2 file attribute set is the "d"
Not on my system:
bodnar42:/home/bodnar42# rm /.journal
rm: remove write-protected file `/.journal'? y
rm: cannot unlink `/.journal': Operation not permit

That's about as immutable as a file can get, and I'm quite sure I did not set 
it immutable manually.

-Ryan

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Tommi Kyntola]

> > And it is not immutable, the only ext2 file attribute set is the "d"
> Not on my system:
> bodnar42:/home/bodnar42# rm /.journal
> rm: remove write-protected file `/.journal'? y
> rm: cannot unlink `/.journal': Operation not permit
>
> That's about as immutable as a file can get, and I'm quite sure I did 
> not set it immutable manually.

Yes my mistake, the .journal is indeed immutable on my other ext3 
partitions, but as you could see from my previous mail, the
immutable wasn't set for that particular partition and
the rm succeeded.

(could be due to the fact that I had to remove it there using
the tune2fs -O to check out wether it's visible or not when
created unmounted)

Even so, I'm wondering wether this removal is standardad
procedure for hiding it once and for all or not?
Since what's there to stop you from 'chattr -i .journal ; rm .journal'.


-- 
  Tommi "Kynde" Kyntola      
     /* A man alone in the forest talking to himself and 
        no women around to hear him. Is he still wrong?  */

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Rik van Riel]

On Tue, 20 Nov 2001, Tommi Kyntola wrote:

> Since what's there to stop you from 'chattr -i .journal ; rm .journal'.

man 1 lart

Rik
-- 
Shortwave goes a long way:  irc.starchat.net  #swl

http://www.surriel.com/		http://distro.conectiva.com/

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Tommi Kyntola]

> > Since what's there to stop you from 'chattr -i .journal ; rm .journal'.
> 
> man 1 lart

:-) naturally

But still, I didnt chattr it to non immutable and rm is something
that the tmp sweeps would do for a .journal file in /tmp anyway.
So it's a pretty typical scenario. 

What I'm wondering is that wether removing it is has any intended (or 
otherwise) consequences?

-- 
  Tommi "Kynde" Kyntola      
     /* A man alone in the forest talking to himself and 
        no women around to hear him. Is he still wrong?  */

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Mike Castle]

On Tue, Nov 20, 2001 at 03:48:00AM +0200, Tommi Kyntola wrote:
> What I'm wondering is that wether removing it is has any intended (or 
> otherwise) consequences?

What are unintented consequences for not removing it?

I.e., backups.

mrc
-- 
     Mike Castle      dalgoda@ix.netcom.com      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Ryan Cumming]

On November 19, 2001 19:43, you wrote:
> What are unintented consequences for not removing it?
>
> I.e., backups.
It already has the "Don't back me up" attribute set ('d'), and I've come 
across absolutely no problems running my system with it lurking in my root 
directory.

-Ryan

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Mike Castle]

On Mon, Nov 19, 2001 at 07:46:13PM -0800, Ryan Cumming wrote:
> On November 19, 2001 19:43, you wrote:
> > What are unintented consequences for not removing it?
> >
> > I.e., backups.
> It already has the "Don't back me up" attribute set ('d'), and I've come 
> across absolutely no problems running my system with it lurking in my root 
> directory.

Which, as I understand it, only applies to dump for ext2fs.

Dump does not reliably work with live file systems with 2.4.x Linus
has even gone on record stating that he does not approve that approach, so
don't expect that to change.

What versions of gtar, star, find|cpio, amanda, or whatever, know how to
handle the `d' attribute?

mrc
-- 
     Mike Castle      dalgoda@ix.netcom.com      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Ryan Cumming]

On November 19, 2001 20:05, Mike Castle wrote:
> On Mon, Nov 19, 2001 at 07:46:13PM -0800, Ryan Cumming wrote:
> > On November 19, 2001 19:43, you wrote:
> > > What are unintented consequences for not removing it?
> > >
> > > I.e., backups.
> >
> > It already has the "Don't back me up" attribute set ('d'), and I've come
> > across absolutely no problems running my system with it lurking in my
> > root directory.
>
> Which, as I understand it, only applies to dump for ext2fs.
Yes, you're absolutely right. Backup tools may attempt to backup the 
.journal, which is fine, because it's safely readable, and usually not large 
enough to take up significant space on the backup media. And then on restore, 
the backup tool would hit hard against .journal's immutable flag, and 
probably end up complaining loudly and skipping the file. Seems harmless 
enough. Of course, tools such as tar accept a list of files to exclude, if 
you're worried about the space wastage.

-Ryan.

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Stephen C. Tweedie]

Hi,

On Mon, Nov 19, 2001 at 08:05:03PM -0800, Mike Castle wrote:

> Dump does not reliably work with live file systems with 2.4.x

It doesn't work reliably on live filesystems on ANY kernel.  That
property is enhanced a bit by 2.4, that's all.  There's no way that a
block-level dump can do things like walk file indirect chains
atomically on a live filesystem, even with 2.2.

Cheers,
 Stephen

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Ryan Cumming]

On November 19, 2001 17:37, you wrote:
> Even so, I'm wondering wether this removal is standardad
> procedure for hiding it once and for all or not?
On my system, the journal appears to have a perfectly normal inode number for 
a root entry (#22), which makes me think that it's just a normal file as far 
as the core filesystem code is concerned. So, when the file is deleted, its 
blocks are freed, and new allocations are free to walk all over the journal. 
That is if the filesystem doesn't barf because the superblock references a 
deleted inode for its journal. Just a theory, though.

Now, I heard (from the same source I vaugely remember reading about hidden 
journals, so take this with a grain of salt) that tune2fs would try to use 
reserved inode #8 for the .journal if possible, and the filesystem could 
handle deletion in that case just fine. So, hopefully the partition of yours 
was using the reserved inode number. Seeing deletion is no longer dangerous, 
tune2fs may have decidedly not set the immutable flag so that you're free to 
'hide' it using rm.

I think this is the part where the ext3 gods step in and free us from our 
ignorance-inspired conjecture.

> Since what's there to stop you from 'chattr -i .journal ; rm .journal'.
I think that's a case of "don't do that, then". I took the immutable flag 
being set as a pretty clear indiction that if I cleared the flag and started 
to play with the file, I pretty much deserve whatever I get. ;)

-Ryan

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Andreas Dilger]

On Nov 19, 2001  17:55 -0800, Ryan Cumming wrote:
> On November 19, 2001 17:37, you wrote:
> > Even so, I'm wondering wether this removal is standardad
> > procedure for hiding it once and for all or not?

Very definitely NOT.  It _may_ work until the filesystem is unmounted,
because the kernel will keep the file "open" so that the inode is not
freed, but the next time you try to mount the filesystem it will
complain about the journal being a bad inode.

> On my system, the journal appears to have a perfectly normal inode number for 
> a root entry (#22), which makes me think that it's just a normal file as far 
> as the core filesystem code is concerned.

Correct.  Normal, except that if you (as root) really work hard to fool with
it, you can potentially cause problems.  Don't do that.  The problems are
99.99% harmless - can't mount as ext3, e2fsck will complain, maybe you can't
boot your system, if it is the root fs.  If you really work at it, maybe
you can corrupt your fs, but that would take serious effort plus a crash.

> Now, I heard (from the same source I vaugely remember reading about hidden 
> journals, so take this with a grain of salt) that tune2fs would try to use 
> reserved inode #8 for the .journal if possible

Correct - IF the filesystem is unmounted when the journal is created.

> So, hopefully the partition of yours was using the reserved inode number.

This can be checked with "tune2fs -l <dev>" to see what it is really using
for the journal inode number.

> Seeing deletion is no longer dangerous, tune2fs may have decidedly not set
> the immutable flag so that you're free to 'hide' it using rm.

Just to repeat, so people don't start doing this - you cannot "hide" your
journal by deleting it.  Bad, bad, bad.  The most recent release of e2fsck
(1.25) will do it properly, by assigning the journal blocks to the reserved
inode #8, and removing the .journal entry from the directory.  This will
hopefully prevent users from trying to do more bad things to their journals.

I'm not 100% sure whether it can do this on the root fs, because the inode
is already in use, and that would seriously confuse things, I think.

> > Since what's there to stop you from 'chattr -i .journal ; rm .journal'.

> I think that's a case of "don't do that, then". I took the immutable flag 
> being set as a pretty clear indiction that if I cleared the flag and started 
> to play with the file, I pretty much deserve whatever I get. ;)

Correct.  We can only do so much to prevent users from shooting themselves
in the foot.  If you work at it, you can do it, at which point you learn.


Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Allan Sandfeld]

On Tuesday 20 November 2001 20:19, Andreas Dilger wrote:
> On Nov 19, 2001  17:55 -0800, Ryan Cumming wrote:
> > On November 19, 2001 17:37, you wrote:
> > > Even so, I'm wondering wether this removal is standardad
> > > procedure for hiding it once and for all or not?
>
> Very definitely NOT.  It _may_ work until the filesystem is unmounted,
> because the kernel will keep the file "open" so that the inode is not
> freed, but the next time you try to mount the filesystem it will
> complain about the journal being a bad inode.
>
> > On my system, the journal appears to have a perfectly normal inode number
> > for a root entry (#22), which makes me think that it's just a normal file
> > as far as the core filesystem code is concerned.
>
> Correct.  Normal, except that if you (as root) really work hard to fool
> with it, you can potentially cause problems.  Don't do that.  The problems
> are 99.99% harmless - can't mount as ext3, e2fsck will complain, maybe you
> can't boot your system, if it is the root fs.  If you really work at it,
> maybe you can corrupt your fs, but that would take serious effort plus a
> crash.
>
I just tried this... :)

First corrupted .journal is reported, then "journal deleted, mounting as 
ext2-only" followed by a forced e2fsck.

Thats what I call a well handled error...

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Mike Fedyk]

On Thu, Nov 22, 2001 at 09:48:37AM +0100, Allan Sandfeld wrote:
> On Tuesday 20 November 2001 20:19, Andreas Dilger wrote:
> > On Nov 19, 2001  17:55 -0800, Ryan Cumming wrote:
> > > On November 19, 2001 17:37, you wrote:
> > > > Even so, I'm wondering wether this removal is standardad
> > > > procedure for hiding it once and for all or not?
> >
> > Very definitely NOT.  It _may_ work until the filesystem is unmounted,
> > because the kernel will keep the file "open" so that the inode is not
> > freed, but the next time you try to mount the filesystem it will
> > complain about the journal being a bad inode.
> >
> > > On my system, the journal appears to have a perfectly normal inode number
> > > for a root entry (#22), which makes me think that it's just a normal file
> > > as far as the core filesystem code is concerned.
> >
> > Correct.  Normal, except that if you (as root) really work hard to fool
> > with it, you can potentially cause problems.  Don't do that.  The problems
> > are 99.99% harmless - can't mount as ext3, e2fsck will complain, maybe you
> > can't boot your system, if it is the root fs.  If you really work at it,
> > maybe you can corrupt your fs, but that would take serious effort plus a
> > crash.
> >
> I just tried this... :)
> 
> First corrupted .journal is reported, then "journal deleted, mounting as 
> ext2-only" followed by a forced e2fsck.
> 
> Thats what I call a well handled error...

Did you also have ext2 linked into your kernel (ie, not as a module)?

If you did, then if you kernel didn't have ext2 it probably would've stopped
right there because ext3 won't mount without a journal.

Mike

Re: Kernel 2.4.15-pre6 / EXT3 / ls shows '.journal' on root-fs.

[Stephen C. Tweedie]

Hi,

On Tue, Nov 20, 2001 at 03:20:52AM +0200, Tommi Kyntola wrote:
 
> Minor corrections though, atleast on my 2.4.15-pre6 with tune2fs 1.23
> the .journal is visible even when created as unmounted.

There is absolutely no code in e2fsprogs to create that file on an
unmounted filesystem.  The only mention of the ".journal" name in
e2fsprogs-1.23 is in the code where it opens a regular file with that
name using normal syscalls to access mounted filesystems.  A
".journal" file might be an artifact resulting from doing a recursive
copy from an online-upgraded ext3 filesystem to the new one, though.

> Is the 'rm .journal' the way preferred way to "hide" it?
> Are there any side-effects caused by removal?
> Could someone shed a little more light on this subject.
> What would happen if root were to write to it?

The .journal is an immutable file.  Root will get -EPERM.

> root@behemoth / # tune2fs -O ^has_journal /dev/hda2
> tune2fs 1.23, 15-Aug-2001 for EXT2 FS 0.5b, 95/08/09
> root@behemoth / # mount /tmp
> root@behemoth / # grep /tmp /etc/mtab
> /dev/hda2 /tmp ext2 rw 0 0
> root@behemoth / # umount /tmp
> root@behemoth / # tune2fs -j /dev/hda2
> root@behemoth / # ls -la /tmp | grep journal
> -rw-------    1 root     root      8388608 Oct 23 17:10 .journal

Just clearing the "has_journal" flag from an online-upgraded
filesystem won't be enough to delete the .journal file.  

"dumpe2fs" will tell you what inode the filesystem is actually using
for the journal: I suspect it won't be using that .journal file.

Cheers,
 Stephen