Moving ext3 journal file

Moving ext3 journal file

[RaúlNúñez de Arenas Coronado]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

    Hello all :))

    Is there any problem on moving the /.journal file (even renaming
it) so it doesn't lives on the root? I mean, maintaining its inode
number, of course ;))

    Anyway, ext3 shouldn't (just an idea) show the journal as a
normal file. It may add some load on the kernel, because the inode
number should be compared with that of the journal every time a file
is accessed, but it's just a suggestion ;))

    And, BTW, ext3 works *really* great :))))

    Raúl

Re: Moving ext3 journal file

[Andreas Dilger]

On Nov 23, 2001  13:58 +0100, RaúlNúñez de Arenas Coronado wrote:
>     Is there any problem on moving the /.journal file (even renaming
> it) so it doesn't lives on the root? I mean, maintaining its inode
> number, of course ;))

The name is irrelevant.  The kernel only accesses the journal by inode
number, and only once at boot time.  Rather than "renaming" it and
causing problems, you should just unmount the filesystem, and run
"e2fsck -f /dev/hdX" (with e2fsck 1.25) and it will hide it for you.

>     Anyway, ext3 shouldn't (just an idea) show the journal as a
> normal file. It may add some load on the kernel, because the inode
> number should be compared with that of the journal every time a file
> is accessed, but it's just a suggestion ;))

???? This doesn't make sense.  Having the .journal file places no load
on the system.  OK, when you do "ls -a /" it has to list an extra file,
but that is so little work as to be unnoticable.  Even with a hidden
journal, it is still in an inode with an inode number, it's just not
in a directory anywhere.

As to "comparing the inode number to that of the journal every time a
file is accessed" it appears you just don't understand how file access
works in the kernel.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: Moving ext3 journal file

[David Gómez]

Hi ;),

>     Is there any problem on moving the /.journal file (even renaming
> it) so it doesn't lives on the root? I mean, maintaining its inode
> number, of course ;))
>
>     Anyway, ext3 shouldn't (just an idea) show the journal as a
> normal file. It may add some load on the kernel, because the inode
> number should be compared with that of the journal every time a file
> is accessed, but it's just a suggestion ;))

AFAIK the .journal it's visible only when you convert an ext2 to an ext3
filesystem on a mounted partition, it was a problem with 2.4.10 kernel
version, but i'm not sure if posterior releases also show this behavior.
Anyway you can solve it recreating a new journal (remount it to ext2
before doing this):

'chattr -i /.journal;rm /.journal;tune2fs -j /dev/whatever'



David Gómez

"The question of whether computers can think is just like the question of
 whether submarines can swim." -- Edsger W. Dijkstra

Re: Moving ext3 journal file

[Andreas Dilger]

On Nov 23, 2001  20:02 -0500, David Gómez wrote:
> AFAIK the .journal it's visible only when you convert an ext2 to an ext3
> filesystem on a mounted partition, it was a problem with 2.4.10 kernel
> version, but i'm not sure if posterior releases also show this behavior.
> Anyway you can solve it recreating a new journal (remount it to ext2
> before doing this):
> 
> 'chattr -i /.journal;rm /.journal;tune2fs -j /dev/whatever'

Don't do that.  That is only good if the filesystem thinks that there
is no journal, or it is using a hidden inode for the journal (i.e. if
you run "tune2fs -l /dev/whatever" and it doesn't have "has_journal"
listed in the filesystem features (this is what happened with 2.4.10).
Otherwise, you will delete your real journal, tune2fs will complain,
and then you will need to run e2fsck to clean up after yourself, before
re-creating your journal again.

If you have a filesystem with a .journal file, and you want to "hide"
it, just run e2fsck 1.25 while the filesystem is unmounted, and it
will do it for you.  If you don't want to have a .journal in the first
place, run tune2fs -j while the filesystem is unmounted.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: Moving ext3 journal file

[H. Peter Anvin]

Followup to:  <20011123155901.C1308@lynx.no>
By author:    Andreas Dilger <adilger@turbolabs.com>
In newsgroup: linux.dev.kernel
> 
> Don't do that.  That is only good if the filesystem thinks that there
> is no journal, or it is using a hidden inode for the journal (i.e. if
> you run "tune2fs -l /dev/whatever" and it doesn't have "has_journal"
> listed in the filesystem features (this is what happened with 2.4.10).
> Otherwise, you will delete your real journal, tune2fs will complain,
> and then you will need to run e2fsck to clean up after yourself, before
> re-creating your journal again.
> 
> If you have a filesystem with a .journal file, and you want to "hide"
> it, just run e2fsck 1.25 while the filesystem is unmounted, and it
> will do it for you.  If you don't want to have a .journal in the first
> place, run tune2fs -j while the filesystem is unmounted.
> 

This is all fine and good except for the root partition (I'm pleased
to hear that e2fsck 1.25 will move the journal to the hidden inode for
non-root partitions.)  It would be nice if this was done automagically
by the mounting code instead of by fsck; that way migration would
truly be painless.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: Moving ext3 journal file

[Andreas Dilger]

On Nov 23, 2001  16:07 -0800, H. Peter Anvin wrote:
> By author:    Andreas Dilger <adilger@turbolabs.com>
> > Don't do that.  That is only good if the filesystem thinks that there
> > is no journal, or it is using a hidden inode for the journal (i.e. if
> > you run "tune2fs -l /dev/whatever" and it doesn't have "has_journal"
> > listed in the filesystem features (this is what happened with 2.4.10).
> > Otherwise, you will delete your real journal, tune2fs will complain,
> > and then you will need to run e2fsck to clean up after yourself, before
> > re-creating your journal again.
> > 
> > If you have a filesystem with a .journal file, and you want to "hide"
> > it, just run e2fsck 1.25 while the filesystem is unmounted, and it
> > will do it for you.  If you don't want to have a .journal in the first
> > place, run tune2fs -j while the filesystem is unmounted.
> 
> This is all fine and good except for the root partition (I'm pleased
> to hear that e2fsck 1.25 will move the journal to the hidden inode for
> non-root partitions.)  It would be nice if this was done automagically
> by the mounting code instead of by fsck; that way migration would
> truly be painless.

Hmm, it could be added into the in-kernel ext3 journal recovery code, maybe,
possibly, but it would be ugly I think.  I really don't see it as being a
problem (other than a purely cosmetic one) to have a .journal file in your
root fs.  I lived with these for a couple of years OK (even some of the
early ext3 tools called them "journal.dat" (i.e. not a leading dot, ugh).

In 2.5 it would be easy (and preferrable for other reasons) to have e2fsck
run from the initramfs on the root fs before it is mounted.  That solves
the problem nicely without adding bloat into the kernel.  We could even
remove the in-kernel journal recovery at that point if we thought that the
initramfs was a reliable environment to host _critical_ boot tools. 

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: Moving ext3 journal file

[H. Peter Anvin]

Followup to:  <20011123174120.Q1308@lynx.no>
By author:    Andreas Dilger <adilger@turbolabs.com>
In newsgroup: linux.dev.kernel
> 
> Hmm, it could be added into the in-kernel ext3 journal recovery code, maybe,
> possibly, but it would be ugly I think.  I really don't see it as being a
> problem (other than a purely cosmetic one) to have a .journal file in your
> root fs.  I lived with these for a couple of years OK (even some of the
> early ext3 tools called them "journal.dat" (i.e. not a leading dot, ugh).
> 

It's either a cosmetic problem or a really problematic one, depending
on how closely controlled that particular part of the namespace is.
The .journal file could potentially try to be copied, backed up, you
name it.  Now, since this is now only an issue for the root partition,
this is much less of a problem (the risk for namespace problems in /
are much less.)

> In 2.5 it would be easy (and preferrable for other reasons) to have e2fsck
> run from the initramfs on the root fs before it is mounted.  That solves
> the problem nicely without adding bloat into the kernel.  We could even
> remove the in-kernel journal recovery at that point if we thought that the
> initramfs was a reliable environment to host _critical_ boot tools. 

You can do that now via initrd as well.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: Moving ext3 journal file

[Andreas Dilger]

On Nov 23, 2001  16:56 -0800, H. Peter Anvin wrote:
> By author:    Andreas Dilger <adilger@turbolabs.com>
> > I really don't see it as being a problem (other than a purely cosmetic
> > one) to have a .journal file in your root fs.  I lived with these for a
> > couple of years OK...
> 
> It's either a cosmetic problem or a really problematic one, depending
> on how closely controlled that particular part of the namespace is.
> The .journal file could potentially try to be copied, backed up, you
> name it.

Actually, unless users are actively trying to shoot themselves in the
foot, none of this really matters.  However, now that ext3 is in the
mainline, the number of users playing with guns has increased a large
amount, it seems, by the number of such reports on ext3-users.

Because .journal is created as immutable, even if it was backed up and
tried to be restored, it would be impossible to write to.  For the
"accursed" ext2 dump, it recognizes the "nodump" flag, but also knows
enough not to back up the journal file.  Sadly, neither cpio or tar
know about ext2 attributes.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: Moving ext3 journal file

[H. Peter Anvin]

Andreas Dilger wrote:

> 
> Actually, unless users are actively trying to shoot themselves in the
> foot, none of this really matters.  However, now that ext3 is in the
> mainline, the number of users playing with guns has increased a large
> amount, it seems, by the number of such reports on ext3-users.
> 
> Because .journal is created as immutable, even if it was backed up and
> tried to be restored, it would be impossible to write to.  For the
> "accursed" ext2 dump, it recognizes the "nodump" flag, but also knows
> enough not to back up the journal file.  Sadly, neither cpio or tar
> know about ext2 attributes.
> 


Nor scp, nor rsync, nor find, nor...

	-hpa

Re: Moving ext3 journal file

[Michael Zimmermann]

Excuse me, if I jump in with almost zero linux-kernel-knowledge,
and take my words as a third-party comment.

May be, I've not seen enough in 35 years of system programming
(including designing and writing journal-systems my own),
but I've never seen a journal beeing part of the data-space to be
journalled. It is simply an ugly thing in the file space. It either belongs
into /proc/fs/ext3 (or the like) or is not to be shown at all. Except
there was a valid neccessity to have it in the normal file space.


Greetings
Michael

Re: Moving ext3 journal file

[Mike Fedyk]

On Sat, Nov 24, 2001 at 12:37:54PM +0100, Michael Zimmermann wrote:
> Excuse me, if I jump in with almost zero linux-kernel-knowledge,
> and take my words as a third-party comment.
> 
> May be, I've not seen enough in 35 years of system programming
> (including designing and writing journal-systems my own),
> but I've never seen a journal beeing part of the data-space to be
> journalled. It is simply an ugly thing in the file space. It either belongs
> into /proc/fs/ext3 (or the like) or is not to be shown at all. Except
> there was a valid neccessity to have it in the normal file space.
> 

You do not understand the reasons behind the thread.

.journal files are only created when an ext2 FS is converted to ext3 *while*
it is mounted.  In this case tune2fs just creates a file (.journal) and
modifies a couple fields in the super block.

The newest e2fsck (1.25) will hide the file for you, just like it would be
if the conversion was done on an unmounted FS.

MF

Re: Moving ext3 journal file

[Mike Castle]

On Sun, Nov 25, 2001 at 02:33:14AM -0800, Mike Fedyk wrote:
> The newest e2fsck (1.25) will hide the file for you, just like it would be
> if the conversion was done on an unmounted FS.


Actually, to re-iterate a recent point:  No, it won't.

The next version of e2fsck should.  But 1.25 does NOT do that.

mrc
-- 
     Mike Castle      dalgoda@ix.netcom.com      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

Re: Moving ext3 journal file

[Mike Fedyk]

On Sun, Nov 25, 2001 at 03:14:53AM -0800, Mike Castle wrote:
> On Sun, Nov 25, 2001 at 02:33:14AM -0800, Mike Fedyk wrote:
> > The newest e2fsck (1.25) will hide the file for you, just like it would be
> > if the conversion was done on an unmounted FS.
> 
> 
> Actually, to re-iterate a recent point:  No, it won't.
> 
> The next version of e2fsck should.  But 1.25 does NOT do that.
> 

Ok...

Strange that aDilger has been one to say that 1.25 does hide the file...

Since I don't know myself, I'll let others say for sure.

MF

Re: Moving ext3 journal file

[Mike Fedyk]

On Sun, Nov 25, 2001 at 02:55:37PM -0800, Mike Fedyk wrote:
> On Sun, Nov 25, 2001 at 03:14:53AM -0800, Mike Castle wrote:
> > On Sun, Nov 25, 2001 at 02:33:14AM -0800, Mike Fedyk wrote:
> > > The newest e2fsck (1.25) will hide the file for you, just like it would be
> > > if the conversion was done on an unmounted FS.
> > 
> > 
> > Actually, to re-iterate a recent point:  No, it won't.
> > 
> > The next version of e2fsck should.  But 1.25 does NOT do that.
> > 
> 
> Ok...
> 
> Strange that aDilger has been one to say that 1.25 does hide the file...
> 
> Since I don't know myself, I'll let others say for sure.
> 

Yep, just finished reading the thread.

You may disregard...

MF

Re: Moving ext3 journal file

[Jamie Lokier]

H. Peter Anvin wrote:
> >  Sadly, neither cpio or tar
> > know about ext2 attributes.

Indeed, I once accidentally copied a .journal file from one filesystem
to another.  Not helpful.

-- Jamie

Re: Moving ext3 journal file

[Thorsten Glaser]

Dixitur de H. Peter Anvin respondebo ad:
> Andreas Dilger wrote:
(...)
> > Because .journal is created as immutable, even if it was backed up and
> > tried to be restored, it would be impossible to write to.  For the
> > "accursed" ext2 dump, it recognizes the "nodump" flag, but also knows
> > enough not to back up the journal file.  Sadly, neither cpio or tar
> > know about ext2 attributes.
> Nor scp, nor rsync, nor find, nor...

Neither they do about UCB FFS attributes... they even have separate
user/system immutable flags :)

What about pax(1)? I guess it doesn't either, because cpio is just pax.

-mirabilos
-- 
| This message body is covered by Germanic and International | OpenBSD30
| Copyright law. Modification of any kind and redistribution | centericq
| via AOL or the Microsoft network are strictly prohibited!! | UIN seems
| Scientific-style quotation permitted if due credits given. | 132315236

Re: Moving ext3 journal file

[Marcel J.E. Mol]

On Fri, Nov 23, 2001 at 04:07:44PM -0800, H. Peter Anvin wrote:
> Followup to:  <20011123155901.C1308@lynx.no>
> By author:    Andreas Dilger <adilger@turbolabs.com>
> In newsgroup: linux.dev.kernel
> > 
> > Don't do that.  That is only good if the filesystem thinks that there
> > is no journal, or it is using a hidden inode for the journal (i.e. if
> > you run "tune2fs -l /dev/whatever" and it doesn't have "has_journal"
> > listed in the filesystem features (this is what happened with 2.4.10).
> > Otherwise, you will delete your real journal, tune2fs will complain,
> > and then you will need to run e2fsck to clean up after yourself, before
> > re-creating your journal again.
> > 
> > If you have a filesystem with a .journal file, and you want to "hide"
> > it, just run e2fsck 1.25 while the filesystem is unmounted, and it
> > will do it for you.  If you don't want to have a .journal in the first
> > place, run tune2fs -j while the filesystem is unmounted.
> > 
> 
> This is all fine and good except for the root partition (I'm pleased
> to hear that e2fsck 1.25 will move the journal to the hidden inode for
> non-root partitions.)  It would be nice if this was done automagically
> by the mounting code instead of by fsck; that way migration would
> truly be painless.

Hm, the e2fsck check does not work for me...
The .journal file still exists after

   # umount /dev/hda11
   # e2fsck -f /dev/hda11
   # mount /dev/hda11
   # rpm -q e2fsprogs
   e2fsprogs-1.25-1

(on redhat rawhide)


-Marcel
-- 
     ======--------         Marcel J.E. Mol                MESA Consulting B.V.
    =======---------        ph. +31-(0)6-54724868          P.O. Box 112
    =======---------        marcel@mesa.nl                 2630 AC  Nootdorp
__==== www.mesa.nl ---____U_n_i_x______I_n_t_e_r_n_e_t____ The Netherlands ____
 They couldn't think of a number,           Linux user 1148  --  counter.li.org
    so they gave me a name!  -- Rupert Hine  --  www.ruperthine.com

Re: Moving ext3 journal file

[Mike Castle]

On Sat, Nov 24, 2001 at 11:54:11AM +0100, Marcel J.E. Mol wrote:
> On Fri, Nov 23, 2001 at 04:07:44PM -0800, H. Peter Anvin wrote:
> > This is all fine and good except for the root partition (I'm pleased
> > to hear that e2fsck 1.25 will move the journal to the hidden inode for
> > non-root partitions.)  It would be nice if this was done automagically
> > by the mounting code instead of by fsck; that way migration would
> > truly be painless.
> 
> Hm, the e2fsck check does not work for me...
> The .journal file still exists after
> 
>    # umount /dev/hda11
>    # e2fsck -f /dev/hda11
>    # mount /dev/hda11
>    # rpm -q e2fsprogs
>    e2fsprogs-1.25-1

I was just going to comment on this as well.  I cannot get this to work as
advertised on a homegrown system as well.

Even tune2fs -l | grep inode and ls -i .journal show it's still using the
same inode.

Actually, a quick look through the code, I can't see where e2fsck would
take measures to hide .journal.  At least not in 1.25.

mrc
-- 
     Mike Castle      dalgoda@ix.netcom.com      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

Re: Moving ext3 journal file

[Andreas Dilger]

On Nov 24, 2001  13:19 -0800, Mike Castle wrote:
> On Sat, Nov 24, 2001 at 11:54:11AM +0100, Marcel J.E. Mol wrote:
> > Hm, the e2fsck check does not work for me...
> > The .journal file still exists after
> > 
> >    # umount /dev/hda11
> >    # e2fsck -f /dev/hda11
> >    # mount /dev/hda11
> >    # rpm -q e2fsprogs
> >    e2fsprogs-1.25-1
> 
> I was just going to comment on this as well.  I cannot get this to work as
> advertised on a homegrown system as well.
> 
> Even tune2fs -l | grep inode and ls -i .journal show it's still using the
> same inode.
> 
> Actually, a quick look through the code, I can't see where e2fsck would
> take measures to hide .journal.  At least not in 1.25.

OK, my bad.  This is in my version of e2fsprogs (from Ted's BitKeeper
repository), but is not in an official release of e2fsprogs yet.  It
will be in 1.26, but for now I don't even think there is a WIP release
which has it.  Sorry for the confusion.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: Moving ext3 journal file

[Mike Castle]

On Sat, Nov 24, 2001 at 03:49:55PM -0700, Andreas Dilger wrote:
> OK, my bad.  This is in my version of e2fsprogs (from Ted's BitKeeper
> repository), but is not in an official release of e2fsprogs yet.  It
> will be in 1.26, but for now I don't even think there is a WIP release
> which has it.  Sorry for the confusion.

Whew.

Better to know a feature is in the works than to think it's not working.
:->

Cheers,
mrc
-- 
     Mike Castle      dalgoda@ix.netcom.com      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc