[RFC] ext3 documentation (lack of)

[RFC] ext3 documentation (lack of)

[Diego Calleja]

Lots of people think that ext3 is very slow. While I'm not claiming that ext3
is the fatest fs in the world, I told some people to look at 
Documentation/filesystem/ext3.txt and try to tweak it before doing some
benchmarks. To my surprise, several ext3 mount options were not documented
(not even in the source) except in some sites spread across the internet, so
it's not a surprise that lots of people ignores some mount options when doing
benchmarks, like the commit interval.

This documents commit (or it tries, sorry for my english), groups the
journal-related options in the same place of the document and adds other
mount options without documenting them (like the ones related to acl, xattr,
resizing, reservations, barriers). Hopefully people will explain those better
than me.

--- stable/Documentation/filesystems/ext3.txt-documentsync	2004-08-18 01:55:48.000000000 +0200
+++ stable/Documentation/filesystems/ext3.txt	2004-08-18 02:54:00.000000000 +0200
@@ -22,6 +22,52 @@
 			the inode which will represent the ext3 file
 			system's journal file.
 
+noload			Don't load the journal on mounting.
+
+data=journal		All data are committed into the journal prior 
+			to being written into the main file system.
+
+data=ordered	(*)	All data are forced directly out to the main file
+			system prior to its metadata being committed to
+			the journal.
+
+data=writeback		Data ordering is not preserved, data may be 
+			written into the main file system after its
+			metadata has been committed to the journal.
+
+commit=nrsec	(*)	Ext3 can be told to write all its data and metadata
+			every 'nrsec' seconds. The default value is 5 seconds.
+			This means that if you lose your power, you will lose,
+			as much, the latest 5 seconds of work. This default
+			value (or any low value) will hurt performance, but
+			it's good for data-safety. Setting it to 0 disables it.
+			Disabling it or setting it to very large values will
+			improve performance, 
+
+barrier=??	(*)??	This enables/disables the "journal barrier"
+
+orlov		(*)	This enables the new Orlov block allocator. It's enabled
+			by default. 
+
+oldalloc		This disables the Orlov block allocator and enables the
+			old block allocator. Orlov should have better performance,
+			we'd like to get some feedback if it's the contrary for
+			you.
+
+user_xattr=
+
+nouser_xattr=
+
+acl=
+
+noacl=
+
+reservation=
+
+noreservation=
+
+resize=
+
 bsddf 		(*)	Make 'df' act like BSD.
 minixdf			Make 'df' act like Minix.
 
@@ -30,8 +76,6 @@
 
 debug			Extra debugging information is sent to syslog.
 
-noload			Don't load the journal on mounting.
-
 errors=remount-ro(*)	Remount the filesystem read-only on an error.
 errors=continue		Keep going on a filesystem error.
 errors=panic		Panic and halt the machine if an error occurs.
@@ -48,17 +92,6 @@
 
 sb=n			Use alternate superblock at this location.
 
-data=journal		All data are committed into the journal prior 
-			to being written into the main file system.
-		
-data=ordered	(*)	All data are forced directly out to the main file 
-			system prior to its metadata being committed to 
-			the journal.
-		
-data=writeback  	Data ordering is not preserved, data may be 
-			written into the main file system after its
-			metadata has been committed to the journal.
-
 quota			Quota options are currently silently ignored.
 noquota			(see fs/ext3/super.c, line 594)
 grpquota

Re: [RFC] ext3 documentation (lack of)

[Ryan Cumming]

[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]

On Tuesday 17 August 2004 17:59, you wrote:
> +commit=nrsec	(*)	Ext3 can be told to write all its data and metadata
> +			every 'nrsec' seconds. The default value is 5 seconds.
> +			This means that if you lose your power, you will lose,
> +			as much, the latest 5 seconds of work. This default
> +			value (or any low value) will hurt performance, but
> +			it's good for data-safety. Setting it to 0 disables it.
> +			Disabling it or setting it to very large values will
> +			improve performance,

Setting commit to zero doesn't seem to disable it, judging from my local 
2.6.8.1-mm1 source.

super.c has:
case Opt_commit:
   if (match_int(&args[0], &option))
      return 0;
   if (option < 0)
      return 0;
   if (option == 0)
      option = JBD_DEFAULT_MAX_COMMIT_AGE;
   sbi->s_commit_interval = HZ * option;
   break;

Where JBD_DEFAULT_COMMIT_AGE is defined to 5 in include/linux/jbd.h. So it 
seems that setting it to zero will just set it to the default commit interval 
of 5 seconds. Am I missing something?

-Ryan

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] ext3 documentation (lack of)

[Diego Calleja]

El Tue, 17 Aug 2004 23:01:06 -0700 Ryan Cumming <ryan@spitfire.gotdns.org> escribió:

> Setting commit to zero doesn't seem to disable it, judging from my local 
> 2.6.8.1-mm1 source.


Ugh, sure, I just assumed it'd disable it. Anyway, here's a updated version.

(It'd be nice if people could provide a description for the rest so we can
recollect and submit them)

Diego Calleja

--- stable/Documentation/filesystems/ext3.txt-documentsync	2004-08-18 01:55:48.000000000 +0200
+++ stable/Documentation/filesystems/ext3.txt	2004-08-18 13:37:37.000000000 +0200
@@ -22,6 +22,54 @@
 			the inode which will represent the ext3 file
 			system's journal file.
 
+noload			Don't load the journal on mounting.
+
+data=journal		All data are committed into the journal prior 
+			to being written into the main file system.
+
+data=ordered	(*)	All data are forced directly out to the main file
+			system prior to its metadata being committed to
+			the journal.
+
+data=writeback		Data ordering is not preserved, data may be 
+			written into the main file system after its
+			metadata has been committed to the journal.
+
+commit=nrsec	(*)	Ext3 can be told to sync all its data and metadata
+			every 'nrsec' seconds. The default value is 5 seconds.
+			This means that if you lose your power, you will lose,
+			as much, the latest 5 seconds of work (your filesystem
+			will not be damaged though, thanks to journaling). This
+			default value (or any low value) will hurt performance,
+			but it's good for data-safety. Setting it to 0 will
+			have the same effect than leaving the default 5 sec.
+			Setting it to very large values will improve
+			performance.
+
+barrier=??	(*)??	This enables/disables the "journal barrier"
+
+orlov		(*)	This enables the new Orlov block allocator. It's enabled
+			by default. 
+
+oldalloc		This disables the Orlov block allocator and enables the
+			old block allocator. Orlov should have better performance,
+			we'd like to get some feedback if it's the contrary for
+			you.
+
+user_xattr=
+
+nouser_xattr=
+
+acl=
+
+noacl=
+
+reservation=
+
+noreservation=
+
+resize=
+
 bsddf 		(*)	Make 'df' act like BSD.
 minixdf			Make 'df' act like Minix.
 
@@ -30,8 +78,6 @@
 
 debug			Extra debugging information is sent to syslog.
 
-noload			Don't load the journal on mounting.
-
 errors=remount-ro(*)	Remount the filesystem read-only on an error.
 errors=continue		Keep going on a filesystem error.
 errors=panic		Panic and halt the machine if an error occurs.
@@ -48,17 +94,6 @@
 
 sb=n			Use alternate superblock at this location.
 
-data=journal		All data are committed into the journal prior 
-			to being written into the main file system.
-		
-data=ordered	(*)	All data are forced directly out to the main file 
-			system prior to its metadata being committed to 
-			the journal.
-		
-data=writeback  	Data ordering is not preserved, data may be 
-			written into the main file system after its
-			metadata has been committed to the journal.
-
 quota			Quota options are currently silently ignored.
 noquota			(see fs/ext3/super.c, line 594)
 grpquota

Re: [RFC] ext3 documentation (lack of)

[Janusz Dziemidowicz]

On Wed, 18 Aug 2004, Diego Calleja wrote:

> (It'd be nice if people could provide a description for the rest so we can
> recollect and submit them)

AFAIK
user_xattr - turns on POSIX Extended attributes
nouser_xattr - obvious
acl - turns on ACL (Access Control Lists)
noacl - obvious

However these features are only present in 2.6.x, they are explained in 
Help during kernel configuration. I'm not sure if acl requires user_xattr 
to be turned on. I remember that I spent some time looking for a way to 
enable these on ext2/ext3 :)

--
Janusz Dziemidowicz
rraptorr@kursor.pl

Re: [RFC] ext3 documentation (lack of)

[Helge Hafting]

Janusz Dziemidowicz wrote:

> AFAIK
> user_xattr - turns on POSIX Extended attributes
> nouser_xattr - obvious
> acl - turns on ACL (Access Control Lists)
> noacl - obvious
>
> However these features are only present in 2.6.x, they are explained 
> in Help during kernel configuration. I'm not sure if acl requires 
> user_xattr to be turned on. I remember that I spent some time looking 
> for a way to enable these on ext2/ext3 :)

ACL's does not require use of the user_xattr mount option - I have tested
that acl's work on ext3 with only the "acl" option.  I guess user_xattr is
turned on automatically as needed - you can use the option if you want
posix extended attributes without acls.

Helge Hafting

Re: [RFC] ext3 documentation (lack of)

[Diego Calleja]

El Wed, 18 Aug 2004 14:24:43 +0200 (CEST) Janusz Dziemidowicz <rraptorr@kursor.pl> escribió:

> AFAIK
> user_xattr - turns on POSIX Extended attributes
> nouser_xattr - obvious
> acl - turns on ACL (Access Control Lists)
> noacl - obvious

Thanks. I've recollected those, explained it a bit (except the barrier bits,
I don't really mean what barriers are), and updated ext2 documentation with
some of them, and deleted a small comentary about ext3 not being available.

unchanged:
--- stable/Documentation/filesystems/ext2.txt-document	2004-08-18 18:17:28.000000000 +0200
+++ stable/Documentation/filesystems/ext2.txt	2004-08-18 18:15:45.000000000 +0200
@@ -17,6 +17,33 @@
 bsddf			(*)	Makes `df' act like BSD.
 minixdf				Makes `df' act like Minix.
 
+barrier=1			This enables/disables barriers. barrier=0 disables it,
+				barrier=1 enables it.
+
+orlov			(*)	This enables the new Orlov block allocator. It's
+				enabled by default.
+
+oldalloc			This disables the Orlov block allocator and
+				enables the old block allocator. Orlov should
+				have better performance, we'd like to get some
+				feedback  if it's the contrary for you.
+
+user_xattr		(*)	Enables POSIX Extended Attributes. It's enabled by
+				default, however you need to confifure its support
+				(CONFIG_EXT2_FS_XATTR). This is neccesary if you want
+				to use POSIX Acces Control Lists support. You can visit
+				http://acl.bestbits.at to know more about POSIX Extended
+				attributes.
+
+nouser_xattr			Disables POSIX Extended Attributes.
+
+acl			(*)	Enables POSIX Access Control Lists support. This is
+				enabled by default, however you need to configure
+				its support (CONFIG_EXT2_FS_POSIX_ACL). If you want
+				to know more about ACLs visit http://acl.bestbits.at
+
+noacl				This option disables POSIX Access Control List support.
+
 check=none, nocheck	(*)	Don't do extra checking of bitmaps on mount
 				(check=normal and check=strict options removed)
 
@@ -336,9 +363,8 @@
 the time of the crash, then there is no guarantee of consistency for
 the blocks in that transaction so they are discarded (which means any
 filesystem changes they represent are also lost).
-
-The ext3 code is currently (Apr 2001) available for 2.2 kernels only,
-and not yet available for 2.4 kernels.
+Check Documentation/filesystems/ext3.txt if you want to read more about
+ext3 and journaling.
 
 References
 ==========
@@ -349,8 +375,6 @@
 Journaling (ext3)	ftp://ftp.uk.linux.org/pub/linux/sct/fs/jfs/
 Hashed Directories	http://kernelnewbies.org/~phillips/htree/
 Filesystem Resizing	http://ext2resize.sourceforge.net/
-Extended Attributes &
-Access Control Lists	http://acl.bestbits.at/
 Compression (*)		http://www.netspace.net.au/~reiter/e2compr/
 
 Implementations for:
only in patch2:
unchanged:
--- stable/Documentation/filesystems/ext3.txt-document	2004-08-18 01:55:48.000000000 +0200
+++ stable/Documentation/filesystems/ext3.txt	2004-08-18 18:15:15.000000000 +0200
@@ -22,6 +22,63 @@
 			the inode which will represent the ext3 file
 			system's journal file.
 
+noload			Don't load the journal on mounting.
+
+data=journal		All data are committed into the journal prior 
+			to being written into the main file system.
+
+data=ordered	(*)	All data are forced directly out to the main file
+			system prior to its metadata being committed to
+			the journal.
+
+data=writeback		Data ordering is not preserved, data may be 
+			written into the main file system after its
+			metadata has been committed to the journal.
+
+commit=nrsec	(*)	Ext3 can be told to sync all its data and metadata
+			every 'nrsec' seconds. The default value is 5 seconds.
+			This means that if you lose your power, you will lose,
+			as much, the latest 5 seconds of work (your filesystem
+			will not be damaged though, thanks to journaling). This
+			default value (or any low value) will hurt performance,
+			but it's good for data-safety. Setting it to 0 will
+			have the same effect than leaving the default 5 sec.
+			Setting it to very large values will improve
+			performance.
+
+barrier=1		This enables/disables barriers. barrier=0 disables it, 
+			barrier=1 enables it.
+
+orlov		(*)	This enables the new Orlov block allocator. It's enabled
+			by default. 
+
+oldalloc		This disables the Orlov block allocator and enables the
+			old block allocator. Orlov should have better performance,
+			we'd like to get some feedback if it's the contrary for
+			you.
+
+user_xattr	(*)	Enables POSIX Extended Attributes. It's enabled by
+			default, however you need to confifure its support
+			(CONFIG_EXT3_FS_XATTR). This is neccesary if you want
+			to use POSIX Acces Control Lists support. You can visit
+			http://acl.bestbits.at to know more about POSIX Extended
+			attributes.
+
+nouser_xattr		Disables POSIX Extended Attributes. 
+
+acl		(*)	Enables POSIX Access Control Lists support. This is
+			enabled by default, however you need to configure
+			its support (CONFIG_EXT3_FS_POSIX_ACL). If you want
+			to know more about ACLs visit http://acl.bestbits.at
+
+noacl			This option disables POSIX Access Control List support.
+
+reservation
+
+noreservation
+
+resize=
+
 bsddf 		(*)	Make 'df' act like BSD.
 minixdf			Make 'df' act like Minix.
 
@@ -30,8 +87,6 @@
 
 debug			Extra debugging information is sent to syslog.
 
-noload			Don't load the journal on mounting.
-
 errors=remount-ro(*)	Remount the filesystem read-only on an error.
 errors=continue		Keep going on a filesystem error.
 errors=panic		Panic and halt the machine if an error occurs.
@@ -48,17 +103,6 @@
 
 sb=n			Use alternate superblock at this location.
 
-data=journal		All data are committed into the journal prior 
-			to being written into the main file system.
-		
-data=ordered	(*)	All data are forced directly out to the main file 
-			system prior to its metadata being committed to 
-			the journal.
-		
-data=writeback  	Data ordering is not preserved, data may be 
-			written into the main file system after its
-			metadata has been committed to the journal.
-
 quota			Quota options are currently silently ignored.
 noquota			(see fs/ext3/super.c, line 594)
 grpquota
@@ -114,7 +158,7 @@
 -------------
 
 Ext2 partitions can be easily convert to ext3, with `tune2fs -j <dev>`.
- Ext3 is fully compatible with Ext2.  Ext3 partitions can easily be
+Ext3 is fully compatible with Ext2.  Ext3 partitions can easily be
 mounted as Ext2.
 
 External Tools

Re: [RFC] ext3 documentation (lack of)

[Andreas Dilger]

[-- Attachment #1: Type: text/plain, Size: 766 bytes --]

On Aug 18, 2004  14:59 +0200, Helge Hafting wrote:
> ACL's does not require use of the user_xattr mount option - I have tested
> that acl's work on ext3 with only the "acl" option.  I guess user_xattr is
> turned on automatically as needed - you can use the option if you want
> posix extended attributes without acls.

Actually, user_xattr is entirely unrelated to acls.  The user_xattr option
is used to enable arbitrary EAs that can be stored by the owner of the file
with setfattr and read with getfattr (along with the syscalls for same).
The acl support uses "in kernel" EAs that the user cannot modify.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://members.shaw.ca/adilger/             http://members.shaw.ca/golinux/


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] ext3 documentation (lack of)

[Janusz Dziemidowicz]

On Wed, 18 Aug 2004, Diego Calleja wrote:

> Thanks. I've recollected those, explained it a bit (except the barrier bits,
> I don't really mean what barriers are), and updated ext2 documentation with
> some of them, and deleted a small comentary about ext3 not being available.

> +user_xattr		(*)	Enables POSIX Extended Attributes. It's enabled by
> +				default, however you need to confifure its support
> +				(CONFIG_EXT2_FS_XATTR). This is neccesary if you want
> +				to use POSIX Acces Control Lists support. You can visit
> +				http://acl.bestbits.at to know more about POSIX Extended
> +				attributes.
> +
> +nouser_xattr			Disables POSIX Extended Attributes.
> +
> +acl			(*)	Enables POSIX Access Control Lists support. This is
> +				enabled by default, however you need to configure
> +				its support (CONFIG_EXT2_FS_POSIX_ACL). If you want
> +				to know more about ACLs visit http://acl.bestbits.at
> +
> +noacl				This option disables POSIX Access Control List support.

Hmmm, after a quick look on source code one more clarification. Since 
2.4.20 one can set default mount options in ext2/ext3 superblock using 
tune2fs. If user_xattr or acl is there, then they are enabled by default 
while mounting such filesystem. However default mount options field in 
superblock is by default empty, so user_xattr and acl are not enabled by 
default for most users. Please correct me if I'm wrong.

Another thing is that a quick grep for reservation and resize options 
shows nothing in latest kernel. Are you sure that these options exist? For 
sure reiserfs has resize option.

--
Janusz Dziemidowicz
rraptorr@kursor.pl