Making it easier for end users to use ext4

Making it easier for end users to use ext4

[Theodore Ts'o]

So I got a comment on my recent blog posting about using ext4 that
Googling ext4 results in a lot of ancient status reports and not a lot
of good information about how to get started with ext4.  The comment was
fair, so I took a few minutes to update this web page:

      http://ext4.wiki.kernel.org/index.php/Ext4_Howto

with some getting started information.

Eric, when you have a chance, could you take a quick peek at the Fedora
Core section of that page and update it appropriately.  It would
probably be a good idea to include the yum repository with the latest
1.41 e2fsprogs RPM's, etc.  One other thought is if we start encouraging
people to use ext4 on FC9, whether it might be a good idea to take all
of the patches to ext4 since 2.6.25, add them to the patch queue, and
then back port the resulting stack of patches to 2.6.25 so that FC9
users who are interested in testing ext4 would get the benefit of the
latest ext4 code, and also so any bug reports we get would also be
against the latest code.  Alternatively, the repository could just
simply include 2.6.26 plus the latest ext4 patch set, whichever would be
easier.

If other ext4 developers have a chance, it would be a good idea to go
through the http://ext4.wiki.kernel.org site, and do some grooming of
the site.  Some of the pages like the FAQ, which were originally
targetted primarily at ext4 developers, may need to have some of the
more developer-centric content moved to other pages now that we will
start having more users coming to look at that site.  And of course,
there is some very badly out-of-date information on that site as well!

      	      	   	 	     		     - Ted

Re: Making it easier for end users to use ext4

[Eric Sandeen]

Theodore Ts'o wrote:
> So I got a comment on my recent blog posting about using ext4 that
> Googling ext4 results in a lot of ancient status reports and not a lot
> of good information about how to get started with ext4.  The comment was
> fair, so I took a few minutes to update this web page:
> 
>       http://ext4.wiki.kernel.org/index.php/Ext4_Howto
> 
> with some getting started information.
> 
> Eric, when you have a chance, could you take a quick peek at the Fedora
> Core section of that page and update it appropriately.  

Sure, first I'll remove "core" since that terminology doesn't exist
anymore ;)

> It would
> probably be a good idea to include the yum repository with the latest
> 1.41 e2fsprogs RPM's, etc.  One other thought is if we start encouraging
> people to use ext4 on FC9, whether it might be a good idea to take all
> of the patches to ext4 since 2.6.25, add them to the patch queue, and
> then back port the resulting stack of patches to 2.6.25 so that FC9
> users who are interested in testing ext4 would get the benefit of the
> latest ext4 code, and also so any bug reports we get would also be
> against the latest code.  

well, left to its own devices, F9 will see 2.6.26 at least as a testing
kernel pretty soon I think.  Waiting for that seems reasonable vs. the
above patch gyrations....

> Alternatively, the repository could just
> simply include 2.6.26 plus the latest ext4 patch set, whichever would be
> easier.

I think very soon just saying "run the latest kernel + e2fsprogs from F9
updates-testing" will be a good start.  It won't have delalloc yet,
though.  Do we want to try to make that more widely available to people
who can't/don't build their own kernels?  I'm slightly hesitant I guess
both for the extra work of maintaining the repo & complete kernel
builds, as well as providing too much rope to people who may not know
just how much rope they've got ...

For pre-built fedora stuff I'd rather those users just follow what gets
packaged from upstream unless someone really, really thinks we should do
otherwise.

-Eric

> If other ext4 developers have a chance, it would be a good idea to go
> through the http://ext4.wiki.kernel.org site, and do some grooming of
> the site.  Some of the pages like the FAQ, which were originally
> targetted primarily at ext4 developers, may need to have some of the
> more developer-centric content moved to other pages now that we will
> start having more users coming to look at that site.  And of course,
> there is some very badly out-of-date information on that site as well!
> 
>       	      	   	 	     		     - Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Making it easier for end users to use ext4

[Theodore Tso]

On Mon, Jul 14, 2008 at 02:35:07PM -0500, Eric Sandeen wrote:
> > Eric, when you have a chance, could you take a quick peek at the Fedora
> > Core section of that page and update it appropriately.  
> 
> Sure, first I'll remove "core" since that terminology doesn't exist
> anymore ;)

Is "FC" acronym as in "FC9" going away as well?  :-)

> I think very soon just saying "run the latest kernel + e2fsprogs from F9
> updates-testing" will be a good start.  It won't have delalloc yet,
> though.  Do we want to try to make that more widely available to people
> who can't/don't build their own kernels?  I'm slightly hesitant I guess
> both for the extra work of maintaining the repo & complete kernel
> builds, as well as providing too much rope to people who may not know
> just how much rope they've got ...

Well, there are two reasons why I was thinking it might be nice to
synch up the ext4 code base.  One was the "more testers good"
argument.  The other was is that while *most* of the various races and
bug fixes that have gone in since 2.6.26 were delalloc related, some
are real bugs that might bite 2.6.25 users if they start using the
ext4 code in a more demanding workload.  That would result in perhaps
preventable data loss, plus which we could end up spending time
debugging a problem which had already been solved.

If Fedora is going to be putting out another test kernel shortly, any
chance we can get the 2.6.26-ext4-1 or 2.6.26-ext4-2 (with mingming's
latest patch, to be released shortly) included in that test kernel?
Again, the goal is to make it as easy as possible for people to test
the latest version of ext4, so we can get those bugs fixed.

Regards,

						- Ted

Re: Making it easier for end users to use ext4

[Eric Sandeen]

Theodore Tso wrote:
> On Mon, Jul 14, 2008 at 02:35:07PM -0500, Eric Sandeen wrote:
>>> Eric, when you have a chance, could you take a quick peek at the Fedora
>>> Core section of that page and update it appropriately.  
>> Sure, first I'll remove "core" since that terminology doesn't exist
>> anymore ;)
> 
> Is "FC" acronym as in "FC9" going away as well?  :-)

the "fc9" you see in rpms now stands for "Fedora Collection" - becase,
well, rpm got confused otherwise ;)

I probably forget this from time to time as well ;)

> 
>> I think very soon just saying "run the latest kernel + e2fsprogs from F9
>> updates-testing" will be a good start.  It won't have delalloc yet,
>> though.  Do we want to try to make that more widely available to people
>> who can't/don't build their own kernels?  I'm slightly hesitant I guess
>> both for the extra work of maintaining the repo & complete kernel
>> builds, as well as providing too much rope to people who may not know
>> just how much rope they've got ...
> 
> Well, there are two reasons why I was thinking it might be nice to
> synch up the ext4 code base.  One was the "more testers good"
> argument.  The other was is that while *most* of the various races and
> bug fixes that have gone in since 2.6.26 were delalloc related, some
> are real bugs that might bite 2.6.25 users if they start using the
> ext4 code in a more demanding workload.  That would result in perhaps
> preventable data loss, plus which we could end up spending time
> debugging a problem which had already been solved.

Sounds reasonable.  Should said fixes also go into the 2.6.2[56].x
stable trees for the same reasons, do you think?  And, well, if they
did, Fedora would also get them rather quickly.  :)

> If Fedora is going to be putting out another test kernel shortly, any
> chance we can get the 2.6.26-ext4-1 or 2.6.26-ext4-2 (with mingming's
> latest patch, to be released shortly) included in that test kernel?
> Again, the goal is to make it as easy as possible for people to test
> the latest version of ext4, so we can get those bugs fixed.

I've gone back and forth about how bleeding-edge to make the Fedora ext4
code, especially for F9.  Putting it all into rawhide would be fine,
that's what it's for.  But for F9 I'd like to stay on that fine line
between "fix the bugs & add the features quickly" and "don't push
unstable or barely-stable code into the release too soon and cause new
problems."

I don't mind pushing the stable patches into F9 on the early side, and
for rawhide we can push as much as is desired.

Thanks,
-Eric

> Regards,
> 
> 						- Ted

Re: Making it easier for end users to use ext4

[Goswin von Brederlow]

"Theodore Ts'o" <tytso@mit.edu> writes:

> So I got a comment on my recent blog posting about using ext4 that
> Googling ext4 results in a lot of ancient status reports and not a lot
> of good information about how to get started with ext4.  The comment was
> fair, so I took a few minutes to update this web page:
>
>       http://ext4.wiki.kernel.org/index.php/Ext4_Howto
>
> with some getting started information.

Many thanks. Going through various pages I found the information to be
seriously out of date ranging from one to two years.

Some thoughts about missing bits from a first read through:

Getting Ext4 code:
- add ext4 and e2fsprogs git repositories
- Add Debian section

64 bit block support:
- limits without it
- state of it
- where to get patches

MfG
        Goswin

[PATCH, RFC] Add more intelligent handling of the mke2fs.conf installation process

[Theodore Tso]

On Wed, Jul 16, 2008 at 10:02:00AM -0700, Mingming Cao wrote:
> I looked at misc/Makefile, it seems if the system already
> have /etc/mke2fs.conf, it will not get updated. This is reasonable for
> ext3.   But not good for ext4.

Good point.  The following patch won't do anything for people building
RPM or dpkg packages (which have their own automatic configuration
file handling); this will provide a poor person's config handling
machinery for people building e2fsprogs and the installing on their
own local system.

						- Ted

>From f8e42224f8cc77afd2b2372dc527428072e28519 Mon Sep 17 00:00:00 2001
From: Theodore Ts'o <tytso@mit.edu>
Date: Wed, 16 Jul 2008 13:39:32 -0400
Subject: [PATCH] Add more intelligent handling of the mke2fs.conf installation process

For people who are compiling mke2fs for their own use outside of a
package manager, we need to make sure the system /etc/mke2fs.conf is
sufficiently up-to-date that it won't cause problems, but at the same
time we don't want to blow away any user-specific customizations.

So if /etc/mk2fs.conf exists, but does not mention ext4dev, we will
move it aside to /etc/mke2fs.conf.e2fsprogs-old and then install the
new mke2fs.conf.  If the /etc/mke2fs.conf file exists but does mention
ext4dev, we install the new mke2fs.conf file as
/etc/mke2fs.conf.e2fsprogs-new.  In both we print warning to the user
so they can manually make any changes as needed.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
---
 misc/Makefile.in |   25 ++++++++++++++++++++++++-
 1 files changed, 24 insertions(+), 1 deletions(-)

diff --git a/misc/Makefile.in b/misc/Makefile.in
index a8d1018..a81df8a 100644
--- a/misc/Makefile.in
+++ b/misc/Makefile.in
@@ -343,7 +343,30 @@ install: all $(SMANPAGES) $(UMANPAGES) installdirs
 		echo "	INSTALL_DATA $(man5dir)/$$i"; \
 		$(INSTALL_DATA) $$i $(DESTDIR)$(man5dir)/$$i; \
 	done
-	@if ! test -f $(DESTDIR)$(root_sysconfdir)/mke2fs.conf; then \
+	@if test -f $(DESTDIR)$(root_sysconfdir)/mke2fs.conf; then \
+		if cmp -s $(DESTDIR)$(root_sysconfdir)/mke2fs.conf \
+			$(srcdir)/mke2fs.conf; then \
+			true; \
+		else \
+			if grep -q ext4dev $(DESTDIR)$(root_sysconfdir)/mke2fs.conf ; then \
+				echo "	INSTALL_DATA $(root_sysconfdir)/mke2fs.conf.e2fsprogs-new"; \
+				$(INSTALL_DATA) $(srcdir)/mke2fs.conf \
+					$(DESTDIR)$(root_sysconfdir)/mke2fs.conf.e2fsprogs-new; \
+				echo "Warning: installing mke2fs.conf in $(DESTDIR)$(root_sysconfdir)/mke2fs.conf.e2fsprogs-new"; \
+				echo "Check to see if you need to update your $(root_sysconfdir)/mke2fs.conf"; \
+			else \
+				echo "	INSTALL_DATA $(root_sysconfdir)/mke2fs.conf"; \
+				mv $(DESTDIR)$(root_sysconfdir)/mke2fs.conf \
+				   $(DESTDIR)$(root_sysconfdir)/mke2fs.conf.e2fsprogs-old; \
+				$(INSTALL_DATA) $(srcdir)/mke2fs.conf \
+					$(DESTDIR)$(root_sysconfdir)/mke2fs.conf; \
+				echo "Your mke2fs.conf is too old.  Backing up old version in"; \
+				echo "$(DESTDIR)$(root_sysconfdir)/mke2fs.conf.e2fsprogs-old.  Please check to see"; \
+				echo "if you have any local customizations that you wish to preserve."; \
+			fi; \
+			echo " "; \
+		fi; \
+	else \
 		echo "	INSTALL_DATA $(root_sysconfdir)/mke2fs.conf"; \
 		$(INSTALL_DATA) $(srcdir)/mke2fs.conf \
 			$(DESTDIR)$(root_sysconfdir)/mke2fs.conf; \
-- 
1.5.6.1.205.ge2c7.dirty

Re: [PATCH, RFC] Add more intelligent handling of the mke2fs.conf installation process

[Mingming Cao]

在 2008-07-16三的 13:52 -0400，Theodore Tso写道：
> On Wed, Jul 16, 2008 at 10:02:00AM -0700, Mingming Cao wrote:
> > I looked at misc/Makefile, it seems if the system already
> > have /etc/mke2fs.conf, it will not get updated. This is reasonable for
> > ext3.   But not good for ext4.
> 
> Good point.  The following patch won't do anything for people building
> RPM or dpkg packages (which have their own automatic configuration
> file handling); this will provide a poor person's config handling
> machinery for people building e2fsprogs and the installing on their
> own local system.
> 

Thanks Ted, 

I verified it works great.

Mingming
> 						- Ted
> 
> From f8e42224f8cc77afd2b2372dc527428072e28519 Mon Sep 17 00:00:00 2001
> From: Theodore Ts'o <tytso@mit.edu>
> Date: Wed, 16 Jul 2008 13:39:32 -0400
> Subject: [PATCH] Add more intelligent handling of the mke2fs.conf installation process
> 
> For people who are compiling mke2fs for their own use outside of a
> package manager, we need to make sure the system /etc/mke2fs.conf is
> sufficiently up-to-date that it won't cause problems, but at the same
> time we don't want to blow away any user-specific customizations.
> 
> So if /etc/mk2fs.conf exists, but does not mention ext4dev, we will
> move it aside to /etc/mke2fs.conf.e2fsprogs-old and then install the
> new mke2fs.conf.  If the /etc/mke2fs.conf file exists but does mention
> ext4dev, we install the new mke2fs.conf file as
> /etc/mke2fs.conf.e2fsprogs-new.  In both we print warning to the user
> so they can manually make any changes as needed.
> 
> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
> ---
>  misc/Makefile.in |   25 ++++++++++++++++++++++++-
>  1 files changed, 24 insertions(+), 1 deletions(-)
> 
> diff --git a/misc/Makefile.in b/misc/Makefile.in
> index a8d1018..a81df8a 100644
> --- a/misc/Makefile.in
> +++ b/misc/Makefile.in
> @@ -343,7 +343,30 @@ install: all $(SMANPAGES) $(UMANPAGES) installdirs
>  		echo "	INSTALL_DATA $(man5dir)/$$i"; \
>  		$(INSTALL_DATA) $$i $(DESTDIR)$(man5dir)/$$i; \
>  	done
> -	@if ! test -f $(DESTDIR)$(root_sysconfdir)/mke2fs.conf; then \
> +	@if test -f $(DESTDIR)$(root_sysconfdir)/mke2fs.conf; then \
> +		if cmp -s $(DESTDIR)$(root_sysconfdir)/mke2fs.conf \
> +			$(srcdir)/mke2fs.conf; then \
> +			true; \
> +		else \
> +			if grep -q ext4dev $(DESTDIR)$(root_sysconfdir)/mke2fs.conf ; then \
> +				echo "	INSTALL_DATA $(root_sysconfdir)/mke2fs.conf.e2fsprogs-new"; \
> +				$(INSTALL_DATA) $(srcdir)/mke2fs.conf \
> +					$(DESTDIR)$(root_sysconfdir)/mke2fs.conf.e2fsprogs-new; \
> +				echo "Warning: installing mke2fs.conf in $(DESTDIR)$(root_sysconfdir)/mke2fs.conf.e2fsprogs-new"; \
> +				echo "Check to see if you need to update your $(root_sysconfdir)/mke2fs.conf"; \
> +			else \
> +				echo "	INSTALL_DATA $(root_sysconfdir)/mke2fs.conf"; \
> +				mv $(DESTDIR)$(root_sysconfdir)/mke2fs.conf \
> +				   $(DESTDIR)$(root_sysconfdir)/mke2fs.conf.e2fsprogs-old; \
> +				$(INSTALL_DATA) $(srcdir)/mke2fs.conf \
> +					$(DESTDIR)$(root_sysconfdir)/mke2fs.conf; \
> +				echo "Your mke2fs.conf is too old.  Backing up old version in"; \
> +				echo "$(DESTDIR)$(root_sysconfdir)/mke2fs.conf.e2fsprogs-old.  Please check to see"; \
> +				echo "if you have any local customizations that you wish to preserve."; \
> +			fi; \
> +			echo " "; \
> +		fi; \
> +	else \
>  		echo "	INSTALL_DATA $(root_sysconfdir)/mke2fs.conf"; \
>  		$(INSTALL_DATA) $(srcdir)/mke2fs.conf \
>  			$(DESTDIR)$(root_sysconfdir)/mke2fs.conf; \

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH, RFC] Add more intelligent handling of the mke2fs.conf installation process

[Andreas Dilger]

On Jul 16, 2008  13:52 -0400, Theodore Ts'o wrote:
> So if /etc/mk2fs.conf exists, but does not mention ext4dev, we will
> move it aside to /etc/mke2fs.conf.e2fsprogs-old and then install the
> new mke2fs.conf.  If the /etc/mke2fs.conf file exists but does mention
> ext4dev, we install the new mke2fs.conf file as
> /etc/mke2fs.conf.e2fsprogs-new.  In both we print warning to the user
> so they can manually make any changes as needed.

What about just adding the ext4/ext4dev stanzas into the existing file?

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

Re: [PATCH, RFC] Add more intelligent handling of the mke2fs.conf installation process

[Theodore Tso]

On Wed, Jul 16, 2008 at 03:29:49PM -0600, Andreas Dilger wrote:
> On Jul 16, 2008  13:52 -0400, Theodore Ts'o wrote:
> > So if /etc/mk2fs.conf exists, but does not mention ext4dev, we will
> > move it aside to /etc/mke2fs.conf.e2fsprogs-old and then install the
> > new mke2fs.conf.  If the /etc/mke2fs.conf file exists but does mention
> > ext4dev, we install the new mke2fs.conf file as
> > /etc/mke2fs.conf.e2fsprogs-new.  In both we print warning to the user
> > so they can manually make any changes as needed.
> 
> What about just adding the ext4/ext4dev stanzas into the existing file?

There are other changes would be highly recommended beyond just the
ext4 and ext4dev stanzas.  Also, it will be a relatively small set of
people that will be installing e2fsprogs via "make install", so I
decided to keep things simple.

I suppose I could check and see if the md5sum of mke2fs matched the
versions shipped in v1.40 and 1.40.1..1.40.8 and if they matched, to
just overwrite it with the current distributed default (which is what
a package manager effectively does), but I figured it wasn't worth the
trouble.

						- Ted