Release of e2fsprogs 1.43.5

Release of e2fsprogs 1.43.5

[Theodore Ts'o]

I've released e2fsprogs 1.43.5 in all of the usual places; it's tagged
in the git trees on git.kernel.org, github, and sourceforge, and
available for download at:

ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/v1.43.5

and

http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.43.5.tar.gz

The release notes for 1.43.5 can be found below.

E2fsprogs 1.43.5 (August 4, 2017)
=================================

Fix a bug which could cause e2fsck to loop forever when rebuilding an
encrypted directory with entries with identical prefixes up to an
embedded NUL character.

Fix build when configured with --disable-threads or --enable-profile.

Fix ext2fs_sync_device() so it returns the proper error code if the
BLKFLSBUF ioctl fails for some reason.

Fix a potential crash in tune2fs when enabling project quota when the
file system has no inodes by fixing error handling in the
quota_write_inode() in libsupport.

Fix a bug in debugfs's "ls -p" handling which could cause it to print
garbage after a file name in a directory entry.

Fix a number of bugs in debugfs, dumpe2fs, e2fsck, tune2fs, and
resize2fs where a carefully/maliciously corrupted file systems (found
by American Fuzzy Lop) could these programs to crash or hang.
(Addresses Debian Bug #868489)

E2fsck no longer issues some harmless bitmap warnings caused by a
corrupted file system; since those corruptions will be fixed up by
e2fsck later, there's no point issuing these internal warnings.

E2fsck will now notice invalid quota inode numbers, and offer to fix
them.

Fix a regression introduced in e2fsprogs 1.43 which broke mke2fs's
hugefile creation so that they aren't contiguous.  (Addresses Google
Bugs 62791459 and 638037607)

Fix how backup superblocks are written in big endian systems so they
are all correctly byte swapped.

Fixed tune2fs's support for enabling and disabling project quota.

The badblock program now properly handles languages which have
multi-column wide character such as Chinese.  (Addresses Debian Bug:
#860926)

The mke2fs -U option now accepts "null", "clear", "random", or "time"
just as tune2fs's -U option.

Fix e2fsck's -E bmap2extent feature (which converts a file to use
extents from indirect block maps) so that it handles sparse files correctly.

Fix libext2fs to correctly handle accounting for an external xattr
block on bigalloc file systems.

Fix e2fsck to correctly handle quota accounting when deleting files
that have mnultiply-claimed blocks pass 1b.

Fix potential buffer overrun bug in e2fsck when using 128 byte inodes.
(Addresses Google Bug 37326362).

E2fsck's extent tree optimization can now be disabled via an
command-line extended option or via an e2fsck.conf configuration
parameter.

The e2fsck program now properly updates the quota accounting when
optimizing the extent tree.  (Addresses Google Bug 36391645)

Fix some error handling in e4crypt when operating on keyring ID's.

Fix e2fsck's quota handling when the bigalloc feature is enabled.

The libext2fs file system now uses readahead when opening file systems
to speed up opening very large file systems with the meta_bg option.

If e2fsck logging is enabled, e2fsck will report the exit code in the
last line of the log file.

Debugfs can now open file systems with a bad sueprblock checksum, and
the show_super_stats command will print the expected checksum.

Fix a regression introduced in e2fsprogs 1.43.4 where tune2fs was not
able to correctly enable the quota feature.  (Addresses Debian Bug
#855417)

E2fsck will only return an exit status code of 1 if it has fixed a
file system corruption.  If it only performed an optimal optimization
of internal file system metadata, it will now return 0.

Fix e2fsck so it won't check inode fields of deleted inodes.  This was
mostly harmless, but the addition of a check in 1.43.4 to enforce the
requirement that inline data inodes have a system.data attribute
caused a regression in xfstests generic/079.  Fix this and a number of
other checks which could lead to false positive reports of file system
corruption.

Removed a debugging printf in libsupport which could result to some
extraneous and confusing output if the user specifies a quota type.

The debugfs logdump command now accepts a new -S option which will
cause it to print information about the journal superblock.

Added a missing space in debugfs's ls -l output to separate the size
and date fields.

Update the Czech, Danish, Dutch, French, Hungarian, Polish, Serbian,
Spanish, Swedish, Ukrainian, and Vietnamese translations.

Update various man pages.  (Addresses Debian Bugs #867895 and #865584)

Fix various compiler and UBSAN warnings.

Fixed various Debian Packaging Issues.  (Addresses Debian Bugs
#864144 and #866623)


Programming notes
-----------------

The blkid library can now recognize exfat file systems.

Add new test, j_recover_csum3_64bit, which checks to make sure we
correctly recovering 64-bit journals using the v3 checksum format.

Fix portability assumption in tests; don't depend on the existence of
/dev/stdin.

Fix portability with newer versions of glibc and with Cygwin builds on
Windows 8 and Windows 10.

Clean up the comments in problem.c match the message that will
actually be printed.

Removed an unused an obsolete header file.

Re: Release of e2fsprogs 1.43.5

[Eric Sandeen]

On 8/5/17 9:34 AM, Theodore Ts'o wrote:
> I've released e2fsprogs 1.43.5 in all of the usual places; it's tagged
> in the git trees on git.kernel.org, github, and sourceforge, and
> available for download at:
> 
> ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/v1.43.5
> 
> and
> 
> http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.43.5.tar.gz
> 
> The release notes for 1.43.5 can be found below.

I'm getting test failures in rawhide with this, FWIW; on i686
and arm:

336 tests succeeded	3 tests failed
Tests failed: f_del_dup_quota f_quota f_quota_extent_opt 

Haven't dug into this yet (failures are on ephemeral build root
on a build host), but has anyone else seen problems?

Thanks,
-Eric

Re: Release of e2fsprogs 1.43.5

[Eric Whitney]

* Eric Sandeen <esandeen@redhat.com>:
> On 8/5/17 9:34 AM, Theodore Ts'o wrote:
> > I've released e2fsprogs 1.43.5 in all of the usual places; it's tagged
> > in the git trees on git.kernel.org, github, and sourceforge, and
> > available for download at:
> > 
> > ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/v1.43.5
> > 
> > and
> > 
> > http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.43.5.tar.gz
> > 
> > The release notes for 1.43.5 can be found below.
> 
> I'm getting test failures in rawhide with this, FWIW; on i686
> and arm:
> 
> 336 tests succeeded	3 tests failed
> Tests failed: f_del_dup_quota f_quota f_quota_extent_opt 
> 
> Haven't dug into this yet (failures are on ephemeral build root
> on a build host), but has anyone else seen problems?
> 

Yes - I can reproduce these failures on Debian Jessie on 32 bit x86, as can
Ted.

Reverting "libext2fs: fix compile errors/warnings" (a850bc56e0aa) resolves
the problem for me.  In particular, undoing the format string changes in
scan_dquots_callback() fixes the failures.  I think we need to use %lld as
dqb_curspace and dqb_curinodes are __le64, as addressed in "e2fsck: fix type
mismatches in quota warning message" (f09669fc6889).

Eric

Re: Release of e2fsprogs 1.43.5

[Darrick J. Wong]

On Thu, Aug 10, 2017 at 01:30:44PM -0400, Eric Whitney wrote:
> * Eric Sandeen <esandeen@redhat.com>:
> > On 8/5/17 9:34 AM, Theodore Ts'o wrote:
> > > I've released e2fsprogs 1.43.5 in all of the usual places; it's tagged
> > > in the git trees on git.kernel.org, github, and sourceforge, and
> > > available for download at:
> > > 
> > > ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/v1.43.5
> > > 
> > > and
> > > 
> > > http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.43.5.tar.gz
> > > 
> > > The release notes for 1.43.5 can be found below.
> > 
> > I'm getting test failures in rawhide with this, FWIW; on i686
> > and arm:
> > 
> > 336 tests succeeded	3 tests failed
> > Tests failed: f_del_dup_quota f_quota f_quota_extent_opt 
> > 
> > Haven't dug into this yet (failures are on ephemeral build root
> > on a build host), but has anyone else seen problems?
> > 
> 
> Yes - I can reproduce these failures on Debian Jessie on 32 bit x86, as can
> Ted.
> 
> Reverting "libext2fs: fix compile errors/warnings" (a850bc56e0aa) resolves
> the problem for me.  In particular, undoing the format string changes in
> scan_dquots_callback() fixes the failures.  I think we need to use %lld as
> dqb_curspace and dqb_curinodes are __le64,

lib/support/quotaio.h does:

  typedef int64_t qsize_t;

  struct util_dqblk {
	...
	qsize_t dqb_curinodes;
	...
	qsize_t dqb_curspace;
	...
  };

and if they're int64_t then you could use PRId64.

--D

> as addressed in "e2fsck: fix type mismatches in quota warning message"
> (f09669fc6889).
> 
> Eric

Re: Release of e2fsprogs 1.43.5

[Eric Whitney]

* Darrick J. Wong <darrick.wong@oracle.com>:
> On Thu, Aug 10, 2017 at 01:30:44PM -0400, Eric Whitney wrote:
> > * Eric Sandeen <esandeen@redhat.com>:
> > > On 8/5/17 9:34 AM, Theodore Ts'o wrote:
> > > > I've released e2fsprogs 1.43.5 in all of the usual places; it's tagged
> > > > in the git trees on git.kernel.org, github, and sourceforge, and
> > > > available for download at:
> > > > 
> > > > ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/v1.43.5
> > > > 
> > > > and
> > > > 
> > > > http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.43.5.tar.gz
> > > > 
> > > > The release notes for 1.43.5 can be found below.
> > > 
> > > I'm getting test failures in rawhide with this, FWIW; on i686
> > > and arm:
> > > 
> > > 336 tests succeeded	3 tests failed
> > > Tests failed: f_del_dup_quota f_quota f_quota_extent_opt 
> > > 
> > > Haven't dug into this yet (failures are on ephemeral build root
> > > on a build host), but has anyone else seen problems?
> > > 
> > 
> > Yes - I can reproduce these failures on Debian Jessie on 32 bit x86, as can
> > Ted.
> > 
> > Reverting "libext2fs: fix compile errors/warnings" (a850bc56e0aa) resolves
> > the problem for me.  In particular, undoing the format string changes in
> > scan_dquots_callback() fixes the failures.  I think we need to use %lld as
> > dqb_curspace and dqb_curinodes are __le64,
> 
> lib/support/quotaio.h does:
> 
>   typedef int64_t qsize_t;
> 
>   struct util_dqblk {
> 	...
> 	qsize_t dqb_curinodes;
> 	...
> 	qsize_t dqb_curspace;
> 	...
>   };
> 
> and if they're int64_t then you could use PRId64.
> 
> --D

My mistake - I grepped the wrong declarations of dqb_curinodes/dqb_curspace to
determine type.  They're indeed int64_t in this case.

Eric

[PATCH] libsupport: fix 32-bit quota test failures

[Theodore Ts'o]

On 32-bit platform some of the util_dqblk structures have a type of
long long.  So we need to use %lld and casts to make sure the right
thing happens on both 32-bit and 64-bit platforms.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 lib/support/mkquota.c | 22 ++++++++++++----------
 1 file changed, 12 insertions(+), 10 deletions(-)

diff --git a/lib/support/mkquota.c b/lib/support/mkquota.c
index 00f3a406e..931a839dd 100644
--- a/lib/support/mkquota.c
+++ b/lib/support/mkquota.c
@@ -50,11 +50,13 @@ static void print_dquot(const char *desc, struct dquot *dq)
 {
 	if (desc)
 		fprintf(stderr, "%s: ", desc);
-	fprintf(stderr, "%u %ld:%ld:%ld %ld:%ld:%ld\n",
-		dq->dq_id, dq->dq_dqb.dqb_curspace,
-		dq->dq_dqb.dqb_bsoftlimit, dq->dq_dqb.dqb_bhardlimit,
-		dq->dq_dqb.dqb_curinodes,
-		dq->dq_dqb.dqb_isoftlimit, dq->dq_dqb.dqb_ihardlimit);
+	fprintf(stderr, "%u %lld:%lld:%lld %lld:%lld:%lld\n",
+		dq->dq_id, (long long) dq->dq_dqb.dqb_curspace,
+		(long long) dq->dq_dqb.dqb_bsoftlimit,
+		(long long) dq->dq_dqb.dqb_bhardlimit,
+		(long long) dq->dq_dqb.dqb_curinodes,
+		(long long) dq->dq_dqb.dqb_isoftlimit,
+		(long long) dq->dq_dqb.dqb_ihardlimit);
 }
 #else
 static void print_dquot(const char *desc EXT2FS_ATTR((unused)),
@@ -524,11 +526,11 @@ static int scan_dquots_callback(struct dquot *dquot, void *cb_data)
 	    dq->dq_dqb.dqb_curinodes != dquot->dq_dqb.dqb_curinodes) {
 		scan_data->usage_is_inconsistent = 1;
 		fprintf(stderr, "[QUOTA WARNING] Usage inconsistent for ID %u:"
-			"actual (%ld, %ld) != expected (%ld, %ld)\n",
-			dq->dq_id, dq->dq_dqb.dqb_curspace,
-			dq->dq_dqb.dqb_curinodes,
-			dquot->dq_dqb.dqb_curspace,
-			dquot->dq_dqb.dqb_curinodes);
+			"actual (%lld, %lld) != expected (%lld, %lld)\n",
+			dq->dq_id, (long long) dq->dq_dqb.dqb_curspace,
+			(long long) dq->dq_dqb.dqb_curinodes,
+			(long long) dquot->dq_dqb.dqb_curspace,
+			(long long) dquot->dq_dqb.dqb_curinodes);
 	}
 
 	if (scan_data->update_limits) {
-- 
2.11.0.rc0.7.gbe5a750

Re: Release of e2fsprogs 1.43.5

[Eric Sandeen]

On 8/10/17 10:30 AM, Eric Whitney wrote:
> * Eric Sandeen <esandeen@redhat.com>:

...

>> 336 tests succeeded	3 tests failed
>> Tests failed: f_del_dup_quota f_quota f_quota_extent_opt 
>>
>> Haven't dug into this yet (failures are on ephemeral build root
>> on a build host), but has anyone else seen problems?
>>
> 
> Yes - I can reproduce these failures on Debian Jessie on 32 bit x86, as can
> Ted.
> 
> Reverting "libext2fs: fix compile errors/warnings" (a850bc56e0aa) resolves
> the problem for me.  In particular, undoing the format string changes in
> scan_dquots_callback() fixes the failures.  I think we need to use %lld as
> dqb_curspace and dqb_curinodes are __le64, as addressed in "e2fsck: fix type
> mismatches in quota warning message" (f09669fc6889).

Cool, thanks Eric.

-Eric