* [PATCH 1/6] tests: use e2fsck -f instead of -p for resize tests
2014-07-26 13:48 [PATCH 0/6] miscellaneous bug fixes for e2fsprogs Theodore Ts'o
@ 2014-07-26 13:48 ` Theodore Ts'o
2014-07-26 13:48 ` [PATCH 2/6] libext2fs: fix rb_resize_bmap to handle the padding bits Theodore Ts'o
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Theodore Ts'o @ 2014-07-26 13:48 UTC (permalink / raw)
To: Ext4 Developers List; +Cc: Theodore Ts'o
Using e2sck -f provides better debugging information if things go
wrong.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
tests/scripts/resize_test | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/tests/scripts/resize_test b/tests/scripts/resize_test
index a12e4b9..0633e0c 100755
--- a/tests/scripts/resize_test
+++ b/tests/scripts/resize_test
@@ -64,8 +64,8 @@ then
return 1
fi
-echo $FSCK -fp $TMPFILE >> $LOG 2>&1
-if ! $FSCK -fp $TMPFILE >> $LOG 2>&1
+echo $FSCK -fy $TMPFILE >> $LOG 2>&1
+if ! $FSCK -fy $TMPFILE >> $LOG 2>&1
then
dumpe2fs $TMPFILE >> $LOG
return 1
@@ -94,8 +94,8 @@ then
return 1
fi
-echo $FSCK -fp $TMPFILE >> $LOG 2>&1
-if ! $FSCK -fp $TMPFILE >> $LOG 2>&1
+echo $FSCK -fy $TMPFILE >> $LOG 2>&1
+if ! $FSCK -fy $TMPFILE >> $LOG 2>&1
then
dumpe2fs $TMPFILE >> $LOG
return 1
@@ -119,8 +119,8 @@ then
return 1
fi
-echo $FSCK -fp $TMPFILE >> $LOG 2>&1
-if ! $FSCK -fp $TMPFILE >> $LOG 2>&1
+echo $FSCK -fy $TMPFILE >> $LOG 2>&1
+if ! $FSCK -fy $TMPFILE >> $LOG 2>&1
then
dumpe2fs $TMPFILE >> $LOG
return 1
@@ -144,8 +144,8 @@ then
return 1
fi
-echo $FSCK -fp $TMPFILE >> $LOG 2>&1
-if ! $FSCK -fp $TMPFILE >> $LOG 2>&1
+echo $FSCK -fy $TMPFILE >> $LOG 2>&1
+if ! $FSCK -fy $TMPFILE >> $LOG 2>&1
then
dumpe2fs $TMPFILE >> $LOG
return 1
--
2.0.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 2/6] libext2fs: fix rb_resize_bmap to handle the padding bits
2014-07-26 13:48 [PATCH 0/6] miscellaneous bug fixes for e2fsprogs Theodore Ts'o
2014-07-26 13:48 ` [PATCH 1/6] tests: use e2fsck -f instead of -p for resize tests Theodore Ts'o
@ 2014-07-26 13:48 ` Theodore Ts'o
2014-07-26 13:48 ` [PATCH 3/6] resize2fs: radically reduce memory utilization by using rbtree bitmaps Theodore Ts'o
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Theodore Ts'o @ 2014-07-26 13:48 UTC (permalink / raw)
To: Ext4 Developers List; +Cc: Theodore Ts'o
The bits between end and real_end are set as a safety measure for the
kernel when it uses the bit scan instructions. We need to take this
into account when shrinking or growing the block allocation bitmap,
before we can safely use rbtree bitmaps in resize2fs.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
lib/ext2fs/blkmap64_rb.c | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/lib/ext2fs/blkmap64_rb.c b/lib/ext2fs/blkmap64_rb.c
index a1dde6d..fdbd330 100644
--- a/lib/ext2fs/blkmap64_rb.c
+++ b/lib/ext2fs/blkmap64_rb.c
@@ -299,21 +299,19 @@ static errcode_t rb_resize_bmap(ext2fs_generic_bitmap bmap,
{
struct ext2fs_rb_private *bp;
- if (new_real_end >= bmap->real_end) {
- bmap->end = new_end;
- bmap->real_end = new_real_end;
- return 0;
- }
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 3/6] resize2fs: radically reduce memory utilization by using rbtree bitmaps
2014-07-26 13:48 [PATCH 0/6] miscellaneous bug fixes for e2fsprogs Theodore Ts'o
2014-07-26 13:48 ` [PATCH 1/6] tests: use e2fsck -f instead of -p for resize tests Theodore Ts'o
2014-07-26 13:48 ` [PATCH 2/6] libext2fs: fix rb_resize_bmap to handle the padding bits Theodore Ts'o
@ 2014-07-26 13:48 ` Theodore Ts'o
2014-07-26 13:48 ` [PATCH 4/6] libext2fs: use C99 initializers for the io_manager structure Theodore Ts'o
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Theodore Ts'o @ 2014-07-26 13:48 UTC (permalink / raw)
To: Ext4 Developers List; +Cc: Theodore Ts'o
When resizing an empty 21T file system to 28T, resize2fs was using
this much CPU time and memory:
216.98user 19.77system 4:02.92elapsed 97%CPU (0avgtext+0avgdata 4485664maxresident)k
8inputs+1068680outputs (0major+800745minor)pagefaults 0swaps
After this one-line change:
222.29user 0.49system 3:48.79elapsed 97%CPU (0avgtext+0avgdata 30080maxresident)k
8inputs+1068552outputs (0major+2497minor)pagefaults 0swaps
So this reduces the max memory utilized from 4.2GB to 29MB!
For future work, the primary place where we are spending the most cpu
time (from resize2fs -d 16) are these two places:
blocks_to_move: Memory used: 2508k/25096k (1903k/606k), time: 91.42/91.53/ 0.00
and
calculate_summary_stats: Memory used: 2508k/25612k (1908k/601k), time: 95.33/95.45/ 0.00
The calculate_summary_stats pass can be sped up by using
ext2fs_find_first_{zero,set}_block_bitmap2(), instead of iterating
over the entire block bitmap one bit at a time.
The blocks_to_move pass can be sped up by using a bitmap to store the
location of fs metadata blocks, to avoid an O(N**2) algorithm where N
is the number of groups in the file system.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
resize/main.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/resize/main.c b/resize/main.c
index e4b4435..ef2a810 100644
--- a/resize/main.c
+++ b/resize/main.c
@@ -318,6 +318,7 @@ int main (int argc, char ** argv)
printf("%s", _("Couldn't find valid filesystem superblock.\n"));
exit (1);
}
+ fs->default_bitmap_type = EXT2FS_BMAP64_RBTREE;
if (!(mount_flags & EXT2_MF_MOUNTED)) {
if (!force && ((fs->super->s_lastcheck < fs->super->s_mtime) ||
--
2.0.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 4/6] libext2fs: use C99 initializers for the io_manager structure
2014-07-26 13:48 [PATCH 0/6] miscellaneous bug fixes for e2fsprogs Theodore Ts'o
` (2 preceding siblings ...)
2014-07-26 13:48 ` [PATCH 3/6] resize2fs: radically reduce memory utilization by using rbtree bitmaps Theodore Ts'o
@ 2014-07-26 13:48 ` Theodore Ts'o
2014-07-26 13:48 ` [PATCH 5/6] Fix 32/64-bit overflow when multiplying by blocks/clusters per group Theodore Ts'o
2014-07-26 13:48 ` [PATCH 6/6] libext2fs: fix free block accounting for 64-bit file systems Theodore Ts'o
5 siblings, 0 replies; 7+ messages in thread
From: Theodore Ts'o @ 2014-07-26 13:48 UTC (permalink / raw)
To: Ext4 Developers List; +Cc: Theodore Ts'o
Using C99 initializers makes the code a bit more readable, and it
avoids some gcc -Wall warnings regarding missing initializers.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
lib/ext2fs/dosio.c | 17 +++++++++--------
lib/ext2fs/inode_io.c | 24 +++++++++++-------------
lib/ext2fs/nt_io.c | 18 ++++++++----------
lib/ext2fs/test_io.c | 28 ++++++++++++++--------------
lib/ext2fs/undo_io.c | 26 +++++++++++++-------------
lib/ext2fs/unix_io.c | 28 ++++++++++++++--------------
util/gcc-wall-cleanup | 1 -
7 files changed, 69 insertions(+), 73 deletions(-)
diff --git a/lib/ext2fs/dosio.c b/lib/ext2fs/dosio.c
index 0defff8..d0cf269 100644
--- a/lib/ext2fs/dosio.c
+++ b/lib/ext2fs/dosio.c
@@ -62,15 +62,16 @@ static errcode_t dos_write_blk(io_channel channel, unsigned long block,
static errcode_t dos_flush(io_channel channel);
static struct struct_io_manager struct_dos_manager = {
- EXT2_ET_MAGIC_IO_MANAGER,
- "DOS I/O Manager",
- dos_open,
- dos_close,
- dos_set_blksize,
- dos_read_blk,
- dos_write_blk,
- dos_flush
+ .magic = EXT2_ET_MAGIC_IO_MANAGER,
+ .name = "DOS I/O Manager",
+ .open = dos_open,
+ .close = dos_close,
+ .set_blksize = dos_set_blksize,
+ .read_blk = dos_read_blk,
+ .write_blk = dos_write_blk,
+ .flush = dos_flush
};
+
io_manager dos_io_manager = &struct_dos_manager;
/*
diff --git a/lib/ext2fs/inode_io.c b/lib/ext2fs/inode_io.c
index 8e0944e..f3d94c8 100644
--- a/lib/ext2fs/inode_io.c
+++ b/lib/ext2fs/inode_io.c
@@ -63,19 +63,17 @@ static errcode_t inode_write_blk64(io_channel channel,
unsigned long long block, int count, const void *data);
static struct struct_io_manager struct_inode_manager = {
- EXT2_ET_MAGIC_IO_MANAGER,
- "Inode I/O Manager",
- inode_open,
- inode_close,
- inode_set_blksize,
- inode_read_blk,
- inode_write_blk,
- inode_flush,
- inode_write_byte,
- NULL,
- NULL,
- inode_read_blk64,
- inode_write_blk64
+ .magic = EXT2_ET_MAGIC_IO_MANAGER,
+ .name = "Inode I/O Manager",
+ .open = inode_open,
+ .close = inode_close,
+ .set_blksize = inode_set_blksize,
+ .read_blk = inode_read_blk,
+ .write_blk = inode_write_blk,
+ .flush = inode_flush,
+ .write_byte = inode_write_byte,
+ .read_blk64 = inode_read_blk64,
+ .write_blk64 = inode_write_blk64
};
io_manager inode_io_manager = &struct_inode_manager;
diff --git a/lib/ext2fs/nt_io.c b/lib/ext2fs/nt_io.c
index 0f10543..f0d16ae 100644
--- a/lib/ext2fs/nt_io.c
+++ b/lib/ext2fs/nt_io.c
@@ -230,18 +230,16 @@ static errcode_t nt_write_blk(io_channel channel, unsigned long block,
static errcode_t nt_flush(io_channel channel);
static struct struct_io_manager struct_nt_manager = {
- EXT2_ET_MAGIC_IO_MANAGER,
- "NT I/O Manager",
- nt_open,
- nt_close,
- nt_set_blksize,
- nt_read_blk,
- nt_write_blk,
- nt_flush
+ .magic = EXT2_ET_MAGIC_IO_MANAGER,
+ .name = "NT I/O Manager",
+ .open = nt_open,
+ .close = nt_close,
+ .set_blksize = nt_set_blksize,
+ .read_blk = nt_read_blk,
+ .write_blk = nt_write_blk,
+ .flush = nt_flush
};
-
-
//
// function to get API
//
diff --git a/lib/ext2fs/test_io.c b/lib/ext2fs/test_io.c
index d16a358..6f0d035 100644
--- a/lib/ext2fs/test_io.c
+++ b/lib/ext2fs/test_io.c
@@ -487,20 +487,20 @@ static errcode_t test_discard(io_channel channel, unsigned long long block,
}
static struct struct_io_manager struct_test_manager = {
- EXT2_ET_MAGIC_IO_MANAGER,
- "Test I/O Manager",
- test_open,
- test_close,
- test_set_blksize,
- test_read_blk,
- test_write_blk,
- test_flush,
- test_write_byte,
- test_set_option,
- test_get_stats,
- test_read_blk64,
- test_write_blk64,
- test_discard,
+ .magic = EXT2_ET_MAGIC_IO_MANAGER,
+ .name = "Test I/O Manager",
+ .open = test_open,
+ .close = test_close,
+ .set_blksize = test_set_blksize,
+ .read_blk = test_read_blk,
+ .write_blk = test_write_blk,
+ .flush = test_flush,
+ .write_byte = test_write_byte,
+ .set_option = test_set_option,
+ .get_stats = test_get_stats,
+ .read_blk64 = test_read_blk64,
+ .write_blk64 = test_write_blk64,
+ .discard = test_discard,
};
io_manager test_io_manager = &struct_test_manager;
diff --git a/lib/ext2fs/undo_io.c b/lib/ext2fs/undo_io.c
index 0e05c93..d6beb02 100644
--- a/lib/ext2fs/undo_io.c
+++ b/lib/ext2fs/undo_io.c
@@ -588,19 +588,19 @@ static errcode_t undo_get_stats(io_channel channel, io_stats *stats)
}
static struct struct_io_manager struct_undo_manager = {
- EXT2_ET_MAGIC_IO_MANAGER,
- "Undo I/O Manager",
- undo_open,
- undo_close,
- undo_set_blksize,
- undo_read_blk,
- undo_write_blk,
- undo_flush,
- undo_write_byte,
- undo_set_option,
- undo_get_stats,
- undo_read_blk64,
- undo_write_blk64,
+ .magic = EXT2_ET_MAGIC_IO_MANAGER,
+ .name = "Undo I/O Manager",
+ .open = undo_open,
+ .close = undo_close,
+ .set_blksize = undo_set_blksize,
+ .read_blk = undo_read_blk,
+ .write_blk = undo_write_blk,
+ .flush = undo_flush,
+ .write_byte = undo_write_byte,
+ .set_option = undo_set_option,
+ .get_stats = undo_get_stats,
+ .read_blk64 = undo_read_blk64,
+ .write_blk64 = undo_write_blk64,
};
io_manager undo_io_manager = &struct_undo_manager;
diff --git a/lib/ext2fs/unix_io.c b/lib/ext2fs/unix_io.c
index c3185b6..e93684e 100644
--- a/lib/ext2fs/unix_io.c
+++ b/lib/ext2fs/unix_io.c
@@ -923,20 +923,20 @@ unimplemented:
}
static struct struct_io_manager struct_unix_manager = {
- EXT2_ET_MAGIC_IO_MANAGER,
- "Unix I/O Manager",
- unix_open,
- unix_close,
- unix_set_blksize,
- unix_read_blk,
- unix_write_blk,
- unix_flush,
- unix_write_byte,
- unix_set_option,
- unix_get_stats,
- unix_read_blk64,
- unix_write_blk64,
- unix_discard,
+ .magic = EXT2_ET_MAGIC_IO_MANAGER,
+ .name = "Unix I/O Manager",
+ .open = unix_open,
+ .close = unix_close,
+ .set_blksize = unix_set_blksize,
+ .read_blk = unix_read_blk,
+ .write_blk = unix_write_blk,
+ .flush = unix_flush,
+ .write_byte = unix_write_byte,
+ .set_option = unix_set_option,
+ .get_stats = unix_get_stats,
+ .read_blk64 = unix_read_blk64,
+ .write_blk64 = unix_write_blk64,
+ .discard = unix_discard,
};
io_manager unix_io_manager = &struct_unix_manager;
diff --git a/util/gcc-wall-cleanup b/util/gcc-wall-cleanup
index 6148e46..c619526 100644
--- a/util/gcc-wall-cleanup
+++ b/util/gcc-wall-cleanup
@@ -16,7 +16,6 @@
/In file included from/d
/In function `.*':/d
/zero-length format string/d
-/warning: missing initializer/d
/warning: (near initialization for/d
/^[ ]*from/d
--
2.0.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 5/6] Fix 32/64-bit overflow when multiplying by blocks/clusters per group
2014-07-26 13:48 [PATCH 0/6] miscellaneous bug fixes for e2fsprogs Theodore Ts'o
` (3 preceding siblings ...)
2014-07-26 13:48 ` [PATCH 4/6] libext2fs: use C99 initializers for the io_manager structure Theodore Ts'o
@ 2014-07-26 13:48 ` Theodore Ts'o
2014-07-26 13:48 ` [PATCH 6/6] libext2fs: fix free block accounting for 64-bit file systems Theodore Ts'o
5 siblings, 0 replies; 7+ messages in thread
From: Theodore Ts'o @ 2014-07-26 13:48 UTC (permalink / raw)
To: Ext4 Developers List; +Cc: Theodore Ts'o
There are a number of places where we need convert groups to blocks or
clusters by multiply the groups by blocks/clusters per group.
Unfortunately, both quantities are 32-bit, but the result needs to be
64-bit, and very often the cast to 64-bit gets lost.
Fix this by adding new macros, EXT2_GROUPS_TO_BLOCKS() and
EXT2_GROUPS_TO_CLUSTERS().
This should fix a bug where resizing a 64bit file system can result in
calculate_minimum_resize_size() looping forever.
Addresses-Launchpad-Bug: #1321958
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
e2fsck/pass5.c | 2 +-
e2fsck/super.c | 2 +-
lib/ext2fs/blknum.c | 2 +-
lib/ext2fs/ext2_fs.h | 5 +++++
lib/ext2fs/imager.c | 14 +++++++-------
lib/ext2fs/rw_bitmaps.c | 4 ++--
misc/tune2fs.c | 2 +-
resize/resize2fs.c | 11 +++++------
8 files changed, 23 insertions(+), 19 deletions(-)
diff --git a/e2fsck/pass5.c b/e2fsck/pass5.c
index d0b1ced..bc9a32a 100644
--- a/e2fsck/pass5.c
+++ b/e2fsck/pass5.c
@@ -776,7 +776,7 @@ static void check_block_end(e2fsck_t ctx)
clear_problem_context(&pctx);
end = ext2fs_get_block_bitmap_start2(fs->block_map) +
- ((blk64_t)EXT2_CLUSTERS_PER_GROUP(fs->super) * fs->group_desc_count) - 1;
+ EXT2_GROUPS_TO_CLUSTERS(fs->super, fs->group_desc_count) - 1;
pctx.errcode = ext2fs_fudge_block_bitmap_end2(fs->block_map, end,
&save_blocks_count);
if (pctx.errcode) {
diff --git a/e2fsck/super.c b/e2fsck/super.c
index 81503d4..8d468e6 100644
--- a/e2fsck/super.c
+++ b/e2fsck/super.c
@@ -421,7 +421,7 @@ void check_resize_inode(e2fsck_t ctx)
for (j = 1; j < fs->group_desc_count; j++) {
if (!ext2fs_bg_has_super(fs, j))
continue;
- expect = pblk + (j * fs->super->s_blocks_per_group);
+ expect = pblk + EXT2_GROUPS_TO_BLOCKS(fs->super, j);
if (ind_buf[ind_off] != expect)
goto resize_inode_invalid;
ind_off++;
diff --git a/lib/ext2fs/blknum.c b/lib/ext2fs/blknum.c
index 8ced1ee..7ce6053 100644
--- a/lib/ext2fs/blknum.c
+++ b/lib/ext2fs/blknum.c
@@ -29,7 +29,7 @@ dgrp_t ext2fs_group_of_blk2(ext2_filsys fs, blk64_t blk)
blk64_t ext2fs_group_first_block2(ext2_filsys fs, dgrp_t group)
{
return fs->super->s_first_data_block +
- ((blk64_t)group * fs->super->s_blocks_per_group);
+ EXT2_GROUPS_TO_BLOCKS(fs->super, group);
}
/*
diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h
index d9e14d7..6c3620c 100644
--- a/lib/ext2fs/ext2_fs.h
+++ b/lib/ext2fs/ext2_fs.h
@@ -264,6 +264,11 @@ struct ext2_dx_countlimit {
#define EXT2_DESC_PER_BLOCK(s) (EXT2_BLOCK_SIZE(s) / EXT2_DESC_SIZE(s))
#endif
+#define EXT2_GROUPS_TO_BLOCKS(s, g) ((blk64_t) EXT2_BLOCKS_PER_GROUP(s) * \
+ (g))
+#define EXT2_GROUPS_TO_CLUSTERS(s, g) ((blk64_t) EXT2_CLUSTERS_PER_GROUP(s) * \
+ (g))
+
/*
* Constants relative to the data blocks
*/
diff --git a/lib/ext2fs/imager.c b/lib/ext2fs/imager.c
index 378a3c8..b643cc6 100644
--- a/lib/ext2fs/imager.c
+++ b/lib/ext2fs/imager.c
@@ -286,8 +286,8 @@ errcode_t ext2fs_image_bitmap_write(ext2_filsys fs, int fd, int flags)
ext2fs_generic_bitmap bmap;
errcode_t retval;
ssize_t actual;
- __u32 itr, cnt, size;
- int c, total_size;
+ size_t c;
+ __u64 itr, cnt, size, total_size;
char buf[1024];
if (flags & IMAGER_FLAG_INODEMAP) {
@@ -308,7 +308,7 @@ errcode_t ext2fs_image_bitmap_write(ext2_filsys fs, int fd, int flags)
}
bmap = fs->block_map;
itr = fs->super->s_first_data_block;
- cnt = EXT2_BLOCKS_PER_GROUP(fs->super) * fs->group_desc_count;
+ cnt = EXT2_GROUPS_TO_BLOCKS(fs->super, fs->group_desc_count);
size = EXT2_BLOCKS_PER_GROUP(fs->super) / 8;
}
total_size = size * fs->group_desc_count;
@@ -342,9 +342,9 @@ errcode_t ext2fs_image_bitmap_write(ext2_filsys fs, int fd, int flags)
if (c > (int) sizeof(buf))
c = sizeof(buf);
actual = write(fd, buf, c);
- if (actual == -1)
+ if (actual < 0)
return errno;
- if (actual != c)
+ if ((size_t) actual != c)
return EXT2_ET_SHORT_WRITE;
size -= c;
}
@@ -360,7 +360,7 @@ errcode_t ext2fs_image_bitmap_read(ext2_filsys fs, int fd, int flags)
{
ext2fs_generic_bitmap bmap;
errcode_t retval;
- __u32 itr, cnt;
+ __u64 itr, cnt;
char buf[1024];
unsigned int size;
ssize_t actual;
@@ -383,7 +383,7 @@ errcode_t ext2fs_image_bitmap_read(ext2_filsys fs, int fd, int flags)
}
bmap = fs->block_map;
itr = fs->super->s_first_data_block;
- cnt = EXT2_BLOCKS_PER_GROUP(fs->super) * fs->group_desc_count;
+ cnt = EXT2_GROUPS_TO_BLOCKS(fs->super, fs->group_desc_count);
size = EXT2_BLOCKS_PER_GROUP(fs->super) / 8;
}
diff --git a/lib/ext2fs/rw_bitmaps.c b/lib/ext2fs/rw_bitmaps.c
index d24ba9a..a07ecd5 100644
--- a/lib/ext2fs/rw_bitmaps.c
+++ b/lib/ext2fs/rw_bitmaps.c
@@ -262,8 +262,8 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
}
blk = (fs->image_header->offset_blockmap /
fs->blocksize);
- blk_cnt = (blk64_t)EXT2_CLUSTERS_PER_GROUP(fs->super) *
- fs->group_desc_count;
+ blk_cnt = EXT2_GROUPS_TO_CLUSTERS(fs->super,
+ fs->group_desc_count);
while (block_nbytes > 0) {
retval = io_channel_read_blk64(fs->image_io, blk++,
1, block_bitmap);
diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index 6b21235..ad0b05f 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -1383,7 +1383,7 @@ static int ext2fs_is_block_in_group(ext2_filsys fs, dgrp_t group, blk64_t blk)
{
blk64_t start_blk, end_blk;
start_blk = fs->super->s_first_data_block +
- EXT2_BLOCKS_PER_GROUP(fs->super) * group;
+ EXT2_GROUPS_TO_BLOCKS(fs->super, group);
/*
* We cannot get new block beyond end_blk for for the last block group
* so we can check with EXT2_BLOCKS_PER_GROUP even for last block group
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 6bd2e1c..9641b1e 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -436,8 +436,7 @@ retry:
fs->inode_map);
if (retval) goto errout;
- real_end = (((blk64_t) EXT2_BLOCKS_PER_GROUP(fs->super) *
- fs->group_desc_count)) - 1 +
+ real_end = EXT2_GROUPS_TO_BLOCKS(fs->super, fs->group_desc_count) - 1 +
fs->super->s_first_data_block;
retval = ext2fs_resize_block_bitmap2(new_size - 1,
real_end, fs->block_map);
@@ -2318,7 +2317,7 @@ blk64_t calculate_minimum_resize_size(ext2_filsys fs, int flags)
fs->super->s_free_inodes_count;
blks_needed = ext2fs_div_ceil(inode_count,
fs->super->s_inodes_per_group) *
- EXT2_BLOCKS_PER_GROUP(fs->super);
+ (blk64_t) EXT2_BLOCKS_PER_GROUP(fs->super);
groups = ext2fs_div64_ceil(blks_needed,
EXT2_BLOCKS_PER_GROUP(fs->super));
#ifdef RESIZE2FS_DEBUG
@@ -2365,7 +2364,7 @@ blk64_t calculate_minimum_resize_size(ext2_filsys fs, int flags)
* figure out how many data blocks we have given the number of groups
* we need for our inodes
*/
- data_blocks = groups * EXT2_BLOCKS_PER_GROUP(fs->super);
+ data_blocks = EXT2_GROUPS_TO_BLOCKS(fs->super, groups);
last_start = 0;
for (grp = 0; grp < flex_groups; grp++) {
overhead = calc_group_overhead(fs, grp, old_desc_blocks);
@@ -2403,7 +2402,7 @@ blk64_t calculate_minimum_resize_size(ext2_filsys fs, int flags)
extra_grps = ext2fs_div64_ceil(remainder,
EXT2_BLOCKS_PER_GROUP(fs->super));
- data_blocks += extra_grps * EXT2_BLOCKS_PER_GROUP(fs->super);
+ data_blocks += EXT2_GROUPS_TO_BLOCKS(fs->super, extra_grps);
/* ok we have to account for the last group */
overhead = calc_group_overhead(fs, groups-1, old_desc_blocks);
@@ -2501,7 +2500,7 @@ blk64_t calculate_minimum_resize_size(ext2_filsys fs, int flags)
* blocks needed to handle the group descriptor metadata+data
* that we need
*/
- blks_needed = (groups-1) * EXT2_BLOCKS_PER_GROUP(fs->super);
+ blks_needed = EXT2_GROUPS_TO_BLOCKS(fs->super, groups - 1);
blks_needed += overhead;
/*
--
2.0.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 6/6] libext2fs: fix free block accounting for 64-bit file systems
2014-07-26 13:48 [PATCH 0/6] miscellaneous bug fixes for e2fsprogs Theodore Ts'o
` (4 preceding siblings ...)
2014-07-26 13:48 ` [PATCH 5/6] Fix 32/64-bit overflow when multiplying by blocks/clusters per group Theodore Ts'o
@ 2014-07-26 13:48 ` Theodore Ts'o
5 siblings, 0 replies; 7+ messages in thread
From: Theodore Ts'o @ 2014-07-26 13:48 UTC (permalink / raw)
To: Ext4 Developers List; +Cc: Theodore Ts'o
We rely on a nasty hack to adjust the free block count where we pass
signed value into ext2fs_free_blocks_count_add(), which takes an
64-bit unsigned value, and relies on overflow and C's signed->unsigned
semantics to do the subtraction. This works, so long as a 64-bit
signed value is used.
Unfortunately, ext2fs_block_alloc_stats2() and
ext2fs_block_alloc_stats_range(), this is not true, so on a 64-bit
file system, the free blocks accounting can get screwed up.
A simple way to demonstrate the problem is:
mke2fs -F -t ext4 -O 64bit /tmp/foo.img 1M
e2fsck -fy /tmp/foo.img
... which will result in the following e2fsck complaint:
Pass 5: Checking group summary information
Free blocks count wrong (4294968278, counted=982).
Fix? yes
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
lib/ext2fs/alloc_stats.c | 4 ++--
tests/m_64bit_flexbg/expect.1 | 55 +++++++++++++++++++++++++++++++++++++++++++
tests/m_64bit_flexbg/script | 4 ++++
3 files changed, 61 insertions(+), 2 deletions(-)
create mode 100644 tests/m_64bit_flexbg/expect.1
create mode 100644 tests/m_64bit_flexbg/script
diff --git a/lib/ext2fs/alloc_stats.c b/lib/ext2fs/alloc_stats.c
index 1f58e00..a449dc3 100644
--- a/lib/ext2fs/alloc_stats.c
+++ b/lib/ext2fs/alloc_stats.c
@@ -79,7 +79,7 @@ void ext2fs_block_alloc_stats2(ext2_filsys fs, blk64_t blk, int inuse)
ext2fs_group_desc_csum_set(fs, group);
ext2fs_free_blocks_count_add(fs->super,
- -inuse * EXT2FS_CLUSTER_RATIO(fs));
+ -inuse * (blk64_t) EXT2FS_CLUSTER_RATIO(fs));
ext2fs_mark_super_dirty(fs);
ext2fs_mark_bb_dirty(fs);
if (fs->block_alloc_stats)
@@ -140,7 +140,7 @@ void ext2fs_block_alloc_stats_range(ext2_filsys fs, blk64_t blk,
inuse*n/EXT2FS_CLUSTER_RATIO(fs));
ext2fs_bg_flags_clear(fs, group, EXT2_BG_BLOCK_UNINIT);
ext2fs_group_desc_csum_set(fs, group);
- ext2fs_free_blocks_count_add(fs->super, -inuse * n);
+ ext2fs_free_blocks_count_add(fs->super, -inuse * (blk64_t) n);
blk += n;
num -= n;
}
diff --git a/tests/m_64bit_flexbg/expect.1 b/tests/m_64bit_flexbg/expect.1
new file mode 100644
index 0000000..3635318
--- /dev/null
+++ b/tests/m_64bit_flexbg/expect.1
@@ -0,0 +1,55 @@
+Creating filesystem with 1024 1k blocks and 128 inodes
+
+Allocating group tables: \b\b\bdone
+Writing inode tables: \b\b\bdone
+Writing superblocks and filesystem accounting information: \b\b\bdone
+
+Filesystem features: ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/128 files (0.0% non-contiguous), 42/1024 blocks
+Exit status is 0
+Filesystem volume name: <none>
+Last mounted on: <not available>
+Filesystem magic number: 0xEF53
+Filesystem revision #: 1 (dynamic)
+Filesystem features: ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super
+Default mount options: (none)
+Filesystem state: clean
+Errors behavior: Continue
+Filesystem OS type: Linux
+Inode count: 128
+Block count: 1024
+Reserved block count: 51
+Free blocks: 982
+Free inodes: 117
+First block: 1
+Block size: 1024
+Fragment size: 1024
+Group descriptor size: 64
+Reserved GDT blocks: 7
+Blocks per group: 8192
+Fragments per group: 8192
+Inodes per group: 128
+Inode blocks per group: 16
+Flex block group size: 16
+Mount count: 0
+Check interval: 15552000 (6 months)
+Reserved blocks uid: 0
+Reserved blocks gid: 0
+First inode: 11
+Inode size: 128
+Default directory hash: half_md4
+
+
+Group 0: (Blocks 1-1023)
+ Primary superblock at 1, Group descriptors at 2-2
+ Reserved GDT blocks at 3-9
+ Block bitmap at 10 (+9), Inode bitmap at 26 (+25)
+ Inode table at 42-57 (+41)
+ 982 free blocks, 117 free inodes, 2 directories
+ Free blocks: 24-25, 28-41, 58-1023
+ Free inodes: 12-128
diff --git a/tests/m_64bit_flexbg/script b/tests/m_64bit_flexbg/script
new file mode 100644
index 0000000..98684e4
--- /dev/null
+++ b/tests/m_64bit_flexbg/script
@@ -0,0 +1,4 @@
+DESCRIPTION="mkfs with 64bit and flex_bg"
+FS_SIZE=1M
+MKE2FS_OPTS="-O 64bit,extents,flex_bg"
+. $cmd_dir/run_mke2fs
--
2.0.0
^ permalink raw reply related [flat|nested] 7+ messages in thread