resize2fs minimum size wrong

resize2fs minimum size wrong

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

So I have a filesystem with nothing beyond group 16 used ( including
metadata ), so it should be able to shrink to ~524288 blocks, yet
resize2fs claims the minimum size is 770083 blocks.  Resize debug output
follows:

resize2fs 1.43-WIP (29-Mar-2015)
fs has 52108 inodes, 7 groups required.
fs requires 492169 data blocks.
With 7 group(s), we have 218842 blocks available.
Added 9 extra group(s), blks_needed 492169, data_blocks 513754,
last_start 224238
Last group's overhead is 10534
Need 267931 data blocks in last group
Final size of last group is 278465
Estimated blocks needed: 769985
Extents safety margin: 98
Estimated minimum size of the filesystem: 770083

So clearly it thinks only 16 block groups are enough, then somehow leaps
from there to 24.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCgAGBQJVYO6xAAoJENRVrw2cjl5RDSUH/1bCpLZ/oxj/zoeFnLgjHDvI
UMiR4sJhtEC2pYxBFatz9DOg3iDsEQvN+tJ3lcVb4l8rn6pFepGaNljxiMk8uWae
U784u/jd5UNbYSftKtC/Y9MkH9bfcWKR9wAmWvGz9gSjJXb5SeqGo4yQZTFkhPK8
vW3jvJ7YCCWqLh/VGGunq6jJaGPGVsstlMOEKvQjdEdd0zhXp3YyMy2+GfBlsvkd
PO1780FjG07s+VS4DRwktNNrWIB8TFxM0TsslDrd8ImB6xSssbbPbVI5T81MrhR1
QwO4q/ql4XAwy57rzmxmzf75HhpjUN69ySBm6VA5N2eyiJqOqY3sBz6wZ01LG/I=
=dht/
-----END PGP SIGNATURE-----

Re: resize2fs minimum size wrong

[Theodore Ts'o]

On Sat, May 23, 2015 at 05:18:41PM -0400, Phillip Susi wrote:
> 
> So I have a filesystem with nothing beyond group 16 used ( including
> metadata ), so it should be able to shrink to ~524288 blocks, yet
> resize2fs claims the minimum size is 770083 blocks.  Resize debug output
> follows:

Figuring out the minimum size in a way where we don't screw up and
pick a number too _low_ is tricky, because if resize2fs aborts in the
middle, we can end up with a corrupted file system.

I have continued to severely regret accepting the patch that
calculates the minimum size necessary, because it's *hard* to get it
right, and if it's too aggressive, users lose data, and get angry, and
if it's too liberal, then users whine.

I'm going to guess the problem has to do with the reserved space
needed so that resize works correctly, and the amount of reserved
space needed drops as we decrease the nmber of block groups.  So if
you start with a very large file system, and then try to shrink it
down to minimum size, you may need to run resize2fs -M multiple times.
Which to be honest, is just fine as far as I'm concerned, because I've
been trying to strongly discourage the use of resize2fs -M in the
first place, because it tends to leave a highly non-optimal file
system.

						- Ted

Re: resize2fs minimum size wrong

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 05/23/2015 07:36 PM, Theodore Ts'o wrote:
> Figuring out the minimum size in a way where we don't screw up and 
> pick a number too _low_ is tricky, because if resize2fs aborts in
> the middle, we can end up with a corrupted file system.

Why is it hard?  Why isn't it simply the number of used blocks, or the
highest fixed metadata block, whichever is higher?

> I'm going to guess the problem has to do with the reserved space 
> needed so that resize works correctly, and the amount of reserved

Reserved for what?


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCgAGBQJVYRsGAAoJENRVrw2cjl5R8fsIAKPy2Z0IKOpKUcQR/2dpuBp+
i1Td++sM3jPfXkshJrds8xe/B1dqxcVKTK6KOjk3YIQg7cr1XfRZW4hNZ4dU/vDi
NK5UiwHMa15HkVGTcF3ic3mEXmO2x3iYpgMb/9kkVJ9/k+VAXOuku1C59BQ+TAaM
bJ75j4xGatN58wMs4+RqTqDHBo/SiSCSCRvlpBmytcZIjmGguhvY9VtgbS7Wqf3l
GN1IjryP5CQaxq563avoIIl93u6ej4l3gYv2cIBDeBfBUv5iXkMx5rA56Z1PRs7v
WRvOA75aKS6EBq1/7PvRkyvfPOBBMUgRJOoYWziMJQKD044HKVNdy72BGjn3hbQ=
=ngJP
-----END PGP SIGNATURE-----

Re: resize2fs minimum size wrong

[Theodore Ts'o]

On Sat, May 23, 2015 at 08:27:50PM -0400, Phillip Susi wrote:
> 
> On 05/23/2015 07:36 PM, Theodore Ts'o wrote:
> > Figuring out the minimum size in a way where we don't screw up and 
> > pick a number too _low_ is tricky, because if resize2fs aborts in
> > the middle, we can end up with a corrupted file system.
> 
> Why is it hard?  Why isn't it simply the number of used blocks, or the
> highest fixed metadata block, whichever is higher?

The amount of space needed for metadata blocks change depending on how
many block groups you have, and as you shrink the block groups, the
amount of space you need the metadata blocks shrink.  You might also
need some of the extra block groups to make sure you have enough
inodes for all of the inodes in the system.

Also, we can't necessarily use all of the space right away; for
example, the inode table for block group #25 lives in block group #16.
If we are shrinking the file system such that block groups #20 through
#25 will no longer be present, we can't use the inode space for those
block groups until *after* the resize has completed successfully,
since we can only free up those blocks after we have made sure we have
evacuated all of the inodes to other parts of the inode table.

> > I'm going to guess the problem has to do with the reserved space 
> > needed so that resize works correctly, and the amount of reserved
> 
> Reserved for what?

Reserved for additional block group descriptors when we are growing
the file system.

					- Ted

[PATCH] resize2fs: fix minimum size calculation

[Phillip Susi]

The size calculation adds an initial number of block groups needed to hold
the inodes on the fs, then augments them with additional groups needed to
reach the minimum number of data blocks, and then calculates the size of
the last block group ( which should be a fraction of one block group ).  It
was calculating a last bg size of several block groups because when adding
the supplementary bgs, it only advanced the position of the start of the
last group by one group, even though it was adding multiple groups.  Add
the missing multiplier to move the last bg start position up to the correct
location, thus computing a proper fractional last bg size.
---
 resize/resize2fs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 041ff75..3359a8f 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -2870,7 +2870,7 @@ blk64_t calculate_minimum_resize_size(ext2_filsys fs, int flags)
 
 		/* ok we have to account for the last group */
 		overhead = calc_group_overhead(fs, groups-1, old_desc_blocks);
-		last_start += EXT2_BLOCKS_PER_GROUP(fs->super) - overhead;
+		last_start += (EXT2_BLOCKS_PER_GROUP(fs->super) * extra_grps) - overhead;
 
 		grp = flex_groups;
 		groups += extra_grps;
-- 
2.1.4

Re: [PATCH] resize2fs: fix minimum size calculation

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 05/23/2015 11:02 PM, Phillip Susi wrote:
> The size calculation adds an initial number of block groups needed
> to hold the inodes on the fs, then augments them with additional
> groups needed to reach the minimum number of data blocks, and then
> calculates the size of the last block group ( which should be a
> fraction of one block group ).  It was calculating a last bg size
> of several block groups because when adding the supplementary bgs,
> it only advanced the position of the start of the last group by one
> group, even though it was adding multiple groups.  Add the missing
> multiplier to move the last bg start position up to the correct 
> location, thus computing a proper fractional last bg size.

Bump.



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCgAGBQJVghcLAAoJENRVrw2cjl5Rc40H/3S52xJvxPV0f9cpXZT1qNLe
KEOi3O8B8FxQdfi3e3DjOk7OVl9t0wB+pob1QrOb8COp8wWEYr0dvQbq+fMLFOCx
8QehpD+q+0O9ZiptcNvhPEa65vNXTWE9Mo+nnK/eSPm+wXJ/XfGISIQfy7U0qzXP
ZJ0rsyqpFqkJKBOBIh3P9vxxaR+wDM+KuRERk0nc/shnH6Yg5Ud2PUzd+ZmGSYHY
DPxT0Gde3T/9whLOMBaD8NSTyqG5zPASBybqqqKHyeasAuWUqGEO8G79KsZvaeJ5
3LCG92vWO6wt7jQPzgf95Z6uj4CHVVufqA7DQwKKDgIE1JGFq7C76/pwfAranG0=
=nqrK
-----END PGP SIGNATURE-----