* [PATCH 01/14] misc: fix Coverity bugs
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
@ 2015-05-14 0:21 ` Darrick J. Wong
2015-05-16 22:36 ` Theodore Ts'o
2015-05-14 0:21 ` [PATCH 02/14] undo-io: write out index block after every write Darrick J. Wong
` (14 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:21 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Fix Coverity bugs 1297093,1297096, 1297489, 1297491, 1297493, 1297506,
1297507, 1297514, 1297516, and 1297517.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
debugfs/util.c | 2 +-
e2fsck/extents.c | 12 +++++++++---
e2fsck/readahead.c | 2 +-
e2fsck/unix.c | 4 ++++
lib/e2p/feature.c | 4 ++--
lib/e2p/mntopts.c | 2 +-
misc/e4defrag.c | 4 +++-
misc/logsave.c | 2 +-
8 files changed, 22 insertions(+), 10 deletions(-)
diff --git a/debugfs/util.c b/debugfs/util.c
index af14539..4fef89a 100644
--- a/debugfs/util.c
+++ b/debugfs/util.c
@@ -390,7 +390,7 @@ int common_block_args_process(int argc, char *argv[],
return 1;
if (*block == 0) {
com_err(argv[0], 0, "Invalid block number 0");
- err = 1;
+ return 1;
}
if (argc > 2) {
diff --git a/e2fsck/extents.c b/e2fsck/extents.c
index a08fa94..407dafb 100644
--- a/e2fsck/extents.c
+++ b/e2fsck/extents.c
@@ -27,6 +27,8 @@ static errcode_t e2fsck_rebuild_extents(e2fsck_t ctx, ext2_ino_t ino);
/* Schedule an inode to have its extent tree rebuilt during pass 1E. */
errcode_t e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino)
{
+ errcode_t retval = 0;
+
if (!EXT2_HAS_INCOMPAT_FEATURE(ctx->fs->super,
EXT3_FEATURE_INCOMPAT_EXTENTS) ||
(ctx->options & E2F_OPT_NO) ||
@@ -37,13 +39,15 @@ errcode_t e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino)
return e2fsck_rebuild_extents(ctx, ino);
if (!ctx->inodes_to_rebuild)
- e2fsck_allocate_inode_bitmap(ctx->fs,
+ retval = e2fsck_allocate_inode_bitmap(ctx->fs,
_("extent rebuild inode map"),
EXT2FS_BMAP64_RBTREE,
"inodes_to_rebuild",
&ctx->inodes_to_rebuild);
- if (ctx->inodes_to_rebuild)
- ext2fs_mark_inode_bitmap2(ctx->inodes_to_rebuild, ino);
+ if (retval)
+ return retval;
+
+ ext2fs_mark_inode_bitmap2(ctx->inodes_to_rebuild, ino);
return 0;
}
@@ -225,6 +229,8 @@ static errcode_t rebuild_extent_tree(e2fsck_t ctx, struct extent_list *list,
/* Collect lblk->pblk mappings */
if (inode.i_flags & EXT4_EXTENTS_FL) {
retval = load_extents(ctx, list);
+ if (retval)
+ goto err;
goto extents_loaded;
}
diff --git a/e2fsck/readahead.c b/e2fsck/readahead.c
index 4429a37..a860f2b 100644
--- a/e2fsck/readahead.c
+++ b/e2fsck/readahead.c
@@ -242,7 +242,7 @@ unsigned long long e2fsck_guess_readahead(ext2_filsys fs)
* worth of inode table blocks seems to yield the largest reductions
* in e2fsck runtime.
*/
- guess = 2 * fs->blocksize * fs->inode_blocks_per_group;
+ guess = 2ULL * fs->blocksize * fs->inode_blocks_per_group;
/* Disable RA if it'd use more 1/50th of RAM. */
if (get_memory_size() > (guess * 50))
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index 4bad553..940ecb4 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -682,6 +682,10 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
}
ctx->ext_attr_ver = ea_ver;
} else if (strcmp(token, "readahead_kb") == 0) {
+ if (!arg) {
+ extended_usage++;
+ continue;
+ }
reada_kb = strtoull(arg, &p, 0);
if (*p) {
fprintf(stderr, "%s",
diff --git a/lib/e2p/feature.c b/lib/e2p/feature.c
index 73884f2..d1d4762 100644
--- a/lib/e2p/feature.c
+++ b/lib/e2p/feature.c
@@ -187,7 +187,7 @@ int e2p_string2feature(char *string, int *compat_type, unsigned int *mask)
if (string[9] == 0)
return 1;
num = strtol(string+9, &eptr, 10);
- if (num > 32 || num < 0)
+ if (num > 31 || num < 0)
return 1;
if (*eptr)
return 1;
@@ -261,7 +261,7 @@ int e2p_jrnl_string2feature(char *string, int *compat_type, unsigned int *mask)
if (string[9] == 0)
return 1;
num = strtol(string+9, &eptr, 10);
- if (num > 32 || num < 0)
+ if (num > 31 || num < 0)
return 1;
if (*eptr)
return 1;
diff --git a/lib/e2p/mntopts.c b/lib/e2p/mntopts.c
index d56cc52..ff2e5de 100644
--- a/lib/e2p/mntopts.c
+++ b/lib/e2p/mntopts.c
@@ -72,7 +72,7 @@ int e2p_string2mntopt(char *string, unsigned int *mask)
if (string[8] == 0)
return 1;
num = strtol(string+8, &eptr, 10);
- if (num > 32 || num < 0)
+ if (num > 31 || num < 0)
return 1;
if (*eptr)
return 1;
diff --git a/misc/e4defrag.c b/misc/e4defrag.c
index d0eac60..3f949d0 100644
--- a/misc/e4defrag.c
+++ b/misc/e4defrag.c
@@ -387,8 +387,10 @@ static int page_in_core(int fd, struct move_extent defrag_data,
*page_num = 0;
*page_num = (length + pagesize - 1) / pagesize;
*vec = (unsigned char *)calloc(*page_num, 1);
- if (*vec == NULL)
+ if (*vec == NULL) {
+ munmap(page, length);
return -1;
+ }
/* Get information on whether pages are in core */
if (mincore(page, (size_t)length, *vec) == -1 ||
diff --git a/misc/logsave.c b/misc/logsave.c
index f6cc42a..6a624de 100644
--- a/misc/logsave.c
+++ b/misc/logsave.c
@@ -219,7 +219,7 @@ static int run_program(char **argv)
sprintf(buffer, "died with signal %d\n",
WTERMSIG(status));
send_output(buffer, 0, SEND_BOTH);
- rc = 1;
+ return 1;
}
rc = 0;
}
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH 01/14] misc: fix Coverity bugs
2015-05-14 0:21 ` [PATCH 01/14] misc: fix Coverity bugs Darrick J. Wong
@ 2015-05-16 22:36 ` Theodore Ts'o
0 siblings, 0 replies; 39+ messages in thread
From: Theodore Ts'o @ 2015-05-16 22:36 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Wed, May 13, 2015 at 05:21:15PM -0700, Darrick J. Wong wrote:
> Fix Coverity bugs 1297093,1297096, 1297489, 1297491, 1297493, 1297506,
> 1297507, 1297514, 1297516, and 1297517.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Thanks, applied.
- Ted
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH 02/14] undo-io: write out index block after every write
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
2015-05-14 0:21 ` [PATCH 01/14] misc: fix Coverity bugs Darrick J. Wong
@ 2015-05-14 0:21 ` Darrick J. Wong
2015-05-17 0:18 ` Theodore Ts'o
2015-05-14 0:21 ` [PATCH 03/14] misc: fix undo file setup Darrick J. Wong
` (13 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:21 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Write out the undo file's index block after writing a block to the
undo file. This ensures that we always have a consistent undo file
in the page cache, even if the program crashes. When we fill up a
key block in the undo file, we'll call fsync to force the whole
thing to storage; this should happen about every 256 blocks given
the usual 4K block size.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/undo_io.c | 47 ++++++++++++++++++++++-------------------------
1 file changed, 22 insertions(+), 25 deletions(-)
diff --git a/lib/ext2fs/undo_io.c b/lib/ext2fs/undo_io.c
index 4a48193..b9294c3 100644
--- a/lib/ext2fs/undo_io.c
+++ b/lib/ext2fs/undo_io.c
@@ -165,7 +165,7 @@ errcode_t set_undo_io_backup_file(char *file_name)
return 0;
}
-static errcode_t write_undo_indexes(struct undo_private_data *data)
+static errcode_t write_undo_indexes(struct undo_private_data *data, int flush)
{
errcode_t retval;
struct ext2_super_block super;
@@ -187,9 +187,14 @@ static errcode_t write_undo_indexes(struct undo_private_data *data)
1, data->keyb);
if (retval)
return retval;
- memset(data->keyb, 0, data->tdb_data_size);
- data->keys_in_block = 0;
- data->key_blk_num = data->undo_blk_num;
+ /* Move on to the next key block if it's full. */
+ if (data->keys_in_block == KEYS_PER_BLOCK(data)) {
+ memset(data->keyb, 0, data->tdb_data_size);
+ data->keys_in_block = 0;
+ data->key_blk_num = data->undo_blk_num;
+ data->undo_blk_num++;
+ flush = 1;
+ }
}
/* Prepare superblock for write */
@@ -230,7 +235,8 @@ static errcode_t write_undo_indexes(struct undo_private_data *data)
if (retval)
goto err_out;
- retval = io_channel_flush(data->undo_file);
+ if (flush)
+ retval = io_channel_flush(data->undo_file);
err_out:
io_channel_set_blksize(channel, block_size);
return retval;
@@ -261,7 +267,7 @@ static errcode_t undo_setup_tdb(struct undo_private_data *data)
retval = ext2fs_get_mem(data->tdb_data_size, &data->keyb);
if (retval)
return retval;
- data->key_blk_num = data->undo_blk_num;
+ data->key_blk_num = data->first_key_blk;
/* Record block size */
dbg_printf("Undo block size %llu\n", data->tdb_data_size);
@@ -348,22 +354,6 @@ static errcode_t undo_write_tdb(io_channel channel,
}
ext2fs_mark_block_bitmap2(data->written_block_map, block_num);
- /* Spit out a key block */
- if (data->keys_in_block == KEYS_PER_BLOCK(data)) {
- retval = write_undo_indexes(data);
- if (retval)
- return retval;
- retval = io_channel_write_blk64(data->undo_file,
- data->key_blk_num, 1,
- data->keyb);
- if (retval)
- return retval;
- }
-
- /* Allocate new key block */
- if (data->keys_in_block == 0)
- data->undo_blk_num++;
-
/*
* Read one block using the backing I/O manager
* The backing I/O manager block size may be
@@ -458,6 +448,12 @@ static errcode_t undo_write_tdb(io_channel channel,
}
data->undo_blk_num++;
free(read_ptr);
+
+ /* Write out the key block */
+ retval = write_undo_indexes(data, 0);
+ if (retval)
+ return retval;
+
/* Next block */
block_num++;
}
@@ -668,7 +664,7 @@ static void undo_atexit(void *p)
struct undo_private_data *data = p;
errcode_t err;
- err = write_undo_indexes(data);
+ err = write_undo_indexes(data, 1);
io_channel_close(data->undo_file);
com_err(data->tdb_file, err, "while force-closing undo file");
@@ -707,7 +703,8 @@ static errcode_t undo_open(const char *name, int flags, io_channel *channel)
memset(data, 0, sizeof(struct undo_private_data));
data->magic = EXT2_ET_MAGIC_UNIX_IO_CHANNEL;
data->super_blk_num = 1;
- data->undo_blk_num = data->first_key_blk = 2;
+ data->first_key_blk = 2;
+ data->undo_blk_num = 3;
if (undo_io_backing_manager) {
retval = undo_io_backing_manager->open(name, flags,
@@ -789,7 +786,7 @@ static errcode_t undo_close(io_channel channel)
/* Before closing write the file system identity */
if (!getenv("UNDO_IO_SIMULATE_UNFINISHED"))
data->hdr.state = ext2fs_cpu_to_le32(E2UNDO_STATE_FINISHED);
- err = write_undo_indexes(data);
+ err = write_undo_indexes(data, 1);
ext2fs_remove_exit_fn(undo_atexit, data);
if (data->real)
retval = io_channel_close(data->real);
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH 02/14] undo-io: write out index block after every write
2015-05-14 0:21 ` [PATCH 02/14] undo-io: write out index block after every write Darrick J. Wong
@ 2015-05-17 0:18 ` Theodore Ts'o
0 siblings, 0 replies; 39+ messages in thread
From: Theodore Ts'o @ 2015-05-17 0:18 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Wed, May 13, 2015 at 05:21:21PM -0700, Darrick J. Wong wrote:
> Write out the undo file's index block after writing a block to the
> undo file. This ensures that we always have a consistent undo file
> in the page cache, even if the program crashes. When we fill up a
> key block in the undo file, we'll call fsync to force the whole
> thing to storage; this should happen about every 256 blocks given
> the usual 4K block size.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Applied, but I'm not sure calling fsync every 256 blocks is worth it
at all; when mke2fs is writing a 128M journal, it's going to be
writing 32,768 blocks. If we call fsync every 256 blocks, then we'll
be calling it 128 times during the course of the mke2fs, and this
doesn't count any of the other blocks overwritten by mke2fs.
The fsync() is only going to help in case of power failure --- and if
we only calling it every 256 blocks, we're going to lose data in
anyway.
This patch doesn't appear to change how often we call fsync(2), so
I'll take this, but I think we should seriously think about only
calling fync() if we're closing the undo file.
- Ted
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH 03/14] misc: fix undo file setup
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
2015-05-14 0:21 ` [PATCH 01/14] misc: fix Coverity bugs Darrick J. Wong
2015-05-14 0:21 ` [PATCH 02/14] undo-io: write out index block after every write Darrick J. Wong
@ 2015-05-14 0:21 ` Darrick J. Wong
2015-05-17 0:20 ` Theodore Ts'o
2015-05-14 0:21 ` [PATCH 04/14] filefrag: fix broken extent emulation and uninitialized variables Darrick J. Wong
` (12 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:21 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Fix Coverity bugs 1297094-1297101 by fixing all the mutations in the
*_setup_tdb() functions, fixing buffer overflows, and checking
return values.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
debugfs/debugfs.c | 29 ++++++++++++------------
e2fsck/unix.c | 12 +++++++---
misc/e2undo.c | 64 +++++++++++++++++++++++++++++++++--------------------
misc/mke2fs.c | 12 +++++++---
misc/tune2fs.c | 63 +++++++++++++++++++++++++++++-----------------------
resize/main.c | 29 ++++++++++++------------
6 files changed, 121 insertions(+), 88 deletions(-)
diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
index 4b88f73..c677f5f 100644
--- a/debugfs/debugfs.c
+++ b/debugfs/debugfs.c
@@ -55,11 +55,12 @@ static int debugfs_setup_tdb(const char *device_name, char *undo_file,
errcode_t retval = ENOMEM;
char *tdb_dir = NULL, *tdb_file = NULL;
char *dev_name, *tmp_name;
- int free_tdb_dir = 0;
/* (re)open a specific undo file */
if (undo_file && undo_file[0] != 0) {
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto err;
*io_ptr = undo_io_manager;
retval = set_undo_io_backup_file(undo_file);
if (retval)
@@ -68,7 +69,7 @@ static int debugfs_setup_tdb(const char *device_name, char *undo_file,
"using the command:\n"
" e2undo %s %s\n\n",
undo_file, device_name);
- return 0;
+ return retval;
}
/*
@@ -76,19 +77,18 @@ static int debugfs_setup_tdb(const char *device_name, char *undo_file,
* nice
*/
tdb_dir = getenv("E2FSPROGS_UNDO_DIR");
+ if (!tdb_dir)
+ tdb_dir = "/var/lib/e2fsprogs";
- if (tdb_dir == NULL || !strcmp(tdb_dir, "none") || (tdb_dir[0] == 0) ||
- access(tdb_dir, W_OK)) {
- if (free_tdb_dir)
- free(tdb_dir);
+ if (!strcmp(tdb_dir, "none") || (tdb_dir[0] == 0) ||
+ access(tdb_dir, W_OK))
return 0;
- }
tmp_name = strdup(device_name);
if (!tmp_name)
goto errout;
dev_name = basename(tmp_name);
- tdb_file = malloc(strlen(tdb_dir) + 8 + strlen(dev_name) + 7 + 1);
+ tdb_file = malloc(strlen(tdb_dir) + 9 + strlen(dev_name) + 7 + 1);
if (!tdb_file) {
free(tmp_name);
goto errout;
@@ -98,10 +98,14 @@ static int debugfs_setup_tdb(const char *device_name, char *undo_file,
if ((unlink(tdb_file) < 0) && (errno != ENOENT)) {
retval = errno;
+ com_err("debugfs", retval,
+ "while trying to delete %s", tdb_file);
goto errout;
}
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto errout;
*io_ptr = undo_io_manager;
retval = set_undo_io_backup_file(tdb_file);
if (retval)
@@ -110,14 +114,9 @@ static int debugfs_setup_tdb(const char *device_name, char *undo_file,
"using the command:\n"
" e2undo %s %s\n\n", tdb_file, device_name);
- if (free_tdb_dir)
- free(tdb_dir);
free(tdb_file);
return 0;
-
errout:
- if (free_tdb_dir)
- free(tdb_dir);
free(tdb_file);
err:
com_err("debugfs", retval, "while trying to setup undo file\n");
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index 940ecb4..9ef4b1e 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -1242,7 +1242,9 @@ static int e2fsck_setup_tdb(e2fsck_t ctx, io_manager *io_ptr)
/* (re)open a specific undo file */
if (ctx->undo_file && ctx->undo_file[0] != 0) {
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto err;
*io_ptr = undo_io_manager;
retval = set_undo_io_backup_file(ctx->undo_file);
if (retval)
@@ -1251,7 +1253,7 @@ static int e2fsck_setup_tdb(e2fsck_t ctx, io_manager *io_ptr)
"using the command:\n"
" e2undo %s %s\n\n"),
ctx->undo_file, ctx->filesystem_name);
- return 0;
+ return retval;
}
/*
@@ -1287,10 +1289,14 @@ static int e2fsck_setup_tdb(e2fsck_t ctx, io_manager *io_ptr)
if ((unlink(tdb_file) < 0) && (errno != ENOENT)) {
retval = errno;
+ com_err(ctx->program_name, retval,
+ _("while trying to delete %s"), tdb_file);
goto errout;
}
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto errout;
*io_ptr = undo_io_manager;
retval = set_undo_io_backup_file(tdb_file);
if (retval)
diff --git a/misc/e2undo.c b/misc/e2undo.c
index 3f312c6..6123c48 100644
--- a/misc/e2undo.c
+++ b/misc/e2undo.c
@@ -204,29 +204,29 @@ static int e2undo_setup_tdb(const char *name, io_manager *io_ptr)
{
errcode_t retval = 0;
const char *tdb_dir;
- char *tdb_file;
+ char *tdb_file = NULL;
char *dev_name, *tmp_name;
/* (re)open a specific undo file */
if (undo_file && undo_file[0] != 0) {
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto err;
*io_ptr = undo_io_manager;
- set_undo_io_backup_file(undo_file);
- printf(_("To undo the e2undo operation please run "
- "the command\n e2undo %s %s\n\n"),
+ retval = set_undo_io_backup_file(undo_file);
+ if (retval)
+ goto err;
+ printf(_("Overwriting existing filesystem; this can be undone "
+ "using the command:\n"
+ " e2undo %s %s\n\n"),
undo_file, name);
return retval;
}
- tmp_name = strdup(name);
- if (!tmp_name) {
- alloc_fn_fail:
- com_err(prg_name, ENOMEM, "%s",
- _("Couldn't allocate memory for tdb filename\n"));
- return ENOMEM;
- }
- dev_name = basename(tmp_name);
-
+ /*
+ * Configuration via a conf file would be
+ * nice
+ */
tdb_dir = getenv("E2FSPROGS_UNDO_DIR");
if (!tdb_dir)
tdb_dir = "/var/lib/e2fsprogs";
@@ -235,27 +235,43 @@ static int e2undo_setup_tdb(const char *name, io_manager *io_ptr)
access(tdb_dir, W_OK))
return 0;
- tdb_file = malloc(strlen(tdb_dir) + 9 + strlen(dev_name) + 7 + 1);
- if (!tdb_file)
- goto alloc_fn_fail;
+ tmp_name = strdup(name);
+ if (!tmp_name)
+ goto errout;
+ dev_name = basename(tmp_name);
+ tdb_file = malloc(strlen(tdb_dir) + 8 + strlen(dev_name) + 7 + 1);
+ if (!tdb_file) {
+ free(tmp_name);
+ goto errout;
+ }
sprintf(tdb_file, "%s/e2undo-%s.e2undo", tdb_dir, dev_name);
+ free(tmp_name);
if ((unlink(tdb_file) < 0) && (errno != ENOENT)) {
retval = errno;
com_err(prg_name, retval,
_("while trying to delete %s"), tdb_file);
- free(tdb_file);
- return retval;
+ goto errout;
}
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto errout;
*io_ptr = undo_io_manager;
- set_undo_io_backup_file(tdb_file);
- printf(_("To undo the e2undo operation please run "
- "the command\n e2undo %s %s\n\n"),
+ retval = set_undo_io_backup_file(tdb_file);
+ if (retval)
+ goto errout;
+ printf(_("Overwriting existing filesystem; this can be undone "
+ "using the command:\n"
+ " e2undo %s %s\n\n"),
tdb_file, name);
+
free(tdb_file);
- free(tmp_name);
+ return 0;
+errout:
+ free(tdb_file);
+err:
+ com_err(prg_name, retval, "while trying to setup undo file\n");
return retval;
}
diff --git a/misc/mke2fs.c b/misc/mke2fs.c
index 05a16d6..78b1252 100644
--- a/misc/mke2fs.c
+++ b/misc/mke2fs.c
@@ -2500,7 +2500,9 @@ static int mke2fs_setup_tdb(const char *name, io_manager *io_ptr)
/* (re)open a specific undo file */
if (undo_file && undo_file[0] != 0) {
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto err;
*io_ptr = undo_io_manager;
retval = set_undo_io_backup_file(undo_file);
if (retval)
@@ -2508,7 +2510,7 @@ static int mke2fs_setup_tdb(const char *name, io_manager *io_ptr)
printf(_("Overwriting existing filesystem; this can be undone "
"using the command:\n"
" e2undo %s %s\n\n"), undo_file, name);
- return 0;
+ return retval;
}
/*
@@ -2544,10 +2546,14 @@ static int mke2fs_setup_tdb(const char *name, io_manager *io_ptr)
if ((unlink(tdb_file) < 0) && (errno != ENOENT)) {
retval = errno;
+ com_err(program_name, retval,
+ _("while trying to delete %s"), tdb_file);
goto errout;
}
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto errout;
*io_ptr = undo_io_manager;
retval = set_undo_io_backup_file(tdb_file);
if (retval)
diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index f97ec25..d2e8b20 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -2529,38 +2529,29 @@ static int tune2fs_setup_tdb(const char *name, io_manager *io_ptr)
{
errcode_t retval = 0;
const char *tdb_dir;
- char *tdb_file;
+ char *tdb_file = NULL;
char *dev_name, *tmp_name;
/* (re)open a specific undo file */
if (undo_file && undo_file[0] != 0) {
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto err;
*io_ptr = undo_io_manager;
- set_undo_io_backup_file(undo_file);
- printf(_("To undo the tune2fs operation please run "
- "the command\n e2undo %s %s\n\n"),
+ retval = set_undo_io_backup_file(undo_file);
+ if (retval)
+ goto err;
+ printf(_("Overwriting existing filesystem; this can be undone "
+ "using the command:\n"
+ " e2undo %s %s\n\n"),
undo_file, name);
return retval;
}
-#if 0 /* FIXME!! */
/*
* Configuration via a conf file would be
* nice
*/
- profile_get_string(profile, "scratch_files",
- "directory", 0, 0,
- &tdb_dir);
-#endif
- tmp_name = strdup(name);
- if (!tmp_name) {
- alloc_fn_fail:
- com_err(program_name, ENOMEM, "%s",
- _("Couldn't allocate memory for tdb filename\n"));
- return ENOMEM;
- }
- dev_name = basename(tmp_name);
-
tdb_dir = getenv("E2FSPROGS_UNDO_DIR");
if (!tdb_dir)
tdb_dir = "/var/lib/e2fsprogs";
@@ -2569,27 +2560,43 @@ static int tune2fs_setup_tdb(const char *name, io_manager *io_ptr)
access(tdb_dir, W_OK))
return 0;
+ tmp_name = strdup(name);
+ if (!tmp_name)
+ goto errout;
+ dev_name = basename(tmp_name);
tdb_file = malloc(strlen(tdb_dir) + 9 + strlen(dev_name) + 7 + 1);
- if (!tdb_file)
- goto alloc_fn_fail;
+ if (!tdb_file) {
+ free(tmp_name);
+ goto errout;
+ }
sprintf(tdb_file, "%s/tune2fs-%s.e2undo", tdb_dir, dev_name);
+ free(tmp_name);
if ((unlink(tdb_file) < 0) && (errno != ENOENT)) {
retval = errno;
com_err(program_name, retval,
_("while trying to delete %s"), tdb_file);
- free(tdb_file);
- return retval;
+ goto errout;
}
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto errout;
*io_ptr = undo_io_manager;
- set_undo_io_backup_file(tdb_file);
- printf(_("To undo the tune2fs operation please run "
- "the command\n e2undo %s %s\n\n"),
+ retval = set_undo_io_backup_file(tdb_file);
+ if (retval)
+ goto errout;
+ printf(_("Overwriting existing filesystem; this can be undone "
+ "using the command:\n"
+ " e2undo %s %s\n\n"),
tdb_file, name);
+
free(tdb_file);
- free(tmp_name);
+ return 0;
+errout:
+ free(tdb_file);
+err:
+ com_err("tune2fs", retval, "while trying to setup undo file\n");
return retval;
}
diff --git a/resize/main.c b/resize/main.c
index a61943e..9da3a95 100644
--- a/resize/main.c
+++ b/resize/main.c
@@ -170,11 +170,12 @@ static int resize2fs_setup_tdb(const char *device_name, char *undo_file,
errcode_t retval = ENOMEM;
char *tdb_dir = NULL, *tdb_file = NULL;
char *dev_name, *tmp_name;
- int free_tdb_dir = 0;
/* (re)open a specific undo file */
if (undo_file && undo_file[0] != 0) {
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto err;
*io_ptr = undo_io_manager;
retval = set_undo_io_backup_file(undo_file);
if (retval)
@@ -183,7 +184,7 @@ static int resize2fs_setup_tdb(const char *device_name, char *undo_file,
"using the command:\n"
" e2undo %s %s\n\n"),
undo_file, device_name);
- return 0;
+ return retval;
}
/*
@@ -191,19 +192,18 @@ static int resize2fs_setup_tdb(const char *device_name, char *undo_file,
* nice
*/
tdb_dir = getenv("E2FSPROGS_UNDO_DIR");
+ if (!tdb_dir)
+ tdb_dir = "/var/lib/e2fsprogs";
- if (tdb_dir == NULL || !strcmp(tdb_dir, "none") || (tdb_dir[0] == 0) ||
- access(tdb_dir, W_OK)) {
- if (free_tdb_dir)
- free(tdb_dir);
+ if (!strcmp(tdb_dir, "none") || (tdb_dir[0] == 0) ||
+ access(tdb_dir, W_OK))
return 0;
- }
tmp_name = strdup(device_name);
if (!tmp_name)
goto errout;
dev_name = basename(tmp_name);
- tdb_file = malloc(strlen(tdb_dir) + 8 + strlen(dev_name) + 7 + 1);
+ tdb_file = malloc(strlen(tdb_dir) + 11 + strlen(dev_name) + 7 + 1);
if (!tdb_file) {
free(tmp_name);
goto errout;
@@ -213,10 +213,14 @@ static int resize2fs_setup_tdb(const char *device_name, char *undo_file,
if ((unlink(tdb_file) < 0) && (errno != ENOENT)) {
retval = errno;
+ com_err(program_name, retval,
+ _("while trying to delete %s"), tdb_file);
goto errout;
}
- set_undo_io_backing_manager(*io_ptr);
+ retval = set_undo_io_backing_manager(*io_ptr);
+ if (retval)
+ goto errout;
*io_ptr = undo_io_manager;
retval = set_undo_io_backup_file(tdb_file);
if (retval)
@@ -225,14 +229,9 @@ static int resize2fs_setup_tdb(const char *device_name, char *undo_file,
"using the command:\n"
" e2undo %s %s\n\n"), tdb_file, device_name);
- if (free_tdb_dir)
- free(tdb_dir);
free(tdb_file);
return 0;
^ permalink raw reply related [flat|nested] 39+ messages in thread* [PATCH 04/14] filefrag: fix broken extent emulation and uninitialized variables
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (2 preceding siblings ...)
2015-05-14 0:21 ` [PATCH 03/14] misc: fix undo file setup Darrick J. Wong
@ 2015-05-14 0:21 ` Darrick J. Wong
2015-05-17 0:26 ` Theodore Ts'o
2015-05-14 0:21 ` [PATCH 05/14] e2fsck: fix buffer overrun in revoke block scanning Darrick J. Wong
` (11 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:21 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
This started with the fm_ext being uninitialized, but upon closer
analysis I discovered that forcing extent emulation in FIBMAP mode
was reporting an extent for every block in the file. Fix both
problems.
The Coverity bug was 1297512.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
misc/filefrag.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/misc/filefrag.c b/misc/filefrag.c
index c1a8684..3b104c6 100644
--- a/misc/filefrag.c
+++ b/misc/filefrag.c
@@ -287,8 +287,8 @@ static int filefrag_fibmap(int fd, int blk_shift, int *num_extents,
const long bpib = st->st_blksize / 4;
int count;
+ memset(&fm_ext, 0, sizeof(fm_ext));
if (force_extent) {
- memset(&fm_ext, 0, sizeof(fm_ext));
fm_ext.fe_flags = FIEMAP_EXTENT_MERGED;
}
@@ -331,15 +331,17 @@ static int filefrag_fibmap(int fd, int blk_shift, int *num_extents,
blk_shift, st);
fm_ext.fe_length = 0;
(*num_extents)++;
+ fm_ext.fe_logical = logical;
+ fm_ext.fe_physical = block * st->st_blksize;
} else if (last_block && (block != last_block + 1)) {
if (verbose)
printf("Discontinuity: Block %ld is at %lu (was "
"%lu)\n", i, block, last_block + 1);
fm_ext.fe_length = 0;
(*num_extents)++;
+ fm_ext.fe_logical = logical;
+ fm_ext.fe_physical = block * st->st_blksize;
}
- fm_ext.fe_logical = logical;
- fm_ext.fe_physical = block * st->st_blksize;
fm_ext.fe_length += st->st_blksize;
last_block = block;
}
^ permalink raw reply related [flat|nested] 39+ messages in thread* [PATCH 05/14] e2fsck: fix buffer overrun in revoke block scanning
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (3 preceding siblings ...)
2015-05-14 0:21 ` [PATCH 04/14] filefrag: fix broken extent emulation and uninitialized variables Darrick J. Wong
@ 2015-05-14 0:21 ` Darrick J. Wong
2015-05-14 19:37 ` [PATCH v2 " Darrick J. Wong
2015-05-14 0:21 ` [PATCH 06/14] e2fsck: convert block-mapped files to extents on bigalloc fs Darrick J. Wong
` (10 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:21 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Check the value of r_count to ensure that we never try to read revoke
records past the end of the revoke block. It turns out that the
journal writing code in debugfs was also playing fast and loose with
the r_count, so fix that as well.
The Coverity bug was 1297508.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
debugfs/do_journal.c | 19 +++++++++++--------
e2fsck/recovery.c | 10 +++++++++-
e2fsck/revoke.c | 26 ++++++++++++++------------
tests/j_corrupt_revoke_rcount/expect.1 | 8 ++++++++
tests/j_corrupt_revoke_rcount/expect.2 | 8 ++++++++
tests/j_corrupt_revoke_rcount/image.gz | Bin
tests/j_corrupt_revoke_rcount/name | 1 +
7 files changed, 51 insertions(+), 21 deletions(-)
create mode 100644 tests/j_corrupt_revoke_rcount/expect.1
create mode 100644 tests/j_corrupt_revoke_rcount/expect.2
create mode 100644 tests/j_corrupt_revoke_rcount/image.gz
create mode 100644 tests/j_corrupt_revoke_rcount/name
diff --git a/debugfs/do_journal.c b/debugfs/do_journal.c
index 46d1793..bdb0440 100644
--- a/debugfs/do_journal.c
+++ b/debugfs/do_journal.c
@@ -175,7 +175,7 @@ static errcode_t journal_add_revoke_to_trans(journal_transaction_t *trans,
void *buf;
size_t i, offset;
blk64_t curr_blk;
- int csum_size = 0;
+ int sz, csum_size = 0;
struct buffer_head *bh;
errcode_t err;
@@ -204,9 +204,15 @@ static errcode_t journal_add_revoke_to_trans(journal_transaction_t *trans,
jrb->r_header.h_sequence = ext2fs_cpu_to_be32(trans->tid);
offset = sizeof(*jrb);
+ if (JFS_HAS_INCOMPAT_FEATURE(trans->journal,
+ JFS_FEATURE_INCOMPAT_64BIT))
+ sz = 8;
+ else
+ sz = 4;
+
for (i = 0; i < revoke_len; i++) {
/* Block full, write to journal */
- if (offset > trans->journal->j_blocksize - csum_size) {
+ if (offset + sz >= trans->journal->j_blocksize - csum_size) {
jrb->r_count = ext2fs_cpu_to_be32(offset);
jbd2_revoke_csum_set(trans->journal, bh);
@@ -233,16 +239,13 @@ static errcode_t journal_add_revoke_to_trans(journal_transaction_t *trans,
}
if (JFS_HAS_INCOMPAT_FEATURE(trans->journal,
- JFS_FEATURE_INCOMPAT_64BIT)) {
+ JFS_FEATURE_INCOMPAT_64BIT))
*((__u64 *)(&((char *)buf)[offset])) =
ext2fs_cpu_to_be64(revoke_list[i]);
- offset += 8;
-
- } else {
+ else
*((__u32 *)(&((char *)buf)[offset])) =
ext2fs_cpu_to_be32(revoke_list[i]);
- offset += 4;
- }
+ offset += sz;
}
if (offset > 0) {
diff --git a/e2fsck/recovery.c b/e2fsck/recovery.c
index b5ce3b3..d5244be 100644
--- a/e2fsck/recovery.c
+++ b/e2fsck/recovery.c
@@ -839,15 +839,23 @@ static int scan_revoke_records(journal_t *journal, struct buffer_head *bh,
{
journal_revoke_header_t *header;
int offset, max;
+ int csum_size = 0;
+ __u32 rcount;
int record_len = 4;
header = (journal_revoke_header_t *) bh->b_data;
offset = sizeof(journal_revoke_header_t);
- max = ext2fs_be32_to_cpu(header->r_count);
+ rcount = ext2fs_be32_to_cpu(header->r_count);
if (!jbd2_revoke_block_csum_verify(journal, header))
return -EINVAL;
+ if (journal_has_csum_v2or3(journal))
+ csum_size = sizeof(struct journal_revoke_tail);
+ if (rcount > journal->j_blocksize - csum_size)
+ return -EINVAL;
+ max = rcount;
+
if (JFS_HAS_INCOMPAT_FEATURE(journal, JFS_FEATURE_INCOMPAT_64BIT))
record_len = 8;
diff --git a/e2fsck/revoke.c b/e2fsck/revoke.c
index b4c3f5f..fa71b26 100644
--- a/e2fsck/revoke.c
+++ b/e2fsck/revoke.c
@@ -583,7 +583,7 @@ static void write_one_revoke_record(journal_t *journal,
{
int csum_size = 0;
struct buffer_head *descriptor;
- int offset;
+ int sz, offset;
journal_header_t *header;
/* If we are already aborting, this all becomes a noop. We
@@ -600,9 +600,14 @@ static void write_one_revoke_record(journal_t *journal,
if (journal_has_csum_v2or3(journal))
csum_size = sizeof(struct journal_revoke_tail);
+ if (JBD2_HAS_INCOMPAT_FEATURE(journal, JBD2_FEATURE_INCOMPAT_64BIT))
+ sz = 8;
+ else
+ sz = 4;
+
/* Make sure we have a descriptor with space left for the record */
if (descriptor) {
- if (offset >= journal->j_blocksize - csum_size) {
+ if (offset + sz >= journal->j_blocksize - csum_size) {
flush_descriptor(journal, descriptor, offset, write_op);
descriptor = NULL;
}
@@ -625,16 +630,13 @@ static void write_one_revoke_record(journal_t *journal,
*descriptorp = descriptor;
}
- if (JFS_HAS_INCOMPAT_FEATURE(journal, JFS_FEATURE_INCOMPAT_64BIT)) {
- * ((__u64 *)(&descriptor->b_data[offset])) =
- ext2fs_cpu_to_be64(record->blocknr);
- offset += 8;
-
- } else {
- * ((__u32 *)(&descriptor->b_data[offset])) =
- ext2fs_cpu_to_be32(record->blocknr);
- offset += 4;
- }
+ if (JBD2_HAS_INCOMPAT_FEATURE(journal, JBD2_FEATURE_INCOMPAT_64BIT)) {
+ * ((__be64 *)(&descriptor->b_data[offset])) =
+ cpu_to_be64(record->blocknr);
+ else
+ * ((__be32 *)(&descriptor->b_data[offset])) =
+ cpu_to_be32(record->blocknr);
+ offset += sz;
*offsetp = offset;
}
diff --git a/tests/j_corrupt_revoke_rcount/expect.1 b/tests/j_corrupt_revoke_rcount/expect.1
new file mode 100644
index 0000000..97324f3
--- /dev/null
+++ b/tests/j_corrupt_revoke_rcount/expect.1
@@ -0,0 +1,8 @@
+test_filesys: recovering journal
+../e2fsck/e2fsck: Invalid argument while recovering ext3 journal of test_filesys
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+
+test_filesys: ********** WARNING: Filesystem still has errors **********
+
+Exit status is 12
diff --git a/tests/j_corrupt_revoke_rcount/expect.2 b/tests/j_corrupt_revoke_rcount/expect.2
new file mode 100644
index 0000000..c569901
--- /dev/null
+++ b/tests/j_corrupt_revoke_rcount/expect.2
@@ -0,0 +1,8 @@
+test_filesys: recovering journal
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/512 files (9.1% non-contiguous), 1066/2048 blocks
+Exit status is 0
diff --git a/tests/j_corrupt_revoke_rcount/image.gz b/tests/j_corrupt_revoke_rcount/image.gz
new file mode 100644
index 0000000000000000000000000000000000000000..c8b19e8c4ebe28f9e368635fa3962d4f0685d762
GIT binary patch
literal 8648
zcmb2|=3v+~Hz<^e`ORJ1Y!OEZh6gkEb{tnNI+!5H;gZYzQpaVY_a%+8i8nG-Cpk51
z{>jKmO4PiOIZ^3am{^&Z=VN_=i}DJBE;>oCioccd>nF0BO1z)+-16Ss+k0iF&6IwY
zRyO^%K!dZkcOZYFq3Zi=#hl~wXMg6<PE9%bO@H-gZpAIP!}lGX@P60DNc}01hYNNV
z*{S9S?`zkbTlMGHwX5Hb3hS?n-T!gp&&<sIUE%L5|9?63HaPpe-rh>Hn^mQiMc-0;
z)*1v&)7E!SUM<#E^(}wO&(B+CCg~il=sFwXUp&=4`)$N*kM$87!|QsIqvRME{%8KE
zw>kRq+>d(eG+X5dZ!4a4N-;4oG#tGCj(w|hTpa@g!;j;iUjJ8COuM{rhj@ZY%FQn&
zh5YZIrzuBA?W@kuD*X9x?(B0@f$9z{kh*93|NlLmXa7MwM&ngnAo`CykTm$u4kUlw
z{IzrQdFK~)U-Gy7Hv8rh{yhGd_PockSCxR0JV*Zg_b}|w-`>4;<FYiMz=H#4|8M><
z`}#-mXW{0bOIH1R(0q#-D2@j_SdbxqHyPyU1@A4Hf&L&D9GHJ2Vs8A4V~JYdug}~f
zu~w^I_lkAq#+;m^23IEQmwmgjyR+-PV>ZjT!>1N+iO3KAQor%d&wuyctowSS{crs8
zKMQyLd6<&_KmPW`|K_LsKf$8l%VLA&|I;1ToG+PG@A~_E*R%iKAI}F!9<wLcoum3j
zLtr!nMnhmU1V%$(Gz3ONU^E0qLtr!nMnhmU1V%$(Gz3ONU^E0qLtr!nMnhmU1V%$(
zGz3ONU^E0qLtr!nMnhmU1V%$(XorCPt!8KN1i$0|_}Mj#V9#)OmhZI$E?!^&01SK{
AN&o-=
literal 0
HcmV?d00001
diff --git a/tests/j_corrupt_revoke_rcount/name b/tests/j_corrupt_revoke_rcount/name
new file mode 100644
index 0000000..92b523e
--- /dev/null
+++ b/tests/j_corrupt_revoke_rcount/name
@@ -0,0 +1 @@
+corrupt revoke r_count buffer overflow
^ permalink raw reply related [flat|nested] 39+ messages in thread* [PATCH v2 05/14] e2fsck: fix buffer overrun in revoke block scanning
2015-05-14 0:21 ` [PATCH 05/14] e2fsck: fix buffer overrun in revoke block scanning Darrick J. Wong
@ 2015-05-14 19:37 ` Darrick J. Wong
2015-05-17 0:50 ` Theodore Ts'o
0 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 19:37 UTC (permalink / raw)
To: tytso; +Cc: linux-ext4
Check the value of r_count to ensure that we never try to read revoke
records past the end of the revoke block. It turns out that the
journal writing code in debugfs was also playing fast and loose with
the r_count, so fix that as well.
The Coverity bug was 1297508.
v2: Allow the revoke block to use all slots, including the last
one.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
debugfs/do_journal.c | 19 +++++++++++--------
e2fsck/recovery.c | 10 +++++++++-
e2fsck/revoke.c | 26 ++++++++++++++------------
tests/j_corrupt_revoke_rcount/expect.1 | 8 ++++++++
tests/j_corrupt_revoke_rcount/expect.2 | 8 ++++++++
tests/j_corrupt_revoke_rcount/image.gz | Bin
tests/j_corrupt_revoke_rcount/name | 1 +
7 files changed, 51 insertions(+), 21 deletions(-)
create mode 100644 tests/j_corrupt_revoke_rcount/expect.1
create mode 100644 tests/j_corrupt_revoke_rcount/expect.2
create mode 100644 tests/j_corrupt_revoke_rcount/image.gz
create mode 100644 tests/j_corrupt_revoke_rcount/name
diff --git a/debugfs/do_journal.c b/debugfs/do_journal.c
index 46d1793..22322e5 100644
--- a/debugfs/do_journal.c
+++ b/debugfs/do_journal.c
@@ -175,7 +175,7 @@ static errcode_t journal_add_revoke_to_trans(journal_transaction_t *trans,
void *buf;
size_t i, offset;
blk64_t curr_blk;
- int csum_size = 0;
+ int sz, csum_size = 0;
struct buffer_head *bh;
errcode_t err;
@@ -204,9 +204,15 @@ static errcode_t journal_add_revoke_to_trans(journal_transaction_t *trans,
jrb->r_header.h_sequence = ext2fs_cpu_to_be32(trans->tid);
offset = sizeof(*jrb);
+ if (JFS_HAS_INCOMPAT_FEATURE(trans->journal,
+ JFS_FEATURE_INCOMPAT_64BIT))
+ sz = 8;
+ else
+ sz = 4;
+
for (i = 0; i < revoke_len; i++) {
/* Block full, write to journal */
- if (offset > trans->journal->j_blocksize - csum_size) {
+ if (offset + sz > trans->journal->j_blocksize - csum_size) {
jrb->r_count = ext2fs_cpu_to_be32(offset);
jbd2_revoke_csum_set(trans->journal, bh);
@@ -233,16 +239,13 @@ static errcode_t journal_add_revoke_to_trans(journal_transaction_t *trans,
}
if (JFS_HAS_INCOMPAT_FEATURE(trans->journal,
- JFS_FEATURE_INCOMPAT_64BIT)) {
+ JFS_FEATURE_INCOMPAT_64BIT))
*((__u64 *)(&((char *)buf)[offset])) =
ext2fs_cpu_to_be64(revoke_list[i]);
- offset += 8;
-
- } else {
+ else
*((__u32 *)(&((char *)buf)[offset])) =
ext2fs_cpu_to_be32(revoke_list[i]);
- offset += 4;
- }
+ offset += sz;
}
if (offset > 0) {
diff --git a/e2fsck/recovery.c b/e2fsck/recovery.c
index b5ce3b3..d5244be 100644
--- a/e2fsck/recovery.c
+++ b/e2fsck/recovery.c
@@ -839,15 +839,23 @@ static int scan_revoke_records(journal_t *journal, struct buffer_head *bh,
{
journal_revoke_header_t *header;
int offset, max;
+ int csum_size = 0;
+ __u32 rcount;
int record_len = 4;
header = (journal_revoke_header_t *) bh->b_data;
offset = sizeof(journal_revoke_header_t);
- max = ext2fs_be32_to_cpu(header->r_count);
+ rcount = ext2fs_be32_to_cpu(header->r_count);
if (!jbd2_revoke_block_csum_verify(journal, header))
return -EINVAL;
+ if (journal_has_csum_v2or3(journal))
+ csum_size = sizeof(struct journal_revoke_tail);
+ if (rcount > journal->j_blocksize - csum_size)
+ return -EINVAL;
+ max = rcount;
+
if (JFS_HAS_INCOMPAT_FEATURE(journal, JFS_FEATURE_INCOMPAT_64BIT))
record_len = 8;
diff --git a/e2fsck/revoke.c b/e2fsck/revoke.c
index b4c3f5f..0543099 100644
--- a/e2fsck/revoke.c
+++ b/e2fsck/revoke.c
@@ -583,7 +583,7 @@ static void write_one_revoke_record(journal_t *journal,
{
int csum_size = 0;
struct buffer_head *descriptor;
- int offset;
+ int sz, offset;
journal_header_t *header;
/* If we are already aborting, this all becomes a noop. We
@@ -600,9 +600,14 @@ static void write_one_revoke_record(journal_t *journal,
if (journal_has_csum_v2or3(journal))
csum_size = sizeof(struct journal_revoke_tail);
+ if (JBD2_HAS_INCOMPAT_FEATURE(journal, JBD2_FEATURE_INCOMPAT_64BIT))
+ sz = 8;
+ else
+ sz = 4;
+
/* Make sure we have a descriptor with space left for the record */
if (descriptor) {
- if (offset >= journal->j_blocksize - csum_size) {
+ if (offset + sz > journal->j_blocksize - csum_size) {
flush_descriptor(journal, descriptor, offset, write_op);
descriptor = NULL;
}
@@ -625,16 +630,13 @@ static void write_one_revoke_record(journal_t *journal,
*descriptorp = descriptor;
}
- if (JFS_HAS_INCOMPAT_FEATURE(journal, JFS_FEATURE_INCOMPAT_64BIT)) {
- * ((__u64 *)(&descriptor->b_data[offset])) =
- ext2fs_cpu_to_be64(record->blocknr);
- offset += 8;
-
- } else {
- * ((__u32 *)(&descriptor->b_data[offset])) =
- ext2fs_cpu_to_be32(record->blocknr);
- offset += 4;
- }
+ if (JBD2_HAS_INCOMPAT_FEATURE(journal, JBD2_FEATURE_INCOMPAT_64BIT)) {
+ * ((__be64 *)(&descriptor->b_data[offset])) =
+ cpu_to_be64(record->blocknr);
+ else
+ * ((__be32 *)(&descriptor->b_data[offset])) =
+ cpu_to_be32(record->blocknr);
+ offset += sz;
*offsetp = offset;
}
diff --git a/tests/j_corrupt_revoke_rcount/expect.1 b/tests/j_corrupt_revoke_rcount/expect.1
new file mode 100644
index 0000000..97324f3
--- /dev/null
+++ b/tests/j_corrupt_revoke_rcount/expect.1
@@ -0,0 +1,8 @@
+test_filesys: recovering journal
+../e2fsck/e2fsck: Invalid argument while recovering ext3 journal of test_filesys
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+
+test_filesys: ********** WARNING: Filesystem still has errors **********
+
+Exit status is 12
diff --git a/tests/j_corrupt_revoke_rcount/expect.2 b/tests/j_corrupt_revoke_rcount/expect.2
new file mode 100644
index 0000000..c569901
--- /dev/null
+++ b/tests/j_corrupt_revoke_rcount/expect.2
@@ -0,0 +1,8 @@
+test_filesys: recovering journal
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/512 files (9.1% non-contiguous), 1066/2048 blocks
+Exit status is 0
diff --git a/tests/j_corrupt_revoke_rcount/image.gz b/tests/j_corrupt_revoke_rcount/image.gz
new file mode 100644
index 0000000000000000000000000000000000000000..c8b19e8c4ebe28f9e368635fa3962d4f0685d762
GIT binary patch
literal 8648
zcmb2|=3v+~Hz<^e`ORJ1Y!OEZh6gkEb{tnNI+!5H;gZYzQpaVY_a%+8i8nG-Cpk51
z{>jKmO4PiOIZ^3am{^&Z=VN_=i}DJBE;>oCioccd>nF0BO1z)+-16Ss+k0iF&6IwY
zRyO^%K!dZkcOZYFq3Zi=#hl~wXMg6<PE9%bO@H-gZpAIP!}lGX@P60DNc}01hYNNV
z*{S9S?`zkbTlMGHwX5Hb3hS?n-T!gp&&<sIUE%L5|9?63HaPpe-rh>Hn^mQiMc-0;
z)*1v&)7E!SUM<#E^(}wO&(B+CCg~il=sFwXUp&=4`)$N*kM$87!|QsIqvRME{%8KE
zw>kRq+>d(eG+X5dZ!4a4N-;4oG#tGCj(w|hTpa@g!;j;iUjJ8COuM{rhj@ZY%FQn&
zh5YZIrzuBA?W@kuD*X9x?(B0@f$9z{kh*93|NlLmXa7MwM&ngnAo`CykTm$u4kUlw
z{IzrQdFK~)U-Gy7Hv8rh{yhGd_PockSCxR0JV*Zg_b}|w-`>4;<FYiMz=H#4|8M><
z`}#-mXW{0bOIH1R(0q#-D2@j_SdbxqHyPyU1@A4Hf&L&D9GHJ2Vs8A4V~JYdug}~f
zu~w^I_lkAq#+;m^23IEQmwmgjyR+-PV>ZjT!>1N+iO3KAQor%d&wuyctowSS{crs8
zKMQyLd6<&_KmPW`|K_LsKf$8l%VLA&|I;1ToG+PG@A~_E*R%iKAI}F!9<wLcoum3j
zLtr!nMnhmU1V%$(Gz3ONU^E0qLtr!nMnhmU1V%$(Gz3ONU^E0qLtr!nMnhmU1V%$(
zGz3ONU^E0qLtr!nMnhmU1V%$(XorCPt!8KN1i$0|_}Mj#V9#)OmhZI$E?!^&01SK{
AN&o-=
literal 0
HcmV?d00001
diff --git a/tests/j_corrupt_revoke_rcount/name b/tests/j_corrupt_revoke_rcount/name
new file mode 100644
index 0000000..92b523e
--- /dev/null
+++ b/tests/j_corrupt_revoke_rcount/name
@@ -0,0 +1 @@
+corrupt revoke r_count buffer overflow
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v2 05/14] e2fsck: fix buffer overrun in revoke block scanning
2015-05-14 19:37 ` [PATCH v2 " Darrick J. Wong
@ 2015-05-17 0:50 ` Theodore Ts'o
0 siblings, 0 replies; 39+ messages in thread
From: Theodore Ts'o @ 2015-05-17 0:50 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Thu, May 14, 2015 at 12:37:43PM -0700, Darrick J. Wong wrote:
> Check the value of r_count to ensure that we never try to read revoke
> records past the end of the revoke block. It turns out that the
> journal writing code in debugfs was also playing fast and loose with
> the r_count, so fix that as well.
>
> The Coverity bug was 1297508.
>
> v2: Allow the revoke block to use all slots, including the last
> one.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Thanks, applied.
- Ted
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH 06/14] e2fsck: convert block-mapped files to extents on bigalloc fs
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (4 preceding siblings ...)
2015-05-14 0:21 ` [PATCH 05/14] e2fsck: fix buffer overrun in revoke block scanning Darrick J. Wong
@ 2015-05-14 0:21 ` Darrick J. Wong
2015-05-17 0:51 ` Theodore Ts'o
2015-05-14 0:21 ` [PATCH 07/14] libext2fs: support allocating uninit blocks in bmap2() Darrick J. Wong
` (9 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:21 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
As of v4.0, the Linux kernel won't add blocks to a block-mapped file
on a bigalloc filesystem. Therefore, convert any such files or
directories we find, to prevent fs errors later on.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/pass1.c | 17 +++++++++++++++++
e2fsck/problem.c | 5 +++++
e2fsck/problem.h | 3 +++
tests/f_badcluster/expect | 26 +++++++++++++-------------
4 files changed, 38 insertions(+), 13 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 5e906c1..e87643a 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -3187,6 +3187,23 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
pctx->num = 0;
}
+ /*
+ * The kernel gets mad if we ask it to allocate bigalloc clusters to
+ * a block mapped file, so rebuild it as an extent file. We can skip
+ * symlinks because they're never rewritten.
+ */
+ if (EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
+ EXT4_FEATURE_RO_COMPAT_BIGALLOC) &&
+ (LINUX_S_ISREG(inode->i_mode) || LINUX_S_ISDIR(inode->i_mode)) &&
+ ext2fs_inode_data_blocks2(fs, inode) > 0 &&
+ (ino == EXT2_ROOT_INO || ino >= EXT2_FIRST_INO(fs->super)) &&
+ !(inode->i_flags & (EXT4_EXTENTS_FL | EXT4_INLINE_DATA_FL)) &&
+ fix_problem(ctx, PR_1_NO_BIGALLOC_BLOCKMAP_FILES, pctx)) {
+ pctx->errcode = e2fsck_rebuild_extents_later(ctx, ino);
+ if (pctx->errcode)
+ goto out;
+ }
+
if (ctx->dirs_to_hash && pb.is_dir &&
!(ctx->lost_and_found && ctx->lost_and_found == ino) &&
!(inode->i_flags & EXT2_INDEX_FL) &&
diff --git a/e2fsck/problem.c b/e2fsck/problem.c
index 7e8ac5b..4f4309e 100644
--- a/e2fsck/problem.c
+++ b/e2fsck/problem.c
@@ -1106,6 +1106,11 @@ static struct e2fsck_problem problem_table[] = {
N_("@i %i @x tree could be more shallow (%b; could be <= %c)\n"),
PROMPT_FIX, PR_NO_OK | PR_PREEN_NO | PR_PREEN_OK },
+ /* Inode extent tree could be more shallow */
+ { PR_1_NO_BIGALLOC_BLOCKMAP_FILES,
+ N_("@i %i on bigalloc @f cannot be @b mapped. "),
+ PROMPT_FIX, 0 },
+
/* Pass 1b errors */
/* Pass 1B: Rescan for duplicate/bad blocks */
diff --git a/e2fsck/problem.h b/e2fsck/problem.h
index 2426021..e0b5d14 100644
--- a/e2fsck/problem.h
+++ b/e2fsck/problem.h
@@ -648,6 +648,9 @@ struct problem_context {
/* extent tree max depth too big */
#define PR_1_EXTENT_BAD_MAX_DEPTH 0x01007F
+/* bigalloc fs cannot have blockmap files */
+#define PR_1_NO_BIGALLOC_BLOCKMAP_FILES 0x010080
+
/*
* Pass 1b errors
*/
diff --git a/tests/f_badcluster/expect b/tests/f_badcluster/expect
index b8ce19d..65a1641 100644
--- a/tests/f_badcluster/expect
+++ b/tests/f_badcluster/expect
@@ -19,6 +19,8 @@ Inode 18 logical block 3 (physical block 1201) violates cluster allocation rules
Will fix in pass 1B.
Inode 18, i_blocks is 32, should be 64. Fix? yes
+Inode 15 on bigalloc filesystem cannot be block mapped. Fix? yes
+
Running additional passes to resolve blocks claimed by more than one inode...
Pass 1B: Rescanning for multiply-claimed blocks
@@ -65,19 +67,20 @@ File /g (inode #18, mod time Tue Jun 17 08:00:50 2014)
has 1 multiply-claimed block(s), shared with 0 file(s):
Clone multiply-claimed blocks? yes
+Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-Free blocks count wrong for group #0 (50, counted=47).
+Free blocks count wrong for group #0 (51, counted=48).
Fix? yes
-Free blocks count wrong (800, counted=752).
+Free blocks count wrong (816, counted=768).
Fix? yes
test_fs: ***** FILE SYSTEM WAS MODIFIED *****
-test_fs: 18/128 files (22.2% non-contiguous), 1296/2048 blocks
+test_fs: 18/128 files (22.2% non-contiguous), 1280/2048 blocks
Pass 1: Checking inodes, blocks, and sizes
Inode 12, i_blocks is 64, should be 32. Fix? yes
@@ -94,21 +97,21 @@ Pass 5: Checking group summary information
Block bitmap differences: -(1168--1200)
Fix? yes
-Free blocks count wrong for group #0 (47, counted=50).
+Free blocks count wrong for group #0 (48, counted=51).
Fix? yes
-Free blocks count wrong (752, counted=800).
+Free blocks count wrong (768, counted=816).
Fix? yes
test_fs: ***** FILE SYSTEM WAS MODIFIED *****
-test_fs: 18/128 files (5.6% non-contiguous), 1248/2048 blocks
+test_fs: 18/128 files (5.6% non-contiguous), 1232/2048 blocks
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_fs: 18/128 files (5.6% non-contiguous), 1248/2048 blocks
+test_fs: 18/128 files (5.6% non-contiguous), 1232/2048 blocks
debugfs: stat /a
Inode: 12 Type: regular Mode: 0644 Flags: 0x80000
Generation: 1117152157 Version: 0x00000001
@@ -146,19 +149,16 @@ mtime: 0x539ff5b2 -- Tue Jun 17 08:00:50 2014
EXTENTS:
(0-1):1216-1217, (2):1218
debugfs: stat /d
-Inode: 15 Type: regular Mode: 0644 Flags: 0x0
+Inode: 15 Type: regular Mode: 0644 Flags: 0x80000
Generation: 1117152160 Version: 0x00000001
User: 0 Group: 0 Size: 3072
File ACL: 0 Directory ACL: 0
-Links: 1 Blockcount: 32
+Links: 1 Blockcount: 0
Fragment: Address: 0 Number: 0 Size: 0
ctime: 0x539ff5b2 -- Tue Jun 17 08:00:50 2014
atime: 0x539ff5b2 -- Tue Jun 17 08:00:50 2014
mtime: 0x539ff5b2 -- Tue Jun 17 08:00:50 2014
-BLOCKS:
-(TIND):1650
-TOTAL: 1
^ permalink raw reply related [flat|nested] 39+ messages in thread* [PATCH 07/14] libext2fs: support allocating uninit blocks in bmap2()
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (5 preceding siblings ...)
2015-05-14 0:21 ` [PATCH 06/14] e2fsck: convert block-mapped files to extents on bigalloc fs Darrick J. Wong
@ 2015-05-14 0:21 ` Darrick J. Wong
2015-05-17 0:54 ` Theodore Ts'o
2015-05-14 0:22 ` [PATCH 08/14] libext2fs: find/alloc a range of empty blocks Darrick J. Wong
` (8 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:21 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
As part of supporting fallocate-like functionality, extend
ext2fs_bmap() with two flags -- BMAP_UNINIT and BMAP_ZERO. The first
will cause it to mark/set a block uninitialized, if it's part of an
extent based file. For a block mapped file, the mapping is put in,
but there is no way to remember the uninitialized status. The second
flag causes the block to be zeroed to support the use case of
emulating uninitialized blocks on a block-map file by zeroing them.
Eventually fallocate or fuse2fs or somebody will use these.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/bmap.c | 9 +++++++--
lib/ext2fs/ext2fs.h | 2 ++
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/lib/ext2fs/bmap.c b/lib/ext2fs/bmap.c
index cb3f5a1..c18f742 100644
--- a/lib/ext2fs/bmap.c
+++ b/lib/ext2fs/bmap.c
@@ -214,10 +214,13 @@ static errcode_t extent_bmap(ext2_filsys fs, ext2_ino_t ino,
errcode_t retval = 0;
blk64_t blk64 = 0;
int alloc = 0;
+ int set_flags;
+
+ set_flags = bmap_flags & BMAP_UNINIT ? EXT2_EXTENT_SET_BMAP_UNINIT : 0;
if (bmap_flags & BMAP_SET) {
retval = ext2fs_extent_set_bmap(handle, block,
- *phys_blk, 0);
+ *phys_blk, set_flags);
return retval;
}
retval = ext2fs_extent_goto(handle, block);
@@ -254,7 +257,7 @@ got_block:
alloc++;
set_extent:
retval = ext2fs_extent_set_bmap(handle, block,
- blk64, 0);
+ blk64, set_flags);
if (retval) {
ext2fs_block_alloc_stats2(fs, blk64, -1);
return retval;
@@ -441,6 +444,8 @@ errcode_t ext2fs_bmap2(ext2_filsys fs, ext2_ino_t ino, struct ext2_inode *inode,
if (retval == 0)
*phys_blk = blk32;
done:
+ if (*phys_blk && retval == 0 && (bmap_flags & BMAP_ZERO))
+ retval = ext2fs_zero_blocks2(fs, *phys_blk, 1, NULL, NULL);
if (buf)
ext2fs_free_mem(&buf);
if (handle)
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 231a951..2039ca0 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -527,6 +527,8 @@ typedef struct ext2_icount *ext2_icount_t;
*/
#define BMAP_ALLOC 0x0001
#define BMAP_SET 0x0002
+#define BMAP_UNINIT 0x0004
+#define BMAP_ZERO 0x0008
/*
* Returned flags from ext2fs_bmap
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH 07/14] libext2fs: support allocating uninit blocks in bmap2()
2015-05-14 0:21 ` [PATCH 07/14] libext2fs: support allocating uninit blocks in bmap2() Darrick J. Wong
@ 2015-05-17 0:54 ` Theodore Ts'o
0 siblings, 0 replies; 39+ messages in thread
From: Theodore Ts'o @ 2015-05-17 0:54 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Wed, May 13, 2015 at 05:21:53PM -0700, Darrick J. Wong wrote:
> As part of supporting fallocate-like functionality, extend
> ext2fs_bmap() with two flags -- BMAP_UNINIT and BMAP_ZERO. The first
> will cause it to mark/set a block uninitialized, if it's part of an
> extent based file. For a block mapped file, the mapping is put in,
> but there is no way to remember the uninitialized status. The second
> flag causes the block to be zeroed to support the use case of
> emulating uninitialized blocks on a block-map file by zeroing them.
>
> Eventually fallocate or fuse2fs or somebody will use these.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Thanks, applied.
- Ted
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH 08/14] libext2fs: find/alloc a range of empty blocks
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (6 preceding siblings ...)
2015-05-14 0:21 ` [PATCH 07/14] libext2fs: support allocating uninit blocks in bmap2() Darrick J. Wong
@ 2015-05-14 0:22 ` Darrick J. Wong
2015-05-17 1:02 ` Theodore Ts'o
2015-05-14 0:22 ` [PATCH 09/14] libext2fs: add new hooks to support large allocations Darrick J. Wong
` (7 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:22 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Provide a function that, given a goal pblk and a range, will try to
find a run of free blocks to satisfy the allocation. By default the
function will look anywhere in the filesystem for the run, though this
can be constrained with optional flags. One flag indicates that the
range must start at the goal block; the other flag indicates that we
should not return a range shorter than len.
v2: Add a second function to allocate a range of blocks.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/alloc.c | 141 +++++++++++++++++++++++++++++++++++++++++++++++++++
lib/ext2fs/ext2fs.h | 11 ++++
2 files changed, 152 insertions(+)
diff --git a/lib/ext2fs/alloc.c b/lib/ext2fs/alloc.c
index 9901ca5..4c3b620 100644
--- a/lib/ext2fs/alloc.c
+++ b/lib/ext2fs/alloc.c
@@ -26,6 +26,16 @@
#include "ext2_fs.h"
#include "ext2fs.h"
+#define min(a, b) ((a) < (b) ? (a) : (b))
+
+#undef DEBUG
+
+#ifdef DEBUG
+# define dbg_printf(f, a...) do {printf(f, ## a); fflush(stdout); } while (0)
+#else
+# define dbg_printf(f, a...)
+#endif
+
/*
* Clear the uninit block bitmap flag if necessary
*/
@@ -346,3 +356,134 @@ no_blocks:
group = group & ~((1 << (log_flex)) - 1);
return ext2fs_group_first_block2(fs, group);
}
+
+/*
+ * Starting at _goal_, scan around the filesystem to find a run of free blocks
+ * that's at least _len_ blocks long. Possible flags:
+ * - EXT2_NEWRANGE_EXACT_GOAL: The range of blocks must start at _goal_.
+ * - EXT2_NEWRANGE_MIN_LENGTH: do not return a allocation shorter than _len_.
+ * - EXT2_NEWRANGE_ZERO_BLOCKS: Zero blocks pblk to pblk+plen before returning.
+ *
+ * The starting block is returned in _pblk_ and the length is returned via
+ * _plen_. The blocks are not marked in the bitmap; the caller must mark
+ * however much of the returned run they actually use, hopefully via
+ * ext2fs_block_alloc_stats_range().
+ *
+ * This function can return a range that is longer than what was requested.
+ */
+errcode_t ext2fs_new_range(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, ext2fs_block_bitmap map, blk64_t *pblk,
+ blk64_t *plen)
+{
+ errcode_t retval;
+ blk64_t start, end, b;
+ int looped = 0;
+ blk64_t max_blocks = ext2fs_blocks_count(fs->super);
+
+ dbg_printf("%s: flags=0x%x goal=%llu len=%llu\n", __func__, flags,
+ goal, len);
+ EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
+ if (len == 0 || (flags & ~EXT2_NEWRANGE_ALL_FLAGS))
+ return EXT2_ET_INVALID_ARGUMENT;
+ if (!map)
+ map = fs->block_map;
+ if (!map)
+ return EXT2_ET_NO_BLOCK_BITMAP;
+ if (!goal || goal >= ext2fs_blocks_count(fs->super))
+ goal = fs->super->s_first_data_block;
+
+ start = goal;
+ while (!looped || start <= goal) {
+ retval = ext2fs_find_first_zero_block_bitmap2(map, start,
+ max_blocks - 1,
+ &start);
+ if (retval == ENOENT) {
+ /*
+ * If there are no free blocks beyond the starting
+ * point, try scanning the whole filesystem, unless the
+ * user told us only to allocate from _goal_, or if
+ * we're already scanning the whole filesystem.
+ */
+ if (flags & EXT2_NEWRANGE_FIXED_GOAL ||
+ start == fs->super->s_first_data_block)
+ goto fail;
+ start = fs->super->s_first_data_block;
+ continue;
+ } else if (retval)
+ goto errout;
+
+ if (flags & EXT2_NEWRANGE_FIXED_GOAL && start != goal)
+ goto fail;
+
+ b = min(start + len - 1, max_blocks - 1);
+ retval = ext2fs_find_first_set_block_bitmap2(map, start, b,
+ &end);
+ if (retval == ENOENT)
+ end = b + 1;
+ else if (retval)
+ goto errout;
+
+ if (!(flags & EXT2_NEWRANGE_MIN_LENGTH) ||
+ (end - start) >= len) {
+ /* Success! */
+ *pblk = start;
+ *plen = end - start;
+ dbg_printf("%s: new_range goal=%llu--%llu "
+ "blk=%llu--%llu %llu\n",
+ __func__, goal, goal + len - 1,
+ *pblk, *pblk + *plen - 1, *plen);
+
+ for (b = start; b < end;
+ b += fs->super->s_blocks_per_group)
+ clear_block_uninit(fs,
+ ext2fs_group_of_blk2(fs, b));
+ return 0;
+ }
+
+ if (flags & EXT2_NEWRANGE_FIXED_GOAL)
+ goto fail;
+ start = end;
+ if (start >= max_blocks) {
+ if (looped)
+ goto fail;
+ looped = 1;
+ start = fs->super->s_first_data_block;
+ }
+ }
+
+fail:
+ retval = EXT2_ET_BLOCK_ALLOC_FAIL;
+errout:
+ return retval;
+}
+
+errcode_t ext2fs_alloc_range(ext2_filsys fs, int flags, blk64_t goal,
+ blk_t len, blk64_t *ret)
+{
+ int newr_flags = EXT2_NEWRANGE_MIN_LENGTH;
+ errcode_t retval;
+ blk64_t plen;
+
+ EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
+ if (len == 0 || (flags & ~EXT2_ALLOCRANGE_ALL_FLAGS))
+ return EXT2_ET_INVALID_ARGUMENT;
+
+ if (flags & EXT2_ALLOCRANGE_FIXED_GOAL)
+ newr_flags |= EXT2_NEWRANGE_FIXED_GOAL;
+
+ retval = ext2fs_new_range(fs, newr_flags, goal, len, NULL, ret, &plen);
+ if (retval)
+ return retval;
+
+ if (plen < len)
+ return EXT2_ET_BLOCK_ALLOC_FAIL;
+
+ if (flags & EXT2_ALLOCRANGE_ZERO_BLOCKS) {
+ retval = ext2fs_zero_blocks2(fs, *ret, len, NULL, NULL);
+ if (retval)
+ return retval;
+ }
+
+ ext2fs_block_alloc_stats_range(fs, *ret, len, +1);
+ return retval;
+}
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 2039ca0..613f0d1 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -674,6 +674,17 @@ extern void ext2fs_set_alloc_block_callback(ext2_filsys fs,
blk64_t *ret));
blk64_t ext2fs_find_inode_goal(ext2_filsys fs, ext2_ino_t ino,
struct ext2_inode *inode, blk64_t lblk);
+#define EXT2_NEWRANGE_FIXED_GOAL (0x1)
+#define EXT2_NEWRANGE_MIN_LENGTH (0x2)
+#define EXT2_NEWRANGE_ALL_FLAGS (0x3)
+errcode_t ext2fs_new_range(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, ext2fs_block_bitmap map, blk64_t *pblk,
+ blk64_t *plen);
+#define EXT2_ALLOCRANGE_FIXED_GOAL (0x1)
+#define EXT2_ALLOCRANGE_ZERO_BLOCKS (0x2)
+#define EXT2_ALLOCRANGE_ALL_FLAGS (0x3)
+errcode_t ext2fs_alloc_range(ext2_filsys fs, int flags, blk64_t goal,
+ blk_t len, blk64_t *ret);
/* alloc_sb.c */
extern int ext2fs_reserve_super_and_bgd(ext2_filsys fs,
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH 08/14] libext2fs: find/alloc a range of empty blocks
2015-05-14 0:22 ` [PATCH 08/14] libext2fs: find/alloc a range of empty blocks Darrick J. Wong
@ 2015-05-17 1:02 ` Theodore Ts'o
0 siblings, 0 replies; 39+ messages in thread
From: Theodore Ts'o @ 2015-05-17 1:02 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Wed, May 13, 2015 at 05:22:00PM -0700, Darrick J. Wong wrote:
> Provide a function that, given a goal pblk and a range, will try to
> find a run of free blocks to satisfy the allocation. By default the
> function will look anywhere in the filesystem for the run, though this
> can be constrained with optional flags. One flag indicates that the
> range must start at the goal block; the other flag indicates that we
> should not return a range shorter than len.
>
> v2: Add a second function to allocate a range of blocks.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Thanks, applied.
- Ted
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH 09/14] libext2fs: add new hooks to support large allocations
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (7 preceding siblings ...)
2015-05-14 0:22 ` [PATCH 08/14] libext2fs: find/alloc a range of empty blocks Darrick J. Wong
@ 2015-05-14 0:22 ` Darrick J. Wong
2015-06-11 0:08 ` Theodore Ts'o
2015-05-14 0:22 ` [PATCH 10/14] libext2fs: implement fallocate Darrick J. Wong
` (6 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:22 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Add a new get_alloc_blocks hook and a block_alloc_stats_range hook so
that e2fsck can capture allocation requests spanning more than a
block to its block_found_map.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/pass1.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
lib/ext2fs/alloc.c | 37 ++++++++++++++++++++++++++++++++++++-
lib/ext2fs/alloc_stats.c | 16 ++++++++++++++++
lib/ext2fs/ext2fs.h | 16 ++++++++++++++++
4 files changed, 113 insertions(+), 1 deletion(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index e87643a..3777e42 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -3934,6 +3934,26 @@ static errcode_t e2fsck_get_alloc_block(ext2_filsys fs, blk64_t goal,
return (0);
}
+static errcode_t e2fsck_new_range(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen)
+{
+ e2fsck_t ctx = (e2fsck_t) fs->priv_data;
+ errcode_t retval;
+
+ if (ctx->block_found_map)
+ return ext2fs_new_range(fs, flags, goal, len,
+ ctx->block_found_map, pblk, plen);
+
+ if (!fs->block_map) {
+ retval = ext2fs_read_block_bitmap(fs);
+ if (retval)
+ return retval;
+ }
+
+ return ext2fs_new_range(fs, flags, goal, len, fs->block_map,
+ pblk, plen);
+}
+
static void e2fsck_block_alloc_stats(ext2_filsys fs, blk64_t blk, int inuse)
{
e2fsck_t ctx = (e2fsck_t) fs->priv_data;
@@ -3953,6 +3973,28 @@ static void e2fsck_block_alloc_stats(ext2_filsys fs, blk64_t blk, int inuse)
}
}
+static void e2fsck_block_alloc_stats_range(ext2_filsys fs, blk64_t blk,
+ blk_t num, int inuse)
+{
+ e2fsck_t ctx = (e2fsck_t) fs->priv_data;
+
+ /* Never free a critical metadata block */
+ if (ctx->block_found_map &&
+ ctx->block_metadata_map &&
+ inuse < 0 &&
+ ext2fs_test_block_bitmap_range2(ctx->block_metadata_map, blk, num))
+ return;
+
+ if (ctx->block_found_map) {
+ if (inuse > 0)
+ ext2fs_mark_block_bitmap_range2(ctx->block_found_map,
+ blk, num);
+ else
+ ext2fs_unmark_block_bitmap_range2(ctx->block_found_map,
+ blk, num);
+ }
+}
+
void e2fsck_use_inode_shortcuts(e2fsck_t ctx, int use_shortcuts)
{
ext2_filsys fs = ctx->fs;
@@ -3976,4 +4018,7 @@ void e2fsck_intercept_block_allocations(e2fsck_t ctx)
ext2fs_set_alloc_block_callback(ctx->fs, e2fsck_get_alloc_block, 0);
ext2fs_set_block_alloc_stats_callback(ctx->fs,
e2fsck_block_alloc_stats, 0);
+ ext2fs_set_new_range_callback(ctx->fs, e2fsck_new_range, NULL);
+ ext2fs_set_block_alloc_stats_range_callback(ctx->fs,
+ e2fsck_block_alloc_stats_range, NULL);
}
diff --git a/lib/ext2fs/alloc.c b/lib/ext2fs/alloc.c
index 4c3b620..86e7f99 100644
--- a/lib/ext2fs/alloc.c
+++ b/lib/ext2fs/alloc.c
@@ -379,12 +379,32 @@ errcode_t ext2fs_new_range(ext2_filsys fs, int flags, blk64_t goal,
blk64_t start, end, b;
int looped = 0;
blk64_t max_blocks = ext2fs_blocks_count(fs->super);
+ errcode_t (*nrf)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen);
dbg_printf("%s: flags=0x%x goal=%llu len=%llu\n", __func__, flags,
goal, len);
EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
if (len == 0 || (flags & ~EXT2_NEWRANGE_ALL_FLAGS))
return EXT2_ET_INVALID_ARGUMENT;
+
+ if (!map && fs->new_range) {
+ /*
+ * In case there are clients out there whose new_range
+ * handlers call ext2fs_new_range with a NULL block map,
+ * temporarily swap out the function pointer so that we don't
+ * end up in an infinite loop.
+ */
+ nrf = fs->new_range;
+ fs->new_range = NULL;
+ retval = nrf(fs, flags, goal, len, pblk, plen);
+ fs->new_range = nrf;
+ if (retval)
+ return retval;
+ start = *pblk;
+ end = *pblk + *plen;
+ goto allocated;
+ }
if (!map)
map = fs->block_map;
if (!map)
@@ -432,7 +452,7 @@ errcode_t ext2fs_new_range(ext2_filsys fs, int flags, blk64_t goal,
"blk=%llu--%llu %llu\n",
__func__, goal, goal + len - 1,
*pblk, *pblk + *plen - 1, *plen);
-
+allocated:
for (b = start; b < end;
b += fs->super->s_blocks_per_group)
clear_block_uninit(fs,
@@ -457,6 +477,21 @@ errout:
return retval;
}
+void ext2fs_set_new_range_callback(ext2_filsys fs,
+ errcode_t (*func)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen),
+ errcode_t (**old)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen))
+{
+ if (!fs || fs->magic != EXT2_ET_MAGIC_EXT2FS_FILSYS)
+ return;
+
+ if (old)
+ *old = fs->new_range;
+
+ fs->new_range = func;
+}
+
errcode_t ext2fs_alloc_range(ext2_filsys fs, int flags, blk64_t goal,
blk_t len, blk64_t *ret)
{
diff --git a/lib/ext2fs/alloc_stats.c b/lib/ext2fs/alloc_stats.c
index aca5004..3949f61 100644
--- a/lib/ext2fs/alloc_stats.c
+++ b/lib/ext2fs/alloc_stats.c
@@ -145,4 +145,20 @@ void ext2fs_block_alloc_stats_range(ext2_filsys fs, blk64_t blk,
}
ext2fs_mark_super_dirty(fs);
ext2fs_mark_bb_dirty(fs);
+ if (fs->block_alloc_stats_range)
+ (fs->block_alloc_stats_range)(fs, blk, num, inuse);
+}
+
+void ext2fs_set_block_alloc_stats_range_callback(ext2_filsys fs,
+ void (*func)(ext2_filsys fs, blk64_t blk,
+ blk_t num, int inuse),
+ void (**old)(ext2_filsys fs, blk64_t blk,
+ blk_t num, int inuse))
+{
+ if (!fs || fs->magic != EXT2_ET_MAGIC_EXT2FS_FILSYS)
+ return;
+ if (old)
+ *old = fs->block_alloc_stats_range;
+
+ fs->block_alloc_stats_range = func;
}
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 613f0d1..9655cd8 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -279,6 +279,12 @@ struct struct_ext2_filsys {
io_channel journal_io;
char *journal_name;
+
+ /* New block range allocation hooks */
+ errcode_t (*new_range)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen);
+ void (*block_alloc_stats_range)(ext2_filsys fs, blk64_t blk, blk_t num,
+ int inuse);
};
#if EXT2_FLAT_INCLUDES
@@ -674,6 +680,16 @@ extern void ext2fs_set_alloc_block_callback(ext2_filsys fs,
blk64_t *ret));
blk64_t ext2fs_find_inode_goal(ext2_filsys fs, ext2_ino_t ino,
struct ext2_inode *inode, blk64_t lblk);
+extern void ext2fs_set_new_range_callback(ext2_filsys fs,
+ errcode_t (*func)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen),
+ errcode_t (**old)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen));
+extern void ext2fs_set_block_alloc_stats_range_callback(ext2_filsys fs,
+ void (*func)(ext2_filsys fs, blk64_t blk,
+ blk_t num, int inuse),
+ void (**old)(ext2_filsys fs, blk64_t blk,
+ blk_t num, int inuse));
#define EXT2_NEWRANGE_FIXED_GOAL (0x1)
#define EXT2_NEWRANGE_MIN_LENGTH (0x2)
#define EXT2_NEWRANGE_ALL_FLAGS (0x3)
^ permalink raw reply related [flat|nested] 39+ messages in thread* [PATCH 10/14] libext2fs: implement fallocate
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (8 preceding siblings ...)
2015-05-14 0:22 ` [PATCH 09/14] libext2fs: add new hooks to support large allocations Darrick J. Wong
@ 2015-05-14 0:22 ` Darrick J. Wong
2015-06-11 0:09 ` Theodore Ts'o
2015-05-14 0:22 ` [PATCH 11/14] libext2fs: use fallocate for creating journals and hugefiles Darrick J. Wong
` (5 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:22 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Create a library function to perform fallocation on arbitrary files,
and wire up a few users for this function. This is a bit more intense
than Ted's original mk_hugefiles implementation since we have to honor
any blocks that may already be allocated to the file.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/Makefile.in | 8
lib/ext2fs/ext2fs.h | 10 +
lib/ext2fs/fallocate.c | 853 ++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 871 insertions(+)
create mode 100644 lib/ext2fs/fallocate.c
diff --git a/lib/ext2fs/Makefile.in b/lib/ext2fs/Makefile.in
index 26aaf6f..67b6a98 100644
--- a/lib/ext2fs/Makefile.in
+++ b/lib/ext2fs/Makefile.in
@@ -80,6 +80,7 @@ OBJS= $(DEBUGFS_LIB_OBJS) $(RESIZE_LIB_OBJS) $(E2IMAGE_LIB_OBJS) \
expanddir.o \
ext_attr.o \
extent.o \
+ fallocate.o \
fileio.o \
finddev.o \
flushb.o \
@@ -801,6 +802,13 @@ extent.o: $(srcdir)/extent.c $(top_builddir)/lib/config.h \
$(top_srcdir)/lib/et/com_err.h $(srcdir)/ext2_io.h \
$(top_builddir)/lib/ext2fs/ext2_err.h $(srcdir)/ext2_ext_attr.h \
$(srcdir)/bitops.h $(srcdir)/e2image.h
+fallocate.o: $(srcdir)/fallocate.c $(top_builddir)/lib/config.h \
+ $(top_builddir)/lib/dirpaths.h $(srcdir)/ext2_fs.h \
+ $(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fsP.h \
+ $(srcdir)/ext2fs.h $(srcdir)/ext2_fs.h $(srcdir)/ext3_extents.h \
+ $(top_srcdir)/lib/et/com_err.h $(srcdir)/ext2_io.h \
+ $(top_builddir)/lib/ext2fs/ext2_err.h $(srcdir)/ext2_ext_attr.h \
+ $(srcdir)/bitops.h $(srcdir)/e2image.h
fileio.o: $(srcdir)/fileio.c $(top_builddir)/lib/config.h \
$(top_builddir)/lib/dirpaths.h $(srcdir)/ext2_fs.h \
$(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h \
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 9655cd8..3784437 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1243,6 +1243,16 @@ extern errcode_t ext2fs_extent_goto2(ext2_extent_handle_t handle,
extern errcode_t ext2fs_extent_fix_parents(ext2_extent_handle_t handle);
size_t ext2fs_max_extent_depth(ext2_extent_handle_t handle);
+/* fallocate.c */
+#define EXT2_FALLOCATE_ZERO_BLOCKS (0x1)
+#define EXT2_FALLOCATE_FORCE_INIT (0x2)
+#define EXT2_FALLOCATE_FORCE_UNINIT (0x4)
+#define EXT2_FALLOCATE_INIT_BEYOND_EOF (0x8)
+#define EXT2_FALLOCATE_ALL_FLAGS (0xF)
+errcode_t ext2fs_fallocate(ext2_filsys fs, int flags, ext2_ino_t ino,
+ struct ext2_inode *inode, blk64_t goal,
+ blk64_t start, blk64_t len);
+
/* fileio.c */
extern errcode_t ext2fs_file_open2(ext2_filsys fs, ext2_ino_t ino,
struct ext2_inode *inode,
diff --git a/lib/ext2fs/fallocate.c b/lib/ext2fs/fallocate.c
new file mode 100644
index 0000000..8b502d5
--- /dev/null
+++ b/lib/ext2fs/fallocate.c
@@ -0,0 +1,853 @@
+/*
+ * fallocate.c -- Allocate large chunks of file.
+ *
+ * Copyright (C) 2014 Oracle.
+ *
+ * %Begin-Header%
+ * This file may be redistributed under the terms of the GNU Library
+ * General Public License, version 2.
+ * %End-Header%
+ */
+
+#include "config.h"
+
+#include "ext2_fs.h"
+#include "ext2fs.h"
+#define min(a, b) ((a) < (b) ? (a) : (b))
+
+#undef DEBUG
+
+#ifdef DEBUG
+# define dbg_printf(f, a...) do {printf(f, ## a); fflush(stdout); } while (0)
+#else
+# define dbg_printf(f, a...)
+#endif
+
+/*
+ * Extent-based fallocate code.
+ *
+ * Find runs of unmapped logical blocks by starting at start and walking the
+ * extents until we reach the end of the range we want.
+ *
+ * For each run of unmapped blocks, try to find the extents on either side of
+ * the range. If there's a left extent that can grow by at least a cluster and
+ * there are lblocks between start and the next lcluster after start, see if
+ * there's an implied cluster allocation; if so, zero the blocks (if the left
+ * extent is initialized) and adjust the extent. Ditto for the blocks between
+ * the end of the last full lcluster and end, if there's a right extent.
+ *
+ * Try to attach as much as we can to the left extent, then try to attach as
+ * much as we can to the right extent. For the remainder, try to allocate the
+ * whole range; map in whatever we get; and repeat until we're done.
+ *
+ * To attach to a left extent, figure out the maximum amount we can add to the
+ * extent and try to allocate that much, and append if successful. To attach
+ * to a right extent, figure out the max we can add to the extent, try to
+ * allocate that much, and prepend if successful.
+ *
+ * We need an alloc_range function that tells us how much we can allocate given
+ * a maximum length and one of a suggested start, a fixed start, or a fixed end
+ * point.
+ *
+ * Every time we modify the extent tree we also need to update the block stats.
+ *
+ * At the end, update i_blocks and i_size appropriately.
+ */
+
+static void dbg_print_extent(char *desc, struct ext2fs_extent *extent)
+{
+#ifdef DEBUG
+ if (desc)
+ printf("%s: ", desc);
+ printf("extent: lblk %llu--%llu, len %u, pblk %llu, flags: ",
+ extent->e_lblk, extent->e_lblk + extent->e_len - 1,
+ extent->e_len, extent->e_pblk);
+ if (extent->e_flags & EXT2_EXTENT_FLAGS_LEAF)
+ fputs("LEAF ", stdout);
+ if (extent->e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ fputs("UNINIT ", stdout);
+ if (extent->e_flags & EXT2_EXTENT_FLAGS_SECOND_VISIT)
+ fputs("2ND_VISIT ", stdout);
+ if (!extent->e_flags)
+ fputs("(none)", stdout);
+ fputc('\n', stdout);
+ fflush(stdout);
+#endif
+}
+
+static errcode_t claim_range(ext2_filsys fs, struct ext2_inode *inode,
+ blk64_t blk, blk64_t len)
+{
+ blk64_t clusters;
+
+ clusters = (len + EXT2FS_CLUSTER_RATIO(fs) - 1) /
+ EXT2FS_CLUSTER_RATIO(fs);
+ ext2fs_block_alloc_stats_range(fs, blk,
+ clusters * EXT2FS_CLUSTER_RATIO(fs), +1);
+ return ext2fs_iblk_add_blocks(fs, inode, clusters);
+}
+
+static errcode_t ext_falloc_helper(ext2_filsys fs,
+ int flags,
+ ext2_ino_t ino,
+ struct ext2_inode *inode,
+ ext2_extent_handle_t handle,
+ struct ext2fs_extent *left_ext,
+ struct ext2fs_extent *right_ext,
+ blk64_t range_start, blk64_t range_len,
+ blk64_t alloc_goal)
+{
+ struct ext2fs_extent newex, ex;
+ int op;
+ blk64_t fillable, pblk, plen, x, y;
+ blk64_t eof_blk = 0, cluster_fill = 0;
+ errcode_t err;
+ blk_t max_extent_len, max_uninit_len, max_init_len;
+
+#ifdef DEBUG
+ printf("%s: ", __func__);
+ if (left_ext)
+ printf("left_ext=%llu--%llu, ", left_ext->e_lblk,
+ left_ext->e_lblk + left_ext->e_len - 1);
+ if (right_ext)
+ printf("right_ext=%llu--%llu, ", right_ext->e_lblk,
+ right_ext->e_lblk + right_ext->e_len - 1);
+ printf("start=%llu len=%llu, goal=%llu\n", range_start, range_len,
+ alloc_goal);
+ fflush(stdout);
+#endif
+ /* Can't create initialized extents past EOF? */
+ if (!(flags & EXT2_FALLOCATE_INIT_BEYOND_EOF))
+ eof_blk = EXT2_I_SIZE(inode) / fs->blocksize;
+
+ /* The allocation goal must be as far into a cluster as range_start. */
+ alloc_goal = (alloc_goal & ~EXT2FS_CLUSTER_MASK(fs)) |
+ (range_start & EXT2FS_CLUSTER_MASK(fs));
+
+ max_uninit_len = EXT_UNINIT_MAX_LEN & ~EXT2FS_CLUSTER_MASK(fs);
+ max_init_len = EXT_INIT_MAX_LEN & ~EXT2FS_CLUSTER_MASK(fs);
+
+ /* We must lengthen the left extent to the end of the cluster */
+ if (left_ext && EXT2FS_CLUSTER_RATIO(fs) > 1) {
+ /* How many more blocks can be attached to left_ext? */
+ if (left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ fillable = max_uninit_len - left_ext->e_len;
+ else
+ fillable = max_init_len - left_ext->e_len;
+
+ if (fillable > range_len)
+ fillable = range_len;
+ if (fillable == 0)
+ goto expand_right;
+
+ /*
+ * If range_start isn't on a cluster boundary, try an
+ * implied cluster allocation for left_ext.
+ */
+ cluster_fill = EXT2FS_CLUSTER_RATIO(fs) -
+ (range_start & EXT2FS_CLUSTER_MASK(fs));
+ cluster_fill &= EXT2FS_CLUSTER_MASK(fs);
+ if (cluster_fill == 0)
+ goto expand_right;
+
+ if (cluster_fill > fillable)
+ cluster_fill = fillable;
+
+ /* Don't expand an initialized left_ext beyond EOF */
+ if (!(flags & EXT2_FALLOCATE_INIT_BEYOND_EOF)) {
+ x = left_ext->e_lblk + left_ext->e_len - 1;
+ dbg_printf("%s: lend=%llu newlend=%llu eofblk=%llu\n",
+ __func__, x, x + cluster_fill, eof_blk);
+ if (eof_blk >= x && eof_blk <= x + cluster_fill)
+ cluster_fill = eof_blk - x;
+ if (cluster_fill == 0)
+ goto expand_right;
+ }
+
+ err = ext2fs_extent_goto(handle, left_ext->e_lblk);
+ if (err)
+ goto expand_right;
+ left_ext->e_len += cluster_fill;
+ range_start += cluster_fill;
+ range_len -= cluster_fill;
+ alloc_goal += cluster_fill;
+
+ dbg_print_extent("ext_falloc clus left+", left_ext);
+ err = ext2fs_extent_replace(handle, 0, left_ext);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ /* Zero blocks */
+ if (!(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)) {
+ err = ext2fs_zero_blocks2(fs, left_ext->e_pblk +
+ left_ext->e_len -
+ cluster_fill, cluster_fill,
+ NULL, NULL);
+ if (err)
+ goto out;
+ }
+ }
+
+expand_right:
+ /* We must lengthen the right extent to the beginning of the cluster */
+ if (right_ext && EXT2FS_CLUSTER_RATIO(fs) > 1) {
+ /* How much can we attach to right_ext? */
+ if (right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ fillable = max_uninit_len - right_ext->e_len;
+ else
+ fillable = max_init_len - right_ext->e_len;
+
+ if (fillable > range_len)
+ fillable = range_len;
+ if (fillable == 0)
+ goto try_merge;
+
+ /*
+ * If range_end isn't on a cluster boundary, try an implied
+ * cluster allocation for right_ext.
+ */
+ cluster_fill = right_ext->e_lblk & EXT2FS_CLUSTER_MASK(fs);
+ if (cluster_fill == 0)
+ goto try_merge;
+
+ err = ext2fs_extent_goto(handle, right_ext->e_lblk);
+ if (err)
+ goto out;
+
+ if (cluster_fill > fillable)
+ cluster_fill = fillable;
+ right_ext->e_lblk -= cluster_fill;
+ right_ext->e_pblk -= cluster_fill;
+ right_ext->e_len += cluster_fill;
+ range_len -= cluster_fill;
+
+ dbg_print_extent("ext_falloc clus right+", right_ext);
+ err = ext2fs_extent_replace(handle, 0, right_ext);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ /* Zero blocks if necessary */
+ if (!(right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)) {
+ err = ext2fs_zero_blocks2(fs, right_ext->e_pblk,
+ cluster_fill, NULL, NULL);
+ if (err)
+ goto out;
+ }
+ }
+
+try_merge:
+ /* Merge both extents together, perhaps? */
+ if (left_ext && right_ext) {
+ /* Are the two extents mergeable? */
+ if ((left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT) !=
+ (right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT))
+ goto try_left;
+
+ /* User requires init/uninit but extent is uninit/init. */
+ if (((flags & EXT2_FALLOCATE_FORCE_INIT) &&
+ (left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)) ||
+ ((flags & EXT2_FALLOCATE_FORCE_UNINIT) &&
+ !(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)))
+ goto try_left;
+
+ /*
+ * Skip initialized extent unless user wants to zero blocks
+ * or requires init extent.
+ */
+ if (!(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (!(flags & EXT2_FALLOCATE_ZERO_BLOCKS) ||
+ !(flags & EXT2_FALLOCATE_FORCE_INIT)))
+ goto try_left;
+
+ /* Will it even fit? */
+ x = left_ext->e_len + range_len + right_ext->e_len;
+ if (x > (left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT ?
+ max_uninit_len : max_init_len))
+ goto try_left;
+
+ err = ext2fs_extent_goto(handle, left_ext->e_lblk);
+ if (err)
+ goto try_left;
+
+ /* Allocate blocks */
+ y = left_ext->e_pblk + left_ext->e_len;
+ err = ext2fs_new_range(fs, EXT2_NEWRANGE_FIXED_GOAL |
+ EXT2_NEWRANGE_MIN_LENGTH, y,
+ right_ext->e_pblk - y + 1, NULL,
+ &pblk, &plen);
+ if (err)
+ goto try_left;
+ if (pblk + plen != right_ext->e_pblk)
+ goto try_left;
+ err = claim_range(fs, inode, pblk, plen);
+ if (err)
+ goto out;
+
+ /* Modify extents */
+ left_ext->e_len = x;
+ dbg_print_extent("ext_falloc merge", left_ext);
+ err = ext2fs_extent_replace(handle, 0, left_ext);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT_LEAF, &newex);
+ if (err)
+ goto out;
+ err = ext2fs_extent_delete(handle, 0);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+ *right_ext = *left_ext;
+
+ /* Zero blocks */
+ if (!(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, range_start, range_len,
+ NULL, NULL);
+ if (err)
+ goto out;
+ }
+
+ return 0;
+ }
+
+try_left:
+ /* Extend the left extent */
+ if (left_ext) {
+ /* How many more blocks can be attached to left_ext? */
+ if (left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ fillable = max_uninit_len - left_ext->e_len;
+ else if (flags & EXT2_FALLOCATE_ZERO_BLOCKS)
+ fillable = max_init_len - left_ext->e_len;
+ else
+ fillable = 0;
+
+ /* User requires init/uninit but extent is uninit/init. */
+ if (((flags & EXT2_FALLOCATE_FORCE_INIT) &&
+ (left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)) ||
+ ((flags & EXT2_FALLOCATE_FORCE_UNINIT) &&
+ !(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)))
+ goto try_right;
+
+ if (fillable > range_len)
+ fillable = range_len;
+
+ /* Don't expand an initialized left_ext beyond EOF */
+ x = left_ext->e_lblk + left_ext->e_len - 1;
+ if (!(flags & EXT2_FALLOCATE_INIT_BEYOND_EOF)) {
+ dbg_printf("%s: lend=%llu newlend=%llu eofblk=%llu\n",
+ __func__, x, x + fillable, eof_blk);
+ if (eof_blk >= x && eof_blk <= x + fillable)
+ fillable = eof_blk - x;
+ }
+
+ if (fillable == 0)
+ goto try_right;
+
+ /* Test if the right edge of the range is already mapped? */
+ if (EXT2FS_CLUSTER_RATIO(fs) > 1) {
+ err = ext2fs_map_cluster_block(fs, ino, inode,
+ x + fillable, &pblk);
+ if (err)
+ goto out;
+ if (pblk)
+ fillable -= 1 + ((x + fillable)
+ & EXT2FS_CLUSTER_MASK(fs));
+ if (fillable == 0)
+ goto try_right;
+ }
+
+ /* Allocate range of blocks */
+ x = left_ext->e_pblk + left_ext->e_len;
+ err = ext2fs_new_range(fs, EXT2_NEWRANGE_FIXED_GOAL |
+ EXT2_NEWRANGE_MIN_LENGTH,
+ x, fillable, NULL, &pblk, &plen);
+ if (err)
+ goto try_right;
+ err = claim_range(fs, inode, pblk, plen);
+ if (err)
+ goto out;
+
+ /* Modify left_ext */
+ err = ext2fs_extent_goto(handle, left_ext->e_lblk);
+ if (err)
+ goto out;
+ range_start += plen;
+ range_len -= plen;
+ left_ext->e_len += plen;
+ dbg_print_extent("ext_falloc left+", left_ext);
+ err = ext2fs_extent_replace(handle, 0, left_ext);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ /* Zero blocks if necessary */
+ if (!(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, pblk, plen, NULL, NULL);
+ if (err)
+ goto out;
+ }
+ }
+
+try_right:
+ /* Extend the right extent */
+ if (right_ext) {
+ /* How much can we attach to right_ext? */
+ if (right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ fillable = max_uninit_len - right_ext->e_len;
+ else if (flags & EXT2_FALLOCATE_ZERO_BLOCKS)
+ fillable = max_init_len - right_ext->e_len;
+ else
+ fillable = 0;
+
+ /* User requires init/uninit but extent is uninit/init. */
+ if (((flags & EXT2_FALLOCATE_FORCE_INIT) &&
+ (right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)) ||
+ ((flags & EXT2_FALLOCATE_FORCE_UNINIT) &&
+ !(right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)))
+ goto try_anywhere;
+
+ if (fillable > range_len)
+ fillable = range_len;
+ if (fillable == 0)
+ goto try_anywhere;
+
+ /* Test if the left edge of the range is already mapped? */
+ if (EXT2FS_CLUSTER_RATIO(fs) > 1) {
+ err = ext2fs_map_cluster_block(fs, ino, inode,
+ right_ext->e_lblk - fillable, &pblk);
+ if (err)
+ goto out;
+ if (pblk)
+ fillable -= EXT2FS_CLUSTER_RATIO(fs) -
+ ((right_ext->e_lblk - fillable)
+ & EXT2FS_CLUSTER_MASK(fs));
+ if (fillable == 0)
+ goto try_anywhere;
+ }
+
+ /*
+ * FIXME: It would be nice if we could handle allocating a
+ * variable range from a fixed end point instead of just
+ * skipping to the general allocator if the whole range is
+ * unavailable.
+ */
+ err = ext2fs_new_range(fs, EXT2_NEWRANGE_FIXED_GOAL |
+ EXT2_NEWRANGE_MIN_LENGTH,
+ right_ext->e_pblk - fillable,
+ fillable, NULL, &pblk, &plen);
+ if (err)
+ goto try_anywhere;
+ err = claim_range(fs, inode,
+ pblk & ~EXT2FS_CLUSTER_MASK(fs),
+ plen + (pblk & EXT2FS_CLUSTER_MASK(fs)));
+ if (err)
+ goto out;
+
+ /* Modify right_ext */
+ err = ext2fs_extent_goto(handle, right_ext->e_lblk);
+ if (err)
+ goto out;
+ range_len -= plen;
+ right_ext->e_lblk -= plen;
+ right_ext->e_pblk -= plen;
+ right_ext->e_len += plen;
+ dbg_print_extent("ext_falloc right+", right_ext);
+ err = ext2fs_extent_replace(handle, 0, right_ext);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ /* Zero blocks if necessary */
+ if (!(right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, pblk,
+ plen + cluster_fill, NULL, NULL);
+ if (err)
+ goto out;
+ }
+ }
+
+try_anywhere:
+ /* Try implied cluster alloc on the left and right ends */
+ if (range_len > 0 && (range_start & EXT2FS_CLUSTER_MASK(fs))) {
+ cluster_fill = EXT2FS_CLUSTER_RATIO(fs) -
+ (range_start & EXT2FS_CLUSTER_MASK(fs));
+ cluster_fill &= EXT2FS_CLUSTER_MASK(fs);
+ if (cluster_fill > range_len)
+ cluster_fill = range_len;
+ newex.e_lblk = range_start;
+ err = ext2fs_map_cluster_block(fs, ino, inode, newex.e_lblk,
+ &pblk);
+ if (err)
+ goto out;
+ if (pblk == 0)
+ goto try_right_implied;
+ newex.e_pblk = pblk;
+ newex.e_len = cluster_fill;
+ newex.e_flags = (flags & EXT2_FALLOCATE_FORCE_INIT ? 0 :
+ EXT2_EXTENT_FLAGS_UNINIT);
+ dbg_print_extent("ext_falloc iclus left+", &newex);
+ ext2fs_extent_goto(handle, newex.e_lblk);
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT,
+ &ex);
+ if (err == EXT2_ET_NO_CURRENT_NODE)
+ ex.e_lblk = 0;
+ else if (err)
+ goto out;
+
+ if (ex.e_lblk > newex.e_lblk)
+ op = 0; /* insert before */
+ else
+ op = EXT2_EXTENT_INSERT_AFTER;
+ dbg_printf("%s: inserting %s lblk %llu newex=%llu\n",
+ __func__, op ? "after" : "before", ex.e_lblk,
+ newex.e_lblk);
+ err = ext2fs_extent_insert(handle, op, &newex);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ if (!(newex.e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, newex.e_pblk,
+ newex.e_len, NULL, NULL);
+ if (err)
+ goto out;
+ }
+
+ range_start += cluster_fill;
+ range_len -= cluster_fill;
+ }
+
+try_right_implied:
+ y = range_start + range_len;
+ if (range_len > 0 && (y & EXT2FS_CLUSTER_MASK(fs))) {
+ cluster_fill = y & EXT2FS_CLUSTER_MASK(fs);
+ if (cluster_fill > range_len)
+ cluster_fill = range_len;
+ newex.e_lblk = y & ~EXT2FS_CLUSTER_MASK(fs);
+ err = ext2fs_map_cluster_block(fs, ino, inode, newex.e_lblk,
+ &pblk);
+ if (err)
+ goto out;
+ if (pblk == 0)
+ goto no_implied;
+ newex.e_pblk = pblk;
+ newex.e_len = cluster_fill;
+ newex.e_flags = (flags & EXT2_FALLOCATE_FORCE_INIT ? 0 :
+ EXT2_EXTENT_FLAGS_UNINIT);
+ dbg_print_extent("ext_falloc iclus right+", &newex);
+ ext2fs_extent_goto(handle, newex.e_lblk);
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT,
+ &ex);
+ if (err == EXT2_ET_NO_CURRENT_NODE)
+ ex.e_lblk = 0;
+ else if (err)
+ goto out;
+
+ if (ex.e_lblk > newex.e_lblk)
+ op = 0; /* insert before */
+ else
+ op = EXT2_EXTENT_INSERT_AFTER;
+ dbg_printf("%s: inserting %s lblk %llu newex=%llu\n",
+ __func__, op ? "after" : "before", ex.e_lblk,
+ newex.e_lblk);
+ err = ext2fs_extent_insert(handle, op, &newex);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ if (!(newex.e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, newex.e_pblk,
+ newex.e_len, NULL, NULL);
+ if (err)
+ goto out;
+ }
+
+ range_len -= cluster_fill;
+ }
+
+no_implied:
+ if (range_len == 0)
+ return 0;
+
+ newex.e_lblk = range_start;
+ if (flags & EXT2_FALLOCATE_FORCE_INIT) {
+ max_extent_len = max_init_len;
+ newex.e_flags = 0;
+ } else {
+ max_extent_len = max_uninit_len;
+ newex.e_flags = EXT2_EXTENT_FLAGS_UNINIT;
+ }
+ pblk = alloc_goal;
+ y = range_len;
+ for (x = 0; x < y;) {
+ cluster_fill = newex.e_lblk & EXT2FS_CLUSTER_MASK(fs);
+ fillable = min(range_len + cluster_fill, max_extent_len);
+ err = ext2fs_new_range(fs, 0, pblk & ~EXT2FS_CLUSTER_MASK(fs),
+ fillable,
+ NULL, &pblk, &plen);
+ if (err)
+ goto out;
+ err = claim_range(fs, inode, pblk, plen);
+ if (err)
+ goto out;
+
+ /* Create extent */
+ newex.e_pblk = pblk + cluster_fill;
+ newex.e_len = plen - cluster_fill;
+ dbg_print_extent("ext_falloc create", &newex);
+ ext2fs_extent_goto(handle, newex.e_lblk);
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT,
+ &ex);
+ if (err == EXT2_ET_NO_CURRENT_NODE)
+ ex.e_lblk = 0;
+ else if (err)
+ goto out;
+
+ if (ex.e_lblk > newex.e_lblk)
+ op = 0; /* insert before */
+ else
+ op = EXT2_EXTENT_INSERT_AFTER;
+ dbg_printf("%s: inserting %s lblk %llu newex=%llu\n",
+ __func__, op ? "after" : "before", ex.e_lblk,
+ newex.e_lblk);
+ err = ext2fs_extent_insert(handle, op, &newex);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ if (!(newex.e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, pblk, plen, NULL, NULL);
+ if (err)
+ goto out;
+ }
+
+ /* Update variables at end of loop */
+ x += plen - cluster_fill;
+ range_len -= plen - cluster_fill;
+ newex.e_lblk += plen - cluster_fill;
+ pblk += plen - cluster_fill;
+ if (pblk >= ext2fs_blocks_count(fs->super))
+ pblk = fs->super->s_first_data_block;
+ }
+
+out:
+ return err;
+}
+
+static errcode_t extent_fallocate(ext2_filsys fs, int flags, ext2_ino_t ino,
+ struct ext2_inode *inode, blk64_t goal,
+ blk64_t start, blk64_t len)
+{
+ ext2_extent_handle_t handle;
+ struct ext2fs_extent left_extent, right_extent;
+ struct ext2fs_extent *left_adjacent, *right_adjacent;
+ errcode_t err;
+ blk64_t range_start, range_end = 0, end, next;
+ blk64_t count, goal_distance;
+
+ end = start + len - 1;
+ err = ext2fs_extent_open2(fs, ino, inode, &handle);
+ if (err)
+ return err;
+
+ /*
+ * Find the extent closest to the start of the alloc range. We don't
+ * check the return value because _goto() sets the current node to the
+ * next-lowest extent if 'start' is in a hole; or the next-highest
+ * extent if there aren't any lower ones; or doesn't set a current node
+ * if there was a real error reading the extent tree. In that case,
+ * _get() will error out.
+ */
+start_again:
+ ext2fs_extent_goto(handle, start);
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT, &left_extent);
+ if (err == EXT2_ET_NO_CURRENT_NODE) {
+ blk64_t max_blocks = ext2fs_blocks_count(fs->super);
+
+ if (goal == ~0ULL)
+ goal = ext2fs_find_inode_goal(fs, ino, inode, start);
+ err = ext2fs_find_first_zero_block_bitmap2(fs->block_map,
+ goal, max_blocks - 1, &goal);
+ goal += start;
+ err = ext_falloc_helper(fs, flags, ino, inode, handle, NULL,
+ NULL, start, len, goal);
+ goto errout;
+ } else if (err)
+ goto errout;
+
+ dbg_print_extent("ext_falloc initial", &left_extent);
+ next = left_extent.e_lblk + left_extent.e_len;
+ if (left_extent.e_lblk > start) {
+ /* The nearest extent we found was beyond start??? */
+ goal = left_extent.e_pblk - (left_extent.e_lblk - start);
+ err = ext_falloc_helper(fs, flags, ino, inode, handle, NULL,
+ &left_extent, start,
+ left_extent.e_lblk - start, goal);
+ if (err)
+ goto errout;
+
+ goto start_again;
+ } else if (next >= start) {
+ range_start = next;
+ left_adjacent = &left_extent;
+ } else {
+ range_start = start;
+ left_adjacent = NULL;
+ }
+ goal = left_extent.e_pblk + (range_start - left_extent.e_lblk);
+ goal_distance = range_start - next;
+
+ do {
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT_LEAF,
+ &right_extent);
+ dbg_printf("%s: ino=%d get next =%d\n", __func__, ino,
+ (int)err);
+ dbg_print_extent("ext_falloc next", &right_extent);
+ /* Stop if we've seen this extent before */
+ if (!err && right_extent.e_lblk <= left_extent.e_lblk)
+ err = EXT2_ET_EXTENT_NO_NEXT;
+
+ if (err && err != EXT2_ET_EXTENT_NO_NEXT)
+ goto errout;
+ if (err == EXT2_ET_EXTENT_NO_NEXT ||
+ right_extent.e_lblk > end + 1) {
+ range_end = end;
+ right_adjacent = NULL;
+ } else {
+ /* Handle right_extent.e_lblk <= end */
+ range_end = right_extent.e_lblk - 1;
+ right_adjacent = &right_extent;
+ }
+ if (err != EXT2_ET_EXTENT_NO_NEXT &&
+ goal_distance > (range_end - right_extent.e_lblk)) {
+ goal = right_extent.e_pblk -
+ (right_extent.e_lblk - range_start);
+ goal_distance = range_end - right_extent.e_lblk;
+ }
+
+ dbg_printf("%s: ino=%d rstart=%llu rend=%llu\n", __func__, ino,
+ range_start, range_end);
+ err = 0;
+ if (range_start <= range_end) {
+ count = range_end - range_start + 1;
+ err = ext_falloc_helper(fs, flags, ino, inode, handle,
+ left_adjacent, right_adjacent,
+ range_start, count, goal);
+ if (err)
+ goto errout;
+ }
+
+ if (range_end == end)
+ break;
+
+ err = ext2fs_extent_goto(handle, right_extent.e_lblk);
+ if (err)
+ goto errout;
+ next = right_extent.e_lblk + right_extent.e_len;
+ left_extent = right_extent;
+ left_adjacent = &left_extent;
+ range_start = next;
+ goal = left_extent.e_pblk + (range_start - left_extent.e_lblk);
+ goal_distance = range_start - next;
+ } while (range_end < end);
+
+errout:
+ ext2fs_extent_free(handle);
+ return err;
+}
+
+/*
+ * Map physical blocks to a range of logical blocks within a file. The range
+ * of logical blocks are (start, start + len). If there are already extents,
+ * the mappings will try to extend the mappings; otherwise, it will try to map
+ * start as if logical block 0 points to goal. If goal is ~0ULL, then the goal
+ * is calculated based on the inode group.
+ *
+ * Flags:
+ * - EXT2_FALLOCATE_ZERO_BLOCKS: Zero the blocks that are allocated.
+ * - EXT2_FALLOCATE_FORCE_INIT: Create only initialized extents.
+ * - EXT2_FALLOCATE_FORCE_UNINIT: Create only uninitialized extents.
+ * - EXT2_FALLOCATE_INIT_BEYOND_EOF: Create extents beyond EOF.
+ *
+ * If neither FORCE_INIT nor FORCE_UNINIT are specified, this function will
+ * try to expand any extents it finds, zeroing blocks as necessary.
+ */
+errcode_t ext2fs_fallocate(ext2_filsys fs, int flags, ext2_ino_t ino,
+ struct ext2_inode *inode, blk64_t goal,
+ blk64_t start, blk64_t len)
+{
+ struct ext2_inode inode_buf;
+ blk64_t blk, x;
+ errcode_t err;
+
+ if (((flags & EXT2_FALLOCATE_FORCE_INIT) &&
+ (flags & EXT2_FALLOCATE_FORCE_UNINIT)) ||
+ (flags & ~EXT2_FALLOCATE_ALL_FLAGS))
+ return EXT2_ET_INVALID_ARGUMENT;
+
+ if (len > ext2fs_blocks_count(fs->super))
+ return EXT2_ET_BLOCK_ALLOC_FAIL;
+ else if (len == 0)
+ return 0;
+
+ /* Read inode structure if necessary */
+ if (!inode) {
+ err = ext2fs_read_inode(fs, ino, &inode_buf);
+ if (err)
+ return err;
+ inode = &inode_buf;
+ }
+ dbg_printf("%s: ino=%d start=%llu len=%llu goal=%llu\n", __func__, ino,
+ start, len, goal);
+
+ if (inode->i_flags & EXT4_EXTENTS_FL) {
+ err = extent_fallocate(fs, flags, ino, inode, goal, start, len);
+ goto out;
+ }
+
+ /* XXX: Allocate a bunch of blocks the slow way */
+ for (blk = start; blk < start + len; blk++) {
+ err = ext2fs_bmap2(fs, ino, inode, NULL, 0, blk, 0, &x);
+ if (err)
+ return err;
+ if (x)
+ continue;
+
+ err = ext2fs_bmap2(fs, ino, inode, NULL,
+ BMAP_ALLOC | BMAP_UNINIT | BMAP_ZERO, blk,
+ 0, &x);
+ if (err)
+ return err;
+ }
+
+out:
+ if (inode == &inode_buf)
+ ext2fs_write_inode(fs, ino, inode);
+ return err;
+}
^ permalink raw reply related [flat|nested] 39+ messages in thread* [PATCH 11/14] libext2fs: use fallocate for creating journals and hugefiles
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (9 preceding siblings ...)
2015-05-14 0:22 ` [PATCH 10/14] libext2fs: implement fallocate Darrick J. Wong
@ 2015-05-14 0:22 ` Darrick J. Wong
2015-05-17 3:39 ` Theodore Ts'o
2015-05-18 21:18 ` [PATCH v2 " Darrick J. Wong
2015-05-14 0:22 ` [PATCH 12/14] debugfs: implement fallocate Darrick J. Wong
` (4 subsequent siblings)
15 siblings, 2 replies; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:22 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Use the new fallocate API for creating the journal and the mk_hugefile
feature.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/mkjournal.c | 134 +++++----------------------------
misc/mk_hugefiles.c | 96 ++----------------------
tests/f_opt_extent/expect | 15 ----
tests/r_32to64bit_meta/expect | 4 -
tests/r_32to64bit_move_itable/expect | 4 -
tests/r_64to32bit/expect | 4 -
tests/r_64to32bit_meta/expect | 4 -
tests/t_disable_mcsum_noinitbg/expect | 6 +
tests/t_enable_mcsum/expect | 15 ----
tests/t_enable_mcsum_ext3/expect | 8 +-
tests/t_enable_mcsum_initbg/expect | 11 +--
tests/t_iexpand_full/expect | 4 -
12 files changed, 56 insertions(+), 249 deletions(-)
diff --git a/lib/ext2fs/mkjournal.c b/lib/ext2fs/mkjournal.c
index c42cb98..02a65cb 100644
--- a/lib/ext2fs/mkjournal.c
+++ b/lib/ext2fs/mkjournal.c
@@ -227,89 +227,6 @@ errcode_t ext2fs_zero_blocks(ext2_filsys fs, blk_t blk, int num,
}
/*
- * Helper function for creating the journal using direct I/O routines
- */
-struct mkjournal_struct {
- int num_blocks;
- int newblocks;
- blk64_t goal;
- blk64_t blk_to_zero;
- int zero_count;
- int flags;
- char *buf;
- errcode_t err;
-};
-
-static int mkjournal_proc(ext2_filsys fs,
- blk64_t *blocknr,
- e2_blkcnt_t blockcnt,
- blk64_t ref_block EXT2FS_ATTR((unused)),
- int ref_offset EXT2FS_ATTR((unused)),
- void *priv_data)
-{
- struct mkjournal_struct *es = (struct mkjournal_struct *) priv_data;
- blk64_t new_blk;
- errcode_t retval;
-
- if (*blocknr) {
- es->goal = *blocknr;
- return 0;
- }
- if (blockcnt &&
- (EXT2FS_B2C(fs, es->goal) == EXT2FS_B2C(fs, es->goal+1)))
- new_blk = es->goal+1;
- else {
- es->goal &= ~EXT2FS_CLUSTER_MASK(fs);
- retval = ext2fs_new_block2(fs, es->goal, 0, &new_blk);
- if (retval) {
- es->err = retval;
- return BLOCK_ABORT;
- }
- ext2fs_block_alloc_stats2(fs, new_blk, +1);
- es->newblocks++;
- }
- if (blockcnt >= 0)
- es->num_blocks--;
-
- retval = 0;
- if (blockcnt <= 0)
- retval = io_channel_write_blk64(fs->io, new_blk, 1, es->buf);
- else if (!(es->flags & EXT2_MKJOURNAL_LAZYINIT)) {
- if (es->zero_count) {
- if ((es->blk_to_zero + es->zero_count == new_blk) &&
- (es->zero_count < 1024))
- es->zero_count++;
- else {
- retval = ext2fs_zero_blocks2(fs,
- es->blk_to_zero,
- es->zero_count,
- 0, 0);
- es->zero_count = 0;
- }
- }
- if (es->zero_count == 0) {
- es->blk_to_zero = new_blk;
- es->zero_count = 1;
- }
- }
-
- if (blockcnt == 0)
- memset(es->buf, 0, fs->blocksize);
-
- if (retval) {
- es->err = retval;
- return BLOCK_ABORT;
- }
- *blocknr = es->goal = new_blk;
-
- if (es->num_blocks == 0)
- return (BLOCK_CHANGED | BLOCK_ABORT);
- else
- return BLOCK_CHANGED;
-
-}
-
-/*
* Calculate the initial goal block to be roughly at the middle of the
* filesystem. Pick a group that has the largest number of free
* blocks.
@@ -350,7 +267,8 @@ static errcode_t write_journal_inode(ext2_filsys fs, ext2_ino_t journal_ino,
errcode_t retval;
struct ext2_inode inode;
unsigned long long inode_size;
- struct mkjournal_struct es;
+ int falloc_flags = EXT2_FALLOCATE_FORCE_INIT;
+ blk64_t zblk;
if ((retval = ext2fs_create_journal_superblock(fs, num_blocks, flags,
&buf)))
@@ -367,40 +285,16 @@ static errcode_t write_journal_inode(ext2_filsys fs, ext2_ino_t journal_ino,
goto out2;
}
- es.num_blocks = num_blocks;
- es.newblocks = 0;
- es.buf = buf;
- es.err = 0;
- es.flags = flags;
- es.zero_count = 0;
- es.goal = (goal != ~0ULL) ? goal : get_midpoint_journal_block(fs);
+ if (goal == ~0ULL)
+ goal = get_midpoint_journal_block(fs);
- if (fs->super->s_feature_incompat & EXT3_FEATURE_INCOMPAT_EXTENTS) {
+ if (fs->super->s_feature_incompat & EXT3_FEATURE_INCOMPAT_EXTENTS)
inode.i_flags |= EXT4_EXTENTS_FL;
- if ((retval = ext2fs_write_inode(fs, journal_ino, &inode)))
- goto out2;
- }
- retval = ext2fs_block_iterate3(fs, journal_ino, BLOCK_FLAG_APPEND,
- 0, mkjournal_proc, &es);
- if (retval)
- goto out2;
- if (es.err) {
- retval = es.err;
- goto out2;
- }
- if (es.zero_count) {
- retval = ext2fs_zero_blocks2(fs, es.blk_to_zero,
- es.zero_count, 0, 0);
- if (retval)
- goto out2;
- }
-
- if ((retval = ext2fs_read_inode(fs, journal_ino, &inode)))
- goto out2;
+ if (!(flags & EXT2_MKJOURNAL_LAZYINIT))
+ falloc_flags |= EXT2_FALLOCATE_ZERO_BLOCKS;
inode_size = (unsigned long long)fs->blocksize * num_blocks;
- ext2fs_iblk_add_blocks(fs, &inode, es.newblocks);
inode.i_mtime = inode.i_ctime = fs->now ? fs->now : time(0);
inode.i_links_count = 1;
inode.i_mode = LINUX_S_IFREG | 0600;
@@ -408,9 +302,21 @@ static errcode_t write_journal_inode(ext2_filsys fs, ext2_ino_t journal_ino,
if (retval)
goto out2;
+ retval = ext2fs_fallocate(fs, falloc_flags, journal_ino,
+ &inode, goal, 0, num_blocks);
+ if (retval)
+ goto out2;
+
if ((retval = ext2fs_write_new_inode(fs, journal_ino, &inode)))
goto out2;
- retval = 0;
+
+ retval = ext2fs_bmap2(fs, journal_ino, &inode, NULL, 0, 0, NULL, &zblk);
+ if (retval)
+ goto out2;
+
+ retval = io_channel_write_blk64(fs->io, zblk, 1, buf);
+ if (retval)
+ goto out2;
memcpy(fs->super->s_jnl_blocks, inode.i_block, EXT2_N_BLOCKS*4);
fs->super->s_jnl_blocks[15] = inode.i_size_high;
diff --git a/misc/mk_hugefiles.c b/misc/mk_hugefiles.c
index e42c0b9..0978d55 100644
--- a/misc/mk_hugefiles.c
+++ b/misc/mk_hugefiles.c
@@ -258,12 +258,7 @@ static errcode_t mk_hugefile(ext2_filsys fs, blk64_t num,
{
errcode_t retval;
- blk64_t lblk, bend = 0;
- __u64 size;
- blk64_t left;
- blk64_t count = 0;
struct ext2_inode inode;
- ext2_extent_handle_t handle;
retval = ext2fs_new_inode(fs, 0, LINUX_S_IFREG, NULL, ino);
if (retval)
@@ -283,85 +278,20 @@ static errcode_t mk_hugefile(ext2_filsys fs, blk64_t num,
ext2fs_inode_alloc_stats2(fs, *ino, +1, 0);
- retval = ext2fs_extent_open2(fs, *ino, &inode, &handle);
+ if (EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+ EXT3_FEATURE_INCOMPAT_EXTENTS))
+ inode.i_flags |= EXT4_EXTENTS_FL;
+ retval = ext2fs_fallocate(fs,
+ EXT2_FALLOCATE_FORCE_INIT |
+ EXT2_FALLOCATE_ZERO_BLOCKS,
+ *ino, &inode, ~0ULL, 0, num);
if (retval)
return retval;
-
- lblk = 0;
- left = num ? num : 1;
- while (left) {
- blk64_t pblk, end;
- blk64_t n = left;
-
- retval = ext2fs_find_first_zero_block_bitmap2(fs->block_map,
- goal, ext2fs_blocks_count(fs->super) - 1, &end);
- if (retval)
- goto errout;
- goal = end;
-
- retval = ext2fs_find_first_set_block_bitmap2(fs->block_map, goal,
- ext2fs_blocks_count(fs->super) - 1, &bend);
- if (retval == ENOENT) {
- bend = ext2fs_blocks_count(fs->super);
- if (num == 0)
- left = 0;
- }
- if (!num || bend - goal < left)
- n = bend - goal;
- pblk = goal;
- if (num)
- left -= n;
- goal += n;
- count += n;
- ext2fs_block_alloc_stats_range(fs, pblk, n, +1);
-
- if (zero_hugefile) {
- blk64_t ret_blk;
- retval = ext2fs_zero_blocks2(fs, pblk, n,
- &ret_blk, NULL);
-
- if (retval)
- com_err(program_name, retval,
- _("while zeroing block %llu "
- "for hugefile"), ret_blk);
- }
-
- while (n) {
- blk64_t l = n;
- struct ext2fs_extent newextent;
-
- if (l > EXT_INIT_MAX_LEN)
- l = EXT_INIT_MAX_LEN;
-
- newextent.e_len = l;
- newextent.e_pblk = pblk;
- newextent.e_lblk = lblk;
- newextent.e_flags = 0;
-
- retval = ext2fs_extent_insert(handle,
- EXT2_EXTENT_INSERT_AFTER, &newextent);
- if (retval)
- return retval;
- pblk += l;
- lblk += l;
- n -= l;
- }
- }
-
- retval = ext2fs_read_inode(fs, *ino, &inode);
- if (retval)
- goto errout;
-
- retval = ext2fs_iblk_add_blocks(fs, &inode,
- count / EXT2FS_CLUSTER_RATIO(fs));
- if (retval)
- goto errout;
- size = (__u64) count * fs->blocksize;
- retval = ext2fs_inode_size_set(fs, &inode, size);
+ retval = ext2fs_inode_size_set(fs, &inode, num * fs->blocksize);
if (retval)
- goto errout;
+ return retval;
- retval = ext2fs_write_new_inode(fs, *ino, &inode);
+ retval = ext2fs_write_inode(fs, *ino, &inode);
if (retval)
goto errout;
@@ -379,13 +309,7 @@ retry:
goto retry;
}
- if (retval)
- goto errout;
-
errout:
- if (handle)
- ext2fs_extent_free(handle);
-
return retval;
}
diff --git a/tests/f_opt_extent/expect b/tests/f_opt_extent/expect
index 6d4863b..f4ed7ff 100644
--- a/tests/f_opt_extent/expect
+++ b/tests/f_opt_extent/expect
@@ -30,22 +30,11 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 570
-+Free blocks: 567
+-Free blocks: 569
++Free blocks: 566
Free inodes: 65047
First block: 1
Block size: 1024
-@@ -47,8 +47,8 @@
- Block bitmap at 262 (+261)
- Inode bitmap at 278 (+277)
- Inode table at 294-549 (+293)
-- 21 free blocks, 535 free inodes, 3 directories, 535 unused inodes
-- Free blocks: 4414-4434
-+ 18 free blocks, 535 free inodes, 3 directories, 535 unused inodes
-+ Free blocks: 4417-4434
- Free inodes: 490-1024
- Group 1: (Blocks 8193-16384) [INODE_UNINIT]
- Backup superblock at 8193, Group descriptors at 8194-8197
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
diff --git a/tests/r_32to64bit_meta/expect b/tests/r_32to64bit_meta/expect
index 0eacd45..8796503 100644
--- a/tests/r_32to64bit_meta/expect
+++ b/tests/r_32to64bit_meta/expect
@@ -35,8 +35,8 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 858
-+Free blocks: 852
+-Free blocks: 857
++Free blocks: 851
Free inodes: 65046
First block: 1
Block size: 1024
diff --git a/tests/r_32to64bit_move_itable/expect b/tests/r_32to64bit_move_itable/expect
index b51663d..999bb8d 100644
--- a/tests/r_32to64bit_move_itable/expect
+++ b/tests/r_32to64bit_move_itable/expect
@@ -35,8 +35,8 @@ Change in FS metadata:
Inode count: 98304
Block count: 786432
Reserved block count: 39321
--Free blocks: 764
-+Free blocks: 734
+-Free blocks: 763
++Free blocks: 733
Free inodes: 97566
First block: 1
Block size: 1024
diff --git a/tests/r_64to32bit/expect b/tests/r_64to32bit/expect
index 13e94a2..5d2ea4b 100644
--- a/tests/r_64to32bit/expect
+++ b/tests/r_64to32bit/expect
@@ -35,8 +35,8 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 571
-+Free blocks: 589
+-Free blocks: 570
++Free blocks: 588
Free inodes: 65048
First block: 1
Block size: 1024
diff --git a/tests/r_64to32bit_meta/expect b/tests/r_64to32bit_meta/expect
index d6e2dcc..1400c6b 100644
--- a/tests/r_64to32bit_meta/expect
+++ b/tests/r_64to32bit_meta/expect
@@ -35,8 +35,8 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 852
-+Free blocks: 858
+-Free blocks: 851
++Free blocks: 857
Free inodes: 65046
First block: 1
Block size: 1024
diff --git a/tests/t_disable_mcsum_noinitbg/expect b/tests/t_disable_mcsum_noinitbg/expect
index a022631..09e4ff1 100644
--- a/tests/t_disable_mcsum_noinitbg/expect
+++ b/tests/t_disable_mcsum_noinitbg/expect
@@ -40,9 +40,9 @@ Change in FS metadata:
Block bitmap at 262 (+261)
Inode bitmap at 278 (+277)
Inode table at 294-549 (+293)
-- 21 free blocks, 536 free inodes, 2 directories, 536 unused inodes
-+ 21 free blocks, 536 free inodes, 2 directories
- Free blocks: 4413-4433
+- 0 free blocks, 536 free inodes, 2 directories, 536 unused inodes
++ 0 free blocks, 536 free inodes, 2 directories
+ Free blocks:
Free inodes: 489-1024
-Group 1: (Blocks 8193-16384) [INODE_UNINIT]
+Group 1: (Blocks 8193-16384)
diff --git a/tests/t_enable_mcsum/expect b/tests/t_enable_mcsum/expect
index 2ee3c27..81e1125 100644
--- a/tests/t_enable_mcsum/expect
+++ b/tests/t_enable_mcsum/expect
@@ -45,8 +45,8 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 571
-+Free blocks: 568
+-Free blocks: 570
++Free blocks: 567
Free inodes: 65048
First block: 1
Block size: 1024
@@ -58,17 +58,6 @@ Change in FS metadata:
Journal features: (none)
Journal size: 16M
Journal length: 16384
-@@ -46,8 +47,8 @@
- Block bitmap at 262 (+261)
- Inode bitmap at 278 (+277)
- Inode table at 294-549 (+293)
-- 21 free blocks, 536 free inodes, 2 directories, 536 unused inodes
-- Free blocks: 4413-4433
-+ 18 free blocks, 536 free inodes, 2 directories, 536 unused inodes
-+ Free blocks: 4413, 4417-4433
- Free inodes: 489-1024
- Group 1: (Blocks 8193-16384) [INODE_UNINIT]
- Backup superblock at 8193, Group descriptors at 8194-8197
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
diff --git a/tests/t_enable_mcsum_ext3/expect b/tests/t_enable_mcsum_ext3/expect
index 5460482..0f761a9 100644
--- a/tests/t_enable_mcsum_ext3/expect
+++ b/tests/t_enable_mcsum_ext3/expect
@@ -49,8 +49,8 @@ Change in FS metadata:
Reserved GDT blocks at 4-259
Block bitmap at 260 (+259)
@@ -45,7 +46,7 @@
- 7789 free blocks, 1013 free inodes, 2 directories
- Free blocks: 404-8192
+ 0 free blocks, 1013 free inodes, 2 directories
+ Free blocks:
Free inodes: 12-1024
-Group 1: (Blocks 8193-16384)
+Group 1: (Blocks 8193-16384) [ITABLE_ZEROED]
@@ -58,8 +58,8 @@ Change in FS metadata:
Reserved GDT blocks at 8196-8451
Block bitmap at 8452 (+259)
@@ -54,6 +55,6 @@
- 7803 free blocks, 1024 free inodes, 0 directories
- Free blocks: 8582-16384
+ 0 free blocks, 1024 free inodes, 0 directories
+ Free blocks:
Free inodes: 1025-2048
-Group 2: (Blocks 16385-24576)
+Group 2: (Blocks 16385-24576) [ITABLE_ZEROED]
diff --git a/tests/t_enable_mcsum_initbg/expect b/tests/t_enable_mcsum_initbg/expect
index d3b4444..3cbb98f 100644
--- a/tests/t_enable_mcsum_initbg/expect
+++ b/tests/t_enable_mcsum_initbg/expect
@@ -45,8 +45,8 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 571
-+Free blocks: 568
+-Free blocks: 570
++Free blocks: 567
Free inodes: 65048
First block: 1
Block size: 1024
@@ -69,10 +69,9 @@ Change in FS metadata:
Block bitmap at 262 (+261)
Inode bitmap at 278 (+277)
Inode table at 294-549 (+293)
-- 21 free blocks, 536 free inodes, 2 directories
-- Free blocks: 4413-4433
-+ 18 free blocks, 536 free inodes, 2 directories, 536 unused inodes
-+ Free blocks: 4413, 4417-4433
+- 0 free blocks, 536 free inodes, 2 directories
++ 0 free blocks, 536 free inodes, 2 directories, 536 unused inodes
+ Free blocks:
Free inodes: 489-1024
-Group 1: (Blocks 8193-16384)
+Group 1: (Blocks 8193-16384) [INODE_UNINIT, ITABLE_ZEROED]
diff --git a/tests/t_iexpand_full/expect b/tests/t_iexpand_full/expect
index 3eb1715..0474827 100644
--- a/tests/t_iexpand_full/expect
+++ b/tests/t_iexpand_full/expect
@@ -21,8 +21,8 @@ Setting inode size 256
Exit status is 0
Change in FS metadata:
@@ -13 +13 @@
--Free blocks: 12301
-+Free blocks: 12
+-Free blocks: 12299
++Free blocks: 10
@@ -22 +22 @@
-Inode blocks per group: 128
+Inode blocks per group: 256
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH 11/14] libext2fs: use fallocate for creating journals and hugefiles
2015-05-14 0:22 ` [PATCH 11/14] libext2fs: use fallocate for creating journals and hugefiles Darrick J. Wong
@ 2015-05-17 3:39 ` Theodore Ts'o
2015-05-18 19:24 ` Darrick J. Wong
2015-05-18 21:18 ` [PATCH v2 " Darrick J. Wong
1 sibling, 1 reply; 39+ messages in thread
From: Theodore Ts'o @ 2015-05-17 3:39 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Wed, May 13, 2015 at 05:22:19PM -0700, Darrick J. Wong wrote:
> Use the new fallocate API for creating the journal and the mk_hugefile
> feature.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
I tried applying patches 9-11, and I found a regression. If you add
the following stanza to /etc/mke2fs.conf:
hugefile = {
features = extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,^resize_inode,sparse_super2
hash_alg = half_md4
num_backup_sb = 0
packed_meta_blocks = 1
make_hugefiles = 1
inode_ratio = 4194304
hugefiles_dir = /store
hugefiles_name = big-data
hugefiles_digits = 0
hugefiles_size = 0
hugefiles_align = 256M
num_hugefiles = 1
zero_hugefiles = false
flex_bg_size = 262144
}
... then "mke2fs -Fq -T hugefile /dev/sdXX" should create a file
system with a single file /store/big-data that starts at offset 256M
and consumes the rest of the space. For example, try the commands
% time mke2fs -Fq -T hugefile /tmp/foo.img 8T
% debugfs -R "extents /store/big-data" /tmp/foo.img
With this patch applied, the file /store/big-data is a zero-length
file, instead of a very big file consuming the whole disk.
Arguably there should have been a test so that this regression would
be detected automatically. I'll take care of adding it.
(BTW, note how quickly the file /store/big-data is created using the
mk_hugefile code. Although I understand the new fallocate code is
more general, hopefully this generality doesn't cause performance
regression in terms of the file system layout or CPU time required to
create the big-data file.)
> --- a/tests/r_32to64bit_meta/expect
> +++ b/tests/r_32to64bit_meta/expect
> @@ -35,8 +35,8 @@ Change in FS metadata:
> Inode count: 65536
> Block count: 524288
> Reserved block count: 26214
> --Free blocks: 858
> -+Free blocks: 852
> +-Free blocks: 857
> ++Free blocks: 851
> Free inodes: 65046
> First block: 1
> Block size: 1024
Why these changes? This implies the new fallocate code isn't creating
an extent tree that isn't quite as efficient as the original code?
- Ted
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH 11/14] libext2fs: use fallocate for creating journals and hugefiles
2015-05-17 3:39 ` Theodore Ts'o
@ 2015-05-18 19:24 ` Darrick J. Wong
0 siblings, 0 replies; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-18 19:24 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Sat, May 16, 2015 at 11:39:25PM -0400, Theodore Ts'o wrote:
> On Wed, May 13, 2015 at 05:22:19PM -0700, Darrick J. Wong wrote:
> > Use the new fallocate API for creating the journal and the mk_hugefile
> > feature.
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
>
> I tried applying patches 9-11, and I found a regression. If you add
> the following stanza to /etc/mke2fs.conf:
>
> hugefile = {
> features = extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,^resize_inode,sparse_super2
> hash_alg = half_md4
> num_backup_sb = 0
> packed_meta_blocks = 1
> make_hugefiles = 1
> inode_ratio = 4194304
> hugefiles_dir = /store
> hugefiles_name = big-data
> hugefiles_digits = 0
> hugefiles_size = 0
> hugefiles_align = 256M
> num_hugefiles = 1
> zero_hugefiles = false
> flex_bg_size = 262144
> }
>
> ... then "mke2fs -Fq -T hugefile /dev/sdXX" should create a file
> system with a single file /store/big-data that starts at offset 256M
> and consumes the rest of the space. For example, try the commands
>
> % time mke2fs -Fq -T hugefile /tmp/foo.img 8T
> % debugfs -R "extents /store/big-data" /tmp/foo.img
>
> With this patch applied, the file /store/big-data is a zero-length
> file, instead of a very big file consuming the whole disk.
Oops. I missed that subtlety; it's a pretty quick fix to make it
fallocate all the way to the end. I also found a small bookkeeping error
that eliminates the churn in the test case expect files.
> Arguably there should have been a test so that this regression would
> be detected automatically. I'll take care of adding it.
>
> (BTW, note how quickly the file /store/big-data is created using the
> mk_hugefile code. Although I understand the new fallocate code is
> more general, hopefully this generality doesn't cause performance
> regression in terms of the file system layout or CPU time required to
> create the big-data file.)
A lot of the complexity deals with figuring out if for a given hole we should
merely try to extent (or merge) the left and right extents. For empty files,
it figures out that there is no left/right extent and simply cuts to the
alloc-range-and-map loop. I noticed that it seemed to slow down maybe a
tenth of a second (out of 5) for a 4TB file; is that too much of a regression?
--D
>
> > --- a/tests/r_32to64bit_meta/expect
> > +++ b/tests/r_32to64bit_meta/expect
> > @@ -35,8 +35,8 @@ Change in FS metadata:
> > Inode count: 65536
> > Block count: 524288
> > Reserved block count: 26214
> > --Free blocks: 858
> > -+Free blocks: 852
> > +-Free blocks: 857
> > ++Free blocks: 851
> > Free inodes: 65046
> > First block: 1
> > Block size: 1024
>
> Why these changes? This implies the new fallocate code isn't creating
> an extent tree that isn't quite as efficient as the original code?
>
> - Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v2 11/14] libext2fs: use fallocate for creating journals and hugefiles
2015-05-14 0:22 ` [PATCH 11/14] libext2fs: use fallocate for creating journals and hugefiles Darrick J. Wong
2015-05-17 3:39 ` Theodore Ts'o
@ 2015-05-18 21:18 ` Darrick J. Wong
2015-06-11 0:12 ` Theodore Ts'o
1 sibling, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-18 21:18 UTC (permalink / raw)
To: tytso; +Cc: linux-ext4
Use the new fallocate API for creating the journal and the mk_hugefile
feature.
v2: Use all remaining disk space when the hugefile size is zero. This
was a corner case that I missed. Also fix a bug where the goal didn't
get used; this fixes a lot of churn in the test case expect files.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/mkjournal.c | 134 ++++++--------------------------------
misc/mk_hugefiles.c | 98 +++-------------------------
tests/t_enable_mcsum_ext3/expect | 8 +-
3 files changed, 36 insertions(+), 204 deletions(-)
diff --git a/lib/ext2fs/mkjournal.c b/lib/ext2fs/mkjournal.c
index c42cb98..02a65cb 100644
--- a/lib/ext2fs/mkjournal.c
+++ b/lib/ext2fs/mkjournal.c
@@ -227,89 +227,6 @@ errcode_t ext2fs_zero_blocks(ext2_filsys fs, blk_t blk, int num,
}
/*
- * Helper function for creating the journal using direct I/O routines
- */
-struct mkjournal_struct {
- int num_blocks;
- int newblocks;
- blk64_t goal;
- blk64_t blk_to_zero;
- int zero_count;
- int flags;
- char *buf;
- errcode_t err;
-};
-
-static int mkjournal_proc(ext2_filsys fs,
- blk64_t *blocknr,
- e2_blkcnt_t blockcnt,
- blk64_t ref_block EXT2FS_ATTR((unused)),
- int ref_offset EXT2FS_ATTR((unused)),
- void *priv_data)
-{
- struct mkjournal_struct *es = (struct mkjournal_struct *) priv_data;
- blk64_t new_blk;
- errcode_t retval;
-
- if (*blocknr) {
- es->goal = *blocknr;
- return 0;
- }
- if (blockcnt &&
- (EXT2FS_B2C(fs, es->goal) == EXT2FS_B2C(fs, es->goal+1)))
- new_blk = es->goal+1;
- else {
- es->goal &= ~EXT2FS_CLUSTER_MASK(fs);
- retval = ext2fs_new_block2(fs, es->goal, 0, &new_blk);
- if (retval) {
- es->err = retval;
- return BLOCK_ABORT;
- }
- ext2fs_block_alloc_stats2(fs, new_blk, +1);
- es->newblocks++;
- }
- if (blockcnt >= 0)
- es->num_blocks--;
-
- retval = 0;
- if (blockcnt <= 0)
- retval = io_channel_write_blk64(fs->io, new_blk, 1, es->buf);
- else if (!(es->flags & EXT2_MKJOURNAL_LAZYINIT)) {
- if (es->zero_count) {
- if ((es->blk_to_zero + es->zero_count == new_blk) &&
- (es->zero_count < 1024))
- es->zero_count++;
- else {
- retval = ext2fs_zero_blocks2(fs,
- es->blk_to_zero,
- es->zero_count,
- 0, 0);
- es->zero_count = 0;
- }
- }
- if (es->zero_count == 0) {
- es->blk_to_zero = new_blk;
- es->zero_count = 1;
- }
- }
-
- if (blockcnt == 0)
- memset(es->buf, 0, fs->blocksize);
-
- if (retval) {
- es->err = retval;
- return BLOCK_ABORT;
- }
- *blocknr = es->goal = new_blk;
-
- if (es->num_blocks == 0)
- return (BLOCK_CHANGED | BLOCK_ABORT);
- else
- return BLOCK_CHANGED;
-
-}
-
-/*
* Calculate the initial goal block to be roughly at the middle of the
* filesystem. Pick a group that has the largest number of free
* blocks.
@@ -350,7 +267,8 @@ static errcode_t write_journal_inode(ext2_filsys fs, ext2_ino_t journal_ino,
errcode_t retval;
struct ext2_inode inode;
unsigned long long inode_size;
- struct mkjournal_struct es;
+ int falloc_flags = EXT2_FALLOCATE_FORCE_INIT;
+ blk64_t zblk;
if ((retval = ext2fs_create_journal_superblock(fs, num_blocks, flags,
&buf)))
@@ -367,40 +285,16 @@ static errcode_t write_journal_inode(ext2_filsys fs, ext2_ino_t journal_ino,
goto out2;
}
- es.num_blocks = num_blocks;
- es.newblocks = 0;
- es.buf = buf;
- es.err = 0;
- es.flags = flags;
- es.zero_count = 0;
- es.goal = (goal != ~0ULL) ? goal : get_midpoint_journal_block(fs);
+ if (goal == ~0ULL)
+ goal = get_midpoint_journal_block(fs);
- if (fs->super->s_feature_incompat & EXT3_FEATURE_INCOMPAT_EXTENTS) {
+ if (fs->super->s_feature_incompat & EXT3_FEATURE_INCOMPAT_EXTENTS)
inode.i_flags |= EXT4_EXTENTS_FL;
- if ((retval = ext2fs_write_inode(fs, journal_ino, &inode)))
- goto out2;
- }
- retval = ext2fs_block_iterate3(fs, journal_ino, BLOCK_FLAG_APPEND,
- 0, mkjournal_proc, &es);
- if (retval)
- goto out2;
- if (es.err) {
- retval = es.err;
- goto out2;
- }
- if (es.zero_count) {
- retval = ext2fs_zero_blocks2(fs, es.blk_to_zero,
- es.zero_count, 0, 0);
- if (retval)
- goto out2;
- }
-
- if ((retval = ext2fs_read_inode(fs, journal_ino, &inode)))
- goto out2;
+ if (!(flags & EXT2_MKJOURNAL_LAZYINIT))
+ falloc_flags |= EXT2_FALLOCATE_ZERO_BLOCKS;
inode_size = (unsigned long long)fs->blocksize * num_blocks;
- ext2fs_iblk_add_blocks(fs, &inode, es.newblocks);
inode.i_mtime = inode.i_ctime = fs->now ? fs->now : time(0);
inode.i_links_count = 1;
inode.i_mode = LINUX_S_IFREG | 0600;
@@ -408,9 +302,21 @@ static errcode_t write_journal_inode(ext2_filsys fs, ext2_ino_t journal_ino,
if (retval)
goto out2;
+ retval = ext2fs_fallocate(fs, falloc_flags, journal_ino,
+ &inode, goal, 0, num_blocks);
+ if (retval)
+ goto out2;
+
if ((retval = ext2fs_write_new_inode(fs, journal_ino, &inode)))
goto out2;
- retval = 0;
+
+ retval = ext2fs_bmap2(fs, journal_ino, &inode, NULL, 0, 0, NULL, &zblk);
+ if (retval)
+ goto out2;
+
+ retval = io_channel_write_blk64(fs->io, zblk, 1, buf);
+ if (retval)
+ goto out2;
memcpy(fs->super->s_jnl_blocks, inode.i_block, EXT2_N_BLOCKS*4);
fs->super->s_jnl_blocks[15] = inode.i_size_high;
diff --git a/misc/mk_hugefiles.c b/misc/mk_hugefiles.c
index e42c0b9..7dcebbd 100644
--- a/misc/mk_hugefiles.c
+++ b/misc/mk_hugefiles.c
@@ -258,12 +258,7 @@ static errcode_t mk_hugefile(ext2_filsys fs, blk64_t num,
{
errcode_t retval;
- blk64_t lblk, bend = 0;
- __u64 size;
- blk64_t left;
- blk64_t count = 0;
struct ext2_inode inode;
- ext2_extent_handle_t handle;
retval = ext2fs_new_inode(fs, 0, LINUX_S_IFREG, NULL, ino);
if (retval)
@@ -283,85 +278,20 @@ static errcode_t mk_hugefile(ext2_filsys fs, blk64_t num,
ext2fs_inode_alloc_stats2(fs, *ino, +1, 0);
- retval = ext2fs_extent_open2(fs, *ino, &inode, &handle);
+ if (EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+ EXT3_FEATURE_INCOMPAT_EXTENTS))
+ inode.i_flags |= EXT4_EXTENTS_FL;
+ retval = ext2fs_fallocate(fs,
+ EXT2_FALLOCATE_FORCE_INIT |
+ EXT2_FALLOCATE_ZERO_BLOCKS,
+ *ino, &inode, goal, 0, num);
if (retval)
return retval;
-
- lblk = 0;
- left = num ? num : 1;
- while (left) {
- blk64_t pblk, end;
- blk64_t n = left;
-
- retval = ext2fs_find_first_zero_block_bitmap2(fs->block_map,
- goal, ext2fs_blocks_count(fs->super) - 1, &end);
- if (retval)
- goto errout;
- goal = end;
-
- retval = ext2fs_find_first_set_block_bitmap2(fs->block_map, goal,
- ext2fs_blocks_count(fs->super) - 1, &bend);
- if (retval == ENOENT) {
- bend = ext2fs_blocks_count(fs->super);
- if (num == 0)
- left = 0;
- }
- if (!num || bend - goal < left)
- n = bend - goal;
- pblk = goal;
- if (num)
- left -= n;
- goal += n;
- count += n;
- ext2fs_block_alloc_stats_range(fs, pblk, n, +1);
-
- if (zero_hugefile) {
- blk64_t ret_blk;
- retval = ext2fs_zero_blocks2(fs, pblk, n,
- &ret_blk, NULL);
-
- if (retval)
- com_err(program_name, retval,
- _("while zeroing block %llu "
- "for hugefile"), ret_blk);
- }
-
- while (n) {
- blk64_t l = n;
- struct ext2fs_extent newextent;
-
- if (l > EXT_INIT_MAX_LEN)
- l = EXT_INIT_MAX_LEN;
-
- newextent.e_len = l;
- newextent.e_pblk = pblk;
- newextent.e_lblk = lblk;
- newextent.e_flags = 0;
-
- retval = ext2fs_extent_insert(handle,
- EXT2_EXTENT_INSERT_AFTER, &newextent);
- if (retval)
- return retval;
- pblk += l;
- lblk += l;
- n -= l;
- }
- }
-
- retval = ext2fs_read_inode(fs, *ino, &inode);
- if (retval)
- goto errout;
-
- retval = ext2fs_iblk_add_blocks(fs, &inode,
- count / EXT2FS_CLUSTER_RATIO(fs));
- if (retval)
- goto errout;
- size = (__u64) count * fs->blocksize;
- retval = ext2fs_inode_size_set(fs, &inode, size);
+ retval = ext2fs_inode_size_set(fs, &inode, num * fs->blocksize);
if (retval)
- goto errout;
+ return retval;
- retval = ext2fs_write_new_inode(fs, *ino, &inode);
+ retval = ext2fs_write_inode(fs, *ino, &inode);
if (retval)
goto errout;
@@ -379,13 +309,7 @@ retry:
goto retry;
}
- if (retval)
- goto errout;
-
errout:
- if (handle)
- ext2fs_extent_free(handle);
-
return retval;
}
@@ -572,6 +496,8 @@ errcode_t mk_hugefiles(ext2_filsys fs, const char *device_name)
printf(_("with %llu blocks each"), num_blocks);
fputs(": ", stdout);
}
+ if (num_blocks == 0)
+ num_blocks = ext2fs_blocks_count(fs->super) - goal;
for (i=0; i < num_files; i++) {
ext2_ino_t ino;
diff --git a/tests/t_enable_mcsum_ext3/expect b/tests/t_enable_mcsum_ext3/expect
index 5460482..0f761a9 100644
--- a/tests/t_enable_mcsum_ext3/expect
+++ b/tests/t_enable_mcsum_ext3/expect
@@ -49,8 +49,8 @@ Change in FS metadata:
Reserved GDT blocks at 4-259
Block bitmap at 260 (+259)
@@ -45,7 +46,7 @@
- 7789 free blocks, 1013 free inodes, 2 directories
- Free blocks: 404-8192
+ 0 free blocks, 1013 free inodes, 2 directories
+ Free blocks:
Free inodes: 12-1024
-Group 1: (Blocks 8193-16384)
+Group 1: (Blocks 8193-16384) [ITABLE_ZEROED]
@@ -58,8 +58,8 @@ Change in FS metadata:
Reserved GDT blocks at 8196-8451
Block bitmap at 8452 (+259)
@@ -54,6 +55,6 @@
- 7803 free blocks, 1024 free inodes, 0 directories
- Free blocks: 8582-16384
+ 0 free blocks, 1024 free inodes, 0 directories
+ Free blocks:
Free inodes: 1025-2048
-Group 2: (Blocks 16385-24576)
+Group 2: (Blocks 16385-24576) [ITABLE_ZEROED]
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v2 11/14] libext2fs: use fallocate for creating journals and hugefiles
2015-05-18 21:18 ` [PATCH v2 " Darrick J. Wong
@ 2015-06-11 0:12 ` Theodore Ts'o
0 siblings, 0 replies; 39+ messages in thread
From: Theodore Ts'o @ 2015-06-11 0:12 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Mon, May 18, 2015 at 02:18:26PM -0700, Darrick J. Wong wrote:
> Use the new fallocate API for creating the journal and the mk_hugefile
> feature.
>
> v2: Use all remaining disk space when the hugefile size is zero. This
> was a corner case that I missed. Also fix a bug where the goal didn't
> get used; this fixes a lot of churn in the test case expect files.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Thanks, applied.
- Ted
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH 12/14] debugfs: implement fallocate
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (10 preceding siblings ...)
2015-05-14 0:22 ` [PATCH 11/14] libext2fs: use fallocate for creating journals and hugefiles Darrick J. Wong
@ 2015-05-14 0:22 ` Darrick J. Wong
2015-06-11 0:12 ` Theodore Ts'o
2015-05-14 0:22 ` [PATCH 13/14] tests: test debugfs punch command Darrick J. Wong
` (3 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:22 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Implement a fallocate function for debugfs, and add some tests to
demonstrate that it works (more or less).
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
debugfs/debug_cmds.ct | 3 +
debugfs/debugfs.8.in | 7 +
debugfs/debugfs.c | 36 +++++++
debugfs/debugfs.h | 1
tests/d_fallocate/expect.gz | Bin
tests/d_fallocate/name | 1
tests/d_fallocate/script | 175 ++++++++++++++++++++++++++++++++++
tests/d_fallocate_bigalloc/expect.gz | Bin
tests/d_fallocate_bigalloc/name | 1
tests/d_fallocate_bigalloc/script | 176 ++++++++++++++++++++++++++++++++++
tests/d_fallocate_blkmap/expect | 58 +++++++++++
tests/d_fallocate_blkmap/name | 1
tests/d_fallocate_blkmap/script | 85 ++++++++++++++++
13 files changed, 544 insertions(+)
create mode 100644 tests/d_fallocate/expect.gz
create mode 100644 tests/d_fallocate/name
create mode 100644 tests/d_fallocate/script
create mode 100644 tests/d_fallocate_bigalloc/expect.gz
create mode 100644 tests/d_fallocate_bigalloc/name
create mode 100644 tests/d_fallocate_bigalloc/script
create mode 100644 tests/d_fallocate_blkmap/expect
create mode 100644 tests/d_fallocate_blkmap/name
create mode 100644 tests/d_fallocate_blkmap/script
diff --git a/debugfs/debug_cmds.ct b/debugfs/debug_cmds.ct
index c6f6d6c..34dad9e 100644
--- a/debugfs/debug_cmds.ct
+++ b/debugfs/debug_cmds.ct
@@ -157,6 +157,9 @@ request do_dirsearch, "Search a directory for a particular filename",
request do_bmap, "Calculate the logical->physical block mapping for an inode",
bmap;
+request do_fallocate, "Allocate uninitialized blocks to an inode",
+ fallocate;
+
request do_punch, "Punch (or truncate) blocks from an inode by deallocating them",
punch, truncate;
diff --git a/debugfs/debugfs.8.in b/debugfs/debugfs.8.in
index 9a09cbf..a463c73 100644
--- a/debugfs/debugfs.8.in
+++ b/debugfs/debugfs.8.in
@@ -306,6 +306,13 @@ from the file \fIfilespec\fR.
Expand the directory
.IR filespec .
.TP
+.BI fallocate " filespec start_block [end_block]
+Allocate and map uninitialized blocks into \fIfilespec\fR between
+logical block \fIstart_block\fR and \fIend_block\fR, inclusive. If
+\fIend_block\fR is not supplied, this function maps until it runs out
+of free disk blocks or the maximum file size is reached. Existing
+mappings are left alone.
+.TP
.BI feature " [fs_feature] [-fs_feature] ..."
Set or clear various filesystem features in the superblock. After setting
or clearing any filesystem features that were requested, print the current
diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
index c677f5f..40ec05f 100644
--- a/debugfs/debugfs.c
+++ b/debugfs/debugfs.c
@@ -2195,6 +2195,42 @@ void do_punch(int argc, char *argv[])
return;
}
}
+
+void do_fallocate(int argc, char *argv[])
+{
+ ext2_ino_t ino;
+ blk64_t start, end;
+ int err;
+ errcode_t errcode;
+
+ if (common_args_process(argc, argv, 3, 4, argv[0],
+ "<file> start_blk [end_blk]",
+ CHECK_FS_RW | CHECK_FS_BITMAPS))
+ return;
+
+ ino = string_to_inode(argv[1]);
+ if (!ino)
+ return;
+ err = strtoblk(argv[0], argv[2], "logical block", &start);
+ if (err)
+ return;
+ if (argc == 4) {
+ err = strtoblk(argv[0], argv[3], "logical block", &end);
+ if (err)
+ return;
+ } else
+ end = ~0;
+
+ errcode = ext2fs_fallocate(current_fs, EXT2_FALLOCATE_INIT_BEYOND_EOF,
+ ino, NULL, ~0ULL, start, end - start + 1);
+
+ if (errcode) {
+ com_err(argv[0], errcode,
+ "while fallocating inode %u from %llu to %llu\n", ino,
+ (unsigned long long) start, (unsigned long long) end);
+ return;
+ }
+}
#endif /* READ_ONLY */
void do_symlink(int argc, char *argv[])
diff --git a/debugfs/debugfs.h b/debugfs/debugfs.h
index e163d0a..76bb22c 100644
--- a/debugfs/debugfs.h
+++ b/debugfs/debugfs.h
@@ -166,6 +166,7 @@ extern void do_imap(int argc, char **argv);
extern void do_set_current_time(int argc, char **argv);
extern void do_supported_features(int argc, char **argv);
extern void do_punch(int argc, char **argv);
+extern void do_fallocate(int argc, char **argv);
extern void do_symlink(int argc, char **argv);
extern void do_dump_mmp(int argc, char **argv);
diff --git a/tests/d_fallocate/expect.gz b/tests/d_fallocate/expect.gz
new file mode 100644
index 0000000000000000000000000000000000000000..3e6ffc38595b7c451e3de0df3a16f795cf33421b
GIT binary patch
literal 3770
zcmZ9Nc{~$-{KqA}=E^ZmjxiFcMP=1u5+X-hxpK>W6gKA^k(*pYteD9aQf!j@-ZF9@
zIa|z%<Svs4zxDn8{`ft9|GXaW_xt(29`DEJ@%Tt50odT_7N+b+X*TO=5_LlRIp3=+
zUYwp2x!ot`yF<Qymm`s%12x3vEwp8VZp|Eg<<)~-o3D>}6x37KoE6w$b-7eoz5PW1
zRlzvTCtl@Ca_ekW)!OBmnT6E_^6ti0RNcX3*ym3z4_+tK^0)YW4x6-X(}3NZ$?Yn;
zpW7c8^!D0oyV(e<b;fMYm3tHG>l)x3GR?cK)g#dQKH*A*28L(XgRF}W72n*d5h_i2
zZ9e<RJ+!35=U3AnvY+-bH}KeC?)G9uRcqsv*F=Bc;CI}<d=w+*;NFHzMDmsp7tF@i
zetoU(#QduFAt`U!TTRm8-R|%2DEDu{KlhuM%ss?&vdQ@kSnHz0&-`rSWEPjL&*eR4
zfZ-G_yMBM4N!t(4Q(fx0J<+sUyT86L(NDvT4f5@-PWVjhF}Fi#6n!VS*p{B8^qNL^
zv*)bWMxuGpGMU!4(UE`U#^Agv)gFc%tNV`Z53JjN2|o1p3vFxC2%{;vH$@F*+!#@?
zZQ1ein+V5NH0^1>y$bUTcpL5)v9~=G=Kb8RG-x^ESF`2eIHJj_#p<x}V5oWW^Xiq=
zJ4=fn*8=?>F=bkIe$#N;VNB1K7B8LOKgb_f9X^)1rFb413{FRo@;zIZ#KF_{?jy9b
zBb}Q*x|a?oy=!xu7{dbhq%|H{Xmp9(nn*bxTkCbB<Q^$SV+>h9MR$43`L9$(j&kDH
z&<}x?UNZ+vW)Dlz@(gyA%(g&#Ebx<%na`i1;W12@jrVioSD_zkpPIi)e+wf~4piyW
z_jVYTIbkb4kv}~UkFsBy-?cr&qEv@(aOQL}hDG9FVtfLcb+*~VaxZ$!N1Qb?-xT*a
z61)lW#%YIRI)o?#2A5J_Q@e>CBK;L3@hN9zs)BK%LZJ#`*aah-hmp0V0zEdI&jeqb
z>oNVoLCeI`6O|p_&!E8=-^Dj@fnZ-eN(%8j77@Jh<!tbeIYm5O-#$*^B?YKFyDE$1
zQyIu|#XEF0j3le{!SgRLoRuh<j}^+la0lYguENQwekEYHN%S>0>Q^m9M{3Zj?|S*N
z0HM=zJ<++_1e-8QWH1&3auoR0Di>FGE@==5IR)SeJZ6N*sA*5loz&PYHVPTs?gwF>
z+eYx(q~NBQTu9gP-e%VZK_m##@D-uIB0%`_V^Qe`huY_NX8+EBBDXIh#Ii6KO{tlY
z;43!&N5kS2f@BDMm<$8LeX3C`IEbWysa7(|5&gIPE#f!t=9dnWGS>PwXZF#N8cMzU
zo86i1ti_>7`b7lyO8qxg0>^(4ENapNT&c&C7*@6k3X|ya8V0+XQftOvbAxKe+AHKR
zmiO<StBH)-kyPtRwDnTjU!|v+tmI>=b8e$<B4`LHL)Y$%3m#hsrl+9J+~%LmLl_{I
zVi;(I1$ub|lalMa6qM&YvT;6ZteyI<ML8;edulCYn@#NMI%uE?thU~c?&8l2;w|!_
z4yd7286ibay9~GB0lP#de3X%j7l`jKd{9cwOqLMWEZyC7O_`gyX`Om7ye2oV4Yx~;
zi|Z|5DniQKd5x`Yj?JBx%02tlF~Y#jgI2NCyXor>H;AvNZAPcKWx=DDe`TL1Npjy*
zqR;!D-r?a1IeCfYQriRSOr8*fgmI`^s>j?dii5I@b!UT=TL?ZWQy$NKHxikTp3_m4
zB+I`&lOh^|->N^61f)NVE&XcriuxWYlAlniF&=5;fGCt&eC#JOD1M&sw2P6<i1{fL
z&t#13-W^X`M=QEgF$FNY@uVy5&nIWu++I#ZR@*hRi8`r9TS#9UJ%M8i2OU-hklKz8
zh%P*5&XdAr=1~hpB^1MP!*6Is5Czi;v#Sy{K*9q`KX^qvvj_7q&{d+ca|w>nuYxAW
zyPQ7M*Mw7q;@>(K88Vn5GHd7ymx)rO#C$uhaWP!w&&JX3Ycdo}C94b0--=c%zncm#
z5;;Q;X~7=3EqiQPp@hk>Q<iJDd?KLS!NoTkhz)JQaw;o%N2}(6t&5O@u8)_QT1x5y
zv~u(!xV`DI-9L8RC`;wfM?_nM`$n?~8hK3Nmg!o4kEXa^B7c@Xf?4p$Prhs75}n(M
zoHpZzdW?YVx6A8~&wexc6+(w=O`lOK0!<8-J{xKfl&@R1KE&+38(cF{3n+B@l^DNw
z>7go{5eJ1<^FX^Jk1llu)zlbi+NqB35D4;w002+a#Q;hx&hiZSUcYU`QM{ts?RIrI
zCfRUij4&|d8q>4*)E23qa80TpQC*d8Q62x88PwkJLyG@Xv|)F>)a}(JK5gClu#4nP
z+20a1qfLn);Q3CGwQXIVOHuc9Ex|_@GtokM)0tw%<FV%2M?4JxMYmTbNC#xbk_KNq
z4!A!Wb3ndx_)k8{zYC>%$gb-`I^f*J8SL|JDQE49ezONH+rNmgE|1{dpE20C{x+*L
z-WGe{RdN6}?=|~As|5}u9e}si7nz59zQP8ZDvOwxf*N}y?9noACmBE8p*z?k(xjSL
zYWGwcrx>LBJgB$_pdw<fH!_$gj5w~cmrgV;yy2-(ED$3Bh<SRnkVp#};qpF>eQK3H
zJ(?bt1{Vf6FpATiFtUP;$=7$=14!2ap<GKP>@q7yuFg6YT+iUjn|5KB;g5V~5P1SX
zfr1huuiq-UPHe+kLIqc#a&8=*qQVJpjsudHAzm(85c0Wmr(PG=hHYP#5#)-jf6tB<
zOsEn`h`;{FF%koy+~c8WL&8bd?~@eqLJ6&`kta){u7A>l@MThA=LD}<xyP*!09%^|
z66jhgg)#UUm0;+yE^q%vM{p+>F8PEKj6AbA(xW4iAaM>x#x0PP!6kGSCpNzxgw`|+
zY2{AG#w|HoHV>hUEkVk#Pjo!K#=n66u-cXtdf=xzIuaW<+8wy5MUJXgdpqP?5WD@3
zmmf%RU`2j2&esi*%Vb^j(Gu0!h3QMN<EVogtHvSmfqe4--Z%6=9`F1ME<p<jK?k2n
z!CT9CSnIW8Y5kPXgepJT#~0}H0bIyk4)K?U%RL)A6p!M`%*bclX$oQ7{G!AMy}vx2
z?6t^KV^+AJouUnk2Y`{j7%H$eChl#y0^Uet3Fr<_iRP#^iR_N$n0MtA?qVldxT`5(
z&w~KB2!W&Q+)k109%J6OAr4T%`p34`AiWd390M=^BX9s^kXtT!01309R-Pl|vjxdP
z;pNjvmW5_nxzWjOCJs}*fZjVLL}TCk6XV>y?G$mVGtlU05AA@?i>RyoA6cP>OEsnz
z|8R}QEyuXby%MhkW=EPl#FP{H@H3iEzLXO`#m1cS>x=@V#<)NPRZhN+Q3HYS)+X>x
zGbaZ+WulL;^NK{Ao6ng7+FxQ3kN;1^bN`p1{*oe~eKgQsC3L=`D=G$fty@R^cySvR
zLVgYLfa(c2{JyF=Vu3|<8a)HarTqK674*b;g%^qPecTzWicz1=n!lOw{lWzf{gawJ
zHkiA9QN`&H&W@-u56uxhY!vS(eoIYJPux}$uzjka;VeL$+fvY(FC!KiE2(tSc{oc8
z#KGGc@K+0QJeGpcgi2sF#6r+k!C8|C2`R_2T0Jw6Vc;7cRKz4hwEG-oBmh&UJYst<
z^h&ToV|o|LhQDiR5%@jlm|$)(&6GaQOF&#_)C=8DBZdx|cTQHW8CwXBB~&ztfH*GA
zXWQ}cMu~3*{F@O*D<MM4gxa=@Ef6s)%Sy4Fd!;drtSZ4%;XNg&F;4GWr-p2ft*aT~
zg4v*E*VsTEII{qO$M5CcMBO+oIqYjMp6!<1sm`+0WD#5Zac7~hsSmFqNprpCu77zT
z+8nE6j(yX|Q#~2v<-&4^ZlW%n@6EbWh+rB(TAeMIn%O4y;Si?SeY<;>@2d}_GQ6Ui
z_iNglVWrzV)c_$c7h|kWg8SLbD5%zrCb(j^?2Vc%eNB)|;hS53`RaTzBuQ}CT%OZ%
zWxVzxi`Ud7Wkkt}>hXNfa7|<S%N<78F9eoW7suT-CX+LArX0m}FYpT_dARnvpKyWM
zSn85b7ttNW`X3(QFM1po&k3W}v_*U|y5ho~|8&5g^TN#%l`vyf?bCS$4oiJ^hxzhF
z`URi3?%W#YlRn;0X+ATm;4z=LX`-W&ESqZz1FF=A<$p5o)K7B~L4JF-9@&nqH!XGw
zmCX3oBzEzMYh!JUpZdUyZgTM-VL3=Ys9#R93;5(YfBm>=$);8sFj+y-HWFKT{vIq7
zHTT}P`92j>GJmR2%IyO3!8{n=m3uBnA1+c<4@cr*B;^zwGs-#Q>xdcjKk)7Sd$l0E
zM87Dj?Gu7cGge2?Mk%}}1bswDMJRs1&gM@Bb3W9j5wuwkQe1^tDr<f%6w}@0#U<I7
zP4sFH-$FD*lU)stNmL}C6zRVqDOu~gY54c|E+k!=T>78`ZLw>V&5cP)Kju#pB_$^r
zgtQdPbyOPN<Y?FDcDr~?M@qgc9G>5&0}X%WE<b-q?xwIho8;gZSy0ZIoJ6=tQ-HoV
zEu2rfMA?yEUidsgd)UhD?g<^Jjgq57KKc>KXv=E@A(CVPUw+TjQ4y(A>I9jeaqJ1v
zBuNB=_tfSLbfA{X<&#E@{(-tehQGi$H2I2_0M_j#_cTP>YRsYFkl@7t|7*AGejO%N
zy$NfuuiEobNcpz#GkoGv=90I-n6%kXN=OmmWm}ccfn!UpeMD18!|&Cqe%$uxxnQ8@
zY|Ii}ow74sH@A>zvA+S5FVpeaq@1XYUXmEKBAoovg9|!n*z>SE*!iyWb2q>0@L;}l
y0k>{<MPq+8UfTX(XFh&w5ttwuF<UycB^qw5mO16~r}$&Uk<ipHmv5P~vHcHq|7W-W
literal 0
HcmV?d00001
diff --git a/tests/d_fallocate/name b/tests/d_fallocate/name
new file mode 100644
index 0000000..72d0ed3
--- /dev/null
+++ b/tests/d_fallocate/name
@@ -0,0 +1 @@
+fallocate sparse files and big files
diff --git a/tests/d_fallocate/script b/tests/d_fallocate/script
new file mode 100644
index 0000000..ae8956e
--- /dev/null
+++ b/tests/d_fallocate/script
@@ -0,0 +1,175 @@
+if test -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fy
+OUT=$test_name.log
+if [ -f $test_dir/expect.gz ]; then
+ EXP=$test_name.tmp
+ gunzip < $test_dir/expect.gz > $EXP
+else
+ EXP=$test_dir/expect
+fi
+
+cp /dev/null $OUT
+
+cat > $TMPFILE.conf << ENDL
+[fs_types]
+ext4 = {
+ base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr,^has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,64bit
+ blocksize = 1024
+ inode_size = 256
+ inode_ratio = 16384
+}
+ENDL
+MKE2FS_CONFIG=$TMPFILE.conf $MKE2FS -F -o Linux -b 1024 -O ^bigalloc -T ext4 $TMPFILE 65536 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT 2>&1
+rm -rf $TMPFILE.conf
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+echo "debugfs write files" >> $OUT
+make_file() {
+ name="$1"
+ start="$2"
+ flag="$3"
+
+ cat << ENDL
+write /dev/null $name
+sif /$name size 40960
+eo /$name
+set_bmap $flag 10 $((start + 10))
+set_bmap $flag 13 $((start + 13))
+set_bmap $flag 26 $((start + 26))
+set_bmap $flag 29 $((start + 29))
+ec
+sif /$name blocks 8
+setb $((start + 10))
+setb $((start + 13))
+setb $((start + 26))
+setb $((start + 29))
+ENDL
+}
+
+#Files we create:
+# a: fallocate a 40k file
+# b*: falloc sparse file starting at b*
+# c*: falloc spare file ending at c*
+# d: midcluster to midcluster, surrounding sparse
+# e: partial middle cluster alloc
+# f: one big file
+# g*: falloc sparse init file starting at g*
+# h*: falloc sparse init file ending at h*
+# i: midcluster to midcluster, surrounding sparse init
+# j: partial middle cluster alloc
+# k: one big init file
+base=5000
+cat > $TMPFILE.cmd << ENDL
+write /dev/null a
+sif /a size 40960
+fallocate /a 0 39
+ENDL
+echo "ex /a" >> $TMPFILE.cmd2
+
+make_file sample $base --uninit >> $TMPFILE.cmd
+echo "ex /sample" >> $TMPFILE.cmd2
+base=10000
+
+for i in 8 9 10 11 12 13 14 15; do
+ make_file b$i $(($base + (40 * ($i - 8)))) --uninit >> $TMPFILE.cmd
+ echo "fallocate /b$i $i 39" >> $TMPFILE.cmd
+ echo "ex /b$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file c$i $(($base + 320 + (40 * ($i - 24)))) --uninit >> $TMPFILE.cmd
+ echo "fallocate /c$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /c$i" >> $TMPFILE.cmd2
+done
+
+make_file d $(($base + 640)) --uninit >> $TMPFILE.cmd
+echo "fallocate /d 4 35" >> $TMPFILE.cmd
+echo "ex /d" >> $TMPFILE.cmd2
+
+make_file e $(($base + 680)) --uninit >> $TMPFILE.cmd
+echo "fallocate /e 19 20" >> $TMPFILE.cmd
+echo "ex /e" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null f
+sif /f size 1024
+eo /f
+set_bmap --uninit 0 9000
+ec
+sif /f blocks 2
+setb 9000
+fallocate /f 0 8999
+ENDL
+echo "ex /f" >> $TMPFILE.cmd2
+
+# Now do it again, but with initialized blocks
+base=20000
+for i in 8 9 10 11 12 13 14 15; do
+ make_file g$i $(($base + (40 * ($i - 8)))) >> $TMPFILE.cmd
+ echo "fallocate /g$i $i 39" >> $TMPFILE.cmd
+ echo "ex /g$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file h$i $(($base + 320 + (40 * ($i - 24)))) >> $TMPFILE.cmd
+ echo "fallocate /h$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /h$i" >> $TMPFILE.cmd2
+done
+
+make_file i $(($base + 640)) >> $TMPFILE.cmd
+echo "fallocate /i 4 35" >> $TMPFILE.cmd
+echo "ex /i" >> $TMPFILE.cmd2
+
+make_file j $(($base + 680)) >> $TMPFILE.cmd
+echo "fallocate /j 19 20" >> $TMPFILE.cmd
+echo "ex /j" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null k
+sif /k size 1024
+eo /k
+set_bmap 0 19000
+ec
+sif /k blocks 2
+setb 19000
+fallocate /k 0 8999
+sif /k size 9216000
+ENDL
+echo "ex /k" >> $TMPFILE.cmd2
+
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE > /dev/null 2>&1
+$DEBUGFS_EXE -f $TMPFILE.cmd2 $TMPFILE >> $OUT.new 2>&1
+sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
+rm -rf $OUT.new $TMPFILE.cmd $TMPFILE.cmd2
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+rm -f $TMPFILE
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+ rm -f $test_name.tmp
+fi
+
+unset IMAGE FSCK_OPT OUT EXP
+
+else #if test -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
diff --git a/tests/d_fallocate_bigalloc/expect.gz b/tests/d_fallocate_bigalloc/expect.gz
new file mode 100644
index 0000000000000000000000000000000000000000..8640bc29dc71b09810f110d07d84d883b1109b88
GIT binary patch
literal 2673
zcmV-%3Xb(3iwFQXkyum$1KnF)Z``;QeGTSU5GV@l7Fk=o<fTLdMX}vYfB;!^fwTzv
zR7`BeR&CE1GviItUtftl8dDm5kloQs2MB0octq(O>GE7&&g$cOx2ZO_SK@G2ici(;
z`r%=FEk2j)!}G2b)n4om)x(2$-rSTsaZ~<QK5U;J%gtf+p)AGS;qbJ-y1XpE9`wbR
z>b|<TFL#^r;bObHy{u};*NdmSr^~+{PrKcT$8B@oP5tNN`cQ2*tJP1tdbIi)#OLav
z+<)62%17~~I@}4vtc;NN;!_R2-;4F;M(E6%8os?L_p8;9%~?J6cDH?g5{LDt`V?29
z{=9ni>Snts#l&azaaTF8JKz5b-akK;yWk3gyI5aeZ=W|9;&{j52&>hv>-}EHEAi7^
zd3}GpnEh+q^lNv{ef6LEeqC1zy53Z~^7^pdeXCX4JzqDH`A%xE)Ae?<ssF5gs}A3M
zKOOYjm7mL9xw$TdyJ!r@K@ZRI{`v87U7sz~@KEj#|H6%52`Mj~OzuC#clJX5QEaxG
zAL{WB)$Q~4dH?<Ex|7zIUSWTyt2bY(LwzIllecdkyliz-etN$Bycb_;rb~ae_W!O#
z`Bhx5SMQrAQoPw5c2zx;|GeMcR@dtXfAH(wx4k=hUv6&eXX@?4`gXq(*`>(pZ{=nE
z&-Kg14^5-2a97qGNm(fOozBER>X)QCbVuK>AD<q|b6kRMx=CC@-3re9k!#&YwwKW2
z=!jFmDc4^Cr$%t9#gXy2x~=>BBi9b&^bu!)v#!4d&h~H}sy-EGqe&FFU8<SNvgW96
z>m&E!S(-?>{>tOX(<CQo(!dq2zXq-t8SjLq`iW}#TaQz=H(r+}dp??MNR!6!44QN#
zO$?e$B25M~nM9flt}~yeNoMDN>l{xu^W$6aq#}8;o+p{5@gy@1TpCX@^E^p8!;@Tw
zC#gJ7Ql;@E6?oE#JZbQx6M4dOFL{P1S|W<q<L2NgpO4}c@|2UuVFIeZjN_>QPeme6
z1$eSEJgH@PBI$Xe_}R|!L<r-Q7{&=<oD#!0A&k>Z7?*k>o)|oFB2S3plo-beahww4
zI3bSHTpZ`$!;=gRoXo=Ui>%|-Rkmt+wg5gQ7J<KMd&B}qN8F3uYWjm%t446@h-2W`
z^=IIC1gDQU6AEID!zoM{QuT4%;yQvTOc-i<!nh`VHbF!vh-!@oK6*e95za7*ydVlQ
z%;}Au2;!b}g&9Vj&M<vJj!(LF{WWlUPLRMiH~k@qHY2zbLC6t;nW<?feDgK&`5a+~
zUju&g6MiQniORum5g$n$PnelhVMZLoP=3+$2T#QePi7gObmn=|GL0vVnMqR`PZ~3m
zrZk>3W+qKDJmm}Vq`?y>@`Ra5C(KN_>kpnxK9bRl@PwJEX(xFq7UHP@Pd1S!Of;OB
zVK6cbj?XY0PmBqpX(xHI%kadA|E$t@V#I$=%rF@7pLHTnh~qpH$5plvPl)537{?iL
zoQp)B5XU(o!^m8J@YEVC8sSMU!;?WAH;HlFAdZ{F48tIfn}iHQy8fOgqhv4scX+xv
zA4@u932%XE?V62b2~pf6<`)K0+$7`|?D~VH9QHXMWT{*xyD+KQg+X><60-|~pl>3w
zi#90v@jN|w)_>Wi&1{|-;L^-&p0(pm2Eor~&n=oqq_DP|TL>A-Ax#HRy9JxMSkSca
z4#j{Rz`4a{>3Z7ZVza|F?SLE&PWqm1;mNJOVpUO(ji+0jPxhJx26qzRiZL9wIR92P
z`CW4o%$<_PaHhp60?w@hVJ?wo441b!cSlXzcrik3rBObMBM!W<OM<}*jq+k!9D^6*
zauDlR#}rnpPH-}7aR$8P4KG@GoK|Bv+2V5Wk~`0c)i|Ax;i$zG;H7AIVd-%!$8g8z
zr2sFM9XY`981DGISkFtd?%d3+z>V_KJ}-XVxo)MKbHIxk<)wXIq+g}1Upe>M`}-^W
zczID2gRlK<alZGyIELZ+q04$<m^xwD^om7<ViASKqD}xdodC?Prvif8*|-yWO=p2d
zxt_)&;1>qpb&m^XLPBAQ!opG~=$f8WAe5Bq_QffL@3%)N3JG&ZbSy%N5xwmB^s?db
zx~G>M(94E;Y24vU)oM;JhF%=47eg<TsF#6WCR#6pyUC;V%Ar?2t=I3XhK>0ga`VrG
z&oRBS5TSfVVfk!MFblzKv|!c?)|T301@j?~!cyCeV7|y!w=z~RA0FvgmKzmJE+beo
z@h1u9>v$B_@n!_`6}+a66-+7!rV|Cz5KKo4hIk=4S}=xSJR=w_B$!DMENcKc1bZnc
z4hTk}U<e|zO_lVlU^xUU5(O(DSP?B)0l{pvU>0{{X9QEr2u4|=V7{_RVP$jX4I_9C
ziuN1?&q0ZvgWx$R+H(**2Ss}hg6E(a&mr|fg6UMj7=m%MU@#pNZ8`|1gAz>#!E{ix
z=^&U6iZ&gDctLZfgBKEP+H~Zng28kIrQ42Rd2d;TP_(76XxkAi@0Du_<yx94*XE}}
z!>p)Lv%)Q|Bea59Q3Gbh91)?=3T8zOn3cRmbc9wgD{9)T+$~cQuE$koswABEmTd?n
z;ijEfwgE0o`xN=KuPr5qWU7+~gk4Vs#OWJ7EVP1=P-7|kfY7?>50f)qc7BojWu=Z#
zcBZiGd@6UaMn@<(Q&@2B3(X4&olgpFPK7oQ+Dr+3n%^q}q4Qy(PeZq_W;gAy(5InW
z!GMgeJq3YShl0Tz1@-UKK-?7!<|wFq_XRV{2&QGCV7^G+v}uBAEXZgXE0~5k(lS;s
z4FfW|=r<ymhB*of@iT&Hn4_Qs-xn-jNH9(nOhYi8D42#|94#2kk>+T@U_eG!TL%Pl
z_XKkk><E|>4CW}<3(yy=SV%CNDp&!*ibTN*2xg-NgE`VRRxpN<7+vNb5Uepd43onn
zCg)TzhRF#wCG-We%Lv9QQ80#CVM-H>VOChh3dS%itfB>jSz%5T3}%IOv|unDJZd^r
zwvb?{rh{QRIMH-4Ob17s4u<LAXw$(k9c&W?gX!QRLNJxNo)GM1n?_$Sxr|^Y)pQt`
z4wGm)3`~cKHXR10!$g}71JhwtnqUT|!%&Q1()F|$=_T*$2v(g>E*;9%vmD9{WN$i@
z3&F$alMQ>5*_(moFdUn`8CVX(vDuq}<uJKTO4AK2hcU5fx`E{|c}$w_?glfzX;)+Q
zV4D8#WmaQyNhw=h2wPn!-0Cv9$%U}Vg~ClPliOPe+gm7cdkbNE3x(TTCO5VaHnvc>
zv87A<{W<nwsa|Z8*r6iB9V(q2SSol;TrIxcn_{h>BAPuh13R!3wym@^{QAzkPbqR|
z-ilyP%m2milHcyivi<eALyIrFZSza>?GC6vWa2x{GkWd6FK_<LhRpXDtGCtHzld+m
zugu3z@?D*GN%I2KT&vJQ_I?-rr{*KxzI*>heE8>wzrXoKeEji)_~mauzkB=c&CmYp
f9)!C;rDnRY1HYwqzH#!$)N=J-JwLQM>OTMg)z2kH
literal 0
HcmV?d00001
diff --git a/tests/d_fallocate_bigalloc/name b/tests/d_fallocate_bigalloc/name
new file mode 100644
index 0000000..915645c
--- /dev/null
+++ b/tests/d_fallocate_bigalloc/name
@@ -0,0 +1 @@
+fallocate sparse files and big files with bigalloc
diff --git a/tests/d_fallocate_bigalloc/script b/tests/d_fallocate_bigalloc/script
new file mode 100644
index 0000000..6b6bf97
--- /dev/null
+++ b/tests/d_fallocate_bigalloc/script
@@ -0,0 +1,176 @@
+if test -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fy
+OUT=$test_name.log
+if [ -f $test_dir/expect.gz ]; then
+ EXP=$test_name.tmp
+ gunzip < $test_dir/expect.gz > $EXP
+else
+ EXP=$test_dir/expect
+fi
+
+cp /dev/null $OUT
+
+cat > $TMPFILE.conf << ENDL
+[fs_types]
+ext4 = {
+ cluster_size = 8192
+ base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr,^has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,64bit
+ blocksize = 1024
+ inode_size = 256
+ inode_ratio = 16384
+}
+ENDL
+MKE2FS_CONFIG=$TMPFILE.conf $MKE2FS -F -o Linux -b 1024 -O bigalloc -T ext4 $TMPFILE 65536 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT 2>&1
+rm -rf $TMPFILE.conf
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+echo "debugfs write files" >> $OUT
+make_file() {
+ name="$1"
+ start="$2"
+ flag="$3"
+
+ cat << ENDL
+write /dev/null $name
+sif /$name size 40960
+eo /$name
+set_bmap $flag 10 $((start + 10))
+set_bmap $flag 13 $((start + 13))
+set_bmap $flag 26 $((start + 26))
+set_bmap $flag 29 $((start + 29))
+ec
+sif /$name blocks 32
+setb $((start + 10))
+setb $((start + 13))
+setb $((start + 26))
+setb $((start + 29))
+ENDL
+}
+
+#Files we create:
+# a: fallocate a 40k file
+# b*: falloc sparse file starting at b*
+# c*: falloc spare file ending at c*
+# d: midcluster to midcluster, surrounding sparse
+# e: partial middle cluster alloc
+# f: one big file
+# g*: falloc sparse init file starting at g*
+# h*: falloc sparse init file ending at h*
+# i: midcluster to midcluster, surrounding sparse init
+# j: partial middle cluster alloc
+# k: one big init file
+base=5000
+cat > $TMPFILE.cmd << ENDL
+write /dev/null a
+sif /a size 40960
+fallocate /a 0 39
+ENDL
+echo "ex /a" >> $TMPFILE.cmd2
+
+make_file sample $base --uninit >> $TMPFILE.cmd
+echo "ex /sample" >> $TMPFILE.cmd2
+base=10000
+
+for i in 8 9 10 11 12 13 14 15; do
+ make_file b$i $(($base + (40 * ($i - 8)))) --uninit >> $TMPFILE.cmd
+ echo "fallocate /b$i $i 39" >> $TMPFILE.cmd
+ echo "ex /b$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file c$i $(($base + 320 + (40 * ($i - 24)))) --uninit >> $TMPFILE.cmd
+ echo "fallocate /c$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /c$i" >> $TMPFILE.cmd2
+done
+
+make_file d $(($base + 640)) --uninit >> $TMPFILE.cmd
+echo "fallocate /d 4 35" >> $TMPFILE.cmd
+echo "ex /d" >> $TMPFILE.cmd2
+
+make_file e $(($base + 680)) --uninit >> $TMPFILE.cmd
+echo "fallocate /e 19 20" >> $TMPFILE.cmd
+echo "ex /e" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null f
+sif /f size 1024
+eo /f
+set_bmap --uninit 0 9000
+ec
+sif /f blocks 16
+setb 9000
+fallocate /f 0 8999
+ENDL
+echo "ex /f" >> $TMPFILE.cmd2
+
+# Now do it again, but with initialized blocks
+base=20000
+for i in 8 9 10 11 12 13 14 15; do
+ make_file g$i $(($base + (40 * ($i - 8)))) >> $TMPFILE.cmd
+ echo "fallocate /g$i $i 39" >> $TMPFILE.cmd
+ echo "ex /g$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file h$i $(($base + 320 + (40 * ($i - 24)))) >> $TMPFILE.cmd
+ echo "fallocate /h$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /h$i" >> $TMPFILE.cmd2
+done
+
+make_file i $(($base + 640)) >> $TMPFILE.cmd
+echo "fallocate /i 4 35" >> $TMPFILE.cmd
+echo "ex /i" >> $TMPFILE.cmd2
+
+make_file j $(($base + 680)) >> $TMPFILE.cmd
+echo "fallocate /j 19 20" >> $TMPFILE.cmd
+echo "ex /j" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null k
+sif /k size 1024
+eo /k
+set_bmap 0 19000
+ec
+sif /k blocks 16
+setb 19000
+fallocate /k 0 8999
+sif /k size 9216000
+ENDL
+echo "ex /k" >> $TMPFILE.cmd2
+
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE > /dev/null 2>&1
+$DEBUGFS_EXE -f $TMPFILE.cmd2 $TMPFILE >> $OUT.new 2>&1
+sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
+rm -rf $OUT.new $TMPFILE.cmd $TMPFILE.cmd2
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+rm -f $TMPFILE
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+ rm -f $test_name.tmp
+fi
+
+unset IMAGE FSCK_OPT OUT EXP
+
+else #if test -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
diff --git a/tests/d_fallocate_blkmap/expect b/tests/d_fallocate_blkmap/expect
new file mode 100644
index 0000000..f7ae606
--- /dev/null
+++ b/tests/d_fallocate_blkmap/expect
@@ -0,0 +1,58 @@
+Creating filesystem with 65536 1k blocks and 4096 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345
+
+Allocating group tables: \b\b\bdone
+Writing inode tables: \b\b\bdone
+Writing superblocks and filesystem accounting information: \b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/4096 files (0.0% non-contiguous), 2340/65536 blocks
+Exit status is 0
+debugfs write files
+debugfs: stat /a
+Inode: 12 Type: regular Mode: 0666 Flags: 0x0
+Generation: 0 Version: 0x00000000:00000000
+User: 0 Group: 0 Size: 40960
+File ACL: 0 Directory ACL: 0
+Links: 1 Blockcount: 82
+Fragment: Address: 0 Number: 0 Size: 0
+Size of extra inode fields: 28
+BLOCKS:
+(0-1):1312-1313, (2-11):8000-8009, (IND):8010, (12-39):8011-8038
+TOTAL: 41
+
+debugfs: stat /b
+Inode: 13 Type: regular Mode: 0666 Flags: 0x0
+Generation: 0 Version: 0x00000000:00000000
+User: 0 Group: 0 Size: 10240000
+File ACL: 0 Directory ACL: 0
+Links: 1 Blockcount: 20082
+Fragment: Address: 0 Number: 0 Size: 0
+Size of extra inode fields: 28
+BLOCKS:
+(0-11):10000-10011, (IND):10012, (12-267):10013-10268, (DIND):10269, (IND):10270, (268-523):10271-10526, (IND):10527, (524-779):10528-10783, (IND):10784, (780-1035):10785-11040, (IND):11041, (1036-1291):11042-11297, (IND):11298, (1292-1547):11299-11554, (IND):11555, (1548-1803):11556-11811, (IND):11812, (1804-2059):11813-12068, (IND):12069, (2060-2315):12070-12325, (IND):12326, (2316-2571):12327-12582, (IND):12583, (2572-2827):12584-12839, (IND):12840, (2828-3083):12841-13096, (IND):13097, (3084-3339):13098-13353, (IND):13354, (3340-3595):13355-13610, (IND):13611, (3596-3851):13612-13867, (IND):13868, (3852-4107):13869-14124, (IND):14125, (4108-4363):14126-14381, (IND):14382, (4364-4619):14383-14638, (IND):14639, (4620-4875):14640-14895, (IND):14896, (4876-5131):14897-15152, (IND):15153,
(5132-5387):15154-15409, (IND):15410, (5388-5643):15411-15666, (IND):15667, (5644-5899):15668-15923, (IND):15924, (5900-6155):15925-16180, (IND):16181, (6156-6411):16182-16437, (IND):16438,!
(6412-6667):16439-16694, (IND):16695, (6668-6923):16696-16951, (IND):16952, (6924-7179):16953-17208, (IND):17209, (7180-7435):17210-17465, (IND):17466, (7436-7691):17467-17722, (IND):17723, (7692-7947):17724-17979, (IND):17980, (7948-8203):17981-18236, (IND):18237, (8204-8459):18238-18493, (IND):18494, (8460-8715):18495-18750, (IND):18751, (8716-8971):18752-19007, (IND):19008, (8972-9227):19009-19264, (IND):19265, (9228-9483):19266-19521, (IND):19522, (9484-9739):19523-19778, (IND):19779, (9740-9995):19780-20035, (IND):20036, (9996-9999):20037-20040
+TOTAL: 10041
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (6841, counted=6840).
+Fix? yes
+
+Free blocks count wrong for group #1 (1551, counted=1550).
+Fix? yes
+
+Free blocks count wrong (53116, counted=53114).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 13/4096 files (7.7% non-contiguous), 12422/65536 blocks
+Exit status is 1
diff --git a/tests/d_fallocate_blkmap/name b/tests/d_fallocate_blkmap/name
new file mode 100644
index 0000000..ba2b61d
--- /dev/null
+++ b/tests/d_fallocate_blkmap/name
@@ -0,0 +1 @@
+fallocate sparse files and big files on a blockmap fs
diff --git a/tests/d_fallocate_blkmap/script b/tests/d_fallocate_blkmap/script
new file mode 100644
index 0000000..9c48cbc
--- /dev/null
+++ b/tests/d_fallocate_blkmap/script
@@ -0,0 +1,85 @@
+if test -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fy
+OUT=$test_name.log
+if [ -f $test_dir/expect.gz ]; then
+ EXP=$test_name.tmp
+ gunzip < $test_dir/expect.gz > $EXP1
+else
+ EXP=$test_dir/expect
+fi
+
+cp /dev/null $OUT
+
+cat > $TMPFILE.conf << ENDL
+[fs_types]
+ext4 = {
+ base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr,^has_journal,^extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,^64bit
+ blocksize = 1024
+ inode_size = 256
+ inode_ratio = 16384
+}
+ENDL
+MKE2FS_CONFIG=$TMPFILE.conf $MKE2FS -F -o Linux -b 1024 -O ^bigalloc -T ext4 $TMPFILE 65536 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT 2>&1
+rm -rf $TMPFILE.conf
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+echo "debugfs write files" >> $OUT
+
+#Files we create:
+# a: fallocate a 40k file
+# k: one big file
+base=5000
+cat > $TMPFILE.cmd << ENDL
+write /dev/null a
+sif /a bmap[2] 8000
+sif /a size 40960
+sif /a i_blocks 2
+setb 8000
+fallocate /a 0 39
+
+write /dev/null b
+sif /b size 10240000
+sif /b bmap[0] 10000
+sif /b i_blocks 2
+setb 10000
+fallocate /b 0 9999
+ENDL
+echo "stat /a" >> $TMPFILE.cmd2
+echo "stat /b" >> $TMPFILE.cmd2
+
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE > /dev/null 2>&1
+$DEBUGFS_EXE -f $TMPFILE.cmd2 $TMPFILE >> $OUT.new 2>&1
+sed -f $cmd_dir/filter.sed -e '/^.*time:.*$/d' < $OUT.new >> $OUT
+rm -rf $OUT.new $TMPFILE.cmd $TMPFILE.cmd2
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+rm -f $TMPFILE
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+ rm -f $test_name.tmp
+fi
+
+unset IMAGE FSCK_OPT OUT EXP
+
+else #if test -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
^ permalink raw reply related [flat|nested] 39+ messages in thread* [PATCH 13/14] tests: test debugfs punch command
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (11 preceding siblings ...)
2015-05-14 0:22 ` [PATCH 12/14] debugfs: implement fallocate Darrick J. Wong
@ 2015-05-14 0:22 ` Darrick J. Wong
2015-06-11 0:13 ` Theodore Ts'o
2015-05-18 21:17 ` [PATCH 15/14] libext2fs: remove unnecessary undo file flush calls Darrick J. Wong
` (2 subsequent siblings)
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-14 0:22 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Test punching out various parts of sparse files.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
tests/d_punch/expect | 176 +++++++++++++++++++++++++++++++++++++++++
tests/d_punch/name | 1
tests/d_punch/script | 129 ++++++++++++++++++++++++++++++
tests/d_punch_bigalloc/expect | 175 +++++++++++++++++++++++++++++++++++++++++
tests/d_punch_bigalloc/name | 1
tests/d_punch_bigalloc/script | 130 ++++++++++++++++++++++++++++++
6 files changed, 612 insertions(+)
create mode 100644 tests/d_punch/expect
create mode 100644 tests/d_punch/name
create mode 100644 tests/d_punch/script
create mode 100644 tests/d_punch_bigalloc/expect
create mode 100644 tests/d_punch_bigalloc/name
create mode 100644 tests/d_punch_bigalloc/script
diff --git a/tests/d_punch/expect b/tests/d_punch/expect
new file mode 100644
index 0000000..2b39f8e
--- /dev/null
+++ b/tests/d_punch/expect
@@ -0,0 +1,176 @@
+Creating filesystem with 65536 1k blocks and 4096 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345
+
+Allocating group tables: \b\b\bdone
+Writing inode tables: \b\b\bdone
+Writing superblocks and filesystem accounting information: \b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/4096 files (0.0% non-contiguous), 2345/65536 blocks
+Exit status is 0
+debugfs write files
+debugfs: ex /a
+Level Entries Logical Physical Length Flags
+debugfs: ex /sample
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1323 0
+ 1/ 1 1/ 5 0 - 9 1313 - 1322 10 Uninit
+ 1/ 1 2/ 5 11 - 12 1324 - 1325 2 Uninit
+ 1/ 1 3/ 5 14 - 25 1327 - 1338 12 Uninit
+ 1/ 1 4/ 5 27 - 28 1340 - 1341 2 Uninit
+ 1/ 1 5/ 5 30 - 39 1343 - 1352 10 Uninit
+debugfs: ex /b8
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1390 0
+ 1/ 1 1/ 4 0 - 0 1326 - 1326 1 Uninit
+ 1/ 1 2/ 4 1 - 1 1339 - 1339 1 Uninit
+ 1/ 1 3/ 4 2 - 2 1342 - 1342 1 Uninit
+ 1/ 1 4/ 4 3 - 7 1353 - 1357 5 Uninit
+debugfs: ex /b9
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1368 0
+ 1/ 1 1/ 1 0 - 8 1358 - 1366 9 Uninit
+debugfs: ex /b10
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1378 0
+ 1/ 1 1/ 2 0 - 0 1367 - 1367 1 Uninit
+ 1/ 1 2/ 2 1 - 9 1369 - 1377 9 Uninit
+debugfs: ex /b11
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1389 0
+ 1/ 1 1/ 1 0 - 9 1379 - 1388 10 Uninit
+debugfs: ex /b12
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1401 0
+ 1/ 1 1/ 2 0 - 9 1391 - 1400 10 Uninit
+ 1/ 1 2/ 2 11 - 11 1402 - 1402 1 Uninit
+debugfs: ex /b13
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1413 0
+ 1/ 1 1/ 2 0 - 9 1403 - 1412 10 Uninit
+ 1/ 1 2/ 2 11 - 12 1414 - 1415 2 Uninit
+debugfs: ex /b14
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1426 0
+ 1/ 1 1/ 2 0 - 9 1416 - 1425 10 Uninit
+ 1/ 1 2/ 2 11 - 12 1427 - 1428 2 Uninit
+debugfs: ex /b15
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1439 0
+ 1/ 1 1/ 3 0 - 9 1429 - 1438 10 Uninit
+ 1/ 1 2/ 3 11 - 12 1440 - 1441 2 Uninit
+ 1/ 1 3/ 3 14 - 14 1443 - 1443 1 Uninit
+debugfs: ex /c24
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 25 - 4294967295 1453 4294967271
+ 1/ 1 1/ 3 25 - 25 1468 - 1468 1 Uninit
+ 1/ 1 2/ 3 27 - 28 1470 - 1471 2 Uninit
+ 1/ 1 3/ 3 30 - 39 1473 - 1482 10 Uninit
+debugfs: ex /c25
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 27 - 4294967295 1483 4294967269
+ 1/ 1 1/ 2 27 - 28 1485 - 1486 2 Uninit
+ 1/ 1 2/ 2 30 - 39 1488 - 1497 10 Uninit
+debugfs: ex /c26
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 27 - 4294967295 1484 4294967269
+ 1/ 1 1/ 2 27 - 28 1498 - 1499 2 Uninit
+ 1/ 1 2/ 2 30 - 39 1501 - 1510 10 Uninit
+debugfs: ex /c27
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 28 - 4294967295 1487 4294967268
+ 1/ 1 1/ 2 28 - 28 1512 - 1512 1 Uninit
+ 1/ 1 2/ 2 30 - 39 1514 - 1523 10 Uninit
+debugfs: ex /c28
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 30 - 4294967295 1500 4294967266
+ 1/ 1 1/ 1 30 - 39 1526 - 1535 10 Uninit
+debugfs: ex /c29
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 30 - 4294967295 1511 4294967266
+ 1/ 1 1/ 1 30 - 39 1537 - 1546 10 Uninit
+debugfs: ex /c30
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 31 - 4294967295 1513 4294967265
+ 1/ 1 1/ 1 31 - 39 1549 - 1557 9 Uninit
+debugfs: ex /c31
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 32 - 4294967295 1524 4294967264
+ 1/ 1 1/ 1 32 - 39 1560 - 1567 8 Uninit
+debugfs: ex /d
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1525 0
+ 1/ 1 1/ 3 0 - 0 1442 - 1442 1 Uninit
+ 1/ 1 2/ 3 1 - 3 1444 - 1446 3 Uninit
+ 1/ 1 3/ 3 36 - 39 1573 - 1576 4 Uninit
+debugfs: ex /e
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1547 0
+ 1/ 1 1/ 11 0 - 5 1447 - 1452 6 Uninit
+ 1/ 1 2/ 11 6 - 9 1454 - 1457 4 Uninit
+ 1/ 1 3/ 11 11 - 12 1459 - 1460 2 Uninit
+ 1/ 1 4/ 11 14 - 18 1462 - 1466 5 Uninit
+ 1/ 1 5/ 11 21 - 21 1472 - 1472 1 Uninit
+ 1/ 1 6/ 11 22 - 22 1536 - 1536 1 Uninit
+ 1/ 1 7/ 11 23 - 23 1548 - 1548 1 Uninit
+ 1/ 1 8/ 11 24 - 25 1558 - 1559 2 Uninit
+ 1/ 1 9/ 11 27 - 28 1569 - 1570 2 Uninit
+ 1/ 1 10/ 11 30 - 30 1572 - 1572 1 Uninit
+ 1/ 1 11/ 11 31 - 39 1577 - 1585 9 Uninit
+debugfs: ex /f
+Level Entries Logical Physical Length Flags
+ 0/ 0 1/ 2 0 - 0 9000 - 9000 1 Uninit
+ 0/ 0 2/ 2 8999 - 8999 17999 - 17999 1 Uninit
+Pass 1: Checking inodes, blocks, and sizes
+Inode 15 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 16 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 17 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 18 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 19 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 20 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 21 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 22 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 23 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 24 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 25 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 26 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 27 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 28 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 29 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 30 extent tree (at level 1) could be shorter. Fix? yes
+
+Pass 1E: Optimizing extent trees
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #1 (7934, counted=7933).
+Fix? yes
+
+Free blocks count wrong (62939, counted=62938).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 32/4096 files (43.8% non-contiguous), 2598/65536 blocks
+Exit status is 1
diff --git a/tests/d_punch/name b/tests/d_punch/name
new file mode 100644
index 0000000..724639f
--- /dev/null
+++ b/tests/d_punch/name
@@ -0,0 +1 @@
+punch sparse files and big files
diff --git a/tests/d_punch/script b/tests/d_punch/script
new file mode 100644
index 0000000..7a77c69
--- /dev/null
+++ b/tests/d_punch/script
@@ -0,0 +1,129 @@
+if test -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fy
+OUT=$test_name.log
+if [ -f $test_dir/expect.gz ]; then
+ EXP=$test_name.tmp
+ gunzip < $test_dir/expect.gz > $EXP1
+else
+ EXP=$test_dir/expect
+fi
+
+cp /dev/null $OUT
+
+cat > $TMPFILE.conf << ENDL
+[fs_types]
+ext4 = {
+ base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr,^has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,64bit
+ blocksize = 1024
+ inode_size = 256
+ inode_ratio = 16384
+}
+ENDL
+MKE2FS_CONFIG=$TMPFILE.conf $MKE2FS -F -o Linux -b 1024 -O ^bigalloc -T ext4 $TMPFILE 65536 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT 2>&1
+rm -rf $TMPFILE.conf
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+echo "debugfs write files" >> $OUT
+make_file() {
+ name="$1"
+ start="$2"
+ flag="$3"
+
+ cat << ENDL
+write /dev/null $name
+fallocate /$name 0 39
+punch /$name 10 10
+punch /$name 13 13
+punch /$name 26 26
+punch /$name 29 29
+ENDL
+}
+
+#Files we create:
+# a: punch a 40k file
+# b*: punch sparse file starting at b*
+# c*: punch spare file ending at c*
+# d: midcluster to midcluster, surrounding sparse
+# e: partial middle cluster alloc
+# f: one big file
+base=5000
+cat > $TMPFILE.cmd << ENDL
+write /dev/null a
+fallocate /a 0 39
+punch /a 0 39
+ENDL
+echo "ex /a" >> $TMPFILE.cmd2
+
+make_file sample $base --uninit >> $TMPFILE.cmd
+echo "ex /sample" >> $TMPFILE.cmd2
+base=10000
+
+for i in 8 9 10 11 12 13 14 15; do
+ make_file b$i $(($base + (40 * ($i - 8)))) --uninit >> $TMPFILE.cmd
+ echo "punch /b$i $i 39" >> $TMPFILE.cmd
+ echo "ex /b$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file c$i $(($base + 320 + (40 * ($i - 24)))) --uninit >> $TMPFILE.cmd
+ echo "punch /c$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /c$i" >> $TMPFILE.cmd2
+done
+
+make_file d $(($base + 640)) --uninit >> $TMPFILE.cmd
+echo "punch /d 4 35" >> $TMPFILE.cmd
+echo "ex /d" >> $TMPFILE.cmd2
+
+make_file e $(($base + 680)) --uninit >> $TMPFILE.cmd
+echo "punch /e 19 20" >> $TMPFILE.cmd
+echo "ex /e" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null f
+sif /f size 1024
+eo /f
+set_bmap --uninit 0 9000
+ec
+sif /f blocks 2
+setb 9000
+fallocate /f 0 8999
+punch /f 1 8998
+ENDL
+echo "ex /f" >> $TMPFILE.cmd2
+
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE > /dev/null 2>&1
+$DEBUGFS_EXE -f $TMPFILE.cmd2 $TMPFILE >> $OUT.new 2>&1
+sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
+rm -rf $OUT.new $TMPFILE.cmd $TMPFILE.cmd2
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+rm -f $TMPFILE
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+ rm -f $test_name.tmp
+fi
+
+unset IMAGE FSCK_OPT OUT EXP
+
+else #if test -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
diff --git a/tests/d_punch_bigalloc/expect b/tests/d_punch_bigalloc/expect
new file mode 100644
index 0000000..0138c3a
--- /dev/null
+++ b/tests/d_punch_bigalloc/expect
@@ -0,0 +1,175 @@
+
+Warning: the bigalloc feature is still under development
+See https://ext4.wiki.kernel.org/index.php/Bigalloc for more information
+
+Creating filesystem with 65536 1k blocks and 4096 inodes
+
+Allocating group tables: \b\b\bdone
+Writing inode tables: \b\b\bdone
+Writing superblocks and filesystem accounting information: \b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/4096 files (9.1% non-contiguous), 1144/65536 blocks
+Exit status is 0
+debugfs write files
+debugfs: ex /a
+Level Entries Logical Physical Length Flags
+debugfs: ex /sample
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1184 0
+ 1/ 1 1/ 5 0 - 9 1144 - 1153 10 Uninit
+ 1/ 1 2/ 5 11 - 12 1155 - 1156 2 Uninit
+ 1/ 1 3/ 5 14 - 25 1158 - 1169 12 Uninit
+ 1/ 1 4/ 5 27 - 28 1171 - 1172 2 Uninit
+ 1/ 1 5/ 5 30 - 39 1174 - 1183 10 Uninit
+debugfs: ex /b8
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1232 0
+ 1/ 1 1/ 1 0 - 7 1192 - 1199 8 Uninit
+debugfs: ex /b9
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1248 0
+ 1/ 1 1/ 1 0 - 8 1200 - 1208 9 Uninit
+debugfs: ex /b10
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1272 0
+ 1/ 1 1/ 1 0 - 9 1216 - 1225 10 Uninit
+debugfs: ex /b11
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1296 0
+ 1/ 1 1/ 2 0 - 7 1240 - 1247 8 Uninit
+ 1/ 1 2/ 2 8 - 9 1256 - 1257 2 Uninit
+debugfs: ex /b12
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1320 0
+ 1/ 1 1/ 3 0 - 7 1264 - 1271 8 Uninit
+ 1/ 1 2/ 3 8 - 9 1280 - 1281 2 Uninit
+ 1/ 1 3/ 3 11 - 11 1283 - 1283 1 Uninit
+debugfs: ex /b13
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1344 0
+ 1/ 1 1/ 3 0 - 7 1288 - 1295 8 Uninit
+ 1/ 1 2/ 3 8 - 9 1304 - 1305 2 Uninit
+ 1/ 1 3/ 3 11 - 12 1307 - 1308 2 Uninit
+debugfs: ex /b14
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1368 0
+ 1/ 1 1/ 3 0 - 7 1312 - 1319 8 Uninit
+ 1/ 1 2/ 3 8 - 9 1328 - 1329 2 Uninit
+ 1/ 1 3/ 3 11 - 12 1331 - 1332 2 Uninit
+debugfs: ex /b15
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1392 0
+ 1/ 1 1/ 4 0 - 7 1336 - 1343 8 Uninit
+ 1/ 1 2/ 4 8 - 9 1352 - 1353 2 Uninit
+ 1/ 1 3/ 4 11 - 12 1355 - 1356 2 Uninit
+ 1/ 1 4/ 4 14 - 14 1358 - 1358 1 Uninit
+debugfs: ex /c24
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 25 - 4294967295 1416 4294967271
+ 1/ 1 1/ 3 25 - 25 1401 - 1401 1 Uninit
+ 1/ 1 2/ 3 27 - 28 1403 - 1404 2 Uninit
+ 1/ 1 3/ 3 30 - 39 1406 - 1415 10 Uninit
+debugfs: ex /c25
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 27 - 4294967295 1440 4294967269
+ 1/ 1 1/ 2 27 - 28 1427 - 1428 2 Uninit
+ 1/ 1 2/ 2 30 - 39 1430 - 1439 10 Uninit
+debugfs: ex /c26
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 27 - 4294967295 1464 4294967269
+ 1/ 1 1/ 2 27 - 28 1451 - 1452 2 Uninit
+ 1/ 1 2/ 2 30 - 39 1454 - 1463 10 Uninit
+debugfs: ex /c27
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 28 - 4294967295 1488 4294967268
+ 1/ 1 1/ 2 28 - 28 1476 - 1476 1 Uninit
+ 1/ 1 2/ 2 30 - 39 1478 - 1487 10 Uninit
+debugfs: ex /c28
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 30 - 4294967295 1512 4294967266
+ 1/ 1 1/ 1 30 - 39 1502 - 1511 10 Uninit
+debugfs: ex /c29
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 30 - 4294967295 1536 4294967266
+ 1/ 1 1/ 1 30 - 39 1526 - 1535 10 Uninit
+debugfs: ex /c30
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 31 - 4294967295 1560 4294967265
+ 1/ 1 1/ 1 31 - 39 1551 - 1559 9 Uninit
+debugfs: ex /c31
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 32 - 4294967295 1584 4294967264
+ 1/ 1 1/ 1 32 - 39 1576 - 1583 8 Uninit
+debugfs: ex /d
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1600 0
+ 1/ 1 1/ 2 0 - 3 1360 - 1363 4 Uninit
+ 1/ 1 2/ 2 36 - 39 1596 - 1599 4 Uninit
+debugfs: ex /e
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1624 0
+ 1/ 1 1/ 8 0 - 9 1376 - 1385 10 Uninit
+ 1/ 1 2/ 8 11 - 12 1387 - 1388 2 Uninit
+ 1/ 1 3/ 8 14 - 15 1390 - 1391 2 Uninit
+ 1/ 1 4/ 8 16 - 18 1568 - 1570 3 Uninit
+ 1/ 1 5/ 8 21 - 23 1573 - 1575 3 Uninit
+ 1/ 1 6/ 8 24 - 25 1608 - 1609 2 Uninit
+ 1/ 1 7/ 8 27 - 28 1611 - 1612 2 Uninit
+ 1/ 1 8/ 8 30 - 39 1614 - 1623 10 Uninit
+debugfs: ex /f
+Level Entries Logical Physical Length Flags
+ 0/ 0 1/ 2 0 - 0 9000 - 9000 1 Uninit
+ 0/ 0 2/ 2 8999 - 8999 17999 - 17999 1 Uninit
+Pass 1: Checking inodes, blocks, and sizes
+Inode 14 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 15 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 16 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 17 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 18 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 19 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 20 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 22 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 23 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 24 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 25 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 26 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 27 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 28 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 29 extent tree (at level 1) could be shorter. Fix? yes
+
+Inode 30 extent tree (at level 1) could be shorter. Fix? yes
+
+Pass 1E: Optimizing extent trees
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (8003, counted=8002).
+Fix? yes
+
+Free blocks count wrong (64024, counted=64016).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 32/4096 files (43.8% non-contiguous), 1520/65536 blocks
+Exit status is 1
diff --git a/tests/d_punch_bigalloc/name b/tests/d_punch_bigalloc/name
new file mode 100644
index 0000000..6d61ebe
--- /dev/null
+++ b/tests/d_punch_bigalloc/name
@@ -0,0 +1 @@
+punch sparse files and big files with bigalloc
diff --git a/tests/d_punch_bigalloc/script b/tests/d_punch_bigalloc/script
new file mode 100644
index 0000000..6eb0571
--- /dev/null
+++ b/tests/d_punch_bigalloc/script
@@ -0,0 +1,130 @@
+if test -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fy
+OUT=$test_name.log
+if [ -f $test_dir/expect.gz ]; then
+ EXP=$test_name.tmp
+ gunzip < $test_dir/expect.gz > $EXP1
+else
+ EXP=$test_dir/expect
+fi
+
+cp /dev/null $OUT
+
+cat > $TMPFILE.conf << ENDL
+[fs_types]
+ext4 = {
+ cluster_size = 8192
+ base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr,^has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,64bit
+ blocksize = 1024
+ inode_size = 256
+ inode_ratio = 16384
+}
+ENDL
+MKE2FS_CONFIG=$TMPFILE.conf $MKE2FS -F -o Linux -b 1024 -O bigalloc -T ext4 $TMPFILE 65536 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT 2>&1
+rm -rf $TMPFILE.conf
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+echo "debugfs write files" >> $OUT
+make_file() {
+ name="$1"
+ start="$2"
+ flag="$3"
+
+ cat << ENDL
+write /dev/null $name
+fallocate /$name 0 39
+punch /$name 10 10
+punch /$name 13 13
+punch /$name 26 26
+punch /$name 29 29
+ENDL
+}
+
+#Files we create:
+# a: punch a 40k file
+# b*: punch sparse file starting at b*
+# c*: punch spare file ending at c*
+# d: midcluster to midcluster, surrounding sparse
+# e: partial middle cluster alloc
+# f: one big file
+base=5000
+cat > $TMPFILE.cmd << ENDL
+write /dev/null a
+fallocate /a 0 39
+punch /a 0 39
+ENDL
+echo "ex /a" >> $TMPFILE.cmd2
+
+make_file sample $base --uninit >> $TMPFILE.cmd
+echo "ex /sample" >> $TMPFILE.cmd2
+base=10000
+
+for i in 8 9 10 11 12 13 14 15; do
+ make_file b$i $(($base + (40 * ($i - 8)))) --uninit >> $TMPFILE.cmd
+ echo "punch /b$i $i 39" >> $TMPFILE.cmd
+ echo "ex /b$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file c$i $(($base + 320 + (40 * ($i - 24)))) --uninit >> $TMPFILE.cmd
+ echo "punch /c$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /c$i" >> $TMPFILE.cmd2
+done
+
+make_file d $(($base + 640)) --uninit >> $TMPFILE.cmd
+echo "punch /d 4 35" >> $TMPFILE.cmd
+echo "ex /d" >> $TMPFILE.cmd2
+
+make_file e $(($base + 680)) --uninit >> $TMPFILE.cmd
+echo "punch /e 19 20" >> $TMPFILE.cmd
+echo "ex /e" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null f
+sif /f size 1024
+eo /f
+set_bmap --uninit 0 9000
+ec
+sif /f blocks 16
+setb 9000
+fallocate /f 0 8999
+punch /f 1 8998
+ENDL
+echo "ex /f" >> $TMPFILE.cmd2
+
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE > /dev/null 2>&1
+$DEBUGFS_EXE -f $TMPFILE.cmd2 $TMPFILE >> $OUT.new 2>&1
+sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
+rm -rf $OUT.new $TMPFILE.cmd $TMPFILE.cmd2
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+rm -f $TMPFILE
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+ rm -f $test_name.tmp
+fi
+
+unset IMAGE FSCK_OPT OUT EXP
+
+else #if test -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
^ permalink raw reply related [flat|nested] 39+ messages in thread* [PATCH 15/14] libext2fs: remove unnecessary undo file flush calls
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (12 preceding siblings ...)
2015-05-14 0:22 ` [PATCH 13/14] tests: test debugfs punch command Darrick J. Wong
@ 2015-05-18 21:17 ` Darrick J. Wong
2015-06-11 0:13 ` Theodore Ts'o
2015-06-05 1:38 ` [PATCH 16/14] libext2fs: require the inline data xattr on all inline data files Darrick J. Wong
[not found] ` <20150514002240.10785.35238.stgit@birch.djwong.org>
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-05-18 21:17 UTC (permalink / raw)
To: tytso; +Cc: linux-ext4
Remove all flushes of the undo file except for the one that happens just
prior to the file being closed; it seems that the arbitrary flushes
aren't sufficiently useful.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/undo_io.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/lib/ext2fs/undo_io.c b/lib/ext2fs/undo_io.c
index b9294c3..492d226 100644
--- a/lib/ext2fs/undo_io.c
+++ b/lib/ext2fs/undo_io.c
@@ -193,7 +193,6 @@ static errcode_t write_undo_indexes(struct undo_private_data *data, int flush)
data->keys_in_block = 0;
data->key_blk_num = data->undo_blk_num;
data->undo_blk_num++;
- flush = 1;
}
}
^ permalink raw reply related [flat|nested] 39+ messages in thread* [PATCH 16/14] libext2fs: require the inline data xattr on all inline data files
2015-05-14 0:21 [PATCH 00/14] e2fsprogs May 2015 patchbomb Darrick J. Wong
` (13 preceding siblings ...)
2015-05-18 21:17 ` [PATCH 15/14] libext2fs: remove unnecessary undo file flush calls Darrick J. Wong
@ 2015-06-05 1:38 ` Darrick J. Wong
2015-06-11 0:15 ` Theodore Ts'o
[not found] ` <20150514002240.10785.35238.stgit@birch.djwong.org>
15 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-06-05 1:38 UTC (permalink / raw)
To: tytso; +Cc: linux-ext4
Turns out that the kernel requires the inline data xattr to exist for
all inline data files, even if the inline data size is zero.
Therefore, never delete the xattr key, and teach e2fsck always to look
for it.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/pass1.c | 19 +++----------------
e2fsck/problem.c | 2 +-
lib/ext2fs/inline_data.c | 3 ---
tests/f_ea_value_crash/expect.1 | 17 +++++++++++++----
tests/f_ea_value_crash/expect.2 | 2 +-
tests/f_inline_no_ea/expect.1 | 22 ++++++++++++++++++++++
tests/f_inline_no_ea/expect.2 | 7 +++++++
tests/f_inline_no_ea/image.gz | Bin
tests/f_inline_no_ea/name | 1 +
tests/f_inlinedata_repair/expect.1 | 14 +++++++++++---
| 21 ++++++++++++++++++---
| 2 +-
| 21 ++++++++++++++++++---
| 2 +-
| 22 +++++++++++++++++-----
| 2 +-
16 files changed, 115 insertions(+), 42 deletions(-)
create mode 100644 tests/f_inline_no_ea/expect.1
create mode 100644 tests/f_inline_no_ea/expect.2
create mode 100644 tests/f_inline_no_ea/image.gz
create mode 100644 tests/f_inline_no_ea/name
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 3777e42..ad3246a 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -1278,7 +1278,6 @@ void e2fsck_pass1(e2fsck_t ctx)
/* Test for inline data flag but no attr */
if ((inode->i_flags & EXT4_INLINE_DATA_FL) && inlinedata_fs &&
- EXT2_I_SIZE(inode) > EXT4_MIN_INLINE_DATA_SIZE &&
(ino >= EXT2_FIRST_INODE(fs->super))) {
size_t size = 0;
errcode_t err;
@@ -1309,22 +1308,10 @@ void e2fsck_pass1(e2fsck_t ctx)
case EXT2_ET_NO_INLINE_DATA:
case EXT2_ET_EXT_ATTR_CSUM_INVALID:
case EXT2_ET_EA_BAD_VALUE_OFFSET:
- /* broken EA or no system.data EA; truncate */
+ /* broken EA or no system.data EA; remove */
if (fix_problem(ctx, PR_1_INLINE_DATA_NO_ATTR,
- &pctx)) {
- err = ext2fs_inode_size_set(fs, inode,
- sizeof(inode->i_block));
- if (err) {
- pctx.errcode = err;
- ctx->flags |= E2F_FLAG_ABORT;
- goto endit;
- }
- if (LINUX_S_ISLNK(inode->i_mode))
- inode->i_flags &= ~EXT4_INLINE_DATA_FL;
- e2fsck_write_inode(ctx, ino, inode,
- "pass1");
- failed_csum = 0;
- }
+ &pctx))
+ goto clear_inode;
break;
default:
/* Some other kind of non-xattr error? */
diff --git a/e2fsck/problem.c b/e2fsck/problem.c
index 4f4309e..80c6492 100644
--- a/e2fsck/problem.c
+++ b/e2fsck/problem.c
@@ -1053,7 +1053,7 @@ static struct e2fsck_problem problem_table[] = {
/* Inode has INLINE_DATA_FL flag but extended attribute not found */
{ PR_1_INLINE_DATA_NO_ATTR,
N_("@i %i has INLINE_DATA_FL flag but @a not found. "),
- PROMPT_TRUNCATE, 0 },
+ PROMPT_CLEAR, 0 },
/* Extents/inlinedata flag set on a device or socket inode */
{ PR_1_SPECIAL_EXTENTS_IDATA,
diff --git a/lib/ext2fs/inline_data.c b/lib/ext2fs/inline_data.c
index 6260c5f..187465c 100644
--- a/lib/ext2fs/inline_data.c
+++ b/lib/ext2fs/inline_data.c
@@ -558,9 +558,6 @@ errcode_t ext2fs_inline_data_set(ext2_filsys fs, ext2_ino_t ino,
}
if (size <= EXT4_MIN_INLINE_DATA_SIZE) {
- retval = ext2fs_inline_data_ea_remove(fs, ino);
- if (retval)
- return retval;
memcpy((void *)inode->i_block, buf, size);
return ext2fs_write_inode(fs, ino, inode);
}
diff --git a/tests/f_ea_value_crash/expect.1 b/tests/f_ea_value_crash/expect.1
index 8315358..3a884d1 100644
--- a/tests/f_ea_value_crash/expect.1
+++ b/tests/f_ea_value_crash/expect.1
@@ -1,15 +1,24 @@
Pass 1: Checking inodes, blocks, and sizes
-Inode 12 has INLINE_DATA_FL flag but extended attribute not found. Truncate? yes
-
-Inode 12 extended attribute is corrupt (allocation collision). Clear? yes
+Inode 12 has INLINE_DATA_FL flag but extended attribute not found. Clear? yes
Inode 13 extended attribute is corrupt (allocation collision). Clear? yes
Pass 2: Checking directory structure
+Entry 'a' in / (2) has deleted/unused inode 12. Clear? yes
+
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
+Inode bitmap differences: -12
+Fix? yes
+
+Free inodes count wrong for group #0 (115, counted=116).
+Fix? yes
+
+Free inodes count wrong (115, counted=116).
+Fix? yes
+
test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
-test_filesys: 13/128 files (0.0% non-contiguous), 17/512 blocks
+test_filesys: 12/128 files (0.0% non-contiguous), 17/512 blocks
Exit status is 1
diff --git a/tests/f_ea_value_crash/expect.2 b/tests/f_ea_value_crash/expect.2
index 06886a4..3b6073e 100644
--- a/tests/f_ea_value_crash/expect.2
+++ b/tests/f_ea_value_crash/expect.2
@@ -3,5 +3,5 @@ Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 13/128 files (0.0% non-contiguous), 17/512 blocks
+test_filesys: 12/128 files (0.0% non-contiguous), 17/512 blocks
Exit status is 0
diff --git a/tests/f_inline_no_ea/expect.1 b/tests/f_inline_no_ea/expect.1
new file mode 100644
index 0000000..9d3700e
--- /dev/null
+++ b/tests/f_inline_no_ea/expect.1
@@ -0,0 +1,22 @@
+Pass 1: Checking inodes, blocks, and sizes
+Inode 12 has INLINE_DATA_FL flag but extended attribute not found. Clear? yes
+
+Pass 2: Checking directory structure
+Entry 'a' in / (2) has deleted/unused inode 12. Clear? yes
+
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Inode bitmap differences: -12
+Fix? yes
+
+Free inodes count wrong for group #0 (116, counted=117).
+Fix? yes
+
+Free inodes count wrong (116, counted=117).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 11/128 files (0.0% non-contiguous), 17/512 blocks
+Exit status is 1
diff --git a/tests/f_inline_no_ea/expect.2 b/tests/f_inline_no_ea/expect.2
new file mode 100644
index 0000000..8025ccb
--- /dev/null
+++ b/tests/f_inline_no_ea/expect.2
@@ -0,0 +1,7 @@
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/128 files (0.0% non-contiguous), 17/512 blocks
+Exit status is 0
diff --git a/tests/f_inline_no_ea/image.gz b/tests/f_inline_no_ea/image.gz
new file mode 100644
index 0000000000000000000000000000000000000000..63c08578eb4296d6fcc78c72f857a8722169cfbf
GIT binary patch
literal 2511
zcmb2|=3rp?RS?R={Px!VY~er|h7aG@F?vU31hTj8+R+)|Dj;*)@y^ty?ga{5Nwb>@
zgSLpua{0ZTpCCT>NUX*?t|0fLQGzal?BdK$HPd6fJ2^An^53qrdswai{$1>(@00e_
zyf0E@u)gZD?}XEh`Ca*jEm<4wv=?1WSQNQuWp96thoJM`Ix|zhvxjR|>Dz|soSL_&
z$Vl}3`?dn(ss8uvfBg7x?Zfo+<?G&6R{lTX^!l}hl~q{m`+f6P&$Ta3y#C=!h)wO&
zi)XrT8}hW?TeCAa)&ANkhh0a_UVCfu{dmF1uz~Z_m-6zi=H5U1XT4hbH23O;52rX7
z85j;+D__1_q^@jj-G3_v28Iv!Pyhe_p4U*C=WI4(mGd-SKC|@??@FJ#x%Jf7R}W85
zl}?`iDOYjNyY7#RE~^36Z&>=J#Nyxi+E>PQAYSelX(k|D@P{8rk^)|@zIQ!{9jIfQ
zY`Oy}8qw7Lkp8^&<HV&mJpcZ)nj`jCagXocIGsg1mui=opL!>7dD9=;wo6@qlNKM{
zFYWO$`P+{WIe)q1<<_}>8qXRV&J-(l&VBYj;!OSO>z9N2WuNW8c3<&5S5D7PoBx*A
z^!}edv`^_jXYQx@Hs3YZ2L8MJ^Y{GPxcC3g`<eaT_#^t?cAMz0&%e8APyLiN|M8K%
zGo9-v{eL>qzCJE5<UcT^N5N<ajE2By2#kinXb6mk08K(*+wAzPxn_G97!())1v3Vl
literal 0
HcmV?d00001
diff --git a/tests/f_inline_no_ea/name b/tests/f_inline_no_ea/name
new file mode 100644
index 0000000..770273e
--- /dev/null
+++ b/tests/f_inline_no_ea/name
@@ -0,0 +1 @@
+remove inline data files which lack the xattr
diff --git a/tests/f_inlinedata_repair/expect.1 b/tests/f_inlinedata_repair/expect.1
index 9c84b14..89f0f7b 100644
--- a/tests/f_inlinedata_repair/expect.1
+++ b/tests/f_inlinedata_repair/expect.1
@@ -1,5 +1,5 @@
Pass 1: Checking inodes, blocks, and sizes
-Inode 12 has INLINE_DATA_FL flag but extended attribute not found. Truncate? yes
+Inode 12 has INLINE_DATA_FL flag but extended attribute not found. Clear? yes
Inode 16, i_size is 56, should be 60. Fix? yes
@@ -21,8 +21,7 @@ Salvage? yes
Directory inode 32, block #0, offset 4: directory corrupted
Salvage? yes
-Symlink /1 (inode #12) is invalid.
-Clear? yes
+Entry '1' in / (2) has deleted/unused inode 12. Clear? yes
Symlink /3 (inode #14) is invalid.
Clear? yes
@@ -52,6 +51,15 @@ Unattached zero-length inode 34. Clear? yes
Unattached zero-length inode 35. Clear? yes
Pass 5: Checking group summary information
+Inode bitmap differences: -12
+Fix? yes
+
+Free inodes count wrong for group #0 (101, counted=102).
+Fix? yes
+
+Free inodes count wrong (101, counted=102).
+Fix? yes
+
test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
test_filesys: 26/128 files (0.0% non-contiguous), 18/512 blocks
--git a/tests/f_write_ea_no_extra_isize/expect.1 b/tests/f_write_ea_no_extra_isize/expect.1
index b7e7438..50c81a9 100644
--- a/tests/f_write_ea_no_extra_isize/expect.1
+++ b/tests/f_write_ea_no_extra_isize/expect.1
@@ -1,12 +1,27 @@
Pass 1: Checking inodes, blocks, and sizes
+Inode 12 has INLINE_DATA_FL flag but extended attribute not found. Clear? yes
+
Pass 2: Checking directory structure
-Directory inode 12, block #0, offset 4: directory corrupted
-Salvage? yes
+Entry 'x' in / (2) has deleted/unused inode 12. Clear? yes
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
+Inode 2 ref count is 4, should be 3. Fix? yes
+
Pass 5: Checking group summary information
+Inode bitmap differences: -12
+Fix? yes
+
+Free inodes count wrong for group #0 (116, counted=117).
+Fix? yes
+
+Directories count wrong for group #0 (3, counted=2).
+Fix? yes
+
+Free inodes count wrong (116, counted=117).
+Fix? yes
+
test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
-test_filesys: 12/128 files (0.0% non-contiguous), 17/512 blocks
+test_filesys: 11/128 files (0.0% non-contiguous), 17/512 blocks
Exit status is 1
--git a/tests/f_write_ea_no_extra_isize/expect.2 b/tests/f_write_ea_no_extra_isize/expect.2
index 3b6073e..8025ccb 100644
--- a/tests/f_write_ea_no_extra_isize/expect.2
+++ b/tests/f_write_ea_no_extra_isize/expect.2
@@ -3,5 +3,5 @@ Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 12/128 files (0.0% non-contiguous), 17/512 blocks
+test_filesys: 11/128 files (0.0% non-contiguous), 17/512 blocks
Exit status is 0
--git a/tests/f_write_ea_toobig_extra_isize/expect.1 b/tests/f_write_ea_toobig_extra_isize/expect.1
index b7e7438..50c81a9 100644
--- a/tests/f_write_ea_toobig_extra_isize/expect.1
+++ b/tests/f_write_ea_toobig_extra_isize/expect.1
@@ -1,12 +1,27 @@
Pass 1: Checking inodes, blocks, and sizes
+Inode 12 has INLINE_DATA_FL flag but extended attribute not found. Clear? yes
+
Pass 2: Checking directory structure
-Directory inode 12, block #0, offset 4: directory corrupted
-Salvage? yes
+Entry 'x' in / (2) has deleted/unused inode 12. Clear? yes
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
+Inode 2 ref count is 4, should be 3. Fix? yes
+
Pass 5: Checking group summary information
+Inode bitmap differences: -12
+Fix? yes
+
+Free inodes count wrong for group #0 (116, counted=117).
+Fix? yes
+
+Directories count wrong for group #0 (3, counted=2).
+Fix? yes
+
+Free inodes count wrong (116, counted=117).
+Fix? yes
+
test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
-test_filesys: 12/128 files (0.0% non-contiguous), 17/512 blocks
+test_filesys: 11/128 files (0.0% non-contiguous), 17/512 blocks
Exit status is 1
--git a/tests/f_write_ea_toobig_extra_isize/expect.2 b/tests/f_write_ea_toobig_extra_isize/expect.2
index 3b6073e..8025ccb 100644
--- a/tests/f_write_ea_toobig_extra_isize/expect.2
+++ b/tests/f_write_ea_toobig_extra_isize/expect.2
@@ -3,5 +3,5 @@ Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 12/128 files (0.0% non-contiguous), 17/512 blocks
+test_filesys: 11/128 files (0.0% non-contiguous), 17/512 blocks
Exit status is 0
--git a/tests/f_write_ea_toosmall_extra_isize/expect.1 b/tests/f_write_ea_toosmall_extra_isize/expect.1
index eecfc9d..50c81a9 100644
--- a/tests/f_write_ea_toosmall_extra_isize/expect.1
+++ b/tests/f_write_ea_toosmall_extra_isize/expect.1
@@ -1,15 +1,27 @@
Pass 1: Checking inodes, blocks, and sizes
-Inode 12 has a extra size (1) which is invalid
-Fix? yes
+Inode 12 has INLINE_DATA_FL flag but extended attribute not found. Clear? yes
Pass 2: Checking directory structure
-Directory inode 12, block #0, offset 4: directory corrupted
-Salvage? yes
+Entry 'x' in / (2) has deleted/unused inode 12. Clear? yes
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
+Inode 2 ref count is 4, should be 3. Fix? yes
+
Pass 5: Checking group summary information
+Inode bitmap differences: -12
+Fix? yes
+
+Free inodes count wrong for group #0 (116, counted=117).
+Fix? yes
+
+Directories count wrong for group #0 (3, counted=2).
+Fix? yes
+
+Free inodes count wrong (116, counted=117).
+Fix? yes
+
test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
-test_filesys: 12/128 files (0.0% non-contiguous), 17/512 blocks
+test_filesys: 11/128 files (0.0% non-contiguous), 17/512 blocks
Exit status is 1
--git a/tests/f_write_ea_toosmall_extra_isize/expect.2 b/tests/f_write_ea_toosmall_extra_isize/expect.2
index 3b6073e..8025ccb 100644
--- a/tests/f_write_ea_toosmall_extra_isize/expect.2
+++ b/tests/f_write_ea_toosmall_extra_isize/expect.2
@@ -3,5 +3,5 @@ Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 12/128 files (0.0% non-contiguous), 17/512 blocks
+test_filesys: 11/128 files (0.0% non-contiguous), 17/512 blocks
Exit status is 0
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH 16/14] libext2fs: require the inline data xattr on all inline data files
2015-06-05 1:38 ` [PATCH 16/14] libext2fs: require the inline data xattr on all inline data files Darrick J. Wong
@ 2015-06-11 0:15 ` Theodore Ts'o
2015-07-23 21:12 ` Darrick J. Wong
0 siblings, 1 reply; 39+ messages in thread
From: Theodore Ts'o @ 2015-06-11 0:15 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Thu, Jun 04, 2015 at 06:38:56PM -0700, Darrick J. Wong wrote:
> Turns out that the kernel requires the inline data xattr to exist for
> all inline data files, even if the inline data size is zero.
> Therefore, never delete the xattr key, and teach e2fsck always to look
> for it.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
It seems reasonable to have e2fsck look for this case, but I wonder if
it might be better to fix it by changing the kernel and e2fsck and
libext2fs's punch function to clear the inline data flag when it
removes the inline data xattr, and to do so if the xattr data size is
zero.
- Ted
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH 16/14] libext2fs: require the inline data xattr on all inline data files
2015-06-11 0:15 ` Theodore Ts'o
@ 2015-07-23 21:12 ` Darrick J. Wong
0 siblings, 0 replies; 39+ messages in thread
From: Darrick J. Wong @ 2015-07-23 21:12 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Wed, Jun 10, 2015 at 08:15:30PM -0400, Theodore Ts'o wrote:
> On Thu, Jun 04, 2015 at 06:38:56PM -0700, Darrick J. Wong wrote:
> > Turns out that the kernel requires the inline data xattr to exist for
> > all inline data files, even if the inline data size is zero.
> > Therefore, never delete the xattr key, and teach e2fsck always to look
> > for it.
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
>
> It seems reasonable to have e2fsck look for this case, but I wonder if
> it might be better to fix it by changing the kernel and e2fsck and
> libext2fs's punch function to clear the inline data flag when it
> removes the inline data xattr, and to do so if the xattr data size is
> zero.
It would be much cleaner to do so, however... I tried doing this and found that
existing kernels expect to be able to maintain a pointer to the value area(!)
and barf when they don't find an xattr, even if it's empty. I suspect the
proper fix is to remove the shortcut and use the xattr fetch functions (since
the rest of the inline data code uses them) and maybe do the 'system.data' name
compression (and feature flag...) at the same time.
That is to say, if we bother changing anything at all. There's nothing
seriously wrong with the (weird) way it is now, and this patch exists to codify
that weird way into e2fsprogs so future programmers won't screw up the hidden
requirement about the xattr existence.
<shrug>
--D
>
> - Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 39+ messages in thread
[parent not found: <20150514002240.10785.35238.stgit@birch.djwong.org>]
* Re: [PATCH 14/14] misc: add fuse2fs, a FUSE server for e2fsprogs (v4.3)
[not found] ` <20150514002240.10785.35238.stgit@birch.djwong.org>
@ 2015-06-11 0:13 ` Theodore Ts'o
2015-06-15 18:37 ` Darrick J. Wong
0 siblings, 1 reply; 39+ messages in thread
From: Theodore Ts'o @ 2015-06-11 0:13 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Wed, May 13, 2015 at 05:22:40PM -0700, Darrick J. Wong wrote:
> This is the initial implementation of a FUSE server based on
> e2fsprogs. The point of this program is to enable ext4 to run on any
> OS that FUSE supports (and doesn't already have a native driver), such
> as MacOS X, BSDs, and Windows. The code requires FUSE API v28, which
> is available in Linux fuse and osxfuse releases that are available as
> of August 2013.
Thanks, applied.
- Ted
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH 14/14] misc: add fuse2fs, a FUSE server for e2fsprogs (v4.3)
2015-06-11 0:13 ` [PATCH 14/14] misc: add fuse2fs, a FUSE server for e2fsprogs (v4.3) Theodore Ts'o
@ 2015-06-15 18:37 ` Darrick J. Wong
2015-06-15 19:21 ` Theodore Ts'o
0 siblings, 1 reply; 39+ messages in thread
From: Darrick J. Wong @ 2015-06-15 18:37 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Wed, Jun 10, 2015 at 08:13:37PM -0400, Theodore Ts'o wrote:
> On Wed, May 13, 2015 at 05:22:40PM -0700, Darrick J. Wong wrote:
> > This is the initial implementation of a FUSE server based on
> > e2fsprogs. The point of this program is to enable ext4 to run on any
> > OS that FUSE supports (and doesn't already have a native driver), such
> > as MacOS X, BSDs, and Windows. The code requires FUSE API v28, which
> > is available in Linux fuse and osxfuse releases that are available as
> > of August 2013.
>
> Thanks, applied.
Hi Ted,
There's a merge error with this patch; the commit in -next removes all
the Makefile dependencies from misc/Makefile.in, whereas the patch
adds entries for {journal,revoke,recovery}.o.
This breaks a clean build because make doesn't know that it has
to generate profile.h for mke2fs.c.
--D
>
> - Ted
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH 14/14] misc: add fuse2fs, a FUSE server for e2fsprogs (v4.3)
2015-06-15 18:37 ` Darrick J. Wong
@ 2015-06-15 19:21 ` Theodore Ts'o
0 siblings, 0 replies; 39+ messages in thread
From: Theodore Ts'o @ 2015-06-15 19:21 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Mon, Jun 15, 2015 at 11:37:05AM -0700, Darrick J. Wong wrote:
> On Wed, Jun 10, 2015 at 08:13:37PM -0400, Theodore Ts'o wrote:
> > On Wed, May 13, 2015 at 05:22:40PM -0700, Darrick J. Wong wrote:
> > > This is the initial implementation of a FUSE server based on
> > > e2fsprogs. The point of this program is to enable ext4 to run on any
> > > OS that FUSE supports (and doesn't already have a native driver), such
> > > as MacOS X, BSDs, and Windows. The code requires FUSE API v28, which
> > > is available in Linux fuse and osxfuse releases that are available as
> > > of August 2013.
> >
> > Thanks, applied.
>
> Hi Ted,
>
> There's a merge error with this patch; the commit in -next removes all
> the Makefile dependencies from misc/Makefile.in, whereas the patch
> adds entries for {journal,revoke,recovery}.o.
>
> This breaks a clean build because make doesn't know that it has
> to generate profile.h for mke2fs.c.
It wasn't caused by a merge error; this patch added
$(srcdir)/do_journal.c to SRCS (I'm guessing this was a cut and paste
error), and this caused the "make depend" to blow out, and this caused
the Makefile to lose all of its dependencies. (To be fair, one of my
commits, 81f95d43d: "libext2fs, libe2p, misc: git rid of jfs_user.h"
also had a bug which caused "make depend" to fail as well.)
Anyway, I'll check in a fix. Thanks for the report!
- Ted
P.S. Could you take a look at my reply to your patch #16/14?
^ permalink raw reply [flat|nested] 39+ messages in thread