* [PATCH 01/24] metadump: Use boolean values true/false instead of 1/0
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 16:31 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 02/24] mdrestore: Fix logic used to check if target device is large enough Chandan Babu R
` (22 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
db/metadump.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/db/metadump.c b/db/metadump.c
index 27d1df432..6bcfd5bba 100644
--- a/db/metadump.c
+++ b/db/metadump.c
@@ -2421,12 +2421,12 @@ process_inode(
case S_IFDIR:
rval = process_inode_data(dip, TYP_DIR2);
if (dip->di_format == XFS_DINODE_FMT_LOCAL)
- need_new_crc = 1;
+ need_new_crc = true;
break;
case S_IFLNK:
rval = process_inode_data(dip, TYP_SYMLINK);
if (dip->di_format == XFS_DINODE_FMT_LOCAL)
- need_new_crc = 1;
+ need_new_crc = true;
break;
case S_IFREG:
rval = process_inode_data(dip, TYP_DATA);
@@ -2436,7 +2436,7 @@ process_inode(
case S_IFBLK:
case S_IFSOCK:
process_dev_inode(dip);
- need_new_crc = 1;
+ need_new_crc = true;
break;
default:
break;
@@ -2450,7 +2450,7 @@ process_inode(
attr_data.remote_val_count = 0;
switch (dip->di_aformat) {
case XFS_DINODE_FMT_LOCAL:
- need_new_crc = 1;
+ need_new_crc = true;
if (obfuscate || zero_stale_data)
process_sf_attr(dip);
break;
@@ -2469,7 +2469,7 @@ process_inode(
done:
/* Heavy handed but low cost; just do it as a catch-all. */
if (zero_stale_data)
- need_new_crc = 1;
+ need_new_crc = true;
if (crc_was_ok && need_new_crc)
libxfs_dinode_calc_crc(mp, dip);
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 01/24] metadump: Use boolean values true/false instead of 1/0
2023-05-23 9:00 ` [PATCH 01/24] metadump: Use boolean values true/false instead of 1/0 Chandan Babu R
@ 2023-05-23 16:31 ` Darrick J. Wong
0 siblings, 0 replies; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 16:31 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:27PM +0530, Chandan Babu R wrote:
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
Looks ok,
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
--D
> ---
> db/metadump.c | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/db/metadump.c b/db/metadump.c
> index 27d1df432..6bcfd5bba 100644
> --- a/db/metadump.c
> +++ b/db/metadump.c
> @@ -2421,12 +2421,12 @@ process_inode(
> case S_IFDIR:
> rval = process_inode_data(dip, TYP_DIR2);
> if (dip->di_format == XFS_DINODE_FMT_LOCAL)
> - need_new_crc = 1;
> + need_new_crc = true;
> break;
> case S_IFLNK:
> rval = process_inode_data(dip, TYP_SYMLINK);
> if (dip->di_format == XFS_DINODE_FMT_LOCAL)
> - need_new_crc = 1;
> + need_new_crc = true;
> break;
> case S_IFREG:
> rval = process_inode_data(dip, TYP_DATA);
> @@ -2436,7 +2436,7 @@ process_inode(
> case S_IFBLK:
> case S_IFSOCK:
> process_dev_inode(dip);
> - need_new_crc = 1;
> + need_new_crc = true;
> break;
> default:
> break;
> @@ -2450,7 +2450,7 @@ process_inode(
> attr_data.remote_val_count = 0;
> switch (dip->di_aformat) {
> case XFS_DINODE_FMT_LOCAL:
> - need_new_crc = 1;
> + need_new_crc = true;
> if (obfuscate || zero_stale_data)
> process_sf_attr(dip);
> break;
> @@ -2469,7 +2469,7 @@ process_inode(
> done:
> /* Heavy handed but low cost; just do it as a catch-all. */
> if (zero_stale_data)
> - need_new_crc = 1;
> + need_new_crc = true;
>
> if (crc_was_ok && need_new_crc)
> libxfs_dinode_calc_crc(mp, dip);
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 02/24] mdrestore: Fix logic used to check if target device is large enough
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
2023-05-23 9:00 ` [PATCH 01/24] metadump: Use boolean values true/false instead of 1/0 Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 16:32 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 03/24] metadump: Define and use struct metadump Chandan Babu R
` (21 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
The device size verification code should be writing XFS_MAX_SECTORSIZE bytes
to the end of the device rather than "sizeof(char *) * XFS_MAX_SECTORSIZE"
bytes.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
mdrestore/xfs_mdrestore.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
index 7c1a66c40..333282ed2 100644
--- a/mdrestore/xfs_mdrestore.c
+++ b/mdrestore/xfs_mdrestore.c
@@ -115,7 +115,7 @@ perform_restore(
} else {
/* ensure device is sufficiently large enough */
- char *lb[XFS_MAX_SECTORSIZE] = { NULL };
+ char lb[XFS_MAX_SECTORSIZE] = { 0 };
off64_t off;
off = sb.sb_dblocks * sb.sb_blocksize - sizeof(lb);
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 02/24] mdrestore: Fix logic used to check if target device is large enough
2023-05-23 9:00 ` [PATCH 02/24] mdrestore: Fix logic used to check if target device is large enough Chandan Babu R
@ 2023-05-23 16:32 ` Darrick J. Wong
0 siblings, 0 replies; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 16:32 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:28PM +0530, Chandan Babu R wrote:
> The device size verification code should be writing XFS_MAX_SECTORSIZE bytes
> to the end of the device rather than "sizeof(char *) * XFS_MAX_SECTORSIZE"
> bytes.
>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
Yikes.
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
--D
> ---
> mdrestore/xfs_mdrestore.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> index 7c1a66c40..333282ed2 100644
> --- a/mdrestore/xfs_mdrestore.c
> +++ b/mdrestore/xfs_mdrestore.c
> @@ -115,7 +115,7 @@ perform_restore(
> } else {
> /* ensure device is sufficiently large enough */
>
> - char *lb[XFS_MAX_SECTORSIZE] = { NULL };
> + char lb[XFS_MAX_SECTORSIZE] = { 0 };
> off64_t off;
>
> off = sb.sb_dblocks * sb.sb_blocksize - sizeof(lb);
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 03/24] metadump: Define and use struct metadump
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
2023-05-23 9:00 ` [PATCH 01/24] metadump: Use boolean values true/false instead of 1/0 Chandan Babu R
2023-05-23 9:00 ` [PATCH 02/24] mdrestore: Fix logic used to check if target device is large enough Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 16:35 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 04/24] metadump: Add initialization and release functions Chandan Babu R
` (20 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
This commit collects all state tracking variables in a new "struct metadump"
structure.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
db/metadump.c | 459 +++++++++++++++++++++++++++-----------------------
1 file changed, 244 insertions(+), 215 deletions(-)
diff --git a/db/metadump.c b/db/metadump.c
index 6bcfd5bba..806cdfd68 100644
--- a/db/metadump.c
+++ b/db/metadump.c
@@ -40,25 +40,27 @@ static const cmdinfo_t metadump_cmd =
N_("[-a] [-e] [-g] [-m max_extent] [-w] [-o] filename"),
N_("dump metadata to a file"), metadump_help };
-static FILE *outf; /* metadump file */
-
-static xfs_metablock_t *metablock; /* header + index + buffers */
-static __be64 *block_index;
-static char *block_buffer;
-
-static int num_indices;
-static int cur_index;
-
-static xfs_ino_t cur_ino;
-
-static int show_progress = 0;
-static int stop_on_read_error = 0;
-static int max_extent_size = DEFAULT_MAX_EXT_SIZE;
-static int obfuscate = 1;
-static int zero_stale_data = 1;
-static int show_warnings = 0;
-static int progress_since_warning = 0;
-static bool stdout_metadump;
+static struct metadump {
+ int version;
+ int show_progress;
+ int stop_on_read_error;
+ int max_extent_size;
+ int show_warnings;
+ int obfuscate;
+ int zero_stale_data;
+ int progress_since_warning;
+ bool dirty_log;
+ bool stdout_metadump;
+ xfs_ino_t cur_ino;
+ /* Metadump file */
+ FILE *outf;
+ /* header + index + buffers */
+ struct xfs_metablock *metablock;
+ __be64 *block_index;
+ char *block_buffer;
+ int num_indices;
+ int cur_index;
+} metadump;
void
metadump_init(void)
@@ -98,9 +100,9 @@ print_warning(const char *fmt, ...)
va_end(ap);
buf[sizeof(buf)-1] = '\0';
- fprintf(stderr, "%s%s: %s\n", progress_since_warning ? "\n" : "",
- progname, buf);
- progress_since_warning = 0;
+ fprintf(stderr, "%s%s: %s\n",
+ metadump.progress_since_warning ? "\n" : "", progname, buf);
+ metadump.progress_since_warning = 0;
}
static void
@@ -118,10 +120,10 @@ print_progress(const char *fmt, ...)
va_end(ap);
buf[sizeof(buf)-1] = '\0';
- f = stdout_metadump ? stderr : stdout;
+ f = metadump.stdout_metadump ? stderr : stdout;
fprintf(f, "\r%-59s", buf);
fflush(f);
- progress_since_warning = 1;
+ metadump.progress_since_warning = 1;
}
/*
@@ -136,17 +138,19 @@ print_progress(const char *fmt, ...)
static int
write_index(void)
{
+ struct xfs_metablock *metablock = metadump.metablock;
/*
* write index block and following data blocks (streaming)
*/
- metablock->mb_count = cpu_to_be16(cur_index);
- if (fwrite(metablock, (cur_index + 1) << BBSHIFT, 1, outf) != 1) {
+ metablock->mb_count = cpu_to_be16(metadump.cur_index);
+ if (fwrite(metablock, (metadump.cur_index + 1) << BBSHIFT, 1,
+ metadump.outf) != 1) {
print_warning("error writing to target file");
return -1;
}
- memset(block_index, 0, num_indices * sizeof(__be64));
- cur_index = 0;
+ memset(metadump.block_index, 0, metadump.num_indices * sizeof(__be64));
+ metadump.cur_index = 0;
return 0;
}
@@ -163,9 +167,10 @@ write_buf_segment(
int ret;
for (i = 0; i < len; i++, off++, data += BBSIZE) {
- block_index[cur_index] = cpu_to_be64(off);
- memcpy(&block_buffer[cur_index << BBSHIFT], data, BBSIZE);
- if (++cur_index == num_indices) {
+ metadump.block_index[metadump.cur_index] = cpu_to_be64(off);
+ memcpy(&metadump.block_buffer[metadump.cur_index << BBSHIFT],
+ data, BBSIZE);
+ if (++metadump.cur_index == metadump.num_indices) {
ret = write_index();
if (ret)
return -EIO;
@@ -388,11 +393,11 @@ scan_btree(
if (iocur_top->data == NULL) {
print_warning("cannot read %s block %u/%u", typtab[btype].name,
agno, agbno);
- rval = !stop_on_read_error;
+ rval = !metadump.stop_on_read_error;
goto pop_out;
}
- if (zero_stale_data) {
+ if (metadump.zero_stale_data) {
zero_btree_block(iocur_top->data, btype);
iocur_top->need_crc = 1;
}
@@ -446,7 +451,7 @@ scanfunc_freesp(
numrecs = be16_to_cpu(block->bb_numrecs);
if (numrecs > mp->m_alloc_mxr[1]) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid numrecs (%u) in %s block %u/%u",
numrecs, typtab[btype].name, agno, agbno);
return 1;
@@ -455,7 +460,7 @@ scanfunc_freesp(
pp = XFS_ALLOC_PTR_ADDR(mp, block, 1, mp->m_alloc_mxr[1]);
for (i = 0; i < numrecs; i++) {
if (!valid_bno(agno, be32_to_cpu(pp[i]))) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u/%u) "
"in %s block %u/%u",
agno, be32_to_cpu(pp[i]),
@@ -482,13 +487,13 @@ copy_free_bno_btree(
/* validate root and levels before processing the tree */
if (root == 0 || root > mp->m_sb.sb_agblocks) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u) in bnobt "
"root in agf %u", root, agno);
return 1;
}
if (levels > mp->m_alloc_maxlevels) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid level (%u) in bnobt root "
"in agf %u", levels, agno);
return 1;
@@ -510,13 +515,13 @@ copy_free_cnt_btree(
/* validate root and levels before processing the tree */
if (root == 0 || root > mp->m_sb.sb_agblocks) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u) in cntbt "
"root in agf %u", root, agno);
return 1;
}
if (levels > mp->m_alloc_maxlevels) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid level (%u) in cntbt root "
"in agf %u", levels, agno);
return 1;
@@ -543,7 +548,7 @@ scanfunc_rmapbt(
numrecs = be16_to_cpu(block->bb_numrecs);
if (numrecs > mp->m_rmap_mxr[1]) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid numrecs (%u) in %s block %u/%u",
numrecs, typtab[btype].name, agno, agbno);
return 1;
@@ -552,7 +557,7 @@ scanfunc_rmapbt(
pp = XFS_RMAP_PTR_ADDR(block, 1, mp->m_rmap_mxr[1]);
for (i = 0; i < numrecs; i++) {
if (!valid_bno(agno, be32_to_cpu(pp[i]))) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u/%u) "
"in %s block %u/%u",
agno, be32_to_cpu(pp[i]),
@@ -582,13 +587,13 @@ copy_rmap_btree(
/* validate root and levels before processing the tree */
if (root == 0 || root > mp->m_sb.sb_agblocks) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u) in rmapbt "
"root in agf %u", root, agno);
return 1;
}
if (levels > mp->m_rmap_maxlevels) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid level (%u) in rmapbt root "
"in agf %u", levels, agno);
return 1;
@@ -615,7 +620,7 @@ scanfunc_refcntbt(
numrecs = be16_to_cpu(block->bb_numrecs);
if (numrecs > mp->m_refc_mxr[1]) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid numrecs (%u) in %s block %u/%u",
numrecs, typtab[btype].name, agno, agbno);
return 1;
@@ -624,7 +629,7 @@ scanfunc_refcntbt(
pp = XFS_REFCOUNT_PTR_ADDR(block, 1, mp->m_refc_mxr[1]);
for (i = 0; i < numrecs; i++) {
if (!valid_bno(agno, be32_to_cpu(pp[i]))) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u/%u) "
"in %s block %u/%u",
agno, be32_to_cpu(pp[i]),
@@ -654,13 +659,13 @@ copy_refcount_btree(
/* validate root and levels before processing the tree */
if (root == 0 || root > mp->m_sb.sb_agblocks) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u) in refcntbt "
"root in agf %u", root, agno);
return 1;
}
if (levels > mp->m_refc_maxlevels) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid level (%u) in refcntbt root "
"in agf %u", levels, agno);
return 1;
@@ -785,7 +790,8 @@ in_lost_found(
/* Record the "lost+found" inode if we haven't done so already */
ASSERT(ino != 0);
- if (!orphanage_ino && is_orphanage_dir(mp, cur_ino, namelen, name))
+ if (!orphanage_ino && is_orphanage_dir(mp, metadump.cur_ino, namelen,
+ name))
orphanage_ino = ino;
/* We don't obfuscate the "lost+found" directory itself */
@@ -795,7 +801,7 @@ in_lost_found(
/* Most files aren't in "lost+found" at all */
- if (cur_ino != orphanage_ino)
+ if (metadump.cur_ino != orphanage_ino)
return 0;
/*
@@ -1219,7 +1225,7 @@ generate_obfuscated_name(
print_warning("duplicate name for inode %llu "
"in dir inode %llu\n",
(unsigned long long) ino,
- (unsigned long long) cur_ino);
+ (unsigned long long) metadump.cur_ino);
return;
}
@@ -1229,7 +1235,7 @@ generate_obfuscated_name(
print_warning("unable to record name for inode %llu "
"in dir inode %llu\n",
(unsigned long long) ino,
- (unsigned long long) cur_ino);
+ (unsigned long long) metadump.cur_ino);
}
static void
@@ -1245,9 +1251,9 @@ process_sf_dir(
ino_dir_size = be64_to_cpu(dip->di_size);
if (ino_dir_size > XFS_DFORK_DSIZE(dip, mp)) {
ino_dir_size = XFS_DFORK_DSIZE(dip, mp);
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid size in dir inode %llu",
- (long long)cur_ino);
+ (long long)metadump.cur_ino);
}
sfep = xfs_dir2_sf_firstentry(sfp);
@@ -1261,9 +1267,9 @@ process_sf_dir(
int namelen = sfep->namelen;
if (namelen == 0) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("zero length entry in dir inode "
- "%llu", (long long)cur_ino);
+ "%llu", (long long)metadump.cur_ino);
if (i != sfp->count - 1)
break;
namelen = ino_dir_size - ((char *)&sfep->name[0] -
@@ -1271,16 +1277,17 @@ process_sf_dir(
} else if ((char *)sfep - (char *)sfp +
libxfs_dir2_sf_entsize(mp, sfp, sfep->namelen) >
ino_dir_size) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("entry length in dir inode %llu "
- "overflows space", (long long)cur_ino);
+ "overflows space",
+ (long long)metadump.cur_ino);
if (i != sfp->count - 1)
break;
namelen = ino_dir_size - ((char *)&sfep->name[0] -
(char *)sfp);
}
- if (obfuscate)
+ if (metadump.obfuscate)
generate_obfuscated_name(
libxfs_dir2_sf_get_ino(mp, sfp, sfep),
namelen, &sfep->name[0]);
@@ -1290,7 +1297,8 @@ process_sf_dir(
}
/* zero stale data in rest of space in data fork, if any */
- if (zero_stale_data && (ino_dir_size < XFS_DFORK_DSIZE(dip, mp)))
+ if (metadump.zero_stale_data &&
+ (ino_dir_size < XFS_DFORK_DSIZE(dip, mp)))
memset(sfep, 0, XFS_DFORK_DSIZE(dip, mp) - ino_dir_size);
}
@@ -1346,18 +1354,18 @@ process_sf_symlink(
len = be64_to_cpu(dip->di_size);
if (len > XFS_DFORK_DSIZE(dip, mp)) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid size (%d) in symlink inode %llu",
- len, (long long)cur_ino);
+ len, (long long)metadump.cur_ino);
len = XFS_DFORK_DSIZE(dip, mp);
}
buf = (char *)XFS_DFORK_DPTR(dip);
- if (obfuscate)
+ if (metadump.obfuscate)
obfuscate_path_components(buf, len);
/* zero stale data in rest of space in data fork, if any */
- if (zero_stale_data && len < XFS_DFORK_DSIZE(dip, mp))
+ if (metadump.zero_stale_data && len < XFS_DFORK_DSIZE(dip, mp))
memset(&buf[len], 0, XFS_DFORK_DSIZE(dip, mp) - len);
}
@@ -1382,9 +1390,9 @@ process_sf_attr(
ino_attr_size = be16_to_cpu(asfp->hdr.totsize);
if (ino_attr_size > XFS_DFORK_ASIZE(dip, mp)) {
ino_attr_size = XFS_DFORK_ASIZE(dip, mp);
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid attr size in inode %llu",
- (long long)cur_ino);
+ (long long)metadump.cur_ino);
}
asfep = &asfp->list[0];
@@ -1394,19 +1402,20 @@ process_sf_attr(
int namelen = asfep->namelen;
if (namelen == 0) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("zero length attr entry in inode "
- "%llu", (long long)cur_ino);
+ "%llu", (long long)metadump.cur_ino);
break;
} else if ((char *)asfep - (char *)asfp +
xfs_attr_sf_entsize(asfep) > ino_attr_size) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("attr entry length in inode %llu "
- "overflows space", (long long)cur_ino);
+ "overflows space",
+ (long long)metadump.cur_ino);
break;
}
- if (obfuscate) {
+ if (metadump.obfuscate) {
generate_obfuscated_name(0, asfep->namelen,
&asfep->nameval[0]);
memset(&asfep->nameval[asfep->namelen], 'v',
@@ -1418,7 +1427,8 @@ process_sf_attr(
}
/* zero stale data in rest of space in attr fork, if any */
- if (zero_stale_data && (ino_attr_size < XFS_DFORK_ASIZE(dip, mp)))
+ if (metadump.zero_stale_data &&
+ (ino_attr_size < XFS_DFORK_ASIZE(dip, mp)))
memset(asfep, 0, XFS_DFORK_ASIZE(dip, mp) - ino_attr_size);
}
@@ -1429,7 +1439,7 @@ process_dir_free_block(
struct xfs_dir2_free *free;
struct xfs_dir3_icfree_hdr freehdr;
- if (!zero_stale_data)
+ if (!metadump.zero_stale_data)
return;
free = (struct xfs_dir2_free *)block;
@@ -1451,10 +1461,10 @@ process_dir_free_block(
break;
}
default:
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid magic in dir inode %llu "
"free block",
- (unsigned long long)cur_ino);
+ (unsigned long long)metadump.cur_ino);
break;
}
}
@@ -1466,7 +1476,7 @@ process_dir_leaf_block(
struct xfs_dir2_leaf *leaf;
struct xfs_dir3_icleaf_hdr leafhdr;
- if (!zero_stale_data)
+ if (!metadump.zero_stale_data)
return;
/* Yes, this works for dir2 & dir3. Difference is padding. */
@@ -1549,10 +1559,10 @@ process_dir_data_block(
}
if (be32_to_cpu(datahdr->magic) != wantmagic) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning(
"invalid magic in dir inode %llu block %ld",
- (unsigned long long)cur_ino, (long)offset);
+ (unsigned long long)metadump.cur_ino, (long)offset);
return;
}
@@ -1572,10 +1582,10 @@ process_dir_data_block(
if (dir_offset + free_length > end_of_data ||
!free_length ||
(free_length & (XFS_DIR2_DATA_ALIGN - 1))) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning(
"invalid length for dir free space in inode %llu",
- (long long)cur_ino);
+ (long long)metadump.cur_ino);
return;
}
if (be16_to_cpu(*xfs_dir2_data_unused_tag_p(dup)) !=
@@ -1588,7 +1598,7 @@ process_dir_data_block(
* actually at a variable offset, so zeroing &dup->tag
* is zeroing the free space in between
*/
- if (zero_stale_data) {
+ if (metadump.zero_stale_data) {
int zlen = free_length -
sizeof(xfs_dir2_data_unused_t);
@@ -1606,23 +1616,23 @@ process_dir_data_block(
if (dir_offset + length > end_of_data ||
ptr + length > endptr) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning(
"invalid length for dir entry name in inode %llu",
- (long long)cur_ino);
+ (long long)metadump.cur_ino);
return;
}
if (be16_to_cpu(*libxfs_dir2_data_entry_tag_p(mp, dep)) !=
dir_offset)
return;
- if (obfuscate)
+ if (metadump.obfuscate)
generate_obfuscated_name(be64_to_cpu(dep->inumber),
dep->namelen, &dep->name[0]);
dir_offset += length;
ptr += length;
/* Zero the unused space after name, up to the tag */
- if (zero_stale_data) {
+ if (metadump.zero_stale_data) {
/* 1 byte for ftype; don't bother with conditional */
int zlen =
(char *)libxfs_dir2_data_entry_tag_p(mp, dep) -
@@ -1658,7 +1668,7 @@ process_symlink_block(
print_warning("cannot read %s block %u/%u (%llu)",
typtab[btype].name, agno, agbno, s);
- rval = !stop_on_read_error;
+ rval = !metadump.stop_on_read_error;
goto out_pop;
}
link = iocur_top->data;
@@ -1666,10 +1676,10 @@ process_symlink_block(
if (xfs_has_crc((mp)))
link += sizeof(struct xfs_dsymlink_hdr);
- if (obfuscate)
+ if (metadump.obfuscate)
obfuscate_path_components(link, XFS_SYMLINK_BUF_SPACE(mp,
mp->m_sb.sb_blocksize));
- if (zero_stale_data) {
+ if (metadump.zero_stale_data) {
size_t linklen, zlen;
linklen = strlen(link);
@@ -1736,7 +1746,8 @@ process_attr_block(
if ((be16_to_cpu(leaf->hdr.info.magic) != XFS_ATTR_LEAF_MAGIC) &&
(be16_to_cpu(leaf->hdr.info.magic) != XFS_ATTR3_LEAF_MAGIC)) {
for (i = 0; i < attr_data.remote_val_count; i++) {
- if (obfuscate && attr_data.remote_vals[i] == offset)
+ if (metadump.obfuscate &&
+ attr_data.remote_vals[i] == offset)
/* Macros to handle both attr and attr3 */
memset(block +
(bs - XFS_ATTR3_RMT_BUF_SPACE(mp, bs)),
@@ -1753,9 +1764,9 @@ process_attr_block(
nentries * sizeof(xfs_attr_leaf_entry_t) +
xfs_attr3_leaf_hdr_size(leaf) >
XFS_ATTR3_RMT_BUF_SPACE(mp, bs)) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid attr count in inode %llu",
- (long long)cur_ino);
+ (long long)metadump.cur_ino);
return;
}
@@ -1770,22 +1781,22 @@ process_attr_block(
first_name = xfs_attr3_leaf_name(leaf, i);
if (be16_to_cpu(entry->nameidx) > mp->m_sb.sb_blocksize) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning(
"invalid attr nameidx in inode %llu",
- (long long)cur_ino);
+ (long long)metadump.cur_ino);
break;
}
if (entry->flags & XFS_ATTR_LOCAL) {
local = xfs_attr3_leaf_name_local(leaf, i);
if (local->namelen == 0) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning(
"zero length for attr name in inode %llu",
- (long long)cur_ino);
+ (long long)metadump.cur_ino);
break;
}
- if (obfuscate) {
+ if (metadump.obfuscate) {
generate_obfuscated_name(0, local->namelen,
&local->nameval[0]);
memset(&local->nameval[local->namelen], 'v',
@@ -1797,18 +1808,18 @@ process_attr_block(
zlen = xfs_attr_leaf_entsize_local(nlen, vlen) -
(sizeof(xfs_attr_leaf_name_local_t) - 1 +
nlen + vlen);
- if (zero_stale_data)
+ if (metadump.zero_stale_data)
memset(&local->nameval[nlen + vlen], 0, zlen);
} else {
remote = xfs_attr3_leaf_name_remote(leaf, i);
if (remote->namelen == 0 || remote->valueblk == 0) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning(
"invalid attr entry in inode %llu",
- (long long)cur_ino);
+ (long long)metadump.cur_ino);
break;
}
- if (obfuscate) {
+ if (metadump.obfuscate) {
generate_obfuscated_name(0, remote->namelen,
&remote->name[0]);
add_remote_vals(be32_to_cpu(remote->valueblk),
@@ -1819,13 +1830,13 @@ process_attr_block(
zlen = xfs_attr_leaf_entsize_remote(nlen) -
(sizeof(xfs_attr_leaf_name_remote_t) - 1 +
nlen);
- if (zero_stale_data)
+ if (metadump.zero_stale_data)
memset(&remote->name[nlen], 0, zlen);
}
}
/* Zero from end of entries array to the first name/val */
- if (zero_stale_data) {
+ if (metadump.zero_stale_data) {
struct xfs_attr_leaf_entry *entries;
entries = xfs_attr3_leaf_entryp(leaf);
@@ -1858,16 +1869,16 @@ process_single_fsb_objects(
print_warning("cannot read %s block %u/%u (%llu)",
typtab[btype].name, agno, agbno, s);
- rval = !stop_on_read_error;
+ rval = !metadump.stop_on_read_error;
goto out_pop;
}
- if (!obfuscate && !zero_stale_data)
+ if (!metadump.obfuscate && !metadump.zero_stale_data)
goto write;
/* Zero unused part of interior nodes */
- if (zero_stale_data) {
+ if (metadump.zero_stale_data) {
xfs_da_intnode_t *node = iocur_top->data;
int magic = be16_to_cpu(node->hdr.info.magic);
@@ -1978,12 +1989,12 @@ process_multi_fsb_dir(
print_warning("cannot read %s block %u/%u (%llu)",
typtab[btype].name, agno, agbno, s);
- rval = !stop_on_read_error;
+ rval = !metadump.stop_on_read_error;
goto out_pop;
}
- if (!obfuscate && !zero_stale_data)
+ if (!metadump.obfuscate && !metadump.zero_stale_data)
goto write;
dp = iocur_top->data;
@@ -2075,25 +2086,27 @@ process_bmbt_reclist(
* one is found, stop processing remaining extents
*/
if (i > 0 && op + cp > o) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("bmap extent %d in %s ino %llu "
"starts at %llu, previous extent "
"ended at %llu", i,
- typtab[btype].name, (long long)cur_ino,
+ typtab[btype].name,
+ (long long)metadump.cur_ino,
o, op + cp - 1);
break;
}
- if (c > max_extent_size) {
+ if (c > metadump.max_extent_size) {
/*
* since we are only processing non-data extents,
* large numbers of blocks in a metadata extent is
* extremely rare and more than likely to be corrupt.
*/
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("suspicious count %u in bmap "
"extent %d in %s ino %llu", c, i,
- typtab[btype].name, (long long)cur_ino);
+ typtab[btype].name,
+ (long long)metadump.cur_ino);
break;
}
@@ -2104,19 +2117,21 @@ process_bmbt_reclist(
agbno = XFS_FSB_TO_AGBNO(mp, s);
if (!valid_bno(agno, agbno)) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number %u/%u "
"(%llu) in bmap extent %d in %s ino "
"%llu", agno, agbno, s, i,
- typtab[btype].name, (long long)cur_ino);
+ typtab[btype].name,
+ (long long)metadump.cur_ino);
break;
}
if (!valid_bno(agno, agbno + c - 1)) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("bmap extent %i in %s inode %llu "
"overflows AG (end is %u/%u)", i,
- typtab[btype].name, (long long)cur_ino,
+ typtab[btype].name,
+ (long long)metadump.cur_ino,
agno, agbno + c - 1);
break;
}
@@ -2152,7 +2167,7 @@ scanfunc_bmap(
if (level == 0) {
if (nrecs > mp->m_bmap_dmxr[0]) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid numrecs (%u) in %s "
"block %u/%u", nrecs,
typtab[btype].name, agno, agbno);
@@ -2163,7 +2178,7 @@ scanfunc_bmap(
}
if (nrecs > mp->m_bmap_dmxr[1]) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid numrecs (%u) in %s block %u/%u",
nrecs, typtab[btype].name, agno, agbno);
return 1;
@@ -2178,7 +2193,7 @@ scanfunc_bmap(
if (bno == 0 || bno > mp->m_sb.sb_agblocks ||
ag > mp->m_sb.sb_agcount) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u/%u) "
"in %s block %u/%u", ag, bno,
typtab[btype].name, agno, agbno);
@@ -2213,10 +2228,10 @@ process_btinode(
nrecs = be16_to_cpu(dib->bb_numrecs);
if (level > XFS_BM_MAXLEVELS(mp, whichfork)) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid level (%u) in inode %lld %s "
- "root", level, (long long)cur_ino,
- typtab[btype].name);
+ "root", level, (long long)metadump.cur_ino,
+ typtab[btype].name);
return 1;
}
@@ -2227,16 +2242,16 @@ process_btinode(
maxrecs = libxfs_bmdr_maxrecs(XFS_DFORK_SIZE(dip, mp, whichfork), 0);
if (nrecs > maxrecs) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid numrecs (%u) in inode %lld %s "
- "root", nrecs, (long long)cur_ino,
- typtab[btype].name);
+ "root", nrecs, (long long)metadump.cur_ino,
+ typtab[btype].name);
return 1;
}
pp = XFS_BMDR_PTR_ADDR(dib, 1, maxrecs);
- if (zero_stale_data) {
+ if (metadump.zero_stale_data) {
char *top;
/* Unused btree key space */
@@ -2257,11 +2272,11 @@ process_btinode(
if (bno == 0 || bno > mp->m_sb.sb_agblocks ||
ag > mp->m_sb.sb_agcount) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u/%u) "
- "in inode %llu %s root", ag,
- bno, (long long)cur_ino,
- typtab[btype].name);
+ "in inode %llu %s root", ag, bno,
+ (long long)metadump.cur_ino,
+ typtab[btype].name);
continue;
}
@@ -2288,14 +2303,16 @@ process_exinode(
whichfork);
used = nex * sizeof(xfs_bmbt_rec_t);
if (nex > max_nex || used > XFS_DFORK_SIZE(dip, mp, whichfork)) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("bad number of extents %llu in inode %lld",
- (unsigned long long)nex, (long long)cur_ino);
+ (unsigned long long)nex,
+ (long long)metadump.cur_ino);
return 1;
}
/* Zero unused data fork past used extents */
- if (zero_stale_data && (used < XFS_DFORK_SIZE(dip, mp, whichfork)))
+ if (metadump.zero_stale_data &&
+ (used < XFS_DFORK_SIZE(dip, mp, whichfork)))
memset(XFS_DFORK_PTR(dip, whichfork) + used, 0,
XFS_DFORK_SIZE(dip, mp, whichfork) - used);
@@ -2311,7 +2328,7 @@ process_inode_data(
{
switch (dip->di_format) {
case XFS_DINODE_FMT_LOCAL:
- if (!(obfuscate || zero_stale_data))
+ if (!(metadump.obfuscate || metadump.zero_stale_data))
break;
/*
@@ -2323,7 +2340,7 @@ process_inode_data(
print_warning(
"Invalid data fork size (%d) in inode %llu, preserving contents!",
XFS_DFORK_DSIZE(dip, mp),
- (long long)cur_ino);
+ (long long)metadump.cur_ino);
break;
}
@@ -2355,9 +2372,9 @@ process_dev_inode(
struct xfs_dinode *dip)
{
if (xfs_dfork_data_extents(dip)) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("inode %llu has unexpected extents",
- (unsigned long long)cur_ino);
+ (unsigned long long)metadump.cur_ino);
return;
}
@@ -2369,11 +2386,11 @@ process_dev_inode(
if (XFS_DFORK_DSIZE(dip, mp) > XFS_LITINO(mp)) {
print_warning(
"Invalid data fork size (%d) in inode %llu, preserving contents!",
- XFS_DFORK_DSIZE(dip, mp), (long long)cur_ino);
+ XFS_DFORK_DSIZE(dip, mp), (long long)metadump.cur_ino);
return;
}
- if (zero_stale_data) {
+ if (metadump.zero_stale_data) {
unsigned int size = sizeof(xfs_dev_t);
memset(XFS_DFORK_DPTR(dip) + size, 0,
@@ -2399,17 +2416,17 @@ process_inode(
bool crc_was_ok = false; /* no recalc by default */
bool need_new_crc = false;
- cur_ino = XFS_AGINO_TO_INO(mp, agno, agino);
+ metadump.cur_ino = XFS_AGINO_TO_INO(mp, agno, agino);
/* we only care about crc recalculation if we will modify the inode. */
- if (obfuscate || zero_stale_data) {
+ if (metadump.obfuscate || metadump.zero_stale_data) {
crc_was_ok = libxfs_verify_cksum((char *)dip,
mp->m_sb.sb_inodesize,
offsetof(struct xfs_dinode, di_crc));
}
if (free_inode) {
- if (zero_stale_data) {
+ if (metadump.zero_stale_data) {
/* Zero all of the inode literal area */
memset(XFS_DFORK_DPTR(dip), 0, XFS_LITINO(mp));
}
@@ -2451,7 +2468,8 @@ process_inode(
switch (dip->di_aformat) {
case XFS_DINODE_FMT_LOCAL:
need_new_crc = true;
- if (obfuscate || zero_stale_data)
+ if (metadump.obfuscate ||
+ metadump.zero_stale_data)
process_sf_attr(dip);
break;
@@ -2468,7 +2486,7 @@ process_inode(
done:
/* Heavy handed but low cost; just do it as a catch-all. */
- if (zero_stale_data)
+ if (metadump.zero_stale_data)
need_new_crc = true;
if (crc_was_ok && need_new_crc)
@@ -2528,7 +2546,7 @@ copy_inode_chunk(
if (agino == 0 || agino == NULLAGINO || !valid_bno(agno, agbno) ||
!valid_bno(agno, XFS_AGINO_TO_AGBNO(mp,
agino + XFS_INODES_PER_CHUNK - 1))) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("bad inode number %llu (%u/%u)",
XFS_AGINO_TO_INO(mp, agno, agino), agno, agino);
return 1;
@@ -2544,7 +2562,7 @@ copy_inode_chunk(
(xfs_has_align(mp) &&
mp->m_sb.sb_inoalignmt != 0 &&
agbno % mp->m_sb.sb_inoalignmt != 0)) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("badly aligned inode (start = %llu)",
XFS_AGINO_TO_INO(mp, agno, agino));
return 1;
@@ -2561,7 +2579,7 @@ copy_inode_chunk(
if (iocur_top->data == NULL) {
print_warning("cannot read inode block %u/%u",
agno, agbno);
- rval = !stop_on_read_error;
+ rval = !metadump.stop_on_read_error;
goto pop_out;
}
@@ -2587,7 +2605,7 @@ next_bp:
ioff += inodes_per_buf;
}
- if (show_progress)
+ if (metadump.show_progress)
print_progress("Copied %u of %u inodes (%u of %u AGs)",
inodes_copied, mp->m_sb.sb_icount, agno,
mp->m_sb.sb_agcount);
@@ -2617,7 +2635,7 @@ scanfunc_ino(
if (level == 0) {
if (numrecs > igeo->inobt_mxr[0]) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid numrecs %d in %s "
"block %u/%u", numrecs,
typtab[btype].name, agno, agbno);
@@ -2640,7 +2658,7 @@ scanfunc_ino(
}
if (numrecs > igeo->inobt_mxr[1]) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid numrecs %d in %s block %u/%u",
numrecs, typtab[btype].name, agno, agbno);
numrecs = igeo->inobt_mxr[1];
@@ -2649,7 +2667,7 @@ scanfunc_ino(
pp = XFS_INOBT_PTR_ADDR(mp, block, 1, igeo->inobt_mxr[1]);
for (i = 0; i < numrecs; i++) {
if (!valid_bno(agno, be32_to_cpu(pp[i]))) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u/%u) "
"in %s block %u/%u",
agno, be32_to_cpu(pp[i]),
@@ -2677,13 +2695,13 @@ copy_inodes(
/* validate root and levels before processing the tree */
if (root == 0 || root > mp->m_sb.sb_agblocks) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u) in inobt "
"root in agi %u", root, agno);
return 1;
}
if (levels > M_IGEO(mp)->inobt_maxlevels) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid level (%u) in inobt root "
"in agi %u", levels, agno);
return 1;
@@ -2697,7 +2715,7 @@ copy_inodes(
levels = be32_to_cpu(agi->agi_free_level);
if (root == 0 || root > mp->m_sb.sb_agblocks) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid block number (%u) in "
"finobt root in agi %u", root,
agno);
@@ -2705,7 +2723,7 @@ copy_inodes(
}
if (levels > M_IGEO(mp)->inobt_maxlevels) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid level (%u) in finobt "
"root in agi %u", levels, agno);
return 1;
@@ -2736,11 +2754,11 @@ scan_ag(
XFS_FSS_TO_BB(mp, 1), DB_RING_IGN, NULL);
if (!iocur_top->data) {
print_warning("cannot read superblock for ag %u", agno);
- if (stop_on_read_error)
+ if (metadump.stop_on_read_error)
goto pop_out;
} else {
/* Replace any filesystem label with "L's" */
- if (obfuscate) {
+ if (metadump.obfuscate) {
struct xfs_sb *sb = iocur_top->data;
memset(sb->sb_fname, 'L',
min(strlen(sb->sb_fname), sizeof(sb->sb_fname)));
@@ -2758,7 +2776,7 @@ scan_ag(
agf = iocur_top->data;
if (iocur_top->data == NULL) {
print_warning("cannot read agf block for ag %u", agno);
- if (stop_on_read_error)
+ if (metadump.stop_on_read_error)
goto pop_out;
} else {
if (write_buf(iocur_top))
@@ -2773,7 +2791,7 @@ scan_ag(
agi = iocur_top->data;
if (iocur_top->data == NULL) {
print_warning("cannot read agi block for ag %u", agno);
- if (stop_on_read_error)
+ if (metadump.stop_on_read_error)
goto pop_out;
} else {
if (write_buf(iocur_top))
@@ -2787,10 +2805,10 @@ scan_ag(
XFS_FSS_TO_BB(mp, 1), DB_RING_IGN, NULL);
if (iocur_top->data == NULL) {
print_warning("cannot read agfl block for ag %u", agno);
- if (stop_on_read_error)
+ if (metadump.stop_on_read_error)
goto pop_out;
} else {
- if (agf && zero_stale_data) {
+ if (agf && metadump.zero_stale_data) {
/* Zero out unused bits of agfl */
int i;
__be32 *agfl_bno;
@@ -2813,7 +2831,7 @@ scan_ag(
/* copy AG free space btrees */
if (agf) {
- if (show_progress)
+ if (metadump.show_progress)
print_progress("Copying free space trees of AG %u",
agno);
if (!copy_free_bno_btree(agno, agf))
@@ -2859,7 +2877,7 @@ copy_ino(
if (agno >= mp->m_sb.sb_agcount || agbno >= mp->m_sb.sb_agblocks ||
offset >= mp->m_sb.sb_inopblock) {
- if (show_warnings)
+ if (metadump.show_warnings)
print_warning("invalid %s inode number (%lld)",
typtab[itype].name, (long long)ino);
return 1;
@@ -2871,12 +2889,12 @@ copy_ino(
if (iocur_top->data == NULL) {
print_warning("cannot read %s inode %lld",
typtab[itype].name, (long long)ino);
- rval = !stop_on_read_error;
+ rval = !metadump.stop_on_read_error;
goto pop_out;
}
off_cur(offset << mp->m_sb.sb_inodelog, mp->m_sb.sb_inodesize);
- cur_ino = ino;
+ metadump.cur_ino = ino;
rval = process_inode_data(iocur_top->data, itype);
pop_out:
pop_cur();
@@ -2912,7 +2930,7 @@ copy_log(void)
int logversion;
int cycle = XLOG_INIT_CYCLE;
- if (show_progress)
+ if (metadump.show_progress)
print_progress("Copying log");
push_cur();
@@ -2921,11 +2939,11 @@ copy_log(void)
if (iocur_top->data == NULL) {
pop_cur();
print_warning("cannot read log");
- return !stop_on_read_error;
+ return !metadump.stop_on_read_error;
}
/* If not obfuscating or zeroing, just copy the log as it is */
- if (!obfuscate && !zero_stale_data)
+ if (!metadump.obfuscate && !metadump.zero_stale_data)
goto done;
dirty = xlog_is_dirty(mp, &log, &x, 0);
@@ -2933,7 +2951,7 @@ copy_log(void)
switch (dirty) {
case 0:
/* clear out a clean log */
- if (show_progress)
+ if (metadump.show_progress)
print_progress("Zeroing clean log");
logstart = XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart);
@@ -2948,7 +2966,7 @@ copy_log(void)
break;
case 1:
/* keep the dirty log */
- if (obfuscate)
+ if (metadump.obfuscate)
print_warning(
_("Warning: log recovery of an obfuscated metadata image can leak "
"unobfuscated metadata and/or cause image corruption. If possible, "
@@ -2956,7 +2974,7 @@ _("Warning: log recovery of an obfuscated metadata image can leak "
break;
case -1:
/* log detection error */
- if (obfuscate)
+ if (metadump.obfuscate)
print_warning(
_("Could not discern log; image will contain unobfuscated metadata in log."));
break;
@@ -2979,9 +2997,15 @@ metadump_f(
char *p;
exitcode = 1;
- show_progress = 0;
- show_warnings = 0;
- stop_on_read_error = 0;
+
+ metadump.version = 1;
+ metadump.show_progress = 0;
+ metadump.stop_on_read_error = 0;
+ metadump.max_extent_size = DEFAULT_MAX_EXT_SIZE;
+ metadump.show_warnings = 0;
+ metadump.obfuscate = 1;
+ metadump.zero_stale_data = 1;
+ metadump.dirty_log = false;
if (mp->m_sb.sb_magicnum != XFS_SB_MAGIC) {
print_warning("bad superblock magic number %x, giving up",
@@ -3002,27 +3026,29 @@ metadump_f(
while ((c = getopt(argc, argv, "aegm:ow")) != EOF) {
switch (c) {
case 'a':
- zero_stale_data = 0;
+ metadump.zero_stale_data = 0;
break;
case 'e':
- stop_on_read_error = 1;
+ metadump.stop_on_read_error = 1;
break;
case 'g':
- show_progress = 1;
+ metadump.show_progress = 1;
break;
case 'm':
- max_extent_size = (int)strtol(optarg, &p, 0);
- if (*p != '\0' || max_extent_size <= 0) {
+ metadump.max_extent_size =
+ (int)strtol(optarg, &p, 0);
+ if (*p != '\0' ||
+ metadump.max_extent_size <= 0) {
print_warning("bad max extent size %s",
optarg);
return 0;
}
break;
case 'o':
- obfuscate = 0;
+ metadump.obfuscate = 0;
break;
case 'w':
- show_warnings = 1;
+ metadump.show_warnings = 1;
break;
default:
print_warning("bad option for metadump command");
@@ -3035,21 +3061,6 @@ metadump_f(
return 0;
}
- metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
- if (metablock == NULL) {
- print_warning("memory allocation failure");
- return 0;
- }
- metablock->mb_blocklog = BBSHIFT;
- metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC);
-
- /* Set flags about state of metadump */
- metablock->mb_info = XFS_METADUMP_INFO_FLAGS;
- if (obfuscate)
- metablock->mb_info |= XFS_METADUMP_OBFUSCATED;
- if (!zero_stale_data)
- metablock->mb_info |= XFS_METADUMP_FULLBLOCKS;
-
/* If we'll copy the log, see if the log is dirty */
if (mp->m_sb.sb_logstart) {
push_cur();
@@ -3060,34 +3071,52 @@ metadump_f(
struct xlog log;
if (xlog_is_dirty(mp, &log, &x, 0))
- metablock->mb_info |= XFS_METADUMP_DIRTYLOG;
+ metadump.dirty_log = true;
}
pop_cur();
}
- block_index = (__be64 *)((char *)metablock + sizeof(xfs_metablock_t));
- block_buffer = (char *)metablock + BBSIZE;
- num_indices = (BBSIZE - sizeof(xfs_metablock_t)) / sizeof(__be64);
+ metadump.metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
+ if (metadump.metablock == NULL) {
+ print_warning("memory allocation failure");
+ return -1;
+ }
+ metadump.metablock->mb_blocklog = BBSHIFT;
+ metadump.metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC);
+
+ /* Set flags about state of metadump */
+ metadump.metablock->mb_info = XFS_METADUMP_INFO_FLAGS;
+ if (metadump.obfuscate)
+ metadump.metablock->mb_info |= XFS_METADUMP_OBFUSCATED;
+ if (!metadump.zero_stale_data)
+ metadump.metablock->mb_info |= XFS_METADUMP_FULLBLOCKS;
+ if (metadump.dirty_log)
+ metadump.metablock->mb_info |= XFS_METADUMP_DIRTYLOG;
+
+ metadump.block_index = (__be64 *)((char *)metadump.metablock +
+ sizeof(xfs_metablock_t));
+ metadump.block_buffer = (char *)metadump.metablock + BBSIZE;
+ metadump.num_indices = (BBSIZE - sizeof(xfs_metablock_t)) /
+ sizeof(__be64);
/*
* A metadump block can hold at most num_indices of BBSIZE sectors;
* do not try to dump a filesystem with a sector size which does not
* fit within num_indices (i.e. within a single metablock).
*/
- if (mp->m_sb.sb_sectsize > num_indices * BBSIZE) {
+ if (mp->m_sb.sb_sectsize > metadump.num_indices * BBSIZE) {
print_warning("Cannot dump filesystem with sector size %u",
mp->m_sb.sb_sectsize);
- free(metablock);
+ free(metadump.metablock);
return 0;
}
- cur_index = 0;
start_iocur_sp = iocur_sp;
if (strcmp(argv[optind], "-") == 0) {
if (isatty(fileno(stdout))) {
print_warning("cannot write to a terminal");
- free(metablock);
+ free(metadump.metablock);
return 0;
}
/*
@@ -3111,17 +3140,17 @@ metadump_f(
close(outfd);
goto out;
}
- outf = fdopen(outfd, "a");
- if (outf == NULL) {
+ metadump.outf = fdopen(outfd, "a");
+ if (metadump.outf == NULL) {
fprintf(stderr, "cannot create dump stream\n");
dup2(outfd, STDOUT_FILENO);
close(outfd);
goto out;
}
- stdout_metadump = true;
+ metadump.stdout_metadump = true;
} else {
- outf = fopen(argv[optind], "wb");
- if (outf == NULL) {
+ metadump.outf = fopen(argv[optind], "wb");
+ if (metadump.outf == NULL) {
print_warning("cannot create dump file");
goto out;
}
@@ -3148,24 +3177,24 @@ metadump_f(
if (!exitcode)
exitcode = write_index() < 0;
- if (progress_since_warning)
- fputc('\n', stdout_metadump ? stderr : stdout);
+ if (metadump.progress_since_warning)
+ fputc('\n', metadump.stdout_metadump ? stderr : stdout);
- if (stdout_metadump) {
- fflush(outf);
+ if (metadump.stdout_metadump) {
+ fflush(metadump.outf);
fflush(stdout);
ret = dup2(outfd, STDOUT_FILENO);
if (ret < 0)
perror("un-redirecting stdout");
- stdout_metadump = false;
+ metadump.stdout_metadump = false;
}
- fclose(outf);
+ fclose(metadump.outf);
/* cleanup iocur stack */
while (iocur_sp > start_iocur_sp)
pop_cur();
out:
- free(metablock);
+ free(metadump.metablock);
return 0;
}
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 03/24] metadump: Define and use struct metadump
2023-05-23 9:00 ` [PATCH 03/24] metadump: Define and use struct metadump Chandan Babu R
@ 2023-05-23 16:35 ` Darrick J. Wong
2023-05-24 4:50 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 16:35 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:29PM +0530, Chandan Babu R wrote:
> This commit collects all state tracking variables in a new "struct metadump"
> structure.
I think this commit message needs to capture the reasons for /why/ all
these global variables are being pulled into a struct definition that
itself is used once to define a global variable.
(Changes look ok, at least)
--D
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> db/metadump.c | 459 +++++++++++++++++++++++++++-----------------------
> 1 file changed, 244 insertions(+), 215 deletions(-)
>
> diff --git a/db/metadump.c b/db/metadump.c
> index 6bcfd5bba..806cdfd68 100644
> --- a/db/metadump.c
> +++ b/db/metadump.c
> @@ -40,25 +40,27 @@ static const cmdinfo_t metadump_cmd =
> N_("[-a] [-e] [-g] [-m max_extent] [-w] [-o] filename"),
> N_("dump metadata to a file"), metadump_help };
>
> -static FILE *outf; /* metadump file */
> -
> -static xfs_metablock_t *metablock; /* header + index + buffers */
> -static __be64 *block_index;
> -static char *block_buffer;
> -
> -static int num_indices;
> -static int cur_index;
> -
> -static xfs_ino_t cur_ino;
> -
> -static int show_progress = 0;
> -static int stop_on_read_error = 0;
> -static int max_extent_size = DEFAULT_MAX_EXT_SIZE;
> -static int obfuscate = 1;
> -static int zero_stale_data = 1;
> -static int show_warnings = 0;
> -static int progress_since_warning = 0;
> -static bool stdout_metadump;
> +static struct metadump {
> + int version;
> + int show_progress;
> + int stop_on_read_error;
> + int max_extent_size;
> + int show_warnings;
> + int obfuscate;
> + int zero_stale_data;
> + int progress_since_warning;
> + bool dirty_log;
> + bool stdout_metadump;
> + xfs_ino_t cur_ino;
> + /* Metadump file */
> + FILE *outf;
> + /* header + index + buffers */
> + struct xfs_metablock *metablock;
> + __be64 *block_index;
> + char *block_buffer;
> + int num_indices;
> + int cur_index;
> +} metadump;
>
> void
> metadump_init(void)
> @@ -98,9 +100,9 @@ print_warning(const char *fmt, ...)
> va_end(ap);
> buf[sizeof(buf)-1] = '\0';
>
> - fprintf(stderr, "%s%s: %s\n", progress_since_warning ? "\n" : "",
> - progname, buf);
> - progress_since_warning = 0;
> + fprintf(stderr, "%s%s: %s\n",
> + metadump.progress_since_warning ? "\n" : "", progname, buf);
> + metadump.progress_since_warning = 0;
> }
>
> static void
> @@ -118,10 +120,10 @@ print_progress(const char *fmt, ...)
> va_end(ap);
> buf[sizeof(buf)-1] = '\0';
>
> - f = stdout_metadump ? stderr : stdout;
> + f = metadump.stdout_metadump ? stderr : stdout;
> fprintf(f, "\r%-59s", buf);
> fflush(f);
> - progress_since_warning = 1;
> + metadump.progress_since_warning = 1;
> }
>
> /*
> @@ -136,17 +138,19 @@ print_progress(const char *fmt, ...)
> static int
> write_index(void)
> {
> + struct xfs_metablock *metablock = metadump.metablock;
> /*
> * write index block and following data blocks (streaming)
> */
> - metablock->mb_count = cpu_to_be16(cur_index);
> - if (fwrite(metablock, (cur_index + 1) << BBSHIFT, 1, outf) != 1) {
> + metablock->mb_count = cpu_to_be16(metadump.cur_index);
> + if (fwrite(metablock, (metadump.cur_index + 1) << BBSHIFT, 1,
> + metadump.outf) != 1) {
> print_warning("error writing to target file");
> return -1;
> }
>
> - memset(block_index, 0, num_indices * sizeof(__be64));
> - cur_index = 0;
> + memset(metadump.block_index, 0, metadump.num_indices * sizeof(__be64));
> + metadump.cur_index = 0;
> return 0;
> }
>
> @@ -163,9 +167,10 @@ write_buf_segment(
> int ret;
>
> for (i = 0; i < len; i++, off++, data += BBSIZE) {
> - block_index[cur_index] = cpu_to_be64(off);
> - memcpy(&block_buffer[cur_index << BBSHIFT], data, BBSIZE);
> - if (++cur_index == num_indices) {
> + metadump.block_index[metadump.cur_index] = cpu_to_be64(off);
> + memcpy(&metadump.block_buffer[metadump.cur_index << BBSHIFT],
> + data, BBSIZE);
> + if (++metadump.cur_index == metadump.num_indices) {
> ret = write_index();
> if (ret)
> return -EIO;
> @@ -388,11 +393,11 @@ scan_btree(
> if (iocur_top->data == NULL) {
> print_warning("cannot read %s block %u/%u", typtab[btype].name,
> agno, agbno);
> - rval = !stop_on_read_error;
> + rval = !metadump.stop_on_read_error;
> goto pop_out;
> }
>
> - if (zero_stale_data) {
> + if (metadump.zero_stale_data) {
> zero_btree_block(iocur_top->data, btype);
> iocur_top->need_crc = 1;
> }
> @@ -446,7 +451,7 @@ scanfunc_freesp(
>
> numrecs = be16_to_cpu(block->bb_numrecs);
> if (numrecs > mp->m_alloc_mxr[1]) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid numrecs (%u) in %s block %u/%u",
> numrecs, typtab[btype].name, agno, agbno);
> return 1;
> @@ -455,7 +460,7 @@ scanfunc_freesp(
> pp = XFS_ALLOC_PTR_ADDR(mp, block, 1, mp->m_alloc_mxr[1]);
> for (i = 0; i < numrecs; i++) {
> if (!valid_bno(agno, be32_to_cpu(pp[i]))) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u/%u) "
> "in %s block %u/%u",
> agno, be32_to_cpu(pp[i]),
> @@ -482,13 +487,13 @@ copy_free_bno_btree(
>
> /* validate root and levels before processing the tree */
> if (root == 0 || root > mp->m_sb.sb_agblocks) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u) in bnobt "
> "root in agf %u", root, agno);
> return 1;
> }
> if (levels > mp->m_alloc_maxlevels) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid level (%u) in bnobt root "
> "in agf %u", levels, agno);
> return 1;
> @@ -510,13 +515,13 @@ copy_free_cnt_btree(
>
> /* validate root and levels before processing the tree */
> if (root == 0 || root > mp->m_sb.sb_agblocks) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u) in cntbt "
> "root in agf %u", root, agno);
> return 1;
> }
> if (levels > mp->m_alloc_maxlevels) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid level (%u) in cntbt root "
> "in agf %u", levels, agno);
> return 1;
> @@ -543,7 +548,7 @@ scanfunc_rmapbt(
>
> numrecs = be16_to_cpu(block->bb_numrecs);
> if (numrecs > mp->m_rmap_mxr[1]) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid numrecs (%u) in %s block %u/%u",
> numrecs, typtab[btype].name, agno, agbno);
> return 1;
> @@ -552,7 +557,7 @@ scanfunc_rmapbt(
> pp = XFS_RMAP_PTR_ADDR(block, 1, mp->m_rmap_mxr[1]);
> for (i = 0; i < numrecs; i++) {
> if (!valid_bno(agno, be32_to_cpu(pp[i]))) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u/%u) "
> "in %s block %u/%u",
> agno, be32_to_cpu(pp[i]),
> @@ -582,13 +587,13 @@ copy_rmap_btree(
>
> /* validate root and levels before processing the tree */
> if (root == 0 || root > mp->m_sb.sb_agblocks) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u) in rmapbt "
> "root in agf %u", root, agno);
> return 1;
> }
> if (levels > mp->m_rmap_maxlevels) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid level (%u) in rmapbt root "
> "in agf %u", levels, agno);
> return 1;
> @@ -615,7 +620,7 @@ scanfunc_refcntbt(
>
> numrecs = be16_to_cpu(block->bb_numrecs);
> if (numrecs > mp->m_refc_mxr[1]) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid numrecs (%u) in %s block %u/%u",
> numrecs, typtab[btype].name, agno, agbno);
> return 1;
> @@ -624,7 +629,7 @@ scanfunc_refcntbt(
> pp = XFS_REFCOUNT_PTR_ADDR(block, 1, mp->m_refc_mxr[1]);
> for (i = 0; i < numrecs; i++) {
> if (!valid_bno(agno, be32_to_cpu(pp[i]))) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u/%u) "
> "in %s block %u/%u",
> agno, be32_to_cpu(pp[i]),
> @@ -654,13 +659,13 @@ copy_refcount_btree(
>
> /* validate root and levels before processing the tree */
> if (root == 0 || root > mp->m_sb.sb_agblocks) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u) in refcntbt "
> "root in agf %u", root, agno);
> return 1;
> }
> if (levels > mp->m_refc_maxlevels) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid level (%u) in refcntbt root "
> "in agf %u", levels, agno);
> return 1;
> @@ -785,7 +790,8 @@ in_lost_found(
> /* Record the "lost+found" inode if we haven't done so already */
>
> ASSERT(ino != 0);
> - if (!orphanage_ino && is_orphanage_dir(mp, cur_ino, namelen, name))
> + if (!orphanage_ino && is_orphanage_dir(mp, metadump.cur_ino, namelen,
> + name))
> orphanage_ino = ino;
>
> /* We don't obfuscate the "lost+found" directory itself */
> @@ -795,7 +801,7 @@ in_lost_found(
>
> /* Most files aren't in "lost+found" at all */
>
> - if (cur_ino != orphanage_ino)
> + if (metadump.cur_ino != orphanage_ino)
> return 0;
>
> /*
> @@ -1219,7 +1225,7 @@ generate_obfuscated_name(
> print_warning("duplicate name for inode %llu "
> "in dir inode %llu\n",
> (unsigned long long) ino,
> - (unsigned long long) cur_ino);
> + (unsigned long long) metadump.cur_ino);
> return;
> }
>
> @@ -1229,7 +1235,7 @@ generate_obfuscated_name(
> print_warning("unable to record name for inode %llu "
> "in dir inode %llu\n",
> (unsigned long long) ino,
> - (unsigned long long) cur_ino);
> + (unsigned long long) metadump.cur_ino);
> }
>
> static void
> @@ -1245,9 +1251,9 @@ process_sf_dir(
> ino_dir_size = be64_to_cpu(dip->di_size);
> if (ino_dir_size > XFS_DFORK_DSIZE(dip, mp)) {
> ino_dir_size = XFS_DFORK_DSIZE(dip, mp);
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid size in dir inode %llu",
> - (long long)cur_ino);
> + (long long)metadump.cur_ino);
> }
>
> sfep = xfs_dir2_sf_firstentry(sfp);
> @@ -1261,9 +1267,9 @@ process_sf_dir(
> int namelen = sfep->namelen;
>
> if (namelen == 0) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("zero length entry in dir inode "
> - "%llu", (long long)cur_ino);
> + "%llu", (long long)metadump.cur_ino);
> if (i != sfp->count - 1)
> break;
> namelen = ino_dir_size - ((char *)&sfep->name[0] -
> @@ -1271,16 +1277,17 @@ process_sf_dir(
> } else if ((char *)sfep - (char *)sfp +
> libxfs_dir2_sf_entsize(mp, sfp, sfep->namelen) >
> ino_dir_size) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("entry length in dir inode %llu "
> - "overflows space", (long long)cur_ino);
> + "overflows space",
> + (long long)metadump.cur_ino);
> if (i != sfp->count - 1)
> break;
> namelen = ino_dir_size - ((char *)&sfep->name[0] -
> (char *)sfp);
> }
>
> - if (obfuscate)
> + if (metadump.obfuscate)
> generate_obfuscated_name(
> libxfs_dir2_sf_get_ino(mp, sfp, sfep),
> namelen, &sfep->name[0]);
> @@ -1290,7 +1297,8 @@ process_sf_dir(
> }
>
> /* zero stale data in rest of space in data fork, if any */
> - if (zero_stale_data && (ino_dir_size < XFS_DFORK_DSIZE(dip, mp)))
> + if (metadump.zero_stale_data &&
> + (ino_dir_size < XFS_DFORK_DSIZE(dip, mp)))
> memset(sfep, 0, XFS_DFORK_DSIZE(dip, mp) - ino_dir_size);
> }
>
> @@ -1346,18 +1354,18 @@ process_sf_symlink(
>
> len = be64_to_cpu(dip->di_size);
> if (len > XFS_DFORK_DSIZE(dip, mp)) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid size (%d) in symlink inode %llu",
> - len, (long long)cur_ino);
> + len, (long long)metadump.cur_ino);
> len = XFS_DFORK_DSIZE(dip, mp);
> }
>
> buf = (char *)XFS_DFORK_DPTR(dip);
> - if (obfuscate)
> + if (metadump.obfuscate)
> obfuscate_path_components(buf, len);
>
> /* zero stale data in rest of space in data fork, if any */
> - if (zero_stale_data && len < XFS_DFORK_DSIZE(dip, mp))
> + if (metadump.zero_stale_data && len < XFS_DFORK_DSIZE(dip, mp))
> memset(&buf[len], 0, XFS_DFORK_DSIZE(dip, mp) - len);
> }
>
> @@ -1382,9 +1390,9 @@ process_sf_attr(
> ino_attr_size = be16_to_cpu(asfp->hdr.totsize);
> if (ino_attr_size > XFS_DFORK_ASIZE(dip, mp)) {
> ino_attr_size = XFS_DFORK_ASIZE(dip, mp);
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid attr size in inode %llu",
> - (long long)cur_ino);
> + (long long)metadump.cur_ino);
> }
>
> asfep = &asfp->list[0];
> @@ -1394,19 +1402,20 @@ process_sf_attr(
> int namelen = asfep->namelen;
>
> if (namelen == 0) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("zero length attr entry in inode "
> - "%llu", (long long)cur_ino);
> + "%llu", (long long)metadump.cur_ino);
> break;
> } else if ((char *)asfep - (char *)asfp +
> xfs_attr_sf_entsize(asfep) > ino_attr_size) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("attr entry length in inode %llu "
> - "overflows space", (long long)cur_ino);
> + "overflows space",
> + (long long)metadump.cur_ino);
> break;
> }
>
> - if (obfuscate) {
> + if (metadump.obfuscate) {
> generate_obfuscated_name(0, asfep->namelen,
> &asfep->nameval[0]);
> memset(&asfep->nameval[asfep->namelen], 'v',
> @@ -1418,7 +1427,8 @@ process_sf_attr(
> }
>
> /* zero stale data in rest of space in attr fork, if any */
> - if (zero_stale_data && (ino_attr_size < XFS_DFORK_ASIZE(dip, mp)))
> + if (metadump.zero_stale_data &&
> + (ino_attr_size < XFS_DFORK_ASIZE(dip, mp)))
> memset(asfep, 0, XFS_DFORK_ASIZE(dip, mp) - ino_attr_size);
> }
>
> @@ -1429,7 +1439,7 @@ process_dir_free_block(
> struct xfs_dir2_free *free;
> struct xfs_dir3_icfree_hdr freehdr;
>
> - if (!zero_stale_data)
> + if (!metadump.zero_stale_data)
> return;
>
> free = (struct xfs_dir2_free *)block;
> @@ -1451,10 +1461,10 @@ process_dir_free_block(
> break;
> }
> default:
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid magic in dir inode %llu "
> "free block",
> - (unsigned long long)cur_ino);
> + (unsigned long long)metadump.cur_ino);
> break;
> }
> }
> @@ -1466,7 +1476,7 @@ process_dir_leaf_block(
> struct xfs_dir2_leaf *leaf;
> struct xfs_dir3_icleaf_hdr leafhdr;
>
> - if (!zero_stale_data)
> + if (!metadump.zero_stale_data)
> return;
>
> /* Yes, this works for dir2 & dir3. Difference is padding. */
> @@ -1549,10 +1559,10 @@ process_dir_data_block(
> }
>
> if (be32_to_cpu(datahdr->magic) != wantmagic) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning(
> "invalid magic in dir inode %llu block %ld",
> - (unsigned long long)cur_ino, (long)offset);
> + (unsigned long long)metadump.cur_ino, (long)offset);
> return;
> }
>
> @@ -1572,10 +1582,10 @@ process_dir_data_block(
> if (dir_offset + free_length > end_of_data ||
> !free_length ||
> (free_length & (XFS_DIR2_DATA_ALIGN - 1))) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning(
> "invalid length for dir free space in inode %llu",
> - (long long)cur_ino);
> + (long long)metadump.cur_ino);
> return;
> }
> if (be16_to_cpu(*xfs_dir2_data_unused_tag_p(dup)) !=
> @@ -1588,7 +1598,7 @@ process_dir_data_block(
> * actually at a variable offset, so zeroing &dup->tag
> * is zeroing the free space in between
> */
> - if (zero_stale_data) {
> + if (metadump.zero_stale_data) {
> int zlen = free_length -
> sizeof(xfs_dir2_data_unused_t);
>
> @@ -1606,23 +1616,23 @@ process_dir_data_block(
>
> if (dir_offset + length > end_of_data ||
> ptr + length > endptr) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning(
> "invalid length for dir entry name in inode %llu",
> - (long long)cur_ino);
> + (long long)metadump.cur_ino);
> return;
> }
> if (be16_to_cpu(*libxfs_dir2_data_entry_tag_p(mp, dep)) !=
> dir_offset)
> return;
>
> - if (obfuscate)
> + if (metadump.obfuscate)
> generate_obfuscated_name(be64_to_cpu(dep->inumber),
> dep->namelen, &dep->name[0]);
> dir_offset += length;
> ptr += length;
> /* Zero the unused space after name, up to the tag */
> - if (zero_stale_data) {
> + if (metadump.zero_stale_data) {
> /* 1 byte for ftype; don't bother with conditional */
> int zlen =
> (char *)libxfs_dir2_data_entry_tag_p(mp, dep) -
> @@ -1658,7 +1668,7 @@ process_symlink_block(
>
> print_warning("cannot read %s block %u/%u (%llu)",
> typtab[btype].name, agno, agbno, s);
> - rval = !stop_on_read_error;
> + rval = !metadump.stop_on_read_error;
> goto out_pop;
> }
> link = iocur_top->data;
> @@ -1666,10 +1676,10 @@ process_symlink_block(
> if (xfs_has_crc((mp)))
> link += sizeof(struct xfs_dsymlink_hdr);
>
> - if (obfuscate)
> + if (metadump.obfuscate)
> obfuscate_path_components(link, XFS_SYMLINK_BUF_SPACE(mp,
> mp->m_sb.sb_blocksize));
> - if (zero_stale_data) {
> + if (metadump.zero_stale_data) {
> size_t linklen, zlen;
>
> linklen = strlen(link);
> @@ -1736,7 +1746,8 @@ process_attr_block(
> if ((be16_to_cpu(leaf->hdr.info.magic) != XFS_ATTR_LEAF_MAGIC) &&
> (be16_to_cpu(leaf->hdr.info.magic) != XFS_ATTR3_LEAF_MAGIC)) {
> for (i = 0; i < attr_data.remote_val_count; i++) {
> - if (obfuscate && attr_data.remote_vals[i] == offset)
> + if (metadump.obfuscate &&
> + attr_data.remote_vals[i] == offset)
> /* Macros to handle both attr and attr3 */
> memset(block +
> (bs - XFS_ATTR3_RMT_BUF_SPACE(mp, bs)),
> @@ -1753,9 +1764,9 @@ process_attr_block(
> nentries * sizeof(xfs_attr_leaf_entry_t) +
> xfs_attr3_leaf_hdr_size(leaf) >
> XFS_ATTR3_RMT_BUF_SPACE(mp, bs)) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid attr count in inode %llu",
> - (long long)cur_ino);
> + (long long)metadump.cur_ino);
> return;
> }
>
> @@ -1770,22 +1781,22 @@ process_attr_block(
> first_name = xfs_attr3_leaf_name(leaf, i);
>
> if (be16_to_cpu(entry->nameidx) > mp->m_sb.sb_blocksize) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning(
> "invalid attr nameidx in inode %llu",
> - (long long)cur_ino);
> + (long long)metadump.cur_ino);
> break;
> }
> if (entry->flags & XFS_ATTR_LOCAL) {
> local = xfs_attr3_leaf_name_local(leaf, i);
> if (local->namelen == 0) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning(
> "zero length for attr name in inode %llu",
> - (long long)cur_ino);
> + (long long)metadump.cur_ino);
> break;
> }
> - if (obfuscate) {
> + if (metadump.obfuscate) {
> generate_obfuscated_name(0, local->namelen,
> &local->nameval[0]);
> memset(&local->nameval[local->namelen], 'v',
> @@ -1797,18 +1808,18 @@ process_attr_block(
> zlen = xfs_attr_leaf_entsize_local(nlen, vlen) -
> (sizeof(xfs_attr_leaf_name_local_t) - 1 +
> nlen + vlen);
> - if (zero_stale_data)
> + if (metadump.zero_stale_data)
> memset(&local->nameval[nlen + vlen], 0, zlen);
> } else {
> remote = xfs_attr3_leaf_name_remote(leaf, i);
> if (remote->namelen == 0 || remote->valueblk == 0) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning(
> "invalid attr entry in inode %llu",
> - (long long)cur_ino);
> + (long long)metadump.cur_ino);
> break;
> }
> - if (obfuscate) {
> + if (metadump.obfuscate) {
> generate_obfuscated_name(0, remote->namelen,
> &remote->name[0]);
> add_remote_vals(be32_to_cpu(remote->valueblk),
> @@ -1819,13 +1830,13 @@ process_attr_block(
> zlen = xfs_attr_leaf_entsize_remote(nlen) -
> (sizeof(xfs_attr_leaf_name_remote_t) - 1 +
> nlen);
> - if (zero_stale_data)
> + if (metadump.zero_stale_data)
> memset(&remote->name[nlen], 0, zlen);
> }
> }
>
> /* Zero from end of entries array to the first name/val */
> - if (zero_stale_data) {
> + if (metadump.zero_stale_data) {
> struct xfs_attr_leaf_entry *entries;
>
> entries = xfs_attr3_leaf_entryp(leaf);
> @@ -1858,16 +1869,16 @@ process_single_fsb_objects(
>
> print_warning("cannot read %s block %u/%u (%llu)",
> typtab[btype].name, agno, agbno, s);
> - rval = !stop_on_read_error;
> + rval = !metadump.stop_on_read_error;
> goto out_pop;
>
> }
>
> - if (!obfuscate && !zero_stale_data)
> + if (!metadump.obfuscate && !metadump.zero_stale_data)
> goto write;
>
> /* Zero unused part of interior nodes */
> - if (zero_stale_data) {
> + if (metadump.zero_stale_data) {
> xfs_da_intnode_t *node = iocur_top->data;
> int magic = be16_to_cpu(node->hdr.info.magic);
>
> @@ -1978,12 +1989,12 @@ process_multi_fsb_dir(
>
> print_warning("cannot read %s block %u/%u (%llu)",
> typtab[btype].name, agno, agbno, s);
> - rval = !stop_on_read_error;
> + rval = !metadump.stop_on_read_error;
> goto out_pop;
>
> }
>
> - if (!obfuscate && !zero_stale_data)
> + if (!metadump.obfuscate && !metadump.zero_stale_data)
> goto write;
>
> dp = iocur_top->data;
> @@ -2075,25 +2086,27 @@ process_bmbt_reclist(
> * one is found, stop processing remaining extents
> */
> if (i > 0 && op + cp > o) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("bmap extent %d in %s ino %llu "
> "starts at %llu, previous extent "
> "ended at %llu", i,
> - typtab[btype].name, (long long)cur_ino,
> + typtab[btype].name,
> + (long long)metadump.cur_ino,
> o, op + cp - 1);
> break;
> }
>
> - if (c > max_extent_size) {
> + if (c > metadump.max_extent_size) {
> /*
> * since we are only processing non-data extents,
> * large numbers of blocks in a metadata extent is
> * extremely rare and more than likely to be corrupt.
> */
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("suspicious count %u in bmap "
> "extent %d in %s ino %llu", c, i,
> - typtab[btype].name, (long long)cur_ino);
> + typtab[btype].name,
> + (long long)metadump.cur_ino);
> break;
> }
>
> @@ -2104,19 +2117,21 @@ process_bmbt_reclist(
> agbno = XFS_FSB_TO_AGBNO(mp, s);
>
> if (!valid_bno(agno, agbno)) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number %u/%u "
> "(%llu) in bmap extent %d in %s ino "
> "%llu", agno, agbno, s, i,
> - typtab[btype].name, (long long)cur_ino);
> + typtab[btype].name,
> + (long long)metadump.cur_ino);
> break;
> }
>
> if (!valid_bno(agno, agbno + c - 1)) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("bmap extent %i in %s inode %llu "
> "overflows AG (end is %u/%u)", i,
> - typtab[btype].name, (long long)cur_ino,
> + typtab[btype].name,
> + (long long)metadump.cur_ino,
> agno, agbno + c - 1);
> break;
> }
> @@ -2152,7 +2167,7 @@ scanfunc_bmap(
>
> if (level == 0) {
> if (nrecs > mp->m_bmap_dmxr[0]) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid numrecs (%u) in %s "
> "block %u/%u", nrecs,
> typtab[btype].name, agno, agbno);
> @@ -2163,7 +2178,7 @@ scanfunc_bmap(
> }
>
> if (nrecs > mp->m_bmap_dmxr[1]) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid numrecs (%u) in %s block %u/%u",
> nrecs, typtab[btype].name, agno, agbno);
> return 1;
> @@ -2178,7 +2193,7 @@ scanfunc_bmap(
>
> if (bno == 0 || bno > mp->m_sb.sb_agblocks ||
> ag > mp->m_sb.sb_agcount) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u/%u) "
> "in %s block %u/%u", ag, bno,
> typtab[btype].name, agno, agbno);
> @@ -2213,10 +2228,10 @@ process_btinode(
> nrecs = be16_to_cpu(dib->bb_numrecs);
>
> if (level > XFS_BM_MAXLEVELS(mp, whichfork)) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid level (%u) in inode %lld %s "
> - "root", level, (long long)cur_ino,
> - typtab[btype].name);
> + "root", level, (long long)metadump.cur_ino,
> + typtab[btype].name);
> return 1;
> }
>
> @@ -2227,16 +2242,16 @@ process_btinode(
>
> maxrecs = libxfs_bmdr_maxrecs(XFS_DFORK_SIZE(dip, mp, whichfork), 0);
> if (nrecs > maxrecs) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid numrecs (%u) in inode %lld %s "
> - "root", nrecs, (long long)cur_ino,
> - typtab[btype].name);
> + "root", nrecs, (long long)metadump.cur_ino,
> + typtab[btype].name);
> return 1;
> }
>
> pp = XFS_BMDR_PTR_ADDR(dib, 1, maxrecs);
>
> - if (zero_stale_data) {
> + if (metadump.zero_stale_data) {
> char *top;
>
> /* Unused btree key space */
> @@ -2257,11 +2272,11 @@ process_btinode(
>
> if (bno == 0 || bno > mp->m_sb.sb_agblocks ||
> ag > mp->m_sb.sb_agcount) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u/%u) "
> - "in inode %llu %s root", ag,
> - bno, (long long)cur_ino,
> - typtab[btype].name);
> + "in inode %llu %s root", ag, bno,
> + (long long)metadump.cur_ino,
> + typtab[btype].name);
> continue;
> }
>
> @@ -2288,14 +2303,16 @@ process_exinode(
> whichfork);
> used = nex * sizeof(xfs_bmbt_rec_t);
> if (nex > max_nex || used > XFS_DFORK_SIZE(dip, mp, whichfork)) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("bad number of extents %llu in inode %lld",
> - (unsigned long long)nex, (long long)cur_ino);
> + (unsigned long long)nex,
> + (long long)metadump.cur_ino);
> return 1;
> }
>
> /* Zero unused data fork past used extents */
> - if (zero_stale_data && (used < XFS_DFORK_SIZE(dip, mp, whichfork)))
> + if (metadump.zero_stale_data &&
> + (used < XFS_DFORK_SIZE(dip, mp, whichfork)))
> memset(XFS_DFORK_PTR(dip, whichfork) + used, 0,
> XFS_DFORK_SIZE(dip, mp, whichfork) - used);
>
> @@ -2311,7 +2328,7 @@ process_inode_data(
> {
> switch (dip->di_format) {
> case XFS_DINODE_FMT_LOCAL:
> - if (!(obfuscate || zero_stale_data))
> + if (!(metadump.obfuscate || metadump.zero_stale_data))
> break;
>
> /*
> @@ -2323,7 +2340,7 @@ process_inode_data(
> print_warning(
> "Invalid data fork size (%d) in inode %llu, preserving contents!",
> XFS_DFORK_DSIZE(dip, mp),
> - (long long)cur_ino);
> + (long long)metadump.cur_ino);
> break;
> }
>
> @@ -2355,9 +2372,9 @@ process_dev_inode(
> struct xfs_dinode *dip)
> {
> if (xfs_dfork_data_extents(dip)) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("inode %llu has unexpected extents",
> - (unsigned long long)cur_ino);
> + (unsigned long long)metadump.cur_ino);
> return;
> }
>
> @@ -2369,11 +2386,11 @@ process_dev_inode(
> if (XFS_DFORK_DSIZE(dip, mp) > XFS_LITINO(mp)) {
> print_warning(
> "Invalid data fork size (%d) in inode %llu, preserving contents!",
> - XFS_DFORK_DSIZE(dip, mp), (long long)cur_ino);
> + XFS_DFORK_DSIZE(dip, mp), (long long)metadump.cur_ino);
> return;
> }
>
> - if (zero_stale_data) {
> + if (metadump.zero_stale_data) {
> unsigned int size = sizeof(xfs_dev_t);
>
> memset(XFS_DFORK_DPTR(dip) + size, 0,
> @@ -2399,17 +2416,17 @@ process_inode(
> bool crc_was_ok = false; /* no recalc by default */
> bool need_new_crc = false;
>
> - cur_ino = XFS_AGINO_TO_INO(mp, agno, agino);
> + metadump.cur_ino = XFS_AGINO_TO_INO(mp, agno, agino);
>
> /* we only care about crc recalculation if we will modify the inode. */
> - if (obfuscate || zero_stale_data) {
> + if (metadump.obfuscate || metadump.zero_stale_data) {
> crc_was_ok = libxfs_verify_cksum((char *)dip,
> mp->m_sb.sb_inodesize,
> offsetof(struct xfs_dinode, di_crc));
> }
>
> if (free_inode) {
> - if (zero_stale_data) {
> + if (metadump.zero_stale_data) {
> /* Zero all of the inode literal area */
> memset(XFS_DFORK_DPTR(dip), 0, XFS_LITINO(mp));
> }
> @@ -2451,7 +2468,8 @@ process_inode(
> switch (dip->di_aformat) {
> case XFS_DINODE_FMT_LOCAL:
> need_new_crc = true;
> - if (obfuscate || zero_stale_data)
> + if (metadump.obfuscate ||
> + metadump.zero_stale_data)
> process_sf_attr(dip);
> break;
>
> @@ -2468,7 +2486,7 @@ process_inode(
>
> done:
> /* Heavy handed but low cost; just do it as a catch-all. */
> - if (zero_stale_data)
> + if (metadump.zero_stale_data)
> need_new_crc = true;
>
> if (crc_was_ok && need_new_crc)
> @@ -2528,7 +2546,7 @@ copy_inode_chunk(
> if (agino == 0 || agino == NULLAGINO || !valid_bno(agno, agbno) ||
> !valid_bno(agno, XFS_AGINO_TO_AGBNO(mp,
> agino + XFS_INODES_PER_CHUNK - 1))) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("bad inode number %llu (%u/%u)",
> XFS_AGINO_TO_INO(mp, agno, agino), agno, agino);
> return 1;
> @@ -2544,7 +2562,7 @@ copy_inode_chunk(
> (xfs_has_align(mp) &&
> mp->m_sb.sb_inoalignmt != 0 &&
> agbno % mp->m_sb.sb_inoalignmt != 0)) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("badly aligned inode (start = %llu)",
> XFS_AGINO_TO_INO(mp, agno, agino));
> return 1;
> @@ -2561,7 +2579,7 @@ copy_inode_chunk(
> if (iocur_top->data == NULL) {
> print_warning("cannot read inode block %u/%u",
> agno, agbno);
> - rval = !stop_on_read_error;
> + rval = !metadump.stop_on_read_error;
> goto pop_out;
> }
>
> @@ -2587,7 +2605,7 @@ next_bp:
> ioff += inodes_per_buf;
> }
>
> - if (show_progress)
> + if (metadump.show_progress)
> print_progress("Copied %u of %u inodes (%u of %u AGs)",
> inodes_copied, mp->m_sb.sb_icount, agno,
> mp->m_sb.sb_agcount);
> @@ -2617,7 +2635,7 @@ scanfunc_ino(
>
> if (level == 0) {
> if (numrecs > igeo->inobt_mxr[0]) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid numrecs %d in %s "
> "block %u/%u", numrecs,
> typtab[btype].name, agno, agbno);
> @@ -2640,7 +2658,7 @@ scanfunc_ino(
> }
>
> if (numrecs > igeo->inobt_mxr[1]) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid numrecs %d in %s block %u/%u",
> numrecs, typtab[btype].name, agno, agbno);
> numrecs = igeo->inobt_mxr[1];
> @@ -2649,7 +2667,7 @@ scanfunc_ino(
> pp = XFS_INOBT_PTR_ADDR(mp, block, 1, igeo->inobt_mxr[1]);
> for (i = 0; i < numrecs; i++) {
> if (!valid_bno(agno, be32_to_cpu(pp[i]))) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u/%u) "
> "in %s block %u/%u",
> agno, be32_to_cpu(pp[i]),
> @@ -2677,13 +2695,13 @@ copy_inodes(
>
> /* validate root and levels before processing the tree */
> if (root == 0 || root > mp->m_sb.sb_agblocks) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u) in inobt "
> "root in agi %u", root, agno);
> return 1;
> }
> if (levels > M_IGEO(mp)->inobt_maxlevels) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid level (%u) in inobt root "
> "in agi %u", levels, agno);
> return 1;
> @@ -2697,7 +2715,7 @@ copy_inodes(
> levels = be32_to_cpu(agi->agi_free_level);
>
> if (root == 0 || root > mp->m_sb.sb_agblocks) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid block number (%u) in "
> "finobt root in agi %u", root,
> agno);
> @@ -2705,7 +2723,7 @@ copy_inodes(
> }
>
> if (levels > M_IGEO(mp)->inobt_maxlevels) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid level (%u) in finobt "
> "root in agi %u", levels, agno);
> return 1;
> @@ -2736,11 +2754,11 @@ scan_ag(
> XFS_FSS_TO_BB(mp, 1), DB_RING_IGN, NULL);
> if (!iocur_top->data) {
> print_warning("cannot read superblock for ag %u", agno);
> - if (stop_on_read_error)
> + if (metadump.stop_on_read_error)
> goto pop_out;
> } else {
> /* Replace any filesystem label with "L's" */
> - if (obfuscate) {
> + if (metadump.obfuscate) {
> struct xfs_sb *sb = iocur_top->data;
> memset(sb->sb_fname, 'L',
> min(strlen(sb->sb_fname), sizeof(sb->sb_fname)));
> @@ -2758,7 +2776,7 @@ scan_ag(
> agf = iocur_top->data;
> if (iocur_top->data == NULL) {
> print_warning("cannot read agf block for ag %u", agno);
> - if (stop_on_read_error)
> + if (metadump.stop_on_read_error)
> goto pop_out;
> } else {
> if (write_buf(iocur_top))
> @@ -2773,7 +2791,7 @@ scan_ag(
> agi = iocur_top->data;
> if (iocur_top->data == NULL) {
> print_warning("cannot read agi block for ag %u", agno);
> - if (stop_on_read_error)
> + if (metadump.stop_on_read_error)
> goto pop_out;
> } else {
> if (write_buf(iocur_top))
> @@ -2787,10 +2805,10 @@ scan_ag(
> XFS_FSS_TO_BB(mp, 1), DB_RING_IGN, NULL);
> if (iocur_top->data == NULL) {
> print_warning("cannot read agfl block for ag %u", agno);
> - if (stop_on_read_error)
> + if (metadump.stop_on_read_error)
> goto pop_out;
> } else {
> - if (agf && zero_stale_data) {
> + if (agf && metadump.zero_stale_data) {
> /* Zero out unused bits of agfl */
> int i;
> __be32 *agfl_bno;
> @@ -2813,7 +2831,7 @@ scan_ag(
>
> /* copy AG free space btrees */
> if (agf) {
> - if (show_progress)
> + if (metadump.show_progress)
> print_progress("Copying free space trees of AG %u",
> agno);
> if (!copy_free_bno_btree(agno, agf))
> @@ -2859,7 +2877,7 @@ copy_ino(
>
> if (agno >= mp->m_sb.sb_agcount || agbno >= mp->m_sb.sb_agblocks ||
> offset >= mp->m_sb.sb_inopblock) {
> - if (show_warnings)
> + if (metadump.show_warnings)
> print_warning("invalid %s inode number (%lld)",
> typtab[itype].name, (long long)ino);
> return 1;
> @@ -2871,12 +2889,12 @@ copy_ino(
> if (iocur_top->data == NULL) {
> print_warning("cannot read %s inode %lld",
> typtab[itype].name, (long long)ino);
> - rval = !stop_on_read_error;
> + rval = !metadump.stop_on_read_error;
> goto pop_out;
> }
> off_cur(offset << mp->m_sb.sb_inodelog, mp->m_sb.sb_inodesize);
>
> - cur_ino = ino;
> + metadump.cur_ino = ino;
> rval = process_inode_data(iocur_top->data, itype);
> pop_out:
> pop_cur();
> @@ -2912,7 +2930,7 @@ copy_log(void)
> int logversion;
> int cycle = XLOG_INIT_CYCLE;
>
> - if (show_progress)
> + if (metadump.show_progress)
> print_progress("Copying log");
>
> push_cur();
> @@ -2921,11 +2939,11 @@ copy_log(void)
> if (iocur_top->data == NULL) {
> pop_cur();
> print_warning("cannot read log");
> - return !stop_on_read_error;
> + return !metadump.stop_on_read_error;
> }
>
> /* If not obfuscating or zeroing, just copy the log as it is */
> - if (!obfuscate && !zero_stale_data)
> + if (!metadump.obfuscate && !metadump.zero_stale_data)
> goto done;
>
> dirty = xlog_is_dirty(mp, &log, &x, 0);
> @@ -2933,7 +2951,7 @@ copy_log(void)
> switch (dirty) {
> case 0:
> /* clear out a clean log */
> - if (show_progress)
> + if (metadump.show_progress)
> print_progress("Zeroing clean log");
>
> logstart = XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart);
> @@ -2948,7 +2966,7 @@ copy_log(void)
> break;
> case 1:
> /* keep the dirty log */
> - if (obfuscate)
> + if (metadump.obfuscate)
> print_warning(
> _("Warning: log recovery of an obfuscated metadata image can leak "
> "unobfuscated metadata and/or cause image corruption. If possible, "
> @@ -2956,7 +2974,7 @@ _("Warning: log recovery of an obfuscated metadata image can leak "
> break;
> case -1:
> /* log detection error */
> - if (obfuscate)
> + if (metadump.obfuscate)
> print_warning(
> _("Could not discern log; image will contain unobfuscated metadata in log."));
> break;
> @@ -2979,9 +2997,15 @@ metadump_f(
> char *p;
>
> exitcode = 1;
> - show_progress = 0;
> - show_warnings = 0;
> - stop_on_read_error = 0;
> +
> + metadump.version = 1;
> + metadump.show_progress = 0;
> + metadump.stop_on_read_error = 0;
> + metadump.max_extent_size = DEFAULT_MAX_EXT_SIZE;
> + metadump.show_warnings = 0;
> + metadump.obfuscate = 1;
> + metadump.zero_stale_data = 1;
> + metadump.dirty_log = false;
>
> if (mp->m_sb.sb_magicnum != XFS_SB_MAGIC) {
> print_warning("bad superblock magic number %x, giving up",
> @@ -3002,27 +3026,29 @@ metadump_f(
> while ((c = getopt(argc, argv, "aegm:ow")) != EOF) {
> switch (c) {
> case 'a':
> - zero_stale_data = 0;
> + metadump.zero_stale_data = 0;
> break;
> case 'e':
> - stop_on_read_error = 1;
> + metadump.stop_on_read_error = 1;
> break;
> case 'g':
> - show_progress = 1;
> + metadump.show_progress = 1;
> break;
> case 'm':
> - max_extent_size = (int)strtol(optarg, &p, 0);
> - if (*p != '\0' || max_extent_size <= 0) {
> + metadump.max_extent_size =
> + (int)strtol(optarg, &p, 0);
> + if (*p != '\0' ||
> + metadump.max_extent_size <= 0) {
> print_warning("bad max extent size %s",
> optarg);
> return 0;
> }
> break;
> case 'o':
> - obfuscate = 0;
> + metadump.obfuscate = 0;
> break;
> case 'w':
> - show_warnings = 1;
> + metadump.show_warnings = 1;
> break;
> default:
> print_warning("bad option for metadump command");
> @@ -3035,21 +3061,6 @@ metadump_f(
> return 0;
> }
>
> - metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
> - if (metablock == NULL) {
> - print_warning("memory allocation failure");
> - return 0;
> - }
> - metablock->mb_blocklog = BBSHIFT;
> - metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC);
> -
> - /* Set flags about state of metadump */
> - metablock->mb_info = XFS_METADUMP_INFO_FLAGS;
> - if (obfuscate)
> - metablock->mb_info |= XFS_METADUMP_OBFUSCATED;
> - if (!zero_stale_data)
> - metablock->mb_info |= XFS_METADUMP_FULLBLOCKS;
> -
> /* If we'll copy the log, see if the log is dirty */
> if (mp->m_sb.sb_logstart) {
> push_cur();
> @@ -3060,34 +3071,52 @@ metadump_f(
> struct xlog log;
>
> if (xlog_is_dirty(mp, &log, &x, 0))
> - metablock->mb_info |= XFS_METADUMP_DIRTYLOG;
> + metadump.dirty_log = true;
> }
> pop_cur();
> }
>
> - block_index = (__be64 *)((char *)metablock + sizeof(xfs_metablock_t));
> - block_buffer = (char *)metablock + BBSIZE;
> - num_indices = (BBSIZE - sizeof(xfs_metablock_t)) / sizeof(__be64);
> + metadump.metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
> + if (metadump.metablock == NULL) {
> + print_warning("memory allocation failure");
> + return -1;
> + }
> + metadump.metablock->mb_blocklog = BBSHIFT;
> + metadump.metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC);
> +
> + /* Set flags about state of metadump */
> + metadump.metablock->mb_info = XFS_METADUMP_INFO_FLAGS;
> + if (metadump.obfuscate)
> + metadump.metablock->mb_info |= XFS_METADUMP_OBFUSCATED;
> + if (!metadump.zero_stale_data)
> + metadump.metablock->mb_info |= XFS_METADUMP_FULLBLOCKS;
> + if (metadump.dirty_log)
> + metadump.metablock->mb_info |= XFS_METADUMP_DIRTYLOG;
> +
> + metadump.block_index = (__be64 *)((char *)metadump.metablock +
> + sizeof(xfs_metablock_t));
> + metadump.block_buffer = (char *)metadump.metablock + BBSIZE;
> + metadump.num_indices = (BBSIZE - sizeof(xfs_metablock_t)) /
> + sizeof(__be64);
>
> /*
> * A metadump block can hold at most num_indices of BBSIZE sectors;
> * do not try to dump a filesystem with a sector size which does not
> * fit within num_indices (i.e. within a single metablock).
> */
> - if (mp->m_sb.sb_sectsize > num_indices * BBSIZE) {
> + if (mp->m_sb.sb_sectsize > metadump.num_indices * BBSIZE) {
> print_warning("Cannot dump filesystem with sector size %u",
> mp->m_sb.sb_sectsize);
> - free(metablock);
> + free(metadump.metablock);
> return 0;
> }
>
> - cur_index = 0;
> start_iocur_sp = iocur_sp;
>
> if (strcmp(argv[optind], "-") == 0) {
> if (isatty(fileno(stdout))) {
> print_warning("cannot write to a terminal");
> - free(metablock);
> + free(metadump.metablock);
> return 0;
> }
> /*
> @@ -3111,17 +3140,17 @@ metadump_f(
> close(outfd);
> goto out;
> }
> - outf = fdopen(outfd, "a");
> - if (outf == NULL) {
> + metadump.outf = fdopen(outfd, "a");
> + if (metadump.outf == NULL) {
> fprintf(stderr, "cannot create dump stream\n");
> dup2(outfd, STDOUT_FILENO);
> close(outfd);
> goto out;
> }
> - stdout_metadump = true;
> + metadump.stdout_metadump = true;
> } else {
> - outf = fopen(argv[optind], "wb");
> - if (outf == NULL) {
> + metadump.outf = fopen(argv[optind], "wb");
> + if (metadump.outf == NULL) {
> print_warning("cannot create dump file");
> goto out;
> }
> @@ -3148,24 +3177,24 @@ metadump_f(
> if (!exitcode)
> exitcode = write_index() < 0;
>
> - if (progress_since_warning)
> - fputc('\n', stdout_metadump ? stderr : stdout);
> + if (metadump.progress_since_warning)
> + fputc('\n', metadump.stdout_metadump ? stderr : stdout);
>
> - if (stdout_metadump) {
> - fflush(outf);
> + if (metadump.stdout_metadump) {
> + fflush(metadump.outf);
> fflush(stdout);
> ret = dup2(outfd, STDOUT_FILENO);
> if (ret < 0)
> perror("un-redirecting stdout");
> - stdout_metadump = false;
> + metadump.stdout_metadump = false;
> }
> - fclose(outf);
> + fclose(metadump.outf);
>
> /* cleanup iocur stack */
> while (iocur_sp > start_iocur_sp)
> pop_cur();
> out:
> - free(metablock);
> + free(metadump.metablock);
>
> return 0;
> }
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 03/24] metadump: Define and use struct metadump
2023-05-23 16:35 ` Darrick J. Wong
@ 2023-05-24 4:50 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-24 4:50 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 09:35:14 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:29PM +0530, Chandan Babu R wrote:
>> This commit collects all state tracking variables in a new "struct metadump"
>> structure.
>
> I think this commit message needs to capture the reasons for /why/ all
> these global variables are being pulled into a struct definition that
> itself is used once to define a global variable.
>
The purpose of moving the global variables into one structure was to collect
them in one place in the code rather than having them spread across the
file. A new member of type "struct metadump_ops *" will be added by a future
commit to support the two versions of metadump.
I will add the above as part of the commit description.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 04/24] metadump: Add initialization and release functions
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (2 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 03/24] metadump: Define and use struct metadump Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 16:36 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 05/24] set_cur: Add support to read from external log device Chandan Babu R
` (19 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
Move metadump initialization and release functionality into corresponding
functions.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
db/metadump.c | 88 ++++++++++++++++++++++++++++++---------------------
1 file changed, 52 insertions(+), 36 deletions(-)
diff --git a/db/metadump.c b/db/metadump.c
index 806cdfd68..e7a433c21 100644
--- a/db/metadump.c
+++ b/db/metadump.c
@@ -2984,6 +2984,54 @@ done:
return !write_buf(iocur_top);
}
+static int
+init_metadump(void)
+{
+ metadump.metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
+ if (metadump.metablock == NULL) {
+ print_warning("memory allocation failure");
+ return -1;
+ }
+ metadump.metablock->mb_blocklog = BBSHIFT;
+ metadump.metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC);
+
+ /* Set flags about state of metadump */
+ metadump.metablock->mb_info = XFS_METADUMP_INFO_FLAGS;
+ if (metadump.obfuscate)
+ metadump.metablock->mb_info |= XFS_METADUMP_OBFUSCATED;
+ if (!metadump.zero_stale_data)
+ metadump.metablock->mb_info |= XFS_METADUMP_FULLBLOCKS;
+ if (metadump.dirty_log)
+ metadump.metablock->mb_info |= XFS_METADUMP_DIRTYLOG;
+
+ metadump.block_index = (__be64 *)((char *)metadump.metablock +
+ sizeof(xfs_metablock_t));
+ metadump.block_buffer = (char *)(metadump.metablock) + BBSIZE;
+ metadump.num_indices = (BBSIZE - sizeof(xfs_metablock_t)) / sizeof(__be64);
+
+ /*
+ * A metadump block can hold at most num_indices of BBSIZE sectors;
+ * do not try to dump a filesystem with a sector size which does not
+ * fit within num_indices (i.e. within a single metablock).
+ */
+ if (mp->m_sb.sb_sectsize > metadump.num_indices * BBSIZE) {
+ print_warning("Cannot dump filesystem with sector size %u",
+ mp->m_sb.sb_sectsize);
+ free(metadump.metablock);
+ return -1;
+ }
+
+ metadump.cur_index = 0;
+
+ return 0;
+}
+
+static void
+release_metadump(void)
+{
+ free(metadump.metablock);
+}
+
static int
metadump_f(
int argc,
@@ -3076,48 +3124,16 @@ metadump_f(
pop_cur();
}
- metadump.metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
- if (metadump.metablock == NULL) {
- print_warning("memory allocation failure");
- return -1;
- }
- metadump.metablock->mb_blocklog = BBSHIFT;
- metadump.metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC);
-
- /* Set flags about state of metadump */
- metadump.metablock->mb_info = XFS_METADUMP_INFO_FLAGS;
- if (metadump.obfuscate)
- metadump.metablock->mb_info |= XFS_METADUMP_OBFUSCATED;
- if (!metadump.zero_stale_data)
- metadump.metablock->mb_info |= XFS_METADUMP_FULLBLOCKS;
- if (metadump.dirty_log)
- metadump.metablock->mb_info |= XFS_METADUMP_DIRTYLOG;
-
- metadump.block_index = (__be64 *)((char *)metadump.metablock +
- sizeof(xfs_metablock_t));
- metadump.block_buffer = (char *)metadump.metablock + BBSIZE;
- metadump.num_indices = (BBSIZE - sizeof(xfs_metablock_t)) /
- sizeof(__be64);
-
- /*
- * A metadump block can hold at most num_indices of BBSIZE sectors;
- * do not try to dump a filesystem with a sector size which does not
- * fit within num_indices (i.e. within a single metablock).
- */
- if (mp->m_sb.sb_sectsize > metadump.num_indices * BBSIZE) {
- print_warning("Cannot dump filesystem with sector size %u",
- mp->m_sb.sb_sectsize);
- free(metadump.metablock);
+ ret = init_metadump();
+ if (ret)
return 0;
- }
start_iocur_sp = iocur_sp;
if (strcmp(argv[optind], "-") == 0) {
if (isatty(fileno(stdout))) {
print_warning("cannot write to a terminal");
- free(metadump.metablock);
- return 0;
+ goto out;
}
/*
* Redirect stdout to stderr for the duration of the
@@ -3194,7 +3210,7 @@ metadump_f(
while (iocur_sp > start_iocur_sp)
pop_cur();
out:
- free(metadump.metablock);
+ release_metadump();
return 0;
}
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 04/24] metadump: Add initialization and release functions
2023-05-23 9:00 ` [PATCH 04/24] metadump: Add initialization and release functions Chandan Babu R
@ 2023-05-23 16:36 ` Darrick J. Wong
2023-05-24 5:03 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 16:36 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:30PM +0530, Chandan Babu R wrote:
> Move metadump initialization and release functionality into corresponding
> functions.
"No functional changes"?
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> db/metadump.c | 88 ++++++++++++++++++++++++++++++---------------------
> 1 file changed, 52 insertions(+), 36 deletions(-)
>
> diff --git a/db/metadump.c b/db/metadump.c
> index 806cdfd68..e7a433c21 100644
> --- a/db/metadump.c
> +++ b/db/metadump.c
> @@ -2984,6 +2984,54 @@ done:
> return !write_buf(iocur_top);
> }
>
> +static int
> +init_metadump(void)
> +{
> + metadump.metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
> + if (metadump.metablock == NULL) {
> + print_warning("memory allocation failure");
> + return -1;
> + }
> + metadump.metablock->mb_blocklog = BBSHIFT;
> + metadump.metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC);
> +
> + /* Set flags about state of metadump */
> + metadump.metablock->mb_info = XFS_METADUMP_INFO_FLAGS;
> + if (metadump.obfuscate)
> + metadump.metablock->mb_info |= XFS_METADUMP_OBFUSCATED;
> + if (!metadump.zero_stale_data)
> + metadump.metablock->mb_info |= XFS_METADUMP_FULLBLOCKS;
> + if (metadump.dirty_log)
> + metadump.metablock->mb_info |= XFS_METADUMP_DIRTYLOG;
> +
> + metadump.block_index = (__be64 *)((char *)metadump.metablock +
> + sizeof(xfs_metablock_t));
> + metadump.block_buffer = (char *)(metadump.metablock) + BBSIZE;
> + metadump.num_indices = (BBSIZE - sizeof(xfs_metablock_t)) / sizeof(__be64);
> +
> + /*
> + * A metadump block can hold at most num_indices of BBSIZE sectors;
> + * do not try to dump a filesystem with a sector size which does not
> + * fit within num_indices (i.e. within a single metablock).
> + */
> + if (mp->m_sb.sb_sectsize > metadump.num_indices * BBSIZE) {
> + print_warning("Cannot dump filesystem with sector size %u",
> + mp->m_sb.sb_sectsize);
> + free(metadump.metablock);
> + return -1;
> + }
> +
> + metadump.cur_index = 0;
> +
> + return 0;
Tabs, not spaces.
With that fixed,
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
--D
> +}
> +
> +static void
> +release_metadump(void)
> +{
> + free(metadump.metablock);
> +}
> +
> static int
> metadump_f(
> int argc,
> @@ -3076,48 +3124,16 @@ metadump_f(
> pop_cur();
> }
>
> - metadump.metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
> - if (metadump.metablock == NULL) {
> - print_warning("memory allocation failure");
> - return -1;
> - }
> - metadump.metablock->mb_blocklog = BBSHIFT;
> - metadump.metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC);
> -
> - /* Set flags about state of metadump */
> - metadump.metablock->mb_info = XFS_METADUMP_INFO_FLAGS;
> - if (metadump.obfuscate)
> - metadump.metablock->mb_info |= XFS_METADUMP_OBFUSCATED;
> - if (!metadump.zero_stale_data)
> - metadump.metablock->mb_info |= XFS_METADUMP_FULLBLOCKS;
> - if (metadump.dirty_log)
> - metadump.metablock->mb_info |= XFS_METADUMP_DIRTYLOG;
> -
> - metadump.block_index = (__be64 *)((char *)metadump.metablock +
> - sizeof(xfs_metablock_t));
> - metadump.block_buffer = (char *)metadump.metablock + BBSIZE;
> - metadump.num_indices = (BBSIZE - sizeof(xfs_metablock_t)) /
> - sizeof(__be64);
> -
> - /*
> - * A metadump block can hold at most num_indices of BBSIZE sectors;
> - * do not try to dump a filesystem with a sector size which does not
> - * fit within num_indices (i.e. within a single metablock).
> - */
> - if (mp->m_sb.sb_sectsize > metadump.num_indices * BBSIZE) {
> - print_warning("Cannot dump filesystem with sector size %u",
> - mp->m_sb.sb_sectsize);
> - free(metadump.metablock);
> + ret = init_metadump();
> + if (ret)
> return 0;
> - }
>
> start_iocur_sp = iocur_sp;
>
> if (strcmp(argv[optind], "-") == 0) {
> if (isatty(fileno(stdout))) {
> print_warning("cannot write to a terminal");
> - free(metadump.metablock);
> - return 0;
> + goto out;
> }
> /*
> * Redirect stdout to stderr for the duration of the
> @@ -3194,7 +3210,7 @@ metadump_f(
> while (iocur_sp > start_iocur_sp)
> pop_cur();
> out:
> - free(metadump.metablock);
> + release_metadump();
>
> return 0;
> }
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 04/24] metadump: Add initialization and release functions
2023-05-23 16:36 ` Darrick J. Wong
@ 2023-05-24 5:03 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-24 5:03 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 09:36:38 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:30PM +0530, Chandan Babu R wrote:
>> Move metadump initialization and release functionality into corresponding
>> functions.
>
> "No functional changes"?
>
Yes, No functional changes are introduced by this patch. I will add that as
part of the commit description.
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> db/metadump.c | 88 ++++++++++++++++++++++++++++++---------------------
>> 1 file changed, 52 insertions(+), 36 deletions(-)
>>
>> diff --git a/db/metadump.c b/db/metadump.c
>> index 806cdfd68..e7a433c21 100644
>> --- a/db/metadump.c
>> +++ b/db/metadump.c
>> @@ -2984,6 +2984,54 @@ done:
>> return !write_buf(iocur_top);
>> }
>>
>> +static int
>> +init_metadump(void)
>> +{
>> + metadump.metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
>> + if (metadump.metablock == NULL) {
>> + print_warning("memory allocation failure");
>> + return -1;
>> + }
>> + metadump.metablock->mb_blocklog = BBSHIFT;
>> + metadump.metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC);
>> +
>> + /* Set flags about state of metadump */
>> + metadump.metablock->mb_info = XFS_METADUMP_INFO_FLAGS;
>> + if (metadump.obfuscate)
>> + metadump.metablock->mb_info |= XFS_METADUMP_OBFUSCATED;
>> + if (!metadump.zero_stale_data)
>> + metadump.metablock->mb_info |= XFS_METADUMP_FULLBLOCKS;
>> + if (metadump.dirty_log)
>> + metadump.metablock->mb_info |= XFS_METADUMP_DIRTYLOG;
>> +
>> + metadump.block_index = (__be64 *)((char *)metadump.metablock +
>> + sizeof(xfs_metablock_t));
>> + metadump.block_buffer = (char *)(metadump.metablock) + BBSIZE;
>> + metadump.num_indices = (BBSIZE - sizeof(xfs_metablock_t)) / sizeof(__be64);
>> +
>> + /*
>> + * A metadump block can hold at most num_indices of BBSIZE sectors;
>> + * do not try to dump a filesystem with a sector size which does not
>> + * fit within num_indices (i.e. within a single metablock).
>> + */
>> + if (mp->m_sb.sb_sectsize > metadump.num_indices * BBSIZE) {
>> + print_warning("Cannot dump filesystem with sector size %u",
>> + mp->m_sb.sb_sectsize);
>> + free(metadump.metablock);
>> + return -1;
>> + }
>> +
>> + metadump.cur_index = 0;
>> +
>> + return 0;
>
> Tabs, not spaces.
>
I tried to include your .vimrc configuration equivalent to my .emacs. But
looks like I didn't cover all the cases. I will fix up the whitespace
problems.
> With that fixed,
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Thank you.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 05/24] set_cur: Add support to read from external log device
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (3 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 04/24] metadump: Add initialization and release functions Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 16:48 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 06/24] metadump: Dump external log device contents Chandan Babu R
` (18 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
This commit changes set_cur() to be able to read from external log
devices. This is required by a future commit which will add the ability to
dump metadata from external log devices.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
db/io.c | 22 +++++++++++++++-------
db/type.c | 2 ++
db/type.h | 2 +-
3 files changed, 18 insertions(+), 8 deletions(-)
diff --git a/db/io.c b/db/io.c
index 3d2572364..e8c8f57e2 100644
--- a/db/io.c
+++ b/db/io.c
@@ -516,12 +516,13 @@ set_cur(
int ring_flag,
bbmap_t *bbmap)
{
- struct xfs_buf *bp;
- xfs_ino_t dirino;
- xfs_ino_t ino;
- uint16_t mode;
+ struct xfs_buftarg *btargp;
+ struct xfs_buf *bp;
+ xfs_ino_t dirino;
+ xfs_ino_t ino;
+ uint16_t mode;
const struct xfs_buf_ops *ops = type ? type->bops : NULL;
- int error;
+ int error;
if (iocur_sp < 0) {
dbprintf(_("set_cur no stack element to set\n"));
@@ -534,7 +535,14 @@ set_cur(
pop_cur();
push_cur();
+ btargp = mp->m_ddev_targp;
+ if (type->typnm == TYP_ELOG) {
+ ASSERT(mp->m_ddev_targp != mp->m_logdev_targp);
+ btargp = mp->m_logdev_targp;
+ }
+
if (bbmap) {
+ ASSERT(btargp == mp->m_ddev_targp);
#ifdef DEBUG_BBMAP
int i;
printf(_("xfs_db got a bbmap for %lld\n"), (long long)blknum);
@@ -548,11 +556,11 @@ set_cur(
if (!iocur_top->bbmap)
return;
memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
- error = -libxfs_buf_read_map(mp->m_ddev_targp, bbmap->b,
+ error = -libxfs_buf_read_map(btargp, bbmap->b,
bbmap->nmaps, LIBXFS_READBUF_SALVAGE, &bp,
ops);
} else {
- error = -libxfs_buf_read(mp->m_ddev_targp, blknum, len,
+ error = -libxfs_buf_read(btargp, blknum, len,
LIBXFS_READBUF_SALVAGE, &bp, ops);
iocur_top->bbmap = NULL;
}
diff --git a/db/type.c b/db/type.c
index efe704456..cc406ae4c 100644
--- a/db/type.c
+++ b/db/type.c
@@ -100,6 +100,7 @@ static const typ_t __typtab_crc[] = {
{ TYP_INODE, "inode", handle_struct, inode_crc_hfld,
&xfs_inode_buf_ops, TYP_F_CRC_FUNC, xfs_inode_set_crc },
{ TYP_LOG, "log", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
+ { TYP_ELOG, "elog", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
{ TYP_RTBITMAP, "rtbitmap", handle_text, NULL, NULL, TYP_F_NO_CRC_OFF },
{ TYP_RTSUMMARY, "rtsummary", handle_text, NULL, NULL, TYP_F_NO_CRC_OFF },
{ TYP_SB, "sb", handle_struct, sb_hfld, &xfs_sb_buf_ops,
@@ -144,6 +145,7 @@ static const typ_t __typtab_spcrc[] = {
{ TYP_INODE, "inode", handle_struct, inode_crc_hfld,
&xfs_inode_buf_ops, TYP_F_CRC_FUNC, xfs_inode_set_crc },
{ TYP_LOG, "log", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
+ { TYP_ELOG, "elog", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
{ TYP_RTBITMAP, "rtbitmap", handle_text, NULL, NULL, TYP_F_NO_CRC_OFF },
{ TYP_RTSUMMARY, "rtsummary", handle_text, NULL, NULL, TYP_F_NO_CRC_OFF },
{ TYP_SB, "sb", handle_struct, sb_hfld, &xfs_sb_buf_ops,
diff --git a/db/type.h b/db/type.h
index 411bfe90d..feb5c8219 100644
--- a/db/type.h
+++ b/db/type.h
@@ -14,7 +14,7 @@ typedef enum typnm
TYP_AGF, TYP_AGFL, TYP_AGI, TYP_ATTR, TYP_BMAPBTA,
TYP_BMAPBTD, TYP_BNOBT, TYP_CNTBT, TYP_RMAPBT, TYP_REFCBT, TYP_DATA,
TYP_DIR2, TYP_DQBLK, TYP_INOBT, TYP_INODATA, TYP_INODE,
- TYP_LOG, TYP_RTBITMAP, TYP_RTSUMMARY, TYP_SB, TYP_SYMLINK,
+ TYP_LOG, TYP_ELOG, TYP_RTBITMAP, TYP_RTSUMMARY, TYP_SB, TYP_SYMLINK,
TYP_TEXT, TYP_FINOBT, TYP_NONE
} typnm_t;
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 05/24] set_cur: Add support to read from external log device
2023-05-23 9:00 ` [PATCH 05/24] set_cur: Add support to read from external log device Chandan Babu R
@ 2023-05-23 16:48 ` Darrick J. Wong
2023-05-25 8:27 ` Chandan Babu R
2023-06-05 9:19 ` Chandan Babu R
0 siblings, 2 replies; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 16:48 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:31PM +0530, Chandan Babu R wrote:
> This commit changes set_cur() to be able to read from external log
> devices. This is required by a future commit which will add the ability to
> dump metadata from external log devices.
>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> db/io.c | 22 +++++++++++++++-------
> db/type.c | 2 ++
> db/type.h | 2 +-
> 3 files changed, 18 insertions(+), 8 deletions(-)
>
> diff --git a/db/io.c b/db/io.c
> index 3d2572364..e8c8f57e2 100644
> --- a/db/io.c
> +++ b/db/io.c
> @@ -516,12 +516,13 @@ set_cur(
> int ring_flag,
> bbmap_t *bbmap)
> {
> - struct xfs_buf *bp;
> - xfs_ino_t dirino;
> - xfs_ino_t ino;
> - uint16_t mode;
> + struct xfs_buftarg *btargp;
> + struct xfs_buf *bp;
> + xfs_ino_t dirino;
> + xfs_ino_t ino;
> + uint16_t mode;
> const struct xfs_buf_ops *ops = type ? type->bops : NULL;
> - int error;
> + int error;
>
> if (iocur_sp < 0) {
> dbprintf(_("set_cur no stack element to set\n"));
> @@ -534,7 +535,14 @@ set_cur(
> pop_cur();
> push_cur();
>
> + btargp = mp->m_ddev_targp;
> + if (type->typnm == TYP_ELOG) {
This feels like a layering violation, see below...
> + ASSERT(mp->m_ddev_targp != mp->m_logdev_targp);
> + btargp = mp->m_logdev_targp;
> + }
> +
> if (bbmap) {
> + ASSERT(btargp == mp->m_ddev_targp);
> #ifdef DEBUG_BBMAP
> int i;
> printf(_("xfs_db got a bbmap for %lld\n"), (long long)blknum);
> @@ -548,11 +556,11 @@ set_cur(
> if (!iocur_top->bbmap)
> return;
> memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
> - error = -libxfs_buf_read_map(mp->m_ddev_targp, bbmap->b,
> + error = -libxfs_buf_read_map(btargp, bbmap->b,
> bbmap->nmaps, LIBXFS_READBUF_SALVAGE, &bp,
> ops);
> } else {
> - error = -libxfs_buf_read(mp->m_ddev_targp, blknum, len,
> + error = -libxfs_buf_read(btargp, blknum, len,
> LIBXFS_READBUF_SALVAGE, &bp, ops);
> iocur_top->bbmap = NULL;
> }
> diff --git a/db/type.c b/db/type.c
> index efe704456..cc406ae4c 100644
> --- a/db/type.c
> +++ b/db/type.c
> @@ -100,6 +100,7 @@ static const typ_t __typtab_crc[] = {
> { TYP_INODE, "inode", handle_struct, inode_crc_hfld,
> &xfs_inode_buf_ops, TYP_F_CRC_FUNC, xfs_inode_set_crc },
> { TYP_LOG, "log", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
> + { TYP_ELOG, "elog", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
It strikes me as a little odd to create a new /metadata type/ to
reference the external log. If we someday want to add a bunch of new
types to xfs_db to allow us to decode/fuzz the log contents, wouldn't we
have to add them twice -- once for decoding an internal log, and again
to decode the external log? And the only difference between the two
would be the buftarg, right? The set_cur caller needs to know the
daddr already, so I don't think it's unreasonable for the caller to have
to know which buftarg too.
IOWs, I think set_cur ought to take the buftarg, the typ_t, and a daddr
as explicit arguments. But maybe others have opinions?
e.g. rename set_cur to __set_cur and make it take a buftarg, and then:
int
set_log_cur(
const typ_t *type,
xfs_daddr_t blknum,
int len,
int ring_flag,
bbmap_t *bbmap)
{
if (!mp->m_logdev_targp->bt_bdev ||
mp->m_logdev_targp->bt_bdev == mp->m_ddev_targp->bt_bdev) {
printf(_("external log device not loaded, use -l.\n"));
return ENODEV;
}
__set_cur(mp->m_logdev_targp, type, blknum, len, ring_flag, bbmap);
return 0;
}
and then metadump can do something like ....
error = set_log_cur(&typtab[TYP_LOG], 0,
mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL);
--D
> { TYP_RTBITMAP, "rtbitmap", handle_text, NULL, NULL, TYP_F_NO_CRC_OFF },
> { TYP_RTSUMMARY, "rtsummary", handle_text, NULL, NULL, TYP_F_NO_CRC_OFF },
> { TYP_SB, "sb", handle_struct, sb_hfld, &xfs_sb_buf_ops,
> @@ -144,6 +145,7 @@ static const typ_t __typtab_spcrc[] = {
> { TYP_INODE, "inode", handle_struct, inode_crc_hfld,
> &xfs_inode_buf_ops, TYP_F_CRC_FUNC, xfs_inode_set_crc },
> { TYP_LOG, "log", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
> + { TYP_ELOG, "elog", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
> { TYP_RTBITMAP, "rtbitmap", handle_text, NULL, NULL, TYP_F_NO_CRC_OFF },
> { TYP_RTSUMMARY, "rtsummary", handle_text, NULL, NULL, TYP_F_NO_CRC_OFF },
> { TYP_SB, "sb", handle_struct, sb_hfld, &xfs_sb_buf_ops,
> diff --git a/db/type.h b/db/type.h
> index 411bfe90d..feb5c8219 100644
> --- a/db/type.h
> +++ b/db/type.h
> @@ -14,7 +14,7 @@ typedef enum typnm
> TYP_AGF, TYP_AGFL, TYP_AGI, TYP_ATTR, TYP_BMAPBTA,
> TYP_BMAPBTD, TYP_BNOBT, TYP_CNTBT, TYP_RMAPBT, TYP_REFCBT, TYP_DATA,
> TYP_DIR2, TYP_DQBLK, TYP_INOBT, TYP_INODATA, TYP_INODE,
> - TYP_LOG, TYP_RTBITMAP, TYP_RTSUMMARY, TYP_SB, TYP_SYMLINK,
> + TYP_LOG, TYP_ELOG, TYP_RTBITMAP, TYP_RTSUMMARY, TYP_SB, TYP_SYMLINK,
> TYP_TEXT, TYP_FINOBT, TYP_NONE
> } typnm_t;
>
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 05/24] set_cur: Add support to read from external log device
2023-05-23 16:48 ` Darrick J. Wong
@ 2023-05-25 8:27 ` Chandan Babu R
2023-06-05 9:19 ` Chandan Babu R
1 sibling, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 8:27 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 09:48:07 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:31PM +0530, Chandan Babu R wrote:
>> This commit changes set_cur() to be able to read from external log
>> devices. This is required by a future commit which will add the ability to
>> dump metadata from external log devices.
>>
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> db/io.c | 22 +++++++++++++++-------
>> db/type.c | 2 ++
>> db/type.h | 2 +-
>> 3 files changed, 18 insertions(+), 8 deletions(-)
>>
>> diff --git a/db/io.c b/db/io.c
>> index 3d2572364..e8c8f57e2 100644
>> --- a/db/io.c
>> +++ b/db/io.c
>> @@ -516,12 +516,13 @@ set_cur(
>> int ring_flag,
>> bbmap_t *bbmap)
>> {
>> - struct xfs_buf *bp;
>> - xfs_ino_t dirino;
>> - xfs_ino_t ino;
>> - uint16_t mode;
>> + struct xfs_buftarg *btargp;
>> + struct xfs_buf *bp;
>> + xfs_ino_t dirino;
>> + xfs_ino_t ino;
>> + uint16_t mode;
>> const struct xfs_buf_ops *ops = type ? type->bops : NULL;
>> - int error;
>> + int error;
>>
>> if (iocur_sp < 0) {
>> dbprintf(_("set_cur no stack element to set\n"));
>> @@ -534,7 +535,14 @@ set_cur(
>> pop_cur();
>> push_cur();
>>
>> + btargp = mp->m_ddev_targp;
>> + if (type->typnm == TYP_ELOG) {
>
> This feels like a layering violation, see below...
>
>> + ASSERT(mp->m_ddev_targp != mp->m_logdev_targp);
>> + btargp = mp->m_logdev_targp;
>> + }
>> +
>> if (bbmap) {
>> + ASSERT(btargp == mp->m_ddev_targp);
>> #ifdef DEBUG_BBMAP
>> int i;
>> printf(_("xfs_db got a bbmap for %lld\n"), (long long)blknum);
>> @@ -548,11 +556,11 @@ set_cur(
>> if (!iocur_top->bbmap)
>> return;
>> memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
>> - error = -libxfs_buf_read_map(mp->m_ddev_targp, bbmap->b,
>> + error = -libxfs_buf_read_map(btargp, bbmap->b,
>> bbmap->nmaps, LIBXFS_READBUF_SALVAGE, &bp,
>> ops);
>> } else {
>> - error = -libxfs_buf_read(mp->m_ddev_targp, blknum, len,
>> + error = -libxfs_buf_read(btargp, blknum, len,
>> LIBXFS_READBUF_SALVAGE, &bp, ops);
>> iocur_top->bbmap = NULL;
>> }
>> diff --git a/db/type.c b/db/type.c
>> index efe704456..cc406ae4c 100644
>> --- a/db/type.c
>> +++ b/db/type.c
>> @@ -100,6 +100,7 @@ static const typ_t __typtab_crc[] = {
>> { TYP_INODE, "inode", handle_struct, inode_crc_hfld,
>> &xfs_inode_buf_ops, TYP_F_CRC_FUNC, xfs_inode_set_crc },
>> { TYP_LOG, "log", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
>> + { TYP_ELOG, "elog", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
>
> It strikes me as a little odd to create a new /metadata type/ to
> reference the external log. If we someday want to add a bunch of new
> types to xfs_db to allow us to decode/fuzz the log contents, wouldn't we
> have to add them twice -- once for decoding an internal log, and again
> to decode the external log? And the only difference between the two
> would be the buftarg, right? The set_cur caller needs to know the
> daddr already, so I don't think it's unreasonable for the caller to have
> to know which buftarg too.
>
> IOWs, I think set_cur ought to take the buftarg, the typ_t, and a daddr
> as explicit arguments. But maybe others have opinions?
You are right about the requirement to add two entries for each possible
operation related to the log.
>
> e.g. rename set_cur to __set_cur and make it take a buftarg, and then:
>
> int
> set_log_cur(
> const typ_t *type,
> xfs_daddr_t blknum,
> int len,
> int ring_flag,
> bbmap_t *bbmap)
> {
> if (!mp->m_logdev_targp->bt_bdev ||
> mp->m_logdev_targp->bt_bdev == mp->m_ddev_targp->bt_bdev) {
> printf(_("external log device not loaded, use -l.\n"));
> return ENODEV;
> }
>
> __set_cur(mp->m_logdev_targp, type, blknum, len, ring_flag, bbmap);
> return 0;
> }
>
> and then metadump can do something like ....
>
> error = set_log_cur(&typtab[TYP_LOG], 0,
> mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL);
The above suggestion looks correct to me. I will include this change unless
others have any objections to it.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 05/24] set_cur: Add support to read from external log device
2023-05-23 16:48 ` Darrick J. Wong
2023-05-25 8:27 ` Chandan Babu R
@ 2023-06-05 9:19 ` Chandan Babu R
2023-06-05 19:22 ` Darrick J. Wong
1 sibling, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-06-05 9:19 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 09:48:07 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:31PM +0530, Chandan Babu R wrote:
>> This commit changes set_cur() to be able to read from external log
>> devices. This is required by a future commit which will add the ability to
>> dump metadata from external log devices.
>>
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> db/io.c | 22 +++++++++++++++-------
>> db/type.c | 2 ++
>> db/type.h | 2 +-
>> 3 files changed, 18 insertions(+), 8 deletions(-)
>>
>> diff --git a/db/io.c b/db/io.c
>> index 3d2572364..e8c8f57e2 100644
>> --- a/db/io.c
>> +++ b/db/io.c
>> @@ -516,12 +516,13 @@ set_cur(
>> int ring_flag,
>> bbmap_t *bbmap)
>> {
>> - struct xfs_buf *bp;
>> - xfs_ino_t dirino;
>> - xfs_ino_t ino;
>> - uint16_t mode;
>> + struct xfs_buftarg *btargp;
>> + struct xfs_buf *bp;
>> + xfs_ino_t dirino;
>> + xfs_ino_t ino;
>> + uint16_t mode;
>> const struct xfs_buf_ops *ops = type ? type->bops : NULL;
>> - int error;
>> + int error;
>>
>> if (iocur_sp < 0) {
>> dbprintf(_("set_cur no stack element to set\n"));
>> @@ -534,7 +535,14 @@ set_cur(
>> pop_cur();
>> push_cur();
>>
>> + btargp = mp->m_ddev_targp;
>> + if (type->typnm == TYP_ELOG) {
>
> This feels like a layering violation, see below...
>
>> + ASSERT(mp->m_ddev_targp != mp->m_logdev_targp);
>> + btargp = mp->m_logdev_targp;
>> + }
>> +
>> if (bbmap) {
>> + ASSERT(btargp == mp->m_ddev_targp);
>> #ifdef DEBUG_BBMAP
>> int i;
>> printf(_("xfs_db got a bbmap for %lld\n"), (long long)blknum);
>> @@ -548,11 +556,11 @@ set_cur(
>> if (!iocur_top->bbmap)
>> return;
>> memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
>> - error = -libxfs_buf_read_map(mp->m_ddev_targp, bbmap->b,
>> + error = -libxfs_buf_read_map(btargp, bbmap->b,
>> bbmap->nmaps, LIBXFS_READBUF_SALVAGE, &bp,
>> ops);
>> } else {
>> - error = -libxfs_buf_read(mp->m_ddev_targp, blknum, len,
>> + error = -libxfs_buf_read(btargp, blknum, len,
>> LIBXFS_READBUF_SALVAGE, &bp, ops);
>> iocur_top->bbmap = NULL;
>> }
>> diff --git a/db/type.c b/db/type.c
>> index efe704456..cc406ae4c 100644
>> --- a/db/type.c
>> +++ b/db/type.c
>> @@ -100,6 +100,7 @@ static const typ_t __typtab_crc[] = {
>> { TYP_INODE, "inode", handle_struct, inode_crc_hfld,
>> &xfs_inode_buf_ops, TYP_F_CRC_FUNC, xfs_inode_set_crc },
>> { TYP_LOG, "log", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
>> + { TYP_ELOG, "elog", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
>
> It strikes me as a little odd to create a new /metadata type/ to
> reference the external log. If we someday want to add a bunch of new
> types to xfs_db to allow us to decode/fuzz the log contents, wouldn't we
> have to add them twice -- once for decoding an internal log, and again
> to decode the external log? And the only difference between the two
> would be the buftarg, right? The set_cur caller needs to know the
> daddr already, so I don't think it's unreasonable for the caller to have
> to know which buftarg too.
>
> IOWs, I think set_cur ought to take the buftarg, the typ_t, and a daddr
> as explicit arguments. But maybe others have opinions?
>
> e.g. rename set_cur to __set_cur and make it take a buftarg, and then:
>
> int
> set_log_cur(
> const typ_t *type,
> xfs_daddr_t blknum,
> int len,
> int ring_flag,
> bbmap_t *bbmap)
> {
> if (!mp->m_logdev_targp->bt_bdev ||
> mp->m_logdev_targp->bt_bdev == mp->m_ddev_targp->bt_bdev) {
> printf(_("external log device not loaded, use -l.\n"));
> return ENODEV;
> }
>
> __set_cur(mp->m_logdev_targp, type, blknum, len, ring_flag, bbmap);
> return 0;
> }
>
> and then metadump can do something like ....
>
> error = set_log_cur(&typtab[TYP_LOG], 0,
> mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL);
>
Darrick, How about implementing the following instead,
static void
__set_cur(
struct xfs_buftarg *btargp,
const typ_t *type,
xfs_daddr_t blknum,
int len,
int ring_flag,
bbmap_t *bbmap)
{
struct xfs_buf *bp;
xfs_ino_t dirino;
xfs_ino_t ino;
uint16_t mode;
const struct xfs_buf_ops *ops = type ? type->bops : NULL;
int error;
if (iocur_sp < 0) {
dbprintf(_("set_cur no stack element to set\n"));
return;
}
ino = iocur_top->ino;
dirino = iocur_top->dirino;
mode = iocur_top->mode;
pop_cur();
push_cur();
if (bbmap) {
#ifdef DEBUG_BBMAP
int i;
printf(_("xfs_db got a bbmap for %lld\n"), (long long)blknum);
printf(_("\tblock map"));
for (i = 0; i < bbmap->nmaps; i++)
printf(" %lld:%d", (long long)bbmap->b[i].bm_bn,
bbmap->b[i].bm_len);
printf("\n");
#endif
iocur_top->bbmap = malloc(sizeof(struct bbmap));
if (!iocur_top->bbmap)
return;
memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
error = -libxfs_buf_read_map(btargp, bbmap->b,
bbmap->nmaps, LIBXFS_READBUF_SALVAGE, &bp,
ops);
} else {
error = -libxfs_buf_read(btargp, blknum, len,
LIBXFS_READBUF_SALVAGE, &bp, ops);
iocur_top->bbmap = NULL;
}
/*
* Salvage mode means that we still get a buffer even if the verifier
* says the metadata is corrupt. Therefore, the only errors we should
* get are for IO errors or runtime errors.
*/
if (error)
return;
iocur_top->buf = bp->b_addr;
iocur_top->bp = bp;
if (!ops) {
bp->b_ops = NULL;
bp->b_flags |= LIBXFS_B_UNCHECKED;
}
iocur_top->bb = blknum;
iocur_top->blen = len;
iocur_top->boff = 0;
iocur_top->data = iocur_top->buf;
iocur_top->len = BBTOB(len);
iocur_top->off = blknum << BBSHIFT;
iocur_top->typ = cur_typ = type;
iocur_top->ino = ino;
iocur_top->dirino = dirino;
iocur_top->mode = mode;
iocur_top->ino_buf = 0;
iocur_top->dquot_buf = 0;
/* store location in ring */
if (ring_flag)
ring_add();
}
void
set_cur(
const typ_t *type,
xfs_daddr_t blknum,
int len,
int ring_flag,
bbmap_t *bbmap)
{
struct xfs_buftarg *btargp = mp->m_ddev_targp;
if (type->typnm == TYP_LOG &&
mp->m_logdev_targp->bt_bdev != mp->m_ddev_targp->bt_bdev) {
ASSERT(mp->m_sb.sb_logstart == 0);
btargp = mp->m_logdev_targp;
}
__set_cur(btargp, type, blknum, len, ring_flag, bbmap);
}
i.e. We continue to have just one type for the log and set_cur() will
internally decide which buftarg to pass to __set_cur(). Please let me know
your opinion on this approach.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 05/24] set_cur: Add support to read from external log device
2023-06-05 9:19 ` Chandan Babu R
@ 2023-06-05 19:22 ` Darrick J. Wong
2023-06-06 4:47 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-06-05 19:22 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Mon, Jun 05, 2023 at 02:49:49PM +0530, Chandan Babu R wrote:
> On Tue, May 23, 2023 at 09:48:07 AM -0700, Darrick J. Wong wrote:
> > On Tue, May 23, 2023 at 02:30:31PM +0530, Chandan Babu R wrote:
> >> This commit changes set_cur() to be able to read from external log
> >> devices. This is required by a future commit which will add the ability to
> >> dump metadata from external log devices.
> >>
> >> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> >> ---
> >> db/io.c | 22 +++++++++++++++-------
> >> db/type.c | 2 ++
> >> db/type.h | 2 +-
> >> 3 files changed, 18 insertions(+), 8 deletions(-)
> >>
> >> diff --git a/db/io.c b/db/io.c
> >> index 3d2572364..e8c8f57e2 100644
> >> --- a/db/io.c
> >> +++ b/db/io.c
> >> @@ -516,12 +516,13 @@ set_cur(
> >> int ring_flag,
> >> bbmap_t *bbmap)
> >> {
> >> - struct xfs_buf *bp;
> >> - xfs_ino_t dirino;
> >> - xfs_ino_t ino;
> >> - uint16_t mode;
> >> + struct xfs_buftarg *btargp;
> >> + struct xfs_buf *bp;
> >> + xfs_ino_t dirino;
> >> + xfs_ino_t ino;
> >> + uint16_t mode;
> >> const struct xfs_buf_ops *ops = type ? type->bops : NULL;
> >> - int error;
> >> + int error;
> >>
> >> if (iocur_sp < 0) {
> >> dbprintf(_("set_cur no stack element to set\n"));
> >> @@ -534,7 +535,14 @@ set_cur(
> >> pop_cur();
> >> push_cur();
> >>
> >> + btargp = mp->m_ddev_targp;
> >> + if (type->typnm == TYP_ELOG) {
> >
> > This feels like a layering violation, see below...
> >
> >> + ASSERT(mp->m_ddev_targp != mp->m_logdev_targp);
> >> + btargp = mp->m_logdev_targp;
> >> + }
> >> +
> >> if (bbmap) {
> >> + ASSERT(btargp == mp->m_ddev_targp);
> >> #ifdef DEBUG_BBMAP
> >> int i;
> >> printf(_("xfs_db got a bbmap for %lld\n"), (long long)blknum);
> >> @@ -548,11 +556,11 @@ set_cur(
> >> if (!iocur_top->bbmap)
> >> return;
> >> memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
> >> - error = -libxfs_buf_read_map(mp->m_ddev_targp, bbmap->b,
> >> + error = -libxfs_buf_read_map(btargp, bbmap->b,
> >> bbmap->nmaps, LIBXFS_READBUF_SALVAGE, &bp,
> >> ops);
> >> } else {
> >> - error = -libxfs_buf_read(mp->m_ddev_targp, blknum, len,
> >> + error = -libxfs_buf_read(btargp, blknum, len,
> >> LIBXFS_READBUF_SALVAGE, &bp, ops);
> >> iocur_top->bbmap = NULL;
> >> }
> >> diff --git a/db/type.c b/db/type.c
> >> index efe704456..cc406ae4c 100644
> >> --- a/db/type.c
> >> +++ b/db/type.c
> >> @@ -100,6 +100,7 @@ static const typ_t __typtab_crc[] = {
> >> { TYP_INODE, "inode", handle_struct, inode_crc_hfld,
> >> &xfs_inode_buf_ops, TYP_F_CRC_FUNC, xfs_inode_set_crc },
> >> { TYP_LOG, "log", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
> >> + { TYP_ELOG, "elog", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
> >
> > It strikes me as a little odd to create a new /metadata type/ to
> > reference the external log. If we someday want to add a bunch of new
> > types to xfs_db to allow us to decode/fuzz the log contents, wouldn't we
> > have to add them twice -- once for decoding an internal log, and again
> > to decode the external log? And the only difference between the two
> > would be the buftarg, right? The set_cur caller needs to know the
> > daddr already, so I don't think it's unreasonable for the caller to have
> > to know which buftarg too.
> >
> > IOWs, I think set_cur ought to take the buftarg, the typ_t, and a daddr
> > as explicit arguments. But maybe others have opinions?
> >
> > e.g. rename set_cur to __set_cur and make it take a buftarg, and then:
> >
> > int
> > set_log_cur(
> > const typ_t *type,
> > xfs_daddr_t blknum,
> > int len,
> > int ring_flag,
> > bbmap_t *bbmap)
> > {
> > if (!mp->m_logdev_targp->bt_bdev ||
> > mp->m_logdev_targp->bt_bdev == mp->m_ddev_targp->bt_bdev) {
> > printf(_("external log device not loaded, use -l.\n"));
> > return ENODEV;
> > }
> >
> > __set_cur(mp->m_logdev_targp, type, blknum, len, ring_flag, bbmap);
> > return 0;
> > }
> >
> > and then metadump can do something like ....
> >
> > error = set_log_cur(&typtab[TYP_LOG], 0,
> > mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL);
> >
>
> Darrick, How about implementing the following instead,
>
> static void
> __set_cur(
> struct xfs_buftarg *btargp,
> const typ_t *type,
> xfs_daddr_t blknum,
> int len,
> int ring_flag,
> bbmap_t *bbmap)
> {
> struct xfs_buf *bp;
> xfs_ino_t dirino;
> xfs_ino_t ino;
> uint16_t mode;
> const struct xfs_buf_ops *ops = type ? type->bops : NULL;
> int error;
>
> if (iocur_sp < 0) {
> dbprintf(_("set_cur no stack element to set\n"));
> return;
> }
>
> ino = iocur_top->ino;
> dirino = iocur_top->dirino;
> mode = iocur_top->mode;
> pop_cur();
> push_cur();
>
> if (bbmap) {
> #ifdef DEBUG_BBMAP
> int i;
> printf(_("xfs_db got a bbmap for %lld\n"), (long long)blknum);
> printf(_("\tblock map"));
> for (i = 0; i < bbmap->nmaps; i++)
> printf(" %lld:%d", (long long)bbmap->b[i].bm_bn,
> bbmap->b[i].bm_len);
> printf("\n");
> #endif
> iocur_top->bbmap = malloc(sizeof(struct bbmap));
> if (!iocur_top->bbmap)
> return;
> memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
> error = -libxfs_buf_read_map(btargp, bbmap->b,
> bbmap->nmaps, LIBXFS_READBUF_SALVAGE, &bp,
> ops);
> } else {
> error = -libxfs_buf_read(btargp, blknum, len,
> LIBXFS_READBUF_SALVAGE, &bp, ops);
> iocur_top->bbmap = NULL;
> }
>
> /*
> * Salvage mode means that we still get a buffer even if the verifier
> * says the metadata is corrupt. Therefore, the only errors we should
> * get are for IO errors or runtime errors.
> */
> if (error)
> return;
> iocur_top->buf = bp->b_addr;
> iocur_top->bp = bp;
> if (!ops) {
> bp->b_ops = NULL;
> bp->b_flags |= LIBXFS_B_UNCHECKED;
> }
>
> iocur_top->bb = blknum;
> iocur_top->blen = len;
> iocur_top->boff = 0;
> iocur_top->data = iocur_top->buf;
> iocur_top->len = BBTOB(len);
> iocur_top->off = blknum << BBSHIFT;
> iocur_top->typ = cur_typ = type;
> iocur_top->ino = ino;
> iocur_top->dirino = dirino;
> iocur_top->mode = mode;
> iocur_top->ino_buf = 0;
> iocur_top->dquot_buf = 0;
>
> /* store location in ring */
> if (ring_flag)
> ring_add();
> }
>
> void
> set_cur(
> const typ_t *type,
> xfs_daddr_t blknum,
> int len,
> int ring_flag,
> bbmap_t *bbmap)
> {
> struct xfs_buftarg *btargp = mp->m_ddev_targp;
>
> if (type->typnm == TYP_LOG &&
> mp->m_logdev_targp->bt_bdev != mp->m_ddev_targp->bt_bdev) {
> ASSERT(mp->m_sb.sb_logstart == 0);
> btargp = mp->m_logdev_targp;
> }
>
> __set_cur(btargp, type, blknum, len, ring_flag, bbmap);
> }
>
> i.e. We continue to have just one type for the log and set_cur() will
> internally decide which buftarg to pass to __set_cur(). Please let me know
> your opinion on this approach.
If I'm understanding this correctly, you're proposing to push the
buftarg decision down into set_cur instead of encoding it in the typ_t
information?
I still don't like this, because that decision should be made by the
callers of set_*cur, not down in the io cursor handling code.
Take a look at the users of set_log_cur and set_rt_cur in the 'dblock'
command as of djwong-wtf:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/tree/db/block.c?h=djwong-wtf_2023-06-05#n217
Notice this bit here:
static inline bool
is_rtfile(
struct xfs_dinode *dip)
{
return dip->di_flags & cpu_to_be16(XFS_DIFLAG_REALTIME);
}
static int
dblock_f(...)
{
...
if (is_rtfile(iocur_top->data))
set_rt_cur(&typtab[type], (int64_t)XFS_FSB_TO_DADDR(mp, dfsbno),
nb * blkbb, DB_RING_ADD,
nex > 1 ? &bbmap : NULL);
else
set_cur(&typtab[type], (int64_t)XFS_FSB_TO_DADDR(mp, dfsbno),
nb * blkbb, DB_RING_ADD,
nex > 1 ? &bbmap : NULL);
xfs_db can now access the data blocks of realtime files, because we have
the high level logic to decide which buftarg based on the di_flags set
in the inode core. TYP_DATA doesn't know anything at all about inodes
or data blocks or whatever -- down at the level of "data block" we don't
actually have the context we need to select a device.
--D
> --
> chandan
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 05/24] set_cur: Add support to read from external log device
2023-06-05 19:22 ` Darrick J. Wong
@ 2023-06-06 4:47 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-06-06 4:47 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Mon, Jun 05, 2023 at 12:22:35 PM -0700, Darrick J. Wong wrote:
> On Mon, Jun 05, 2023 at 02:49:49PM +0530, Chandan Babu R wrote:
>> On Tue, May 23, 2023 at 09:48:07 AM -0700, Darrick J. Wong wrote:
>> > On Tue, May 23, 2023 at 02:30:31PM +0530, Chandan Babu R wrote:
>> >> This commit changes set_cur() to be able to read from external log
>> >> devices. This is required by a future commit which will add the ability to
>> >> dump metadata from external log devices.
>> >>
>> >> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> >> ---
>> >> db/io.c | 22 +++++++++++++++-------
>> >> db/type.c | 2 ++
>> >> db/type.h | 2 +-
>> >> 3 files changed, 18 insertions(+), 8 deletions(-)
>> >>
>> >> diff --git a/db/io.c b/db/io.c
>> >> index 3d2572364..e8c8f57e2 100644
>> >> --- a/db/io.c
>> >> +++ b/db/io.c
>> >> @@ -516,12 +516,13 @@ set_cur(
>> >> int ring_flag,
>> >> bbmap_t *bbmap)
>> >> {
>> >> - struct xfs_buf *bp;
>> >> - xfs_ino_t dirino;
>> >> - xfs_ino_t ino;
>> >> - uint16_t mode;
>> >> + struct xfs_buftarg *btargp;
>> >> + struct xfs_buf *bp;
>> >> + xfs_ino_t dirino;
>> >> + xfs_ino_t ino;
>> >> + uint16_t mode;
>> >> const struct xfs_buf_ops *ops = type ? type->bops : NULL;
>> >> - int error;
>> >> + int error;
>> >>
>> >> if (iocur_sp < 0) {
>> >> dbprintf(_("set_cur no stack element to set\n"));
>> >> @@ -534,7 +535,14 @@ set_cur(
>> >> pop_cur();
>> >> push_cur();
>> >>
>> >> + btargp = mp->m_ddev_targp;
>> >> + if (type->typnm == TYP_ELOG) {
>> >
>> > This feels like a layering violation, see below...
>> >
>> >> + ASSERT(mp->m_ddev_targp != mp->m_logdev_targp);
>> >> + btargp = mp->m_logdev_targp;
>> >> + }
>> >> +
>> >> if (bbmap) {
>> >> + ASSERT(btargp == mp->m_ddev_targp);
>> >> #ifdef DEBUG_BBMAP
>> >> int i;
>> >> printf(_("xfs_db got a bbmap for %lld\n"), (long long)blknum);
>> >> @@ -548,11 +556,11 @@ set_cur(
>> >> if (!iocur_top->bbmap)
>> >> return;
>> >> memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
>> >> - error = -libxfs_buf_read_map(mp->m_ddev_targp, bbmap->b,
>> >> + error = -libxfs_buf_read_map(btargp, bbmap->b,
>> >> bbmap->nmaps, LIBXFS_READBUF_SALVAGE, &bp,
>> >> ops);
>> >> } else {
>> >> - error = -libxfs_buf_read(mp->m_ddev_targp, blknum, len,
>> >> + error = -libxfs_buf_read(btargp, blknum, len,
>> >> LIBXFS_READBUF_SALVAGE, &bp, ops);
>> >> iocur_top->bbmap = NULL;
>> >> }
>> >> diff --git a/db/type.c b/db/type.c
>> >> index efe704456..cc406ae4c 100644
>> >> --- a/db/type.c
>> >> +++ b/db/type.c
>> >> @@ -100,6 +100,7 @@ static const typ_t __typtab_crc[] = {
>> >> { TYP_INODE, "inode", handle_struct, inode_crc_hfld,
>> >> &xfs_inode_buf_ops, TYP_F_CRC_FUNC, xfs_inode_set_crc },
>> >> { TYP_LOG, "log", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
>> >> + { TYP_ELOG, "elog", NULL, NULL, NULL, TYP_F_NO_CRC_OFF },
>> >
>> > It strikes me as a little odd to create a new /metadata type/ to
>> > reference the external log. If we someday want to add a bunch of new
>> > types to xfs_db to allow us to decode/fuzz the log contents, wouldn't we
>> > have to add them twice -- once for decoding an internal log, and again
>> > to decode the external log? And the only difference between the two
>> > would be the buftarg, right? The set_cur caller needs to know the
>> > daddr already, so I don't think it's unreasonable for the caller to have
>> > to know which buftarg too.
>> >
>> > IOWs, I think set_cur ought to take the buftarg, the typ_t, and a daddr
>> > as explicit arguments. But maybe others have opinions?
>> >
>> > e.g. rename set_cur to __set_cur and make it take a buftarg, and then:
>> >
>> > int
>> > set_log_cur(
>> > const typ_t *type,
>> > xfs_daddr_t blknum,
>> > int len,
>> > int ring_flag,
>> > bbmap_t *bbmap)
>> > {
>> > if (!mp->m_logdev_targp->bt_bdev ||
>> > mp->m_logdev_targp->bt_bdev == mp->m_ddev_targp->bt_bdev) {
>> > printf(_("external log device not loaded, use -l.\n"));
>> > return ENODEV;
>> > }
>> >
>> > __set_cur(mp->m_logdev_targp, type, blknum, len, ring_flag, bbmap);
>> > return 0;
>> > }
>> >
>> > and then metadump can do something like ....
>> >
>> > error = set_log_cur(&typtab[TYP_LOG], 0,
>> > mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL);
>> >
>>
>> Darrick, How about implementing the following instead,
>>
>> static void
>> __set_cur(
>> struct xfs_buftarg *btargp,
>> const typ_t *type,
>> xfs_daddr_t blknum,
>> int len,
>> int ring_flag,
>> bbmap_t *bbmap)
>> {
>> struct xfs_buf *bp;
>> xfs_ino_t dirino;
>> xfs_ino_t ino;
>> uint16_t mode;
>> const struct xfs_buf_ops *ops = type ? type->bops : NULL;
>> int error;
>>
>> if (iocur_sp < 0) {
>> dbprintf(_("set_cur no stack element to set\n"));
>> return;
>> }
>>
>> ino = iocur_top->ino;
>> dirino = iocur_top->dirino;
>> mode = iocur_top->mode;
>> pop_cur();
>> push_cur();
>>
>> if (bbmap) {
>> #ifdef DEBUG_BBMAP
>> int i;
>> printf(_("xfs_db got a bbmap for %lld\n"), (long long)blknum);
>> printf(_("\tblock map"));
>> for (i = 0; i < bbmap->nmaps; i++)
>> printf(" %lld:%d", (long long)bbmap->b[i].bm_bn,
>> bbmap->b[i].bm_len);
>> printf("\n");
>> #endif
>> iocur_top->bbmap = malloc(sizeof(struct bbmap));
>> if (!iocur_top->bbmap)
>> return;
>> memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
>> error = -libxfs_buf_read_map(btargp, bbmap->b,
>> bbmap->nmaps, LIBXFS_READBUF_SALVAGE, &bp,
>> ops);
>> } else {
>> error = -libxfs_buf_read(btargp, blknum, len,
>> LIBXFS_READBUF_SALVAGE, &bp, ops);
>> iocur_top->bbmap = NULL;
>> }
>>
>> /*
>> * Salvage mode means that we still get a buffer even if the verifier
>> * says the metadata is corrupt. Therefore, the only errors we should
>> * get are for IO errors or runtime errors.
>> */
>> if (error)
>> return;
>> iocur_top->buf = bp->b_addr;
>> iocur_top->bp = bp;
>> if (!ops) {
>> bp->b_ops = NULL;
>> bp->b_flags |= LIBXFS_B_UNCHECKED;
>> }
>>
>> iocur_top->bb = blknum;
>> iocur_top->blen = len;
>> iocur_top->boff = 0;
>> iocur_top->data = iocur_top->buf;
>> iocur_top->len = BBTOB(len);
>> iocur_top->off = blknum << BBSHIFT;
>> iocur_top->typ = cur_typ = type;
>> iocur_top->ino = ino;
>> iocur_top->dirino = dirino;
>> iocur_top->mode = mode;
>> iocur_top->ino_buf = 0;
>> iocur_top->dquot_buf = 0;
>>
>> /* store location in ring */
>> if (ring_flag)
>> ring_add();
>> }
>>
>> void
>> set_cur(
>> const typ_t *type,
>> xfs_daddr_t blknum,
>> int len,
>> int ring_flag,
>> bbmap_t *bbmap)
>> {
>> struct xfs_buftarg *btargp = mp->m_ddev_targp;
>>
>> if (type->typnm == TYP_LOG &&
>> mp->m_logdev_targp->bt_bdev != mp->m_ddev_targp->bt_bdev) {
>> ASSERT(mp->m_sb.sb_logstart == 0);
>> btargp = mp->m_logdev_targp;
>> }
>>
>> __set_cur(btargp, type, blknum, len, ring_flag, bbmap);
>> }
>>
>> i.e. We continue to have just one type for the log and set_cur() will
>> internally decide which buftarg to pass to __set_cur(). Please let me know
>> your opinion on this approach.
>
> If I'm understanding this correctly, you're proposing to push the
> buftarg decision down into set_cur instead of encoding it in the typ_t
> information?
>
> I still don't like this, because that decision should be made by the
> callers of set_*cur, not down in the io cursor handling code.
>
> Take a look at the users of set_log_cur and set_rt_cur in the 'dblock'
> command as of djwong-wtf:
> https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/tree/db/block.c?h=djwong-wtf_2023-06-05#n217
>
> Notice this bit here:
>
> static inline bool
> is_rtfile(
> struct xfs_dinode *dip)
> {
> return dip->di_flags & cpu_to_be16(XFS_DIFLAG_REALTIME);
> }
>
> static int
> dblock_f(...)
> {
> ...
>
> if (is_rtfile(iocur_top->data))
> set_rt_cur(&typtab[type], (int64_t)XFS_FSB_TO_DADDR(mp, dfsbno),
> nb * blkbb, DB_RING_ADD,
> nex > 1 ? &bbmap : NULL);
> else
> set_cur(&typtab[type], (int64_t)XFS_FSB_TO_DADDR(mp, dfsbno),
> nb * blkbb, DB_RING_ADD,
> nex > 1 ? &bbmap : NULL);
>
> xfs_db can now access the data blocks of realtime files, because we have
> the high level logic to decide which buftarg based on the di_flags set
> in the inode core. TYP_DATA doesn't know anything at all about inodes
> or data blocks or whatever -- down at the level of "data block" we don't
> actually have the context we need to select a device.
>
Ok. Now I understand. Thanks for the explaination.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 06/24] metadump: Dump external log device contents
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (4 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 05/24] set_cur: Add support to read from external log device Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:02 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 07/24] metadump: Postpone invocation of init_metadump() Chandan Babu R
` (17 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
metadump will now read and dump from external log device when the log is
placed on an external device and metadump v2 is supported by xfsprogs.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
db/metadump.c | 20 +++++++++++++-------
1 file changed, 13 insertions(+), 7 deletions(-)
diff --git a/db/metadump.c b/db/metadump.c
index e7a433c21..62a36427d 100644
--- a/db/metadump.c
+++ b/db/metadump.c
@@ -2921,7 +2921,7 @@ copy_sb_inodes(void)
}
static int
-copy_log(void)
+copy_log(enum typnm log_type)
{
struct xlog log;
int dirty;
@@ -2934,7 +2934,7 @@ copy_log(void)
print_progress("Copying log");
push_cur();
- set_cur(&typtab[TYP_LOG], XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart),
+ set_cur(&typtab[log_type], XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart),
mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL);
if (iocur_top->data == NULL) {
pop_cur();
@@ -3038,6 +3038,7 @@ metadump_f(
char **argv)
{
xfs_agnumber_t agno;
+ enum typnm log_type;
int c;
int start_iocur_sp;
int outfd = -1;
@@ -3110,9 +3111,13 @@ metadump_f(
}
/* If we'll copy the log, see if the log is dirty */
- if (mp->m_sb.sb_logstart) {
+ if (mp->m_logdev_targp == mp->m_ddev_targp || metadump.version == 2) {
+ log_type = TYP_LOG;
+ if (mp->m_logdev_targp != mp->m_ddev_targp)
+ log_type = TYP_ELOG;
+
push_cur();
- set_cur(&typtab[TYP_LOG],
+ set_cur(&typtab[log_type],
XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart),
mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL);
if (iocur_top->data) { /* best effort */
@@ -3185,9 +3190,10 @@ metadump_f(
if (!exitcode)
exitcode = !copy_sb_inodes();
- /* copy log if it's internal */
- if ((mp->m_sb.sb_logstart != 0) && !exitcode)
- exitcode = !copy_log();
+ /* copy log */
+ if (!exitcode && (mp->m_logdev_targp == mp->m_ddev_targp ||
+ metadump.version == 2))
+ exitcode = !copy_log(log_type);
/* write the remaining index */
if (!exitcode)
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 06/24] metadump: Dump external log device contents
2023-05-23 9:00 ` [PATCH 06/24] metadump: Dump external log device contents Chandan Babu R
@ 2023-05-23 17:02 ` Darrick J. Wong
2023-05-26 6:54 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:02 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:32PM +0530, Chandan Babu R wrote:
> metadump will now read and dump from external log device when the log is
> placed on an external device and metadump v2 is supported by xfsprogs.
>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> db/metadump.c | 20 +++++++++++++-------
> 1 file changed, 13 insertions(+), 7 deletions(-)
>
> diff --git a/db/metadump.c b/db/metadump.c
> index e7a433c21..62a36427d 100644
> --- a/db/metadump.c
> +++ b/db/metadump.c
> @@ -2921,7 +2921,7 @@ copy_sb_inodes(void)
> }
>
> static int
> -copy_log(void)
> +copy_log(enum typnm log_type)
> {
> struct xlog log;
> int dirty;
> @@ -2934,7 +2934,7 @@ copy_log(void)
> print_progress("Copying log");
>
> push_cur();
> - set_cur(&typtab[TYP_LOG], XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart),
> + set_cur(&typtab[log_type], XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart),
> mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL);
> if (iocur_top->data == NULL) {
> pop_cur();
> @@ -3038,6 +3038,7 @@ metadump_f(
> char **argv)
> {
> xfs_agnumber_t agno;
> + enum typnm log_type;
> int c;
> int start_iocur_sp;
> int outfd = -1;
> @@ -3110,9 +3111,13 @@ metadump_f(
> }
>
> /* If we'll copy the log, see if the log is dirty */
> - if (mp->m_sb.sb_logstart) {
> + if (mp->m_logdev_targp == mp->m_ddev_targp || metadump.version == 2) {
> + log_type = TYP_LOG;
> + if (mp->m_logdev_targp != mp->m_ddev_targp)
> + log_type = TYP_ELOG;
> +
> push_cur();
> - set_cur(&typtab[TYP_LOG],
> + set_cur(&typtab[log_type],
> XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart),
> mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL);
> if (iocur_top->data) { /* best effort */
> @@ -3185,9 +3190,10 @@ metadump_f(
> if (!exitcode)
> exitcode = !copy_sb_inodes();
>
> - /* copy log if it's internal */
> - if ((mp->m_sb.sb_logstart != 0) && !exitcode)
> - exitcode = !copy_log();
> + /* copy log */
> + if (!exitcode && (mp->m_logdev_targp == mp->m_ddev_targp ||
> + metadump.version == 2))
Version 2? I don't think that's been introduced yet. ;)
--D
> + exitcode = !copy_log(log_type);
>
> /* write the remaining index */
> if (!exitcode)
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 06/24] metadump: Dump external log device contents
2023-05-23 17:02 ` Darrick J. Wong
@ 2023-05-26 6:54 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-26 6:54 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 10:02:12 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:32PM +0530, Chandan Babu R wrote:
>> metadump will now read and dump from external log device when the log is
>> placed on an external device and metadump v2 is supported by xfsprogs.
>>
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> db/metadump.c | 20 +++++++++++++-------
>> 1 file changed, 13 insertions(+), 7 deletions(-)
>>
>> diff --git a/db/metadump.c b/db/metadump.c
>> index e7a433c21..62a36427d 100644
>> --- a/db/metadump.c
>> +++ b/db/metadump.c
>> @@ -2921,7 +2921,7 @@ copy_sb_inodes(void)
>> }
>>
>> static int
>> -copy_log(void)
>> +copy_log(enum typnm log_type)
>> {
>> struct xlog log;
>> int dirty;
>> @@ -2934,7 +2934,7 @@ copy_log(void)
>> print_progress("Copying log");
>>
>> push_cur();
>> - set_cur(&typtab[TYP_LOG], XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart),
>> + set_cur(&typtab[log_type], XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart),
>> mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL);
>> if (iocur_top->data == NULL) {
>> pop_cur();
>> @@ -3038,6 +3038,7 @@ metadump_f(
>> char **argv)
>> {
>> xfs_agnumber_t agno;
>> + enum typnm log_type;
>> int c;
>> int start_iocur_sp;
>> int outfd = -1;
>> @@ -3110,9 +3111,13 @@ metadump_f(
>> }
>>
>> /* If we'll copy the log, see if the log is dirty */
>> - if (mp->m_sb.sb_logstart) {
>> + if (mp->m_logdev_targp == mp->m_ddev_targp || metadump.version == 2) {
>> + log_type = TYP_LOG;
>> + if (mp->m_logdev_targp != mp->m_ddev_targp)
>> + log_type = TYP_ELOG;
>> +
>> push_cur();
>> - set_cur(&typtab[TYP_LOG],
>> + set_cur(&typtab[log_type],
>> XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart),
>> mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL);
>> if (iocur_top->data) { /* best effort */
>> @@ -3185,9 +3190,10 @@ metadump_f(
>> if (!exitcode)
>> exitcode = !copy_sb_inodes();
>>
>> - /* copy log if it's internal */
>> - if ((mp->m_sb.sb_logstart != 0) && !exitcode)
>> - exitcode = !copy_log();
>> + /* copy log */
>> + if (!exitcode && (mp->m_logdev_targp == mp->m_ddev_targp ||
>> + metadump.version == 2))
>
> Version 2? I don't think that's been introduced yet. ;)
>
I will move the aboves changes to the patch which adds the "-v" option.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 07/24] metadump: Postpone invocation of init_metadump()
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (5 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 06/24] metadump: Dump external log device contents Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:13 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 08/24] metadump: Introduce struct metadump_ops Chandan Babu R
` (16 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
A future commit will require that the metadump file be opened before execution
of init_metadump().
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
db/metadump.c | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/db/metadump.c b/db/metadump.c
index 62a36427d..212b484a2 100644
--- a/db/metadump.c
+++ b/db/metadump.c
@@ -3129,10 +3129,6 @@ metadump_f(
pop_cur();
}
- ret = init_metadump();
- if (ret)
- return 0;
-
start_iocur_sp = iocur_sp;
if (strcmp(argv[optind], "-") == 0) {
@@ -3177,6 +3173,10 @@ metadump_f(
}
}
+ ret = init_metadump();
+ if (ret)
+ return 0;
+
exitcode = 0;
for (agno = 0; agno < mp->m_sb.sb_agcount; agno++) {
@@ -3215,8 +3215,9 @@ metadump_f(
/* cleanup iocur stack */
while (iocur_sp > start_iocur_sp)
pop_cur();
-out:
+
release_metadump();
+out:
return 0;
}
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 07/24] metadump: Postpone invocation of init_metadump()
2023-05-23 9:00 ` [PATCH 07/24] metadump: Postpone invocation of init_metadump() Chandan Babu R
@ 2023-05-23 17:13 ` Darrick J. Wong
2023-05-25 8:45 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:13 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:33PM +0530, Chandan Babu R wrote:
> A future commit will require that the metadump file be opened before execution
> of init_metadump().
Why is that?
--D
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> db/metadump.c | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/db/metadump.c b/db/metadump.c
> index 62a36427d..212b484a2 100644
> --- a/db/metadump.c
> +++ b/db/metadump.c
> @@ -3129,10 +3129,6 @@ metadump_f(
> pop_cur();
> }
>
> - ret = init_metadump();
> - if (ret)
> - return 0;
> -
> start_iocur_sp = iocur_sp;
>
> if (strcmp(argv[optind], "-") == 0) {
> @@ -3177,6 +3173,10 @@ metadump_f(
> }
> }
>
> + ret = init_metadump();
> + if (ret)
> + return 0;
> +
> exitcode = 0;
>
> for (agno = 0; agno < mp->m_sb.sb_agcount; agno++) {
> @@ -3215,8 +3215,9 @@ metadump_f(
> /* cleanup iocur stack */
> while (iocur_sp > start_iocur_sp)
> pop_cur();
> -out:
> +
> release_metadump();
>
> +out:
> return 0;
> }
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 07/24] metadump: Postpone invocation of init_metadump()
2023-05-23 17:13 ` Darrick J. Wong
@ 2023-05-25 8:45 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 8:45 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 10:13:42 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:33PM +0530, Chandan Babu R wrote:
>> A future commit will require that the metadump file be opened before execution
>> of init_metadump().
>
> Why is that?
>
The metadump v2 initialization function (i.e. init_metadump_v2()) writes the
header structure (i.e. struct xfs_metadump_header) into the metadump
file. This will require the program to open the metadump file before
initialization function has been invoked.
I will add the above to the commit description.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 08/24] metadump: Introduce struct metadump_ops
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (6 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 07/24] metadump: Postpone invocation of init_metadump() Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:15 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 09/24] metadump: Introduce metadump v1 operations Chandan Babu R
` (15 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
We will need two sets of functions to implement two versions of metadump. This
commit adds the definition for 'struct metadump_ops' to hold pointers to
version specific metadump functions.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
db/metadump.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/db/metadump.c b/db/metadump.c
index 212b484a2..56d8c3bdf 100644
--- a/db/metadump.c
+++ b/db/metadump.c
@@ -40,6 +40,14 @@ static const cmdinfo_t metadump_cmd =
N_("[-a] [-e] [-g] [-m max_extent] [-w] [-o] filename"),
N_("dump metadata to a file"), metadump_help };
+struct metadump_ops {
+ int (*init_metadump)(void);
+ int (*write_metadump)(enum typnm type, char *data, int64_t off,
+ int len);
+ int (*end_write_metadump)(void);
+ void (*release_metadump)(void);
+};
+
static struct metadump {
int version;
int show_progress;
@@ -54,6 +62,7 @@ static struct metadump {
xfs_ino_t cur_ino;
/* Metadump file */
FILE *outf;
+ struct metadump_ops *mdops;
/* header + index + buffers */
struct xfs_metablock *metablock;
__be64 *block_index;
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 08/24] metadump: Introduce struct metadump_ops
2023-05-23 9:00 ` [PATCH 08/24] metadump: Introduce struct metadump_ops Chandan Babu R
@ 2023-05-23 17:15 ` Darrick J. Wong
2023-05-25 8:48 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:15 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:34PM +0530, Chandan Babu R wrote:
> We will need two sets of functions to implement two versions of metadump. This
> commit adds the definition for 'struct metadump_ops' to hold pointers to
> version specific metadump functions.
>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> db/metadump.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/db/metadump.c b/db/metadump.c
> index 212b484a2..56d8c3bdf 100644
> --- a/db/metadump.c
> +++ b/db/metadump.c
> @@ -40,6 +40,14 @@ static const cmdinfo_t metadump_cmd =
> N_("[-a] [-e] [-g] [-m max_extent] [-w] [-o] filename"),
> N_("dump metadata to a file"), metadump_help };
>
> +struct metadump_ops {
> + int (*init_metadump)(void);
> + int (*write_metadump)(enum typnm type, char *data, int64_t off,
> + int len);
> + int (*end_write_metadump)(void);
> + void (*release_metadump)(void);
Needs comments describing what each of these do. Does each
->write_metadump have to have a ->end_write_metadump?
You could probably remove the _metadump suffix too.
--D
> +};
> +
> static struct metadump {
> int version;
> int show_progress;
> @@ -54,6 +62,7 @@ static struct metadump {
> xfs_ino_t cur_ino;
> /* Metadump file */
> FILE *outf;
> + struct metadump_ops *mdops;
> /* header + index + buffers */
> struct xfs_metablock *metablock;
> __be64 *block_index;
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 08/24] metadump: Introduce struct metadump_ops
2023-05-23 17:15 ` Darrick J. Wong
@ 2023-05-25 8:48 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 8:48 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 10:15:25 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:34PM +0530, Chandan Babu R wrote:
>> We will need two sets of functions to implement two versions of metadump. This
>> commit adds the definition for 'struct metadump_ops' to hold pointers to
>> version specific metadump functions.
>>
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> db/metadump.c | 9 +++++++++
>> 1 file changed, 9 insertions(+)
>>
>> diff --git a/db/metadump.c b/db/metadump.c
>> index 212b484a2..56d8c3bdf 100644
>> --- a/db/metadump.c
>> +++ b/db/metadump.c
>> @@ -40,6 +40,14 @@ static const cmdinfo_t metadump_cmd =
>> N_("[-a] [-e] [-g] [-m max_extent] [-w] [-o] filename"),
>> N_("dump metadata to a file"), metadump_help };
>>
>> +struct metadump_ops {
>> + int (*init_metadump)(void);
>> + int (*write_metadump)(enum typnm type, char *data, int64_t off,
>> + int len);
>> + int (*end_write_metadump)(void);
>> + void (*release_metadump)(void);
>
> Needs comments describing what each of these do.
Sure. I will add the comments to describe each function pointer.
> Does each ->write_metadump have to have a ->end_write_metadump?
No. The v1 format code collects metadata blocks in memory and writes them to
the disk after sufficient number of blocks have been collected. We might not
have reached this limit for the final set of 512 byte metadata blocks. Hence,
a write operation has to be explicitly issued for such metadata blocks.
>
> You could probably remove the _metadump suffix too.
Yes, I will remove them.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 09/24] metadump: Introduce metadump v1 operations
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (7 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 08/24] metadump: Introduce struct metadump_ops Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:25 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 10/24] metadump: Rename XFS_MD_MAGIC to XFS_MD_MAGIC_V1 Chandan Babu R
` (14 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
This commit moves functionality associated with writing metadump to disk into
a new function. It also renames metadump initialization, write and release
functions to reflect the fact that they work with v1 metadump files.
The metadump initialization, write and release functions are now invoked via
metadump_ops->init_metadump(), metadump_ops->write_metadump() and
metadump_ops->release_metadump() respectively.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
db/metadump.c | 124 +++++++++++++++++++++++++-------------------------
1 file changed, 61 insertions(+), 63 deletions(-)
diff --git a/db/metadump.c b/db/metadump.c
index 56d8c3bdf..7265f73ec 100644
--- a/db/metadump.c
+++ b/db/metadump.c
@@ -135,59 +135,6 @@ print_progress(const char *fmt, ...)
metadump.progress_since_warning = 1;
}
-/*
- * A complete dump file will have a "zero" entry in the last index block,
- * even if the dump is exactly aligned, the last index will be full of
- * zeros. If the last index entry is non-zero, the dump is incomplete.
- * Correspondingly, the last chunk will have a count < num_indices.
- *
- * Return 0 for success, -1 for failure.
- */
-
-static int
-write_index(void)
-{
- struct xfs_metablock *metablock = metadump.metablock;
- /*
- * write index block and following data blocks (streaming)
- */
- metablock->mb_count = cpu_to_be16(metadump.cur_index);
- if (fwrite(metablock, (metadump.cur_index + 1) << BBSHIFT, 1,
- metadump.outf) != 1) {
- print_warning("error writing to target file");
- return -1;
- }
-
- memset(metadump.block_index, 0, metadump.num_indices * sizeof(__be64));
- metadump.cur_index = 0;
- return 0;
-}
-
-/*
- * Return 0 for success, -errno for failure.
- */
-static int
-write_buf_segment(
- char *data,
- int64_t off,
- int len)
-{
- int i;
- int ret;
-
- for (i = 0; i < len; i++, off++, data += BBSIZE) {
- metadump.block_index[metadump.cur_index] = cpu_to_be64(off);
- memcpy(&metadump.block_buffer[metadump.cur_index << BBSHIFT],
- data, BBSIZE);
- if (++metadump.cur_index == metadump.num_indices) {
- ret = write_index();
- if (ret)
- return -EIO;
- }
- }
- return 0;
-}
-
/*
* we want to preserve the state of the metadata in the dump - whether it is
* intact or corrupt, so even if the buffer has a verifier attached to it we
@@ -224,15 +171,16 @@ write_buf(
/* handle discontiguous buffers */
if (!buf->bbmap) {
- ret = write_buf_segment(buf->data, buf->bb, buf->blen);
+ ret = metadump.mdops->write_metadump(buf->typ->typnm, buf->data,
+ buf->bb, buf->blen);
if (ret)
return ret;
} else {
int len = 0;
for (i = 0; i < buf->bbmap->nmaps; i++) {
- ret = write_buf_segment(buf->data + BBTOB(len),
- buf->bbmap->b[i].bm_bn,
- buf->bbmap->b[i].bm_len);
+ ret = metadump.mdops->write_metadump(buf->typ->typnm,
+ buf->data + BBTOB(len), buf->bbmap->b[i].bm_bn,
+ buf->bbmap->b[i].bm_len);
if (ret)
return ret;
len += buf->bbmap->b[i].bm_len;
@@ -2994,7 +2942,7 @@ done:
}
static int
-init_metadump(void)
+init_metadump_v1(void)
{
metadump.metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
if (metadump.metablock == NULL) {
@@ -3035,12 +2983,60 @@ init_metadump(void)
return 0;
}
+static int
+end_write_metadump_v1(void)
+{
+ /*
+ * write index block and following data blocks (streaming)
+ */
+ metadump.metablock->mb_count = cpu_to_be16(metadump.cur_index);
+ if (fwrite(metadump.metablock, (metadump.cur_index + 1) << BBSHIFT, 1, metadump.outf) != 1) {
+ print_warning("error writing to target file");
+ return -1;
+ }
+
+ memset(metadump.block_index, 0, metadump.num_indices * sizeof(__be64));
+ metadump.cur_index = 0;
+ return 0;
+}
+
+static int
+write_metadump_v1(
+ enum typnm type,
+ char *data,
+ int64_t off,
+ int len)
+{
+ int i;
+ int ret;
+
+ for (i = 0; i < len; i++, off++, data += BBSIZE) {
+ metadump.block_index[metadump.cur_index] = cpu_to_be64(off);
+ memcpy(&metadump.block_buffer[metadump.cur_index << BBSHIFT],
+ data, BBSIZE);
+ if (++metadump.cur_index == metadump.num_indices) {
+ ret = end_write_metadump_v1();
+ if (ret)
+ return -EIO;
+ }
+ }
+
+ return 0;
+}
+
static void
-release_metadump(void)
+release_metadump_v1(void)
{
free(metadump.metablock);
}
+static struct metadump_ops metadump1_ops = {
+ .init_metadump = init_metadump_v1,
+ .write_metadump = write_metadump_v1,
+ .end_write_metadump = end_write_metadump_v1,
+ .release_metadump = release_metadump_v1,
+};
+
static int
metadump_f(
int argc,
@@ -3182,9 +3178,11 @@ metadump_f(
}
}
- ret = init_metadump();
+ metadump.mdops = &metadump1_ops;
+
+ ret = metadump.mdops->init_metadump();
if (ret)
- return 0;
+ goto out;
exitcode = 0;
@@ -3206,7 +3204,7 @@ metadump_f(
/* write the remaining index */
if (!exitcode)
- exitcode = write_index() < 0;
+ exitcode = metadump.mdops->end_write_metadump() < 0;
if (metadump.progress_since_warning)
fputc('\n', metadump.stdout_metadump ? stderr : stdout);
@@ -3225,7 +3223,7 @@ metadump_f(
while (iocur_sp > start_iocur_sp)
pop_cur();
- release_metadump();
+ metadump.mdops->release_metadump();
out:
return 0;
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 09/24] metadump: Introduce metadump v1 operations
2023-05-23 9:00 ` [PATCH 09/24] metadump: Introduce metadump v1 operations Chandan Babu R
@ 2023-05-23 17:25 ` Darrick J. Wong
2023-05-25 14:19 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:25 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:35PM +0530, Chandan Babu R wrote:
> This commit moves functionality associated with writing metadump to disk into
> a new function. It also renames metadump initialization, write and release
> functions to reflect the fact that they work with v1 metadump files.
>
> The metadump initialization, write and release functions are now invoked via
> metadump_ops->init_metadump(), metadump_ops->write_metadump() and
> metadump_ops->release_metadump() respectively.
>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> db/metadump.c | 124 +++++++++++++++++++++++++-------------------------
> 1 file changed, 61 insertions(+), 63 deletions(-)
>
> diff --git a/db/metadump.c b/db/metadump.c
> index 56d8c3bdf..7265f73ec 100644
> --- a/db/metadump.c
> +++ b/db/metadump.c
> @@ -135,59 +135,6 @@ print_progress(const char *fmt, ...)
> metadump.progress_since_warning = 1;
> }
>
> -/*
> - * A complete dump file will have a "zero" entry in the last index block,
> - * even if the dump is exactly aligned, the last index will be full of
> - * zeros. If the last index entry is non-zero, the dump is incomplete.
> - * Correspondingly, the last chunk will have a count < num_indices.
> - *
> - * Return 0 for success, -1 for failure.
> - */
> -
> -static int
> -write_index(void)
> -{
> - struct xfs_metablock *metablock = metadump.metablock;
> - /*
> - * write index block and following data blocks (streaming)
> - */
> - metablock->mb_count = cpu_to_be16(metadump.cur_index);
> - if (fwrite(metablock, (metadump.cur_index + 1) << BBSHIFT, 1,
> - metadump.outf) != 1) {
> - print_warning("error writing to target file");
> - return -1;
> - }
> -
> - memset(metadump.block_index, 0, metadump.num_indices * sizeof(__be64));
> - metadump.cur_index = 0;
> - return 0;
> -}
> -
> -/*
> - * Return 0 for success, -errno for failure.
> - */
> -static int
> -write_buf_segment(
> - char *data,
> - int64_t off,
> - int len)
> -{
> - int i;
> - int ret;
> -
> - for (i = 0; i < len; i++, off++, data += BBSIZE) {
> - metadump.block_index[metadump.cur_index] = cpu_to_be64(off);
> - memcpy(&metadump.block_buffer[metadump.cur_index << BBSHIFT],
> - data, BBSIZE);
> - if (++metadump.cur_index == metadump.num_indices) {
> - ret = write_index();
> - if (ret)
> - return -EIO;
> - }
> - }
> - return 0;
> -}
> -
> /*
> * we want to preserve the state of the metadata in the dump - whether it is
> * intact or corrupt, so even if the buffer has a verifier attached to it we
> @@ -224,15 +171,16 @@ write_buf(
>
> /* handle discontiguous buffers */
> if (!buf->bbmap) {
> - ret = write_buf_segment(buf->data, buf->bb, buf->blen);
> + ret = metadump.mdops->write_metadump(buf->typ->typnm, buf->data,
> + buf->bb, buf->blen);
> if (ret)
> return ret;
> } else {
> int len = 0;
> for (i = 0; i < buf->bbmap->nmaps; i++) {
> - ret = write_buf_segment(buf->data + BBTOB(len),
> - buf->bbmap->b[i].bm_bn,
> - buf->bbmap->b[i].bm_len);
> + ret = metadump.mdops->write_metadump(buf->typ->typnm,
> + buf->data + BBTOB(len), buf->bbmap->b[i].bm_bn,
> + buf->bbmap->b[i].bm_len);
> if (ret)
> return ret;
> len += buf->bbmap->b[i].bm_len;
> @@ -2994,7 +2942,7 @@ done:
> }
>
> static int
> -init_metadump(void)
> +init_metadump_v1(void)
> {
> metadump.metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
> if (metadump.metablock == NULL) {
> @@ -3035,12 +2983,60 @@ init_metadump(void)
> return 0;
> }
>
> +static int
> +end_write_metadump_v1(void)
> +{
> + /*
> + * write index block and following data blocks (streaming)
> + */
> + metadump.metablock->mb_count = cpu_to_be16(metadump.cur_index);
> + if (fwrite(metadump.metablock, (metadump.cur_index + 1) << BBSHIFT, 1, metadump.outf) != 1) {
> + print_warning("error writing to target file");
> + return -1;
> + }
> +
> + memset(metadump.block_index, 0, metadump.num_indices * sizeof(__be64));
> + metadump.cur_index = 0;
> + return 0;
> +}
> +
> +static int
> +write_metadump_v1(
> + enum typnm type,
> + char *data,
> + int64_t off,
This really ought to be an xfs_daddr_t, right?
> + int len)
> +{
> + int i;
> + int ret;
> +
> + for (i = 0; i < len; i++, off++, data += BBSIZE) {
> + metadump.block_index[metadump.cur_index] = cpu_to_be64(off);
> + memcpy(&metadump.block_buffer[metadump.cur_index << BBSHIFT],
> + data, BBSIZE);
Wondering if this ought to be called ->record_segment or something, since
it's not really writing anything to disk, merely copying it to the index
buffer.
> + if (++metadump.cur_index == metadump.num_indices) {
> + ret = end_write_metadump_v1();
> + if (ret)
> + return -EIO;
This is "generic" code for "Have we filled up the index table? If so,
then write the index block the indexed data". Shouldn't it go in
write_buf? And then write_buf does something like:
while (len > 0) {
segment_len = min(len, metadump.num_indices - metadump.cur_index);
metadump.ops->record_segment(type, buf, daddr, segment_len);
metadump.cur_index += segment_len;
if (metadump.cur_index == metadump.num_indices) {
metadump.ops->write_index(...);
metadump.cur_index = 0;
}
len -= segment_len;
daddr += segment_len;
buf += (segment_len << 9);
}
if (metadump.cur_index)
metadump.ops->write_index(...);
metadump.cur_index = 0;
--D
> + }
> + }
> +
> + return 0;
> +}
> +
> static void
> -release_metadump(void)
> +release_metadump_v1(void)
> {
> free(metadump.metablock);
> }
>
> +static struct metadump_ops metadump1_ops = {
> + .init_metadump = init_metadump_v1,
> + .write_metadump = write_metadump_v1,
> + .end_write_metadump = end_write_metadump_v1,
> + .release_metadump = release_metadump_v1,
> +};
> +
> static int
> metadump_f(
> int argc,
> @@ -3182,9 +3178,11 @@ metadump_f(
> }
> }
>
> - ret = init_metadump();
> + metadump.mdops = &metadump1_ops;
> +
> + ret = metadump.mdops->init_metadump();
> if (ret)
> - return 0;
> + goto out;
>
> exitcode = 0;
>
> @@ -3206,7 +3204,7 @@ metadump_f(
>
> /* write the remaining index */
> if (!exitcode)
> - exitcode = write_index() < 0;
> + exitcode = metadump.mdops->end_write_metadump() < 0;
>
> if (metadump.progress_since_warning)
> fputc('\n', metadump.stdout_metadump ? stderr : stdout);
> @@ -3225,7 +3223,7 @@ metadump_f(
> while (iocur_sp > start_iocur_sp)
> pop_cur();
>
> - release_metadump();
> + metadump.mdops->release_metadump();
>
> out:
> return 0;
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 09/24] metadump: Introduce metadump v1 operations
2023-05-23 17:25 ` Darrick J. Wong
@ 2023-05-25 14:19 ` Chandan Babu R
2023-06-02 14:34 ` Darrick J. Wong
0 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 14:19 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 10:25:13 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:35PM +0530, Chandan Babu R wrote:
>> This commit moves functionality associated with writing metadump to disk into
>> a new function. It also renames metadump initialization, write and release
>> functions to reflect the fact that they work with v1 metadump files.
>>
>> The metadump initialization, write and release functions are now invoked via
>> metadump_ops->init_metadump(), metadump_ops->write_metadump() and
>> metadump_ops->release_metadump() respectively.
>>
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> db/metadump.c | 124 +++++++++++++++++++++++++-------------------------
>> 1 file changed, 61 insertions(+), 63 deletions(-)
>>
>> diff --git a/db/metadump.c b/db/metadump.c
>> index 56d8c3bdf..7265f73ec 100644
>> --- a/db/metadump.c
>> +++ b/db/metadump.c
>> @@ -135,59 +135,6 @@ print_progress(const char *fmt, ...)
>> metadump.progress_since_warning = 1;
>> }
>>
>> -/*
>> - * A complete dump file will have a "zero" entry in the last index block,
>> - * even if the dump is exactly aligned, the last index will be full of
>> - * zeros. If the last index entry is non-zero, the dump is incomplete.
>> - * Correspondingly, the last chunk will have a count < num_indices.
>> - *
>> - * Return 0 for success, -1 for failure.
>> - */
>> -
>> -static int
>> -write_index(void)
>> -{
>> - struct xfs_metablock *metablock = metadump.metablock;
>> - /*
>> - * write index block and following data blocks (streaming)
>> - */
>> - metablock->mb_count = cpu_to_be16(metadump.cur_index);
>> - if (fwrite(metablock, (metadump.cur_index + 1) << BBSHIFT, 1,
>> - metadump.outf) != 1) {
>> - print_warning("error writing to target file");
>> - return -1;
>> - }
>> -
>> - memset(metadump.block_index, 0, metadump.num_indices * sizeof(__be64));
>> - metadump.cur_index = 0;
>> - return 0;
>> -}
>> -
>> -/*
>> - * Return 0 for success, -errno for failure.
>> - */
>> -static int
>> -write_buf_segment(
>> - char *data,
>> - int64_t off,
>> - int len)
>> -{
>> - int i;
>> - int ret;
>> -
>> - for (i = 0; i < len; i++, off++, data += BBSIZE) {
>> - metadump.block_index[metadump.cur_index] = cpu_to_be64(off);
>> - memcpy(&metadump.block_buffer[metadump.cur_index << BBSHIFT],
>> - data, BBSIZE);
>> - if (++metadump.cur_index == metadump.num_indices) {
>> - ret = write_index();
>> - if (ret)
>> - return -EIO;
>> - }
>> - }
>> - return 0;
>> -}
>> -
>> /*
>> * we want to preserve the state of the metadata in the dump - whether it is
>> * intact or corrupt, so even if the buffer has a verifier attached to it we
>> @@ -224,15 +171,16 @@ write_buf(
>>
>> /* handle discontiguous buffers */
>> if (!buf->bbmap) {
>> - ret = write_buf_segment(buf->data, buf->bb, buf->blen);
>> + ret = metadump.mdops->write_metadump(buf->typ->typnm, buf->data,
>> + buf->bb, buf->blen);
>> if (ret)
>> return ret;
>> } else {
>> int len = 0;
>> for (i = 0; i < buf->bbmap->nmaps; i++) {
>> - ret = write_buf_segment(buf->data + BBTOB(len),
>> - buf->bbmap->b[i].bm_bn,
>> - buf->bbmap->b[i].bm_len);
>> + ret = metadump.mdops->write_metadump(buf->typ->typnm,
>> + buf->data + BBTOB(len), buf->bbmap->b[i].bm_bn,
>> + buf->bbmap->b[i].bm_len);
>> if (ret)
>> return ret;
>> len += buf->bbmap->b[i].bm_len;
>> @@ -2994,7 +2942,7 @@ done:
>> }
>>
>> static int
>> -init_metadump(void)
>> +init_metadump_v1(void)
>> {
>> metadump.metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
>> if (metadump.metablock == NULL) {
>> @@ -3035,12 +2983,60 @@ init_metadump(void)
>> return 0;
>> }
>>
>> +static int
>> +end_write_metadump_v1(void)
>> +{
>> + /*
>> + * write index block and following data blocks (streaming)
>> + */
>> + metadump.metablock->mb_count = cpu_to_be16(metadump.cur_index);
>> + if (fwrite(metadump.metablock, (metadump.cur_index + 1) << BBSHIFT, 1, metadump.outf) != 1) {
>> + print_warning("error writing to target file");
>> + return -1;
>> + }
>> +
>> + memset(metadump.block_index, 0, metadump.num_indices * sizeof(__be64));
>> + metadump.cur_index = 0;
>> + return 0;
>> +}
>> +
>> +static int
>> +write_metadump_v1(
>> + enum typnm type,
>> + char *data,
>> + int64_t off,
>
> This really ought to be an xfs_daddr_t, right?
>
Yes, you are right. I will make the required change.
>> + int len)
>> +{
>> + int i;
>> + int ret;
>> +
>> + for (i = 0; i < len; i++, off++, data += BBSIZE) {
>> + metadump.block_index[metadump.cur_index] = cpu_to_be64(off);
>> + memcpy(&metadump.block_buffer[metadump.cur_index << BBSHIFT],
>> + data, BBSIZE);
>
> Wondering if this ought to be called ->record_segment or something, since
> it's not really writing anything to disk, merely copying it to the index
> buffer.
>
>> + if (++metadump.cur_index == metadump.num_indices) {
>> + ret = end_write_metadump_v1();
>> + if (ret)
>> + return -EIO;
>
> This is "generic" code for "Have we filled up the index table? If so,
> then write the index block the indexed data". Shouldn't it go in
> write_buf? And then write_buf does something like:
>
> while (len > 0) {
> segment_len = min(len, metadump.num_indices - metadump.cur_index);
>
> metadump.ops->record_segment(type, buf, daddr, segment_len);
>
> metadump.cur_index += segment_len;
> if (metadump.cur_index == metadump.num_indices) {
> metadump.ops->write_index(...);
> metadump.cur_index = 0;
> }
>
> len -= segment_len;
> daddr += segment_len;
> buf += (segment_len << 9);
> }
>
> if (metadump.cur_index)
> metadump.ops->write_index(...);
> metadump.cur_index = 0;
>
The above change would require write_buf() to know about the internals of
metadump v1 format. This change can be performed as long as the metadump
supports only the v1 format. However, supporting the v2 format later requires
that write_buf() invoke version specific functions via function pointers and
to not perform any other version specific operation (e.g. checking to see if
the v1 format's in-memory index is filled) on its own.
>> + }
>> + }
>> +
>> + return 0;
>> +}
>> +
>> static void
>> -release_metadump(void)
>> +release_metadump_v1(void)
>> {
>> free(metadump.metablock);
>> }
>>
>> +static struct metadump_ops metadump1_ops = {
>> + .init_metadump = init_metadump_v1,
>> + .write_metadump = write_metadump_v1,
>> + .end_write_metadump = end_write_metadump_v1,
>> + .release_metadump = release_metadump_v1,
>> +};
>> +
>> static int
>> metadump_f(
>> int argc,
>> @@ -3182,9 +3178,11 @@ metadump_f(
>> }
>> }
>>
>> - ret = init_metadump();
>> + metadump.mdops = &metadump1_ops;
>> +
>> + ret = metadump.mdops->init_metadump();
>> if (ret)
>> - return 0;
>> + goto out;
>>
>> exitcode = 0;
>>
>> @@ -3206,7 +3204,7 @@ metadump_f(
>>
>> /* write the remaining index */
>> if (!exitcode)
>> - exitcode = write_index() < 0;
>> + exitcode = metadump.mdops->end_write_metadump() < 0;
>>
>> if (metadump.progress_since_warning)
>> fputc('\n', metadump.stdout_metadump ? stderr : stdout);
>> @@ -3225,7 +3223,7 @@ metadump_f(
>> while (iocur_sp > start_iocur_sp)
>> pop_cur();
>>
>> - release_metadump();
>> + metadump.mdops->release_metadump();
>>
>> out:
>> return 0;
>> --
>> 2.39.1
>>
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 09/24] metadump: Introduce metadump v1 operations
2023-05-25 14:19 ` Chandan Babu R
@ 2023-06-02 14:34 ` Darrick J. Wong
0 siblings, 0 replies; 76+ messages in thread
From: Darrick J. Wong @ 2023-06-02 14:34 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Thu, May 25, 2023 at 07:49:32PM +0530, Chandan Babu R wrote:
> On Tue, May 23, 2023 at 10:25:13 AM -0700, Darrick J. Wong wrote:
> > On Tue, May 23, 2023 at 02:30:35PM +0530, Chandan Babu R wrote:
> >> This commit moves functionality associated with writing metadump to disk into
> >> a new function. It also renames metadump initialization, write and release
> >> functions to reflect the fact that they work with v1 metadump files.
> >>
> >> The metadump initialization, write and release functions are now invoked via
> >> metadump_ops->init_metadump(), metadump_ops->write_metadump() and
> >> metadump_ops->release_metadump() respectively.
> >>
> >> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> >> ---
> >> db/metadump.c | 124 +++++++++++++++++++++++++-------------------------
> >> 1 file changed, 61 insertions(+), 63 deletions(-)
> >>
> >> diff --git a/db/metadump.c b/db/metadump.c
> >> index 56d8c3bdf..7265f73ec 100644
> >> --- a/db/metadump.c
> >> +++ b/db/metadump.c
> >> @@ -135,59 +135,6 @@ print_progress(const char *fmt, ...)
> >> metadump.progress_since_warning = 1;
> >> }
> >>
> >> -/*
> >> - * A complete dump file will have a "zero" entry in the last index block,
> >> - * even if the dump is exactly aligned, the last index will be full of
> >> - * zeros. If the last index entry is non-zero, the dump is incomplete.
> >> - * Correspondingly, the last chunk will have a count < num_indices.
> >> - *
> >> - * Return 0 for success, -1 for failure.
> >> - */
> >> -
> >> -static int
> >> -write_index(void)
> >> -{
> >> - struct xfs_metablock *metablock = metadump.metablock;
> >> - /*
> >> - * write index block and following data blocks (streaming)
> >> - */
> >> - metablock->mb_count = cpu_to_be16(metadump.cur_index);
> >> - if (fwrite(metablock, (metadump.cur_index + 1) << BBSHIFT, 1,
> >> - metadump.outf) != 1) {
> >> - print_warning("error writing to target file");
> >> - return -1;
> >> - }
> >> -
> >> - memset(metadump.block_index, 0, metadump.num_indices * sizeof(__be64));
> >> - metadump.cur_index = 0;
> >> - return 0;
> >> -}
> >> -
> >> -/*
> >> - * Return 0 for success, -errno for failure.
> >> - */
> >> -static int
> >> -write_buf_segment(
> >> - char *data,
> >> - int64_t off,
> >> - int len)
> >> -{
> >> - int i;
> >> - int ret;
> >> -
> >> - for (i = 0; i < len; i++, off++, data += BBSIZE) {
> >> - metadump.block_index[metadump.cur_index] = cpu_to_be64(off);
> >> - memcpy(&metadump.block_buffer[metadump.cur_index << BBSHIFT],
> >> - data, BBSIZE);
> >> - if (++metadump.cur_index == metadump.num_indices) {
> >> - ret = write_index();
> >> - if (ret)
> >> - return -EIO;
> >> - }
> >> - }
> >> - return 0;
> >> -}
> >> -
> >> /*
> >> * we want to preserve the state of the metadata in the dump - whether it is
> >> * intact or corrupt, so even if the buffer has a verifier attached to it we
> >> @@ -224,15 +171,16 @@ write_buf(
> >>
> >> /* handle discontiguous buffers */
> >> if (!buf->bbmap) {
> >> - ret = write_buf_segment(buf->data, buf->bb, buf->blen);
> >> + ret = metadump.mdops->write_metadump(buf->typ->typnm, buf->data,
> >> + buf->bb, buf->blen);
> >> if (ret)
> >> return ret;
> >> } else {
> >> int len = 0;
> >> for (i = 0; i < buf->bbmap->nmaps; i++) {
> >> - ret = write_buf_segment(buf->data + BBTOB(len),
> >> - buf->bbmap->b[i].bm_bn,
> >> - buf->bbmap->b[i].bm_len);
> >> + ret = metadump.mdops->write_metadump(buf->typ->typnm,
> >> + buf->data + BBTOB(len), buf->bbmap->b[i].bm_bn,
> >> + buf->bbmap->b[i].bm_len);
> >> if (ret)
> >> return ret;
> >> len += buf->bbmap->b[i].bm_len;
> >> @@ -2994,7 +2942,7 @@ done:
> >> }
> >>
> >> static int
> >> -init_metadump(void)
> >> +init_metadump_v1(void)
> >> {
> >> metadump.metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
> >> if (metadump.metablock == NULL) {
> >> @@ -3035,12 +2983,60 @@ init_metadump(void)
> >> return 0;
> >> }
> >>
> >> +static int
> >> +end_write_metadump_v1(void)
> >> +{
> >> + /*
> >> + * write index block and following data blocks (streaming)
> >> + */
> >> + metadump.metablock->mb_count = cpu_to_be16(metadump.cur_index);
> >> + if (fwrite(metadump.metablock, (metadump.cur_index + 1) << BBSHIFT, 1, metadump.outf) != 1) {
> >> + print_warning("error writing to target file");
> >> + return -1;
> >> + }
> >> +
> >> + memset(metadump.block_index, 0, metadump.num_indices * sizeof(__be64));
> >> + metadump.cur_index = 0;
> >> + return 0;
> >> +}
> >> +
> >> +static int
> >> +write_metadump_v1(
> >> + enum typnm type,
> >> + char *data,
> >> + int64_t off,
> >
> > This really ought to be an xfs_daddr_t, right?
> >
>
> Yes, you are right. I will make the required change.
>
> >> + int len)
> >> +{
> >> + int i;
> >> + int ret;
> >> +
> >> + for (i = 0; i < len; i++, off++, data += BBSIZE) {
> >> + metadump.block_index[metadump.cur_index] = cpu_to_be64(off);
> >> + memcpy(&metadump.block_buffer[metadump.cur_index << BBSHIFT],
> >> + data, BBSIZE);
> >
> > Wondering if this ought to be called ->record_segment or something, since
> > it's not really writing anything to disk, merely copying it to the index
> > buffer.
> >
> >> + if (++metadump.cur_index == metadump.num_indices) {
> >> + ret = end_write_metadump_v1();
> >> + if (ret)
> >> + return -EIO;
> >
> > This is "generic" code for "Have we filled up the index table? If so,
> > then write the index block the indexed data". Shouldn't it go in
> > write_buf? And then write_buf does something like:
> >
> > while (len > 0) {
> > segment_len = min(len, metadump.num_indices - metadump.cur_index);
> >
> > metadump.ops->record_segment(type, buf, daddr, segment_len);
> >
> > metadump.cur_index += segment_len;
> > if (metadump.cur_index == metadump.num_indices) {
> > metadump.ops->write_index(...);
> > metadump.cur_index = 0;
> > }
> >
> > len -= segment_len;
> > daddr += segment_len;
> > buf += (segment_len << 9);
> > }
> >
> > if (metadump.cur_index)
> > metadump.ops->write_index(...);
> > metadump.cur_index = 0;
> >
>
> The above change would require write_buf() to know about the internals of
> metadump v1 format. This change can be performed as long as the metadump
> supports only the v1 format. However, supporting the v2 format later requires
> that write_buf() invoke version specific functions via function pointers and
> to not perform any other version specific operation (e.g. checking to see if
> the v1 format's in-memory index is filled) on its own.
Ah, right. I made that comment before you pointed out that v2
interleaves meta_extent records with dump data instead of having
separate index blocks. Question withdrawn.
--D
> >> + }
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +
> >> static void
> >> -release_metadump(void)
> >> +release_metadump_v1(void)
> >> {
> >> free(metadump.metablock);
> >> }
> >>
> >> +static struct metadump_ops metadump1_ops = {
> >> + .init_metadump = init_metadump_v1,
> >> + .write_metadump = write_metadump_v1,
> >> + .end_write_metadump = end_write_metadump_v1,
> >> + .release_metadump = release_metadump_v1,
> >> +};
> >> +
> >> static int
> >> metadump_f(
> >> int argc,
> >> @@ -3182,9 +3178,11 @@ metadump_f(
> >> }
> >> }
> >>
> >> - ret = init_metadump();
> >> + metadump.mdops = &metadump1_ops;
> >> +
> >> + ret = metadump.mdops->init_metadump();
> >> if (ret)
> >> - return 0;
> >> + goto out;
> >>
> >> exitcode = 0;
> >>
> >> @@ -3206,7 +3204,7 @@ metadump_f(
> >>
> >> /* write the remaining index */
> >> if (!exitcode)
> >> - exitcode = write_index() < 0;
> >> + exitcode = metadump.mdops->end_write_metadump() < 0;
> >>
> >> if (metadump.progress_since_warning)
> >> fputc('\n', metadump.stdout_metadump ? stderr : stdout);
> >> @@ -3225,7 +3223,7 @@ metadump_f(
> >> while (iocur_sp > start_iocur_sp)
> >> pop_cur();
> >>
> >> - release_metadump();
> >> + metadump.mdops->release_metadump();
> >>
> >> out:
> >> return 0;
> >> --
> >> 2.39.1
> >>
>
>
> --
> chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 10/24] metadump: Rename XFS_MD_MAGIC to XFS_MD_MAGIC_V1
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (8 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 09/24] metadump: Introduce metadump v1 operations Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:27 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 11/24] metadump: Define metadump v2 ondisk format structures and macros Chandan Babu R
` (13 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
db/metadump.c | 2 +-
include/xfs_metadump.h | 2 +-
mdrestore/xfs_mdrestore.c | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/db/metadump.c b/db/metadump.c
index 7265f73ec..9d7ad76ae 100644
--- a/db/metadump.c
+++ b/db/metadump.c
@@ -2950,7 +2950,7 @@ init_metadump_v1(void)
return -1;
}
metadump.metablock->mb_blocklog = BBSHIFT;
- metadump.metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC);
+ metadump.metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC_V1);
/* Set flags about state of metadump */
metadump.metablock->mb_info = XFS_METADUMP_INFO_FLAGS;
diff --git a/include/xfs_metadump.h b/include/xfs_metadump.h
index fbd990232..a4dca25cb 100644
--- a/include/xfs_metadump.h
+++ b/include/xfs_metadump.h
@@ -7,7 +7,7 @@
#ifndef _XFS_METADUMP_H_
#define _XFS_METADUMP_H_
-#define XFS_MD_MAGIC 0x5846534d /* 'XFSM' */
+#define XFS_MD_MAGIC_V1 0x5846534d /* 'XFSM' */
typedef struct xfs_metablock {
__be32 mb_magic;
diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
index 333282ed2..481dd00c2 100644
--- a/mdrestore/xfs_mdrestore.c
+++ b/mdrestore/xfs_mdrestore.c
@@ -240,7 +240,7 @@ main(
if (fread(&mb, sizeof(mb), 1, src_f) != 1)
fatal("error reading from metadump file\n");
- if (mb.mb_magic != cpu_to_be32(XFS_MD_MAGIC))
+ if (mb.mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
fatal("specified file is not a metadata dump\n");
if (show_info) {
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 10/24] metadump: Rename XFS_MD_MAGIC to XFS_MD_MAGIC_V1
2023-05-23 9:00 ` [PATCH 10/24] metadump: Rename XFS_MD_MAGIC to XFS_MD_MAGIC_V1 Chandan Babu R
@ 2023-05-23 17:27 ` Darrick J. Wong
0 siblings, 0 replies; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:27 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:36PM +0530, Chandan Babu R wrote:
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
Good thing we didn't publish XFS_MD_MAGIC in xfslibs-dev...
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
--D
> ---
> db/metadump.c | 2 +-
> include/xfs_metadump.h | 2 +-
> mdrestore/xfs_mdrestore.c | 2 +-
> 3 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/db/metadump.c b/db/metadump.c
> index 7265f73ec..9d7ad76ae 100644
> --- a/db/metadump.c
> +++ b/db/metadump.c
> @@ -2950,7 +2950,7 @@ init_metadump_v1(void)
> return -1;
> }
> metadump.metablock->mb_blocklog = BBSHIFT;
> - metadump.metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC);
> + metadump.metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC_V1);
>
> /* Set flags about state of metadump */
> metadump.metablock->mb_info = XFS_METADUMP_INFO_FLAGS;
> diff --git a/include/xfs_metadump.h b/include/xfs_metadump.h
> index fbd990232..a4dca25cb 100644
> --- a/include/xfs_metadump.h
> +++ b/include/xfs_metadump.h
> @@ -7,7 +7,7 @@
> #ifndef _XFS_METADUMP_H_
> #define _XFS_METADUMP_H_
>
> -#define XFS_MD_MAGIC 0x5846534d /* 'XFSM' */
> +#define XFS_MD_MAGIC_V1 0x5846534d /* 'XFSM' */
>
> typedef struct xfs_metablock {
> __be32 mb_magic;
> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> index 333282ed2..481dd00c2 100644
> --- a/mdrestore/xfs_mdrestore.c
> +++ b/mdrestore/xfs_mdrestore.c
> @@ -240,7 +240,7 @@ main(
>
> if (fread(&mb, sizeof(mb), 1, src_f) != 1)
> fatal("error reading from metadump file\n");
> - if (mb.mb_magic != cpu_to_be32(XFS_MD_MAGIC))
> + if (mb.mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
> fatal("specified file is not a metadata dump\n");
>
> if (show_info) {
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 11/24] metadump: Define metadump v2 ondisk format structures and macros
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (9 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 10/24] metadump: Rename XFS_MD_MAGIC to XFS_MD_MAGIC_V1 Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:34 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 12/24] metadump: Define metadump ops for v2 format Chandan Babu R
` (12 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
include/xfs_metadump.h | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
diff --git a/include/xfs_metadump.h b/include/xfs_metadump.h
index a4dca25cb..1d8d7008c 100644
--- a/include/xfs_metadump.h
+++ b/include/xfs_metadump.h
@@ -8,7 +8,9 @@
#define _XFS_METADUMP_H_
#define XFS_MD_MAGIC_V1 0x5846534d /* 'XFSM' */
+#define XFS_MD_MAGIC_V2 0x584D4432 /* 'XMD2' */
+/* Metadump v1 */
typedef struct xfs_metablock {
__be32 mb_magic;
__be16 mb_count;
@@ -23,4 +25,34 @@ typedef struct xfs_metablock {
#define XFS_METADUMP_FULLBLOCKS (1 << 2)
#define XFS_METADUMP_DIRTYLOG (1 << 3)
+/* Metadump v2 */
+struct xfs_metadump_header {
+ __be32 xmh_magic;
+ __be32 xmh_version;
+ __be32 xmh_compat_flags;
+ __be32 xmh_incompat_flags;
+ __be64 xmh_reserved;
+} __packed;
+
+#define XFS_MD2_INCOMPAT_OBFUSCATED (1 << 0)
+#define XFS_MD2_INCOMPAT_FULLBLOCKS (1 << 1)
+#define XFS_MD2_INCOMPAT_DIRTYLOG (1 << 2)
+
+struct xfs_meta_extent {
+ /*
+ * Lowest 54 bits are used to store 512 byte addresses.
+ * Next 2 bits is used for indicating the device.
+ * 00 - Data device
+ * 01 - External log
+ */
+ __be64 xme_addr;
+ /* In units of 512 byte blocks */
+ __be32 xme_len;
+} __packed;
+
+#define XME_ADDR_DATA_DEVICE (1UL << 54)
+#define XME_ADDR_LOG_DEVICE (1UL << 55)
+
+#define XME_ADDR_DEVICE_MASK (~(XME_ADDR_DATA_DEVICE | XME_ADDR_LOG_DEVICE))
+
#endif /* _XFS_METADUMP_H_ */
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 11/24] metadump: Define metadump v2 ondisk format structures and macros
2023-05-23 9:00 ` [PATCH 11/24] metadump: Define metadump v2 ondisk format structures and macros Chandan Babu R
@ 2023-05-23 17:34 ` Darrick J. Wong
2023-05-25 9:26 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:34 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:37PM +0530, Chandan Babu R wrote:
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> include/xfs_metadump.h | 32 ++++++++++++++++++++++++++++++++
> 1 file changed, 32 insertions(+)
>
> diff --git a/include/xfs_metadump.h b/include/xfs_metadump.h
> index a4dca25cb..1d8d7008c 100644
> --- a/include/xfs_metadump.h
> +++ b/include/xfs_metadump.h
> @@ -8,7 +8,9 @@
> #define _XFS_METADUMP_H_
>
> #define XFS_MD_MAGIC_V1 0x5846534d /* 'XFSM' */
> +#define XFS_MD_MAGIC_V2 0x584D4432 /* 'XMD2' */
>
> +/* Metadump v1 */
> typedef struct xfs_metablock {
> __be32 mb_magic;
> __be16 mb_count;
> @@ -23,4 +25,34 @@ typedef struct xfs_metablock {
> #define XFS_METADUMP_FULLBLOCKS (1 << 2)
> #define XFS_METADUMP_DIRTYLOG (1 << 3)
>
> +/* Metadump v2 */
> +struct xfs_metadump_header {
> + __be32 xmh_magic;
> + __be32 xmh_version;
> + __be32 xmh_compat_flags;
> + __be32 xmh_incompat_flags;
> + __be64 xmh_reserved;
__be32 xmh_crc; ?
Otherwise there's nothing to check for bitflips in the index blocks
themselves.
> +} __packed;
Does an array of xfs_meta_extent come immediately after
xfs_metadump_header, or do they go in a separate block after the header?
How big is the index block supposed to be?
> +
> +#define XFS_MD2_INCOMPAT_OBFUSCATED (1 << 0)
> +#define XFS_MD2_INCOMPAT_FULLBLOCKS (1 << 1)
> +#define XFS_MD2_INCOMPAT_DIRTYLOG (1 << 2)
Should the header declare when some of the xfs_meta_extents will have
XME_ADDR_LOG_DEVICE set?
> +
> +struct xfs_meta_extent {
> + /*
Tabs not spaces.
> + * Lowest 54 bits are used to store 512 byte addresses.
> + * Next 2 bits is used for indicating the device.
> + * 00 - Data device
> + * 01 - External log
So if you were to (say) add the realtime device, would that be bit 56,
or would you define 0xC0000000000000 (aka DATA|LOG) to mean realtime?
> + */
> + __be64 xme_addr;
> + /* In units of 512 byte blocks */
> + __be32 xme_len;
> +} __packed;
> +
> +#define XME_ADDR_DATA_DEVICE (1UL << 54)
> +#define XME_ADDR_LOG_DEVICE (1UL << 55)
1ULL, because "UL" means unsigned long, which is 32-bits on i386.
--D
> +
> +#define XME_ADDR_DEVICE_MASK (~(XME_ADDR_DATA_DEVICE | XME_ADDR_LOG_DEVICE))
> +
> #endif /* _XFS_METADUMP_H_ */
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 11/24] metadump: Define metadump v2 ondisk format structures and macros
2023-05-23 17:34 ` Darrick J. Wong
@ 2023-05-25 9:26 ` Chandan Babu R
2023-06-02 14:46 ` Darrick J. Wong
0 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 9:26 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 10:34:35 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:37PM +0530, Chandan Babu R wrote:
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> include/xfs_metadump.h | 32 ++++++++++++++++++++++++++++++++
>> 1 file changed, 32 insertions(+)
>>
>> diff --git a/include/xfs_metadump.h b/include/xfs_metadump.h
>> index a4dca25cb..1d8d7008c 100644
>> --- a/include/xfs_metadump.h
>> +++ b/include/xfs_metadump.h
>> @@ -8,7 +8,9 @@
>> #define _XFS_METADUMP_H_
>>
>> #define XFS_MD_MAGIC_V1 0x5846534d /* 'XFSM' */
>> +#define XFS_MD_MAGIC_V2 0x584D4432 /* 'XMD2' */
>>
>> +/* Metadump v1 */
>> typedef struct xfs_metablock {
>> __be32 mb_magic;
>> __be16 mb_count;
>> @@ -23,4 +25,34 @@ typedef struct xfs_metablock {
>> #define XFS_METADUMP_FULLBLOCKS (1 << 2)
>> #define XFS_METADUMP_DIRTYLOG (1 << 3)
>>
>> +/* Metadump v2 */
>> +struct xfs_metadump_header {
>> + __be32 xmh_magic;
>> + __be32 xmh_version;
>> + __be32 xmh_compat_flags;
>> + __be32 xmh_incompat_flags;
>> + __be64 xmh_reserved;
>
> __be32 xmh_crc; ?
>
> Otherwise there's nothing to check for bitflips in the index blocks
> themselves.
The user could generate a sha1sum of the metadump file and share it with the
receiver for verifying the integrity of the metadump file right?
>
>> +} __packed;
>
> Does an array of xfs_meta_extent come immediately after
> xfs_metadump_header, or do they go in a separate block after the header?
> How big is the index block supposed to be?
>
A metadump file in V2 format is structured as shown below,
|------------------------------|
| struct xfs_metadump_header |
|------------------------------|
| struct xfs_meta_extent 0 |
| Extent 0's data |
| struct xfs_meta_extent 1 |
| Extent 1's data |
| ... |
| struct xfs_meta_extent (n-1) |
| Extent (n-1)'s data |
|------------------------------|
If there are no objections, I will add the above diagram to
include/xfs_metadump.h.
>> +
>> +#define XFS_MD2_INCOMPAT_OBFUSCATED (1 << 0)
>> +#define XFS_MD2_INCOMPAT_FULLBLOCKS (1 << 1)
>> +#define XFS_MD2_INCOMPAT_DIRTYLOG (1 << 2)
>
> Should the header declare when some of the xfs_meta_extents will have
> XME_ADDR_LOG_DEVICE set?
>
I will add a comment describing that extents captured from an external log
device will have XME_ADDR_LOG_DEVICE set.
>> +
>> +struct xfs_meta_extent {
>> + /*
>
> Tabs not spaces.
>
>> + * Lowest 54 bits are used to store 512 byte addresses.
>> + * Next 2 bits is used for indicating the device.
>> + * 00 - Data device
>> + * 01 - External log
>
> So if you were to (say) add the realtime device, would that be bit 56,
> or would you define 0xC0000000000000 (aka DATA|LOG) to mean realtime?
>
I am sorry, the comment I have written above is incorrect. I forgot to update it
before posting the patchset. Realtime device has to be (1ULL << 56).
But, Your comments on "[PATCH 22/24] mdrestore: Define mdrestore ops for v2
format" has convinced me that we could use the 2 bits at bit posistions 54 and
55 as a counter. i.e 00 maps to XME_ADDR_DATA_DEVICE and 01 maps to
XME_ADDR_LOG_DEVICE.
>> + */
>> + __be64 xme_addr;
>> + /* In units of 512 byte blocks */
>> + __be32 xme_len;
>> +} __packed;
>> +
>> +#define XME_ADDR_DATA_DEVICE (1UL << 54)
>> +#define XME_ADDR_LOG_DEVICE (1UL << 55)
>
> 1ULL, because "UL" means unsigned long, which is 32-bits on i386.
Ok. I will fix that.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 11/24] metadump: Define metadump v2 ondisk format structures and macros
2023-05-25 9:26 ` Chandan Babu R
@ 2023-06-02 14:46 ` Darrick J. Wong
0 siblings, 0 replies; 76+ messages in thread
From: Darrick J. Wong @ 2023-06-02 14:46 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Thu, May 25, 2023 at 02:56:38PM +0530, Chandan Babu R wrote:
> On Tue, May 23, 2023 at 10:34:35 AM -0700, Darrick J. Wong wrote:
> > On Tue, May 23, 2023 at 02:30:37PM +0530, Chandan Babu R wrote:
> >> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> >> ---
> >> include/xfs_metadump.h | 32 ++++++++++++++++++++++++++++++++
> >> 1 file changed, 32 insertions(+)
> >>
> >> diff --git a/include/xfs_metadump.h b/include/xfs_metadump.h
> >> index a4dca25cb..1d8d7008c 100644
> >> --- a/include/xfs_metadump.h
> >> +++ b/include/xfs_metadump.h
> >> @@ -8,7 +8,9 @@
> >> #define _XFS_METADUMP_H_
> >>
> >> #define XFS_MD_MAGIC_V1 0x5846534d /* 'XFSM' */
> >> +#define XFS_MD_MAGIC_V2 0x584D4432 /* 'XMD2' */
> >>
> >> +/* Metadump v1 */
> >> typedef struct xfs_metablock {
> >> __be32 mb_magic;
> >> __be16 mb_count;
> >> @@ -23,4 +25,34 @@ typedef struct xfs_metablock {
> >> #define XFS_METADUMP_FULLBLOCKS (1 << 2)
> >> #define XFS_METADUMP_DIRTYLOG (1 << 3)
> >>
> >> +/* Metadump v2 */
> >> +struct xfs_metadump_header {
> >> + __be32 xmh_magic;
> >> + __be32 xmh_version;
> >> + __be32 xmh_compat_flags;
> >> + __be32 xmh_incompat_flags;
> >> + __be64 xmh_reserved;
> >
> > __be32 xmh_crc; ?
> >
> > Otherwise there's nothing to check for bitflips in the index blocks
> > themselves.
>
> The user could generate a sha1sum of the metadump file and share it with the
> receiver for verifying the integrity of the metadump file right?
Heh, sorry, this is another erroneous comment based on me thinking that
xfs_metadump_header would be followed by an array of xfs_meta_extent,
like how v1 does things. Question withdrawn.
> >
> >> +} __packed;
> >
> > Does an array of xfs_meta_extent come immediately after
> > xfs_metadump_header, or do they go in a separate block after the header?
> > How big is the index block supposed to be?
> >
>
> A metadump file in V2 format is structured as shown below,
>
> |------------------------------|
> | struct xfs_metadump_header |
> |------------------------------|
> | struct xfs_meta_extent 0 |
> | Extent 0's data |
> | struct xfs_meta_extent 1 |
> | Extent 1's data |
> | ... |
> | struct xfs_meta_extent (n-1) |
> | Extent (n-1)'s data |
> |------------------------------|
>
> If there are no objections, I will add the above diagram to
> include/xfs_metadump.h.
Yes, please.
> >> +
> >> +#define XFS_MD2_INCOMPAT_OBFUSCATED (1 << 0)
> >> +#define XFS_MD2_INCOMPAT_FULLBLOCKS (1 << 1)
> >> +#define XFS_MD2_INCOMPAT_DIRTYLOG (1 << 2)
> >
> > Should the header declare when some of the xfs_meta_extents will have
> > XME_ADDR_LOG_DEVICE set?
> >
>
> I will add a comment describing that extents captured from an external log
> device will have XME_ADDR_LOG_DEVICE set.
<nod> I get that, but I'm asking about declaring in the
xfs_metadump_header that eventually there will be an xfs_meta_extent
with XME_ADDR_LOG_DEVICE in it.
The scenario that I'm thinking about is on the mdrestore side of things.
mdrestore reads its CLI arguments, but it doesn't know if the log device
argument is actually required until it reads the metadump header.
Once it has read the metadump header, it ought to be able to tell the
user "This metadump has external log contents but you didn't pass in
--log-device=/dev/XXX" and error out before actually writing anything.
What I think we ought to avoid is the situation where mid-stream we
discover a meta_extent with XME_ADDR_LOG_DEVICE set and only /then/
error out, having already written a bunch of stuff to the data device.
> >> +
> >> +struct xfs_meta_extent {
> >> + /*
> >
> > Tabs not spaces.
> >
>
> >> + * Lowest 54 bits are used to store 512 byte addresses.
> >> + * Next 2 bits is used for indicating the device.
> >> + * 00 - Data device
> >> + * 01 - External log
> >
> > So if you were to (say) add the realtime device, would that be bit 56,
> > or would you define 0xC0000000000000 (aka DATA|LOG) to mean realtime?
> >
>
> I am sorry, the comment I have written above is incorrect. I forgot to update it
> before posting the patchset. Realtime device has to be (1ULL << 56).
>
> But, Your comments on "[PATCH 22/24] mdrestore: Define mdrestore ops for v2
> format" has convinced me that we could use the 2 bits at bit posistions 54 and
> 55 as a counter. i.e 00 maps to XME_ADDR_DATA_DEVICE and 01 maps to
> XME_ADDR_LOG_DEVICE.
<nod>
> >> + */
> >> + __be64 xme_addr;
> >> + /* In units of 512 byte blocks */
> >> + __be32 xme_len;
> >> +} __packed;
> >> +
> >> +#define XME_ADDR_DATA_DEVICE (1UL << 54)
> >> +#define XME_ADDR_LOG_DEVICE (1UL << 55)
> >
> > 1ULL, because "UL" means unsigned long, which is 32-bits on i386.
>
> Ok. I will fix that.
<nod>
--D
>
> --
> chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 12/24] metadump: Define metadump ops for v2 format
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (10 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 11/24] metadump: Define metadump v2 ondisk format structures and macros Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:37 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 13/24] metadump: Add support for passing version option Chandan Babu R
` (11 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
This commit adds functionality to dump metadata from an XFS filesystem in
newly introduced v2 format.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
db/metadump.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 70 insertions(+), 3 deletions(-)
diff --git a/db/metadump.c b/db/metadump.c
index 9d7ad76ae..627436e68 100644
--- a/db/metadump.c
+++ b/db/metadump.c
@@ -3037,6 +3037,69 @@ static struct metadump_ops metadump1_ops = {
.release_metadump = release_metadump_v1,
};
+static int
+init_metadump_v2(void)
+{
+ struct xfs_metadump_header xmh = {0};
+ uint32_t compat_flags = 0;
+
+ xmh.xmh_magic = cpu_to_be32(XFS_MD_MAGIC_V2);
+ xmh.xmh_version = 2;
+
+ if (metadump.obfuscate)
+ compat_flags |= XFS_MD2_INCOMPAT_OBFUSCATED;
+ if (!metadump.zero_stale_data)
+ compat_flags |= XFS_MD2_INCOMPAT_FULLBLOCKS;
+ if (metadump.dirty_log)
+ compat_flags |= XFS_MD2_INCOMPAT_DIRTYLOG;
+
+ xmh.xmh_compat_flags = cpu_to_be32(compat_flags);
+
+ if (fwrite(&xmh, sizeof(xmh), 1, metadump.outf) != 1) {
+ print_warning("error writing to target file");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+write_metadump_v2(
+ enum typnm type,
+ char *data,
+ int64_t off,
+ int len)
+{
+ struct xfs_meta_extent xme;
+ uint64_t addr;
+
+ addr = off;
+ if (type == TYP_ELOG)
+ addr |= XME_ADDR_LOG_DEVICE;
+ else
+ addr |= XME_ADDR_DATA_DEVICE;
+
+ xme.xme_addr = cpu_to_be64(addr);
+ xme.xme_len = cpu_to_be32(len);
+
+ if (fwrite(&xme, sizeof(xme), 1, metadump.outf) != 1) {
+ print_warning("error writing to target file");
+ return -EIO;
+ }
+
+ if (fwrite(data, len << BBSHIFT, 1, metadump.outf) != 1) {
+ print_warning("error writing to target file");
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static struct metadump_ops metadump2_ops = {
+ .init_metadump = init_metadump_v2,
+ .write_metadump = write_metadump_v2,
+};
+
static int
metadump_f(
int argc,
@@ -3178,7 +3241,10 @@ metadump_f(
}
}
- metadump.mdops = &metadump1_ops;
+ if (metadump.version == 1)
+ metadump.mdops = &metadump1_ops;
+ else
+ metadump.mdops = &metadump2_ops;
ret = metadump.mdops->init_metadump();
if (ret)
@@ -3203,7 +3269,7 @@ metadump_f(
exitcode = !copy_log(log_type);
/* write the remaining index */
- if (!exitcode)
+ if (!exitcode && metadump.mdops->end_write_metadump)
exitcode = metadump.mdops->end_write_metadump() < 0;
if (metadump.progress_since_warning)
@@ -3223,7 +3289,8 @@ metadump_f(
while (iocur_sp > start_iocur_sp)
pop_cur();
- metadump.mdops->release_metadump();
+ if (metadump.mdops->release_metadump)
+ metadump.mdops->release_metadump();
out:
return 0;
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 12/24] metadump: Define metadump ops for v2 format
2023-05-23 9:00 ` [PATCH 12/24] metadump: Define metadump ops for v2 format Chandan Babu R
@ 2023-05-23 17:37 ` Darrick J. Wong
0 siblings, 0 replies; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:37 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:38PM +0530, Chandan Babu R wrote:
> This commit adds functionality to dump metadata from an XFS filesystem in
> newly introduced v2 format.
>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> db/metadump.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 70 insertions(+), 3 deletions(-)
>
> diff --git a/db/metadump.c b/db/metadump.c
> index 9d7ad76ae..627436e68 100644
> --- a/db/metadump.c
> +++ b/db/metadump.c
> @@ -3037,6 +3037,69 @@ static struct metadump_ops metadump1_ops = {
> .release_metadump = release_metadump_v1,
> };
>
> +static int
> +init_metadump_v2(void)
> +{
> + struct xfs_metadump_header xmh = {0};
> + uint32_t compat_flags = 0;
> +
> + xmh.xmh_magic = cpu_to_be32(XFS_MD_MAGIC_V2);
> + xmh.xmh_version = 2;
> +
> + if (metadump.obfuscate)
> + compat_flags |= XFS_MD2_INCOMPAT_OBFUSCATED;
> + if (!metadump.zero_stale_data)
> + compat_flags |= XFS_MD2_INCOMPAT_FULLBLOCKS;
> + if (metadump.dirty_log)
> + compat_flags |= XFS_MD2_INCOMPAT_DIRTYLOG;
> +
> + xmh.xmh_compat_flags = cpu_to_be32(compat_flags);
> +
> + if (fwrite(&xmh, sizeof(xmh), 1, metadump.outf) != 1) {
> + print_warning("error writing to target file");
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +static int
> +write_metadump_v2(
> + enum typnm type,
> + char *data,
> + int64_t off,
> + int len)
> +{
> + struct xfs_meta_extent xme;
> + uint64_t addr;
> +
> + addr = off;
> + if (type == TYP_ELOG)
> + addr |= XME_ADDR_LOG_DEVICE;
> + else
> + addr |= XME_ADDR_DATA_DEVICE;
> +
> + xme.xme_addr = cpu_to_be64(addr);
> + xme.xme_len = cpu_to_be32(len);
> +
> + if (fwrite(&xme, sizeof(xme), 1, metadump.outf) != 1) {
> + print_warning("error writing to target file");
> + return -EIO;
> + }
> +
> + if (fwrite(data, len << BBSHIFT, 1, metadump.outf) != 1) {
> + print_warning("error writing to target file");
> + return -EIO;
> + }
Oh, I see, the v2 format is a straight stream -- first the v2 header,
then a xfs_meta_extent, then the buffer; and repeat #2 and #3 until
finished. That ought to be documented in the previous patch.
--D
> +
> + return 0;
> +}
> +
> +static struct metadump_ops metadump2_ops = {
> + .init_metadump = init_metadump_v2,
> + .write_metadump = write_metadump_v2,
> +};
> +
> static int
> metadump_f(
> int argc,
> @@ -3178,7 +3241,10 @@ metadump_f(
> }
> }
>
> - metadump.mdops = &metadump1_ops;
> + if (metadump.version == 1)
> + metadump.mdops = &metadump1_ops;
> + else
> + metadump.mdops = &metadump2_ops;
>
> ret = metadump.mdops->init_metadump();
> if (ret)
> @@ -3203,7 +3269,7 @@ metadump_f(
> exitcode = !copy_log(log_type);
>
> /* write the remaining index */
> - if (!exitcode)
> + if (!exitcode && metadump.mdops->end_write_metadump)
> exitcode = metadump.mdops->end_write_metadump() < 0;
>
> if (metadump.progress_since_warning)
> @@ -3223,7 +3289,8 @@ metadump_f(
> while (iocur_sp > start_iocur_sp)
> pop_cur();
>
> - metadump.mdops->release_metadump();
> + if (metadump.mdops->release_metadump)
> + metadump.mdops->release_metadump();
>
> out:
> return 0;
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 13/24] metadump: Add support for passing version option
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (11 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 12/24] metadump: Define metadump ops for v2 format Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:41 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 14/24] xfs_metadump.sh: " Chandan Babu R
` (10 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
The new option allows the user to explicitly specify the version of metadump
to use. However, we will default to using the v1 format.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
db/metadump.c | 18 ++++++++++++++++--
1 file changed, 16 insertions(+), 2 deletions(-)
diff --git a/db/metadump.c b/db/metadump.c
index 627436e68..df508b987 100644
--- a/db/metadump.c
+++ b/db/metadump.c
@@ -37,7 +37,7 @@ static void metadump_help(void);
static const cmdinfo_t metadump_cmd =
{ "metadump", NULL, metadump_f, 0, -1, 0,
- N_("[-a] [-e] [-g] [-m max_extent] [-w] [-o] filename"),
+ N_("[-a] [-e] [-g] [-m max_extent] [-w] [-o] [-v 1|2] filename"),
N_("dump metadata to a file"), metadump_help };
struct metadump_ops {
@@ -91,6 +91,7 @@ metadump_help(void)
" -g -- Display dump progress\n"
" -m -- Specify max extent size in blocks to copy (default = %d blocks)\n"
" -o -- Don't obfuscate names and extended attributes\n"
+" -v -- Metadump version to be used\n"
" -w -- Show warnings of bad metadata information\n"
"\n"), DEFAULT_MAX_EXT_SIZE);
}
@@ -3112,6 +3113,7 @@ metadump_f(
int outfd = -1;
int ret;
char *p;
+ bool version_opt_set = false;
exitcode = 1;
@@ -3140,7 +3142,7 @@ metadump_f(
return 0;
}
- while ((c = getopt(argc, argv, "aegm:ow")) != EOF) {
+ while ((c = getopt(argc, argv, "aegm:ov:w")) != EOF) {
switch (c) {
case 'a':
metadump.zero_stale_data = 0;
@@ -3164,6 +3166,15 @@ metadump_f(
case 'o':
metadump.obfuscate = 0;
break;
+ case 'v':
+ metadump.version = (int)strtol(optarg, &p, 0);
+ if (*p != '\0' || (metadump.version != 1 && metadump.version != 2)) {
+ print_warning("bad metadump version: %s",
+ optarg);
+ return 0;
+ }
+ version_opt_set = true;
+ break;
case 'w':
metadump.show_warnings = 1;
break;
@@ -3178,6 +3189,9 @@ metadump_f(
return 0;
}
+ if (mp->m_logdev_targp != mp->m_ddev_targp && version_opt_set == false)
+ metadump.version = 2;
+
/* If we'll copy the log, see if the log is dirty */
if (mp->m_logdev_targp == mp->m_ddev_targp || metadump.version == 2) {
log_type = TYP_LOG;
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 13/24] metadump: Add support for passing version option
2023-05-23 9:00 ` [PATCH 13/24] metadump: Add support for passing version option Chandan Babu R
@ 2023-05-23 17:41 ` Darrick J. Wong
0 siblings, 0 replies; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:41 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:39PM +0530, Chandan Babu R wrote:
> The new option allows the user to explicitly specify the version of metadump
> to use. However, we will default to using the v1 format.
>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
Looks fine,
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
--D
> ---
> db/metadump.c | 18 ++++++++++++++++--
> 1 file changed, 16 insertions(+), 2 deletions(-)
>
> diff --git a/db/metadump.c b/db/metadump.c
> index 627436e68..df508b987 100644
> --- a/db/metadump.c
> +++ b/db/metadump.c
> @@ -37,7 +37,7 @@ static void metadump_help(void);
>
> static const cmdinfo_t metadump_cmd =
> { "metadump", NULL, metadump_f, 0, -1, 0,
> - N_("[-a] [-e] [-g] [-m max_extent] [-w] [-o] filename"),
> + N_("[-a] [-e] [-g] [-m max_extent] [-w] [-o] [-v 1|2] filename"),
> N_("dump metadata to a file"), metadump_help };
>
> struct metadump_ops {
> @@ -91,6 +91,7 @@ metadump_help(void)
> " -g -- Display dump progress\n"
> " -m -- Specify max extent size in blocks to copy (default = %d blocks)\n"
> " -o -- Don't obfuscate names and extended attributes\n"
> +" -v -- Metadump version to be used\n"
> " -w -- Show warnings of bad metadata information\n"
> "\n"), DEFAULT_MAX_EXT_SIZE);
> }
> @@ -3112,6 +3113,7 @@ metadump_f(
> int outfd = -1;
> int ret;
> char *p;
> + bool version_opt_set = false;
>
> exitcode = 1;
>
> @@ -3140,7 +3142,7 @@ metadump_f(
> return 0;
> }
>
> - while ((c = getopt(argc, argv, "aegm:ow")) != EOF) {
> + while ((c = getopt(argc, argv, "aegm:ov:w")) != EOF) {
> switch (c) {
> case 'a':
> metadump.zero_stale_data = 0;
> @@ -3164,6 +3166,15 @@ metadump_f(
> case 'o':
> metadump.obfuscate = 0;
> break;
> + case 'v':
> + metadump.version = (int)strtol(optarg, &p, 0);
> + if (*p != '\0' || (metadump.version != 1 && metadump.version != 2)) {
> + print_warning("bad metadump version: %s",
> + optarg);
> + return 0;
> + }
> + version_opt_set = true;
> + break;
> case 'w':
> metadump.show_warnings = 1;
> break;
> @@ -3178,6 +3189,9 @@ metadump_f(
> return 0;
> }
>
> + if (mp->m_logdev_targp != mp->m_ddev_targp && version_opt_set == false)
> + metadump.version = 2;
> +
> /* If we'll copy the log, see if the log is dirty */
> if (mp->m_logdev_targp == mp->m_ddev_targp || metadump.version == 2) {
> log_type = TYP_LOG;
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 14/24] xfs_metadump.sh: Add support for passing version option
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (12 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 13/24] metadump: Add support for passing version option Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:39 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 15/24] xfs_metadump.8: Add description for the newly introduced -v option Chandan Babu R
` (9 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
The new option allows the user to explicitly specify which version of
metadump to use. However, we will default to using the v1 format.
---
db/xfs_metadump.sh | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/db/xfs_metadump.sh b/db/xfs_metadump.sh
index 9852a5bc2..9e8f86e53 100755
--- a/db/xfs_metadump.sh
+++ b/db/xfs_metadump.sh
@@ -8,7 +8,7 @@ OPTS=" "
DBOPTS=" "
USAGE="Usage: xfs_metadump [-aefFogwV] [-m max_extents] [-l logdev] source target"
-while getopts "aefgl:m:owFV" c
+while getopts "aefgl:m:owFv:V" c
do
case $c in
a) OPTS=$OPTS"-a ";;
@@ -20,6 +20,7 @@ do
f) DBOPTS=$DBOPTS" -f";;
l) DBOPTS=$DBOPTS" -l "$OPTARG" ";;
F) DBOPTS=$DBOPTS" -F";;
+ v) OPTS=$OPTS"-v "$OPTARG" ";;
V) xfs_db -p xfs_metadump -V
status=$?
exit $status
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 14/24] xfs_metadump.sh: Add support for passing version option
2023-05-23 9:00 ` [PATCH 14/24] xfs_metadump.sh: " Chandan Babu R
@ 2023-05-23 17:39 ` Darrick J. Wong
2023-05-25 9:31 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:39 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:40PM +0530, Chandan Babu R wrote:
> The new option allows the user to explicitly specify which version of
> metadump to use. However, we will default to using the v1 format.
Needs SOB tag.
--D
> ---
> db/xfs_metadump.sh | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/db/xfs_metadump.sh b/db/xfs_metadump.sh
> index 9852a5bc2..9e8f86e53 100755
> --- a/db/xfs_metadump.sh
> +++ b/db/xfs_metadump.sh
> @@ -8,7 +8,7 @@ OPTS=" "
> DBOPTS=" "
> USAGE="Usage: xfs_metadump [-aefFogwV] [-m max_extents] [-l logdev] source target"
>
> -while getopts "aefgl:m:owFV" c
> +while getopts "aefgl:m:owFv:V" c
> do
> case $c in
> a) OPTS=$OPTS"-a ";;
> @@ -20,6 +20,7 @@ do
> f) DBOPTS=$DBOPTS" -f";;
> l) DBOPTS=$DBOPTS" -l "$OPTARG" ";;
> F) DBOPTS=$DBOPTS" -F";;
> + v) OPTS=$OPTS"-v "$OPTARG" ";;
> V) xfs_db -p xfs_metadump -V
> status=$?
> exit $status
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH 14/24] xfs_metadump.sh: Add support for passing version option
2023-05-23 17:39 ` Darrick J. Wong
@ 2023-05-25 9:31 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 9:31 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 10:39:30 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:40PM +0530, Chandan Babu R wrote:
>> The new option allows the user to explicitly specify which version of
>> metadump to use. However, we will default to using the v1 format.
>
> Needs SOB tag.
I will add my SOB tag and resend the patchset.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 15/24] xfs_metadump.8: Add description for the newly introduced -v option
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (13 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 14/24] xfs_metadump.sh: " Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:40 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 16/24] mdrestore: Define and use struct mdrestore Chandan Babu R
` (8 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
man/man8/xfs_metadump.8 | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/man/man8/xfs_metadump.8 b/man/man8/xfs_metadump.8
index c0e79d779..23695c768 100644
--- a/man/man8/xfs_metadump.8
+++ b/man/man8/xfs_metadump.8
@@ -11,6 +11,9 @@ xfs_metadump \- copy XFS filesystem metadata to a file
] [
.B \-l
.I logdev
+] [
+.B \-v
+.I version
]
.I source
.I target
@@ -74,6 +77,9 @@ metadata such as filenames is not considered sensitive. If obfuscation
is required on a metadump with a dirty log, please inform the recipient
of the metadump image about this situation.
.PP
+The contents of an external log device can be dumped only when using the v2
+format. Metadump in v2 format can be generated by passing the "-v 2" option.
+.PP
.B xfs_metadump
should not be used for any purposes other than for debugging and reporting
filesystem problems. The most common usage scenario for this tool is when
@@ -134,6 +140,10 @@ this value. The default size is 2097151 blocks.
.B \-o
Disables obfuscation of file names and extended attributes.
.TP
+.B \-v
+The format of the metadump file to be produced. Valid values are 1 and 2. The
+default metadump format is 1.
+.TP
.B \-w
Prints warnings of inconsistent metadata encountered to stderr. Bad metadata
is still copied.
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 15/24] xfs_metadump.8: Add description for the newly introduced -v option
2023-05-23 9:00 ` [PATCH 15/24] xfs_metadump.8: Add description for the newly introduced -v option Chandan Babu R
@ 2023-05-23 17:40 ` Darrick J. Wong
2023-05-25 10:04 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:40 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:41PM +0530, Chandan Babu R wrote:
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
This should be in the previous patch.
> ---
> man/man8/xfs_metadump.8 | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/man/man8/xfs_metadump.8 b/man/man8/xfs_metadump.8
> index c0e79d779..23695c768 100644
> --- a/man/man8/xfs_metadump.8
> +++ b/man/man8/xfs_metadump.8
> @@ -11,6 +11,9 @@ xfs_metadump \- copy XFS filesystem metadata to a file
> ] [
> .B \-l
> .I logdev
> +] [
> +.B \-v
> +.I version
> ]
> .I source
> .I target
> @@ -74,6 +77,9 @@ metadata such as filenames is not considered sensitive. If obfuscation
> is required on a metadump with a dirty log, please inform the recipient
> of the metadump image about this situation.
> .PP
> +The contents of an external log device can be dumped only when using the v2
> +format. Metadump in v2 format can be generated by passing the "-v 2" option.
Please start each sentence on a separate line.
This also should mention that metadump will pick v2 if there's no
explicit -v option and the fs has an external log.
--D
> +.PP
> .B xfs_metadump
> should not be used for any purposes other than for debugging and reporting
> filesystem problems. The most common usage scenario for this tool is when
> @@ -134,6 +140,10 @@ this value. The default size is 2097151 blocks.
> .B \-o
> Disables obfuscation of file names and extended attributes.
> .TP
> +.B \-v
> +The format of the metadump file to be produced. Valid values are 1 and 2. The
> +default metadump format is 1.
> +.TP
> .B \-w
> Prints warnings of inconsistent metadata encountered to stderr. Bad metadata
> is still copied.
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH 15/24] xfs_metadump.8: Add description for the newly introduced -v option
2023-05-23 17:40 ` Darrick J. Wong
@ 2023-05-25 10:04 ` Chandan Babu R
2023-06-02 14:58 ` Darrick J. Wong
0 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 10:04 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 10:40:53 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:41PM +0530, Chandan Babu R wrote:
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>
> This should be in the previous patch.
>
Ok.
>> ---
>> man/man8/xfs_metadump.8 | 10 ++++++++++
>> 1 file changed, 10 insertions(+)
>>
>> diff --git a/man/man8/xfs_metadump.8 b/man/man8/xfs_metadump.8
>> index c0e79d779..23695c768 100644
>> --- a/man/man8/xfs_metadump.8
>> +++ b/man/man8/xfs_metadump.8
>> @@ -11,6 +11,9 @@ xfs_metadump \- copy XFS filesystem metadata to a file
>> ] [
>> .B \-l
>> .I logdev
>> +] [
>> +.B \-v
>> +.I version
>> ]
>> .I source
>> .I target
>> @@ -74,6 +77,9 @@ metadata such as filenames is not considered sensitive. If obfuscation
>> is required on a metadump with a dirty log, please inform the recipient
>> of the metadump image about this situation.
>> .PP
>> +The contents of an external log device can be dumped only when using the v2
>> +format. Metadump in v2 format can be generated by passing the "-v 2" option.
>
> Please start each sentence on a separate line.
Does it need to be formatted as shown below?
The contents of an external log device can be dumped only when using the v2
format.
Metadump in v2 format can be generated by passing the "-v 2" option.
>
> This also should mention that metadump will pick v2 if there's no
> explicit -v option and the fs has an external log.
Ok. I will include that as part of the description.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH 15/24] xfs_metadump.8: Add description for the newly introduced -v option
2023-05-25 10:04 ` Chandan Babu R
@ 2023-06-02 14:58 ` Darrick J. Wong
0 siblings, 0 replies; 76+ messages in thread
From: Darrick J. Wong @ 2023-06-02 14:58 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Thu, May 25, 2023 at 03:34:47PM +0530, Chandan Babu R wrote:
> On Tue, May 23, 2023 at 10:40:53 AM -0700, Darrick J. Wong wrote:
> > On Tue, May 23, 2023 at 02:30:41PM +0530, Chandan Babu R wrote:
> >> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> >
> > This should be in the previous patch.
> >
>
> Ok.
>
> >> ---
> >> man/man8/xfs_metadump.8 | 10 ++++++++++
> >> 1 file changed, 10 insertions(+)
> >>
> >> diff --git a/man/man8/xfs_metadump.8 b/man/man8/xfs_metadump.8
> >> index c0e79d779..23695c768 100644
> >> --- a/man/man8/xfs_metadump.8
> >> +++ b/man/man8/xfs_metadump.8
> >> @@ -11,6 +11,9 @@ xfs_metadump \- copy XFS filesystem metadata to a file
> >> ] [
> >> .B \-l
> >> .I logdev
> >> +] [
> >> +.B \-v
> >> +.I version
> >> ]
> >> .I source
> >> .I target
> >> @@ -74,6 +77,9 @@ metadata such as filenames is not considered sensitive. If obfuscation
> >> is required on a metadump with a dirty log, please inform the recipient
> >> of the metadump image about this situation.
> >> .PP
> >> +The contents of an external log device can be dumped only when using the v2
> >> +format. Metadump in v2 format can be generated by passing the "-v 2" option.
> >
> > Please start each sentence on a separate line.
>
> Does it need to be formatted as shown below?
>
> The contents of an external log device can be dumped only when using the v2
> format.
> Metadump in v2 format can be generated by passing the "-v 2" option.
Yes, this is correct enough for me. From man-pages(7):
Use semantic newlines
In the source of a manual page, new sentences should be started
on new lines, long sentences should be split into lines at
clause breaks (commas, semicolons, colons, and so on), and long
clauses should be split at phrase boundaries. This convention,
sometimes known as "semantic newlines", makes it easier to see
the effect of patches, which often operate at the level of in‐
dividual sentences, clauses, or phrases.
Frankly I don't care to split at the *clause* level because I think
newlines should come after a complete thought. Note that the source of
that paragraph looks like this:
.SS Use semantic newlines
In the source of a manual page,
new sentences should be started on new lines,
and long sentences should split into lines at clause breaks
(commas, semicolons, colons, and so on).
This convention, sometimes known as "semantic newlines",
makes it easier to see the effect of patches,
which often operate at the level of individual sentences or sentence clauses.
Which is (IMHO) harder to read when reviewing diffs.
Oddly it also misses the fact that “sometimes known as "semantic
newlines",” is a parenthetical clause and should be a separate line.
I'm also pretty sure that “...should be started on new lines, and long
sentences should...” is a compound sentence and doesn't need the comma.
This whole thing also assumes that writers have had as comprehensive an
education in English grammar as I have forgotten over the past 30 years.
So.
I'm only going to go as far as asking for newlines after each sentence
so that manpage diffs don't write-amplify into adjacent thoughts that
haven't actually changed.
> >
> > This also should mention that metadump will pick v2 if there's no
> > explicit -v option and the fs has an external log.
>
> Ok. I will include that as part of the description.
Thank you!
--D
> --
> chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 16/24] mdrestore: Define and use struct mdrestore
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (14 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 15/24] xfs_metadump.8: Add description for the newly introduced -v option Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:42 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 17/24] mdrestore: Add open_device(), read_header() and show_info() functions Chandan Babu R
` (7 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
This commit collects all state tracking variables in a new "struct mdrestore"
structure.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
mdrestore/xfs_mdrestore.c | 27 +++++++++++++++++----------
1 file changed, 17 insertions(+), 10 deletions(-)
diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
index 481dd00c2..de9175a08 100644
--- a/mdrestore/xfs_mdrestore.c
+++ b/mdrestore/xfs_mdrestore.c
@@ -7,9 +7,11 @@
#include "libxfs.h"
#include "xfs_metadump.h"
-static int show_progress = 0;
-static int show_info = 0;
-static int progress_since_warning = 0;
+static struct mdrestore {
+ int show_progress;
+ int show_info;
+ int progress_since_warning;
+} mdrestore;
static void
fatal(const char *msg, ...)
@@ -35,7 +37,7 @@ print_progress(const char *fmt, ...)
printf("\r%-59s", buf);
fflush(stdout);
- progress_since_warning = 1;
+ mdrestore.progress_since_warning = 1;
}
/*
@@ -127,7 +129,8 @@ perform_restore(
bytes_read = 0;
for (;;) {
- if (show_progress && (bytes_read & ((1 << 20) - 1)) == 0)
+ if (mdrestore.show_progress &&
+ (bytes_read & ((1 << 20) - 1)) == 0)
print_progress("%lld MB read", bytes_read >> 20);
for (cur_index = 0; cur_index < mb_count; cur_index++) {
@@ -158,7 +161,7 @@ perform_restore(
bytes_read += block_size + (mb_count << mbp->mb_blocklog);
}
- if (progress_since_warning)
+ if (mdrestore.progress_since_warning)
putchar('\n');
memset(block_buffer, 0, sb.sb_sectsize);
@@ -197,15 +200,19 @@ main(
int is_target_file;
struct xfs_metablock mb;
+ mdrestore.show_progress = 0;
+ mdrestore.show_info = 0;
+ mdrestore.progress_since_warning = 0;
+
progname = basename(argv[0]);
while ((c = getopt(argc, argv, "giV")) != EOF) {
switch (c) {
case 'g':
- show_progress = 1;
+ mdrestore.show_progress = 1;
break;
case 'i':
- show_info = 1;
+ mdrestore.show_info = 1;
break;
case 'V':
printf("%s version %s\n", progname, VERSION);
@@ -219,7 +226,7 @@ main(
usage();
/* show_info without a target is ok */
- if (!show_info && argc - optind != 2)
+ if (!mdrestore.show_info && argc - optind != 2)
usage();
/*
@@ -243,7 +250,7 @@ main(
if (mb.mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
fatal("specified file is not a metadata dump\n");
- if (show_info) {
+ if (mdrestore.show_info) {
if (mb.mb_info & XFS_METADUMP_INFO_FLAGS) {
printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
argv[optind],
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 16/24] mdrestore: Define and use struct mdrestore
2023-05-23 9:00 ` [PATCH 16/24] mdrestore: Define and use struct mdrestore Chandan Babu R
@ 2023-05-23 17:42 ` Darrick J. Wong
2023-05-26 8:38 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:42 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:42PM +0530, Chandan Babu R wrote:
> This commit collects all state tracking variables in a new "struct mdrestore"
> structure.
Same comment as patch 3.
--D
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> mdrestore/xfs_mdrestore.c | 27 +++++++++++++++++----------
> 1 file changed, 17 insertions(+), 10 deletions(-)
>
> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> index 481dd00c2..de9175a08 100644
> --- a/mdrestore/xfs_mdrestore.c
> +++ b/mdrestore/xfs_mdrestore.c
> @@ -7,9 +7,11 @@
> #include "libxfs.h"
> #include "xfs_metadump.h"
>
> -static int show_progress = 0;
> -static int show_info = 0;
> -static int progress_since_warning = 0;
> +static struct mdrestore {
> + int show_progress;
> + int show_info;
> + int progress_since_warning;
> +} mdrestore;
>
> static void
> fatal(const char *msg, ...)
> @@ -35,7 +37,7 @@ print_progress(const char *fmt, ...)
>
> printf("\r%-59s", buf);
> fflush(stdout);
> - progress_since_warning = 1;
> + mdrestore.progress_since_warning = 1;
> }
>
> /*
> @@ -127,7 +129,8 @@ perform_restore(
> bytes_read = 0;
>
> for (;;) {
> - if (show_progress && (bytes_read & ((1 << 20) - 1)) == 0)
> + if (mdrestore.show_progress &&
> + (bytes_read & ((1 << 20) - 1)) == 0)
> print_progress("%lld MB read", bytes_read >> 20);
>
> for (cur_index = 0; cur_index < mb_count; cur_index++) {
> @@ -158,7 +161,7 @@ perform_restore(
> bytes_read += block_size + (mb_count << mbp->mb_blocklog);
> }
>
> - if (progress_since_warning)
> + if (mdrestore.progress_since_warning)
> putchar('\n');
>
> memset(block_buffer, 0, sb.sb_sectsize);
> @@ -197,15 +200,19 @@ main(
> int is_target_file;
> struct xfs_metablock mb;
>
> + mdrestore.show_progress = 0;
> + mdrestore.show_info = 0;
> + mdrestore.progress_since_warning = 0;
> +
> progname = basename(argv[0]);
>
> while ((c = getopt(argc, argv, "giV")) != EOF) {
> switch (c) {
> case 'g':
> - show_progress = 1;
> + mdrestore.show_progress = 1;
> break;
> case 'i':
> - show_info = 1;
> + mdrestore.show_info = 1;
> break;
> case 'V':
> printf("%s version %s\n", progname, VERSION);
> @@ -219,7 +226,7 @@ main(
> usage();
>
> /* show_info without a target is ok */
> - if (!show_info && argc - optind != 2)
> + if (!mdrestore.show_info && argc - optind != 2)
> usage();
>
> /*
> @@ -243,7 +250,7 @@ main(
> if (mb.mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
> fatal("specified file is not a metadata dump\n");
>
> - if (show_info) {
> + if (mdrestore.show_info) {
> if (mb.mb_info & XFS_METADUMP_INFO_FLAGS) {
> printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
> argv[optind],
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 16/24] mdrestore: Define and use struct mdrestore
2023-05-23 17:42 ` Darrick J. Wong
@ 2023-05-26 8:38 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-26 8:38 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 10:42:48 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:42PM +0530, Chandan Babu R wrote:
>> This commit collects all state tracking variables in a new "struct mdrestore"
>> structure.
>
> Same comment as patch 3.
I will add a commit description explaining the reason for creating a new
structure to collect all the global variables.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 17/24] mdrestore: Add open_device(), read_header() and show_info() functions
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (15 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 16/24] mdrestore: Define and use struct mdrestore Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:44 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 18/24] mdrestore: Introduce struct mdrestore_ops Chandan Babu R
` (6 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
This commit moves functionality associated with opening the target device,
reading metadump header information and printing information about the
metadump into their respective functions.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
mdrestore/xfs_mdrestore.c | 114 +++++++++++++++++++++++---------------
1 file changed, 68 insertions(+), 46 deletions(-)
diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
index de9175a08..8c847c5a3 100644
--- a/mdrestore/xfs_mdrestore.c
+++ b/mdrestore/xfs_mdrestore.c
@@ -40,8 +40,67 @@ print_progress(const char *fmt, ...)
mdrestore.progress_since_warning = 1;
}
+extern int platform_check_ismounted(char *, char *, struct stat *, int);
+
+static int
+open_device(
+ char *path,
+ bool *is_file)
+{
+ struct stat statbuf;
+ int open_flags;
+ int fd;
+
+ open_flags = O_RDWR;
+ *is_file = false;
+
+ if (stat(path, &statbuf) < 0) {
+ /* ok, assume it's a file and create it */
+ open_flags |= O_CREAT;
+ *is_file = true;
+ } else if (S_ISREG(statbuf.st_mode)) {
+ open_flags |= O_TRUNC;
+ *is_file = true;
+ } else {
+ /*
+ * check to make sure a filesystem isn't mounted on the device
+ */
+ if (platform_check_ismounted(path, NULL, &statbuf, 0))
+ fatal("a filesystem is mounted on target device \"%s\","
+ " cannot restore to a mounted filesystem.\n",
+ path);
+ }
+
+ fd = open(path, open_flags, 0644);
+ if (fd < 0)
+ fatal("couldn't open \"%s\"\n", path);
+
+ return fd;
+}
+
+static void read_header(struct xfs_metablock *mb, FILE *src_f)
+{
+ if (fread(mb, sizeof(*mb), 1, src_f) != 1)
+ fatal("error reading from metadump file\n");
+ if (mb->mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
+ fatal("specified file is not a metadata dump\n");
+}
+
+static void show_info(struct xfs_metablock *mb, const char *mdfile)
+{
+ if (mb->mb_info & XFS_METADUMP_INFO_FLAGS) {
+ printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
+ mdfile,
+ mb->mb_info & XFS_METADUMP_OBFUSCATED ? "":"not ",
+ mb->mb_info & XFS_METADUMP_DIRTYLOG ? "dirty":"clean",
+ mb->mb_info & XFS_METADUMP_FULLBLOCKS ? "full":"zeroed");
+ } else {
+ printf("%s: no informational flags present\n", mdfile);
+ }
+}
+
/*
- * perform_restore() -- do the actual work to restore the metadump
+ * restore() -- do the actual work to restore the metadump
*
* @src_f: A FILE pointer to the source metadump
* @dst_fd: the file descriptor for the target file
@@ -51,7 +110,7 @@ print_progress(const char *fmt, ...)
* src_f should be positioned just past a read the previously validated metablock
*/
static void
-perform_restore(
+restore(
FILE *src_f,
int dst_fd,
int is_target_file,
@@ -185,8 +244,6 @@ usage(void)
exit(1);
}
-extern int platform_check_ismounted(char *, char *, struct stat *, int);
-
int
main(
int argc,
@@ -195,9 +252,7 @@ main(
FILE *src_f;
int dst_fd;
int c;
- int open_flags;
- struct stat statbuf;
- int is_target_file;
+ bool is_target_file;
struct xfs_metablock mb;
mdrestore.show_progress = 0;
@@ -230,8 +285,8 @@ main(
usage();
/*
- * open source and test if this really is a dump. The first metadump block
- * will be passed to perform_restore() which will continue to read the
+ * open source and test if this really is a dump. The first metadump
+ * block will be passed to restore() which will continue to read the
* file from this point. This avoids rewind the stream, which causes
* restore to fail when source was being read from stdin.
*/
@@ -245,22 +300,10 @@ main(
fatal("cannot open source dump file\n");
}
- if (fread(&mb, sizeof(mb), 1, src_f) != 1)
- fatal("error reading from metadump file\n");
- if (mb.mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
- fatal("specified file is not a metadata dump\n");
+ read_header(&mb, src_f);
if (mdrestore.show_info) {
- if (mb.mb_info & XFS_METADUMP_INFO_FLAGS) {
- printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
- argv[optind],
- mb.mb_info & XFS_METADUMP_OBFUSCATED ? "":"not ",
- mb.mb_info & XFS_METADUMP_DIRTYLOG ? "dirty":"clean",
- mb.mb_info & XFS_METADUMP_FULLBLOCKS ? "full":"zeroed");
- } else {
- printf("%s: no informational flags present\n",
- argv[optind]);
- }
+ show_info(&mb, argv[optind]);
if (argc - optind == 1)
exit(0);
@@ -269,30 +312,9 @@ main(
optind++;
/* check and open target */
- open_flags = O_RDWR;
- is_target_file = 0;
- if (stat(argv[optind], &statbuf) < 0) {
- /* ok, assume it's a file and create it */
- open_flags |= O_CREAT;
- is_target_file = 1;
- } else if (S_ISREG(statbuf.st_mode)) {
- open_flags |= O_TRUNC;
- is_target_file = 1;
- } else {
- /*
- * check to make sure a filesystem isn't mounted on the device
- */
- if (platform_check_ismounted(argv[optind], NULL, &statbuf, 0))
- fatal("a filesystem is mounted on target device \"%s\","
- " cannot restore to a mounted filesystem.\n",
- argv[optind]);
- }
-
- dst_fd = open(argv[optind], open_flags, 0644);
- if (dst_fd < 0)
- fatal("couldn't open target \"%s\"\n", argv[optind]);
+ dst_fd = open_device(argv[optind], &is_target_file);
- perform_restore(src_f, dst_fd, is_target_file, &mb);
+ restore(src_f, dst_fd, is_target_file, &mb);
close(dst_fd);
if (src_f != stdin)
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 17/24] mdrestore: Add open_device(), read_header() and show_info() functions
2023-05-23 9:00 ` [PATCH 17/24] mdrestore: Add open_device(), read_header() and show_info() functions Chandan Babu R
@ 2023-05-23 17:44 ` Darrick J. Wong
2023-05-25 10:11 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:44 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:43PM +0530, Chandan Babu R wrote:
> This commit moves functionality associated with opening the target device,
> reading metadump header information and printing information about the
> metadump into their respective functions.
"No functional changes"?
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> mdrestore/xfs_mdrestore.c | 114 +++++++++++++++++++++++---------------
> 1 file changed, 68 insertions(+), 46 deletions(-)
>
> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> index de9175a08..8c847c5a3 100644
> --- a/mdrestore/xfs_mdrestore.c
> +++ b/mdrestore/xfs_mdrestore.c
> @@ -40,8 +40,67 @@ print_progress(const char *fmt, ...)
> mdrestore.progress_since_warning = 1;
> }
>
> +extern int platform_check_ismounted(char *, char *, struct stat *, int);
#include <libfrog/platform.h> ?
--D
> +
> +static int
> +open_device(
> + char *path,
> + bool *is_file)
> +{
> + struct stat statbuf;
> + int open_flags;
> + int fd;
> +
> + open_flags = O_RDWR;
> + *is_file = false;
> +
> + if (stat(path, &statbuf) < 0) {
> + /* ok, assume it's a file and create it */
> + open_flags |= O_CREAT;
> + *is_file = true;
> + } else if (S_ISREG(statbuf.st_mode)) {
> + open_flags |= O_TRUNC;
> + *is_file = true;
> + } else {
> + /*
> + * check to make sure a filesystem isn't mounted on the device
> + */
> + if (platform_check_ismounted(path, NULL, &statbuf, 0))
> + fatal("a filesystem is mounted on target device \"%s\","
> + " cannot restore to a mounted filesystem.\n",
> + path);
> + }
> +
> + fd = open(path, open_flags, 0644);
> + if (fd < 0)
> + fatal("couldn't open \"%s\"\n", path);
> +
> + return fd;
> +}
> +
> +static void read_header(struct xfs_metablock *mb, FILE *src_f)
> +{
> + if (fread(mb, sizeof(*mb), 1, src_f) != 1)
> + fatal("error reading from metadump file\n");
> + if (mb->mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
> + fatal("specified file is not a metadata dump\n");
> +}
> +
> +static void show_info(struct xfs_metablock *mb, const char *mdfile)
> +{
> + if (mb->mb_info & XFS_METADUMP_INFO_FLAGS) {
> + printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
> + mdfile,
> + mb->mb_info & XFS_METADUMP_OBFUSCATED ? "":"not ",
> + mb->mb_info & XFS_METADUMP_DIRTYLOG ? "dirty":"clean",
> + mb->mb_info & XFS_METADUMP_FULLBLOCKS ? "full":"zeroed");
> + } else {
> + printf("%s: no informational flags present\n", mdfile);
> + }
> +}
> +
> /*
> - * perform_restore() -- do the actual work to restore the metadump
> + * restore() -- do the actual work to restore the metadump
> *
> * @src_f: A FILE pointer to the source metadump
> * @dst_fd: the file descriptor for the target file
> @@ -51,7 +110,7 @@ print_progress(const char *fmt, ...)
> * src_f should be positioned just past a read the previously validated metablock
> */
> static void
> -perform_restore(
> +restore(
> FILE *src_f,
> int dst_fd,
> int is_target_file,
> @@ -185,8 +244,6 @@ usage(void)
> exit(1);
> }
>
> -extern int platform_check_ismounted(char *, char *, struct stat *, int);
> -
> int
> main(
> int argc,
> @@ -195,9 +252,7 @@ main(
> FILE *src_f;
> int dst_fd;
> int c;
> - int open_flags;
> - struct stat statbuf;
> - int is_target_file;
> + bool is_target_file;
> struct xfs_metablock mb;
>
> mdrestore.show_progress = 0;
> @@ -230,8 +285,8 @@ main(
> usage();
>
> /*
> - * open source and test if this really is a dump. The first metadump block
> - * will be passed to perform_restore() which will continue to read the
> + * open source and test if this really is a dump. The first metadump
> + * block will be passed to restore() which will continue to read the
> * file from this point. This avoids rewind the stream, which causes
> * restore to fail when source was being read from stdin.
> */
> @@ -245,22 +300,10 @@ main(
> fatal("cannot open source dump file\n");
> }
>
> - if (fread(&mb, sizeof(mb), 1, src_f) != 1)
> - fatal("error reading from metadump file\n");
> - if (mb.mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
> - fatal("specified file is not a metadata dump\n");
> + read_header(&mb, src_f);
>
> if (mdrestore.show_info) {
> - if (mb.mb_info & XFS_METADUMP_INFO_FLAGS) {
> - printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
> - argv[optind],
> - mb.mb_info & XFS_METADUMP_OBFUSCATED ? "":"not ",
> - mb.mb_info & XFS_METADUMP_DIRTYLOG ? "dirty":"clean",
> - mb.mb_info & XFS_METADUMP_FULLBLOCKS ? "full":"zeroed");
> - } else {
> - printf("%s: no informational flags present\n",
> - argv[optind]);
> - }
> + show_info(&mb, argv[optind]);
>
> if (argc - optind == 1)
> exit(0);
> @@ -269,30 +312,9 @@ main(
> optind++;
>
> /* check and open target */
> - open_flags = O_RDWR;
> - is_target_file = 0;
> - if (stat(argv[optind], &statbuf) < 0) {
> - /* ok, assume it's a file and create it */
> - open_flags |= O_CREAT;
> - is_target_file = 1;
> - } else if (S_ISREG(statbuf.st_mode)) {
> - open_flags |= O_TRUNC;
> - is_target_file = 1;
> - } else {
> - /*
> - * check to make sure a filesystem isn't mounted on the device
> - */
> - if (platform_check_ismounted(argv[optind], NULL, &statbuf, 0))
> - fatal("a filesystem is mounted on target device \"%s\","
> - " cannot restore to a mounted filesystem.\n",
> - argv[optind]);
> - }
> -
> - dst_fd = open(argv[optind], open_flags, 0644);
> - if (dst_fd < 0)
> - fatal("couldn't open target \"%s\"\n", argv[optind]);
> + dst_fd = open_device(argv[optind], &is_target_file);
>
> - perform_restore(src_f, dst_fd, is_target_file, &mb);
> + restore(src_f, dst_fd, is_target_file, &mb);
>
> close(dst_fd);
> if (src_f != stdin)
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 17/24] mdrestore: Add open_device(), read_header() and show_info() functions
2023-05-23 17:44 ` Darrick J. Wong
@ 2023-05-25 10:11 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 10:11 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 10:44:15 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:43PM +0530, Chandan Babu R wrote:
>> This commit moves functionality associated with opening the target device,
>> reading metadump header information and printing information about the
>> metadump into their respective functions.
>
> "No functional changes"?
>
Yes, this commit does not include any functional changes. I will update the
commit message to reflect this fact.
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> mdrestore/xfs_mdrestore.c | 114 +++++++++++++++++++++++---------------
>> 1 file changed, 68 insertions(+), 46 deletions(-)
>>
>> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
>> index de9175a08..8c847c5a3 100644
>> --- a/mdrestore/xfs_mdrestore.c
>> +++ b/mdrestore/xfs_mdrestore.c
>> @@ -40,8 +40,67 @@ print_progress(const char *fmt, ...)
>> mdrestore.progress_since_warning = 1;
>> }
>>
>> +extern int platform_check_ismounted(char *, char *, struct stat *, int);
>
> #include <libfrog/platform.h> ?
I will include the above mentioned header file.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 18/24] mdrestore: Introduce struct mdrestore_ops
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (16 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 17/24] mdrestore: Add open_device(), read_header() and show_info() functions Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:44 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 19/24] mdrestore: Introduce mdrestore v1 operations Chandan Babu R
` (5 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
We will need two sets of functions to work with two versions of metadump
formats. This commit adds the definition for 'struct mdrestore_ops' to hold
pointers to version specific mdrestore functions.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
mdrestore/xfs_mdrestore.c | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
index 8c847c5a3..895e5cdab 100644
--- a/mdrestore/xfs_mdrestore.c
+++ b/mdrestore/xfs_mdrestore.c
@@ -7,10 +7,18 @@
#include "libxfs.h"
#include "xfs_metadump.h"
+struct mdrestore_ops {
+ void (*read_header)(void *header, FILE *mdfp);
+ void (*show_info)(void *header, const char *mdfile);
+ void (*restore)(void *header, FILE *mdfp, int data_fd,
+ bool is_target_file);
+};
+
static struct mdrestore {
- int show_progress;
- int show_info;
- int progress_since_warning;
+ struct mdrestore_ops *mdrops;
+ int show_progress;
+ int show_info;
+ int progress_since_warning;
} mdrestore;
static void
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 18/24] mdrestore: Introduce struct mdrestore_ops
2023-05-23 9:00 ` [PATCH 18/24] mdrestore: Introduce struct mdrestore_ops Chandan Babu R
@ 2023-05-23 17:44 ` Darrick J. Wong
2023-05-25 10:34 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:44 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:44PM +0530, Chandan Babu R wrote:
> We will need two sets of functions to work with two versions of metadump
> formats. This commit adds the definition for 'struct mdrestore_ops' to hold
> pointers to version specific mdrestore functions.
>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
Looks ok, though I think those three int fields are really bool, right?
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
--D
> ---
> mdrestore/xfs_mdrestore.c | 14 +++++++++++---
> 1 file changed, 11 insertions(+), 3 deletions(-)
>
> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> index 8c847c5a3..895e5cdab 100644
> --- a/mdrestore/xfs_mdrestore.c
> +++ b/mdrestore/xfs_mdrestore.c
> @@ -7,10 +7,18 @@
> #include "libxfs.h"
> #include "xfs_metadump.h"
>
> +struct mdrestore_ops {
> + void (*read_header)(void *header, FILE *mdfp);
> + void (*show_info)(void *header, const char *mdfile);
> + void (*restore)(void *header, FILE *mdfp, int data_fd,
> + bool is_target_file);
> +};
> +
> static struct mdrestore {
> - int show_progress;
> - int show_info;
> - int progress_since_warning;
> + struct mdrestore_ops *mdrops;
> + int show_progress;
> + int show_info;
> + int progress_since_warning;
> } mdrestore;
>
> static void
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 18/24] mdrestore: Introduce struct mdrestore_ops
2023-05-23 17:44 ` Darrick J. Wong
@ 2023-05-25 10:34 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 10:34 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 10:44:49 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:44PM +0530, Chandan Babu R wrote:
>> We will need two sets of functions to work with two versions of metadump
>> formats. This commit adds the definition for 'struct mdrestore_ops' to hold
>> pointers to version specific mdrestore functions.
>>
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>
> Looks ok, though I think those three int fields are really bool, right?
>
Yes, you are right. I will change the data type of these variables.
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
>
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 19/24] mdrestore: Introduce mdrestore v1 operations
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (17 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 18/24] mdrestore: Introduce struct mdrestore_ops Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 17:48 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 20/24] mdrestore: Detect metadump version from metadump image Chandan Babu R
` (4 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
In order to indicate the version of metadump files that they can work with,
this commit renames read_header(), show_info() and restore() functions to
read_header_v1(), show_info_v1() and restore_v1() respectively.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
mdrestore/xfs_mdrestore.c | 76 ++++++++++++++++++++++-----------------
1 file changed, 43 insertions(+), 33 deletions(-)
diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
index 895e5cdab..5ec1a47b0 100644
--- a/mdrestore/xfs_mdrestore.c
+++ b/mdrestore/xfs_mdrestore.c
@@ -86,16 +86,26 @@ open_device(
return fd;
}
-static void read_header(struct xfs_metablock *mb, FILE *src_f)
+static void
+read_header_v1(
+ void *header,
+ FILE *mdfp)
{
- if (fread(mb, sizeof(*mb), 1, src_f) != 1)
+ struct xfs_metablock *mb = header;
+
+ if (fread(mb, sizeof(*mb), 1, mdfp) != 1)
fatal("error reading from metadump file\n");
if (mb->mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
fatal("specified file is not a metadata dump\n");
}
-static void show_info(struct xfs_metablock *mb, const char *mdfile)
+static void
+show_info_v1(
+ void *header,
+ const char *mdfile)
{
+ struct xfs_metablock *mb = header;
+
if (mb->mb_info & XFS_METADUMP_INFO_FLAGS) {
printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
mdfile,
@@ -107,24 +117,15 @@ static void show_info(struct xfs_metablock *mb, const char *mdfile)
}
}
-/*
- * restore() -- do the actual work to restore the metadump
- *
- * @src_f: A FILE pointer to the source metadump
- * @dst_fd: the file descriptor for the target file
- * @is_target_file: designates whether the target is a regular file
- * @mbp: pointer to metadump's first xfs_metablock, read and verified by the caller
- *
- * src_f should be positioned just past a read the previously validated metablock
- */
static void
-restore(
- FILE *src_f,
- int dst_fd,
- int is_target_file,
- const struct xfs_metablock *mbp)
+restore_v1(
+ void *header,
+ FILE *mdfp,
+ int data_fd,
+ bool is_target_file)
{
- struct xfs_metablock *metablock; /* header + index + blocks */
+ struct xfs_metablock *mbp = header;
+ struct xfs_metablock *metablock;
__be64 *block_index;
char *block_buffer;
int block_size;
@@ -148,14 +149,15 @@ restore(
block_index = (__be64 *)((char *)metablock + sizeof(xfs_metablock_t));
block_buffer = (char *)metablock + block_size;
- if (fread(block_index, block_size - sizeof(struct xfs_metablock), 1, src_f) != 1)
+ if (fread(block_index, block_size - sizeof(struct xfs_metablock), 1,
+ mdfp) != 1)
fatal("error reading from metadump file\n");
if (block_index[0] != 0)
fatal("first block is not the primary superblock\n");
- if (fread(block_buffer, mb_count << mbp->mb_blocklog, 1, src_f) != 1)
+ if (fread(block_buffer, mb_count << mbp->mb_blocklog, 1, mdfp) != 1)
fatal("error reading from metadump file\n");
libxfs_sb_from_disk(&sb, (struct xfs_dsb *)block_buffer);
@@ -178,7 +180,7 @@ restore(
if (is_target_file) {
/* ensure regular files are correctly sized */
- if (ftruncate(dst_fd, sb.sb_dblocks * sb.sb_blocksize))
+ if (ftruncate(data_fd, sb.sb_dblocks * sb.sb_blocksize))
fatal("cannot set filesystem image size: %s\n",
strerror(errno));
} else {
@@ -188,7 +190,7 @@ restore(
off64_t off;
off = sb.sb_dblocks * sb.sb_blocksize - sizeof(lb);
- if (pwrite(dst_fd, lb, sizeof(lb), off) < 0)
+ if (pwrite(data_fd, lb, sizeof(lb), off) < 0)
fatal("failed to write last block, is target too "
"small? (error: %s)\n", strerror(errno));
}
@@ -201,7 +203,7 @@ restore(
print_progress("%lld MB read", bytes_read >> 20);
for (cur_index = 0; cur_index < mb_count; cur_index++) {
- if (pwrite(dst_fd, &block_buffer[cur_index <<
+ if (pwrite(data_fd, &block_buffer[cur_index <<
mbp->mb_blocklog], block_size,
be64_to_cpu(block_index[cur_index]) <<
BBSHIFT) < 0)
@@ -212,7 +214,7 @@ restore(
if (mb_count < max_indices)
break;
- if (fread(metablock, block_size, 1, src_f) != 1)
+ if (fread(metablock, block_size, 1, mdfp) != 1)
fatal("error reading from metadump file\n");
mb_count = be16_to_cpu(metablock->mb_count);
@@ -222,7 +224,7 @@ restore(
fatal("bad block count: %u\n", mb_count);
if (fread(block_buffer, mb_count << mbp->mb_blocklog,
- 1, src_f) != 1)
+ 1, mdfp) != 1)
fatal("error reading from metadump file\n");
bytes_read += block_size + (mb_count << mbp->mb_blocklog);
@@ -239,12 +241,18 @@ restore(
offsetof(struct xfs_sb, sb_crc));
}
- if (pwrite(dst_fd, block_buffer, sb.sb_sectsize, 0) < 0)
+ if (pwrite(data_fd, block_buffer, sb.sb_sectsize, 0) < 0)
fatal("error writing primary superblock: %s\n", strerror(errno));
free(metablock);
}
+static struct mdrestore_ops mdrestore_ops_v1 = {
+ .read_header = read_header_v1,
+ .show_info = show_info_v1,
+ .restore = restore_v1,
+};
+
static void
usage(void)
{
@@ -294,9 +302,9 @@ main(
/*
* open source and test if this really is a dump. The first metadump
- * block will be passed to restore() which will continue to read the
- * file from this point. This avoids rewind the stream, which causes
- * restore to fail when source was being read from stdin.
+ * block will be passed to mdrestore_ops->restore() which will continue
+ * to read the file from this point. This avoids rewind the stream,
+ * which causes restore to fail when source was being read from stdin.
*/
if (strcmp(argv[optind], "-") == 0) {
src_f = stdin;
@@ -308,10 +316,12 @@ main(
fatal("cannot open source dump file\n");
}
- read_header(&mb, src_f);
+ mdrestore.mdrops = &mdrestore_ops_v1;
+
+ mdrestore.mdrops->read_header(&mb, src_f);
if (mdrestore.show_info) {
- show_info(&mb, argv[optind]);
+ mdrestore.mdrops->show_info(&mb, argv[optind]);
if (argc - optind == 1)
exit(0);
@@ -322,7 +332,7 @@ main(
/* check and open target */
dst_fd = open_device(argv[optind], &is_target_file);
- restore(src_f, dst_fd, is_target_file, &mb);
+ mdrestore.mdrops->restore(&mb, src_f, dst_fd, is_target_file);
close(dst_fd);
if (src_f != stdin)
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 19/24] mdrestore: Introduce mdrestore v1 operations
2023-05-23 9:00 ` [PATCH 19/24] mdrestore: Introduce mdrestore v1 operations Chandan Babu R
@ 2023-05-23 17:48 ` Darrick J. Wong
2023-05-25 10:39 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 17:48 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:45PM +0530, Chandan Babu R wrote:
> In order to indicate the version of metadump files that they can work with,
> this commit renames read_header(), show_info() and restore() functions to
> read_header_v1(), show_info_v1() and restore_v1() respectively.
>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> mdrestore/xfs_mdrestore.c | 76 ++++++++++++++++++++++-----------------
> 1 file changed, 43 insertions(+), 33 deletions(-)
>
> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> index 895e5cdab..5ec1a47b0 100644
> --- a/mdrestore/xfs_mdrestore.c
> +++ b/mdrestore/xfs_mdrestore.c
> @@ -86,16 +86,26 @@ open_device(
> return fd;
> }
>
> -static void read_header(struct xfs_metablock *mb, FILE *src_f)
> +static void
> +read_header_v1(
> + void *header,
> + FILE *mdfp)
> {
> - if (fread(mb, sizeof(*mb), 1, src_f) != 1)
> + struct xfs_metablock *mb = header;
> +
> + if (fread(mb, sizeof(*mb), 1, mdfp) != 1)
> fatal("error reading from metadump file\n");
> if (mb->mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
> fatal("specified file is not a metadata dump\n");
> }
>
> -static void show_info(struct xfs_metablock *mb, const char *mdfile)
> +static void
> +show_info_v1(
> + void *header,
> + const char *mdfile)
> {
> + struct xfs_metablock *mb = header;
> +
> if (mb->mb_info & XFS_METADUMP_INFO_FLAGS) {
> printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
> mdfile,
> @@ -107,24 +117,15 @@ static void show_info(struct xfs_metablock *mb, const char *mdfile)
> }
> }
>
> -/*
> - * restore() -- do the actual work to restore the metadump
> - *
> - * @src_f: A FILE pointer to the source metadump
> - * @dst_fd: the file descriptor for the target file
> - * @is_target_file: designates whether the target is a regular file
> - * @mbp: pointer to metadump's first xfs_metablock, read and verified by the caller
> - *
> - * src_f should be positioned just past a read the previously validated metablock
> - */
> static void
> -restore(
> - FILE *src_f,
> - int dst_fd,
> - int is_target_file,
> - const struct xfs_metablock *mbp)
> +restore_v1(
> + void *header,
> + FILE *mdfp,
> + int data_fd,
Umm. mdfp == "FILE * stream for reading the source" and "data_fd" == "fd
pointing to data device for writing the filesystem"?
I think I'd prefer md_fp and ddev_fd...
> + bool is_target_file)
> {
> - struct xfs_metablock *metablock; /* header + index + blocks */
> + struct xfs_metablock *mbp = header;
> + struct xfs_metablock *metablock;
> __be64 *block_index;
> char *block_buffer;
> int block_size;
> @@ -148,14 +149,15 @@ restore(
> block_index = (__be64 *)((char *)metablock + sizeof(xfs_metablock_t));
> block_buffer = (char *)metablock + block_size;
>
> - if (fread(block_index, block_size - sizeof(struct xfs_metablock), 1, src_f) != 1)
> + if (fread(block_index, block_size - sizeof(struct xfs_metablock), 1,
> + mdfp) != 1)
> fatal("error reading from metadump file\n");
>
> if (block_index[0] != 0)
> fatal("first block is not the primary superblock\n");
>
>
> - if (fread(block_buffer, mb_count << mbp->mb_blocklog, 1, src_f) != 1)
> + if (fread(block_buffer, mb_count << mbp->mb_blocklog, 1, mdfp) != 1)
> fatal("error reading from metadump file\n");
>
> libxfs_sb_from_disk(&sb, (struct xfs_dsb *)block_buffer);
> @@ -178,7 +180,7 @@ restore(
> if (is_target_file) {
> /* ensure regular files are correctly sized */
>
> - if (ftruncate(dst_fd, sb.sb_dblocks * sb.sb_blocksize))
> + if (ftruncate(data_fd, sb.sb_dblocks * sb.sb_blocksize))
> fatal("cannot set filesystem image size: %s\n",
> strerror(errno));
> } else {
> @@ -188,7 +190,7 @@ restore(
> off64_t off;
>
> off = sb.sb_dblocks * sb.sb_blocksize - sizeof(lb);
> - if (pwrite(dst_fd, lb, sizeof(lb), off) < 0)
> + if (pwrite(data_fd, lb, sizeof(lb), off) < 0)
> fatal("failed to write last block, is target too "
> "small? (error: %s)\n", strerror(errno));
> }
> @@ -201,7 +203,7 @@ restore(
> print_progress("%lld MB read", bytes_read >> 20);
>
> for (cur_index = 0; cur_index < mb_count; cur_index++) {
> - if (pwrite(dst_fd, &block_buffer[cur_index <<
> + if (pwrite(data_fd, &block_buffer[cur_index <<
> mbp->mb_blocklog], block_size,
> be64_to_cpu(block_index[cur_index]) <<
> BBSHIFT) < 0)
> @@ -212,7 +214,7 @@ restore(
> if (mb_count < max_indices)
> break;
>
> - if (fread(metablock, block_size, 1, src_f) != 1)
> + if (fread(metablock, block_size, 1, mdfp) != 1)
> fatal("error reading from metadump file\n");
>
> mb_count = be16_to_cpu(metablock->mb_count);
> @@ -222,7 +224,7 @@ restore(
> fatal("bad block count: %u\n", mb_count);
>
> if (fread(block_buffer, mb_count << mbp->mb_blocklog,
> - 1, src_f) != 1)
> + 1, mdfp) != 1)
> fatal("error reading from metadump file\n");
>
> bytes_read += block_size + (mb_count << mbp->mb_blocklog);
> @@ -239,12 +241,18 @@ restore(
> offsetof(struct xfs_sb, sb_crc));
> }
>
> - if (pwrite(dst_fd, block_buffer, sb.sb_sectsize, 0) < 0)
> + if (pwrite(data_fd, block_buffer, sb.sb_sectsize, 0) < 0)
> fatal("error writing primary superblock: %s\n", strerror(errno));
>
> free(metablock);
> }
>
> +static struct mdrestore_ops mdrestore_ops_v1 = {
> + .read_header = read_header_v1,
> + .show_info = show_info_v1,
> + .restore = restore_v1,
> +};
> +
> static void
> usage(void)
> {
> @@ -294,9 +302,9 @@ main(
>
> /*
> * open source and test if this really is a dump. The first metadump
> - * block will be passed to restore() which will continue to read the
> - * file from this point. This avoids rewind the stream, which causes
> - * restore to fail when source was being read from stdin.
> + * block will be passed to mdrestore_ops->restore() which will continue
> + * to read the file from this point. This avoids rewind the stream,
> + * which causes restore to fail when source was being read from stdin.
> */
> if (strcmp(argv[optind], "-") == 0) {
> src_f = stdin;
> @@ -308,10 +316,12 @@ main(
> fatal("cannot open source dump file\n");
> }
>
> - read_header(&mb, src_f);
> + mdrestore.mdrops = &mdrestore_ops_v1;
> +
> + mdrestore.mdrops->read_header(&mb, src_f);
Starting to wonder if it's a mistake to continue to declare a struct
xfs_metablock directly on the stack in main() but I guess I'll see what
happens when you introduce the v2 code.
--D
>
> if (mdrestore.show_info) {
> - show_info(&mb, argv[optind]);
> + mdrestore.mdrops->show_info(&mb, argv[optind]);
>
> if (argc - optind == 1)
> exit(0);
> @@ -322,7 +332,7 @@ main(
> /* check and open target */
> dst_fd = open_device(argv[optind], &is_target_file);
>
> - restore(src_f, dst_fd, is_target_file, &mb);
> + mdrestore.mdrops->restore(&mb, src_f, dst_fd, is_target_file);
>
> close(dst_fd);
> if (src_f != stdin)
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 19/24] mdrestore: Introduce mdrestore v1 operations
2023-05-23 17:48 ` Darrick J. Wong
@ 2023-05-25 10:39 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 10:39 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 10:48:11 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:45PM +0530, Chandan Babu R wrote:
>> In order to indicate the version of metadump files that they can work with,
>> this commit renames read_header(), show_info() and restore() functions to
>> read_header_v1(), show_info_v1() and restore_v1() respectively.
>>
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> mdrestore/xfs_mdrestore.c | 76 ++++++++++++++++++++++-----------------
>> 1 file changed, 43 insertions(+), 33 deletions(-)
>>
>> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
>> index 895e5cdab..5ec1a47b0 100644
>> --- a/mdrestore/xfs_mdrestore.c
>> +++ b/mdrestore/xfs_mdrestore.c
>> @@ -86,16 +86,26 @@ open_device(
>> return fd;
>> }
>>
>> -static void read_header(struct xfs_metablock *mb, FILE *src_f)
>> +static void
>> +read_header_v1(
>> + void *header,
>> + FILE *mdfp)
>> {
>> - if (fread(mb, sizeof(*mb), 1, src_f) != 1)
>> + struct xfs_metablock *mb = header;
>> +
>> + if (fread(mb, sizeof(*mb), 1, mdfp) != 1)
>> fatal("error reading from metadump file\n");
>> if (mb->mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
>> fatal("specified file is not a metadata dump\n");
>> }
>>
>> -static void show_info(struct xfs_metablock *mb, const char *mdfile)
>> +static void
>> +show_info_v1(
>> + void *header,
>> + const char *mdfile)
>> {
>> + struct xfs_metablock *mb = header;
>> +
>> if (mb->mb_info & XFS_METADUMP_INFO_FLAGS) {
>> printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
>> mdfile,
>> @@ -107,24 +117,15 @@ static void show_info(struct xfs_metablock *mb, const char *mdfile)
>> }
>> }
>>
>> -/*
>> - * restore() -- do the actual work to restore the metadump
>> - *
>> - * @src_f: A FILE pointer to the source metadump
>> - * @dst_fd: the file descriptor for the target file
>> - * @is_target_file: designates whether the target is a regular file
>> - * @mbp: pointer to metadump's first xfs_metablock, read and verified by the caller
>> - *
>> - * src_f should be positioned just past a read the previously validated metablock
>> - */
>> static void
>> -restore(
>> - FILE *src_f,
>> - int dst_fd,
>> - int is_target_file,
>> - const struct xfs_metablock *mbp)
>> +restore_v1(
>> + void *header,
>> + FILE *mdfp,
>> + int data_fd,
>
> Umm. mdfp == "FILE * stream for reading the source" and "data_fd" == "fd
> pointing to data device for writing the filesystem"?
>
> I think I'd prefer md_fp and ddev_fd...
>
Ok. This is more readable that what I had written.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 20/24] mdrestore: Detect metadump version from metadump image
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (18 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 19/24] mdrestore: Introduce mdrestore v1 operations Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 18:11 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 21/24] mdrestore: Extract target device size verification into a function Chandan Babu R
` (3 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
mdrestore/xfs_mdrestore.c | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
index 5ec1a47b0..52081a6ca 100644
--- a/mdrestore/xfs_mdrestore.c
+++ b/mdrestore/xfs_mdrestore.c
@@ -8,7 +8,7 @@
#include "xfs_metadump.h"
struct mdrestore_ops {
- void (*read_header)(void *header, FILE *mdfp);
+ int (*read_header)(void *header, FILE *mdfp);
void (*show_info)(void *header, const char *mdfile);
void (*restore)(void *header, FILE *mdfp, int data_fd,
bool is_target_file);
@@ -86,7 +86,7 @@ open_device(
return fd;
}
-static void
+static int
read_header_v1(
void *header,
FILE *mdfp)
@@ -96,7 +96,9 @@ read_header_v1(
if (fread(mb, sizeof(*mb), 1, mdfp) != 1)
fatal("error reading from metadump file\n");
if (mb->mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
- fatal("specified file is not a metadata dump\n");
+ return -1;
+
+ return 0;
}
static void
@@ -316,9 +318,10 @@ main(
fatal("cannot open source dump file\n");
}
- mdrestore.mdrops = &mdrestore_ops_v1;
-
- mdrestore.mdrops->read_header(&mb, src_f);
+ if (mdrestore_ops_v1.read_header(&mb, src_f) == 0)
+ mdrestore.mdrops = &mdrestore_ops_v1;
+ else
+ fatal("Invalid metadump format\n");
if (mdrestore.show_info) {
mdrestore.mdrops->show_info(&mb, argv[optind]);
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 20/24] mdrestore: Detect metadump version from metadump image
2023-05-23 9:00 ` [PATCH 20/24] mdrestore: Detect metadump version from metadump image Chandan Babu R
@ 2023-05-23 18:11 ` Darrick J. Wong
0 siblings, 0 replies; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 18:11 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:46PM +0530, Chandan Babu R wrote:
I'll have more to say about this in patch 22.
--D
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> mdrestore/xfs_mdrestore.c | 15 +++++++++------
> 1 file changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> index 5ec1a47b0..52081a6ca 100644
> --- a/mdrestore/xfs_mdrestore.c
> +++ b/mdrestore/xfs_mdrestore.c
> @@ -8,7 +8,7 @@
> #include "xfs_metadump.h"
>
> struct mdrestore_ops {
> - void (*read_header)(void *header, FILE *mdfp);
> + int (*read_header)(void *header, FILE *mdfp);
> void (*show_info)(void *header, const char *mdfile);
> void (*restore)(void *header, FILE *mdfp, int data_fd,
> bool is_target_file);
> @@ -86,7 +86,7 @@ open_device(
> return fd;
> }
>
> -static void
> +static int
> read_header_v1(
> void *header,
> FILE *mdfp)
> @@ -96,7 +96,9 @@ read_header_v1(
> if (fread(mb, sizeof(*mb), 1, mdfp) != 1)
> fatal("error reading from metadump file\n");
> if (mb->mb_magic != cpu_to_be32(XFS_MD_MAGIC_V1))
> - fatal("specified file is not a metadata dump\n");
> + return -1;
> +
> + return 0;
> }
>
> static void
> @@ -316,9 +318,10 @@ main(
> fatal("cannot open source dump file\n");
> }
>
> - mdrestore.mdrops = &mdrestore_ops_v1;
> -
> - mdrestore.mdrops->read_header(&mb, src_f);
> + if (mdrestore_ops_v1.read_header(&mb, src_f) == 0)
> + mdrestore.mdrops = &mdrestore_ops_v1;
> + else
> + fatal("Invalid metadump format\n");
>
> if (mdrestore.show_info) {
> mdrestore.mdrops->show_info(&mb, argv[optind]);
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 21/24] mdrestore: Extract target device size verification into a function
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (19 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 20/24] mdrestore: Detect metadump version from metadump image Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 18:07 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 22/24] mdrestore: Define mdrestore ops for v2 format Chandan Babu R
` (2 subsequent siblings)
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
mdrestore/xfs_mdrestore.c | 43 +++++++++++++++++++++++----------------
1 file changed, 26 insertions(+), 17 deletions(-)
diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
index 52081a6ca..615ecdc77 100644
--- a/mdrestore/xfs_mdrestore.c
+++ b/mdrestore/xfs_mdrestore.c
@@ -86,6 +86,30 @@ open_device(
return fd;
}
+static void
+verify_device_size(
+ int dev_fd,
+ bool is_file,
+ xfs_rfsblock_t nr_blocks,
+ uint32_t blocksize)
+{
+ if (is_file) {
+ /* ensure regular files are correctly sized */
+ if (ftruncate(dev_fd, nr_blocks * blocksize))
+ fatal("cannot set filesystem image size: %s\n",
+ strerror(errno));
+ } else {
+ /* ensure device is sufficiently large enough */
+ char lb[XFS_MAX_SECTORSIZE] = { 0 };
+ off64_t off;
+
+ off = nr_blocks * blocksize - sizeof(lb);
+ if (pwrite(dev_fd, lb, sizeof(lb), off) < 0)
+ fatal("failed to write last block, is target too "
+ "small? (error: %s)\n", strerror(errno));
+ }
+}
+
static int
read_header_v1(
void *header,
@@ -179,23 +203,8 @@ restore_v1(
((struct xfs_dsb*)block_buffer)->sb_inprogress = 1;
- if (is_target_file) {
- /* ensure regular files are correctly sized */
-
- if (ftruncate(data_fd, sb.sb_dblocks * sb.sb_blocksize))
- fatal("cannot set filesystem image size: %s\n",
- strerror(errno));
- } else {
- /* ensure device is sufficiently large enough */
-
- char lb[XFS_MAX_SECTORSIZE] = { 0 };
- off64_t off;
-
- off = sb.sb_dblocks * sb.sb_blocksize - sizeof(lb);
- if (pwrite(data_fd, lb, sizeof(lb), off) < 0)
- fatal("failed to write last block, is target too "
- "small? (error: %s)\n", strerror(errno));
- }
+ verify_device_size(data_fd, is_target_file, sb.sb_dblocks,
+ sb.sb_blocksize);
bytes_read = 0;
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 21/24] mdrestore: Extract target device size verification into a function
2023-05-23 9:00 ` [PATCH 21/24] mdrestore: Extract target device size verification into a function Chandan Babu R
@ 2023-05-23 18:07 ` Darrick J. Wong
2023-05-25 12:02 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 18:07 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
"No functional changes" ?
With a better commit message,
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
--D
On Tue, May 23, 2023 at 02:30:47PM +0530, Chandan Babu R wrote:
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> mdrestore/xfs_mdrestore.c | 43 +++++++++++++++++++++++----------------
> 1 file changed, 26 insertions(+), 17 deletions(-)
>
> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> index 52081a6ca..615ecdc77 100644
> --- a/mdrestore/xfs_mdrestore.c
> +++ b/mdrestore/xfs_mdrestore.c
> @@ -86,6 +86,30 @@ open_device(
> return fd;
> }
>
> +static void
> +verify_device_size(
> + int dev_fd,
> + bool is_file,
> + xfs_rfsblock_t nr_blocks,
> + uint32_t blocksize)
> +{
> + if (is_file) {
> + /* ensure regular files are correctly sized */
> + if (ftruncate(dev_fd, nr_blocks * blocksize))
> + fatal("cannot set filesystem image size: %s\n",
> + strerror(errno));
> + } else {
> + /* ensure device is sufficiently large enough */
> + char lb[XFS_MAX_SECTORSIZE] = { 0 };
> + off64_t off;
> +
> + off = nr_blocks * blocksize - sizeof(lb);
> + if (pwrite(dev_fd, lb, sizeof(lb), off) < 0)
> + fatal("failed to write last block, is target too "
> + "small? (error: %s)\n", strerror(errno));
> + }
> +}
> +
> static int
> read_header_v1(
> void *header,
> @@ -179,23 +203,8 @@ restore_v1(
>
> ((struct xfs_dsb*)block_buffer)->sb_inprogress = 1;
>
> - if (is_target_file) {
> - /* ensure regular files are correctly sized */
> -
> - if (ftruncate(data_fd, sb.sb_dblocks * sb.sb_blocksize))
> - fatal("cannot set filesystem image size: %s\n",
> - strerror(errno));
> - } else {
> - /* ensure device is sufficiently large enough */
> -
> - char lb[XFS_MAX_SECTORSIZE] = { 0 };
> - off64_t off;
> -
> - off = sb.sb_dblocks * sb.sb_blocksize - sizeof(lb);
> - if (pwrite(data_fd, lb, sizeof(lb), off) < 0)
> - fatal("failed to write last block, is target too "
> - "small? (error: %s)\n", strerror(errno));
> - }
> + verify_device_size(data_fd, is_target_file, sb.sb_dblocks,
> + sb.sb_blocksize);
>
> bytes_read = 0;
>
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 21/24] mdrestore: Extract target device size verification into a function
2023-05-23 18:07 ` Darrick J. Wong
@ 2023-05-25 12:02 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 12:02 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 11:07:01 AM -0700, Darrick J. Wong wrote:
> "No functional changes" ?
>
I will add a commit description which will also indicate that there were no
functional changes made in this patch.
> With a better commit message,
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 22/24] mdrestore: Define mdrestore ops for v2 format
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (20 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 21/24] mdrestore: Extract target device size verification into a function Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 18:06 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 23/24] mdrestore: Add support for passing log device as an argument Chandan Babu R
2023-05-23 9:00 ` [PATCH 24/24] xfs_mdrestore.8: Add description for the newly introduced -l option Chandan Babu R
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
This commit adds functionality to restore metadump stored in v2 format.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
mdrestore/xfs_mdrestore.c | 209 +++++++++++++++++++++++++++++++++++---
1 file changed, 194 insertions(+), 15 deletions(-)
diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
index 615ecdc77..9e06d37dc 100644
--- a/mdrestore/xfs_mdrestore.c
+++ b/mdrestore/xfs_mdrestore.c
@@ -11,7 +11,8 @@ struct mdrestore_ops {
int (*read_header)(void *header, FILE *mdfp);
void (*show_info)(void *header, const char *mdfile);
void (*restore)(void *header, FILE *mdfp, int data_fd,
- bool is_target_file);
+ bool is_data_target_file, int log_fd,
+ bool is_log_target_file);
};
static struct mdrestore {
@@ -148,7 +149,9 @@ restore_v1(
void *header,
FILE *mdfp,
int data_fd,
- bool is_target_file)
+ bool is_data_target_file,
+ int log_fd,
+ bool is_log_target_file)
{
struct xfs_metablock *mbp = header;
struct xfs_metablock *metablock;
@@ -203,7 +206,7 @@ restore_v1(
((struct xfs_dsb*)block_buffer)->sb_inprogress = 1;
- verify_device_size(data_fd, is_target_file, sb.sb_dblocks,
+ verify_device_size(data_fd, is_data_target_file, sb.sb_dblocks,
sb.sb_blocksize);
bytes_read = 0;
@@ -264,6 +267,163 @@ static struct mdrestore_ops mdrestore_ops_v1 = {
.restore = restore_v1,
};
+static int
+read_header_v2(
+ void *header,
+ FILE *mdfp)
+{
+ struct xfs_metadump_header *xmh = header;
+
+ rewind(mdfp);
+
+ if (fread(xmh, sizeof(*xmh), 1, mdfp) != 1)
+ fatal("error reading from metadump file\n");
+ if (xmh->xmh_magic != cpu_to_be32(XFS_MD_MAGIC_V2))
+ return -1;
+
+ return 0;
+}
+
+static void
+show_info_v2(
+ void *header,
+ const char *mdfile)
+{
+ struct xfs_metadump_header *xmh;
+ uint32_t incompat_flags;
+
+ xmh = header;
+ incompat_flags = be32_to_cpu(xmh->xmh_incompat_flags);
+
+ printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
+ mdfile,
+ incompat_flags & XFS_MD2_INCOMPAT_OBFUSCATED ? "":"not ",
+ incompat_flags & XFS_MD2_INCOMPAT_DIRTYLOG ? "dirty":"clean",
+ incompat_flags & XFS_MD2_INCOMPAT_FULLBLOCKS ? "full":"zeroed");
+}
+
+static void
+restore_v2(
+ void *header,
+ FILE *mdfp,
+ int data_fd,
+ bool is_data_target_file,
+ int log_fd,
+ bool is_log_target_file)
+{
+ struct xfs_sb sb;
+ struct xfs_meta_extent xme;
+ char *block_buffer;
+ int64_t bytes_read;
+ uint64_t offset;
+ int prev_len;
+ int len;
+
+ if (fread(&xme, sizeof(xme), 1, mdfp) != 1)
+ fatal("error reading from metadump file\n");
+
+ len = be32_to_cpu(xme.xme_len);
+ len <<= BBSHIFT;
+
+ block_buffer = calloc(1, len);
+ if (block_buffer == NULL)
+ fatal("memory allocation failure\n");
+
+ if (fread(block_buffer, len, 1, mdfp) != 1)
+ fatal("error reading from metadump file\n");
+
+ libxfs_sb_from_disk(&sb, (struct xfs_dsb *)block_buffer);
+
+ if (sb.sb_magicnum != XFS_SB_MAGIC)
+ fatal("bad magic number for primary superblock\n");
+
+ if (sb.sb_logstart == 0 && log_fd == -1)
+ fatal("External Log device is required\n");
+
+ ((struct xfs_dsb *)block_buffer)->sb_inprogress = 1;
+
+ verify_device_size(data_fd, is_data_target_file, sb.sb_dblocks,
+ sb.sb_blocksize);
+
+ if (sb.sb_logstart == 0)
+ verify_device_size(log_fd, is_log_target_file, sb.sb_logblocks,
+ sb.sb_blocksize);
+
+ bytes_read = 0;
+
+ do {
+ int fd;
+
+ if (mdrestore.show_progress &&
+ (bytes_read & ((1 << 20) - 1)) == 0)
+ print_progress("%lld MB read", bytes_read >> 20);
+
+ offset = be64_to_cpu(xme.xme_addr) & XME_ADDR_DEVICE_MASK;
+ offset <<= BBSHIFT;
+
+ if (be64_to_cpu(xme.xme_addr) & XME_ADDR_DATA_DEVICE)
+ fd = data_fd;
+ else if (be64_to_cpu(xme.xme_addr) & XME_ADDR_LOG_DEVICE)
+ fd = log_fd;
+ else
+ ASSERT(0);
+
+ if (pwrite(fd, block_buffer, len, offset) < 0)
+ fatal("error writing to %s device at offset %llu: %s\n",
+ fd == data_fd ? "data": "log", offset,
+ strerror(errno));
+
+ if (fread(&xme, sizeof(xme), 1, mdfp) != 1) {
+ if (feof(mdfp))
+ break;
+ fatal("error reading from metadump file\n");
+ }
+
+ prev_len = len;
+ len = be32_to_cpu(xme.xme_len);
+ len <<= BBSHIFT;
+ if (len > prev_len) {
+ void *p;
+ p = realloc(block_buffer, len);
+ if (p == NULL) {
+ free(block_buffer);
+ fatal("memory allocation failure\n");
+ }
+ block_buffer = p;
+ }
+
+ if (fread(block_buffer, len, 1, mdfp) != 1)
+ fatal("error reading from metadump file\n");
+
+ bytes_read += len;
+ } while (1);
+
+ if (mdrestore.progress_since_warning)
+ putchar('\n');
+
+ memset(block_buffer, 0, sb.sb_sectsize);
+ sb.sb_inprogress = 0;
+ libxfs_sb_to_disk((struct xfs_dsb *)block_buffer, &sb);
+ if (xfs_sb_version_hascrc(&sb)) {
+ xfs_update_cksum(block_buffer, sb.sb_sectsize,
+ offsetof(struct xfs_sb, sb_crc));
+ }
+
+ if (pwrite(data_fd, block_buffer, sb.sb_sectsize, 0) < 0)
+ fatal("error writing primary superblock: %s\n",
+ strerror(errno));
+
+ free(block_buffer);
+
+ return;
+}
+
+static struct mdrestore_ops mdrestore_ops_v2 = {
+ .read_header = read_header_v2,
+ .show_info = show_info_v2,
+ .restore = restore_v2,
+};
+
static void
usage(void)
{
@@ -276,11 +436,16 @@ main(
int argc,
char **argv)
{
- FILE *src_f;
- int dst_fd;
- int c;
- bool is_target_file;
- struct xfs_metablock mb;
+ struct xfs_metadump_header xmh;
+ struct xfs_metablock mb;
+ FILE *src_f;
+ char *logdev = NULL;
+ void *header;
+ int data_dev_fd;
+ int log_dev_fd;
+ int c;
+ bool is_data_dev_file;
+ bool is_log_dev_file;
mdrestore.show_progress = 0;
mdrestore.show_info = 0;
@@ -327,13 +492,18 @@ main(
fatal("cannot open source dump file\n");
}
- if (mdrestore_ops_v1.read_header(&mb, src_f) == 0)
+ if (mdrestore_ops_v1.read_header(&mb, src_f) == 0) {
mdrestore.mdrops = &mdrestore_ops_v1;
- else
+ header = &mb;
+ } else if (mdrestore_ops_v2.read_header(&xmh, src_f) == 0) {
+ mdrestore.mdrops = &mdrestore_ops_v2;
+ header = &xmh;
+ } else {
fatal("Invalid metadump format\n");
+ }
if (mdrestore.show_info) {
- mdrestore.mdrops->show_info(&mb, argv[optind]);
+ mdrestore.mdrops->show_info(header, argv[optind]);
if (argc - optind == 1)
exit(0);
@@ -341,12 +511,21 @@ main(
optind++;
- /* check and open target */
- dst_fd = open_device(argv[optind], &is_target_file);
+ /* check and open data device */
+ data_dev_fd = open_device(argv[optind], &is_data_dev_file);
+
+ log_dev_fd = -1;
+ if (logdev)
+ /* check and open log device */
+ log_dev_fd = open_device(logdev, &is_log_dev_file);
+
+ mdrestore.mdrops->restore(header, src_f, data_dev_fd, is_data_dev_file,
+ log_dev_fd, is_log_dev_file);
- mdrestore.mdrops->restore(&mb, src_f, dst_fd, is_target_file);
+ close(data_dev_fd);
+ if (logdev)
+ close(log_dev_fd);
- close(dst_fd);
if (src_f != stdin)
fclose(src_f);
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 22/24] mdrestore: Define mdrestore ops for v2 format
2023-05-23 9:00 ` [PATCH 22/24] mdrestore: Define mdrestore ops for v2 format Chandan Babu R
@ 2023-05-23 18:06 ` Darrick J. Wong
2023-05-25 12:10 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 18:06 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:48PM +0530, Chandan Babu R wrote:
> This commit adds functionality to restore metadump stored in v2 format.
>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> mdrestore/xfs_mdrestore.c | 209 +++++++++++++++++++++++++++++++++++---
> 1 file changed, 194 insertions(+), 15 deletions(-)
>
> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> index 615ecdc77..9e06d37dc 100644
> --- a/mdrestore/xfs_mdrestore.c
> +++ b/mdrestore/xfs_mdrestore.c
> @@ -11,7 +11,8 @@ struct mdrestore_ops {
> int (*read_header)(void *header, FILE *mdfp);
> void (*show_info)(void *header, const char *mdfile);
> void (*restore)(void *header, FILE *mdfp, int data_fd,
> - bool is_target_file);
> + bool is_data_target_file, int log_fd,
> + bool is_log_target_file);
> };
>
> static struct mdrestore {
> @@ -148,7 +149,9 @@ restore_v1(
> void *header,
> FILE *mdfp,
> int data_fd,
> - bool is_target_file)
> + bool is_data_target_file,
> + int log_fd,
> + bool is_log_target_file)
> {
> struct xfs_metablock *mbp = header;
> struct xfs_metablock *metablock;
> @@ -203,7 +206,7 @@ restore_v1(
>
> ((struct xfs_dsb*)block_buffer)->sb_inprogress = 1;
>
> - verify_device_size(data_fd, is_target_file, sb.sb_dblocks,
> + verify_device_size(data_fd, is_data_target_file, sb.sb_dblocks,
> sb.sb_blocksize);
>
> bytes_read = 0;
> @@ -264,6 +267,163 @@ static struct mdrestore_ops mdrestore_ops_v1 = {
> .restore = restore_v1,
> };
>
> +static int
> +read_header_v2(
> + void *header,
> + FILE *mdfp)
> +{
> + struct xfs_metadump_header *xmh = header;
> +
> + rewind(mdfp);
Does rewind() work if @mdfp is a pipe?
I suspect the best you can do is read the first 4 bytes in main, pick
the read_header function from that, and have the read_header_v[12] read
in the rest of the header from there. I use a lot of:
xfs_metadump -ago /dev/sda - | gzip > foo.md.gz
gzip -d < foo.md.gz | xfs_mdrestore -g - /dev/sdb
to store compressed metadumps for future reference.
(Well ok I use xz or zstd, but you get the point.)
> +
> + if (fread(xmh, sizeof(*xmh), 1, mdfp) != 1)
> + fatal("error reading from metadump file\n");
> + if (xmh->xmh_magic != cpu_to_be32(XFS_MD_MAGIC_V2))
> + return -1;
> +
> + return 0;
> +}
> +
> +static void
> +show_info_v2(
> + void *header,
> + const char *mdfile)
> +{
> + struct xfs_metadump_header *xmh;
> + uint32_t incompat_flags;
> +
> + xmh = header;
> + incompat_flags = be32_to_cpu(xmh->xmh_incompat_flags);
> +
> + printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
> + mdfile,
> + incompat_flags & XFS_MD2_INCOMPAT_OBFUSCATED ? "":"not ",
> + incompat_flags & XFS_MD2_INCOMPAT_DIRTYLOG ? "dirty":"clean",
> + incompat_flags & XFS_MD2_INCOMPAT_FULLBLOCKS ? "full":"zeroed");
> +}
> +
> +static void
> +restore_v2(
> + void *header,
> + FILE *mdfp,
> + int data_fd,
> + bool is_data_target_file,
> + int log_fd,
> + bool is_log_target_file)
> +{
> + struct xfs_sb sb;
> + struct xfs_meta_extent xme;
> + char *block_buffer;
> + int64_t bytes_read;
> + uint64_t offset;
> + int prev_len;
> + int len;
> +
> + if (fread(&xme, sizeof(xme), 1, mdfp) != 1)
> + fatal("error reading from metadump file\n");
> +
> + len = be32_to_cpu(xme.xme_len);
> + len <<= BBSHIFT;
Do we need to validate xme_addr==0 and xme_len==1 here?
> +
> + block_buffer = calloc(1, len);
> + if (block_buffer == NULL)
> + fatal("memory allocation failure\n");
> +
> + if (fread(block_buffer, len, 1, mdfp) != 1)
> + fatal("error reading from metadump file\n");
> +
> + libxfs_sb_from_disk(&sb, (struct xfs_dsb *)block_buffer);
> +
> + if (sb.sb_magicnum != XFS_SB_MAGIC)
> + fatal("bad magic number for primary superblock\n");
> +
> + if (sb.sb_logstart == 0 && log_fd == -1)
> + fatal("External Log device is required\n");
> +
> + ((struct xfs_dsb *)block_buffer)->sb_inprogress = 1;
> +
> + verify_device_size(data_fd, is_data_target_file, sb.sb_dblocks,
> + sb.sb_blocksize);
> +
> + if (sb.sb_logstart == 0)
> + verify_device_size(log_fd, is_log_target_file, sb.sb_logblocks,
> + sb.sb_blocksize);
> +
> + bytes_read = 0;
> +
> + do {
> + int fd;
> +
> + if (mdrestore.show_progress &&
> + (bytes_read & ((1 << 20) - 1)) == 0)
> + print_progress("%lld MB read", bytes_read >> 20);
Doesn't this miss a progress report if a metadata extent bumps
bytes_read across a MB boundary without actually landing on it? Say
you've written 1020K, and the next xfs_meta_extent is 8k long.
if (metadump.show_progress) {
static int64_t mb_read;
int64_t mb_now = bytes_read >> 20;
if (mb_now != mb_read) {
print_progress("%lld MB read", mb_now);
mb_read = mb_now;
}
}
> +
> + offset = be64_to_cpu(xme.xme_addr) & XME_ADDR_DEVICE_MASK;
> + offset <<= BBSHIFT;
offset = BBTOB(be64_to_cpu() ... ); ?
Also, I'd have thought that XME_ADDR_DEVICE_MASK is what you use to
decode the device, not what you use to decode the address within a
device.
> +
> + if (be64_to_cpu(xme.xme_addr) & XME_ADDR_DATA_DEVICE)
> + fd = data_fd;
> + else if (be64_to_cpu(xme.xme_addr) & XME_ADDR_LOG_DEVICE)
> + fd = log_fd;
> + else
> + ASSERT(0);
If you instead defined the constants like this:
#define XME_ADDR_DEVICE_SHIFT 54
#define XME_ADDR_DEVICE_MASK ((1ULL << XME_ADDR_DEVICE_SHIFT) - 1)
#define XME_ADDR_DATA_DEVICE (0 << XME_ADDR_DEVICE_SHIFT)
#define XME_ADDR_LOG_DEVICE (1 << XME_ADDR_DEVICE_SHIFT)
#define XME_ADDR_RT_DEVICE (2 << XME_ADDR_DEVICE_SHIFT)
#define XME_ADDR_DEVICE_MASK (3 << XME_ADDR_DEVICE_SHIFT)
Then the above becomes:
offset = BBTOB(be64_to_cpu(xme.xme_addr) & XME_ADDR_DADDR_MASK);
switch (be64_to_cpu(xme.xme_addr) & XME_ADDR_DEVICE_MASK) {
case XME_ADDR_DATA_DEVICE:
fd = data_fd;
break;
...
}
> +
> + if (pwrite(fd, block_buffer, len, offset) < 0)
> + fatal("error writing to %s device at offset %llu: %s\n",
> + fd == data_fd ? "data": "log", offset,
> + strerror(errno));
> +
> + if (fread(&xme, sizeof(xme), 1, mdfp) != 1) {
> + if (feof(mdfp))
> + break;
> + fatal("error reading from metadump file\n");
> + }
> +
> + prev_len = len;
> + len = be32_to_cpu(xme.xme_len);
> + len <<= BBSHIFT;
> + if (len > prev_len) {
> + void *p;
> + p = realloc(block_buffer, len);
Would it be preferable to declare an 8MB buffer and only copy contents
in that granularity? Technically speaking, xme_len == -1U would require
us to allocate a 2TB buffer, wouldn't it?
> + if (p == NULL) {
> + free(block_buffer);
> + fatal("memory allocation failure\n");
> + }
> + block_buffer = p;
> + }
> +
> + if (fread(block_buffer, len, 1, mdfp) != 1)
> + fatal("error reading from metadump file\n");
> +
> + bytes_read += len;
> + } while (1);
> +
> + if (mdrestore.progress_since_warning)
> + putchar('\n');
> +
> + memset(block_buffer, 0, sb.sb_sectsize);
Tabs not spaces.
> + sb.sb_inprogress = 0;
> + libxfs_sb_to_disk((struct xfs_dsb *)block_buffer, &sb);
> + if (xfs_sb_version_hascrc(&sb)) {
> + xfs_update_cksum(block_buffer, sb.sb_sectsize,
> + offsetof(struct xfs_sb, sb_crc));
> + }
> +
> + if (pwrite(data_fd, block_buffer, sb.sb_sectsize, 0) < 0)
> + fatal("error writing primary superblock: %s\n",
> + strerror(errno));
> +
> + free(block_buffer);
> +
> + return;
> +}
> +
> +static struct mdrestore_ops mdrestore_ops_v2 = {
> + .read_header = read_header_v2,
> + .show_info = show_info_v2,
> + .restore = restore_v2,
> +};
> +
> static void
> usage(void)
> {
> @@ -276,11 +436,16 @@ main(
> int argc,
> char **argv)
> {
> - FILE *src_f;
> - int dst_fd;
> - int c;
> - bool is_target_file;
> - struct xfs_metablock mb;
> + struct xfs_metadump_header xmh;
> + struct xfs_metablock mb;
Hmm...
> + FILE *src_f;
> + char *logdev = NULL;
> + void *header;
> + int data_dev_fd;
> + int log_dev_fd;
> + int c;
> + bool is_data_dev_file;
> + bool is_log_dev_file;
>
> mdrestore.show_progress = 0;
> mdrestore.show_info = 0;
> @@ -327,13 +492,18 @@ main(
> fatal("cannot open source dump file\n");
> }
>
> - if (mdrestore_ops_v1.read_header(&mb, src_f) == 0)
> + if (mdrestore_ops_v1.read_header(&mb, src_f) == 0) {
> mdrestore.mdrops = &mdrestore_ops_v1;
> - else
> + header = &mb;
> + } else if (mdrestore_ops_v2.read_header(&xmh, src_f) == 0) {
> + mdrestore.mdrops = &mdrestore_ops_v2;
> + header = &xmh;
Perhaps define a union of both header formats, then pass that to
->read_header, ->show_info, and ->restore?
--D
> + } else {
> fatal("Invalid metadump format\n");
> + }
>
> if (mdrestore.show_info) {
> - mdrestore.mdrops->show_info(&mb, argv[optind]);
> + mdrestore.mdrops->show_info(header, argv[optind]);
>
> if (argc - optind == 1)
> exit(0);
> @@ -341,12 +511,21 @@ main(
>
> optind++;
>
> - /* check and open target */
> - dst_fd = open_device(argv[optind], &is_target_file);
> + /* check and open data device */
> + data_dev_fd = open_device(argv[optind], &is_data_dev_file);
> +
> + log_dev_fd = -1;
> + if (logdev)
> + /* check and open log device */
> + log_dev_fd = open_device(logdev, &is_log_dev_file);
> +
> + mdrestore.mdrops->restore(header, src_f, data_dev_fd, is_data_dev_file,
> + log_dev_fd, is_log_dev_file);
>
> - mdrestore.mdrops->restore(&mb, src_f, dst_fd, is_target_file);
> + close(data_dev_fd);
> + if (logdev)
> + close(log_dev_fd);
>
> - close(dst_fd);
> if (src_f != stdin)
> fclose(src_f);
>
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 22/24] mdrestore: Define mdrestore ops for v2 format
2023-05-23 18:06 ` Darrick J. Wong
@ 2023-05-25 12:10 ` Chandan Babu R
2023-06-02 15:01 ` Darrick J. Wong
0 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 12:10 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 11:06:02 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:48PM +0530, Chandan Babu R wrote:
>> This commit adds functionality to restore metadump stored in v2 format.
>>
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> mdrestore/xfs_mdrestore.c | 209 +++++++++++++++++++++++++++++++++++---
>> 1 file changed, 194 insertions(+), 15 deletions(-)
>>
>> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
>> index 615ecdc77..9e06d37dc 100644
>> --- a/mdrestore/xfs_mdrestore.c
>> +++ b/mdrestore/xfs_mdrestore.c
>> @@ -11,7 +11,8 @@ struct mdrestore_ops {
>> int (*read_header)(void *header, FILE *mdfp);
>> void (*show_info)(void *header, const char *mdfile);
>> void (*restore)(void *header, FILE *mdfp, int data_fd,
>> - bool is_target_file);
>> + bool is_data_target_file, int log_fd,
>> + bool is_log_target_file);
>> };
>>
>> static struct mdrestore {
>> @@ -148,7 +149,9 @@ restore_v1(
>> void *header,
>> FILE *mdfp,
>> int data_fd,
>> - bool is_target_file)
>> + bool is_data_target_file,
>> + int log_fd,
>> + bool is_log_target_file)
>> {
>> struct xfs_metablock *mbp = header;
>> struct xfs_metablock *metablock;
>> @@ -203,7 +206,7 @@ restore_v1(
>>
>> ((struct xfs_dsb*)block_buffer)->sb_inprogress = 1;
>>
>> - verify_device_size(data_fd, is_target_file, sb.sb_dblocks,
>> + verify_device_size(data_fd, is_data_target_file, sb.sb_dblocks,
>> sb.sb_blocksize);
>>
>> bytes_read = 0;
>> @@ -264,6 +267,163 @@ static struct mdrestore_ops mdrestore_ops_v1 = {
>> .restore = restore_v1,
>> };
>>
>> +static int
>> +read_header_v2(
>> + void *header,
>> + FILE *mdfp)
>> +{
>> + struct xfs_metadump_header *xmh = header;
>> +
>> + rewind(mdfp);
>
> Does rewind() work if @mdfp is a pipe?
>
> I suspect the best you can do is read the first 4 bytes in main, pick
> the read_header function from that, and have the read_header_v[12] read
> in the rest of the header from there. I use a lot of:
>
> xfs_metadump -ago /dev/sda - | gzip > foo.md.gz
> gzip -d < foo.md.gz | xfs_mdrestore -g - /dev/sdb
>
> to store compressed metadumps for future reference.
>
> (Well ok I use xz or zstd, but you get the point.)
>
Thanks for pointing out the bug and suggesting the fix.
>> +
>> + if (fread(xmh, sizeof(*xmh), 1, mdfp) != 1)
>> + fatal("error reading from metadump file\n");
>> + if (xmh->xmh_magic != cpu_to_be32(XFS_MD_MAGIC_V2))
>> + return -1;
>> +
>> + return 0;
>> +}
>> +
>> +static void
>> +show_info_v2(
>> + void *header,
>> + const char *mdfile)
>> +{
>> + struct xfs_metadump_header *xmh;
>> + uint32_t incompat_flags;
>> +
>> + xmh = header;
>> + incompat_flags = be32_to_cpu(xmh->xmh_incompat_flags);
>> +
>> + printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
>> + mdfile,
>> + incompat_flags & XFS_MD2_INCOMPAT_OBFUSCATED ? "":"not ",
>> + incompat_flags & XFS_MD2_INCOMPAT_DIRTYLOG ? "dirty":"clean",
>> + incompat_flags & XFS_MD2_INCOMPAT_FULLBLOCKS ? "full":"zeroed");
>> +}
>> +
>> +static void
>> +restore_v2(
>> + void *header,
>> + FILE *mdfp,
>> + int data_fd,
>> + bool is_data_target_file,
>> + int log_fd,
>> + bool is_log_target_file)
>> +{
>> + struct xfs_sb sb;
>> + struct xfs_meta_extent xme;
>> + char *block_buffer;
>> + int64_t bytes_read;
>> + uint64_t offset;
>> + int prev_len;
>> + int len;
>> +
>> + if (fread(&xme, sizeof(xme), 1, mdfp) != 1)
>> + fatal("error reading from metadump file\n");
>> +
>> + len = be32_to_cpu(xme.xme_len);
>> + len <<= BBSHIFT;
>
> Do we need to validate xme_addr==0 and xme_len==1 here?
>
Yes, I will add the check.
>> +
>> + block_buffer = calloc(1, len);
>> + if (block_buffer == NULL)
>> + fatal("memory allocation failure\n");
>> +
>> + if (fread(block_buffer, len, 1, mdfp) != 1)
>> + fatal("error reading from metadump file\n");
>> +
>> + libxfs_sb_from_disk(&sb, (struct xfs_dsb *)block_buffer);
>> +
>> + if (sb.sb_magicnum != XFS_SB_MAGIC)
>> + fatal("bad magic number for primary superblock\n");
>> +
>> + if (sb.sb_logstart == 0 && log_fd == -1)
>> + fatal("External Log device is required\n");
>> +
>> + ((struct xfs_dsb *)block_buffer)->sb_inprogress = 1;
>> +
>> + verify_device_size(data_fd, is_data_target_file, sb.sb_dblocks,
>> + sb.sb_blocksize);
>> +
>> + if (sb.sb_logstart == 0)
>> + verify_device_size(log_fd, is_log_target_file, sb.sb_logblocks,
>> + sb.sb_blocksize);
>> +
>> + bytes_read = 0;
>> +
>> + do {
>> + int fd;
>> +
>> + if (mdrestore.show_progress &&
>> + (bytes_read & ((1 << 20) - 1)) == 0)
>> + print_progress("%lld MB read", bytes_read >> 20);
>
> Doesn't this miss a progress report if a metadata extent bumps
> bytes_read across a MB boundary without actually landing on it? Say
> you've written 1020K, and the next xfs_meta_extent is 8k long.
>
> if (metadump.show_progress) {
> static int64_t mb_read;
> int64_t mb_now = bytes_read >> 20;
>
> if (mb_now != mb_read) {
> print_progress("%lld MB read", mb_now);
> mb_read = mb_now;
> }
> }
>
I will include the above suggestion in the patchset.
>> +
>> + offset = be64_to_cpu(xme.xme_addr) & XME_ADDR_DEVICE_MASK;
>> + offset <<= BBSHIFT;
>
> offset = BBTOB(be64_to_cpu() ... ); ?
Yes, this is much better than what I had written.
>
> Also, I'd have thought that XME_ADDR_DEVICE_MASK is what you use to
> decode the device, not what you use to decode the address within a
> device.
>
>> +
>> + if (be64_to_cpu(xme.xme_addr) & XME_ADDR_DATA_DEVICE)
>> + fd = data_fd;
>> + else if (be64_to_cpu(xme.xme_addr) & XME_ADDR_LOG_DEVICE)
>> + fd = log_fd;
>> + else
>> + ASSERT(0);
>
> If you instead defined the constants like this:
>
> #define XME_ADDR_DEVICE_SHIFT 54
> #define XME_ADDR_DEVICE_MASK ((1ULL << XME_ADDR_DEVICE_SHIFT) - 1)
You probably meant to define XME_ADDR_DADDR_MASK.
> #define XME_ADDR_DATA_DEVICE (0 << XME_ADDR_DEVICE_SHIFT)
> #define XME_ADDR_LOG_DEVICE (1 << XME_ADDR_DEVICE_SHIFT)
> #define XME_ADDR_RT_DEVICE (2 << XME_ADDR_DEVICE_SHIFT)
>
> #define XME_ADDR_DEVICE_MASK (3 << XME_ADDR_DEVICE_SHIFT)
>
> Then the above becomes:
>
> offset = BBTOB(be64_to_cpu(xme.xme_addr) & XME_ADDR_DADDR_MASK);
>
> switch (be64_to_cpu(xme.xme_addr) & XME_ADDR_DEVICE_MASK) {
> case XME_ADDR_DATA_DEVICE:
> fd = data_fd;
> break;
> ...
> }
Yes, this looks much better. Thanks for your suggestion.
>> +
>> + if (pwrite(fd, block_buffer, len, offset) < 0)
>> + fatal("error writing to %s device at offset %llu: %s\n",
>> + fd == data_fd ? "data": "log", offset,
>> + strerror(errno));
>> +
>> + if (fread(&xme, sizeof(xme), 1, mdfp) != 1) {
>> + if (feof(mdfp))
>> + break;
>> + fatal("error reading from metadump file\n");
>> + }
>> +
>> + prev_len = len;
>> + len = be32_to_cpu(xme.xme_len);
>> + len <<= BBSHIFT;
>> + if (len > prev_len) {
>> + void *p;
>> + p = realloc(block_buffer, len);
>
> Would it be preferable to declare an 8MB buffer and only copy contents
> in that granularity? Technically speaking, xme_len == -1U would require
> us to allocate a 2TB buffer, wouldn't it?
>
Yes, I agree. I will make the required changes.
>> + if (p == NULL) {
>> + free(block_buffer);
>> + fatal("memory allocation failure\n");
>> + }
>> + block_buffer = p;
>> + }
>> +
>> + if (fread(block_buffer, len, 1, mdfp) != 1)
>> + fatal("error reading from metadump file\n");
>> +
>> + bytes_read += len;
>> + } while (1);
>> +
>> + if (mdrestore.progress_since_warning)
>> + putchar('\n');
>> +
>> + memset(block_buffer, 0, sb.sb_sectsize);
>
> Tabs not spaces.
>
>> + sb.sb_inprogress = 0;
>> + libxfs_sb_to_disk((struct xfs_dsb *)block_buffer, &sb);
>> + if (xfs_sb_version_hascrc(&sb)) {
>> + xfs_update_cksum(block_buffer, sb.sb_sectsize,
>> + offsetof(struct xfs_sb, sb_crc));
>> + }
>> +
>> + if (pwrite(data_fd, block_buffer, sb.sb_sectsize, 0) < 0)
>> + fatal("error writing primary superblock: %s\n",
>> + strerror(errno));
>> +
>> + free(block_buffer);
>> +
>> + return;
>> +}
>> +
>> +static struct mdrestore_ops mdrestore_ops_v2 = {
>> + .read_header = read_header_v2,
>> + .show_info = show_info_v2,
>> + .restore = restore_v2,
>> +};
>> +
>> static void
>> usage(void)
>> {
>> @@ -276,11 +436,16 @@ main(
>> int argc,
>> char **argv)
>> {
>> - FILE *src_f;
>> - int dst_fd;
>> - int c;
>> - bool is_target_file;
>> - struct xfs_metablock mb;
>> + struct xfs_metadump_header xmh;
>> + struct xfs_metablock mb;
>
> Hmm...
>
>> + FILE *src_f;
>> + char *logdev = NULL;
>> + void *header;
>> + int data_dev_fd;
>> + int log_dev_fd;
>> + int c;
>> + bool is_data_dev_file;
>> + bool is_log_dev_file;
>>
>> mdrestore.show_progress = 0;
>> mdrestore.show_info = 0;
>> @@ -327,13 +492,18 @@ main(
>> fatal("cannot open source dump file\n");
>> }
>>
>> - if (mdrestore_ops_v1.read_header(&mb, src_f) == 0)
>> + if (mdrestore_ops_v1.read_header(&mb, src_f) == 0) {
>> mdrestore.mdrops = &mdrestore_ops_v1;
>> - else
>> + header = &mb;
>> + } else if (mdrestore_ops_v2.read_header(&xmh, src_f) == 0) {
>> + mdrestore.mdrops = &mdrestore_ops_v2;
>> + header = &xmh;
>
> Perhaps define a union of both header formats, then pass that to
> ->read_header, ->show_info, and ->restore?
Sure. I will do that.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 22/24] mdrestore: Define mdrestore ops for v2 format
2023-05-25 12:10 ` Chandan Babu R
@ 2023-06-02 15:01 ` Darrick J. Wong
0 siblings, 0 replies; 76+ messages in thread
From: Darrick J. Wong @ 2023-06-02 15:01 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Thu, May 25, 2023 at 05:40:24PM +0530, Chandan Babu R wrote:
> On Tue, May 23, 2023 at 11:06:02 AM -0700, Darrick J. Wong wrote:
> > On Tue, May 23, 2023 at 02:30:48PM +0530, Chandan Babu R wrote:
> >> This commit adds functionality to restore metadump stored in v2 format.
> >>
> >> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> >> ---
> >> mdrestore/xfs_mdrestore.c | 209 +++++++++++++++++++++++++++++++++++---
> >> 1 file changed, 194 insertions(+), 15 deletions(-)
> >>
> >> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> >> index 615ecdc77..9e06d37dc 100644
> >> --- a/mdrestore/xfs_mdrestore.c
> >> +++ b/mdrestore/xfs_mdrestore.c
> >> @@ -11,7 +11,8 @@ struct mdrestore_ops {
> >> int (*read_header)(void *header, FILE *mdfp);
> >> void (*show_info)(void *header, const char *mdfile);
> >> void (*restore)(void *header, FILE *mdfp, int data_fd,
> >> - bool is_target_file);
> >> + bool is_data_target_file, int log_fd,
> >> + bool is_log_target_file);
> >> };
> >>
> >> static struct mdrestore {
> >> @@ -148,7 +149,9 @@ restore_v1(
> >> void *header,
> >> FILE *mdfp,
> >> int data_fd,
> >> - bool is_target_file)
> >> + bool is_data_target_file,
> >> + int log_fd,
> >> + bool is_log_target_file)
> >> {
> >> struct xfs_metablock *mbp = header;
> >> struct xfs_metablock *metablock;
> >> @@ -203,7 +206,7 @@ restore_v1(
> >>
> >> ((struct xfs_dsb*)block_buffer)->sb_inprogress = 1;
> >>
> >> - verify_device_size(data_fd, is_target_file, sb.sb_dblocks,
> >> + verify_device_size(data_fd, is_data_target_file, sb.sb_dblocks,
> >> sb.sb_blocksize);
> >>
> >> bytes_read = 0;
> >> @@ -264,6 +267,163 @@ static struct mdrestore_ops mdrestore_ops_v1 = {
> >> .restore = restore_v1,
> >> };
> >>
> >> +static int
> >> +read_header_v2(
> >> + void *header,
> >> + FILE *mdfp)
> >> +{
> >> + struct xfs_metadump_header *xmh = header;
> >> +
> >> + rewind(mdfp);
> >
> > Does rewind() work if @mdfp is a pipe?
> >
> > I suspect the best you can do is read the first 4 bytes in main, pick
> > the read_header function from that, and have the read_header_v[12] read
> > in the rest of the header from there. I use a lot of:
> >
> > xfs_metadump -ago /dev/sda - | gzip > foo.md.gz
> > gzip -d < foo.md.gz | xfs_mdrestore -g - /dev/sdb
> >
> > to store compressed metadumps for future reference.
> >
> > (Well ok I use xz or zstd, but you get the point.)
> >
>
> Thanks for pointing out the bug and suggesting the fix.
>
> >> +
> >> + if (fread(xmh, sizeof(*xmh), 1, mdfp) != 1)
> >> + fatal("error reading from metadump file\n");
> >> + if (xmh->xmh_magic != cpu_to_be32(XFS_MD_MAGIC_V2))
> >> + return -1;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static void
> >> +show_info_v2(
> >> + void *header,
> >> + const char *mdfile)
> >> +{
> >> + struct xfs_metadump_header *xmh;
> >> + uint32_t incompat_flags;
> >> +
> >> + xmh = header;
> >> + incompat_flags = be32_to_cpu(xmh->xmh_incompat_flags);
> >> +
> >> + printf("%s: %sobfuscated, %s log, %s metadata blocks\n",
> >> + mdfile,
> >> + incompat_flags & XFS_MD2_INCOMPAT_OBFUSCATED ? "":"not ",
> >> + incompat_flags & XFS_MD2_INCOMPAT_DIRTYLOG ? "dirty":"clean",
> >> + incompat_flags & XFS_MD2_INCOMPAT_FULLBLOCKS ? "full":"zeroed");
> >> +}
> >> +
> >> +static void
> >> +restore_v2(
> >> + void *header,
> >> + FILE *mdfp,
> >> + int data_fd,
> >> + bool is_data_target_file,
> >> + int log_fd,
> >> + bool is_log_target_file)
> >> +{
> >> + struct xfs_sb sb;
> >> + struct xfs_meta_extent xme;
> >> + char *block_buffer;
> >> + int64_t bytes_read;
> >> + uint64_t offset;
> >> + int prev_len;
> >> + int len;
> >> +
> >> + if (fread(&xme, sizeof(xme), 1, mdfp) != 1)
> >> + fatal("error reading from metadump file\n");
> >> +
> >> + len = be32_to_cpu(xme.xme_len);
> >> + len <<= BBSHIFT;
> >
> > Do we need to validate xme_addr==0 and xme_len==1 here?
> >
>
> Yes, I will add the check.
>
> >> +
> >> + block_buffer = calloc(1, len);
> >> + if (block_buffer == NULL)
> >> + fatal("memory allocation failure\n");
> >> +
> >> + if (fread(block_buffer, len, 1, mdfp) != 1)
> >> + fatal("error reading from metadump file\n");
> >> +
> >> + libxfs_sb_from_disk(&sb, (struct xfs_dsb *)block_buffer);
> >> +
> >> + if (sb.sb_magicnum != XFS_SB_MAGIC)
> >> + fatal("bad magic number for primary superblock\n");
> >> +
> >> + if (sb.sb_logstart == 0 && log_fd == -1)
> >> + fatal("External Log device is required\n");
> >> +
> >> + ((struct xfs_dsb *)block_buffer)->sb_inprogress = 1;
> >> +
> >> + verify_device_size(data_fd, is_data_target_file, sb.sb_dblocks,
> >> + sb.sb_blocksize);
> >> +
> >> + if (sb.sb_logstart == 0)
> >> + verify_device_size(log_fd, is_log_target_file, sb.sb_logblocks,
> >> + sb.sb_blocksize);
> >> +
> >> + bytes_read = 0;
> >> +
> >> + do {
> >> + int fd;
> >> +
> >> + if (mdrestore.show_progress &&
> >> + (bytes_read & ((1 << 20) - 1)) == 0)
> >> + print_progress("%lld MB read", bytes_read >> 20);
> >
> > Doesn't this miss a progress report if a metadata extent bumps
> > bytes_read across a MB boundary without actually landing on it? Say
> > you've written 1020K, and the next xfs_meta_extent is 8k long.
> >
> > if (metadump.show_progress) {
> > static int64_t mb_read;
> > int64_t mb_now = bytes_read >> 20;
> >
> > if (mb_now != mb_read) {
> > print_progress("%lld MB read", mb_now);
> > mb_read = mb_now;
> > }
> > }
> >
>
> I will include the above suggestion in the patchset.
>
> >> +
> >> + offset = be64_to_cpu(xme.xme_addr) & XME_ADDR_DEVICE_MASK;
> >> + offset <<= BBSHIFT;
> >
> > offset = BBTOB(be64_to_cpu() ... ); ?
>
> Yes, this is much better than what I had written.
>
> >
> > Also, I'd have thought that XME_ADDR_DEVICE_MASK is what you use to
> > decode the device, not what you use to decode the address within a
> > device.
> >
> >> +
> >> + if (be64_to_cpu(xme.xme_addr) & XME_ADDR_DATA_DEVICE)
> >> + fd = data_fd;
> >> + else if (be64_to_cpu(xme.xme_addr) & XME_ADDR_LOG_DEVICE)
> >> + fd = log_fd;
> >> + else
> >> + ASSERT(0);
> >
> > If you instead defined the constants like this:
> >
> > #define XME_ADDR_DEVICE_SHIFT 54
> > #define XME_ADDR_DEVICE_MASK ((1ULL << XME_ADDR_DEVICE_SHIFT) - 1)
>
> You probably meant to define XME_ADDR_DADDR_MASK.
Yep. Sorry about that.
> > #define XME_ADDR_DATA_DEVICE (0 << XME_ADDR_DEVICE_SHIFT)
> > #define XME_ADDR_LOG_DEVICE (1 << XME_ADDR_DEVICE_SHIFT)
> > #define XME_ADDR_RT_DEVICE (2 << XME_ADDR_DEVICE_SHIFT)
> >
> > #define XME_ADDR_DEVICE_MASK (3 << XME_ADDR_DEVICE_SHIFT)
> >
> > Then the above becomes:
> >
> > offset = BBTOB(be64_to_cpu(xme.xme_addr) & XME_ADDR_DADDR_MASK);
> >
> > switch (be64_to_cpu(xme.xme_addr) & XME_ADDR_DEVICE_MASK) {
> > case XME_ADDR_DATA_DEVICE:
> > fd = data_fd;
> > break;
> > ...
> > }
>
> Yes, this looks much better. Thanks for your suggestion.
>
> >> +
> >> + if (pwrite(fd, block_buffer, len, offset) < 0)
> >> + fatal("error writing to %s device at offset %llu: %s\n",
> >> + fd == data_fd ? "data": "log", offset,
> >> + strerror(errno));
> >> +
> >> + if (fread(&xme, sizeof(xme), 1, mdfp) != 1) {
> >> + if (feof(mdfp))
> >> + break;
> >> + fatal("error reading from metadump file\n");
> >> + }
> >> +
> >> + prev_len = len;
> >> + len = be32_to_cpu(xme.xme_len);
> >> + len <<= BBSHIFT;
> >> + if (len > prev_len) {
> >> + void *p;
> >> + p = realloc(block_buffer, len);
> >
> > Would it be preferable to declare an 8MB buffer and only copy contents
> > in that granularity? Technically speaking, xme_len == -1U would require
> > us to allocate a 2TB buffer, wouldn't it?
> >
>
> Yes, I agree. I will make the required changes.
I don't know if 8MB is a reasonable buffer size; I picked it
arbitrarily.
> >> + if (p == NULL) {
> >> + free(block_buffer);
> >> + fatal("memory allocation failure\n");
> >> + }
> >> + block_buffer = p;
> >> + }
> >> +
> >> + if (fread(block_buffer, len, 1, mdfp) != 1)
> >> + fatal("error reading from metadump file\n");
> >> +
> >> + bytes_read += len;
> >> + } while (1);
> >> +
> >> + if (mdrestore.progress_since_warning)
> >> + putchar('\n');
> >> +
> >> + memset(block_buffer, 0, sb.sb_sectsize);
> >
> > Tabs not spaces.
> >
> >> + sb.sb_inprogress = 0;
> >> + libxfs_sb_to_disk((struct xfs_dsb *)block_buffer, &sb);
> >> + if (xfs_sb_version_hascrc(&sb)) {
> >> + xfs_update_cksum(block_buffer, sb.sb_sectsize,
> >> + offsetof(struct xfs_sb, sb_crc));
> >> + }
> >> +
> >> + if (pwrite(data_fd, block_buffer, sb.sb_sectsize, 0) < 0)
> >> + fatal("error writing primary superblock: %s\n",
> >> + strerror(errno));
> >> +
> >> + free(block_buffer);
> >> +
> >> + return;
> >> +}
> >> +
> >> +static struct mdrestore_ops mdrestore_ops_v2 = {
> >> + .read_header = read_header_v2,
> >> + .show_info = show_info_v2,
> >> + .restore = restore_v2,
> >> +};
> >> +
> >> static void
> >> usage(void)
> >> {
> >> @@ -276,11 +436,16 @@ main(
> >> int argc,
> >> char **argv)
> >> {
> >> - FILE *src_f;
> >> - int dst_fd;
> >> - int c;
> >> - bool is_target_file;
> >> - struct xfs_metablock mb;
> >> + struct xfs_metadump_header xmh;
> >> + struct xfs_metablock mb;
> >
> > Hmm...
> >
> >> + FILE *src_f;
> >> + char *logdev = NULL;
> >> + void *header;
> >> + int data_dev_fd;
> >> + int log_dev_fd;
> >> + int c;
> >> + bool is_data_dev_file;
> >> + bool is_log_dev_file;
> >>
> >> mdrestore.show_progress = 0;
> >> mdrestore.show_info = 0;
> >> @@ -327,13 +492,18 @@ main(
> >> fatal("cannot open source dump file\n");
> >> }
> >>
> >> - if (mdrestore_ops_v1.read_header(&mb, src_f) == 0)
> >> + if (mdrestore_ops_v1.read_header(&mb, src_f) == 0) {
> >> mdrestore.mdrops = &mdrestore_ops_v1;
> >> - else
> >> + header = &mb;
> >> + } else if (mdrestore_ops_v2.read_header(&xmh, src_f) == 0) {
> >> + mdrestore.mdrops = &mdrestore_ops_v2;
> >> + header = &xmh;
> >
> > Perhaps define a union of both header formats, then pass that to
> > ->read_header, ->show_info, and ->restore?
>
> Sure. I will do that.
Ok.
--D
> --
> chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 23/24] mdrestore: Add support for passing log device as an argument
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (21 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 22/24] mdrestore: Define mdrestore ops for v2 format Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 18:09 ` Darrick J. Wong
2023-05-23 9:00 ` [PATCH 24/24] xfs_mdrestore.8: Add description for the newly introduced -l option Chandan Babu R
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
metadump v2 format allows dumping metadata from external log devices. This
commit allows passing the device file to which log data must be restored from
the corresponding metadump file.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
mdrestore/xfs_mdrestore.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
index 9e06d37dc..f5eff62ef 100644
--- a/mdrestore/xfs_mdrestore.c
+++ b/mdrestore/xfs_mdrestore.c
@@ -427,7 +427,8 @@ static struct mdrestore_ops mdrestore_ops_v2 = {
static void
usage(void)
{
- fprintf(stderr, "Usage: %s [-V] [-g] [-i] source target\n", progname);
+ fprintf(stderr, "Usage: %s [-V] [-g] [-i] [-l logdev] source target\n",
+ progname);
exit(1);
}
@@ -453,7 +454,7 @@ main(
progname = basename(argv[0]);
- while ((c = getopt(argc, argv, "giV")) != EOF) {
+ while ((c = getopt(argc, argv, "gil:V")) != EOF) {
switch (c) {
case 'g':
mdrestore.show_progress = 1;
@@ -461,6 +462,9 @@ main(
case 'i':
mdrestore.show_info = 1;
break;
+ case 'l':
+ logdev = optarg;
+ break;
case 'V':
printf("%s version %s\n", progname, VERSION);
exit(0);
@@ -493,6 +497,8 @@ main(
}
if (mdrestore_ops_v1.read_header(&mb, src_f) == 0) {
+ if (logdev != NULL)
+ usage();
mdrestore.mdrops = &mdrestore_ops_v1;
header = &mb;
} else if (mdrestore_ops_v2.read_header(&xmh, src_f) == 0) {
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 23/24] mdrestore: Add support for passing log device as an argument
2023-05-23 9:00 ` [PATCH 23/24] mdrestore: Add support for passing log device as an argument Chandan Babu R
@ 2023-05-23 18:09 ` Darrick J. Wong
2023-05-25 13:43 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 18:09 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:49PM +0530, Chandan Babu R wrote:
> metadump v2 format allows dumping metadata from external log devices. This
> commit allows passing the device file to which log data must be restored from
> the corresponding metadump file.
>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> mdrestore/xfs_mdrestore.c | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> index 9e06d37dc..f5eff62ef 100644
> --- a/mdrestore/xfs_mdrestore.c
> +++ b/mdrestore/xfs_mdrestore.c
> @@ -427,7 +427,8 @@ static struct mdrestore_ops mdrestore_ops_v2 = {
> static void
> usage(void)
> {
> - fprintf(stderr, "Usage: %s [-V] [-g] [-i] source target\n", progname);
> + fprintf(stderr, "Usage: %s [-V] [-g] [-i] [-l logdev] source target\n",
> + progname);
> exit(1);
> }
>
> @@ -453,7 +454,7 @@ main(
>
> progname = basename(argv[0]);
>
> - while ((c = getopt(argc, argv, "giV")) != EOF) {
> + while ((c = getopt(argc, argv, "gil:V")) != EOF) {
> switch (c) {
> case 'g':
> mdrestore.show_progress = 1;
> @@ -461,6 +462,9 @@ main(
> case 'i':
> mdrestore.show_info = 1;
> break;
> + case 'l':
> + logdev = optarg;
> + break;
> case 'V':
> printf("%s version %s\n", progname, VERSION);
> exit(0);
> @@ -493,6 +497,8 @@ main(
> }
>
> if (mdrestore_ops_v1.read_header(&mb, src_f) == 0) {
> + if (logdev != NULL)
> + usage();
> mdrestore.mdrops = &mdrestore_ops_v1;
> header = &mb;
> } else if (mdrestore_ops_v2.read_header(&xmh, src_f) == 0) {
What if we have a v2 with XME_ADDR_LOG_DEVICE meta_extents but the
caller doesn't specify -l? Do we proceed with the metadump, only to
fail midway through the restore?
--D
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 23/24] mdrestore: Add support for passing log device as an argument
2023-05-23 18:09 ` Darrick J. Wong
@ 2023-05-25 13:43 ` Chandan Babu R
2023-06-02 15:02 ` Darrick J. Wong
0 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 13:43 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 11:09:59 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:49PM +0530, Chandan Babu R wrote:
>> metadump v2 format allows dumping metadata from external log devices. This
>> commit allows passing the device file to which log data must be restored from
>> the corresponding metadump file.
>>
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> mdrestore/xfs_mdrestore.c | 10 ++++++++--
>> 1 file changed, 8 insertions(+), 2 deletions(-)
>>
>> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
>> index 9e06d37dc..f5eff62ef 100644
>> --- a/mdrestore/xfs_mdrestore.c
>> +++ b/mdrestore/xfs_mdrestore.c
>> @@ -427,7 +427,8 @@ static struct mdrestore_ops mdrestore_ops_v2 = {
>> static void
>> usage(void)
>> {
>> - fprintf(stderr, "Usage: %s [-V] [-g] [-i] source target\n", progname);
>> + fprintf(stderr, "Usage: %s [-V] [-g] [-i] [-l logdev] source target\n",
>> + progname);
>> exit(1);
>> }
>>
>> @@ -453,7 +454,7 @@ main(
>>
>> progname = basename(argv[0]);
>>
>> - while ((c = getopt(argc, argv, "giV")) != EOF) {
>> + while ((c = getopt(argc, argv, "gil:V")) != EOF) {
>> switch (c) {
>> case 'g':
>> mdrestore.show_progress = 1;
>> @@ -461,6 +462,9 @@ main(
>> case 'i':
>> mdrestore.show_info = 1;
>> break;
>> + case 'l':
>> + logdev = optarg;
>> + break;
>> case 'V':
>> printf("%s version %s\n", progname, VERSION);
>> exit(0);
>> @@ -493,6 +497,8 @@ main(
>> }
>>
>> if (mdrestore_ops_v1.read_header(&mb, src_f) == 0) {
>> + if (logdev != NULL)
>> + usage();
>> mdrestore.mdrops = &mdrestore_ops_v1;
>> header = &mb;
>> } else if (mdrestore_ops_v2.read_header(&xmh, src_f) == 0) {
>
> What if we have a v2 with XME_ADDR_LOG_DEVICE meta_extents but the
> caller doesn't specify -l? Do we proceed with the metadump, only to
> fail midway through the restore?
restore_v2() has the following statement just after reading in the superblock,
if (sb.sb_logstart == 0 && log_fd == -1)
fatal("External Log device is required\n");
Hence, In the case of a missing log device argument, the program exits before
any metadata is written to the target device.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 23/24] mdrestore: Add support for passing log device as an argument
2023-05-25 13:43 ` Chandan Babu R
@ 2023-06-02 15:02 ` Darrick J. Wong
2023-06-05 6:19 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-06-02 15:02 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Thu, May 25, 2023 at 07:13:03PM +0530, Chandan Babu R wrote:
> On Tue, May 23, 2023 at 11:09:59 AM -0700, Darrick J. Wong wrote:
> > On Tue, May 23, 2023 at 02:30:49PM +0530, Chandan Babu R wrote:
> >> metadump v2 format allows dumping metadata from external log devices. This
> >> commit allows passing the device file to which log data must be restored from
> >> the corresponding metadump file.
> >>
> >> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> >> ---
> >> mdrestore/xfs_mdrestore.c | 10 ++++++++--
> >> 1 file changed, 8 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
> >> index 9e06d37dc..f5eff62ef 100644
> >> --- a/mdrestore/xfs_mdrestore.c
> >> +++ b/mdrestore/xfs_mdrestore.c
> >> @@ -427,7 +427,8 @@ static struct mdrestore_ops mdrestore_ops_v2 = {
> >> static void
> >> usage(void)
> >> {
> >> - fprintf(stderr, "Usage: %s [-V] [-g] [-i] source target\n", progname);
> >> + fprintf(stderr, "Usage: %s [-V] [-g] [-i] [-l logdev] source target\n",
> >> + progname);
> >> exit(1);
> >> }
> >>
> >> @@ -453,7 +454,7 @@ main(
> >>
> >> progname = basename(argv[0]);
> >>
> >> - while ((c = getopt(argc, argv, "giV")) != EOF) {
> >> + while ((c = getopt(argc, argv, "gil:V")) != EOF) {
> >> switch (c) {
> >> case 'g':
> >> mdrestore.show_progress = 1;
> >> @@ -461,6 +462,9 @@ main(
> >> case 'i':
> >> mdrestore.show_info = 1;
> >> break;
> >> + case 'l':
> >> + logdev = optarg;
> >> + break;
> >> case 'V':
> >> printf("%s version %s\n", progname, VERSION);
> >> exit(0);
> >> @@ -493,6 +497,8 @@ main(
> >> }
> >>
> >> if (mdrestore_ops_v1.read_header(&mb, src_f) == 0) {
> >> + if (logdev != NULL)
> >> + usage();
> >> mdrestore.mdrops = &mdrestore_ops_v1;
> >> header = &mb;
> >> } else if (mdrestore_ops_v2.read_header(&xmh, src_f) == 0) {
> >
> > What if we have a v2 with XME_ADDR_LOG_DEVICE meta_extents but the
> > caller doesn't specify -l? Do we proceed with the metadump, only to
> > fail midway through the restore?
>
> restore_v2() has the following statement just after reading in the superblock,
>
> if (sb.sb_logstart == 0 && log_fd == -1)
> fatal("External Log device is required\n");
>
> Hence, In the case of a missing log device argument, the program exits before
> any metadata is written to the target device.
Ah, ok, that's how you handle that. In that case the only reason for a
flag in the v2 metadump header would be the principle of declaring
things that happen later in the metadump stream/file. Your call. :)
--D
> --
> chandan
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [PATCH 23/24] mdrestore: Add support for passing log device as an argument
2023-06-02 15:02 ` Darrick J. Wong
@ 2023-06-05 6:19 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-06-05 6:19 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Fri, Jun 02, 2023 at 08:02:26 AM -0700, Darrick J. Wong wrote:
> On Thu, May 25, 2023 at 07:13:03PM +0530, Chandan Babu R wrote:
>> On Tue, May 23, 2023 at 11:09:59 AM -0700, Darrick J. Wong wrote:
>> > On Tue, May 23, 2023 at 02:30:49PM +0530, Chandan Babu R wrote:
>> >> metadump v2 format allows dumping metadata from external log devices. This
>> >> commit allows passing the device file to which log data must be restored from
>> >> the corresponding metadump file.
>> >>
>> >> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> >> ---
>> >> mdrestore/xfs_mdrestore.c | 10 ++++++++--
>> >> 1 file changed, 8 insertions(+), 2 deletions(-)
>> >>
>> >> diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c
>> >> index 9e06d37dc..f5eff62ef 100644
>> >> --- a/mdrestore/xfs_mdrestore.c
>> >> +++ b/mdrestore/xfs_mdrestore.c
>> >> @@ -427,7 +427,8 @@ static struct mdrestore_ops mdrestore_ops_v2 = {
>> >> static void
>> >> usage(void)
>> >> {
>> >> - fprintf(stderr, "Usage: %s [-V] [-g] [-i] source target\n", progname);
>> >> + fprintf(stderr, "Usage: %s [-V] [-g] [-i] [-l logdev] source target\n",
>> >> + progname);
>> >> exit(1);
>> >> }
>> >>
>> >> @@ -453,7 +454,7 @@ main(
>> >>
>> >> progname = basename(argv[0]);
>> >>
>> >> - while ((c = getopt(argc, argv, "giV")) != EOF) {
>> >> + while ((c = getopt(argc, argv, "gil:V")) != EOF) {
>> >> switch (c) {
>> >> case 'g':
>> >> mdrestore.show_progress = 1;
>> >> @@ -461,6 +462,9 @@ main(
>> >> case 'i':
>> >> mdrestore.show_info = 1;
>> >> break;
>> >> + case 'l':
>> >> + logdev = optarg;
>> >> + break;
>> >> case 'V':
>> >> printf("%s version %s\n", progname, VERSION);
>> >> exit(0);
>> >> @@ -493,6 +497,8 @@ main(
>> >> }
>> >>
>> >> if (mdrestore_ops_v1.read_header(&mb, src_f) == 0) {
>> >> + if (logdev != NULL)
>> >> + usage();
>> >> mdrestore.mdrops = &mdrestore_ops_v1;
>> >> header = &mb;
>> >> } else if (mdrestore_ops_v2.read_header(&xmh, src_f) == 0) {
>> >
>> > What if we have a v2 with XME_ADDR_LOG_DEVICE meta_extents but the
>> > caller doesn't specify -l? Do we proceed with the metadump, only to
>> > fail midway through the restore?
>>
>> restore_v2() has the following statement just after reading in the superblock,
>>
>> if (sb.sb_logstart == 0 && log_fd == -1)
>> fatal("External Log device is required\n");
>>
>> Hence, In the case of a missing log device argument, the program exits before
>> any metadata is written to the target device.
>
> Ah, ok, that's how you handle that. In that case the only reason for a
> flag in the v2 metadump header would be the principle of declaring
> things that happen later in the metadump stream/file. Your call. :)
I think I will implement your suggestion regarding introducing a new header
flag to indicate that the metadump contains data which was copied from an
external log device.
This will make it easier for mdrestore to detect the requirement for the "-l
logdev" option much earlier in the restore process.
Thanks for the suggestion.
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH 24/24] xfs_mdrestore.8: Add description for the newly introduced -l option
2023-05-23 9:00 [PATCH 00/24] Metadump v2 Chandan Babu R
` (22 preceding siblings ...)
2023-05-23 9:00 ` [PATCH 23/24] mdrestore: Add support for passing log device as an argument Chandan Babu R
@ 2023-05-23 9:00 ` Chandan Babu R
2023-05-23 18:10 ` Darrick J. Wong
23 siblings, 1 reply; 76+ messages in thread
From: Chandan Babu R @ 2023-05-23 9:00 UTC (permalink / raw)
To: cem; +Cc: Chandan Babu R, linux-xfs
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
man/man8/xfs_mdrestore.8 | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/man/man8/xfs_mdrestore.8 b/man/man8/xfs_mdrestore.8
index 72f3b2977..a53ac84d0 100644
--- a/man/man8/xfs_mdrestore.8
+++ b/man/man8/xfs_mdrestore.8
@@ -5,6 +5,9 @@ xfs_mdrestore \- restores an XFS metadump image to a filesystem image
.B xfs_mdrestore
[
.B \-gi
+] [
+.B \-l
+.I logdev
]
.I source
.I target
@@ -49,6 +52,11 @@ Shows metadump information on stdout. If no
is specified, exits after displaying information. Older metadumps man not
include any descriptive information.
.TP
+.B \-l " logdev"
+Metadump in v2 format can contain metadata dumped from an external log. In
+such a scenario, the user has to provide a device to which the log device
+contents from the metadump file are copied.
+.TP
.B \-V
Prints the version number and exits.
.SH DIAGNOSTICS
--
2.39.1
^ permalink raw reply related [flat|nested] 76+ messages in thread* Re: [PATCH 24/24] xfs_mdrestore.8: Add description for the newly introduced -l option
2023-05-23 9:00 ` [PATCH 24/24] xfs_mdrestore.8: Add description for the newly introduced -l option Chandan Babu R
@ 2023-05-23 18:10 ` Darrick J. Wong
2023-05-25 13:45 ` Chandan Babu R
0 siblings, 1 reply; 76+ messages in thread
From: Darrick J. Wong @ 2023-05-23 18:10 UTC (permalink / raw)
To: Chandan Babu R; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 02:30:50PM +0530, Chandan Babu R wrote:
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
> man/man8/xfs_mdrestore.8 | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/man/man8/xfs_mdrestore.8 b/man/man8/xfs_mdrestore.8
> index 72f3b2977..a53ac84d0 100644
> --- a/man/man8/xfs_mdrestore.8
> +++ b/man/man8/xfs_mdrestore.8
> @@ -5,6 +5,9 @@ xfs_mdrestore \- restores an XFS metadump image to a filesystem image
> .B xfs_mdrestore
> [
> .B \-gi
> +] [
> +.B \-l
> +.I logdev
> ]
> .I source
> .I target
> @@ -49,6 +52,11 @@ Shows metadump information on stdout. If no
> is specified, exits after displaying information. Older metadumps man not
> include any descriptive information.
> .TP
> +.B \-l " logdev"
> +Metadump in v2 format can contain metadata dumped from an external log. In
> +such a scenario, the user has to provide a device to which the log device
> +contents from the metadump file are copied.
Please start sentences on a new line.
Also, this ought to be folded into the previous patch.
Otherwise the manpage additions look reasonable to me.
--D
> +.TP
> .B \-V
> Prints the version number and exits.
> .SH DIAGNOSTICS
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH 24/24] xfs_mdrestore.8: Add description for the newly introduced -l option
2023-05-23 18:10 ` Darrick J. Wong
@ 2023-05-25 13:45 ` Chandan Babu R
0 siblings, 0 replies; 76+ messages in thread
From: Chandan Babu R @ 2023-05-25 13:45 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: cem, linux-xfs
On Tue, May 23, 2023 at 11:10:56 AM -0700, Darrick J. Wong wrote:
> On Tue, May 23, 2023 at 02:30:50PM +0530, Chandan Babu R wrote:
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>> man/man8/xfs_mdrestore.8 | 8 ++++++++
>> 1 file changed, 8 insertions(+)
>>
>> diff --git a/man/man8/xfs_mdrestore.8 b/man/man8/xfs_mdrestore.8
>> index 72f3b2977..a53ac84d0 100644
>> --- a/man/man8/xfs_mdrestore.8
>> +++ b/man/man8/xfs_mdrestore.8
>> @@ -5,6 +5,9 @@ xfs_mdrestore \- restores an XFS metadump image to a filesystem image
>> .B xfs_mdrestore
>> [
>> .B \-gi
>> +] [
>> +.B \-l
>> +.I logdev
>> ]
>> .I source
>> .I target
>> @@ -49,6 +52,11 @@ Shows metadump information on stdout. If no
>> is specified, exits after displaying information. Older metadumps man not
>> include any descriptive information.
>> .TP
>> +.B \-l " logdev"
>> +Metadump in v2 format can contain metadata dumped from an external log. In
>> +such a scenario, the user has to provide a device to which the log device
>> +contents from the metadump file are copied.
>
> Please start sentences on a new line.
>
> Also, this ought to be folded into the previous patch.
>
> Otherwise the manpage additions look reasonable to me.
Thanks for your valuable review comments!
--
chandan
^ permalink raw reply [flat|nested] 76+ messages in thread