* [Cluster-devel] [fsck.gfs2 v2 PATCH 01/40] fsck.gfs2: Move pass5 to immediately follow pass1
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
@ 2016-05-06 17:38 ` Bob Peterson
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 02/40] fsck.gfs2: Convert block_type to bitmap_type after pass1 and 5 Bob Peterson
` (40 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:38 UTC (permalink / raw)
To: cluster-devel.redhat.com
The purpose of pass1 is to make sure all bits are represented
properly for inodes. After that, the bitmaps should be kept in
sync by subsequent passes. The purpose of the old pass5 was really
to free any blocks that were not linked somehow to dinodes.
This patch moves that reconciliation to immediately follow pass1.
That allows us to free the bitmap memory sooner, making subsequent
passes use the rgrp bitmaps for their work. The purpose is to
reduce the memory footprint of fsck.gfs2.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/main.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/gfs2/fsck/main.c b/gfs2/fsck/main.c
index 08343ca..8f2c733 100644
--- a/gfs2/fsck/main.c
+++ b/gfs2/fsck/main.c
@@ -236,11 +236,11 @@ struct fsck_pass {
static const struct fsck_pass passes[] = {
{ .name = "pass1", .f = pass1 },
+ { .name = "reconcile_bitmaps", .f = pass5 },
{ .name = "pass1b", .f = pass1b },
{ .name = "pass2", .f = pass2 },
{ .name = "pass3", .f = pass3 },
{ .name = "pass4", .f = pass4 },
- { .name = "pass5", .f = pass5 },
{ .name = "check_statfs", .f = check_statfs },
{ .name = NULL, }
};
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 02/40] fsck.gfs2: Convert block_type to bitmap_type after pass1 and 5
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 01/40] fsck.gfs2: Move pass5 to immediately follow pass1 Bob Peterson
@ 2016-05-06 17:38 ` Bob Peterson
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 03/40] fsck.gfs2: Change bitmap_type variables to int Bob Peterson
` (39 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:38 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch changes all the passes that happen after pass1 and 5
from using block_type to using bitmap_type. This is a step toward
eliminating the blockmap after pass1 and pass5 are complete.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/lost_n_found.c | 4 ++--
gfs2/fsck/metawalk.c | 12 ++++++------
gfs2/fsck/pass1b.c | 6 +++---
gfs2/fsck/pass2.c | 6 +++---
gfs2/fsck/pass3.c | 8 ++++----
gfs2/fsck/pass4.c | 2 +-
gfs2/fsck/util.c | 2 +-
gfs2/fsck/util.h | 8 ++++++++
8 files changed, 28 insertions(+), 20 deletions(-)
diff --git a/gfs2/fsck/lost_n_found.c b/gfs2/fsck/lost_n_found.c
index ba3ae1d..4816b3c 100644
--- a/gfs2/fsck/lost_n_found.c
+++ b/gfs2/fsck/lost_n_found.c
@@ -131,14 +131,14 @@ void make_sure_lf_exists(struct gfs2_inode *ip)
them in sync so that pass4 can detect and fix any descrepancies. */
set_di_nlink(sdp->md.rooti);
- q = block_type(lf_dip->i_di.di_num.no_addr);
+ q = bitmap_type(sdp, lf_dip->i_di.di_num.no_addr);
if (q != GFS2_BLKST_DINODE) {
lf_was_created = 1;
/* This is a new lost+found directory, so set its block type
and increment link counts for the directories */
/* FIXME: i'd feel better about this if fs_mkdir returned
whether it created a new directory or just found an old one,
- and we used that instead of the block_type to run this */
+ and we used that instead of the bitmap_type to run this */
fsck_blockmap_set(ip, lf_dip->i_di.di_num.no_addr,
_("lost+found dinode"), GFS2_BLKST_DINODE);
dirtree_insert(lf_dip->i_di.di_num);
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 8fcaea3..f401f2c 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -1045,7 +1045,7 @@ static int delete_block_if_notdup(struct gfs2_inode *ip, uint64_t block,
if (!valid_block(ip->i_sbd, block))
return meta_error;
- q = block_type(block);
+ q = bitmap_type(ip->i_sbd, block);
if (q == GFS2_BLKST_FREE) {
log_info( _("%s block %lld (0x%llx), part of inode "
"%lld (0x%llx), was already free.\n"),
@@ -1774,7 +1774,7 @@ int remove_dentry_from_dir(struct gfs2_sbd *sdp, uint64_t dir,
remove_dentry_fxns.private = &dentryblock;
remove_dentry_fxns.check_dentry = remove_dentry;
- q = block_type(dir);
+ q = bitmap_type(sdp, dir);
if (q != GFS2_BLKST_DINODE) {
log_info( _("Parent block is not an inode...ignoring\n"));
return 1;
@@ -1825,7 +1825,7 @@ static int del_eattr_generic(struct gfs2_inode *ip, uint64_t block,
uint8_t q;
if (valid_block(ip->i_sbd, block)) {
- q = block_type(block);
+ q = bitmap_type(ip->i_sbd, block);
if (q == GFS2_BLKST_FREE)
was_free = 1;
ret = delete_block_if_notdup(ip, block, NULL, eatype,
@@ -1925,7 +1925,7 @@ static int alloc_metalist(struct gfs2_inode *ip, uint64_t block,
*is_valid = 1;
*was_duplicate = 0;
*bh = bread(ip->i_sbd, block);
- q = block_type(block);
+ q = bitmap_type(ip->i_sbd, block);
if (q == GFS2_BLKST_FREE) {
log_debug(_("%s reference to new metadata block "
"%lld (0x%llx) is now marked as indirect.\n"),
@@ -1947,7 +1947,7 @@ static int alloc_data(struct gfs2_inode *ip, uint64_t metablock,
/* No need to range_check here--if it was added, it's in range. */
/* We can't check the bitmap here because this function is called
after the bitmap has been set but before the blockmap has. */
- q = block_type(block);
+ q = bitmap_type(ip->i_sbd, block);
if (q == GFS2_BLKST_FREE) {
log_debug(_("%s reference to new data block "
"%lld (0x%llx) is now marked as data.\n"),
@@ -1965,7 +1965,7 @@ static int alloc_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
/* No need to range_check here--if it was added, it's in range. */
/* We can't check the bitmap here because this function is called
after the bitmap has been set but before the blockmap has. */
- q = block_type(block);
+ q = bitmap_type(ip->i_sbd, block);
if (q == GFS2_BLKST_FREE)
fsck_blockmap_set(ip, block, _("newly allocated leaf"),
ip->i_sbd->gfs1 ? GFS2_BLKST_DINODE :
diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
index 16c0aec..66dd0a8 100644
--- a/gfs2/fsck/pass1b.c
+++ b/gfs2/fsck/pass1b.c
@@ -216,7 +216,7 @@ static void resolve_dup_references(struct gfs2_sbd *sdp, struct duptree *dt,
return;
this_ref = get_ref_type(id);
- q = block_type(id->block_no);
+ q = bitmap_type(sdp, id->block_no);
if (inval)
log_warn( _("Invalid "));
/* FIXME: If we already found an acceptable reference to this
@@ -525,7 +525,7 @@ static void resolve_last_reference(struct gfs2_sbd *sdp, struct duptree *dt,
if (dt->dup_flags & DUPFLAG_REF1_IS_DUPL)
clone_dup_ref_in_inode(ip, dt);
- q = block_type(id->block_no);
+ q = bitmap_type(sdp, id->block_no);
if (q == GFS2_BLKST_UNLINKED) {
log_debug( _("The remaining reference inode %lld (0x%llx) is "
"marked invalid: Marking the block as free.\n"),
@@ -883,7 +883,7 @@ int pass1b(struct gfs2_sbd *sdp)
"duplicates.\n"), dups_found);
break;
}
- q = block_type(i);
+ q = bitmap_type(sdp, i);
if (q == GFS2_BLKST_FREE || q == GFS2_BLKST_USED)
continue;
diff --git a/gfs2/fsck/pass2.c b/gfs2/fsck/pass2.c
index 11777eb..afee8dc 100644
--- a/gfs2/fsck/pass2.c
+++ b/gfs2/fsck/pass2.c
@@ -506,7 +506,7 @@ static int basic_dentry_checks(struct gfs2_inode *ip, struct gfs2_dirent *dent,
tmp_name);
}
- *q = block_type(entry->no_addr);
+ *q = bitmap_type(sdp, entry->no_addr);
/* Get the status of the directory inode */
/**
* 1. Blocks marked "invalid" were invalidated due to duplicate
@@ -1720,7 +1720,7 @@ static int check_system_dir(struct gfs2_inode *sysinode, const char *dirname,
}
iblock = sysinode->i_di.di_num.no_addr;
- ds.q = block_type(iblock);
+ ds.q = bitmap_type(sysinode->i_sbd, iblock);
pass2_fxns.private = (void *) &ds;
if (ds.q == GFS2_BLKST_UNLINKED) {
@@ -1998,7 +1998,7 @@ int pass2(struct gfs2_sbd *sdp)
if (is_system_dir(sdp, dirblk))
continue;
- q = block_type(dirblk);
+ q = bitmap_type(sdp, dirblk);
if (q != GFS2_BLKST_DINODE)
continue;
diff --git a/gfs2/fsck/pass3.c b/gfs2/fsck/pass3.c
index 054953f..7732178 100644
--- a/gfs2/fsck/pass3.c
+++ b/gfs2/fsck/pass3.c
@@ -75,7 +75,7 @@ static struct dir_info *mark_and_return_parent(struct gfs2_sbd *sdp,
return NULL;
if (di->dotdot_parent.no_addr == di->treewalk_parent) {
- q_dotdot = block_type(di->dotdot_parent.no_addr);
+ q_dotdot = bitmap_type(sdp, di->dotdot_parent.no_addr);
if (q_dotdot != GFS2_BLKST_DINODE) {
log_err( _("Orphaned directory at block %llu (0x%llx) "
"moved to lost+found\n"),
@@ -95,9 +95,9 @@ static struct dir_info *mark_and_return_parent(struct gfs2_sbd *sdp,
(unsigned long long)di->dotdot_parent.no_addr,
(unsigned long long)di->treewalk_parent,
(unsigned long long)di->treewalk_parent);
- q_dotdot = block_type(di->dotdot_parent.no_addr);
+ q_dotdot = bitmap_type(sdp, di->dotdot_parent.no_addr);
dt_dotdot = dirtree_find(di->dotdot_parent.no_addr);
- q_treewalk = block_type(di->treewalk_parent);
+ q_treewalk = bitmap_type(sdp, di->treewalk_parent);
dt_treewalk = dirtree_find(di->treewalk_parent);
/* if the dotdot entry isn't a directory, but the
* treewalk is, treewalk is correct - if the treewalk
@@ -249,7 +249,7 @@ int pass3(struct gfs2_sbd *sdp)
di = tdi;
continue;
}
- q = block_type(di->dinode.no_addr);
+ q = bitmap_type(sdp, di->dinode.no_addr);
if (q == GFS2_BLKST_UNLINKED) {
log_err( _("Found unlinked directory "
"containing bad block at block %llu"
diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
index 850a2fe..c3fb30a 100644
--- a/gfs2/fsck/pass4.c
+++ b/gfs2/fsck/pass4.c
@@ -70,7 +70,7 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
log_err( _("Found unlinked inode at %llu (0x%llx)\n"),
(unsigned long long)ii->di_num.no_addr,
(unsigned long long)ii->di_num.no_addr);
- q = block_type(ii->di_num.no_addr);
+ q = bitmap_type(sdp, ii->di_num.no_addr);
if (q == GFS2_BLKST_UNLINKED) {
log_err( _("Unlinked inode %llu (0x%llx) contains "
"bad blocks\n"),
diff --git a/gfs2/fsck/util.c b/gfs2/fsck/util.c
index 84402fc..44a41ba 100644
--- a/gfs2/fsck/util.c
+++ b/gfs2/fsck/util.c
@@ -372,7 +372,7 @@ int add_duplicate_ref(struct gfs2_inode *ip, uint64_t block,
return meta_error;
}
id->block_no = ip->i_di.di_num.no_addr;
- q = block_type(ip->i_di.di_num.no_addr);
+ q = bitmap_type(ip->i_sbd, ip->i_di.di_num.no_addr);
/* If it's an invalid dinode, put it first on the invalid
inode reference list otherwise put it on the normal list. */
if (!inode_valid || q == GFS2_BLKST_UNLINKED)
diff --git a/gfs2/fsck/util.h b/gfs2/fsck/util.h
index 62f8ca6..7d842e2 100644
--- a/gfs2/fsck/util.h
+++ b/gfs2/fsck/util.h
@@ -59,6 +59,14 @@ static inline uint8_t block_type(uint64_t bblock)
return btype;
}
+static inline uint8_t bitmap_type(struct gfs2_sbd *sdp, uint64_t bblock)
+{
+ struct rgrp_tree *rgd;
+
+ rgd = gfs2_blk2rgrpd(sdp, bblock);
+ return lgfs2_get_bitmap(sdp, bblock, rgd);
+}
+
static const inline char *block_type_string(uint8_t q)
{
const char *blktyp[] = {"free", "data", "other", "inode", "invalid"};
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 03/40] fsck.gfs2: Change bitmap_type variables to int
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 01/40] fsck.gfs2: Move pass5 to immediately follow pass1 Bob Peterson
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 02/40] fsck.gfs2: Convert block_type to bitmap_type after pass1 and 5 Bob Peterson
@ 2016-05-06 17:38 ` Bob Peterson
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 04/40] fsck.gfs2: Use di_entries to determine if lost+found was created Bob Peterson
` (38 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:38 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch changes all references of "q" from a uint8_t to a normal
int. This will allow us to check for bad return codes at some point.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/fsck.h | 2 +-
gfs2/fsck/lost_n_found.c | 2 +-
gfs2/fsck/metawalk.c | 12 ++++++------
gfs2/fsck/pass1.c | 18 +++++++++---------
gfs2/fsck/pass1b.c | 6 +++---
gfs2/fsck/pass2.c | 16 ++++++++--------
gfs2/fsck/pass3.c | 4 ++--
gfs2/fsck/pass4.c | 2 +-
gfs2/fsck/pass5.c | 2 +-
gfs2/fsck/util.c | 2 +-
gfs2/fsck/util.h | 10 +++++-----
11 files changed, 38 insertions(+), 38 deletions(-)
diff --git a/gfs2/fsck/fsck.h b/gfs2/fsck/fsck.h
index a3c5f72..4cf4ce1 100644
--- a/gfs2/fsck/fsck.h
+++ b/gfs2/fsck/fsck.h
@@ -53,7 +53,7 @@ struct dir_info
struct dir_status {
uint8_t dotdir:1;
uint8_t dotdotdir:1;
- uint8_t q;
+ int q;
uint32_t entry_count;
};
diff --git a/gfs2/fsck/lost_n_found.c b/gfs2/fsck/lost_n_found.c
index 4816b3c..0b57d21 100644
--- a/gfs2/fsck/lost_n_found.c
+++ b/gfs2/fsck/lost_n_found.c
@@ -99,7 +99,7 @@ static void add_dotdot(struct gfs2_inode *ip)
void make_sure_lf_exists(struct gfs2_inode *ip)
{
- uint8_t q;
+ int q;
struct dir_info *di;
struct gfs2_sbd *sdp = ip->i_sbd;
uint32_t mode;
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index f401f2c..3cb5a7a 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -1040,7 +1040,7 @@ static int delete_block_if_notdup(struct gfs2_inode *ip, uint64_t block,
const char *btype, int *was_duplicate,
void *private)
{
- uint8_t q;
+ int q;
if (!valid_block(ip->i_sbd, block))
return meta_error;
@@ -1760,7 +1760,7 @@ int remove_dentry_from_dir(struct gfs2_sbd *sdp, uint64_t dir,
{
struct metawalk_fxns remove_dentry_fxns = {0};
struct gfs2_inode *ip;
- uint8_t q;
+ int q;
int error;
log_debug( _("Removing dentry %llu (0x%llx) from directory %llu"
@@ -1822,7 +1822,7 @@ static int del_eattr_generic(struct gfs2_inode *ip, uint64_t block,
{
int ret = 0;
int was_free = 0;
- uint8_t q;
+ int q;
if (valid_block(ip->i_sbd, block)) {
q = bitmap_type(ip->i_sbd, block);
@@ -1916,7 +1916,7 @@ static int alloc_metalist(struct gfs2_inode *ip, uint64_t block,
struct gfs2_buffer_head **bh, int h, int *is_valid,
int *was_duplicate, void *private)
{
- uint8_t q;
+ int q;
const char *desc = (const char *)private;
/* No need to range_check here--if it was added, it's in range. */
@@ -1941,7 +1941,7 @@ static int alloc_data(struct gfs2_inode *ip, uint64_t metablock,
uint64_t block, void *private,
struct gfs2_buffer_head *bh, uint64_t *ptr)
{
- uint8_t q;
+ int q;
const char *desc = (const char *)private;
/* No need to range_check here--if it was added, it's in range. */
@@ -1960,7 +1960,7 @@ static int alloc_data(struct gfs2_inode *ip, uint64_t metablock,
static int alloc_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
{
- uint8_t q;
+ int q;
/* No need to range_check here--if it was added, it's in range. */
/* We can't check the bitmap here because this function is called
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index 875f3d3..cc70d5f 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -214,7 +214,7 @@ struct metawalk_fxns sysdir_fxns = {
static int p1check_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
{
struct block_count *bc = (struct block_count *) private;
- uint8_t q;
+ int q;
/* Note if we've gotten this far, the block has already passed the
check in metawalk: gfs2_check_meta(lbh, GFS2_METATYPE_LF).
@@ -247,7 +247,7 @@ static int check_metalist(struct gfs2_inode *ip, uint64_t block,
struct gfs2_buffer_head **bh, int h, int *is_valid,
int *was_duplicate, void *private)
{
- uint8_t q;
+ int q;
int iblk_type;
struct gfs2_buffer_head *nbh;
struct block_count *bc = (struct block_count *)private;
@@ -445,7 +445,7 @@ static int check_data(struct gfs2_inode *ip, uint64_t metablock,
uint64_t block, void *private,
struct gfs2_buffer_head *bbh, uint64_t *ptr)
{
- uint8_t q;
+ int q;
struct block_count *bc = (struct block_count *) private;
if (!valid_block(ip->i_sbd, block)) {
@@ -581,7 +581,7 @@ static int undo_eattr_indir_or_leaf(struct gfs2_inode *ip, uint64_t block,
void *private)
{
struct gfs2_sbd *sdp = ip->i_sbd;
- uint8_t q;
+ int q;
int error;
struct block_count *bc = (struct block_count *) private;
@@ -629,7 +629,7 @@ static int check_eattr_indir(struct gfs2_inode *ip, uint64_t indirect,
{
struct gfs2_sbd *sdp = ip->i_sbd;
int ret = 0;
- uint8_t q;
+ int q;
struct block_count *bc = (struct block_count *) private;
/* This inode contains an eattr - it may be invalid, but the
@@ -720,7 +720,7 @@ static int check_ealeaf_block(struct gfs2_inode *ip, uint64_t block, int btype,
{
struct gfs2_buffer_head *leaf_bh = NULL;
struct gfs2_sbd *sdp = ip->i_sbd;
- uint8_t q;
+ int q;
struct block_count *bc = (struct block_count *) private;
q = block_type(block);
@@ -948,7 +948,7 @@ static int mark_block_invalid(struct gfs2_inode *ip, uint64_t block,
enum dup_ref_type reftype, const char *btype,
int *is_valid, int *was_duplicate)
{
- uint8_t q;
+ int q;
/* If the block isn't valid, we obviously can't invalidate it.
* However, if we return an error, invalidating will stop, and
@@ -1051,7 +1051,7 @@ static int rangecheck_block(struct gfs2_inode *ip, uint64_t block,
void *private)
{
long *bad_pointers = (long *)private;
- uint8_t q;
+ int q;
if (!valid_block(ip->i_sbd, block)) {
(*bad_pointers)++;
@@ -1583,7 +1583,7 @@ static int pass1_process_bitmap(struct gfs2_sbd *sdp, struct rgrp_tree *rgd, uin
unsigned i;
uint64_t block;
struct gfs2_inode *ip;
- uint8_t q;
+ int q;
/* Readahead numbers arrived at by experiment */
unsigned rawin = 50;
unsigned ralen = 100 * sdp->bsize;
diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
index 66dd0a8..a7b3e42 100644
--- a/gfs2/fsck/pass1b.c
+++ b/gfs2/fsck/pass1b.c
@@ -203,7 +203,7 @@ static void resolve_dup_references(struct gfs2_sbd *sdp, struct duptree *dt,
enum dup_ref_type this_ref;
struct inode_info *ii;
int found_good_ref = 0;
- uint8_t q;
+ int q;
osi_list_foreach_safe(tmp, ref_list, x) {
if (skip_this_pass || fsck_abort)
@@ -502,7 +502,7 @@ static void resolve_last_reference(struct gfs2_sbd *sdp, struct duptree *dt,
struct gfs2_inode *ip;
struct inode_with_dups *id;
osi_list_t *tmp;
- uint8_t q;
+ int q;
log_notice( _("Block %llu (0x%llx) has only one remaining "
"valid inode referencing it.\n"),
@@ -856,7 +856,7 @@ int pass1b(struct gfs2_sbd *sdp)
{
struct duptree *dt;
uint64_t i;
- uint8_t q;
+ int q;
struct osi_node *n;
int rc = FSCK_OK;
diff --git a/gfs2/fsck/pass2.c b/gfs2/fsck/pass2.c
index afee8dc..f114d2b 100644
--- a/gfs2/fsck/pass2.c
+++ b/gfs2/fsck/pass2.c
@@ -135,7 +135,7 @@ static const char *de_type_string(uint8_t de_type)
return de_types[3]; /* invalid */
}
-static int check_file_type(uint64_t block, uint8_t de_type, uint8_t q,
+static int check_file_type(uint64_t block, uint8_t de_type, int q,
int gfs1, int *isdir)
{
struct dir_info *dt;
@@ -176,7 +176,7 @@ struct metawalk_fxns pass2_fxns_delete = {
*/
static int bad_formal_ino(struct gfs2_inode *ip, struct gfs2_dirent *dent,
struct gfs2_inum entry, const char *tmp_name,
- uint8_t q, struct gfs2_dirent *de,
+ int q, struct gfs2_dirent *de,
struct gfs2_buffer_head *bh)
{
struct inode_info *ii;
@@ -299,7 +299,7 @@ static int wrong_leaf(struct gfs2_inode *ip, struct gfs2_inum *entry,
int hash_index, struct gfs2_buffer_head *bh,
struct dir_status *ds, struct gfs2_dirent *dent,
struct gfs2_dirent *de, struct gfs2_dirent *prev_de,
- uint32_t *count, uint8_t q)
+ uint32_t *count, int q)
{
struct gfs2_sbd *sdp = ip->i_sbd;
struct gfs2_buffer_head *dest_lbh;
@@ -437,7 +437,7 @@ static int wrong_leaf(struct gfs2_inode *ip, struct gfs2_inum *entry,
static int basic_dentry_checks(struct gfs2_inode *ip, struct gfs2_dirent *dent,
struct gfs2_inum *entry, const char *tmp_name,
uint32_t *count, struct gfs2_dirent *de,
- struct dir_status *ds, uint8_t *q,
+ struct dir_status *ds, int *q,
struct gfs2_buffer_head *bh, int *isdir)
{
struct gfs2_sbd *sdp = ip->i_sbd;
@@ -643,7 +643,7 @@ static int check_dentry(struct gfs2_inode *ip, struct gfs2_dirent *dent,
uint32_t *count, int *lindex, void *priv)
{
struct gfs2_sbd *sdp = ip->i_sbd;
- uint8_t q = 0;
+ int q = 0;
char tmp_name[MAX_FILENAME];
struct gfs2_inum entry;
struct dir_status *ds = (struct dir_status *) priv;
@@ -955,7 +955,7 @@ static int lost_leaf(struct gfs2_inode *ip, uint64_t *tbl, uint64_t leafno,
} else {
uint32_t count;
struct dir_status ds = {0};
- uint8_t q = 0;
+ int q = 0;
error = basic_dentry_checks(ip, dent, &de.de_inum,
tmp_name, &count, &de,
@@ -1013,7 +1013,7 @@ static int basic_check_dentry(struct gfs2_inode *ip, struct gfs2_dirent *dent,
struct gfs2_buffer_head *bh, char *filename,
uint32_t *count, int *lindex, void *priv)
{
- uint8_t q = 0;
+ int q = 0;
char tmp_name[MAX_FILENAME];
struct gfs2_inum entry;
struct dir_status *ds = (struct dir_status *) priv;
@@ -1953,7 +1953,7 @@ static int pass2_check_dir(struct gfs2_sbd *sdp, struct gfs2_inode *ip)
int pass2(struct gfs2_sbd *sdp)
{
uint64_t dirblk;
- uint8_t q;
+ int q;
/* Check all the system directory inodes. */
if (!sdp->gfs1 &&
diff --git a/gfs2/fsck/pass3.c b/gfs2/fsck/pass3.c
index 7732178..a3c0a60 100644
--- a/gfs2/fsck/pass3.c
+++ b/gfs2/fsck/pass3.c
@@ -65,7 +65,7 @@ static struct dir_info *mark_and_return_parent(struct gfs2_sbd *sdp,
struct dir_info *di)
{
struct dir_info *pdi;
- uint8_t q_dotdot, q_treewalk;
+ int q_dotdot, q_treewalk;
int error = 0;
struct dir_info *dt_dotdot, *dt_treewalk;
@@ -181,7 +181,7 @@ int pass3(struct gfs2_sbd *sdp)
struct osi_node *tmp, *next = NULL;
struct dir_info *di, *tdi;
struct gfs2_inode *ip;
- uint8_t q;
+ int q;
struct alloc_state lf_as = {.as_blocks = 0, .as_meta_goal = 0};
di = dirtree_find(sdp->md.rooti->i_di.di_num.no_addr);
diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
index c3fb30a..dcd5ba7 100644
--- a/gfs2/fsck/pass4.c
+++ b/gfs2/fsck/pass4.c
@@ -47,7 +47,7 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
struct inode_info *ii;
struct gfs2_inode *ip;
int lf_addition = 0;
- uint8_t q;
+ int q;
struct alloc_state lf_as = {.as_blocks = 0, .as_meta_goal = 0};
/* FIXME: should probably factor this out into a generic
diff --git a/gfs2/fsck/pass5.c b/gfs2/fsck/pass5.c
index fab3e5c..99ff0a1 100644
--- a/gfs2/fsck/pass5.c
+++ b/gfs2/fsck/pass5.c
@@ -19,7 +19,7 @@ static int check_block_status(struct gfs2_sbd *sdp, char *buffer,
unsigned char *byte, *end;
unsigned int bit;
unsigned char rg_status;
- uint8_t q;
+ int q;
uint64_t block;
/* FIXME verify cast */
diff --git a/gfs2/fsck/util.c b/gfs2/fsck/util.c
index 44a41ba..55bd050 100644
--- a/gfs2/fsck/util.c
+++ b/gfs2/fsck/util.c
@@ -364,7 +364,7 @@ int add_duplicate_ref(struct gfs2_inode *ip, uint64_t block,
if (id == NULL) {
/* Check for the inode on the invalid inode reference list. */
- uint8_t q;
+ int q;
id = calloc(1, sizeof(*id));
if (!id) {
diff --git a/gfs2/fsck/util.h b/gfs2/fsck/util.h
index 7d842e2..8709193 100644
--- a/gfs2/fsck/util.h
+++ b/gfs2/fsck/util.h
@@ -47,11 +47,11 @@ static inline int astate_changed(struct gfs2_inode *ip, struct alloc_state *as)
return 0;
}
-static inline uint8_t block_type(uint64_t bblock)
+static inline int block_type(uint64_t bblock)
{
static unsigned char *byte;
static uint64_t b;
- static uint8_t btype;
+ static int btype;
byte = bl->map + BLOCKMAP_SIZE2(bblock);
b = BLOCKMAP_BYTE_OFFSET2(bblock);
@@ -59,7 +59,7 @@ static inline uint8_t block_type(uint64_t bblock)
return btype;
}
-static inline uint8_t bitmap_type(struct gfs2_sbd *sdp, uint64_t bblock)
+static inline int bitmap_type(struct gfs2_sbd *sdp, uint64_t bblock)
{
struct rgrp_tree *rgd;
@@ -67,10 +67,10 @@ static inline uint8_t bitmap_type(struct gfs2_sbd *sdp, uint64_t bblock)
return lgfs2_get_bitmap(sdp, bblock, rgd);
}
-static const inline char *block_type_string(uint8_t q)
+static const inline char *block_type_string(int q)
{
const char *blktyp[] = {"free", "data", "other", "inode", "invalid"};
- if (q <= GFS2_BLKST_DINODE)
+ if (q >= GFS2_BLKST_FREE && q <= GFS2_BLKST_DINODE)
return (blktyp[q]);
return blktyp[4];
}
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 04/40] fsck.gfs2: Use di_entries to determine if lost+found was created
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (2 preceding siblings ...)
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 03/40] fsck.gfs2: Change bitmap_type variables to int Bob Peterson
@ 2016-05-06 17:38 ` Bob Peterson
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 05/40] fsck.gfs2: pass1b shouldn't complain about non-bitmap blocks Bob Peterson
` (37 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:38 UTC (permalink / raw)
To: cluster-devel.redhat.com
Before this patch, function make_sure_lf_exists compared the bitmap
to the blockmap to determine if the lost+found directory was created.
Since we're phasing out the blockmap, and only want to use the
bitmap, we need an alternative. This patch changes it to check the
number of directory entries in the root directory has increased.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/lost_n_found.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/gfs2/fsck/lost_n_found.c b/gfs2/fsck/lost_n_found.c
index 0b57d21..cd847f8 100644
--- a/gfs2/fsck/lost_n_found.c
+++ b/gfs2/fsck/lost_n_found.c
@@ -99,14 +99,15 @@ static void add_dotdot(struct gfs2_inode *ip)
void make_sure_lf_exists(struct gfs2_inode *ip)
{
- int q;
struct dir_info *di;
struct gfs2_sbd *sdp = ip->i_sbd;
uint32_t mode;
+ int root_entries;
if (lf_dip)
return;
+ root_entries = sdp->md.rooti->i_di.di_entries;
log_info( _("Locating/Creating lost+found directory\n"));
/* if this is gfs1, we have to trick createi into using
@@ -131,8 +132,7 @@ void make_sure_lf_exists(struct gfs2_inode *ip)
them in sync so that pass4 can detect and fix any descrepancies. */
set_di_nlink(sdp->md.rooti);
- q = bitmap_type(sdp, lf_dip->i_di.di_num.no_addr);
- if (q != GFS2_BLKST_DINODE) {
+ if (sdp->md.rooti->i_di.di_entries > root_entries) {
lf_was_created = 1;
/* This is a new lost+found directory, so set its block type
and increment link counts for the directories */
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 05/40] fsck.gfs2: pass1b shouldn't complain about non-bitmap blocks
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (3 preceding siblings ...)
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 04/40] fsck.gfs2: Use di_entries to determine if lost+found was created Bob Peterson
@ 2016-05-06 17:38 ` Bob Peterson
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 06/40] fsck.gfs2: Change all fsck_blockmap_set to fsck_bitmap_set Bob Peterson
` (36 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:38 UTC (permalink / raw)
To: cluster-devel.redhat.com
One of the previous patches switched pass1b to stop using the
blockmap and start using the bitmap. One major difference between
them is that the blockmap has a value (0 - 3) for every block on
the device, whereas the bitmap has a value (0 - 3) for every block
in the file system. It does not have a value for blocks that are
on the device but aren't in the file system, such as the superblock
and the blocks preceding it, the blocks reserved for the rgrps
and their respective bitmaps.
That means the block_map function returned 0 for non-fs blocks,
but the bitmap function returned -1 (meaning invalid block).
The result was that pass1b complained about every non-fs block.
This patch makes pass1b check the return code from function
bitmap_type() and if it's a negative number, it skips over it.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass1b.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
index a7b3e42..ee636fe 100644
--- a/gfs2/fsck/pass1b.c
+++ b/gfs2/fsck/pass1b.c
@@ -885,7 +885,7 @@ int pass1b(struct gfs2_sbd *sdp)
}
q = bitmap_type(sdp, i);
- if (q == GFS2_BLKST_FREE || q == GFS2_BLKST_USED)
+ if (q == GFS2_BLKST_FREE || q == GFS2_BLKST_USED || q < 0)
continue;
if (q == GFS2_BLKST_UNLINKED)
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 06/40] fsck.gfs2: Change all fsck_blockmap_set to fsck_bitmap_set
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (4 preceding siblings ...)
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 05/40] fsck.gfs2: pass1b shouldn't complain about non-bitmap blocks Bob Peterson
@ 2016-05-06 17:38 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 07/40] fsck.gfs2: Move set_ip_blockmap to pass1 Bob Peterson
` (35 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:38 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch changes all functions past pass1 so that they set the
bits in the bitmap rather than bits in the blockmap. This is a
step toward being able to free the bitmap after pass1.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/lost_n_found.c | 4 +--
gfs2/fsck/metawalk.c | 36 +++++++++++--------
gfs2/fsck/metawalk.h | 7 ++++
gfs2/fsck/pass1b.c | 90 ++++++++++++++++++++++++++++++++++--------------
gfs2/fsck/pass2.c | 33 ++++++++----------
gfs2/fsck/pass3.c | 17 +++------
gfs2/fsck/pass4.c | 16 ++++-----
7 files changed, 122 insertions(+), 81 deletions(-)
diff --git a/gfs2/fsck/lost_n_found.c b/gfs2/fsck/lost_n_found.c
index cd847f8..9c85f52 100644
--- a/gfs2/fsck/lost_n_found.c
+++ b/gfs2/fsck/lost_n_found.c
@@ -139,8 +139,8 @@ void make_sure_lf_exists(struct gfs2_inode *ip)
/* FIXME: i'd feel better about this if fs_mkdir returned
whether it created a new directory or just found an old one,
and we used that instead of the bitmap_type to run this */
- fsck_blockmap_set(ip, lf_dip->i_di.di_num.no_addr,
- _("lost+found dinode"), GFS2_BLKST_DINODE);
+ fsck_bitmap_set(ip, lf_dip->i_di.di_num.no_addr,
+ _("lost+found dinode"), GFS2_BLKST_DINODE);
dirtree_insert(lf_dip->i_di.di_num);
/* root inode links to lost+found */
incr_link_count(sdp->md.rooti->i_di.di_num, lf_dip, _("root"));
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 3cb5a7a..ce23220 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -110,12 +110,11 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
}
/*
- * _fsck_blockmap_set - Mark a block in the 4-bit blockmap and the 2-bit
- * bitmap, and adjust free space accordingly.
+ * _fsck_bitmap_set - Mark a block in the bitmap, and adjust free space.
*/
-int _fsck_blockmap_set(struct gfs2_inode *ip, uint64_t bblock,
- const char *btype, int mark,
- int error_on_dinode, const char *caller, int fline)
+int _fsck_bitmap_set(struct gfs2_inode *ip, uint64_t bblock,
+ const char *btype, int mark,
+ int error_on_dinode, const char *caller, int fline)
{
int error;
static int prev_ino_addr = 0;
@@ -172,19 +171,26 @@ int _fsck_blockmap_set(struct gfs2_inode *ip, uint64_t bblock,
prev_mark = mark;
prev_caller = caller;
}
-
- /* First, check the rgrp bitmap against what we think it should be.
- If that fails, it's an invalid block--part of an rgrp. */
error = check_n_fix_bitmap(ip->i_sbd, bblock, error_on_dinode, mark);
- if (error) {
- if (error < 0)
- log_err( _("This block is not represented in the "
- "bitmap.\n"));
+ if (error < 0)
+ log_err(_("This block is not represented in the bitmap.\n"));
+ return error;
+}
+
+/*
+ * _fsck_blockmap_set - Mark a block in the 4-bit blockmap and the 2-bit
+ * bitmap, and adjust free space accordingly.
+ */
+int _fsck_blockmap_set(struct gfs2_inode *ip, uint64_t bblock,
+ const char *btype, int mark,
+ int error_on_dinode, const char *caller, int fline)
+{
+ int error = _fsck_bitmap_set(ip, bblock, btype, mark, error_on_dinode,
+ caller, fline);
+ if (error)
return error;
- }
- error = gfs2_blockmap_set(bl, bblock, mark);
- return error;
+ return gfs2_blockmap_set(bl, bblock, mark);
}
struct duptree *dupfind(uint64_t block)
diff --git a/gfs2/fsck/metawalk.h b/gfs2/fsck/metawalk.h
index 8ec38d0..90bd028 100644
--- a/gfs2/fsck/metawalk.h
+++ b/gfs2/fsck/metawalk.h
@@ -49,6 +49,9 @@ extern int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
extern int _fsck_blockmap_set(struct gfs2_inode *ip, uint64_t bblock,
const char *btype, int mark, int error_on_dinode,
const char *caller, int line);
+extern int _fsck_bitmap_set(struct gfs2_inode *ip, uint64_t bblock,
+ const char *btype, int mark, int error_on_dinode,
+ const char *caller, int line);
extern int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk,
int error_on_dinode, int new_blockmap_state);
extern void reprocess_inode(struct gfs2_inode *ip, const char *desc);
@@ -65,6 +68,10 @@ extern int repair_leaf(struct gfs2_inode *ip, uint64_t *leaf_no, int lindex,
#define is_duplicate(dblock) ((dupfind(dblock)) ? 1 : 0)
+#define fsck_bitmap_set(ip, b, bt, m) \
+ _fsck_bitmap_set(ip, b, bt, m, 0, __FUNCTION__, __LINE__)
+#define fsck_bitmap_set_noino(ip, b, bt, m) \
+ _fsck_bitmap_set(ip, b, bt, m, 1, __FUNCTION__, __LINE__)
#define fsck_blockmap_set(ip, b, bt, m) \
_fsck_blockmap_set(ip, b, bt, m, 0, __FUNCTION__, __LINE__)
#define fsck_blkmap_set_noino(ip, b, bt, m) \
diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
index ee636fe..20b603c 100644
--- a/gfs2/fsck/pass1b.c
+++ b/gfs2/fsck/pass1b.c
@@ -348,9 +348,9 @@ static void resolve_dup_references(struct gfs2_sbd *sdp, struct duptree *dt,
ii = inodetree_find(ip->i_di.di_num.no_addr);
if (ii)
inodetree_delete(ii);
- fsck_blockmap_set(ip, ip->i_di.di_num.no_addr,
+ fsck_bitmap_set(ip, ip->i_di.di_num.no_addr,
_("duplicate referencing bad"),
- GFS2_BLKST_UNLINKED);
+ GFS2_BLKST_UNLINKED);
/* We delete the dup_handler inode count and
duplicate id BEFORE clearing the metadata,
because if this is the last reference to
@@ -428,8 +428,8 @@ static int clone_data(struct gfs2_inode *ip, uint64_t metablock,
if (!error) {
clone_bh = bread(ip->i_sbd, clonet->dup_block);
if (clone_bh) {
- fsck_blockmap_set(ip, cloneblock, _("data"),
- GFS2_BLKST_USED);
+ fsck_bitmap_set(ip, cloneblock, _("data"),
+ GFS2_BLKST_USED);
clone_bh->b_blocknr = cloneblock;
bmodified(clone_bh);
brelse(clone_bh);
@@ -496,6 +496,46 @@ static void clone_dup_ref_in_inode(struct gfs2_inode *ip, struct duptree *dt)
}
}
+static int set_ip_bitmap(struct gfs2_inode *ip)
+{
+ uint64_t block = ip->i_bh->b_blocknr;
+ uint32_t mode;
+ const char *ty;
+
+ if (ip->i_sbd->gfs1)
+ mode = gfs_to_gfs2_mode(ip);
+ else
+ mode = ip->i_di.di_mode & S_IFMT;
+
+ switch (mode) {
+ case S_IFDIR:
+ ty = _("directory");
+ break;
+ case S_IFREG:
+ ty = _("file");
+ break;
+ case S_IFLNK:
+ ty = _("symlink");
+ break;
+ case S_IFBLK:
+ ty = _("block device");
+ break;
+ case S_IFCHR:
+ ty = _("character device");
+ break;
+ case S_IFIFO:
+ ty = _("fifo");
+ break;
+ case S_IFSOCK:
+ ty = _("socket");
+ break;
+ default:
+ return -EINVAL;
+ }
+ fsck_bitmap_set(ip, block, ty, GFS2_BLKST_DINODE);
+ return 0;
+}
+
static void resolve_last_reference(struct gfs2_sbd *sdp, struct duptree *dt,
enum dup_ref_type acceptable_ref)
{
@@ -531,31 +571,31 @@ static void resolve_last_reference(struct gfs2_sbd *sdp, struct duptree *dt,
"marked invalid: Marking the block as free.\n"),
(unsigned long long)id->block_no,
(unsigned long long)id->block_no);
- fsck_blockmap_set(ip, dt->block, _("reference-repaired leaf"),
+ fsck_bitmap_set(ip, dt->block, _("reference-repaired leaf"),
GFS2_BLKST_FREE);
} else if (id->reftypecount[ref_is_inode]) {
- set_ip_blockmap(ip, 0); /* 0=do not add to dirtree */
+ set_ip_bitmap(ip);
} else if (id->reftypecount[ref_as_data]) {
- fsck_blockmap_set(ip, dt->block, _("reference-repaired data"),
- GFS2_BLKST_USED);
+ fsck_bitmap_set(ip, dt->block, _("reference-repaired data"),
+ GFS2_BLKST_USED);
} else if (id->reftypecount[ref_as_meta]) {
if (is_dir(&ip->i_di, sdp->gfs1))
- fsck_blockmap_set(ip, dt->block,
- _("reference-repaired leaf"),
- sdp->gfs1 ? GFS2_BLKST_DINODE :
- GFS2_BLKST_USED);
+ fsck_bitmap_set(ip, dt->block,
+ _("reference-repaired leaf"),
+ sdp->gfs1 ? GFS2_BLKST_DINODE :
+ GFS2_BLKST_USED);
else
- fsck_blockmap_set(ip, dt->block,
- _("reference-repaired indirect"),
- sdp->gfs1 ? GFS2_BLKST_DINODE :
- GFS2_BLKST_USED);
+ fsck_bitmap_set(ip, dt->block,
+ _("reference-repaired indirect"),
+ sdp->gfs1 ? GFS2_BLKST_DINODE :
+ GFS2_BLKST_USED);
} else {
if (acceptable_ref == ref_as_ea)
- fsck_blockmap_set(ip, dt->block,
- _("reference-repaired extended "
- "attribute"),
- sdp->gfs1 ? GFS2_BLKST_DINODE :
- GFS2_BLKST_USED);
+ fsck_bitmap_set(ip, dt->block,
+ _("reference-repaired extended "
+ "attribute"),
+ sdp->gfs1 ? GFS2_BLKST_DINODE :
+ GFS2_BLKST_USED);
else {
log_err(_("Error: The remaining reference to block "
" %lld (0x%llx) is as extended attribute, "
@@ -574,9 +614,9 @@ static void resolve_last_reference(struct gfs2_sbd *sdp, struct duptree *dt,
ip->i_di.di_flags &= ~GFS2_DIF_EA_INDIRECT;
ip->i_di.di_blocks--;
bmodified(ip->i_bh);
- fsck_blockmap_set(ip, dt->block,
- _("reference-repaired EA"),
- GFS2_BLKST_FREE);
+ fsck_bitmap_set(ip, dt->block,
+ _("reference-repaired EA"),
+ GFS2_BLKST_FREE);
log_err(_("The bad extended attribute was "
"removed.\n"));
} else {
@@ -725,8 +765,6 @@ static int handle_dup_blk(struct gfs2_sbd *sdp, struct duptree *dt)
(unsigned long long)dup_blk);
if (dh.dt)
dup_delete(dh.dt);
- /* Now fix the block type of the block in question. */
- gfs2_blockmap_set(bl, dup_blk, GFS2_BLKST_FREE);
check_n_fix_bitmap(sdp, dup_blk, 0, GFS2_BLKST_FREE);
}
}
diff --git a/gfs2/fsck/pass2.c b/gfs2/fsck/pass2.c
index f114d2b..214fa03 100644
--- a/gfs2/fsck/pass2.c
+++ b/gfs2/fsck/pass2.c
@@ -375,9 +375,9 @@ static int wrong_leaf(struct gfs2_inode *ip, struct gfs2_inum *entry,
(unsigned long long)real_leaf,
(unsigned long long)real_leaf,
(unsigned long long)ip->i_di.di_blocks);
- fsck_blockmap_set(ip, real_leaf, _("split leaf"),
- sdp->gfs1 ? GFS2_BLKST_DINODE :
- GFS2_BLKST_USED);
+ fsck_bitmap_set(ip, real_leaf, _("split leaf"),
+ sdp->gfs1 ? GFS2_BLKST_DINODE :
+ GFS2_BLKST_USED);
}
/* If the misplaced dirent was supposed to be earlier in the
hash table, we need to adjust our counts for the blocks
@@ -475,9 +475,9 @@ static int basic_dentry_checks(struct gfs2_inode *ip, struct gfs2_dirent *dent,
return 0;
}
/* Don't be tempted to do this:
- fsck_blockmap_set(ip, ip->i_di.di_num.no_addr,
- _("corrupt directory entry"),
- GFS2_BLKST_FREE);
+ fsck_bitmap_set(ip, ip->i_di.di_num.no_addr,
+ _("corrupt directory entry"),
+ GFS2_BLKST_FREE);
We can't free it because another dir may have a valid reference
to it. Just return 1 so we can delete the bad dirent. */
log_err( _("Bad directory entry deleted.\n"));
@@ -893,9 +893,9 @@ static void pad_with_leafblks(struct gfs2_inode *ip, uint64_t *tbl,
(unsigned long long)new_leaf_blk,
(unsigned long long)new_leaf_blk,
lindex, lindex, new_len);
- fsck_blockmap_set(ip, new_leaf_blk, _("pad leaf"),
- ip->i_sbd->gfs1 ?
- GFS2_BLKST_DINODE : GFS2_BLKST_USED);
+ fsck_bitmap_set(ip, new_leaf_blk, _("pad leaf"),
+ ip->i_sbd->gfs1 ?
+ GFS2_BLKST_DINODE : GFS2_BLKST_USED);
/* Fix the hash table in memory to have the new leaf */
for (i = 0; i < new_len; i++)
tbl[lindex + i] = cpu_to_be64(new_leaf_blk);
@@ -997,7 +997,7 @@ static int lost_leaf(struct gfs2_inode *ip, uint64_t *tbl, uint64_t leafno,
log_err(_("Directory entries from misplaced leaf block were relocated "
"to lost+found.\n"));
/* Free the lost leaf. */
- fsck_blockmap_set(ip, leafno, _("lost leaf"), GFS2_BLKST_FREE);
+ fsck_bitmap_set(ip, leafno, _("lost leaf"), GFS2_BLKST_FREE);
ip->i_di.di_blocks--;
bmodified(ip->i_bh);
/* Now we have to deal with the bad hash table entries pointing to the
@@ -1228,9 +1228,9 @@ static int fix_hashtable(struct gfs2_inode *ip, uint64_t *tbl, unsigned hsize,
"(0x%llx) for index %d (0x%x)\n"),
(unsigned long long)new_leaf_blk,
(unsigned long long)new_leaf_blk, lindex, lindex);
- fsck_blockmap_set(ip, new_leaf_blk, _("split leaf"),
- ip->i_sbd->gfs1 ?
- GFS2_BLKST_DINODE : GFS2_BLKST_USED);
+ fsck_bitmap_set(ip, new_leaf_blk, _("split leaf"),
+ ip->i_sbd->gfs1 ?
+ GFS2_BLKST_DINODE : GFS2_BLKST_USED);
log_err(_("Hash table repaired.\n"));
/* Fix up the hash table in memory to include the new leaf */
for (i = 0; i < *proper_len; i++)
@@ -1695,7 +1695,7 @@ build_it:
log_err(_("Error rebuilding %s.\n"), fn);
return -1;
}
- fsck_blockmap_set(ip, ip->i_di.di_num.no_addr, fn, GFS2_BLKST_DINODE);
+ fsck_bitmap_set(ip, ip->i_di.di_num.no_addr, fn, GFS2_BLKST_DINODE);
reprocess_inode(ip, fn);
log_err(_("System file %s rebuilt.\n"), fn);
goto out_good;
@@ -1742,7 +1742,7 @@ static int check_system_dir(struct gfs2_inode *sysinode, const char *dirname,
return -1;
}
if (error > 0)
- fsck_blockmap_set(sysinode, iblock, dirname, GFS2_BLKST_FREE);
+ fsck_bitmap_set(sysinode, iblock, dirname, GFS2_BLKST_FREE);
if (check_inode_eattr(sysinode, &pass2_fxns)) {
stack;
@@ -1888,9 +1888,6 @@ static int pass2_check_dir(struct gfs2_sbd *sdp, struct gfs2_inode *ip)
log_debug(_("Directory block %lld (0x%llx) is now marked as 'invalid'\n"),
(unsigned long long)dirblk, (unsigned long long)dirblk);
- /* Can't use fsck_blockmap_set here because we don't
- have an inode in memory. */
- gfs2_blockmap_set(bl, dirblk, GFS2_BLKST_FREE);
check_n_fix_bitmap(sdp, dirblk, 0, GFS2_BLKST_FREE);
}
diff --git a/gfs2/fsck/pass3.c b/gfs2/fsck/pass3.c
index a3c0a60..dad1f5c 100644
--- a/gfs2/fsck/pass3.c
+++ b/gfs2/fsck/pass3.c
@@ -250,6 +250,7 @@ int pass3(struct gfs2_sbd *sdp)
continue;
}
q = bitmap_type(sdp, di->dinode.no_addr);
+ ip = fsck_load_inode(sdp, di->dinode.no_addr);
if (q == GFS2_BLKST_UNLINKED) {
log_err( _("Found unlinked directory "
"containing bad block at block %llu"
@@ -264,12 +265,9 @@ int pass3(struct gfs2_sbd *sdp)
di->dinode.no_addr,
(unsigned long long)
di->dinode.no_addr);
- /* Can't use fsck_blockmap_set
- because we don't have ip */
- gfs2_blockmap_set(bl, di->dinode.no_addr,
- GFS2_BLKST_FREE);
check_n_fix_bitmap(sdp, di->dinode.no_addr,
0, GFS2_BLKST_FREE);
+ fsck_inode_put(&ip);
break;
} else
log_err( _("Unlinked directory with bad block remains\n"));
@@ -287,13 +285,10 @@ int pass3(struct gfs2_sbd *sdp)
"marked as free\n"),
(unsigned long long)di->dinode.no_addr,
(unsigned long long)di->dinode.no_addr);
- /* Can't use fsck_blockmap_set
- because we don't have ip */
- gfs2_blockmap_set(bl, di->dinode.no_addr,
- GFS2_BLKST_FREE);
check_n_fix_bitmap(sdp, di->dinode.no_addr, 0,
GFS2_BLKST_FREE);
log_err( _("The block was cleared\n"));
+ fsck_inode_put(&ip);
break;
}
@@ -301,17 +296,15 @@ int pass3(struct gfs2_sbd *sdp)
" (0x%llx)\n"),
(unsigned long long)di->dinode.no_addr,
(unsigned long long)di->dinode.no_addr);
- ip = fsck_load_inode(sdp, di->dinode.no_addr);
/* Don't skip zero size directories with eattrs */
if (!ip->i_di.di_size && !ip->i_di.di_eattr){
log_err( _("Unlinked directory has zero "
"size.\n"));
if (query( _("Remove zero-size unlinked "
"directory? (y/n) "))) {
- fsck_blockmap_set(ip,
- di->dinode.no_addr,
+ fsck_bitmap_set(ip, di->dinode.no_addr,
_("zero-sized unlinked inode"),
- GFS2_BLKST_FREE);
+ GFS2_BLKST_FREE);
fsck_inode_put(&ip);
break;
} else {
diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
index dcd5ba7..25c63b7 100644
--- a/gfs2/fsck/pass4.c
+++ b/gfs2/fsck/pass4.c
@@ -82,9 +82,9 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
check_inode_eattr(ip,
&pass4_fxns_delete);
check_metatree(ip, &pass4_fxns_delete);
- fsck_blockmap_set(ip, ii->di_num.no_addr,
- _("bad unlinked"),
- GFS2_BLKST_FREE);
+ fsck_bitmap_set(ip, ii->di_num.no_addr,
+ _("bad unlinked"),
+ GFS2_BLKST_FREE);
fsck_inode_put(&ip);
continue;
} else
@@ -101,9 +101,9 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
check_inode_eattr(ip,
&pass4_fxns_delete);
check_metatree(ip, &pass4_fxns_delete);
- fsck_blockmap_set(ip, ii->di_num.no_addr,
- _("invalid unlinked"),
- GFS2_BLKST_FREE);
+ fsck_bitmap_set(ip, ii->di_num.no_addr,
+ _("invalid unlinked"),
+ GFS2_BLKST_FREE);
fsck_inode_put(&ip);
log_err( _("The inode was deleted\n"));
} else {
@@ -122,9 +122,9 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
log_err( _("Unlinked inode has zero size\n"));
if (query(_("Clear zero-size unlinked inode? "
"(y/n) "))) {
- fsck_blockmap_set(ip, ii->di_num.no_addr,
+ fsck_bitmap_set(ip, ii->di_num.no_addr,
_("unlinked zero-length"),
- GFS2_BLKST_FREE);
+ GFS2_BLKST_FREE);
fsck_inode_put(&ip);
continue;
}
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 07/40] fsck.gfs2: Move set_ip_blockmap to pass1
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (5 preceding siblings ...)
2016-05-06 17:38 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 06/40] fsck.gfs2: Change all fsck_blockmap_set to fsck_bitmap_set Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 08/40] fsck.gfs2: Remove unneeded parameter instree from set_ip_blockmap Bob Peterson
` (34 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
Since function set_ip_blockmap is now only called from pass1,
move it to pass1 and make it static.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass1.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
gfs2/fsck/util.c | 50 --------------------------------------------------
gfs2/fsck/util.h | 1 -
3 files changed, 49 insertions(+), 51 deletions(-)
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index cc70d5f..e09bcb4 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -1141,6 +1141,55 @@ struct metawalk_fxns eattr_undo_fxns = {
.check_eattr_leaf = undo_eattr_indir_or_leaf,
.finish_eattr_indir = finish_eattr_indir,
};
+/* set_ip_blockmap - set the blockmap for a dinode
+ *
+ * instree: Set to 1 if directories should be inserted into the directory tree
+ * otherwise 0.
+ * returns: 0 if no error, -EINVAL if dinode has a bad mode, -EPERM on error
+ */
+static int set_ip_blockmap(struct gfs2_inode *ip, int instree)
+{
+ uint64_t block = ip->i_bh->b_blocknr;
+ uint32_t mode;
+ const char *ty;
+
+ if (ip->i_sbd->gfs1)
+ mode = gfs_to_gfs2_mode(ip);
+ else
+ mode = ip->i_di.di_mode & S_IFMT;
+
+ switch (mode) {
+ case S_IFDIR:
+ ty = _("directory");
+ break;
+ case S_IFREG:
+ ty = _("file");
+ break;
+ case S_IFLNK:
+ ty = _("symlink");
+ break;
+ case S_IFBLK:
+ ty = _("block device");
+ break;
+ case S_IFCHR:
+ ty = _("character device");
+ break;
+ case S_IFIFO:
+ ty = _("fifo");
+ break;
+ case S_IFSOCK:
+ ty = _("socket");
+ break;
+ default:
+ return -EINVAL;
+ }
+ if (fsck_blockmap_set(ip, block, ty, GFS2_BLKST_DINODE) ||
+ (mode == S_IFDIR && instree && !dirtree_insert(ip->i_di.di_num))) {
+ stack;
+ return -EPERM;
+ }
+ return 0;
+}
/*
* handle_ip - process an incore structure representing a dinode.
diff --git a/gfs2/fsck/util.c b/gfs2/fsck/util.c
index 55bd050..7ca7b91 100644
--- a/gfs2/fsck/util.c
+++ b/gfs2/fsck/util.c
@@ -591,56 +591,6 @@ void *gfs2_bmap_destroy(struct gfs2_sbd *sdp, struct gfs2_bmap *il)
return il;
}
-/* set_ip_blockmap - set the blockmap for a dinode
- *
- * instree: Set to 1 if directories should be inserted into the directory tree
- * otherwise 0.
- * returns: 0 if no error, -EINVAL if dinode has a bad mode, -EPERM on error
- */
-int set_ip_blockmap(struct gfs2_inode *ip, int instree)
-{
- uint64_t block = ip->i_bh->b_blocknr;
- uint32_t mode;
- const char *ty;
-
- if (ip->i_sbd->gfs1)
- mode = gfs_to_gfs2_mode(ip);
- else
- mode = ip->i_di.di_mode & S_IFMT;
-
- switch (mode) {
- case S_IFDIR:
- ty = _("directory");
- break;
- case S_IFREG:
- ty = _("file");
- break;
- case S_IFLNK:
- ty = _("symlink");
- break;
- case S_IFBLK:
- ty = _("block device");
- break;
- case S_IFCHR:
- ty = _("character device");
- break;
- case S_IFIFO:
- ty = _("fifo");
- break;
- case S_IFSOCK:
- ty = _("socket");
- break;
- default:
- return -EINVAL;
- }
- if (fsck_blockmap_set(ip, block, ty, GFS2_BLKST_DINODE) ||
- (mode == S_IFDIR && instree && !dirtree_insert(ip->i_di.di_num))) {
- stack;
- return -EPERM;
- }
- return 0;
-}
-
uint64_t find_free_blk(struct gfs2_sbd *sdp)
{
struct osi_node *n, *next = NULL;
diff --git a/gfs2/fsck/util.h b/gfs2/fsck/util.h
index 8709193..45a9000 100644
--- a/gfs2/fsck/util.h
+++ b/gfs2/fsck/util.h
@@ -119,7 +119,6 @@ extern struct gfs2_bmap *gfs2_bmap_create(struct gfs2_sbd *sdp, uint64_t size,
uint64_t *addl_mem_needed);
extern void *gfs2_bmap_destroy(struct gfs2_sbd *sdp, struct gfs2_bmap *il);
extern int gfs2_blockmap_set(struct gfs2_bmap *il, uint64_t block, int mark);
-extern int set_ip_blockmap(struct gfs2_inode *ip, int instree);
extern char generic_interrupt(const char *caller, const char *where,
const char *progress, const char *question,
const char *answers);
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 08/40] fsck.gfs2: Remove unneeded parameter instree from set_ip_blockmap
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (6 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 07/40] fsck.gfs2: Move set_ip_blockmap to pass1 Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 09/40] fsck.gfs2: Move leaf repair to pass2 Bob Peterson
` (33 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch removes parameter instree from function set_ip_blockmap,
since it is only called in one place.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass1.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index e09bcb4..e1653f1 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -1143,11 +1143,9 @@ struct metawalk_fxns eattr_undo_fxns = {
};
/* set_ip_blockmap - set the blockmap for a dinode
*
- * instree: Set to 1 if directories should be inserted into the directory tree
- * otherwise 0.
* returns: 0 if no error, -EINVAL if dinode has a bad mode, -EPERM on error
*/
-static int set_ip_blockmap(struct gfs2_inode *ip, int instree)
+static int set_ip_blockmap(struct gfs2_inode *ip)
{
uint64_t block = ip->i_bh->b_blocknr;
uint32_t mode;
@@ -1184,7 +1182,7 @@ static int set_ip_blockmap(struct gfs2_inode *ip, int instree)
return -EINVAL;
}
if (fsck_blockmap_set(ip, block, ty, GFS2_BLKST_DINODE) ||
- (mode == S_IFDIR && instree && !dirtree_insert(ip->i_di.di_num))) {
+ (mode == S_IFDIR && !dirtree_insert(ip->i_di.di_num))) {
stack;
return -EPERM;
}
@@ -1220,7 +1218,7 @@ static int handle_ip(struct gfs2_sbd *sdp, struct gfs2_inode *ip)
return 0;
}
- error = set_ip_blockmap(ip, 1);
+ error = set_ip_blockmap(ip);
if (error == -EINVAL) {
/* We found a dinode that has an invalid mode. At this point
set_ip_blockmap returned an error, which means it never
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 09/40] fsck.gfs2: Move leaf repair to pass2
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (7 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 08/40] fsck.gfs2: Remove unneeded parameter instree from set_ip_blockmap Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 10/40] fsck.gfs2: Eliminate astate code Bob Peterson
` (32 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
We don't want to try to repair directory leaf blocks from pass1
because we haven't verified the bitmap yet, so we cannot do
block allocations if necessary. Therefore, we postpone leaf
repair to pass2, where all the other directory integrity checks
are done. We also change fsck_blockmap_set to fsck_bitmap_set.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/metawalk.c | 179 +--------------------------------------------------
gfs2/fsck/metawalk.h | 8 +--
gfs2/fsck/pass1.c | 9 ---
gfs2/fsck/pass2.c | 147 +++++++++++++++++++++++++++++++++++++++++-
4 files changed, 146 insertions(+), 197 deletions(-)
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index ce23220..c37110f 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -629,8 +629,7 @@ bad_leaf:
brelse(lbh);
if (pass->repair_leaf) {
/* The leaf we read in is bad so we need to repair it. */
- fix = pass->repair_leaf(ip, leaf_no, lindex, *ref_count, msg,
- pass->private);
+ fix = pass->repair_leaf(ip, leaf_no, lindex, *ref_count, msg);
if (fix < 0)
return fix;
@@ -2024,179 +2023,3 @@ void reprocess_inode(struct gfs2_inode *ip, const char *desc)
log_err( _("Error %d reprocessing the %s metadata tree.\n"),
error, desc);
}
-
-/*
- * write_new_leaf - allocate and write a new leaf to cover a gap in hash table
- * @dip: the directory inode
- * @start_lindex: where in the hash table to start writing
- * @num_copies: number of copies of the pointer to write into hash table
- * @before_or_after: desc. of whether this is being added before/after/etc.
- * @bn: pointer to return the newly allocated leaf's block number
- */
-int write_new_leaf(struct gfs2_inode *dip, int start_lindex, int num_copies,
- const char *before_or_after, uint64_t *bn)
-{
- struct gfs2_buffer_head *nbh;
- struct gfs2_leaf *leaf;
- struct gfs2_dirent *dent;
- int count, i;
- int factor = 0, pad_size;
- uint64_t *cpyptr;
- char *padbuf;
- int divisor = num_copies;
- int end_lindex = start_lindex + num_copies;
-
- padbuf = malloc(num_copies * sizeof(uint64_t));
- /* calculate the depth needed for the new leaf */
- while (divisor > 1) {
- factor++;
- divisor /= 2;
- }
- /* Make sure the number of copies is properly a factor of 2 */
- if ((1 << factor) != num_copies) {
- log_err(_("Program error: num_copies not a factor of 2.\n"));
- log_err(_("num_copies=%d, dinode = %lld (0x%llx)\n"),
- num_copies,
- (unsigned long long)dip->i_di.di_num.no_addr,
- (unsigned long long)dip->i_di.di_num.no_addr);
- log_err(_("lindex = %d (0x%x)\n"), start_lindex, start_lindex);
- stack;
- free(padbuf);
- return -1;
- }
-
- /* allocate and write out a new leaf block */
- if (lgfs2_meta_alloc(dip, bn)) {
- log_err( _("Error: allocation failed while fixing directory leaf "
- "pointers.\n"));
- free(padbuf);
- return -1;
- }
- fsck_blockmap_set(dip, *bn, _("directory leaf"), dip->i_sbd->gfs1 ?
- GFS2_BLKST_DINODE : GFS2_BLKST_USED);
- log_err(_("A new directory leaf was allocated at block %lld "
- "(0x%llx) to fill the %d (0x%x) pointer gap %s the existing "
- "pointer at index %d (0x%x).\n"), (unsigned long long)*bn,
- (unsigned long long)*bn, num_copies, num_copies,
- before_or_after, start_lindex, start_lindex);
- dip->i_di.di_blocks++;
- bmodified(dip->i_bh);
- nbh = bget(dip->i_sbd, *bn);
- memset(nbh->b_data, 0, dip->i_sbd->bsize);
- leaf = (struct gfs2_leaf *)nbh->b_data;
- leaf->lf_header.mh_magic = cpu_to_be32(GFS2_MAGIC);
- leaf->lf_header.mh_type = cpu_to_be32(GFS2_METATYPE_LF);
- leaf->lf_header.mh_format = cpu_to_be32(GFS2_FORMAT_LF);
- leaf->lf_depth = cpu_to_be16(dip->i_di.di_depth - factor);
-
- /* initialize the first dirent on the new leaf block */
- dent = (struct gfs2_dirent *)(nbh->b_data + sizeof(struct gfs2_leaf));
- dent->de_rec_len = cpu_to_be16(dip->i_sbd->bsize -
- sizeof(struct gfs2_leaf));
- bmodified(nbh);
- brelse(nbh);
-
- /* pad the hash table with the new leaf block */
- cpyptr = (uint64_t *)padbuf;
- for (i = start_lindex; i < end_lindex; i++) {
- *cpyptr = cpu_to_be64(*bn);
- cpyptr++;
- }
- pad_size = num_copies * sizeof(uint64_t);
- log_err(_("Writing to the hash table of directory %lld "
- "(0x%llx)@index: 0x%x for 0x%lx pointers.\n"),
- (unsigned long long)dip->i_di.di_num.no_addr,
- (unsigned long long)dip->i_di.di_num.no_addr,
- start_lindex, (unsigned long)pad_size / sizeof(uint64_t));
- if (dip->i_sbd->gfs1)
- count = gfs1_writei(dip, padbuf, start_lindex *
- sizeof(uint64_t), pad_size);
- else
- count = gfs2_writei(dip, padbuf, start_lindex *
- sizeof(uint64_t), pad_size);
- free(padbuf);
- if (count != pad_size) {
- log_err( _("Error: bad write while fixing directory leaf "
- "pointers.\n"));
- return -1;
- }
- return 0;
-}
-
-/* repair_leaf - Warn the user of an error and ask permission to fix it
- * Process a bad leaf pointer and ask to repair the first time.
- * The repair process involves extending the previous leaf's entries
- * so that they replace the bad ones. We have to hack up the old
- * leaf a bit, but it's better than deleting the whole directory,
- * which is what used to happen before. */
-int repair_leaf(struct gfs2_inode *ip, uint64_t *leaf_no, int lindex,
- int ref_count, const char *msg, int allow_alloc)
-{
- int new_leaf_blks = 0, error, refs;
- uint64_t bn = 0;
-
- log_err( _("Directory Inode %llu (0x%llx) points to leaf %llu"
- " (0x%llx) %s.\n"),
- (unsigned long long)ip->i_di.di_num.no_addr,
- (unsigned long long)ip->i_di.di_num.no_addr,
- (unsigned long long)*leaf_no,
- (unsigned long long)*leaf_no, msg);
- if (!query( _("Attempt to patch around it? (y/n) "))) {
- log_err( _("Bad leaf left in place.\n"));
- goto out;
- }
- if (!allow_alloc) {
- uint64_t *cpyptr;
- char *padbuf;
- int pad_size, i;
-
- padbuf = malloc(ref_count * sizeof(uint64_t));
- cpyptr = (uint64_t *)padbuf;
- for (i = 0; i < ref_count; i++) {
- *cpyptr = 0;
- cpyptr++;
- }
- pad_size = ref_count * sizeof(uint64_t);
- log_err(_("Writing zeros to the hash table of directory %lld "
- "(0x%llx)@index: 0x%x for 0x%x pointers.\n"),
- (unsigned long long)ip->i_di.di_num.no_addr,
- (unsigned long long)ip->i_di.di_num.no_addr,
- lindex, ref_count);
- if (ip->i_sbd->gfs1)
- gfs1_writei(ip, padbuf, lindex * sizeof(uint64_t),
- pad_size);
- else
- gfs2_writei(ip, padbuf, lindex * sizeof(uint64_t),
- pad_size);
- free(padbuf);
- log_err( _("Directory Inode %llu (0x%llx) patched.\n"),
- (unsigned long long)ip->i_di.di_num.no_addr,
- (unsigned long long)ip->i_di.di_num.no_addr);
- goto out;
- }
- /* We can only write leafs in quantities that are factors of
- two, since leaves are doubled, not added sequentially.
- So if we have a hole that's not a factor of 2, we have to
- break it down into separate leaf blocks that are. */
- while (ref_count) {
- refs = 1;
- while (refs <= ref_count) {
- if (refs * 2 > ref_count)
- break;
- refs *= 2;
- }
- error = write_new_leaf(ip, lindex, refs, _("replacing"), &bn);
- if (error)
- return error;
-
- new_leaf_blks++;
- lindex += refs;
- ref_count -= refs;
- }
- log_err( _("Directory Inode %llu (0x%llx) repaired.\n"),
- (unsigned long long)ip->i_di.di_num.no_addr,
- (unsigned long long)ip->i_di.di_num.no_addr);
-out:
- *leaf_no = bn;
- return new_leaf_blks;
-}
diff --git a/gfs2/fsck/metawalk.h b/gfs2/fsck/metawalk.h
index 90bd028..6d6c3ff 100644
--- a/gfs2/fsck/metawalk.h
+++ b/gfs2/fsck/metawalk.h
@@ -60,11 +60,6 @@ extern struct gfs2_inode *fsck_system_inode(struct gfs2_sbd *sdp,
uint64_t block);
extern int find_remove_dup(struct gfs2_inode *ip, uint64_t block,
const char *btype);
-extern int write_new_leaf(struct gfs2_inode *dip, int start_lindex,
- int num_copies, const char *before_or_after,
- uint64_t *bn);
-extern int repair_leaf(struct gfs2_inode *ip, uint64_t *leaf_no, int lindex,
- int ref_count, const char *msg, int allow_alloc);
#define is_duplicate(dblock) ((dupfind(dblock)) ? 1 : 0)
@@ -155,8 +150,7 @@ struct metawalk_fxns {
int (*check_hash_tbl) (struct gfs2_inode *ip, uint64_t *tbl,
unsigned hsize, void *private);
int (*repair_leaf) (struct gfs2_inode *ip, uint64_t *leaf_no,
- int lindex, int ref_count, const char *msg,
- void *private);
+ int lindex, int ref_count, const char *msg);
int (*undo_check_meta) (struct gfs2_inode *ip, uint64_t block,
int h, void *private);
int (*undo_check_data) (struct gfs2_inode *ip, uint64_t block,
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index e1653f1..8d68028 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -84,14 +84,6 @@ static int invalidate_eattr_leaf(struct gfs2_inode *ip, uint64_t block,
void *private);
static int handle_ip(struct gfs2_sbd *sdp, struct gfs2_inode *ip);
-static int pass1_repair_leaf(struct gfs2_inode *ip, uint64_t *leaf_no,
- int lindex, int ref_count, const char *msg,
- void *private)
-{
- repair_leaf(ip, leaf_no, lindex, ref_count, msg, 0);
- return 0;
-}
-
struct metawalk_fxns pass1_fxns = {
.private = NULL,
.check_leaf = p1check_leaf,
@@ -103,7 +95,6 @@ struct metawalk_fxns pass1_fxns = {
.check_eattr_entry = check_eattr_entries,
.check_eattr_extentry = check_extended_leaf_eattr,
.big_file_msg = big_file_comfort,
- .repair_leaf = pass1_repair_leaf,
.undo_check_meta = undo_check_metalist,
.undo_check_data = undo_check_data,
};
diff --git a/gfs2/fsck/pass2.c b/gfs2/fsck/pass2.c
index 214fa03..96074d2 100644
--- a/gfs2/fsck/pass2.c
+++ b/gfs2/fsck/pass2.c
@@ -853,6 +853,105 @@ nuke_dentry:
return 1;
}
+/*
+ * write_new_leaf - allocate and write a new leaf to cover a gap in hash table
+ * @dip: the directory inode
+ * @start_lindex: where in the hash table to start writing
+ * @num_copies: number of copies of the pointer to write into hash table
+ * @before_or_after: desc. of whether this is being added before/after/etc.
+ * @bn: pointer to return the newly allocated leaf's block number
+ */
+static int write_new_leaf(struct gfs2_inode *dip, int start_lindex,
+ int num_copies, const char *before_or_after,
+ uint64_t *bn)
+{
+ struct gfs2_buffer_head *nbh;
+ struct gfs2_leaf *leaf;
+ struct gfs2_dirent *dent;
+ int count, i;
+ int factor = 0, pad_size;
+ uint64_t *cpyptr;
+ char *padbuf;
+ int divisor = num_copies;
+ int end_lindex = start_lindex + num_copies;
+
+ padbuf = malloc(num_copies * sizeof(uint64_t));
+ /* calculate the depth needed for the new leaf */
+ while (divisor > 1) {
+ factor++;
+ divisor /= 2;
+ }
+ /* Make sure the number of copies is properly a factor of 2 */
+ if ((1 << factor) != num_copies) {
+ log_err(_("Program error: num_copies not a factor of 2.\n"));
+ log_err(_("num_copies=%d, dinode = %lld (0x%llx)\n"),
+ num_copies,
+ (unsigned long long)dip->i_di.di_num.no_addr,
+ (unsigned long long)dip->i_di.di_num.no_addr);
+ log_err(_("lindex = %d (0x%x)\n"), start_lindex, start_lindex);
+ stack;
+ free(padbuf);
+ return -1;
+ }
+
+ /* allocate and write out a new leaf block */
+ if (lgfs2_meta_alloc(dip, bn)) {
+ log_err( _("Error: allocation failed while fixing directory leaf "
+ "pointers.\n"));
+ free(padbuf);
+ return -1;
+ }
+ fsck_bitmap_set(dip, *bn, _("directory leaf"), dip->i_sbd->gfs1 ?
+ GFS2_BLKST_DINODE : GFS2_BLKST_USED);
+ log_err(_("A new directory leaf was allocated at block %lld "
+ "(0x%llx) to fill the %d (0x%x) pointer gap %s the existing "
+ "pointer at index %d (0x%x).\n"), (unsigned long long)*bn,
+ (unsigned long long)*bn, num_copies, num_copies,
+ before_or_after, start_lindex, start_lindex);
+ dip->i_di.di_blocks++;
+ bmodified(dip->i_bh);
+ nbh = bget(dip->i_sbd, *bn);
+ memset(nbh->b_data, 0, dip->i_sbd->bsize);
+ leaf = (struct gfs2_leaf *)nbh->b_data;
+ leaf->lf_header.mh_magic = cpu_to_be32(GFS2_MAGIC);
+ leaf->lf_header.mh_type = cpu_to_be32(GFS2_METATYPE_LF);
+ leaf->lf_header.mh_format = cpu_to_be32(GFS2_FORMAT_LF);
+ leaf->lf_depth = cpu_to_be16(dip->i_di.di_depth - factor);
+
+ /* initialize the first dirent on the new leaf block */
+ dent = (struct gfs2_dirent *)(nbh->b_data + sizeof(struct gfs2_leaf));
+ dent->de_rec_len = cpu_to_be16(dip->i_sbd->bsize -
+ sizeof(struct gfs2_leaf));
+ bmodified(nbh);
+ brelse(nbh);
+
+ /* pad the hash table with the new leaf block */
+ cpyptr = (uint64_t *)padbuf;
+ for (i = start_lindex; i < end_lindex; i++) {
+ *cpyptr = cpu_to_be64(*bn);
+ cpyptr++;
+ }
+ pad_size = num_copies * sizeof(uint64_t);
+ log_err(_("Writing to the hash table of directory %lld "
+ "(0x%llx)@index: 0x%x for 0x%lx pointers.\n"),
+ (unsigned long long)dip->i_di.di_num.no_addr,
+ (unsigned long long)dip->i_di.di_num.no_addr,
+ start_lindex, (unsigned long)pad_size / sizeof(uint64_t));
+ if (dip->i_sbd->gfs1)
+ count = gfs1_writei(dip, padbuf, start_lindex *
+ sizeof(uint64_t), pad_size);
+ else
+ count = gfs2_writei(dip, padbuf, start_lindex *
+ sizeof(uint64_t), pad_size);
+ free(padbuf);
+ if (count != pad_size) {
+ log_err( _("Error: bad write while fixing directory leaf "
+ "pointers.\n"));
+ return -1;
+ }
+ return 0;
+}
+
/* pad_with_leafblks - pad a hash table with pointers to new leaf blocks
*
* @ip: pointer to the dinode structure
@@ -1047,11 +1146,53 @@ static int basic_check_dentry(struct gfs2_inode *ip, struct gfs2_dirent *dent,
}
}
+/* pass2_repair_leaf - Warn the user of an error and ask permission to fix it
+ * Process a bad leaf pointer and ask to repair the first time.
+ * The repair process involves extending the previous leaf's entries
+ * so that they replace the bad ones. We have to hack up the old
+ * leaf a bit, but it's better than deleting the whole directory,
+ * which is what used to happen before. */
static int pass2_repair_leaf(struct gfs2_inode *ip, uint64_t *leaf_no,
- int lindex, int ref_count, const char *msg,
- void *private)
+ int lindex, int ref_count, const char *msg)
{
- return repair_leaf(ip, leaf_no, lindex, ref_count, msg, 1);
+ int new_leaf_blks = 0, error, refs;
+ uint64_t bn = 0;
+
+ log_err( _("Directory Inode %llu (0x%llx) points to leaf %llu"
+ " (0x%llx) %s.\n"),
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)*leaf_no,
+ (unsigned long long)*leaf_no, msg);
+ if (!query( _("Attempt to patch around it? (y/n) "))) {
+ log_err( _("Bad leaf left in place.\n"));
+ goto out;
+ }
+ /* We can only write leafs in quantities that are factors of
+ two, since leaves are doubled, not added sequentially.
+ So if we have a hole that's not a factor of 2, we have to
+ break it down into separate leaf blocks that are. */
+ while (ref_count) {
+ refs = 1;
+ while (refs <= ref_count) {
+ if (refs * 2 > ref_count)
+ break;
+ refs *= 2;
+ }
+ error = write_new_leaf(ip, lindex, refs, _("replacing"), &bn);
+ if (error)
+ return error;
+
+ new_leaf_blks++;
+ lindex += refs;
+ ref_count -= refs;
+ }
+ log_err( _("Directory Inode %llu (0x%llx) repaired.\n"),
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)ip->i_di.di_num.no_addr);
+out:
+ *leaf_no = bn;
+ return new_leaf_blks;
}
/* The purpose of leafck_fxns is to provide a means for function fix_hashtable
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 10/40] fsck.gfs2: Eliminate astate code
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (8 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 09/40] fsck.gfs2: Move leaf repair to pass2 Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 11/40] fsck.gfs2: Move reprocess code to pass1 Bob Peterson
` (31 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
Since all the passes after pass1 now use the rgrp bitmaps for the
definitive state of a block, there's no more need to re-sync the
blockmap with the bitmap. Therefore, we can completely eliminate
the astate save/restore and reprocessing code.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/metawalk.c | 6 ------
gfs2/fsck/pass2.c | 21 ---------------------
gfs2/fsck/pass3.c | 18 ------------------
gfs2/fsck/pass4.c | 5 -----
gfs2/fsck/util.h | 20 --------------------
5 files changed, 70 deletions(-)
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index c37110f..cbc73e9 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -1718,9 +1718,6 @@ int check_linear_dir(struct gfs2_inode *ip, struct gfs2_buffer_head *bh,
int check_dir(struct gfs2_sbd *sdp, struct gfs2_inode *ip, struct metawalk_fxns *pass)
{
int error = 0;
- struct alloc_state as;
-
- astate_save(ip, &as);
if (ip->i_di.di_flags & GFS2_DIF_EXHASH)
error = check_leaf_blks(ip, pass);
@@ -1730,9 +1727,6 @@ int check_dir(struct gfs2_sbd *sdp, struct gfs2_inode *ip, struct metawalk_fxns
if (error < 0)
stack;
- if (astate_changed(ip, &as))
- reprocess_inode(ip, _("Current"));
-
return error;
}
diff --git a/gfs2/fsck/pass2.c b/gfs2/fsck/pass2.c
index 96074d2..a215b3c 100644
--- a/gfs2/fsck/pass2.c
+++ b/gfs2/fsck/pass2.c
@@ -1837,7 +1837,6 @@ build_it:
return -1;
}
fsck_bitmap_set(ip, ip->i_di.di_num.no_addr, fn, GFS2_BLKST_DINODE);
- reprocess_inode(ip, fn);
log_err(_("System file %s rebuilt.\n"), fn);
goto out_good;
}
@@ -1848,7 +1847,6 @@ static int check_system_dir(struct gfs2_inode *sysinode, const char *dirname,
int builder(struct gfs2_sbd *sdp))
{
uint64_t iblock = 0;
- struct alloc_state as;
struct dir_status ds = {0};
int error = 0;
@@ -1865,15 +1863,12 @@ static int check_system_dir(struct gfs2_inode *sysinode, const char *dirname,
pass2_fxns.private = (void *) &ds;
if (ds.q == GFS2_BLKST_UNLINKED) {
- astate_save(sysinode, &as);
/* First check that the directory's metatree is valid */
error = check_metatree(sysinode, &pass2_fxns);
if (error < 0) {
stack;
return error;
}
- if (astate_changed(sysinode, &as))
- reprocess_inode(sysinode, _("System inode"));
}
error = check_dir(sysinode->i_sbd, sysinode, &pass2_fxns);
if (skip_this_pass || fsck_abort) /* if asked to skip the rest */
@@ -1892,7 +1887,6 @@ static int check_system_dir(struct gfs2_inode *sysinode, const char *dirname,
if (!ds.dotdir) {
log_err( _("No '.' entry found for %s directory.\n"), dirname);
if (query( _("Is it okay to add '.' entry? (y/n) "))) {
- astate_save(sysinode, &as);
log_warn( _("Adding '.' entry\n"));
error = dir_add(sysinode, ".", 1, &(sysinode->i_di.di_num),
(sysinode->i_sbd->gfs1 ? GFS_FILE_DIR : DT_DIR));
@@ -1901,8 +1895,6 @@ static int check_system_dir(struct gfs2_inode *sysinode, const char *dirname,
strerror(errno));
return -errno;
}
- if (astate_changed(sysinode, &as))
- reprocess_inode(sysinode, dirname);
/* This system inode is linked to itself via '.' */
incr_link_count(sysinode->i_di.di_num, sysinode,
"sysinode \".\"");
@@ -1974,20 +1966,16 @@ static int pass2_check_dir(struct gfs2_sbd *sdp, struct gfs2_inode *ip)
{
uint64_t dirblk = ip->i_di.di_num.no_addr;
struct dir_status ds = {0};
- struct alloc_state as;
int error;
pass2_fxns.private = &ds;
if (ds.q == GFS2_BLKST_UNLINKED) {
/* First check that the directory's metatree is valid */
- astate_save(ip, &as);
error = check_metatree(ip, &pass2_fxns);
if (error < 0) {
stack;
return error;
}
- if (astate_changed(ip, &as))
- reprocess_inode(ip, "current");
}
error = check_dir(sdp, ip, &pass2_fxns);
if (skip_this_pass || fsck_abort) /* if asked to skip the rest */
@@ -2037,7 +2025,6 @@ static int pass2_check_dir(struct gfs2_sbd *sdp, struct gfs2_inode *ip)
(unsigned long long)dirblk, (unsigned long long)dirblk);
if (query( _("Is it okay to add '.' entry? (y/n) "))) {
- astate_save(ip, &as);
error = dir_add(ip, ".", 1, &(ip->i_di.di_num),
(sdp->gfs1 ? GFS_FILE_DIR : DT_DIR));
if (error) {
@@ -2045,14 +2032,6 @@ static int pass2_check_dir(struct gfs2_sbd *sdp, struct gfs2_inode *ip)
strerror(errno));
return -errno;
}
- if (astate_changed(ip, &as)) {
- char dirname[80];
-
- sprintf(dirname, _("Directory at %lld (0x%llx)"),
- (unsigned long long)dirblk,
- (unsigned long long)dirblk);
- reprocess_inode(ip, dirname);
- }
/* directory links to itself via '.' */
incr_link_count(ip->i_di.di_num, ip, _("\". (itself)\""));
ds.entry_count++;
diff --git a/gfs2/fsck/pass3.c b/gfs2/fsck/pass3.c
index dad1f5c..0df427d 100644
--- a/gfs2/fsck/pass3.c
+++ b/gfs2/fsck/pass3.c
@@ -24,7 +24,6 @@ static int attach_dotdot_to(struct gfs2_sbd *sdp, uint64_t newdotdot,
int filename_len = 2;
int err;
struct gfs2_inode *ip, *pip;
- struct alloc_state as;
ip = fsck_load_inode(sdp, block);
pip = fsck_load_inode(sdp, newdotdot);
@@ -39,7 +38,6 @@ static int attach_dotdot_to(struct gfs2_sbd *sdp, uint64_t newdotdot,
log_warn( _("Unable to remove \"..\" directory entry.\n"));
else
decr_link_count(olddotdot, block, _("old \"..\""));
- astate_save(ip, &as);
err = dir_add(ip, filename, filename_len, &pip->i_di.di_num,
(sdp->gfs1 ? GFS_FILE_DIR : DT_DIR));
if (err) {
@@ -47,14 +45,6 @@ static int attach_dotdot_to(struct gfs2_sbd *sdp, uint64_t newdotdot,
filename, strerror(errno));
exit(FSCK_ERROR);
}
- if (astate_changed(ip, &as)) {
- char dirname[80];
-
- sprintf(dirname, _("Directory at %lld (0x%llx)"),
- (unsigned long long)ip->i_di.di_num.no_addr,
- (unsigned long long)ip->i_di.di_num.no_addr);
- reprocess_inode(ip, dirname);
- }
incr_link_count(pip->i_di.di_num, ip, _("new \"..\""));
fsck_inode_put(&ip);
fsck_inode_put(&pip);
@@ -182,7 +172,6 @@ int pass3(struct gfs2_sbd *sdp)
struct dir_info *di, *tdi;
struct gfs2_inode *ip;
int q;
- struct alloc_state lf_as = {.as_blocks = 0, .as_meta_goal = 0};
di = dirtree_find(sdp->md.rooti->i_di.di_num.no_addr);
if (di) {
@@ -229,8 +218,6 @@ int pass3(struct gfs2_sbd *sdp)
*/
log_info( _("Checking directory linkage.\n"));
for (tmp = osi_first(&dirtree); tmp; tmp = next) {
- if (lf_dip && lf_as.as_blocks == 0)
- astate_save(lf_dip, &lf_as);
next = osi_next(tmp);
di = (struct dir_info *)tmp;
while (!di->checked) {
@@ -328,11 +315,6 @@ int pass3(struct gfs2_sbd *sdp)
}
}
if (lf_dip) {
- /* If the lf directory had new blocks added we have to mark
- them properly in the blockmap so they're not freed. */
- if (astate_changed(lf_dip, &lf_as))
- reprocess_inode(lf_dip, "lost+found");
-
log_debug( _("At end of pass3, lost+found entries is %u\n"),
lf_dip->i_di.di_entries);
}
diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
index 25c63b7..ba38e8c 100644
--- a/gfs2/fsck/pass4.c
+++ b/gfs2/fsck/pass4.c
@@ -48,15 +48,12 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
struct gfs2_inode *ip;
int lf_addition = 0;
int q;
- struct alloc_state lf_as = {.as_blocks = 0, .as_meta_goal = 0};
/* FIXME: should probably factor this out into a generic
* scanning fxn */
for (tmp = osi_first(&inodetree); tmp; tmp = next) {
if (skip_this_pass || fsck_abort) /* if asked to skip the rest */
return 0;
- if (lf_dip && lf_as.as_blocks == 0)
- astate_save(lf_dip, &lf_as);
next = osi_next(tmp);
ii = (struct inode_info *)tmp;
/* Don't check reference counts on the special gfs files */
@@ -179,8 +176,6 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
if (lf_dip == NULL)
return 0;
- if (astate_changed(lf_dip, &lf_as))
- reprocess_inode(lf_dip, "lost+found");
if (lf_addition) {
if (!(ii = inodetree_find(lf_dip->i_di.di_num.no_addr))) {
diff --git a/gfs2/fsck/util.h b/gfs2/fsck/util.h
index 45a9000..7753daf 100644
--- a/gfs2/fsck/util.h
+++ b/gfs2/fsck/util.h
@@ -12,11 +12,6 @@
#define INODE_VALID 1
#define INODE_INVALID 0
-struct alloc_state {
- uint64_t as_blocks;
- uint64_t as_meta_goal;
-};
-
struct di_info *search_list(osi_list_t *list, uint64_t addr);
void big_file_comfort(struct gfs2_inode *ip, uint64_t blks_checked);
void warm_fuzzy_stuff(uint64_t block);
@@ -32,21 +27,6 @@ extern const char *reftypes[ref_types + 1];
#define BLOCKMAP_BYTE_OFFSET2(x) ((x & 0x0000000000000003) << 1)
#define BLOCKMAP_MASK2 (0x3)
-static inline void astate_save(struct gfs2_inode *ip, struct alloc_state *as)
-{
- as->as_blocks = ip->i_di.di_blocks;
- as->as_meta_goal = ip->i_di.di_goal_meta;
-}
-
-static inline int astate_changed(struct gfs2_inode *ip, struct alloc_state *as)
-{
- if (as->as_blocks != ip->i_di.di_blocks)
- return 1;
- if (as->as_meta_goal != ip->i_di.di_goal_meta)
- return 1;
- return 0;
-}
-
static inline int block_type(uint64_t bblock)
{
static unsigned char *byte;
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 11/40] fsck.gfs2: Move reprocess code to pass1
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (9 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 10/40] fsck.gfs2: Eliminate astate code Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 12/40] fsck.gfs2: Separate out functions that may only be done after pass1 Bob Peterson
` (30 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
Since all the passes after pass1 no longer care about the blockmap,
we only need to keep the blockmap in sync with the bitmap in pass1.
Therefore, we can safely move the "reprocess_inode" stuff to pass1.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/metawalk.c | 107 ---------------------------------------------------
gfs2/fsck/metawalk.h | 1 -
gfs2/fsck/pass1.c | 107 +++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 107 insertions(+), 108 deletions(-)
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index cbc73e9..7a77dc9 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -1910,110 +1910,3 @@ int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
return delete_block_if_notdup(ip, block, NULL, _("extended attribute"),
NULL, private);
}
-
-static int alloc_metalist(struct gfs2_inode *ip, uint64_t block,
- struct gfs2_buffer_head **bh, int h, int *is_valid,
- int *was_duplicate, void *private)
-{
- int q;
- const char *desc = (const char *)private;
-
- /* No need to range_check here--if it was added, it's in range. */
- /* We can't check the bitmap here because this function is called
- after the bitmap has been set but before the blockmap has. */
- *is_valid = 1;
- *was_duplicate = 0;
- *bh = bread(ip->i_sbd, block);
- q = bitmap_type(ip->i_sbd, block);
- if (q == GFS2_BLKST_FREE) {
- log_debug(_("%s reference to new metadata block "
- "%lld (0x%llx) is now marked as indirect.\n"),
- desc, (unsigned long long)block,
- (unsigned long long)block);
- gfs2_blockmap_set(bl, block, ip->i_sbd->gfs1 ?
- GFS2_BLKST_DINODE : GFS2_BLKST_USED);
- }
- return meta_is_good;
-}
-
-static int alloc_data(struct gfs2_inode *ip, uint64_t metablock,
- uint64_t block, void *private,
- struct gfs2_buffer_head *bh, uint64_t *ptr)
-{
- int q;
- const char *desc = (const char *)private;
-
- /* No need to range_check here--if it was added, it's in range. */
- /* We can't check the bitmap here because this function is called
- after the bitmap has been set but before the blockmap has. */
- q = bitmap_type(ip->i_sbd, block);
- if (q == GFS2_BLKST_FREE) {
- log_debug(_("%s reference to new data block "
- "%lld (0x%llx) is now marked as data.\n"),
- desc, (unsigned long long)block,
- (unsigned long long)block);
- gfs2_blockmap_set(bl, block, GFS2_BLKST_USED);
- }
- return 0;
-}
-
-static int alloc_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
-{
- int q;
-
- /* No need to range_check here--if it was added, it's in range. */
- /* We can't check the bitmap here because this function is called
- after the bitmap has been set but before the blockmap has. */
- q = bitmap_type(ip->i_sbd, block);
- if (q == GFS2_BLKST_FREE)
- fsck_blockmap_set(ip, block, _("newly allocated leaf"),
- ip->i_sbd->gfs1 ? GFS2_BLKST_DINODE :
- GFS2_BLKST_USED);
- return 0;
-}
-
-struct metawalk_fxns alloc_fxns = {
- .private = NULL,
- .check_leaf = alloc_leaf,
- .check_metalist = alloc_metalist,
- .check_data = alloc_data,
- .check_eattr_indir = NULL,
- .check_eattr_leaf = NULL,
- .check_dentry = NULL,
- .check_eattr_entry = NULL,
- .check_eattr_extentry = NULL,
- .finish_eattr_indir = NULL,
-};
-
-/*
- * reprocess_inode - fixes the blockmap to match the bitmap due to an
- * unexpected block allocation via libgfs2.
- *
- * The problem we're trying to overcome here is when a new block must be
- * added to a dinode because of a write. This will happen when lost+found
- * needs a new indirect block for its hash table. In that case, the write
- * causes a new block to be assigned in the bitmap but that block is not yet
- * accurately reflected in the fsck blockmap. We need to compensate here.
- *
- * We can't really use fsck_blockmap_set here because the new block
- * was already allocated by libgfs2 and therefore it took care of
- * the rgrp free space variable. fsck_blockmap_set adjusts the free space
- * in the rgrp according to the change, which has already been done.
- * So it's only our blockmap that now disagrees with the rgrp bitmap, so we
- * need to fix only that.
- */
-void reprocess_inode(struct gfs2_inode *ip, const char *desc)
-{
- int error;
-
- alloc_fxns.private = (void *)desc;
- log_info( _("%s inode %llu (0x%llx) had blocks added; reprocessing "
- "its metadata tree at height=%d.\n"), desc,
- (unsigned long long)ip->i_di.di_num.no_addr,
- (unsigned long long)ip->i_di.di_num.no_addr,
- ip->i_di.di_height);
- error = check_metatree(ip, &alloc_fxns);
- if (error)
- log_err( _("Error %d reprocessing the %s metadata tree.\n"),
- error, desc);
-}
diff --git a/gfs2/fsck/metawalk.h b/gfs2/fsck/metawalk.h
index 6d6c3ff..d505945 100644
--- a/gfs2/fsck/metawalk.h
+++ b/gfs2/fsck/metawalk.h
@@ -54,7 +54,6 @@ extern int _fsck_bitmap_set(struct gfs2_inode *ip, uint64_t bblock,
const char *caller, int line);
extern int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk,
int error_on_dinode, int new_blockmap_state);
-extern void reprocess_inode(struct gfs2_inode *ip, const char *desc);
extern struct duptree *dupfind(uint64_t block);
extern struct gfs2_inode *fsck_system_inode(struct gfs2_sbd *sdp,
uint64_t block);
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index 8d68028..d12e03e 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -1180,6 +1180,113 @@ static int set_ip_blockmap(struct gfs2_inode *ip)
return 0;
}
+static int alloc_metalist(struct gfs2_inode *ip, uint64_t block,
+ struct gfs2_buffer_head **bh, int h, int *is_valid,
+ int *was_duplicate, void *private)
+{
+ int q;
+ const char *desc = (const char *)private;
+
+ /* No need to range_check here--if it was added, it's in range. */
+ /* We can't check the bitmap here because this function is called
+ after the bitmap has been set but before the blockmap has. */
+ *is_valid = 1;
+ *was_duplicate = 0;
+ *bh = bread(ip->i_sbd, block);
+ q = bitmap_type(ip->i_sbd, block);
+ if (q == GFS2_BLKST_FREE) {
+ log_debug(_("%s reference to new metadata block "
+ "%lld (0x%llx) is now marked as indirect.\n"),
+ desc, (unsigned long long)block,
+ (unsigned long long)block);
+ gfs2_blockmap_set(bl, block, ip->i_sbd->gfs1 ?
+ GFS2_BLKST_DINODE : GFS2_BLKST_USED);
+ }
+ return meta_is_good;
+}
+
+static int alloc_data(struct gfs2_inode *ip, uint64_t metablock,
+ uint64_t block, void *private,
+ struct gfs2_buffer_head *bh, uint64_t *ptr)
+{
+ int q;
+ const char *desc = (const char *)private;
+
+ /* No need to range_check here--if it was added, it's in range. */
+ /* We can't check the bitmap here because this function is called
+ after the bitmap has been set but before the blockmap has. */
+ q = bitmap_type(ip->i_sbd, block);
+ if (q == GFS2_BLKST_FREE) {
+ log_debug(_("%s reference to new data block "
+ "%lld (0x%llx) is now marked as data.\n"),
+ desc, (unsigned long long)block,
+ (unsigned long long)block);
+ gfs2_blockmap_set(bl, block, GFS2_BLKST_USED);
+ }
+ return 0;
+}
+
+static int alloc_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
+{
+ int q;
+
+ /* No need to range_check here--if it was added, it's in range. */
+ /* We can't check the bitmap here because this function is called
+ after the bitmap has been set but before the blockmap has. */
+ q = bitmap_type(ip->i_sbd, block);
+ if (q == GFS2_BLKST_FREE)
+ fsck_blockmap_set(ip, block, _("newly allocated leaf"),
+ ip->i_sbd->gfs1 ? GFS2_BLKST_DINODE :
+ GFS2_BLKST_USED);
+ return 0;
+}
+
+struct metawalk_fxns alloc_fxns = {
+ .private = NULL,
+ .check_leaf = alloc_leaf,
+ .check_metalist = alloc_metalist,
+ .check_data = alloc_data,
+ .check_eattr_indir = NULL,
+ .check_eattr_leaf = NULL,
+ .check_dentry = NULL,
+ .check_eattr_entry = NULL,
+ .check_eattr_extentry = NULL,
+ .finish_eattr_indir = NULL,
+};
+
+/*
+ * reprocess_inode - fixes the blockmap to match the bitmap due to an
+ * unexpected block allocation via libgfs2.
+ *
+ * The problem we're trying to overcome here is when a new block must be
+ * added to a dinode because of a write. This will happen when lost+found
+ * needs a new indirect block for its hash table. In that case, the write
+ * causes a new block to be assigned in the bitmap but that block is not yet
+ * accurately reflected in the fsck blockmap. We need to compensate here.
+ *
+ * We can't really use fsck_blockmap_set here because the new block
+ * was already allocated by libgfs2 and therefore it took care of
+ * the rgrp free space variable. fsck_blockmap_set adjusts the free space
+ * in the rgrp according to the change, which has already been done.
+ * So it's only our blockmap that now disagrees with the rgrp bitmap, so we
+ * need to fix only that.
+ */
+static void reprocess_inode(struct gfs2_inode *ip, const char *desc)
+{
+ int error;
+
+ alloc_fxns.private = (void *)desc;
+ log_info( _("%s inode %llu (0x%llx) had blocks added; reprocessing "
+ "its metadata tree at height=%d.\n"), desc,
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ ip->i_di.di_height);
+ error = check_metatree(ip, &alloc_fxns);
+ if (error)
+ log_err( _("Error %d reprocessing the %s metadata tree.\n"),
+ error, desc);
+}
+
/*
* handle_ip - process an incore structure representing a dinode.
*/
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 12/40] fsck.gfs2: Separate out functions that may only be done after pass1
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (10 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 11/40] fsck.gfs2: Move reprocess code to pass1 Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-09 14:55 ` Andrew Price
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 13/40] fsck.gfs2: Divest check_metatree from fsck_blockmap_set Bob Peterson
` (29 subsequent siblings)
41 siblings, 1 reply; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch adds a new header file and source file which has the
purpose of separating out the functions that may only be called
after pass1 from the functions that pass1 may call. This is to
ensure that there will never be inconsistencies between blockmap
(which pass1 uses) and bitmap (which subsequent passes use).
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/Makefile.am | 1 +
gfs2/fsck/afterpass1_common.c | 272 ++++++++++++++++++++++++++++++++++++++++++
gfs2/fsck/afterpass1_common.h | 31 +++++
gfs2/fsck/metawalk.c | 272 +-----------------------------------------
gfs2/fsck/metawalk.h | 32 +----
gfs2/fsck/pass1.c | 22 ++++
gfs2/fsck/pass1b.c | 1 +
gfs2/fsck/pass2.c | 1 +
gfs2/fsck/pass3.c | 1 +
gfs2/fsck/pass4.c | 1 +
10 files changed, 335 insertions(+), 299 deletions(-)
create mode 100644 gfs2/fsck/afterpass1_common.c
create mode 100644 gfs2/fsck/afterpass1_common.h
diff --git a/gfs2/fsck/Makefile.am b/gfs2/fsck/Makefile.am
index 73d957e..ecd05d8 100644
--- a/gfs2/fsck/Makefile.am
+++ b/gfs2/fsck/Makefile.am
@@ -19,6 +19,7 @@ fsck_gfs2_SOURCES = \
lost_n_found.c \
main.c \
metawalk.c \
+ afterpass1_common.c \
pass1b.c \
pass1.c \
pass2.c \
diff --git a/gfs2/fsck/afterpass1_common.c b/gfs2/fsck/afterpass1_common.c
new file mode 100644
index 0000000..36646f8
--- /dev/null
+++ b/gfs2/fsck/afterpass1_common.c
@@ -0,0 +1,272 @@
+#include <inttypes.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <libintl.h>
+#include <ctype.h>
+#include <fcntl.h>
+#define _(String) gettext(String)
+
+#include <logging.h>
+#include "libgfs2.h"
+#include "fsck.h"
+#include "afterpass1_common.h"
+#include "metawalk.h"
+#include "util.h"
+
+/**
+ * find_remove_dup - find out if this is a duplicate ref. If so, remove it.
+ *
+ * Returns: 1 if there are any remaining references to this block, else 0.
+ */
+int find_remove_dup(struct gfs2_inode *ip, uint64_t block, const char *btype)
+{
+ struct duptree *dt;
+ struct inode_with_dups *id;
+
+ dt = dupfind(block);
+ if (!dt)
+ return 0;
+
+ /* remove the inode reference id structure for this reference. */
+ id = find_dup_ref_inode(dt, ip);
+ if (!id)
+ goto more_refs;
+
+ dup_listent_delete(dt, id);
+ if (dt->refs == 0) {
+ log_info( _("This was the last reference: it's no longer a "
+ "duplicate.\n"));
+ dup_delete(dt); /* not duplicate now */
+ return 0;
+ }
+more_refs:
+ log_info( _("%d block reference(s) remain.\n"), dt->refs);
+ return 1; /* references still exist so do not free the block. */
+}
+
+/**
+ * delete_block_if_notdup - delete blocks associated with an inode
+ *
+ * Ignore blocks that are already marked free.
+ * If it has been identified as duplicate, remove the duplicate reference.
+ * If all duplicate references have been removed, delete the block.
+ */
+static int delete_block_if_notdup(struct gfs2_inode *ip, uint64_t block,
+ struct gfs2_buffer_head **bh,
+ const char *btype, int *was_duplicate,
+ void *private)
+{
+ int q;
+
+ if (!valid_block(ip->i_sbd, block))
+ return meta_error;
+
+ q = bitmap_type(ip->i_sbd, block);
+ if (q == GFS2_BLKST_FREE) {
+ log_info( _("%s block %lld (0x%llx), part of inode "
+ "%lld (0x%llx), was already free.\n"),
+ btype, (unsigned long long)block,
+ (unsigned long long)block,
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)ip->i_di.di_num.no_addr);
+ return meta_is_good;
+ }
+ if (find_remove_dup(ip, block, btype)) { /* a dup */
+ if (was_duplicate)
+ *was_duplicate = 1;
+ log_err( _("Not clearing duplicate reference in inode "
+ "at block #%llu (0x%llx) to block #%llu (0x%llx) "
+ "because it's referenced by another inode.\n"),
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)block, (unsigned long long)block);
+ } else {
+ check_n_fix_bitmap(ip->i_sbd, block, 0, GFS2_BLKST_FREE);
+ }
+ return meta_is_good;
+}
+
+static int remove_dentry(struct gfs2_inode *ip, struct gfs2_dirent *dent,
+ struct gfs2_dirent *prev_de,
+ struct gfs2_buffer_head *bh,
+ char *filename, uint32_t *count, int *lindex,
+ void *private)
+{
+ /* the metawalk_fxn's private field must be set to the dentry
+ * block we want to clear */
+ uint64_t *dentryblock = (uint64_t *) private;
+ struct gfs2_dirent dentry, *de;
+
+ memset(&dentry, 0, sizeof(struct gfs2_dirent));
+ gfs2_dirent_in(&dentry, (char *)dent);
+ de = &dentry;
+
+ if (de->de_inum.no_addr == *dentryblock)
+ dirent2_del(ip, bh, prev_de, dent);
+ else
+ (*count)++;
+
+ return 0;
+
+}
+
+int remove_dentry_from_dir(struct gfs2_sbd *sdp, uint64_t dir,
+ uint64_t dentryblock)
+{
+ struct metawalk_fxns remove_dentry_fxns = {0};
+ struct gfs2_inode *ip;
+ int q;
+ int error;
+
+ log_debug( _("Removing dentry %llu (0x%llx) from directory %llu"
+ " (0x%llx)\n"), (unsigned long long)dentryblock,
+ (unsigned long long)dentryblock,
+ (unsigned long long)dir, (unsigned long long)dir);
+ if (!valid_block(sdp, dir)) {
+ log_err( _("Parent directory is invalid\n"));
+ return 1;
+ }
+ remove_dentry_fxns.private = &dentryblock;
+ remove_dentry_fxns.check_dentry = remove_dentry;
+
+ q = bitmap_type(sdp, dir);
+ if (q != GFS2_BLKST_DINODE) {
+ log_info( _("Parent block is not an inode...ignoring\n"));
+ return 1;
+ }
+
+ ip = fsck_load_inode(sdp, dir);
+ if (ip == NULL) {
+ stack;
+ return -1;
+ }
+ /* Need to run check_dir with a private var of dentryblock,
+ * and fxns that remove that dentry if found */
+ error = check_dir(sdp, ip, &remove_dentry_fxns);
+ fsck_inode_put(&ip);
+ return error;
+}
+
+int delete_metadata(struct gfs2_inode *ip, uint64_t block,
+ struct gfs2_buffer_head **bh, int h, int *is_valid,
+ int *was_duplicate, void *private)
+{
+ *is_valid = 1;
+ *was_duplicate = 0;
+ return delete_block_if_notdup(ip, block, bh, _("metadata"),
+ was_duplicate, private);
+}
+
+int delete_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
+{
+ return delete_block_if_notdup(ip, block, NULL, _("leaf"), NULL,
+ private);
+}
+
+int delete_data(struct gfs2_inode *ip, uint64_t metablock,
+ uint64_t block, void *private, struct gfs2_buffer_head *bh,
+ uint64_t *ptr)
+{
+ return delete_block_if_notdup(ip, block, NULL, _("data"), NULL,
+ private);
+}
+
+static int del_eattr_generic(struct gfs2_inode *ip, uint64_t block,
+ uint64_t parent, struct gfs2_buffer_head **bh,
+ void *private, const char *eatype)
+{
+ int ret = 0;
+ int was_free = 0;
+ int q;
+
+ if (valid_block(ip->i_sbd, block)) {
+ q = bitmap_type(ip->i_sbd, block);
+ if (q == GFS2_BLKST_FREE)
+ was_free = 1;
+ ret = delete_block_if_notdup(ip, block, NULL, eatype,
+ NULL, private);
+ if (!ret) {
+ *bh = bread(ip->i_sbd, block);
+ if (!was_free)
+ ip->i_di.di_blocks--;
+ bmodified(ip->i_bh);
+ }
+ }
+ /* Even if it's a duplicate reference, we want to eliminate the
+ reference itself, and adjust di_blocks accordingly. */
+ if (ip->i_di.di_eattr) {
+ if (block == ip->i_di.di_eattr)
+ ip->i_di.di_eattr = 0;
+ bmodified(ip->i_bh);
+ }
+ return ret;
+}
+
+int delete_eattr_indir(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
+ struct gfs2_buffer_head **bh, void *private)
+{
+ return del_eattr_generic(ip, block, parent, bh, private,
+ _("extended attribute"));
+}
+
+int delete_eattr_leaf(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
+ struct gfs2_buffer_head **bh, void *private)
+{
+ return del_eattr_generic(ip, block, parent, bh, private,
+ _("indirect extended attribute"));
+}
+
+int delete_eattr_entry(struct gfs2_inode *ip, struct gfs2_buffer_head *leaf_bh,
+ struct gfs2_ea_header *ea_hdr,
+ struct gfs2_ea_header *ea_hdr_prev, void *private)
+{
+ struct gfs2_sbd *sdp = ip->i_sbd;
+ char ea_name[256];
+ uint32_t avail_size;
+ int max_ptrs;
+
+ if (!ea_hdr->ea_name_len){
+ /* Skip this entry for now */
+ return 1;
+ }
+
+ memset(ea_name, 0, sizeof(ea_name));
+ strncpy(ea_name, (char *)ea_hdr + sizeof(struct gfs2_ea_header),
+ ea_hdr->ea_name_len);
+
+ if (!GFS2_EATYPE_VALID(ea_hdr->ea_type) &&
+ ((ea_hdr_prev) || (!ea_hdr_prev && ea_hdr->ea_type))){
+ /* Skip invalid entry */
+ return 1;
+ }
+
+ if (!ea_hdr->ea_num_ptrs)
+ return 0;
+
+ avail_size = sdp->sd_sb.sb_bsize - sizeof(struct gfs2_meta_header);
+ max_ptrs = (be32_to_cpu(ea_hdr->ea_data_len) + avail_size - 1) /
+ avail_size;
+
+ if (max_ptrs > ea_hdr->ea_num_ptrs)
+ return 1;
+
+ log_debug( _(" Pointers Required: %d\n Pointers Reported: %d\n"),
+ max_ptrs, ea_hdr->ea_num_ptrs);
+
+ return 0;
+}
+
+int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
+ struct gfs2_buffer_head *leaf_bh,
+ struct gfs2_ea_header *ea_hdr,
+ struct gfs2_ea_header *ea_hdr_prev, void *private)
+{
+ uint64_t block = be64_to_cpu(*ea_data_ptr);
+
+ return delete_block_if_notdup(ip, block, NULL, _("extended attribute"),
+ NULL, private);
+}
diff --git a/gfs2/fsck/afterpass1_common.h b/gfs2/fsck/afterpass1_common.h
new file mode 100644
index 0000000..2a422c7
--- /dev/null
+++ b/gfs2/fsck/afterpass1_common.h
@@ -0,0 +1,31 @@
+#ifndef _AFTERPASS1_H
+#define _AFTERPASS1_H
+
+#include "util.h"
+
+extern int delete_metadata(struct gfs2_inode *ip, uint64_t block,
+ struct gfs2_buffer_head **bh, int h, int *is_valid,
+ int *was_duplicate, void *private);
+extern int delete_leaf(struct gfs2_inode *ip, uint64_t block, void *private);
+extern int delete_data(struct gfs2_inode *ip, uint64_t metablock,
+ uint64_t block, void *private,
+ struct gfs2_buffer_head *bh, uint64_t *ptr);
+extern int delete_eattr_indir(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
+ struct gfs2_buffer_head **bh, void *private);
+extern int delete_eattr_leaf(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
+ struct gfs2_buffer_head **bh, void *private);
+extern int delete_eattr_entry(struct gfs2_inode *ip,
+ struct gfs2_buffer_head *leaf_bh,
+ struct gfs2_ea_header *ea_hdr,
+ struct gfs2_ea_header *ea_hdr_prev,
+ void *private);
+extern int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
+ struct gfs2_buffer_head *leaf_bh,
+ struct gfs2_ea_header *ea_hdr,
+ struct gfs2_ea_header *ea_hdr_prev,
+ void *private);
+extern int find_remove_dup(struct gfs2_inode *ip, uint64_t block,
+ const char *btype);
+extern int remove_dentry_from_dir(struct gfs2_sbd *sdp, uint64_t dir,
+ uint64_t dentryblock);
+#endif
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 7a77dc9..1f2bc10 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -989,93 +989,6 @@ static int check_leaf_eattr(struct gfs2_inode *ip, uint64_t block,
}
/**
- * delete_block - delete a block associated with an inode
- */
-int delete_block(struct gfs2_inode *ip, uint64_t block,
- struct gfs2_buffer_head **bh, const char *btype,
- void *private)
-{
- if (valid_block(ip->i_sbd, block)) {
- fsck_blockmap_set(ip, block, btype, GFS2_BLKST_FREE);
- return 0;
- }
- return -1;
-}
-
-/**
- * find_remove_dup - find out if this is a duplicate ref. If so, remove it.
- *
- * Returns: 1 if there are any remaining references to this block, else 0.
- */
-int find_remove_dup(struct gfs2_inode *ip, uint64_t block, const char *btype)
-{
- struct duptree *dt;
- struct inode_with_dups *id;
-
- dt = dupfind(block);
- if (!dt)
- return 0;
-
- /* remove the inode reference id structure for this reference. */
- id = find_dup_ref_inode(dt, ip);
- if (!id)
- goto more_refs;
-
- dup_listent_delete(dt, id);
- if (dt->refs == 0) {
- log_info( _("This was the last reference: it's no longer a "
- "duplicate.\n"));
- dup_delete(dt); /* not duplicate now */
- return 0;
- }
-more_refs:
- log_info( _("%d block reference(s) remain.\n"), dt->refs);
- return 1; /* references still exist so do not free the block. */
-}
-
-/**
- * delete_block_if_notdup - delete blocks associated with an inode
- *
- * Ignore blocks that are already marked free.
- * If it has been identified as duplicate, remove the duplicate reference.
- * If all duplicate references have been removed, delete the block.
- */
-static int delete_block_if_notdup(struct gfs2_inode *ip, uint64_t block,
- struct gfs2_buffer_head **bh,
- const char *btype, int *was_duplicate,
- void *private)
-{
- int q;
-
- if (!valid_block(ip->i_sbd, block))
- return meta_error;
-
- q = bitmap_type(ip->i_sbd, block);
- if (q == GFS2_BLKST_FREE) {
- log_info( _("%s block %lld (0x%llx), part of inode "
- "%lld (0x%llx), was already free.\n"),
- btype, (unsigned long long)block,
- (unsigned long long)block,
- (unsigned long long)ip->i_di.di_num.no_addr,
- (unsigned long long)ip->i_di.di_num.no_addr);
- return meta_is_good;
- }
- if (find_remove_dup(ip, block, btype)) { /* a dup */
- if (was_duplicate)
- *was_duplicate = 1;
- log_err( _("Not clearing duplicate reference in inode "
- "at block #%llu (0x%llx) to block #%llu (0x%llx) "
- "because it's referenced by another inode.\n"),
- (unsigned long long)ip->i_di.di_num.no_addr,
- (unsigned long long)ip->i_di.di_num.no_addr,
- (unsigned long long)block, (unsigned long long)block);
- } else {
- fsck_blockmap_set(ip, block, btype, GFS2_BLKST_FREE);
- }
- return meta_is_good;
-}
-
-/**
* check_indirect_eattr
* @ip: the inode the eattr comes from
* @indirect_block
@@ -1159,9 +1072,9 @@ static int check_indirect_eattr(struct gfs2_inode *ip, uint64_t indirect,
leaf_pointer_errors,
pass->private);
}
- if (leaf_pointer_errors &&
+ if (pass->delete_block && leaf_pointer_errors &&
leaf_pointer_errors == leaf_pointers) {
- delete_block(ip, indirect, NULL, "leaf", NULL);
+ pass->delete_block(ip, indirect, NULL, "leaf", NULL);
error = 1;
}
}
@@ -1729,184 +1642,3 @@ int check_dir(struct gfs2_sbd *sdp, struct gfs2_inode *ip, struct metawalk_fxns
return error;
}
-
-static int remove_dentry(struct gfs2_inode *ip, struct gfs2_dirent *dent,
- struct gfs2_dirent *prev_de,
- struct gfs2_buffer_head *bh,
- char *filename, uint32_t *count, int *lindex,
- void *private)
-{
- /* the metawalk_fxn's private field must be set to the dentry
- * block we want to clear */
- uint64_t *dentryblock = (uint64_t *) private;
- struct gfs2_dirent dentry, *de;
-
- memset(&dentry, 0, sizeof(struct gfs2_dirent));
- gfs2_dirent_in(&dentry, (char *)dent);
- de = &dentry;
-
- if (de->de_inum.no_addr == *dentryblock)
- dirent2_del(ip, bh, prev_de, dent);
- else
- (*count)++;
-
- return 0;
-
-}
-
-int remove_dentry_from_dir(struct gfs2_sbd *sdp, uint64_t dir,
- uint64_t dentryblock)
-{
- struct metawalk_fxns remove_dentry_fxns = {0};
- struct gfs2_inode *ip;
- int q;
- int error;
-
- log_debug( _("Removing dentry %llu (0x%llx) from directory %llu"
- " (0x%llx)\n"), (unsigned long long)dentryblock,
- (unsigned long long)dentryblock,
- (unsigned long long)dir, (unsigned long long)dir);
- if (!valid_block(sdp, dir)) {
- log_err( _("Parent directory is invalid\n"));
- return 1;
- }
- remove_dentry_fxns.private = &dentryblock;
- remove_dentry_fxns.check_dentry = remove_dentry;
-
- q = bitmap_type(sdp, dir);
- if (q != GFS2_BLKST_DINODE) {
- log_info( _("Parent block is not an inode...ignoring\n"));
- return 1;
- }
-
- ip = fsck_load_inode(sdp, dir);
- if (ip == NULL) {
- stack;
- return -1;
- }
- /* Need to run check_dir with a private var of dentryblock,
- * and fxns that remove that dentry if found */
- error = check_dir(sdp, ip, &remove_dentry_fxns);
- fsck_inode_put(&ip);
- return error;
-}
-
-int delete_metadata(struct gfs2_inode *ip, uint64_t block,
- struct gfs2_buffer_head **bh, int h, int *is_valid,
- int *was_duplicate, void *private)
-{
- *is_valid = 1;
- *was_duplicate = 0;
- return delete_block_if_notdup(ip, block, bh, _("metadata"),
- was_duplicate, private);
-}
-
-int delete_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
-{
- return delete_block_if_notdup(ip, block, NULL, _("leaf"), NULL,
- private);
-}
-
-int delete_data(struct gfs2_inode *ip, uint64_t metablock,
- uint64_t block, void *private, struct gfs2_buffer_head *bh,
- uint64_t *ptr)
-{
- return delete_block_if_notdup(ip, block, NULL, _("data"), NULL,
- private);
-}
-
-static int del_eattr_generic(struct gfs2_inode *ip, uint64_t block,
- uint64_t parent, struct gfs2_buffer_head **bh,
- void *private, const char *eatype)
-{
- int ret = 0;
- int was_free = 0;
- int q;
-
- if (valid_block(ip->i_sbd, block)) {
- q = bitmap_type(ip->i_sbd, block);
- if (q == GFS2_BLKST_FREE)
- was_free = 1;
- ret = delete_block_if_notdup(ip, block, NULL, eatype,
- NULL, private);
- if (!ret) {
- *bh = bread(ip->i_sbd, block);
- if (!was_free)
- ip->i_di.di_blocks--;
- bmodified(ip->i_bh);
- }
- }
- /* Even if it's a duplicate reference, we want to eliminate the
- reference itself, and adjust di_blocks accordingly. */
- if (ip->i_di.di_eattr) {
- if (block == ip->i_di.di_eattr)
- ip->i_di.di_eattr = 0;
- bmodified(ip->i_bh);
- }
- return ret;
-}
-
-int delete_eattr_indir(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
- struct gfs2_buffer_head **bh, void *private)
-{
- return del_eattr_generic(ip, block, parent, bh, private,
- _("extended attribute"));
-}
-
-int delete_eattr_leaf(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
- struct gfs2_buffer_head **bh, void *private)
-{
- return del_eattr_generic(ip, block, parent, bh, private,
- _("indirect extended attribute"));
-}
-
-int delete_eattr_entry(struct gfs2_inode *ip, struct gfs2_buffer_head *leaf_bh,
- struct gfs2_ea_header *ea_hdr,
- struct gfs2_ea_header *ea_hdr_prev, void *private)
-{
- struct gfs2_sbd *sdp = ip->i_sbd;
- char ea_name[256];
- uint32_t avail_size;
- int max_ptrs;
-
- if (!ea_hdr->ea_name_len){
- /* Skip this entry for now */
- return 1;
- }
-
- memset(ea_name, 0, sizeof(ea_name));
- strncpy(ea_name, (char *)ea_hdr + sizeof(struct gfs2_ea_header),
- ea_hdr->ea_name_len);
-
- if (!GFS2_EATYPE_VALID(ea_hdr->ea_type) &&
- ((ea_hdr_prev) || (!ea_hdr_prev && ea_hdr->ea_type))){
- /* Skip invalid entry */
- return 1;
- }
-
- if (!ea_hdr->ea_num_ptrs)
- return 0;
-
- avail_size = sdp->sd_sb.sb_bsize - sizeof(struct gfs2_meta_header);
- max_ptrs = (be32_to_cpu(ea_hdr->ea_data_len) + avail_size - 1) /
- avail_size;
-
- if (max_ptrs > ea_hdr->ea_num_ptrs)
- return 1;
-
- log_debug( _(" Pointers Required: %d\n Pointers Reported: %d\n"),
- max_ptrs, ea_hdr->ea_num_ptrs);
-
- return 0;
-}
-
-int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
- struct gfs2_buffer_head *leaf_bh,
- struct gfs2_ea_header *ea_hdr,
- struct gfs2_ea_header *ea_hdr_prev, void *private)
-{
- uint64_t block = be64_to_cpu(*ea_data_ptr);
-
- return delete_block_if_notdup(ip, block, NULL, _("extended attribute"),
- NULL, private);
-}
diff --git a/gfs2/fsck/metawalk.h b/gfs2/fsck/metawalk.h
index d505945..d2f0d97 100644
--- a/gfs2/fsck/metawalk.h
+++ b/gfs2/fsck/metawalk.h
@@ -19,33 +19,6 @@ extern int check_linear_dir(struct gfs2_inode *ip, struct gfs2_buffer_head *bh,
extern int check_leaf(struct gfs2_inode *ip, int lindex,
struct metawalk_fxns *pass, uint64_t *leaf_no,
struct gfs2_leaf *leaf, int *ref_count);
-extern int remove_dentry_from_dir(struct gfs2_sbd *sdp, uint64_t dir,
- uint64_t dentryblock);
-extern int delete_block(struct gfs2_inode *ip, uint64_t block,
- struct gfs2_buffer_head **bh, const char *btype,
- void *private);
-extern int delete_metadata(struct gfs2_inode *ip, uint64_t block,
- struct gfs2_buffer_head **bh, int h, int *is_valid,
- int *was_duplicate, void *private);
-extern int delete_leaf(struct gfs2_inode *ip, uint64_t block, void *private);
-extern int delete_data(struct gfs2_inode *ip, uint64_t metablock,
- uint64_t block, void *private,
- struct gfs2_buffer_head *bh, uint64_t *ptr);
-extern int delete_eattr_indir(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
- struct gfs2_buffer_head **bh, void *private);
-extern int delete_eattr_leaf(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
- struct gfs2_buffer_head **bh, void *private);
-extern int delete_eattr_entry(struct gfs2_inode *ip,
- struct gfs2_buffer_head *leaf_bh,
- struct gfs2_ea_header *ea_hdr,
- struct gfs2_ea_header *ea_hdr_prev,
- void *private);
-extern int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
- struct gfs2_buffer_head *leaf_bh,
- struct gfs2_ea_header *ea_hdr,
- struct gfs2_ea_header *ea_hdr_prev,
- void *private);
-
extern int _fsck_blockmap_set(struct gfs2_inode *ip, uint64_t bblock,
const char *btype, int mark, int error_on_dinode,
const char *caller, int line);
@@ -57,8 +30,6 @@ extern int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk,
extern struct duptree *dupfind(uint64_t block);
extern struct gfs2_inode *fsck_system_inode(struct gfs2_sbd *sdp,
uint64_t block);
-extern int find_remove_dup(struct gfs2_inode *ip, uint64_t block,
- const char *btype);
#define is_duplicate(dblock) ((dupfind(dblock)) ? 1 : 0)
@@ -154,6 +125,9 @@ struct metawalk_fxns {
int h, void *private);
int (*undo_check_data) (struct gfs2_inode *ip, uint64_t block,
void *private);
+ int (*delete_block) (struct gfs2_inode *ip, uint64_t block,
+ struct gfs2_buffer_head **bh, const char *btype,
+ void *private);
};
#endif /* _METAWALK_H */
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index d12e03e..fbacfea 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -83,6 +83,22 @@ static int invalidate_eattr_leaf(struct gfs2_inode *ip, uint64_t block,
uint64_t parent, struct gfs2_buffer_head **bh,
void *private);
static int handle_ip(struct gfs2_sbd *sdp, struct gfs2_inode *ip);
+static int delete_block(struct gfs2_inode *ip, uint64_t block,
+ struct gfs2_buffer_head **bh, const char *btype,
+ void *private);
+/**
+ * delete_block - delete a block associated with an inode
+ */
+static int delete_block(struct gfs2_inode *ip, uint64_t block,
+ struct gfs2_buffer_head **bh, const char *btype,
+ void *private)
+{
+ if (valid_block(ip->i_sbd, block)) {
+ fsck_blockmap_set(ip, block, btype, GFS2_BLKST_FREE);
+ return 0;
+ }
+ return -1;
+}
struct metawalk_fxns pass1_fxns = {
.private = NULL,
@@ -97,6 +113,7 @@ struct metawalk_fxns pass1_fxns = {
.big_file_msg = big_file_comfort,
.undo_check_meta = undo_check_metalist,
.undo_check_data = undo_check_data,
+ .delete_block = delete_block,
};
struct metawalk_fxns invalidate_fxns = {
@@ -106,6 +123,7 @@ struct metawalk_fxns invalidate_fxns = {
.check_leaf = invalidate_leaf,
.check_eattr_indir = invalidate_eattr_indir,
.check_eattr_leaf = invalidate_eattr_leaf,
+ .delete_block = delete_block,
};
/*
@@ -200,6 +218,7 @@ struct metawalk_fxns sysdir_fxns = {
.private = NULL,
.check_metalist = resuscitate_metalist,
.check_dentry = resuscitate_dentry,
+ .delete_block = delete_block,
};
static int p1check_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
@@ -1124,6 +1143,7 @@ struct metawalk_fxns rangecheck_fxns = {
.check_leaf = rangecheck_leaf,
.check_eattr_indir = rangecheck_eattr_indir,
.check_eattr_leaf = rangecheck_eattr_leaf,
+ .delete_block = delete_block,
};
struct metawalk_fxns eattr_undo_fxns = {
@@ -1131,6 +1151,7 @@ struct metawalk_fxns eattr_undo_fxns = {
.check_eattr_indir = undo_eattr_indir_or_leaf,
.check_eattr_leaf = undo_eattr_indir_or_leaf,
.finish_eattr_indir = finish_eattr_indir,
+ .delete_block = delete_block,
};
/* set_ip_blockmap - set the blockmap for a dinode
*
@@ -1252,6 +1273,7 @@ struct metawalk_fxns alloc_fxns = {
.check_eattr_entry = NULL,
.check_eattr_extentry = NULL,
.finish_eattr_indir = NULL,
+ .delete_block = delete_block,
};
/*
diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
index 20b603c..a9632d8 100644
--- a/gfs2/fsck/pass1b.c
+++ b/gfs2/fsck/pass1b.c
@@ -15,6 +15,7 @@
#include "util.h"
#include "metawalk.h"
#include "inode_hash.h"
+#include "afterpass1_common.h"
struct fxn_info {
uint64_t block;
diff --git a/gfs2/fsck/pass2.c b/gfs2/fsck/pass2.c
index a215b3c..02d82e4 100644
--- a/gfs2/fsck/pass2.c
+++ b/gfs2/fsck/pass2.c
@@ -17,6 +17,7 @@
#include "link.h"
#include "lost_n_found.h"
#include "inode_hash.h"
+#include "afterpass1_common.h"
#define MAX_FILENAME 256
diff --git a/gfs2/fsck/pass3.c b/gfs2/fsck/pass3.c
index 0df427d..d5f4ea2 100644
--- a/gfs2/fsck/pass3.c
+++ b/gfs2/fsck/pass3.c
@@ -16,6 +16,7 @@
#include "link.h"
#include "metawalk.h"
#include "util.h"
+#include "afterpass1_common.h"
static int attach_dotdot_to(struct gfs2_sbd *sdp, uint64_t newdotdot,
uint64_t olddotdot, uint64_t block)
diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
index ba38e8c..0d6cc9d 100644
--- a/gfs2/fsck/pass4.c
+++ b/gfs2/fsck/pass4.c
@@ -13,6 +13,7 @@
#include "inode_hash.h"
#include "metawalk.h"
#include "util.h"
+#include "afterpass1_common.h"
struct metawalk_fxns pass4_fxns_delete = {
.private = NULL,
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 12/40] fsck.gfs2: Separate out functions that may only be done after pass1
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 12/40] fsck.gfs2: Separate out functions that may only be done after pass1 Bob Peterson
@ 2016-05-09 14:55 ` Andrew Price
2016-05-09 16:35 ` Bob Peterson
0 siblings, 1 reply; 45+ messages in thread
From: Andrew Price @ 2016-05-09 14:55 UTC (permalink / raw)
To: cluster-devel.redhat.com
On 06/05/16 18:39, Bob Peterson wrote:
> This patch adds a new header file and source file which has the
> purpose of separating out the functions that may only be called
> after pass1 from the functions that pass1 may call. This is to
> ensure that there will never be inconsistencies between blockmap
> (which pass1 uses) and bitmap (which subsequent passes use).
>
> Signed-off-by: Bob Peterson <rpeterso@redhat.com>
> ---
> gfs2/fsck/Makefile.am | 1 +
> gfs2/fsck/afterpass1_common.c | 272 ++++++++++++++++++++++++++++++++++++++++++
> gfs2/fsck/afterpass1_common.h | 31 +++++
> gfs2/fsck/metawalk.c | 272 +-----------------------------------------
> gfs2/fsck/metawalk.h | 32 +----
> gfs2/fsck/pass1.c | 22 ++++
> gfs2/fsck/pass1b.c | 1 +
> gfs2/fsck/pass2.c | 1 +
> gfs2/fsck/pass3.c | 1 +
> gfs2/fsck/pass4.c | 1 +
> 10 files changed, 335 insertions(+), 299 deletions(-)
> create mode 100644 gfs2/fsck/afterpass1_common.c
> create mode 100644 gfs2/fsck/afterpass1_common.h
The .h file should be added to noinst_HEADERS in gfs2/fsck/Makefile.am
Andy
> diff --git a/gfs2/fsck/Makefile.am b/gfs2/fsck/Makefile.am
> index 73d957e..ecd05d8 100644
> --- a/gfs2/fsck/Makefile.am
> +++ b/gfs2/fsck/Makefile.am
> @@ -19,6 +19,7 @@ fsck_gfs2_SOURCES = \
> lost_n_found.c \
> main.c \
> metawalk.c \
> + afterpass1_common.c \
> pass1b.c \
> pass1.c \
> pass2.c \
> diff --git a/gfs2/fsck/afterpass1_common.c b/gfs2/fsck/afterpass1_common.c
> new file mode 100644
> index 0000000..36646f8
> --- /dev/null
> +++ b/gfs2/fsck/afterpass1_common.c
> @@ -0,0 +1,272 @@
> +#include <inttypes.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <unistd.h>
> +#include <libintl.h>
> +#include <ctype.h>
> +#include <fcntl.h>
> +#define _(String) gettext(String)
> +
> +#include <logging.h>
> +#include "libgfs2.h"
> +#include "fsck.h"
> +#include "afterpass1_common.h"
> +#include "metawalk.h"
> +#include "util.h"
> +
> +/**
> + * find_remove_dup - find out if this is a duplicate ref. If so, remove it.
> + *
> + * Returns: 1 if there are any remaining references to this block, else 0.
> + */
> +int find_remove_dup(struct gfs2_inode *ip, uint64_t block, const char *btype)
> +{
> + struct duptree *dt;
> + struct inode_with_dups *id;
> +
> + dt = dupfind(block);
> + if (!dt)
> + return 0;
> +
> + /* remove the inode reference id structure for this reference. */
> + id = find_dup_ref_inode(dt, ip);
> + if (!id)
> + goto more_refs;
> +
> + dup_listent_delete(dt, id);
> + if (dt->refs == 0) {
> + log_info( _("This was the last reference: it's no longer a "
> + "duplicate.\n"));
> + dup_delete(dt); /* not duplicate now */
> + return 0;
> + }
> +more_refs:
> + log_info( _("%d block reference(s) remain.\n"), dt->refs);
> + return 1; /* references still exist so do not free the block. */
> +}
> +
> +/**
> + * delete_block_if_notdup - delete blocks associated with an inode
> + *
> + * Ignore blocks that are already marked free.
> + * If it has been identified as duplicate, remove the duplicate reference.
> + * If all duplicate references have been removed, delete the block.
> + */
> +static int delete_block_if_notdup(struct gfs2_inode *ip, uint64_t block,
> + struct gfs2_buffer_head **bh,
> + const char *btype, int *was_duplicate,
> + void *private)
> +{
> + int q;
> +
> + if (!valid_block(ip->i_sbd, block))
> + return meta_error;
> +
> + q = bitmap_type(ip->i_sbd, block);
> + if (q == GFS2_BLKST_FREE) {
> + log_info( _("%s block %lld (0x%llx), part of inode "
> + "%lld (0x%llx), was already free.\n"),
> + btype, (unsigned long long)block,
> + (unsigned long long)block,
> + (unsigned long long)ip->i_di.di_num.no_addr,
> + (unsigned long long)ip->i_di.di_num.no_addr);
> + return meta_is_good;
> + }
> + if (find_remove_dup(ip, block, btype)) { /* a dup */
> + if (was_duplicate)
> + *was_duplicate = 1;
> + log_err( _("Not clearing duplicate reference in inode "
> + "at block #%llu (0x%llx) to block #%llu (0x%llx) "
> + "because it's referenced by another inode.\n"),
> + (unsigned long long)ip->i_di.di_num.no_addr,
> + (unsigned long long)ip->i_di.di_num.no_addr,
> + (unsigned long long)block, (unsigned long long)block);
> + } else {
> + check_n_fix_bitmap(ip->i_sbd, block, 0, GFS2_BLKST_FREE);
> + }
> + return meta_is_good;
> +}
> +
> +static int remove_dentry(struct gfs2_inode *ip, struct gfs2_dirent *dent,
> + struct gfs2_dirent *prev_de,
> + struct gfs2_buffer_head *bh,
> + char *filename, uint32_t *count, int *lindex,
> + void *private)
> +{
> + /* the metawalk_fxn's private field must be set to the dentry
> + * block we want to clear */
> + uint64_t *dentryblock = (uint64_t *) private;
> + struct gfs2_dirent dentry, *de;
> +
> + memset(&dentry, 0, sizeof(struct gfs2_dirent));
> + gfs2_dirent_in(&dentry, (char *)dent);
> + de = &dentry;
> +
> + if (de->de_inum.no_addr == *dentryblock)
> + dirent2_del(ip, bh, prev_de, dent);
> + else
> + (*count)++;
> +
> + return 0;
> +
> +}
> +
> +int remove_dentry_from_dir(struct gfs2_sbd *sdp, uint64_t dir,
> + uint64_t dentryblock)
> +{
> + struct metawalk_fxns remove_dentry_fxns = {0};
> + struct gfs2_inode *ip;
> + int q;
> + int error;
> +
> + log_debug( _("Removing dentry %llu (0x%llx) from directory %llu"
> + " (0x%llx)\n"), (unsigned long long)dentryblock,
> + (unsigned long long)dentryblock,
> + (unsigned long long)dir, (unsigned long long)dir);
> + if (!valid_block(sdp, dir)) {
> + log_err( _("Parent directory is invalid\n"));
> + return 1;
> + }
> + remove_dentry_fxns.private = &dentryblock;
> + remove_dentry_fxns.check_dentry = remove_dentry;
> +
> + q = bitmap_type(sdp, dir);
> + if (q != GFS2_BLKST_DINODE) {
> + log_info( _("Parent block is not an inode...ignoring\n"));
> + return 1;
> + }
> +
> + ip = fsck_load_inode(sdp, dir);
> + if (ip == NULL) {
> + stack;
> + return -1;
> + }
> + /* Need to run check_dir with a private var of dentryblock,
> + * and fxns that remove that dentry if found */
> + error = check_dir(sdp, ip, &remove_dentry_fxns);
> + fsck_inode_put(&ip);
> + return error;
> +}
> +
> +int delete_metadata(struct gfs2_inode *ip, uint64_t block,
> + struct gfs2_buffer_head **bh, int h, int *is_valid,
> + int *was_duplicate, void *private)
> +{
> + *is_valid = 1;
> + *was_duplicate = 0;
> + return delete_block_if_notdup(ip, block, bh, _("metadata"),
> + was_duplicate, private);
> +}
> +
> +int delete_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
> +{
> + return delete_block_if_notdup(ip, block, NULL, _("leaf"), NULL,
> + private);
> +}
> +
> +int delete_data(struct gfs2_inode *ip, uint64_t metablock,
> + uint64_t block, void *private, struct gfs2_buffer_head *bh,
> + uint64_t *ptr)
> +{
> + return delete_block_if_notdup(ip, block, NULL, _("data"), NULL,
> + private);
> +}
> +
> +static int del_eattr_generic(struct gfs2_inode *ip, uint64_t block,
> + uint64_t parent, struct gfs2_buffer_head **bh,
> + void *private, const char *eatype)
> +{
> + int ret = 0;
> + int was_free = 0;
> + int q;
> +
> + if (valid_block(ip->i_sbd, block)) {
> + q = bitmap_type(ip->i_sbd, block);
> + if (q == GFS2_BLKST_FREE)
> + was_free = 1;
> + ret = delete_block_if_notdup(ip, block, NULL, eatype,
> + NULL, private);
> + if (!ret) {
> + *bh = bread(ip->i_sbd, block);
> + if (!was_free)
> + ip->i_di.di_blocks--;
> + bmodified(ip->i_bh);
> + }
> + }
> + /* Even if it's a duplicate reference, we want to eliminate the
> + reference itself, and adjust di_blocks accordingly. */
> + if (ip->i_di.di_eattr) {
> + if (block == ip->i_di.di_eattr)
> + ip->i_di.di_eattr = 0;
> + bmodified(ip->i_bh);
> + }
> + return ret;
> +}
> +
> +int delete_eattr_indir(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
> + struct gfs2_buffer_head **bh, void *private)
> +{
> + return del_eattr_generic(ip, block, parent, bh, private,
> + _("extended attribute"));
> +}
> +
> +int delete_eattr_leaf(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
> + struct gfs2_buffer_head **bh, void *private)
> +{
> + return del_eattr_generic(ip, block, parent, bh, private,
> + _("indirect extended attribute"));
> +}
> +
> +int delete_eattr_entry(struct gfs2_inode *ip, struct gfs2_buffer_head *leaf_bh,
> + struct gfs2_ea_header *ea_hdr,
> + struct gfs2_ea_header *ea_hdr_prev, void *private)
> +{
> + struct gfs2_sbd *sdp = ip->i_sbd;
> + char ea_name[256];
> + uint32_t avail_size;
> + int max_ptrs;
> +
> + if (!ea_hdr->ea_name_len){
> + /* Skip this entry for now */
> + return 1;
> + }
> +
> + memset(ea_name, 0, sizeof(ea_name));
> + strncpy(ea_name, (char *)ea_hdr + sizeof(struct gfs2_ea_header),
> + ea_hdr->ea_name_len);
> +
> + if (!GFS2_EATYPE_VALID(ea_hdr->ea_type) &&
> + ((ea_hdr_prev) || (!ea_hdr_prev && ea_hdr->ea_type))){
> + /* Skip invalid entry */
> + return 1;
> + }
> +
> + if (!ea_hdr->ea_num_ptrs)
> + return 0;
> +
> + avail_size = sdp->sd_sb.sb_bsize - sizeof(struct gfs2_meta_header);
> + max_ptrs = (be32_to_cpu(ea_hdr->ea_data_len) + avail_size - 1) /
> + avail_size;
> +
> + if (max_ptrs > ea_hdr->ea_num_ptrs)
> + return 1;
> +
> + log_debug( _(" Pointers Required: %d\n Pointers Reported: %d\n"),
> + max_ptrs, ea_hdr->ea_num_ptrs);
> +
> + return 0;
> +}
> +
> +int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
> + struct gfs2_buffer_head *leaf_bh,
> + struct gfs2_ea_header *ea_hdr,
> + struct gfs2_ea_header *ea_hdr_prev, void *private)
> +{
> + uint64_t block = be64_to_cpu(*ea_data_ptr);
> +
> + return delete_block_if_notdup(ip, block, NULL, _("extended attribute"),
> + NULL, private);
> +}
> diff --git a/gfs2/fsck/afterpass1_common.h b/gfs2/fsck/afterpass1_common.h
> new file mode 100644
> index 0000000..2a422c7
> --- /dev/null
> +++ b/gfs2/fsck/afterpass1_common.h
> @@ -0,0 +1,31 @@
> +#ifndef _AFTERPASS1_H
> +#define _AFTERPASS1_H
> +
> +#include "util.h"
> +
> +extern int delete_metadata(struct gfs2_inode *ip, uint64_t block,
> + struct gfs2_buffer_head **bh, int h, int *is_valid,
> + int *was_duplicate, void *private);
> +extern int delete_leaf(struct gfs2_inode *ip, uint64_t block, void *private);
> +extern int delete_data(struct gfs2_inode *ip, uint64_t metablock,
> + uint64_t block, void *private,
> + struct gfs2_buffer_head *bh, uint64_t *ptr);
> +extern int delete_eattr_indir(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
> + struct gfs2_buffer_head **bh, void *private);
> +extern int delete_eattr_leaf(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
> + struct gfs2_buffer_head **bh, void *private);
> +extern int delete_eattr_entry(struct gfs2_inode *ip,
> + struct gfs2_buffer_head *leaf_bh,
> + struct gfs2_ea_header *ea_hdr,
> + struct gfs2_ea_header *ea_hdr_prev,
> + void *private);
> +extern int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
> + struct gfs2_buffer_head *leaf_bh,
> + struct gfs2_ea_header *ea_hdr,
> + struct gfs2_ea_header *ea_hdr_prev,
> + void *private);
> +extern int find_remove_dup(struct gfs2_inode *ip, uint64_t block,
> + const char *btype);
> +extern int remove_dentry_from_dir(struct gfs2_sbd *sdp, uint64_t dir,
> + uint64_t dentryblock);
> +#endif
> diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
> index 7a77dc9..1f2bc10 100644
> --- a/gfs2/fsck/metawalk.c
> +++ b/gfs2/fsck/metawalk.c
> @@ -989,93 +989,6 @@ static int check_leaf_eattr(struct gfs2_inode *ip, uint64_t block,
> }
>
> /**
> - * delete_block - delete a block associated with an inode
> - */
> -int delete_block(struct gfs2_inode *ip, uint64_t block,
> - struct gfs2_buffer_head **bh, const char *btype,
> - void *private)
> -{
> - if (valid_block(ip->i_sbd, block)) {
> - fsck_blockmap_set(ip, block, btype, GFS2_BLKST_FREE);
> - return 0;
> - }
> - return -1;
> -}
> -
> -/**
> - * find_remove_dup - find out if this is a duplicate ref. If so, remove it.
> - *
> - * Returns: 1 if there are any remaining references to this block, else 0.
> - */
> -int find_remove_dup(struct gfs2_inode *ip, uint64_t block, const char *btype)
> -{
> - struct duptree *dt;
> - struct inode_with_dups *id;
> -
> - dt = dupfind(block);
> - if (!dt)
> - return 0;
> -
> - /* remove the inode reference id structure for this reference. */
> - id = find_dup_ref_inode(dt, ip);
> - if (!id)
> - goto more_refs;
> -
> - dup_listent_delete(dt, id);
> - if (dt->refs == 0) {
> - log_info( _("This was the last reference: it's no longer a "
> - "duplicate.\n"));
> - dup_delete(dt); /* not duplicate now */
> - return 0;
> - }
> -more_refs:
> - log_info( _("%d block reference(s) remain.\n"), dt->refs);
> - return 1; /* references still exist so do not free the block. */
> -}
> -
> -/**
> - * delete_block_if_notdup - delete blocks associated with an inode
> - *
> - * Ignore blocks that are already marked free.
> - * If it has been identified as duplicate, remove the duplicate reference.
> - * If all duplicate references have been removed, delete the block.
> - */
> -static int delete_block_if_notdup(struct gfs2_inode *ip, uint64_t block,
> - struct gfs2_buffer_head **bh,
> - const char *btype, int *was_duplicate,
> - void *private)
> -{
> - int q;
> -
> - if (!valid_block(ip->i_sbd, block))
> - return meta_error;
> -
> - q = bitmap_type(ip->i_sbd, block);
> - if (q == GFS2_BLKST_FREE) {
> - log_info( _("%s block %lld (0x%llx), part of inode "
> - "%lld (0x%llx), was already free.\n"),
> - btype, (unsigned long long)block,
> - (unsigned long long)block,
> - (unsigned long long)ip->i_di.di_num.no_addr,
> - (unsigned long long)ip->i_di.di_num.no_addr);
> - return meta_is_good;
> - }
> - if (find_remove_dup(ip, block, btype)) { /* a dup */
> - if (was_duplicate)
> - *was_duplicate = 1;
> - log_err( _("Not clearing duplicate reference in inode "
> - "at block #%llu (0x%llx) to block #%llu (0x%llx) "
> - "because it's referenced by another inode.\n"),
> - (unsigned long long)ip->i_di.di_num.no_addr,
> - (unsigned long long)ip->i_di.di_num.no_addr,
> - (unsigned long long)block, (unsigned long long)block);
> - } else {
> - fsck_blockmap_set(ip, block, btype, GFS2_BLKST_FREE);
> - }
> - return meta_is_good;
> -}
> -
> -/**
> * check_indirect_eattr
> * @ip: the inode the eattr comes from
> * @indirect_block
> @@ -1159,9 +1072,9 @@ static int check_indirect_eattr(struct gfs2_inode *ip, uint64_t indirect,
> leaf_pointer_errors,
> pass->private);
> }
> - if (leaf_pointer_errors &&
> + if (pass->delete_block && leaf_pointer_errors &&
> leaf_pointer_errors == leaf_pointers) {
> - delete_block(ip, indirect, NULL, "leaf", NULL);
> + pass->delete_block(ip, indirect, NULL, "leaf", NULL);
> error = 1;
> }
> }
> @@ -1729,184 +1642,3 @@ int check_dir(struct gfs2_sbd *sdp, struct gfs2_inode *ip, struct metawalk_fxns
>
> return error;
> }
> -
> -static int remove_dentry(struct gfs2_inode *ip, struct gfs2_dirent *dent,
> - struct gfs2_dirent *prev_de,
> - struct gfs2_buffer_head *bh,
> - char *filename, uint32_t *count, int *lindex,
> - void *private)
> -{
> - /* the metawalk_fxn's private field must be set to the dentry
> - * block we want to clear */
> - uint64_t *dentryblock = (uint64_t *) private;
> - struct gfs2_dirent dentry, *de;
> -
> - memset(&dentry, 0, sizeof(struct gfs2_dirent));
> - gfs2_dirent_in(&dentry, (char *)dent);
> - de = &dentry;
> -
> - if (de->de_inum.no_addr == *dentryblock)
> - dirent2_del(ip, bh, prev_de, dent);
> - else
> - (*count)++;
> -
> - return 0;
> -
> -}
> -
> -int remove_dentry_from_dir(struct gfs2_sbd *sdp, uint64_t dir,
> - uint64_t dentryblock)
> -{
> - struct metawalk_fxns remove_dentry_fxns = {0};
> - struct gfs2_inode *ip;
> - int q;
> - int error;
> -
> - log_debug( _("Removing dentry %llu (0x%llx) from directory %llu"
> - " (0x%llx)\n"), (unsigned long long)dentryblock,
> - (unsigned long long)dentryblock,
> - (unsigned long long)dir, (unsigned long long)dir);
> - if (!valid_block(sdp, dir)) {
> - log_err( _("Parent directory is invalid\n"));
> - return 1;
> - }
> - remove_dentry_fxns.private = &dentryblock;
> - remove_dentry_fxns.check_dentry = remove_dentry;
> -
> - q = bitmap_type(sdp, dir);
> - if (q != GFS2_BLKST_DINODE) {
> - log_info( _("Parent block is not an inode...ignoring\n"));
> - return 1;
> - }
> -
> - ip = fsck_load_inode(sdp, dir);
> - if (ip == NULL) {
> - stack;
> - return -1;
> - }
> - /* Need to run check_dir with a private var of dentryblock,
> - * and fxns that remove that dentry if found */
> - error = check_dir(sdp, ip, &remove_dentry_fxns);
> - fsck_inode_put(&ip);
> - return error;
> -}
> -
> -int delete_metadata(struct gfs2_inode *ip, uint64_t block,
> - struct gfs2_buffer_head **bh, int h, int *is_valid,
> - int *was_duplicate, void *private)
> -{
> - *is_valid = 1;
> - *was_duplicate = 0;
> - return delete_block_if_notdup(ip, block, bh, _("metadata"),
> - was_duplicate, private);
> -}
> -
> -int delete_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
> -{
> - return delete_block_if_notdup(ip, block, NULL, _("leaf"), NULL,
> - private);
> -}
> -
> -int delete_data(struct gfs2_inode *ip, uint64_t metablock,
> - uint64_t block, void *private, struct gfs2_buffer_head *bh,
> - uint64_t *ptr)
> -{
> - return delete_block_if_notdup(ip, block, NULL, _("data"), NULL,
> - private);
> -}
> -
> -static int del_eattr_generic(struct gfs2_inode *ip, uint64_t block,
> - uint64_t parent, struct gfs2_buffer_head **bh,
> - void *private, const char *eatype)
> -{
> - int ret = 0;
> - int was_free = 0;
> - int q;
> -
> - if (valid_block(ip->i_sbd, block)) {
> - q = bitmap_type(ip->i_sbd, block);
> - if (q == GFS2_BLKST_FREE)
> - was_free = 1;
> - ret = delete_block_if_notdup(ip, block, NULL, eatype,
> - NULL, private);
> - if (!ret) {
> - *bh = bread(ip->i_sbd, block);
> - if (!was_free)
> - ip->i_di.di_blocks--;
> - bmodified(ip->i_bh);
> - }
> - }
> - /* Even if it's a duplicate reference, we want to eliminate the
> - reference itself, and adjust di_blocks accordingly. */
> - if (ip->i_di.di_eattr) {
> - if (block == ip->i_di.di_eattr)
> - ip->i_di.di_eattr = 0;
> - bmodified(ip->i_bh);
> - }
> - return ret;
> -}
> -
> -int delete_eattr_indir(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
> - struct gfs2_buffer_head **bh, void *private)
> -{
> - return del_eattr_generic(ip, block, parent, bh, private,
> - _("extended attribute"));
> -}
> -
> -int delete_eattr_leaf(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
> - struct gfs2_buffer_head **bh, void *private)
> -{
> - return del_eattr_generic(ip, block, parent, bh, private,
> - _("indirect extended attribute"));
> -}
> -
> -int delete_eattr_entry(struct gfs2_inode *ip, struct gfs2_buffer_head *leaf_bh,
> - struct gfs2_ea_header *ea_hdr,
> - struct gfs2_ea_header *ea_hdr_prev, void *private)
> -{
> - struct gfs2_sbd *sdp = ip->i_sbd;
> - char ea_name[256];
> - uint32_t avail_size;
> - int max_ptrs;
> -
> - if (!ea_hdr->ea_name_len){
> - /* Skip this entry for now */
> - return 1;
> - }
> -
> - memset(ea_name, 0, sizeof(ea_name));
> - strncpy(ea_name, (char *)ea_hdr + sizeof(struct gfs2_ea_header),
> - ea_hdr->ea_name_len);
> -
> - if (!GFS2_EATYPE_VALID(ea_hdr->ea_type) &&
> - ((ea_hdr_prev) || (!ea_hdr_prev && ea_hdr->ea_type))){
> - /* Skip invalid entry */
> - return 1;
> - }
> -
> - if (!ea_hdr->ea_num_ptrs)
> - return 0;
> -
> - avail_size = sdp->sd_sb.sb_bsize - sizeof(struct gfs2_meta_header);
> - max_ptrs = (be32_to_cpu(ea_hdr->ea_data_len) + avail_size - 1) /
> - avail_size;
> -
> - if (max_ptrs > ea_hdr->ea_num_ptrs)
> - return 1;
> -
> - log_debug( _(" Pointers Required: %d\n Pointers Reported: %d\n"),
> - max_ptrs, ea_hdr->ea_num_ptrs);
> -
> - return 0;
> -}
> -
> -int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
> - struct gfs2_buffer_head *leaf_bh,
> - struct gfs2_ea_header *ea_hdr,
> - struct gfs2_ea_header *ea_hdr_prev, void *private)
> -{
> - uint64_t block = be64_to_cpu(*ea_data_ptr);
> -
> - return delete_block_if_notdup(ip, block, NULL, _("extended attribute"),
> - NULL, private);
> -}
> diff --git a/gfs2/fsck/metawalk.h b/gfs2/fsck/metawalk.h
> index d505945..d2f0d97 100644
> --- a/gfs2/fsck/metawalk.h
> +++ b/gfs2/fsck/metawalk.h
> @@ -19,33 +19,6 @@ extern int check_linear_dir(struct gfs2_inode *ip, struct gfs2_buffer_head *bh,
> extern int check_leaf(struct gfs2_inode *ip, int lindex,
> struct metawalk_fxns *pass, uint64_t *leaf_no,
> struct gfs2_leaf *leaf, int *ref_count);
> -extern int remove_dentry_from_dir(struct gfs2_sbd *sdp, uint64_t dir,
> - uint64_t dentryblock);
> -extern int delete_block(struct gfs2_inode *ip, uint64_t block,
> - struct gfs2_buffer_head **bh, const char *btype,
> - void *private);
> -extern int delete_metadata(struct gfs2_inode *ip, uint64_t block,
> - struct gfs2_buffer_head **bh, int h, int *is_valid,
> - int *was_duplicate, void *private);
> -extern int delete_leaf(struct gfs2_inode *ip, uint64_t block, void *private);
> -extern int delete_data(struct gfs2_inode *ip, uint64_t metablock,
> - uint64_t block, void *private,
> - struct gfs2_buffer_head *bh, uint64_t *ptr);
> -extern int delete_eattr_indir(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
> - struct gfs2_buffer_head **bh, void *private);
> -extern int delete_eattr_leaf(struct gfs2_inode *ip, uint64_t block, uint64_t parent,
> - struct gfs2_buffer_head **bh, void *private);
> -extern int delete_eattr_entry(struct gfs2_inode *ip,
> - struct gfs2_buffer_head *leaf_bh,
> - struct gfs2_ea_header *ea_hdr,
> - struct gfs2_ea_header *ea_hdr_prev,
> - void *private);
> -extern int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
> - struct gfs2_buffer_head *leaf_bh,
> - struct gfs2_ea_header *ea_hdr,
> - struct gfs2_ea_header *ea_hdr_prev,
> - void *private);
> -
> extern int _fsck_blockmap_set(struct gfs2_inode *ip, uint64_t bblock,
> const char *btype, int mark, int error_on_dinode,
> const char *caller, int line);
> @@ -57,8 +30,6 @@ extern int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk,
> extern struct duptree *dupfind(uint64_t block);
> extern struct gfs2_inode *fsck_system_inode(struct gfs2_sbd *sdp,
> uint64_t block);
> -extern int find_remove_dup(struct gfs2_inode *ip, uint64_t block,
> - const char *btype);
>
> #define is_duplicate(dblock) ((dupfind(dblock)) ? 1 : 0)
>
> @@ -154,6 +125,9 @@ struct metawalk_fxns {
> int h, void *private);
> int (*undo_check_data) (struct gfs2_inode *ip, uint64_t block,
> void *private);
> + int (*delete_block) (struct gfs2_inode *ip, uint64_t block,
> + struct gfs2_buffer_head **bh, const char *btype,
> + void *private);
> };
>
> #endif /* _METAWALK_H */
> diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
> index d12e03e..fbacfea 100644
> --- a/gfs2/fsck/pass1.c
> +++ b/gfs2/fsck/pass1.c
> @@ -83,6 +83,22 @@ static int invalidate_eattr_leaf(struct gfs2_inode *ip, uint64_t block,
> uint64_t parent, struct gfs2_buffer_head **bh,
> void *private);
> static int handle_ip(struct gfs2_sbd *sdp, struct gfs2_inode *ip);
> +static int delete_block(struct gfs2_inode *ip, uint64_t block,
> + struct gfs2_buffer_head **bh, const char *btype,
> + void *private);
> +/**
> + * delete_block - delete a block associated with an inode
> + */
> +static int delete_block(struct gfs2_inode *ip, uint64_t block,
> + struct gfs2_buffer_head **bh, const char *btype,
> + void *private)
> +{
> + if (valid_block(ip->i_sbd, block)) {
> + fsck_blockmap_set(ip, block, btype, GFS2_BLKST_FREE);
> + return 0;
> + }
> + return -1;
> +}
>
> struct metawalk_fxns pass1_fxns = {
> .private = NULL,
> @@ -97,6 +113,7 @@ struct metawalk_fxns pass1_fxns = {
> .big_file_msg = big_file_comfort,
> .undo_check_meta = undo_check_metalist,
> .undo_check_data = undo_check_data,
> + .delete_block = delete_block,
> };
>
> struct metawalk_fxns invalidate_fxns = {
> @@ -106,6 +123,7 @@ struct metawalk_fxns invalidate_fxns = {
> .check_leaf = invalidate_leaf,
> .check_eattr_indir = invalidate_eattr_indir,
> .check_eattr_leaf = invalidate_eattr_leaf,
> + .delete_block = delete_block,
> };
>
> /*
> @@ -200,6 +218,7 @@ struct metawalk_fxns sysdir_fxns = {
> .private = NULL,
> .check_metalist = resuscitate_metalist,
> .check_dentry = resuscitate_dentry,
> + .delete_block = delete_block,
> };
>
> static int p1check_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
> @@ -1124,6 +1143,7 @@ struct metawalk_fxns rangecheck_fxns = {
> .check_leaf = rangecheck_leaf,
> .check_eattr_indir = rangecheck_eattr_indir,
> .check_eattr_leaf = rangecheck_eattr_leaf,
> + .delete_block = delete_block,
> };
>
> struct metawalk_fxns eattr_undo_fxns = {
> @@ -1131,6 +1151,7 @@ struct metawalk_fxns eattr_undo_fxns = {
> .check_eattr_indir = undo_eattr_indir_or_leaf,
> .check_eattr_leaf = undo_eattr_indir_or_leaf,
> .finish_eattr_indir = finish_eattr_indir,
> + .delete_block = delete_block,
> };
> /* set_ip_blockmap - set the blockmap for a dinode
> *
> @@ -1252,6 +1273,7 @@ struct metawalk_fxns alloc_fxns = {
> .check_eattr_entry = NULL,
> .check_eattr_extentry = NULL,
> .finish_eattr_indir = NULL,
> + .delete_block = delete_block,
> };
>
> /*
> diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
> index 20b603c..a9632d8 100644
> --- a/gfs2/fsck/pass1b.c
> +++ b/gfs2/fsck/pass1b.c
> @@ -15,6 +15,7 @@
> #include "util.h"
> #include "metawalk.h"
> #include "inode_hash.h"
> +#include "afterpass1_common.h"
>
> struct fxn_info {
> uint64_t block;
> diff --git a/gfs2/fsck/pass2.c b/gfs2/fsck/pass2.c
> index a215b3c..02d82e4 100644
> --- a/gfs2/fsck/pass2.c
> +++ b/gfs2/fsck/pass2.c
> @@ -17,6 +17,7 @@
> #include "link.h"
> #include "lost_n_found.h"
> #include "inode_hash.h"
> +#include "afterpass1_common.h"
>
> #define MAX_FILENAME 256
>
> diff --git a/gfs2/fsck/pass3.c b/gfs2/fsck/pass3.c
> index 0df427d..d5f4ea2 100644
> --- a/gfs2/fsck/pass3.c
> +++ b/gfs2/fsck/pass3.c
> @@ -16,6 +16,7 @@
> #include "link.h"
> #include "metawalk.h"
> #include "util.h"
> +#include "afterpass1_common.h"
>
> static int attach_dotdot_to(struct gfs2_sbd *sdp, uint64_t newdotdot,
> uint64_t olddotdot, uint64_t block)
> diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
> index ba38e8c..0d6cc9d 100644
> --- a/gfs2/fsck/pass4.c
> +++ b/gfs2/fsck/pass4.c
> @@ -13,6 +13,7 @@
> #include "inode_hash.h"
> #include "metawalk.h"
> #include "util.h"
> +#include "afterpass1_common.h"
>
> struct metawalk_fxns pass4_fxns_delete = {
> .private = NULL,
>
^ permalink raw reply [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 12/40] fsck.gfs2: Separate out functions that may only be done after pass1
2016-05-09 14:55 ` Andrew Price
@ 2016-05-09 16:35 ` Bob Peterson
0 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-09 16:35 UTC (permalink / raw)
To: cluster-devel.redhat.com
----- Original Message -----
| On 06/05/16 18:39, Bob Peterson wrote:
| > This patch adds a new header file and source file which has the
| > purpose of separating out the functions that may only be called
| > after pass1 from the functions that pass1 may call. This is to
| > ensure that there will never be inconsistencies between blockmap
| > (which pass1 uses) and bitmap (which subsequent passes use).
| >
| > Signed-off-by: Bob Peterson <rpeterso@redhat.com>
| > ---
| > gfs2/fsck/Makefile.am | 1 +
| > gfs2/fsck/afterpass1_common.c | 272
| > ++++++++++++++++++++++++++++++++++++++++++
| > gfs2/fsck/afterpass1_common.h | 31 +++++
| > gfs2/fsck/metawalk.c | 272
| > +-----------------------------------------
| > gfs2/fsck/metawalk.h | 32 +----
| > gfs2/fsck/pass1.c | 22 ++++
| > gfs2/fsck/pass1b.c | 1 +
| > gfs2/fsck/pass2.c | 1 +
| > gfs2/fsck/pass3.c | 1 +
| > gfs2/fsck/pass4.c | 1 +
| > 10 files changed, 335 insertions(+), 299 deletions(-)
| > create mode 100644 gfs2/fsck/afterpass1_common.c
| > create mode 100644 gfs2/fsck/afterpass1_common.h
|
| The .h file should be added to noinst_HEADERS in gfs2/fsck/Makefile.am
|
| Andy
Hi Andy,
Good catch. I'll add it before pushing.
Bob Peterson
Red Hat File Systems
^ permalink raw reply [flat|nested] 45+ messages in thread
* [Cluster-devel] [fsck.gfs2 v2 PATCH 13/40] fsck.gfs2: Divest check_metatree from fsck_blockmap_set
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (11 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 12/40] fsck.gfs2: Separate out functions that may only be done after pass1 Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 14/40] fsck.gfs2: eliminate fsck_blockmap_set from check_eattr_entries Bob Peterson
` (28 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch replaces the call to function fsck_blockmap_set in
function check_metatree (which is used in all passes) to a
corresponding call to fsck_bitmap_set. This is another step to
having pass1 be the only function to manipulate the blockmap.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/metawalk.c | 4 ++--
gfs2/fsck/pass1.c | 7 ++++++-
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 1f2bc10..529c86d 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -1604,8 +1604,8 @@ undo_metalist:
to undo. */
delete_all_dups(ip);
/* Set the dinode as "bad" so it gets deleted */
- fsck_blockmap_set(ip, ip->i_di.di_num.no_addr,
- _("corrupt"), GFS2_BLKST_FREE);
+ fsck_bitmap_set(ip, ip->i_di.di_num.no_addr, _("corrupt"),
+ GFS2_BLKST_FREE);
log_err(_("The corrupt inode was invalidated.\n"));
out:
free_metalist(ip, &metalist[0]);
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index fbacfea..0ae842e 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -1304,9 +1304,14 @@ static void reprocess_inode(struct gfs2_inode *ip, const char *desc)
(unsigned long long)ip->i_di.di_num.no_addr,
ip->i_di.di_height);
error = check_metatree(ip, &alloc_fxns);
- if (error)
+ if (error) {
+ /* check_metatree will have fixed the bitmap, but not the
+ blockmap. */
+ gfs2_blockmap_set(bl, ip->i_di.di_num.no_addr,
+ GFS2_BLKST_FREE);
log_err( _("Error %d reprocessing the %s metadata tree.\n"),
error, desc);
+ }
}
/*
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 14/40] fsck.gfs2: eliminate fsck_blockmap_set from check_eattr_entries
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (12 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 13/40] fsck.gfs2: Divest check_metatree from fsck_blockmap_set Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 15/40] fsck.gfs2: Move blockmap stuff to pass1.c Bob Peterson
` (27 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
Before this patch, metawalk function check_eattr_entries was calling
function fsck_blockmap_set, but only on error conditions.
This patch makes it the caller's responsibility. Since pass1 called
check_eattr_entries, it needed to set the blockmap and bitmap, so
that's now included in the pass1->check_eattr_extentry caller.
Subsequent passes don't need to set the blockmap, but they do need
to set the bitmap. That's also moved to the pass->check_eattr_extentry
but in reality, it's only delete_eattr_extentry that needs to do this.
The only other caller is check_eattr_extentry_refs in pass1b, but
that's a special case in which we don't want to delete the eattrs.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/afterpass1_common.c | 32 ++++++++++++++++++++++++++++----
gfs2/fsck/afterpass1_common.h | 4 +++-
gfs2/fsck/metawalk.c | 43 +++++++------------------------------------
gfs2/fsck/metawalk.h | 3 ++-
gfs2/fsck/pass1.c | 39 ++++++++++++++++++++++++++++++++++-----
gfs2/fsck/pass1b.c | 9 ++++++++-
6 files changed, 82 insertions(+), 48 deletions(-)
diff --git a/gfs2/fsck/afterpass1_common.c b/gfs2/fsck/afterpass1_common.c
index 36646f8..bdea242 100644
--- a/gfs2/fsck/afterpass1_common.c
+++ b/gfs2/fsck/afterpass1_common.c
@@ -260,13 +260,37 @@ int delete_eattr_entry(struct gfs2_inode *ip, struct gfs2_buffer_head *leaf_bh,
return 0;
}
-int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
- struct gfs2_buffer_head *leaf_bh,
+int delete_eattr_extentry(struct gfs2_inode *ip, int i, uint64_t *ea_data_ptr,
+ struct gfs2_buffer_head *leaf_bh, uint32_t tot_ealen,
struct gfs2_ea_header *ea_hdr,
struct gfs2_ea_header *ea_hdr_prev, void *private)
{
uint64_t block = be64_to_cpu(*ea_data_ptr);
+ int error;
- return delete_block_if_notdup(ip, block, NULL, _("extended attribute"),
- NULL, private);
+ error = delete_block_if_notdup(ip, block, NULL,
+ _("extended attribute"), NULL, private);
+ if (error) {
+ log_err(_("Bad extended attribute found at block %lld "
+ "(0x%llx)"),
+ (unsigned long long)be64_to_cpu(*ea_data_ptr),
+ (unsigned long long)be64_to_cpu(*ea_data_ptr));
+ if (query( _("Repair the bad Extended Attribute? (y/n) "))) {
+ ea_hdr->ea_num_ptrs = i;
+ ea_hdr->ea_data_len = cpu_to_be32(tot_ealen);
+ *ea_data_ptr = 0;
+ bmodified(leaf_bh);
+ /* Endianness doesn't matter in this case because it's
+ a single byte. */
+ fsck_bitmap_set(ip, ip->i_di.di_eattr,
+ _("extended attribute"),
+ ip->i_sbd->gfs1 ? GFS2_BLKST_DINODE :
+ GFS2_BLKST_USED);
+ log_err( _("The EA was fixed.\n"));
+ } else {
+ error = 1;
+ log_err( _("The bad EA was not fixed.\n"));
+ }
+ }
+ return error;
}
diff --git a/gfs2/fsck/afterpass1_common.h b/gfs2/fsck/afterpass1_common.h
index 2a422c7..8b345ee 100644
--- a/gfs2/fsck/afterpass1_common.h
+++ b/gfs2/fsck/afterpass1_common.h
@@ -19,8 +19,10 @@ extern int delete_eattr_entry(struct gfs2_inode *ip,
struct gfs2_ea_header *ea_hdr,
struct gfs2_ea_header *ea_hdr_prev,
void *private);
-extern int delete_eattr_extentry(struct gfs2_inode *ip, uint64_t *ea_data_ptr,
+extern int delete_eattr_extentry(struct gfs2_inode *ip, int i,
+ uint64_t *ea_data_ptr,
struct gfs2_buffer_head *leaf_bh,
+ uint32_t tot_ealen,
struct gfs2_ea_header *ea_hdr,
struct gfs2_ea_header *ea_hdr_prev,
void *private);
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 529c86d..ac345f2 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -854,7 +854,7 @@ static int check_eattr_entries(struct gfs2_inode *ip,
struct gfs2_ea_header *ea_hdr, *ea_hdr_prev = NULL;
uint64_t *ea_data_ptr = NULL;
int i;
- int error = 0;
+ int error = 0, err;
uint32_t offset = (uint32_t)sizeof(struct gfs2_meta_header);
if (!pass->check_eattr_entry)
@@ -891,41 +891,12 @@ static int check_eattr_entries(struct gfs2_inode *ip,
** reuse........... */
for(i = 0; i < ea_hdr->ea_num_ptrs; i++){
- if (pass->check_eattr_extentry(ip,
- ea_data_ptr,
- bh, ea_hdr,
- ea_hdr_prev,
- pass->private)) {
- log_err(_("Bad extended attribute "
- "found at block %lld "
- "(0x%llx)"),
- (unsigned long long)
- be64_to_cpu(*ea_data_ptr),
- (unsigned long long)
- be64_to_cpu(*ea_data_ptr));
- if (query( _("Repair the bad Extended "
- "Attribute? (y/n) "))) {
- ea_hdr->ea_num_ptrs = i;
- ea_hdr->ea_data_len =
- cpu_to_be32(tot_ealen);
- *ea_data_ptr = 0;
- bmodified(bh);
- /* Endianness doesn't matter
- in this case because it's
- a single byte. */
- fsck_blockmap_set(ip,
- ip->i_di.di_eattr,
- _("extended attribute"),
- sdp->gfs1 ? GFS2_BLKST_DINODE :
- GFS2_BLKST_USED);
- log_err( _("The EA was "
- "fixed.\n"));
- } else {
- error = 1;
- log_err( _("The bad EA was "
- "not fixed.\n"));
- }
- }
+ err = pass->check_eattr_extentry(ip, i,
+ ea_data_ptr, bh, tot_ealen,
+ ea_hdr, ea_hdr_prev,
+ pass->private);
+ if (err)
+ error = err;
tot_ealen += sdp->sd_sb.sb_bsize -
sizeof(struct gfs2_meta_header);
ea_data_ptr++;
diff --git a/gfs2/fsck/metawalk.h b/gfs2/fsck/metawalk.h
index d2f0d97..e47a871 100644
--- a/gfs2/fsck/metawalk.h
+++ b/gfs2/fsck/metawalk.h
@@ -108,9 +108,10 @@ struct metawalk_fxns {
struct gfs2_ea_header *ea_hdr,
struct gfs2_ea_header *ea_hdr_prev,
void *private);
- int (*check_eattr_extentry) (struct gfs2_inode *ip,
+ int (*check_eattr_extentry) (struct gfs2_inode *ip, int i,
uint64_t *ea_data_ptr,
struct gfs2_buffer_head *leaf_bh,
+ uint32_t tot_ealen,
struct gfs2_ea_header *ea_hdr,
struct gfs2_ea_header *ea_hdr_prev,
void *private);
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index 0ae842e..aae5a06 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -59,8 +59,10 @@ static int check_eattr_entries(struct gfs2_inode *ip,
struct gfs2_ea_header *ea_hdr,
struct gfs2_ea_header *ea_hdr_prev,
void *private);
-static int check_extended_leaf_eattr(struct gfs2_inode *ip, uint64_t *data_ptr,
+static int check_extended_leaf_eattr(struct gfs2_inode *ip, int i,
+ uint64_t *data_ptr,
struct gfs2_buffer_head *leaf_bh,
+ uint32_t tot_ealen,
struct gfs2_ea_header *ea_hdr,
struct gfs2_ea_header *ea_hdr_prev,
void *private);
@@ -789,8 +791,10 @@ static int check_ealeaf_block(struct gfs2_inode *ip, uint64_t block, int btype,
*
* Returns: 0 if correct[able], -1 if removal is needed
*/
-static int check_extended_leaf_eattr(struct gfs2_inode *ip, uint64_t *data_ptr,
+static int check_extended_leaf_eattr(struct gfs2_inode *ip, int i,
+ uint64_t *data_ptr,
struct gfs2_buffer_head *leaf_bh,
+ uint32_t tot_ealen,
struct gfs2_ea_header *ea_hdr,
struct gfs2_ea_header *ea_hdr_prev,
void *private)
@@ -798,7 +802,7 @@ static int check_extended_leaf_eattr(struct gfs2_inode *ip, uint64_t *data_ptr,
uint64_t el_blk = be64_to_cpu(*data_ptr);
struct gfs2_sbd *sdp = ip->i_sbd;
struct gfs2_buffer_head *bh = NULL;
- int error;
+ int error = 0;
if (!valid_block(sdp, el_blk)) {
log_err( _("Inode #%llu (0x%llx): Extended Attribute block "
@@ -813,11 +817,36 @@ static int check_extended_leaf_eattr(struct gfs2_inode *ip, uint64_t *data_ptr,
fsck_blockmap_set(ip, ip->i_di.di_eattr,
_("bad (out of range) Extended Attribute "),
GFS2_BLKST_UNLINKED);
- return 1;
+ error = 1;
+ } else {
+ error = check_ealeaf_block(ip, el_blk, GFS2_METATYPE_ED, &bh,
+ private);
}
- error = check_ealeaf_block(ip, el_blk, GFS2_METATYPE_ED, &bh, private);
if (bh)
brelse(bh);
+ if (error) {
+ log_err(_("Bad extended attribute found at block %lld "
+ "(0x%llx)"),
+ (unsigned long long)be64_to_cpu(*data_ptr),
+ (unsigned long long)be64_to_cpu(*data_ptr));
+ if (query( _("Repair the bad Extended Attribute? (y/n) "))) {
+ ea_hdr->ea_num_ptrs = i;
+ ea_hdr->ea_data_len = cpu_to_be32(tot_ealen);
+ *data_ptr = 0;
+ bmodified(leaf_bh);
+ /* Endianness doesn't matter in this case because it's
+ a single byte. */
+ fsck_blockmap_set(ip, ip->i_di.di_eattr,
+ _("extended attribute"),
+ sdp->gfs1 ? GFS2_BLKST_DINODE :
+ GFS2_BLKST_USED);
+ log_err( _("The EA was fixed.\n"));
+ error = 0;
+ } else {
+ error = 1;
+ log_err( _("The bad EA was not fixed.\n"));
+ }
+ }
return error;
}
diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
index a9632d8..182c6cb 100644
--- a/gfs2/fsck/pass1b.c
+++ b/gfs2/fsck/pass1b.c
@@ -831,15 +831,22 @@ static int check_eattr_entry_refs(struct gfs2_inode *ip,
return 0;
}
-static int check_eattr_extentry_refs(struct gfs2_inode *ip,
+static int check_eattr_extentry_refs(struct gfs2_inode *ip, int i,
uint64_t *ea_data_ptr,
struct gfs2_buffer_head *leaf_bh,
+ uint32_t tot_ealen,
struct gfs2_ea_header *ea_hdr,
struct gfs2_ea_header *ea_hdr_prev,
void *private)
{
uint64_t block = be64_to_cpu(*ea_data_ptr);
+ /* This is a case where a bad return code may be sent back, and
+ behavior has changed. Before, if add_duplicate_ref returned a
+ non-zero return code, the caller would delete the eattr from
+ the blockmap. In this case, we should be okay because the only
+ error possible is a malloc that fails, in which case we don't
+ want to delete the eattr anyway. */
return add_duplicate_ref(ip, block, ref_as_ea, 1, INODE_VALID);
}
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 15/40] fsck.gfs2: Move blockmap stuff to pass1.c
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (13 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 14/40] fsck.gfs2: eliminate fsck_blockmap_set from check_eattr_entries Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 16/40] fsck: make pass1 call bitmap reconciliation AKA pass5 Bob Peterson
` (26 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
Now that we've eliminated all blockmap dependencies from the
other passes, we can move the blockmap-related functions from
metawalk to pass1.c. This will ensure there won't be new calls
added.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/metawalk.c | 16 ----------------
gfs2/fsck/metawalk.h | 8 --------
gfs2/fsck/pass1.c | 21 +++++++++++++++++++++
3 files changed, 21 insertions(+), 24 deletions(-)
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index ac345f2..3217711 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -177,22 +177,6 @@ int _fsck_bitmap_set(struct gfs2_inode *ip, uint64_t bblock,
return error;
}
-/*
- * _fsck_blockmap_set - Mark a block in the 4-bit blockmap and the 2-bit
- * bitmap, and adjust free space accordingly.
- */
-int _fsck_blockmap_set(struct gfs2_inode *ip, uint64_t bblock,
- const char *btype, int mark,
- int error_on_dinode, const char *caller, int fline)
-{
- int error = _fsck_bitmap_set(ip, bblock, btype, mark, error_on_dinode,
- caller, fline);
- if (error)
- return error;
-
- return gfs2_blockmap_set(bl, bblock, mark);
-}
-
struct duptree *dupfind(uint64_t block)
{
struct osi_node *node = dup_blocks.osi_node;
diff --git a/gfs2/fsck/metawalk.h b/gfs2/fsck/metawalk.h
index e47a871..813736a 100644
--- a/gfs2/fsck/metawalk.h
+++ b/gfs2/fsck/metawalk.h
@@ -19,9 +19,6 @@ extern int check_linear_dir(struct gfs2_inode *ip, struct gfs2_buffer_head *bh,
extern int check_leaf(struct gfs2_inode *ip, int lindex,
struct metawalk_fxns *pass, uint64_t *leaf_no,
struct gfs2_leaf *leaf, int *ref_count);
-extern int _fsck_blockmap_set(struct gfs2_inode *ip, uint64_t bblock,
- const char *btype, int mark, int error_on_dinode,
- const char *caller, int line);
extern int _fsck_bitmap_set(struct gfs2_inode *ip, uint64_t bblock,
const char *btype, int mark, int error_on_dinode,
const char *caller, int line);
@@ -37,11 +34,6 @@ extern struct gfs2_inode *fsck_system_inode(struct gfs2_sbd *sdp,
_fsck_bitmap_set(ip, b, bt, m, 0, __FUNCTION__, __LINE__)
#define fsck_bitmap_set_noino(ip, b, bt, m) \
_fsck_bitmap_set(ip, b, bt, m, 1, __FUNCTION__, __LINE__)
-#define fsck_blockmap_set(ip, b, bt, m) \
- _fsck_blockmap_set(ip, b, bt, m, 0, __FUNCTION__, __LINE__)
-#define fsck_blkmap_set_noino(ip, b, bt, m) \
- _fsck_blockmap_set(ip, b, bt, m, 1, __FUNCTION__, __LINE__)
-
enum meta_check_rc {
meta_error = -1,
meta_is_good = 0,
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index aae5a06..ca6f25f 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -88,6 +88,27 @@ static int handle_ip(struct gfs2_sbd *sdp, struct gfs2_inode *ip);
static int delete_block(struct gfs2_inode *ip, uint64_t block,
struct gfs2_buffer_head **bh, const char *btype,
void *private);
+/*
+ * _fsck_blockmap_set - Mark a block in the 4-bit blockmap and the 2-bit
+ * bitmap, and adjust free space accordingly.
+ */
+static int _fsck_blockmap_set(struct gfs2_inode *ip, uint64_t bblock,
+ const char *btype, int mark, int error_on_dinode,
+ const char *caller, int fline)
+{
+ int error = _fsck_bitmap_set(ip, bblock, btype, mark, error_on_dinode,
+ caller, fline);
+ if (error)
+ return error;
+
+ return gfs2_blockmap_set(bl, bblock, mark);
+}
+
+#define fsck_blockmap_set(ip, b, bt, m) \
+ _fsck_blockmap_set(ip, b, bt, m, 0, __FUNCTION__, __LINE__)
+#define fsck_blkmap_set_noino(ip, b, bt, m) \
+ _fsck_blockmap_set(ip, b, bt, m, 1, __FUNCTION__, __LINE__)
+
/**
* delete_block - delete a block associated with an inode
*/
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 16/40] fsck: make pass1 call bitmap reconciliation AKA pass5
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (14 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 15/40] fsck.gfs2: Move blockmap stuff to pass1.c Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 17/40] fsck.gfs2: make blockmap global variable only to pass1 Bob Peterson
` (25 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch makes pass1 call what used to be pass5. That enables
pass1 to pass its private blockmap in as a private variable.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/main.c | 34 ----------------------------------
gfs2/fsck/pass1.c | 5 +++++
gfs2/fsck/util.c | 29 +++++++++++++++++++++++++++++
gfs2/fsck/util.h | 6 ++++++
4 files changed, 40 insertions(+), 34 deletions(-)
diff --git a/gfs2/fsck/main.c b/gfs2/fsck/main.c
index 8f2c733..2cedb30 100644
--- a/gfs2/fsck/main.c
+++ b/gfs2/fsck/main.c
@@ -229,14 +229,8 @@ static int check_statfs(struct gfs2_sbd *sdp)
return 0;
}
-struct fsck_pass {
- const char *name;
- int (*f)(struct gfs2_sbd *sdp);
-};
-
static const struct fsck_pass passes[] = {
{ .name = "pass1", .f = pass1 },
- { .name = "reconcile_bitmaps", .f = pass5 },
{ .name = "pass1b", .f = pass1b },
{ .name = "pass2", .f = pass2 },
{ .name = "pass3", .f = pass3 },
@@ -245,34 +239,6 @@ static const struct fsck_pass passes[] = {
{ .name = NULL, }
};
-static void print_pass_duration(const char *name, struct timeval *start)
-{
- char duration[17] = ""; /* strlen("XXdXXhXXmXX.XXXs") + 1 */
- struct timeval end, diff;
- unsigned d, h, m, s;
- char *p = duration;
-
- gettimeofday(&end, NULL);
- timersub(&end, start, &diff);
-
- s = diff.tv_sec % 60;
- diff.tv_sec /= 60;
- m = diff.tv_sec % 60;
- diff.tv_sec /= 60;
- h = diff.tv_sec % 24;
- d = diff.tv_sec / 24;
-
- if (d)
- p += snprintf(p, 4, "%ud", d > 99 ? 99U : d);
- if (h)
- p += snprintf(p, 4, "%uh", h);
- if (m)
- p += snprintf(p, 4, "%um", m);
-
- snprintf(p, 8, "%u.%03lus", s, diff.tv_usec / 1000);
- log_notice(_("%s completed in %s\n"), name, duration);
-}
-
static int fsck_pass(const struct fsck_pass *p, struct gfs2_sbd *sdp)
{
int ret;
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index ca6f25f..876e565 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -1991,6 +1991,7 @@ int pass1(struct gfs2_sbd *sdp)
uint64_t i;
uint64_t rg_count = 0;
int ret;
+ struct timeval timer;
osi_list_init(&gfs1_rindex_blks.list);
@@ -2040,6 +2041,10 @@ int pass1(struct gfs2_sbd *sdp)
return ret;
}
+ log_notice(_("Reconciling bitmaps.\n"));
+ gettimeofday(&timer, NULL);
+ pass5(sdp);
+ print_pass_duration("reconcile_bitmaps", &timer);
gfs2_special_free(&gfs1_rindex_blks);
return FSCK_OK;
}
diff --git a/gfs2/fsck/util.c b/gfs2/fsck/util.c
index 7ca7b91..d6d664a 100644
--- a/gfs2/fsck/util.c
+++ b/gfs2/fsck/util.c
@@ -707,3 +707,32 @@ void delete_all_dups(struct gfs2_inode *ip)
}
}
}
+
+void print_pass_duration(const char *name, struct timeval *start)
+{
+ char duration[17] = ""; /* strlen("XXdXXhXXmXX.XXXs") + 1 */
+ struct timeval end, diff;
+ unsigned d, h, m, s;
+ char *p = duration;
+
+ gettimeofday(&end, NULL);
+ timersub(&end, start, &diff);
+
+ s = diff.tv_sec % 60;
+ diff.tv_sec /= 60;
+ m = diff.tv_sec % 60;
+ diff.tv_sec /= 60;
+ h = diff.tv_sec % 24;
+ d = diff.tv_sec / 24;
+
+ if (d)
+ p += snprintf(p, 4, "%ud", d > 99 ? 99U : d);
+ if (h)
+ p += snprintf(p, 4, "%uh", h);
+ if (m)
+ p += snprintf(p, 4, "%um", m);
+
+ snprintf(p, 8, "%u.%03lus", s, diff.tv_usec / 1000);
+ log_notice(_("%s completed in %s\n"), name, duration);
+}
+
diff --git a/gfs2/fsck/util.h b/gfs2/fsck/util.h
index 7753daf..50a3c8d 100644
--- a/gfs2/fsck/util.h
+++ b/gfs2/fsck/util.h
@@ -27,6 +27,11 @@ extern const char *reftypes[ref_types + 1];
#define BLOCKMAP_BYTE_OFFSET2(x) ((x & 0x0000000000000003) << 1)
#define BLOCKMAP_MASK2 (0x3)
+struct fsck_pass {
+ const char *name;
+ int (*f)(struct gfs2_sbd *sdp);
+};
+
static inline int block_type(uint64_t bblock)
{
static unsigned char *byte;
@@ -106,6 +111,7 @@ extern char gfs2_getch(void);
extern uint64_t find_free_blk(struct gfs2_sbd *sdp);
extern uint64_t *get_dir_hash(struct gfs2_inode *ip);
extern void delete_all_dups(struct gfs2_inode *ip);
+extern void print_pass_duration(const char *name, struct timeval *start);
#define stack log_debug("<backtrace> - %s()\n", __func__)
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 17/40] fsck.gfs2: make blockmap global variable only to pass1
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (15 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 16/40] fsck: make pass1 call bitmap reconciliation AKA pass5 Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 18/40] fsck.gfs2: Add wrapper function pass1_check_metatree Bob Peterson
` (24 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch changes the variable "bl" from being a global variable
to being global only to pass1. This is another step toward allowing
pass1 to free the blockmap when it's done.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/fsck.h | 3 +-
gfs2/fsck/initialize.c | 11 -----
gfs2/fsck/main.c | 1 -
gfs2/fsck/pass1.c | 120 +++++++++++++++++++++++++++++++++++++++++--------
gfs2/fsck/pass5.c | 17 +++----
gfs2/fsck/util.c | 66 ---------------------------
gfs2/fsck/util.h | 6 +--
7 files changed, 112 insertions(+), 112 deletions(-)
diff --git a/gfs2/fsck/fsck.h b/gfs2/fsck/fsck.h
index 4cf4ce1..d57ccfd 100644
--- a/gfs2/fsck/fsck.h
+++ b/gfs2/fsck/fsck.h
@@ -119,7 +119,7 @@ extern int pass1c(struct gfs2_sbd *sdp);
extern int pass2(struct gfs2_sbd *sdp);
extern int pass3(struct gfs2_sbd *sdp);
extern int pass4(struct gfs2_sbd *sdp);
-extern int pass5(struct gfs2_sbd *sdp);
+extern int pass5(struct gfs2_sbd *sdp, struct gfs2_bmap *bl);
extern int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count,
int *sane);
extern int fsck_query(const char *format, ...)
@@ -142,7 +142,6 @@ struct gfs2_options {
extern struct gfs2_options opts;
extern struct gfs2_inode *lf_dip; /* Lost and found directory inode */
extern int lf_was_created;
-extern struct gfs2_bmap *bl;
extern uint64_t last_fs_block, last_reported_block;
extern int64_t last_reported_fblock;
extern int skip_this_pass, fsck_abort;
diff --git a/gfs2/fsck/initialize.c b/gfs2/fsck/initialize.c
index 84d39cd..d54ab0d 100644
--- a/gfs2/fsck/initialize.c
+++ b/gfs2/fsck/initialize.c
@@ -115,8 +115,6 @@ static void empty_super_block(struct gfs2_sbd *sdp)
log_info( _("Freeing buffers.\n"));
gfs2_rgrp_free(&sdp->rgtree);
- if (bl)
- gfs2_bmap_destroy(sdp, bl);
gfs2_inodetree_free();
gfs2_dirtree_free();
gfs2_dup_free();
@@ -674,7 +672,6 @@ static int init_system_inodes(struct gfs2_sbd *sdp)
uint64_t inumbuf = 0;
char *buf;
struct gfs2_statfs_change sc;
- uint64_t addl_mem_needed;
int err;
/*******************************************************************
@@ -841,14 +838,6 @@ static int init_system_inodes(struct gfs2_sbd *sdp)
goto fail;
}
- bl = gfs2_bmap_create(sdp, last_fs_block+1, &addl_mem_needed);
- if (!bl) {
- log_crit( _("This system doesn't have enough memory and swap space to fsck this file system.\n"));
- log_crit( _("Additional memory needed is approximately: %lluMB\n"),
- (unsigned long long)(addl_mem_needed / 1048576ULL));
- log_crit( _("Please increase your swap space by that amount and run gfs2_fsck again.\n"));
- goto fail;
- }
return 0;
fail:
empty_super_block(sdp);
diff --git a/gfs2/fsck/main.c b/gfs2/fsck/main.c
index 2cedb30..91ebe28 100644
--- a/gfs2/fsck/main.c
+++ b/gfs2/fsck/main.c
@@ -27,7 +27,6 @@
struct gfs2_options opts = {0};
struct gfs2_inode *lf_dip = NULL; /* Lost and found directory inode */
int lf_was_created = 0;
-struct gfs2_bmap *bl = NULL;
uint64_t last_fs_block, last_reported_block = -1;
int64_t last_reported_fblock = -1000000;
int skip_this_pass = FALSE, fsck_abort = FALSE;
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index 876e565..49f6c6e 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -30,6 +30,7 @@
#include "fs_recovery.h"
struct special_blocks gfs1_rindex_blks;
+struct gfs2_bmap *bl = NULL;
struct block_count {
uint64_t indir_count;
@@ -88,6 +89,24 @@ static int handle_ip(struct gfs2_sbd *sdp, struct gfs2_inode *ip);
static int delete_block(struct gfs2_inode *ip, uint64_t block,
struct gfs2_buffer_head **bh, const char *btype,
void *private);
+
+static int gfs2_blockmap_set(struct gfs2_bmap *bmap, uint64_t bblock, int mark)
+{
+ static unsigned char *byte;
+ static uint64_t b;
+
+ if (!bmap)
+ return 0;
+ if (bblock > bmap->size)
+ return -1;
+
+ byte = bmap->map + BLOCKMAP_SIZE2(bblock);
+ b = BLOCKMAP_BYTE_OFFSET2(bblock);
+ *byte &= ~(BLOCKMAP_MASK2 << b);
+ *byte |= (mark & BLOCKMAP_MASK2) << b;
+ return 0;
+}
+
/*
* _fsck_blockmap_set - Mark a block in the 4-bit blockmap and the 2-bit
* bitmap, and adjust free space accordingly.
@@ -253,7 +272,7 @@ static int p1check_leaf(struct gfs2_inode *ip, uint64_t block, void *private)
check in metawalk: gfs2_check_meta(lbh, GFS2_METATYPE_LF).
So we know it's a leaf block. */
bc->indir_count++;
- q = block_type(block);
+ q = block_type(bl, block);
if (q != GFS2_BLKST_FREE) {
log_err( _("Found duplicate block #%llu (0x%llx) referenced "
"as a directory leaf in dinode "
@@ -311,7 +330,7 @@ static int check_metalist(struct gfs2_inode *ip, uint64_t block,
iblk_type = GFS2_METATYPE_IN;
blktypedesc = _("a journaled data block");
}
- q = block_type(block);
+ q = block_type(bl, block);
if (q != GFS2_BLKST_FREE) {
log_err( _("Found duplicate block #%llu (0x%llx) referenced "
"as metadata in indirect block for dinode "
@@ -502,7 +521,7 @@ static int check_data(struct gfs2_inode *ip, uint64_t metablock,
return -1;
}
bc->data_count++; /* keep the count sane anyway */
- q = block_type(block);
+ q = block_type(bl, block);
if (q != GFS2_BLKST_FREE) {
struct gfs2_buffer_head *bh;
struct gfs2_meta_header mh;
@@ -623,7 +642,7 @@ static int undo_eattr_indir_or_leaf(struct gfs2_inode *ip, uint64_t block,
/* Need to check block_type before undoing the reference, which can
set it to free, which would cause the test below to fail. */
- q = block_type(block);
+ q = block_type(bl, block);
error = undo_reference(ip, block, 0, private);
if (error)
@@ -672,7 +691,7 @@ static int check_eattr_indir(struct gfs2_inode *ip, uint64_t indirect,
* in pass1c */
return 1;
}
- q = block_type(indirect);
+ q = block_type(bl, indirect);
/* Special duplicate processing: If we have an EA block,
check if it really is an EA. If it is, let duplicate
@@ -756,7 +775,7 @@ static int check_ealeaf_block(struct gfs2_inode *ip, uint64_t block, int btype,
int q;
struct block_count *bc = (struct block_count *) private;
- q = block_type(block);
+ q = block_type(bl, block);
/* Special duplicate processing: If we have an EA block, check if it
really is an EA. If it is, let duplicate handling sort it out.
If it isn't, clear it but don't count it as a duplicate. */
@@ -1027,7 +1046,7 @@ static int mark_block_invalid(struct gfs2_inode *ip, uint64_t block,
return meta_is_good;
}
- q = block_type(block);
+ q = block_type(bl, block);
if (q != GFS2_BLKST_FREE) {
if (was_duplicate)
*was_duplicate = 1;
@@ -1126,7 +1145,7 @@ static int rangecheck_block(struct gfs2_inode *ip, uint64_t block,
return meta_error; /* Exits check_metatree quicker */
}
/* See how many duplicate blocks it has */
- q = block_type(block);
+ q = block_type(bl, block);
if (q != GFS2_BLKST_FREE) {
(*bad_pointers)++;
log_info( _("Duplicated %s block pointer (violation %ld, block"
@@ -1595,7 +1614,7 @@ static int check_system_inode(struct gfs2_sbd *sdp,
}
}
if (*sysinode) {
- ds.q = block_type(iblock);
+ ds.q = block_type(bl, iblock);
/* If the inode exists but the block is marked free, we might
be recovering from a corrupt bitmap. In that case, don't
rebuild the inode. Just reuse the inode and fix the
@@ -1858,7 +1877,7 @@ static int pass1_process_bitmap(struct gfs2_sbd *sdp, struct rgrp_tree *rgd, uin
check_magic = ((struct gfs2_meta_header *)
(bh->b_data))->mh_magic;
- q = block_type(block);
+ q = block_type(bl, block);
if (q != GFS2_BLKST_FREE) {
if (be32_to_cpu(check_magic) == GFS2_MAGIC &&
sdp->gfs1 && !is_inode) {
@@ -1971,6 +1990,55 @@ out:
return ret;
}
+static int gfs2_blockmap_create(struct gfs2_bmap *bmap, uint64_t size)
+{
+ bmap->size = size;
+
+ /* Have to add 1 to BLOCKMAP_SIZE since it's 0-based and mallocs
+ * must be 1-based */
+ bmap->mapsize = BLOCKMAP_SIZE2(size) + 1;
+
+ if (!(bmap->map = calloc(bmap->mapsize, sizeof(char))))
+ return -ENOMEM;
+ return 0;
+}
+
+static struct gfs2_bmap *gfs2_bmap_create(struct gfs2_sbd *sdp, uint64_t size,
+ uint64_t *addl_mem_needed)
+{
+ struct gfs2_bmap *il;
+
+ *addl_mem_needed = 0L;
+ il = calloc(1, sizeof(*il));
+ if (!il)
+ return NULL;
+
+ if (gfs2_blockmap_create(il, size)) {
+ *addl_mem_needed = il->mapsize;
+ free(il);
+ il = NULL;
+ }
+ return il;
+}
+
+static void gfs2_blockmap_destroy(struct gfs2_bmap *bmap)
+{
+ if (bmap->map)
+ free(bmap->map);
+ bmap->size = 0;
+ bmap->mapsize = 0;
+}
+
+static void *gfs2_bmap_destroy(struct gfs2_sbd *sdp, struct gfs2_bmap *il)
+{
+ if (il) {
+ gfs2_blockmap_destroy(il);
+ free(il);
+ il = NULL;
+ }
+ return il;
+}
+
/**
* pass1 - walk through inodes and check inode state
*
@@ -1990,9 +2058,18 @@ int pass1(struct gfs2_sbd *sdp)
struct rgrp_tree *rgd;
uint64_t i;
uint64_t rg_count = 0;
- int ret;
struct timeval timer;
-
+ int ret = FSCK_OK;
+ uint64_t addl_mem_needed;
+
+ bl = gfs2_bmap_create(sdp, last_fs_block+1, &addl_mem_needed);
+ if (!bl) {
+ log_crit( _("This system doesn't have enough memory and swap space to fsck this file system.\n"));
+ log_crit( _("Additional memory needed is approximately: %lluMB\n"),
+ (unsigned long long)(addl_mem_needed / 1048576ULL));
+ log_crit( _("Please increase your swap space by that amount and run gfs2_fsck again.\n"));
+ return FSCK_ERROR;
+ }
osi_list_init(&gfs1_rindex_blks.list);
/* FIXME: In the gfs fsck, we had to mark things like the
@@ -2014,8 +2091,10 @@ int pass1(struct gfs2_sbd *sdp)
* things - we can change the method later if necessary.
*/
for (n = osi_first(&sdp->rgtree); n; n = next, rg_count++) {
- if (fsck_abort)
- return FSCK_CANCELED;
+ if (fsck_abort) {
+ ret = FSCK_CANCELED;
+ goto out;
+ }
next = osi_next(n);
log_debug( _("Checking metadata in Resource Group #%llu\n"),
(unsigned long long)rg_count);
@@ -2028,7 +2107,8 @@ int pass1(struct gfs2_sbd *sdp)
GFS2_BLKST_USED)) {
stack;
gfs2_special_free(&gfs1_rindex_blks);
- return FSCK_ERROR;
+ ret = FSCK_ERROR;
+ goto out;
}
/* rgrps and bitmaps don't have bits to represent
their blocks, so don't do this:
@@ -2038,13 +2118,15 @@ int pass1(struct gfs2_sbd *sdp)
ret = pass1_process_rgrp(sdp, rgd);
if (ret)
- return ret;
-
+ goto out;
}
log_notice(_("Reconciling bitmaps.\n"));
gettimeofday(&timer, NULL);
- pass5(sdp);
+ pass5(sdp, bl);
print_pass_duration("reconcile_bitmaps", &timer);
+out:
gfs2_special_free(&gfs1_rindex_blks);
- return FSCK_OK;
+ if (bl)
+ gfs2_bmap_destroy(sdp, bl);
+ return ret;
}
diff --git a/gfs2/fsck/pass5.c b/gfs2/fsck/pass5.c
index 99ff0a1..7d81bfa 100644
--- a/gfs2/fsck/pass5.c
+++ b/gfs2/fsck/pass5.c
@@ -12,9 +12,10 @@
#include "fsck.h"
#include "util.h"
-static int check_block_status(struct gfs2_sbd *sdp, char *buffer,
- unsigned int buflen, uint64_t *rg_block,
- uint64_t rg_data, uint32_t *count)
+static int check_block_status(struct gfs2_sbd *sdp, struct gfs2_bmap *bl,
+ char *buffer, unsigned int buflen,
+ uint64_t *rg_block, uint64_t rg_data,
+ uint32_t *count)
{
unsigned char *byte, *end;
unsigned int bit;
@@ -34,7 +35,7 @@ static int check_block_status(struct gfs2_sbd *sdp, char *buffer,
if (skip_this_pass || fsck_abort) /* if asked to skip the rest */
return 0;
- q = block_type(block);
+ q = block_type(bl, block);
/* GFS1 file systems will have to suffer from slower fsck run
* times because in GFS, there's no 1:1 relationship between
* bits and counts. If a bit is marked "dinode" in GFS1, it
@@ -124,7 +125,7 @@ static int check_block_status(struct gfs2_sbd *sdp, char *buffer,
}
static void update_rgrp(struct gfs2_sbd *sdp, struct rgrp_tree *rgp,
- uint32_t *count)
+ struct gfs2_bmap *bl, uint32_t *count)
{
uint32_t i;
struct gfs2_bitmap *bits;
@@ -136,7 +137,7 @@ static void update_rgrp(struct gfs2_sbd *sdp, struct rgrp_tree *rgp,
bits = &rgp->bits[i];
/* update the bitmaps */
- if (check_block_status(sdp, bits->bi_bh->b_data + bits->bi_offset,
+ if (check_block_status(sdp, bl, bits->bi_bh->b_data + bits->bi_offset,
bits->bi_len, &rg_block, rgp->ri.ri_data0, count))
return;
if (skip_this_pass || fsck_abort) /* if asked to skip the rest */
@@ -209,7 +210,7 @@ static void update_rgrp(struct gfs2_sbd *sdp, struct rgrp_tree *rgp,
* fix free block maps
* fix used inode maps
*/
-int pass5(struct gfs2_sbd *sdp)
+int pass5(struct gfs2_sbd *sdp, struct gfs2_bmap *bl)
{
struct osi_node *n, *next = NULL;
struct rgrp_tree *rgp = NULL;
@@ -227,7 +228,7 @@ int pass5(struct gfs2_sbd *sdp)
rg_count++;
/* Compare the bitmaps and report the differences */
- update_rgrp(sdp, rgp, count);
+ update_rgrp(sdp, rgp, bl, count);
}
/* Fix up superblock info based on this - don't think there's
* anything to do here... */
diff --git a/gfs2/fsck/util.c b/gfs2/fsck/util.c
index d6d664a..2e77000 100644
--- a/gfs2/fsck/util.c
+++ b/gfs2/fsck/util.c
@@ -525,72 +525,6 @@ void dirtree_delete(struct dir_info *b)
free(b);
}
-static int gfs2_blockmap_create(struct gfs2_bmap *bmap, uint64_t size)
-{
- bmap->size = size;
-
- /* Have to add 1 to BLOCKMAP_SIZE since it's 0-based and mallocs
- * must be 1-based */
- bmap->mapsize = BLOCKMAP_SIZE2(size) + 1;
-
- if (!(bmap->map = calloc(bmap->mapsize, sizeof(char))))
- return -ENOMEM;
- return 0;
-}
-
-static void gfs2_blockmap_destroy(struct gfs2_bmap *bmap)
-{
- if (bmap->map)
- free(bmap->map);
- bmap->size = 0;
- bmap->mapsize = 0;
-}
-
-struct gfs2_bmap *gfs2_bmap_create(struct gfs2_sbd *sdp, uint64_t size,
- uint64_t *addl_mem_needed)
-{
- struct gfs2_bmap *il;
-
- *addl_mem_needed = 0L;
- il = calloc(1, sizeof(*il));
- if (!il)
- return NULL;
-
- if (gfs2_blockmap_create(il, size)) {
- *addl_mem_needed = il->mapsize;
- free(il);
- il = NULL;
- }
- return il;
-}
-
-int gfs2_blockmap_set(struct gfs2_bmap *bmap, uint64_t bblock, int mark)
-{
- static unsigned char *byte;
- static uint64_t b;
-
- if (!bmap)
- return 0;
- if (bblock > bmap->size)
- return -1;
-
- byte = bmap->map + BLOCKMAP_SIZE2(bblock);
- b = BLOCKMAP_BYTE_OFFSET2(bblock);
- *byte &= ~(BLOCKMAP_MASK2 << b);
- *byte |= (mark & BLOCKMAP_MASK2) << b;
- return 0;
-}
-
-void *gfs2_bmap_destroy(struct gfs2_sbd *sdp, struct gfs2_bmap *il)
-{
- if (il) {
- gfs2_blockmap_destroy(il);
- free(il);
- il = NULL;
- }
- return il;
-}
-
uint64_t find_free_blk(struct gfs2_sbd *sdp)
{
struct osi_node *n, *next = NULL;
diff --git a/gfs2/fsck/util.h b/gfs2/fsck/util.h
index 50a3c8d..aa01552 100644
--- a/gfs2/fsck/util.h
+++ b/gfs2/fsck/util.h
@@ -32,7 +32,7 @@ struct fsck_pass {
int (*f)(struct gfs2_sbd *sdp);
};
-static inline int block_type(uint64_t bblock)
+static inline int block_type(struct gfs2_bmap *bl, uint64_t bblock)
{
static unsigned char *byte;
static uint64_t b;
@@ -100,10 +100,6 @@ static inline uint32_t gfs_to_gfs2_mode(struct gfs2_inode *ip)
}
extern enum dup_ref_type get_ref_type(struct inode_with_dups *id);
-extern struct gfs2_bmap *gfs2_bmap_create(struct gfs2_sbd *sdp, uint64_t size,
- uint64_t *addl_mem_needed);
-extern void *gfs2_bmap_destroy(struct gfs2_sbd *sdp, struct gfs2_bmap *il);
-extern int gfs2_blockmap_set(struct gfs2_bmap *il, uint64_t block, int mark);
extern char generic_interrupt(const char *caller, const char *where,
const char *progress, const char *question,
const char *answers);
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 18/40] fsck.gfs2: Add wrapper function pass1_check_metatree
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (16 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 17/40] fsck.gfs2: make blockmap global variable only to pass1 Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 19/40] fsck.gfs2: pass counted_links into fix_link_count in pass4 Bob Peterson
` (23 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch adds a wrapper function pass1_check_metatree, which
simply calls the normal check_metatree. Since the regular function
now frees inodes from the bitmap but not the blockmap, pass1 needs
the wrapper function to keep its blockmap in sync with the bitmap.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass1.c | 35 +++++++++++++++++++++++++----------
1 file changed, 25 insertions(+), 10 deletions(-)
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index 49f6c6e..f4eea49 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -1346,6 +1346,26 @@ struct metawalk_fxns alloc_fxns = {
};
/*
+ * pass1_check_metatree - wrapper function for check_metatree
+ *
+ * Generic function check_metatree sets the bitmap values, but not the
+ * corresponding values in the blockmap. If we get an error, the inode will
+ * have been freed in the bitmap. We need to set the inode address as free
+ * as well.
+ */
+static int pass1_check_metatree(struct gfs2_inode *ip,
+ struct metawalk_fxns *pass)
+{
+ int error;
+
+ error = check_metatree(ip, pass);
+ if (error)
+ gfs2_blockmap_set(bl, ip->i_di.di_num.no_addr,
+ GFS2_BLKST_FREE);
+ return error;
+}
+
+/*
* reprocess_inode - fixes the blockmap to match the bitmap due to an
* unexpected block allocation via libgfs2.
*
@@ -1372,15 +1392,10 @@ static void reprocess_inode(struct gfs2_inode *ip, const char *desc)
(unsigned long long)ip->i_di.di_num.no_addr,
(unsigned long long)ip->i_di.di_num.no_addr,
ip->i_di.di_height);
- error = check_metatree(ip, &alloc_fxns);
- if (error) {
- /* check_metatree will have fixed the bitmap, but not the
- blockmap. */
- gfs2_blockmap_set(bl, ip->i_di.di_num.no_addr,
- GFS2_BLKST_FREE);
+ error = pass1_check_metatree(ip, &alloc_fxns);
+ if (error)
log_err( _("Error %d reprocessing the %s metadata tree.\n"),
error, desc);
- }
}
/*
@@ -1400,7 +1415,7 @@ static int handle_ip(struct gfs2_sbd *sdp, struct gfs2_inode *ip)
so it's better to check it up front and delete the inode if
there is corruption. */
rangecheck_fxns.private = &bad_pointers;
- error = check_metatree(ip, &rangecheck_fxns);
+ error = pass1_check_metatree(ip, &rangecheck_fxns);
if (bad_pointers > BAD_POINTER_TOLERANCE) {
log_err( _("Error: inode %llu (0x%llx) has more than "
"%d bad pointers.\n"),
@@ -1433,7 +1448,7 @@ static int handle_ip(struct gfs2_sbd *sdp, struct gfs2_inode *ip)
lf_blks = lf_dip->i_di.di_blocks;
pass1_fxns.private = &bc;
- error = check_metatree(ip, &pass1_fxns);
+ error = pass1_check_metatree(ip, &pass1_fxns);
/* Pass1 may have added some blocks to lost+found by virtue of leafs
that were misplaced. If it did, we need to reprocess lost+found
@@ -1688,7 +1703,7 @@ static int check_system_inode(struct gfs2_sbd *sdp,
sysdir_fxns.private = &bc;
if ((*sysinode)->i_di.di_flags & GFS2_DIF_EXHASH)
- check_metatree(*sysinode, &sysdir_fxns);
+ pass1_check_metatree(*sysinode, &sysdir_fxns);
else {
err = check_linear_dir(*sysinode, (*sysinode)->i_bh,
&sysdir_fxns);
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 19/40] fsck.gfs2: pass counted_links into fix_link_count in pass4
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (17 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 18/40] fsck.gfs2: Add wrapper function pass1_check_metatree Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 20/40] fsck.gfs2: refactor pass4 function scan_inode_list Bob Peterson
` (22 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch changes function fix_link_count so that it accepts the
counted_links rather than a pointer to the inode info structure.
That makes it more versatile for future expansion.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass4.c | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)
diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
index 0d6cc9d..14cce55 100644
--- a/gfs2/fsck/pass4.c
+++ b/gfs2/fsck/pass4.c
@@ -25,21 +25,20 @@ struct metawalk_fxns pass4_fxns_delete = {
/* Updates the link count of an inode to what the fsck has seen for
* link count */
-static int fix_link_count(struct inode_info *ii, struct gfs2_inode *ip)
+static int fix_link_count(uint32_t counted_links, struct gfs2_inode *ip)
{
log_info( _("Fixing inode link count (%d->%d) for %llu (0x%llx) \n"),
- ip->i_di.di_nlink, ii->counted_links,
+ ip->i_di.di_nlink, counted_links,
(unsigned long long)ip->i_di.di_num.no_addr,
(unsigned long long)ip->i_di.di_num.no_addr);
- if (ip->i_di.di_nlink == ii->counted_links)
+ if (ip->i_di.di_nlink == counted_links)
return 0;
- ip->i_di.di_nlink = ii->counted_links;
+ ip->i_di.di_nlink = counted_links;
bmodified(ip->i_bh);
log_debug( _("Changing inode %llu (0x%llx) to have %u links\n"),
(unsigned long long)ip->i_di.di_num.no_addr,
- (unsigned long long)ip->i_di.di_num.no_addr,
- ii->counted_links);
+ (unsigned long long)ip->i_di.di_num.no_addr, counted_links);
return 0;
}
@@ -135,7 +134,7 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
fsck_inode_put(&ip);
return -1;
} else {
- fix_link_count(ii, ip);
+ fix_link_count(ii->counted_links, ip);
lf_addition = 1;
}
} else
@@ -155,7 +154,7 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
(unsigned long long)ii->di_num.no_addr,
(unsigned long long)ii->di_num.no_addr)) {
ip = fsck_load_inode(sdp, ii->di_num.no_addr); /* bread, inode_get */
- fix_link_count(ii, ip);
+ fix_link_count(ii->counted_links, ip);
ii->di_nlink = ii->counted_links;
fsck_inode_put(&ip); /* out, brelse, free */
log_warn( _("Link count updated to %d for "
@@ -183,7 +182,7 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
log_crit( _("Unable to find lost+found inode in inode_hash!!\n"));
return -1;
} else {
- fix_link_count(ii, lf_dip);
+ fix_link_count(ii->counted_links, lf_dip);
}
}
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 20/40] fsck.gfs2: refactor pass4 function scan_inode_list
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (18 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 19/40] fsck.gfs2: pass counted_links into fix_link_count in pass4 Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 21/40] fsck.gfs2: More refactoring of " Bob Peterson
` (21 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch makes no functional changes. It just cuts a large swath
of function scan_inode_list and puts it in a separate function,
handle_unlinked. This makes it more versatile for later in order
to save memory.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass4.c | 161 ++++++++++++++++++++++++++++--------------------------
1 file changed, 85 insertions(+), 76 deletions(-)
diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
index 14cce55..8c11761 100644
--- a/gfs2/fsck/pass4.c
+++ b/gfs2/fsck/pass4.c
@@ -42,12 +42,94 @@ static int fix_link_count(uint32_t counted_links, struct gfs2_inode *ip)
return 0;
}
+/**
+ * handle_unlinked - handle an unlinked dinode
+ *
+ * Note: We need to pass in *counted_links here, not counted_links because
+ * add_inode_to_lf may be called here, and that might change the original
+ * value, whether that's in the dirtree or the inodetree.
+ *
+ * Returns: 1 if caller should do "continue", 0 if not.
+ */
+static int handle_unlinked(struct gfs2_sbd *sdp, uint64_t no_addr,
+ uint32_t *counted_links, int *lf_addition)
+{
+ struct gfs2_inode *ip;
+ int q;
+
+ log_err( _("Found unlinked inode at %llu (0x%llx)\n"),
+ (unsigned long long)no_addr, (unsigned long long)no_addr);
+ q = bitmap_type(sdp, no_addr);
+ if (q == GFS2_BLKST_UNLINKED) {
+ log_err( _("Unlinked inode %llu (0x%llx) contains bad "
+ "blocks\n"), (unsigned long long)no_addr,
+ (unsigned long long)no_addr);
+ if (query(_("Delete unlinked inode with bad blocks? "
+ "(y/n) "))) {
+ ip = fsck_load_inode(sdp, no_addr);
+ check_inode_eattr(ip, &pass4_fxns_delete);
+ check_metatree(ip, &pass4_fxns_delete);
+ fsck_bitmap_set(ip, no_addr, _("bad unlinked"),
+ GFS2_BLKST_FREE);
+ fsck_inode_put(&ip);
+ return 1;
+ } else {
+ log_err( _("Unlinked inode with bad blocks not "
+ "cleared\n"));
+ }
+ }
+ if (q != GFS2_BLKST_DINODE) {
+ log_err( _("Unlinked block %lld (0x%llx) marked as inode is "
+ "not an inode (%d)\n"),
+ (unsigned long long)no_addr,
+ (unsigned long long)no_addr, q);
+ ip = fsck_load_inode(sdp, no_addr);
+ if (query(_("Delete unlinked inode? (y/n) "))) {
+ check_inode_eattr(ip, &pass4_fxns_delete);
+ check_metatree(ip, &pass4_fxns_delete);
+ fsck_bitmap_set(ip, no_addr, _("invalid unlinked"),
+ GFS2_BLKST_FREE);
+ fsck_inode_put(&ip);
+ log_err( _("The inode was deleted\n"));
+ } else {
+ log_err( _("The inode was not deleted\n"));
+ fsck_inode_put(&ip);
+ }
+ return 1;
+ }
+ ip = fsck_load_inode(sdp, no_addr);
+
+ /* We don't want to clear zero-size files with eattrs - there might be
+ relevent info in them. */
+ if (!ip->i_di.di_size && !ip->i_di.di_eattr){
+ log_err( _("Unlinked inode has zero size\n"));
+ if (query(_("Clear zero-size unlinked inode? (y/n) "))) {
+ fsck_bitmap_set(ip, no_addr, _("unlinked zero-length"),
+ GFS2_BLKST_FREE);
+ fsck_inode_put(&ip);
+ return 1;
+ }
+ }
+ if (query( _("Add unlinked inode to lost+found? (y/n)"))) {
+ if (add_inode_to_lf(ip)) {
+ stack;
+ fsck_inode_put(&ip);
+ return -1;
+ } else {
+ fix_link_count(*counted_links, ip);
+ *lf_addition = 1;
+ }
+ } else
+ log_err( _("Unlinked inode left unlinked\n"));
+ fsck_inode_put(&ip);
+ return 0;
+}
+
static int scan_inode_list(struct gfs2_sbd *sdp) {
struct osi_node *tmp, *next = NULL;
struct inode_info *ii;
struct gfs2_inode *ip;
int lf_addition = 0;
- int q;
/* FIXME: should probably factor this out into a generic
* scanning fxn */
@@ -64,82 +146,9 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
(ii->di_num.no_addr == sdp->md.statfs->i_di.di_num.no_addr)))
continue;
if (ii->counted_links == 0) {
- log_err( _("Found unlinked inode at %llu (0x%llx)\n"),
- (unsigned long long)ii->di_num.no_addr,
- (unsigned long long)ii->di_num.no_addr);
- q = bitmap_type(sdp, ii->di_num.no_addr);
- if (q == GFS2_BLKST_UNLINKED) {
- log_err( _("Unlinked inode %llu (0x%llx) contains "
- "bad blocks\n"),
- (unsigned long long)ii->di_num.no_addr,
- (unsigned long long)ii->di_num.no_addr);
- if (query( _("Delete unlinked inode with bad "
- "blocks? (y/n) "))) {
- ip = fsck_load_inode(sdp, ii->di_num.no_addr);
- check_inode_eattr(ip,
- &pass4_fxns_delete);
- check_metatree(ip, &pass4_fxns_delete);
- fsck_bitmap_set(ip, ii->di_num.no_addr,
- _("bad unlinked"),
- GFS2_BLKST_FREE);
- fsck_inode_put(&ip);
- continue;
- } else
- log_err( _("Unlinked inode with bad blocks not cleared\n"));
- }
- if (q != GFS2_BLKST_DINODE) {
- log_err( _("Unlinked block %lld (0x%llx) "
- "marked as inode is "
- "not an inode (%d)\n"),
- (unsigned long long)ii->di_num.no_addr,
- (unsigned long long)ii->di_num.no_addr, q);
- ip = fsck_load_inode(sdp, ii->di_num.no_addr);
- if (query(_("Delete unlinked inode? (y/n) "))) {
- check_inode_eattr(ip,
- &pass4_fxns_delete);
- check_metatree(ip, &pass4_fxns_delete);
- fsck_bitmap_set(ip, ii->di_num.no_addr,
- _("invalid unlinked"),
- GFS2_BLKST_FREE);
- fsck_inode_put(&ip);
- log_err( _("The inode was deleted\n"));
- } else {
- log_err( _("The inode was not "
- "deleted\n"));
- fsck_inode_put(&ip);
- }
+ if (handle_unlinked(sdp, ii->di_num.no_addr,
+ &ii->counted_links, &lf_addition))
continue;
- }
- ip = fsck_load_inode(sdp, ii->di_num.no_addr);
-
- /* We don't want to clear zero-size files with
- * eattrs - there might be relevent info in
- * them. */
- if (!ip->i_di.di_size && !ip->i_di.di_eattr){
- log_err( _("Unlinked inode has zero size\n"));
- if (query(_("Clear zero-size unlinked inode? "
- "(y/n) "))) {
- fsck_bitmap_set(ip, ii->di_num.no_addr,
- _("unlinked zero-length"),
- GFS2_BLKST_FREE);
- fsck_inode_put(&ip);
- continue;
- }
-
- }
- if (query( _("Add unlinked inode to lost+found? "
- "(y/n)"))) {
- if (add_inode_to_lf(ip)) {
- stack;
- fsck_inode_put(&ip);
- return -1;
- } else {
- fix_link_count(ii->counted_links, ip);
- lf_addition = 1;
- }
- } else
- log_err( _("Unlinked inode left unlinked\n"));
- fsck_inode_put(&ip);
} /* if (ii->counted_links == 0) */
else if (ii->di_nlink != ii->counted_links) {
log_err( _("Link count inconsistent for inode %llu"
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 21/40] fsck.gfs2: More refactoring of pass4 function scan_inode_list
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (19 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 20/40] fsck.gfs2: refactor pass4 function scan_inode_list Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 22/40] fsck.gfs2: Fix white space problems Bob Peterson
` (20 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch also makes no changes to functionality. It just factors
more code out of function scan_inode_list in pass4 into a new
function called handle_inconsist.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass4.c | 60 +++++++++++++++++++++++++++++--------------------------
1 file changed, 32 insertions(+), 28 deletions(-)
diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
index 8c11761..5dc6e73 100644
--- a/gfs2/fsck/pass4.c
+++ b/gfs2/fsck/pass4.c
@@ -125,10 +125,38 @@ static int handle_unlinked(struct gfs2_sbd *sdp, uint64_t no_addr,
return 0;
}
-static int scan_inode_list(struct gfs2_sbd *sdp) {
+static void handle_inconsist(struct gfs2_sbd *sdp, uint64_t no_addr,
+ uint32_t *di_nlink, uint32_t counted_links)
+{
+ log_err( _("Link count inconsistent for inode %llu"
+ " (0x%llx) has %u but fsck found %u.\n"),
+ (unsigned long long)no_addr, (unsigned long long)no_addr,
+ *di_nlink, counted_links);
+ /* Read in the inode, adjust the link count, and write it back out */
+ if (query( _("Update link count for inode %llu (0x%llx) ? (y/n) "),
+ (unsigned long long)no_addr, (unsigned long long)no_addr)) {
+ struct gfs2_inode *ip;
+
+ ip = fsck_load_inode(sdp, no_addr); /* bread, inode_get */
+ fix_link_count(counted_links, ip);
+ *di_nlink = counted_links;
+ fsck_inode_put(&ip); /* out, brelse, free */
+ log_warn(_("Link count updated to %d for inode %llu "
+ "(0x%llx)\n"), *di_nlink,
+ (unsigned long long)no_addr,
+ (unsigned long long)no_addr);
+ } else {
+ log_err( _("Link count for inode %llu (0x%llx) still "
+ "incorrect\n"),
+ (unsigned long long)no_addr,
+ (unsigned long long)no_addr);
+ }
+}
+
+static int scan_inode_list(struct gfs2_sbd *sdp)
+{
struct osi_node *tmp, *next = NULL;
struct inode_info *ii;
- struct gfs2_inode *ip;
int lf_addition = 0;
/* FIXME: should probably factor this out into a generic
@@ -151,32 +179,8 @@ static int scan_inode_list(struct gfs2_sbd *sdp) {
continue;
} /* if (ii->counted_links == 0) */
else if (ii->di_nlink != ii->counted_links) {
- log_err( _("Link count inconsistent for inode %llu"
- " (0x%llx) has %u but fsck found %u.\n"),
- (unsigned long long)ii->di_num.no_addr,
- (unsigned long long)ii->di_num.no_addr, ii->di_nlink,
- ii->counted_links);
- /* Read in the inode, adjust the link count,
- * and write it back out */
- if (query( _("Update link count for inode %llu"
- " (0x%llx) ? (y/n) "),
- (unsigned long long)ii->di_num.no_addr,
- (unsigned long long)ii->di_num.no_addr)) {
- ip = fsck_load_inode(sdp, ii->di_num.no_addr); /* bread, inode_get */
- fix_link_count(ii->counted_links, ip);
- ii->di_nlink = ii->counted_links;
- fsck_inode_put(&ip); /* out, brelse, free */
- log_warn( _("Link count updated to %d for "
- "inode %llu (0x%llx)\n"),
- ii->di_nlink,
- (unsigned long long)ii->di_num.no_addr,
- (unsigned long long)ii->di_num.no_addr);
- } else {
- log_err( _("Link count for inode %llu (0x%llx"
- ") still incorrect\n"),
- (unsigned long long)ii->di_num.no_addr,
- (unsigned long long)ii->di_num.no_addr);
- }
+ handle_inconsist(sdp, ii->di_num.no_addr,
+ &ii->di_nlink, ii->counted_links);
}
log_debug( _("block %llu (0x%llx) has link count %d\n"),
(unsigned long long)ii->di_num.no_addr,
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 22/40] fsck.gfs2: Fix white space problems
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (20 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 21/40] fsck.gfs2: More refactoring of " Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 23/40] fsck.gfs2: move link count info for directories to directory tree Bob Peterson
` (19 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch changes some white space from spaces to tabs.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/fsck.h | 19 +++++++++----------
1 file changed, 9 insertions(+), 10 deletions(-)
diff --git a/gfs2/fsck/fsck.h b/gfs2/fsck/fsck.h
index d57ccfd..54b40ea 100644
--- a/gfs2/fsck/fsck.h
+++ b/gfs2/fsck/fsck.h
@@ -33,21 +33,20 @@ struct gfs2_bmap {
struct inode_info
{
- struct osi_node node;
- struct gfs2_inum di_num;
- uint32_t di_nlink; /* the number of links the inode
+ struct osi_node node;
+ struct gfs2_inum di_num;
+ uint32_t di_nlink; /* the number of links the inode
* thinks it has */
- uint32_t counted_links; /* the number of links we've found */
+ uint32_t counted_links; /* the number of links we've found */
};
struct dir_info
{
- struct osi_node node;
- struct gfs2_inum dinode;
- uint64_t treewalk_parent;
- struct gfs2_inum dotdot_parent;
- uint8_t checked:1;
-
+ struct osi_node node;
+ struct gfs2_inum dinode;
+ uint64_t treewalk_parent;
+ struct gfs2_inum dotdot_parent;
+ uint8_t checked:1;
};
struct dir_status {
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 23/40] fsck.gfs2: move link count info for directories to directory tree
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (21 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 22/40] fsck.gfs2: Fix white space problems Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 24/40] fsck.gfs2: Use bitmaps instead of linked list for inodes w/nlink == 1 Bob Peterson
` (18 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch saves memory in fsck.gfs2. Instead of keeping an
inode_info structure in a tree for directories, we keep the info
we need--the link counts--in the directory tree structure instead.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/fsck.h | 2 ++
gfs2/fsck/inode_hash.c | 3 +--
gfs2/fsck/link.c | 49 +++++++++++++++++++++++++++++++++++++++++-
gfs2/fsck/link.h | 2 +-
gfs2/fsck/lost_n_found.c | 2 +-
gfs2/fsck/pass1b.c | 4 ++++
gfs2/fsck/pass2.c | 34 +++++++++++++++++++++++-------
gfs2/fsck/pass3.c | 2 +-
gfs2/fsck/pass4.c | 55 +++++++++++++++++++++++++++++++++++++++++++++---
9 files changed, 136 insertions(+), 17 deletions(-)
diff --git a/gfs2/fsck/fsck.h b/gfs2/fsck/fsck.h
index 54b40ea..09b96ed 100644
--- a/gfs2/fsck/fsck.h
+++ b/gfs2/fsck/fsck.h
@@ -46,6 +46,8 @@ struct dir_info
struct gfs2_inum dinode;
uint64_t treewalk_parent;
struct gfs2_inum dotdot_parent;
+ uint32_t di_nlink;
+ uint32_t counted_links;
uint8_t checked:1;
};
diff --git a/gfs2/fsck/inode_hash.c b/gfs2/fsck/inode_hash.c
index eeb0a84..09303d7 100644
--- a/gfs2/fsck/inode_hash.c
+++ b/gfs2/fsck/inode_hash.c
@@ -53,8 +53,7 @@ struct inode_info *inodetree_insert(struct gfs2_inum di_num)
return NULL;
}
/* Add new node and rebalance tree. */
- data->di_num.no_addr = di_num.no_addr;
- data->di_num.no_formal_ino = di_num.no_formal_ino;
+ data->di_num = di_num;
osi_link_node(&data->node, parent, newn);
osi_insert_color(&data->node, &inodetree);
diff --git a/gfs2/fsck/link.c b/gfs2/fsck/link.c
index 9a958b4..7e8acb7 100644
--- a/gfs2/fsck/link.c
+++ b/gfs2/fsck/link.c
@@ -13,11 +13,25 @@
#include "fsck.h"
#include "inode_hash.h"
#include "link.h"
+#include "util.h"
int set_di_nlink(struct gfs2_inode *ip)
{
struct inode_info *ii;
+ struct dir_info *di;
+ if (is_dir(&ip->i_di, ip->i_sbd->gfs1)) {
+ di = dirtree_find(ip->i_di.di_num.no_addr);
+ if (di == NULL) {
+ log_err(_("Error: directory %lld (0x%llx) is not "
+ "in the dir_tree (set).\n"),
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)ip->i_di.di_num.no_addr);
+ return -1;
+ }
+ di->di_nlink = ip->i_di.di_nlink;
+ return 0;
+ }
/*log_debug( _("Setting link count to %u for %" PRIu64
" (0x%" PRIx64 ")\n"), count, inode_no, inode_no);*/
/* If the list has entries, look for one that matches inode_no */
@@ -36,7 +50,21 @@ int incr_link_count(struct gfs2_inum no, struct gfs2_inode *ip,
{
struct inode_info *ii = NULL;
uint64_t referenced_from = ip ? ip->i_di.di_num.no_addr : 0;
+ struct dir_info *di;
+
+ di = dirtree_find(no.no_addr);
+ if (di) {
+ if (di->dinode.no_formal_ino != no.no_formal_ino)
+ return 1;
+ di->counted_links++;
+ log_debug( _("Dir (0x%llx) incremented counted links to %u "
+ "for (0x%llx) via %s\n"),
+ (unsigned long long)referenced_from,
+ di->counted_links,
+ (unsigned long long)no.no_addr, why);
+ return 0;
+ }
ii = inodetree_find(no.no_addr);
/* If the list has entries, look for one that matches inode_no */
if (ii) {
@@ -65,10 +93,29 @@ int incr_link_count(struct gfs2_inum no, struct gfs2_inode *ip,
return 0;
}
-int decr_link_count(uint64_t inode_no, uint64_t referenced_from,
+int decr_link_count(uint64_t inode_no, uint64_t referenced_from, int gfs1,
const char *why)
{
struct inode_info *ii = NULL;
+ struct dir_info *di;
+
+ di = dirtree_find(inode_no);
+ if (di) {
+ if (!di->counted_links) {
+ log_debug( _("Dir (0x%llx)'s link to "
+ "(0x%llx) via %s is zero!\n"),
+ (unsigned long long)referenced_from,
+ (unsigned long long)inode_no, why);
+ return 0;
+ }
+ di->counted_links--;
+ log_debug( _("Dir (0x%llx) decremented counted links to %u "
+ "for (0x%llx) via %s\n"),
+ (unsigned long long)referenced_from,
+ di->counted_links, (unsigned long long)inode_no,
+ why);
+ return 0;
+ }
ii = inodetree_find(inode_no);
/* If the list has entries, look for one that matches
diff --git a/gfs2/fsck/link.h b/gfs2/fsck/link.h
index 842afb9..76a195a 100644
--- a/gfs2/fsck/link.h
+++ b/gfs2/fsck/link.h
@@ -4,7 +4,7 @@
int set_di_nlink(struct gfs2_inode *ip);
int incr_link_count(struct gfs2_inum no, struct gfs2_inode *ip,
const char *why);
-int decr_link_count(uint64_t inode_no, uint64_t referenced_from,
+int decr_link_count(uint64_t inode_no, uint64_t referenced_from, int gfs1,
const char *why);
#endif /* _LINK_H */
diff --git a/gfs2/fsck/lost_n_found.c b/gfs2/fsck/lost_n_found.c
index 9c85f52..197ae7d 100644
--- a/gfs2/fsck/lost_n_found.c
+++ b/gfs2/fsck/lost_n_found.c
@@ -43,7 +43,7 @@ static void add_dotdot(struct gfs2_inode *ip)
if (dip->i_di.di_num.no_formal_ino ==
di->dotdot_parent.no_formal_ino) {
decr_link_count(di->dotdot_parent.no_addr,
- ip->i_di.di_num.no_addr,
+ ip->i_di.di_num.no_addr, sdp->gfs1,
_(".. unlinked, moving to lost+found"));
if (dip->i_di.di_nlink > 0) {
dip->i_di.di_nlink--;
diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
index 182c6cb..db477ff 100644
--- a/gfs2/fsck/pass1b.c
+++ b/gfs2/fsck/pass1b.c
@@ -203,6 +203,7 @@ static void resolve_dup_references(struct gfs2_sbd *sdp, struct duptree *dt,
};
enum dup_ref_type this_ref;
struct inode_info *ii;
+ struct dir_info *di;
int found_good_ref = 0;
int q;
@@ -349,6 +350,9 @@ static void resolve_dup_references(struct gfs2_sbd *sdp, struct duptree *dt,
ii = inodetree_find(ip->i_di.di_num.no_addr);
if (ii)
inodetree_delete(ii);
+ di = dirtree_find(ip->i_di.di_num.no_addr);
+ if (di)
+ dirtree_delete(di);
fsck_bitmap_set(ip, ip->i_di.di_num.no_addr,
_("duplicate referencing bad"),
GFS2_BLKST_UNLINKED);
diff --git a/gfs2/fsck/pass2.c b/gfs2/fsck/pass2.c
index 02d82e4..3a8646f 100644
--- a/gfs2/fsck/pass2.c
+++ b/gfs2/fsck/pass2.c
@@ -181,12 +181,21 @@ static int bad_formal_ino(struct gfs2_inode *ip, struct gfs2_dirent *dent,
struct gfs2_buffer_head *bh)
{
struct inode_info *ii;
+ struct dir_info *di = NULL;
struct gfs2_inode *child_ip;
struct gfs2_inum childs_dotdot;
struct gfs2_sbd *sdp = ip->i_sbd;
int error;
+ struct gfs2_inum inum = { 0 };
ii = inodetree_find(entry.no_addr);
+ if (ii)
+ inum = ii->di_num;
+ else {
+ di = dirtree_find(entry.no_addr);
+ if (di)
+ inum = di->dinode;
+ }
log_err( _("Directory entry '%s' pointing to block %llu (0x%llx) in "
"directory %llu (0x%llx) has the wrong 'formal' inode "
"number.\n"), tmp_name, (unsigned long long)entry.no_addr,
@@ -196,8 +205,8 @@ static int bad_formal_ino(struct gfs2_inode *ip, struct gfs2_dirent *dent,
log_err( _("The directory entry has %llu (0x%llx) but the inode has "
"%llu (0x%llx)\n"), (unsigned long long)entry.no_formal_ino,
(unsigned long long)entry.no_formal_ino,
- (unsigned long long)ii->di_num.no_formal_ino,
- (unsigned long long)ii->di_num.no_formal_ino);
+ (unsigned long long)inum.no_formal_ino,
+ (unsigned long long)inum.no_formal_ino);
if (q != GFS2_BLKST_DINODE || !strcmp("..", tmp_name)) {
if (query( _("Remove the corrupt directory entry? (y/n) ")))
return 1;
@@ -214,7 +223,7 @@ static int bad_formal_ino(struct gfs2_inode *ip, struct gfs2_dirent *dent,
"linkage.\n"));
if (query( _("Fix the bad directory entry? (y/n) "))) {
log_err( _("Fixing the corrupt directory entry.\n"));
- entry.no_formal_ino = ii->di_num.no_formal_ino;
+ entry.no_formal_ino = inum.no_formal_ino;
de->de_inum.no_formal_ino = entry.no_formal_ino;
gfs2_dirent_out(de, (char *)dent);
bmodified(bh);
@@ -446,6 +455,8 @@ static int basic_dentry_checks(struct gfs2_inode *ip, struct gfs2_dirent *dent,
struct gfs2_inode *entry_ip = NULL;
int error;
struct inode_info *ii;
+ struct dir_info *di = NULL;
+ struct gfs2_inum inum = { 0 };
*isdir = 0;
if (!valid_block(ip->i_sbd, entry->no_addr)) {
@@ -541,7 +552,7 @@ static int basic_dentry_checks(struct gfs2_inode *ip, struct gfs2_dirent *dent,
/* Don't decrement the link here: Here in pass2, we increment
only when we know it's okay.
- decr_link_count(ip->i_di.di_num.no_addr); */
+ decr_link_count(ip->i_di.di_num.no_addr, blah); */
/* If it was previously marked invalid (i.e. known
to be bad, not just a free block, etc.) then the temptation
would be to delete any metadata it holds. The trouble is:
@@ -596,7 +607,14 @@ static int basic_dentry_checks(struct gfs2_inode *ip, struct gfs2_dirent *dent,
/* We need to verify the formal inode number matches. If it doesn't,
it needs to be deleted. */
ii = inodetree_find(entry->no_addr);
- if (ii && ii->di_num.no_formal_ino != entry->no_formal_ino) {
+ if (ii)
+ inum = ii->di_num;
+ else {
+ di = dirtree_find(entry->no_addr);
+ if (di)
+ inum = di->dinode;
+ }
+ if (inum.no_formal_ino != entry->no_formal_ino) {
log_err( _("Directory entry '%s' pointing to block %llu "
"(0x%llx) in directory %llu (0x%llx) has the "
"wrong 'formal' inode number.\n"), tmp_name,
@@ -608,15 +626,15 @@ static int basic_dentry_checks(struct gfs2_inode *ip, struct gfs2_dirent *dent,
"inode has %llu (0x%llx)\n"),
(unsigned long long)entry->no_formal_ino,
(unsigned long long)entry->no_formal_ino,
- (unsigned long long)ii->di_num.no_formal_ino,
- (unsigned long long)ii->di_num.no_formal_ino);
+ (unsigned long long)inum.no_formal_ino,
+ (unsigned long long)inum.no_formal_ino);
return 1;
}
/* Check for a special case where a (bad) GFS1 dirent points to what
* is not a known inode. It could be other GFS1 metadata, such as an
* eattr or indirect block, but marked "dinode" in the bitmap because
* gfs1 marked all gfs1 metadata that way. */
- if (ii == NULL && sdp->gfs1) {
+ if (ii == NULL && di == NULL && sdp->gfs1) {
struct gfs2_buffer_head *tbh;
tbh = bread(sdp, entry->no_addr);
diff --git a/gfs2/fsck/pass3.c b/gfs2/fsck/pass3.c
index d5f4ea2..2fa0bdb 100644
--- a/gfs2/fsck/pass3.c
+++ b/gfs2/fsck/pass3.c
@@ -38,7 +38,7 @@ static int attach_dotdot_to(struct gfs2_sbd *sdp, uint64_t newdotdot,
if (gfs2_dirent_del(ip, filename, filename_len))
log_warn( _("Unable to remove \"..\" directory entry.\n"));
else
- decr_link_count(olddotdot, block, _("old \"..\""));
+ decr_link_count(olddotdot, block, sdp->gfs1, _("old \"..\""));
err = dir_add(ip, filename, filename_len, &pip->i_di.di_num,
(sdp->gfs1 ? GFS_FILE_DIR : DT_DIR));
if (err) {
diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
index 5dc6e73..c65d321 100644
--- a/gfs2/fsck/pass4.c
+++ b/gfs2/fsck/pass4.c
@@ -157,6 +157,7 @@ static int scan_inode_list(struct gfs2_sbd *sdp)
{
struct osi_node *tmp, *next = NULL;
struct inode_info *ii;
+ struct dir_info *lf_di;
int lf_addition = 0;
/* FIXME: should probably factor this out into a generic
@@ -169,7 +170,6 @@ static int scan_inode_list(struct gfs2_sbd *sdp)
/* Don't check reference counts on the special gfs files */
if (sdp->gfs1 &&
((ii->di_num.no_addr == sdp->md.riinode->i_di.di_num.no_addr) ||
- (ii->di_num.no_addr == sdp->md.jiinode->i_di.di_num.no_addr) ||
(ii->di_num.no_addr == sdp->md.qinode->i_di.di_num.no_addr) ||
(ii->di_num.no_addr == sdp->md.statfs->i_di.di_num.no_addr)))
continue;
@@ -191,11 +191,56 @@ static int scan_inode_list(struct gfs2_sbd *sdp)
return 0;
if (lf_addition) {
- if (!(ii = inodetree_find(lf_dip->i_di.di_num.no_addr))) {
+ if (!(lf_di = dirtree_find(lf_dip->i_di.di_num.no_addr))) {
log_crit( _("Unable to find lost+found inode in inode_hash!!\n"));
return -1;
} else {
- fix_link_count(ii->counted_links, lf_dip);
+ fix_link_count(lf_di->counted_links, lf_dip);
+ }
+ }
+
+ return 0;
+}
+
+static int scan_dir_list(struct gfs2_sbd *sdp)
+{
+ struct osi_node *tmp, *next = NULL;
+ struct dir_info *di, *lf_di;
+ int lf_addition = 0;
+
+ /* FIXME: should probably factor this out into a generic
+ * scanning fxn */
+ for (tmp = osi_first(&dirtree); tmp; tmp = next) {
+ if (skip_this_pass || fsck_abort) /* if asked to skip the rest */
+ return 0;
+ next = osi_next(tmp);
+ di = (struct dir_info *)tmp;
+ /* Don't check reference counts on the special gfs files */
+ if (sdp->gfs1 &&
+ di->dinode.no_addr == sdp->md.jiinode->i_di.di_num.no_addr)
+ continue;
+ if (di->counted_links == 0) {
+ if (handle_unlinked(sdp, di->dinode.no_addr,
+ &di->counted_links, &lf_addition))
+ continue;
+ } else if (di->di_nlink != di->counted_links) {
+ handle_inconsist(sdp, di->dinode.no_addr,
+ &di->di_nlink, di->counted_links);
+ }
+ log_debug( _("block %llu (0x%llx) has link count %d\n"),
+ (unsigned long long)di->dinode.no_addr,
+ (unsigned long long)di->dinode.no_addr, di->di_nlink);
+ } /* osi_list_foreach(tmp, list) */
+
+ if (lf_dip == NULL)
+ return 0;
+
+ if (lf_addition) {
+ if (!(lf_di = dirtree_find(lf_dip->i_di.di_num.no_addr))) {
+ log_crit( _("Unable to find lost+found inode in inode_hash!!\n"));
+ return -1;
+ } else {
+ fix_link_count(lf_di->counted_links, lf_dip);
}
}
@@ -221,6 +266,10 @@ int pass4(struct gfs2_sbd *sdp)
stack;
return FSCK_ERROR;
}
+ if (scan_dir_list(sdp)) {
+ stack;
+ return FSCK_ERROR;
+ }
if (lf_dip)
log_debug( _("At end of pass4, lost+found entries is %u\n"),
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 24/40] fsck.gfs2: Use bitmaps instead of linked list for inodes w/nlink == 1
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (22 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 23/40] fsck.gfs2: move link count info for directories to directory tree Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 25/40] fsck.gfs2: Refactor check_n_fix_bitmap to make it more readable Bob Peterson
` (17 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch adds two new in-core bitmaps. Their purpose is to replace
the most common case where inodes have an nlink count equal to 1.
If we've got petabytes of inodes, this will save us a lot of memory.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/link.c | 110 +++++++++++++++++++++++++++++++++++++--------------
gfs2/fsck/link.h | 4 ++
gfs2/fsck/main.c | 3 ++
gfs2/fsck/metawalk.c | 2 +
gfs2/fsck/pass1.c | 40 +++++++++++++++++--
gfs2/fsck/pass1b.c | 3 ++
gfs2/fsck/pass2.c | 8 ++++
gfs2/fsck/pass4.c | 80 +++++++++++++++++++++++++------------
gfs2/fsck/util.h | 23 +++++++++++
9 files changed, 214 insertions(+), 59 deletions(-)
diff --git a/gfs2/fsck/link.c b/gfs2/fsck/link.c
index 7e8acb7..0243d85 100644
--- a/gfs2/fsck/link.c
+++ b/gfs2/fsck/link.c
@@ -15,6 +15,26 @@
#include "link.h"
#include "util.h"
+struct gfs2_bmap nlink1map = { 0 }; /* map of dinodes with nlink == 1 */
+struct gfs2_bmap clink1map = { 0 }; /* map of dinodes w/counted links == 1 */
+
+int link1_set(struct gfs2_bmap *bmap, uint64_t bblock, int mark)
+{
+ static unsigned char *byte;
+ static uint64_t b;
+
+ if (!bmap)
+ return 0;
+ if (bblock > bmap->size)
+ return -1;
+
+ byte = bmap->map + BLOCKMAP_SIZE1(bblock);
+ b = BLOCKMAP_BYTE_OFFSET1(bblock);
+ *byte &= ~(BLOCKMAP_MASK1 << b);
+ *byte |= (mark & BLOCKMAP_MASK1) << b;
+ return 0;
+}
+
int set_di_nlink(struct gfs2_inode *ip)
{
struct inode_info *ii;
@@ -32,6 +52,10 @@ int set_di_nlink(struct gfs2_inode *ip)
di->di_nlink = ip->i_di.di_nlink;
return 0;
}
+ if (ip->i_di.di_nlink == 1) {
+ link1_set(&nlink1map, ip->i_di.di_num.no_addr, 1);
+ return 0;
+ }
/*log_debug( _("Setting link count to %u for %" PRIu64
" (0x%" PRIx64 ")\n"), count, inode_no, inode_no);*/
/* If the list has entries, look for one that matches inode_no */
@@ -45,12 +69,21 @@ int set_di_nlink(struct gfs2_inode *ip)
return 0;
}
+/* I'm making whyincr a macro rather than function so that the debug output
+ * matches older versions. */
+#define whyincr(no_addr, why, referenced_from, counted_links) \
+ log_debug(_("Dir (0x%llx) incremented counted links to %u " \
+ "for (0x%llx) via %s\n"), \
+ (unsigned long long)referenced_from, counted_links, \
+ (unsigned long long)no_addr, why);
+
int incr_link_count(struct gfs2_inum no, struct gfs2_inode *ip,
const char *why)
{
struct inode_info *ii = NULL;
uint64_t referenced_from = ip ? ip->i_di.di_num.no_addr : 0;
struct dir_info *di;
+ struct gfs2_inode *link_ip;
di = dirtree_find(no.no_addr);
if (di) {
@@ -58,11 +91,7 @@ int incr_link_count(struct gfs2_inum no, struct gfs2_inode *ip,
return 1;
di->counted_links++;
- log_debug( _("Dir (0x%llx) incremented counted links to %u "
- "for (0x%llx) via %s\n"),
- (unsigned long long)referenced_from,
- di->counted_links,
- (unsigned long long)no.no_addr, why);
+ whyincr(no.no_addr, why, referenced_from, di->counted_links);
return 0;
}
ii = inodetree_find(no.no_addr);
@@ -72,27 +101,52 @@ int incr_link_count(struct gfs2_inum no, struct gfs2_inode *ip,
return 1;
ii->counted_links++;
- log_debug( _("Dir (0x%llx) incremented counted "
- "links to %u for (0x%llx) via %s\n"),
- (unsigned long long)referenced_from,
- ii->counted_links, (unsigned long long)no.no_addr,
- why);
+ whyincr(no.no_addr, why, referenced_from, ii->counted_links);
return 0;
}
- log_debug( _("Ref: (0x%llx) No match found when incrementing "
- "link for (0x%llx)!\n"),
- (unsigned long long)referenced_from,
- (unsigned long long)no.no_addr);
- /* If no match was found, add a new entry and set its
- * counted links to 1 */
+ if (link1_type(&clink1map, no.no_addr) != 1) {
+ link1_set(&clink1map, no.no_addr, 1);
+ whyincr(no.no_addr, why, referenced_from, 1);
+ return 0;
+ }
+
+ link_ip = fsck_load_inode(ip->i_sbd, no.no_addr);
+ /* Check formal ino against dinode before adding to inode tree. */
+ if (no.no_formal_ino != ip->i_di.di_num.no_formal_ino) {
+ fsck_inode_put(&link_ip);
+ return 1;
+ }
+ /* Move it from the link1 maps to a real inode tree entry */
+ link1_set(&nlink1map, no.no_addr, 0);
+ link1_set(&clink1map, no.no_addr, 0);
+
+ /* If no match was found, it must be a hard link. In theory, it can't
+ be a duplicate because those were resolved in pass1b. Add a new
+ inodetree entry and set its counted links to 2 */
ii = inodetree_insert(no);
- if (ii)
- ii->counted_links = 1;
- else
+ if (!ii) {
+ log_debug( _("Ref: (0x%llx) Error incrementing link for "
+ "(0x%llx)!\n"),
+ (unsigned long long)referenced_from,
+ (unsigned long long)no.no_addr);
+ fsck_inode_put(&link_ip);
return -1;
+ }
+ ii->di_num = link_ip->i_di.di_num;
+ fsck_inode_put(&link_ip);
+ ii->di_nlink = 1; /* Must be 1 or it wouldn't have gotten into the
+ nlink1map */
+ ii->counted_links = 2;
+ whyincr(no.no_addr, why, referenced_from, ii->counted_links);
return 0;
}
+#define whydecr(no_addr, why, referenced_from, counted_links) \
+ log_debug(_("Dir (0x%llx) decremented counted links to %u " \
+ "for (0x%llx) via %s\n"), \
+ (unsigned long long)referenced_from, counted_links, \
+ (unsigned long long)no_addr, why);
+
int decr_link_count(uint64_t inode_no, uint64_t referenced_from, int gfs1,
const char *why)
{
@@ -109,11 +163,7 @@ int decr_link_count(uint64_t inode_no, uint64_t referenced_from, int gfs1,
return 0;
}
di->counted_links--;
- log_debug( _("Dir (0x%llx) decremented counted links to %u "
- "for (0x%llx) via %s\n"),
- (unsigned long long)referenced_from,
- di->counted_links, (unsigned long long)inode_no,
- why);
+ whydecr(inode_no, why, referenced_from, di->counted_links);
return 0;
}
@@ -129,13 +179,15 @@ int decr_link_count(uint64_t inode_no, uint64_t referenced_from, int gfs1,
return 0;
}
ii->counted_links--;
- log_debug( _("Dir (0x%llx) decremented counted "
- "links to %u for (0x%llx) via %s\n"),
- (unsigned long long)referenced_from,
- ii->counted_links, (unsigned long long)inode_no,
- why);
+ whydecr(inode_no, why, referenced_from, ii->counted_links);
return 0;
}
+ if (link1_type(&clink1map, inode_no) == 1) { /* 1 -> 0 */
+ link1_set(&clink1map, inode_no, 0);
+ whydecr(inode_no, why, referenced_from, 0);
+ return 0;
+ }
+
log_debug( _("No match found when decrementing link for (0x%llx)!\n"),
(unsigned long long)inode_no);
return -1;
diff --git a/gfs2/fsck/link.h b/gfs2/fsck/link.h
index 76a195a..14534e5 100644
--- a/gfs2/fsck/link.h
+++ b/gfs2/fsck/link.h
@@ -1,6 +1,10 @@
#ifndef _LINK_H
#define _LINK_H
+extern struct gfs2_bmap nlink1map; /* map of dinodes with nlink == 1 */
+extern struct gfs2_bmap clink1map; /* map of dinodes w/counted links == 1 */
+
+int link1_set(struct gfs2_bmap *bmap, uint64_t bblock, int mark);
int set_di_nlink(struct gfs2_inode *ip);
int incr_link_count(struct gfs2_inum no, struct gfs2_inode *ip,
const char *why);
diff --git a/gfs2/fsck/main.c b/gfs2/fsck/main.c
index 91ebe28..c5967bc 100644
--- a/gfs2/fsck/main.c
+++ b/gfs2/fsck/main.c
@@ -20,6 +20,7 @@
#include "copyright.cf"
#include "libgfs2.h"
#include "fsck.h"
+#include "link.h"
#include "osi_list.h"
#include "metawalk.h"
#include "util.h"
@@ -352,6 +353,8 @@ int main(int argc, char **argv)
if (!opts.no && errors_corrected)
log_notice( _("Writing changes to disk\n"));
fsync(sdp->device_fd);
+ link1_destroy(&nlink1map);
+ link1_destroy(&clink1map);
destroy(sdp);
if (sb_fixed)
log_warn(_("Superblock was reset. Use tunegfs2 to manually "
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 3217711..6b7146e 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -14,6 +14,7 @@
#include <logging.h>
#include "libgfs2.h"
+#include "link.h"
#include "osi_tree.h"
#include "fsck.h"
#include "util.h"
@@ -88,6 +89,7 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
ii = inodetree_find(blk);
if (ii)
inodetree_delete(ii);
+ link1_set(&nlink1map, blk, 0);
}
rgd->rg.rg_free++;
if (sdp->gfs1)
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index f4eea49..ee20b9e 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -2018,6 +2018,20 @@ static int gfs2_blockmap_create(struct gfs2_bmap *bmap, uint64_t size)
return 0;
}
+
+static int link1_create(struct gfs2_bmap *bmap, uint64_t size)
+{
+ bmap->size = size;
+
+ /* Have to add 1 to BLOCKMAP_SIZE since it's 0-based and mallocs
+ * must be 1-based */
+ bmap->mapsize = BLOCKMAP_SIZE1(size) + 1;
+
+ if (!(bmap->map = calloc(bmap->mapsize, sizeof(char))))
+ return -ENOMEM;
+ return 0;
+}
+
static struct gfs2_bmap *gfs2_bmap_create(struct gfs2_sbd *sdp, uint64_t size,
uint64_t *addl_mem_needed)
{
@@ -2054,6 +2068,14 @@ static void *gfs2_bmap_destroy(struct gfs2_sbd *sdp, struct gfs2_bmap *il)
return il;
}
+static void enomem(uint64_t addl_mem_needed)
+{
+ log_crit( _("This system doesn't have enough memory and swap space to fsck this file system.\n"));
+ log_crit( _("Additional memory needed is approximately: %lluMB\n"),
+ (unsigned long long)(addl_mem_needed / 1048576ULL));
+ log_crit( _("Please increase your swap space by that amount and run gfs2_fsck again.\n"));
+}
+
/**
* pass1 - walk through inodes and check inode state
*
@@ -2079,10 +2101,20 @@ int pass1(struct gfs2_sbd *sdp)
bl = gfs2_bmap_create(sdp, last_fs_block+1, &addl_mem_needed);
if (!bl) {
- log_crit( _("This system doesn't have enough memory and swap space to fsck this file system.\n"));
- log_crit( _("Additional memory needed is approximately: %lluMB\n"),
- (unsigned long long)(addl_mem_needed / 1048576ULL));
- log_crit( _("Please increase your swap space by that amount and run gfs2_fsck again.\n"));
+ enomem(addl_mem_needed);
+ return FSCK_ERROR;
+ }
+ addl_mem_needed = link1_create(&nlink1map, last_fs_block+1);
+ if (addl_mem_needed) {
+ enomem(addl_mem_needed);
+ gfs2_bmap_destroy(sdp, bl);
+ return FSCK_ERROR;
+ }
+ addl_mem_needed = link1_create(&clink1map, last_fs_block+1);
+ if (addl_mem_needed) {
+ enomem(addl_mem_needed);
+ link1_destroy(&nlink1map);
+ gfs2_bmap_destroy(sdp, bl);
return FSCK_ERROR;
}
osi_list_init(&gfs1_rindex_blks.list);
diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
index db477ff..d6ab285 100644
--- a/gfs2/fsck/pass1b.c
+++ b/gfs2/fsck/pass1b.c
@@ -10,6 +10,7 @@
#include <logging.h>
#include "libgfs2.h"
+#include "link.h"
#include "fsck.h"
#include "osi_list.h"
#include "util.h"
@@ -353,6 +354,8 @@ static void resolve_dup_references(struct gfs2_sbd *sdp, struct duptree *dt,
di = dirtree_find(ip->i_di.di_num.no_addr);
if (di)
dirtree_delete(di);
+ link1_set(&nlink1map, ip->i_di.di_num.no_addr,
+ 0);
fsck_bitmap_set(ip, ip->i_di.di_num.no_addr,
_("duplicate referencing bad"),
GFS2_BLKST_UNLINKED);
diff --git a/gfs2/fsck/pass2.c b/gfs2/fsck/pass2.c
index 3a8646f..d4f50d2 100644
--- a/gfs2/fsck/pass2.c
+++ b/gfs2/fsck/pass2.c
@@ -195,6 +195,8 @@ static int bad_formal_ino(struct gfs2_inode *ip, struct gfs2_dirent *dent,
di = dirtree_find(entry.no_addr);
if (di)
inum = di->dinode;
+ else if (link1_type(&clink1map, entry.no_addr) == 1)
+ inum = entry;
}
log_err( _("Directory entry '%s' pointing to block %llu (0x%llx) in "
"directory %llu (0x%llx) has the wrong 'formal' inode "
@@ -613,6 +615,12 @@ static int basic_dentry_checks(struct gfs2_inode *ip, struct gfs2_dirent *dent,
di = dirtree_find(entry->no_addr);
if (di)
inum = di->dinode;
+ else if (link1_type(&nlink1map, entry->no_addr) == 1) {
+ /* Since we don't have ii or di, the only way to
+ validate formal_ino is to read in the inode, which
+ would kill performance. So skip it for now. */
+ return 0;
+ }
}
if (inum.no_formal_ino != entry->no_formal_ino) {
log_err( _("Directory entry '%s' pointing to block %llu "
diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
index c65d321..16339cc 100644
--- a/gfs2/fsck/pass4.c
+++ b/gfs2/fsck/pass4.c
@@ -9,6 +9,7 @@
#include <logging.h>
#include "libgfs2.h"
#include "fsck.h"
+#include "link.h"
#include "lost_n_found.h"
#include "inode_hash.h"
#include "metawalk.h"
@@ -153,11 +154,30 @@ static void handle_inconsist(struct gfs2_sbd *sdp, uint64_t no_addr,
}
}
+static int adjust_lf_links(int lf_addition)
+{
+ struct dir_info *lf_di;
+
+ if (lf_dip == NULL)
+ return 0;
+
+ if (!lf_addition)
+ return 0;
+
+ if (!(lf_di = dirtree_find(lf_dip->i_di.di_num.no_addr))) {
+ log_crit(_("Unable to find lost+found inode in "
+ "inode_hash!!\n"));
+ return -1;
+ } else {
+ fix_link_count(lf_di->counted_links, lf_dip);
+ }
+ return 0;
+}
+
static int scan_inode_list(struct gfs2_sbd *sdp)
{
struct osi_node *tmp, *next = NULL;
struct inode_info *ii;
- struct dir_info *lf_di;
int lf_addition = 0;
/* FIXME: should probably factor this out into a generic
@@ -187,25 +207,13 @@ static int scan_inode_list(struct gfs2_sbd *sdp)
(unsigned long long)ii->di_num.no_addr, ii->di_nlink);
} /* osi_list_foreach(tmp, list) */
- if (lf_dip == NULL)
- return 0;
-
- if (lf_addition) {
- if (!(lf_di = dirtree_find(lf_dip->i_di.di_num.no_addr))) {
- log_crit( _("Unable to find lost+found inode in inode_hash!!\n"));
- return -1;
- } else {
- fix_link_count(lf_di->counted_links, lf_dip);
- }
- }
-
- return 0;
+ return adjust_lf_links(lf_addition);
}
static int scan_dir_list(struct gfs2_sbd *sdp)
{
struct osi_node *tmp, *next = NULL;
- struct dir_info *di, *lf_di;
+ struct dir_info *di;
int lf_addition = 0;
/* FIXME: should probably factor this out into a generic
@@ -232,19 +240,33 @@ static int scan_dir_list(struct gfs2_sbd *sdp)
(unsigned long long)di->dinode.no_addr, di->di_nlink);
} /* osi_list_foreach(tmp, list) */
- if (lf_dip == NULL)
- return 0;
+ return adjust_lf_links(lf_addition);
+}
- if (lf_addition) {
- if (!(lf_di = dirtree_find(lf_dip->i_di.di_num.no_addr))) {
- log_crit( _("Unable to find lost+found inode in inode_hash!!\n"));
- return -1;
- } else {
- fix_link_count(lf_di->counted_links, lf_dip);
+static int scan_nlink1_list(struct gfs2_sbd *sdp)
+{
+ uint64_t blk;
+ uint32_t counted_links;
+ int lf_addition = 0;
+
+ for (blk = 0; blk < last_fs_block; blk++) {
+ if (skip_this_pass || fsck_abort)
+ return 0;
+ if (link1_type(&nlink1map, blk) == 0)
+ continue;
+
+ if (link1_type(&clink1map, blk) == 0) {
+ /* In other cases, counted_links is a pointer to a
+ real count that gets incremented when it's added
+ to lost+found. In this case, however, there's not a
+ real count, so we fake it out to be 1. */
+ counted_links = 1;
+ if (handle_unlinked(sdp, blk, &counted_links,
+ &lf_addition))
+ continue;
}
}
-
- return 0;
+ return adjust_lf_links(lf_addition);
}
/**
@@ -261,15 +283,21 @@ int pass4(struct gfs2_sbd *sdp)
if (lf_dip)
log_debug( _("At beginning of pass4, lost+found entries is %u\n"),
lf_dip->i_di.di_entries);
- log_info( _("Checking inode reference counts.\n"));
+ log_info( _("Checking inode reference counts: multi-links.\n"));
if (scan_inode_list(sdp)) {
stack;
return FSCK_ERROR;
}
+ log_info( _("Checking inode reference counts: directories.\n"));
if (scan_dir_list(sdp)) {
stack;
return FSCK_ERROR;
}
+ log_info( _("Checking inode reference counts: normal links.\n"));
+ if (scan_nlink1_list(sdp)) {
+ stack;
+ return FSCK_ERROR;
+ }
if (lf_dip)
log_debug( _("At end of pass4, lost+found entries is %u\n"),
diff --git a/gfs2/fsck/util.h b/gfs2/fsck/util.h
index aa01552..4545594 100644
--- a/gfs2/fsck/util.h
+++ b/gfs2/fsck/util.h
@@ -23,9 +23,12 @@ extern void dup_listent_delete(struct duptree *dt, struct inode_with_dups *id);
extern const char *reftypes[ref_types + 1];
+#define BLOCKMAP_SIZE1(size) ((size) >> 3)
#define BLOCKMAP_SIZE2(size) ((size) >> 2)
#define BLOCKMAP_BYTE_OFFSET2(x) ((x & 0x0000000000000003) << 1)
+#define BLOCKMAP_BYTE_OFFSET1(x) (x & 0x0000000000000007)
#define BLOCKMAP_MASK2 (0x3)
+#define BLOCKMAP_MASK1 (1)
struct fsck_pass {
const char *name;
@@ -44,6 +47,26 @@ static inline int block_type(struct gfs2_bmap *bl, uint64_t bblock)
return btype;
}
+static inline int link1_type(struct gfs2_bmap *bl, uint64_t bblock)
+{
+ static unsigned char *byte;
+ static uint64_t b;
+ static int btype;
+
+ byte = bl->map + BLOCKMAP_SIZE1(bblock);
+ b = BLOCKMAP_BYTE_OFFSET1(bblock);
+ btype = (*byte & (BLOCKMAP_MASK1 << b )) >> b;
+ return btype;
+}
+
+static inline void link1_destroy(struct gfs2_bmap *bmap)
+{
+ if (bmap->map)
+ free(bmap->map);
+ bmap->size = 0;
+ bmap->mapsize = 0;
+}
+
static inline int bitmap_type(struct gfs2_sbd *sdp, uint64_t bblock)
{
struct rgrp_tree *rgd;
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 25/40] fsck.gfs2: Refactor check_n_fix_bitmap to make it more readable
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (23 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 24/40] fsck.gfs2: Use bitmaps instead of linked list for inodes w/nlink == 1 Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 26/40] fsck.gfs2: adjust rgrp inode count when fixing bitmap Bob Peterson
` (16 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch should not change functionality. It just refactors the
function check_n_fix_bitmap so it's easier to read and change.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/metawalk.c | 122 +++++++++++++++++++++++++--------------------------
1 file changed, 60 insertions(+), 62 deletions(-)
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 6b7146e..9ec4bea 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -34,6 +34,10 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
{
int old_bitmap_state;
struct rgrp_tree *rgd;
+ const char *allocdesc[2][5] = { /* gfs2 descriptions */
+ {"free", "data", "unlinked", "inode", "reserved"},
+ /* gfs1 descriptions: */
+ {"free", "data", "free meta", "metadata", "reserved"}};
rgd = gfs2_blk2rgrpd(sdp, blk);
@@ -44,70 +48,64 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
(unsigned long long)blk, (unsigned long long)blk);
return -1;
}
- if (old_bitmap_state != new_blockmap_state) {
- const char *allocdesc[2][5] = { /* gfs2 descriptions */
- {"free", "data", "unlinked", "inode", "reserved"},
- /* gfs1 descriptions: */
- {"free", "data", "free meta", "metadata", "reserved"}};
-
- if (error_on_dinode && old_bitmap_state == GFS2_BLKST_DINODE &&
- new_blockmap_state != GFS2_BLKST_FREE) {
- log_debug(_("Reference as '%s' to block %llu (0x%llx) "
- "which was marked as dinode. Needs "
- "further investigation.\n"),
- allocdesc[sdp->gfs1][new_blockmap_state],
- (unsigned long long)blk,
- (unsigned long long)blk);
- return 1;
- }
- /* Keep these messages as short as possible, or the output
- gets to be huge and unmanageable. */
- log_err( _("Block %llu (0x%llx) was '%s', should be %s.\n"),
- (unsigned long long)blk, (unsigned long long)blk,
- allocdesc[sdp->gfs1][old_bitmap_state],
- allocdesc[sdp->gfs1][new_blockmap_state]);
- if (query( _("Fix the bitmap? (y/n)"))) {
- /* If the new bitmap state is free (and therefore the
- old state was not) we have to add to the free
- space in the rgrp. If the old bitmap state was
- free (and therefore it no longer is) we have to
- subtract to the free space. If the type changed
- from dinode to data or data to dinode, no change in
- free space. */
- gfs2_set_bitmap(rgd, blk, new_blockmap_state);
- if (new_blockmap_state == GFS2_BLKST_FREE) {
- /* If we're freeing a dinode, get rid of
- the hash table entries for it. */
- if (old_bitmap_state == GFS2_BLKST_DINODE ||
- old_bitmap_state == GFS2_BLKST_UNLINKED) {
- struct dir_info *dt;
- struct inode_info *ii;
-
- dt = dirtree_find(blk);
- if (dt)
- dirtree_delete(dt);
- ii = inodetree_find(blk);
- if (ii)
- inodetree_delete(ii);
- link1_set(&nlink1map, blk, 0);
- }
- rgd->rg.rg_free++;
- if (sdp->gfs1)
- gfs_rgrp_out((struct gfs_rgrp *)&rgd->rg, rgd->bits[0].bi_bh);
- else
- gfs2_rgrp_out_bh(&rgd->rg, rgd->bits[0].bi_bh);
- } else if (old_bitmap_state == GFS2_BLKST_FREE) {
- rgd->rg.rg_free--;
- if (sdp->gfs1)
- gfs_rgrp_out((struct gfs_rgrp *)&rgd->rg, rgd->bits[0].bi_bh);
- else
- gfs2_rgrp_out_bh(&rgd->rg, rgd->bits[0].bi_bh);
- }
- log_err( _("The bitmap was fixed.\n"));
- } else {
- log_err( _("The bitmap inconsistency was ignored.\n"));
+ if (old_bitmap_state == new_blockmap_state)
+ return 0;
+
+ if (error_on_dinode && old_bitmap_state == GFS2_BLKST_DINODE &&
+ new_blockmap_state != GFS2_BLKST_FREE) {
+ log_debug(_("Reference as '%s' to block %llu (0x%llx) which "
+ "was marked as dinode. Needs further "
+ "investigation.\n"),
+ allocdesc[sdp->gfs1][new_blockmap_state],
+ (unsigned long long)blk, (unsigned long long)blk);
+ return 1;
+ }
+ /* Keep these messages as short as possible, or the output gets to be
+ huge and unmanageable. */
+ log_err( _("Block %llu (0x%llx) was '%s', should be %s.\n"),
+ (unsigned long long)blk, (unsigned long long)blk,
+ allocdesc[sdp->gfs1][old_bitmap_state],
+ allocdesc[sdp->gfs1][new_blockmap_state]);
+ if (!query( _("Fix the bitmap? (y/n)"))) {
+ log_err( _("The bitmap inconsistency was ignored.\n"));
+ return 0;
+ }
+ /* If the new bitmap state is free (and therefore the old state was
+ not) we have to add to the free space in the rgrp. If the old
+ bitmap state was free (and therefore it no longer is) we have to
+ subtract to the free space. If the type changed from dinode to
+ data or data to dinode, no change in free space. */
+ gfs2_set_bitmap(rgd, blk, new_blockmap_state);
+ if (new_blockmap_state == GFS2_BLKST_FREE) {
+ /* If we're freeing a dinode, get rid of the hash table
+ entries for it. */
+ if (old_bitmap_state == GFS2_BLKST_DINODE ||
+ old_bitmap_state == GFS2_BLKST_UNLINKED) {
+ struct dir_info *dt;
+ struct inode_info *ii;
+
+ dt = dirtree_find(blk);
+ if (dt)
+ dirtree_delete(dt);
+ ii = inodetree_find(blk);
+ if (ii)
+ inodetree_delete(ii);
+ link1_set(&nlink1map, blk, 0);
}
+ rgd->rg.rg_free++;
+ if (sdp->gfs1)
+ gfs_rgrp_out((struct gfs_rgrp *)&rgd->rg,
+ rgd->bits[0].bi_bh);
+ else
+ gfs2_rgrp_out_bh(&rgd->rg, rgd->bits[0].bi_bh);
+ } else if (old_bitmap_state == GFS2_BLKST_FREE) {
+ rgd->rg.rg_free--;
+ if (sdp->gfs1)
+ gfs_rgrp_out((struct gfs_rgrp *)&rgd->rg, rgd->bits[0].bi_bh);
+ else
+ gfs2_rgrp_out_bh(&rgd->rg, rgd->bits[0].bi_bh);
}
+ log_err( _("The bitmap was fixed.\n"));
return 0;
}
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 26/40] fsck.gfs2: adjust rgrp inode count when fixing bitmap
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (24 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 25/40] fsck.gfs2: Refactor check_n_fix_bitmap to make it more readable Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 27/40] fsck.gfs2: blocks cannot be UNLINKED in pass1b or after that Bob Peterson
` (15 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
Before this patch, fsck.gfs2 would adjust the rgrp bitmaps and
rgrp free space when freeing a block. Later, in pass5, it would
fix the rgrp count of dinodes as needed. Now that pass5 is done
immediately after pass1, we need the other passes to make sure
to keep the dinode count in the rgrps in sync. Therefore, we
adjust it as we go.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/metawalk.c | 45 +++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 43 insertions(+), 2 deletions(-)
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 9ec4bea..2d52217 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -34,6 +34,7 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
{
int old_bitmap_state;
struct rgrp_tree *rgd;
+ int treat_as_inode = 0;
const char *allocdesc[2][5] = { /* gfs2 descriptions */
{"free", "data", "unlinked", "inode", "reserved"},
/* gfs1 descriptions: */
@@ -85,11 +86,33 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
struct inode_info *ii;
dt = dirtree_find(blk);
- if (dt)
+ if (dt) {
dirtree_delete(dt);
+ treat_as_inode = 1;
+ }
ii = inodetree_find(blk);
- if (ii)
+ if (ii) {
inodetree_delete(ii);
+ treat_as_inode = 1;
+ } else if (!sdp->gfs1) {
+ treat_as_inode = 1;
+ } else {
+ /* This is a GFS1 fs (so all metadata is marked
+ inode). We need to verify it is an inode
+ before we can decr the rgrp inode count. */
+ if (link1_type(&nlink1map, blk) == 1)
+ treat_as_inode = 1;
+ }
+ if (old_bitmap_state == GFS2_BLKST_DINODE) {
+ if (treat_as_inode && rgd->rg.rg_dinodes > 0)
+ rgd->rg.rg_dinodes--;
+ else if (sdp->gfs1) {
+ struct gfs_rgrp *gfs1rg =
+ (struct gfs_rgrp *)&rgd->rg;
+ if (gfs1rg->rg_usedmeta > 0)
+ gfs1rg->rg_usedmeta--;
+ }
+ }
link1_set(&nlink1map, blk, 0);
}
rgd->rg.rg_free++;
@@ -99,6 +122,24 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
else
gfs2_rgrp_out_bh(&rgd->rg, rgd->bits[0].bi_bh);
} else if (old_bitmap_state == GFS2_BLKST_FREE) {
+ if (!sdp->gfs1) {
+ treat_as_inode = 1;
+ } else {
+ /* This is a GFS1 fs (so all metadata is marked inode).
+ We need to verify it is an inode before we can decr
+ the rgrp inode count. */
+ if (link1_type(&nlink1map, blk) == 1)
+ treat_as_inode = 1;
+ }
+ if (new_blockmap_state == GFS2_BLKST_DINODE) {
+ if (treat_as_inode)
+ rgd->rg.rg_dinodes++;
+ else if (sdp->gfs1) {
+ struct gfs_rgrp *gfs1rg =
+ (struct gfs_rgrp *)&rgd->rg;
+ gfs1rg->rg_usedmeta++;
+ }
+ }
rgd->rg.rg_free--;
if (sdp->gfs1)
gfs_rgrp_out((struct gfs_rgrp *)&rgd->rg, rgd->bits[0].bi_bh);
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 27/40] fsck.gfs2: blocks cannot be UNLINKED in pass1b or after that
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (25 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 26/40] fsck.gfs2: adjust rgrp inode count when fixing bitmap Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 28/40] fsck.gfs2: Add error checks to get_next_leaf Bob Peterson
` (14 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch basically changes some checks for blocks in an UNLINKED
state to checking the FREE state instead. If any blocks were in the
UNLINKED state, they would have already been resolved by the time
we get to pass1b, and therefore the checks can only be in error.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass1b.c | 18 +++++++++---------
gfs2/fsck/pass2.c | 10 +---------
gfs2/fsck/pass3.c | 2 +-
gfs2/fsck/pass4.c | 2 +-
4 files changed, 12 insertions(+), 20 deletions(-)
diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
index d6ab285..f5f286a 100644
--- a/gfs2/fsck/pass1b.c
+++ b/gfs2/fsck/pass1b.c
@@ -358,7 +358,7 @@ static void resolve_dup_references(struct gfs2_sbd *sdp, struct duptree *dt,
0);
fsck_bitmap_set(ip, ip->i_di.di_num.no_addr,
_("duplicate referencing bad"),
- GFS2_BLKST_UNLINKED);
+ GFS2_BLKST_FREE);
/* We delete the dup_handler inode count and
duplicate id BEFORE clearing the metadata,
because if this is the last reference to
@@ -574,13 +574,11 @@ static void resolve_last_reference(struct gfs2_sbd *sdp, struct duptree *dt,
clone_dup_ref_in_inode(ip, dt);
q = bitmap_type(sdp, id->block_no);
- if (q == GFS2_BLKST_UNLINKED) {
- log_debug( _("The remaining reference inode %lld (0x%llx) is "
- "marked invalid: Marking the block as free.\n"),
+ if (q == GFS2_BLKST_FREE) {
+ log_debug( _("The remaining reference inode %lld (0x%llx) was "
+ "already marked free.\n"),
(unsigned long long)id->block_no,
(unsigned long long)id->block_no);
- fsck_bitmap_set(ip, dt->block, _("reference-repaired leaf"),
- GFS2_BLKST_FREE);
} else if (id->reftypecount[ref_is_inode]) {
set_ip_bitmap(ip);
} else if (id->reftypecount[ref_as_data]) {
@@ -941,11 +939,13 @@ int pass1b(struct gfs2_sbd *sdp)
if (q == GFS2_BLKST_FREE || q == GFS2_BLKST_USED || q < 0)
continue;
- if (q == GFS2_BLKST_UNLINKED)
- log_debug( _("Checking invalidated duplicate block "
- "%lld (0x%llx)\n"),
+ if (q == GFS2_BLKST_UNLINKED) {
+ log_debug( _("Error: block %lld (0x%llx) is still "
+ "marked UNLINKED.\n"),
(unsigned long long)i,
(unsigned long long)i);
+ return FSCK_ERROR;
+ }
warm_fuzzy_stuff(i);
if (find_block_ref(sdp, i) < 0) {
diff --git a/gfs2/fsck/pass2.c b/gfs2/fsck/pass2.c
index d4f50d2..ffd58dc 100644
--- a/gfs2/fsck/pass2.c
+++ b/gfs2/fsck/pass2.c
@@ -1889,7 +1889,7 @@ static int check_system_dir(struct gfs2_inode *sysinode, const char *dirname,
ds.q = bitmap_type(sysinode->i_sbd, iblock);
pass2_fxns.private = (void *) &ds;
- if (ds.q == GFS2_BLKST_UNLINKED) {
+ if (ds.q == GFS2_BLKST_FREE) {
/* First check that the directory's metatree is valid */
error = check_metatree(sysinode, &pass2_fxns);
if (error < 0) {
@@ -1996,14 +1996,6 @@ static int pass2_check_dir(struct gfs2_sbd *sdp, struct gfs2_inode *ip)
int error;
pass2_fxns.private = &ds;
- if (ds.q == GFS2_BLKST_UNLINKED) {
- /* First check that the directory's metatree is valid */
- error = check_metatree(ip, &pass2_fxns);
- if (error < 0) {
- stack;
- return error;
- }
- }
error = check_dir(sdp, ip, &pass2_fxns);
if (skip_this_pass || fsck_abort) /* if asked to skip the rest */
return FSCK_OK;
diff --git a/gfs2/fsck/pass3.c b/gfs2/fsck/pass3.c
index 2fa0bdb..fbf8318 100644
--- a/gfs2/fsck/pass3.c
+++ b/gfs2/fsck/pass3.c
@@ -239,7 +239,7 @@ int pass3(struct gfs2_sbd *sdp)
}
q = bitmap_type(sdp, di->dinode.no_addr);
ip = fsck_load_inode(sdp, di->dinode.no_addr);
- if (q == GFS2_BLKST_UNLINKED) {
+ if (q == GFS2_BLKST_FREE) {
log_err( _("Found unlinked directory "
"containing bad block at block %llu"
" (0x%llx)\n"),
diff --git a/gfs2/fsck/pass4.c b/gfs2/fsck/pass4.c
index 16339cc..313ff0f 100644
--- a/gfs2/fsck/pass4.c
+++ b/gfs2/fsck/pass4.c
@@ -61,7 +61,7 @@ static int handle_unlinked(struct gfs2_sbd *sdp, uint64_t no_addr,
log_err( _("Found unlinked inode at %llu (0x%llx)\n"),
(unsigned long long)no_addr, (unsigned long long)no_addr);
q = bitmap_type(sdp, no_addr);
- if (q == GFS2_BLKST_UNLINKED) {
+ if (q == GFS2_BLKST_FREE) {
log_err( _("Unlinked inode %llu (0x%llx) contains bad "
"blocks\n"), (unsigned long long)no_addr,
(unsigned long long)no_addr);
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 28/40] fsck.gfs2: Add error checks to get_next_leaf
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (26 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 27/40] fsck.gfs2: blocks cannot be UNLINKED in pass1b or after that Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 29/40] fsck.gfs2: re-add a non-allocating repair_leaf to pass1 Bob Peterson
` (13 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch adds some error checking to function get_next_leaf.
Since a leaf block's "lf_next" pointer happens to have the same
block offset as an inode's inode number, we can otherwise get into
an infinite loop whereby a corrupt file system has a leaf block
mistakenly pointing to a dinode, and the dinode has what appears
to be a "lf_next" that points to itself.
I added two checks: First, to guard against situations where a
leaf block points to itself as lf_next, and second, to guard against
situations where a lf_next points to a non-leaf block.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/libgfs2/fs_ops.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/gfs2/libgfs2/fs_ops.c b/gfs2/libgfs2/fs_ops.c
index 2ae27d1..a43a973 100644
--- a/gfs2/libgfs2/fs_ops.c
+++ b/gfs2/libgfs2/fs_ops.c
@@ -1139,7 +1139,18 @@ static int get_next_leaf(struct gfs2_inode *dip,struct gfs2_buffer_head *bh_in,
if (!leaf->lf_next)
return -1;
+ /* Check for a leaf that points to itself as "next" */
+ if (be64_to_cpu(leaf->lf_next) == bh_in->b_blocknr)
+ return -1;
*bh_out = bread(dip->i_sbd, be64_to_cpu(leaf->lf_next));
+ if (*bh_out == NULL)
+ return -ENOENT;
+ /* Check for a leaf pointing to a non-leaf */
+ if (gfs2_check_meta(*bh_out, GFS2_METATYPE_LF)) {
+ brelse(*bh_out);
+ *bh_out = NULL;
+ return -ENOENT;
+ }
return 0;
}
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 29/40] fsck.gfs2: re-add a non-allocating repair_leaf to pass1
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (27 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 28/40] fsck.gfs2: Add error checks to get_next_leaf Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 30/40] libgfs2: Allocate new GFS1 metadata as type 3, not type 1 Bob Peterson
` (12 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
In an earlier patch, the repair_leaf function was removed from
pass1. This turns out to cause a problem where there is leaf block
corruption. This patch adds back in a non-allocating function so
things work as they used to.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass1.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index ee20b9e..6908ac7 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -142,6 +142,52 @@ static int delete_block(struct gfs2_inode *ip, uint64_t block,
return -1;
}
+/* This is a pass1-specific leaf repair. Since we are not allowed to do
+ * block allocations, we do what we can. */
+static int pass1_repair_leaf(struct gfs2_inode *ip, uint64_t *leaf_no,
+ int lindex, int ref_count, const char *msg)
+{
+ uint64_t *cpyptr;
+ char *padbuf;
+ int pad_size, i;
+
+ log_err( _("Directory Inode %llu (0x%llx) points to leaf %llu"
+ " (0x%llx) %s.\n"),
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)*leaf_no,
+ (unsigned long long)*leaf_no, msg);
+ if (!query( _("Attempt to patch around it? (y/n) "))) {
+ log_err( _("Bad leaf left in place.\n"));
+ goto out;
+ }
+
+ padbuf = malloc(ref_count * sizeof(uint64_t));
+ cpyptr = (uint64_t *)padbuf;
+ for (i = 0; i < ref_count; i++) {
+ *cpyptr = 0;
+ cpyptr++;
+ }
+ pad_size = ref_count * sizeof(uint64_t);
+ log_err(_("Writing zeros to the hash table of directory %lld "
+ "(0x%llx)@index: 0x%x for 0x%x pointers.\n"),
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)ip->i_di.di_num.no_addr, lindex,
+ ref_count);
+ if (ip->i_sbd->gfs1)
+ gfs1_writei(ip, padbuf, lindex * sizeof(uint64_t), pad_size);
+ else
+ gfs2_writei(ip, padbuf, lindex * sizeof(uint64_t), pad_size);
+ free(padbuf);
+ log_err( _("Directory Inode %llu (0x%llx) patched.\n"),
+ (unsigned long long)ip->i_di.di_num.no_addr,
+ (unsigned long long)ip->i_di.di_num.no_addr);
+
+out:
+ *leaf_no = 0;
+ return 0;
+}
+
struct metawalk_fxns pass1_fxns = {
.private = NULL,
.check_leaf = p1check_leaf,
@@ -153,6 +199,7 @@ struct metawalk_fxns pass1_fxns = {
.check_eattr_entry = check_eattr_entries,
.check_eattr_extentry = check_extended_leaf_eattr,
.big_file_msg = big_file_comfort,
+ .repair_leaf = pass1_repair_leaf,
.undo_check_meta = undo_check_metalist,
.undo_check_data = undo_check_data,
.delete_block = delete_block,
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 30/40] libgfs2: Allocate new GFS1 metadata as type 3, not type 1
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (28 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 29/40] fsck.gfs2: re-add a non-allocating repair_leaf to pass1 Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 31/40] fsck.gfs2: Undo partially done metadata records Bob Peterson
` (11 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
GFS1 and GFS2 treat metadata differently. In GFS2, only dinodes are
marked as type 3 (dinode). All other GFS2 metadata is considered
data (type 1). In GFS1, all metadata is considered "metadata" which
is type 3. So this patch makes function lgfs2_meta_alloc set the
block type in the bitmap according to the sdp->gfs1 setting.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/libgfs2/fs_ops.c | 22 +++++++++++++++-------
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/gfs2/libgfs2/fs_ops.c b/gfs2/libgfs2/fs_ops.c
index a43a973..3acfb67 100644
--- a/gfs2/libgfs2/fs_ops.c
+++ b/gfs2/libgfs2/fs_ops.c
@@ -137,7 +137,7 @@ static uint64_t find_free_block(struct rgrp_tree *rgd)
return blkno;
}
-static int blk_alloc_in_rg(struct gfs2_sbd *sdp, unsigned state, struct rgrp_tree *rgd, uint64_t blkno)
+static int blk_alloc_in_rg(struct gfs2_sbd *sdp, unsigned state, struct rgrp_tree *rgd, uint64_t blkno, int dinode)
{
if (blkno == 0)
return -1;
@@ -145,8 +145,14 @@ static int blk_alloc_in_rg(struct gfs2_sbd *sdp, unsigned state, struct rgrp_tre
if (gfs2_set_bitmap(rgd, blkno, state))
return -1;
- if (state == GFS2_BLKST_DINODE)
- rgd->rg.rg_dinodes++;
+ if (state == GFS2_BLKST_DINODE) {
+ struct gfs_rgrp *gfs1rg = (struct gfs_rgrp *)&rgd->rg;
+
+ if (dinode)
+ rgd->rg.rg_dinodes++;
+ else if (sdp->gfs1)
+ gfs1rg->rg_usedmeta++;
+ }
rgd->rg.rg_free--;
if (sdp->gfs1)
@@ -163,7 +169,7 @@ static int blk_alloc_in_rg(struct gfs2_sbd *sdp, unsigned state, struct rgrp_tre
* resource group with blksreq free blocks but only allocate the one block.
* Returns 0 on success with the allocated block number in *blkno or non-zero otherwise.
*/
-static int block_alloc(struct gfs2_sbd *sdp, const uint64_t blksreq, int state, uint64_t *blkno)
+static int block_alloc(struct gfs2_sbd *sdp, const uint64_t blksreq, int state, uint64_t *blkno, int dinode)
{
int ret;
int release = 0;
@@ -186,7 +192,7 @@ static int block_alloc(struct gfs2_sbd *sdp, const uint64_t blksreq, int state,
}
bn = find_free_block(rgt);
- ret = blk_alloc_in_rg(sdp, state, rgt, bn);
+ ret = blk_alloc_in_rg(sdp, state, rgt, bn, dinode);
if (release)
gfs2_rgrp_relse(rgt);
*blkno = bn;
@@ -195,7 +201,7 @@ static int block_alloc(struct gfs2_sbd *sdp, const uint64_t blksreq, int state,
int lgfs2_dinode_alloc(struct gfs2_sbd *sdp, const uint64_t blksreq, uint64_t *blkno)
{
- int ret = block_alloc(sdp, blksreq, GFS2_BLKST_DINODE, blkno);
+ int ret = block_alloc(sdp, blksreq, GFS2_BLKST_DINODE, blkno, TRUE);
if (ret == 0)
sdp->dinodes_alloced++;
return ret;
@@ -203,7 +209,9 @@ int lgfs2_dinode_alloc(struct gfs2_sbd *sdp, const uint64_t blksreq, uint64_t *b
int lgfs2_meta_alloc(struct gfs2_inode *ip, uint64_t *blkno)
{
- int ret = block_alloc(ip->i_sbd, 1, GFS2_BLKST_USED, blkno);
+ int ret = block_alloc(ip->i_sbd, 1,
+ ip->i_sbd->gfs1 ? GFS2_BLKST_DINODE :
+ GFS2_BLKST_USED, blkno, FALSE);
if (ret == 0) {
ip->i_di.di_goal_meta = *blkno;
bmodified(ip->i_bh);
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 31/40] fsck.gfs2: Undo partially done metadata records
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (29 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 30/40] libgfs2: Allocate new GFS1 metadata as type 3, not type 1 Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 32/40] fsck.gfs2: Eliminate redundant code in _fsck_bitmap_set Bob Peterson
` (10 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch fixes a problem with duplicate block processing where
one reference is as metadata and another is not (reference as data
or ea or something). The problem is that the metadata block may have
been processed as metadata (for example, an indirect block pointing
to a bunch of data blocks, so all blocks it references are now
marked as metadata) but the last resolved duplicate is not as
metadata, so the sub-blocks are never reprocessed, or "undone".
This patch makes it so that basically if the first reference is
as metadata (so we know we processed its sub-blocks) and the inode
referencing it as metadata is the LAST inode to reference it as
metadata, the sub-blocks are sent through "undo" processing.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/afterpass1_common.c | 33 ++++++++++++++++++++++++++++-----
gfs2/fsck/afterpass1_common.h | 2 --
gfs2/fsck/metawalk.c | 43 ++++++++++++++++++++++++++++++++++---------
gfs2/fsck/util.c | 22 ++++++++++++++++++++++
gfs2/fsck/util.h | 2 +-
5 files changed, 85 insertions(+), 17 deletions(-)
diff --git a/gfs2/fsck/afterpass1_common.c b/gfs2/fsck/afterpass1_common.c
index bdea242..95fdbf8 100644
--- a/gfs2/fsck/afterpass1_common.c
+++ b/gfs2/fsck/afterpass1_common.c
@@ -22,10 +22,13 @@
*
* Returns: 1 if there are any remaining references to this block, else 0.
*/
-int find_remove_dup(struct gfs2_inode *ip, uint64_t block, const char *btype)
+static int find_remove_dup(struct gfs2_inode *ip, uint64_t block,
+ const char *btype, int *removed_last_meta)
{
struct duptree *dt;
struct inode_with_dups *id;
+ int deleted_a_meta_ref = 0;
+ int meta_refs_left = 0;
dt = dupfind(block);
if (!dt)
@@ -36,15 +39,30 @@ int find_remove_dup(struct gfs2_inode *ip, uint64_t block, const char *btype)
if (!id)
goto more_refs;
+ if (id->reftypecount[ref_as_meta])
+ deleted_a_meta_ref = 1;
dup_listent_delete(dt, id);
if (dt->refs == 0) {
log_info( _("This was the last reference: it's no longer a "
"duplicate.\n"));
dup_delete(dt); /* not duplicate now */
+ if (deleted_a_meta_ref) {
+ log_debug("Removed the last reference as metadata.\n");
+ *removed_last_meta = 1;
+ }
return 0;
+ } else if (deleted_a_meta_ref) {
+ /* If we deleted a metadata reference, see if there are more
+ references as meta, or if it was the last one. */
+ meta_refs_left = count_dup_meta_refs(dt);
}
more_refs:
- log_info( _("%d block reference(s) remain.\n"), dt->refs);
+ log_info(_("%d block reference(s) remain (%d as metadata).\n"),
+ dt->refs, meta_refs_left);
+ if (deleted_a_meta_ref && meta_refs_left == 0) {
+ log_debug("Removed the last reference as metadata.\n");
+ *removed_last_meta = 1;
+ }
return 1; /* references still exist so do not free the block. */
}
@@ -61,6 +79,7 @@ static int delete_block_if_notdup(struct gfs2_inode *ip, uint64_t block,
void *private)
{
int q;
+ int removed_lastmeta;
if (!valid_block(ip->i_sbd, block))
return meta_error;
@@ -75,9 +94,13 @@ static int delete_block_if_notdup(struct gfs2_inode *ip, uint64_t block,
(unsigned long long)ip->i_di.di_num.no_addr);
return meta_is_good;
}
- if (find_remove_dup(ip, block, btype)) { /* a dup */
- if (was_duplicate)
- *was_duplicate = 1;
+ if (find_remove_dup(ip, block, btype, &removed_lastmeta)) { /* a dup */
+ if (was_duplicate) {
+ if (removed_lastmeta)
+ log_debug("Removed last reference as meta.\n");
+ else
+ *was_duplicate = 1;
+ }
log_err( _("Not clearing duplicate reference in inode "
"at block #%llu (0x%llx) to block #%llu (0x%llx) "
"because it's referenced by another inode.\n"),
diff --git a/gfs2/fsck/afterpass1_common.h b/gfs2/fsck/afterpass1_common.h
index 8b345ee..829828f 100644
--- a/gfs2/fsck/afterpass1_common.h
+++ b/gfs2/fsck/afterpass1_common.h
@@ -26,8 +26,6 @@ extern int delete_eattr_extentry(struct gfs2_inode *ip, int i,
struct gfs2_ea_header *ea_hdr,
struct gfs2_ea_header *ea_hdr_prev,
void *private);
-extern int find_remove_dup(struct gfs2_inode *ip, uint64_t block,
- const char *btype);
extern int remove_dentry_from_dir(struct gfs2_sbd *sdp, uint64_t dir,
uint64_t dentryblock);
#endif
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 2d52217..8a1748b 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -1217,7 +1217,7 @@ static int build_and_check_metalist(struct gfs2_inode *ip, osi_list_t *mlp,
struct gfs2_buffer_head *bh, *nbh, *metabh = ip->i_bh;
osi_list_t *prev_list, *cur_list, *tmp;
int h, head_size, iblk_type;
- uint64_t *ptr, block;
+ uint64_t *ptr, block, *undoptr;
int error, was_duplicate, is_valid;
int maxptrs;
@@ -1297,7 +1297,7 @@ static int build_and_check_metalist(struct gfs2_inode *ip, osi_list_t *mlp,
"(0x%llx).\n"),
(unsigned long long)block,
(unsigned long long)block);
- return error;
+ goto error_undo;
}
if (error == meta_skip_further) {
log_info(_("\nUnrecoverable metadata "
@@ -1306,18 +1306,24 @@ static int build_and_check_metalist(struct gfs2_inode *ip, osi_list_t *mlp,
" will be skipped.\n"),
(unsigned long long)block,
(unsigned long long)block);
- return error;
+ goto error_undo;
}
if (!is_valid) {
log_debug( _("Skipping rejected block "
"%llu (0x%llx)\n"),
(unsigned long long)block,
(unsigned long long)block);
- if (pass->invalid_meta_is_fatal)
- return meta_error;
-
+ if (pass->invalid_meta_is_fatal) {
+ error = meta_error;
+ goto error_undo;
+ }
continue;
}
+ /* Note that there's a special case in which
+ we need to process the metadata block, even
+ if it was a duplicate. That's for cases
+ where we deleted the last reference as
+ metadata. */
if (was_duplicate) {
log_debug( _("Skipping duplicate %llu "
"(0x%llx)\n"),
@@ -1330,9 +1336,10 @@ static int build_and_check_metalist(struct gfs2_inode *ip, osi_list_t *mlp,
"%lld (0x%llx)\n"),
(unsigned long long)block,
(unsigned long long)block);
- if (pass->invalid_meta_is_fatal)
- return meta_error;
-
+ if (pass->invalid_meta_is_fatal) {
+ error = meta_error;
+ goto error_undo;
+ }
continue;
}
if (!nbh)
@@ -1342,6 +1349,24 @@ static int build_and_check_metalist(struct gfs2_inode *ip, osi_list_t *mlp,
} /* for blocks at that height */
} /* for height */
return 0;
+
+error_undo: /* undo what we've done so far for this block */
+ if (pass->undo_check_meta == NULL)
+ return error;
+
+ log_info(_("Undoing the work we did before the error on block %llu "
+ "(0x%llx).\n"), (unsigned long long)bh->b_blocknr,
+ (unsigned long long)bh->b_blocknr);
+ for (undoptr = (uint64_t *)(bh->b_data + head_size); undoptr < ptr &&
+ (char *)undoptr < (bh->b_data + ip->i_sbd->bsize);
+ undoptr++) {
+ if (!*undoptr)
+ continue;
+
+ block = be64_to_cpu(*undoptr);
+ pass->undo_check_meta(ip, block, h, pass->private);
+ }
+ return error;
}
/**
diff --git a/gfs2/fsck/util.c b/gfs2/fsck/util.c
index 2e77000..8a8220b 100644
--- a/gfs2/fsck/util.c
+++ b/gfs2/fsck/util.c
@@ -293,6 +293,28 @@ struct inode_with_dups *find_dup_ref_inode(struct duptree *dt,
return NULL;
}
+/**
+ * count_dup_meta_refs - count the number of remaining references as metadata
+ */
+int count_dup_meta_refs(struct duptree *dt)
+{
+ osi_list_t *ref;
+ struct inode_with_dups *id;
+ int metarefs = 0;
+
+ osi_list_foreach(ref, &dt->ref_invinode_list) {
+ id = osi_list_entry(ref, struct inode_with_dups, list);
+ if (id->reftypecount[ref_as_meta])
+ metarefs++;
+ }
+ osi_list_foreach(ref, &dt->ref_inode_list) {
+ id = osi_list_entry(ref, struct inode_with_dups, list);
+ if (id->reftypecount[ref_as_meta])
+ metarefs++;
+ }
+ return metarefs;
+}
+
/*
* add_duplicate_ref - Add a duplicate reference to the duplicates tree list
* A new element of the tree will be created as needed
diff --git a/gfs2/fsck/util.h b/gfs2/fsck/util.h
index 4545594..d93b65d 100644
--- a/gfs2/fsck/util.h
+++ b/gfs2/fsck/util.h
@@ -20,7 +20,7 @@ int add_duplicate_ref(struct gfs2_inode *ip, uint64_t block,
extern struct inode_with_dups *find_dup_ref_inode(struct duptree *dt,
struct gfs2_inode *ip);
extern void dup_listent_delete(struct duptree *dt, struct inode_with_dups *id);
-
+extern int count_dup_meta_refs(struct duptree *dt);
extern const char *reftypes[ref_types + 1];
#define BLOCKMAP_SIZE1(size) ((size) >> 3)
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 32/40] fsck.gfs2: Eliminate redundant code in _fsck_bitmap_set
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (30 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 31/40] fsck.gfs2: Undo partially done metadata records Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 33/40] fsck.gfs2: Fix inode counting bug Bob Peterson
` (9 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
Function _fsck_bitmap_set was checking if the block is unlinked,
and printing out messages identical to the alternative code path.
Therefore, the redundant code was eliminated.
rhbz#1268045
---
gfs2/fsck/metawalk.c | 11 -----------
1 file changed, 11 deletions(-)
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 8a1748b..0df54df 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -185,17 +185,6 @@ int _fsck_bitmap_set(struct gfs2_inode *ip, uint64_t bblock,
(unsigned long long)ip->i_di.di_num.no_addr,
block_type_string(mark));
- } else if (mark == GFS2_BLKST_UNLINKED) {
- if (prevcount) {
- log_info("\n");
- prevcount = 0;
- }
- printf( _("(%s:%d) inode (0x%llx) references %s block"
- " (0x%llx): marking as '%s'\n"),
- caller, fline,
- (unsigned long long)ip->i_di.di_num.no_addr,
- btype, (unsigned long long)bblock,
- block_type_string(mark));
} else {
if (prevcount) {
log_info("\n");
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 33/40] fsck.gfs2: Fix inode counting bug
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (31 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 32/40] fsck.gfs2: Eliminate redundant code in _fsck_bitmap_set Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 34/40] fsck.gfs2: Adjust bitmap for lost+found after adding to dirtree Bob Peterson
` (8 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch fixes a previous patch that caused the inode count to
get off when repairing GFS1 file systems with corrupt system dinodes.
rhbz#1268045
---
gfs2/fsck/metawalk.c | 46 +++++++++++++++++++++++++++-------------------
1 file changed, 27 insertions(+), 19 deletions(-)
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 0df54df..668690f 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -35,12 +35,14 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
int old_bitmap_state;
struct rgrp_tree *rgd;
int treat_as_inode = 0;
+ struct gfs_rgrp *gfs1rg;
const char *allocdesc[2][5] = { /* gfs2 descriptions */
{"free", "data", "unlinked", "inode", "reserved"},
/* gfs1 descriptions: */
{"free", "data", "free meta", "metadata", "reserved"}};
rgd = gfs2_blk2rgrpd(sdp, blk);
+ gfs1rg = (struct gfs_rgrp *)&rgd->rg;
old_bitmap_state = lgfs2_get_bitmap(sdp, blk, rgd);
if (old_bitmap_state < 0) {
@@ -106,12 +108,8 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
if (old_bitmap_state == GFS2_BLKST_DINODE) {
if (treat_as_inode && rgd->rg.rg_dinodes > 0)
rgd->rg.rg_dinodes--;
- else if (sdp->gfs1) {
- struct gfs_rgrp *gfs1rg =
- (struct gfs_rgrp *)&rgd->rg;
- if (gfs1rg->rg_usedmeta > 0)
- gfs1rg->rg_usedmeta--;
- }
+ if (sdp->gfs1 && gfs1rg->rg_usedmeta > 0)
+ gfs1rg->rg_usedmeta--;
}
link1_set(&nlink1map, blk, 0);
}
@@ -122,23 +120,33 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
else
gfs2_rgrp_out_bh(&rgd->rg, rgd->bits[0].bi_bh);
} else if (old_bitmap_state == GFS2_BLKST_FREE) {
- if (!sdp->gfs1) {
- treat_as_inode = 1;
- } else {
- /* This is a GFS1 fs (so all metadata is marked inode).
- We need to verify it is an inode before we can decr
- the rgrp inode count. */
- if (link1_type(&nlink1map, blk) == 1)
- treat_as_inode = 1;
- }
if (new_blockmap_state == GFS2_BLKST_DINODE) {
+ if (!sdp->gfs1) {
+ treat_as_inode = 1;
+ } else {
+ /* This is GFS1 (so all metadata is marked
+ inode). We need to verify it is an inode
+ before we can decr the rgrp inode count. */
+ if (link1_type(&nlink1map, blk) == 1)
+ treat_as_inode = 1;
+ else {
+ struct dir_info *dt;
+ struct inode_info *ii;
+
+ dt = dirtree_find(blk);
+ if (dt)
+ treat_as_inode = 1;
+ else {
+ ii = inodetree_find(blk);
+ if (ii)
+ treat_as_inode = 1;
+ }
+ }
+ }
if (treat_as_inode)
rgd->rg.rg_dinodes++;
- else if (sdp->gfs1) {
- struct gfs_rgrp *gfs1rg =
- (struct gfs_rgrp *)&rgd->rg;
+ if (sdp->gfs1)
gfs1rg->rg_usedmeta++;
- }
}
rgd->rg.rg_free--;
if (sdp->gfs1)
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 34/40] fsck.gfs2: Adjust bitmap for lost+found after adding to dirtree
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (32 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 33/40] fsck.gfs2: Fix inode counting bug Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 35/40] GFS2: Add initialization checks for GFS1 used metadata Bob Peterson
` (7 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch changes the order in which things are done in function
make_sure_lf_exists. Before, it was setting the bitmap and then
adding lost+found to the directory tree. Now the order is reversed:
lost+found is added to the directory tree, then the bitmap is
twiddled. Doing it in this order ensures that the block is treated
as an inode block for the purposes of adjusting the rgrp counts.
This is only a problem for GFS1 file systems where a bitmap
designation might mean either dinode or other metadata.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/lost_n_found.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/gfs2/fsck/lost_n_found.c b/gfs2/fsck/lost_n_found.c
index 197ae7d..bd05223 100644
--- a/gfs2/fsck/lost_n_found.c
+++ b/gfs2/fsck/lost_n_found.c
@@ -139,9 +139,12 @@ void make_sure_lf_exists(struct gfs2_inode *ip)
/* FIXME: i'd feel better about this if fs_mkdir returned
whether it created a new directory or just found an old one,
and we used that instead of the bitmap_type to run this */
+ dirtree_insert(lf_dip->i_di.di_num);
+ /* Set the bitmap AFTER the dirtree insert so that function
+ check_n_fix_bitmap will realize it's a dinode and adjust
+ the rgrp counts properly. */
fsck_bitmap_set(ip, lf_dip->i_di.di_num.no_addr,
_("lost+found dinode"), GFS2_BLKST_DINODE);
- dirtree_insert(lf_dip->i_di.di_num);
/* root inode links to lost+found */
incr_link_count(sdp->md.rooti->i_di.di_num, lf_dip, _("root"));
/* lost+found link for '.' from itself */
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 35/40] GFS2: Add initialization checks for GFS1 used metadata
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (33 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 34/40] fsck.gfs2: Adjust bitmap for lost+found after adding to dirtree Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 36/40] fsck.gfs2: Use BLKST constants to make pass5 more clear Bob Peterson
` (6 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
Before this patch, fsck.gfs2's initialization code that verified
rgrp counts was not counting the "used metadata" count used in
GFS1 file systems. This patch adds those checks.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/initialize.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 49 insertions(+), 3 deletions(-)
diff --git a/gfs2/fsck/initialize.c b/gfs2/fsck/initialize.c
index d54ab0d..652eec3 100644
--- a/gfs2/fsck/initialize.c
+++ b/gfs2/fsck/initialize.c
@@ -198,14 +198,14 @@ static void check_rgrp_integrity(struct gfs2_sbd *sdp, struct rgrp_tree *rgd,
int *fixit, int *this_rg_fixed,
int *this_rg_bad, int *this_rg_cleaned)
{
- uint32_t rg_free, rg_reclaimed, rg_unlinked;
+ uint32_t rg_free, rg_reclaimed, rg_unlinked, rg_usedmeta, rg_useddi;
int rgb, x, y, off, bytes_to_check, total_bytes_to_check, asked = 0;
unsigned int state;
struct gfs_rgrp *gfs1rg = (struct gfs_rgrp *)&rgd->rg;
uint64_t diblock;
struct gfs2_buffer_head *bh;
- rg_free = rg_reclaimed = rg_unlinked = 0;
+ rg_free = rg_reclaimed = rg_unlinked = rg_usedmeta = rg_useddi = 0;
total_bytes_to_check = rgd->ri.ri_bitbytes;
*this_rg_fixed = *this_rg_bad = *this_rg_cleaned = 0;
@@ -241,6 +241,15 @@ static void check_rgrp_integrity(struct gfs2_sbd *sdp, struct rgrp_tree *rgd,
continue;
}
if (state == GFS2_BLKST_DINODE) {
+ if (sdp->gfs1) {
+ bh = bread(sdp, diblock);
+ if (!gfs2_check_meta(bh,
+ GFS2_METATYPE_DI))
+ rg_useddi++;
+ else
+ rg_usedmeta++;
+ brelse(bh);
+ }
diblock++;
continue;
}
@@ -344,7 +353,10 @@ static void check_rgrp_integrity(struct gfs2_sbd *sdp, struct rgrp_tree *rgd,
} else
log_err( _("The rgrp was not fixed.\n"));
}
- if (sdp->gfs1 && gfs1rg->rg_freemeta != rg_unlinked) {
+ if (!sdp->gfs1)
+ return;
+
+ if (gfs1rg->rg_freemeta != rg_unlinked) {
*this_rg_bad = 1;
*this_rg_cleaned = 0;
log_err( _("Error: resource group %lld (0x%llx): "
@@ -360,6 +372,40 @@ static void check_rgrp_integrity(struct gfs2_sbd *sdp, struct rgrp_tree *rgd,
} else
log_err( _("The rgrp was not fixed.\n"));
}
+ if (gfs1rg->rg_useddi != rg_useddi) {
+ *this_rg_bad = 1;
+ *this_rg_cleaned = 0;
+ log_err( _("Error: resource group %lld (0x%llx): used dinode "
+ "count (%d) does not match bitmap (%d)\n"),
+ (unsigned long long)rgd->ri.ri_addr,
+ (unsigned long long)rgd->ri.ri_addr,
+ gfs1rg->rg_useddi, rg_useddi);
+ if (query( _("Fix the rgrp used dinode block count? (y/n)"))) {
+ gfs1rg->rg_useddi = rg_useddi;
+ gfs_rgrp_out((struct gfs_rgrp *)&rgd->rg,
+ rgd->bits[0].bi_bh);
+ *this_rg_fixed = 1;
+ log_err( _("The rgrp was fixed.\n"));
+ } else
+ log_err( _("The rgrp was not fixed.\n"));
+ }
+ if (gfs1rg->rg_usedmeta != rg_usedmeta) {
+ *this_rg_bad = 1;
+ *this_rg_cleaned = 0;
+ log_err( _("Error: resource group %lld (0x%llx): used "
+ "metadata (%d) does not match bitmap (%d)\n"),
+ (unsigned long long)rgd->ri.ri_addr,
+ (unsigned long long)rgd->ri.ri_addr,
+ gfs1rg->rg_usedmeta, rg_usedmeta);
+ if (query( _("Fix the rgrp used meta blocks count? (y/n)"))) {
+ gfs1rg->rg_usedmeta = rg_usedmeta;
+ gfs_rgrp_out((struct gfs_rgrp *)&rgd->rg,
+ rgd->bits[0].bi_bh);
+ *this_rg_fixed = 1;
+ log_err( _("The rgrp was fixed.\n"));
+ } else
+ log_err( _("The rgrp was not fixed.\n"));
+ }
/*
else {
log_debug( _("Resource group %lld (0x%llx) free space "
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 36/40] fsck.gfs2: Use BLKST constants to make pass5 more clear
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (34 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 35/40] GFS2: Add initialization checks for GFS1 used metadata Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 37/40] fsck.gfs2: Fix GFS1 "used meta" accounting bug Bob Peterson
` (5 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch makes no functional changes. It just changes the array
references for "count" use the equivalent BLKST_* constants. This
makes the code more intuitive to understand and read.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass5.c | 35 +++++++++++++++++++----------------
1 file changed, 19 insertions(+), 16 deletions(-)
diff --git a/gfs2/fsck/pass5.c b/gfs2/fsck/pass5.c
index 7d81bfa..bab06be 100644
--- a/gfs2/fsck/pass5.c
+++ b/gfs2/fsck/pass5.c
@@ -145,50 +145,53 @@ static void update_rgrp(struct gfs2_sbd *sdp, struct rgrp_tree *rgp,
}
/* actually adjust counters and write out to disk */
- if (rgp->rg.rg_free != count[0]) {
+ if (rgp->rg.rg_free != count[GFS2_BLKST_FREE]) {
log_err( _("RG #%llu (0x%llx) free count inconsistent: "
"is %u should be %u\n"),
(unsigned long long)rgp->ri.ri_addr,
(unsigned long long)rgp->ri.ri_addr,
- rgp->rg.rg_free, count[0]);
- rgp->rg.rg_free = count[0];
+ rgp->rg.rg_free, count[GFS2_BLKST_FREE]);
+ rgp->rg.rg_free = count[GFS2_BLKST_FREE];
update = 1;
}
- if (rgp->rg.rg_dinodes != count[3]) {
+ if (rgp->rg.rg_dinodes != count[GFS2_BLKST_DINODE]) {
log_err( _("RG #%llu (0x%llx) Inode count inconsistent: is "
"%u should be %u\n"),
(unsigned long long)rgp->ri.ri_addr,
(unsigned long long)rgp->ri.ri_addr,
- rgp->rg.rg_dinodes, count[3]);
- rgp->rg.rg_dinodes = count[3];
+ rgp->rg.rg_dinodes, count[GFS2_BLKST_DINODE]);
+ rgp->rg.rg_dinodes = count[GFS2_BLKST_DINODE];
update = 1;
}
- if (sdp->gfs1 && gfs1rg->rg_usedmeta != count[1]) {
+ if (sdp->gfs1 && gfs1rg->rg_usedmeta != count[GFS2_BLKST_USED]) {
log_err( _("RG #%llu (0x%llx) Used metadata count "
"inconsistent: is %u should be %u\n"),
(unsigned long long)rgp->ri.ri_addr,
(unsigned long long)rgp->ri.ri_addr,
- gfs1rg->rg_usedmeta, count[1]);
- gfs1rg->rg_usedmeta = count[1];
+ gfs1rg->rg_usedmeta, count[GFS2_BLKST_USED]);
+ gfs1rg->rg_usedmeta = count[GFS2_BLKST_USED];
update = 1;
}
- if (sdp->gfs1 && gfs1rg->rg_freemeta != count[2]) {
+ if (sdp->gfs1 && gfs1rg->rg_freemeta != count[GFS2_BLKST_UNLINKED]) {
log_err( _("RG #%llu (0x%llx) Free metadata count "
"inconsistent: is %u should be %u\n"),
(unsigned long long)rgp->ri.ri_addr,
(unsigned long long)rgp->ri.ri_addr,
- gfs1rg->rg_freemeta, count[2]);
- gfs1rg->rg_freemeta = count[2];
+ gfs1rg->rg_freemeta, count[GFS2_BLKST_UNLINKED]);
+ gfs1rg->rg_freemeta = count[GFS2_BLKST_UNLINKED];
update = 1;
}
- if (!sdp->gfs1 && (rgp->ri.ri_data != count[0] + count[1] +
- count[2] + count[3])) {
+ if (!sdp->gfs1 && (rgp->ri.ri_data != count[GFS2_BLKST_FREE] +
+ count[GFS2_BLKST_USED] +
+ count[GFS2_BLKST_UNLINKED] +
+ count[GFS2_BLKST_DINODE])) {
/* FIXME not sure how to handle this case ATM - it
* means that the total number of blocks we've counted
* exceeds the blocks in the rg */
log_err( _("Internal fsck error: %u != %u + %u + %u + %u\n"),
- rgp->ri.ri_data, count[0], count[1], count[2],
- count[3]);
+ rgp->ri.ri_data, count[GFS2_BLKST_FREE],
+ count[GFS2_BLKST_USED], count[GFS2_BLKST_UNLINKED],
+ count[GFS2_BLKST_DINODE]);
exit(FSCK_ERROR);
}
if (update) {
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 37/40] fsck.gfs2: Fix GFS1 "used meta" accounting bug
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (35 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 36/40] fsck.gfs2: Use BLKST constants to make pass5 more clear Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 38/40] fsck.gfs2: pass1b is too noisy wrt gfs1 non-dinode metadata Bob Peterson
` (4 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
Before this patch, fsck.gfs2 was grouping all data blocks and
non-inode metadata blocks in the same category. That's wrong
because the GFS1 "used meta" block count counts data blocks as
metadata blocks. This patch fixes it so that GFS1 "used metadata"
has its own count and the accounting is done properly.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass5.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/gfs2/fsck/pass5.c b/gfs2/fsck/pass5.c
index bab06be..554f39f 100644
--- a/gfs2/fsck/pass5.c
+++ b/gfs2/fsck/pass5.c
@@ -12,6 +12,8 @@
#include "fsck.h"
#include "util.h"
+#define GFS1_BLKST_USEDMETA 4
+
static int check_block_status(struct gfs2_sbd *sdp, struct gfs2_bmap *bl,
char *buffer, unsigned int buflen,
uint64_t *rg_block, uint64_t rg_data,
@@ -50,7 +52,7 @@ static int check_block_status(struct gfs2_sbd *sdp, struct gfs2_bmap *bl,
if (gfs2_check_meta(bh, GFS2_METATYPE_DI) == 0)
count[GFS2_BLKST_DINODE]++;
else
- count[GFS2_BLKST_USED]++;
+ count[GFS1_BLKST_USEDMETA]++;
brelse(bh);
} else {
count[q]++;
@@ -163,13 +165,13 @@ static void update_rgrp(struct gfs2_sbd *sdp, struct rgrp_tree *rgp,
rgp->rg.rg_dinodes = count[GFS2_BLKST_DINODE];
update = 1;
}
- if (sdp->gfs1 && gfs1rg->rg_usedmeta != count[GFS2_BLKST_USED]) {
+ if (sdp->gfs1 && gfs1rg->rg_usedmeta != count[GFS1_BLKST_USEDMETA]) {
log_err( _("RG #%llu (0x%llx) Used metadata count "
"inconsistent: is %u should be %u\n"),
(unsigned long long)rgp->ri.ri_addr,
(unsigned long long)rgp->ri.ri_addr,
- gfs1rg->rg_usedmeta, count[GFS2_BLKST_USED]);
- gfs1rg->rg_usedmeta = count[GFS2_BLKST_USED];
+ gfs1rg->rg_usedmeta, count[GFS1_BLKST_USEDMETA]);
+ gfs1rg->rg_usedmeta = count[GFS1_BLKST_USEDMETA];
update = 1;
}
if (sdp->gfs1 && gfs1rg->rg_freemeta != count[GFS2_BLKST_UNLINKED]) {
@@ -217,7 +219,7 @@ int pass5(struct gfs2_sbd *sdp, struct gfs2_bmap *bl)
{
struct osi_node *n, *next = NULL;
struct rgrp_tree *rgp = NULL;
- uint32_t count[5];
+ uint32_t count[5]; /* we need 5 because of GFS1 usedmeta */
uint64_t rg_count = 0;
/* Reconcile RG bitmaps with fsck bitmap */
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 38/40] fsck.gfs2: pass1b is too noisy wrt gfs1 non-dinode metadata
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (36 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 37/40] fsck.gfs2: Fix GFS1 "used meta" accounting bug Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 39/40] fsck.gfs2: Fix rgrp dinode accounting bug Bob Peterson
` (3 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
For GFS1 file systems, a block marked as metadata may either be a
dinode or any other kind of metadata. For GFS2 file systems, it makes
sense to complain when we find something that we think is a dinode,
but isn't. For GFS1 file systems, non-dinode metadata is a common
thing, so it makes sense to keep quiet about it.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass1b.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
index f5f286a..9d5fc04 100644
--- a/gfs2/fsck/pass1b.c
+++ b/gfs2/fsck/pass1b.c
@@ -877,9 +877,10 @@ static int find_block_ref(struct gfs2_sbd *sdp, uint64_t inode)
/* double-check the meta header just to be sure it's metadata */
if (ip->i_di.di_header.mh_magic != GFS2_MAGIC ||
ip->i_di.di_header.mh_type != GFS2_METATYPE_DI) {
- log_debug( _("Block %lld (0x%llx) is not gfs2 metadata.\n"),
- (unsigned long long)inode,
- (unsigned long long)inode);
+ if (!sdp->gfs1)
+ log_debug( _("Block %lld (0x%llx) is not a dinode.\n"),
+ (unsigned long long)inode,
+ (unsigned long long)inode);
error = 1;
goto out;
}
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 39/40] fsck.gfs2: Fix rgrp dinode accounting bug
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (37 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 38/40] fsck.gfs2: pass1b is too noisy wrt gfs1 non-dinode metadata Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 40/40] fsck.gfs2: Fix rgrp accounting in check_n_fix_bitmap Bob Peterson
` (2 subsequent siblings)
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch fixes a bug whereby the count of dinodes in an rgrp can
get off because it's not adjusted properly when the bit is changed
from dinode to free. It's not adjusted because it's not found in
the inode tree or dirtree. The patch changes it so that the bitmap
is changed before deleting those things.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/pass1b.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/gfs2/fsck/pass1b.c b/gfs2/fsck/pass1b.c
index 9d5fc04..6dec193 100644
--- a/gfs2/fsck/pass1b.c
+++ b/gfs2/fsck/pass1b.c
@@ -347,6 +347,13 @@ static void resolve_dup_references(struct gfs2_sbd *sdp, struct duptree *dt,
if ((this_ref != ref_as_ea) &&
(inval || id->reftypecount[ref_as_data] ||
id->reftypecount[ref_as_meta])) {
+ /* Fix the bitmap first, while the inodetree
+ and dirtree entries exist. That way, the
+ bitmap_set will do proper accounting for
+ the rgrp dinode count. */
+ fsck_bitmap_set(ip, ip->i_di.di_num.no_addr,
+ _("duplicate referencing bad"),
+ GFS2_BLKST_FREE);
/* Remove the inode from the inode tree */
ii = inodetree_find(ip->i_di.di_num.no_addr);
if (ii)
@@ -356,9 +363,6 @@ static void resolve_dup_references(struct gfs2_sbd *sdp, struct duptree *dt,
dirtree_delete(di);
link1_set(&nlink1map, ip->i_di.di_num.no_addr,
0);
- fsck_bitmap_set(ip, ip->i_di.di_num.no_addr,
- _("duplicate referencing bad"),
- GFS2_BLKST_FREE);
/* We delete the dup_handler inode count and
duplicate id BEFORE clearing the metadata,
because if this is the last reference to
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 40/40] fsck.gfs2: Fix rgrp accounting in check_n_fix_bitmap
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (38 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 39/40] fsck.gfs2: Fix rgrp dinode accounting bug Bob Peterson
@ 2016-05-06 17:39 ` Bob Peterson
2016-05-09 14:55 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Andrew Price
2016-05-09 17:48 ` Abhijith Das
41 siblings, 0 replies; 45+ messages in thread
From: Bob Peterson @ 2016-05-06 17:39 UTC (permalink / raw)
To: cluster-devel.redhat.com
This patch fixes function check_n_fix_bitmap for proper accounting,
especially for GFS1 file systems. It did proper accounting for
normal transitions from dinode-to-free and free-to-dinode, but it
stumbled when it did transitions from data-to-metadata and back.
This patch has the philosophy of "decrement dinode count and/or
metadata count" whenever a bit is transitioned from dinode to
something else. Likewise, it increments when a bit is transitioned
from anything else to dinode.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
gfs2/fsck/metawalk.c | 119 +++++++++++++++++++++++++--------------------------
1 file changed, 59 insertions(+), 60 deletions(-)
diff --git a/gfs2/fsck/metawalk.c b/gfs2/fsck/metawalk.c
index 668690f..253ebcb 100644
--- a/gfs2/fsck/metawalk.c
+++ b/gfs2/fsck/metawalk.c
@@ -35,6 +35,7 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
int old_bitmap_state;
struct rgrp_tree *rgd;
int treat_as_inode = 0;
+ int rewrite_rgrp = 0;
struct gfs_rgrp *gfs1rg;
const char *allocdesc[2][5] = { /* gfs2 descriptions */
{"free", "data", "unlinked", "inode", "reserved"},
@@ -80,75 +81,73 @@ int check_n_fix_bitmap(struct gfs2_sbd *sdp, uint64_t blk, int error_on_dinode,
data or data to dinode, no change in free space. */
gfs2_set_bitmap(rgd, blk, new_blockmap_state);
if (new_blockmap_state == GFS2_BLKST_FREE) {
- /* If we're freeing a dinode, get rid of the hash table
- entries for it. */
- if (old_bitmap_state == GFS2_BLKST_DINODE ||
- old_bitmap_state == GFS2_BLKST_UNLINKED) {
- struct dir_info *dt;
- struct inode_info *ii;
-
- dt = dirtree_find(blk);
- if (dt) {
- dirtree_delete(dt);
- treat_as_inode = 1;
- }
- ii = inodetree_find(blk);
- if (ii) {
- inodetree_delete(ii);
- treat_as_inode = 1;
- } else if (!sdp->gfs1) {
- treat_as_inode = 1;
- } else {
- /* This is a GFS1 fs (so all metadata is marked
- inode). We need to verify it is an inode
- before we can decr the rgrp inode count. */
- if (link1_type(&nlink1map, blk) == 1)
- treat_as_inode = 1;
- }
- if (old_bitmap_state == GFS2_BLKST_DINODE) {
- if (treat_as_inode && rgd->rg.rg_dinodes > 0)
- rgd->rg.rg_dinodes--;
- if (sdp->gfs1 && gfs1rg->rg_usedmeta > 0)
- gfs1rg->rg_usedmeta--;
- }
- link1_set(&nlink1map, blk, 0);
- }
rgd->rg.rg_free++;
- if (sdp->gfs1)
- gfs_rgrp_out((struct gfs_rgrp *)&rgd->rg,
- rgd->bits[0].bi_bh);
- else
- gfs2_rgrp_out_bh(&rgd->rg, rgd->bits[0].bi_bh);
+ rewrite_rgrp = 1;
} else if (old_bitmap_state == GFS2_BLKST_FREE) {
- if (new_blockmap_state == GFS2_BLKST_DINODE) {
- if (!sdp->gfs1) {
+ rgd->rg.rg_free--;
+ rewrite_rgrp = 1;
+ }
+ /* If we're freeing a dinode, get rid of the data structs for it. */
+ if (old_bitmap_state == GFS2_BLKST_DINODE ||
+ old_bitmap_state == GFS2_BLKST_UNLINKED) {
+ struct dir_info *dt;
+ struct inode_info *ii;
+
+ dt = dirtree_find(blk);
+ if (dt) {
+ dirtree_delete(dt);
+ treat_as_inode = 1;
+ }
+ ii = inodetree_find(blk);
+ if (ii) {
+ inodetree_delete(ii);
+ treat_as_inode = 1;
+ } else if (!sdp->gfs1) {
+ treat_as_inode = 1;
+ } else if (link1_type(&nlink1map, blk) == 1) {
+ /* This is a GFS1 fs (so all metadata is marked inode).
+ We need to verify it is an inode before we can decr
+ the rgrp inode count. */
+ treat_as_inode = 1;
+ }
+ if (old_bitmap_state == GFS2_BLKST_DINODE) {
+ if (treat_as_inode && rgd->rg.rg_dinodes > 0)
+ rgd->rg.rg_dinodes--;
+ else if (sdp->gfs1 && gfs1rg->rg_usedmeta > 0)
+ gfs1rg->rg_usedmeta--;
+ rewrite_rgrp = 1;
+ }
+ link1_set(&nlink1map, blk, 0);
+ } else if (new_blockmap_state == GFS2_BLKST_DINODE) {
+ if (!sdp->gfs1) {
+ treat_as_inode = 1;
+ } else {
+ /* This is GFS1 (so all metadata is marked inode). We
+ need to verify it is an inode before we can decr
+ the rgrp inode count. */
+ if (link1_type(&nlink1map, blk) == 1)
treat_as_inode = 1;
- } else {
- /* This is GFS1 (so all metadata is marked
- inode). We need to verify it is an inode
- before we can decr the rgrp inode count. */
- if (link1_type(&nlink1map, blk) == 1)
+ else {
+ struct dir_info *dt;
+ struct inode_info *ii;
+
+ dt = dirtree_find(blk);
+ if (dt)
treat_as_inode = 1;
else {
- struct dir_info *dt;
- struct inode_info *ii;
-
- dt = dirtree_find(blk);
- if (dt)
+ ii = inodetree_find(blk);
+ if (ii)
treat_as_inode = 1;
- else {
- ii = inodetree_find(blk);
- if (ii)
- treat_as_inode = 1;
- }
}
}
- if (treat_as_inode)
- rgd->rg.rg_dinodes++;
- if (sdp->gfs1)
- gfs1rg->rg_usedmeta++;
}
- rgd->rg.rg_free--;
+ if (treat_as_inode)
+ rgd->rg.rg_dinodes++;
+ else if (sdp->gfs1)
+ gfs1rg->rg_usedmeta++;
+ rewrite_rgrp = 1;
+ }
+ if (rewrite_rgrp) {
if (sdp->gfs1)
gfs_rgrp_out((struct gfs_rgrp *)&rgd->rg, rgd->bits[0].bi_bh);
else
--
2.5.5
^ permalink raw reply related [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (39 preceding siblings ...)
2016-05-06 17:39 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 40/40] fsck.gfs2: Fix rgrp accounting in check_n_fix_bitmap Bob Peterson
@ 2016-05-09 14:55 ` Andrew Price
2016-05-09 17:48 ` Abhijith Das
41 siblings, 0 replies; 45+ messages in thread
From: Andrew Price @ 2016-05-09 14:55 UTC (permalink / raw)
To: cluster-devel.redhat.com
Hi Bob,
On 06/05/16 18:38, Bob Peterson wrote:
> On 15 April, I posted a set of 30 patches for fsck.gfs2 to reduce its
> memory requirements. Since that time, I've been testing and fixing
> problems. Here is version 2.
> ---
> Right now, fsck.gfs2 is a memory hog with regard to very large file
> systems. A file system of 250TB would require an absurd about of memory.
> That's because it uses memory for (1) rgrp bitmaps, (2) its own internal
> blockmap, (3) a directory tree, and (4) a tree of ALL inodes in the
> file system, for reconciling link counts.
>
> This series of 40 patches is designed to reduce the memory footprint
> of fsck.gfs2. It does so by almost entirely eliminating the inode
> tree in favor of two much smaller bitmaps. In order to do that, I
> rearranged the order in which things are done.
>
> In order to save on memory, some things will be slower, so this will
> cause a performance hit to fsck.gfs2. However, I'm confident that
> we can fix them and make it fast again. The slowness is caused because
> most bitmap manipulation is now done on the bitmaps in the resource
> groups, which means the code needs to search out the rgrp for these
> operations. This can be optimized to pass in the rgrp, since it will
> be known in most cases (for example, pass1 operates on one rgrp at a
> time, and chances are quite good that the rgrp needed will be that one.)
> These further optimizations will be done in later patches.
>
> I'm still testing these changes against my collection of gfs2 and gfs1
> metadata, but the results are good so far. So there may be changes.
I've read through the patches and the code looks good to me. It's
difficult to comment on the design of the solution but nothing scary
jumped out at me, and the test results look promising. There's some
really good refactoring work in this set too.
> Signed-off-by: Bob Peterson <rpeterso@redhat.com>
> ---
> Bob Peterson (40):
> fsck.gfs2: Move pass5 to immediately follow pass1
> fsck.gfs2: Convert block_type to bitmap_type after pass1 and 5
> fsck.gfs2: Change bitmap_type variables to int
> fsck.gfs2: Use di_entries to determine if lost+found was created
> fsck.gfs2: pass1b shouldn't complain about non-bitmap blocks
> fsck.gfs2: Change all fsck_blockmap_set to fsck_bitmap_set
> fsck.gfs2: Move set_ip_blockmap to pass1
> fsck.gfs2: Remove unneeded parameter instree from set_ip_blockmap
> fsck.gfs2: Move leaf repair to pass2
> fsck.gfs2: Eliminate astate code
> fsck.gfs2: Move reprocess code to pass1
> fsck.gfs2: Separate out functions that may only be done after pass1
> fsck.gfs2: Divest check_metatree from fsck_blockmap_set
> fsck.gfs2: eliminate fsck_blockmap_set from check_eattr_entries
> fsck.gfs2: Move blockmap stuff to pass1.c
> fsck: make pass1 call bitmap reconciliation AKA pass5
> fsck.gfs2: make blockmap global variable only to pass1
> fsck.gfs2: Add wrapper function pass1_check_metatree
> fsck.gfs2: pass counted_links into fix_link_count in pass4
> fsck.gfs2: refactor pass4 function scan_inode_list
> fsck.gfs2: More refactoring of pass4 function scan_inode_list
> fsck.gfs2: Fix white space problems
> fsck.gfs2: move link count info for directories to directory tree
> fsck.gfs2: Use bitmaps instead of linked list for inodes w/nlink == 1
> fsck.gfs2: Refactor check_n_fix_bitmap to make it more readable
> fsck.gfs2: adjust rgrp inode count when fixing bitmap
> fsck.gfs2: blocks cannot be UNLINKED in pass1b or after that
> fsck.gfs2: Add error checks to get_next_leaf
> fsck.gfs2: re-add a non-allocating repair_leaf to pass1
> libgfs2: Allocate new GFS1 metadata as type 3, not type 1
> fsck.gfs2: Undo partially done metadata records
> fsck.gfs2: Eliminate redundant code in _fsck_bitmap_set
> fsck.gfs2: Fix inode counting bug
> fsck.gfs2: Adjust bitmap for lost+found after adding to dirtree
> GFS2: Add initialization checks for GFS1 used metadata
s/GFS2/fsck.gfs2/ :)
Thanks,
Andy
> fsck.gfs2: Use BLKST constants to make pass5 more clear
> fsck.gfs2: Fix GFS1 "used meta" accounting bug
> fsck.gfs2: pass1b is too noisy wrt gfs1 non-dinode metadata
> fsck.gfs2: Fix rgrp dinode accounting bug
> fsck.gfs2: Fix rgrp accounting in check_n_fix_bitmap
>
> gfs2/fsck/Makefile.am | 1 +
> gfs2/fsck/afterpass1_common.c | 319 ++++++++++++++++
> gfs2/fsck/afterpass1_common.h | 31 ++
> gfs2/fsck/fsck.h | 26 +-
> gfs2/fsck/initialize.c | 63 +++-
> gfs2/fsck/inode_hash.c | 3 +-
> gfs2/fsck/link.c | 139 ++++++-
> gfs2/fsck/link.h | 6 +-
> gfs2/fsck/lost_n_found.c | 17 +-
> gfs2/fsck/main.c | 38 +-
> gfs2/fsck/metawalk.c | 847 ++++++++----------------------------------
> gfs2/fsck/metawalk.h | 59 +--
> gfs2/fsck/pass1.c | 479 ++++++++++++++++++++++--
> gfs2/fsck/pass1b.c | 148 +++++---
> gfs2/fsck/pass2.c | 276 ++++++++++----
> gfs2/fsck/pass3.c | 52 +--
> gfs2/fsck/pass4.c | 339 ++++++++++-------
> gfs2/fsck/pass5.c | 60 +--
> gfs2/fsck/util.c | 171 +++------
> gfs2/fsck/util.h | 70 ++--
> gfs2/libgfs2/fs_ops.c | 33 +-
> 21 files changed, 1849 insertions(+), 1328 deletions(-)
> create mode 100644 gfs2/fsck/afterpass1_common.c
> create mode 100644 gfs2/fsck/afterpass1_common.h
>
^ permalink raw reply [flat|nested] 45+ messages in thread* [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches
2016-05-06 17:38 [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Bob Peterson
` (40 preceding siblings ...)
2016-05-09 14:55 ` [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches Andrew Price
@ 2016-05-09 17:48 ` Abhijith Das
41 siblings, 0 replies; 45+ messages in thread
From: Abhijith Das @ 2016-05-09 17:48 UTC (permalink / raw)
To: cluster-devel.redhat.com
Hi,
Looks good to me. ACK.
Cheers!
--Abhi
----- Original Message -----
> From: "Bob Peterson" <rpeterso@redhat.com>
> To: "cluster-devel" <cluster-devel@redhat.com>
> Sent: Friday, May 6, 2016 12:38:53 PM
> Subject: [Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches
>
> On 15 April, I posted a set of 30 patches for fsck.gfs2 to reduce its
> memory requirements. Since that time, I've been testing and fixing
> problems. Here is version 2.
> ---
> Right now, fsck.gfs2 is a memory hog with regard to very large file
> systems. A file system of 250TB would require an absurd about of memory.
> That's because it uses memory for (1) rgrp bitmaps, (2) its own internal
> blockmap, (3) a directory tree, and (4) a tree of ALL inodes in the
> file system, for reconciling link counts.
>
> This series of 40 patches is designed to reduce the memory footprint
> of fsck.gfs2. It does so by almost entirely eliminating the inode
> tree in favor of two much smaller bitmaps. In order to do that, I
> rearranged the order in which things are done.
>
> In order to save on memory, some things will be slower, so this will
> cause a performance hit to fsck.gfs2. However, I'm confident that
> we can fix them and make it fast again. The slowness is caused because
> most bitmap manipulation is now done on the bitmaps in the resource
> groups, which means the code needs to search out the rgrp for these
> operations. This can be optimized to pass in the rgrp, since it will
> be known in most cases (for example, pass1 operates on one rgrp at a
> time, and chances are quite good that the rgrp needed will be that one.)
> These further optimizations will be done in later patches.
>
> I'm still testing these changes against my collection of gfs2 and gfs1
> metadata, but the results are good so far. So there may be changes.
>
> Signed-off-by: Bob Peterson <rpeterso@redhat.com>
^ permalink raw reply [flat|nested] 45+ messages in thread