[PATCH v6] f2fs: support in-memory inode checksum when checking consistency

[PATCH v6] f2fs: support in-memory inode checksum when checking consistency

[Weichao Guo]

Enable in-memory inode checksum to protect metadata blocks from
in-memory scribbles when checking consistency, which has no
performance requirements.

Signed-off-by: Weichao Guo <guoweichao@huawei.com>
---
 fs/f2fs/inode.c |  7 +++++++
 fs/f2fs/node.c  | 10 +++++++++-
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
index 205add3..e405b28 100644
--- a/fs/f2fs/inode.c
+++ b/fs/f2fs/inode.c
@@ -159,8 +159,12 @@ bool f2fs_inode_chksum_verify(struct f2fs_sb_info *sbi, struct page *page)
 	struct f2fs_inode *ri;
 	__u32 provided, calculated;
 
+#ifdef CONFIG_F2FS_CHECK_FS
+	if (!f2fs_enable_inode_chksum(sbi, page))
+#else
 	if (!f2fs_enable_inode_chksum(sbi, page) ||
 			PageDirty(page) || PageWriteback(page))
+#endif
 		return true;
 
 	ri = &F2FS_NODE(page)->i;
@@ -445,6 +449,9 @@ void update_inode(struct inode *inode, struct page *node_page)
 	if (inode->i_nlink == 0)
 		clear_inline_node(node_page);
 
+#ifdef CONFIG_F2FS_CHECK_FS
+	f2fs_inode_chksum_set(F2FS_I_SB(inode), node_page);
+#endif
 }
 
 void update_inode_page(struct inode *inode)
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index 177c438..6107935 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -1113,8 +1113,12 @@ static int read_node_page(struct page *page, int op_flags)
 		.encrypted_page = NULL,
 	};
 
-	if (PageUptodate(page))
+	if (PageUptodate(page)) {
+#ifdef CONFIG_F2FS_CHECK_FS
+		f2fs_bug_on(sbi, !f2fs_inode_chksum_verify(sbi, page));
+#endif
 		return LOCKED_PAGE;
+	}
 
 	get_node_info(sbi, page->index, &ni);
 
@@ -1731,6 +1735,10 @@ static int f2fs_set_node_page_dirty(struct page *page)
 
 	if (!PageUptodate(page))
 		SetPageUptodate(page);
+#ifdef CONFIG_F2FS_CHECK_FS
+	if (IS_INODE(page))
+		f2fs_inode_chksum_set(F2FS_P_SB(page), page);
+#endif
 	if (!PageDirty(page)) {
 		f2fs_set_page_dirty_nobuffers(page);
 		inc_page_count(F2FS_P_SB(page), F2FS_DIRTY_NODES);
-- 
2.10.1


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Re: [PATCH v6] f2fs: support in-memory inode checksum when checking consistency

[Chao Yu]

On 2018/3/9 23:10, Weichao Guo wrote:
> Enable in-memory inode checksum to protect metadata blocks from
> in-memory scribbles when checking consistency, which has no
> performance requirements.
> 
> Signed-off-by: Weichao Guo <guoweichao@huawei.com>

Reviewed-by: Chao Yu <yuchao0@huawei.com>

Thanks,

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Re: [PATCH v6] f2fs: support in-memory inode checksum when checking consistency

[Chao Yu]

Ping,

Jaegeuk, sometimes, we suffer data corruption caused by bit-transition of cache,
we troubleshoot for weeks or months, and finally, find that it is caused by
hardware. So for this case, maybe in-memory checksum can help to troubleshoot
more quickly.

Could you consider this?

Thanks,

On 2018/3/9 23:10, Weichao Guo wrote:
> Enable in-memory inode checksum to protect metadata blocks from
> in-memory scribbles when checking consistency, which has no
> performance requirements.
> 
> Signed-off-by: Weichao Guo <guoweichao@huawei.com>
> ---
>  fs/f2fs/inode.c |  7 +++++++
>  fs/f2fs/node.c  | 10 +++++++++-
>  2 files changed, 16 insertions(+), 1 deletion(-)
> 
> diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
> index 205add3..e405b28 100644
> --- a/fs/f2fs/inode.c
> +++ b/fs/f2fs/inode.c
> @@ -159,8 +159,12 @@ bool f2fs_inode_chksum_verify(struct f2fs_sb_info *sbi, struct page *page)
>  	struct f2fs_inode *ri;
>  	__u32 provided, calculated;
>  
> +#ifdef CONFIG_F2FS_CHECK_FS
> +	if (!f2fs_enable_inode_chksum(sbi, page))
> +#else
>  	if (!f2fs_enable_inode_chksum(sbi, page) ||
>  			PageDirty(page) || PageWriteback(page))
> +#endif
>  		return true;
>  
>  	ri = &F2FS_NODE(page)->i;
> @@ -445,6 +449,9 @@ void update_inode(struct inode *inode, struct page *node_page)
>  	if (inode->i_nlink == 0)
>  		clear_inline_node(node_page);
>  
> +#ifdef CONFIG_F2FS_CHECK_FS
> +	f2fs_inode_chksum_set(F2FS_I_SB(inode), node_page);
> +#endif
>  }
>  
>  void update_inode_page(struct inode *inode)
> diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
> index 177c438..6107935 100644
> --- a/fs/f2fs/node.c
> +++ b/fs/f2fs/node.c
> @@ -1113,8 +1113,12 @@ static int read_node_page(struct page *page, int op_flags)
>  		.encrypted_page = NULL,
>  	};
>  
> -	if (PageUptodate(page))
> +	if (PageUptodate(page)) {
> +#ifdef CONFIG_F2FS_CHECK_FS
> +		f2fs_bug_on(sbi, !f2fs_inode_chksum_verify(sbi, page));
> +#endif
>  		return LOCKED_PAGE;
> +	}
>  
>  	get_node_info(sbi, page->index, &ni);
>  
> @@ -1731,6 +1735,10 @@ static int f2fs_set_node_page_dirty(struct page *page)
>  
>  	if (!PageUptodate(page))
>  		SetPageUptodate(page);
> +#ifdef CONFIG_F2FS_CHECK_FS
> +	if (IS_INODE(page))
> +		f2fs_inode_chksum_set(F2FS_P_SB(page), page);
> +#endif
>  	if (!PageDirty(page)) {
>  		f2fs_set_page_dirty_nobuffers(page);
>  		inc_page_count(F2FS_P_SB(page), F2FS_DIRTY_NODES);
> 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Re: [PATCH v6] f2fs: support in-memory inode checksum when checking consistency

[Chao Yu]

On 2018/6/12 22:36, Chao Yu wrote:
> Ping,
> 
> Jaegeuk, sometimes, we suffer data corruption caused by bit-transition of cache,
> we troubleshoot for weeks or months, and finally, find that it is caused by
> hardware. So for this case, maybe in-memory checksum can help to troubleshoot
> more quickly.
> 
> Could you consider this?
> 
> Thanks,
> 
> On 2018/3/9 23:10, Weichao Guo wrote:
>> Enable in-memory inode checksum to protect metadata blocks from
>> in-memory scribbles when checking consistency, which has no
>> performance requirements.
>>
>> Signed-off-by: Weichao Guo <guoweichao@huawei.com>

Reviewed-by: Chao Yu <yuchao0@huawei.com>

Thanks,


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Re: [PATCH v6] f2fs: support in-memory inode checksum when checking consistency

[Chao Yu]

Hi Weichao and Jaegeuk,

On 2018/6/22 15:16, Chao Yu wrote:
> On 2018/6/12 22:36, Chao Yu wrote:
>> Ping,
>>
>> Jaegeuk, sometimes, we suffer data corruption caused by bit-transition of cache,
>> we troubleshoot for weeks or months, and finally, find that it is caused by
>> hardware. So for this case, maybe in-memory checksum can help to troubleshoot
>> more quickly.

I encounter mismatch on checksum when I test generic/475, I think we should add
missing calculation in below place:

diff --git a/fs/f2fs/node.h b/fs/f2fs/node.h
index b0da4c2..0f4db7a 100644
--- a/fs/f2fs/node.h
+++ b/fs/f2fs/node.h
@@ -449,6 +449,10 @@ static inline void set_mark(struct page *page, int mark,
int type)
 	else
 		flag &= ~(0x1 << type);
 	rn->footer.flag = cpu_to_le32(flag);
+
+#ifdef CONFIG_F2FS_CHECK_FS
+	f2fs_inode_chksum_set(F2FS_P_SB(page), page);
+#endif
 }
 #define set_dentry_mark(page, mark)	set_mark(page, mark, DENT_BIT_SHIFT)
 #define set_fsync_mark(page, mark)	set_mark(page, mark, FSYNC_BIT_SHIFT)

>>
>> Could you consider this?
>>
>> Thanks,
>>
>> On 2018/3/9 23:10, Weichao Guo wrote:
>>> Enable in-memory inode checksum to protect metadata blocks from
>>> in-memory scribbles when checking consistency, which has no
>>> performance requirements.
>>>
>>> Signed-off-by: Weichao Guo <guoweichao@huawei.com>
> 
> Reviewed-by: Chao Yu <yuchao0@huawei.com>
> 
> Thanks,
> 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Re: [PATCH v6] f2fs: support in-memory inode checksum when checking consistency

[Jaegeuk Kim]

On 07/15, Chao Yu wrote:
> Hi Weichao and Jaegeuk,
> 
> On 2018/6/22 15:16, Chao Yu wrote:
> > On 2018/6/12 22:36, Chao Yu wrote:
> >> Ping,
> >>
> >> Jaegeuk, sometimes, we suffer data corruption caused by bit-transition of cache,
> >> we troubleshoot for weeks or months, and finally, find that it is caused by
> >> hardware. So for this case, maybe in-memory checksum can help to troubleshoot
> >> more quickly.
> 
> I encounter mismatch on checksum when I test generic/475, I think we should add
> missing calculation in below place:

Hi Chao,

I added the below.

Thanks,

> 
> diff --git a/fs/f2fs/node.h b/fs/f2fs/node.h
> index b0da4c2..0f4db7a 100644
> --- a/fs/f2fs/node.h
> +++ b/fs/f2fs/node.h
> @@ -449,6 +449,10 @@ static inline void set_mark(struct page *page, int mark,
> int type)
>  	else
>  		flag &= ~(0x1 << type);
>  	rn->footer.flag = cpu_to_le32(flag);
> +
> +#ifdef CONFIG_F2FS_CHECK_FS
> +	f2fs_inode_chksum_set(F2FS_P_SB(page), page);
> +#endif
>  }
>  #define set_dentry_mark(page, mark)	set_mark(page, mark, DENT_BIT_SHIFT)
>  #define set_fsync_mark(page, mark)	set_mark(page, mark, FSYNC_BIT_SHIFT)
> 
> >>
> >> Could you consider this?
> >>
> >> Thanks,
> >>
> >> On 2018/3/9 23:10, Weichao Guo wrote:
> >>> Enable in-memory inode checksum to protect metadata blocks from
> >>> in-memory scribbles when checking consistency, which has no
> >>> performance requirements.
> >>>
> >>> Signed-off-by: Weichao Guo <guoweichao@huawei.com>
> > 
> > Reviewed-by: Chao Yu <yuchao0@huawei.com>
> > 
> > Thanks,
> > 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot