From: hujianyang <hujianyang@huawei.com>
To: Jaegeuk Kim <jaegeuk@kernel.org>
Cc: Chao Yu <chao2.yu@samsung.com>,
cm224.lee@samsung.com, linux-f2fs-devel@lists.sourceforge.net,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>
Subject: Space leak in f2fs
Date: Wed, 13 May 2015 15:17:17 +0800 [thread overview]
Message-ID: <5552FA7D.7000704@huawei.com> (raw)
Hi Jaegeuk,
I found a space leak problem in f2fs. This problem could lead to
ENOSPC error during stress tests, e.g. ltp.
<<<test_output>>>
growfiles(gf15): 11656 growfiles.c/2249: 16920 tlibio.c/739 write(6, buf, 1352) ret:-1, errno=28 No space left on device
gf15 1 TFAIL : growfiles.c:132: Test failed
...
And can be reproduced by these steps whether background_gc is on
or not:
1) format a 4GB f2fs partition
2) dd a 3G file,
3) unlink it.
Do these steps again and again. Soon, after one unlink operation,
you can see the space of the 3G file is not free.
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 301064 3854328 8% /mnt/f2fs
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
3072+0 records in
3072+0 records out
3221225472 bytes (3.2 GB) copied, 3.1892 s, 1.0 GB/s
Fs-Server:/mnt/f2fs # unlink ./test
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
3072+0 records in
3072+0 records out
3221225472 bytes (3.2 GB) copied, 3.44288 s, 936 MB/s
Fs-Server:/mnt/f2fs # unlink ./test
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 3449888 705504 84% /mnt/f2fs
Fs-Server:/mnt/f2fs # ls
Fs-Server:/mnt/f2fs # ls
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 3449888 705504 84% /mnt/f2fs
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
dd: writing `./test': No space left on device
689+0 records in
688+0 records out
721719296 bytes (722 MB) copied, 0.618972 s, 1.2 GB/s
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 4155392 0 100% /mnt/f2fs
Fs-Server:/mnt/f2fs # ls -l test
-rw-r--r-- 1 root root 721719296 May 13 14:52 test
We can reuse the leaking space after a sync call:
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 4155392 0 100% /mnt/f2fs
Fs-Server:/mnt/f2fs # sync
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 1006568 3148824 25% /mnt/f2fs
I found this may caused by .drop_inode in f2fs. see f2fs_drop_inode()
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 19438f2..7646d2a 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -424,15 +424,6 @@ static struct inode *f2fs_alloc_inode(struct super_block *sb)
static int f2fs_drop_inode(struct inode *inode)
{
- /*
- * This is to avoid a deadlock condition like below.
- * writeback_single_inode(inode)
- * - f2fs_write_data_page
- * - f2fs_gc -> iput -> evict
- * - inode_wait_for_writeback(inode)
- */
- if (!inode_unhashed(inode) && inode->i_state & I_SYNC)
- return 0;
return generic_drop_inode(inode);
}
After removing these code, this problem is fixed. But this function is
introduced by commit 531ad7d58c6476c5856653448b4c7d26427502b4 to fix
a deadlock problem.
I wish you and other developers in this list could help me to fix this
problem in a correct way.
Thanks,
Hu
WARNING: multiple messages have this Message-ID (diff)
From: hujianyang <hujianyang@huawei.com>
To: Jaegeuk Kim <jaegeuk@kernel.org>
Cc: Chao Yu <chao2.yu@samsung.com>, <cm224.lee@samsung.com>,
<linux-f2fs-devel@lists.sourceforge.net>,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>
Subject: Space leak in f2fs
Date: Wed, 13 May 2015 15:17:17 +0800 [thread overview]
Message-ID: <5552FA7D.7000704@huawei.com> (raw)
Hi Jaegeuk,
I found a space leak problem in f2fs. This problem could lead to
ENOSPC error during stress tests, e.g. ltp.
<<<test_output>>>
growfiles(gf15): 11656 growfiles.c/2249: 16920 tlibio.c/739 write(6, buf, 1352) ret:-1, errno=28 No space left on device
gf15 1 TFAIL : growfiles.c:132: Test failed
...
And can be reproduced by these steps whether background_gc is on
or not:
1) format a 4GB f2fs partition
2) dd a 3G file,
3) unlink it.
Do these steps again and again. Soon, after one unlink operation,
you can see the space of the 3G file is not free.
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 301064 3854328 8% /mnt/f2fs
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
3072+0 records in
3072+0 records out
3221225472 bytes (3.2 GB) copied, 3.1892 s, 1.0 GB/s
Fs-Server:/mnt/f2fs # unlink ./test
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
3072+0 records in
3072+0 records out
3221225472 bytes (3.2 GB) copied, 3.44288 s, 936 MB/s
Fs-Server:/mnt/f2fs # unlink ./test
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 3449888 705504 84% /mnt/f2fs
Fs-Server:/mnt/f2fs # ls
Fs-Server:/mnt/f2fs # ls
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 3449888 705504 84% /mnt/f2fs
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
dd: writing `./test': No space left on device
689+0 records in
688+0 records out
721719296 bytes (722 MB) copied, 0.618972 s, 1.2 GB/s
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 4155392 0 100% /mnt/f2fs
Fs-Server:/mnt/f2fs # ls -l test
-rw-r--r-- 1 root root 721719296 May 13 14:52 test
We can reuse the leaking space after a sync call:
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 4155392 0 100% /mnt/f2fs
Fs-Server:/mnt/f2fs # sync
Fs-Server:/mnt/f2fs # df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdd3 4193280 1006568 3148824 25% /mnt/f2fs
I found this may caused by .drop_inode in f2fs. see f2fs_drop_inode()
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 19438f2..7646d2a 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -424,15 +424,6 @@ static struct inode *f2fs_alloc_inode(struct super_block *sb)
static int f2fs_drop_inode(struct inode *inode)
{
- /*
- * This is to avoid a deadlock condition like below.
- * writeback_single_inode(inode)
- * - f2fs_write_data_page
- * - f2fs_gc -> iput -> evict
- * - inode_wait_for_writeback(inode)
- */
- if (!inode_unhashed(inode) && inode->i_state & I_SYNC)
- return 0;
return generic_drop_inode(inode);
}
After removing these code, this problem is fixed. But this function is
introduced by commit 531ad7d58c6476c5856653448b4c7d26427502b4 to fix
a deadlock problem.
I wish you and other developers in this list could help me to fix this
problem in a correct way.
Thanks,
Hu
next reply other threads:[~2015-05-13 7:17 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-05-13 7:17 hujianyang [this message]
2015-05-13 7:17 ` Space leak in f2fs hujianyang
2015-05-13 17:46 ` [f2fs-dev] " Jaegeuk Kim
2015-05-14 0:24 ` Jaegeuk Kim
2015-05-14 1:40 ` hujianyang
2015-05-14 1:45 ` [f2fs-dev] " Jaegeuk Kim
2015-05-14 21:14 ` Jaegeuk Kim
2015-05-15 8:31 ` Chao Yu
2015-05-16 0:55 ` Jaegeuk Kim
2015-05-18 2:43 ` Chao Yu
2015-05-18 2:50 ` Nicholas Krause
2015-05-18 2:50 ` Nicholas Krause
2015-05-18 5:44 ` [f2fs-dev] " Jaegeuk Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5552FA7D.7000704@huawei.com \
--to=hujianyang@huawei.com \
--cc=chao2.yu@samsung.com \
--cc=cm224.lee@samsung.com \
--cc=jaegeuk@kernel.org \
--cc=linux-f2fs-devel@lists.sourceforge.net \
--cc=linux-fsdevel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.