All of lore.kernel.org
 help / color / mirror / Atom feed
From: hujianyang <hujianyang@huawei.com>
To: Jaegeuk Kim <jaegeuk@kernel.org>
Cc: Chao Yu <chao2.yu@samsung.com>,
	cm224.lee@samsung.com, linux-f2fs-devel@lists.sourceforge.net,
	"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>
Subject: Space leak in f2fs
Date: Wed, 13 May 2015 15:17:17 +0800	[thread overview]
Message-ID: <5552FA7D.7000704@huawei.com> (raw)

Hi Jaegeuk,

I found a space leak problem in f2fs. This problem could lead to
ENOSPC error during stress tests, e.g. ltp.

<<<test_output>>>
growfiles(gf15): 11656 growfiles.c/2249: 16920 tlibio.c/739 write(6, buf, 1352) ret:-1, errno=28 No space left on device
gf15        1  TFAIL  :  growfiles.c:132: Test failed

...

And can be reproduced by these steps whether background_gc is on
or not:

1) format a 4GB f2fs partition
2) dd a 3G file,
3) unlink it.

Do these steps again and again. Soon, after one unlink operation,
you can see the space of the 3G file is not free.

Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks   Used Available Use% Mounted on
/dev/sdd3        4193280 301064   3854328   8% /mnt/f2fs
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
3072+0 records in
3072+0 records out
3221225472 bytes (3.2 GB) copied, 3.1892 s, 1.0 GB/s
Fs-Server:/mnt/f2fs # unlink ./test
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
3072+0 records in
3072+0 records out
3221225472 bytes (3.2 GB) copied, 3.44288 s, 936 MB/s
Fs-Server:/mnt/f2fs # unlink ./test
Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sdd3        4193280 3449888    705504  84% /mnt/f2fs
Fs-Server:/mnt/f2fs # ls
Fs-Server:/mnt/f2fs # ls
Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sdd3        4193280 3449888    705504  84% /mnt/f2fs
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
dd: writing `./test': No space left on device
689+0 records in
688+0 records out
721719296 bytes (722 MB) copied, 0.618972 s, 1.2 GB/s
Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sdd3        4193280 4155392         0 100% /mnt/f2fs
Fs-Server:/mnt/f2fs # ls -l test
-rw-r--r-- 1 root root 721719296 May 13 14:52 test


We can reuse the leaking space after a sync call:


Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sdd3        4193280 4155392         0 100% /mnt/f2fs
Fs-Server:/mnt/f2fs # sync
Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sdd3        4193280 1006568   3148824  25% /mnt/f2fs


I found this may caused by .drop_inode in f2fs. see f2fs_drop_inode()

diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 19438f2..7646d2a 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -424,15 +424,6 @@ static struct inode *f2fs_alloc_inode(struct super_block *sb)

 static int f2fs_drop_inode(struct inode *inode)
 {
-       /*
-        * This is to avoid a deadlock condition like below.
-        * writeback_single_inode(inode)
-        *  - f2fs_write_data_page
-        *    - f2fs_gc -> iput -> evict
-        *       - inode_wait_for_writeback(inode)
-        */
-       if (!inode_unhashed(inode) && inode->i_state & I_SYNC)
-               return 0;
        return generic_drop_inode(inode);
 }

After removing these code, this problem is fixed. But this function is
introduced by commit 531ad7d58c6476c5856653448b4c7d26427502b4 to fix
a deadlock problem.

I wish you and other developers in this list could help me to fix this
problem in a correct way.

Thanks,
Hu


WARNING: multiple messages have this Message-ID (diff)
From: hujianyang <hujianyang@huawei.com>
To: Jaegeuk Kim <jaegeuk@kernel.org>
Cc: Chao Yu <chao2.yu@samsung.com>, <cm224.lee@samsung.com>,
	<linux-f2fs-devel@lists.sourceforge.net>,
	"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>
Subject: Space leak in f2fs
Date: Wed, 13 May 2015 15:17:17 +0800	[thread overview]
Message-ID: <5552FA7D.7000704@huawei.com> (raw)

Hi Jaegeuk,

I found a space leak problem in f2fs. This problem could lead to
ENOSPC error during stress tests, e.g. ltp.

<<<test_output>>>
growfiles(gf15): 11656 growfiles.c/2249: 16920 tlibio.c/739 write(6, buf, 1352) ret:-1, errno=28 No space left on device
gf15        1  TFAIL  :  growfiles.c:132: Test failed

...

And can be reproduced by these steps whether background_gc is on
or not:

1) format a 4GB f2fs partition
2) dd a 3G file,
3) unlink it.

Do these steps again and again. Soon, after one unlink operation,
you can see the space of the 3G file is not free.

Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks   Used Available Use% Mounted on
/dev/sdd3        4193280 301064   3854328   8% /mnt/f2fs
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
3072+0 records in
3072+0 records out
3221225472 bytes (3.2 GB) copied, 3.1892 s, 1.0 GB/s
Fs-Server:/mnt/f2fs # unlink ./test
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
3072+0 records in
3072+0 records out
3221225472 bytes (3.2 GB) copied, 3.44288 s, 936 MB/s
Fs-Server:/mnt/f2fs # unlink ./test
Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sdd3        4193280 3449888    705504  84% /mnt/f2fs
Fs-Server:/mnt/f2fs # ls
Fs-Server:/mnt/f2fs # ls
Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sdd3        4193280 3449888    705504  84% /mnt/f2fs
Fs-Server:/mnt/f2fs # dd if=/dev/zero of=./test bs=1M count=3072
dd: writing `./test': No space left on device
689+0 records in
688+0 records out
721719296 bytes (722 MB) copied, 0.618972 s, 1.2 GB/s
Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sdd3        4193280 4155392         0 100% /mnt/f2fs
Fs-Server:/mnt/f2fs # ls -l test
-rw-r--r-- 1 root root 721719296 May 13 14:52 test


We can reuse the leaking space after a sync call:


Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sdd3        4193280 4155392         0 100% /mnt/f2fs
Fs-Server:/mnt/f2fs # sync
Fs-Server:/mnt/f2fs # df .
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sdd3        4193280 1006568   3148824  25% /mnt/f2fs


I found this may caused by .drop_inode in f2fs. see f2fs_drop_inode()

diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 19438f2..7646d2a 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -424,15 +424,6 @@ static struct inode *f2fs_alloc_inode(struct super_block *sb)

 static int f2fs_drop_inode(struct inode *inode)
 {
-       /*
-        * This is to avoid a deadlock condition like below.
-        * writeback_single_inode(inode)
-        *  - f2fs_write_data_page
-        *    - f2fs_gc -> iput -> evict
-        *       - inode_wait_for_writeback(inode)
-        */
-       if (!inode_unhashed(inode) && inode->i_state & I_SYNC)
-               return 0;
        return generic_drop_inode(inode);
 }

After removing these code, this problem is fixed. But this function is
introduced by commit 531ad7d58c6476c5856653448b4c7d26427502b4 to fix
a deadlock problem.

I wish you and other developers in this list could help me to fix this
problem in a correct way.

Thanks,
Hu


             reply	other threads:[~2015-05-13  7:17 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-05-13  7:17 hujianyang [this message]
2015-05-13  7:17 ` Space leak in f2fs hujianyang
2015-05-13 17:46 ` [f2fs-dev] " Jaegeuk Kim
2015-05-14  0:24   ` Jaegeuk Kim
2015-05-14  1:40     ` hujianyang
2015-05-14  1:45       ` [f2fs-dev] " Jaegeuk Kim
2015-05-14 21:14       ` Jaegeuk Kim
2015-05-15  8:31         ` Chao Yu
2015-05-16  0:55           ` Jaegeuk Kim
2015-05-18  2:43             ` Chao Yu
2015-05-18  2:50               ` Nicholas Krause
2015-05-18  2:50                 ` Nicholas Krause
2015-05-18  5:44               ` [f2fs-dev] " Jaegeuk Kim

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5552FA7D.7000704@huawei.com \
    --to=hujianyang@huawei.com \
    --cc=chao2.yu@samsung.com \
    --cc=cm224.lee@samsung.com \
    --cc=jaegeuk@kernel.org \
    --cc=linux-f2fs-devel@lists.sourceforge.net \
    --cc=linux-fsdevel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.