From: ching <lsching17@gmail.com>
To: cwillu <cwillu@cwillu.com>
Cc: Hugo Mills <hugo@carfax.org.uk>,
Felix Pepinghege <postfach@pepinghege.net>,
"linux-btrfs@vger.kernel.org" <linux-btrfs@vger.kernel.org>
Subject: Re: Why btrfs inline small file by default?
Date: Thu, 01 Nov 2012 05:05:04 +0800 [thread overview]
Message-ID: <50919280.1030606@gmail.com> (raw)
In-Reply-To: <CAE5mzvgMskhBgH506BageXREifs63Zn_HKDaWBMuesS8Np66SQ@mail.gmail.com>
On 10/31/2012 08:18 AM, cwillu wrote:
> import os
> import sys
>
> data = "1" * 1024 * 3
>
> for x in xrange(100 * 1000):
> with open('%s/%s' % (sys.argv[1], x), 'a') as f:
> f.write(data)
>
> root@repository:~$ mount -o loop ~/inline /mnt
> root@repository:~$ mount -o loop,max_inline=0 ~/noninline /mnt2
>
> root@repository:~$ time python test.py /mnt
> real 0m11.105s
> user 0m1.328s
> sys 0m5.416s
> root@repository:~$ time python test.py /mnt2
> real 0m21.905s
> user 0m1.292s
> sys 0m5.460s
>
> root@repository:/$ btrfs fi df /mnt
> Data: total=1.01GB, used=256.00KB
> System, DUP: total=8.00MB, used=4.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=652.70MB
> Metadata: total=8.00MB, used=0.00
>
> root@repository:/$ btrfs fi df /mnt2
> Data: total=1.01GB, used=391.12MB
> System, DUP: total=8.00MB, used=4.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=60.98MB
> Metadata: total=8.00MB, used=0.00
>
> 3k data, 4k leaf: inline is twice the speed, but 1.4x bigger.
>
> ----
>
> root@repository:~$ mkfs.btrfs inline -l 64k
> root@repository:~$ mkfs.btrfs noninline -l 64k
> ...
> root@repository:~$ time python test.py /mnt
> real 0m12.244s
> user 0m1.396s
> sys 0m8.101s
> root@repository:~$ time python test.py /mnt2
> real 0m13.047s
> user 0m1.436s
> sys 0m7.772s
>
> root@repository:/$ btr\fs fi df /mnt
> Data: total=8.00MB, used=256.00KB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=342.06MB
> Metadata: total=8.00MB, used=0.00
>
> root@repository:/$ btr\fs fi df /mnt2
> Data: total=1.01GB, used=391.10MB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=50.06MB
> Metadata: total=8.00MB, used=0.00
>
> 3k data, 64k leaf: inline is still 10% faster, and is now 25% smaller
>
> ----
>
> data = "1" * 1024 * 32
>
> ... (mkfs, mount, etc)
>
> root@repository:~$ time python test.py /mnt
> real 0m17.834s
> user 0m1.224s
> sys 0m4.772s
> root@repository:~$ time python test.py /mnt2
> real 0m20.521s
> user 0m1.304s
> sys 0m6.344s
>
> root@repository:/$ btrfs fi df /mnt
> Data: total=4.01GB, used=3.05GB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=54.00MB
> Metadata: total=8.00MB, used=0.00
>
> root@repository:/$ btrfs fi df /mnt2
> Data: total=4.01GB, used=3.05GB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=53.56MB
> Metadata: total=8.00MB, used=0.00
>
> 32k data, 64k leaf: inline is still 10% faster, and is now the same
> size (not dead sure why, probably some interaction with the size of
> the actual write that happens)
>
> ----
>
> data = "1" * 1024 * 7
>
> ... etc
>
>
> root@repository:~$ time python test.py /mnt
> real 0m9.628s
> user 0m1.368s
> sys 0m4.188s
> root@repository:~$ time python test.py /mnt2
> real 0m13.455s
> user 0m1.608s
> sys 0m7.884s
>
> root@repository:/$ btrfs fi df /mnt
> Data: total=3.01GB, used=1.91GB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=74.69MB
> Metadata: total=8.00MB, used=0.00
>
> root@repository:/$ btrfs fi df /mnt2
> Data: total=3.01GB, used=1.91GB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=74.69MB
> Metadata: total=8.00MB, used=0.00
>
> 7k data, 64k leaf: 30% faster, same data usage.
>
> ----
>
> Are we done yet? Can I go home now? ;p
>
thanks for the test.
but the result just indicate the inline small file is not a "safe" optimization to be turned on by default.
next prev parent reply other threads:[~2012-10-31 21:05 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-30 11:04 Why btrfs inline small file by default? ching
2012-10-30 12:04 ` Felix Pepinghege
2012-10-30 12:17 ` cwillu
2012-10-30 21:40 ` ching
2012-10-30 22:14 ` Hugo Mills
2012-10-30 22:19 ` Hugo Mills
2012-10-30 23:47 ` ching
2012-10-31 0:12 ` David Sterba
2012-10-31 21:07 ` ching
2012-10-31 0:18 ` cwillu
2012-10-31 8:48 ` Ahmet Inan
2012-10-31 9:39 ` cwillu
2012-10-31 10:48 ` Ahmet Inan
2012-10-31 10:55 ` Michael Kjörling
2012-10-31 11:10 ` Ahmet Inan
2012-10-31 10:57 ` cwillu
2012-10-31 11:56 ` Michael Kjörling
2012-10-31 13:27 ` Ahmet Inan
2012-10-31 13:44 ` Roman Mamedov
2012-10-31 21:05 ` ching [this message]
2012-10-30 22:16 ` cwillu
2012-10-30 23:41 ` ching
2012-10-30 21:39 ` ching
2012-10-30 12:11 ` Felix Pepinghege
2012-10-30 12:13 ` Mitch Harder
2012-10-30 16:38 ` David Sterba
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=50919280.1030606@gmail.com \
--to=lsching17@gmail.com \
--cc=cwillu@cwillu.com \
--cc=hugo@carfax.org.uk \
--cc=linux-btrfs@vger.kernel.org \
--cc=postfach@pepinghege.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).