From: ching <lsching17@gmail.com>
To: cwillu <cwillu@cwillu.com>
Cc: Hugo Mills <hugo@carfax.org.uk>,
Felix Pepinghege <postfach@pepinghege.net>,
"linux-btrfs@vger.kernel.org" <linux-btrfs@vger.kernel.org>
Subject: Re: Why btrfs inline small file by default?
Date: Thu, 01 Nov 2012 05:05:04 +0800 [thread overview]
Message-ID: <50919280.1030606@gmail.com> (raw)
In-Reply-To: <CAE5mzvgMskhBgH506BageXREifs63Zn_HKDaWBMuesS8Np66SQ@mail.gmail.com>
On 10/31/2012 08:18 AM, cwillu wrote:
> import os
> import sys
>
> data = "1" * 1024 * 3
>
> for x in xrange(100 * 1000):
> with open('%s/%s' % (sys.argv[1], x), 'a') as f:
> f.write(data)
>
> root@repository:~$ mount -o loop ~/inline /mnt
> root@repository:~$ mount -o loop,max_inline=0 ~/noninline /mnt2
>
> root@repository:~$ time python test.py /mnt
> real 0m11.105s
> user 0m1.328s
> sys 0m5.416s
> root@repository:~$ time python test.py /mnt2
> real 0m21.905s
> user 0m1.292s
> sys 0m5.460s
>
> root@repository:/$ btrfs fi df /mnt
> Data: total=1.01GB, used=256.00KB
> System, DUP: total=8.00MB, used=4.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=652.70MB
> Metadata: total=8.00MB, used=0.00
>
> root@repository:/$ btrfs fi df /mnt2
> Data: total=1.01GB, used=391.12MB
> System, DUP: total=8.00MB, used=4.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=60.98MB
> Metadata: total=8.00MB, used=0.00
>
> 3k data, 4k leaf: inline is twice the speed, but 1.4x bigger.
>
> ----
>
> root@repository:~$ mkfs.btrfs inline -l 64k
> root@repository:~$ mkfs.btrfs noninline -l 64k
> ...
> root@repository:~$ time python test.py /mnt
> real 0m12.244s
> user 0m1.396s
> sys 0m8.101s
> root@repository:~$ time python test.py /mnt2
> real 0m13.047s
> user 0m1.436s
> sys 0m7.772s
>
> root@repository:/$ btr\fs fi df /mnt
> Data: total=8.00MB, used=256.00KB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=342.06MB
> Metadata: total=8.00MB, used=0.00
>
> root@repository:/$ btr\fs fi df /mnt2
> Data: total=1.01GB, used=391.10MB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=50.06MB
> Metadata: total=8.00MB, used=0.00
>
> 3k data, 64k leaf: inline is still 10% faster, and is now 25% smaller
>
> ----
>
> data = "1" * 1024 * 32
>
> ... (mkfs, mount, etc)
>
> root@repository:~$ time python test.py /mnt
> real 0m17.834s
> user 0m1.224s
> sys 0m4.772s
> root@repository:~$ time python test.py /mnt2
> real 0m20.521s
> user 0m1.304s
> sys 0m6.344s
>
> root@repository:/$ btrfs fi df /mnt
> Data: total=4.01GB, used=3.05GB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=54.00MB
> Metadata: total=8.00MB, used=0.00
>
> root@repository:/$ btrfs fi df /mnt2
> Data: total=4.01GB, used=3.05GB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=53.56MB
> Metadata: total=8.00MB, used=0.00
>
> 32k data, 64k leaf: inline is still 10% faster, and is now the same
> size (not dead sure why, probably some interaction with the size of
> the actual write that happens)
>
> ----
>
> data = "1" * 1024 * 7
>
> ... etc
>
>
> root@repository:~$ time python test.py /mnt
> real 0m9.628s
> user 0m1.368s
> sys 0m4.188s
> root@repository:~$ time python test.py /mnt2
> real 0m13.455s
> user 0m1.608s
> sys 0m7.884s
>
> root@repository:/$ btrfs fi df /mnt
> Data: total=3.01GB, used=1.91GB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=74.69MB
> Metadata: total=8.00MB, used=0.00
>
> root@repository:/$ btrfs fi df /mnt2
> Data: total=3.01GB, used=1.91GB
> System, DUP: total=8.00MB, used=64.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=1.00GB, used=74.69MB
> Metadata: total=8.00MB, used=0.00
>
> 7k data, 64k leaf: 30% faster, same data usage.
>
> ----
>
> Are we done yet? Can I go home now? ;p
>
thanks for the test.
but the result just indicate the inline small file is not a "safe" optimization to be turned on by default.
next prev parent reply other threads:[~2012-10-31 21:05 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-30 11:04 Why btrfs inline small file by default? ching
2012-10-30 12:04 ` Felix Pepinghege
2012-10-30 12:17 ` cwillu
2012-10-30 21:40 ` ching
2012-10-30 22:14 ` Hugo Mills
2012-10-30 22:19 ` Hugo Mills
2012-10-30 23:47 ` ching
2012-10-31 0:12 ` David Sterba
2012-10-31 21:07 ` ching
2012-10-31 0:18 ` cwillu
2012-10-31 8:48 ` Ahmet Inan
2012-10-31 9:39 ` cwillu
2012-10-31 10:48 ` Ahmet Inan
2012-10-31 10:55 ` Michael Kjörling
2012-10-31 11:10 ` Ahmet Inan
2012-10-31 10:57 ` cwillu
2012-10-31 11:56 ` Michael Kjörling
2012-10-31 13:27 ` Ahmet Inan
2012-10-31 13:44 ` Roman Mamedov
2012-10-31 21:05 ` ching [this message]
2012-10-30 22:16 ` cwillu
2012-10-30 23:41 ` ching
2012-10-30 21:39 ` ching
2012-10-30 12:11 ` Felix Pepinghege
2012-10-30 12:13 ` Mitch Harder
2012-10-30 16:38 ` David Sterba
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=50919280.1030606@gmail.com \
--to=lsching17@gmail.com \
--cc=cwillu@cwillu.com \
--cc=hugo@carfax.org.uk \
--cc=linux-btrfs@vger.kernel.org \
--cc=postfach@pepinghege.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.