* billion file testing of btrfs
@ 2010-08-16 12:29 Ric Wheeler
2010-08-16 12:37 ` Chris Mason
2010-08-19 0:37 ` Chris Mason
0 siblings, 2 replies; 4+ messages in thread
From: Ric Wheeler @ 2010-08-16 12:29 UTC (permalink / raw)
To: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 2379 bytes --]
I decided to try btrfs on F13 (2.6.33.6-147.2.4.fc13.x86_64 kernel) with the
following fs_mark command and a 1.5 TB Seagate S-ATA disk:
# fs_mark -s 0 -S 0 -D 1000 -n 1000000 -L 1000 -d /test/ -l btrfs_log.txt
btrfs starts off at a fantastic rate - roughly 3-4 times the speed of ext4:
FSUse% Count Size Files/sec App Overhead
0 1000000 0 20815.9 6257344
0 2000000 0 17531.2 6310061
0 3000000 0 10656.4 6473972
0 4000000 0 17933.6 6291377
0 5000000 0 6687.9 6708750
0 6000000 0 6494.3 6792701
0 7000000 0 18211.1 6266870
0 8000000 0 18518.4 6231522
Then it chugs along, but hits relatively long periods of slowness:
20 231000000 0 16991.8 6216216
20 232000000 0 17524.1 6290540
20 233000000 0 17088.5 6456494
20 234000000 0 1259.1 10185066
20 235000000 0 1419.0 8144068
20 236000000 0 1670.9 12154381
20 237000000 0 1863.6 8582337
20 238000000 0 929.1 9714860
21 239000000 0 4100.7 7385278
21 240000000 0 3486.6 8773568
21 241000000 0 2224.7 7320514
21 242000000 0 3761.8 7166617
21 243000000 0 3197.0 6918920
21 244000000 0 1293.1 8486926
21 245000000 0 3189.7 7396151
21 246000000 0 3164.7 7103912
21 247000000 0 4028.8 6770544
21 248000000 0 3698.7 7133084
21 249000000 0 2247.9 7421000
22 250000000 0 1579.3 9833236
I will fill this and get some plots, etc to do relative performance & then
wanted to try the current RC kernel just to refresh.
Ric
(Whole log file appended)
[-- Attachment #2: btrfs_log.txt --]
[-- Type: text/plain, Size: 16430 bytes --]
# fs_mark -s 0 -S 0 -D 1000 -n 1000000 -L 1000 -d /test/ -l btrfs_log.txt
# Version 3.3, 1 thread(s) starting at Sun Aug 15 22:42:42 2010
# Sync method: NO SYNC: Test does not issue sync() or fsync() calls.
# Directories: Time based hash between directories across 1000 subdirectories with 180 seconds per subdirectory.
# File names: 40 bytes long, (16 initial bytes of time stamp with 24 random bytes at end of name)
# Files info: size 0 bytes, written with an IO size of 16384 bytes per write
# App overhead is time in microseconds spent in the test not doing file writing related system calls.
#
FSUse% Count Size Files/sec App Overhead
0 1000000 0 20815.9 6257344
0 2000000 0 17531.2 6310061
0 3000000 0 10656.4 6473972
0 4000000 0 17933.6 6291377
0 5000000 0 6687.9 6708750
0 6000000 0 6494.3 6792701
0 7000000 0 18211.1 6266870
0 8000000 0 18518.4 6231522
0 9000000 0 18291.6 6242354
0 10000000 0 18797.3 6331131
1 11000000 0 17585.3 6323020
1 12000000 0 18160.9 6293072
1 13000000 0 17452.3 7033092
1 14000000 0 4050.4 7114239
1 15000000 0 3829.6 7220560
1 16000000 0 4115.2 7118247
1 17000000 0 12020.7 6364758
1 18000000 0 16576.1 6295093
1 19000000 0 16852.4 6293392
1 20000000 0 15563.5 6603340
1 21000000 0 13275.7 6546154
1 22000000 0 17274.9 6244028
2 23000000 0 17825.4 6328352
2 24000000 0 17159.0 6291665
2 25000000 0 16936.0 6280647
2 26000000 0 16816.1 6307828
2 27000000 0 16358.0 6374361
2 28000000 0 16899.2 6305802
2 29000000 0 18116.2 6281832
2 30000000 0 16591.6 6218579
2 31000000 0 17737.5 6234369
2 32000000 0 17487.4 6674911
2 33000000 0 15949.1 6289806
3 34000000 0 18070.0 6294203
3 35000000 0 17549.3 7035713
3 36000000 0 17365.5 6302612
3 37000000 0 17120.4 6309273
3 38000000 0 16950.7 6291340
3 39000000 0 16162.7 6350948
3 40000000 0 18372.7 6275539
3 41000000 0 16830.7 6268292
3 42000000 0 15992.6 6373618
3 43000000 0 18168.9 6301528
3 44000000 0 18061.7 6264057
3 45000000 0 16048.7 6346317
4 46000000 0 17334.9 6321253
4 47000000 0 3448.3 6838377
4 48000000 0 3516.7 8038983
4 49000000 0 3867.1 7227485
4 50000000 0 2888.4 7243821
4 51000000 0 2394.8 7216079
4 52000000 0 2498.6 6843414
4 53000000 0 2819.5 7060184
4 54000000 0 17219.0 6259943
4 55000000 0 15260.4 6367936
4 56000000 0 17522.6 6266963
5 57000000 0 17278.3 6306790
5 58000000 0 17827.9 6275829
5 59000000 0 4285.3 7437493
5 60000000 0 16448.4 6289245
5 61000000 0 16113.5 6325499
5 62000000 0 18155.8 6239094
5 63000000 0 4179.2 6651598
5 64000000 0 17690.2 6284524
5 65000000 0 17225.0 7155034
5 66000000 0 17310.0 6309914
5 67000000 0 17925.5 6297320
6 68000000 0 16592.5 6323214
6 69000000 0 17267.0 6282377
6 70000000 0 17570.8 6300443
6 71000000 0 17495.1 6309743
6 72000000 0 17521.6 6315327
6 73000000 0 17210.1 6620004
6 74000000 0 17117.6 6308142
6 75000000 0 17990.8 6269538
6 76000000 0 17220.2 6268183
6 77000000 0 16806.9 6299533
6 78000000 0 17071.6 6283797
6 79000000 0 16890.7 6660269
7 80000000 0 16579.1 6287810
7 81000000 0 17684.8 6286511
7 82000000 0 17263.7 6256444
7 83000000 0 15841.5 6315280
7 84000000 0 18360.0 6250738
7 85000000 0 17432.3 6307927
7 86000000 0 17294.0 6283318
7 87000000 0 17683.4 6281639
7 88000000 0 17109.2 6284513
7 89000000 0 17802.2 6365485
7 90000000 0 17277.2 6257247
8 91000000 0 16790.4 6354296
8 92000000 0 17396.6 6299773
8 93000000 0 16301.2 6326910
8 94000000 0 15742.4 6342880
8 95000000 0 17721.4 6245928
8 96000000 0 16865.1 6560790
8 97000000 0 16640.5 6282039
8 98000000 0 4287.5 6489531
8 99000000 0 16553.8 6301800
8 100000000 0 16489.6 6314946
8 101000000 0 18035.2 6280337
9 102000000 0 16251.7 6356805
9 103000000 0 16520.5 6327830
9 104000000 0 16970.1 6330138
9 105000000 0 16501.6 6343194
9 106000000 0 16790.0 6329598
9 107000000 0 17150.5 6225618
9 108000000 0 17503.5 6230408
9 109000000 0 16969.3 6322322
9 110000000 0 17535.4 6224087
9 111000000 0 2431.8 7065238
9 112000000 0 4203.4 6464506
9 113000000 0 15603.5 6334619
10 114000000 0 17903.2 6298115
10 115000000 0 16947.8 6283959
10 116000000 0 18118.8 6223771
10 117000000 0 1250.3 11497573
10 118000000 0 2168.3 8677721
10 119000000 0 2921.5 6998525
10 120000000 0 3902.7 7088629
10 121000000 0 3142.4 7104538
10 122000000 0 2621.8 7479713
10 123000000 0 3495.4 6812262
10 124000000 0 1280.9 9670585
11 125000000 0 4146.8 7053423
11 126000000 0 3079.9 7218964
11 127000000 0 2314.8 7201122
11 128000000 0 3771.3 7342388
11 129000000 0 2868.1 7491468
11 130000000 0 3842.0 7056393
11 131000000 0 2290.7 7607168
11 132000000 0 3135.0 7477374
11 133000000 0 5423.2 6864242
11 134000000 0 11873.0 6531883
11 135000000 0 15301.9 6541432
11 136000000 0 17808.3 6329129
12 137000000 0 3988.5 6745565
12 138000000 0 5161.1 6541221
12 139000000 0 16439.4 6287890
12 140000000 0 16342.1 6323441
12 141000000 0 17439.4 6330809
12 142000000 0 16882.0 6316980
12 143000000 0 16365.0 6297142
12 144000000 0 17550.7 6305619
12 145000000 0 17242.1 6352472
12 146000000 0 16119.6 6305376
12 147000000 0 17830.1 6284454
13 148000000 0 17229.8 6264212
13 149000000 0 16436.5 6333343
13 150000000 0 16922.7 6333859
13 151000000 0 3278.8 6877742
13 152000000 0 16787.3 6319630
13 153000000 0 16442.3 6295449
13 154000000 0 4177.2 6425930
13 155000000 0 17959.6 6305964
13 156000000 0 17138.8 6282213
13 157000000 0 16580.9 6294517
13 158000000 0 4248.8 6447632
14 159000000 0 18054.5 6275990
14 160000000 0 16402.8 6298766
14 161000000 0 16938.3 6255453
14 162000000 0 16445.8 6279203
14 163000000 0 4183.4 6504925
14 164000000 0 16150.3 6335906
14 165000000 0 17317.9 6326524
14 166000000 0 16635.8 6334818
14 167000000 0 16870.7 6343965
14 168000000 0 17101.9 6278377
14 169000000 0 4131.7 6447477
14 170000000 0 2790.1 6957939
15 171000000 0 4258.3 6547637
15 172000000 0 4144.9 6517516
15 173000000 0 16753.4 6310725
15 174000000 0 16321.8 6355113
15 175000000 0 1530.6 7590414
15 176000000 0 5685.5 6650047
15 177000000 0 16874.9 6295606
15 178000000 0 17090.0 6302893
15 179000000 0 16433.3 6297855
15 180000000 0 16141.8 6253448
15 181000000 0 16165.8 6309024
16 182000000 0 17838.1 6244481
16 183000000 0 16783.3 6256981
16 184000000 0 16437.4 6325421
16 185000000 0 17549.6 6279683
16 186000000 0 16268.9 6303004
16 187000000 0 16265.2 6294318
16 188000000 0 16747.9 6299414
16 189000000 0 16479.7 6309647
16 190000000 0 16483.0 6342135
16 191000000 0 16504.6 6305356
16 192000000 0 17565.2 7524570
17 193000000 0 16896.9 6330856
17 194000000 0 17781.8 6288532
17 195000000 0 16574.7 6288047
17 196000000 0 16541.5 6330765
17 197000000 0 17394.4 6264580
17 198000000 0 16538.1 6261857
17 199000000 0 16501.2 6291159
17 200000000 0 16685.6 6259176
17 201000000 0 17689.4 7061755
17 202000000 0 17507.0 6304072
17 203000000 0 18049.2 6304002
17 204000000 0 16178.9 6300915
18 205000000 0 9445.2 6416147
18 206000000 0 4183.0 6501230
18 207000000 0 17040.1 6280866
18 208000000 0 17637.4 6288235
18 209000000 0 5166.5 6693375
18 210000000 0 16564.9 6266877
18 211000000 0 16242.0 6280384
18 212000000 0 2387.9 6597650
18 213000000 0 15703.4 6325825
18 214000000 0 17280.9 6278357
18 215000000 0 16712.3 6374671
19 216000000 0 15901.2 6356488
19 217000000 0 17927.5 6272836
19 218000000 0 4185.8 6621414
19 219000000 0 7264.8 6474084
19 220000000 0 4190.6 6501562
19 221000000 0 17371.4 6284265
19 222000000 0 16956.5 6284733
19 223000000 0 16041.5 6273139
19 224000000 0 17289.1 6291473
19 225000000 0 18191.0 6305787
19 226000000 0 16150.2 6311278
20 227000000 0 17520.5 6290320
20 228000000 0 16998.6 6223947
20 229000000 0 16410.2 6252458
20 230000000 0 17867.6 6275663
20 231000000 0 16991.8 6216216
20 232000000 0 17524.1 6290540
20 233000000 0 17088.5 6456494
20 234000000 0 1259.1 10185066
20 235000000 0 1419.0 8144068
20 236000000 0 1670.9 12154381
20 237000000 0 1863.6 8582337
20 238000000 0 929.1 9714860
21 239000000 0 4100.7 7385278
21 240000000 0 3486.6 8773568
21 241000000 0 2224.7 7320514
21 242000000 0 3761.8 7166617
21 243000000 0 3197.0 6918920
21 244000000 0 1293.1 8486926
21 245000000 0 3189.7 7396151
21 246000000 0 3164.7 7103912
21 247000000 0 4028.8 6770544
21 248000000 0 3698.7 7133084
21 249000000 0 2247.9 7421000
22 250000000 0 1579.3 9833236
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: billion file testing of btrfs
2010-08-16 12:29 billion file testing of btrfs Ric Wheeler
@ 2010-08-16 12:37 ` Chris Mason
2010-08-16 13:10 ` Ric Wheeler
2010-08-19 0:37 ` Chris Mason
1 sibling, 1 reply; 4+ messages in thread
From: Chris Mason @ 2010-08-16 12:37 UTC (permalink / raw)
To: Ric Wheeler; +Cc: linux-btrfs
On Mon, Aug 16, 2010 at 08:29:24AM -0400, Ric Wheeler wrote:
>
> I decided to try btrfs on F13 (2.6.33.6-147.2.4.fc13.x86_64 kernel)
> with the following fs_mark command and a 1.5 TB Seagate S-ATA disk:
>
> # fs_mark -s 0 -S 0 -D 1000 -n 1000000 -L 1000 -d /test/ -l btrfs_log.txt
>
> btrfs starts off at a fantastic rate - roughly 3-4 times the speed of ext4:
>
[ log ]
>
> I will fill this and get some plots, etc to do relative performance
> & then wanted to try the current RC kernel just to refresh.
Thanks Ric, I think this is related to the perf problem delalloc
flushing performance bug. I'll nail it down shortly.
-chris
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: billion file testing of btrfs
2010-08-16 12:37 ` Chris Mason
@ 2010-08-16 13:10 ` Ric Wheeler
0 siblings, 0 replies; 4+ messages in thread
From: Ric Wheeler @ 2010-08-16 13:10 UTC (permalink / raw)
To: Chris Mason, linux-btrfs
On 08/16/2010 08:37 AM, Chris Mason wrote:
> On Mon, Aug 16, 2010 at 08:29:24AM -0400, Ric Wheeler wrote:
>> I decided to try btrfs on F13 (2.6.33.6-147.2.4.fc13.x86_64 kernel)
>> with the following fs_mark command and a 1.5 TB Seagate S-ATA disk:
>>
>> # fs_mark -s 0 -S 0 -D 1000 -n 1000000 -L 1000 -d /test/ -l btrfs_log.txt
>>
>> btrfs starts off at a fantastic rate - roughly 3-4 times the speed of ext4:
>>
> [ log ]
>
>> I will fill this and get some plots, etc to do relative performance
>> & then wanted to try the current RC kernel just to refresh.
> Thanks Ric, I think this is related to the perf problem delalloc
> flushing performance bug. I'll nail it down shortly.
>
> -chris
Great, I will be happy to test patches when you have them. No particular hurry
on my end.
I also hope to get our new arrays online and should be able to steal a couple of
days run there with luck,
Ric
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: billion file testing of btrfs
2010-08-16 12:29 billion file testing of btrfs Ric Wheeler
2010-08-16 12:37 ` Chris Mason
@ 2010-08-19 0:37 ` Chris Mason
1 sibling, 0 replies; 4+ messages in thread
From: Chris Mason @ 2010-08-19 0:37 UTC (permalink / raw)
To: Ric Wheeler; +Cc: linux-btrfs
On Mon, Aug 16, 2010 at 08:29:24AM -0400, Ric Wheeler wrote:
>
> I decided to try btrfs on F13 (2.6.33.6-147.2.4.fc13.x86_64 kernel)
> with the following fs_mark command and a 1.5 TB Seagate S-ATA disk:
>
> # fs_mark -s 0 -S 0 -D 1000 -n 1000000 -L 1000 -d /test/ -l btrfs_log.txt
>
> btrfs starts off at a fantastic rate - roughly 3-4 times the speed of ext4:
The results are interesting, both ext4 and btrfs seem to have a little
more than a million inodes in slab cache on my box. The ext4 inode is
a little fatter so it's about 2.2GB instead of 1.7GB.
I'll let btrfs run overnight. ext4 isn't doing many reads at all but is
instead stuck in the log, but there's very little IO. I'm not actually
sure what it is doing.
Btrfs is doing a healthy amount of writes and batches of reads.
-chris
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2010-08-19 0:37 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-08-16 12:29 billion file testing of btrfs Ric Wheeler
2010-08-16 12:37 ` Chris Mason
2010-08-16 13:10 ` Ric Wheeler
2010-08-19 0:37 ` Chris Mason
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).