* benchmark: write barrier/write cache on XFS
@ 2008-12-08 20:39 Martin Steigerwald
2008-12-08 21:43 ` Martin Steigerwald
0 siblings, 1 reply; 2+ messages in thread
From: Martin Steigerwald @ 2008-12-08 20:39 UTC (permalink / raw)
To: xfs; +Cc: linux-fsdevel, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 6836 bytes --]
Hi!
I got curious about the recent discussions about the write barrier
feature[1][2].
Thus I did my own benchmark this evening. Since I use XFS and tested with
XFS for now only. Write barrier + write cache, no barrier + write cache,
no write barrier + no write cache. I just did tar -xf
linux-2.6.27.tar.bz2 and rm -rf linux-2.6.27.
My conclusion is: At least for this metadata intensive workload enabling
write barries is complete nonsense for XFS filesystems, cause it runs way
faster without write cache and without barriers. I am completely puzzled
about this, cause I always thought that barriers where meant to provide a
performance improvement versus disabling write cache. I actually
advertised them as such in my Linux magazine article, should have benched
them before it seems. Write barrier seem to be a slow down feature for
XFS.
See for yourself. I am interested in other benchmarks like this.
Automating this would be good and running it for different filesystems.
Maybe adding some different workloads as these are highly selective
tests, testing with RAID systems as Justin did and of course testing with
different filesystems.
I think I will disable write cache and disable barriers on that ThinkPad
T42 and I think also other machines.
IBM ThinkPad T42 with 160 GB IDE hitachi drive via libata:
shambhala:~> grep "model name" /proc/cpuinfo
model name : Intel(R) Pentium(R) M processor 1.80GHz
shambhala:~> cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
ondemand
shambhala:~> cat /proc/version
Linux version 2.6.27.7-tp42-toi-3.0-2008-11-25 (martin@shambhala) (gcc
version 4.3.2 (Debian 4.3.2-1) ) #1 PREEMPT Sun Nov 30 10:29:09 CET 2008
shambhala:~> hdparm -I /dev/sda | egrep "(Model Num|device size|power
manage|acoustic manage|DMA:|120 ns)"
Model Number: Hitachi HTS541616J9AT00
device size with M = 1024*1024: 152627 MBytes
device size with M = 1000*1000: 160041 MBytes (160 GB)
Advanced power management level: 254
Recommended acoustic management value: 128, current value: 128
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5
The default - write barrier and write cache enabled:
shambhala:~> hdparm -W /dev/sda
/dev/sda:
write-caching = 1 (on)
shambhala:~> hdparm -I /dev/sda | grep cache
* Write cache
shambhala:~> mkfs.xfs -f -L xfs -l lazy-count=1 /dev/sda6
meta-data=/dev/sda6 isize=256 agcount=4, agsize=1344188
blks
= sectsz=512 attr=2
data = bsize=4096 blocks=5376750, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096
log =internal log bsize=4096 blocks=2625, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
shambhala:~> mount /dev/sda6 /mnt/zeit
shambhala:~> grep zeit /proc/mounts
/dev/sda6 /mnt/zeit xfs rw,attr2,noquota 0 0
shambhala:~> cd /mnt/zeit
shambhala:/mnt/zeit> sync ; time
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2; time
sync
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 46,01s
user 5,59s system 41% cpu 2:03,01 total
sync 0,00s user 0,18s system 3% cpu 5,303 total
shambhala:/mnt/zeit> sync ; time rm -rf linux-2.6.27 ; time sync
rm -rf linux-2.6.27 0,08s user 3,49s system 6% cpu 51,482 total
sync 0,00s user 0,15s system 45% cpu 0,320 total
shambhala:/mnt/zeit> cd
shambhala:~> umount /mnt/zeit
Write barriers disabled, write cache enabled:
shambhala:~> hdparm -W /dev/sda
/dev/sda:
write-caching = 1 (on)
shambhala:~> hdparm -I /dev/sda | grep cache
* Write cache
shambhala:~> mkfs.xfs -f -L xfs -l lazy-count=1 /dev/sda6
meta-data=/dev/sda6 isize=256 agcount=4, agsize=1344188
blks
= sectsz=512 attr=2
data = bsize=4096 blocks=5376750, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096
log =internal log bsize=4096 blocks=2625, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
shambhala:~> mount -o nobarrier /dev/sda6 /mnt/zeit
shambhala:~> cd /mnt/zeit
shambhala:/mnt/zeit> sync ; time
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2; time
sync
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 26,77s
user 3,20s system 67% cpu 44,236 total
sync 0,00s user 0,17s system 5% cpu 3,224 total
shambhala:/mnt/zeit> sync ; time rm -rf linux-2.6.27 ; time sync
rm -rf linux-2.6.27 0,05s user 3,20s system 43% cpu 7,442 total
sync 0,00s user 0,14s system 44% cpu 0,309 total
shambhala:/mnt/zeit> cd
shambhala:~> umount /mnt/zeit
Write barriers and write cache disabled:
shambhala:~> hdparm -W0 /dev/sda
/dev/sda:
setting drive write-caching to 0 (off)
write-caching = 0 (off)
shambhala:~> hdparm -I /dev/sda | grep cache
Write cache
shambhala:~> mkfs.xfs -f -L xfs -l lazy-count=1 /dev/sda6
meta-data=/dev/sda6 isize=256 agcount=4, agsize=1344188
blks
= sectsz=512 attr=2
data = bsize=4096 blocks=5376750, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096
log =internal log bsize=4096 blocks=2625, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
shambhala:~> mount -o nobarrier /dev/sda6 /mnt/zeit
shambhala:~> cd /mnt/zeit
shambhala:/mnt/zeit> sync ; time
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2; time
sync
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 35,22s
user 3,89s system 53% cpu 1:12,71 total
sync 0,00s user 0,17s system 7% cpu 2,355 total
shambhala:/mnt/zeit> sync ; time rm -rf linux-2.6.27 ; time sync
rm -rf linux-2.6.27 0,07s user 2,82s system 12% cpu 23,391 total
sync 0,00s user 0,14s system 44% cpu 0,329 total
shambhala:/mnt/zeit> cd
shambhala:~> umount /mnt/zeit
shambhala:~> date
Mo 8. Dez 21:24:42 CET 2008
[1] http://oss.sgi.com/archives/xfs/2008-12/msg00219.html
[2] http://oss.sgi.com/archives/xfs/2008-12/msg00161.html or
http://lkml.org/lkml/2008/12/4/169
Ciao,
--
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: benchmark: write barrier/write cache on XFS
2008-12-08 20:39 benchmark: write barrier/write cache on XFS Martin Steigerwald
@ 2008-12-08 21:43 ` Martin Steigerwald
0 siblings, 0 replies; 2+ messages in thread
From: Martin Steigerwald @ 2008-12-08 21:43 UTC (permalink / raw)
To: xfs; +Cc: linux-fsdevel, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 4792 bytes --]
Am Montag 08 Dezember 2008 schrieb Martin Steigerwald:
> Hi!
>
> I got curious about the recent discussions about the write barrier
> feature[1][2].
>
> Thus I did my own benchmark this evening. Since I use XFS and tested
> with XFS for now only. Write barrier + write cache, no barrier + write
> cache, no write barrier + no write cache. I just did tar -xf
> linux-2.6.27.tar.bz2 and rm -rf linux-2.6.27.
I disabled write cache and barriers for my XFS filesystems on my TP 42
shambhala. During that I did further tests. This time I also include IO
scheduler settings. They are default and have been the same during my
previous tests.
Write cache has about 13 seconds benefit on the barrier enabled XFS for
the tar -xf linux-2.6.27.tar.bz2 workload but no benefit at all for the
rm -rf linux-2.6.27 workload!
And again nobarrier and no cache wins.
On a heavily used /home-XFS - I know it should have more free space:
martin@shambhala:~ -> LANG=C df -hT /home/
Filesystem Type Size Used Avail Use% Mounted on
/dev/sda5 xfs 112G 107G 5.1G 96% /home
It has been grown two times - since 6 AGs instead of the default of 4:
shambhala:~> xfs_info /home
meta-data=/dev/sda5 isize=256 agcount=6, agsize=4883256
blks
= sectsz=512 attr=2
data = bsize=4096 blocks=29299536, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096
log =internal bsize=4096 blocks=32768, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
IO scheduler settings:
martin@shambhala:~> cat /sys/block/sda/queue/scheduler
noop anticipatory deadline [cfq]
martin@shambhala:~> grep "" /sys/block/sda/queue/iosched/*
/sys/block/sda/queue/iosched/back_seek_max:16384
/sys/block/sda/queue/iosched/back_seek_penalty:2
/sys/block/sda/queue/iosched/fifo_expire_async:250
/sys/block/sda/queue/iosched/fifo_expire_sync:123
/sys/block/sda/queue/iosched/quantum:4
/sys/block/sda/queue/iosched/slice_async:40
/sys/block/sda/queue/iosched/slice_async_rq:2
/sys/block/sda/queue/iosched/slice_idle:6
/sys/block/sda/queue/iosched/slice_sync:100
Write barriers and no write cache:
shambhala:~> hdparm -W /dev/sda
/dev/sda:
write-caching = 0 (off)
shambhala:> cat /proc/mounts | egrep "(/ |home)"
rootfs / rootfs rw 0 0
/dev/disk/by-uuid/7fcd9766-bf1a-426a-8a07-2c3e0c510898 / xfs
rw,relatime,attr2,noquota 0 0
/dev/sda5 /home xfs rw,relatime,attr2,logbufs=8,logbsize=256k,noquota 0 0
martin@shambhala:~/Zeit> sync; time tar -xf
~/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 ; time sync
tar -xf ~/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 36,71s user 4,22s
system 36% cpu 1:51,58 total
sync 0,00s user 0,05s system 0% cpu 12,689 total
martin@shambhala:~/Zeit> sync; time rm -rf linux-2.6.27 ; time sync
rm -rf linux-2.6.27 0,06s user 3,90s system 17% cpu 22,906 total
sync 0,00s user 0,01s system 6% cpu 0,103 total
Write barriers and write cache:
shambhala:~> hdparm -W1 /dev/sda
/dev/sda:
setting drive write-caching to 1 (on)
write-caching = 1 (on)
martin@shambhala:~/Zeit> sync; time tar -xf
~/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 ; time sync
tar -xf ~/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 34,38s user 3,91s
system 38% cpu 1:38,84 total
sync 0,00s user 0,04s system 0% cpu 10,493 total
martin@shambhala:~/Zeit> sync; time rm -rf linux-2.6.27 ; time sync
rm -rf linux-2.6.27 0,07s user 3,98s system 17% cpu 23,511 total
sync 0,00s user 0,01s system 5% cpu 0,126 total
No write barriers and no write cache:
shambhala:> vim fstab
shambhala:> mount -o remount /
shambhala:> mount -o remount /home
shambhala:> cat /proc/mounts | egrep "(/ |home)"
rootfs / rootfs rw 0 0
/dev/disk/by-uuid/7fcd9766-bf1a-426a-8a07-2c3e0c510898 / xfs
rw,relatime,attr2,nobarrier,noquota 0 0
/dev/sda5 /home xfs
rw,relatime,attr2,nobarrier,logbufs=8,logbsize=256k,noquota 0 0
martin@shambhala:~/Zeit> sync; time tar -xf
~/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 ; time sync
tar -xf ~/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 30,36s user 3,31s
system 48% cpu 1:08,94 total
sync 0,00s user 0,08s system 0% cpu 17,462 total
martin@shambhala:~/Zeit> sync; time rm -rf linux-2.6.27 ; time sync
rm -rf linux-2.6.27 0,07s user 3,87s system 20% cpu 19,172 total
sync 0,00s user 0,01s system 4% cpu 0,142 total
martin@shambhala:~> date
Mo 8. Dez 22:38:06 CET 2008
Ciao,
--
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2008-12-08 21:43 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-12-08 20:39 benchmark: write barrier/write cache on XFS Martin Steigerwald
2008-12-08 21:43 ` Martin Steigerwald
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox