From: David Howells <dhowells@redhat.com>
Cc: dhowells@redhat.com, Daniel Phillips <phillips@phunq.net>,
Trond.Myklebust@netapp.com, chuck.lever@oracle.com,
casey@schaufler-ca.com, nfsv4@linux-nfs.org,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
selinux@tycho.nsa.gov, linux-security-module@vger.kernel.org
Subject: Re: [PATCH 00/37] Permit filesystem local caching
Date: Fri, 22 Feb 2008 16:14:35 +0000 [thread overview]
Message-ID: <19020.1203696875@redhat.com> (raw)
In-Reply-To: <17916.1203636833@redhat.com>
David Howells <dhowells@redhat.com> wrote:
> > > Have you got before/after benchmark results?
> >
> > See attached.
>
> Attached here are results using BTRFS (patched so that it'll work at all)
> rather than Ext3 on the client on the partition backing the cache.
And here are XFS results.
Tuning XFS makes a *really* big difference for the lots of small/medium files
being tarred case. However, in general BTRFS is much better.
David
---
=========================
FEW BIG FILES TEST ON XFS
=========================
Completely cold caches:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m2.286s
user 0m0.000s
sys 0m1.828s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m4.228s
user 0m0.000s
sys 0m1.360s
Warm NFS pagecache:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m0.058s
user 0m0.000s
sys 0m0.060s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m0.122s
user 0m0.000s
sys 0m0.120s
Warm XFS pagecache, cold NFS pagecache:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m0.181s
user 0m0.000s
sys 0m0.180s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m1.034s
user 0m0.000s
sys 0m0.404s
Warm on-disk cache, cold pagecaches:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m1.540s
user 0m0.000s
sys 0m0.256s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m3.003s
user 0m0.000s
sys 0m0.532s
==========================================
MANY SMALL/MEDIUM FILE READING TEST ON XFS
==========================================
Completely cold caches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 4m56.827s
user 0m0.180s
sys 0m6.668s
Warm NFS pagecache:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 0m15.084s
user 0m0.212s
sys 0m5.008s
Warm XFS pagecache, cold NFS pagecache:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 0m13.547s
user 0m0.220s
sys 0m5.652s
Warm on-disk cache, cold pagecaches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 4m36.316s
user 0m0.148s
sys 0m4.440s
=======================================================
MANY SMALL/MEDIUM FILE READING TEST ON AN OPTIMISED XFS
=======================================================
mkfs.xfs -d agcount=4 -l size=128m,version=2 /dev/sda6
Completely cold caches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 3m44.033s
user 0m0.248s
sys 0m6.632s
Warm on-disk cache, cold pagecaches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 3m8.582s
user 0m0.108s
sys 0m3.420s
--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.
WARNING: multiple messages have this Message-ID (diff)
From: David Howells <dhowells@redhat.com>
To: unlisted-recipients:; (no To-header on input)
Cc: dhowells@redhat.com, Daniel Phillips <phillips@phunq.net>,
Trond.Myklebust@netapp.com, chuck.lever@oracle.com,
casey@schaufler-ca.com, nfsv4@linux-nfs.org,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
selinux@tycho.nsa.gov, linux-security-module@vger.kernel.org
Subject: Re: [PATCH 00/37] Permit filesystem local caching
Date: Fri, 22 Feb 2008 16:14:35 +0000 [thread overview]
Message-ID: <19020.1203696875@redhat.com> (raw)
In-Reply-To: <17916.1203636833@redhat.com>
David Howells <dhowells@redhat.com> wrote:
> > > Have you got before/after benchmark results?
> >
> > See attached.
>
> Attached here are results using BTRFS (patched so that it'll work at all)
> rather than Ext3 on the client on the partition backing the cache.
And here are XFS results.
Tuning XFS makes a *really* big difference for the lots of small/medium files
being tarred case. However, in general BTRFS is much better.
David
---
=========================
FEW BIG FILES TEST ON XFS
=========================
Completely cold caches:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m2.286s
user 0m0.000s
sys 0m1.828s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m4.228s
user 0m0.000s
sys 0m1.360s
Warm NFS pagecache:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m0.058s
user 0m0.000s
sys 0m0.060s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m0.122s
user 0m0.000s
sys 0m0.120s
Warm XFS pagecache, cold NFS pagecache:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m0.181s
user 0m0.000s
sys 0m0.180s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m1.034s
user 0m0.000s
sys 0m0.404s
Warm on-disk cache, cold pagecaches:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m1.540s
user 0m0.000s
sys 0m0.256s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m3.003s
user 0m0.000s
sys 0m0.532s
==========================================
MANY SMALL/MEDIUM FILE READING TEST ON XFS
==========================================
Completely cold caches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 4m56.827s
user 0m0.180s
sys 0m6.668s
Warm NFS pagecache:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 0m15.084s
user 0m0.212s
sys 0m5.008s
Warm XFS pagecache, cold NFS pagecache:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 0m13.547s
user 0m0.220s
sys 0m5.652s
Warm on-disk cache, cold pagecaches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 4m36.316s
user 0m0.148s
sys 0m4.440s
=======================================================
MANY SMALL/MEDIUM FILE READING TEST ON AN OPTIMISED XFS
=======================================================
mkfs.xfs -d agcount=4 -l size=128m,version=2 /dev/sda6
Completely cold caches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 3m44.033s
user 0m0.248s
sys 0m6.632s
Warm on-disk cache, cold pagecaches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 3m8.582s
user 0m0.108s
sys 0m3.420s
WARNING: multiple messages have this Message-ID (diff)
From: David Howells <dhowells@redhat.com>
Cc: Trond.Myklebust@netapp.com, nfsv4@linux-nfs.org,
linux-kernel@vger.kernel.org, dhowells@redhat.com,
linux-fsdevel@vger.kernel.org,
linux-security-module@vger.kernel.org, selinux@tycho.nsa.gov,
casey@schaufler-ca.com, Daniel Phillips <phillips@phunq.net>
Subject: Re: [PATCH 00/37] Permit filesystem local caching
Date: Fri, 22 Feb 2008 16:14:35 +0000 [thread overview]
Message-ID: <19020.1203696875@redhat.com> (raw)
In-Reply-To: <17916.1203636833@redhat.com>
David Howells <dhowells@redhat.com> wrote:
> > > Have you got before/after benchmark results?
> >
> > See attached.
>
> Attached here are results using BTRFS (patched so that it'll work at all)
> rather than Ext3 on the client on the partition backing the cache.
And here are XFS results.
Tuning XFS makes a *really* big difference for the lots of small/medium files
being tarred case. However, in general BTRFS is much better.
David
---
=========================
FEW BIG FILES TEST ON XFS
=========================
Completely cold caches:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m2.286s
user 0m0.000s
sys 0m1.828s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m4.228s
user 0m0.000s
sys 0m1.360s
Warm NFS pagecache:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m0.058s
user 0m0.000s
sys 0m0.060s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m0.122s
user 0m0.000s
sys 0m0.120s
Warm XFS pagecache, cold NFS pagecache:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m0.181s
user 0m0.000s
sys 0m0.180s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m1.034s
user 0m0.000s
sys 0m0.404s
Warm on-disk cache, cold pagecaches:
[root@andromeda ~]# time cat /warthog/bigfile >/dev/null
real 0m1.540s
user 0m0.000s
sys 0m0.256s
[root@andromeda ~]# time cat /warthog/biggerfile >/dev/null
real 0m3.003s
user 0m0.000s
sys 0m0.532s
==========================================
MANY SMALL/MEDIUM FILE READING TEST ON XFS
==========================================
Completely cold caches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 4m56.827s
user 0m0.180s
sys 0m6.668s
Warm NFS pagecache:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 0m15.084s
user 0m0.212s
sys 0m5.008s
Warm XFS pagecache, cold NFS pagecache:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 0m13.547s
user 0m0.220s
sys 0m5.652s
Warm on-disk cache, cold pagecaches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 4m36.316s
user 0m0.148s
sys 0m4.440s
=======================================================
MANY SMALL/MEDIUM FILE READING TEST ON AN OPTIMISED XFS
=======================================================
mkfs.xfs -d agcount=4 -l size=128m,version=2 /dev/sda6
Completely cold caches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 3m44.033s
user 0m0.248s
sys 0m6.632s
Warm on-disk cache, cold pagecaches:
[root@andromeda ~]# time tar cf - /warthog/aaa >/dev/zero
real 3m8.582s
user 0m0.108s
sys 0m3.420s
next prev parent reply other threads:[~2008-02-22 16:16 UTC|newest]
Thread overview: 168+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-02-20 16:05 [PATCH 00/37] Permit filesystem local caching David Howells
2008-02-20 16:05 ` David Howells
2008-02-20 16:05 ` David Howells
2008-02-20 16:06 ` [PATCH 01/37] KEYS: Increase the payload size when instantiating a key David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` [PATCH 02/37] KEYS: Check starting keyring as part of search David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` [PATCH 03/37] KEYS: Allow the callout data to be passed as a blob rather than a string David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` [PATCH 04/37] KEYS: Add keyctl function to get a security label David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` [PATCH 05/37] Security: Change current->fs[ug]id to current_fs[ug]id() David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` [PATCH 06/37] Security: Separate task security context from task_struct David Howells
2008-02-20 16:06 ` David Howells
2008-02-22 4:47 ` Casey Schaufler
2008-02-22 4:47 ` Casey Schaufler
2008-02-20 16:06 ` [PATCH 07/37] Security: De-embed task security record from task and use refcounting David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` David Howells
2008-02-22 4:57 ` Casey Schaufler
2008-02-22 4:57 ` Casey Schaufler
2008-02-22 4:57 ` Casey Schaufler
2008-02-20 16:06 ` [PATCH 08/37] Security: Add a kernel_service object class to SELinux David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` [PATCH 09/37] Security: Allow kernel services to override LSM settings for task actions David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` David Howells
2008-02-22 5:06 ` Casey Schaufler
2008-02-22 5:06 ` Casey Schaufler
2008-02-22 5:06 ` Casey Schaufler
2008-02-22 13:06 ` David Howells
2008-02-22 13:06 ` David Howells
2008-02-22 13:06 ` David Howells
2008-02-20 16:06 ` [PATCH 10/37] Security: Make NFSD work with detached security David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` [PATCH 11/37] FS-Cache: Release page->private after failed readahead David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:06 ` David Howells
2008-02-20 16:07 ` [PATCH 12/37] FS-Cache: Recruit a couple of page flags for cache management David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` [PATCH 13/37] FS-Cache: Provide an add_wait_queue_tail() function David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` [PATCH 14/37] FS-Cache: Generic filesystem caching facility David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` [PATCH 15/37] CacheFiles: Add missing copy_page export for ia64 David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` [PATCH 16/37] CacheFiles: Be consistent about the use of mapping vs file->f_mapping in Ext3 David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` [PATCH 17/37] CacheFiles: Add a hook to write a single page of data to an inode David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` [PATCH 18/37] CacheFiles: Permit the page lock state to be monitored David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` [PATCH 19/37] CacheFiles: Export things for CacheFiles David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` [PATCH 20/37] CacheFiles: A cache that backs onto a mounted filesystem David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` [PATCH 21/37] NFS: Add comment banners to some NFS functions David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:07 ` [PATCH 22/37] NFS: Add FS-Cache option bit and debug bit David Howells
2008-02-20 16:07 ` David Howells
2008-02-20 16:08 ` [PATCH 23/37] NFS: Permit local filesystem caching to be enabled for NFS David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:08 ` [PATCH 24/37] NFS: Register NFS for caching and retrieve the top-level index David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:08 ` [PATCH 25/37] NFS: Define and create server-level objects David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:08 ` [PATCH 26/37] NFS: Define and create superblock-level objects David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:08 ` [PATCH 27/37] NFS: Define and create inode-level cache objects David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:08 ` [PATCH 28/37] NFS: Use local disk inode cache David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:08 ` [PATCH 29/37] NFS: Invalidate FsCache page flags when cache removed David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:08 ` [PATCH 30/37] NFS: Add some new I/O event counters for FS-Cache events David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:08 ` [PATCH 31/37] NFS: FS-Cache page management David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:08 ` [PATCH 32/37] NFS: Add read context retention for FS-Cache to call back with David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:08 ` [PATCH 33/37] NFS: nfs_readpage_async() needs to be accessible as a fallback for local caching David Howells
2008-02-20 16:08 ` David Howells
2008-02-20 16:09 ` [PATCH 34/37] NFS: Read pages from FS-Cache into an NFS inode David Howells
2008-02-20 16:09 ` David Howells
2008-02-20 16:09 ` [PATCH 35/37] NFS: Store pages from an NFS inode into a local cache David Howells
2008-02-20 16:09 ` David Howells
2008-02-20 16:09 ` [PATCH 36/37] NFS: Display local caching state David Howells
2008-02-20 16:09 ` David Howells
2008-02-20 16:09 ` [PATCH 37/37] NFS: Add mount options to enable local caching on NFS David Howells
2008-02-20 16:09 ` David Howells
2008-02-20 19:58 ` [PATCH 00/37] Permit filesystem local caching Serge E. Hallyn
2008-02-20 19:58 ` Serge E. Hallyn
2008-02-20 20:11 ` David Howells
2008-02-20 20:11 ` David Howells
2008-02-20 20:11 ` David Howells
2008-02-20 20:18 ` James Carter
2008-02-21 3:07 ` Daniel Phillips
2008-02-21 12:31 ` David Howells
2008-02-21 12:31 ` David Howells
2008-02-21 12:31 ` David Howells
2008-02-21 14:55 ` David Howells
2008-02-21 14:55 ` David Howells
2008-02-21 14:55 ` David Howells
2008-02-21 15:17 ` Kevin Coffman
2008-02-21 22:44 ` Daniel Phillips
2008-02-21 22:44 ` Daniel Phillips
2008-02-21 22:52 ` Muntz, Daniel
2008-02-22 0:07 ` David Howells
2008-02-22 0:07 ` David Howells
2008-02-22 0:07 ` David Howells
2008-02-22 0:57 ` Daniel Phillips
2008-02-22 0:57 ` Daniel Phillips
2008-02-22 12:48 ` David Howells
2008-02-22 12:48 ` David Howells
2008-02-22 12:48 ` David Howells
2008-02-22 22:25 ` Daniel Phillips
2008-02-22 22:25 ` Daniel Phillips
2008-02-23 1:22 ` David Howells
2008-02-23 1:22 ` David Howells
2008-02-23 1:22 ` David Howells
2008-02-21 23:33 ` David Howells
2008-02-21 23:33 ` David Howells
2008-02-21 23:33 ` David Howells
2008-02-22 13:52 ` Chris Mason
2008-02-22 16:12 ` David Howells
2008-02-22 16:12 ` David Howells
2008-02-22 16:12 ` David Howells
2008-02-22 16:47 ` David Howells
2008-02-22 16:47 ` David Howells
2008-02-22 16:47 ` David Howells
2008-02-22 16:14 ` David Howells [this message]
2008-02-22 16:14 ` David Howells
2008-02-22 16:14 ` David Howells
[not found] ` <200802251401.16413.phillips@phunq.net>
2008-02-25 23:19 ` David Howells
2008-02-25 23:19 ` David Howells
2008-02-25 23:19 ` David Howells
2008-02-26 0:43 ` Daniel Phillips
2008-02-26 0:43 ` Daniel Phillips
2008-02-26 2:00 ` David Howells
2008-02-26 2:00 ` David Howells
2008-02-26 2:00 ` David Howells
2008-02-26 10:26 ` Daniel Phillips
2008-02-26 14:33 ` David Howells
2008-02-26 14:33 ` David Howells
2008-02-26 14:33 ` David Howells
2008-02-26 19:43 ` Daniel Phillips
2008-02-26 19:43 ` Daniel Phillips
2008-02-26 21:09 ` David Howells
2008-02-26 21:09 ` David Howells
2008-02-26 21:09 ` David Howells
-- strict thread matches above, loose matches on Subject: below --
2008-02-22 16:01 Rick Macklem
2008-02-22 16:01 ` Rick Macklem
2008-02-08 16:51 David Howells
2008-02-08 16:51 ` David Howells
2008-02-08 16:51 ` David Howells
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=19020.1203696875@redhat.com \
--to=dhowells@redhat.com \
--cc=Trond.Myklebust@netapp.com \
--cc=casey@schaufler-ca.com \
--cc=chuck.lever@oracle.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-security-module@vger.kernel.org \
--cc=nfsv4@linux-nfs.org \
--cc=phillips@phunq.net \
--cc=selinux@tycho.nsa.gov \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.