Re: large fs testing

[Ric Wheeler <rwheeler@redhat.com>]

I think that this kind of regression test should be fine, the key to avoiding 
the no benchmark issue is not to compare results on one array against a second....

ric


On 05/26/2009 08:21 AM, Joshua Giles wrote:
> Hi Ric,
>
> I'm wondering if we should include a "regression" performance test as part of the tools you'll give out for large fs testing?  Given some simple tools to run and some numbers output, we could request they (or as part of the test) measure the difference between fedora major releases or as part of the test cycle and send us that info?  Would we need some agreements such that they don't share this info with others or is this dirty laundry ok to air?
>
> -Josh Giles
> ----- Original Message -----
> From: "Ric Wheeler"<rwheeler@redhat.com>
> To: linux-fsdevel@vger.kernel.org
> Cc: "Christoph Hellwig"<hch@infradead.org>, "Douglas Shakshober"<dshaks@redhat.com>, "Joshua Giles"<jgiles@redhat.com>, "Valerie Aurora"<vaurora@redhat.com>, "Eric Sandeen"<esandeen@redhat.com>, "Steven Whitehouse"<swhiteho@redhat.com>, "Edward Shishkin"<edward@redhat.com>, "Josef Bacik"<jbacik@redhat.com>, "Jeff Moyer"<jmoyer@redhat.com>, "Chris Mason"<chris.mason@oracle.com>, "Eric Whitney"<eric.whitney@hp.com>, "Theodore Tso"<tytso@mit.edu>
> Sent: Saturday, May 23, 2009 9:53:28 AM GMT -05:00 US/Canada Eastern
> Subject: large fs testing
>
> Jeff Moyer&  I have been working with EMC elab over the last week or so testing
> ext4, xfs and gfs2 at roughly 80TB striped across a set of 12TB LUNs (single
> server, 6GB of DRAM, 2 quad core HT enabled CPU's).
>
> The goal of the testing is (in decreasing priority) is to validate Val's 64 bit
> patches for ext4 e2fsprogs, do a very quick sanity check that XFS does indeed
> scale as well as I hear (and it has so far :-)) and to test gfs2 tools at that
> high capacity. Not enough time to get it all done and significant fumbling on my
> part made it go even slower.
>
> Never the less, I have come to a rough idea of what a useful benchmark would be.
> If this sounds sane to all, I would like to try and put something together that
> we could provide to places like the EMC people who have large storage
> occasionally, are not kernel hackers, but would be willing to test for us. It
> will need to be fairly bullet proof and avoid doing performance numbers on the
> storage for normal things I assume (to avoid leaking competitive benchmarks out).
>
> Motivation - all things being equal, users benefit from having all storage
> consumed by one massive file system since that single file system manages space
> allocation, avoids seekiness, etc (something that applications have to do
> manually when using sets of file systems, the current state of the art for ext3
> for example).
>
> The challenges are:
>
> (1) object count - how many files can you pack into that file system with
> reasonable performance? (The test to date filled the single ext4 fs with 207
> million 20KB files)
>
> (2) files per directory - how many files per directory?
>
> (3) FS creation time - can you create a file system in reasonable time?
> (mkfs.xfs took seconds, mkfs.ext4 took 90 minutes). I think that 90 minutes is
> definitely on the painful side, but usable for most.
>
> (4) FS check time at a given fill rate for a healthy device (no IO errors).
> Testing at empty, 25%, 50%, 75% and 95% and full would all be interesting. Can
> you run these checks with a reasonable amount of DRAM - if not, what guidance do
> we need to give to customers on how big the servers need to be?
>
> It would seem to be a nice goal to be able to fsck a file system in one working
> day - say 8 hours - so that you could get a customer back on their feet, but
> maybe 24 hours would be an outside goal?
>
> (5) Write rate as the fs fills (picking the same set of fill rates?)
>
> To make is some how a tractable problem, I wanted to define small (20KB), medium
> (MP3 sized, say 4MB) and large (video sized, 4GB?) files to do the test with. I
> used fs_mark (no fsync's and 256 directories) to fill the file system (at least
> until my patience/time ran out!). With these options, it still hits very high
> file/directory counts (I am thinking about tweaking fs_mark to dynamically
> create a time based directory scheme, something like day/hour/min and giving it
> an option to stop at a specified fill rate).
>
> Sorry for the long ramble, I was curious to see if this makes sense to the
> broader set of you all&  if you have had any similar experiences to share.
>
> Thanks!
>
> Ric
>
>
>
>
>
>
>
>