All of lore.kernel.org
 help / color / mirror / Atom feed
From: Peter Niemayer <niemayer@isg.de>
To: linux-xfs@oss.sgi.com
Subject: Re: observed significant performance improvement using "delaylog" in a real-world application
Date: Wed, 11 Aug 2010 11:45:41 +0200	[thread overview]
Message-ID: <i3trg6$eps$1@dough.gmane.org> (raw)
In-Reply-To: <201008111003.36890@zmi.at>

On 08/11/2010 10:03 AM, Michael Monnerie wrote:
> On Dienstag, 10. August 2010 Peter Niemayer wrote:
>> 17% performance increase for our benchmark scenario
>
> Can you say what kind of data accesses you have?

The application basically writes a large data set to ~ 100,000 files,
which consists of many small messages. Each message consists of a 
primary key (an integer in the range of 0 to approx. 1,000,000) and
an (almost random) number of data bytes (length in the range from 10 to
~1000 byte).

For each message, the application opens the file that is determined by
the primary key with O_APPEND, write()s the data bytes to the file.
Then it closes the file. (There are usually a few to many messages per
primary key / file).
The application runs 4 threads in parallel to spread the above action
over 4 CPU cores, each thread processes a quarter of the primary keys
(primary_key & 0x03).

This description is, of course, somewhat simplified, but it should
get you an idea on the kind of I/O-operations.

Regards,

Peter Niemayer


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

  reply	other threads:[~2010-08-11  9:45 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-08-10 16:01 observed significant performance improvement using "delaylog" in a real-world application Peter Niemayer
2010-08-11  0:08 ` Dave Chinner
2010-08-11 18:08   ` Peter Niemayer
2010-08-11  8:03 ` Michael Monnerie
2010-08-11  9:45   ` Peter Niemayer [this message]
2010-08-11 12:28     ` Michael Monnerie
2010-08-11 17:01       ` Peter Niemayer
2010-08-11 22:31         ` Michael Monnerie
2010-08-11 23:44         ` Dave Chinner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='i3trg6$eps$1@dough.gmane.org' \
    --to=niemayer@isg.de \
    --cc=linux-xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.