From: Mark Nelson <mark.nelson@inktank.com>
To: Andreas Joachim Peters <Andreas.Joachim.Peters@cern.ch>
Cc: Dan van der Ster <dan@vanderster.com>,
Sage Weil <sage@inktank.com>,
"ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: Object Write Latency
Date: Mon, 23 Sep 2013 11:03:58 -0500 [thread overview]
Message-ID: <5240666E.7010009@inktank.com> (raw)
In-Reply-To: <3472A07E6605974CBC9BC573F1BC02E4A52736BE@PLOXCHG03.cern.ch>
On 09/23/2013 10:38 AM, Andreas Joachim Peters wrote:
> We deployed 3 OSDs with an EXT4 using RapidDisk in-memory.
>
> The FS does 140k/s append+sync and the latency is now:
>
> ~1 ms for few byte objects with single replica
> ~2 ms for few byte objects three replica (instead of 65-80ms)
Interesting! If you look at the slowest operations in the ceph admin
socket now with dump_historic_ops, where are those operations spending
their time?
>
> This gives probably the base-line of the best you can do with the current implementation.
>
> ==> the 80ms are probably just a 'feature' of the hardware (JBOD disks/controller) and we might try to find some tuning parameters to improve the latency slightly.
Hardware definitely plays a huge part in terms of Ceph performance. You
can run Ceph on just about anything, but it's surprising how different
two roughly similar systems can perform.
>
> Could you just explain how the async api functions (is_complete, is_safe) map to the three states
>
> 1) object is transferred from client to all OSDs and is present in memory there
> 2) object is written to the OSD journal
> 3) object is committed from OSD journal to the OSD filesystem
>
> Is it correct that the object is visible by clients only when 3) has happened?
Yes, afaik.
>
> Thanks for your help,
> Andreas.
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
next prev parent reply other threads:[~2013-09-23 16:03 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-09-20 12:27 Object Write Latency Andreas Joachim Peters
2013-09-20 13:11 ` Mark Nelson
2013-09-23 7:34 ` Dan van der Ster
2013-09-20 14:47 ` Gregory Farnum
2013-09-23 7:37 ` Dan van der Ster
2013-09-23 18:18 ` Sage Weil
2013-09-24 9:13 ` Dan van der Ster
2013-09-24 10:04 ` Dan van der Ster
[not found] ` <etPan.52417ad3.3804823e.f19a@leseb-mba.local>
2013-09-24 12:30 ` Sébastien Han
2013-09-24 12:33 ` Dan van der Ster
2013-09-24 12:39 ` Sébastien Han
2013-09-20 15:34 ` Sage Weil
2013-09-23 7:39 ` Dan van der Ster
2013-09-23 15:38 ` Andreas Joachim Peters
2013-09-23 16:03 ` Mark Nelson [this message]
2013-09-23 21:40 ` Andreas Joachim Peters
2013-09-24 8:55 ` Dan van der Ster
2013-09-24 15:20 ` Sage Weil
2013-09-23 16:19 ` Sage Weil
2013-09-23 15:40 ` Sage Weil
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5240666E.7010009@inktank.com \
--to=mark.nelson@inktank.com \
--cc=Andreas.Joachim.Peters@cern.ch \
--cc=ceph-devel@vger.kernel.org \
--cc=dan@vanderster.com \
--cc=sage@inktank.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.