* Re: ceph from poc to production
@ 2013-01-16 18:41 Stefan Priebe
2013-01-17 16:24 ` Sébastien Han
0 siblings, 1 reply; 6+ messages in thread
From: Stefan Priebe @ 2013-01-16 18:41 UTC (permalink / raw)
To: ceph-devel@vger.kernel.org
Hello Sebastien,
hello list,
first nice article sebastien ;-) thanks for contributing.
I've two question regarding the tips:
Can somebody explain me why i should do this?:
- Increase the osdmax value
- Set “filestore flusher” option to false, in your [osd] section
Greets,
Stefan
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ceph from poc to production
2013-01-16 18:41 ceph from poc to production Stefan Priebe
@ 2013-01-17 16:24 ` Sébastien Han
2013-01-17 16:29 ` Mark Nelson
2013-01-17 18:59 ` Stefan Priebe - Profihost AG
0 siblings, 2 replies; 6+ messages in thread
From: Sébastien Han @ 2013-01-17 16:24 UTC (permalink / raw)
To: Stefan Priebe; +Cc: ceph-devel@vger.kernel.org
Hi Stephan,
> - Increase the osdmax value
Well actually this doesn't seem to be necessary because when you issue
a "ceph osd create" the osdmax value is automatically incremented. It
just that a while ago I saw on the ML or IRC that it was a good
practice to do this. And this documentation
http://ceph.com/docs/master/rados/operations/control/ says:
"Set the max_osd parameter in the osd map. This is necessary when
expanding the storage cluster."
Maybe the doc is outdated on this option, I don't know.
> - Set “filestore flusher” option to false, in your [osd] section
During my benchmarks sessions I noticed that disabling this option
brought way better performance, at least for old hardware. Fo recent
hardware I haven't seen any difference, but I kept the habit to
disable it. There are some discussions about this option on the ML. As
far as I can remember, it's harmless to disable it.
From the doc here
http://ceph.com/docs/master/rados/configuration/filestore-config-ref/:
"The filestore flusher forces data from large writes to be written out
using sync file range before the sync in order to (hopefully) reduce
the cost of the eventual sync. In practice, disabling ‘filestore
flusher’ seems to improve performance in some cases."
Cheers,
--
Regards,
Sébastien Han.
On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe <s.priebe@profihost.ag> wrote:
>
> Hello Sebastien,
> hello list,
>
> first nice article sebastien ;-) thanks for contributing.
>
> I've two question regarding the tips:
>
> Can somebody explain me why i should do this?:
> - Increase the osdmax value
> - Set “filestore flusher” option to false, in your [osd] section
>
> Greets,
> Stefan
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ceph from poc to production
2013-01-17 16:24 ` Sébastien Han
@ 2013-01-17 16:29 ` Mark Nelson
2013-01-17 16:34 ` Sébastien Han
2013-01-17 18:58 ` Stefan Priebe - Profihost AG
2013-01-17 18:59 ` Stefan Priebe - Profihost AG
1 sibling, 2 replies; 6+ messages in thread
From: Mark Nelson @ 2013-01-17 16:29 UTC (permalink / raw)
To: Sébastien Han; +Cc: Stefan Priebe, ceph-devel@vger.kernel.org
On 01/17/2013 10:24 AM, Sébastien Han wrote:
> Hi Stephan,
>
>> - Increase the osdmax value
>
> Well actually this doesn't seem to be necessary because when you issue
> a "ceph osd create" the osdmax value is automatically incremented. It
> just that a while ago I saw on the ML or IRC that it was a good
> practice to do this. And this documentation
> http://ceph.com/docs/master/rados/operations/control/ says:
>
> "Set the max_osd parameter in the osd map. This is necessary when
> expanding the storage cluster."
>
> Maybe the doc is outdated on this option, I don't know.
>
>> - Set “filestore flusher” option to false, in your [osd] section
>
> During my benchmarks sessions I noticed that disabling this option
> brought way better performance, at least for old hardware. Fo recent
> hardware I haven't seen any difference, but I kept the habit to
> disable it. There are some discussions about this option on the ML. As
> far as I can remember, it's harmless to disable it.
> From the doc here
> http://ceph.com/docs/master/rados/configuration/filestore-config-ref/:
I actually have been doing parametric sweeps over the various ceph
tunables to try to see what affects performance. I'll have an article
coming out soonish documenting some of the findings. Typically
disabling the flusher seems to be beneficial on XFS and BTRFS, but it
seems to be the opposite on EXT4.
>
> "The filestore flusher forces data from large writes to be written out
> using sync file range before the sync in order to (hopefully) reduce
> the cost of the eventual sync. In practice, disabling ‘filestore
> flusher’ seems to improve performance in some cases."
>
> Cheers,
>
> --
> Regards,
> Sébastien Han.
>
>
> On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe <s.priebe@profihost.ag> wrote:
>>
>> Hello Sebastien,
>> hello list,
>>
>> first nice article sebastien ;-) thanks for contributing.
>>
>> I've two question regarding the tips:
>>
>> Can somebody explain me why i should do this?:
>> - Increase the osdmax value
>> - Set “filestore flusher” option to false, in your [osd] section
>>
>> Greets,
>> Stefan
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ceph from poc to production
2013-01-17 16:29 ` Mark Nelson
@ 2013-01-17 16:34 ` Sébastien Han
2013-01-17 18:58 ` Stefan Priebe - Profihost AG
1 sibling, 0 replies; 6+ messages in thread
From: Sébastien Han @ 2013-01-17 16:34 UTC (permalink / raw)
To: Mark Nelson; +Cc: Stefan Priebe, ceph-devel@vger.kernel.org
Cool, I look forward to reading it!
--
Regards,
Sébastien Han.
On Thu, Jan 17, 2013 at 5:29 PM, Mark Nelson <mark.nelson@inktank.com> wrote:
> On 01/17/2013 10:24 AM, Sébastien Han wrote:
>>
>> Hi Stephan,
>>
>>> - Increase the osdmax value
>>
>>
>> Well actually this doesn't seem to be necessary because when you issue
>> a "ceph osd create" the osdmax value is automatically incremented. It
>> just that a while ago I saw on the ML or IRC that it was a good
>> practice to do this. And this documentation
>> http://ceph.com/docs/master/rados/operations/control/ says:
>>
>> "Set the max_osd parameter in the osd map. This is necessary when
>> expanding the storage cluster."
>>
>> Maybe the doc is outdated on this option, I don't know.
>>
>>> - Set “filestore flusher” option to false, in your [osd] section
>>
>>
>> During my benchmarks sessions I noticed that disabling this option
>> brought way better performance, at least for old hardware. Fo recent
>> hardware I haven't seen any difference, but I kept the habit to
>> disable it. There are some discussions about this option on the ML. As
>> far as I can remember, it's harmless to disable it.
>> From the doc here
>> http://ceph.com/docs/master/rados/configuration/filestore-config-ref/:
>
>
> I actually have been doing parametric sweeps over the various ceph tunables
> to try to see what affects performance. I'll have an article coming out
> soonish documenting some of the findings. Typically disabling the flusher
> seems to be beneficial on XFS and BTRFS, but it seems to be the opposite on
> EXT4.
>
>
>>
>> "The filestore flusher forces data from large writes to be written out
>> using sync file range before the sync in order to (hopefully) reduce
>> the cost of the eventual sync. In practice, disabling ‘filestore
>> flusher’ seems to improve performance in some cases."
>>
>> Cheers,
>>
>> --
>> Regards,
>> Sébastien Han.
>>
>>
>> On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe <s.priebe@profihost.ag>
>> wrote:
>>>
>>>
>>> Hello Sebastien,
>>> hello list,
>>>
>>> first nice article sebastien ;-) thanks for contributing.
>>>
>>> I've two question regarding the tips:
>>>
>>> Can somebody explain me why i should do this?:
>>> - Increase the osdmax value
>>> - Set “filestore flusher” option to false, in your [osd] section
>>>
>>> Greets,
>>> Stefan
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ceph from poc to production
2013-01-17 16:29 ` Mark Nelson
2013-01-17 16:34 ` Sébastien Han
@ 2013-01-17 18:58 ` Stefan Priebe - Profihost AG
1 sibling, 0 replies; 6+ messages in thread
From: Stefan Priebe - Profihost AG @ 2013-01-17 18:58 UTC (permalink / raw)
To: Mark Nelson; +Cc: Sébastien Han, ceph-devel@vger.kernel.org
Great! Any ETA?
Am 17.01.2013 um 17:29 schrieb Mark Nelson <mark.nelson@inktank.com>:
> On 01/17/2013 10:24 AM, Sébastien Han wrote:
>> Hi Stephan,
>>
>>> - Increase the osdmax value
>>
>> Well actually this doesn't seem to be necessary because when you issue
>> a "ceph osd create" the osdmax value is automatically incremented. It
>> just that a while ago I saw on the ML or IRC that it was a good
>> practice to do this. And this documentation
>> http://ceph.com/docs/master/rados/operations/control/ says:
>>
>> "Set the max_osd parameter in the osd map. This is necessary when
>> expanding the storage cluster."
>>
>> Maybe the doc is outdated on this option, I don't know.
>>
>>> - Set “filestore flusher” option to false, in your [osd] section
>>
>> During my benchmarks sessions I noticed that disabling this option
>> brought way better performance, at least for old hardware. Fo recent
>> hardware I haven't seen any difference, but I kept the habit to
>> disable it. There are some discussions about this option on the ML. As
>> far as I can remember, it's harmless to disable it.
>> From the doc here
>> http://ceph.com/docs/master/rados/configuration/filestore-config-ref/:
>
> I actually have been doing parametric sweeps over the various ceph tunables to try to see what affects performance. I'll have an article coming out soonish documenting some of the findings. Typically disabling the flusher seems to be beneficial on XFS and BTRFS, but it seems to be the opposite on EXT4.
>
>>
>> "The filestore flusher forces data from large writes to be written out
>> using sync file range before the sync in order to (hopefully) reduce
>> the cost of the eventual sync. In practice, disabling ‘filestore
>> flusher’ seems to improve performance in some cases."
>>
>> Cheers,
>>
>> --
>> Regards,
>> Sébastien Han.
>>
>>
>> On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe <s.priebe@profihost.ag> wrote:
>>>
>>> Hello Sebastien,
>>> hello list,
>>>
>>> first nice article sebastien ;-) thanks for contributing.
>>>
>>> I've two question regarding the tips:
>>>
>>> Can somebody explain me why i should do this?:
>>> - Increase the osdmax value
>>> - Set “filestore flusher” option to false, in your [osd] section
>>>
>>> Greets,
>>> Stefan
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ceph from poc to production
2013-01-17 16:24 ` Sébastien Han
2013-01-17 16:29 ` Mark Nelson
@ 2013-01-17 18:59 ` Stefan Priebe - Profihost AG
1 sibling, 0 replies; 6+ messages in thread
From: Stefan Priebe - Profihost AG @ 2013-01-17 18:59 UTC (permalink / raw)
To: Sébastien Han; +Cc: ceph-devel@vger.kernel.org
Thx!
Am 17.01.2013 um 17:24 schrieb Sébastien Han <han.sebastien@gmail.com>:
> Hi Stephan,
>
>> - Increase the osdmax value
>
> Well actually this doesn't seem to be necessary because when you issue
> a "ceph osd create" the osdmax value is automatically incremented. It
> just that a while ago I saw on the ML or IRC that it was a good
> practice to do this. And this documentation
> http://ceph.com/docs/master/rados/operations/control/ says:
>
> "Set the max_osd parameter in the osd map. This is necessary when
> expanding the storage cluster."
>
> Maybe the doc is outdated on this option, I don't know.
>
>> - Set “filestore flusher” option to false, in your [osd] section
>
> During my benchmarks sessions I noticed that disabling this option
> brought way better performance, at least for old hardware. Fo recent
> hardware I haven't seen any difference, but I kept the habit to
> disable it. There are some discussions about this option on the ML. As
> far as I can remember, it's harmless to disable it.
> From the doc here
> http://ceph.com/docs/master/rados/configuration/filestore-config-ref/:
>
> "The filestore flusher forces data from large writes to be written out
> using sync file range before the sync in order to (hopefully) reduce
> the cost of the eventual sync. In practice, disabling ‘filestore
> flusher’ seems to improve performance in some cases."
>
> Cheers,
>
> --
> Regards,
> Sébastien Han.
>
>
> On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe <s.priebe@profihost.ag> wrote:
>>
>> Hello Sebastien,
>> hello list,
>>
>> first nice article sebastien ;-) thanks for contributing.
>>
>> I've two question regarding the tips:
>>
>> Can somebody explain me why i should do this?:
>> - Increase the osdmax value
>> - Set “filestore flusher” option to false, in your [osd] section
>>
>> Greets,
>> Stefan
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-01-17 18:59 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-01-16 18:41 ceph from poc to production Stefan Priebe
2013-01-17 16:24 ` Sébastien Han
2013-01-17 16:29 ` Mark Nelson
2013-01-17 16:34 ` Sébastien Han
2013-01-17 18:58 ` Stefan Priebe - Profihost AG
2013-01-17 18:59 ` Stefan Priebe - Profihost AG
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.