* Re: ceph from poc to production @ 2013-01-16 18:41 Stefan Priebe 2013-01-17 16:24 ` Sébastien Han 0 siblings, 1 reply; 6+ messages in thread From: Stefan Priebe @ 2013-01-16 18:41 UTC (permalink / raw) To: ceph-devel@vger.kernel.org Hello Sebastien, hello list, first nice article sebastien ;-) thanks for contributing. I've two question regarding the tips: Can somebody explain me why i should do this?: - Increase the osdmax value - Set “filestore flusher” option to false, in your [osd] section Greets, Stefan -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ceph from poc to production 2013-01-16 18:41 ceph from poc to production Stefan Priebe @ 2013-01-17 16:24 ` Sébastien Han 2013-01-17 16:29 ` Mark Nelson 2013-01-17 18:59 ` Stefan Priebe - Profihost AG 0 siblings, 2 replies; 6+ messages in thread From: Sébastien Han @ 2013-01-17 16:24 UTC (permalink / raw) To: Stefan Priebe; +Cc: ceph-devel@vger.kernel.org Hi Stephan, > - Increase the osdmax value Well actually this doesn't seem to be necessary because when you issue a "ceph osd create" the osdmax value is automatically incremented. It just that a while ago I saw on the ML or IRC that it was a good practice to do this. And this documentation http://ceph.com/docs/master/rados/operations/control/ says: "Set the max_osd parameter in the osd map. This is necessary when expanding the storage cluster." Maybe the doc is outdated on this option, I don't know. > - Set “filestore flusher” option to false, in your [osd] section During my benchmarks sessions I noticed that disabling this option brought way better performance, at least for old hardware. Fo recent hardware I haven't seen any difference, but I kept the habit to disable it. There are some discussions about this option on the ML. As far as I can remember, it's harmless to disable it. From the doc here http://ceph.com/docs/master/rados/configuration/filestore-config-ref/: "The filestore flusher forces data from large writes to be written out using sync file range before the sync in order to (hopefully) reduce the cost of the eventual sync. In practice, disabling ‘filestore flusher’ seems to improve performance in some cases." Cheers, -- Regards, Sébastien Han. On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe <s.priebe@profihost.ag> wrote: > > Hello Sebastien, > hello list, > > first nice article sebastien ;-) thanks for contributing. > > I've two question regarding the tips: > > Can somebody explain me why i should do this?: > - Increase the osdmax value > - Set “filestore flusher” option to false, in your [osd] section > > Greets, > Stefan > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ceph from poc to production 2013-01-17 16:24 ` Sébastien Han @ 2013-01-17 16:29 ` Mark Nelson 2013-01-17 16:34 ` Sébastien Han 2013-01-17 18:58 ` Stefan Priebe - Profihost AG 2013-01-17 18:59 ` Stefan Priebe - Profihost AG 1 sibling, 2 replies; 6+ messages in thread From: Mark Nelson @ 2013-01-17 16:29 UTC (permalink / raw) To: Sébastien Han; +Cc: Stefan Priebe, ceph-devel@vger.kernel.org On 01/17/2013 10:24 AM, Sébastien Han wrote: > Hi Stephan, > >> - Increase the osdmax value > > Well actually this doesn't seem to be necessary because when you issue > a "ceph osd create" the osdmax value is automatically incremented. It > just that a while ago I saw on the ML or IRC that it was a good > practice to do this. And this documentation > http://ceph.com/docs/master/rados/operations/control/ says: > > "Set the max_osd parameter in the osd map. This is necessary when > expanding the storage cluster." > > Maybe the doc is outdated on this option, I don't know. > >> - Set “filestore flusher” option to false, in your [osd] section > > During my benchmarks sessions I noticed that disabling this option > brought way better performance, at least for old hardware. Fo recent > hardware I haven't seen any difference, but I kept the habit to > disable it. There are some discussions about this option on the ML. As > far as I can remember, it's harmless to disable it. > From the doc here > http://ceph.com/docs/master/rados/configuration/filestore-config-ref/: I actually have been doing parametric sweeps over the various ceph tunables to try to see what affects performance. I'll have an article coming out soonish documenting some of the findings. Typically disabling the flusher seems to be beneficial on XFS and BTRFS, but it seems to be the opposite on EXT4. > > "The filestore flusher forces data from large writes to be written out > using sync file range before the sync in order to (hopefully) reduce > the cost of the eventual sync. In practice, disabling ‘filestore > flusher’ seems to improve performance in some cases." > > Cheers, > > -- > Regards, > Sébastien Han. > > > On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe <s.priebe@profihost.ag> wrote: >> >> Hello Sebastien, >> hello list, >> >> first nice article sebastien ;-) thanks for contributing. >> >> I've two question regarding the tips: >> >> Can somebody explain me why i should do this?: >> - Increase the osdmax value >> - Set “filestore flusher” option to false, in your [osd] section >> >> Greets, >> Stefan >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ceph from poc to production 2013-01-17 16:29 ` Mark Nelson @ 2013-01-17 16:34 ` Sébastien Han 2013-01-17 18:58 ` Stefan Priebe - Profihost AG 1 sibling, 0 replies; 6+ messages in thread From: Sébastien Han @ 2013-01-17 16:34 UTC (permalink / raw) To: Mark Nelson; +Cc: Stefan Priebe, ceph-devel@vger.kernel.org Cool, I look forward to reading it! -- Regards, Sébastien Han. On Thu, Jan 17, 2013 at 5:29 PM, Mark Nelson <mark.nelson@inktank.com> wrote: > On 01/17/2013 10:24 AM, Sébastien Han wrote: >> >> Hi Stephan, >> >>> - Increase the osdmax value >> >> >> Well actually this doesn't seem to be necessary because when you issue >> a "ceph osd create" the osdmax value is automatically incremented. It >> just that a while ago I saw on the ML or IRC that it was a good >> practice to do this. And this documentation >> http://ceph.com/docs/master/rados/operations/control/ says: >> >> "Set the max_osd parameter in the osd map. This is necessary when >> expanding the storage cluster." >> >> Maybe the doc is outdated on this option, I don't know. >> >>> - Set “filestore flusher” option to false, in your [osd] section >> >> >> During my benchmarks sessions I noticed that disabling this option >> brought way better performance, at least for old hardware. Fo recent >> hardware I haven't seen any difference, but I kept the habit to >> disable it. There are some discussions about this option on the ML. As >> far as I can remember, it's harmless to disable it. >> From the doc here >> http://ceph.com/docs/master/rados/configuration/filestore-config-ref/: > > > I actually have been doing parametric sweeps over the various ceph tunables > to try to see what affects performance. I'll have an article coming out > soonish documenting some of the findings. Typically disabling the flusher > seems to be beneficial on XFS and BTRFS, but it seems to be the opposite on > EXT4. > > >> >> "The filestore flusher forces data from large writes to be written out >> using sync file range before the sync in order to (hopefully) reduce >> the cost of the eventual sync. In practice, disabling ‘filestore >> flusher’ seems to improve performance in some cases." >> >> Cheers, >> >> -- >> Regards, >> Sébastien Han. >> >> >> On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe <s.priebe@profihost.ag> >> wrote: >>> >>> >>> Hello Sebastien, >>> hello list, >>> >>> first nice article sebastien ;-) thanks for contributing. >>> >>> I've two question regarding the tips: >>> >>> Can somebody explain me why i should do this?: >>> - Increase the osdmax value >>> - Set “filestore flusher” option to false, in your [osd] section >>> >>> Greets, >>> Stefan >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ceph from poc to production 2013-01-17 16:29 ` Mark Nelson 2013-01-17 16:34 ` Sébastien Han @ 2013-01-17 18:58 ` Stefan Priebe - Profihost AG 1 sibling, 0 replies; 6+ messages in thread From: Stefan Priebe - Profihost AG @ 2013-01-17 18:58 UTC (permalink / raw) To: Mark Nelson; +Cc: Sébastien Han, ceph-devel@vger.kernel.org Great! Any ETA? Am 17.01.2013 um 17:29 schrieb Mark Nelson <mark.nelson@inktank.com>: > On 01/17/2013 10:24 AM, Sébastien Han wrote: >> Hi Stephan, >> >>> - Increase the osdmax value >> >> Well actually this doesn't seem to be necessary because when you issue >> a "ceph osd create" the osdmax value is automatically incremented. It >> just that a while ago I saw on the ML or IRC that it was a good >> practice to do this. And this documentation >> http://ceph.com/docs/master/rados/operations/control/ says: >> >> "Set the max_osd parameter in the osd map. This is necessary when >> expanding the storage cluster." >> >> Maybe the doc is outdated on this option, I don't know. >> >>> - Set “filestore flusher” option to false, in your [osd] section >> >> During my benchmarks sessions I noticed that disabling this option >> brought way better performance, at least for old hardware. Fo recent >> hardware I haven't seen any difference, but I kept the habit to >> disable it. There are some discussions about this option on the ML. As >> far as I can remember, it's harmless to disable it. >> From the doc here >> http://ceph.com/docs/master/rados/configuration/filestore-config-ref/: > > I actually have been doing parametric sweeps over the various ceph tunables to try to see what affects performance. I'll have an article coming out soonish documenting some of the findings. Typically disabling the flusher seems to be beneficial on XFS and BTRFS, but it seems to be the opposite on EXT4. > >> >> "The filestore flusher forces data from large writes to be written out >> using sync file range before the sync in order to (hopefully) reduce >> the cost of the eventual sync. In practice, disabling ‘filestore >> flusher’ seems to improve performance in some cases." >> >> Cheers, >> >> -- >> Regards, >> Sébastien Han. >> >> >> On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe <s.priebe@profihost.ag> wrote: >>> >>> Hello Sebastien, >>> hello list, >>> >>> first nice article sebastien ;-) thanks for contributing. >>> >>> I've two question regarding the tips: >>> >>> Can somebody explain me why i should do this?: >>> - Increase the osdmax value >>> - Set “filestore flusher” option to false, in your [osd] section >>> >>> Greets, >>> Stefan >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ceph from poc to production 2013-01-17 16:24 ` Sébastien Han 2013-01-17 16:29 ` Mark Nelson @ 2013-01-17 18:59 ` Stefan Priebe - Profihost AG 1 sibling, 0 replies; 6+ messages in thread From: Stefan Priebe - Profihost AG @ 2013-01-17 18:59 UTC (permalink / raw) To: Sébastien Han; +Cc: ceph-devel@vger.kernel.org Thx! Am 17.01.2013 um 17:24 schrieb Sébastien Han <han.sebastien@gmail.com>: > Hi Stephan, > >> - Increase the osdmax value > > Well actually this doesn't seem to be necessary because when you issue > a "ceph osd create" the osdmax value is automatically incremented. It > just that a while ago I saw on the ML or IRC that it was a good > practice to do this. And this documentation > http://ceph.com/docs/master/rados/operations/control/ says: > > "Set the max_osd parameter in the osd map. This is necessary when > expanding the storage cluster." > > Maybe the doc is outdated on this option, I don't know. > >> - Set “filestore flusher” option to false, in your [osd] section > > During my benchmarks sessions I noticed that disabling this option > brought way better performance, at least for old hardware. Fo recent > hardware I haven't seen any difference, but I kept the habit to > disable it. There are some discussions about this option on the ML. As > far as I can remember, it's harmless to disable it. > From the doc here > http://ceph.com/docs/master/rados/configuration/filestore-config-ref/: > > "The filestore flusher forces data from large writes to be written out > using sync file range before the sync in order to (hopefully) reduce > the cost of the eventual sync. In practice, disabling ‘filestore > flusher’ seems to improve performance in some cases." > > Cheers, > > -- > Regards, > Sébastien Han. > > > On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe <s.priebe@profihost.ag> wrote: >> >> Hello Sebastien, >> hello list, >> >> first nice article sebastien ;-) thanks for contributing. >> >> I've two question regarding the tips: >> >> Can somebody explain me why i should do this?: >> - Increase the osdmax value >> - Set “filestore flusher” option to false, in your [osd] section >> >> Greets, >> Stefan >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-01-17 18:59 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-01-16 18:41 ceph from poc to production Stefan Priebe 2013-01-17 16:24 ` Sébastien Han 2013-01-17 16:29 ` Mark Nelson 2013-01-17 16:34 ` Sébastien Han 2013-01-17 18:58 ` Stefan Priebe - Profihost AG 2013-01-17 18:59 ` Stefan Priebe - Profihost AG
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.