os recommendations

os recommendations

[Dominik Mostowiec]

Hi,
I found in doc: http://ceph.com/docs/master/start/os-recommendations/
"Putting multiple ceph-osd daemons using XFS or ext4 on the same host
will not perform as well as they could."

For now recommended filesystem is XFS.
This means that for the best performance setup should be 1 OSD per host?

-- 
Regards
Dominik

Re: os recommendations

[Christoph Hellwig]

On Tue, Nov 26, 2013 at 11:43:07AM +0100, Dominik Mostowiec wrote:
> Hi,
> I found in doc: http://ceph.com/docs/master/start/os-recommendations/
> "Putting multiple ceph-osd daemons using XFS or ext4 on the same host
> will not perform as well as they could."
> 
> For now recommended filesystem is XFS.
> This means that for the best performance setup should be 1 OSD per host?

Btw, could anyone clarify where that stance comes from, that is numbers
to back it up.

Re: os recommendations

[Sage Weil]

On Tue, 26 Nov 2013, Christoph Hellwig wrote:
> On Tue, Nov 26, 2013 at 11:43:07AM +0100, Dominik Mostowiec wrote:
> > Hi,
> > I found in doc: http://ceph.com/docs/master/start/os-recommendations/
> > "Putting multiple ceph-osd daemons using XFS or ext4 on the same host
> > will not perform as well as they could."
> > 
> > For now recommended filesystem is XFS.
> > This means that for the best performance setup should be 1 OSD per host?
> 
> Btw, could anyone clarify where that stance comes from, that is numbers
> to back it up.

If syncfs(2) is not present, we have to use sync(2).  That means you have 
N daemons calling sync(2) to force a commit on a single fs, but all other 
mounted fs's are also synced... which means N times the sync(2) calls.

Fortunately syncfs(2) has been around for a while now, so this only 
affects really old distros.  And even when glibc does not have a syscall 
wrapper for it, we try to call the syscall directly.

sage

Re: os recommendations

[Dominik Mostowiec]

Thanks.

--
Regards
Dominik

2013/11/26 Sage Weil <sage@inktank.com>:
> On Tue, 26 Nov 2013, Christoph Hellwig wrote:
>> On Tue, Nov 26, 2013 at 11:43:07AM +0100, Dominik Mostowiec wrote:
>> > Hi,
>> > I found in doc: http://ceph.com/docs/master/start/os-recommendations/
>> > "Putting multiple ceph-osd daemons using XFS or ext4 on the same host
>> > will not perform as well as they could."
>> >
>> > For now recommended filesystem is XFS.
>> > This means that for the best performance setup should be 1 OSD per host?
>>
>> Btw, could anyone clarify where that stance comes from, that is numbers
>> to back it up.
>
> If syncfs(2) is not present, we have to use sync(2).  That means you have
> N daemons calling sync(2) to force a commit on a single fs, but all other
> mounted fs's are also synced... which means N times the sync(2) calls.
>
> Fortunately syncfs(2) has been around for a while now, so this only
> affects really old distros.  And even when glibc does not have a syscall
> wrapper for it, we try to call the syscall directly.
>
> sage



-- 
Pozdrawiam
Dominik

Re: os recommendations

[Christoph Hellwig]

On Tue, Nov 26, 2013 at 06:50:33AM -0800, Sage Weil wrote:
> If syncfs(2) is not present, we have to use sync(2).  That means you have 
> N daemons calling sync(2) to force a commit on a single fs, but all other 
> mounted fs's are also synced... which means N times the sync(2) calls.
> 
> Fortunately syncfs(2) has been around for a while now, so this only 
> affects really old distros.  And even when glibc does not have a syscall 
> wrapper for it, we try to call the syscall directly.

And for btrfs you were/are using magic ioctls, right.

Looks like the page reference in the last post has already been updated,
thanks!