mon disk access pattern

mon disk access pattern

[Stefan Priebe - Profihost AG]

Hello list,

i read somewhere that the mon has "special" disk access patterns - even 
though it does not write much data.

I'm using a patched debian squeeze with syncfs support (thanks to amon). 
Are there any recommandations where to put the mon data? (SSD? Raid? 
Which FS?)

Greets,
Stefan

Re: mon disk access pattern

[Wido den Hollander]

Hi,

On 15-06-12 07:52, Stefan Priebe - Profihost AG wrote:
> Hello list,
>
> i read somewhere that the mon has "special" disk access patterns - even
> though it does not write much data.

"special"? Where did you read that? The monitor has about 1 ~ 2GB of 
storage.

So if your monitor has something like 4GB ~ 8GB of RAM, your kernel 
should cache almost all your monitor data.

>
> I'm using a patched debian squeeze with syncfs support (thanks to amon).
> Are there any recommandations where to put the mon data? (SSD? Raid?
> Which FS?)
>

Something like a regular disk with ext4 should be fine, but you can 
always put in an SSD if you want that.

Wido

> Greets,
> Stefan
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: mon disk access pattern

[Mark Nelson]

On 06/15/2012 04:51 AM, Wido den Hollander wrote:
> Hi,
>
> On 15-06-12 07:52, Stefan Priebe - Profihost AG wrote:
>> Hello list,
>>
>> i read somewhere that the mon has "special" disk access patterns - even
>> though it does not write much data.
>
> "special"? Where did you read that? The monitor has about 1 ~ 2GB of
> storage.
>
> So if your monitor has something like 4GB ~ 8GB of RAM, your kernel
> should cache almost all your monitor data.
>

I think at some point someone mentioned to me that the mon can cause a 
lot of syncs, so running them on the OSDs without syncfs might be 
detrimental.  For the majority of our internal performance testing I've 
kept them off the OSDs just to be sure.

Re: mon disk access pattern

[Sage Weil]

On Fri, 15 Jun 2012, Mark Nelson wrote:
> On 06/15/2012 04:51 AM, Wido den Hollander wrote:
> > Hi,
> > 
> > On 15-06-12 07:52, Stefan Priebe - Profihost AG wrote:
> > > Hello list,
> > > 
> > > i read somewhere that the mon has "special" disk access patterns - even
> > > though it does not write much data.
> > 
> > "special"? Where did you read that? The monitor has about 1 ~ 2GB of
> > storage.
> > 
> > So if your monitor has something like 4GB ~ 8GB of RAM, your kernel
> > should cache almost all your monitor data.
> > 
> 
> I think at some point someone mentioned to me that the mon can cause a lot of
> syncs, so running them on the OSDs without syncfs might be detrimental.  For
> the majority of our internal performance testing I've kept them off the OSDs
> just to be sure.

Right.  The only thing interesting about the mon access pattern is that it 
calls fsync() a lot when healthy, and will call syncfs() or sync() during 
recovery.

sage

Re: mon disk access pattern

[Stefan Priebe]

Am 15.06.2012 18:30, schrieb Sage Weil:
> On Fri, 15 Jun 2012, Mark Nelson wrote:
>> On 06/15/2012 04:51 AM, Wido den Hollander wrote:
>> I think at some point someone mentioned to me that the mon can cause a lot of
>> syncs, so running them on the OSDs without syncfs might be detrimental.  For
>> the majority of our internal performance testing I've kept them off the OSDs
>> just to be sure.
>
> Right.  The only thing interesting about the mon access pattern is that it
> calls fsync() a lot when healthy, and will call syncfs() or sync() during
> recovery.

But if i have syncfs support does it still matter if they sit on an osd 
with a seperate disk? Do you have any iops or bandwith knowledge for the 
mon daemon?

Stefan

Re: mon disk access pattern

[Stefan Priebe - Profihost AG]

Am 15.06.2012 13:34, schrieb Mark Nelson:
> On 06/15/2012 04:51 AM, Wido den Hollander wrote:
>> Hi,
>>
>> On 15-06-12 07:52, Stefan Priebe - Profihost AG wrote:
>>> Hello list,
>>>
>>> i read somewhere that the mon has "special" disk access patterns - even
>>> though it does not write much data.
>>
>> "special"? Where did you read that? The monitor has about 1 ~ 2GB of
>> storage.
>>
>> So if your monitor has something like 4GB ~ 8GB of RAM, your kernel
>> should cache almost all your monitor data.
>>
>
> I think at some point someone mentioned to me that the mon can cause a
> lot of syncs, so running them on the OSDs without syncfs might be
> detrimental. For the majority of our internal performance testing I've
> kept them off the OSDs just to be sure.

My main idea behind that was what type of disk i need for a performant 
mon system. SATA, SAS, SSD? Raid?

Stefan