Bobtail vs Argonaut Performance Preview

Bobtail vs Argonaut Performance Preview

[Patrick McGarry]

Hey All,

Inktank's Mark Nelson just posted a great performance preview of
Bobtail with comparison to Argonaut.  Feel free to check it out:

http://ow.ly/gg87B


Best Regards,

Patrick

Re: Bobtail vs Argonaut Performance Preview

[Christoph Hellwig]

On Thu, Dec 20, 2012 at 11:08:19AM -0500, Patrick McGarry wrote:
> Hey All,
> 
> Inktank's Mark Nelson just posted a great performance preview of
> Bobtail with comparison to Argonaut.  Feel free to check it out:
> 
> http://ow.ly/gg87B

What's the problem with using a proper link instead of these idiotic
shortening services?

Re: Bobtail vs Argonaut Performance Preview

[Mark Nelson]

On 12/22/2012 02:32 AM, Christoph Hellwig wrote:
> On Thu, Dec 20, 2012 at 11:08:19AM -0500, Patrick McGarry wrote:
>> Hey All,
>>
>> Inktank's Mark Nelson just posted a great performance preview of
>> Bobtail with comparison to Argonaut.  Feel free to check it out:
>>
>> http://ow.ly/gg87B
>
> What's the problem with using a proper link instead of these idiotic
> shortening services?

I think it's just the twitter world leaking out into other mediums. 
Some times it's slightly bewildering to me that in this day and age so 
much conversation seems to happen on a medium that only allows 140 
characters at a time.  That's apparently the world we live in though.  :)

Btw Christoph, thank you for taking the time to read my article.  If 
I've done anything dumb or suboptimal regarding xfs, please do let me 
know.  Soon I will be doing parametric sweeps over ceph parameter spaces 
to see how performance varies on different hardware configurations.  I 
want to make sure the tests are setup as optimally as possible.

>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Mark

Re: Bobtail vs Argonaut Performance Preview

[Patrick McGarry]

Actually one of the chief reasons is the shortened URLs give you
embedded tracking for how many people click through your link.  This
means we can see who clicks on the link from where and get a better
picture of stats w/o having to depend entirely on the imperfect
referrer information provided by google analytics.

On Sat, Dec 22, 2012 at 8:36 AM, Mark Nelson <mark.nelson@inktank.com> wrote:
> On 12/22/2012 02:32 AM, Christoph Hellwig wrote:
>>
>> On Thu, Dec 20, 2012 at 11:08:19AM -0500, Patrick McGarry wrote:
>>>
>>> Hey All,
>>>
>>> Inktank's Mark Nelson just posted a great performance preview of
>>> Bobtail with comparison to Argonaut.  Feel free to check it out:
>>>
>>> http://ow.ly/gg87B
>>
>>
>> What's the problem with using a proper link instead of these idiotic
>> shortening services?
>
>
> I think it's just the twitter world leaking out into other mediums. Some
> times it's slightly bewildering to me that in this day and age so much
> conversation seems to happen on a medium that only allows 140 characters at
> a time.  That's apparently the world we live in though.  :)
>
> Btw Christoph, thank you for taking the time to read my article.  If I've
> done anything dumb or suboptimal regarding xfs, please do let me know.  Soon
> I will be doing parametric sweeps over ceph parameter spaces to see how
> performance varies on different hardware configurations.  I want to make
> sure the tests are setup as optimally as possible.
>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> Mark

Re: Bobtail vs Argonaut Performance Preview

[Christoph Hellwig]

On Sat, Dec 22, 2012 at 07:36:41AM -0600, Mark Nelson wrote:
> Btw Christoph, thank you for taking the time to read my article.  If
> I've done anything dumb or suboptimal regarding xfs, please do let
> me know.  Soon I will be doing parametric sweeps over ceph parameter
> spaces to see how performance varies on different hardware
> configurations.  I want to make sure the tests are setup as
> optimally as possible.

You're defintively missing the "inode64" mount option, which we've
always recommended, and which finally made it to be the default in
Linux 3.7.

Some other things worth playing with, but which aren't guaranteed to
be a win are:

 - use a larger than default log size (e.g. mkfs.xfs -l size=2g)
 - use large directory blocks, similar to what you already do for btrfs
   (mkfs.xfs -n size=16k or 64k)

Also at least for the benchmarks doing concurrent I/O (or any real life
setup) you're probably much better off with a concatenation than a RAID 0
for the multiple disk setup.

Re: Bobtail vs Argonaut Performance Preview

[Mark Nelson]

On 12/22/2012 12:45 PM, Christoph Hellwig wrote:
> On Sat, Dec 22, 2012 at 07:36:41AM -0600, Mark Nelson wrote:
>> Btw Christoph, thank you for taking the time to read my article.  If
>> I've done anything dumb or suboptimal regarding xfs, please do let
>> me know.  Soon I will be doing parametric sweeps over ceph parameter
>> spaces to see how performance varies on different hardware
>> configurations.  I want to make sure the tests are setup as
>> optimally as possible.
>
> You're defintively missing the "inode64" mount option, which we've
> always recommended, and which finally made it to be the default in
> Linux 3.7.
>

Is inode64 typically faster than inode32?  I thought I remembered 
dchinner saying that the situation wasn't always particularly clear and 
it depended on the workload.  Having said that, I can't really see it 
not being a good thing for Ceph to spread metadata out over all of the 
AGs, especially in the multi-disk raid config.  I'll use it for the next 
set of tests.

> Some other things worth playing with, but which aren't guaranteed to
> be a win are:
>
>   - use a larger than default log size (e.g. mkfs.xfs -l size=2g)
>   - use large directory blocks, similar to what you already do for btrfs
>     (mkfs.xfs -n size=16k or 64k)

I'll definitely give them a try at some point.  Thanks for the tips 
Christoph!

>
> Also at least for the benchmarks doing concurrent I/O (or any real life
> setup) you're probably much better off with a concatenation than a RAID 0
> for the multiple disk setup.
>

Re: Bobtail vs Argonaut Performance Preview

[Christoph Hellwig]

On Sat, Dec 22, 2012 at 01:44:15PM -0600, Mark Nelson wrote:
> Is inode64 typically faster than inode32?  I thought I remembered
> dchinner saying that the situation wasn't always particularly clear
> and it depended on the workload.  Having said that, I can't really
> see it not being a good thing for Ceph to spread metadata out over
> all of the AGs, especially in the multi-disk raid config.  I'll use
> it for the next set of tests.

Not for all workloads, but for the vast majority.  Especially in the
case where you have an inode for every 4MB of OSD data you'd much rather
have the inode close to the actual file data.

Re: Bobtail vs Argonaut Performance Preview

[Stefan Priebe]

Hi Christoph,
Am 22.12.2012 19:45, schrieb Christoph Hellwig:
> You're defintively missing the "inode64" mount option, which we've
> always recommended, and which finally made it to be the default in
> Linux 3.7.
>
> Some other things worth playing with, but which aren't guaranteed to
> be a win are:
>
>   - use a larger than default log size (e.g. mkfs.xfs -l size=2g)
>   - use large directory blocks, similar to what you already do for btrfs
>     (mkfs.xfs -n size=16k or 64k)
Thanks for recommendations.

Right now i'm using this in my ceph setup with 240GB SSDs for each OSD. 
May i ask you to comment them? Do they make sense in your eyes?

# i have 12 cores per machine
mkfs.xfs -f -d agcount=12 /dev/sdX2

Mount options: noatime,nodiratime,nobarrier,logbsize=256k,logbufs=8,inode64

Greets,
Stefan