Re: Infiniband 40GB

[Mark Nelson <mark.nelson@inktank.com>]

Hi Alexandre,

A lot of our testing is on Ubuntu right now.  I'm using the ceph and 
kernel debs from ceph.gitbuilder.com for my tests.  Post some results to 
the list once you get your cluster setup!

Thanks,
Mark


On 6/4/12 8:07 AM, Alexandre DERUMIER wrote:
> Thanks Mark,
> I'll rebuild my cluster with ubuntu precise tomorrow. (Don't have time to backport/maintain libc6 ;)
>
>
> BTW, do you use mainly ubuntu at intank for your tests ?
>
> I'd like to have a setup as close as possible of intank setup.
>
>
> ----- Mail original -----
>
> De: "Mark Nelson"<mark.nelson@inktank.com>
> À: "Alexandre DERUMIER"<aderumier@odiso.com>
> Cc: "Amon Ott"<a.ott@m-privacy.de>, ceph-devel@vger.kernel.org, "Yann Dupont"<Yann.Dupont@univ-nantes.fr>
> Envoyé: Lundi 4 Juin 2012 14:59:58
> Objet: Re: Infiniband 40GB
>
> On 6/4/12 6:40 AM, Alexandre DERUMIER wrote:
>> Hi,
>>
>> I'm currently doing some tests with xfs, debian wheezy with standard libc6 (2.11.3-3) and 3.2 kernel.
>>
>> I'm doing some iostats(3 nodes with 5 osd), and I see constant writes to disks.(as the datas are flushed each second from journal to disk).
>>
>> Journal is big enough (20GB tmpfs) to handle 30s of write.
>>
>> Do you think it's related to the missing syncfs() support ?
>>
>> -Alexandre
>
> Hi Alexandre,
>
> I've included some seekwatcher results for rados bench tests using 16
> concurrent 4MB writes on XFS OSD. One shows ubuntu oneiric and the
> other precise (ie no syncfs support vs syncfs support in libc).
> Unfortunately the original test was on 0.46 and the second test was on
> 0.47.2, so multiple things changed between the tests. Both were tested
> with kernel 3.4. Interestingly the seeks/second don't seem to drop much
> but the overall performance has about doubled. This was using a single
> 7200rpm disk for the OSD data disk and a seperate 7200rpm disk for the
> journal in both cases. I'd definitely try 0.47.2 with a new libc though
> and see how that works for you.
>
> ceph 0.46/oneiric:
> http://nhm.ceph.com/movies/mailinglist-tests/xfs-osd0-oneiric-3.4.mpg
>
> ceph 0.47.2/precise:
> http://nhm.ceph.com/movies/mailinglist-tests/xfs-osd0-precise-3.4.mpg
>
> Mark
>
>
>

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html