Re: Infiniband 40GB

[Mark Nelson <mark.nelson@inktank.com>]

On 6/7/12 6:25 AM, Alexandre DERUMIER wrote:
> others tests done today: (kernel 3.4 - ubuntu precise)
>
> 3 nodes with 5 osd with btrfs, 1GB journal in tmps forced in writeahead
> 3 nodes with 1 osd with xfs,8GB journal in tmpfs
> 3 nodes with 1 osd with btfs,8GB journal in tmpfs forced in writeahead
>
> 3 nodes with 5 osd with btrfs, 20G journal on disk forced in writeahead
> 3 nodes with 1 osd with xfs,20GB journal on disk
> 3 nodes with 1 osd with btfs,20GB journal on disk forced in writeahead
>
>
>
>
> same behaviour for all cases, writes are constant to disk.
>
> benched with:
> rados -p pool3 bench 60 write -t 16
>
> also with
> fio, bonnie  , random/seq write from guest vm with differents block size.
>
>
>
>
> ----- Mail original -----
>
> De: "Alexandre DERUMIER"<aderumier@odiso.com>
> À: "Mark Nelson"<mark.nelson@inktank.com>
> Cc: "Amon Ott"<a.ott@m-privacy.de>, ceph-devel@vger.kernel.org, "Yann Dupont"<Yann.Dupont@univ-nantes.fr>
> Envoyé: Jeudi 7 Juin 2012 05:31:15
> Objet: Re: Infiniband 40GB
>
> Hi again,
> I have done some tests with journals on a real disk, I have same behaviour.
>
> iostat show constant write to journal and write to disks at the same time since the beginning of the benchmark.
>
>
> maybe can I try to use differents partitions for each journal ? (currently I have 1 partition with 5 journal files of each osd)
>
> -Alexandre
>
>
>
> ----- Mail original -----
>
> De: "Alexandre DERUMIER"<aderumier@odiso.com>
> À: "Mark Nelson"<mark.nelson@inktank.com>
> Cc: "Amon Ott"<a.ott@m-privacy.de>, ceph-devel@vger.kernel.org, "Yann Dupont"<Yann.Dupont@univ-nantes.fr>
> Envoyé: Jeudi 7 Juin 2012 05:11:15
> Objet: Re: Infiniband 40GB
>
> Hi mark,
> I have attached a blktrace of /dev/sdb1 of node1 (osd.0)
>
> and also iostat (showing constant writes)
>
> bench used:
>
> rados -p pool3 bench 60 write -t 16
>
>
> kernel use : 3.4 from intank
>
> I'll do tests with journal on an xfs partition today
>

Hi Alexandre,

I'll try to take a look at the data you sent me later today.

Thanks!
Mark

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html