Re: Slow request warnings on 0.48

[Mark Nelson <mark.nelson@inktank.com>]

On 07/04/2012 11:58 AM, Alexandre DERUMIER wrote:
> Hi, I see same messages here after upgrade to 0.48.
>
> with random write benchmark.
>
> I have more lags than before with 0.47 (but disks are at 100% usage, so can't tell if it's normal or not)
>
>
> ----- Mail original -----
>
> De: "David Blundell"<David.Blundell@100percentit.com>
> À: ceph-devel@vger.kernel.org
> Envoyé: Mercredi 4 Juillet 2012 18:53:02
> Objet: Slow request warnings on 0.48
>
> I have three servers running mon and osd using Ubuntu 12.04 that I have been testing with RADOS storing RBD KVM instances
>
> 0.47.3 worked extremely well (once I got over a few btrfs issues). The same servers running 0.48 give a large number of "[WRN] slow request" messages whenever I generate a lot of random IO in the KVM instances using iozone. The slow responses eventually leads to disk timeouts on the KVM instances.
>
> I have erased the osds and recreated on new btrfs volumes with the same result.
>
> I have also tried switching to xfs using mkfs.xfs -n size=64k with noatime, inode64,delaylog,logbufs=8,logbsize=256k
>
> Xfs gives the same result - the iozone tests run fine until the random IO starts and then there are lots of slow request warnings.
>
> Does anyone have any ideas about the best place to start troubleshooting / debugging?
>
> Thanks,
>
> David

Hi David and Alexandre,

Does this only happen with random writes or also sequential writes?  If 
it happens with sequential writes as well, does it happen with rados bench?

-- 
Mark Nelson
Performance Engineer
Inktank
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html