From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Jim Schutt" Subject: Re: [RFC PATCH 0/6] Understanding delays due to throttling under very heavy write load Date: Mon, 6 Feb 2012 11:20:25 -0700 Message-ID: <4F3019E9.80607@sandia.gov> References: <1328111668-10068-1-git-send-email-jaschut@sandia.gov> <4F29CDAA.408@sandia.gov> <4F2AABF5.6050803@sandia.gov> <4F2ADEAE.8010403@sandia.gov> <4F2AF085.6000405@sandia.gov> <4F2C08A7.2050507@sandia.gov> <3032884323297001561@unknownmsgid> <4F2C6EE6.4050008@sandia.gov> <4F2FFDD3.1010100@sandia.gov> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Return-path: Received: from sentry-two.sandia.gov ([132.175.109.14]:37850 "EHLO sentry-two.sandia.gov" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754593Ab2BFSUy (ORCPT ); Mon, 6 Feb 2012 13:20:54 -0500 In-Reply-To: Sender: ceph-devel-owner@vger.kernel.org List-ID: To: Yehuda Sadeh Weinraub Cc: Gregory Farnum , "ceph-devel@vger.kernel.org" On 02/06/2012 10:22 AM, Yehuda Sadeh Weinraub wrote: > On Mon, Feb 6, 2012 at 8:20 AM, Jim Schutt wrote: >> >> The above suggests to me that the slowdown is a result >> of requests not getting submitted at the same rate as >> when things are running well. >> > > Yeah, it really looks like that. My suggestions wouldn't help there. > > I do see that when things go well the number of writes per device is > capped at ~200 writes per second and the throughput per device is > ~100MB/sec. Is 100MB/sec the expected device throughput? Pretty much, at least for the outer tracks on a drive. I've seen ~108 MB/s with dd to a block device. Also, I've got 8 drives per SAS adapter with 6 Gb/s links, so it seems unlikely to me that my disk subsystem is any sort of significant bottleneck. -- Jim