From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from asav21.altibox.net ([109.247.116.8]:47418 "EHLO asav21.altibox.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751862Ab3KYJPZ (ORCPT ); Mon, 25 Nov 2013 04:15:25 -0500 Message-ID: <52931527.8030002@hesbynett.no> Date: Mon, 25 Nov 2013 10:15:19 +0100 From: David Brown MIME-Version: 1.0 To: stan@hardwarefreak.com, John Williams CC: NeilBrown , James Plank , Ric Wheeler , Andrea Mazzoleni , "H. Peter Anvin" , Linux RAID Mailing List , Btrfs BTRFS , David Smith Subject: Re: Triple parity and beyond References: <528A90B7.5010905@zytor.com> <528AA1EB.3010909@zytor.com> <528BCA2D.5010500@redhat.com> <73BEB41F-0FAC-4108-BEA9-DB6D921F6F55@cs.utk.edu> <528D61C5.70902@hardwarefreak.com> <528DADB1.8010604@hardwarefreak.com> <528E8FEC.2070204@hardwarefreak.com> <20131123100753.1820ab7c@notabene.brown> <5290252A.8020508@hardwarefreak.com> <20131123160428.6f1c5898@notabene.brown> <20131123181208.5103bee4@notabene.brown> <52917AA8.1040300@hardwarefreak.com> <52926BE3.1000301@hardwarefreak.com> In-Reply-To: <52926BE3.1000301@hardwarefreak.com> Content-Type: text/plain; charset=ISO-8859-1 Sender: linux-btrfs-owner@vger.kernel.org List-ID: On 24/11/13 22:13, Stan Hoeppner wrote: > On 11/23/2013 11:14 PM, John Williams wrote: >> On Sat, Nov 23, 2013 at 8:03 PM, Stan Hoeppner wrote: >> >>> Parity array rebuilds are read-modify-write operations. The main >>> difference from normal operation RMWs is that the write is always to the >>> same disk. As long as the stripe reads and chunk reconstruction outrun >>> the write throughput then the rebuild speed should be as fast as a >>> mirror rebuild. But this doesn't appear to be what people are >>> experiencing. Parity rebuilds would seem to take much longer. >> >> "This" doesn't appear to be what SOME people, who have reported >> issues, are experiencing. Their issues must be examined on a case by >> case basis. > > Given what you state below this may very well be the case. > >> But I, and a number of other people I have talked to or corresponded >> with, have had mdadm RAID 5 or RAID 6 rebuilds of one drive run at >> approximately the optimal sequential write speed of the replacement >> drive. It is not unusual on a reasonably configured system. > > I freely admit I may have drawn an incorrect conclusion about md parity > rebuild performance based on incomplete data. I simply don't recall > anyone stating here in ~3 years that their parity rebuilds were speedy, > but quite the opposite. I guess it's possible that each one of those > cases was due to another factor, such as user load, slow CPU, bus > bottleneck, wonky disk firmware, backplane issues, etc. > Maybe this is just reporting bias - people are quick to post about problems such as slow rebuilds, but very seldom send a message saying everything worked perfectly! There /are/ reasons why parity raid rebuilds are going to be slower than mirror rebuilds - delays on one disk reading is one issue, and I expect that simultaneous use of the array for normal work will have more impact on parity raid rebuild times than on a mirror array (certainly compared to raid10 with multiple pairs). I just don't think it is quite as bad as you think. From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Brown Subject: Re: Triple parity and beyond Date: Mon, 25 Nov 2013 10:15:19 +0100 Message-ID: <52931527.8030002@hesbynett.no> References: <528A90B7.5010905@zytor.com> <528AA1EB.3010909@zytor.com> <528BCA2D.5010500@redhat.com> <73BEB41F-0FAC-4108-BEA9-DB6D921F6F55@cs.utk.edu> <528D61C5.70902@hardwarefreak.com> <528DADB1.8010604@hardwarefreak.com> <528E8FEC.2070204@hardwarefreak.com> <20131123100753.1820ab7c@notabene.brown> <5290252A.8020508@hardwarefreak.com> <20131123160428.6f1c5898@notabene.brown> <2013112 3181208.5103bee4@notabene.brown> <52917AA8.1040300@hardwarefreak.com> <52926BE3.1000301@hardwarefreak.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <52926BE3.1000301@hardwarefreak.com> Sender: linux-btrfs-owner@vger.kernel.org To: stan@hardwarefreak.com, John Williams Cc: NeilBrown , James Plank , Ric Wheeler , Andrea Mazzoleni , "H. Peter Anvin" , Linux RAID Mailing List , Btrfs BTRFS , David Smith List-Id: linux-raid.ids On 24/11/13 22:13, Stan Hoeppner wrote: > On 11/23/2013 11:14 PM, John Williams wrote: >> On Sat, Nov 23, 2013 at 8:03 PM, Stan Hoeppner wrote: >> >>> Parity array rebuilds are read-modify-write operations. The main >>> difference from normal operation RMWs is that the write is always to the >>> same disk. As long as the stripe reads and chunk reconstruction outrun >>> the write throughput then the rebuild speed should be as fast as a >>> mirror rebuild. But this doesn't appear to be what people are >>> experiencing. Parity rebuilds would seem to take much longer. >> >> "This" doesn't appear to be what SOME people, who have reported >> issues, are experiencing. Their issues must be examined on a case by >> case basis. > > Given what you state below this may very well be the case. > >> But I, and a number of other people I have talked to or corresponded >> with, have had mdadm RAID 5 or RAID 6 rebuilds of one drive run at >> approximately the optimal sequential write speed of the replacement >> drive. It is not unusual on a reasonably configured system. > > I freely admit I may have drawn an incorrect conclusion about md parity > rebuild performance based on incomplete data. I simply don't recall > anyone stating here in ~3 years that their parity rebuilds were speedy, > but quite the opposite. I guess it's possible that each one of those > cases was due to another factor, such as user load, slow CPU, bus > bottleneck, wonky disk firmware, backplane issues, etc. > Maybe this is just reporting bias - people are quick to post about problems such as slow rebuilds, but very seldom send a message saying everything worked perfectly! There /are/ reasons why parity raid rebuilds are going to be slower than mirror rebuilds - delays on one disk reading is one issue, and I expect that simultaneous use of the array for normal work will have more impact on parity raid rebuild times than on a mirror array (certainly compared to raid10 with multiple pairs). I just don't think it is quite as bad as you think.