From mboxrd@z Thu Jan 1 00:00:00 1970 From: Martin Steigerwald Subject: Re: 12x performance drop on md/linux+sw raid1 due to barriers [xfs] Date: Tue, 16 Dec 2008 10:39:07 +0100 Message-ID: <200812161039.07700.Martin@lichtvoll.de> References: <18757.33373.744917.457587@tree.ty.sabi.co.uk> <20081215223857.GF32301@disturbed> (sfid-20081216_091051_242821_0787460D) Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Return-path: In-Reply-To: <20081215223857.GF32301@disturbed> Content-Disposition: inline Sender: linux-raid-owner@vger.kernel.org To: linux-xfs@oss.sgi.com Cc: Dave Chinner , Peter Grandi , Linux RAID , Linux XFS List-Id: linux-raid.ids Am Montag 15 Dezember 2008 schrieb Dave Chinner: > On Sun, Dec 14, 2008 at 10:02:05PM +0000, Peter Grandi wrote: > > [ ... ] > > > > > But - as far as I understood - the filesystem doesn't have to > > > wait for barriers to complete, but could continue issuing IO > > > requests happily. A barrier only means, any request prior to > > > that have to land before and any after it after it. > > > > > > It doesn't mean that the barrier has to land immediately and > > > the filesystem has to wait for this. At least that always was > > > the whole point of barriers for me. If thats not the case I > > > misunderstood the purpose of barriers to the maximum extent > > > possible. > > > > Unfortunately that seems the case. > > > > The purpose of barriers is to guarantee that relevant data is > > known to be on persistent storage (kind of hardware 'fsync'). > > > > In effect write barrier means "tell me when relevant data is on > > persistent storage", or less precisely "flush/sync writes now > > and tell me when it is done". Properties as to ordering are just > > a side effect. > > No, that is incorrect. > > Barriers provide strong ordering semantics. I/Os issued before the > barrier must be completed before the barrier I/O, and I/Os issued > after the barrier write must not be started before the barrier write > completes. The elevators are not allowed to re-=D0=BErder I/Os around > barriers. > > This is all documented in Documentation/block/barrier.txt. Please > read it because most of what you are saying appears to be based on > incorrect assumptions about what barriers do. Hmmm, so I am not completely off track it seems ;-). What I still do not understand then is: How can write barriers + write=20 cache be slower than no write barriers + no cache? I still would expect= =20 write barriers + write cache be in between no barriers + write cache an= d=20 no barriers + no cache performance wise. And would see anything else as= a=20 regression basically. This doesn't go into my brain yet and I thought I understood=20 Documentation/block/barrier.txt well enough before writing my article. Ciao, --=20 Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-raid" i= n the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.12.11.20060308/8.12.11/SuSE Linux 0.7) with ESMTP id mBG9dAFs031844 for ; Tue, 16 Dec 2008 03:39:11 -0600 From: Martin Steigerwald Subject: Re: 12x performance drop on md/linux+sw raid1 due to barriers [xfs] Date: Tue, 16 Dec 2008 10:39:07 +0100 References: <18757.33373.744917.457587@tree.ty.sabi.co.uk> <20081215223857.GF32301@disturbed> (sfid-20081216_091051_242821_0787460D) In-Reply-To: <20081215223857.GF32301@disturbed> MIME-Version: 1.0 Content-Disposition: inline Message-Id: <200812161039.07700.Martin@lichtvoll.de> List-Id: XFS Filesystem from SGI List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: xfs-bounces@oss.sgi.com Errors-To: xfs-bounces@oss.sgi.com To: linux-xfs@oss.sgi.com Cc: Linux RAID QW0gTW9udGFnIDE1IERlemVtYmVyIDIwMDggc2NocmllYiBEYXZlIENoaW5uZXI6Cj4gT24gU3Vu LCBEZWMgMTQsIDIwMDggYXQgMTA6MDI6MDVQTSArMDAwMCwgUGV0ZXIgR3JhbmRpIHdyb3RlOgo+ ID4gWyAuLi4gXQo+ID4KPiA+ID4gQnV0IC0gYXMgZmFyIGFzIEkgdW5kZXJzdG9vZCAtIHRoZSBm aWxlc3lzdGVtIGRvZXNuJ3QgaGF2ZSB0bwo+ID4gPiB3YWl0IGZvciBiYXJyaWVycyB0byBjb21w bGV0ZSwgYnV0IGNvdWxkIGNvbnRpbnVlIGlzc3VpbmcgSU8KPiA+ID4gcmVxdWVzdHMgaGFwcGls eS4gQSBiYXJyaWVyIG9ubHkgbWVhbnMsIGFueSByZXF1ZXN0IHByaW9yIHRvCj4gPiA+IHRoYXQg aGF2ZSB0byBsYW5kIGJlZm9yZSBhbmQgYW55IGFmdGVyIGl0IGFmdGVyIGl0Lgo+ID4gPgo+ID4g PiBJdCBkb2Vzbid0IG1lYW4gdGhhdCB0aGUgYmFycmllciBoYXMgdG8gbGFuZCBpbW1lZGlhdGVs eSBhbmQKPiA+ID4gdGhlIGZpbGVzeXN0ZW0gaGFzIHRvIHdhaXQgZm9yIHRoaXMuIEF0IGxlYXN0 IHRoYXQgYWx3YXlzIHdhcwo+ID4gPiB0aGUgd2hvbGUgcG9pbnQgb2YgYmFycmllcnMgZm9yIG1l LiBJZiB0aGF0cyBub3QgdGhlIGNhc2UgSQo+ID4gPiBtaXN1bmRlcnN0b29kIHRoZSBwdXJwb3Nl IG9mIGJhcnJpZXJzIHRvIHRoZSBtYXhpbXVtIGV4dGVudAo+ID4gPiBwb3NzaWJsZS4KPiA+Cj4g PiBVbmZvcnR1bmF0ZWx5IHRoYXQgc2VlbXMgdGhlIGNhc2UuCj4gPgo+ID4gVGhlIHB1cnBvc2Ug b2YgYmFycmllcnMgaXMgdG8gZ3VhcmFudGVlIHRoYXQgcmVsZXZhbnQgZGF0YSBpcwo+ID4ga25v d24gdG8gYmUgb24gcGVyc2lzdGVudCBzdG9yYWdlIChraW5kIG9mIGhhcmR3YXJlICdmc3luYycp Lgo+ID4KPiA+IEluIGVmZmVjdCB3cml0ZSBiYXJyaWVyIG1lYW5zICJ0ZWxsIG1lIHdoZW4gcmVs ZXZhbnQgZGF0YSBpcyBvbgo+ID4gcGVyc2lzdGVudCBzdG9yYWdlIiwgb3IgbGVzcyBwcmVjaXNl bHkgImZsdXNoL3N5bmMgd3JpdGVzIG5vdwo+ID4gYW5kIHRlbGwgbWUgd2hlbiBpdCBpcyBkb25l Ii4gUHJvcGVydGllcyBhcyB0byBvcmRlcmluZyBhcmUganVzdAo+ID4gYSBzaWRlIGVmZmVjdC4K Pgo+IE5vLCB0aGF0IGlzIGluY29ycmVjdC4KPgo+IEJhcnJpZXJzIHByb3ZpZGUgc3Ryb25nIG9y ZGVyaW5nIHNlbWFudGljcy4gIEkvT3MgaXNzdWVkIGJlZm9yZSB0aGUKPiBiYXJyaWVyIG11c3Qg YmUgY29tcGxldGVkIGJlZm9yZSB0aGUgYmFycmllciBJL08sIGFuZCBJL09zIGlzc3VlZAo+IGFm dGVyIHRoZSBiYXJyaWVyIHdyaXRlIG11c3Qgbm90IGJlIHN0YXJ0ZWQgYmVmb3JlIHRoZSBiYXJy aWVyIHdyaXRlCj4gY29tcGxldGVzLiBUaGUgZWxldmF0b3JzIGFyZSBub3QgYWxsb3dlZCB0byBy ZS3QvnJkZXIgSS9PcyBhcm91bmQKPiBiYXJyaWVycy4KPgo+IFRoaXMgaXMgYWxsIGRvY3VtZW50 ZWQgaW4gRG9jdW1lbnRhdGlvbi9ibG9jay9iYXJyaWVyLnR4dC4gUGxlYXNlCj4gcmVhZCBpdCBi ZWNhdXNlIG1vc3Qgb2Ygd2hhdCB5b3UgYXJlIHNheWluZyBhcHBlYXJzIHRvIGJlIGJhc2VkIG9u Cj4gaW5jb3JyZWN0IGFzc3VtcHRpb25zIGFib3V0IHdoYXQgYmFycmllcnMgZG8uCgpIbW1tLCBz byBJIGFtIG5vdCBjb21wbGV0ZWx5IG9mZiB0cmFjayBpdCBzZWVtcyA7LSkuCgpXaGF0IEkgc3Rp bGwgZG8gbm90IHVuZGVyc3RhbmQgdGhlbiBpczogSG93IGNhbiB3cml0ZSBiYXJyaWVycyArIHdy aXRlIApjYWNoZSBiZSBzbG93ZXIgdGhhbiBubyB3cml0ZSBiYXJyaWVycyArIG5vIGNhY2hlPyBJ IHN0aWxsIHdvdWxkIGV4cGVjdCAKd3JpdGUgYmFycmllcnMgKyB3cml0ZSBjYWNoZSBiZSBpbiBi ZXR3ZWVuIG5vIGJhcnJpZXJzICsgd3JpdGUgY2FjaGUgYW5kIApubyBiYXJyaWVycyArIG5vIGNh Y2hlIHBlcmZvcm1hbmNlIHdpc2UuIEFuZCB3b3VsZCBzZWUgYW55dGhpbmcgZWxzZSBhcyBhIApy ZWdyZXNzaW9uIGJhc2ljYWxseS4KClRoaXMgZG9lc24ndCBnbyBpbnRvIG15IGJyYWluIHlldCBh bmQgSSB0aG91Z2h0IEkgdW5kZXJzdG9vZCAKRG9jdW1lbnRhdGlvbi9ibG9jay9iYXJyaWVyLnR4 dCB3ZWxsIGVub3VnaCBiZWZvcmUgd3JpdGluZyBteSBhcnRpY2xlLgoKQ2lhbywKLS0gCk1hcnRp biAnSGVsaW9zJyBTdGVpZ2Vyd2FsZCAtIGh0dHA6Ly93d3cuTGljaHR2b2xsLmRlCkdQRzogMDNC MCAwRDZDIDAwNDAgMDcxMCA0QUZBICBCODJGIDk5MUIgRUFBQyBBNTk5IDg0QzcKCl9fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCnhmcyBtYWlsaW5nIGxpc3QK eGZzQG9zcy5zZ2kuY29tCmh0dHA6Ly9vc3Muc2dpLmNvbS9tYWlsbWFuL2xpc3RpbmZvL3hmcwo=