From mboxrd@z Thu Jan 1 00:00:00 1970 From: Evgeniy Didin Subject: Re: mmc: block: bonnie++ runs with errors on arc/hsdk board Date: Fri, 16 Mar 2018 17:10:37 +0000 Message-ID: <1521220236.10304.10.camel@synopsys.com> References: <1520942674.10285.8.camel@synopsys.com> <486fe92e-9b53-53a4-f98d-129a97032470@intel.com> <1520956602.10285.10.camel@synopsys.com> <1521030769.10285.16.camel@synopsys.com> <83de0f90-0919-f449-b432-494b9bf524a1@intel.com> <1521126524.10304.5.camel@synopsys.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: "ulf.hansson@linaro.org" , "linux-ext4@vger.kernel.org" , "tytso@mit.edu" , "linus.walleij@linaro.org" , "Alexey.Brodkin@synopsys.com" , "linux-mmc@vger.kernel.org" , "jh80.chung@samsung.com" , "adilger.kernel@dilger.ca" , "linux-snps-arc@lists.infradead.org" , "Eugeniy.Paltsev@synopsys.com" To: "adrian.hunter@intel.com" , "Evgeniy.Didin@synopsys.com" Return-path: In-Reply-To: Content-Language: en-US Content-ID: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-snps-arc" Errors-To: linux-snps-arc-bounces+gla-linux-snps-arc=m.gmane.org@lists.infradead.org List-Id: linux-ext4.vger.kernel.org SGVsbG8gQWRyaWFuLA0KDQo+IFllcy7CoMKgVW5mb3J0dW5hdGVseSB0aGUgY2xvY2sgdXNlZCBp cyBub3QgYWNjdXJhdGUgZW5vdWdoIHRvIGNvcnJlY3RseSBvcmRlcg0KPiB0aGUgZXZlbnRzIGFj cm9zcyBkaWZmZXJlbnQgQ1BVcywgd2hpY2ggbWFrZXMgaXQgdmVyeSBoYXJkIHRvIHNlZSBkZWxh eXMNCj4gYmV0d2VlbiByZXF1ZXN0cy7CoMKgWW91IGNvdWxkIHRyeSBhIGRpZmZlcmVudCBjbG9j ayAtIHJlZmVyIHRoZSAtLWNsb2NraWQNCj4gb3B0aW9uIHRvIHBlcmYgcmVjb3JkLg0KPiANCj4g TmV2ZXJ0aGVsZXNzIGl0IHNob3dzIHRoZXJlIGFyZSBubyBJL08gZXJyb3JzIHdoaWNoIG1lYW5z IHRoZSBlcnJvciByZWNvdmVyeQ0KPiBjYW4gYmUgcnVsZWQgb3V0IGFzIGEgcHJvYmxlbS4NCj4g DQo+IFRoZSBpc3N1ZSBjb3VsZCBiZSBjYXVzZWQgYnkgdGhlIEkvTyBzY2hlZHVsZXIuwqDCoFVu ZGVyIGJsay1tcSB0aGUgZGVmYXVsdA0KPiBzY2hlZHVsZXIgaXMgdGhlIG1xLWRlYWRsaW5lIHNj aGVkdWxlciB3aGVyZWFzIHdpdGhvdXQgYmxrLW1xIHlvdSB3b3VsZA0KPiBwcm9iYWJseSBoYXZl IGJlZW4gdXNpbmcgY2ZxIGJ5IGRlZmF1bHQuwqDCoFlvdSBjb3VsZCB0cnkgdGhlIGJmcSBzY2hl ZHVsZXI6DQo+IA0KPiAJZWNobyBiZnEgPiAvc3lzL2Jsb2NrL21tY2JsazAvcXVldWUvc2NoZWR1 bGVyDQo+IA0KPiBCdXQgeW91IG1pZ2h0IG5lZWQgdG8gYWRkIGl0IHRvIHRoZSBrZXJuZWwgY29u ZmlnIGkuZS4NCj4gDQo+IAlDT05GSUdfSU9TQ0hFRF9CRlE9eQ0KPiANClN3aXRjaGluZyBmcm9t IG1xLWRlYWRsaW5lIHNjaGVkdWxlciB0byBiZnEgZml4ZWQgdGhlIGlzc3VlLg0KQWxzbyBib25u aWUrKyByZXN1bHRzIGhhdmUgY2hhbmdlZDoNCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tPDgtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQpiZnEgc2NoZWR1bGVyOg0K QVJDTGludXgsNTEyTSw2NDYzLDg3LDcyOTcsMCw1NDUwLDAsOTgyNyw5OSwzNDI5NTIsOTksKysr KyssKysrLDE2LDE3NTI1LDEwMCwrKysrKywrKyssMjQzMjksOTksMTc2MjEsMTAwLCsrKysrLCsr KywyNDAwMSwxMDENCg0KbXEtZGVhZGxpbmUgc2NoZWR1bGVyOg0KQVJDTGludXgsNTEyTSw0NDUz LDM2LDY0NzQsMSw1ODUyLDAsMTI5NDAsOTksMzQ0MzI5LDEwMCwrKysrKywrKyssMTYsMjIxNjgs OTgsKysrKyssKysrLDMyNzYwLDk5LDIyNzU1LDEwMCwrKysrKywrKyssMzIyMDUsMTAwDQotLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLTw4LS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLQ0KQXMgSSBzZWUsIHRoZSBwZXJmb3JtYW5jZSBvZiBzZXF1ZW50aWFsIGlucHV0IHBl ciBjaGFyIGFuZCBvZiBmaWxlIG9wZXJhdGlvbnMgaGF2ZSBkZWNyZWFzZWQgZm9yIH4yNSUuDQoN CkRvIHlvdSBoYXZlIGFueSBpZGVhIHdoYXQgY291bGQgYmUgYSByZWFzb24gZm9yIHN1Y2ggYSBs b25nIHN0YWxsaW5nIGluIGNhc2Ugb2YgbXEtZGVhZGxpbmUgSU9zY2hlZHVsZXI/IEkgd291bGQg ZXhwZWN0IGlmIHRoZXJlIGlzIA0Kc29tZSBsb25nIGFzeW5jIG9wZXJhdGlvbiwga2VybmVsIHNo b3VsZCBub3QgYmUgYmxvY2tlZC4gQnV0IHdoYXQgd2Ugc2VlIHVzaW5nIG1xLWRlYWRsaW5lIGlz IGtlcm5lbCBibG9ja2VkIGluIGJpdF93YWl0X2lvKCkuDQpEbyB5b3UgdGhpbmsgdGhpcyBpcyBh IHZhbGlkIGJlaGF2aW9yIGF0IGxlYXN0IGluIGNhc2Ugb2YgbXEtZGVhZGxpbmUgSU9zY2hlZHVs ZXI/DQoNCj4gQWx0ZXJuYXRpdmVseSB5b3UgY291bGQgZmlkZGxlIHdpdGggdGhlIHNjaGVkdWxl ciBwYXJhbWV0ZXJzOg0KPiANCj4gV2l0aCBtcS1kZWFkbGluZSB0aGV5IGFyZToNCj4gDQo+ICMg Z3JlcCAtSCAuIC9zeXMvYmxvY2svbW1jYmxrMC9xdWV1ZS9pb3NjaGVkLyoNCj4gL3N5cy9ibG9j ay9tbWNibGswL3F1ZXVlL2lvc2NoZWQvZmlmb19iYXRjaDoxNg0KPiAvc3lzL2Jsb2NrL21tY2Js azAvcXVldWUvaW9zY2hlZC9mcm9udF9tZXJnZXM6MQ0KPiAvc3lzL2Jsb2NrL21tY2JsazAvcXVl dWUvaW9zY2hlZC9yZWFkX2V4cGlyZTo1MDANCj4gL3N5cy9ibG9jay9tbWNibGswL3F1ZXVlL2lv c2NoZWQvd3JpdGVfZXhwaXJlOjUwMDANCj4gL3N5cy9ibG9jay9tbWNibGswL3F1ZXVlL2lvc2No ZWQvd3JpdGVzX3N0YXJ2ZWQ6Mg0KPiANCj4gWW91IGNvdWxkIHRyeSBkZWNyZWFzaW5nIHRoZSB3 cml0ZV9leHBpcmUgYW5kL29yIGZpZm9fYmF0Y2guDQpJdCBzZWVtcyB0aGF0IGRlY3JlYXNpbmcg ZG9lc24ndCBhZmZlY3Qgb24gdGhpcyBpc3N1ZS4NCg0KQmVzdCByZWdhcmRzLA0KRXZnZW5peSBE aWRpbgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51 eC1zbnBzLWFyYyBtYWlsaW5nIGxpc3QKbGludXgtc25wcy1hcmNAbGlzdHMuaW5mcmFkZWFkLm9y ZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LXNucHMt YXJj From mboxrd@z Thu Jan 1 00:00:00 1970 From: Evgeniy.Didin@synopsys.com (Evgeniy Didin) Date: Fri, 16 Mar 2018 17:10:37 +0000 Subject: mmc: block: bonnie++ runs with errors on arc/hsdk board In-Reply-To: References: <1520942674.10285.8.camel@synopsys.com> <486fe92e-9b53-53a4-f98d-129a97032470@intel.com> <1520956602.10285.10.camel@synopsys.com> <1521030769.10285.16.camel@synopsys.com> <83de0f90-0919-f449-b432-494b9bf524a1@intel.com> <1521126524.10304.5.camel@synopsys.com> List-ID: Message-ID: <1521220236.10304.10.camel@synopsys.com> To: linux-snps-arc@lists.infradead.org Hello Adrian, > Yes.??Unfortunately the clock used is not accurate enough to correctly order > the events across different CPUs, which makes it very hard to see delays > between requests.??You could try a different clock - refer the --clockid > option to perf record. > > Nevertheless it shows there are no I/O errors which means the error recovery > can be ruled out as a problem. > > The issue could be caused by the I/O scheduler.??Under blk-mq the default > scheduler is the mq-deadline scheduler whereas without blk-mq you would > probably have been using cfq by default.??You could try the bfq scheduler: > > echo bfq > /sys/block/mmcblk0/queue/scheduler > > But you might need to add it to the kernel config i.e. > > CONFIG_IOSCHED_BFQ=y > Switching from mq-deadline scheduler to bfq fixed the issue. Also bonnie++ results have changed: -----------------------------------------------<8---------------------------------------------------------------------------- bfq scheduler: ARCLinux,512M,6463,87,7297,0,5450,0,9827,99,342952,99,+++++,+++,16,17525,100,+++++,+++,24329,99,17621,100,+++++,+++,24001,101 mq-deadline scheduler: ARCLinux,512M,4453,36,6474,1,5852,0,12940,99,344329,100,+++++,+++,16,22168,98,+++++,+++,32760,99,22755,100,+++++,+++,32205,100 -----------------------------------------------<8---------------------------------------------------------------------------- As I see, the performance of sequential input per char and of file operations have decreased for ~25%. Do you have any idea what could be a reason for such a long stalling in case of mq-deadline IOscheduler? I would expect if there is some long async operation, kernel should not be blocked. But what we see using mq-deadline is kernel blocked in bit_wait_io(). Do you think this is a valid behavior at least in case of mq-deadline IOscheduler? > Alternatively you could fiddle with the scheduler parameters: > > With mq-deadline they are: > > # grep -H . /sys/block/mmcblk0/queue/iosched/* > /sys/block/mmcblk0/queue/iosched/fifo_batch:16 > /sys/block/mmcblk0/queue/iosched/front_merges:1 > /sys/block/mmcblk0/queue/iosched/read_expire:500 > /sys/block/mmcblk0/queue/iosched/write_expire:5000 > /sys/block/mmcblk0/queue/iosched/writes_starved:2 > > You could try decreasing the write_expire and/or fifo_batch. It seems that decreasing doesn't affect on this issue. Best regards, Evgeniy Didin