diff for duplicates of <1484873646.4857.3.camel@intel.com> diff --git a/a/1.txt b/N1/1.txt index 6fbb95f..acf92eb 100644 --- a/a/1.txt +++ b/N1/1.txt @@ -1,132 +1,167 @@ -T24gVHVlLCAyMDE3LTAxLTE3IGF0IDE4OjAxIC0wODAwLCBBbmRpcnkgWHUgd3JvdGU6DQo+IE9u -IFR1ZSwgSmFuIDE3LCAyMDE3IGF0IDQ6MTYgUE0sIEFuZHJlYXMgRGlsZ2VyIDxhZGlsZ2VyQGRp -bGdlci5jYT4NCj4gd3JvdGU6DQo+ID4gT24gSmFuIDE3LCAyMDE3LCBhdCAzOjE1IFBNLCBBbmRp -cnkgWHUgPGFuZGlyeUBnbWFpbC5jb20+IHdyb3RlOg0KPiA+ID4gT24gVHVlLCBKYW4gMTcsIDIw -MTcgYXQgMTozNSBQTSwgVmlzaGFsIFZlcm1hIDx2aXNoYWwubC52ZXJtYUBpbnRlDQo+ID4gPiBs -LmNvbT4gd3JvdGU6DQo+ID4gPiA+IE9uIDAxLzE2LCBEYXJyaWNrIEouIFdvbmcgd3JvdGU6DQo+ -ID4gPiA+ID4gT24gRnJpLCBKYW4gMTMsIDIwMTcgYXQgMDU6NDk6MTBQTSAtMDcwMCwgVmlzaGFs -IFZlcm1hIHdyb3RlOg0KPiA+ID4gPiA+ID4gT24gMDEvMTQsIFNsYXZhIER1YmV5a28gd3JvdGU6 -DQo+ID4gPiA+ID4gPiA+IA0KPiA+ID4gPiA+ID4gPiAtLS0tIE9yaWdpbmFsIE1lc3NhZ2UgLS0t -LQ0KPiA+ID4gPiA+ID4gPiBTdWJqZWN0OiBbTFNGL01NIFRPUElDXSBCYWRibG9ja3MgY2hlY2tp -bmcvcmVwcmVzZW50YXRpb24NCj4gPiA+ID4gPiA+ID4gaW4gZmlsZXN5c3RlbXMNCj4gPiA+ID4g -PiA+ID4gU2VudDogSmFuIDEzLCAyMDE3IDE6NDAgUE0NCj4gPiA+ID4gPiA+ID4gRnJvbTogIlZl -cm1hLCBWaXNoYWwgTCIgPHZpc2hhbC5sLnZlcm1hQGludGVsLmNvbT4NCj4gPiA+ID4gPiA+ID4g -VG86IGxzZi1wY0BsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZw0KPiA+ID4gPiA+ID4gPiBDYzog -bGludXgtbnZkaW1tQGxpc3RzLjAxLm9yZywgbGludXgtYmxvY2tAdmdlci5rZXJuZWwub3JnDQo+ -ID4gPiA+ID4gPiA+ICwgbGludXgtZnNkZXZlbEB2Z2VyLmtlcm5lbC5vcmcNCj4gPiA+ID4gPiA+ -ID4gDQo+ID4gPiA+ID4gPiA+ID4gVGhlIGN1cnJlbnQgaW1wbGVtZW50YXRpb24gb2YgYmFkYmxv -Y2tzLCB3aGVyZSB3ZQ0KPiA+ID4gPiA+ID4gPiA+IGNvbnN1bHQgdGhlDQo+ID4gPiA+ID4gPiA+ -ID4gYmFkYmxvY2tzIGxpc3QgZm9yIGV2ZXJ5IElPIGluIHRoZSBibG9jayBkcml2ZXIgd29ya3Ms -DQo+ID4gPiA+ID4gPiA+ID4gYW5kIGlzIGENCj4gPiA+ID4gPiA+ID4gPiBsYXN0IG9wdGlvbiBm -YWlsc2FmZSwgYnV0IGZyb20gYSB1c2VyIHBlcnNwZWN0aXZlLCBpdA0KPiA+ID4gPiA+ID4gPiA+ -IGlzbid0IHRoZQ0KPiA+ID4gPiA+ID4gPiA+IGVhc2llc3QgaW50ZXJmYWNlIHRvIHdvcmsgd2l0 -aC4NCj4gPiA+ID4gPiA+ID4gDQo+ID4gPiA+ID4gPiA+IEFzIEkgcmVtZW1iZXIsIEZBVCBhbmQg -SEZTKyBzcGVjaWZpY2F0aW9ucyBjb250YWluDQo+ID4gPiA+ID4gPiA+IGRlc2NyaXB0aW9uIG9m -IGJhZCBibG9ja3MNCj4gPiA+ID4gPiA+ID4gKHBoeXNpY2FsIHNlY3RvcnMpIHRhYmxlLiBJIGJl -bGlldmUgdGhhdCB0aGlzIHRhYmxlIHdhcw0KPiA+ID4gPiA+ID4gPiB1c2VkIGZvciB0aGUgY2Fz -ZSBvZg0KPiA+ID4gPiA+ID4gPiBmbG9wcHkgbWVkaWEuIEJ1dCwgZmluYWxseSwgdGhpcyB0YWJs -ZSBiZWNvbWVzIHRvIGJlIHRoZQ0KPiA+ID4gPiA+ID4gPiBjb21wbGV0ZWx5IG9ic29sZXRlDQo+ -ID4gPiA+ID4gPiA+IGFydGVmYWN0IGJlY2F1c2UgbW9zdGx5IHN0b3JhZ2UgZGV2aWNlcyBhcmUg -cmVsaWFibHkNCj4gPiA+ID4gPiA+ID4gZW5vdWdoLiBXaHkgZG8geW91IG5lZWQNCj4gPiA+ID4g -PiANCj4gPiA+ID4gPiBleHQ0IGhhcyBhIGJhZGJsb2NrcyBpbm9kZSB0byBvd24gYWxsIHRoZSBi -YWQgc3BvdHMgb24gZGlzaywNCj4gPiA+ID4gPiBidXQgSVNUUiBpdA0KPiA+ID4gPiA+IGRvZXNu -J3Qgc3VwcG9ydCg/PykgZXh0ZW50cyBvciA2NC1iaXQgZmlsZXN5c3RlbXMsIGFuZCBtaWdodA0K -PiA+ID4gPiA+IGp1c3QgYmUgYQ0KPiA+ID4gPiA+IHZlc3RpZ2lhbCBvcmdhbiBhdCB0aGlzIHBv -aW50LsKgwqBYRlMgZG9lc24ndCBoYXZlIGFueXRoaW5nIHRvDQo+ID4gPiA+ID4gdHJhY2sgYmFk -DQo+ID4gPiA+ID4gYmxvY2tzIGN1cnJlbnRseS4uLi4NCj4gPiA+ID4gPiANCj4gPiA+ID4gPiA+ -ID4gaW4gZXhwb3NpbmcgdGhlIGJhZCBibG9ja3Mgb24gdGhlIGZpbGUgc3lzdGVtIGxldmVsP8Kg -wqBEbw0KPiA+ID4gPiA+ID4gPiB5b3UgZXhwZWN0IHRoYXQgbmV4dA0KPiA+ID4gPiA+ID4gPiBn -ZW5lcmF0aW9uIG9mIE5WTSBtZW1vcnkgd2lsbCBiZSBzbyB1bnJlbGlhYmxlIHRoYXQgZmlsZQ0K -PiA+ID4gPiA+ID4gPiBzeXN0ZW0gbmVlZHMgdG8gbWFuYWdlDQo+ID4gPiA+ID4gPiA+IGJhZCBi -bG9ja3M/IFdoYXQncyBhYm91dCBlcmFzdXJlIGNvZGluZyBzY2hlbWVzPyBEbyBmaWxlDQo+ID4g -PiA+ID4gPiA+IHN5c3RlbSByZWFsbHkgbmVlZCB0byBzdWZmZXINCj4gPiA+ID4gPiA+ID4gZnJv -bSB0aGUgYmFkIGJsb2NrIGlzc3VlPw0KPiA+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+ID4gVXN1 -YWxseSwgd2UgYXJlIHVzaW5nIExCQXMgYW5kIGl0IGlzIHRoZSByZXNwb25zaWJpbGl0eSBvZg0K -PiA+ID4gPiA+ID4gPiBzdG9yYWdlIGRldmljZSB0byBtYXANCj4gPiA+ID4gPiA+ID4gYSBiYWQg -cGh5c2ljYWwgYmxvY2svcGFnZS9zZWN0b3IgaW50byB2YWxpZCBvbmUuIERvIHlvdQ0KPiA+ID4g -PiA+ID4gPiBtZWFuIHRoYXQgd2UgaGF2ZQ0KPiA+ID4gPiA+ID4gPiBhY2Nlc3MgdG8gcGh5c2lj -YWwgTlZNIG1lbW9yeSBhZGRyZXNzIGRpcmVjdGx5PyBCdXQgaXQNCj4gPiA+ID4gPiA+ID4gbG9v -a3MgbGlrZSB0aGF0IHdlIGNhbg0KPiA+ID4gPiA+ID4gPiBoYXZlIGEgImJhZCBibG9jayIgaXNz -dWUgZXZlbiB3ZSB3aWxsIGFjY2VzcyBkYXRhIGludG8NCj4gPiA+ID4gPiA+ID4gcGFnZSBjYWNo -ZSdzIG1lbW9yeQ0KPiA+ID4gPiA+ID4gPiBwYWdlIChpZiB3ZSB3aWxsIHVzZSBOVk0gbWVtb3J5 -IGZvciBwYWdlIGNhY2hlLCBvZg0KPiA+ID4gPiA+ID4gPiBjb3Vyc2UpLiBTbywgd2hhdCBkbyB5 -b3UNCj4gPiA+ID4gPiA+ID4gaW1wbHkgYnkgImJhZCBibG9jayIgaXNzdWU/DQo+ID4gPiA+ID4g -PiANCj4gPiA+ID4gPiA+IFdlIGRvbid0IGhhdmUgZGlyZWN0IHBoeXNpY2FsIGFjY2VzcyB0byB0 -aGUgZGV2aWNlJ3MgYWRkcmVzcw0KPiA+ID4gPiA+ID4gc3BhY2UsIGluDQo+ID4gPiA+ID4gPiB0 -aGUgc2Vuc2UgdGhlIGRldmljZSBpcyBzdGlsbCBmcmVlIHRvIHBlcmZvcm0gcmVtYXBwaW5nIG9m -DQo+ID4gPiA+ID4gPiBjaHVua3Mgb2YgTlZNDQo+ID4gPiA+ID4gPiB1bmRlcm5lYXRoIHVzLiBU -aGUgcHJvYmxlbSBpcyB0aGF0IHdoZW4gYSBibG9jayBvciBhZGRyZXNzDQo+ID4gPiA+ID4gPiBy -YW5nZSAoYXMNCj4gPiA+ID4gPiA+IHNtYWxsIGFzIGEgY2FjaGUgbGluZSkgZ29lcyBiYWQsIHRo -ZSBkZXZpY2UgbWFpbnRhaW5zIGENCj4gPiA+ID4gPiA+IHBvaXNvbiBiaXQgZm9yDQo+ID4gPiA+ -ID4gPiBldmVyeSBhZmZlY3RlZCBjYWNoZSBsaW5lLiBCZWhpbmQgdGhlIHNjZW5lcywgaXQgbWF5 -IGhhdmUNCj4gPiA+ID4gPiA+IGFscmVhZHkNCj4gPiA+ID4gPiA+IHJlbWFwcGVkIHRoZSByYW5n -ZSwgYnV0IHRoZSBjYWNoZSBsaW5lIHBvaXNvbiBoYXMgdG8gYmUga2VwdA0KPiA+ID4gPiA+ID4g -c28gdGhhdA0KPiA+ID4gPiA+ID4gdGhlcmUgaXMgYSBub3RpZmljYXRpb24gdG8gdGhlIHVzZXIv -b3duZXIgb2YgdGhlIGRhdGEgdGhhdA0KPiA+ID4gPiA+ID4gc29tZXRoaW5nIGhhcw0KPiA+ID4g -PiA+ID4gYmVlbiBsb3N0LiBTaW5jZSBOVk0gaXMgYnl0ZSBhZGRyZXNzYWJsZSBtZW1vcnkgc2l0 -dGluZyBvbg0KPiA+ID4gPiA+ID4gdGhlIG1lbW9yeQ0KPiA+ID4gPiA+ID4gYnVzLCBzdWNoIGEg -cG9pc29uZWQgY2FjaGUgbGluZSByZXN1bHRzIGluIG1lbW9yeSBlcnJvcnMgYW5kDQo+ID4gPiA+ -ID4gPiBTSUdCVVNlcy4NCj4gPiA+ID4gPiA+IENvbXBhcmVkIHRvIHRyYWRhdGlvbmFsIHN0b3Jh -Z2Ugd2hlcmUgYW4gYXBwIHdpbGwgZ2V0IG5pY2UNCj4gPiA+ID4gPiA+IGFuZCBmcmllbmRseQ0K -PiA+ID4gPiA+ID4gKHJlbGF0aXZlbHkgc3BlYWtpbmcuLikgLUVJT3MuIFRoZSB3aG9sZSBiYWRi -bG9ja3MNCj4gPiA+ID4gPiA+IGltcGxlbWVudGF0aW9uIHdhcw0KPiA+ID4gPiA+ID4gZG9uZSBz -byB0aGF0IHRoZSBkcml2ZXIgY2FuIGludGVyY2VwdCBJTyAoaS5lLiByZWFkcykgdG8NCj4gPiA+ -ID4gPiA+IF9rbm93bl8gYmFkDQo+ID4gPiA+ID4gPiBsb2NhdGlvbnMsIGFuZCBzaG9ydC1jaXJj -dWl0IHRoZW0gd2l0aCBhbiBFSU8uIElmIHRoZSBkcml2ZXINCj4gPiA+ID4gPiA+IGRvZXNuJ3QN -Cj4gPiA+ID4gPiA+IGNhdGNoIHRoZXNlLCB0aGUgcmVhZHMgd2lsbCB0dXJuIGludG8gYSBtZW1v -cnkgYnVzIGFjY2VzcywNCj4gPiA+ID4gPiA+IGFuZCB0aGUNCj4gPiA+ID4gPiA+IHBvaXNvbiB3 -aWxsIGNhdXNlIGEgU0lHQlVTLg0KPiA+ID4gPiA+IA0KPiA+ID4gPiA+ICJkcml2ZXIiIC4uLiB5 -b3UgbWVhbiBYRlM/wqDCoE9yIGRvIHlvdSBtZWFuIHRoZSB0aGluZyB0aGF0DQo+ID4gPiA+ID4g -bWFrZXMgcG1lbQ0KPiA+ID4gPiA+IGxvb2sga2luZCBvZiBsaWtlIGEgdHJhZGl0aW9uYWwgYmxv -Y2sgZGV2aWNlPyA6KQ0KPiA+ID4gPiANCj4gPiA+ID4gWWVzLCB0aGUgdGhpbmcgdGhhdCBtYWtl -cyBwbWVtIGxvb2sgbGlrZSBhIGJsb2NrIGRldmljZSA6KSAtLQ0KPiA+ID4gPiBkcml2ZXJzL252 -ZGltbS9wbWVtLmMNCj4gPiA+ID4gDQo+ID4gPiA+ID4gDQo+ID4gPiA+ID4gPiBUaGlzIGVmZm9y -dCBpcyB0byB0cnkgYW5kIG1ha2UgdGhpcyBiYWRibG9jayBjaGVja2luZw0KPiA+ID4gPiA+ID4g -c21hcnRlciAtIGFuZCB0cnkNCj4gPiA+ID4gPiA+IGFuZCByZWR1Y2UgdGhlIHBlbmFsdHkgb24g -ZXZlcnkgSU8gdG8gYSBzbWFsbGVyIHJhbmdlLCB3aGljaA0KPiA+ID4gPiA+ID4gb25seSB0aGUN -Cj4gPiA+ID4gPiA+IGZpbGVzeXN0ZW0gY2FuIGRvLg0KPiA+ID4gPiA+IA0KPiA+ID4gPiA+IFRo -b3VnaC4uLiBub3cgdGhhdCBYRlMgbWVyZ2VkIHRoZSByZXZlcnNlIG1hcHBpbmcgc3VwcG9ydCwN -Cj4gPiA+ID4gPiBJJ3ZlIGJlZW4NCj4gPiA+ID4gPiB3b25kZXJpbmcgaWYgdGhlcmUnbGwgYmUg -YSByZXN1Ym1pc3Npb24gb2YgdGhlIGRldmljZSBlcnJvcnMNCj4gPiA+ID4gPiBjYWxsYmFjaz8N -Cj4gPiA+ID4gPiBJdCBzdGlsbCB3b3VsZCBiZSB1c2VmdWwgdG8gYmUgYWJsZSB0byBpbmZvcm0g -dGhlIHVzZXIgdGhhdA0KPiA+ID4gPiA+IHBhcnQgb2YNCj4gPiA+ID4gPiB0aGVpciBmcyBoYXMg -Z29uZSBiYWQsIG9yLCBiZXR0ZXIgeWV0LCBpZiB0aGUgYnVmZmVyIGlzIHN0aWxsDQo+ID4gPiA+ -ID4gaW4gbWVtb3J5DQo+ID4gPiA+ID4gc29tZXBsYWNlIGVsc2UsIGp1c3Qgd3JpdGUgaXQgYmFj -ayBvdXQuDQo+ID4gPiA+ID4gDQo+ID4gPiA+ID4gT3IgSSBzdXBwb3NlIGlmIHdlIGhhZCBzb21l -IGtpbmQgb2YgcmFpZDEgc2V0IHVwIGJldHdlZW4NCj4gPiA+ID4gPiBtZW1vcmllcyB3ZQ0KPiA+ -ID4gPiA+IGNvdWxkIHJlYWQgb25lIG9mIHRoZSBvdGhlciBjb3BpZXMgYW5kIHJld3JpdGUgaXQg -aW50byB0aGUNCj4gPiA+ID4gPiBmYWlsaW5nDQo+ID4gPiA+ID4gcmVnaW9uIGltbWVkaWF0ZWx5 -Lg0KPiA+ID4gPiANCj4gPiA+ID4gWWVzLCB0aGF0IGlzIGtpbmQgb2Ygd2hhdCBJIHdhcyBob3Bp -bmcgdG8gYWNjb21wbGlzaCB2aWEgdGhpcw0KPiA+ID4gPiBkaXNjdXNzaW9uLiBIb3cgbXVjaCB3 -b3VsZCBmaWxlc3lzdGVtcyB3YW50IHRvIGJlIGludm9sdmVkIGluDQo+ID4gPiA+IHRoaXMgc29y -dA0KPiA+ID4gPiBvZiBiYWRibG9ja3MgaGFuZGxpbmcsIGlmIGF0IGFsbC4gSSBjYW4gcmVmcmVz -aCBteSBwYXRjaGVzIHRoYXQNCj4gPiA+ID4gcHJvdmlkZQ0KPiA+ID4gPiB0aGUgZnMgbm90aWZp -Y2F0aW9uLCBidXQgdGhhdCdzIHRoZSBlYXN5IGJpdCwgYW5kIGEgc3RhcnRpbmcNCj4gPiA+ID4g -cG9pbnQuDQo+ID4gPiA+IA0KPiA+ID4gDQo+ID4gPiBJIGhhdmUgc29tZSBxdWVzdGlvbnMuIFdo -eSBtb3ZpbmcgYmFkYmxvY2sgaGFuZGxpbmcgdG8gZmlsZSBzeXN0ZW0NCj4gPiA+IGxldmVsIGF2 -b2lkIHRoZSBjaGVja2luZyBwaGFzZT8gSW4gZmlsZSBzeXN0ZW0gbGV2ZWwgZm9yIGVhY2ggSS9P -DQo+ID4gPiBJDQo+ID4gPiBzdGlsbCBoYXZlIHRvIGNoZWNrIHRoZSBiYWRibG9jayBsaXN0LCBy -aWdodD8gRG8geW91IG1lYW4gZHVyaW5nDQo+ID4gPiBtb3VudA0KPiA+ID4gaXQgY2FuIGdvIHRo -cm91Z2ggdGhlIHBtZW0gZGV2aWNlIGFuZCBsb2NhdGVzIGFsbCB0aGUgZGF0YQ0KPiA+ID4gc3Ry -dWN0dXJlcw0KPiA+ID4gbWFuZ2xlZCBieSBiYWRibG9ja3MgYW5kIGhhbmRsZSB0aGVtIGFjY29y -ZGluZ2x5LCBzbyB0aGF0IGR1cmluZw0KPiA+ID4gbm9ybWFsIHJ1bm5pbmcgdGhlIGJhZGJsb2Nr -cyB3aWxsIG5ldmVyIGJlIGFjY2Vzc2VkPyBPciwgaWYgdGhlcmUNCj4gPiA+IGlzDQo+ID4gPiBy -ZXBsaWNhdGFpb24vc25hcHNob3Qgc3VwcG9ydCwgdXNlIGEgY29weSB0byByZWNvdmVyIHRoZQ0K -PiA+ID4gYmFkYmxvY2tzPw0KPiA+IA0KPiA+IFdpdGggZXh0NCBiYWRibG9ja3MsIHRoZSBtYWlu -IG91dGNvbWUgaXMgdGhhdCB0aGUgYmFkIGJsb2NrcyB3b3VsZA0KPiA+IGJlDQo+ID4gcGVtYW5l -bnRseSBtYXJrZWQgaW4gdGhlIGFsbG9jYXRpb24gYml0bWFwIGFzIGJlaW5nIHVzZWQsIGFuZCB0 -aGV5DQo+ID4gd291bGQNCj4gPiBuZXZlciBiZSBhbGxvY2F0ZWQgdG8gYSBmaWxlLCBzbyB0aGV5 -IHNob3VsZCBuZXZlciBiZSBhY2Nlc3NlZA0KPiA+IHVubGVzcw0KPiA+IGRvaW5nIGEgZnVsbCBk -ZXZpY2Ugc2NhbiAod2hpY2ggZXh0NCBhbmQgZTJmc2NrIG5ldmVyIGRvKS7CoMKgVGhhdA0KPiA+ -IHdvdWxkDQo+ID4gYXZvaWQgdGhlIG5lZWQgdG8gY2hlY2sgZXZlcnkgSS9PIGFnYWluc3QgdGhl -IGJhZCBibG9ja3MgbGlzdCwgaWYNCj4gPiB0aGUNCj4gPiBkcml2ZXIga25vd3MgdGhhdCB0aGUg -ZmlsZXN5c3RlbSB3aWxsIGhhbmRsZSB0aGlzLg0KPiA+IA0KPiANCj4gVGhhbmsgeW91IGZvciBl -eHBsYW5hdGlvbi4gSG93ZXZlciB0aGlzIG9ubHkgd29ya3MgZm9yIGZyZWUgYmxvY2tzLA0KPiBy -aWdodD8gV2hhdCBhYm91dCBhbGxvY2F0ZWQgYmxvY2tzLCBsaWtlIGZpbGUgZGF0YSBhbmQgbWV0 -YWRhdGE/DQo+IA0KTGlrZSBBbmRyZWFzIHNhaWQsIHRoZSBleHQ0IGJhZGJsb2NrcyBmZWF0dXJl -IGhhcyBub3QgYmVlbiBpbiB1c2UsIGFuZA0KdGhlIGN1cnJlbnQgYmxvY2sgbGF5ZXIgYmFkYmxv -Y2tzIGFyZSBkaXN0aW5jdCBhbmQgdW5yZWxhdGVkIHRvIHRoZXNlLg0KQ2FuIHRoZSBleHQ0IGJh -ZGJsb2NrcyBpbmZyYXN0cnVjdHVyZSBiZSByZXZpdmVkIGFuZCBleHRlbmRlZCBpZiB3ZQ0KZGVj -aWRlIHRvIGFkZCBiYWRibG9ja3MgdG8gZmlsZXN5c3RlbXM/IE1heWJlIC0gdGhhdCB3YXMgb25l -IG9mIHRoZQ0KdG9waWNzIEkgd2FzIGhvcGluZyB0byBkaXNjdXNzL2ZpbmQgb3V0IG1vcmUgYWJv -dXQu +On Tue, 2017-01-17 at 18:01 -0800, Andiry Xu wrote: +> On Tue, Jan 17, 2017 at 4:16 PM, Andreas Dilger <adilger@dilger.ca> +> wrote: +> > On Jan 17, 2017, at 3:15 PM, Andiry Xu <andiry@gmail.com> wrote: +> > > On Tue, Jan 17, 2017 at 1:35 PM, Vishal Verma <vishal.l.verma@inte +> > > l.com> wrote: +> > > > On 01/16, Darrick J. Wong wrote: +> > > > > On Fri, Jan 13, 2017 at 05:49:10PM -0700, Vishal Verma wrote: +> > > > > > On 01/14, Slava Dubeyko wrote: +> > > > > > > +> > > > > > > ---- Original Message ---- +> > > > > > > Subject: [LSF/MM TOPIC] Badblocks checking/representation +> > > > > > > in filesystems +> > > > > > > Sent: Jan 13, 2017 1:40 PM +> > > > > > > From: "Verma, Vishal L" <vishal.l.verma@intel.com> +> > > > > > > To: lsf-pc@lists.linux-foundation.org +> > > > > > > Cc: linux-nvdimm@lists.01.org, linux-block@vger.kernel.org +> > > > > > > , linux-fsdevel@vger.kernel.org +> > > > > > > +> > > > > > > > The current implementation of badblocks, where we +> > > > > > > > consult the +> > > > > > > > badblocks list for every IO in the block driver works, +> > > > > > > > and is a +> > > > > > > > last option failsafe, but from a user perspective, it +> > > > > > > > isn't the +> > > > > > > > easiest interface to work with. +> > > > > > > +> > > > > > > As I remember, FAT and HFS+ specifications contain +> > > > > > > description of bad blocks +> > > > > > > (physical sectors) table. I believe that this table was +> > > > > > > used for the case of +> > > > > > > floppy media. But, finally, this table becomes to be the +> > > > > > > completely obsolete +> > > > > > > artefact because mostly storage devices are reliably +> > > > > > > enough. Why do you need +> > > > > +> > > > > ext4 has a badblocks inode to own all the bad spots on disk, +> > > > > but ISTR it +> > > > > doesn't support(??) extents or 64-bit filesystems, and might +> > > > > just be a +> > > > > vestigial organ at this point. XFS doesn't have anything to +> > > > > track bad +> > > > > blocks currently.... +> > > > > +> > > > > > > in exposing the bad blocks on the file system level? Do +> > > > > > > you expect that next +> > > > > > > generation of NVM memory will be so unreliable that file +> > > > > > > system needs to manage +> > > > > > > bad blocks? What's about erasure coding schemes? Do file +> > > > > > > system really need to suffer +> > > > > > > from the bad block issue? +> > > > > > > +> > > > > > > Usually, we are using LBAs and it is the responsibility of +> > > > > > > storage device to map +> > > > > > > a bad physical block/page/sector into valid one. Do you +> > > > > > > mean that we have +> > > > > > > access to physical NVM memory address directly? But it +> > > > > > > looks like that we can +> > > > > > > have a "bad block" issue even we will access data into +> > > > > > > page cache's memory +> > > > > > > page (if we will use NVM memory for page cache, of +> > > > > > > course). So, what do you +> > > > > > > imply by "bad block" issue? +> > > > > > +> > > > > > We don't have direct physical access to the device's address +> > > > > > space, in +> > > > > > the sense the device is still free to perform remapping of +> > > > > > chunks of NVM +> > > > > > underneath us. The problem is that when a block or address +> > > > > > range (as +> > > > > > small as a cache line) goes bad, the device maintains a +> > > > > > poison bit for +> > > > > > every affected cache line. Behind the scenes, it may have +> > > > > > already +> > > > > > remapped the range, but the cache line poison has to be kept +> > > > > > so that +> > > > > > there is a notification to the user/owner of the data that +> > > > > > something has +> > > > > > been lost. Since NVM is byte addressable memory sitting on +> > > > > > the memory +> > > > > > bus, such a poisoned cache line results in memory errors and +> > > > > > SIGBUSes. +> > > > > > Compared to tradational storage where an app will get nice +> > > > > > and friendly +> > > > > > (relatively speaking..) -EIOs. The whole badblocks +> > > > > > implementation was +> > > > > > done so that the driver can intercept IO (i.e. reads) to +> > > > > > _known_ bad +> > > > > > locations, and short-circuit them with an EIO. If the driver +> > > > > > doesn't +> > > > > > catch these, the reads will turn into a memory bus access, +> > > > > > and the +> > > > > > poison will cause a SIGBUS. +> > > > > +> > > > > "driver" ... you mean XFS? Or do you mean the thing that +> > > > > makes pmem +> > > > > look kind of like a traditional block device? :) +> > > > +> > > > Yes, the thing that makes pmem look like a block device :) -- +> > > > drivers/nvdimm/pmem.c +> > > > +> > > > > +> > > > > > This effort is to try and make this badblock checking +> > > > > > smarter - and try +> > > > > > and reduce the penalty on every IO to a smaller range, which +> > > > > > only the +> > > > > > filesystem can do. +> > > > > +> > > > > Though... now that XFS merged the reverse mapping support, +> > > > > I've been +> > > > > wondering if there'll be a resubmission of the device errors +> > > > > callback? +> > > > > It still would be useful to be able to inform the user that +> > > > > part of +> > > > > their fs has gone bad, or, better yet, if the buffer is still +> > > > > in memory +> > > > > someplace else, just write it back out. +> > > > > +> > > > > Or I suppose if we had some kind of raid1 set up between +> > > > > memories we +> > > > > could read one of the other copies and rewrite it into the +> > > > > failing +> > > > > region immediately. +> > > > +> > > > Yes, that is kind of what I was hoping to accomplish via this +> > > > discussion. How much would filesystems want to be involved in +> > > > this sort +> > > > of badblocks handling, if at all. I can refresh my patches that +> > > > provide +> > > > the fs notification, but that's the easy bit, and a starting +> > > > point. +> > > > +> > > +> > > I have some questions. Why moving badblock handling to file system +> > > level avoid the checking phase? In file system level for each I/O +> > > I +> > > still have to check the badblock list, right? Do you mean during +> > > mount +> > > it can go through the pmem device and locates all the data +> > > structures +> > > mangled by badblocks and handle them accordingly, so that during +> > > normal running the badblocks will never be accessed? Or, if there +> > > is +> > > replicataion/snapshot support, use a copy to recover the +> > > badblocks? +> > +> > With ext4 badblocks, the main outcome is that the bad blocks would +> > be +> > pemanently marked in the allocation bitmap as being used, and they +> > would +> > never be allocated to a file, so they should never be accessed +> > unless +> > doing a full device scan (which ext4 and e2fsck never do). That +> > would +> > avoid the need to check every I/O against the bad blocks list, if +> > the +> > driver knows that the filesystem will handle this. +> > +> +> Thank you for explanation. However this only works for free blocks, +> right? What about allocated blocks, like file data and metadata? +> +Like Andreas said, the ext4 badblocks feature has not been in use, and +the current block layer badblocks are distinct and unrelated to these. +Can the ext4 badblocks infrastructure be revived and extended if we +decide to add badblocks to filesystems? Maybe - that was one of the +topics I was hoping to discuss/find out more about. diff --git a/a/content_digest b/N1/content_digest index d947e20..5fcf540 100644 --- a/a/content_digest +++ b/N1/content_digest @@ -21,137 +21,172 @@ " linux-fsdevel@vger.kernel.org <linux-fsdevel@vger.kernel.org>\0" "\00:1\0" "b\0" - "T24gVHVlLCAyMDE3LTAxLTE3IGF0IDE4OjAxIC0wODAwLCBBbmRpcnkgWHUgd3JvdGU6DQo+IE9u\n" - "IFR1ZSwgSmFuIDE3LCAyMDE3IGF0IDQ6MTYgUE0sIEFuZHJlYXMgRGlsZ2VyIDxhZGlsZ2VyQGRp\n" - "bGdlci5jYT4NCj4gd3JvdGU6DQo+ID4gT24gSmFuIDE3LCAyMDE3LCBhdCAzOjE1IFBNLCBBbmRp\n" - "cnkgWHUgPGFuZGlyeUBnbWFpbC5jb20+IHdyb3RlOg0KPiA+ID4gT24gVHVlLCBKYW4gMTcsIDIw\n" - "MTcgYXQgMTozNSBQTSwgVmlzaGFsIFZlcm1hIDx2aXNoYWwubC52ZXJtYUBpbnRlDQo+ID4gPiBs\n" - "LmNvbT4gd3JvdGU6DQo+ID4gPiA+IE9uIDAxLzE2LCBEYXJyaWNrIEouIFdvbmcgd3JvdGU6DQo+\n" - "ID4gPiA+ID4gT24gRnJpLCBKYW4gMTMsIDIwMTcgYXQgMDU6NDk6MTBQTSAtMDcwMCwgVmlzaGFs\n" - "IFZlcm1hIHdyb3RlOg0KPiA+ID4gPiA+ID4gT24gMDEvMTQsIFNsYXZhIER1YmV5a28gd3JvdGU6\n" - "DQo+ID4gPiA+ID4gPiA+IA0KPiA+ID4gPiA+ID4gPiAtLS0tIE9yaWdpbmFsIE1lc3NhZ2UgLS0t\n" - "LQ0KPiA+ID4gPiA+ID4gPiBTdWJqZWN0OiBbTFNGL01NIFRPUElDXSBCYWRibG9ja3MgY2hlY2tp\n" - "bmcvcmVwcmVzZW50YXRpb24NCj4gPiA+ID4gPiA+ID4gaW4gZmlsZXN5c3RlbXMNCj4gPiA+ID4g\n" - "PiA+ID4gU2VudDogSmFuIDEzLCAyMDE3IDE6NDAgUE0NCj4gPiA+ID4gPiA+ID4gRnJvbTogIlZl\n" - "cm1hLCBWaXNoYWwgTCIgPHZpc2hhbC5sLnZlcm1hQGludGVsLmNvbT4NCj4gPiA+ID4gPiA+ID4g\n" - "VG86IGxzZi1wY0BsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZw0KPiA+ID4gPiA+ID4gPiBDYzog\n" - "bGludXgtbnZkaW1tQGxpc3RzLjAxLm9yZywgbGludXgtYmxvY2tAdmdlci5rZXJuZWwub3JnDQo+\n" - "ID4gPiA+ID4gPiA+ICwgbGludXgtZnNkZXZlbEB2Z2VyLmtlcm5lbC5vcmcNCj4gPiA+ID4gPiA+\n" - "ID4gDQo+ID4gPiA+ID4gPiA+ID4gVGhlIGN1cnJlbnQgaW1wbGVtZW50YXRpb24gb2YgYmFkYmxv\n" - "Y2tzLCB3aGVyZSB3ZQ0KPiA+ID4gPiA+ID4gPiA+IGNvbnN1bHQgdGhlDQo+ID4gPiA+ID4gPiA+\n" - "ID4gYmFkYmxvY2tzIGxpc3QgZm9yIGV2ZXJ5IElPIGluIHRoZSBibG9jayBkcml2ZXIgd29ya3Ms\n" - "DQo+ID4gPiA+ID4gPiA+ID4gYW5kIGlzIGENCj4gPiA+ID4gPiA+ID4gPiBsYXN0IG9wdGlvbiBm\n" - "YWlsc2FmZSwgYnV0IGZyb20gYSB1c2VyIHBlcnNwZWN0aXZlLCBpdA0KPiA+ID4gPiA+ID4gPiA+\n" - "IGlzbid0IHRoZQ0KPiA+ID4gPiA+ID4gPiA+IGVhc2llc3QgaW50ZXJmYWNlIHRvIHdvcmsgd2l0\n" - "aC4NCj4gPiA+ID4gPiA+ID4gDQo+ID4gPiA+ID4gPiA+IEFzIEkgcmVtZW1iZXIsIEZBVCBhbmQg\n" - "SEZTKyBzcGVjaWZpY2F0aW9ucyBjb250YWluDQo+ID4gPiA+ID4gPiA+IGRlc2NyaXB0aW9uIG9m\n" - "IGJhZCBibG9ja3MNCj4gPiA+ID4gPiA+ID4gKHBoeXNpY2FsIHNlY3RvcnMpIHRhYmxlLiBJIGJl\n" - "bGlldmUgdGhhdCB0aGlzIHRhYmxlIHdhcw0KPiA+ID4gPiA+ID4gPiB1c2VkIGZvciB0aGUgY2Fz\n" - "ZSBvZg0KPiA+ID4gPiA+ID4gPiBmbG9wcHkgbWVkaWEuIEJ1dCwgZmluYWxseSwgdGhpcyB0YWJs\n" - "ZSBiZWNvbWVzIHRvIGJlIHRoZQ0KPiA+ID4gPiA+ID4gPiBjb21wbGV0ZWx5IG9ic29sZXRlDQo+\n" - "ID4gPiA+ID4gPiA+IGFydGVmYWN0IGJlY2F1c2UgbW9zdGx5IHN0b3JhZ2UgZGV2aWNlcyBhcmUg\n" - "cmVsaWFibHkNCj4gPiA+ID4gPiA+ID4gZW5vdWdoLiBXaHkgZG8geW91IG5lZWQNCj4gPiA+ID4g\n" - "PiANCj4gPiA+ID4gPiBleHQ0IGhhcyBhIGJhZGJsb2NrcyBpbm9kZSB0byBvd24gYWxsIHRoZSBi\n" - "YWQgc3BvdHMgb24gZGlzaywNCj4gPiA+ID4gPiBidXQgSVNUUiBpdA0KPiA+ID4gPiA+IGRvZXNu\n" - "J3Qgc3VwcG9ydCg/PykgZXh0ZW50cyBvciA2NC1iaXQgZmlsZXN5c3RlbXMsIGFuZCBtaWdodA0K\n" - "PiA+ID4gPiA+IGp1c3QgYmUgYQ0KPiA+ID4gPiA+IHZlc3RpZ2lhbCBvcmdhbiBhdCB0aGlzIHBv\n" - "aW50LsKgwqBYRlMgZG9lc24ndCBoYXZlIGFueXRoaW5nIHRvDQo+ID4gPiA+ID4gdHJhY2sgYmFk\n" - "DQo+ID4gPiA+ID4gYmxvY2tzIGN1cnJlbnRseS4uLi4NCj4gPiA+ID4gPiANCj4gPiA+ID4gPiA+\n" - "ID4gaW4gZXhwb3NpbmcgdGhlIGJhZCBibG9ja3Mgb24gdGhlIGZpbGUgc3lzdGVtIGxldmVsP8Kg\n" - "wqBEbw0KPiA+ID4gPiA+ID4gPiB5b3UgZXhwZWN0IHRoYXQgbmV4dA0KPiA+ID4gPiA+ID4gPiBn\n" - "ZW5lcmF0aW9uIG9mIE5WTSBtZW1vcnkgd2lsbCBiZSBzbyB1bnJlbGlhYmxlIHRoYXQgZmlsZQ0K\n" - "PiA+ID4gPiA+ID4gPiBzeXN0ZW0gbmVlZHMgdG8gbWFuYWdlDQo+ID4gPiA+ID4gPiA+IGJhZCBi\n" - "bG9ja3M/IFdoYXQncyBhYm91dCBlcmFzdXJlIGNvZGluZyBzY2hlbWVzPyBEbyBmaWxlDQo+ID4g\n" - "PiA+ID4gPiA+IHN5c3RlbSByZWFsbHkgbmVlZCB0byBzdWZmZXINCj4gPiA+ID4gPiA+ID4gZnJv\n" - "bSB0aGUgYmFkIGJsb2NrIGlzc3VlPw0KPiA+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+ID4gVXN1\n" - "YWxseSwgd2UgYXJlIHVzaW5nIExCQXMgYW5kIGl0IGlzIHRoZSByZXNwb25zaWJpbGl0eSBvZg0K\n" - "PiA+ID4gPiA+ID4gPiBzdG9yYWdlIGRldmljZSB0byBtYXANCj4gPiA+ID4gPiA+ID4gYSBiYWQg\n" - "cGh5c2ljYWwgYmxvY2svcGFnZS9zZWN0b3IgaW50byB2YWxpZCBvbmUuIERvIHlvdQ0KPiA+ID4g\n" - "PiA+ID4gPiBtZWFuIHRoYXQgd2UgaGF2ZQ0KPiA+ID4gPiA+ID4gPiBhY2Nlc3MgdG8gcGh5c2lj\n" - "YWwgTlZNIG1lbW9yeSBhZGRyZXNzIGRpcmVjdGx5PyBCdXQgaXQNCj4gPiA+ID4gPiA+ID4gbG9v\n" - "a3MgbGlrZSB0aGF0IHdlIGNhbg0KPiA+ID4gPiA+ID4gPiBoYXZlIGEgImJhZCBibG9jayIgaXNz\n" - "dWUgZXZlbiB3ZSB3aWxsIGFjY2VzcyBkYXRhIGludG8NCj4gPiA+ID4gPiA+ID4gcGFnZSBjYWNo\n" - "ZSdzIG1lbW9yeQ0KPiA+ID4gPiA+ID4gPiBwYWdlIChpZiB3ZSB3aWxsIHVzZSBOVk0gbWVtb3J5\n" - "IGZvciBwYWdlIGNhY2hlLCBvZg0KPiA+ID4gPiA+ID4gPiBjb3Vyc2UpLiBTbywgd2hhdCBkbyB5\n" - "b3UNCj4gPiA+ID4gPiA+ID4gaW1wbHkgYnkgImJhZCBibG9jayIgaXNzdWU/DQo+ID4gPiA+ID4g\n" - "PiANCj4gPiA+ID4gPiA+IFdlIGRvbid0IGhhdmUgZGlyZWN0IHBoeXNpY2FsIGFjY2VzcyB0byB0\n" - "aGUgZGV2aWNlJ3MgYWRkcmVzcw0KPiA+ID4gPiA+ID4gc3BhY2UsIGluDQo+ID4gPiA+ID4gPiB0\n" - "aGUgc2Vuc2UgdGhlIGRldmljZSBpcyBzdGlsbCBmcmVlIHRvIHBlcmZvcm0gcmVtYXBwaW5nIG9m\n" - "DQo+ID4gPiA+ID4gPiBjaHVua3Mgb2YgTlZNDQo+ID4gPiA+ID4gPiB1bmRlcm5lYXRoIHVzLiBU\n" - "aGUgcHJvYmxlbSBpcyB0aGF0IHdoZW4gYSBibG9jayBvciBhZGRyZXNzDQo+ID4gPiA+ID4gPiBy\n" - "YW5nZSAoYXMNCj4gPiA+ID4gPiA+IHNtYWxsIGFzIGEgY2FjaGUgbGluZSkgZ29lcyBiYWQsIHRo\n" - "ZSBkZXZpY2UgbWFpbnRhaW5zIGENCj4gPiA+ID4gPiA+IHBvaXNvbiBiaXQgZm9yDQo+ID4gPiA+\n" - "ID4gPiBldmVyeSBhZmZlY3RlZCBjYWNoZSBsaW5lLiBCZWhpbmQgdGhlIHNjZW5lcywgaXQgbWF5\n" - "IGhhdmUNCj4gPiA+ID4gPiA+IGFscmVhZHkNCj4gPiA+ID4gPiA+IHJlbWFwcGVkIHRoZSByYW5n\n" - "ZSwgYnV0IHRoZSBjYWNoZSBsaW5lIHBvaXNvbiBoYXMgdG8gYmUga2VwdA0KPiA+ID4gPiA+ID4g\n" - "c28gdGhhdA0KPiA+ID4gPiA+ID4gdGhlcmUgaXMgYSBub3RpZmljYXRpb24gdG8gdGhlIHVzZXIv\n" - "b3duZXIgb2YgdGhlIGRhdGEgdGhhdA0KPiA+ID4gPiA+ID4gc29tZXRoaW5nIGhhcw0KPiA+ID4g\n" - "PiA+ID4gYmVlbiBsb3N0LiBTaW5jZSBOVk0gaXMgYnl0ZSBhZGRyZXNzYWJsZSBtZW1vcnkgc2l0\n" - "dGluZyBvbg0KPiA+ID4gPiA+ID4gdGhlIG1lbW9yeQ0KPiA+ID4gPiA+ID4gYnVzLCBzdWNoIGEg\n" - "cG9pc29uZWQgY2FjaGUgbGluZSByZXN1bHRzIGluIG1lbW9yeSBlcnJvcnMgYW5kDQo+ID4gPiA+\n" - "ID4gPiBTSUdCVVNlcy4NCj4gPiA+ID4gPiA+IENvbXBhcmVkIHRvIHRyYWRhdGlvbmFsIHN0b3Jh\n" - "Z2Ugd2hlcmUgYW4gYXBwIHdpbGwgZ2V0IG5pY2UNCj4gPiA+ID4gPiA+IGFuZCBmcmllbmRseQ0K\n" - "PiA+ID4gPiA+ID4gKHJlbGF0aXZlbHkgc3BlYWtpbmcuLikgLUVJT3MuIFRoZSB3aG9sZSBiYWRi\n" - "bG9ja3MNCj4gPiA+ID4gPiA+IGltcGxlbWVudGF0aW9uIHdhcw0KPiA+ID4gPiA+ID4gZG9uZSBz\n" - "byB0aGF0IHRoZSBkcml2ZXIgY2FuIGludGVyY2VwdCBJTyAoaS5lLiByZWFkcykgdG8NCj4gPiA+\n" - "ID4gPiA+IF9rbm93bl8gYmFkDQo+ID4gPiA+ID4gPiBsb2NhdGlvbnMsIGFuZCBzaG9ydC1jaXJj\n" - "dWl0IHRoZW0gd2l0aCBhbiBFSU8uIElmIHRoZSBkcml2ZXINCj4gPiA+ID4gPiA+IGRvZXNuJ3QN\n" - "Cj4gPiA+ID4gPiA+IGNhdGNoIHRoZXNlLCB0aGUgcmVhZHMgd2lsbCB0dXJuIGludG8gYSBtZW1v\n" - "cnkgYnVzIGFjY2VzcywNCj4gPiA+ID4gPiA+IGFuZCB0aGUNCj4gPiA+ID4gPiA+IHBvaXNvbiB3\n" - "aWxsIGNhdXNlIGEgU0lHQlVTLg0KPiA+ID4gPiA+IA0KPiA+ID4gPiA+ICJkcml2ZXIiIC4uLiB5\n" - "b3UgbWVhbiBYRlM/wqDCoE9yIGRvIHlvdSBtZWFuIHRoZSB0aGluZyB0aGF0DQo+ID4gPiA+ID4g\n" - "bWFrZXMgcG1lbQ0KPiA+ID4gPiA+IGxvb2sga2luZCBvZiBsaWtlIGEgdHJhZGl0aW9uYWwgYmxv\n" - "Y2sgZGV2aWNlPyA6KQ0KPiA+ID4gPiANCj4gPiA+ID4gWWVzLCB0aGUgdGhpbmcgdGhhdCBtYWtl\n" - "cyBwbWVtIGxvb2sgbGlrZSBhIGJsb2NrIGRldmljZSA6KSAtLQ0KPiA+ID4gPiBkcml2ZXJzL252\n" - "ZGltbS9wbWVtLmMNCj4gPiA+ID4gDQo+ID4gPiA+ID4gDQo+ID4gPiA+ID4gPiBUaGlzIGVmZm9y\n" - "dCBpcyB0byB0cnkgYW5kIG1ha2UgdGhpcyBiYWRibG9jayBjaGVja2luZw0KPiA+ID4gPiA+ID4g\n" - "c21hcnRlciAtIGFuZCB0cnkNCj4gPiA+ID4gPiA+IGFuZCByZWR1Y2UgdGhlIHBlbmFsdHkgb24g\n" - "ZXZlcnkgSU8gdG8gYSBzbWFsbGVyIHJhbmdlLCB3aGljaA0KPiA+ID4gPiA+ID4gb25seSB0aGUN\n" - "Cj4gPiA+ID4gPiA+IGZpbGVzeXN0ZW0gY2FuIGRvLg0KPiA+ID4gPiA+IA0KPiA+ID4gPiA+IFRo\n" - "b3VnaC4uLiBub3cgdGhhdCBYRlMgbWVyZ2VkIHRoZSByZXZlcnNlIG1hcHBpbmcgc3VwcG9ydCwN\n" - "Cj4gPiA+ID4gPiBJJ3ZlIGJlZW4NCj4gPiA+ID4gPiB3b25kZXJpbmcgaWYgdGhlcmUnbGwgYmUg\n" - "YSByZXN1Ym1pc3Npb24gb2YgdGhlIGRldmljZSBlcnJvcnMNCj4gPiA+ID4gPiBjYWxsYmFjaz8N\n" - "Cj4gPiA+ID4gPiBJdCBzdGlsbCB3b3VsZCBiZSB1c2VmdWwgdG8gYmUgYWJsZSB0byBpbmZvcm0g\n" - "dGhlIHVzZXIgdGhhdA0KPiA+ID4gPiA+IHBhcnQgb2YNCj4gPiA+ID4gPiB0aGVpciBmcyBoYXMg\n" - "Z29uZSBiYWQsIG9yLCBiZXR0ZXIgeWV0LCBpZiB0aGUgYnVmZmVyIGlzIHN0aWxsDQo+ID4gPiA+\n" - "ID4gaW4gbWVtb3J5DQo+ID4gPiA+ID4gc29tZXBsYWNlIGVsc2UsIGp1c3Qgd3JpdGUgaXQgYmFj\n" - "ayBvdXQuDQo+ID4gPiA+ID4gDQo+ID4gPiA+ID4gT3IgSSBzdXBwb3NlIGlmIHdlIGhhZCBzb21l\n" - "IGtpbmQgb2YgcmFpZDEgc2V0IHVwIGJldHdlZW4NCj4gPiA+ID4gPiBtZW1vcmllcyB3ZQ0KPiA+\n" - "ID4gPiA+IGNvdWxkIHJlYWQgb25lIG9mIHRoZSBvdGhlciBjb3BpZXMgYW5kIHJld3JpdGUgaXQg\n" - "aW50byB0aGUNCj4gPiA+ID4gPiBmYWlsaW5nDQo+ID4gPiA+ID4gcmVnaW9uIGltbWVkaWF0ZWx5\n" - "Lg0KPiA+ID4gPiANCj4gPiA+ID4gWWVzLCB0aGF0IGlzIGtpbmQgb2Ygd2hhdCBJIHdhcyBob3Bp\n" - "bmcgdG8gYWNjb21wbGlzaCB2aWEgdGhpcw0KPiA+ID4gPiBkaXNjdXNzaW9uLiBIb3cgbXVjaCB3\n" - "b3VsZCBmaWxlc3lzdGVtcyB3YW50IHRvIGJlIGludm9sdmVkIGluDQo+ID4gPiA+IHRoaXMgc29y\n" - "dA0KPiA+ID4gPiBvZiBiYWRibG9ja3MgaGFuZGxpbmcsIGlmIGF0IGFsbC4gSSBjYW4gcmVmcmVz\n" - "aCBteSBwYXRjaGVzIHRoYXQNCj4gPiA+ID4gcHJvdmlkZQ0KPiA+ID4gPiB0aGUgZnMgbm90aWZp\n" - "Y2F0aW9uLCBidXQgdGhhdCdzIHRoZSBlYXN5IGJpdCwgYW5kIGEgc3RhcnRpbmcNCj4gPiA+ID4g\n" - "cG9pbnQuDQo+ID4gPiA+IA0KPiA+ID4gDQo+ID4gPiBJIGhhdmUgc29tZSBxdWVzdGlvbnMuIFdo\n" - "eSBtb3ZpbmcgYmFkYmxvY2sgaGFuZGxpbmcgdG8gZmlsZSBzeXN0ZW0NCj4gPiA+IGxldmVsIGF2\n" - "b2lkIHRoZSBjaGVja2luZyBwaGFzZT8gSW4gZmlsZSBzeXN0ZW0gbGV2ZWwgZm9yIGVhY2ggSS9P\n" - "DQo+ID4gPiBJDQo+ID4gPiBzdGlsbCBoYXZlIHRvIGNoZWNrIHRoZSBiYWRibG9jayBsaXN0LCBy\n" - "aWdodD8gRG8geW91IG1lYW4gZHVyaW5nDQo+ID4gPiBtb3VudA0KPiA+ID4gaXQgY2FuIGdvIHRo\n" - "cm91Z2ggdGhlIHBtZW0gZGV2aWNlIGFuZCBsb2NhdGVzIGFsbCB0aGUgZGF0YQ0KPiA+ID4gc3Ry\n" - "dWN0dXJlcw0KPiA+ID4gbWFuZ2xlZCBieSBiYWRibG9ja3MgYW5kIGhhbmRsZSB0aGVtIGFjY29y\n" - "ZGluZ2x5LCBzbyB0aGF0IGR1cmluZw0KPiA+ID4gbm9ybWFsIHJ1bm5pbmcgdGhlIGJhZGJsb2Nr\n" - "cyB3aWxsIG5ldmVyIGJlIGFjY2Vzc2VkPyBPciwgaWYgdGhlcmUNCj4gPiA+IGlzDQo+ID4gPiBy\n" - "ZXBsaWNhdGFpb24vc25hcHNob3Qgc3VwcG9ydCwgdXNlIGEgY29weSB0byByZWNvdmVyIHRoZQ0K\n" - "PiA+ID4gYmFkYmxvY2tzPw0KPiA+IA0KPiA+IFdpdGggZXh0NCBiYWRibG9ja3MsIHRoZSBtYWlu\n" - "IG91dGNvbWUgaXMgdGhhdCB0aGUgYmFkIGJsb2NrcyB3b3VsZA0KPiA+IGJlDQo+ID4gcGVtYW5l\n" - "bnRseSBtYXJrZWQgaW4gdGhlIGFsbG9jYXRpb24gYml0bWFwIGFzIGJlaW5nIHVzZWQsIGFuZCB0\n" - "aGV5DQo+ID4gd291bGQNCj4gPiBuZXZlciBiZSBhbGxvY2F0ZWQgdG8gYSBmaWxlLCBzbyB0aGV5\n" - "IHNob3VsZCBuZXZlciBiZSBhY2Nlc3NlZA0KPiA+IHVubGVzcw0KPiA+IGRvaW5nIGEgZnVsbCBk\n" - "ZXZpY2Ugc2NhbiAod2hpY2ggZXh0NCBhbmQgZTJmc2NrIG5ldmVyIGRvKS7CoMKgVGhhdA0KPiA+\n" - "IHdvdWxkDQo+ID4gYXZvaWQgdGhlIG5lZWQgdG8gY2hlY2sgZXZlcnkgSS9PIGFnYWluc3QgdGhl\n" - "IGJhZCBibG9ja3MgbGlzdCwgaWYNCj4gPiB0aGUNCj4gPiBkcml2ZXIga25vd3MgdGhhdCB0aGUg\n" - "ZmlsZXN5c3RlbSB3aWxsIGhhbmRsZSB0aGlzLg0KPiA+IA0KPiANCj4gVGhhbmsgeW91IGZvciBl\n" - "eHBsYW5hdGlvbi4gSG93ZXZlciB0aGlzIG9ubHkgd29ya3MgZm9yIGZyZWUgYmxvY2tzLA0KPiBy\n" - "aWdodD8gV2hhdCBhYm91dCBhbGxvY2F0ZWQgYmxvY2tzLCBsaWtlIGZpbGUgZGF0YSBhbmQgbWV0\n" - "YWRhdGE/DQo+IA0KTGlrZSBBbmRyZWFzIHNhaWQsIHRoZSBleHQ0IGJhZGJsb2NrcyBmZWF0dXJl\n" - "IGhhcyBub3QgYmVlbiBpbiB1c2UsIGFuZA0KdGhlIGN1cnJlbnQgYmxvY2sgbGF5ZXIgYmFkYmxv\n" - "Y2tzIGFyZSBkaXN0aW5jdCBhbmQgdW5yZWxhdGVkIHRvIHRoZXNlLg0KQ2FuIHRoZSBleHQ0IGJh\n" - "ZGJsb2NrcyBpbmZyYXN0cnVjdHVyZSBiZSByZXZpdmVkIGFuZCBleHRlbmRlZCBpZiB3ZQ0KZGVj\n" - "aWRlIHRvIGFkZCBiYWRibG9ja3MgdG8gZmlsZXN5c3RlbXM/IE1heWJlIC0gdGhhdCB3YXMgb25l\n" - "IG9mIHRoZQ0KdG9waWNzIEkgd2FzIGhvcGluZyB0byBkaXNjdXNzL2ZpbmQgb3V0IG1vcmUgYWJv\n" - dXQu + "On Tue, 2017-01-17 at 18:01 -0800, Andiry Xu wrote:\n" + "> On Tue, Jan 17, 2017 at 4:16 PM, Andreas Dilger <adilger@dilger.ca>\n" + "> wrote:\n" + "> > On Jan 17, 2017, at 3:15 PM, Andiry Xu <andiry@gmail.com> wrote:\n" + "> > > On Tue, Jan 17, 2017 at 1:35 PM, Vishal Verma <vishal.l.verma@inte\n" + "> > > l.com> wrote:\n" + "> > > > On 01/16, Darrick J. Wong wrote:\n" + "> > > > > On Fri, Jan 13, 2017 at 05:49:10PM -0700, Vishal Verma wrote:\n" + "> > > > > > On 01/14, Slava Dubeyko wrote:\n" + "> > > > > > > \n" + "> > > > > > > ---- Original Message ----\n" + "> > > > > > > Subject: [LSF/MM TOPIC] Badblocks checking/representation\n" + "> > > > > > > in filesystems\n" + "> > > > > > > Sent: Jan 13, 2017 1:40 PM\n" + "> > > > > > > From: \"Verma, Vishal L\" <vishal.l.verma@intel.com>\n" + "> > > > > > > To: lsf-pc@lists.linux-foundation.org\n" + "> > > > > > > Cc: linux-nvdimm@lists.01.org, linux-block@vger.kernel.org\n" + "> > > > > > > , linux-fsdevel@vger.kernel.org\n" + "> > > > > > > \n" + "> > > > > > > > The current implementation of badblocks, where we\n" + "> > > > > > > > consult the\n" + "> > > > > > > > badblocks list for every IO in the block driver works,\n" + "> > > > > > > > and is a\n" + "> > > > > > > > last option failsafe, but from a user perspective, it\n" + "> > > > > > > > isn't the\n" + "> > > > > > > > easiest interface to work with.\n" + "> > > > > > > \n" + "> > > > > > > As I remember, FAT and HFS+ specifications contain\n" + "> > > > > > > description of bad blocks\n" + "> > > > > > > (physical sectors) table. I believe that this table was\n" + "> > > > > > > used for the case of\n" + "> > > > > > > floppy media. But, finally, this table becomes to be the\n" + "> > > > > > > completely obsolete\n" + "> > > > > > > artefact because mostly storage devices are reliably\n" + "> > > > > > > enough. Why do you need\n" + "> > > > > \n" + "> > > > > ext4 has a badblocks inode to own all the bad spots on disk,\n" + "> > > > > but ISTR it\n" + "> > > > > doesn't support(??) extents or 64-bit filesystems, and might\n" + "> > > > > just be a\n" + "> > > > > vestigial organ at this point.\302\240\302\240XFS doesn't have anything to\n" + "> > > > > track bad\n" + "> > > > > blocks currently....\n" + "> > > > > \n" + "> > > > > > > in exposing the bad blocks on the file system level?\302\240\302\240Do\n" + "> > > > > > > you expect that next\n" + "> > > > > > > generation of NVM memory will be so unreliable that file\n" + "> > > > > > > system needs to manage\n" + "> > > > > > > bad blocks? What's about erasure coding schemes? Do file\n" + "> > > > > > > system really need to suffer\n" + "> > > > > > > from the bad block issue?\n" + "> > > > > > > \n" + "> > > > > > > Usually, we are using LBAs and it is the responsibility of\n" + "> > > > > > > storage device to map\n" + "> > > > > > > a bad physical block/page/sector into valid one. Do you\n" + "> > > > > > > mean that we have\n" + "> > > > > > > access to physical NVM memory address directly? But it\n" + "> > > > > > > looks like that we can\n" + "> > > > > > > have a \"bad block\" issue even we will access data into\n" + "> > > > > > > page cache's memory\n" + "> > > > > > > page (if we will use NVM memory for page cache, of\n" + "> > > > > > > course). So, what do you\n" + "> > > > > > > imply by \"bad block\" issue?\n" + "> > > > > > \n" + "> > > > > > We don't have direct physical access to the device's address\n" + "> > > > > > space, in\n" + "> > > > > > the sense the device is still free to perform remapping of\n" + "> > > > > > chunks of NVM\n" + "> > > > > > underneath us. The problem is that when a block or address\n" + "> > > > > > range (as\n" + "> > > > > > small as a cache line) goes bad, the device maintains a\n" + "> > > > > > poison bit for\n" + "> > > > > > every affected cache line. Behind the scenes, it may have\n" + "> > > > > > already\n" + "> > > > > > remapped the range, but the cache line poison has to be kept\n" + "> > > > > > so that\n" + "> > > > > > there is a notification to the user/owner of the data that\n" + "> > > > > > something has\n" + "> > > > > > been lost. Since NVM is byte addressable memory sitting on\n" + "> > > > > > the memory\n" + "> > > > > > bus, such a poisoned cache line results in memory errors and\n" + "> > > > > > SIGBUSes.\n" + "> > > > > > Compared to tradational storage where an app will get nice\n" + "> > > > > > and friendly\n" + "> > > > > > (relatively speaking..) -EIOs. The whole badblocks\n" + "> > > > > > implementation was\n" + "> > > > > > done so that the driver can intercept IO (i.e. reads) to\n" + "> > > > > > _known_ bad\n" + "> > > > > > locations, and short-circuit them with an EIO. If the driver\n" + "> > > > > > doesn't\n" + "> > > > > > catch these, the reads will turn into a memory bus access,\n" + "> > > > > > and the\n" + "> > > > > > poison will cause a SIGBUS.\n" + "> > > > > \n" + "> > > > > \"driver\" ... you mean XFS?\302\240\302\240Or do you mean the thing that\n" + "> > > > > makes pmem\n" + "> > > > > look kind of like a traditional block device? :)\n" + "> > > > \n" + "> > > > Yes, the thing that makes pmem look like a block device :) --\n" + "> > > > drivers/nvdimm/pmem.c\n" + "> > > > \n" + "> > > > > \n" + "> > > > > > This effort is to try and make this badblock checking\n" + "> > > > > > smarter - and try\n" + "> > > > > > and reduce the penalty on every IO to a smaller range, which\n" + "> > > > > > only the\n" + "> > > > > > filesystem can do.\n" + "> > > > > \n" + "> > > > > Though... now that XFS merged the reverse mapping support,\n" + "> > > > > I've been\n" + "> > > > > wondering if there'll be a resubmission of the device errors\n" + "> > > > > callback?\n" + "> > > > > It still would be useful to be able to inform the user that\n" + "> > > > > part of\n" + "> > > > > their fs has gone bad, or, better yet, if the buffer is still\n" + "> > > > > in memory\n" + "> > > > > someplace else, just write it back out.\n" + "> > > > > \n" + "> > > > > Or I suppose if we had some kind of raid1 set up between\n" + "> > > > > memories we\n" + "> > > > > could read one of the other copies and rewrite it into the\n" + "> > > > > failing\n" + "> > > > > region immediately.\n" + "> > > > \n" + "> > > > Yes, that is kind of what I was hoping to accomplish via this\n" + "> > > > discussion. How much would filesystems want to be involved in\n" + "> > > > this sort\n" + "> > > > of badblocks handling, if at all. I can refresh my patches that\n" + "> > > > provide\n" + "> > > > the fs notification, but that's the easy bit, and a starting\n" + "> > > > point.\n" + "> > > > \n" + "> > > \n" + "> > > I have some questions. Why moving badblock handling to file system\n" + "> > > level avoid the checking phase? In file system level for each I/O\n" + "> > > I\n" + "> > > still have to check the badblock list, right? Do you mean during\n" + "> > > mount\n" + "> > > it can go through the pmem device and locates all the data\n" + "> > > structures\n" + "> > > mangled by badblocks and handle them accordingly, so that during\n" + "> > > normal running the badblocks will never be accessed? Or, if there\n" + "> > > is\n" + "> > > replicataion/snapshot support, use a copy to recover the\n" + "> > > badblocks?\n" + "> > \n" + "> > With ext4 badblocks, the main outcome is that the bad blocks would\n" + "> > be\n" + "> > pemanently marked in the allocation bitmap as being used, and they\n" + "> > would\n" + "> > never be allocated to a file, so they should never be accessed\n" + "> > unless\n" + "> > doing a full device scan (which ext4 and e2fsck never do).\302\240\302\240That\n" + "> > would\n" + "> > avoid the need to check every I/O against the bad blocks list, if\n" + "> > the\n" + "> > driver knows that the filesystem will handle this.\n" + "> > \n" + "> \n" + "> Thank you for explanation. However this only works for free blocks,\n" + "> right? What about allocated blocks, like file data and metadata?\n" + "> \n" + "Like Andreas said, the ext4 badblocks feature has not been in use, and\n" + "the current block layer badblocks are distinct and unrelated to these.\n" + "Can the ext4 badblocks infrastructure be revived and extended if we\n" + "decide to add badblocks to filesystems? Maybe - that was one of the\n" + topics I was hoping to discuss/find out more about. -fe0d0d5e26337192ac9b3a9c09447a4d5837f9606c2c2351fab57a50891f87ae +e0c491d896ed21e4d6346dc7c9d16b007291205663be8f9624606925ee8f4f1f
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.