From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andreas Pflug Subject: Re: kernel BUG at nvme/host/pci.c Date: Sat, 15 Jul 2017 15:34:24 +0200 Message-ID: <2fdb1d68-254b-cdc2-c60c-febd1f7785e0@pse-consulting.de> References: <28fcb21a-35b6-61c1-29e0-9adcc954c98c@pse-consulting.de> <20170710190818.GA13671@localhost.localdomain> <64da8c4d-2d69-22fa-06a0-aa4f11903e39@pse-consulting.de> <20170711194523.GD4604@localhost.localdomain> <458f2115-4a07-e23c-2f1f-a90a95668f74@pse-consulting.de> <20170712195012.GA8388@localhost.localdomain> <5ac79312-b9e3-7cbb-8a82-382fb99705b5@pse-consulting.de> <20170713134754.GD8388@localhost.localdomain> <39459fb9-ea2c-0718-10a1-a7e96b9ec21f@pse-consulting.de> <20170714170846.GA24365@localhost.localdomain> <20170715085141.GC25753@infradead.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------6CE11F35966F15FD32C2A157" Return-path: Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dWNDR-0002Iq-Oi for xen-devel@lists.xenproject.org; Sat, 15 Jul 2017 13:34:37 +0000 In-Reply-To: <20170715085141.GC25753@infradead.org> Content-Language: de-DE List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Christoph Hellwig , Keith Busch Cc: xen-devel@lists.xenproject.org, linux-nvme@lists.infradead.org List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --------------6CE11F35966F15FD32C2A157 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Am 15.07.17 um 10:51 schrieb Christoph Hellwig: > On Fri, Jul 14, 2017 at 01:08:47PM -0400, Keith Busch wrote: >>> So LVM2 backed by md raid1 isn't compatible with newer hardware... Any >>> suggestions? >> It's not that LVM2 or RAID isn't compatible. Either the IOMMU isn't >> compatible if can use different page offsets for DMA addresses than the >> physical aaddresses, or the driver for it is broken. The DMA addresses >> in this mapped SGL look completely broken, at least, since the last 4 >> entries are all the same address. That'll corrupt data. > Given that this is a Xen system I wonder if swiotlb-xen is involved > here, which does some odd chunking of dma translations? I did some more testing now. With data stored on SATA disks with md1 and lvm2 (i.e. just replacing NVME by SATA), there's nothing happening. With data stored on /dev/nvme1n1p1, i.e. without any device mapping stuff, I get the same problem. Log attached. Regards, Andreas --------------6CE11F35966F15FD32C2A157 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="x.log" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="x.log" SnVsIDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE0OTIxNV0gSW52YWxpZCBTR0wgZm9yIHBh eWxvYWQ6MjA5OTIgbmVudHM6NQpKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQzNzYuMTUwMzgy XSAtLS0tLS0tLS0tLS1bIGN1dCBoZXJlIF0tLS0tLS0tLS0tLS0KSnVsIDE1IDE1OjI1OjA2 IHhlbjIgWyA0Mzc2LjE1MTI2MV0gV0FSTklORzogQ1BVOiAwIFBJRDogMjkwOTUgYXQgZHJp dmVycy9udm1lL2hvc3QvcGNpLmM6NjIzIG52bWVfcXVldWVfcnErMHg4MWIvMHg4NDAgW252 bWVdCkp1bCAxNSAxNToyNTowNiB4ZW4yIFsgNDM3Ni4xNTIxOTRdIE1vZHVsZXMgbGlua2Vk IGluOiB4dF9waHlzZGV2IGJyX25ldGZpbHRlciBpcHRhYmxlX2ZpbHRlciB4ZW5fbmV0YmFj ayB4ZW5fYmxrYmFjayBuZXRjb25zb2xlIGNvbmZpZ2ZzIGJyaWRnZSB4ZW5fZ250ZGV2IHhl bl9ldnRjaG4geGVuZnMgeGVuX3ByaXZjbWQgaVRDT193ZHQgaW50ZWxfcmFwbCBpVENPX3Zl bmRvcl9zdXBwb3J0IG14bV93bWkgeDg2X3BrZ190ZW1wX3RoZXJtYWwgaW50ZWxfcG93ZXJj bGFtcCBjb3JldGVtcCBjcmN0MTBkaWZfcGNsbXVsIGNyYzMyX3BjbG11bCBnaGFzaF9jbG11 bG5pX2ludGVsIHBjYmMgYWVzbmlfaW50ZWwgYWVzX3g4Nl82NCBjcnlwdG9fc2ltZCBnbHVl X2hlbHBlciBjcnlwdGQgaW50ZWxfcmFwbF9wZXJmIHNuZF9wY20gc25kX3RpbWVyIHNuZCBz b3VuZGNvcmUgcGNzcGtyIGkyY19pODAxIGpveWRldiBhc3QgdHRtIGRybV9rbXNfaGVscGVy IGRybSBzZyBpMmNfYWxnb19iaXQgbHBjX2ljaCBlaGNpX3BjaSBtZmRfY29yZSBlaGNpX2hj ZCBtZWlfbWUgbWVpIGUxMDAwZSBpeGdiZSBwdHAgbnZtZSBwcHNfY29yZSBtZGlvIG52bWVf Y29yZSBpb2F0ZG1hIHNocGNocCBkY2Egd21pIGFjcGlfcG93ZXJfbWV0ZXIgODAyMXEgZ2Fy cCBtcnAgc3RwIGxsYyBidXR0b24gaXBtaV9zaSBpcG1pX2RldmludGYgaXBtaV9tc2doYW5k bGVyIHN1bnJwYyBkcmJkIGxydV9jYWNoZSBpcF90YWJsZXMgeF90YWJsZXMgYXV0b2ZzNCBl eHQ0IGNyYzE2IG1iY2FjaGUgamJkMiBmc2NyeXB0byByYWlkMTAgcmFpZDQ1NiBsaWJjcmMz MmMgY3JjMzJjX2dlbmVyaWMgYXN5bmNfcmFpZDZfcmVjb3YKSnVsIDE1IDE1OjI1OjA2IHhl bjIgWyA0Mzc2LjE1ODU4Ml0gIGFzeW5jX21lbWNweSBhc3luY19wcSBhc3luY194b3IgeG9y IGFzeW5jX3R4IHJhaWQ2X3BxIHJhaWQwIG11bHRpcGF0aCBsaW5lYXIgZXZkZXYgaGlkX2dl bmVyaWMgdXNiaGlkIGhpZCBiY2FjaGUgZG1fbW9kIHJhaWQxIG1kX21vZCBzZF9tb2QgY3Jj MzJjX2ludGVsIGFoY2kgbGliYWhjaSB4aGNpX3BjaSB4aGNpX2hjZCBsaWJhdGEgdXNiY29y ZSBzY3NpX21vZApKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQzNzYuMTYwNTkzXSBDUFU6IDAg UElEOiAyOTA5NSBDb21tOiA4LmhkYS0wIFRhaW50ZWQ6IEcgICAgICBEIFcgICAgICAgNC4x Mi4wLTIwMTcwNzEzKyAjMQpKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQzNzYuMTYxNjc4XSBI YXJkd2FyZSBuYW1lOiBTdXBlcm1pY3JvIFgxMERSaS9YMTBEUkktVCwgQklPUyAyLjEgMDkv MTMvMjAxNgpKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQzNzYuMTYyNjQ5XSB0YXNrOiBmZmZm ODgwMTVmZGM1MDAwIHRhc2suc3RhY2s6IGZmZmZjOTAwNDgxMzQwMDAKSnVsIDE1IDE1OjI1 OjA2IHhlbjIgWyA0Mzc2LjE2MzY3Nl0gUklQOiBlMDMwOm52bWVfcXVldWVfcnErMHg4MWIv MHg4NDAgW252bWVdCkp1bCAxNSAxNToyNTowNiB4ZW4yIFsgNDM3Ni4xNjQ4MDRdIFJTUDog ZTAyYjpmZmZmYzkwMDQ4MTM3YTAwIEVGTEFHUzogMDAwMTAyODYKSnVsIDE1IDE1OjI1OjA2 IHhlbjIgWyA0Mzc2LjE2NTg5MF0gUkFYOiAwMDAwMDAwMDAwMDAwMDI1IFJCWDogMDAwMDAw MDBmZmZmZjIwMCBSQ1g6IDAwMDAwMDAwMDAwMDAwMDAKSnVsIDE1IDE1OjI1OjA2IHhlbjIg WyA0Mzc2LjE2Njk4Ml0gUkRYOiAwMDAwMDAwMDAwMDAwMDAwIFJTSTogZmZmZjg4MDE4NmEw ZGU5OCBSREk6IGZmZmY4ODAxODZhMGRlOTgKSnVsIDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2 LjE2ODA5OV0gUkJQOiBmZmZmODgwMTczMmZmMDAwIFIwODogMDAwMDAwMDAwMDAwMDAwMSBS MDk6IDAwMDAwMDAwMDAwMDBhNTcKSnVsIDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE2OTA4 MV0gUjEwOiAwMDAwMDAwMDAwMDAxMDAwIFIxMTogMDAwMDAwMDAwMDAwMDAwMSBSMTI6IDAw MDAwMDAwMDAwMDAyMDAKSnVsIDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE3MDE5OF0gUjEz OiAwMDAwMDAwMDAwMDAxMDAwIFIxNDogZmZmZjg4MDE1ZjlkNzgwMCBSMTU6IGZmZmY4ODAx NmZjZTE4MDAKSnVsIDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE3MTMzMF0gRlM6ICAwMDAw MDAwMDAwMDAwMDAwKDAwMDApIEdTOmZmZmY4ODAxODZhMDAwMDAoMDAwMCkga25sR1M6ZmZm Zjg4MDE4NmEwMDAwMApKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQzNzYuMTcyNDc0XSBDUzog IGUwMzMgRFM6IDAwMDAgRVM6IDAwMDAgQ1IwOiAwMDAwMDAwMDgwMDUwMDMzCkp1bCAxNSAx NToyNTowNiB4ZW4yIFsgNDM3Ni4xNzM2MDBdIENSMjogMDAwMDAwYjBmOThkMTk3MCBDUjM6 IDAwMDAwMDAxNzVkNGYwMDAgQ1I0OiAwMDAwMDAwMDAwMDQyNjYwCkp1bCAxNSAxNToyNTow NiB4ZW4yIFsgNDM3Ni4xNzQ2NDNdIENhbGwgVHJhY2U6Ckp1bCAxNSAxNToyNTowNiB4ZW4y IFsgNDM3Ni4xNzU3NDNdICA/IF9fc2JpdG1hcF9nZXRfd29yZCsweDJhLzB4ODAKSnVsIDE1 IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE3NjgxNF0gID8gYmxrX21xX2Rpc3BhdGNoX3JxX2xp c3QrMHgyMDAvMHgzZDAKSnVsIDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE3NzkzMl0gID8g YmxrX21xX2ZsdXNoX2J1c3lfY3R4cysweGQxLzB4MTIwCkp1bCAxNSAxNToyNTowNiB4ZW4y IFsgNDM3Ni4xNzg5NjFdICA/IGJsa19tcV9zY2hlZF9kaXNwYXRjaF9yZXF1ZXN0cysweDFj MC8weDFmMApKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQzNzYuMTc5OTQyXSAgPyBfX2Jsa19t cV9kZWxheV9ydW5faHdfcXVldWUrMHg4Zi8weGEwCkp1bCAxNSAxNToyNTowNiB4ZW4yIFsg NDM3Ni4xODA5NDFdICA/IGJsa19tcV9mbHVzaF9wbHVnX2xpc3QrMHgxODQvMHgyNjAKSnVs IDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE4MTkzNV0gID8gYmxrX2ZsdXNoX3BsdWdfbGlz dCsweGYyLzB4MjgwCkp1bCAxNSAxNToyNTowNiB4ZW4yIFsgNDM3Ni4xODI5NTJdICA/IGJs a19maW5pc2hfcGx1ZysweDI3LzB4NDAKSnVsIDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE4 Mzk4NV0gID8gZGlzcGF0Y2hfcndfYmxvY2tfaW8rMHg3MzIvMHg5YzAgW3hlbl9ibGtiYWNr XQpKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQzNzYuMTg1MDU5XSAgPyBfcmF3X3NwaW5fbG9j a19pcnFzYXZlKzB4MTcvMHgzOQpKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQzNzYuMTg2MTAz XSAgPyBfX2RvX2Jsb2NrX2lvX29wKzB4MzYyLzB4NjkwIFt4ZW5fYmxrYmFja10KSnVsIDE1 IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE4NzE2N10gID8gX3Jhd19zcGluX3VubG9ja19pcnFy ZXN0b3JlKzB4MTYvMHgyMApKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQzNzYuMTg4MjE2XSAg PyBfX2RvX2Jsb2NrX2lvX29wKzB4MzYyLzB4NjkwIFt4ZW5fYmxrYmFja10KSnVsIDE1IDE1 OjI1OjA2IHhlbjIgWyA0Mzc2LjE4OTI5NF0gID8geGVuX2Jsa2lmX3NjaGVkdWxlKzB4MTE2 LzB4N2YwIFt4ZW5fYmxrYmFja10KSnVsIDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE5MDI0 N10gID8gX19zY2hlZHVsZSsweDNjZC8weDg1MApKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQz NzYuMTkxMTUyXSAgPyByZW1vdmVfd2FpdF9xdWV1ZSsweDYwLzB4NjAKSnVsIDE1IDE1OjI1 OjA2IHhlbjIgWyA0Mzc2LjE5MjExMl0gID8ga3RocmVhZCsweGZjLzB4MTMwCkp1bCAxNSAx NToyNTowNiB4ZW4yIFsgNDM3Ni4xOTMxNjldICA/IHhlbl9ibGtpZl9iZV9pbnQrMHgzMC8w eDMwIFt4ZW5fYmxrYmFja10KSnVsIDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE5NDEwNV0g ID8ga3RocmVhZF9jcmVhdGVfb25fbm9kZSsweDcwLzB4NzAKSnVsIDE1IDE1OjI1OjA2IHhl bjIgWyA0Mzc2LjE5NTA1OV0gID8gZG9fZ3JvdXBfZXhpdCsweDNhLzB4YTAKSnVsIDE1IDE1 OjI1OjA2IHhlbjIgWyA0Mzc2LjE5NjA0OV0gID8gcmV0X2Zyb21fZm9yaysweDI1LzB4MzAK SnVsIDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE5NzA1MF0gQ29kZTogZjkgZmYgZmYgNDEg ZjYgNDcgNGEgMDQgYzYgMDUgN2EgM2UgMDAgMDAgMDEgNDEgOGIgOTcgNzAgMDEgMDAgMDAg NzQgMjggNDEgOGIgYjcgOTAgMDAgMDAgMDAgNDggYzcgYzcgYjggODcgNDggYzAgZTggNDAg YTQgYzQgYzAgPDBmPiBmZiBlOSA0ZCBmZSBmZiBmZiAwZiAwYiA0YyA4YiAyZCBjNSA5NSA3 OSBjMSBlOSA1MyBmZiBmZiBmZiAKSnVsIDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjE5ODk0 N10gLS0tWyBlbmQgdHJhY2UgNmQ3ZDM5NWEyOWM5MzFiNSBdLS0tCkp1bCAxNSAxNToyNTow NiB4ZW4yIFsgNDM3Ni4yMDAwMTJdIHNnWzBdIHBoeXNfYWRkcjoweDAwMDAwMDBhZmY1NDll MDAgb2Zmc2V0OjM1ODQgbGVuZ3RoOjQ2MDggZG1hX2FkZHJlc3M6MHgwMDAwMDAwMDAwNGEz MDAwIGRtYV9sZW5ndGg6NDYwOApKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQzNzYuMjAwOTUx XSBzZ1sxXSBwaHlzX2FkZHI6MHgwMDAwMDAwYWZmNWMzMDAwIG9mZnNldDowIGxlbmd0aDo0 MDk2IGRtYV9hZGRyZXNzOjB4MDAwMDAwMDlmNGE4MDAwMCBkbWFfbGVuZ3RoOjQwOTYKSnVs IDE1IDE1OjI1OjA2IHhlbjIgWyA0Mzc2LjIwMjAxNV0gc2dbMl0gcGh5c19hZGRyOjB4MDAw MDAwMGFmZjYxNTAwMCBvZmZzZXQ6MCBsZW5ndGg6NDA5NiBkbWFfYWRkcmVzczoweDAwMDAw MDA5ZjRhODAwMDAgZG1hX2xlbmd0aDo0MDk2Ckp1bCAxNSAxNToyNTowNiB4ZW4yIFsgNDM3 Ni4yMDMwMDZdIHNnWzNdIHBoeXNfYWRkcjoweDAwMDAwMDBhZmY2MDgwMDAgb2Zmc2V0OjAg bGVuZ3RoOjQwOTYgZG1hX2FkZHJlc3M6MHgwMDAwMDAwOWY0YTgwMDAwIGRtYV9sZW5ndGg6 NDA5NgpKdWwgMTUgMTU6MjU6MDYgeGVuMiBbIDQzNzYuMjAzODg5XSBzZ1s0XSBwaHlzX2Fk ZHI6MHgwMDAwMDAwYWZmNTBlMDAwIG9mZnNldDowIGxlbmd0aDo0MDk2IGRtYV9hZGRyZXNz OjB4MDAwMDAwMDlmNWE0ZTAwMCBkbWFfbGVuZ3RoOjQwOTYKSnVsIDE1IDE1OjI1OjA2IHhl bjIgWyA0Mzc2LjIwNDcyMl0gcHJpbnRfcmVxX2Vycm9yOiBJL08gZXJyb3IsIGRldiBudm1l MW4xLCBzZWN0b3IgMTQzMTg5NTEK --------------6CE11F35966F15FD32C2A157 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --------------6CE11F35966F15FD32C2A157--