From mboxrd@z Thu Jan 1 00:00:00 1970 From: Wan Zongshun Subject: Re: [PATCH] iommu/amd: Fix unity mapping initialization race Date: Sun, 10 Jul 2016 19:40:53 +0800 Message-ID: <57823445.1020808@iommu.org> References: <1467820838-5059-1-git-send-email-joro@8bytes.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <1467820838-5059-1-git-send-email-joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Joerg Roedel , iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Cc: Joerg Roedel , linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, stable-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-Id: iommu@lists.linux-foundation.org CgpPbiAyMDE25bm0MDfmnIgwN+aXpSAwMDowMCwgSm9lcmcgUm9lZGVsIHdyb3RlOgo+IEZyb206 IEpvZXJnIFJvZWRlbCA8anJvZWRlbEBzdXNlLmRlPgo+Cj4gVGhlcmUgaXMgYSByYWNlIGNvbmRp dGlvbiBpbiB0aGUgQU1EIElPTU1VIGluaXQgY29kZSB0aGF0Cj4gY2F1c2VzIHJlcXVlc3RlZCB1 bml0eSBtYXBwaW5ncyB0byBiZSBibG9ja2VkIGJ5IHRoZSBJT01NVQo+IGZvciBhIHNob3J0IHBl cmlvZCBvZiB0aW1lLiBUaGlzIHJlc3VsdHMgb24gYm9vdCBmYWlsdXJlcwo+IGFuZCBJT19QQUdF X0ZBVUxUcyBvbiBzb21lIG1hY2hpbmVzLgo+Cj4gRml4IHRoaXMgYnkgbWFraW5nIHN1cmUgdGhl IHVuaXR5IG1hcHBpbmdzIGFyZSBpbnN0YWxsZWQKPiBiZWZvcmUgYWxsIG90aGVyIERNQSBpcyBi bG9ja2VkLgo+Cj4gRml4ZXM6IGFhZmQ4YmEwY2E3NCAoJ2lvbW11L2FtZDogSW1wbGVtZW50IGFk ZF9kZXZpY2UgYW5kIHJlbW92ZV9kZXZpY2UnKQo+IENjOiBzdGFibGVAdmdlci5rZXJuZWwub3Jn ICMgdjQuMisKPiBTaWduZWQtb2ZmLWJ5OiBKb2VyZyBSb2VkZWwgPGpyb2VkZWxAc3VzZS5kZT4K PiAtLS0KPiAgIGRyaXZlcnMvaW9tbXUvYW1kX2lvbW11X2luaXQuYyB8IDE0ICsrKysrKysrKysr Ky0tCj4gICAxIGZpbGUgY2hhbmdlZCwgMTIgaW5zZXJ0aW9ucygrKSwgMiBkZWxldGlvbnMoLSkK Pgo+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2lvbW11L2FtZF9pb21tdV9pbml0LmMgYi9kcml2ZXJz L2lvbW11L2FtZF9pb21tdV9pbml0LmMKPiBpbmRleCBkMDkxZGVmLi41OTc0MWVhIDEwMDY0NAo+ IC0tLSBhL2RyaXZlcnMvaW9tbXUvYW1kX2lvbW11X2luaXQuYwo+ICsrKyBiL2RyaXZlcnMvaW9t bXUvYW1kX2lvbW11X2luaXQuYwo+IEBAIC0xNTY4LDEzICsxNTY4LDIzIEBAIHN0YXRpYyBpbnQg X19pbml0IGFtZF9pb21tdV9pbml0X3BjaSh2b2lkKQo+ICAgCQkJYnJlYWs7Cj4gICAJfQo+Cj4g KwkvKgo+ICsJICogT3JkZXIgaXMgaW1wb3J0YW50IGhlcmUgdG8gbWFrZSBzdXJlIGFueSB1bml0 eSBtYXAgcmVxdWlyZW1lbnRzIGFyZQo+ICsJICogZnVsZmlsbGVkLiBUaGUgdW5pdHkgbWFwcGlu Z3MgYXJlIGNyZWF0ZWQgYW5kIHdyaXR0ZW4gdG8gdGhlIGRldmljZQo+ICsJICogdGFibGUgZHVy aW5nIHRoZSBhbWRfaW9tbXVfaW5pdF9hcGkoKSBjYWxsLgo+ICsJICoKPiArCSAqIEFmdGVyIHRo YXQgd2UgY2FsbCBpbml0X2RldmljZV90YWJsZV9kbWEoKSB0byBtYWtlIHN1cmUgYW55Cj4gKwkg KiB1bmluaXRpYWxpemVkIERURSB3aWxsIGJsb2NrIERNQSwgYW5kIGluIHRoZSBlbmQgd2UgZmx1 c2ggdGhlIGNhY2hlcwo+ICsJICogb2YgYWxsIElPTU1VcyB0byBtYWtlIHN1cmUgdGhlIGNoYW5n ZXMgdG8gdGhlIGRldmljZSB0YWJsZSBhcmUKPiArCSAqIGFjdGl2ZS4KPiArCSAqLwoKSm9lcmcs CgpEbyB5b3UgbWVhbiB3ZSBuZWVkIGVuYWJsZSB0aGUgViBhbmQgVFYgYml0cyB0byBEVEUgZW50 cnkgYWZ0ZXIgYWxsIERURXMgCnRhYmxlcyB3ZXJlIGluaXRpYWxpemVkIGNvbXBsZXRlbHk/CgpJ IGNoZWNrZWQgdGhpcyBmdW5jdGlvbiAnaW5pdF9kZXZpY2VfdGFibGVfZG1hJywgYW5kIGZpbmQg aXQganVzdCBzZXQKViBhbmQgVFYgYml0LCB0byBzZXQgdHJhbnNsYXRpb24gaW5mbyB2YWxpZCBh bmQgRFRFIGJpdHMxMjc6MSB2YWxpZC4KClNvIEkganVzdCB0aGluayBhbGwgdGhpbmdzIGl0IHNo b3VsZCB0byBkbyBhcmUgdG8gYWxsb3cgRE1BIGFjY2VzcywKR1BBLXRvLVNQQSB0cmFuc2xhdGlv biBzaG91bGQgYmUgYWN0aXZlLCB3aHkgeW91IGFkZCBmdW5jdGlvbiBjb21tZW50cyAKYmVsb3cg aXMgdG8gbm90IGFsbG93IERNQSBhY2Nlc3MgYW5kIHN1cHByZXNzIGFsbCBwYWdlIGZhdWx0cz8K Ci8qCiAgKiBJbml0IHRoZSBkZXZpY2UgdGFibGUgdG8gbm90IGFsbG93IERNQSBhY2Nlc3MgZm9y IGRldmljZXMgYW5kCiAgKiBzdXBwcmVzcyBhbGwgcGFnZSBmYXVsdHMKICAqLwpzdGF0aWMgdm9p ZCBpbml0X2RldmljZV90YWJsZV9kbWEodm9pZCkKCj4gKwlyZXQgPSBhbWRfaW9tbXVfaW5pdF9h cGkoKTsKPiArCj4gICAJaW5pdF9kZXZpY2VfdGFibGVfZG1hKCk7Cj4KPiAgIAlmb3JfZWFjaF9p b21tdShpb21tdSkKPiAgIAkJaW9tbXVfZmx1c2hfYWxsX2NhY2hlcyhpb21tdSk7Cj4KPiAtCXJl dCA9IGFtZF9pb21tdV9pbml0X2FwaSgpOwo+IC0KPiAgIAlpZiAoIXJldCkKPiAgIAkJcHJpbnRf aW9tbXVfaW5mbygpOwo+Cj4KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18KaW9tbXUgbWFpbGluZyBsaXN0CmlvbW11QGxpc3RzLmxpbnV4LWZvdW5kYXRpb24u b3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2lv bW11 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757120AbcGJLlA (ORCPT ); Sun, 10 Jul 2016 07:41:00 -0400 Received: from mr213139.mail.yeah.net ([223.252.213.139]:5289 "EHLO mr213139.mail.yeah.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750834AbcGJLk6 (ORCPT ); Sun, 10 Jul 2016 07:40:58 -0400 Subject: Re: [PATCH] iommu/amd: Fix unity mapping initialization race To: Joerg Roedel , iommu@lists.linux-foundation.org References: <1467820838-5059-1-git-send-email-joro@8bytes.org> Cc: Joerg Roedel , linux-kernel@vger.kernel.org, stable@vger.kernel.org From: Wan Zongshun Message-ID: <57823445.1020808@iommu.org> Date: Sun, 10 Jul 2016 19:40:53 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 MIME-Version: 1.0 In-Reply-To: <1467820838-5059-1-git-send-email-joro@8bytes.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-HM-Spam-Status: e1koWUFPN1dZCBgUCR5ZQUNVTU1NQkJCQ0pNQ0JPTkhKV1kJDhceCFlBWS grPSQ1NDU#JDI1JDM#Oj8#KUFLVUtANiMkIj4oJDI1JDM#Oj8#KUFLVUtAKy8pJCI#KCQyNSQzPj o*PilBS1VLQDg0LjUvKSIkODVBS1VLQCk#PDI0NSQ6KDI6QUtVS0ArKTQtMjU4PiQoMzo1PDM6Mk FLVUtAPyI1OjYyOCQyKyQ1NCQyNSQzPjo*PilBS1VLQCk#OjckMiskMjUkKTk3JDI1JDM#Oj8#KU FKSVVLQDYuNy8yJCk4Ky8kPzI9PT4pPjUvJDI1JDM#Oj8#KUFJVUtAMiskSEskNjI1Li8#JDg1Ly RLJE5LQUtVS0AyKyRKJDYyNS4vPiQ4NS8kSyRKS0FLVUtAMiskLzQ*OiIkODUvJEskSktLQUtVS0 AyKyROJDYyNS4vPiQ4NS8kSyRKS0FLVUtAMiskSiQzNC4pJDg1LyRLJEpLS0FLVUtAKC45MT44Ly QvND86IiQ4NS8kSyRKS0tBS1VLQCguOTE#OC8kTiQ2MjUuLz4kODUvJEskSktBS1VLQCguOTE#OC 8kSiQzNC4pJDg1LyRLJEpLS0FLVUtAKDkxJDc#NSRPSyQjQUtVS0tKS0AvPiMvJDc#NSRKSUNLJC NBS1VLS0pLQDg1LyQzLzY3JDc#NSRKMCROMEFLVUtLSktAKT44PjItPikkODUvJEokTkFLVUtLSk tAIzY6Mjc#KSQ1Ljc3QUtVS0tKS0A*PjcyLT4pJDc#NSRLQUtVS0tKS0A3OjU8JD41QUtVS0tKS0 A9NSQ2OiIkT0pCJDM3MSRKJEtDS0hLT0FLVUhIQD0rJCk#JD0sJDM3MSRLQ0tIS01BVkxVTkA9NS QoLjkkPjUsNCk*KCQzNzEkSktLSUtKQUtVSUNAKC45JD5BSlVOTkA9NSQ2OiIkT0pCJDM3MSRJJE tDS0hLT0FLVUtZBg++ X-HM-Sender-Digest: e1kSHx4VD1lBWUc6Mjo6Pio5PDohMjNIQ08RGTEzURoKCzZVSlVKT01D Sk5LQ05IT0xMVTMWGhIXVQ0MOxIUFhYOVRQJHEVZV1kMHhlZQR0aFwgeV1kIAVlBSE5LTzdXWRIL WUFZSUlJVU1OVU1PVUNZBg++ X-HM-Tid: 0a55d49c2f297d8b5c0651c13b4 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2016年07月07日 00:00, Joerg Roedel wrote: > From: Joerg Roedel > > There is a race condition in the AMD IOMMU init code that > causes requested unity mappings to be blocked by the IOMMU > for a short period of time. This results on boot failures > and IO_PAGE_FAULTs on some machines. > > Fix this by making sure the unity mappings are installed > before all other DMA is blocked. > > Fixes: aafd8ba0ca74 ('iommu/amd: Implement add_device and remove_device') > Cc: stable@vger.kernel.org # v4.2+ > Signed-off-by: Joerg Roedel > --- > drivers/iommu/amd_iommu_init.c | 14 ++++++++++++-- > 1 file changed, 12 insertions(+), 2 deletions(-) > > diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c > index d091def..59741ea 100644 > --- a/drivers/iommu/amd_iommu_init.c > +++ b/drivers/iommu/amd_iommu_init.c > @@ -1568,13 +1568,23 @@ static int __init amd_iommu_init_pci(void) > break; > } > > + /* > + * Order is important here to make sure any unity map requirements are > + * fulfilled. The unity mappings are created and written to the device > + * table during the amd_iommu_init_api() call. > + * > + * After that we call init_device_table_dma() to make sure any > + * uninitialized DTE will block DMA, and in the end we flush the caches > + * of all IOMMUs to make sure the changes to the device table are > + * active. > + */ Joerg, Do you mean we need enable the V and TV bits to DTE entry after all DTEs tables were initialized completely? I checked this function 'init_device_table_dma', and find it just set V and TV bit, to set translation info valid and DTE bits127:1 valid. So I just think all things it should to do are to allow DMA access, GPA-to-SPA translation should be active, why you add function comments below is to not allow DMA access and suppress all page faults? /* * Init the device table to not allow DMA access for devices and * suppress all page faults */ static void init_device_table_dma(void) > + ret = amd_iommu_init_api(); > + > init_device_table_dma(); > > for_each_iommu(iommu) > iommu_flush_all_caches(iommu); > > - ret = amd_iommu_init_api(); > - > if (!ret) > print_iommu_info(); > >