From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11DC4C2D0A3 for ; Thu, 29 Oct 2020 20:59:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B4DD920791 for ; Thu, 29 Oct 2020 20:59:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726320AbgJ2U7F convert rfc822-to-8bit (ORCPT ); Thu, 29 Oct 2020 16:59:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725862AbgJ2U7E (ORCPT ); Thu, 29 Oct 2020 16:59:04 -0400 X-Greylist: delayed 70663 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Thu, 29 Oct 2020 13:59:04 PDT Received: from mail.nic.cz (lists.nic.cz [IPv6:2001:1488:800:400::400]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97CAFC0613CF for ; Thu, 29 Oct 2020 13:59:04 -0700 (PDT) Received: from localhost (unknown [IPv6:2a0e:b107:ae1:0:3e97:eff:fe61:c680]) by mail.nic.cz (Postfix) with ESMTPSA id C059F140837; Thu, 29 Oct 2020 21:59:01 +0100 (CET) Date: Thu, 29 Oct 2020 21:58:53 +0100 From: Marek Behun To: Bjorn Helgaas Cc: Toke =?UTF-8?B?SMO4aWxhbmQtSsO4cmdlbnNlbg==?= , Pali =?UTF-8?B?Um9ow6Fy?= , vtolkm@gmail.com, linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Rob Herring , Ilias Apalodimas , Thomas Petazzoni , Jason Cooper Subject: Re: PCI trouble on mvebu (Turris Omnia) Message-ID: <20201029215853.6ccce4e0@nic.cz> In-Reply-To: <20201029193022.GA476048@bjorn-Precision-5520> References: <871rhhmgkq.fsf@toke.dk> <20201029193022.GA476048@bjorn-Precision-5520> X-Mailer: Claws Mail 3.17.6 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-Virus-Scanned: clamav-milter 0.102.2 at mail X-Virus-Status: Clean Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Thu, 29 Oct 2020 14:30:22 -0500 Bjorn Helgaas wrote: > On Thu, Oct 29, 2020 at 12:12:21PM +0100, Toke Høiland-Jørgensen wrote: > > Pali Rohár writes: > > > > I have been testing mainline kernel on Turris Omnia with two PCIe > > > default cards (WLE200 and WLE900) and it worked fine. But I do not know > > > if I had ASPM enabled or not. > > > > > > So it is working fine for you when CONFIG_PCIEASPM is disabled and whole > > > issue is only when CONFIG_PCIEASPM is enabled? > > > > Yup, exactly. And I'm also currently testing with the default WLE200/900 > > cards... I just tried sticking an MT76-based WiFi card into the third > > PCI slot, and that doesn't come up either when I enable PCIEASPM. > > Huh. So IIUC, the following cases all try to retrain the link and it > fails to come up again: > > - aardvark + WLE900VX (see commit 43fc679ced18) > - mvebu + WLE200 > - mvebu + WLE900 > - mvebu + MT76 Bjorn, IIRC Pali's patches fix the WLE900VX card for Aardvark (both in kernel and in U-Boot). IMO mvebu has similar issues. Both these drivers handle the PCIe reset signal incorrectly (or at least Aardvark did before Pali's work). mvebu is used on Turris Omnia, and our HW guys first solved the WLE900VX not working issue by using different capacitors for the SerDeses (this was 5 years ago). But after Pali's work on Aardvark I think this could also be solved for mvebu driver in software. BTW the WLE900VX card has problems on many systems, it won't work for example on Thinkpad X230. There is a bug on kernel bugzilla reported for this. My opinion is that many drivers do not respect the PCIe specification for reset and link training totally correctly (Pali was talking about this when he was looking at Aardvark) and that WLE900VX has a bug that in combination with those drivers causes the fail. If you look at the drivers, they are incompatible in how they handle the reset signal and link training. I am curious what Pali will tell us, he said that he will look into the mvebu driver. Marek From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 688A5C2D0A3 for ; Thu, 29 Oct 2020 20:59:43 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CBA3320790 for ; Thu, 29 Oct 2020 20:59:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="ArPk2vyP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CBA3320790 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nic.cz Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Q3b2IJBaW1It9aFYyiiADST9+1mRyX3teA0/C5DlVoY=; b=ArPk2vyPF3awT1y6NbaV8gAbB YOl20ONKIOQUiJQyln5f0N25vOnreZoYOcEWUEXhDPinLLC72QiPaO6TyzoHRC8q1E3IlFw6F8cNR 4RjudL/UbcuFyWWepJJp7XdSUYBrO6jyZ6mymhTo/IabLlmSvyFbQiCalB9XRgZLIbQNcy0Y92OMo vXsBVNPSvuaG1yMWQm9pOSFmb1mFfuQeBXyqc81o1IoZpq7lDV+uCTGUBox4JqBTEWKredRNsoidA MTHDuXZRAsqknKZNS4eKxtNSqT5uPXNlYIP/e+3OM7xV5UL+LMg2jbGkzs4QzIze4TI2Rsv2Y1yiP K/Y6vo2vQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kYF0o-0001QB-8Q; Thu, 29 Oct 2020 20:59:10 +0000 Received: from lists.nic.cz ([217.31.204.67] helo=mail.nic.cz) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kYF0l-0001PB-2K for linux-arm-kernel@lists.infradead.org; Thu, 29 Oct 2020 20:59:08 +0000 Received: from localhost (unknown [IPv6:2a0e:b107:ae1:0:3e97:eff:fe61:c680]) by mail.nic.cz (Postfix) with ESMTPSA id C059F140837; Thu, 29 Oct 2020 21:59:01 +0100 (CET) Date: Thu, 29 Oct 2020 21:58:53 +0100 From: Marek Behun To: Bjorn Helgaas Subject: Re: PCI trouble on mvebu (Turris Omnia) Message-ID: <20201029215853.6ccce4e0@nic.cz> In-Reply-To: <20201029193022.GA476048@bjorn-Precision-5520> References: <871rhhmgkq.fsf@toke.dk> <20201029193022.GA476048@bjorn-Precision-5520> X-Mailer: Claws Mail 3.17.6 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.102.2 at mail X-Virus-Status: Clean X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201029_165907_260401_4CA8F0A5 X-CRM114-Status: GOOD ( 18.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Rob Herring , Ilias Apalodimas , Jason Cooper , Pali =?UTF-8?B?Um9ow6Fy?= , Toke =?UTF-8?B?SMO4aWxhbmQtSsO4cmdlbnNlbg==?= , Thomas Petazzoni , linux-pci@vger.kernel.org, vtolkm@gmail.com, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gVGh1LCAyOSBPY3QgMjAyMCAxNDozMDoyMiAtMDUwMApCam9ybiBIZWxnYWFzIDxoZWxnYWFz QGtlcm5lbC5vcmc+IHdyb3RlOgoKPiBPbiBUaHUsIE9jdCAyOSwgMjAyMCBhdCAxMjoxMjoyMVBN ICswMTAwLCBUb2tlIEjDuGlsYW5kLUrDuHJnZW5zZW4gd3JvdGU6Cj4gPiBQYWxpIFJvaMOhciA8 cGFsaUBrZXJuZWwub3JnPiB3cml0ZXM6ICAKPiAKPiA+ID4gSSBoYXZlIGJlZW4gdGVzdGluZyBt YWlubGluZSBrZXJuZWwgb24gVHVycmlzIE9tbmlhIHdpdGggdHdvIFBDSWUKPiA+ID4gZGVmYXVs dCBjYXJkcyAoV0xFMjAwIGFuZCBXTEU5MDApIGFuZCBpdCB3b3JrZWQgZmluZS4gQnV0IEkgZG8g bm90IGtub3cKPiA+ID4gaWYgSSBoYWQgQVNQTSBlbmFibGVkIG9yIG5vdC4KPiA+ID4KPiA+ID4g U28gaXQgaXMgd29ya2luZyBmaW5lIGZvciB5b3Ugd2hlbiBDT05GSUdfUENJRUFTUE0gaXMgZGlz YWJsZWQgYW5kIHdob2xlCj4gPiA+IGlzc3VlIGlzIG9ubHkgd2hlbiBDT05GSUdfUENJRUFTUE0g aXMgZW5hYmxlZD8gIAo+ID4gCj4gPiBZdXAsIGV4YWN0bHkuIEFuZCBJJ20gYWxzbyBjdXJyZW50 bHkgdGVzdGluZyB3aXRoIHRoZSBkZWZhdWx0IFdMRTIwMC85MDAKPiA+IGNhcmRzLi4uIEkganVz dCB0cmllZCBzdGlja2luZyBhbiBNVDc2LWJhc2VkIFdpRmkgY2FyZCBpbnRvIHRoZSB0aGlyZAo+ ID4gUENJIHNsb3QsIGFuZCB0aGF0IGRvZXNuJ3QgY29tZSB1cCBlaXRoZXIgd2hlbiBJIGVuYWJs ZSBQQ0lFQVNQTS4gIAo+IAo+IEh1aC4gIFNvIElJVUMsIHRoZSBmb2xsb3dpbmcgY2FzZXMgYWxs IHRyeSB0byByZXRyYWluIHRoZSBsaW5rIGFuZCBpdAo+IGZhaWxzIHRvIGNvbWUgdXAgYWdhaW46 Cj4gCj4gICAtIGFhcmR2YXJrICsgV0xFOTAwVlggKHNlZSBjb21taXQgNDNmYzY3OWNlZDE4KQo+ ICAgLSBtdmVidSArIFdMRTIwMAo+ICAgLSBtdmVidSArIFdMRTkwMAo+ICAgLSBtdmVidSArIE1U NzYKCkJqb3JuLCBJSVJDIFBhbGkncyBwYXRjaGVzIGZpeCB0aGUgV0xFOTAwVlggY2FyZCBmb3Ig QWFyZHZhcmsgKGJvdGggaW4Ka2VybmVsIGFuZCBpbiBVLUJvb3QpLgpJTU8gbXZlYnUgaGFzIHNp bWlsYXIgaXNzdWVzLiBCb3RoIHRoZXNlIGRyaXZlcnMgaGFuZGxlIHRoZSBQQ0llIHJlc2V0CnNp Z25hbCBpbmNvcnJlY3RseSAob3IgYXQgbGVhc3QgQWFyZHZhcmsgZGlkIGJlZm9yZSBQYWxpJ3Mg d29yaykuCgptdmVidSBpcyB1c2VkIG9uIFR1cnJpcyBPbW5pYSwgYW5kIG91ciBIVyBndXlzIGZp cnN0IHNvbHZlZCB0aGUgV0xFOTAwVlgKbm90IHdvcmtpbmcgaXNzdWUgYnkgdXNpbmcgZGlmZmVy ZW50IGNhcGFjaXRvcnMgZm9yIHRoZSBTZXJEZXNlcyAodGhpcwp3YXMgNSB5ZWFycyBhZ28pLiBC dXQgYWZ0ZXIgUGFsaSdzIHdvcmsgb24gQWFyZHZhcmsgSSB0aGluayB0aGlzIGNvdWxkCmFsc28g YmUgc29sdmVkIGZvciBtdmVidSBkcml2ZXIgaW4gc29mdHdhcmUuCgpCVFcgdGhlIFdMRTkwMFZY IGNhcmQgaGFzIHByb2JsZW1zIG9uIG1hbnkgc3lzdGVtcywgaXQgd29uJ3Qgd29yayBmb3IKZXhh bXBsZSBvbiBUaGlua3BhZCBYMjMwLiBUaGVyZSBpcyBhIGJ1ZyBvbiBrZXJuZWwgYnVnemlsbGEg cmVwb3J0ZWQKZm9yIHRoaXMuCgpNeSBvcGluaW9uIGlzIHRoYXQgbWFueSBkcml2ZXJzIGRvIG5v dCByZXNwZWN0IHRoZSBQQ0llIHNwZWNpZmljYXRpb24KZm9yIHJlc2V0IGFuZCBsaW5rIHRyYWlu aW5nIHRvdGFsbHkgY29ycmVjdGx5IChQYWxpIHdhcyB0YWxraW5nIGFib3V0CnRoaXMgd2hlbiBo ZSB3YXMgbG9va2luZyBhdCBBYXJkdmFyaykgYW5kIHRoYXQgV0xFOTAwVlggaGFzIGEgYnVnIHRo YXQKaW4gY29tYmluYXRpb24gd2l0aCB0aG9zZSBkcml2ZXJzIGNhdXNlcyB0aGUgZmFpbC4gSWYg eW91IGxvb2sgYXQgdGhlCmRyaXZlcnMsIHRoZXkgYXJlIGluY29tcGF0aWJsZSBpbiBob3cgdGhl eSBoYW5kbGUgdGhlIHJlc2V0IHNpZ25hbCBhbmQKbGluayB0cmFpbmluZy4KCkkgYW0gY3VyaW91 cyB3aGF0IFBhbGkgd2lsbCB0ZWxsIHVzLCBoZSBzYWlkIHRoYXQgaGUgd2lsbCBsb29rIGludG8g dGhlCm12ZWJ1IGRyaXZlci4KCk1hcmVrCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51eC1hcm0t a2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFp bG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==