From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pandora.armlinux.org.uk (pandora.armlinux.org.uk [78.32.30.218]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 488651AA8B for ; Fri, 4 Aug 2023 22:27:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2019; h=Sender:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=oe3R1YKPMSS61Kql1VVF1wpgWu78Kb9TP4tIQw5QCgw=; b=SomGhhLqgNBWYw3yU37FFm8NyK s731vyDc2/mwtHBDHIGN42h/VWqKhR7ykKos0RTCgVNWEeRLTW6cjMRygXWyWOw0tRkaJNoU8ihMt joVIvLvs/SLsrlOSbDgQwuDcUf2tUgAXvZ06MdHxlSEi4slkyS8RXcw3xQXQpGACzxue2JTNuaUKE 8UjyA+Ru1k97X35qsFFUdXPQi408YX9k/nEMxwhC5z9COVi+ma3TUCk9he6zHNWSdYUXob5U+7FAk +gTTRgTTtyhAfUAnGBToBl5TlJnE/mL9gLGS99weNkN5FiRMeCGbXfp9pnNeM6Behz7BUwJueWHfK JWUbifAw==; Received: from shell.armlinux.org.uk ([fd8f:7570:feb6:1:5054:ff:fe00:4ec]:47482) by pandora.armlinux.org.uk with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1qS3Gf-0000pF-0M; Fri, 04 Aug 2023 23:27:33 +0100 Received: from linux by shell.armlinux.org.uk with local (Exim 4.94.2) (envelope-from ) id 1qS3Ge-0004MT-Q1; Fri, 04 Aug 2023 23:27:32 +0100 Date: Fri, 4 Aug 2023 23:27:32 +0100 From: "Russell King (Oracle)" To: Nick Bowler Cc: Rob Herring , Saravana Kannan , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, netdev@vger.kernel.org, regressions@lists.linux.dev Subject: Re: PROBLEM: Broken or delayed ethernet on Xilinx ZCU104 since 5.18 (regression) Message-ID: References: Precedence: bulk X-Mailing-List: regressions@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Sender: Russell King (Oracle) On Fri, Aug 04, 2023 at 05:31:21PM -0400, Nick Bowler wrote: > On 2023-08-04, Rob Herring wrote: > > On Fri, Aug 4, 2023 at 11:52 AM Nick Bowler wrote: > >> I don't know about the deferred probe timeout, but I bisected the 6.5-rc4 > >> breakage to this commit: > >> > >> commit c720a1f5e6ee8cb39c28435efc0819cec84d6ee2 > >> Author: Michal Simek > >> Date: Mon May 22 16:59:48 2023 +0200 > >> > >> arm64: zynqmp: Describe TI phy as ethernet-phy-id > > > > I don't see anything obviously problematic with that commit. (The > > #phy-cells property added is wrong as ethernet phys don't use the phy > > binding, but that should just be ignored). I'd check if the phy probed > > and has a DT node associated with it. > > I think the answer is "no, the phy was not probed". Without reverting > that commit, there is absolutely nothing in /sys/bus/mdio_bus/devices. > There is no phy device link under /sys/bus/mdio_bus/drivers/"TI DP83867", > and there is no mdio_bus under /sys/bus/platform/devices/ff0e0000.ethernet. > > When I revert that commit, I can locate the phy device under all these > locations. > > > fw_devlink tracks parent-child dependencies and maybe changing to > > parent-grandchild affected that. We don't yet track 'phy-handle' > > dependencies, but we'd have a circular one here if we did (though that > > should be handled). Does "fw_devlink=off" help? > > Booting with fw_devlink=off results in no obvious change in behaviour. I think we need to rewind a tad. My understanding is that this uses the Cadence macb driver. In your original message, you said that the ethernet driver wasn't being bound to the driver. Since the ethernet driver is responsible for spotting the "mdio" sub-node and creating the MDIO bus, if the driver isn't being successfully bound, then the MDIO bus and the PHYs on the bus won't be created, so you won't find them in /sys/bus/mdio_bus/devices. Moreover, the Cadence macb driver, and this doesn't care about the presence of the PHY at probe time, only when the network interface is brought up. See macb_phylink_connect() which is called from macb_open(). So, I think that the deferred probing has nothing to do with PHYs, and that's just a wild goose chase. I think instead we need to be concentrating on what's going on with the ethernet driver, and why the ethernet driver is deferring its probe. Is macb_probe() getting called at all? How far through macb_probe() do we get before we defer? I think those are the key questions that need answering. Maybe, if you can get access to the machine while the driver is deferring, /sys/kernel/debug/devices_deferred might give some useful information, but that's just a hope. -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last! From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 239B3C04A6A for ; Fri, 4 Aug 2023 22:28:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=r9Z2HgIMSRWseK3pLY7UgN3BalBwGutCy1aE15K+//4=; b=wWjTqMM9JO4LgC gfzwp8rYzIWVr7AR7s1dfL7GRf6MSv45lGtQFfTBTQ9boozd9uw4d9+2xYUe7qI0gmOVa/iqQWRc5 mIlgJGle1e8zYjzT5mpEUJ2UGZp1ZiVkZlJXkT2mT+GG8r3qaJZaE4eP9l+orYEo44OhNgndNns2m QXFLAChhH7FP5xQ/OhveC+QnA6ZJmUDyGWHa5TWLKjQbOwdUE/kqyBU3MKjh3VgSVau3ZyJCjE7ww Ckhh6wOJfZJel5wkt1SFf3QBmkt9XYHkEPHMhYjfYCC0mMcGDS+igKX8RZpcMEw08DDRZplO7V/Wj 3DpVm6WP5IUQJ0JKncQg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qS3Gr-00DMck-2b; Fri, 04 Aug 2023 22:27:45 +0000 Received: from [2001:4d48:ad52:32c8:5054:ff:fe00:142] (helo=pandora.armlinux.org.uk) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qS3Gn-00DMbs-2f for linux-arm-kernel@lists.infradead.org; Fri, 04 Aug 2023 22:27:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2019; h=Sender:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=oe3R1YKPMSS61Kql1VVF1wpgWu78Kb9TP4tIQw5QCgw=; b=SomGhhLqgNBWYw3yU37FFm8NyK s731vyDc2/mwtHBDHIGN42h/VWqKhR7ykKos0RTCgVNWEeRLTW6cjMRygXWyWOw0tRkaJNoU8ihMt joVIvLvs/SLsrlOSbDgQwuDcUf2tUgAXvZ06MdHxlSEi4slkyS8RXcw3xQXQpGACzxue2JTNuaUKE 8UjyA+Ru1k97X35qsFFUdXPQi408YX9k/nEMxwhC5z9COVi+ma3TUCk9he6zHNWSdYUXob5U+7FAk +gTTRgTTtyhAfUAnGBToBl5TlJnE/mL9gLGS99weNkN5FiRMeCGbXfp9pnNeM6Behz7BUwJueWHfK JWUbifAw==; Received: from shell.armlinux.org.uk ([fd8f:7570:feb6:1:5054:ff:fe00:4ec]:47482) by pandora.armlinux.org.uk with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1qS3Gf-0000pF-0M; Fri, 04 Aug 2023 23:27:33 +0100 Received: from linux by shell.armlinux.org.uk with local (Exim 4.94.2) (envelope-from ) id 1qS3Ge-0004MT-Q1; Fri, 04 Aug 2023 23:27:32 +0100 Date: Fri, 4 Aug 2023 23:27:32 +0100 From: "Russell King (Oracle)" To: Nick Bowler Cc: Rob Herring , Saravana Kannan , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, netdev@vger.kernel.org, regressions@lists.linux.dev Subject: Re: PROBLEM: Broken or delayed ethernet on Xilinx ZCU104 since 5.18 (regression) Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230804_152741_867332_8214E5B3 X-CRM114-Status: GOOD ( 29.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gRnJpLCBBdWcgMDQsIDIwMjMgYXQgMDU6MzE6MjFQTSAtMDQwMCwgTmljayBCb3dsZXIgd3Jv dGU6Cj4gT24gMjAyMy0wOC0wNCwgUm9iIEhlcnJpbmcgPHJvYmhAa2VybmVsLm9yZz4gd3JvdGU6 Cj4gPiBPbiBGcmksIEF1ZyA0LCAyMDIzIGF0IDExOjUy4oCvQU0gTmljayBCb3dsZXIgPG5ib3ds ZXJAZHJhY29ueC5jYT4gd3JvdGU6Cj4gPj4gSSBkb24ndCBrbm93IGFib3V0IHRoZSBkZWZlcnJl ZCBwcm9iZSB0aW1lb3V0LCBidXQgSSBiaXNlY3RlZCB0aGUgNi41LXJjNAo+ID4+IGJyZWFrYWdl IHRvIHRoaXMgY29tbWl0Ogo+ID4+Cj4gPj4gICBjb21taXQgYzcyMGExZjVlNmVlOGNiMzljMjg0 MzVlZmMwODE5Y2VjODRkNmVlMgo+ID4+ICAgQXV0aG9yOiBNaWNoYWwgU2ltZWsgPG1pY2hhbC5z aW1la0BhbWQuY29tPgo+ID4+ICAgRGF0ZTogICBNb24gTWF5IDIyIDE2OjU5OjQ4IDIwMjMgKzAy MDAKPiA+Pgo+ID4+ICAgICAgIGFybTY0OiB6eW5xbXA6IERlc2NyaWJlIFRJIHBoeSBhcyBldGhl cm5ldC1waHktaWQKPiA+Cj4gPiBJIGRvbid0IHNlZSBhbnl0aGluZyBvYnZpb3VzbHkgcHJvYmxl bWF0aWMgd2l0aCB0aGF0IGNvbW1pdC4gKFRoZQo+ID4gI3BoeS1jZWxscyBwcm9wZXJ0eSBhZGRl ZCBpcyB3cm9uZyBhcyBldGhlcm5ldCBwaHlzIGRvbid0IHVzZSB0aGUgcGh5Cj4gPiBiaW5kaW5n LCBidXQgdGhhdCBzaG91bGQganVzdCBiZSBpZ25vcmVkKS4gSSdkIGNoZWNrIGlmIHRoZSBwaHkg cHJvYmVkCj4gPiBhbmQgaGFzIGEgRFQgbm9kZSBhc3NvY2lhdGVkIHdpdGggaXQuCj4gCj4gSSB0 aGluayB0aGUgYW5zd2VyIGlzICJubywgdGhlIHBoeSB3YXMgbm90IHByb2JlZCIuICBXaXRob3V0 IHJldmVydGluZwo+IHRoYXQgY29tbWl0LCB0aGVyZSBpcyBhYnNvbHV0ZWx5IG5vdGhpbmcgaW4g L3N5cy9idXMvbWRpb19idXMvZGV2aWNlcy4KPiBUaGVyZSBpcyBubyBwaHkgZGV2aWNlIGxpbmsg dW5kZXIgL3N5cy9idXMvbWRpb19idXMvZHJpdmVycy8iVEkgRFA4Mzg2NyIsCj4gYW5kIHRoZXJl IGlzIG5vIG1kaW9fYnVzIHVuZGVyIC9zeXMvYnVzL3BsYXRmb3JtL2RldmljZXMvZmYwZTAwMDAu ZXRoZXJuZXQuCj4gCj4gV2hlbiBJIHJldmVydCB0aGF0IGNvbW1pdCwgSSBjYW4gbG9jYXRlIHRo ZSBwaHkgZGV2aWNlIHVuZGVyIGFsbCB0aGVzZQo+IGxvY2F0aW9ucy4KPiAKPiA+IGZ3X2Rldmxp bmsgdHJhY2tzIHBhcmVudC1jaGlsZCBkZXBlbmRlbmNpZXMgYW5kIG1heWJlIGNoYW5naW5nIHRv Cj4gPiBwYXJlbnQtZ3JhbmRjaGlsZCBhZmZlY3RlZCB0aGF0LiBXZSBkb24ndCB5ZXQgdHJhY2sg J3BoeS1oYW5kbGUnCj4gPiBkZXBlbmRlbmNpZXMsIGJ1dCB3ZSdkIGhhdmUgYSBjaXJjdWxhciBv bmUgaGVyZSBpZiB3ZSBkaWQgKHRob3VnaCB0aGF0Cj4gPiBzaG91bGQgYmUgaGFuZGxlZCkuIERv ZXMgImZ3X2Rldmxpbms9b2ZmIiBoZWxwPwo+IAo+IEJvb3Rpbmcgd2l0aCBmd19kZXZsaW5rPW9m ZiByZXN1bHRzIGluIG5vIG9idmlvdXMgY2hhbmdlIGluIGJlaGF2aW91ci4KCkkgdGhpbmsgd2Ug bmVlZCB0byByZXdpbmQgYSB0YWQuCgpNeSB1bmRlcnN0YW5kaW5nIGlzIHRoYXQgdGhpcyB1c2Vz IHRoZSBDYWRlbmNlIG1hY2IgZHJpdmVyLgoKSW4geW91ciBvcmlnaW5hbCBtZXNzYWdlLCB5b3Ug c2FpZCB0aGF0IHRoZSBldGhlcm5ldCBkcml2ZXIgd2Fzbid0CmJlaW5nIGJvdW5kIHRvIHRoZSBk cml2ZXIuCgpTaW5jZSB0aGUgZXRoZXJuZXQgZHJpdmVyIGlzIHJlc3BvbnNpYmxlIGZvciBzcG90 dGluZyB0aGUgIm1kaW8iCnN1Yi1ub2RlIGFuZCBjcmVhdGluZyB0aGUgTURJTyBidXMsIGlmIHRo ZSBkcml2ZXIgaXNuJ3QgYmVpbmcKc3VjY2Vzc2Z1bGx5IGJvdW5kLCB0aGVuIHRoZSBNRElPIGJ1 cyBhbmQgdGhlIFBIWXMgb24gdGhlIGJ1cyB3b24ndCBiZQpjcmVhdGVkLCBzbyB5b3Ugd29uJ3Qg ZmluZCB0aGVtIGluIC9zeXMvYnVzL21kaW9fYnVzL2RldmljZXMuCgpNb3Jlb3ZlciwgdGhlIENh ZGVuY2UgbWFjYiBkcml2ZXIsIGFuZCB0aGlzIGRvZXNuJ3QgY2FyZSBhYm91dCB0aGUKcHJlc2Vu Y2Ugb2YgdGhlIFBIWSBhdCBwcm9iZSB0aW1lLCBvbmx5IHdoZW4gdGhlIG5ldHdvcmsgaW50ZXJm YWNlIGlzCmJyb3VnaHQgdXAuIFNlZSBtYWNiX3BoeWxpbmtfY29ubmVjdCgpIHdoaWNoIGlzIGNh bGxlZCBmcm9tCm1hY2Jfb3BlbigpLgoKU28sIEkgdGhpbmsgdGhhdCB0aGUgZGVmZXJyZWQgcHJv YmluZyBoYXMgbm90aGluZyB0byBkbyB3aXRoIFBIWXMsIGFuZAp0aGF0J3MganVzdCBhIHdpbGQg Z29vc2UgY2hhc2UuCgpJIHRoaW5rIGluc3RlYWQgd2UgbmVlZCB0byBiZSBjb25jZW50cmF0aW5n IG9uIHdoYXQncyBnb2luZyBvbiB3aXRoCnRoZSBldGhlcm5ldCBkcml2ZXIsIGFuZCB3aHkgdGhl IGV0aGVybmV0IGRyaXZlciBpcyBkZWZlcnJpbmcgaXRzCnByb2JlLiBJcyBtYWNiX3Byb2JlKCkg Z2V0dGluZyBjYWxsZWQgYXQgYWxsPyBIb3cgZmFyIHRocm91Z2gKbWFjYl9wcm9iZSgpIGRvIHdl IGdldCBiZWZvcmUgd2UgZGVmZXI/CgpJIHRoaW5rIHRob3NlIGFyZSB0aGUga2V5IHF1ZXN0aW9u cyB0aGF0IG5lZWQgYW5zd2VyaW5nLgoKTWF5YmUsIGlmIHlvdSBjYW4gZ2V0IGFjY2VzcyB0byB0 aGUgbWFjaGluZSB3aGlsZSB0aGUgZHJpdmVyIGlzCmRlZmVycmluZywgL3N5cy9rZXJuZWwvZGVi dWcvZGV2aWNlc19kZWZlcnJlZCBtaWdodCBnaXZlIHNvbWUKdXNlZnVsIGluZm9ybWF0aW9uLCBi dXQgdGhhdCdzIGp1c3QgYSBob3BlLgoKLS0gClJNSydzIFBhdGNoIHN5c3RlbTogaHR0cHM6Ly93 d3cuYXJtbGludXgub3JnLnVrL2RldmVsb3Blci9wYXRjaGVzLwpGVFRQIGlzIGhlcmUhIDgwTWJw cyBkb3duIDEwTWJwcyB1cC4gRGVjZW50IGNvbm5lY3Rpdml0eSBhdCBsYXN0IQoKX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBt YWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9s aXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo=