From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0420CC433E0 for ; Fri, 26 Feb 2021 10:11:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9EB2C64EED for ; Fri, 26 Feb 2021 10:11:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230466AbhBZKLi (ORCPT ); Fri, 26 Feb 2021 05:11:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231175AbhBZKJf (ORCPT ); Fri, 26 Feb 2021 05:09:35 -0500 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6810C06174A for ; Fri, 26 Feb 2021 02:08:54 -0800 (PST) Received: by mail-wr1-x42d.google.com with SMTP id n4so8021843wrx.1 for ; Fri, 26 Feb 2021 02:08:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=oJii/GddMEKHlkJYx9Vk+QfCd96uLYBQX13xDE6YMvQ=; b=lI3fUHW0MQFXjxhIXQ/KNATYwxgpbjWykk6Mjl2KBkLHblX9UKO9ucwfG+ndP+SrnB rWO5QI/rKcVJL2FI/x2FoP4875QyJbRLpLlZ53j93xvwd1Eak1wVK3KWn8iJcYSqxxv2 SHAzt4RMZYI5s6+1fwuf+FEPVM5l23cN+AoIp+mOKIF83o0e7nSn1SWFfluieyrpoCnd yKHZGVq2OrgdWA4jNFLFQuy8gFJWbB9c6BE8vKG1MG7Z0smwgrcKbV88B2miWSj83HEP 06rQ/FpR6oZ36Nq+zITujtpA2RRi/1E82fxKRZNLmgEeMbF4eDIDoSVCgVfFf14+HLay OHDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=oJii/GddMEKHlkJYx9Vk+QfCd96uLYBQX13xDE6YMvQ=; b=SN53FjbF/VKWjlZd5OZ63vZ9bYo4v8BM6+HTszf/VxkZ14wayqUqL9Sn3rkjhsSFhn he7AAeV4K6bhfZnDN1kNtoEJz27Mvf/FlmLy2jnnnehKNzaGxoN26SSWepAnS5gJAw0G rsxa5MmWds3cnmutAig0cVNEaIVQSabd0AqOHvV4M6ILyHkuY0x3Eo90Gm+KQMsTkYKX 4xJlq2qPc0kmXXTOY2ml1DfnXSpJVPU9RCBDzQ8HNa+BfHGGriIxmvJY28pWW0l16IJW wlw3jA9AYUwaSElvX6c/jEomdEp8RIPdPFEqwxg+QlxBjLEOVH3sFac87cOFNzWISNKv ocsA== X-Gm-Message-State: AOAM531ZoWnINSt+Jz09n7zeeBI72/tAncyP2+X3w9TLFLk50NMdhIHA 1BQFck0JNc6H7XJdCCRFhs8= X-Google-Smtp-Source: ABdhPJxqeQFTzHiBtG8cUg7UpXau+VTGkj1GRpixkIGSEWq4g28CkoxMvlkYsgFSJwWuOJttZzzc1w== X-Received: by 2002:adf:b313:: with SMTP id j19mr2278778wrd.188.1614334133389; Fri, 26 Feb 2021 02:08:53 -0800 (PST) Received: from ?IPv6:2003:ea:8f39:5b00:3483:8cf6:25ff:155b? (p200300ea8f395b0034838cf625ff155b.dip0.t-ipconnect.de. [2003:ea:8f39:5b00:3483:8cf6:25ff:155b]) by smtp.googlemail.com with ESMTPSA id s23sm10715557wmc.35.2021.02.26.02.08.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 26 Feb 2021 02:08:53 -0800 (PST) Subject: Re: [PATCH v2] bcm63xx_enet: fix internal phy IRQ assignment To: =?UTF-8?Q?Daniel_Gonz=c3=a1lez_Cabanelas?= Cc: Andrew Lunn , Florian Fainelli , "David S. Miller" , Jakub Kicinski , gregkh@linuxfoundation.org, netdev@vger.kernel.org, =?UTF-8?Q?=c3=81lvaro_Fern=c3=a1ndez_Rojas?= References: <2323124.5UR7tLNZLG@tool> <9d9f3077-9c5c-e7bc-0c77-8e8353be7732@gmail.com> <0e75a5c3-f6bd-6039-3cfd-8708da963d20@gmail.com> <7fc4933f-36d4-99dc-f968-9ca3b8758a9b@gmail.com> From: Heiner Kallweit Message-ID: <286fb043-b812-a5ba-c66e-eef63fe5cc98@gmail.com> Date: Fri, 26 Feb 2021 11:08:43 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 26.02.2021 10:49, Daniel González Cabanelas wrote: > El vie, 26 feb 2021 a las 10:32, Heiner Kallweit > () escribió: >> >> On 26.02.2021 10:10, Daniel González Cabanelas wrote: >>> El vie, 26 feb 2021 a las 8:13, Heiner Kallweit >>> () escribió: >>>> >>>> On 25.02.2021 23:28, Daniel González Cabanelas wrote: >>>>> El jue, 25 feb 2021 a las 21:05, Heiner Kallweit >>>>> () escribió: >>>>>> >>>>>> On 25.02.2021 17:36, Daniel González Cabanelas wrote: >>>>>>> El jue, 25 feb 2021 a las 8:22, Heiner Kallweit >>>>>>> () escribió: >>>>>>>> >>>>>>>> On 25.02.2021 00:54, Daniel González Cabanelas wrote: >>>>>>>>> El mié, 24 feb 2021 a las 23:01, Florian Fainelli >>>>>>>>> () escribió: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2/24/2021 1:44 PM, Heiner Kallweit wrote: >>>>>>>>>>> On 24.02.2021 16:44, Daniel González Cabanelas wrote: >>>>>>>>>>>> The current bcm63xx_enet driver doesn't asign the internal phy IRQ. As a >>>>>>>>>>>> result of this it works in polling mode. >>>>>>>>>>>> >>>>>>>>>>>> Fix it using the phy_device structure to assign the platform IRQ. >>>>>>>>>>>> >>>>>>>>>>>> Tested under a BCM6348 board. Kernel dmesg before the patch: >>>>>>>>>>>> Broadcom BCM63XX (1) bcm63xx_enet-0:01: attached PHY driver [Broadcom >>>>>>>>>>>> BCM63XX (1)] (mii_bus:phy_addr=bcm63xx_enet-0:01, irq=POLL) >>>>>>>>>>>> >>>>>>>>>>>> After the patch: >>>>>>>>>>>> Broadcom BCM63XX (1) bcm63xx_enet-0:01: attached PHY driver [Broadcom >>>>>>>>>>>> BCM63XX (1)] (mii_bus:phy_addr=bcm63xx_enet-0:01, irq=17) >>>>>>>>>>>> >>>>>>>>>>>> Pluging and uplugging the ethernet cable now generates interrupts and the >>>>>>>>>>>> PHY goes up and down as expected. >>>>>>>>>>>> >>>>>>>>>>>> Signed-off-by: Daniel González Cabanelas >>>>>>>>>>>> --- >>>>>>>>>>>> changes in V2: >>>>>>>>>>>> - snippet moved after the mdiobus registration >>>>>>>>>>>> - added missing brackets >>>>>>>>>>>> >>>>>>>>>>>> drivers/net/ethernet/broadcom/bcm63xx_enet.c | 13 +++++++++++-- >>>>>>>>>>>> 1 file changed, 11 insertions(+), 2 deletions(-) >>>>>>>>>>>> >>>>>>>>>>>> diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.c b/drivers/net/ethernet/broadcom/bcm63xx_enet.c >>>>>>>>>>>> index fd876721316..dd218722560 100644 >>>>>>>>>>>> --- a/drivers/net/ethernet/broadcom/bcm63xx_enet.c >>>>>>>>>>>> +++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.c >>>>>>>>>>>> @@ -1818,10 +1818,19 @@ static int bcm_enet_probe(struct platform_device *pdev) >>>>>>>>>>>> * if a slave is not present on hw */ >>>>>>>>>>>> bus->phy_mask = ~(1 << priv->phy_id); >>>>>>>>>>>> >>>>>>>>>>>> - if (priv->has_phy_interrupt) >>>>>>>>>>>> + ret = mdiobus_register(bus); >>>>>>>>>>>> + >>>>>>>>>>>> + if (priv->has_phy_interrupt) { >>>>>>>>>>>> + phydev = mdiobus_get_phy(bus, priv->phy_id); >>>>>>>>>>>> + if (!phydev) { >>>>>>>>>>>> + dev_err(&dev->dev, "no PHY found\n"); >>>>>>>>>>>> + goto out_unregister_mdio; >>>>>>>>>>>> + } >>>>>>>>>>>> + >>>>>>>>>>>> bus->irq[priv->phy_id] = priv->phy_interrupt; >>>>>>>>>>>> + phydev->irq = priv->phy_interrupt; >>>>>>>>>>>> + } >>>>>>>>>>>> >>>>>>>>>>>> - ret = mdiobus_register(bus); >>>>>>>>>>> >>>>>>>>>>> You shouldn't have to set phydev->irq, this is done by phy_device_create(). >>>>>>>>>>> For this to work bus->irq[] needs to be set before calling mdiobus_register(). >>>>>>>>>> >>>>>>>>>> Yes good point, and that is what the unchanged code does actually. >>>>>>>>>> Daniel, any idea why that is not working? >>>>>>>>> >>>>>>>>> Hi Florian, I don't know. bus->irq[] has no effect, only assigning the >>>>>>>>> IRQ through phydev->irq works. >>>>>>>>> >>>>>>>>> I can resend the patch without the bus->irq[] line since it's >>>>>>>>> pointless in this scenario. >>>>>>>>> >>>>>>>> >>>>>>>> It's still an ugly workaround and a proper root cause analysis should be done >>>>>>>> first. I can only imagine that phydev->irq is overwritten in phy_probe() >>>>>>>> because phy_drv_supports_irq() is false. Can you please check whether >>>>>>>> phydev->irq is properly set in phy_device_create(), and if yes, whether >>>>>>>> it's reset to PHY_POLL in phy_probe()?. >>>>>>>> >>>>>>> >>>>>>> Hi Heiner, I added some kernel prints: >>>>>>> >>>>>>> [ 2.712519] libphy: Fixed MDIO Bus: probed >>>>>>> [ 2.721969] =======phy_device_create=========== >>>>>>> [ 2.726841] phy_device_create: dev->irq = 17 >>>>>>> [ 2.726841] >>>>>>> [ 2.832620] =======phy_probe=========== >>>>>>> [ 2.836846] phy_probe: phydev->irq = 17 >>>>>>> [ 2.840950] phy_probe: phy_drv_supports_irq = 0, phy_interrupt_is_valid = 1 >>>>>>> [ 2.848267] phy_probe: phydev->irq = -1 >>>>>>> [ 2.848267] >>>>>>> [ 2.854059] =======phy_probe=========== >>>>>>> [ 2.858174] phy_probe: phydev->irq = -1 >>>>>>> [ 2.862253] phy_probe: phydev->irq = -1 >>>>>>> [ 2.862253] >>>>>>> [ 2.868121] libphy: bcm63xx_enet MII bus: probed >>>>>>> [ 2.873320] Broadcom BCM63XX (1) bcm63xx_enet-0:01: attached PHY >>>>>>> driver [Broadcom BCM63XX (1)] (mii_bus:phy_addr=bcm63xx_enet-0:01, >>>>>>> irq=POLL) >>>>>>> >>>>>>> Currently using kernel 5.4.99. I still have no idea what's going on. >>>>>>> >>>>>> Thanks for debugging. This confirms my assumption that the interrupt >>>>>> is overwritten in phy_probe(). I'm just scratching my head how >>>>>> phy_drv_supports_irq() can return 0. In 5.4.99 it's defined as: >>>>>> >>>>>> static bool phy_drv_supports_irq(struct phy_driver *phydrv) >>>>>> { >>>>>> return phydrv->config_intr && phydrv->ack_interrupt; >>>>>> } >>>>>> >>>>>> And that's the PHY driver: >>>>>> >>>>>> static struct phy_driver bcm63xx_driver[] = { >>>>>> { >>>>>> .phy_id = 0x00406000, >>>>>> .phy_id_mask = 0xfffffc00, >>>>>> .name = "Broadcom BCM63XX (1)", >>>>>> /* PHY_BASIC_FEATURES */ >>>>>> .flags = PHY_IS_INTERNAL, >>>>>> .config_init = bcm63xx_config_init, >>>>>> .ack_interrupt = bcm_phy_ack_intr, >>>>>> .config_intr = bcm63xx_config_intr, >>>>>> } >>>>>> >>>>>> So both callbacks are set. Can you extend your debugging and check >>>>>> in phy_drv_supports_irq() which of the callbacks is missing? >>>>>> >>>>> >>>>> Hi, both callbacks are missing on the first check. However on the next >>>>> calls they're there. >>>>> >>>>> [ 2.263909] libphy: Fixed MDIO Bus: probed >>>>> [ 2.273026] =======phy_device_create=========== >>>>> [ 2.277908] phy_device_create: dev->irq = 17 >>>>> [ 2.277908] >>>>> [ 2.373104] =======phy_probe=========== >>>>> [ 2.377336] phy_probe: phydev->irq = 17 >>>>> [ 2.381445] phy_drv_supports_irq: phydrv->config_intr = 0, >>>>> phydrv->ack_interrupt = 0 >>>>> [ 2.389554] phydev->irq = PHY_POLL; >>>>> [ 2.393186] phy_probe: phydev->irq = -1 >>>>> [ 2.393186] >>>>> [ 2.398987] =======phy_probe=========== >>>>> [ 2.403108] phy_probe: phydev->irq = -1 >>>>> [ 2.407195] phy_drv_supports_irq: phydrv->config_intr = 1, >>>>> phydrv->ack_interrupt = 1 >>>>> [ 2.415314] phy_probe: phydev->irq = -1 >>>>> [ 2.415314] >>>>> [ 2.421189] libphy: bcm63xx_enet MII bus: probed >>>>> [ 2.426129] =======phy_connect=========== >>>>> [ 2.430410] phy_drv_supports_irq: phydrv->config_intr = 1, >>>>> phydrv->ack_interrupt = 1 >>>>> [ 2.438537] phy_connect: phy_drv_supports_irq = 1 >>>>> [ 2.438537] >>>>> [ 2.445284] Broadcom BCM63XX (1) bcm63xx_enet-0:01: attached PHY >>>>> driver [Broadcom BCM63XX (1)] (mii_bus:phy_addr=bcm63xx_enet-0:01, >>>>> irq=POLL) >>>>> >>>> >>>> I'd like to understand why the phy_device is probed twice, >>>> with which drivers it's probed. >>>> Could you please add printing phydrv->name to phy_probe() ? >>>> >>> >>> Hi Heiner, indeed there are two different probed devices. The B53 >>> switch driver is causing this issue. >>> >>> [ 2.269595] libphy: Fixed MDIO Bus: probed >>> [ 2.278706] =======phy_device_create=========== >>> [ 2.283594] phy_device_create: dev->irq = 17 >>> [ 2.283594] >>> [ 2.379554] =======phy_probe=========== >>> [ 2.383780] phy_probe: phydrv->name = Broadcom B53 (3) >> >> Is this an out-of-tree driver? I can't find this string in any >> DSA or PHY driver. >> > > Yes it is. > https://github.com/openwrt/openwrt/blob/master/target/linux/generic/files/drivers/net/phy/b53/b53_mdio.c#L421 > OK, I see. Then there's no reason to complain upstream. Either use the mainline B53 DSA driver of fix interrupt mode downstream. >> >>> [ 2.389235] phy_probe: phydev->irq = 17 >>> [ 2.393332] phy_drv_supports_irq: phydrv->config_intr = 0, >>> phydrv->ack_interrupt = 0 >>> [ 2.401445] phydev->irq = PHY_POLL >>> [ 2.405080] phy_probe: phydev->irq = -1 >>> [ 2.405080] >>> [ 2.410878] =======phy_probe=========== >>> [ 2.414996] phy_probe: phydrv->name = Broadcom BCM63XX (1) >>> [ 2.420791] phy_probe: phydev->irq = -1 >>> [ 2.424876] phy_drv_supports_irq: phydrv->config_intr = 1, >>> phydrv->ack_interrupt = 1 >>> [ 2.432994] phy_probe: phydev->irq = -1 >>> [ 2.432994] >>> [ 2.438862] libphy: bcm63xx_enet MII bus: probed >>> [ 2.443809] =======phy_connect=========== >>> [ 2.448092] phy_drv_supports_irq: phydrv->config_intr = 1, >>> phydrv->ack_interrupt = 1 >>> [ 2.456215] phy_connect: phy_drv_supports_irq = 1 >>> [ 2.456215] >>> [ 2.462961] Broadcom BCM63XX (1) bcm63xx_enet-0:01: attached PHY >>> driver [Broadcom BCM63XX (1)] (mii_bus:phy_addr=bcm63xx_enet-0:01, >>> irq=POLL) >>> >>> The board has no switch, it's a driver for other boards in OpenWrt. I >>> forgot it wasn't upstreamed: >>> https://github.com/openwrt/openwrt/tree/master/target/linux/generic/files/drivers/net/phy/b53 >>> >>> I tested a kernel compiled without this driver, now the IRQ is >>> detected as it should be: >>> >>> [ 2.270707] libphy: Fixed MDIO Bus: probed >>> [ 2.279715] =======phy_device_create=========== >>> [ 2.284600] phy_device_create: dev->irq = 17 >>> [ 2.284600] >>> [ 2.373763] =======phy_probe=========== >>> [ 2.377989] phy_probe: phydrv->name = Broadcom BCM63XX (1) >>> [ 2.383803] phy_probe: phydev->irq = 17 >>> [ 2.387888] phy_drv_supports_irq: phydrv->config_intr = 1, >>> phydrv->ack_interrupt = 1 >>> [ 2.396007] phy_probe: phydev->irq = 17 >>> [ 2.396007] >>> [ 2.401877] libphy: bcm63xx_enet MII bus: probed >>> [ 2.406820] =======phy_connect=========== >>> [ 2.411099] phy_drv_supports_irq: phydrv->config_intr = 1, >>> phydrv->ack_interrupt = 1 >>> [ 2.419226] phy_connect: phy_drv_supports_irq = 1 >>> [ 2.419226] >>> [ 2.429857] Broadcom BCM63XX (1) bcm63xx_enet-0:01: attached PHY >>> driver [Broadcom BCM63XX (1)] (mii_bus:phy_addr=bcm63xx_enet-0:01, >>> irq=17) >>> >>> Then, maybe this is an OpenWrt bug itself? >>> >>> Regards >>> Daniel >>> >>>> >>>>> I also added the prints to phy_connect. >>>>> >>>>>> Last but not least: Do you use a mainline kernel, or is it maybe >>>>>> a modified downstream kernel? In the latter case, please check >>>>>> in your kernel sources whether both callbacks are set. >>>>>> >>>>> >>>>> It's a modified kernel, and the the callbacks are set. BTW I also >>>>> tested the kernel with no patches concerning to the ethernet driver. >>>>> >>>>> Regards, >>>>> Daniel >>>>> >>>>>> >>>>>> >>>>>>>> On which kernel version do you face this problem? >>>>>>>> >>>>>>> The kernel version 4.4 works ok. The minimum version where I found the >>>>>>> problem were the kernel 4.9.111, now using 5.4. And 5.10 also tested. >>>>>>> >>>>>>> Regards >>>>>>> Daniel >>>>>>> >>>>>>>>> Regards >>>>>>>>>> -- >>>>>>>>>> Florian >>>>>>>> >>>>>> >>>> >>