From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A81FCC74A5B for ; Thu, 16 Mar 2023 22:10:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tAelWylzeVmkH9JgyflG/Wi1sgjxjgmepC0cUHWrbLA=; b=ERKFGCiIX1hGrK vutfuh3mtXllPluy5UHNPHPW7xArL9R7axD6Sltbp+mzOTmCm6qsVRASJ17gKeRC7Ht7qkMlmiyQB dtLTmgX5jmAOoCO4YXsd23cnGDanp2io0psd9ZkbV+924nXoBa3xyrBItI3ldER9xAT01hz6MJVre 7hrTZBbv+scaIt5HN6jRedCVqeq0Wa9QIb0X+GP65lO1AW0x9Ue4YkX3izYJgVpS7fkcWOpELTtU4 E8BMfF24/YApmkO9s2omQesDIgxaGG6OVjY4mP//gkvIZRI1Qh22OruvYNta1z7fAFrZR1XSzvKFf 94ng8sbhrBrruEm8RUsw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pcvml-00095o-1q; Thu, 16 Mar 2023 22:09:23 +0000 Received: from esa6.hgst.iphmx.com ([216.71.154.45]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pcvmi-00093j-2a for linux-arm-kernel@lists.infradead.org; Thu, 16 Mar 2023 22:09:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1679004560; x=1710540560; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=BwhFewWBTGHcR2L3lWIABhEO48EgoTHYXLzjzl/RGDE=; b=MTkaXkA+JvdbGHOZh9Nx+NtdHvvcTP21xKDAtgqayoQuXwHPz8kGl1DL XhTzPW0HzJW3jM4Lh6ehKS5qBmNsW0bOBdYRKSfA5rej5obzukzkgWyu2 kF/d6k/zyGGgHmH6fR0xRdmfbOkzeXDfPXzsygTdcQQItdyNZq1EZjw9G HRrCbNaghQlxQLCSthuf1GksQdkQi51gnlSwHmU0DAe3gi+f3Kg/qvjcp vYWHnnt0iYnKxhUXqyQY0VviYvBDYqgvATHvZ1aNF27dJ/bVCaROO87YX Vv6SZAQv2Feki6pkurpwopY7kdGAYQ1wiXVq1SuOKG53Bu1qDOE9gTfKi w==; X-IronPort-AV: E=Sophos;i="5.98,265,1673884800"; d="scan'208";a="225814596" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 17 Mar 2023 06:09:10 +0800 IronPort-SDR: fvptHz2jlY+uPJ+NE+5Y4E48t0e+V7ImLxKyYBVMvHcLW2qQjHAq2Gmhsp9s8nPnV4G4K/M8rN HHv+zktM76xT/z1twjp2PEzKOpolm9RqU5M8Xtp7BuJ4uQiHmpVqA+Cm+gTzA88EjcmgrxVwjv ZbIqDVYsm25DRAbUZXFXmHLWmOKNgWndDfw8mq+1WZ8YuRjnj4rVxPuJcsZ1ugDiHOPcI0wCrs nUihQv7wPmHG6Bgm8RV0UY75xrrjSvjGzlQvDJ/YPYmw0A0a2r+Ptyq3KbZFvStZSWcLjz6QGw t8A= Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 16 Mar 2023 14:19:54 -0700 IronPort-SDR: oyEWhWd0V0XzWESiPYxZVulM0GpnHttRARyyA+kr1nOB1MTmQZ082xLJxqQTOwNa3nTwtt5LEb E4O+6h5md0b7rQrXMYNORQznCupY1bIzLFAz/CiVZXoq6/IV8nDJLhh330p5i8im9u4ASIp5uu C0qxZY+pHaOuBTMEPJThNqUdfS3TAYkLk9PQ5zv/IEZAOINnd+ihZwGctsZcTUwsWhfIYFbpG2 9Y3NrkcRD4IJCkx51AvhnqTZAmuYWVN/DJ7x5lW82WvOZa/awt7evXhzcgt3qVCExR4BN+esvQ cwc= WDCIronportException: Internal Received: from usg-ed-osssrv.wdc.com ([10.3.10.180]) by uls-op-cesaip02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 16 Mar 2023 15:09:11 -0700 Received: from usg-ed-osssrv.wdc.com (usg-ed-osssrv.wdc.com [127.0.0.1]) by usg-ed-osssrv.wdc.com (Postfix) with ESMTP id 4Pd1d169RXz1RtVv for ; Thu, 16 Mar 2023 15:09:09 -0700 (PDT) Authentication-Results: usg-ed-osssrv.wdc.com (amavisd-new); dkim=pass reason="pass (just generated, assumed good)" header.d=opensource.wdc.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= opensource.wdc.com; h=content-transfer-encoding:content-type :in-reply-to:organization:from:content-language:references:to :subject:user-agent:mime-version:date:message-id; s=dkim; t= 1679004548; x=1681596549; bh=BwhFewWBTGHcR2L3lWIABhEO48EgoTHYXLz jzl/RGDE=; b=aokn1OA+f94YRY/4ymkhcYE3bMoK8reCMjT/KpuQH4ZVr+2mq9w 5e7pZgOHZWubNv4okXqZdY+Y8qXjm6sLsAowCBJ1rAuCPFwxGMt3t/K9oqwyBpu9 QtPZ+65osCY2fA8Z+k57qDR+MdICOJzpwp3dNRpRZ/bvm1mfKIbtPX3PHffijL1B 36LuyYyV5fDYN5k6mMmx38XuBxqIoTTj0yQ5+Gj0EB+n/M1I75c9FH7GQznNLpx4 l+G543xbtjd9v5buOaU3OqNDNsbgLQcw/2q2dz4IdVYNdMjBFt5L9tL/L2j69CNr WB6Oh11cwKBj/jmzvdLM2N3EJzPmKfdkDQA== X-Virus-Scanned: amavisd-new at usg-ed-osssrv.wdc.com Received: from usg-ed-osssrv.wdc.com ([127.0.0.1]) by usg-ed-osssrv.wdc.com (usg-ed-osssrv.wdc.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 7Zfyf_73Esty for ; Thu, 16 Mar 2023 15:09:08 -0700 (PDT) Received: from [10.225.163.88] (unknown [10.225.163.88]) by usg-ed-osssrv.wdc.com (Postfix) with ESMTPSA id 4Pd1cx56X6z1RtVm; Thu, 16 Mar 2023 15:09:05 -0700 (PDT) Message-ID: <15208569-b3d7-b9f7-6676-9d9122cac84a@opensource.wdc.com> Date: Fri, 17 Mar 2023 07:09:04 +0900 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Subject: Re: [PATCH v2 0/9] PCI: rockchip: Fix RK3399 PCIe endpoint controller driver To: Rick Wertenbroek Cc: alberto.dassatti@heig-vd.ch, xxm@rock-chips.com, rick.wertenbroek@heig-vd.ch, Rob Herring , Krzysztof Kozlowski , Heiko Stuebner , Shawn Lin , Lorenzo Pieralisi , =?UTF-8?Q?Krzysztof_Wilczy=c5=84ski?= , Bjorn Helgaas , Jani Nikula , Rodrigo Vivi , Mikko Kovanen , Greg Kroah-Hartman , devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org References: <20230214140858.1133292-1-rick.wertenbroek@gmail.com> <3c4ed614-f088-928f-2807-deaa5e4b668a@opensource.wdc.com> <8392a7de-666a-bce6-dc9f-b60d6dd93013@opensource.wdc.com> <1e8184e9-7e0b-2598-cc5a-e46d6c2f152a@opensource.wdc.com> Content-Language: en-US From: Damien Le Moal Organization: Western Digital Research In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230316_150920_993326_9D94B8A9 X-CRM114-Status: GOOD ( 34.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 3/17/23 01:34, Rick Wertenbroek wrote: >>> By the way, enabling the interrupts to see the error notifications, I do see a >>> lot of retry timeout and other recoverable errors. So the issues I am seeing >>> could be due to my PCI cable setup that is not ideal (bad signal, ground loops, >>> ... ?). Not sure. I do not have a PCI analyzer handy :) > > I have enabled the IRQs and messages thanks to your patches but I don't get > messages from the IRQs (it seems no IRQs are fired). My PCIe link seems stable. > The main issue I face is still that after a random amount of time, the BARs are > reset to 0, I don't have a PCIe analyzer so I cannot chase config space TLPs > (e.g., host writing the BAR values to the config header), but I don't think that > the problem comes from a TLP issued from the host. (it might be). Hmmm... I am getting lots of IRQs, especially the ones signaling "replay timer timed out" and "replay timer rolled over after 4 transmissions of the same TLP" but also some "phy error detected on receive side"... Need to try to rework my cable setup I guess. As for the BARs being reset to 0, I have not checked, but it may be why I see things not working after some inactivity. Will check that. We may be seeing the same regarding that. > I don't think it's a buffer overflow / out-of-bounds access by kernel > code for two reasons > 1) The values in the config space around the BARs is coherent and unchanged > 2) The bars are reset to 0 and not a random value > > I suspect a hardware reset of those registers issued internally in the > PCIe controller, > I don't know why (it might be a link related event or power state > related event). > > I have also experienced very slow behavior with the PCI endpoint test driver, > e.g., pcitest -w 1024 -d would take tens of seconds to complete. It seems to > come from LCRC errors, when I check the "LCRC Error count register" > @0xFD90'0214 I can see it drastically increase between two calls of pcitest > (when I mean drastically it means by 6607 (0x19CF) for example). > > The "ECC Correctable Error Count Register" @0xFD90'0218 reads 0 though. > > I have tried to shorten the cabling by removing one of the PCIe extenders, that > didn't change the issues much. > > Any ideas as to why I see a large number of TLPs with LCRC errors in them ? > Do you experience the same ? What are your values in 0xFD90'0214 when > running e.g., pcitest -w 1024 -d (note: you can reset the counter by writing > 0xFFFF to it in case it reaches the maximum value of 0xFFFF). I have not checked. But I will look at these counters to see what I have there. -- Damien Le Moal Western Digital Research _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel