From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F3478C4829E for ; Thu, 15 Feb 2024 11:26:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:From:References:Cc:To: Subject:Date:Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=cNMbeCAeWiFnl9TY+pmikS3RqfIO58wgaAIHyRo07U8=; b=kcjuAAudcz+wDPh4dXyGUt9TJ/ 5pnJ1SAHKsMV08MO9ab867np4SO+nUxfd/9Kajx8I4pOu755uTuXmdGmmbbTPJS6IwpcSWyuMhfz5 x6w2oEhfodqYmdnt0jj4yFA65Zr2iLZbRp+79hbVqMKy3F3coZfwuGbpavUQ5186wMalUgPRZWto7 8s2ENpmXYYVJxc3tIBvMNCjuvFxtGNLZ3f0SaXSwUy/LO+LM2uLhN27eekkT8VKwaVs327vP7T92w JFQErk0QNHU3MHfq8SFHgC91vEjuomGXGYsd4UQ1FpdM9t0og72w3NCoLOKlbo/UbeHGHsIBLT6WO j7oeUsMw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1raZsb-0000000G1kw-3uxC; Thu, 15 Feb 2024 11:26:13 +0000 Received: from mail-bn8nam11on20601.outbound.protection.outlook.com ([2a01:111:f403:2414::601] helo=NAM11-BN8-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1raZsY-0000000G1ib-3YNr for linux-arm-kernel@lists.infradead.org; Thu, 15 Feb 2024 11:26:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IM5CPOHa3lPE+Q6htE+6kc98MHMlSiX2H7h/EMqiR+3wn8AgHCWRL+t3yNvmVyRLfdy6gU+T5ujUUy/u0EkUiZjBu8f4tJNPPD83aMYqiL0gl5qBCcFDLcafNyM5nvrSJclxvbBOgdQfTqVX1Eq3Q9oUFMTHzg1wvqY6J0wpTsJa8XlNFI8zqxwj6eQVyn7onkkZEuLKEft+Tr/gks+Zu2eH5+NhsnDcEhqOSn2dE9cjuu0F8X8vMssmteBVI55GJaf0lRt7wjvgt+FdJ6XgIal0amvoo4bjakK7povMWzDLO6hdW9EESqHFYd1/+XLze/KQAGSAccJHZ2QSp5nM2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CLgPQX0pJPCezfrcHy5k4d0b4WLCVJvdcQ9Aut+yoF4=; b=TYF2wmB7724cmfNFMvTpKt0VrQGDCtS/ALpdD5/OTBXh3wBuNuuixtA5y9LuEycRDAmJ+IMU2vsNUkcioGnlzpH5lRf8D5SVWpXW0B2cC4sTVrzohWO8YkX0MHX4u7rNe/LsYXFf4Nw3iu3Fy7/sjsyjp6ucJ1GDrF2SE0fmFVKB+atdLjKIpH2ToycZ7n2fjSWXy7Kor289xoyAXPeUA6ejTM/sgAoWyM7m5HuSyUzFKIUSdiFtoASStmb7jNUkXAl4V0rTSokg9PpxlJffYJ+CHUwhDTSdIyI8sWog6jTud1YwNS7Mm7bKmpB2ZLiU/62ynU+sdamRBP1cIC6gHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CLgPQX0pJPCezfrcHy5k4d0b4WLCVJvdcQ9Aut+yoF4=; b=MLsQhn7fdX2ZUZ5Tp60U57upwxlfnib2yoCfNxCctA7dgtjRAMr9A5eoGbo8DUgtNN8OXIEKI3DCJa/Rn3TCL74ljV0FgsAABW3Y75QJg7DY8LpVaeos4TD7s8dpvRAVL+1rmrWR9tIQ3fV2OKMLN2ULb0eWWu3TtDK8qJy/neIAVsPEOrPy/GM7WR0TO4clAzXhCPfoa5u1VAaM9FBu6wQNO8znISwBmJtThh/GYhom7DgrDxl8YACldRP7K49WXdIdCz416tDxGaLS32cYszpKcWfoHJUcks2gYjyZd3lagGB550g3G3uW6/KYzx907vPPwURXaOIbkk+eyczLSQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from PH8PR12MB6674.namprd12.prod.outlook.com (2603:10b6:510:1c1::18) by DM4PR12MB6661.namprd12.prod.outlook.com (2603:10b6:8:b7::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.11; Thu, 15 Feb 2024 11:25:59 +0000 Received: from PH8PR12MB6674.namprd12.prod.outlook.com ([fe80::ad63:96df:9a9a:83e4]) by PH8PR12MB6674.namprd12.prod.outlook.com ([fe80::ad63:96df:9a9a:83e4%5]) with mapi id 15.20.7292.022; Thu, 15 Feb 2024 11:25:59 +0000 Message-ID: <17dadf66-055c-4796-a905-44e37b6fcfe3@nvidia.com> Date: Thu, 15 Feb 2024 16:55:47 +0530 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH V16 13/13] PCI: tegra: Add Tegra194 PCIe support To: Bjorn Helgaas Cc: Lorenzo Pieralisi , bhelgaas@google.com, robh+dt@kernel.org, mark.rutland@arm.com, thierry.reding@gmail.com, jonathanh@nvidia.com, Kishon Vijay Abraham I , catalin.marinas@arm.com, Will Deacon , jingoohan1@gmail.com, gustavo.pimentel@synopsys.com, digetx@gmail.com, mperttunen@nvidia.com, linux-pci@vger.kernel.org, devicetree@vger.kernel.org, linux-tegra@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kthota@nvidia.com, mmaddireddy@nvidia.com, sagar.tv@gmail.com References: <20240214191249.GA1263222@bhelgaas> Content-Language: en-US From: Vidya Sagar In-Reply-To: <20240214191249.GA1263222@bhelgaas> X-ClientProxiedBy: MA0PR01CA0025.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a01:b8::13) To PH8PR12MB6674.namprd12.prod.outlook.com (2603:10b6:510:1c1::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH8PR12MB6674:EE_|DM4PR12MB6661:EE_ X-MS-Office365-Filtering-Correlation-Id: 394e01f4-36e1-4c7f-b780-08dc2e18e201 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aDI9/cS4KGgBDoQDKGIyC3STbkrh9sGoI1enyvAiDzrAfmlOcgO2T/SuJ/WlQ+gJ37xwCg4+0jFDNdHbWLBGCQv8liK03XDOlqlSbZ9HaPO3eFr4YFBGWSVU0uT94JcbqnhiCNa+kSH48382WHPS0KvmQ3/yApUC5xIYGmvDHfq/KBMAOKo5GmH4VEp554uFNDhDNioRsVIOcWTwU12RdjiN2rrZYm7lvNMpeDOqMlze/871q9eMTmKbYzltC3Kc/OK2VDrukJuBIEF6Lk36FPg6OfilzMLqgaNpUWe6RHljaYvEr9avXnJTQ1ENDaLy/plyS0rkUGyvawvPmPsu6IQ1QouXpTk86E04LVJD3rkdl9DrL9GaX6RDa2wZiCd7ODmXEktg8q3uufiFsV8bPvOddvJc25af49tENenH88qcqgJz5Zp+NyXsoUB8QZPdI4CRvM5oBuseivuzrwPs86qcNFjIX1b0aidmBX64OMY9cCz5d09ASri3oKxPG9cyJvumF0Hu+YYUW2esVE3+3bzZt3xjE5bQ5WeTP5ZhqQtGAjJfWAkGXfcSMHuMteBa X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH8PR12MB6674.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(136003)(39860400002)(346002)(366004)(396003)(230922051799003)(230273577357003)(1800799012)(64100799003)(186009)(451199024)(66946007)(2906002)(36756003)(53546011)(478600001)(66476007)(2616005)(6666004)(6506007)(6512007)(26005)(5660300002)(7416002)(66556008)(6916009)(4326008)(31696002)(316002)(8936002)(8676002)(86362001)(54906003)(38100700002)(6486002)(83380400001)(41300700001)(31686004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MHNuTW1TOGpqRzFyU3Q5TXY5VE1kTGd6RDdIZ2FLblE2VkJidEZHejBkYTZG?= =?utf-8?B?cFkzZnlRbWYxZDVYSSs1Q2tkZENFb0g3WExUb1c0Z3RzQ1VvMWRjeCtKdTVi?= =?utf-8?B?SDNVNDh1aktrd2hsWHl3Q1RTOWxXN29qb1haRTBKQ05RUmFLSlU5R2pCcENm?= =?utf-8?B?b0F4Y2NKWGFOUWVJTWpkTTNBRFVYRzNLV2NleHNmQW93TjZ6L0xHM3ZPRm1J?= =?utf-8?B?ZDl1RDdBbHJ3dU1NQll5amd3T2hOMEh5NGRISHJYSDV3NWtMYlZLUjdEMUs0?= =?utf-8?B?N3JFRnlPL0YyYk8zemM3NCtOT3laQ1NjQ0V5WmZsdjRyaXUvYlBPSlJWQVVk?= =?utf-8?B?WjhaZkRvUVMxTXFJZ3ZWamVGZStEV1NVZGVtRllqZUhhb2VlWnJxRHhZSTVq?= =?utf-8?B?emRRZ0tNRmQwZElhOU0vcmpxV05NaFZwUnZHZXpscXUyS2RwMjJ6ZmNSSjZw?= =?utf-8?B?Ny9ZTHhJZzRBZU1TZ3Bab2RBZ2RxVjlpM2djdWFWa21TTlJIRmN5bTFMdEcw?= =?utf-8?B?YjBIelVRQjVxa0VQZElLdWVFeERyOHpEQm13UUs2a2IyQkRJUGluQThDMm5i?= =?utf-8?B?eXhTbWdMdFVuWjRaWWZDR05mMFcyZisyQmNwT2MvdFI0alpvWFA2RlBWZXpo?= =?utf-8?B?djdSTWVjNXdhT1hyL3NXY2RtUnY1WWswaW9YcXp3MkxpOXMwSHRPbE5KU0x2?= =?utf-8?B?UWhOMTJNWHNXTkhBNHo4SUJITWxCK3l5cnZRd2hsNUVubk40dVJCSjhwL2g4?= =?utf-8?B?TkRHQ25ZcHk2RFk3ZEdYOHk2VGs0ZzI3ZEU2b2l2S1Fwd2JtbzRLV21QZ0lI?= =?utf-8?B?K09xY0RmS2srT3h0T0ZySldVNC9FSTdRQStiRVRzRzROUVhtOU9RWkFIek93?= =?utf-8?B?Sk9VY1BuWml3Zk1tS1ByakRXVENYUnhBSnk1eTFGQ3FKQWE0czU5cDdoS3Z6?= =?utf-8?B?OUUxVEFpUVVmS1hkbHRUM3U5d2lBb2JNdHZ2T1hYNDdCMy93UUY5U0JLUE1h?= =?utf-8?B?cFBGd2s1aG9mZWo1OVp2bU8zZlRWcGpkSDZabDNVV3dzSWpmZThkZExJL2gv?= =?utf-8?B?aitWTWVHNks4TFdOdGlyL0dkL29IMU9ZanozNitORHdkNVdGbW84UVo1ay9q?= =?utf-8?B?cjZvU3c0eHloRzhuUkI5WkY4NnZzU29FbUN2L0gvTDRSL3IwaVZpN0hhdDdK?= =?utf-8?B?b2puVWlFa1l6R01Wc1dZL3llYzZ6VFlWL3c5cVNoOFE1WnBOeGpKRmJ3MldR?= =?utf-8?B?aDZkbjB6a2V5SGUzL1lVYXVvYllpdW1GT0FyRHJUbkhzNVdTWm8vT3A4U0RR?= =?utf-8?B?WW8wSHp1MTAvYmtJWmttSG9rM1plQjQ2bjNTMk1KZjRLRnZvK29meWpJTGZh?= =?utf-8?B?VXRINE14bkhFWGRoWmxQdGtSWndzRk1ZcGw2MVM2Qk1BblU4NFpTNWVJU3dB?= =?utf-8?B?VVNyT0hDdm85WGV3NmpPd2hJeHMyMkJZWG43NVk4OFhldS9nSHhENFVqWjV3?= =?utf-8?B?bnhLRmlmZE02Ky83Q04rREhkOG82SHMzUXAyYW5TQWxwL3h1TDJQT1lHY09S?= =?utf-8?B?bDFmUFF1aDI1S0lUQlIzaGMrS0gvcWlyRSs2d2V2dUp3Z3JjTWR6UDJOTHZW?= =?utf-8?B?L1FjUkZWSVRHdkxjMDQ4b3hxTzhOWE9ER2piOVpWTDhxK0N4Tk5PaFhEODJx?= =?utf-8?B?Qm9wdnoyblZaQzdxVlNNUDVsM2hrUjJoa2ZWMmEzeGpOblVhQ2VXdzFGakxF?= =?utf-8?B?R3BBYjVQMy9QVmN5SWtRWE81d09TMGU5blhrUzNOOE4xMUlwWi94YXYrenJl?= =?utf-8?B?dGtpczMvVmdTU3lmRmNpNlFUaEQ1bHJiSDJPOFpsN01QNk5QT3JiaDBJNXF2?= =?utf-8?B?L3ZpQjNNeHYwYml1YUdTbnNXWDArb3NoVFFTeXFjUDlUanhRcjVGbUkwVlhU?= =?utf-8?B?Ym9PSm13ZEx5UXF3bVVXS1hsaGJKa1VWcE1Dd1JnL3VlWGdSam1VTGIwMllI?= =?utf-8?B?NG03Umg0K2l4bUJ5WlBrYVpvd0lZMThTTXRrZzZPclJwWnEvSXFqZEthdjd0?= =?utf-8?B?SVNtanA5eVAxb3YyeTduUFNtbE04TjdCYWZla3FyWVpLcVo4a0RMbERnTHVm?= =?utf-8?Q?ZzoIFDAcoMaSr8wjg3QswyqUp?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 394e01f4-36e1-4c7f-b780-08dc2e18e201 X-MS-Exchange-CrossTenant-AuthSource: PH8PR12MB6674.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2024 11:25:58.9251 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: urtg9EGQwTj1iO0BEOirQTQg2ogPTIdVguHBs7L+JNLwfuTF9fwC7zxB9ALxrzYS2L3jvxpWoACxyx94kJoRiQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6661 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240215_032610_932383_EAA5A3E5 X-CRM114-Status: GOOD ( 23.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 15-02-2024 00:42, Bjorn Helgaas wrote: > External email: Use caution opening links or attachments > > > Hi Vidya, question about ancient history: > > On Tue, Aug 13, 2019 at 05:06:27PM +0530, Vidya Sagar wrote: >> Add support for Synopsys DesignWare core IP based PCIe host controller >> present in Tegra194 SoC. >> ... >> +static int tegra_pcie_dw_host_init(struct pcie_port *pp) >> +{ >> + struct dw_pcie *pci = to_dw_pcie_from_pp(pp); >> + struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); >> + u32 val, tmp, offset, speed; >> + >> + tegra_pcie_prepare_host(pp); >> + >> + if (dw_pcie_wait_for_link(pci)) { >> + /* >> + * There are some endpoints which can't get the link up if >> + * root port has Data Link Feature (DLF) enabled. >> + * Refer Spec rev 4.0 ver 1.0 sec 3.4.2 & 7.7.4 for more info >> + * on Scaled Flow Control and DLF. >> + * So, need to confirm that is indeed the case here and attempt >> + * link up once again with DLF disabled. > This comment suggests that there's an issue with *Endpoints*, not an > issue with the Root Port. If so, it seems like this problem could > occur with all Root Ports, not just Tegra194. Do you remember any > details about this? > > I don't remember hearing about any similar issues, and this driver is > the only place PCI_EXT_CAP_ID_DLF is referenced, so maybe it is > actually something related to Tegra194? We noticed PCIe link-up issues with some endpoints. link-up at the physical layer level but NOT at the Data link layer level precisely. We further figured out that it is the DLFE DLLPs that the root port sends during the link up process which are causing the endpoints get confused and preventing them from sending the InitFC DLLPs leading to the link not being up at Data Link Layer level. PCIe spec rev4.0, Rev 4.0, ver 1.0, Section-3.3 says that (verbatim) "The Data Link Feature Exchange Enable field permits systems to disable the Data Link Feature Exchange. This can be used to work around legacy hardware that does not correctly ignore the DLLP". Based on this, we added the following code to retry link-up with DLFE disabled, only upon confirming that the link came up at phy level but not at the DL level in the previous attempt. Thanks, Vidya Sagar > >> + val = appl_readl(pcie, APPL_DEBUG); >> + val &= APPL_DEBUG_LTSSM_STATE_MASK; >> + val >>= APPL_DEBUG_LTSSM_STATE_SHIFT; >> + tmp = appl_readl(pcie, APPL_LINK_STATUS); >> + tmp &= APPL_LINK_STATUS_RDLH_LINK_UP; >> + if (!(val == 0x11 && !tmp)) { >> + /* Link is down for all good reasons */ >> + return 0; >> + } >> + >> + dev_info(pci->dev, "Link is down in DLL"); >> + dev_info(pci->dev, "Trying again with DLFE disabled\n"); >> + /* Disable LTSSM */ >> + val = appl_readl(pcie, APPL_CTRL); >> + val &= ~APPL_CTRL_LTSSM_EN; >> + appl_writel(pcie, val, APPL_CTRL); >> + >> + reset_control_assert(pcie->core_rst); >> + reset_control_deassert(pcie->core_rst); >> + >> + offset = dw_pcie_find_ext_capability(pci, PCI_EXT_CAP_ID_DLF); >> + val = dw_pcie_readl_dbi(pci, offset + PCI_DLF_CAP); >> + val &= ~PCI_DLF_EXCHANGE_ENABLE; >> + dw_pcie_writel_dbi(pci, offset, val); >> + >> + tegra_pcie_prepare_host(pp); >> + >> + if (dw_pcie_wait_for_link(pci)) >> + return 0; >> + } >> + >> + speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) & >> + PCI_EXP_LNKSTA_CLS; >> + clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]); >> + >> + tegra_pcie_enable_interrupts(pp); >> + >> + return 0; >> +} _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel