From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 79A02D0D79F for ; Fri, 11 Oct 2024 14:40:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=C/UtyB2uWIm739F4MXYMRShLe9+F8H6vI52MdDvVTyA=; b=11ybYXaH7vKxQHVCZA/afQZfrT 1zPdm4WNTWa0ykbK2Q6HoU+Cf2rOQre0U8Dtf2B+cCz/+JdBwVqb5nmqlPsJNnMp0la5CujYAyZwL QN5/6jAPyEQF+AiI0RWJo+jaou/kPI64H46UIlBKIgasgFb6lh0k9+mkDyIgnwb0hLEi/KVYxnVbf YqQ/BSmKOqbK2uL5p5LcwnacfwHER/YN8PrlgTrxfnD+m7iG27sUX2JeYphq5IBMV97ASQ5uaMw9s z0PB9wrlqe0NGGz0WZ5f6qdckSp496QVbBMujDGSeb5rcFWycthtXToJuXXL8IhefCBxm9yXgQuND aIGTSREQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1szGor-0000000GdjM-3aCu; Fri, 11 Oct 2024 14:40:41 +0000 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1szGkk-0000000GceZ-17p6 for linux-arm-kernel@lists.infradead.org; Fri, 11 Oct 2024 14:36:30 +0000 Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-43117ed8adbso19738425e9.2 for ; Fri, 11 Oct 2024 07:36:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728657384; x=1729262184; darn=lists.infradead.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=C/UtyB2uWIm739F4MXYMRShLe9+F8H6vI52MdDvVTyA=; b=hGLAoXMgMa/J6Ue4w8WCNrhl+R4Nw22Q0CoKbi79imY1EeY3XNIEWcpwZgpWSPCTc4 +45Yj+OU7lJClHT55OwFN1yEHAUv4ayjsZdIXDZVe4DVbv92upjnVdOALZ1XgBTD9dOO yDtUK0aXPN0M/M9x/rFZf8R/My1DoGubCNoFK+1S5Qt4ivfC4JlYh1GEvwcFUrdD3Nwl 4q8AfMpVxxvz9/8gSIcwzO6DNAvgTm5xHoobgmqTlKmE8LpI5rPzGoFJxXobAPtngpKf UCsEyHK1DgaTnBXu7ccKIF8QmQ3i/NRlCwVHGTPlFSfK+Zuh56gWtzD9/ISpctUYn0Qy iVXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728657384; x=1729262184; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=C/UtyB2uWIm739F4MXYMRShLe9+F8H6vI52MdDvVTyA=; b=G9MY7C/ylO7jCF0bIvQebh0HQgUQHu+FVrZHFyikvsaKU/xIqS0vitOjAdT6CyJl1N CQwDfeYwuaow80vJWuHKEmozruR0HfzGFehQji13ORssiMbHkpDlSKSy2UfFDFQVNRzD u/FibCIEAQ7jItIo0w9JevR8CnjNmZbMfMK8beZR5CviQwQc8ldo5N0549ge60+6hcK2 Cc5dPPuEQeBB7PlcnyWS9w3jXEJ2kJ3Vk6nwwdTP45LnwbUKSpg+5XTsT6uvcrlGrhrR JW+pqptV9V71H/6YjPDbgwNJNb0umkk/YfT1NqkGJPzLG+IYTiKfKiySFexKfb4fpB9o 4gcQ== X-Forwarded-Encrypted: i=1; AJvYcCXuK25nQz0BERm9rSOhqRvnQVbN+hST5BXN+sORbbnBqj1qwMSAgCoMuyDmwKZIcH/JBQ2kq3pjFmFZz8oPvWwa@lists.infradead.org X-Gm-Message-State: AOJu0YwcgEFWkMdJ6wJ6Qcg5ke76KRB9/MMYF7unyVNrFR4r473fvM/f ejguZPVYkof8GTaC0TmabuYDiMs0jKRYRPExi16OXFWb0LEKfq146cPGjNzY X-Google-Smtp-Source: AGHT+IFlnrR3OqGd8XdhiXG/glDXZS1+OWZdJWElmbQjUA5D2o+fvQ2WDzXsVd68UTgo7ViCacocYQ== X-Received: by 2002:a5d:62c2:0:b0:37d:39bf:37e3 with SMTP id ffacd0b85a97d-37d55199c2dmr2624425f8f.16.1728657383878; Fri, 11 Oct 2024 07:36:23 -0700 (PDT) Received: from eichest-laptop ([2a02:168:af72:0:5e49:8e15:b881:5968]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-37d4b9180f8sm4071123f8f.115.2024.10.11.07.36.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Oct 2024 07:36:23 -0700 (PDT) Date: Fri, 11 Oct 2024 16:36:21 +0200 From: Stefan Eichenberger To: Frank Li Cc: Francesco Dolcini , hongxing.zhu@nxp.com, l.stach@pengutronix.de, lpieralisi@kernel.org, kw@linux.com, manivannan.sadhasivam@linaro.org, robh@kernel.org, bhelgaas@google.com, shawnguo@kernel.org, s.hauer@pengutronix.de, kernel@pengutronix.de, festevam@gmail.com, francesco.dolcini@toradex.com, linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, imx@lists.linux.dev, linux-kernel@vger.kernel.org, Stefan Eichenberger Subject: Re: [PATCH v2] PCI: imx6: Add suspend/resume support for i.MX6QDL Message-ID: References: <20241009131659.29616-1-eichest@gmail.com> <20241010201121.GA88411@francesco-nb> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241011_073626_339730_41563D8C X-CRM114-Status: GOOD ( 34.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Frank, On Thu, Oct 10, 2024 at 06:45:17PM -0400, Frank Li wrote: > On Thu, Oct 10, 2024 at 10:11:21PM +0200, Francesco Dolcini wrote: > > Hello Frank, > > > > On Thu, Oct 10, 2024 at 04:01:21PM -0400, Frank Li wrote: > > > On Wed, Oct 09, 2024 at 03:14:05PM +0200, Stefan Eichenberger wrote: > > > > From: Stefan Eichenberger > > > > > > > > The suspend/resume support is broken on the i.MX6QDL platform. This > > > > patch resets the link upon resuming to recover functionality. It shares > > > > most of the sequences with other i.MX devices but does not touch the > > > > critical registers, which might break PCIe. This patch addresses the > > > > same issue as the following downstream commit: > > > > https://github.com/nxp-imx/linux-imx/commit/4e92355e1f79d225ea842511fcfd42b343b32995 > > > > In comparison this patch will also reset the device if possible. Without > > > > this patch suspend/resume will not work if a PCIe device is connected. > > > > The kernel will hang on resume and print an error: > > > > ath10k_pci 0000:01:00.0: Unable to change power state from D3hot to D0, device inaccessible > > > > 8<--- cut here --- > > > > Unhandled fault: imprecise external abort (0x1406) at 0x0106f944 > > > > > > > > Signed-off-by: Stefan Eichenberger > > > > --- > > > > > > Thank you for your patch. > > > > > > But it may conflict with another suspend/resume patch > > > https://lore.kernel.org/imx/1727245477-15961-8-git-send-email-hongxing.zhu@nxp.com/ > > > > Thanks for the head-up. > > > > Do you see any issue with this patch apart that? Because this patch is > > fixing a crash, so I would expect this to be merged, once ready, and > > such a series rebased afterward. > > > > I am writing this explicitly since you wrote a similar comment on the > > v1 (https://lore.kernel.org/all/ZsNXDq%2FkidZdyhvD@lizhi-Precision-Tower-5810/) > > and I would like to prevent to have this fix starving for long just because > > multiple people is working on the same driver. > > My key comment for this patch is use flags IMX_PCIE_FLAG_SKIP_TURN_OFF > in suspend()/resume(), it is up to kw to pick which one firstly. I will try to implement it as you proposed with the new flag. However, what I figured out with kernel v6.12-rc1 I get the following warning when booting on an i.MX6QDL even without my patch applied: [ 1.901199] PCI: bus0: Fast back to back transfers disabled [ 1.904754] mmc1: SDHCI controller on 2190000.mmc [2190000.mmc] using ADMA [ 1.904914] mmc2: SDHCI controller on 2194000.mmc [2194000.mmc] using ADMA [ 1.910686] pci 0000:01:00.0: [168c:003c] type 00 class 0x028000 PCIe Endpoint [ 1.918390] NET: Registered PF_PACKET protocol family [ 1.918573] mmc0: SDHCI controller on 2198000.mmc [2198000.mmc] using ADMA [ 1.924322] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x001fffff 64bit] [ 1.931635] Key type dns_resolver registered [ 1.936764] pci 0000:01:00.0: ROM [mem 0x00000000-0x0000ffff pref] [ 1.961043] pci 0000:01:00.0: supports D1 D2 [ 1.961526] Registering SWP/SWPB emulation handler [ 1.965601] mmc0: new DDR MMC card at address 0001 [ 1.976575] mmcblk0: mmc0:0001 Q2J54A 3.59 GiB [ 1.979794] Loading compiled-in X.509 certificates [ 1.985036] PCI: bus1: Fast back to back transfers disabled [ 1.991531] pci 0000:00:00.0: bridge window [mem 0x01000000-0x011fffff]: assigned [ 1.998742] mmcblk0: p1 p2 p3 [ 1.999163] pci 0000:00:00.0: BAR 0 [mem 0x01200000-0x012fffff]: assigned [ 2.003947] mmcblk0boot0: mmc0:0001 Q2J54A 16.0 MiB [ 2.008990] pci 0000:00:00.0: bridge window [mem 0x01300000-0x013fffff pref]: assigned [ 2.009023] pci 0000:00:00.0: ROM [mem 0x01400000-0x0140ffff pref]: assigned [ 2.009054] pci 0000:01:00.0: BAR 0 [mem 0x01000000-0x011fffff 64bit]: assigned [ 2.017526] mmcblk0boot1: mmc0:0001 Q2J54A 16.0 MiB [ 2.022015] pci 0000:01:00.0: ROM [mem 0x01300000-0x0130ffff pref]: assigned [ 2.032133] mmcblk0rpmb: mmc0:0001 Q2J54A 512 KiB, chardev (242:0) [ 2.036347] pci 0000:00:00.0: PCI bridge to [bus 01-ff] [ 2.036370] pci 0000:00:00.0: bridge window [mem 0x01000000-0x011fffff] [ 2.036384] pci 0000:00:00.0: bridge window [mem 0x01300000-0x013fffff pref] [ 2.042552] pps pps0: new PPS source ptp0 [ 2.048338] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] [ 2.083626] pci_bus 0000:00: resource 5 [mem 0x01000000-0x01efffff] [ 2.089972] pci_bus 0000:01: resource 1 [mem 0x01000000-0x011fffff] [ 2.093461] fec 2188000.ethernet eth0: registered PHC device 0 [ 2.096283] pci_bus 0000:01: resource 2 [mem 0x01300000-0x013fffff pref] [ 2.096352] sysfs: cannot create duplicate filename '/devices/platform/soc/1ffc000.pcie/pci0000:00/0000:00:00.0/resource0' [ 2.096365] CPU: 3 UID: 0 PID: 52 Comm: kworker/u19:2 Not tainted 6.12.0-rc1 #54 [ 2.096381] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree) [ 2.096391] Workqueue: async async_run_entry_fn [ 2.103025] imx_thermal 20c8000.anatop:tempmon: Industrial CPU temperature grade - max:105C critical:100C passive:95C [ 2.108932] [ 2.108940] Call trace: [ 2.108950] unwind_backtrace from show_stack+0x10/0x14 [ 2.121391] clk: Disabling unused clocks [ 2.127423] show_stack from dump_stack_lvl+0x54/0x68 [ 2.127451] dump_stack_lvl from sysfs_warn_dup+0x58/0x64 [ 2.134265] PM: genpd: Disabling unused power domains [ 2.138525] sysfs_warn_dup from sysfs_add_bin_file_mode_ns+0xbc/0xcc [ 2.138547] sysfs_add_bin_file_mode_ns from sysfs_create_bin_file+0xac/0xb4 [ 2.149242] ALSA device list: [ 2.150647] sysfs_create_bin_file from pci_create_resource_files+0x84/0x148 [ 2.153183] No soundcards found. [ 2.158407] pci_create_resource_files from pci_bus_add_device+0x24/0xe4 [ 2.211699] pci_bus_add_device from pci_bus_add_devices+0x2c/0x70 [ 2.217930] pci_bus_add_devices from pci_host_probe+0x7c/0xa4 [ 2.223806] pci_host_probe from dw_pcie_host_init+0x4ec/0x71c [ 2.229682] dw_pcie_host_init from imx_pcie_probe+0x3a8/0x75c [ 2.235559] imx_pcie_probe from platform_probe+0x5c/0xb0 [ 2.241010] platform_probe from really_probe+0xd0/0x3a4 [ 2.246364] really_probe from __driver_probe_device+0x8c/0x1d4 [ 2.252321] __driver_probe_device from driver_probe_device+0x30/0xc0 [ 2.258803] driver_probe_device from __driver_attach_async_helper+0x50/0xd8 [ 2.265892] __driver_attach_async_helper from async_run_entry_fn+0x30/0x144 [ 2.272980] async_run_entry_fn from process_one_work+0x154/0x2dc [ 2.279115] process_one_work from worker_thread+0x250/0x3f0 [ 2.284811] worker_thread from kthread+0x110/0x12c [ 2.289726] kthread from ret_from_fork+0x14/0x28 [ 2.294461] Exception stack(0xe6a0dfb0 to 0xe6a0dff8) [ 2.299535] dfa0: 00000000 00000000 00000000 00000000 [ 2.307740] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 2.315942] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 2.323156] pcieport 0000:00:00.0: PME: Signaling with IRQ 293 [ 2.329645] pcieport 0000:00:00.0: AER: enabled with IRQ 293 [ 2.335553] sysfs: cannot create duplicate filename '/devices/platform/soc/1ffc000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/resource0' [ 2.347794] CPU: 3 UID: 0 PID: 52 Comm: kworker/u19:2 Not tainted 6.12.0-rc1 #54 [ 2.355229] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree) [ 2.361779] Workqueue: async async_run_entry_fn [ 2.366349] Call trace: [ 2.366362] unwind_backtrace from show_stack+0x10/0x14 [ 2.374183] show_stack from dump_stack_lvl+0x54/0x68 [ 2.379273] dump_stack_lvl from sysfs_warn_dup+0x58/0x64 [ 2.384706] sysfs_warn_dup from sysfs_add_bin_file_mode_ns+0xbc/0xcc [ 2.391183] sysfs_add_bin_file_mode_ns from sysfs_create_bin_file+0xac/0xb4 [ 2.398270] sysfs_create_bin_file from pci_create_resource_files+0x84/0x148 [ 2.405360] pci_create_resource_files from pci_bus_add_device+0x24/0xe4 [ 2.412113] pci_bus_add_device from pci_bus_add_devices+0x2c/0x70 [ 2.418334] pci_bus_add_devices from pci_bus_add_devices+0x60/0x70 [ 2.424642] pci_bus_add_devices from pci_host_probe+0x7c/0xa4 [ 2.430511] pci_host_probe from dw_pcie_host_init+0x4ec/0x71c [ 2.436384] dw_pcie_host_init from imx_pcie_probe+0x3a8/0x75c [ 2.442258] imx_pcie_probe from platform_probe+0x5c/0xb0 [ 2.447704] platform_probe from really_probe+0xd0/0x3a4 [ 2.453057] really_probe from __driver_probe_device+0x8c/0x1d4 [ 2.459014] __driver_probe_device from driver_probe_device+0x30/0xc0 [ 2.465493] driver_probe_device from __driver_attach_async_helper+0x50/0xd8 [ 2.472580] __driver_attach_async_helper from async_run_entry_fn+0x30/0x144 [ 2.479666] async_run_entry_fn from process_one_work+0x154/0x2dc [ 2.485800] process_one_work from worker_thread+0x250/0x3f0 [ 2.491494] worker_thread from kthread+0x110/0x12c [ 2.496408] kthread from ret_from_fork+0x14/0x28 [ 2.501140] Exception stack(0xe6a0dfb0 to 0xe6a0dff8) [ 2.506214] dfa0: 00000000 00000000 00000000 00000000 [ 2.514418] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 2.522620] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 This was not the case with kernel v6.11, do you have an idea where this comes from? I did not dig into more detail yet but it looks a bit like a regression. The driver still works, it just prints this duplicate filename warning. Regards, Stefan