From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 009122AE6C for ; Wed, 24 Jul 2024 19:18:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721848721; cv=none; b=O0e6e60pqGhsOydVtrco74E23yQokGwspzLplvTXgvS3TXtdBhL67bsE4tL49jXAKzfosPOML3KzdbaTGHmMPzMYcWBczJjhWjFpfkNbdIPRJ/MLApsFQsjLHfDBIAP1zHjECf5Obh+JK9YOLGXKAymTmaFmXTwhZ+v7oLeWe9E= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721848721; c=relaxed/simple; bh=a02GUhiWbAY4bVUVygn/E4p/MC1dnbGGAe2U0twtjyY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=t64cl8Sk6rC2P46D22+aiJfcQ1nmdFeA559bbADcoWipYS3TqKiHykqthNS7NjsqznrzJaihYNwUc1p8XJgdBy1CXP/BDUE0cEyTQ3Lw7RnrXI9txeqVSXP994tWNarWuEbUZoHlAj6skxcN/0a60df4gxw63Q1FLcii9EJWkkE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=QjBQFpoD; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="QjBQFpoD" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-70d138e06e3so125162b3a.2 for ; Wed, 24 Jul 2024 12:18:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1721848719; x=1722453519; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=a02GUhiWbAY4bVUVygn/E4p/MC1dnbGGAe2U0twtjyY=; b=QjBQFpoDwueoj4zaWBfhpIt1PmvjhP2Y9gai1cprtX9q/W0xgDhg6D1r7Fe0BfRb5T Whvz+oHWbAyt+QN0zkgz2uqn33X5T6cyrbEk8yESXM6lfO0vuo6qCPx/awdTKjqgWXRE QJU70uNzrK0fyWJ1tMFTihcXFcAj0cxZjBwBCYUojvQ5agXyw9jk76WXAS8M+IrPf/3H +k/I9TV/SIcUiM6wXrq0QcU2NkAau0VGiN6HrDOo2C9fsNtcx2BtgasM0sST07zD6Pe4 9f6PHzti41HELRptWhxDVjE9h1npUjjUOMY0YQVLieuILaIAzF6Xfml0dPynXZeZMPXN Z4tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721848719; x=1722453519; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=a02GUhiWbAY4bVUVygn/E4p/MC1dnbGGAe2U0twtjyY=; b=gqoxjwQNOZ66zCPFb3mrJz1qewO4ni7y+R7WwXV6HloAlXo/0PLYWmY6BchLx+mNDG HoycNFyaP/J80VKs8/qdV1Ud2+eGKVwTLSG1/st6ELUN7RGuPmJ0Guoo5vldeQlWp2kR lw9eJPuu/OWuhaBDf0qBF8ZARQoeg1zdpJTj9k7ZiNKSJLCIxD/DV7V0eNyN0z90WZC6 GmS2gSC8J30mB2e8eH+eK9rWcFtpeVDYixeUblCeotxUTteUlJNu30hHU/z4t58Xvmk0 wpOlmpufvYMIW5nrDPI75+GRRGouPr8EZ4iKQY8y4KWOKY3LAAmIkOm06fLfj6cz42Hz yGxQ== X-Forwarded-Encrypted: i=1; AJvYcCWkTTftq0vWJQZtiWWMj508T5SSrSGQpSF2QOTA5GxnwmSE88IJXRnUP+Mx9m4iIWxLsuQI+By8ScH4XeNXoXOX2Sw5awlKaSuSDW1v X-Gm-Message-State: AOJu0Yz3dIGZI5zqGz9n8o0UDIJHQXy8M1wnyC0eTv++ZQ3tnMVDHFVT C17wOl2LzADHtoPY5du10uGraAeUCVqca1g2n6YhEQnRmW9WucxNq5Ee3331KPY= X-Google-Smtp-Source: AGHT+IFUwWJ+8obz221hJG5x/jzoeJXTY4KS7z9HkU5UcBMvPmWxLk9KdioZD32zMoXRgep0BAh/AA== X-Received: by 2002:a05:6a00:3cd5:b0:70d:15b9:3ece with SMTP id d2e1a72fcca58-70eaa946a3bmr578402b3a.29.1721848719142; Wed, 24 Jul 2024 12:18:39 -0700 (PDT) Received: from dev-mattc2.dev.purestorage.com ([208.88.159.128]) by smtp.googlemail.com with ESMTPSA id d2e1a72fcca58-70cff4b8166sm8862385b3a.85.2024.07.24.12.18.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jul 2024 12:18:38 -0700 (PDT) From: Matthew W Carlis To: macro@orcam.me.uk Cc: alex.williamson@redhat.com, bhelgaas@google.com, christophe.leroy@csgroup.eu, davem@davemloft.net, david.abdurachmanov@gmail.com, edumazet@google.com, ilpo.jarvinen@linux.intel.com, kuba@kernel.org, leon@kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-rdma@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, lukas@wunner.de, mahesh@linux.ibm.com, mattc@purestorage.com, mika.westerberg@linux.intel.com, mpe@ellerman.id.au, netdev@vger.kernel.org, npiggin@gmail.com, oohall@gmail.com, pabeni@redhat.com, pali@kernel.org, saeedm@nvidia.com, sr@denx.de, wilson@tuliptree.org Subject: PCI: Work around PCIe link training failures Date: Wed, 24 Jul 2024 13:18:30 -0600 Message-Id: <20240724191830.4807-1-mattc@purestorage.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Sorry for belated response. I wasn't really sure when you first asked & I still only have a 'hand wavy' theory here. I think one thing that is getting us in trouble is when we turn the endpoint device on, then off, wait for a little while then turn it back on. It seems that the port here in this case is forced to Gen1 & there is not any path for the kernel to allow it to try another alternative again without an informed user to write the register. I'm still trying to barter for the time to really deeply dive into this so must apologize if this sounds crazy or couldn't be correct. - Matt