From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753054AbdJSL7c (ORCPT ); Thu, 19 Oct 2017 07:59:32 -0400 Received: from mail-sn1nam01on0043.outbound.protection.outlook.com ([104.47.32.43]:31296 "EHLO NAM01-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751680AbdJSL7a (ORCPT ); Thu, 19 Oct 2017 07:59:30 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Vadim.Lomovtsev@cavium.com; Date: Thu, 19 Oct 2017 04:59:21 -0700 From: Vadim Lomovtsev To: Bjorn Helgaas Cc: bhelgaas@google.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, alex.williamson@redhat.com, Wilson.Snyder@cavium.com, robert.richter@cavium.com, david.daney@cavium.com, mjaggi@caviumnetworks.com, Vadim Lomovtsev Subject: Re: [PATCH v7 0/2] PCI: quirks: Cavium ThunderX ACS quirk update Message-ID: <20171019115921.GA6199@localhost.localdomain> References: <1506536439-29318-1-git-send-email-Vadim.Lomovtsev@caviumnetworks.com> <1508244459-2615-1-git-send-email-Vadim.Lomovtsev@caviumnetworks.com> <20171019112645.GD6332@bhelgaas-glaptop.roam.corp.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171019112645.GD6332@bhelgaas-glaptop.roam.corp.google.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: SN4PR0501CA0100.namprd05.prod.outlook.com (10.167.128.17) To MWHPR07MB3008.namprd07.prod.outlook.com (10.172.96.14) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 22a6d4b2-49cc-4457-e735-08d516e8d8e1 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075);SRVR:MWHPR07MB3008; X-Microsoft-Exchange-Diagnostics: 1;MWHPR07MB3008;3:9osntMTqDgjYGrUye8QHEvVlAkcok2mLQFh9hC1vH6ZjI5Nnsc8C0JL0cpp7wnUpZdwAU22Ds/CsXtf4HERfTJNBVjHM1IEmMbAqkg9dxQzzQuRwLSBSCsE/My56ec87kTqxTJ0+WpoQyebas33NipgSK9FjrNWVHwAP/gxOkv4M9jnKE7g+YUlu3Cv9wIYX9Ie3/DYxeT/cfbwz0J5Kxtlrgk4bmG1MKlcS79qdUsjNoZISM7kRdgFNJ2CodoAQ;25:40LZ54bjA0PJJsXUcAQQ86OxEFZ+A+pNaiUpWWiVH951rl2WratNy4mBlfh+wi19M9OVjyf+TVtoaOXXsT6ECrm87nR03bl8YIEmNS+Ep/d4mM5qrkchTJH/TC53eereCC6GxRaiPOD4w+I2x9UJ14N+pkvvQdeXi/iT6KSHnsMLtoqJ9yUsW+hWfHjau6vPs6sG3L+ISt7IacMUeS6IgO1kpNdmZJXmMqIH0mbBddAYhnoRH7wSYKDzl5P/iuOOrCPbUTwh/AuydHtI0JMuauGN+ZgnC96SFsIj7wjpn1DOAN/eBQiBl9ErUDa8DVLejBmy/lnSObGY49D/j8yeYdlbn+QTBBcRPE3HPzFTXhY=;31:9utMAyVC49Eb9yrrzcCnxGmeNE2/q14BIuedm66gsTLfnqo3xRchcNQxbrp0dp7ZWf3uTc8dFXNu58tUS18fENYXnNCJ5VZioGbi24XhuN5FTbIIBczlxkmJB4ejzNM1wKCU9uTib7gqAOmftiqXZ8RNUeqBkW/pkU8FvwCISMZwZ6FmvoeS6IVK0sj/TvIj7Am2I7t2F/xOYLLyDFuVkVAXjUNANHHbGAxBaLepi3U= X-MS-TrafficTypeDiagnostic: MWHPR07MB3008: X-Microsoft-Exchange-Diagnostics: 1;MWHPR07MB3008;20:cHC3cDjwsKMBE7QGZK+aiec9B/VQwf66pMC5yI3s+wdY/zNlwgcI/xfEr2P2Il/4mn/QnDWTWCBAyBVNpB+alk+F/I152u4E/A2YJFXMREQMdZyGV17mFJTNbvwbmDD/XO9s0387l44So/Jv64pSi7Wh1PWwOTCdGFUPgMiuNGTa7UKsWPwKUx4zRnD4vgWKIBh+Is7gowm0UrzQtz1YsF3XBbnHrcDdLO9Sk4XYRGO/OjITrQGiT4I5bSIy4CFsalhECm/NFn4bmMcWE+Tot0xIGM4J2pt3DBOpDsLcn5/5ZgF0z5yiX4XHSi6+nyNtgWEO3joqQ/UZGzIAmHNrrKTjgRPDxcbsdWCMmrcNdGBozyx5AF47AJBiwkxb1bHYMFRujz6U1wuUwW2E+w/bN6EbtktJnndKKY5fnmS0jzrpdSbeCbakjAPc7PbK6h1ux97ZG+NMy+4d3MGcfHFCZnJsB4fngIOvya1IYFtIcp2kyVhz+119sa+IZDx1byHQwmDVSR4tznT95d69p3ETzOxT1/KBzZnWlQWPuXIYS3EenwRuqiyD+AiFrJVhGbj9SgJ2G5omfAVfd4Yibf48H7V1S05NlSzNxv3QynTxb60= X-Exchange-Antispam-Report-Test: UriScan:(788757137089)(211936372134217)(153496737603132); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(100000703101)(100105400095)(3002001)(93006095)(10201501046)(920507026)(6041248)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123560025)(20161123555025)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:MWHPR07MB3008;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:MWHPR07MB3008; X-Microsoft-Exchange-Diagnostics: 1;MWHPR07MB3008;4:Gc2B28zGQbO6Qx1nie5wJnwIVYCIMfeaimrAseP6SqUaEfu0VtdxaVAoHQ6iLkpT7U5xBWivw8fqD21lIv+IYFWMcXdT8dqVXCpT7vOvAhneXVrtVvtsefD8/xnq6+uI9ABtuthwYMdx9pwPwaOZKpgHndliAbXnV179FO+6iw7w9uviaS3poLz76jv5DLerAtKD6ZpklD9Jt8Wo1fICq3cyd3gKXBp22NjNzxBbYw+9hGNMM21dBt+FLFim9z5G+ymde6O7HwrlUD+ytrfj4MgLzTcHsjkxdLgdBs3B1ObwpJv3hbn76MnOUnasgkNaH1y9SOrQ3a71lvw0Rlz6339yYuzzmRyE63OVQlyJY34= X-Forefront-PRVS: 0465429B7F X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6009001)(6069001)(346002)(376002)(189002)(24454002)(199003)(52314003)(54534003)(54356999)(50466002)(478600001)(33656002)(97736004)(42882006)(575784001)(61506002)(316002)(15650500001)(2950100002)(66066001)(81156014)(8676002)(8936002)(6116002)(16586007)(6916009)(9686003)(81166006)(2906002)(72206003)(1076002)(3846002)(47776003)(23726003)(53936002)(83506001)(6506006)(107886003)(305945005)(105586002)(34040400001)(55016002)(5660300001)(76176999)(106356001)(58126008)(7736002)(4326008)(25786009)(50986999)(189998001)(229853002)(6666003)(101416001)(16526018)(68736007)(6246003)(18370500001)(19627235001);DIR:OUT;SFP:1101;SCL:1;SRVR:MWHPR07MB3008;H:localhost.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;MWHPR07MB3008;23:xn98hA+fFY5jYDCW6+P6tw65ojL1Z94V/tskDBSbD?= =?us-ascii?Q?q/J2EQKvAut6YSBif4aiO3P7PuBrM78B0+G/erzkHHIrBpvj35CVrB1/bXzs?= =?us-ascii?Q?9cX6Mmn760FqCowSvWNq+X9yJtDRlX26BnvMyGFKzduZi62jAohoP1vy0xOy?= =?us-ascii?Q?d4iYy3ZRmfiDhoZLzVCIbhJQoxz6OsJh+7evqYlcherWL7Dteivux+CcaYLR?= =?us-ascii?Q?2qwlMX/tM9qWgUzYZi2CHQ/0RhfxVRW6iwsCVq89Jt75KpvqJq0+mxZhTXGj?= =?us-ascii?Q?W5yOE+nOkD0uEeB2WCsBp/gjwdBetbOR7SLV9VGAIUH/e9RYOclg2Kd9XO1Q?= =?us-ascii?Q?Q3/RT9MU7nJ6KDHWpvVvC7vceKvZxRaMJQK03LAGGoH84ltlc3CSI3z6Ysza?= =?us-ascii?Q?M8axN8KM4cdXd/3xKdSmzun4oGq1BEpElSIRSATPOGs18F8Va4TtKsLIPHBw?= =?us-ascii?Q?/G8BTIalRumYxAU+CWlV7qs75PfgYXJKNssbanrsajoIxW15BwjBaI3AR8WZ?= =?us-ascii?Q?lXJNKvsnCm/NwJoW4o0/lzjrIYcjhVvBTQ9RPrIj95A+cjktHc9h71MIOzdR?= =?us-ascii?Q?Ih1R++ywcSY+wsnEVcqcljzbRgfILC7Mb63zOQXYMndcaDLCKn2p2atw6TED?= =?us-ascii?Q?4POg8EZR4w+k8coXnVJOQG+uhwcVdUrpcHhjdxB0IIQd/DlxgpUooCNabm3n?= =?us-ascii?Q?cRpU10srMKy0Iwgzx1e9YsDDq110SIozpJVr+bX4zm3G8TEqcoNMwEUQx8E8?= =?us-ascii?Q?ZQg+Fmf4RPo64DhWIy9+L1X4YRiV40XVoUdrdNByX+9lflhAmUxekoyFKXVG?= =?us-ascii?Q?DE7r7XPan+Xrkj97n/cc79lJiQX/7bPykGNoU9c9Kc1g0naSLHq0gXPRfYXa?= =?us-ascii?Q?V6eHJcyakOnbyViHdeEG7R3w+s+WqhCfORQhLUyioEcw2VWT1Df6QatX9IlN?= =?us-ascii?Q?peSZOlC2p3FocQvyyEEZEgrrZBMjNa4Us98CeHJCmLam8/OQdO69RAfqEwmJ?= =?us-ascii?Q?b1BQD2o7CdiEpr6M0CwORd4zFrek/dvb/yWYyj9y2kmhrb/QQhnQDvFzJpIV?= =?us-ascii?Q?DyiuNF1MTpobfQ+4b5yJJ28jOnDpksBHYhUrXRa8SG6V0iY/nBB3jDhpN/63?= =?us-ascii?Q?nRMi9oylbIadHpTq9H8EzQqJg8oIqltE3uZMVEK057YyC1WOBjRYWXFi30nL?= =?us-ascii?Q?CqoulnpGTPHQ1/mkzl33EnjirxFq93V2jlnnCHjJNxDtCoiyqw3bOfOqp+N7?= =?us-ascii?Q?cnaXWLAbDRZIM6hsuo2Jb751G9z4ECNhaeQm5nTu+BqrwpTMeoGCGgyL/9QE?= =?us-ascii?Q?fIByKFmEG+oL42CnKRzdZk2lIXIMN5Cn+hlvjDqk+VzQcmnRC2URE1Fkd/MO?= =?us-ascii?Q?hoAzZwDO8mxzP2EK9kE3HWLn45FUowJcso24sIv5Fs2ued/?= X-Microsoft-Exchange-Diagnostics: 1;MWHPR07MB3008;6:t23iTLh25HJnB3ay19fzve/vks/LuEu85l9FdrPHE8PKDpjXMegVktgA9DrXqowPLFlXCYe9UK0/sL+qyMQsP8XVo7hBMjOvciwrETTW9qbCXCoJclH3S6fZQL/x+ARmklxd+D4Dn0JX6PW659P2Ari8lNiDgNK2+3t60aLalKIQbqtRp6aQ9OW91ig9cQW4pASCYm5HwvQHjNaqVj0HOTDm0k+H/HOVo0viEpYSs1jiMC7umhmlSrEqYBe7yhHu16yarYo4sA31cugL59tTJhrkHZ4dPXeiS0J6OFuzwykhxVle5lywJ0dnNLixM36kAm1bi3J32Zt99ZlClCgQVQ==;5:WdMWVIcHhS5ayNML6rh2DGrH+SQf5ZhK0eKWNW9fYivo4baSU6ryiUi3ZU5fbywlx/bgKHXWsHDPe9RRl5IbE842hVfECbnPWLiE5ZmvR7q+iWxCntmahewhbwC9Hf/x2UUocGiTWiDAlZBDyZDNCg==;24:07Tl3WBsOTehXOibxau6LAtwKLOOwAOYd70viLhzU+c+UovtVYD/3FEaBOZDUPokBCb6P+bRY2yYNk/+ePCa9ZoQEmaIBXM6jkY3LY1hW2o=;7:KoTKnuUwDWeG1bwqCB9hkmBuoiVgeF5MOUG6ifwZTUW/hg3QmYfARDafxH4ixVNvr0vDnuoTM+GCJoZJ6WPNwB1yavt71jlAhtQSTlU8bEDij6QpJOvK/dKmNG7oRtwSTF7+l1f1vt0j5H7D5oDbarjJ8c1AoB/gj4Md+zqKk0vnM5uB5qjkaiFUSufWWd1B8Q/8TQJA+8zQKqNf3ifmH5oCPEU4kMxTZudJklmS9gM= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Oct 2017 11:59:25.9231 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR07MB3008 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Oct 19, 2017 at 06:26:45AM -0500, Bjorn Helgaas wrote: > On Tue, Oct 17, 2017 at 05:47:37AM -0700, Vadim Lomovtsev wrote: > > From: Vadim Lomovtsev > > > > version 7: > > - update patch description accordingly to review comments; > > - split for two patches: for ACS mask change and device id match change; > > - remove macro #define of ACS flags, put it back to quirk function; > > - remove '__inline_' from device id matching function; > > - wrap code comments to fit into 80 columns; > > - comment fix: change 0xa00 to 0xf800 for matching function description; > > > > version 6: > > - comment typo fix: change 0xa00 to 0xa000; > > > > version 5: > > - update code comments accordingly to review comments; > > > > version 4: > > - update ACS mask (remove TB and TD bits), update commit message (remove > > ACS register printout); > > > > version 3: > > - update subject: remove CN8XXX from subject line, replace it with ThunderX; > > > > version 2: > > - update match function in order to filter only ThunderX devices by device > > ids to properly filter CN8XXX devices, update subject & description with > > ACS register info (v2 was rejected by maillist due to triple X in subject); > > > > Vadim Lomovtsev (2): > > PCI: quirks: Set Cavium ACS capability quirk flags to assert > > RR/CR/SV/UF. > > PCI: quirks: Apply Cavium ThunderX ACS quirk only to Root Ports > > > > drivers/pci/quirks.c | 26 ++++++++++++++++++++------ > > 1 file changed, 20 insertions(+), 6 deletions(-) > > If I'm reading this correctly, the first patch is basically fixing a bug in > the original Cavium ACS quirk (b404bcfbf035 ("PCI: Add ACS quirk for all > Cavium devices")). Right? Yes, first patch fixes improper ACS flags in the patch you mentioned. The second one fixes devid check from another patch (b77d537d00d0 PCI: Apply Cavium ACS quirk only to CN81xx/CN83xx/CN88xx devices). > > I put them on pci/virtualization for v4.15 as follows (patches unchanged, > changelogs wordsmithed). > Good, thank you. > You mentioned stable backports. b404bcfbf035 ("PCI: Add ACS quirk for all > Cavium devices") appeared in v4.6. b77d537d00d0 ("PCI: Apply Cavium ACS > quirk only to CN81xx/CN83xx/CN88xx devices") appeared in v4.12. I assume > you would ideally want all this backported as far as v4.6? We'd have to > figure out how to express that in stable tags. Ideally yes, I think we would need to have them backported onto stable(s), but so far I didn't see any bug-reports which could require these fixes. Anyway, in term of backporting them - are there anything I can help with ? WBR, Vadim > > > commit 60f6d5f7ebc4bba83d73115f58b805f2f2618667 > Author: Vadim Lomovtsev > Date: Tue Oct 17 05:47:38 2017 -0700 > > PCI: Set Cavium ACS capability quirk flags to assert RR/CR/SV/UF > > The Cavium ThunderX (CN8XXX) family of PCIe Root Ports does not advertise > an ACS capability. However, the RTL internally implements similar > protection as if ACS had Request Redirection, Completion Redirection, > Source Validation, and Upstream Forwarding features enabled. > > Change Cavium ACS capabilities quirk flags accordingly. > > Fixes: b404bcfbf035 ("PCI: Add ACS quirk for all Cavium devices") > Signed-off-by: Vadim Lomovtsev > Signed-off-by: Bjorn Helgaas > > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c > index a4d33619a7bb..c23650cfd5a1 100644 > --- a/drivers/pci/quirks.c > +++ b/drivers/pci/quirks.c > @@ -4214,12 +4214,14 @@ static int pci_quirk_amd_sb_acs(struct pci_dev *dev, u16 acs_flags) > static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags) > { > /* > - * Cavium devices matching this quirk do not perform peer-to-peer > - * with other functions, allowing masking out these bits as if they > - * were unimplemented in the ACS capability. > + * Cavium root ports don't advertise an ACS capability. However, > + * the RTL internally implements similar protection as if ACS had > + * Request Redirection, Completion Redirection, Source Validation, > + * and Upstream Forwarding features enabled. Assert that the > + * hardware implements and enables equivalent ACS functionality for > + * these flags. > */ > - acs_flags &= ~(PCI_ACS_SV | PCI_ACS_TB | PCI_ACS_RR | > - PCI_ACS_CR | PCI_ACS_UF | PCI_ACS_DT); > + acs_flags &= ~(PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_SV | PCI_ACS_UF); > > if (!((dev->device >= 0xa000) && (dev->device <= 0xa0ff))) > return -ENOTTY; > > commit 35da518fcad24f7be515976238a4667b5ccd1711 > Author: Vadim Lomovtsev > Date: Tue Oct 17 05:47:39 2017 -0700 > > PCI: Apply Cavium ThunderX ACS quirk to more Root Ports > > Extend the Cavium ThunderX ACS quirk to cover more device IDs and restrict > it to only Root Ports. > > Signed-off-by: Vadim Lomovtsev > [bhelgaas: changelog] > Signed-off-by: Bjorn Helgaas > > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c > index c23650cfd5a1..0e22cce05742 100644 > --- a/drivers/pci/quirks.c > +++ b/drivers/pci/quirks.c > @@ -4211,6 +4211,19 @@ static int pci_quirk_amd_sb_acs(struct pci_dev *dev, u16 acs_flags) > #endif > } > > +static bool pci_quirk_cavium_acs_match(struct pci_dev *dev) > +{ > + /* > + * Effectively selects all downstream ports for whole ThunderX 1 > + * family by 0xf800 mask (which represents 8 SoCs), while the lower > + * bits of device ID are used to indicate which subdevice is used > + * within the SoC. > + */ > + return (pci_is_pcie(dev) && > + (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) && > + ((dev->device & 0xf800) == 0xa000)); > +} > + > static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags) > { > /* > @@ -4223,7 +4236,7 @@ static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags) > */ > acs_flags &= ~(PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_SV | PCI_ACS_UF); > > - if (!((dev->device >= 0xa000) && (dev->device <= 0xa0ff))) > + if (!pci_quirk_cavium_acs_match(dev)) > return -ENOTTY; > > return acs_flags ? 0 : 1;