From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030302AbcHaQKK (ORCPT ); Wed, 31 Aug 2016 12:10:10 -0400 Received: from mail-by2nam01on0082.outbound.protection.outlook.com ([104.47.34.82]:38098 "EHLO NAM01-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S934855AbcHaQKI (ORCPT ); Wed, 31 Aug 2016 12:10:08 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=David.Daney@cavium.com; Message-ID: <57C7015A.2090509@caviumnetworks.com> Date: Wed, 31 Aug 2016 09:10:02 -0700 From: David Daney User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130625 Thunderbird/17.0.7 MIME-Version: 1.0 To: Aaro Koskinen CC: Ed Swierk , driverdev-devel , Greg Kroah-Hartman , lkml Subject: Re: [PATCH 0/9] staging: octeon: multi rx group (queue) support References: <20160830184753.21006-1-aaro.koskinen@iki.fi> <20160831150655.GF14316@raspberrypi.musicnaut.iki.fi> In-Reply-To: <20160831150655.GF14316@raspberrypi.musicnaut.iki.fi> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: MWHPR07CA0013.namprd07.prod.outlook.com (10.172.94.23) To CY1PR07MB2135.namprd07.prod.outlook.com (10.164.112.13) X-MS-Office365-Filtering-Correlation-Id: a22fe58c-cf1d-4057-30f9-08d3d1b94523 X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2135;2:x13cP40DM+810UG51fiJuoOj+J3HwVYnrD/SreGbvBcG4/EgEXoXo0AFbyHq5Aw54g9+ctP/oonHxYA2pDXDl5BUd9JK56U6y3pOA3GVzljiWSe1ipfR66hrHwopK/FuaKqOs0ku6MBmsT02p3vfJ63iv8ItZPbDhynWFB6AF540InIVuerl+wFBZkA+ai4Z;3:d/OaO39y8EprF4beQfaRn7fZisZiIF1g4yvUyW2mIrbvcBRBGQcoRTXBWQs/2awM9G/kfTKWhk4EbdqPDSXfdAgTRSNO4vLtPV594FoNjKV3p9Ul2lKyl+XnnLvCLfEr X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR07MB2135; X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2135;25:Hf1kUPne84qOwm38k6aLJ7TtyI2HX6mSH+7W5/QZxNd/V1VKuWHV+gNTKEGwdZOAyrihHwyqSwqFkPadX+YYqd9dx1ur1b7o3qjb2ZWq4gxWVA9scOxvKygAI+b4B/DCkxP5ZphWw4d0t0O1rUTE+k/tXUHfsGtjr49XXEk+9bCGZPANEB87RuRK8RJoc3h0BSul3aXa2yPwovWHQ7p9x1jPMWbKyh2GxkxroRKpMq2ivVXtEiC0rh/CVfIlMbr0IMHpFy+wgSV+o5YIvUdMjq1YySaJggnTXLgtKmyBXH4diWWnmxPwvCChDWBTsWPgdzzn/r9GzVuNgs8GlWMc8LFo0iJ5LhywwPMSJ+z63YQnfn+4wmUcUMiRfFDFnn7gTPOsHEOSrQucKSpZuZWUdqgm15QGeuCsZsIi9RcKg0i3speur7C9Z0zw3wo0+YMJFp8jxYuVIUBmBCABrCzT8Gt37Igkk7TWSFIwHR1slb2xsbPIz+rsRugDzZBbhkKBftSS8jngcnEoNPkSME/DppV2N72Bh7IQaJNIPC6y0lElHkSLJUeWitoHJv66nBkUY05bq0K7rY7pbolF9i4m+7H4yn/XhKB7PBMK3sTl0VgRlisrMrnvKZvrL0Qi2XhUwqzy7MXpXznNT4EHv4KwPX09TZXybZX7cfepdHqxHdvjPp5NgJbfCZvmPLBsfR9hN9NITnaX9z1bm/cQYqDKEpKX4OOWOOldVmtoUi639KlUymPzQgB7iwryfjt/5ai30loqpW48qQb7F2XdBkTyzg== X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2135;31:mCuARGOSoZTGS7hAf9+RduUwKCWpAnjoXjqbUuYbl0ymMrQUuh1E+JwyvY0cOSCNuRj1+AGvpA3oHQJRZ8FxRvQnyOwOxwnCZq8oWfR6GYtmUlCJ0PxqJVKxEgVQC6sTS/TqNjdiNhRIEt+OWDCvkcbxccCQ18TNAwZP2+EytCCgOMRqzhtFapSf5u/viRBuPwqFaDvULLfM2fj17DGP1FP/qQ9G5OLFaRfjPlWIuUs=;20:XIBNd3HRXXVgIxputFCgf3y0Vb4R24tdF92IDPY7ltdvJxSQAunhkefkRuu13WxfiGsdeHySSwQbf/eEVVEIiv91GK18VAUYlIFUbvTvrjjED4GfNAQ1CrS1KeVj9o7He7p4t+r9cU7ZVVy19qrc8VKc8qmNi7GGT0+g1J7nSsGIFFfzxYaPIwo2TeNEGglvTmfYf9TM2F4U6IdkdUminG34vHWZnXg9GrJe+pBZbAW9KDC93H2/e91rvXL5KozbSeu/JsOikg7+INxTwuPA/ibN+kascB/wDvkyel4RuKdH9ruAQWp080QJhnz/WZR4YXarnPGAs4GM5K2HJGRNSc+9WSF36A1kIaMCpUcYa+ygea2ESrgT28n3M++dhJ6FYsDkd+IRVomVp9kqeCYwEBhEaGTCCkchXNyargEzSmqqvfTlnbsR3r3gZxeSUIndfE/g+Xok3ohwS5/VPnVIP+D2ol0pJAGJMUdF2XK1wrVqkjH9Z1MFuhSGB1rxAB7yUKKanh8lCwXLvALG2W4G3j0jY34X/ONNmGn/jPPntwywsWBn0Wk7iSK8wQE/sNeEa6sGeGClpMRPpQnBPvavtBg57E/skTftPTkdGtYakgY= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046);SRVR:CY1PR07MB2135;BCL:0;PCL:0;RULEID:;SRVR:CY1PR07MB2135; X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2135;4:aFCQCepkK5NEUUH5xzbcTRAs/fbs7DD1KAGpVUdTHrBZcAUEQH8nD1+Inl9aGLmQAmJJX7rlOvbuiDahDJXlZ7ueWOOXM3FBBiKM6w2jMAPpGI+zHwYSHuYC2jMxx5A9YsbbQfYoxJNkhZfaA3JYifRFAWnJwAZwnLob4GPhbr8QciBNaDllKohtKCsl9GM6zbpfoxc4ofBpY0u6YS47eUVLCCGetghB9cfdejqUCn/fcPwme77d2XhP2tA0td3+MFkUWFKQxk7kac/32eEs65jbMcXiokNRHzbmR1xa1DgSGwz/gyMXw2B1z55e/pLsNajWrfMVY9D5TCAQ2r8SkheIke4YPpzEChpw0x+NTCdxIPUd/Dtqict8Gv/3gpY5yl8tI0NvG3HU9dTduLFfgg== X-Forefront-PRVS: 00514A2FE6 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(24454002)(199003)(189002)(377454003)(230700001)(6116002)(65806001)(68736007)(3846002)(64126003)(69596002)(36756003)(87266999)(76176999)(50986999)(586003)(54356999)(19580395003)(2950100001)(4326007)(65816999)(50466002)(5660300001)(7736002)(33656002)(81156014)(105586002)(47776003)(77096005)(66066001)(65956001)(110136002)(83506001)(81166006)(2906002)(23756003)(59896002)(42186005)(4001350100001)(106356001)(53416004)(92566002)(7846002)(101416001)(8676002)(189998001)(97736004)(305945005)(80316001)(19580405001);DIR:OUT;SFP:1101;SCL:1;SRVR:CY1PR07MB2135;H:dl.caveonetworks.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;CY1PR07MB2135;23:quLsWSsybqGTz7IVGah5GMY1YlkYqRafEVrnM1O?= =?iso-8859-1?Q?JpJnqkicIxhiW9DBIlzYVRHjPUQYb49UXtUk7u77hLl+rhUz7VxbXjr+cB?= =?iso-8859-1?Q?GXTzJEmFP8Tpt9r2SiwtM5Y4dowBhLIvlHSvAu15jnZstQll6j5GF5h16/?= =?iso-8859-1?Q?DtqmmRr5nBGCPFA4X2VwA3H9qTAXQey39J4JZjvgxByVOmMOJK8NgPuNQM?= =?iso-8859-1?Q?xPybLNb6RpoqIc5IE/qh4DNKSTary8blxvs/FdANL2P4mtE3ihoSs6PqUI?= =?iso-8859-1?Q?8qqOrvSFCKP1BRKvJ0MV/eY8XxAS/2U0ilQRTcJG8VoWapaRNxajNbHbEv?= =?iso-8859-1?Q?W1CpLeB2BBTFPdGP1pmsLiUJEd+eXX8/nQDIT+4itUMwpAQPpgAbFJFA07?= =?iso-8859-1?Q?+kjq2oGh7P8jDErR0LTYJ5ZngMORcLZTwSpQIR/HPOsySzVrzGG7/GzbP7?= =?iso-8859-1?Q?MI6b1Ud4J3j2ZTLSh7hXfN0fUFQVwNtkSRPFhU63d8UXBlnJ4xZDp8wwJm?= =?iso-8859-1?Q?YZeedqrx3vEJzczCSYcM9DysiyL1Q0jOsDa5xv9UfV4qNzgfxWcJRfgdw7?= =?iso-8859-1?Q?rU8b5pr/Hmudo8QB4Boal6zLPXaBuaLhJ6rdJMtt+VO2IlM5dxhw3F1j2V?= =?iso-8859-1?Q?7yVzShNkaklm8sp1cPK4h1zc1dkmt9IUadQRXZE4qV9eqInbkNRZaxgNKq?= =?iso-8859-1?Q?9d+mcKnfldKlps1UXlEO54qgcsdfA8mtDCXYfwT9UZwff3nfjZ75SrAEmq?= =?iso-8859-1?Q?gyG9pSF6s/3jYvmtTz7Trkd36oMNj16mUXkOgJrnwYYgS5UMsVrY3Fx245?= =?iso-8859-1?Q?oUAOtkJgbSVEotqz+kwNu2VQU+yrkC6bSDg2ARHiKG3/MqpAjOjndkIyGt?= =?iso-8859-1?Q?ARHGfyM3nN8q+yP0q0D54S49E3kaIOozVpYH5/KN2k5T+MyoTaGtV3AfK0?= =?iso-8859-1?Q?vbdyXw40Su/+4KsEWEFODxX8UuXYJz1MVgdkXBWCxCa3NmzZIYx1sOl+Vp?= =?iso-8859-1?Q?E4R2ewlwApf57EsIMXoZe/VKmTFnPKQXj2b3qj/w2Cp12z3PRt1WaX7xMy?= =?iso-8859-1?Q?DZNVGs/7e/wBiuLDQt82lmK8pgoj2fJr0gVclhndYgSE2Ie6U8xb+PLBDZ?= =?iso-8859-1?Q?2+rY+6Pdzn5M71ZoYyt2N8bbN1BZagSdKOKBuO/pssAdb/a+tz5rUnclfl?= =?iso-8859-1?Q?XKDI0G3txfTAX10mALY3tbU6mkMOoY3Nm0l3as2O/oaOFGRUfdfyGVzXSF?= =?iso-8859-1?Q?6n3PtVeSXd53E95YMtF5xVPIzaHcMq8vydGs738ujSC2Uxn94rj6lnNMTH?= =?iso-8859-1?Q?hF3wMFtEz/Zd9slRifjFX15rsKAnDIsLEdRwAeOw3Dtfqy8zawqxEvFHTd?= =?iso-8859-1?Q?PY1GtPNk=3D?= X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2135;6:tzvj6Ftw15pvs6POLr0vFmnsc1hTRpgeSniA2uEneYtY2jKE+ikpSMbnXpRj7Xt9ZHP50FeuLeY5zENNzLHsooUE1qlZFaM+09DY628xSWWzdeMIZw3UqgPnQFM5JPTelNDZC1Girgabfy0WGSGx2dqjEjhUdMUm+MsnsIT/RsEEtR14D+nB8fl4tldzuQ7R1nQKM6fNiQJtQpuzNkxFgq5nSdKSSTcWFip0e2sLqp8S1xMrBM9xXKYH7Ai9ueHGR4DxwM/l+blWB88U1nNAvOoYjNQllfXsUBiOXUSnLIk=;5:XxTib9jgMzinWQfXUKvt6gfzNwC3vJDiW78nLnL1tZG9P9wQRtZZxO/6bovN3YgBxh0RjIndZ2NyDCzer9BqaeldRnTDl9e9nMaueSugU2VsPWU8uBG6hKySiTL0nxI8sWyCkUzEE6gUOhTCtZQAUQ==;24:WhbKmxBFKeQbV57/coBVxGTRWRacNjIKNkyIhXo86RV1/kCr24vQiOm0TG4H7430IaXqOaPspk8Qb4nVsOpYLsgZgQfk+wVmn2opaVxE3pQ=;7:565BEZZp4vexO4FrBBLXVPxxjqa60D9QAM6Je2WD9FTmDb+U9jYf4HFZLDMR2XYTHGr5ZIy+IpIQz7DqsljddxJetLXBOrKwwmTNt/DjyEn/Cryj2ny4rt7wD+errfreyS/VXA2vW8zc9rLPut02chAmxM79uq4bCDjRqjthBx4K9ad/cAPkz/UVhQroLMOoopCDF7TMp66l94ziGpU1SGVTEZ4R9FsmFEadZ8M5reyKlvwSW34Twm7A+CSj/Ckn SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2016 16:10:04.4828 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR07MB2135 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 08/31/2016 08:06 AM, Aaro Koskinen wrote: > Hi, > > On Tue, Aug 30, 2016 at 06:12:17PM -0700, Ed Swierk wrote: >> On Tue, Aug 30, 2016 at 11:47 AM, Aaro Koskinen wrote: >>> This series implements multiple RX group support that should improve >>> the networking performance on multi-core OCTEONs. Basically we register >>> IRQ and NAPI for each group, and ask the HW to select the group for >>> the incoming packets based on hash. >>> >>> Tested on EdgeRouter Lite with a simple forwarding test using two flows >>> and 16 RX groups distributed between two cores - the routing throughput >>> is roughly doubled. >> >> I applied the series to my 4.4.19 tree, which involved backporting a >> bunch of other patches from master, most of them trivial. >> >> When I test it on a Cavium Octeon 2 (CN6880) board, I get an immediate >> crash (bus error) in the netif_receive_skb() call from cvm_oct_poll(). >> Replacing the rx_group argument to cvm_oct_poll() with int group, and >> dereferencing rx_group->group in the caller (cvm_oct_napi_poll()) >> instead makes the crash disappear. Apparently there's some race in >> dereferencing rx_group from within cvm_oct_poll(). >> >> With this workaround in place, I can send and receive on XAUI >> interfaces, but don't see any performance improvement. I'm guessing I >> need to set receive_group_order > 0. But any value between 1 and 4 >> seems to break rx altogether. When I ping another host I see both >> request and response on the wire, and the interface counters increase, >> but the response doesn't make it back to ping. > > This happens only on CN68XX, and I found the root cause. > > I will send a new series later today. Thanks, cn68xx does have a slightly different SSO and interrupt implementation, so it will probably need some special case code to handle both styles of hardware. David. > > A. >