From mboxrd@z Thu Jan 1 00:00:00 1970 From: Suravee Suthikulpanit Subject: Re: [PATCH V4 5/6] perf/amd/iommu: Enable support for multiple IOMMUs Date: Tue, 23 Feb 2016 16:56:22 +0700 Message-ID: <56CC2CC6.6090803@amd.com> References: <1455182127-17551-1-git-send-email-Suravee.Suthikulpanit@amd.com> <1455182127-17551-6-git-send-email-Suravee.Suthikulpanit@amd.com> <20160218131853.GU6357@twins.programming.kicks-ass.net> <56CAC01F.8090800@amd.com> <20160222140741.GH6357@twins.programming.kicks-ass.net> <56CBEA4A.8070001@amd.com> <20160222222457.2824f7e0@t450s.home> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20160222222457.2824f7e0-1yVPhWWZRC1BDLzU/O5InQ@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Alex Williamson Cc: Peter Zijlstra , iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, acme-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, andihartmann-KuiJ5kEpwI6ELgA04lAiVw@public.gmane.org, mingo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, bp-Gina5bIWoIWzQB+pC5nmwQ@public.gmane.org List-Id: iommu@lists.linux-foundation.org On 02/23/2016 12:24 PM, Alex Williamson wrote: > On Tue, 23 Feb 2016 12:12:42 +0700 > Suravee Suthikulpanit wrote: > >> Hi >> >> On 02/22/2016 09:07 PM, Peter Zijlstra wrote: >>> On Mon, Feb 22, 2016 at 03:00:31PM +0700, Suravee Suthikulpanit wrote: >>>>> So I really don't have time to review new muck while I'm hunting perf >>>>> core fail, but Boris made me look at this. >>>>> >>>>> This is crazy, if you have multiple IOMMUs then create an event per >>>>> IOMMU, do _NOT_ fold them all into a single event. >>>> >>>> These are system-wide events, which are programmed on every IOMMU the same >>>> way. I am not sure what you meant by creating an event per IOMMU. Do you >>>> mean I should create internal per-IOMMU struct perf_event for each event? >>> >>> No, I meant to expose each IOMMU individually to userspace, as a >>> separate device. >>> >>> Is there never a case to profile just one of the IOMMUs ? >>> >> >> I see. That's definitely doable and simpler to implement. >> >> I was not sure if making users specify the IOMMU instance (e.g. >> amd_iommu_0/ , amd_iommu_1/, ....) would be too >> tedious. However, this would actually give users better control of the >> performance events, which is a good trade-off. I think it is acceptable. >> >> I'll make the change and send this out in V5. > > We already expose individual IOMMU hardware units in /sys/class/iommu/, > you might consider trying to match the names there for the convenience > of the user. Looks like we use ivhd%d for AMD. Thanks, > > Alex > Hm, the PMU for AMD IOMMU is currently shown as /sys/device/amd_iommu. I am not sure if /sys/device/ivhd[0|1|...] would be obvious to users that IVHD is really the AMD IOMMU. Suravee From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751982AbcBWJ4q (ORCPT ); Tue, 23 Feb 2016 04:56:46 -0500 Received: from mail-by2on0056.outbound.protection.outlook.com ([207.46.100.56]:12096 "EHLO na01-by2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751616AbcBWJ4n (ORCPT ); Tue, 23 Feb 2016 04:56:43 -0500 Authentication-Results: alien8.de; dkim=none (message not signed) header.d=none;alien8.de; dmarc=none action=none header.from=amd.com; Subject: Re: [PATCH V4 5/6] perf/amd/iommu: Enable support for multiple IOMMUs To: Alex Williamson References: <1455182127-17551-1-git-send-email-Suravee.Suthikulpanit@amd.com> <1455182127-17551-6-git-send-email-Suravee.Suthikulpanit@amd.com> <20160218131853.GU6357@twins.programming.kicks-ass.net> <56CAC01F.8090800@amd.com> <20160222140741.GH6357@twins.programming.kicks-ass.net> <56CBEA4A.8070001@amd.com> <20160222222457.2824f7e0@t450s.home> CC: Peter Zijlstra , , , , , , From: Suravee Suthikulpanit Message-ID: <56CC2CC6.6090803@amd.com> Date: Tue, 23 Feb 2016 16:56:22 +0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: <20160222222457.2824f7e0@t450s.home> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [124.121.8.20] X-ClientProxiedBy: KL1PR02CA0024.apcprd02.prod.outlook.com (25.165.15.34) To SN1PR12MB0448.namprd12.prod.outlook.com (25.162.105.141) X-MS-Office365-Filtering-Correlation-Id: 82ca32e3-3b7e-4ab3-925c-08d33c37a0d3 X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0448;2:o2vjjoerQ29itDoGIU3BOM63wVx3OqTweEd9lnekZt0L+sR4x/DQHjF3x4GgOpl5RplF/p0QSHtqTa57GHbAyYIZuifn2f2VXHGw4khNMdwRBH44adcitKCijAlARnBG6i4xS+QJGX9iu36EduLZ4CWEzb/oRciAZqCcWmXIs/Ym2vUQXdWuMSv91TPm6y6A;3:NctZ8nztLbabujd4kxorLY5MJ6StyoUSKlGQwMAsAtW4WH9+0lsUgzA3MUzQZX7Pz/kbwL6/hrHZr12qAQ3katP4EpFPO254bq3gHxRTlUd5x1QNXKd53JK/71xZikul;25:0ViNNkwYL5n1kxzyWs1hs9x9lnOyUkHvjzcz1v8YNYzNGrxSGmisig72x/57etJH0+Z1a1Feh/r3o90S+oxTMLcgEnikA7xFRn9ki+PlkFKV13uWq+LVVybkbFAVoQX6/ND3nlXK0BW6NxYmVqzXG44DVMrehdBDbmiEuM91tk3o7lqGdnr27DOgYNKONZwZ3TVayrcgC3Dfc7oIGx/BS1k+sXuwVvJVrbY15mmFE5UKj8kprlJ20hzFvYAmRkhXGfThaCNmj5VO2DTfOYIFtarOpzO1NgGOkzfcanP2P7Tcl5lHLmZl3OUZlRJRKUq0In2naPL8v+FlsnoqHDLM8VAsJD1iTs55MvTL7m5e2to= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:SN1PR12MB0448; X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0448;20:ob/XHveAFr5bsejPw2H27MJldR36vZ7MqwdHKsTxkoOB09a/5MuTGvohlYaRVCBQ6uZt6eIQuYzi3L3x4U6ZdpC/I8Tcocz/I+HsjjeneGwU7bvLnREZYWGXNvf7Nk+8DidHXQ4SNQJky62MdOVPV6kZVsysFid1RjTo+NbqY6bnZPA53K0APS1KCArxK/XKbRr7o5nY++v788NWmPVnu9z40mZSI81AL/xQzgTq2aRg+LncH5o3AK2KOwR3fxmgYIz04nnq+ue/xfdVEpx536Zn2ucDPg1loXGP9RsTm6GeouRc6Iwj7mLCLanJALqQCC7u9oiN8b4StvY7RW/p6i5E0AQRP23QnK8OdZEMxePJILloqaOIUqRfGosnU3AAr/t/mFDOWXyVzJcDvdr8iXhBS7NtBQLPQCsYuc20sb8FKHQWZV5VTnBFSwU0C4ttbPBZuKqafhWptJY7XVfIG23fVj1Ry8+HZedlZtIoznjEy0dKiLiIHPry/za3iMA7;4:LTIjYJAYsIrB1iZI7CqRtiSbE+X5gvWVpRSBjtk9mG6UUe6GbH13/m30Pk5HJF5I8KcudYfLXe8vsBmbIFoPsZ5fZH6VpW2Gqil1uU9TbhJMEacdiYtALf03EV9zfgQd4zggB4TzFcNwjs6At+bBJmUorXxqkqjJttQBW8w2Xmu5fpZsTsnIpHbPiDoIttKTIqdp6ZUYn8Cv/iATiJSN4N9uTnI4rkjDqpx65bh+hsmCMMiDfn0Ea4u6ST6dciyq9dsTJ1fUo3ekml5R1wmLySP8UHTiqp9g6kv17NUctuA16R2S51T16X+F6f/x5nw1awgIW4ytP3lZwZroT/elvo4g8iDOI0RfTBiS26R5AIoRnpBqEPboJWySpQtBnWr/Pr8roLQ+HIcoJ7WjvAQBRHaRw4GcUTUMcc9kb4CizEE= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001);SRVR:SN1PR12MB0448;BCL:0;PCL:0;RULEID:;SRVR:SN1PR12MB0448; X-Forefront-PRVS: 08617F610C X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(6049001)(377454003)(24454002)(164054003)(479174004)(92566002)(83506001)(6116002)(77096005)(19580395003)(80316001)(65806001)(19580405001)(40100003)(64126003)(33656002)(2950100001)(122386002)(23746002)(65816999)(87266999)(54356999)(76176999)(189998001)(5001960100002)(110136002)(42186005)(87976001)(3846002)(50986999)(230700001)(4326007)(1096002)(117156001)(47776003)(2906002)(5008740100001)(586003)(86362001)(93886004)(36756003)(59896002)(66066001)(5004730100002)(65956001);DIR:OUT;SFP:1101;SCL:1;SRVR:SN1PR12MB0448;H:[192.168.0.19];FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;SN1PR12MB0448;23:+Owp3pHZwD28TIg6Oxrl/fkeLR6A51k1NqePr?= =?Windows-1252?Q?Wo3Anj3ohk7/pK2qUjCwsa/04DNPSLotJ6scfKjbEteQ/xjqdXDui+Mv?= =?Windows-1252?Q?fJKNrL694BcXu/G6qjChn8NL19pjHwHkHM/NV+gTzOK5H62W5H/LzZxd?= =?Windows-1252?Q?PxniwM8zeVyV7MgJ74mjuBmdRjU/g+ilYY2ulNVpO03RO+Vb/WdMoLjk?= =?Windows-1252?Q?ElfPUJZf4STfrzQJ1F6f2wPnwepvvyeqn5FdUw7KSTe5Cr0zK8Jzpsks?= =?Windows-1252?Q?EjzV9dY2jZVDsXBSYOTAg/EsbFpvDQZ5Idrx8ba6AMNWMorQQCpOb++N?= =?Windows-1252?Q?5kxGZlK2Cb96yORPaZUPtoGC6UWE1iOKRo0ApSMaYJvHkd0vU8FjotO9?= =?Windows-1252?Q?wCjaeprLMH1kQTijVGlEfqGMxiv5Ap6AcX2V4HCXgGYjZmLncWfvX1w6?= =?Windows-1252?Q?kYLX1LYtOLy89J46+ucvGR6Vf4BfW53P4HSo2/xZ7Pl5iiJji1Qu77fu?= =?Windows-1252?Q?UzRGiFe5U46HfYsPOIeAhRz0T7oBJerpu9l2J3h/PZfvAh0vTIcEsjcE?= =?Windows-1252?Q?Ko9E2OglpuF56hPdaIhjjzLPozj7oP/SD95xssDyBfq4Sllr7B9ILW4z?= =?Windows-1252?Q?VQGUJDAU9wsGrXbk5OpPvmI6ffs4QbXuQbmUamySS+cIS1kiB2EdT63B?= =?Windows-1252?Q?ga+OryQnpRTX+j+4JRZnIivj0pctYh6YCNqv4QC2RdGfQr5nCL8/Y0KN?= =?Windows-1252?Q?U7FWGYL0nGGJ2WWPjfaHudBPrLUXM9zwplZMWaUJ5WHNd/Ug5gkBbE0k?= =?Windows-1252?Q?sDTeCcai4/2C5FilDzFhmXxsDYnEmA5wl7GkPRl8qt8MdSttKijbt/2/?= =?Windows-1252?Q?ZcBqY6Qxmmizuu/Ga+Ro/Z+RXDKdWUqTQ5P+Cnkhe43lqxrleGq/2aXq?= =?Windows-1252?Q?osBhLjlDutmW4DyLFxDkw1LVfRStIEHLE9LJ+j0ghWA4TOrb0Dr1iemG?= =?Windows-1252?Q?93tVjClDi/oya19d2Ok4KNn4XQEqj/ixaMOIRSWQnfvLdPRoeJ/KdiOa?= =?Windows-1252?Q?FrwecHIUWu2yPElpxXMZixI7PvcgefuK9LOfHdwhQicNuU6Dw9PkZO8b?= =?Windows-1252?Q?ulnrCBJIeavORIef16G1t6pbnPq5V9yZbfde3ojRhMb7Bl7MIPSYylwR?= =?Windows-1252?Q?GLTor/6XeT1YtsLvzSF++a30SMfkQn59OmbzEZD9xMeDlRGDBEISJlkO?= =?Windows-1252?Q?cFimaCGxEhAheKgtiQjqOf6JF88r8G7krEfJV6qY53I4dZGh1/i3OZJV?= =?Windows-1252?Q?n5z3Lo/SFUPSW3aR8dz7YDWcQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0448;5:oLX2H3wAQYhrsAfuBrSNxpgj3Q6u1kWB/m7eUhqVcldzHuJRklI8LX4mAaUmvmHN4+aoDY9zqgG2WRxgcN2st47f5zdgMM+UuBDpzuztlsnB/vu8LHHma9lB6SAHZuguqt2dj7o/M6bqWLRANGNf2Q==;24:Bldt8IPiF0BeFXMUdu+uPsmoPj/3yXFppBOy23CwN7mB5/UTJ/ogn+x+6uKvQ3Ih9cn/SI54i+QRFlNnwi7MD/L/NLOqssaI+jXiZ7wsgeE=;20:H9FXF+uzy05AZ9rXJso0sEAVBUnt5W7gdyEdpod6gNFrdK4yn9KhOv9Z+seOLDZzf/NzJpIUIyYezfmQaWDH2Sie3AFNMMgeGrC82GCETiiPSgGTWX9raGz9Mg72DG/Q8jaTpDGe+82hJ4oLZMckIP5AxQg5B/njvNxsSPto5uIGDsShYE210ZnskidnOFAYmEWkiW599y99SZRJlWWsO18g4DcsPLwT4K1Oo0KqlhAFXfMCNAyVqzx6MjnTmxF8 SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2016 09:56:38.2521 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB0448 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 02/23/2016 12:24 PM, Alex Williamson wrote: > On Tue, 23 Feb 2016 12:12:42 +0700 > Suravee Suthikulpanit wrote: > >> Hi >> >> On 02/22/2016 09:07 PM, Peter Zijlstra wrote: >>> On Mon, Feb 22, 2016 at 03:00:31PM +0700, Suravee Suthikulpanit wrote: >>>>> So I really don't have time to review new muck while I'm hunting perf >>>>> core fail, but Boris made me look at this. >>>>> >>>>> This is crazy, if you have multiple IOMMUs then create an event per >>>>> IOMMU, do _NOT_ fold them all into a single event. >>>> >>>> These are system-wide events, which are programmed on every IOMMU the same >>>> way. I am not sure what you meant by creating an event per IOMMU. Do you >>>> mean I should create internal per-IOMMU struct perf_event for each event? >>> >>> No, I meant to expose each IOMMU individually to userspace, as a >>> separate device. >>> >>> Is there never a case to profile just one of the IOMMUs ? >>> >> >> I see. That's definitely doable and simpler to implement. >> >> I was not sure if making users specify the IOMMU instance (e.g. >> amd_iommu_0/ , amd_iommu_1/, ....) would be too >> tedious. However, this would actually give users better control of the >> performance events, which is a good trade-off. I think it is acceptable. >> >> I'll make the change and send this out in V5. > > We already expose individual IOMMU hardware units in /sys/class/iommu/, > you might consider trying to match the names there for the convenience > of the user. Looks like we use ivhd%d for AMD. Thanks, > > Alex > Hm, the PMU for AMD IOMMU is currently shown as /sys/device/amd_iommu. I am not sure if /sys/device/ivhd[0|1|...] would be obvious to users that IVHD is really the AMD IOMMU. Suravee