From mboxrd@z Thu Jan 1 00:00:00 1970 From: Haggai Eran Subject: Re: [PATCHv3 1/3] rdmacg: Added rdma cgroup controller. Date: Wed, 10 Feb 2016 18:27:46 +0200 Message-ID: <56BB6502.8040507@mellanox.com> References: <1454167387-31260-1-git-send-email-pandit.parav@gmail.com> <1454167387-31260-2-git-send-email-pandit.parav@gmail.com> <20160130183015.GP32380@htj.duckdns.org> <20160131100237.GS32380@htj.duckdns.org> <20160131110420.GV32380@htj.duckdns.org> <20160201184046.GD14091@mtj.duckdns.org> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=c3gBjSS3eymYMB6p0BzYp08uHLJTJJnpfqOv3rq/Uh0=; b=sCbUMUZ2EWb/Y2M/1EjXzP4AQnqjmOSN1/V94BWzir86S7nvDdwkbdOudt+5SyBlavj0cazih8optTRljeAOs0C6mmiKQ1SXu+5dmyNUDwlSasE2ENr5OGSXUEU3Z4PEGq5Ut/4HdrS1LquaeF2NCB999r5IKdU/hPOFBy/7+/o= In-Reply-To: Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-ID: Content-Type: text/plain; charset="us-ascii" To: Parav Pandit , Tejun Heo Cc: cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-doc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, lizefan-hv44wF8Li93QT0dZR+AlfA@public.gmane.org, Johannes Weiner , Doug Ledford , Liran Liss , "Hefty, Sean" , Jason Gunthorpe , Jonathan Corbet , james.l.morris-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org, serge-A9i7LUbDfNHQT0dZR+AlfA@public.gmane.org, Or Gerlitz , Matan Barak , raindel-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org, akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org, linux-security-module-u79uwXL29TY76Z2rM5mHXA@public.gmane.org On 01/02/2016 20:59, Parav Pandit wrote: > On Tue, Feb 2, 2016 at 12:10 AM, Tejun Heo wrote: >> So, I'm really not gonna go for individual drivers defining resources >> on their own. That's a trainwreck waiting to happen. There needs to >> be a lot more scrutiny than that. >> > Not every low level driver. I started with that infrastructure in > v2,v3 but I got your inputs and > I align with that. It could be just single IB stack in one header file > in one enum list would be sufficient. > You have already given that example. > With that mostly two resource type that I have can also shrink to just > single type. > Will wait to hear from them, in case if they have any different thought. Hi, Sorry for the late reply. I think that starting with the standard set of resources that uverbs provide is good, and if we need in the future new types of resources we can add them later. On 31/01/2016 19:50, Parav Pandit wrote: > How would you like to see RDMA verb resources being defined - in RDMA > cgroup or in IB stack? > In current patch v5, its defined by the IB stack which is often > shipped as different package due to high amount of changes, bug fixes, > features. > In v0 patch it was defined by the RDMA cgroup, which means any new > resource addition/definition requires kernel upgrade. Which is hard to > change often. There is indeed an effort to backport the latest RDMA subsystem modules to older kernels, and it would be preferable to be able to introduce new resources through these modules. However, I understand that there are no other cgroups that are modules or depend on modules this way, so I would understand if you decide against it. > If resources are defined by RDMA cgroup kernel than adding/removing > resource means, someone have to do lot of engineering with different > versions of kernel support and loadable module support using compat.h > etc at driver level, which in my mind is some amount of engineering > compare to what v5 has to offer and its already available. With one > round of cleanup in resource definition, it should be usable. If I understand correctly, if the resources are defined in the cgroup, you simply won't be able to add new resources with a module update, no matter how hard you work. I agree that if the cgroup code is changed for cleanup or whatever reason, the backporting may become difficult, but that's just life. > Its important to provide this feedback to Tejun and me, so that we > take informed design decision. Sure. I hope this patchset gets accepted eventually, as I believe it solves a real problem. Today RDMA application can easily hog these resources and the rdma cgroup allows users to prevent that. Regards, Haggai -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752867AbcBJRCs (ORCPT ); Wed, 10 Feb 2016 12:02:48 -0500 Received: from mail-am1on0100.outbound.protection.outlook.com ([157.56.112.100]:43360 "EHLO emea01-am1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751486AbcBJRCm (ORCPT ); Wed, 10 Feb 2016 12:02:42 -0500 X-Greylist: delayed 1067 seconds by postgrey-1.27 at vger.kernel.org; Wed, 10 Feb 2016 12:02:41 EST Authentication-Results: spf=pass (sender IP is 193.47.165.134) smtp.mailfrom=mellanox.com; vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=pass action=none header.from=mellanox.com; Subject: Re: [PATCHv3 1/3] rdmacg: Added rdma cgroup controller. To: Parav Pandit , Tejun Heo References: <1454167387-31260-1-git-send-email-pandit.parav@gmail.com> <1454167387-31260-2-git-send-email-pandit.parav@gmail.com> <20160130183015.GP32380@htj.duckdns.org> <20160131100237.GS32380@htj.duckdns.org> <20160131110420.GV32380@htj.duckdns.org> <20160201184046.GD14091@mtj.duckdns.org> CC: , , , , , Johannes Weiner , Doug Ledford , Liran Liss , "Hefty, Sean" , Jason Gunthorpe , Jonathan Corbet , , , Or Gerlitz , Matan Barak , , , From: Haggai Eran Message-ID: <56BB6502.8040507@mellanox.com> Date: Wed, 10 Feb 2016 18:27:46 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.0.52.254] X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1;DB3FFO11FD050;1:1yAzCTA3gx4rxCFdxnXo3B5w1jc3aoFDYvWM2LX2/IdLuRrUH+fI2tuGEMemEHwsfrhBXURkIVNIZbDcZApHxmsMnQTUjxgT84XnDyQbG9sshYZ8zbCXlRNtBVPm5FRGLbDCe24hghkCefjXfOjpbqWjesm4Z8cLOFMh9MhwvbFT6mxXlfS8x4yPD+XuAkfgqWevYF3SWHz2wNfRD7w4v2wtQ9aMqYAGbumKLC8ypxTG4AL+JxEmmxxPY7TJmoThHVr9hN6wfZ1ZAHP3I/X/UBjtOr9mY7lG0XyHgq4OJZil02FVyItYYoKUuUnMwT3Zmde6NoGnkSERvdbVaHbmfSNGAryl3FOGKZCdkSchA6TXqLprQZcvZnukjzkp474LcwRn8rT7aRaBnsexYGnH6IM4UFmjF1U0vBYWjdLOjRBwSTXwx7mDgq/ZKJv1tTalGusiHao7AXbPDSGlv3CKsg== X-Forefront-Antispam-Report: CIP:193.47.165.134;CTRY:IL;IPV:NLI;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(2980300002)(438002)(479174004)(189002)(199003)(377454003)(87936001)(83506001)(36756003)(4001350100001)(59896002)(5001770100001)(23676002)(2950100001)(77096005)(5004730100002)(11100500001)(93886004)(106466001)(1220700001)(586003)(1096002)(50466002)(92566002)(19580405001)(65956001)(65806001)(64126003)(3846002)(6116002)(80316001)(86362001)(19580395003)(6806005)(189998001)(47776003)(54356999)(87266999)(33656002)(2906002)(76176999)(4326007)(5008740100001)(50986999)(65816999)(230700001)(3940600001);DIR:OUT;SFP:1101;SCL:1;SRVR:AMSPR05MB211;H:mtlcas13.mtl.com;FPR:;SPF:Pass;MLV:sfv;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;AMSPR05MB211;2:DU+ipQh+YXLvNiBviFdjTcfXkuyzh/DWwNCbH9415sMbX1uywMHXcHvWb+NmIXaTE6OY18iywaytwTjUKPj23lSe0TuRR051HWUVRmTzy/ceRT3ePwW3p55XWa+bVfOgfsLIoQ6k3mT3R4pCu6TUag==;3:ZCuT41z4aTg5gac1pAO69oKMMl4Anv3JIR16+/tPQlUhATU/qfxNErmmtxQgZg6J775hZrInN29HG/ennGO9EVWvrXSPzdKrxeljMNjzd4Jyhj/1LuLGTXCqoN9WeLg8BsODRiTEd+4mI4aus0s+QPyx33tjUuLG9Di+VAalbxz3Htac/SjNiP60aJa9b96AVzYJF23ngZqKh/hqx8Qj0j8xD61vATbUGtN15B/eR3S90QfQi8d6NBTv4N9l+dXWB9hd/avIFpIUHOUTVH1VDg==;25:316T0MJJ3WGsXrBkh9bSmybPQQagLt9eSAcUw7BjQGl/PqHhUgqcbjh/I5cCCZb/RLkiPE4G1mlbWeWWuSgz8pWBPJT4jUyuUHZPg4riL8OU6NH3gI8dFkfBLKoFDifghYFegjK5jUwH8e4qF1y6/Z+9e9cRRJ9f0tyyr3Lvtu7A+TTwqAYz+AADAfyKIpEmXseDuSjySVveV1MQ5ILrI9yao7/LNgyxKn0/R8TFqklAst/NCVSIK+IQozosC3gliYQeZGtw5DcayKaQSPgAj3zjkjUgGOiglPbwLqBvKkfc3kZvRYd3vUuew+wdvVVN X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(8251501001);SRVR:AMSPR05MB211; X-MS-Office365-Filtering-Correlation-Id: e9868926-a6ea-4220-6bad-08d332375323 X-Microsoft-Exchange-Diagnostics: 1;AMSPR05MB211;20:TPa9vJ8z046e+cZk+y34PRkgjhP5koIfakwipbSM5AHqGXLFOikJCNOPoTJpw5ELXHSeYGUrXgFAl0TCZggwB9VkgolE4PiQzHlc8XWz6S/btdxj+xIsScl97I28tv6jtqD6+9DQspenQ5BWBFqz+7W0hT3bfxnqRLvXN4/1HW46ZPCbYs6I6GFups7HKyLGf8g1p9gpNW5sR+p7O/AItQl9QXfOq+weXF36qVESKxAZuY0DZbrjtklOOBIK2VlcG94VrPyVtvsVNDZJ8f02UY5GvvpnU4sd5cqy3GIfbFhaarVkm5/xrHA5LtMHxU8CZJ/rjFw+1BRG1FaSiRvQxIrnN27DaVYpCnmpgArZYIpqSlfaEfFOBxbh2eRPOzTBEV/Z2TicLzm8ECI06Zf6srHOPbgXfwMAg91L/fgu02o31Cm8O2n0Htz8CK6d5QUlSc+Njjz50M4ELdykdNVlp4+ZGcVkWdtaFmqOFNgnoyrzABN9YqjoOW3EqEMl/Clq X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(13024025)(13023025)(13015025)(13017025)(8121501046)(13018025)(3002001)(10201501046);SRVR:AMSPR05MB211;BCL:0;PCL:0;RULEID:;SRVR:AMSPR05MB211; X-Microsoft-Exchange-Diagnostics: 1;AMSPR05MB211;4:OdJT30pB1dqefFAA4YtMtVVBW/sHM8KiRPqffmPYAcJN0RlehgPolPmzgENxs9SfHz7c06FhRuOyoymprXn9j1Ykz3u2Y0oxI1dezQBW7uKod7Jt/1PliMTsVf54tBr1cKIJtC1eeaUvHXR+Q/AAAETjsASpKT2yqB/rYtfmjjMIWns78whk7x0g7r9+2SKjXfXbMFqUKq3I+v4VWeO3Oz5mAF+/EPMT9lHwwjSBNBTG/JipalcfjdXiOwKS6OyllGjEw5bPBfX7ERbyO6fkaLS/OnkpOr82nzV+n43BONLCx18s3PSxicXZaQgie1AfYy3IOwNg32ISeCw3lPNvhq7ynxIyzo2EqySten1A8BfdrwM7tfohHJsWfwqeFHwRgSscShyGbX2J8rBX/pj/qJZQghhGxJso07jITrUzrodyT/yEf+z+hA3CECTq4DrS72mSufNLzwAf/bUDwKPpwQ== X-Forefront-PRVS: 0848C1A6AA X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtBTVNQUjA1TUIyMTE7MjM6TVgrd2VxK3FHTVVoM21FQm1WQXY2UWhORmxr?= =?utf-8?B?VE01bFpPdHJ1c0JaK1hwUE1oY21hc3ljdVorVXl1L0N0WWtYbVJuTjFDUFpz?= =?utf-8?B?bFRhQUROWnNITllvKzZBK1NVSHRFdlNQMTlPUVh4dWpOaGRLdVpwM0k0ekRT?= =?utf-8?B?RDVEWWZiMElqSDU4TU1VMlJSTVNvNXhaVUN4YWp3bmZ4SEM3QURLb0lWWHVP?= =?utf-8?B?dWFBZFV1dWZIZ0c5RnRqQk05MnZCdE1pMVlLQ0dJL3hzSmVxanVuQ0ZnNEJ5?= =?utf-8?B?RUpqZVVXMTYyZjljYUdCZWQ4MlRkZlFLK2N0VEc2UmVhS1Y1dDNubDNiSkFi?= =?utf-8?B?di9KWlQveE56YkNPYU9ZMUp6OHY0TDF4RTMxenpEWHBGVGNxZ1BOdSsvQita?= =?utf-8?B?TEdMU0Z2c24xWFYxbnFWVXQ2QUFJbi8rRnV2M3gzN1VDOGZsWVQrOG13NlV0?= =?utf-8?B?RENCT21lSkxYWmpJUUhZd1krWXN4NnhwdlJ1cDN0VmtucE9EK1B2Y2FpTHJH?= =?utf-8?B?STFRemxIL2FiOUFuYkh5dHVrNUlkNlNNcHNDRlorTGN4R0c3ZmtleGRDTUsx?= =?utf-8?B?aDZCUlM1cDM2S1VRWEttV2lSYXRzZXN2bkVITHVhRXhjNVhBc3FzU1ZVV0lq?= =?utf-8?B?Z2xrd1hvVk5NNllpOFJhbytrM3g4dVBJQUdpeHU3VWU5a3dFTFZuVkJWb3dY?= =?utf-8?B?NlpZdmZNQURrdFNsOE52OUF3RnFUbzY5c0gyaHRpZFdhMGlyK1lvQ0pWZ1I0?= =?utf-8?B?VlVxeWxVeW9DbHhNSXVuNUwzbTZiMTg1bmxVNC9IbTlqSGJsQndiUnZQdkhm?= =?utf-8?B?SXVsTmRWUk10cC8wMlY1SEJDY0JPc1VPUVFEaTZiZmhvUkxkMGs3YUN6VzVE?= =?utf-8?B?Y2haRFdvREs5clRDeVNJQ2F1UzQrc1hGWHlwd1N6YTI3ckJzOXBxbURSZVho?= =?utf-8?B?K2xhL1pZTEV5SWxkTmFJY0ZqbWQ1K3dMd3FjRFZvcXhoVi8wVFVMTzJvZ0Ny?= =?utf-8?B?YVlZeTg4SDZtSFZ0REJ5Q0dDdjk1NElFWllOZTlCajNaK1FIa0JoRHVGWXJT?= =?utf-8?B?VEx2ZlVBcXpWZmFaL1VUSEpQTldTcXplaFZCTmpLZVk2SmNsWEpGcFFVaEU3?= =?utf-8?B?MktEaTc5Ym5IaEZ0RjZPYkp6REN6c3Vxa29UeVNiZ0FkbGlZYi9UQUtMa3Zw?= =?utf-8?B?VEVCVExwYlhWTmd2dUFiMVAvSHYwYVM4V0tZckdPaGtYSWdoU0NOdlRHNjRO?= =?utf-8?B?VFhBZThuQ21wem1mc1N4ZFF4Y24yVHB1dWt1NklaUThJRFlwVE1DQWhLZ0l0?= =?utf-8?B?YnNYR3NtcmlKRUZsdG1uZ1BQM3lGV2ltUTl4UDFZb3dyU254SHU3Y0IwTEZZ?= =?utf-8?B?T1FCdmtHVUpDSVpFZlhjQko2N3dnSjBIRm9yYlJ1NzYvQ01WaHVJQUp3OE44?= =?utf-8?B?R3EwSFJteEVPVldJc29lU3BvYW5XV0pjL011M1FoeTNZSTdJV2hhcUdaeHRR?= =?utf-8?B?UzNRVXZBWjhmVGRXbGhPVnk3K0VETHhZZGxyMnVkMjFFYm5kSkJCSkF4ZWFq?= =?utf-8?B?cHVmMUlEd2V4cElwRFIwTENhVElnOWRTMEJ2YTljNmpsYWRaSCtHb2ltVWh2?= =?utf-8?B?L01zV0RjM0ppZXgyZGR6RjQ0NnlheHhmRCtMR2l4V3FBb01BU0NhNVF5Ynp4?= =?utf-8?Q?PRO5+2Cw82jK7R5v0=3D?= X-Microsoft-Exchange-Diagnostics: 1;AMSPR05MB211;5:YWByhdmNLbnHLIjvWktfvJfs5qijpcAKoIFkE/Z4DF/cnBMC8aGs5svH3BRfqCDXsi5ewjsCnPZKLdKwTwciRHK1UAY+/FtmyArfyOSBFWmafsovUbtW791+XefNn/cikgFh7nVYO6JeH7dcrQqBlA==;24:MhzypS1x7ILDvgUL7lL3EgkdefICJVDelhM4npJhqaqBCMV15gjQwc2qzUZ2l3zKk+85dFARfGLTj13gL69AqcKP1SfMfkobf7GG4ci6VaA= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Feb 2016 16:29:18.6452 (UTC) X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a652971c-7d2e-4d9b-a6a4-d149256f461b;Ip=[193.47.165.134];Helo=[mtlcas13.mtl.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AMSPR05MB211 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 01/02/2016 20:59, Parav Pandit wrote: > On Tue, Feb 2, 2016 at 12:10 AM, Tejun Heo wrote: >> So, I'm really not gonna go for individual drivers defining resources >> on their own. That's a trainwreck waiting to happen. There needs to >> be a lot more scrutiny than that. >> > Not every low level driver. I started with that infrastructure in > v2,v3 but I got your inputs and > I align with that. It could be just single IB stack in one header file > in one enum list would be sufficient. > You have already given that example. > With that mostly two resource type that I have can also shrink to just > single type. > Will wait to hear from them, in case if they have any different thought. Hi, Sorry for the late reply. I think that starting with the standard set of resources that uverbs provide is good, and if we need in the future new types of resources we can add them later. On 31/01/2016 19:50, Parav Pandit wrote: > How would you like to see RDMA verb resources being defined - in RDMA > cgroup or in IB stack? > In current patch v5, its defined by the IB stack which is often > shipped as different package due to high amount of changes, bug fixes, > features. > In v0 patch it was defined by the RDMA cgroup, which means any new > resource addition/definition requires kernel upgrade. Which is hard to > change often. There is indeed an effort to backport the latest RDMA subsystem modules to older kernels, and it would be preferable to be able to introduce new resources through these modules. However, I understand that there are no other cgroups that are modules or depend on modules this way, so I would understand if you decide against it. > If resources are defined by RDMA cgroup kernel than adding/removing > resource means, someone have to do lot of engineering with different > versions of kernel support and loadable module support using compat.h > etc at driver level, which in my mind is some amount of engineering > compare to what v5 has to offer and its already available. With one > round of cleanup in resource definition, it should be usable. If I understand correctly, if the resources are defined in the cgroup, you simply won't be able to add new resources with a module update, no matter how hard you work. I agree that if the cgroup code is changed for cleanup or whatever reason, the backporting may become difficult, but that's just life. > Its important to provide this feedback to Tejun and me, so that we > take informed design decision. Sure. I hope this patchset gets accepted eventually, as I believe it solves a real problem. Today RDMA application can easily hog these resources and the rdma cgroup allows users to prevent that. Regards, Haggai