From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932250AbcGAO0p (ORCPT ); Fri, 1 Jul 2016 10:26:45 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:16778 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752510AbcGAO0m (ORCPT ); Fri, 1 Jul 2016 10:26:42 -0400 Subject: Re: [PATCH] nbd: convert to blkmq To: Josef Bacik , , , , , References: <1467382507-523615-1-git-send-email-jbacik@fb.com> From: Jens Axboe Message-ID: <57767D8E.40808@fb.com> Date: Fri, 1 Jul 2016 08:26:22 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 MIME-Version: 1.0 In-Reply-To: <1467382507-523615-1-git-send-email-jbacik@fb.com> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [216.160.245.98] X-ClientProxiedBy: DM2PR21CA0024.namprd21.prod.outlook.com (10.161.137.162) To BLUPR15MB0451.namprd15.prod.outlook.com (10.163.214.17) X-MS-Office365-Filtering-Correlation-Id: 2f3cd88d-432a-4323-0c3b-08d3a1bbb0e9 X-Microsoft-Exchange-Diagnostics: 1;BLUPR15MB0451;2:asII/+/LtOij+EwEN0ThSY/x7H2dDErq726gCShlHfEvhxnONdJbhdLpoUf4fVvIyJ+sQb36rR8nDrldSUigwer9gZWNps9Tf5n+sSzIO/HC6uqcVp9Km2CNTB6G93lJ2tIERBYPFylmxVW3m3qrAe2ck6yA2D21VI+lZmQrjthkiAa6zbJaMIUpfgeLxMHz;3:Sc3wxeouaT432kolBXhMs77IGG5miFLgp4uJ5GsNQwWogSDyp76656iOogf5kEHtSS3UL/dTnoKtIWBBI8feor4c8FWftoFo3ST9Ma2/onSI2EU7FTe3RXBIYNF/Hn8h X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR15MB0451; X-Microsoft-Exchange-Diagnostics: 1;BLUPR15MB0451;25:oiLqs7IeTs9su8Dv2RTrnLy7xMeP6iaSB9VKBsvjOjRDKLIq3+xtBtn5jSTp+nA4ETVsVLyvl3ZBAhmzmYIVwRCL0+RuocTLF4Yp3QyCqSUzp8H1barEeqr7wdcJeO2r2Sk2MxGmDs7DjNh2Dx27FqTUD7wXdrKwh82l1cXwdXxGaHaZ5CLxOlN67+FzUrQ0rAwz6mWgCjRAPDNDV0b/1L0vKuSpnR0pPinU024ZaUrx5rsLF3nM12xWtu39bB5i8hwp7Y9B+qOBQeqn9EjSkXkQylIBPIy6yJyabx4pnD7V8vtyQ6BLE8fUCQcKsWwRKN2XKKNeWcLp2EfUlJBiRkpgSzfMMGVtRDXjfJOkQN5jcV/qawnJhfD/YFUN3O2x9rj75zrnDATDmnnOvFlWyc3MTyjgtFOShlMoStJs+qx3JnDWLeKmJ6fq/lLNHBcKAr30g+M6fTqDgnw+Yc4qGaj9rn9JKSqwkq+u8esiQjYYRb+xml2A0tcjgkTB2QRmW7aLPHL8kua7QpuLWKEE6AllrvRV2M8v6iz2XHilxNnTX1Qi/5ywz3YoWqDWXp7rSDU5JRcKgRExX20egdsRwjHICsA1TIV3b91izMpC/FwpF2O2+guZvUjZPXO0ECW1dAeUBo4ES3lJj3XQBqAiaG4LtblCQNUCOR2/Ks7lJtwMq1jRnkgkb9KB8J6ZKkhszyywvZT21uF6erI+uqrDgw==;31:AlswV3YY1/fTe2Qs89i+InqogAQ1TBChOSSD2F7EPWJq86rJI/xyBudTfACmaiL459AwM2L/yfT3/behmXq6y5GuksbLCOFlnzlgEZ4MchBMOeGcOa3ThxC9j7iIpTLrB7Sh6DwOh3YGu5j7AfksJQZi2mUbkXin4C0unhyudPlYhPJ2ReQx3B/g8iwTjk1mpovE17fOuJxH4Oq7yG5/jQ== X-LD-Processed: 8ae927fe-1255-47a7-a2af-5f3a069daaa2,ExtAddr X-Microsoft-Exchange-Diagnostics: 1;BLUPR15MB0451;20:0NoIrgEW84GzVVi6AWKQzYNCkXfKo/Fr5t7yP93eTsxxVwhk9yOpnWL+hgWV9f2VrHWaLfQeOjDtjz897263YWOBnYPTM2VcR1XZMftIqmhrLMhGvVBySN5f6tBCNd49dZ9UHs4ztpL2SUmyIaTs10+OpWjQcLG1YbBaT5Icx2M=;4:CWWr9hcKNtEsc5wzM9MxFWXQv4Xg8R6PLjRoGbpWFAhub4V8erIegkdglc5QtliTHkGK00kpbzLDNkrssgoveJO6gjmm/5O9TEja6x87XK/e6ruNzLAbuzN7mUuKzuBvRhhywm9IfUKIsyqTo3mfgf6+gJZJwDGeWUl0sa6EpR9Kr/AB0dx6jFFYnxLzpHtmwnHFQVXkTuRG60laIFhqDtbZvkx9Dx0CYQglLktn6u39oteYa9q2B1KZJ0iiSHDNzrXhCOeP6krxOVBJTVb4Pf1DYREFqk+HSydAG0sQtlANeuSsEYm+6Ryep6vzOoJNKOS/n5YKj4VMP2N2yjrWsazl0N8dEft7bSzmuEfFZ1T3O1PmFkfxMW/1I+pnkQBP X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001);SRVR:BLUPR15MB0451;BCL:0;PCL:0;RULEID:;SRVR:BLUPR15MB0451; X-Forefront-PRVS: 0990C54589 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6049001)(6009001)(7916002)(189002)(24454002)(199003)(377454003)(64126003)(97736004)(305945005)(107886002)(5001770100001)(50466002)(81156014)(7736002)(68736007)(8676002)(23746002)(7846002)(4001350100001)(189998001)(81166006)(2906002)(50986999)(3846002)(2201001)(230700001)(42186005)(92566002)(6116002)(586003)(33656002)(117156001)(76176999)(2950100001)(65816999)(101416001)(77096005)(86362001)(54356999)(36756003)(106356001)(105586002)(83506001)(65806001)(47776003)(65956001)(66066001)(129583001)(17423001);DIR:OUT;SFP:1102;SCL:1;SRVR:BLUPR15MB0451;H:[192.168.1.130];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;BLUPR15MB0451;23:Nw3j4vxGwOfZuG0A1Nd+KsMVVPOtRXWgSg4jA?= =?Windows-1252?Q?nV+B0JeEATytED0b/7k5U1UKmiq7ImKeagQsuBmVJIZE3ye4RPsTC1gk?= =?Windows-1252?Q?F5/BxpfDf0czNZ3Dpf2C2A/Nl+7/6jfbfyZFpxEebnFiX6CJifZn7AjM?= =?Windows-1252?Q?hXTpSilkpB/V5/V0AtBbhbwtyDunUsmlK+A2tNRn6GPGP3aClliR4pnx?= =?Windows-1252?Q?Y45ZhWOIIMABINMRVM+KebHDTyANwgDyVuatUyCZ35l2XiaoAIw0DvfY?= =?Windows-1252?Q?jSsP8MAv863FAHDNVVz7eFXXntmv44V5RPwqsQ49wWMklsvIeP6f3EeX?= =?Windows-1252?Q?ZU5PEU/JEwpZ/Jlp5/4fCKvzKBqZOqPkHqJJtkYFmR9hzZyLlGIkHloW?= =?Windows-1252?Q?D3nVGm6xUDSA8RYs8LIRIcZwgtKLfRFu8+TcLWI3RogXzI9QiczIjX8x?= =?Windows-1252?Q?2Uck7pFP3E18b+02uHFBj4sbxjzlEEO5abCeuNXhanwrAhPt9HE86EFW?= =?Windows-1252?Q?Xro+GsXdQVp29jNQ0o/cG8/dhL9KlzyrUXtqB68ZvUcgDHMciOUpg8mJ?= =?Windows-1252?Q?D+MLPvYb+zbinKVeSiSLJC3MFisVbIPIjm0l74YEze1tPIKLelNznUbj?= =?Windows-1252?Q?iwOKPhvTTDZrHR2fqw00XzzbAOS8fkqqMVgIf7uwuZ9bnA5b83jAf5tp?= =?Windows-1252?Q?6nDV3Vy7Fv5iBnur2vWoYHmh6EV7PNdMRrcbJWli4O2bs1EuCHN6fCAv?= =?Windows-1252?Q?LJdCxOBuFpml1rhC4AJiiU215/kxM8auaeso9uaAAaHB/fly39rer37T?= =?Windows-1252?Q?68idN3LbGKwzcWSHoAg5ou5tRmi9GOBL03k66Xmkszs9lCD4Ge340Z5x?= =?Windows-1252?Q?z0K1qp+h9STVugnqC7As5Kr5nHcItDp1x0aJ/EJ85JUkYuDYTiaIML8h?= =?Windows-1252?Q?peN7IwnCMjMuT7XAwJNMU1N+VHwb6ImlvQclQIiaUyfDfXoGnrLg5Mq0?= =?Windows-1252?Q?qx1YI0Mnao7vOsqovFYVcsXNb5Fvlh9s4RlENmBVirEEruqU+DbZXXHJ?= =?Windows-1252?Q?9+RQAXk9D44iAzSPE/UABv84NJ133s2kbItYwXFrnDNsDuWD2KMDKpaE?= =?Windows-1252?Q?+x6KftH/KCT4wmWF7WRyNhweGplg4dvvPuNpz9IPFQVpL8GREzI20aBu?= =?Windows-1252?Q?gctbfI4W2FzxPu3EyaQ4HEVTBBnFYQL+k9C6XBsgeKaZ9BW2FtDysLqp?= =?Windows-1252?Q?uYr7iRMcg2Z5INrZam67t+6UwJ89pH2/zh8jvgOQL+TTtM0XgQuDfFh2?= =?Windows-1252?Q?GSjUMArFkaFFchLAH/oRJhFPcnZb96ABZVqjV9eggCPAEjzyF4IBX1jt?= =?Windows-1252?Q?SnQXS8XrqEU?= X-Microsoft-Exchange-Diagnostics: 1;BLUPR15MB0451;6:tZOX4BPKuxAQ2nISA0oBJb+Zu2jM6VYm4c3Us7WIACOPmLNpuQB9B2QMBd3sTRPwxd5YuZs+osE3Gpj4OAhxtFYAFudKhyUrE8/lmOEbbZvs3p8BLZW9Mt0NTiuL2BvnF4qoQHxP4BTxn61hqs9f5sBPwNwBPmcj95wOVzRrG1QR33axSxF9Pg6gtkZHjRqV1HmMLeWHe2V6bedjTbKyBi+IwG50D3jDB1bdokKKBUllxxDs7RVemZwSXVeUHuuYloFe0VIKfBkgRFvxgGlp1k0LAisint0EDqd+deZWvFU=;5:rMqoSMpT58nAQfVuZmu4RhNbt6cFxEh5FwQ4yaWhy8q2d4tGMVuHa9UW2TLTg0rPpTbQS5z+Hxqaf5BQluA7WKatkyhw04YpckT1zeV4f0t+s1eK/DpYiGKqPKYd7AbQ7Vc6Qe5efpsiYRZQQ/qtjQ==;24:J6/MDYf5zlTwXpQ37BKTv7F+O/oTgYi35xsYBXgpv9RPzDKPwTLCzCo/1VbxgOJU4eR3tTWAzdUXHJeMeopiIcpU/50fNEQ0pdlzCeAjCuo=;7:/3t90vYVQNS+sbXIJsbk2YjHkGfITAgabTDwRcL/26BZ/OT48cgwdYz93oZ2K9bvM3EPrrlpx4xZbqSyKKQ5YcfJgHZigNFxWCx+QvpdE6krShXGBV+w/b61KKXInAw8nMHThOUwF2hntmuGq1qgj7AL446ZP0VaCaJs9lCr1Cz9V+xp7yFpLa9GdooxTF/g7oO2Ryk+rwayDOJj4G7ulq7gj6fg0AC3r4oK58L13EuvEVV7S8gPp3XLfAP4M54O SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;BLUPR15MB0451;20:CenEgzkx/fvXTAi85TlAJfH0hdZgjtDXbVV605U+guCqh2qZT9fPSe/5haQClmZi/hLK/nunD+NTu6sXQEx6lxScyivhq5J7Oe8+C7m8k60Jdpv99f9zDsxuJANR7de9mW72cgYYYq8rY1BQwId7CtxuDLKo92arAmQGjY5o9e8= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2016 14:26:28.0734 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR15MB0451 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-07-01_04:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 07/01/2016 08:15 AM, Josef Bacik wrote: > This moves NBD over to using blkmq, which allows us to get rid of the NBD > wide queue lock and the async submit kthread. We will start with 1 hw > queue for now, but I plan to add multiple tcp connection support in the > future and we'll fix how we set the hwqueue's. Looks pretty clean. I'd get rid of that horrible nbd_find_request() as well, though. Store the request tag in the reply.handle, and then just lookup the request from the tag, if the tag is valid. The resulting request can then be checked for proper state (started). nbd->active_cmd needs to die as well. -- Jens Axboe