From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934303AbcI2RAW (ORCPT ); Thu, 29 Sep 2016 13:00:22 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:47565 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932469AbcI2RAM (ORCPT ); Thu, 29 Sep 2016 13:00:12 -0400 Subject: Re: [Nbd] [PATCH][V3] nbd: add multi-connection support To: Wouter Verhelst References: <1475092892-8230-1-git-send-email-jbacik@fb.com> <20160929095204.mexr6wpypo3bl6mx@grep.be> <87908d95-0b7c-bc3f-f69d-94d006829daf@fb.com> <20160929164100.akytbkbtvziwaqqj@grep.be> CC: , , , , From: Josef Bacik Message-ID: Date: Thu, 29 Sep 2016 12:59:18 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160929164100.akytbkbtvziwaqqj@grep.be> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [107.15.72.49] X-ClientProxiedBy: DM2PR0501CA0023.namprd05.prod.outlook.com (10.162.29.161) To CY4PR15MB1318.namprd15.prod.outlook.com (10.172.181.136) X-MS-Office365-Filtering-Correlation-Id: c1aa980b-8dae-41e9-dd5c-08d3e889f672 X-Microsoft-Exchange-Diagnostics: 1;CY4PR15MB1318;2:dYGEkQgkUn98jvZwjfA4fIhe1yMrx4MbQBU6L7ATkPxYUvj6OfVIViNVXYP0LGCZgufOU3DX2kvYrqVNsXw+LDE+37N41pvUFHCwOpDE9cpvbmqZretDbJWoB5qs/qJz5sE1WPJZYkkFSxOOjbgEcQCOr2jFNDRBPZP1qOmzJUmYGVntLi/DJDLEc0Y+Jjic;3:J8cPC6l39qdJJurT60CAK33182QZA8dh+NnN5qRcD7Pm8k2GAKTzTBlcEKvPUa7XaCr4ti04TwTCZbn4Td6DVKLrntoqb5G6bWzbuiFtWZJM68QflUG8DNMvxeCgXzmp X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY4PR15MB1318; X-Microsoft-Exchange-Diagnostics: 1;CY4PR15MB1318;25:Dc9nFKwscQSCjRjDvqhoOiKxw+B8F8EpLa+jBBmXlnuDjQFH1fEZsSWYtYQxTJafr9V0KO08ylttdwgfa4lj3ExBe8EF3aiC9hsud3mI4Cbdh6LS5LH2yWpH3BaTC5JN/JnchjtZWH3dDI0S3CbLgtg6bCWJFKVczC/3s7r6SKb2SDixpnlzRyjlpJ1DJIiB5dQfplCHs9QhZE3q6pnt7MSTa0RVtnuuZXovbKfbeusrLo1Adi8PeFc3ghRNFacdd5zf2S2Je3XzH3Pny9HTsYvh3V6k8L9+R42jZkwdU54ZlnLorSzRmzOgCaHh6eIuZ+nGGXJ6P+2O7AfS6pSSFlQ9aMX7Z0lXnv6lyN6yp0OHFcEzvUpJ2xILwKX3bvPRJ2AChq7I6wg46WTr3tkGcnsPfvEy1aZwLSy3A1zxuwiAfbDnkt6vA2f1NVseGNqIzbnBbQhxjwXlOG/cAKqGUsc13JFbo3dvrZzSUV/NSHJ57bhhQ8D3MxAddrKFv4GzONZsfWErUeeFqyQ6FUpi8gVD2TVO+JMzODGaUEHwNQ8j3bPFEJr3qDpuGo4Oq+XC2Oc/H77gmiL6q3I7Keb+74RLXgQdD/PpADFIQLOI1Fu4C3uTU5C24hBa78oXV9edQuOcAKYGWqgJ9b58Ckv5Mh51liwFEcVsUThE5qc66P9RJ4WEhgcNPEncJS0mdDlWnRUrc65/UZWwQdipfdmXg2A/WtyDAEr4RJQ1AF5prwg= X-Microsoft-Exchange-Diagnostics: 1;CY4PR15MB1318;31:zMnkfvFHQM/1K0ql7iz7KtHRbgsHinhkqcWisbl6GSPtJtg5QWX+0wWE/xBLKNupy79jIafwRiEFh3CgG+mUfTZOhhjo/CFhaMbxoaLRzyDdyWXK8Spyo25D2dHFnDj4Z7vRggyg9Dk5nelXM2mvvfeY/WReXY4fjwH6tDu/4WGtHvZZjhb0AeSiJB6jr32ZMFIGYkcO5kFt/XyUNh5oZdHYZRPQOQBNJx8PZ1vTTFY=;20:sEd5SmpMGMjNCfrYAOlJSnNe0uVwIbmbEY/8e/+0koFiuW5aK24qZSMhWCPHAk+1Ew2YM/MH3hBM2xi2RIn3qXww27/FTJuadaGdk8ERFdaWweLy8v3Ot4UsgXNKEytVka5ZGZVMGVpC3toZ08SGtA8lSgPSpGLnbIFL9x+BXN4=;4:mpSnAjS1FyG5nlvx4OTfAdtQgTVe7onjtEHpSbejaXxMGVIo93kyzwmhdcS4SPlPjrE4chjFi/lyR6uvPp7w0o7WgivQg3DWLUW/Bem6gJrKVC0BaSD+yXt0mJqQwn4OQEkJ284fQGAwAC+6qwnOwHnxnmbra4VMDsp/18fW6r54MLPWRBvKJpoZzxU7YRKhD76LUfy+Xj0xB2T/ebcUWXRGMXPL7W5DSv0iiO88KNXWQGEwnJhbyztb7o8AOWzb4jnjBpaq0QmSCD5exWqzU+h+2LY156kgs3c2a8EWK6XTpHsHXi1bhSSYEEblMqkyGZJyuS+oEFwavPMxoX1PWJgvVcVxtL7240F6ekFGVvSu2lFKivWhr1SwSOZhc52i7lbhynFnnkSwyAinf7KN9A== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046);SRVR:CY4PR15MB1318;BCL:0;PCL:0;RULEID:;SRVR:CY4PR15MB1318; X-Forefront-PRVS: 00808B16F3 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6069001)(7916002)(189002)(24454002)(377454003)(199003)(86362001)(4326007)(50466002)(8676002)(31696002)(2906002)(4001350100001)(64126003)(97736004)(83506001)(5660300001)(230700001)(6916009)(81166006)(6666003)(106356001)(2950100002)(81156014)(189998001)(105586002)(77096005)(93886004)(47776003)(586003)(42186005)(31686004)(33646002)(6116002)(3846002)(23746002)(65956001)(19580405001)(19580395003)(36756003)(305945005)(50986999)(66066001)(65826007)(54356999)(68736007)(7736002)(76176999)(65806001)(92566002)(101416001)(7846002)(110136003)(781001);DIR:OUT;SFP:1102;SCL:1;SRVR:CY4PR15MB1318;H:localhost.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;CY4PR15MB1318;23:ZKJknjj4Fv2kiR67loqoFMz6Dt/6bpi0pAbcx?= =?Windows-1252?Q?Zo70A9LOTqYcx/oCFyLIjKtW2aQ4oWhy3RSmWgZGXSeLy9o2Bmxm0fJo?= =?Windows-1252?Q?eVk2AVwdWcx+9rKkA5gwrjntg9B0alntI4f7U3iciDt+/Ymp+XP80667?= =?Windows-1252?Q?Jhq0K18drptr8avn++8Z/uGDtaHTlBxsWKUuNAMnOdOplbFus0YiGLcs?= =?Windows-1252?Q?bUyc5Ih9a5is+2C7nCDsIZPsRvF9InjQ8v1xsNkWX7Mf8kAPhkRmzxKq?= =?Windows-1252?Q?c60IJyjzZ/juF6dnbw1m6SbGwnJUp7wGgufd8+8cUcpCyROmjRjYxH7Z?= =?Windows-1252?Q?NoeIpmJ9xH++jzzJC7AdcaUxDdO4/r0RZ5A9QJqhEOACROyxFEp8UXcP?= =?Windows-1252?Q?JUz9GdLLtwyHph37cnD/oJcZWWqkPcbJx3aABpZwEVxTTPDFJ3q5lM8N?= =?Windows-1252?Q?9mEyBXVlTz2xB+LBf4tFXY+8+EJXSJ7dUdwLq6VFMg17uVmruxqHR5eG?= =?Windows-1252?Q?l1vpizoJa6Y6vn1KUXI/5mUAvmDwfUeeXqNBCGGXe6LZSd4IzmpqVFPg?= =?Windows-1252?Q?UKLwn5z1vf6vhtvYLaUmwIYPz+x/f7Kuc6HBTCEvB+Y/YJ8HwGraZ6nh?= =?Windows-1252?Q?wH4JJCpD0QuCEIITp9d2k7Wlxgu4axUmPql/YD7G8vULEPLJJZXr7Jsk?= =?Windows-1252?Q?sRzOrnXNCVfphGxIZ90S8K2vkOkGPDn3EPxH6vX5B5brW1T4HVw47dZa?= =?Windows-1252?Q?/B/2gSdATf1vMY2Jyglmg4VG0YGPZs4R3QPVFaeAlg9bG6E46H+D0BFG?= =?Windows-1252?Q?eE27YtCEVV4KL4tXTfNmmWzbcILp28kQ2FUFe/66gsQXOEJGay8PR/oy?= =?Windows-1252?Q?ZFGu7Q2I6ejKBCla5IwJXBr5aKWuSBYwlOMRzWZ2mEwzrC4q4mOJXaW5?= =?Windows-1252?Q?6pXbV8A+YTnMOjhx89pUszNXivnmo+TspK600FGLYu4W4youpm2+SHw+?= =?Windows-1252?Q?ypwijFK0zLKEuIcdl4n3P2gctNcywTHc3rxIbV9eDQ/J+SVhY+a6K5xq?= =?Windows-1252?Q?2ZJr/kMDfmWuUmcOc8xOXvNb707ZmTg2moyzn5MmZhQKC9mGFL6VeQe5?= =?Windows-1252?Q?GsmLPEpeEfS1QJ2BvgdSr/UZunvMtFYizAnrQlPSb2Ylm3Zh0P/wlJC/?= =?Windows-1252?Q?hP699VawZZOLgVxHMotVaP2WMnLoYctcFy+bGlGK66gOsYHj+EJ+61fm?= =?Windows-1252?Q?V+cnbRXVdFsNrXEyTpB16OJiMOdsy/4Q0o3YFuZrThL2WnQ0FVdq/cKk?= =?Windows-1252?Q?vE8eInP+3K64j0oxS588qzocJnEh8/gvKTQlTf6Rl4D1NjrhoNZW0dXM?= =?Windows-1252?Q?8C8dwwatBllPf/mgPVATsHbnwRfA1wZjb4T86teiuTDbC0SRmqOIfUNo?= =?Windows-1252?Q?kqJOM4Bn7E/E3BaBBBqzWa8KDxzv2XWaEPatseW/Q=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;CY4PR15MB1318;6:fKMaAN5p8MYFDvHoaNcnJqPvKeC+FX/AGp5wbbuReuEeNM8UXt6539A9cC5z+/YLhHmKJoFYd3IQ+yjtnw9vS0GNWqp/BnPgJ8ewghlnZyZ+6VlqwXuxWdudNSYIOxC0T3uAqL1bFVy14c1cA902g1UZNZxVQPxFsblXaC8XnG+Tax8er05uYALsUQPsvF+UfLXpvTwECMJAKBlXM66mhRbdcTu13unlyOoswK34a6fP8daqH/hR5ZTHr15JRyYS0rHC7FiLft9vI7f8KqMe5xPunB6DKK5duMrOby8h7rs=;5:6VLX+x9ewZhg15EPmibfkzkxJnGvYri6+pv3tqpeQYAIHFV6gVFvqpQlpPWlqD8fqkzEqQTh8m4dZpEwyVoXIdCBeVvijhaObpgphiJBNpsavdUEVRvx6BJkfD5y7OnkzEmffWjcMtzB5iGioF6lTQ==;24:FTqL3X8mSN+c34UoO7sSmuBqRAyFNXZ3nqD1+99ndZnywrksDFQFvccv7AbY4LaVSUDda69hYiRLM6aJEa0OIZdVs+hmyTn0ASEvMyVwL0Y=;7:EgHJsjPMtJLmiyD3hn8c3TeE6LX93yAjCF8uWrdNQ6m5JAvh7MbSIXf3J/CceVVTOXtGHeqi7gZkxjOsT9IGKXdKPJkTnrG3voTcRAfSIasac+SKGfdndqhz8m0wjihlBv8UA1g8JYA4qE0FeXWhffRNtkFSSeby/bzAtn8myik+eIkO1W8hirZf2V90IriyLNgCZpxXB/1er8Bo29hxGuFJwFHy/eRYuAgqrozIwjPGdqPKpRjVwaj/VXbvB0kwmQmHjIvV+X5/nWOm9OrZUwRh24pcSwV0Z3rs06TxPABKOEGpn7CBI17fbmLUoK5A SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;CY4PR15MB1318;20:ppO9rQIZFLyp225NeK22veJNri6oyG4cYrW0eKoSHAT5KE417yeQrM7EmoTE9JXWjNWvyF1P4EFhgDAA8JX/7y/zM9xOcJrR3TidgiYL3PN38ii9vSGRoyNmvmLH2WClZUOFROz1ZS7fHpy7SC7UH4TFWkHI8cHMwTs+OnILUAg= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2016 16:59:22.6433 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR15MB1318 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-09-29_11:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/29/2016 12:41 PM, Wouter Verhelst wrote: > On Thu, Sep 29, 2016 at 10:03:50AM -0400, Josef Bacik wrote: >> So think of it like normal disks with multiple channels. We don't send flushes >> down all the hwq's to make sure they are clear, we leave that decision up to the >> application (usually a FS of course). > > Well, when I asked earlier, Christoph said[1] that blk-mq assumes that > when a FLUSH is sent over one channel, and the reply comes in, that all > commands which have been received, regardless of which channel they were > received over, have reched disk. > > [1] Message-ID: <20160915122304.GA15501@infradead.org> > > It is impossible for nbd to make such a guarantee, due to head-of-line > blocking on TCP. > Huh I missed that. Yeah that's not possible for us for sure, I think my option idea is the less awful way forward if we want to address that limitation. Thanks, Josef