From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754725AbcEBRp3 (ORCPT ); Mon, 2 May 2016 13:45:29 -0400 Received: from mail-bl2on0135.outbound.protection.outlook.com ([65.55.169.135]:58712 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753818AbcEBRpU (ORCPT ); Mon, 2 May 2016 13:45:20 -0400 Authentication-Results: infradead.org; dkim=none (message not signed) header.d=none;infradead.org; dmarc=none action=none header.from=hpe.com; Message-ID: <57279224.7030702@hpe.com> Date: Mon, 2 May 2016 13:45:08 -0400 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: Christoph Hellwig CC: "Theodore Ts'o" , Andreas Dilger , Alexander Viro , Matthew Wilcox , , , Dave Chinner , Scott J Norton , Douglas Hatch , Toshimitsu Kani Subject: Re: [PATCH v5 0/2] ext4: Improve parallel I/O performance on NVDIMM References: <1461947276-25988-1-git-send-email-Waiman.Long@hpe.com> <57238DFC.6010108@hpe.com> <20160501172854.GA19601@infradead.org> In-Reply-To: <20160501172854.GA19601@infradead.org> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [72.71.243.175] X-ClientProxiedBy: CY1PR14CA0006.namprd14.prod.outlook.com (10.163.13.144) To AT5PR84MB0308.NAMPRD84.PROD.OUTLOOK.COM (10.162.138.30) X-MS-Office365-Filtering-Correlation-Id: 8e58c750-85dd-4829-4881-08d372b185b9 X-Microsoft-Exchange-Diagnostics: 1;AT5PR84MB0308;2:jdwSQM/zU0Pia2K2qHKFlj25luZ+ss+bp64GEXdr741pZ2m60v7WuLw6wByXFFLfgCxWNg0YWjdSUdb28QWigKeeVD0abL0jYMLd2CoEpU2rBoCv827eiJTfqhDAOu2pF0yzHCmcq4QP1G2nNDjdv7Q6GBLFTxXfBghMxQ7Wzvi82pU6cW+229VyPP9B9nfj;3:lr4h3lCXWV3bzrpEjqpVrHZrf16kv61nNMzFJbtJiOCrTzJYU1eXcme2FTBJUD3QnatxLGaj9dKxwrWVeYtk7rsZVj4zZmL76CIkc6bEUuB/DQ0IrZiMgZpsrW5feuaZ X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AT5PR84MB0308; X-Microsoft-Exchange-Diagnostics: 1;AT5PR84MB0308;25:S9gjT/4zcvj/iGykq7ZKqF+rMLINDuZajiOs5eTnsBRESUeggHfrfQxPvjgcrzoxFZZxDQMDXEi42BLYDliy57HhxQL24we/ZGCf1YbccQYVYiMt562mCM01lqKuodDK4fSdWVDIrcgXKf25KM9q3RQfHucaHx6HtqD9yulhefLhtxRu9DoAHveszkW7t6epROy85FE9TkuLQTfF8wZ0+NnVJAkJK4VymV6WwLy6yTxRUpRxw90UdruoOg4W3y9BS1XAqNuuY8ycpamlgNfonBhM169DQ9fvLBDDBi00lPvhSbHRCDsPRfCa1KXEGBvARZ2CTknA9ao/g8vfbcz9byZR5yvzGWIqzsQ3TPRqI2laHct1C6viwD7OXtNNJOHsQIpAWATLTRrcR/eRyRZ8KdqVFLrcHjLeLKmV8qO7nQw11v1JsT6FYJ+zvfQFfe4A1Qtut8x7yVvs7Edod/QaqDlRwSJzDL+wejxa3gAhCOtPBrhrE6W/NzNlxR2kJwjVO6mGfl6pdMLgTtBmew9iPRxMQCUr2iuPX66pIxCdTCLtGOQhePSLTkLSCHttRotPa3iAo+fk6cHFhJty78nNvGzh8Jnng5XOCsc/JKwn+Gd9CA+KEWud9zRI9JQ8cl/CEiofWUTaWX5Y/+B0reOXpLfcBTYq9JTISfuzDrpG7H2Eg13YPD9/s/r5h2MtBvcAjJrO8e6c22dBJrUihy8Yz457E36452jo1KG/OCq44rByLgSSc3G7WCCeideyg3zG X-LD-Processed: 105b2061-b669-4b31-92ac-24d304d195dc,ExtAddr X-Microsoft-Exchange-Diagnostics: 1;AT5PR84MB0308;20:d9PcpYaNbcPundaAAqZEAoy4FQQK0eR0+TeB/S/vookwA+g1XqsTMshyz7JfRFCXQrkqxNGYBhgZjK+lecVwaP+Gftu+gXyNO4VAD88qqfgs8Ofj764BeZGCbp+eal8Rf3eEteb1II10SzBs3HJd9ahVATnVjhrBw2pHD3tA0GXOiWfiF8vGeMteOofellIdiw1Vxfn7VRg/Gv2498oH7BwppNgoTcZx2gG1lFgYXtMZ7na2N7QvVjPstWMx8PJkPa2A2ihZaBuek88WSjQ9f5Ia4blyNPypB2zgznhlhFmmBNuAoIZ2ZsAXscuniCh/SEhRNFSDkQukQ3AeIaKsRw==;4:+aHRnFDBsV3Ax6FUuskXbrLKCi3uPoFw9hTxEHNcippR9zIJjx6LWSX6fsue6uL3x0u8UfMFXI3Yu1r2bpn3K/OjcHOHIf0bxR1gdAZ4zE0QDPHZaG6re1CEbf4Q5ECG8iUYXMN6N3dSgPZAu8JnRC8TCjV7SLFpearwaB6lLKm02zqD03Gm+Hs7VDPweiUp17EfcHV3TNTRVRlgsqhJRN6+s+gz43myL7RboGg/mp3lWhN4mDiBdU2HFZPq+iZjFgN/IqgRXa16Zp/nkSB1MHVi/doX+OriB+6aQrLeWMGUtrWsdbHQyu04khtqu1dC+3K8ibRnikpkA4h5/oKm/cglDOU3mpswEDcM6WApTpW6Gdqn6JnUnpvoZSYXE+AfOzfz5XZNVc3s+o9rE/8p2g== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(9101521072)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001);SRVR:AT5PR84MB0308;BCL:0;PCL:0;RULEID:;SRVR:AT5PR84MB0308; X-Forefront-PRVS: 0930AAFAD9 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6049001)(6009001)(377454003)(24454002)(3846002)(6116002)(586003)(4001350100001)(5004730100002)(77096005)(1096002)(65956001)(66066001)(76176999)(65816999)(87266999)(65806001)(54356999)(47776003)(4326007)(122286003)(2950100001)(42186005)(81166005)(92566002)(83506001)(5008740100001)(110136002)(50466002)(36756003)(23756003)(117636001)(50986999)(117156001)(189998001)(59896002)(230700001)(2906002)(86362001)(64126003)(62816006);DIR:OUT;SFP:1102;SCL:1;SRVR:AT5PR84MB0308;H:[192.168.142.166];FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;AT5PR84MB0308;23:rAJbSg+J3SreBAoqenPzpVrg8So1AGdOQzBM/gs?= =?iso-8859-1?Q?Cv0Q1fn5deo6tOCMmBLwkOlU7VJN6hyftOYJr8hE/xvv+keBd0nJYXTmgo?= =?iso-8859-1?Q?Soqh6wswtGcF87a2YIvucKq+lJU+Vb4yOdMw0VHSIMNYKvwb9GIDlWPqRJ?= =?iso-8859-1?Q?AMrKe9wE++Z9IMdqkJnLcWL+IHClO3m3Z5Ukjh6W2fUdL41FC8w0AysC9N?= =?iso-8859-1?Q?PHd8LskAXL0dl4X6Na/fg5gtSdULvZjxvGOqcHZ+z+UpZN0hWG3euwG0J6?= =?iso-8859-1?Q?zifUuoWMWfQdCdzasLEURIYnUSfgoAu9rSM+rWhMF5Epw7KP+AGFnS8Qhp?= =?iso-8859-1?Q?5axRUmFdgrYlTRxL4uKxi2p8zDLJuYGCZbD42I592EpW9nDaXSV+o9Jeun?= =?iso-8859-1?Q?jW/CCODabVl8HchujurxxOjPkj0eKrg+7bMFV5HQlZgjj+pUW4u+dpa9Jr?= =?iso-8859-1?Q?ZW8Rrx7DHMCLwatCay9/k8+0Iy9E4+RbgZosTjTM1PWP/pNIsQtuQ7Q+59?= =?iso-8859-1?Q?vPEJpzM83QIeYPKHIXj+bFVkmkOcdjqX7hjhlzenzXCH3pjm1UmrA8d6vg?= =?iso-8859-1?Q?XlPNH/ZxbXVoFvbQbc6NZlMcIDk4+cRizwGSE0ZYlaPNJHTnVEUtqTMZ/Y?= =?iso-8859-1?Q?7mqIVl6YGqq5SIPHtd6Akj8USN8iB1JN8EP+TX2QJyiO58a0r/4r1414iC?= =?iso-8859-1?Q?pUB4IjlUdP0BytI2Go6s+MF+/y48/b51ststBJMwUtJ+aC7MRDcBbedp/f?= =?iso-8859-1?Q?0QOZYV70cZcB+YjXcRCzGWsc4ye8rJzw2ju282Id2VIL6UehHdyBNu9Pcn?= =?iso-8859-1?Q?L8ujMNbVfYV3hjHoAOsuAKknwSXRnq9nr/H8JSkMOtdKvfUaSS2MTfNKN9?= =?iso-8859-1?Q?QbGEg/Phjmg77N38cRnfVp0PocZTh4W8WKO72SFi7oB3KAUA5dueoaeXb4?= =?iso-8859-1?Q?CuvH+sRGFwIyMfcKXWfrQYgUXxVF16QwdUIxS0UJQVdnw4GM8yGYksClXA?= =?iso-8859-1?Q?hvRPKICWKqVc1NLvPpr7d6PgeskYHGHDv7GLg3YXty+SJTMTU/rfbP5TYe?= =?iso-8859-1?Q?YPSNdTPwELnGBrSmFYcDHjOIhCdMAQ2tREKB3Y4ho2RHSBDd5ePIGf8j4+?= =?iso-8859-1?Q?Qm6LINs4UYp/OHRwpn6B72IAL1g=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;AT5PR84MB0308;5:+6Sf+mWW6LjlsPUNslQOnmiqm8ITnO0F8ulMjS0tH6C+kuE+CiKR8cjTGwsNaXIoN61araKTRQwRxAOQQvlM78ND+EO4sAaYzZXF+HK87JgYFokO9yCGlw1Zb2Tq4V6tLn0FUNu8fuiIewbia+1BbQ==;24:mZ9rYzS0yxxID24IQmi9JTXrYvB4YqzCPMzrQ5aCMUo2miGblfiSTHKvRLz4hPgPWFJE0L06VtRPlUgKWpXvL95QEUALgBHxni3m68O+R64=;7:mYbNtE83FnBu8Vf5s9d6RIgUIAVOgiWEqyx3VpZjzKNjJ0Pyw7qt0gQ/8JbGOg9uayOLNUvA1u1c7nCf7iHkBiLLtRfjRlZN82/yJm9V8EvclK6C8kRz6hCKbPQcdXWdrJ0qDeTGcP1yYY7gNTC5txBpjZkl6nlEubx+rYwVxhhPyF+OHP6wprjYKjlAXoSu SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2016 17:45:15.1255 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: AT5PR84MB0308 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/01/2016 01:28 PM, Christoph Hellwig wrote: > On Fri, Apr 29, 2016 at 12:38:20PM -0400, Waiman Long wrote: >> From my testing, it looked like that parallel overwrites to the same file in >> an ext4 filesystem on DAX can happen in parallel even if their range >> overlaps. It was mainly because the code will drop the i_mutex before the >> write. That means the overlapped blocks can get garbage. I think this is a >> problem, but I am not expert in the ext4 filesystem to say for sure. I would >> like to know your thought on that. > That's another issue with dax I/O pretending to be direct I/O.. Because > it isn't we'll need to synchronize it like buffered I/O and not like > direct I/O in all file systems. From what I saw in the code, I think filemap_write_and_wait_range() should have prevented concurrent overwrites from stepping on each other for non-DAX I/O. However it is essentially a no-op for DAX I/O and so the protection is gone. I am planning to send out a patch to disable mutex dropping for DAX overwrite. There is still an issue on the read side. If journal is disabled and the dioread_nolock mount option is used, read will done without locking. Again, the filemap_write_and_wait_range() check on the read side will not protect against write. Cheers, Longman