From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759822AbcHaWJW (ORCPT ); Wed, 31 Aug 2016 18:09:22 -0400 Received: from mail-bn3nam01on0096.outbound.protection.outlook.com ([104.47.33.96]:56722 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753126AbcHaWJT (ORCPT ); Wed, 31 Aug 2016 18:09:19 -0400 X-Greylist: delayed 6507 seconds by postgrey-1.27 at vger.kernel.org; Wed, 31 Aug 2016 18:09:19 EDT From: "Kani, Toshimitsu" To: "ross.zwisler@linux.intel.com" CC: "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "adilger.kernel@dilger.ca" , "viro@zeniv.linux.org.uk" , "dan.j.williams@intel.com" , "akpm@linux-foundation.org" , "mawilcox@microsoft.com" , "linux-nvdimm@lists.01.org" , "linux-fsdevel@vger.kernel.org" , "jack@suse.com" , "tytso@mit.edu" , "linux-ext4@vger.kernel.org" , "david@fromorbit.com" Subject: Re: [PATCH v2 0/9] re-enable DAX PMD support Thread-Topic: [PATCH v2 0/9] re-enable DAX PMD support Thread-Index: AQHR/YpWM0cWcWykREGQ2zcAz/gn16BiKeYAgAFlHYCAABVFgIAACO0A Date: Wed, 31 Aug 2016 22:08:59 +0000 Message-ID: <1472681284.2092.30.camel@hpe.com> References: <20160823220419.11717-1-ross.zwisler@linux.intel.com> <20160830230150.GA12173@linux.intel.com> <1472674799.2092.19.camel@hpe.com> <20160831213607.GA6921@linux.intel.com> In-Reply-To: <20160831213607.GA6921@linux.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=toshi.kani@hpe.com; x-originating-ip: [15.219.163.9] x-ms-office365-filtering-correlation-id: 57fda9ed-0041-40db-de14-08d3d1eb68e8 x-microsoft-exchange-diagnostics: 1;DF4PR84MB0010;6:GVtK+eeMnoaDezmXG0OuhLQWQA2oAOhPMSmDButJuNU1pF/0m1v2xB6eyzT3LNiU6aCs9s0k+95t9sPW8qQKnZOu8jAdi9pLdCnG4dN8qOLLsAV4HAjz+hdIED56+x+5azf5FUabKnq+rkwcnyv4AO4N0O8wPkzs5co5mym3VfuNSM3UNc5oJWRNcIcU1ML+rWSOd98LGwvKGe4zIQovIrCVAa4jkDYhpmDST+VRI3w4HbwRIhXtPjrY+aA9dW+qPJiL41/nd4wkYKaV/WF4WIFLnFT5LAWv1uM178IpTehjs8OgtgYJo9iK1xPGpe8qo8aEDvCuSxY1grAqedj7jg==;5:OUJoCrL1/Jp5ePMQCLfnMWo7sBcxDMoZDeU+AEA4EAW8DSec+K1yBs9iwb1UoXSgF41e7HsJaPTHuD6qQSVbtsX6nxcKwhYwo94xXyjZHwrUtpFNCYrl4KARysjucZjFrAkg+tBIDMlfoLHJyLYjPw==;24:y2IALXIYZIrZapozwYsvEdn4nCamA9UDLXR1ce4OSzz6h6DvWtEQ2cpPRPru7hnhBYZlgrE1FXnkLGXWpBhKIcYNgNaG9S7X3z9Vn6KYohw=;7:cJx6pKmDM4B3YrMUHfhHeLoTPY+5ZNBX+M8Y6nfBqCWmdGcx9IZuG+Wswr4nFLFOzzhv0aAr93VXuX8wXvqDejBWYCOgQbDuLKZBGi/OOskQ2oTS59Z7Hap3TGoyRcu3dv81q2mJ11cqOWqENYeIEstrPpdEnFE1YtqitTO7IwUmPV4DrQBOEJJxQZC/sYtipkVKlEnagPpRu8dgFyb1lpUNRy1tJMvkqh73AtJ0d532c/FkHxl2WdPJesEsIt4j x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DF4PR84MB0010; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026);SRVR:DF4PR84MB0010;BCL:0;PCL:0;RULEID:;SRVR:DF4PR84MB0010; x-forefront-prvs: 00514A2FE6 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6009001)(7916002)(199003)(189002)(51914003)(377424004)(24454002)(2900100001)(19580395003)(105586002)(305945005)(7846002)(5002640100001)(122556002)(8936002)(3280700002)(106356001)(2950100001)(7416002)(5660300001)(92566002)(110136002)(3846002)(36756003)(2351001)(6116002)(3660700001)(189998001)(8666005)(7736002)(102836003)(33646002)(68736007)(10400500002)(81156014)(99286002)(77096005)(76176999)(11100500001)(103116003)(50986999)(97736004)(586003)(5890100001)(86362001)(54356999)(8676002)(2501003)(106116001)(66066001)(4326007)(81166006)(93886004)(2906002)(101416001)(87936001)(217873001);DIR:OUT;SFP:1102;SCL:1;SRVR:DF4PR84MB0010;H:DF4PR84MB0010.NAMPRD84.PROD.OUTLOOK.COM;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: <9CF8BC9D254DF144AF101EE6F7D0B3D0@NAMPRD84.PROD.OUTLOOK.COM> MIME-Version: 1.0 X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2016 22:08:59.4170 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 105b2061-b669-4b31-92ac-24d304d195dc X-MS-Exchange-Transport-CrossTenantHeadersStamped: DF4PR84MB0010 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id u7VM9R5Q008043 On Wed, 2016-08-31 at 15:36 -0600, Ross Zwisler wrote: > On Wed, Aug 31, 2016 at 08:20:48PM +0000, Kani, Toshimitsu wrote: > > > > On Tue, 2016-08-30 at 17:01 -0600, Ross Zwisler wrote: > > > > > > On Tue, Aug 23, 2016 at 04:04:10PM -0600, Ross Zwisler wrote:  : > > > > > > Ping on this series?  Any objections or comments? > > > > Hi Ross, > > > > I am seeing a major performance loss in fio mmap test with this > > patch-set applied.  This happens with or without my patches [1] > > applied on top of yours.  Without my patches, dax_pmd_fault() falls > > back to the pte handler since an mmap'ed address is not 2MB- > > aligned. > > > > I have attached three test results. > >  o rc4.log - 4.8.0-rc4 (base) > >  o non-pmd.log - 4.8.0-rc4 + your patchset (fall back to pte) > >  o pmd.log - 4.8.0-rc4 + your patchset + my patchset (use pmd maps) > > > > My test steps are as follows. > > > > mkfs.ext4 -O bigalloc -C 2M /dev/pmem0 > > mount -o dax /dev/pmem0 /mnt/pmem0 > > numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio > > test.fio > > > > "test.fio" > > --- > > [global] > > bs=4k > > size=2G > > directory=/mnt/pmem0 > > ioengine=mmap > > [randrw] > > rw=randrw > > --- > > > > Can you please take a look? > > Yep, thanks for the report. I have some more observations.  It seems this issue is related with pmd mappings after all.  fio creates "randrw.0.0" file.  In my setup, an initial test run creates pmd mappings and hits this issue.  Subsequent test runs (i.e. randrw.0.0 exists), without my patches, fall back to pte mappings and do not hit this issue.  With my patches applied, subsequent runs still create pmd mappings and hit this issue. Thanks, -Toshi