From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ross Zwisler Subject: Re: DAX can not work on virtual nvdimm device Date: Thu, 1 Sep 2016 20:57:38 -0600 Message-ID: <20160902025738.GA26108@linux.intel.com> References: <436d7526-bf06-633d-afce-4333552d9e31@linux.intel.com> <20160819183047.GA7216@linux.intel.com> <600ac51c-0f61-6e53-9bfa-669c85494d1f@linux.intel.com> <20160829193014.GB16738@linux.intel.com> <61d5c470-a8bf-86d5-d1fe-b4b577a85967@linux.intel.com> <25098f37-53f7-6d5d-0b1a-8469bab51a9f@linux.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from mga06.intel.com ([134.134.136.31]:55485 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750786AbcIBC5m (ORCPT ); Thu, 1 Sep 2016 22:57:42 -0400 Content-Disposition: inline In-Reply-To: <25098f37-53f7-6d5d-0b1a-8469bab51a9f@linux.intel.com> Sender: linux-acpi-owner@vger.kernel.org List-Id: linux-acpi@vger.kernel.org To: Xiao Guangrong , linux-ext4@vger.kernel.org, Jan Kara , Theodore Ts'o Cc: Dan Williams , Ross Zwisler , Yumei Huang , KVM , "linux-nvdimm@lists.01.org" , "qemu-devel@nongnu.org" , LKML , Linux ACPI , Stefan Hajnoczi On Wed, Aug 31, 2016 at 04:44:47PM +0800, Xiao Guangrong wrote: > On 08/31/2016 01:09 AM, Dan Williams wrote: > > > > Can you post your exact reproduction steps? This test is not failing for me. > > > > Sure. > > 1. make the guest kernel based on your tree, the top commit is > 10d7902fa0e82b (dax: unmap/truncate on device shutdown) and > the config file can be found in this thread. > > 2. add guest kernel command line: memmap=6G!10G > > 3: start the guest: > x86_64-softmmu/qemu-system-x86_64 -machine pc,nvdimm --enable-kvm \ > -smp 16 -m 32G,maxmem=100G,slots=100 /other/VMs/centos6.img -monitor stdio > > 4: in guest: > mkfs.ext4 /dev/pmem0 > mount -o dax /dev/pmem0 /mnt/pmem/ > echo > /mnt/pmem/xxx > ./mmap /mnt/pmem/xxx > ./read /mnt/pmem/xxx > > The source code of mmap and read has been attached in this mail. > > Hopefully, you can detect the error triggered by read test. > > Thanks! Okay, I think I've isolated this issue. Xiao's VM was an old CentOS 6 system, and for some reason ext4+DAX with the old tools found in that VM fails. I was able to reproduce this failure with a freshly installed CentOS 6.8 VM. You can see the failure with his tests, or perhaps more easily with this series of commands: # mkfs.ext4 /dev/pmem0 # mount -o dax /dev/pmem0 /mnt/pmem/ # touch /mnt/pmem/x # md5sum /mnt/pmem/x md5sum: /mnt/pmem/x: Bad address This sequence of commands works fine in the old CentOS 6 system if you use XFS instead of ext4, and it works fine with both ext4 and XFS in CentOS 7 and with recent versions of Fedora. I've added the ext4 folks to this mail in case they care, but my guess is that the tools in CentOS 6 are so old that it's not worth worrying about. For reference, the kernel in CentOS 6 is based on 2.6.32. :) DAX was introduced in v4.0. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) (using TLSv1 with cipher CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 4A0001A1E74 for ; Thu, 1 Sep 2016 19:57:40 -0700 (PDT) Date: Thu, 1 Sep 2016 20:57:38 -0600 From: Ross Zwisler Subject: Re: DAX can not work on virtual nvdimm device Message-ID: <20160902025738.GA26108@linux.intel.com> References: <436d7526-bf06-633d-afce-4333552d9e31@linux.intel.com> <20160819183047.GA7216@linux.intel.com> <600ac51c-0f61-6e53-9bfa-669c85494d1f@linux.intel.com> <20160829193014.GB16738@linux.intel.com> <61d5c470-a8bf-86d5-d1fe-b4b577a85967@linux.intel.com> <25098f37-53f7-6d5d-0b1a-8469bab51a9f@linux.intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <25098f37-53f7-6d5d-0b1a-8469bab51a9f@linux.intel.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Xiao Guangrong , linux-ext4@vger.kernel.org, Jan Kara , Theodore Ts'o Cc: Yumei Huang , KVM , "linux-nvdimm@lists.01.org" , "qemu-devel@nongnu.org" , LKML , Linux ACPI , Stefan Hajnoczi List-ID: On Wed, Aug 31, 2016 at 04:44:47PM +0800, Xiao Guangrong wrote: > On 08/31/2016 01:09 AM, Dan Williams wrote: > > > > Can you post your exact reproduction steps? This test is not failing for me. > > > > Sure. > > 1. make the guest kernel based on your tree, the top commit is > 10d7902fa0e82b (dax: unmap/truncate on device shutdown) and > the config file can be found in this thread. > > 2. add guest kernel command line: memmap=6G!10G > > 3: start the guest: > x86_64-softmmu/qemu-system-x86_64 -machine pc,nvdimm --enable-kvm \ > -smp 16 -m 32G,maxmem=100G,slots=100 /other/VMs/centos6.img -monitor stdio > > 4: in guest: > mkfs.ext4 /dev/pmem0 > mount -o dax /dev/pmem0 /mnt/pmem/ > echo > /mnt/pmem/xxx > ./mmap /mnt/pmem/xxx > ./read /mnt/pmem/xxx > > The source code of mmap and read has been attached in this mail. > > Hopefully, you can detect the error triggered by read test. > > Thanks! Okay, I think I've isolated this issue. Xiao's VM was an old CentOS 6 system, and for some reason ext4+DAX with the old tools found in that VM fails. I was able to reproduce this failure with a freshly installed CentOS 6.8 VM. You can see the failure with his tests, or perhaps more easily with this series of commands: # mkfs.ext4 /dev/pmem0 # mount -o dax /dev/pmem0 /mnt/pmem/ # touch /mnt/pmem/x # md5sum /mnt/pmem/x md5sum: /mnt/pmem/x: Bad address This sequence of commands works fine in the old CentOS 6 system if you use XFS instead of ext4, and it works fine with both ext4 and XFS in CentOS 7 and with recent versions of Fedora. I've added the ext4 folks to this mail in case they care, but my guess is that the tools in CentOS 6 are so old that it's not worth worrying about. For reference, the kernel in CentOS 6 is based on 2.6.32. :) DAX was introduced in v4.0. _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752203AbcIBC5r (ORCPT ); Thu, 1 Sep 2016 22:57:47 -0400 Received: from mga06.intel.com ([134.134.136.31]:55485 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750786AbcIBC5m (ORCPT ); Thu, 1 Sep 2016 22:57:42 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.30,269,1470726000"; d="scan'208";a="163685724" Date: Thu, 1 Sep 2016 20:57:38 -0600 From: Ross Zwisler To: Xiao Guangrong , linux-ext4@vger.kernel.org, Jan Kara , "Theodore Ts'o" Cc: Dan Williams , Ross Zwisler , Yumei Huang , KVM , "linux-nvdimm@lists.01.org" , "qemu-devel@nongnu.org" , LKML , Linux ACPI , Stefan Hajnoczi Subject: Re: DAX can not work on virtual nvdimm device Message-ID: <20160902025738.GA26108@linux.intel.com> References: <436d7526-bf06-633d-afce-4333552d9e31@linux.intel.com> <20160819183047.GA7216@linux.intel.com> <600ac51c-0f61-6e53-9bfa-669c85494d1f@linux.intel.com> <20160829193014.GB16738@linux.intel.com> <61d5c470-a8bf-86d5-d1fe-b4b577a85967@linux.intel.com> <25098f37-53f7-6d5d-0b1a-8469bab51a9f@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <25098f37-53f7-6d5d-0b1a-8469bab51a9f@linux.intel.com> User-Agent: Mutt/1.7.0 (2016-08-17) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Aug 31, 2016 at 04:44:47PM +0800, Xiao Guangrong wrote: > On 08/31/2016 01:09 AM, Dan Williams wrote: > > > > Can you post your exact reproduction steps? This test is not failing for me. > > > > Sure. > > 1. make the guest kernel based on your tree, the top commit is > 10d7902fa0e82b (dax: unmap/truncate on device shutdown) and > the config file can be found in this thread. > > 2. add guest kernel command line: memmap=6G!10G > > 3: start the guest: > x86_64-softmmu/qemu-system-x86_64 -machine pc,nvdimm --enable-kvm \ > -smp 16 -m 32G,maxmem=100G,slots=100 /other/VMs/centos6.img -monitor stdio > > 4: in guest: > mkfs.ext4 /dev/pmem0 > mount -o dax /dev/pmem0 /mnt/pmem/ > echo > /mnt/pmem/xxx > ./mmap /mnt/pmem/xxx > ./read /mnt/pmem/xxx > > The source code of mmap and read has been attached in this mail. > > Hopefully, you can detect the error triggered by read test. > > Thanks! Okay, I think I've isolated this issue. Xiao's VM was an old CentOS 6 system, and for some reason ext4+DAX with the old tools found in that VM fails. I was able to reproduce this failure with a freshly installed CentOS 6.8 VM. You can see the failure with his tests, or perhaps more easily with this series of commands: # mkfs.ext4 /dev/pmem0 # mount -o dax /dev/pmem0 /mnt/pmem/ # touch /mnt/pmem/x # md5sum /mnt/pmem/x md5sum: /mnt/pmem/x: Bad address This sequence of commands works fine in the old CentOS 6 system if you use XFS instead of ext4, and it works fine with both ext4 and XFS in CentOS 7 and with recent versions of Fedora. I've added the ext4 folks to this mail in case they care, but my guess is that the tools in CentOS 6 are so old that it's not worth worrying about. For reference, the kernel in CentOS 6 is based on 2.6.32. :) DAX was introduced in v4.0. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50980) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bfegB-0007Eu-JS for qemu-devel@nongnu.org; Thu, 01 Sep 2016 22:58:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bfeg8-0000HH-Do for qemu-devel@nongnu.org; Thu, 01 Sep 2016 22:58:07 -0400 Received: from mga07.intel.com ([134.134.136.100]:2228) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bfeg8-000065-7u for qemu-devel@nongnu.org; Thu, 01 Sep 2016 22:58:04 -0400 Date: Thu, 1 Sep 2016 20:57:38 -0600 From: Ross Zwisler Message-ID: <20160902025738.GA26108@linux.intel.com> References: <436d7526-bf06-633d-afce-4333552d9e31@linux.intel.com> <20160819183047.GA7216@linux.intel.com> <600ac51c-0f61-6e53-9bfa-669c85494d1f@linux.intel.com> <20160829193014.GB16738@linux.intel.com> <61d5c470-a8bf-86d5-d1fe-b4b577a85967@linux.intel.com> <25098f37-53f7-6d5d-0b1a-8469bab51a9f@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <25098f37-53f7-6d5d-0b1a-8469bab51a9f@linux.intel.com> Subject: Re: [Qemu-devel] DAX can not work on virtual nvdimm device List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Xiao Guangrong , linux-ext4@vger.kernel.org, Jan Kara , Theodore Ts'o Cc: Dan Williams , Ross Zwisler , Yumei Huang , KVM , "linux-nvdimm@lists.01.org" , "qemu-devel@nongnu.org" , LKML , Linux ACPI , Stefan Hajnoczi On Wed, Aug 31, 2016 at 04:44:47PM +0800, Xiao Guangrong wrote: > On 08/31/2016 01:09 AM, Dan Williams wrote: > > > > Can you post your exact reproduction steps? This test is not failing for me. > > > > Sure. > > 1. make the guest kernel based on your tree, the top commit is > 10d7902fa0e82b (dax: unmap/truncate on device shutdown) and > the config file can be found in this thread. > > 2. add guest kernel command line: memmap=6G!10G > > 3: start the guest: > x86_64-softmmu/qemu-system-x86_64 -machine pc,nvdimm --enable-kvm \ > -smp 16 -m 32G,maxmem=100G,slots=100 /other/VMs/centos6.img -monitor stdio > > 4: in guest: > mkfs.ext4 /dev/pmem0 > mount -o dax /dev/pmem0 /mnt/pmem/ > echo > /mnt/pmem/xxx > ./mmap /mnt/pmem/xxx > ./read /mnt/pmem/xxx > > The source code of mmap and read has been attached in this mail. > > Hopefully, you can detect the error triggered by read test. > > Thanks! Okay, I think I've isolated this issue. Xiao's VM was an old CentOS 6 system, and for some reason ext4+DAX with the old tools found in that VM fails. I was able to reproduce this failure with a freshly installed CentOS 6.8 VM. You can see the failure with his tests, or perhaps more easily with this series of commands: # mkfs.ext4 /dev/pmem0 # mount -o dax /dev/pmem0 /mnt/pmem/ # touch /mnt/pmem/x # md5sum /mnt/pmem/x md5sum: /mnt/pmem/x: Bad address This sequence of commands works fine in the old CentOS 6 system if you use XFS instead of ext4, and it works fine with both ext4 and XFS in CentOS 7 and with recent versions of Fedora. I've added the ext4 folks to this mail in case they care, but my guess is that the tools in CentOS 6 are so old that it's not worth worrying about. For reference, the kernel in CentOS 6 is based on 2.6.32. :) DAX was introduced in v4.0.