From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Message-ID: <1454013825.2576.31.camel@hpe.com> Subject: Re: [PATCH 0/2] Fix BTT data corruptions after crash From: Toshi Kani Date: Thu, 28 Jan 2016 13:43:45 -0700 In-Reply-To: References: <1454004770-6318-1-git-send-email-toshi.kani@hpe.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org To: Dan Williams Cc: Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Borislav Petkov , Ross Zwisler , Vishal L Verma , micah.parrish@hpe.com, brian.boylston@hpe.com, X86 ML , "linux-nvdimm@lists.01.org" , "linux-kernel@vger.kernel.org" List-ID: On Thu, 2016-01-28 at 12:12 -0800, Dan Williams wrote: > On Thu, Jan 28, 2016 at 10:12 AM, Toshi Kani wrote: > > Data corruption issues were observed in tests which initiated a system > > crash/reset while accessing BTT devices. This problem is reproducible. > > > > The BTT driver calls pmem_rw_bytes() to update data in pmem devices. > > This interface calls __copy_user_nocache(), which uses non-temporal > > stores so that the stores to pmem are persistent. > > > > __copy_user_nocache() uses non-temporal stores when a request size is > > 8 bytes or larger (and is aligned by 8 bytes). The BTT driver updates > > the BTT map table, which entry size is 4 bytes. Therefore, updates to > > the map table entries remain cached, and are not written to pmem after > > a crash. Since the BTT driver makes previous blocks free and uses them > > for subsequent writes, the map table ends up pointing to blocks > > allocated for other LBAs after a crash. > > > > Patch 1 extends __copy_user_nocache() to use non-temporal store for > > 4 byte copy. This patch fixes the BTT data corruption issue. > > > > Nice find! :-) > > Patch 2 changes arch_memcpy_to_pmem() to flush processor caches when > > a request is not naturally aligned or is less than 4 bytes. This is > > defensive change. > > I'm wondering if we should just document that this routine does not > support unaligned transfers? Maybe backed by a debug mode that does > the alignment check. Yes, I agree. For this debug mode, do you have something in mind? Or should we add a new CONFIG option like CONFIG_PMEM_DEBUG? Thanks, -Toshi From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1161111AbcA1Uos (ORCPT ); Thu, 28 Jan 2016 15:44:48 -0500 Received: from g9t5008.houston.hp.com ([15.240.92.66]:49630 "EHLO g9t5008.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753825AbcA1Uoq (ORCPT ); Thu, 28 Jan 2016 15:44:46 -0500 Message-ID: <1454013825.2576.31.camel@hpe.com> Subject: Re: [PATCH 0/2] Fix BTT data corruptions after crash From: Toshi Kani To: Dan Williams Cc: Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Borislav Petkov , Ross Zwisler , Vishal L Verma , micah.parrish@hpe.com, brian.boylston@hpe.com, X86 ML , "linux-nvdimm@lists.01.org" , "linux-kernel@vger.kernel.org" Date: Thu, 28 Jan 2016 13:43:45 -0700 In-Reply-To: References: <1454004770-6318-1-git-send-email-toshi.kani@hpe.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.16.5 (3.16.5-3.fc22) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2016-01-28 at 12:12 -0800, Dan Williams wrote: > On Thu, Jan 28, 2016 at 10:12 AM, Toshi Kani wrote: > > Data corruption issues were observed in tests which initiated a system > > crash/reset while accessing BTT devices. This problem is reproducible. > > > > The BTT driver calls pmem_rw_bytes() to update data in pmem devices. > > This interface calls __copy_user_nocache(), which uses non-temporal > > stores so that the stores to pmem are persistent. > > > > __copy_user_nocache() uses non-temporal stores when a request size is > > 8 bytes or larger (and is aligned by 8 bytes). The BTT driver updates > > the BTT map table, which entry size is 4 bytes. Therefore, updates to > > the map table entries remain cached, and are not written to pmem after > > a crash. Since the BTT driver makes previous blocks free and uses them > > for subsequent writes, the map table ends up pointing to blocks > > allocated for other LBAs after a crash. > > > > Patch 1 extends __copy_user_nocache() to use non-temporal store for > > 4 byte copy. This patch fixes the BTT data corruption issue. > > > > Nice find! :-) > > Patch 2 changes arch_memcpy_to_pmem() to flush processor caches when > > a request is not naturally aligned or is less than 4 bytes. This is > > defensive change. > > I'm wondering if we should just document that this routine does not > support unaligned transfers? Maybe backed by a debug mode that does > the alignment check. Yes, I agree. For this debug mode, do you have something in mind? Or should we add a new CONFIG option like CONFIG_PMEM_DEBUG? Thanks, -Toshi