From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-f175.google.com (mail-ob0-f175.google.com [209.85.214.175]) by kanga.kvack.org (Postfix) with ESMTP id 243346B0032 for ; Tue, 24 Feb 2015 19:15:36 -0500 (EST) Received: by mail-ob0-f175.google.com with SMTP id va2so517353obc.6 for ; Tue, 24 Feb 2015 16:15:35 -0800 (PST) Received: from g4t3426.houston.hp.com (g4t3426.houston.hp.com. [15.201.208.54]) by mx.google.com with ESMTPS id s8si4335987obk.81.2015.02.24.16.15.34 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 16:15:35 -0800 (PST) From: Toshi Kani Subject: [PATCH v8 0/7] Support Write-Through mapping on x86 Date: Tue, 24 Feb 2015 17:14:54 -0700 Message-Id: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: owner-linux-mm@kvack.org List-ID: To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com This patchset adds support of Write-Through (WT) mapping on x86. The study below shows that using WT mapping may be useful for non-volatile memory. http://www.hpl.hp.com/techreports/2012/HPL-2012-236.pdf All new/modified interfaces have been tested. v8: - Rebased to 4.0-rc1 and resolved conflicts with 9d34cfdf4 in patch 5/7. v7: - Rebased to 3.19-rc3 as Juergen's patchset for the PAT management has been accepted. v6: - Dropped the patch moving [set|get]_page_memtype() to pat.c since the tip branch already has this change. - Fixed an issue when CONFIG_X86_PAT is not defined. v5: - Clarified comment of why using slot 7. (Andy Lutomirski, Thomas Gleixner) - Moved [set|get]_page_memtype() to pat.c. (Thomas Gleixner) - Removed BUG() from set_page_memtype(). (Thomas Gleixner) v4: - Added set_memory_wt() by adding WT support of regular memory. v3: - Dropped the set_memory_wt() patch. (Andy Lutomirski) - Refactored the !pat_enabled handling. (H. Peter Anvin, Andy Lutomirski) - Added the picture of PTE encoding. (Konrad Rzeszutek Wilk) v2: - Changed WT to use slot 7 of the PAT MSR. (H. Peter Anvin, Andy Lutomirski) - Changed to have conservative checks to exclude all Pentium 2, 3, M, and 4 families. (Ingo Molnar, Henrique de Moraes Holschuh, Andy Lutomirski) - Updated documentation to cover WT interfaces and usages. (Andy Lutomirski, Yigal Korman) --- Toshi Kani (7): 1/7 x86, mm, pat: Set WT to PA7 slot of PAT MSR 2/7 x86, mm, pat: Change reserve_memtype() to handle WT 3/7 x86, mm, asm-gen: Add ioremap_wt() for WT 4/7 x86, mm, pat: Add pgprot_writethrough() for WT 5/7 x86, mm, pat: Refactor !pat_enable handling 6/7 x86, mm, asm: Add WT support to set_page_memtype() 7/7 x86, mm: Add set_memory_wt() for WT --- Documentation/x86/pat.txt | 13 ++- arch/x86/include/asm/cacheflush.h | 6 +- arch/x86/include/asm/io.h | 2 + arch/x86/include/asm/pgtable_types.h | 3 + arch/x86/mm/init.c | 6 +- arch/x86/mm/iomap_32.c | 12 +-- arch/x86/mm/ioremap.c | 26 ++++- arch/x86/mm/pageattr.c | 61 +++++++++-- arch/x86/mm/pat.c | 194 ++++++++++++++++++++++++----------- include/asm-generic/io.h | 9 ++ include/asm-generic/iomap.h | 4 + include/asm-generic/pgtable.h | 4 + 12 files changed, 251 insertions(+), 89 deletions(-) -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f43.google.com (mail-oi0-f43.google.com [209.85.218.43]) by kanga.kvack.org (Postfix) with ESMTP id D93786B006C for ; Tue, 24 Feb 2015 19:15:38 -0500 (EST) Received: by mail-oi0-f43.google.com with SMTP id z81so457079oif.2 for ; Tue, 24 Feb 2015 16:15:38 -0800 (PST) Received: from g4t3426.houston.hp.com (g4t3426.houston.hp.com. [15.201.208.54]) by mx.google.com with ESMTPS id i4si1100441obk.23.2015.02.24.16.15.37 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 16:15:38 -0800 (PST) From: Toshi Kani Subject: [PATCH v8 1/7] x86, mm, pat: Set WT to PA7 slot of PAT MSR Date: Tue, 24 Feb 2015 17:14:55 -0700 Message-Id: <1424823301-30927-2-git-send-email-toshi.kani@hp.com> In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: owner-linux-mm@kvack.org List-ID: To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani This patch sets WT to the PA7 slot in the PAT MSR when the processor is not affected by the PAT errata. The PA7 slot is chosen to improve robustness in the presence of errata that might cause the high PAT bit to be ignored. This way a buggy PA7 slot access will hit the PA3 slot, which is UC, so at worst we lose performance without causing a correctness issue. The following Intel processors are affected by the PAT errata. errata cpuid ---------------------------------------------------- Pentium 2, A52 family 0x6, model 0x5 Pentium 3, E27 family 0x6, model 0x7, 0x8 Pentium 3 Xenon, G26 family 0x6, model 0x7, 0x8, 0xa Pentium M, Y26 family 0x6, model 0x9 Pentium M 90nm, X9 family 0x6, model 0xd Pentium 4, N46 family 0xf, model 0x0 Instead of making sharp boundary checks, this patch makes conservative checks to exclude all Pentium 2, 3, M and 4 family processors. For such processors, _PAGE_CACHE_MODE_WT is redirected to UC- per the default setup in __cachemode2pte_tbl[]. Signed-off-by: Toshi Kani Reviewed-by: Juergen Gross --- arch/x86/mm/pat.c | 71 ++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 56 insertions(+), 15 deletions(-) diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c index 7ac6869..76b56bc 100644 --- a/arch/x86/mm/pat.c +++ b/arch/x86/mm/pat.c @@ -197,6 +197,7 @@ void pat_init(void) { u64 pat; bool boot_cpu = !boot_pat_state; + struct cpuinfo_x86 *c = &boot_cpu_data; if (!pat_enabled) return; @@ -217,21 +218,61 @@ void pat_init(void) } } - /* Set PWT to Write-Combining. All other bits stay the same */ - /* - * PTE encoding used in Linux: - * PAT - * |PCD - * ||PWT - * ||| - * 000 WB _PAGE_CACHE_WB - * 001 WC _PAGE_CACHE_WC - * 010 UC- _PAGE_CACHE_UC_MINUS - * 011 UC _PAGE_CACHE_UC - * PAT bit unused - */ - pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) | - PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, UC); + if ((c->x86_vendor == X86_VENDOR_INTEL) && + (((c->x86 == 0x6) && (c->x86_model <= 0xd)) || + ((c->x86 == 0xf) && (c->x86_model <= 0x6)))) { + /* + * PAT support with the lower four entries. Intel Pentium 2, + * 3, M, and 4 are affected by PAT errata, which makes the + * upper four entries unusable. We do not use the upper four + * entries for all the affected processor families for safe. + * + * PTE encoding used in Linux: + * PAT + * |PCD + * ||PWT PAT + * ||| slot + * 000 0 WB : _PAGE_CACHE_MODE_WB + * 001 1 WC : _PAGE_CACHE_MODE_WC + * 010 2 UC-: _PAGE_CACHE_MODE_UC_MINUS + * 011 3 UC : _PAGE_CACHE_MODE_UC + * PAT bit unused + * + * NOTE: When WT or WP is used, it is redirected to UC- per + * the default setup in __cachemode2pte_tbl[]. + */ + pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) | + PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, UC); + } else { + /* + * PAT full support. We put WT in slot 7 to improve + * robustness in the presence of errata that might cause + * the high PAT bit to be ignored. This way a buggy slot 7 + * access will hit slot 3, and slot 3 is UC, so at worst + * we lose performance without causing a correctness issue. + * Pentium 4 erratum N46 is an example of such an erratum, + * although we try not to use PAT at all on affected CPUs. + * + * PTE encoding used in Linux: + * PAT + * |PCD + * ||PWT PAT + * ||| slot + * 000 0 WB : _PAGE_CACHE_MODE_WB + * 001 1 WC : _PAGE_CACHE_MODE_WC + * 010 2 UC-: _PAGE_CACHE_MODE_UC_MINUS + * 011 3 UC : _PAGE_CACHE_MODE_UC + * 100 4 WB : Reserved + * 101 5 WC : Reserved + * 110 6 UC-: Reserved + * 111 7 WT : _PAGE_CACHE_MODE_WT + * + * The reserved slots are unused, but mapped to their + * corresponding types in the presence of PAT errata. + */ + pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) | + PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, WT); + } /* Boot CPU check */ if (!boot_pat_state) { -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f53.google.com (mail-oi0-f53.google.com [209.85.218.53]) by kanga.kvack.org (Postfix) with ESMTP id CC18B6B006E for ; Tue, 24 Feb 2015 19:15:41 -0500 (EST) Received: by mail-oi0-f53.google.com with SMTP id u20so383584oif.12 for ; Tue, 24 Feb 2015 16:15:41 -0800 (PST) Received: from g4t3427.houston.hp.com (g4t3427.houston.hp.com. [15.201.208.55]) by mx.google.com with ESMTPS id y132si16974864oia.96.2015.02.24.16.15.40 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 16:15:40 -0800 (PST) From: Toshi Kani Subject: [PATCH v8 2/7] x86, mm, pat: Change reserve_memtype() to handle WT Date: Tue, 24 Feb 2015 17:14:56 -0700 Message-Id: <1424823301-30927-3-git-send-email-toshi.kani@hp.com> In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: owner-linux-mm@kvack.org List-ID: To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani This patch changes reserve_memtype() to handle the WT cache mode with PAT. When PAT is not enabled, WB and UC- are the only types supported. When a target range is in RAM, reserve_ram_pages_type() verifies the requested type. In this case, WT and WP requests fail with -EINVAL and UC gets redirected to UC- since set_page_memtype() is limited to handle three types, WB, WC and UC-. Signed-off-by: Toshi Kani Reviewed-by: Konrad Rzeszutek Wilk --- arch/x86/mm/pat.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c index 76b56bc..b4db0d8 100644 --- a/arch/x86/mm/pat.c +++ b/arch/x86/mm/pat.c @@ -365,6 +365,8 @@ static int pat_pagerange_is_ram(resource_size_t start, resource_size_t end) /* * For RAM pages, we use page flags to mark the pages with appropriate type. + * The page flags are limited to three types, WB, WC and UC-. + * WT and WP requests fail with -EINVAL, and UC gets redirected to UC-. * Here we do two pass: * - Find the memtype of all the pages in the range, look for any conflicts * - In case of no conflicts, set the new memtype for pages in the range @@ -376,6 +378,13 @@ static int reserve_ram_pages_type(u64 start, u64 end, struct page *page; u64 pfn; + if ((req_type == _PAGE_CACHE_MODE_WT) || + (req_type == _PAGE_CACHE_MODE_WP)) { + if (new_type) + *new_type = _PAGE_CACHE_MODE_UC_MINUS; + return -EINVAL; + } + if (req_type == _PAGE_CACHE_MODE_UC) { /* We do not support strong UC */ WARN_ON_ONCE(1); @@ -425,6 +434,7 @@ static int free_ram_pages_type(u64 start, u64 end) * - _PAGE_CACHE_MODE_WC * - _PAGE_CACHE_MODE_UC_MINUS * - _PAGE_CACHE_MODE_UC + * - _PAGE_CACHE_MODE_WT * * If new_type is NULL, function will return an error if it cannot reserve the * region with req_type. If new_type is non-NULL, function will return @@ -442,12 +452,12 @@ int reserve_memtype(u64 start, u64 end, enum page_cache_mode req_type, BUG_ON(start >= end); /* end is exclusive */ if (!pat_enabled) { - /* This is identical to page table setting without PAT */ + /* WB and UC- are the only types supported without PAT */ if (new_type) { - if (req_type == _PAGE_CACHE_MODE_WC) - *new_type = _PAGE_CACHE_MODE_UC_MINUS; + if (req_type == _PAGE_CACHE_MODE_WB) + *new_type = _PAGE_CACHE_MODE_WB; else - *new_type = req_type; + *new_type = _PAGE_CACHE_MODE_UC_MINUS; } return 0; } -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-f170.google.com (mail-ob0-f170.google.com [209.85.214.170]) by kanga.kvack.org (Postfix) with ESMTP id 5C7326B0070 for ; Tue, 24 Feb 2015 19:15:44 -0500 (EST) Received: by mail-ob0-f170.google.com with SMTP id va2so560533obc.1 for ; Tue, 24 Feb 2015 16:15:44 -0800 (PST) Received: from g4t3425.houston.hp.com (g4t3425.houston.hp.com. [15.201.208.53]) by mx.google.com with ESMTPS id s8si4336104obk.81.2015.02.24.16.15.43 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 16:15:43 -0800 (PST) From: Toshi Kani Subject: [PATCH v8 3/7] x86, mm, asm-gen: Add ioremap_wt() for WT Date: Tue, 24 Feb 2015 17:14:57 -0700 Message-Id: <1424823301-30927-4-git-send-email-toshi.kani@hp.com> In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: owner-linux-mm@kvack.org List-ID: To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani This patch adds ioremap_wt() for creating WT mapping on x86. It follows the same model as ioremap_wc() for multi-architecture support. ARCH_HAS_IOREMAP_WT is defined in the x86 version of io.h to indicate that ioremap_wt() is implemented on x86. Also update the PAT documentation file to cover ioremap_wt(). Signed-off-by: Toshi Kani Reviewed-by: Konrad Rzeszutek Wilk --- Documentation/x86/pat.txt | 4 +++- arch/x86/include/asm/io.h | 2 ++ arch/x86/mm/ioremap.c | 24 ++++++++++++++++++++++++ include/asm-generic/io.h | 9 +++++++++ include/asm-generic/iomap.h | 4 ++++ 5 files changed, 42 insertions(+), 1 deletion(-) diff --git a/Documentation/x86/pat.txt b/Documentation/x86/pat.txt index cf08c9f..be7b8c2 100644 --- a/Documentation/x86/pat.txt +++ b/Documentation/x86/pat.txt @@ -12,7 +12,7 @@ virtual addresses. PAT allows for different types of memory attributes. The most commonly used ones that will be supported at this time are Write-back, Uncached, -Write-combined and Uncached Minus. +Write-combined, Write-through and Uncached Minus. PAT APIs @@ -38,6 +38,8 @@ ioremap_nocache | -- | UC- | UC- | | | | | ioremap_wc | -- | -- | WC | | | | | +ioremap_wt | -- | -- | WT | + | | | | set_memory_uc | UC- | -- | -- | set_memory_wb | | | | | | | | diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h index 34a5b93..81942ef 100644 --- a/arch/x86/include/asm/io.h +++ b/arch/x86/include/asm/io.h @@ -35,6 +35,7 @@ */ #define ARCH_HAS_IOREMAP_WC +#define ARCH_HAS_IOREMAP_WT #include #include @@ -320,6 +321,7 @@ extern void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr); extern int ioremap_change_attr(unsigned long vaddr, unsigned long size, enum page_cache_mode pcm); extern void __iomem *ioremap_wc(resource_size_t offset, unsigned long size); +extern void __iomem *ioremap_wt(resource_size_t offset, unsigned long size); extern bool is_early_ioremap_ptep(pte_t *ptep); diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index fdf617c..4f794b2 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -167,6 +167,10 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr, prot = __pgprot(pgprot_val(prot) | cachemode2protval(_PAGE_CACHE_MODE_WC)); break; + case _PAGE_CACHE_MODE_WT: + prot = __pgprot(pgprot_val(prot) | + cachemode2protval(_PAGE_CACHE_MODE_WT)); + break; case _PAGE_CACHE_MODE_WB: break; } @@ -261,6 +265,26 @@ void __iomem *ioremap_wc(resource_size_t phys_addr, unsigned long size) } EXPORT_SYMBOL(ioremap_wc); +/** + * ioremap_wt - map memory into CPU space write through + * @phys_addr: bus address of the memory + * @size: size of the resource to map + * + * This version of ioremap ensures that the memory is marked write through. + * Write through stores data into memory while keeping the cache up-to-date. + * + * Must be freed with iounmap. + */ +void __iomem *ioremap_wt(resource_size_t phys_addr, unsigned long size) +{ + if (pat_enabled) + return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WT, + __builtin_return_address(0)); + else + return ioremap_nocache(phys_addr, size); +} +EXPORT_SYMBOL(ioremap_wt); + void __iomem *ioremap_cache(resource_size_t phys_addr, unsigned long size) { return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WB, diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h index 9db0423..bae62dc 100644 --- a/include/asm-generic/io.h +++ b/include/asm-generic/io.h @@ -777,8 +777,17 @@ static inline void __iomem *ioremap_wc(phys_addr_t offset, size_t size) } #endif +#ifndef ioremap_wt +#define ioremap_wt ioremap_wt +static inline void __iomem *ioremap_wt(phys_addr_t offset, size_t size) +{ + return ioremap_nocache(offset, size); +} +#endif + #ifndef iounmap #define iounmap iounmap + static inline void iounmap(void __iomem *addr) { } diff --git a/include/asm-generic/iomap.h b/include/asm-generic/iomap.h index 1b41011..d8f8622 100644 --- a/include/asm-generic/iomap.h +++ b/include/asm-generic/iomap.h @@ -66,6 +66,10 @@ extern void ioport_unmap(void __iomem *); #define ioremap_wc ioremap_nocache #endif +#ifndef ARCH_HAS_IOREMAP_WT +#define ioremap_wt ioremap_nocache +#endif + #ifdef CONFIG_PCI /* Destroy a virtual mapping cookie for a PCI BAR (memory or IO) */ struct pci_dev; -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-f171.google.com (mail-ob0-f171.google.com [209.85.214.171]) by kanga.kvack.org (Postfix) with ESMTP id 3A4286B0071 for ; Tue, 24 Feb 2015 19:15:45 -0500 (EST) Received: by mail-ob0-f171.google.com with SMTP id gq1so553367obb.2 for ; Tue, 24 Feb 2015 16:15:44 -0800 (PST) Received: from g4t3425.houston.hp.com (g4t3425.houston.hp.com. [15.201.208.53]) by mx.google.com with ESMTPS id ms3si9453975obb.68.2015.02.24.16.15.44 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 16:15:44 -0800 (PST) From: Toshi Kani Subject: [PATCH v8 4/7] x86, mm, pat: Add pgprot_writethrough() for WT Date: Tue, 24 Feb 2015 17:14:58 -0700 Message-Id: <1424823301-30927-5-git-send-email-toshi.kani@hp.com> In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: owner-linux-mm@kvack.org List-ID: To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani This patch adds pgprot_writethrough() for setting WT to a given pgprot_t. Signed-off-by: Toshi Kani Reviewed-by: Konrad Rzeszutek Wilk --- arch/x86/include/asm/pgtable_types.h | 3 +++ arch/x86/mm/pat.c | 10 ++++++++++ include/asm-generic/pgtable.h | 4 ++++ 3 files changed, 17 insertions(+) diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h index 8c7c108..e962f06 100644 --- a/arch/x86/include/asm/pgtable_types.h +++ b/arch/x86/include/asm/pgtable_types.h @@ -367,6 +367,9 @@ extern int nx_enabled; #define pgprot_writecombine pgprot_writecombine extern pgprot_t pgprot_writecombine(pgprot_t prot); +#define pgprot_writethrough pgprot_writethrough +extern pgprot_t pgprot_writethrough(pgprot_t prot); + /* Indicate that x86 has its own track and untrack pfn vma functions */ #define __HAVE_PFNMAP_TRACKING diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c index b4db0d8..d3f7058 100644 --- a/arch/x86/mm/pat.c +++ b/arch/x86/mm/pat.c @@ -972,6 +972,16 @@ pgprot_t pgprot_writecombine(pgprot_t prot) } EXPORT_SYMBOL_GPL(pgprot_writecombine); +pgprot_t pgprot_writethrough(pgprot_t prot) +{ + if (pat_enabled) + return __pgprot(pgprot_val(prot) | + cachemode2protval(_PAGE_CACHE_MODE_WT)); + else + return pgprot_noncached(prot); +} +EXPORT_SYMBOL_GPL(pgprot_writethrough); + #if defined(CONFIG_DEBUG_FS) && defined(CONFIG_X86_PAT) static struct memtype *memtype_get_idx(loff_t pos) diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 4d46085..d1e59a4 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -256,6 +256,10 @@ static inline int pmd_same(pmd_t pmd_a, pmd_t pmd_b) #define pgprot_writecombine pgprot_noncached #endif +#ifndef pgprot_writethrough +#define pgprot_writethrough pgprot_noncached +#endif + #ifndef pgprot_device #define pgprot_device pgprot_noncached #endif -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f52.google.com (mail-oi0-f52.google.com [209.85.218.52]) by kanga.kvack.org (Postfix) with ESMTP id 3B7B96B0072 for ; Tue, 24 Feb 2015 19:15:47 -0500 (EST) Received: by mail-oi0-f52.google.com with SMTP id u20so393913oif.11 for ; Tue, 24 Feb 2015 16:15:46 -0800 (PST) Received: from g4t3425.houston.hp.com (g4t3425.houston.hp.com. [15.201.208.53]) by mx.google.com with ESMTPS id u11si4225791oib.67.2015.02.24.16.15.46 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 16:15:46 -0800 (PST) From: Toshi Kani Subject: [PATCH v8 5/7] x86, mm, pat: Refactor !pat_enabled handling Date: Tue, 24 Feb 2015 17:14:59 -0700 Message-Id: <1424823301-30927-6-git-send-email-toshi.kani@hp.com> In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: owner-linux-mm@kvack.org List-ID: To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani This patch refactors the !pat_enabled handling code and integrates it into the PAT abstraction code. The PAT table is emulated by corresponding to the two cache attribute bits, PWT (Write Through) and PCD (Cache Disable). The emulated PAT table is the same as the BIOS default setup when the system has PAT but the "nopat" boot option is specified. The emulated PAT table is also used when MSR_IA32_CR_PAT returns 0 (9d34cfdf4). Signed-off-by: Toshi Kani Reviewed-by: Juergen Gross --- arch/x86/mm/init.c | 6 ++- arch/x86/mm/iomap_32.c | 12 +++---- arch/x86/mm/ioremap.c | 10 +----- arch/x86/mm/pageattr.c | 3 -- arch/x86/mm/pat.c | 85 ++++++++++++++++++++++++++++-------------------- 5 files changed, 61 insertions(+), 55 deletions(-) diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index a110efc..98b9518 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -37,7 +37,7 @@ */ uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { [_PAGE_CACHE_MODE_WB] = 0, - [_PAGE_CACHE_MODE_WC] = _PAGE_PWT, + [_PAGE_CACHE_MODE_WC] = _PAGE_PCD, [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD, [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT, [_PAGE_CACHE_MODE_WT] = _PAGE_PCD, @@ -46,11 +46,11 @@ uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { EXPORT_SYMBOL(__cachemode2pte_tbl); uint8_t __pte2cachemode_tbl[8] = { [__pte2cm_idx(0)] = _PAGE_CACHE_MODE_WB, - [__pte2cm_idx(_PAGE_PWT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx(_PAGE_PWT)] = _PAGE_CACHE_MODE_UC_MINUS, [__pte2cm_idx(_PAGE_PCD)] = _PAGE_CACHE_MODE_UC_MINUS, [__pte2cm_idx(_PAGE_PWT | _PAGE_PCD)] = _PAGE_CACHE_MODE_UC, [__pte2cm_idx(_PAGE_PAT)] = _PAGE_CACHE_MODE_WB, - [__pte2cm_idx(_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx(_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS, [__pte2cm_idx(_PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS, [__pte2cm_idx(_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC, }; diff --git a/arch/x86/mm/iomap_32.c b/arch/x86/mm/iomap_32.c index 9ca35fc..2c51a2b 100644 --- a/arch/x86/mm/iomap_32.c +++ b/arch/x86/mm/iomap_32.c @@ -77,13 +77,13 @@ void __iomem * iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot) { /* - * For non-PAT systems, promote PAGE_KERNEL_WC to PAGE_KERNEL_UC_MINUS. - * PAGE_KERNEL_WC maps to PWT, which translates to uncached if the - * MTRR is UC or WC. UC_MINUS gets the real intention, of the - * user, which is "WC if the MTRR is WC, UC if you can't do that." + * For non-PAT systems, translate non-WB request to UC- just in + * case the caller set the PWT bit to prot directly without using + * pgprot_writecombine(). UC- translates to uncached if the MTRR + * is UC or WC. UC- gets the real intention, of the user, which is + * "WC if the MTRR is WC, UC if you can't do that." */ - if (!pat_enabled && pgprot_val(prot) == - (__PAGE_KERNEL | cachemode2protval(_PAGE_CACHE_MODE_WC))) + if (!pat_enabled && pgprot2cachemode(prot) != _PAGE_CACHE_MODE_WB) prot = __pgprot(__PAGE_KERNEL | cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS)); diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index 4f794b2..f73db48 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -257,11 +257,8 @@ EXPORT_SYMBOL(ioremap_nocache); */ void __iomem *ioremap_wc(resource_size_t phys_addr, unsigned long size) { - if (pat_enabled) - return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WC, + return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WC, __builtin_return_address(0)); - else - return ioremap_nocache(phys_addr, size); } EXPORT_SYMBOL(ioremap_wc); @@ -277,11 +274,8 @@ EXPORT_SYMBOL(ioremap_wc); */ void __iomem *ioremap_wt(resource_size_t phys_addr, unsigned long size) { - if (pat_enabled) - return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WT, + return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WT, __builtin_return_address(0)); - else - return ioremap_nocache(phys_addr, size); } EXPORT_SYMBOL(ioremap_wt); diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 536ea2f..1965fc8 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -1573,9 +1573,6 @@ int set_memory_wc(unsigned long addr, int numpages) { int ret; - if (!pat_enabled) - return set_memory_uc(addr, numpages); - ret = reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE, _PAGE_CACHE_MODE_WC, NULL); if (ret) diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c index d3f7058..db3cc89 100644 --- a/arch/x86/mm/pat.c +++ b/arch/x86/mm/pat.c @@ -181,7 +181,11 @@ void pat_init_cache_modes(void) char pat_msg[33]; u64 pat; - rdmsrl(MSR_IA32_CR_PAT, pat); + if (pat_enabled) + rdmsrl(MSR_IA32_CR_PAT, pat); + else + pat = boot_pat_state; + pat_msg[32] = 0; for (i = 7; i >= 0; i--) { cache = pat_get_cache_mode((pat >> (i * 8)) & 7, @@ -199,28 +203,58 @@ void pat_init(void) bool boot_cpu = !boot_pat_state; struct cpuinfo_x86 *c = &boot_cpu_data; - if (!pat_enabled) - return; - if (!cpu_has_pat) { if (!boot_pat_state) { pat_disable("PAT not supported by CPU."); - return; - } else { + } else if (pat_enabled) { /* * If this happens we are on a secondary CPU, but * switched to PAT on the boot CPU. We have no way to * undo PAT. */ - printk(KERN_ERR "PAT enabled, " + pr_err("PAT enabled, " "but not supported by secondary CPU\n"); BUG(); } } - if ((c->x86_vendor == X86_VENDOR_INTEL) && - (((c->x86 == 0x6) && (c->x86_model <= 0xd)) || - ((c->x86 == 0xf) && (c->x86_model <= 0x6)))) { + /* Boot CPU check */ + if (pat_enabled && !boot_pat_state) { + rdmsrl(MSR_IA32_CR_PAT, boot_pat_state); + if (!boot_pat_state) + pat_disable("PAT read returns always zero, disabled."); + + } + + if (!pat_enabled) { + /* + * No PAT. Emulate the PAT table that corresponds to the two + * cache bits, PWT (Write Through) and PCD (Cache Disable). + * This setup is the same as the BIOS default setup when the + * system has PAT but the "nopat" boot option is specified. + * This emulated PAT table is also used when MSR_IA32_CR_PAT + * returns 0. + * + * PTE encoding used in Linux: + * PCD + * |PWT PAT + * || slot + * 00 0 WB : _PAGE_CACHE_MODE_WB + * 01 1 WT : _PAGE_CACHE_MODE_WT + * 10 2 UC-: _PAGE_CACHE_MODE_UC_MINUS + * 11 3 UC : _PAGE_CACHE_MODE_UC + * + * NOTE: When WC or WP is used, it is redirected to UC- per + * the default setup in __cachemode2pte_tbl[]. + */ + pat = PAT(0, WB) | PAT(1, WT) | PAT(2, UC_MINUS) | PAT(3, UC) | + PAT(4, WB) | PAT(5, WT) | PAT(6, UC_MINUS) | PAT(7, UC); + if (!boot_pat_state) + boot_pat_state = pat; + + } else if ((c->x86_vendor == X86_VENDOR_INTEL) && + (((c->x86 == 0x6) && (c->x86_model <= 0xd)) || + ((c->x86 == 0xf) && (c->x86_model <= 0x6)))) { /* * PAT support with the lower four entries. Intel Pentium 2, * 3, M, and 4 are affected by PAT errata, which makes the @@ -274,16 +308,8 @@ void pat_init(void) PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, WT); } - /* Boot CPU check */ - if (!boot_pat_state) { - rdmsrl(MSR_IA32_CR_PAT, boot_pat_state); - if (!boot_pat_state) { - pat_disable("PAT read returns always zero, disabled."); - return; - } - } - - wrmsrl(MSR_IA32_CR_PAT, pat); + if (pat_enabled) + wrmsrl(MSR_IA32_CR_PAT, pat); if (boot_cpu) pat_init_cache_modes(); @@ -452,13 +478,8 @@ int reserve_memtype(u64 start, u64 end, enum page_cache_mode req_type, BUG_ON(start >= end); /* end is exclusive */ if (!pat_enabled) { - /* WB and UC- are the only types supported without PAT */ - if (new_type) { - if (req_type == _PAGE_CACHE_MODE_WB) - *new_type = _PAGE_CACHE_MODE_WB; - else - *new_type = _PAGE_CACHE_MODE_UC_MINUS; - } + if (new_type) + *new_type = req_type; return 0; } @@ -964,21 +985,15 @@ void untrack_pfn(struct vm_area_struct *vma, unsigned long pfn, pgprot_t pgprot_writecombine(pgprot_t prot) { - if (pat_enabled) - return __pgprot(pgprot_val(prot) | + return __pgprot(pgprot_val(prot) | cachemode2protval(_PAGE_CACHE_MODE_WC)); - else - return pgprot_noncached(prot); } EXPORT_SYMBOL_GPL(pgprot_writecombine); pgprot_t pgprot_writethrough(pgprot_t prot) { - if (pat_enabled) - return __pgprot(pgprot_val(prot) | + return __pgprot(pgprot_val(prot) | cachemode2protval(_PAGE_CACHE_MODE_WT)); - else - return pgprot_noncached(prot); } EXPORT_SYMBOL_GPL(pgprot_writethrough); -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f49.google.com (mail-oi0-f49.google.com [209.85.218.49]) by kanga.kvack.org (Postfix) with ESMTP id 209956B0073 for ; Tue, 24 Feb 2015 19:15:49 -0500 (EST) Received: by mail-oi0-f49.google.com with SMTP id v63so415471oia.8 for ; Tue, 24 Feb 2015 16:15:48 -0800 (PST) Received: from g4t3425.houston.hp.com (g4t3425.houston.hp.com. [15.201.208.53]) by mx.google.com with ESMTPS id r9si4016901obu.106.2015.02.24.16.15.48 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 16:15:48 -0800 (PST) From: Toshi Kani Subject: [PATCH v8 6/7] x86, mm, asm: Add WT support to set_page_memtype() Date: Tue, 24 Feb 2015 17:15:00 -0700 Message-Id: <1424823301-30927-7-git-send-email-toshi.kani@hp.com> In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: owner-linux-mm@kvack.org List-ID: To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani As set_memory_wb() calls set_page_memtype() with -1, _PGMT_DEFAULT is solely used for tracking the WB type. _PGMT_WB is defined but unused. Hence, this patch renames _PGMT_DEFAULT to _PGMT_WB to clarify its usage, and releases the slot used by _PGMT_WB before. As a result, set_memory_wb() is changed to call set_page_memtype() with _PGMT_WB, and get_page_memtype() returns _PAGE_CACHE_MODE_WB for _PGMT_WB. This patch then defines _PGMT_WT to the released slot. This enables set_page_memtype() to track the WT type. Signed-off-by: Toshi Kani --- arch/x86/mm/pat.c | 48 +++++++++++++++++++++--------------------------- 1 file changed, 21 insertions(+), 27 deletions(-) diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c index db3cc89..cfb14ac 100644 --- a/arch/x86/mm/pat.c +++ b/arch/x86/mm/pat.c @@ -69,18 +69,18 @@ static u64 __read_mostly boot_pat_state; #ifdef CONFIG_X86_PAT /* - * X86 PAT uses page flags WC and Uncached together to keep track of - * memory type of pages that have backing page struct. X86 PAT supports 3 - * different memory types, _PAGE_CACHE_MODE_WB, _PAGE_CACHE_MODE_WC and - * _PAGE_CACHE_MODE_UC_MINUS and fourth state where page's memory type has not - * been changed from its default (value of -1 used to denote this). + * X86 PAT uses page flags arch_1 and uncached together to keep track of + * memory type of pages that have backing page struct. X86 PAT supports 4 + * different memory types, _PAGE_CACHE_MODE_WT, _PAGE_CACHE_MODE_WC, + * _PAGE_CACHE_MODE_UC_MINUS and _PAGE_CACHE_MODE_WB where page's memory + * type has not been changed from its default. * Note we do not support _PAGE_CACHE_MODE_UC here. */ -#define _PGMT_DEFAULT 0 +#define _PGMT_WB 0 /* default */ #define _PGMT_WC (1UL << PG_arch_1) #define _PGMT_UC_MINUS (1UL << PG_uncached) -#define _PGMT_WB (1UL << PG_uncached | 1UL << PG_arch_1) +#define _PGMT_WT (1UL << PG_uncached | 1UL << PG_arch_1) #define _PGMT_MASK (1UL << PG_uncached | 1UL << PG_arch_1) #define _PGMT_CLEAR_MASK (~_PGMT_MASK) @@ -88,14 +88,14 @@ static inline enum page_cache_mode get_page_memtype(struct page *pg) { unsigned long pg_flags = pg->flags & _PGMT_MASK; - if (pg_flags == _PGMT_DEFAULT) - return -1; + if (pg_flags == _PGMT_WB) + return _PAGE_CACHE_MODE_WB; else if (pg_flags == _PGMT_WC) return _PAGE_CACHE_MODE_WC; else if (pg_flags == _PGMT_UC_MINUS) return _PAGE_CACHE_MODE_UC_MINUS; else - return _PAGE_CACHE_MODE_WB; + return _PAGE_CACHE_MODE_WT; } static inline void set_page_memtype(struct page *pg, @@ -112,11 +112,12 @@ static inline void set_page_memtype(struct page *pg, case _PAGE_CACHE_MODE_UC_MINUS: memtype_flags = _PGMT_UC_MINUS; break; - case _PAGE_CACHE_MODE_WB: - memtype_flags = _PGMT_WB; + case _PAGE_CACHE_MODE_WT: + memtype_flags = _PGMT_WT; break; + case _PAGE_CACHE_MODE_WB: default: - memtype_flags = _PGMT_DEFAULT; + memtype_flags = _PGMT_WB; /* default */ break; } @@ -391,8 +392,9 @@ static int pat_pagerange_is_ram(resource_size_t start, resource_size_t end) /* * For RAM pages, we use page flags to mark the pages with appropriate type. - * The page flags are limited to three types, WB, WC and UC-. - * WT and WP requests fail with -EINVAL, and UC gets redirected to UC-. + * The page flags are limited to four types, WB (default), WC, WT and UC-. + * WP request fails with -EINVAL, and UC gets redirected to UC-. + * A new memtype can only be set to the default memtype WB. * Here we do two pass: * - Find the memtype of all the pages in the range, look for any conflicts * - In case of no conflicts, set the new memtype for pages in the range @@ -404,8 +406,7 @@ static int reserve_ram_pages_type(u64 start, u64 end, struct page *page; u64 pfn; - if ((req_type == _PAGE_CACHE_MODE_WT) || - (req_type == _PAGE_CACHE_MODE_WP)) { + if (req_type == _PAGE_CACHE_MODE_WP) { if (new_type) *new_type = _PAGE_CACHE_MODE_UC_MINUS; return -EINVAL; @@ -422,7 +423,7 @@ static int reserve_ram_pages_type(u64 start, u64 end, page = pfn_to_page(pfn); type = get_page_memtype(page); - if (type != -1) { + if (type != _PAGE_CACHE_MODE_WB) { pr_info("reserve_ram_pages_type failed [mem %#010Lx-%#010Lx], track 0x%x, req 0x%x\n", start, end - 1, type, req_type); if (new_type) @@ -449,7 +450,7 @@ static int free_ram_pages_type(u64 start, u64 end) for (pfn = (start >> PAGE_SHIFT); pfn < (end >> PAGE_SHIFT); ++pfn) { page = pfn_to_page(pfn); - set_page_memtype(page, -1); + set_page_memtype(page, _PAGE_CACHE_MODE_WB); } return 0; } @@ -589,7 +590,7 @@ int free_memtype(u64 start, u64 end) * Only to be called when PAT is enabled * * Returns _PAGE_CACHE_MODE_WB, _PAGE_CACHE_MODE_WC, _PAGE_CACHE_MODE_UC_MINUS - * or _PAGE_CACHE_MODE_UC + * or _PAGE_CACHE_MODE_WT. */ static enum page_cache_mode lookup_memtype(u64 paddr) { @@ -603,13 +604,6 @@ static enum page_cache_mode lookup_memtype(u64 paddr) struct page *page; page = pfn_to_page(paddr >> PAGE_SHIFT); rettype = get_page_memtype(page); - /* - * -1 from get_page_memtype() implies RAM page is in its - * default state and not reserved, and hence of type WB - */ - if (rettype == -1) - rettype = _PAGE_CACHE_MODE_WB; - return rettype; } -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-f176.google.com (mail-ob0-f176.google.com [209.85.214.176]) by kanga.kvack.org (Postfix) with ESMTP id BA4CA6B0074 for ; Tue, 24 Feb 2015 19:15:50 -0500 (EST) Received: by mail-ob0-f176.google.com with SMTP id wo20so506379obc.7 for ; Tue, 24 Feb 2015 16:15:50 -0800 (PST) Received: from g4t3425.houston.hp.com (g4t3425.houston.hp.com. [15.201.208.53]) by mx.google.com with ESMTPS id k124si2092378oif.1.2015.02.24.16.15.50 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 16:15:50 -0800 (PST) From: Toshi Kani Subject: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT Date: Tue, 24 Feb 2015 17:15:01 -0700 Message-Id: <1424823301-30927-8-git-send-email-toshi.kani@hp.com> In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: owner-linux-mm@kvack.org List-ID: To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani This patch adds set_memory_wt(), set_memory_array_wt() and set_pages_array_wt() for setting specified range(s) of the regular memory to WT. Signed-off-by: Toshi Kani --- Documentation/x86/pat.txt | 9 ++++-- arch/x86/include/asm/cacheflush.h | 6 +++- arch/x86/mm/pageattr.c | 58 +++++++++++++++++++++++++++++++++---- 3 files changed, 63 insertions(+), 10 deletions(-) diff --git a/Documentation/x86/pat.txt b/Documentation/x86/pat.txt index be7b8c2..bf4339c 100644 --- a/Documentation/x86/pat.txt +++ b/Documentation/x86/pat.txt @@ -46,6 +46,9 @@ set_memory_uc | UC- | -- | -- | set_memory_wc | WC | -- | -- | set_memory_wb | | | | | | | | +set_memory_wt | WT | -- | -- | + set_memory_wb | | | | + | | | | pci sysfs resource | -- | -- | UC- | | | | | pci sysfs resource_wc | -- | -- | WC | @@ -117,8 +120,8 @@ can be more restrictive, in case of any existing aliasing for that address. For example: If there is an existing uncached mapping, a new ioremap_wc can return uncached mapping in place of write-combine requested. -set_memory_[uc|wc] and set_memory_wb should be used in pairs, where driver will -first make a region uc or wc and switch it back to wb after use. +set_memory_[uc|wc|wt] and set_memory_wb should be used in pairs, where driver +will first make a region uc, wc or wt and switch it back to wb after use. Over time writes to /proc/mtrr will be deprecated in favor of using PAT based interfaces. Users writing to /proc/mtrr are suggested to use above interfaces. @@ -126,7 +129,7 @@ interfaces. Users writing to /proc/mtrr are suggested to use above interfaces. Drivers should use ioremap_[uc|wc] to access PCI BARs with [uc|wc] access types. -Drivers should use set_memory_[uc|wc] to set access type for RAM ranges. +Drivers should use set_memory_[uc|wc|wt] to set access type for RAM ranges. PAT debugging diff --git a/arch/x86/include/asm/cacheflush.h b/arch/x86/include/asm/cacheflush.h index 47c8e32..b6f7457 100644 --- a/arch/x86/include/asm/cacheflush.h +++ b/arch/x86/include/asm/cacheflush.h @@ -8,7 +8,7 @@ /* * The set_memory_* API can be used to change various attributes of a virtual * address range. The attributes include: - * Cachability : UnCached, WriteCombining, WriteBack + * Cachability : UnCached, WriteCombining, WriteThrough, WriteBack * Executability : eXeutable, NoteXecutable * Read/Write : ReadOnly, ReadWrite * Presence : NotPresent @@ -35,9 +35,11 @@ int _set_memory_uc(unsigned long addr, int numpages); int _set_memory_wc(unsigned long addr, int numpages); +int _set_memory_wt(unsigned long addr, int numpages); int _set_memory_wb(unsigned long addr, int numpages); int set_memory_uc(unsigned long addr, int numpages); int set_memory_wc(unsigned long addr, int numpages); +int set_memory_wt(unsigned long addr, int numpages); int set_memory_wb(unsigned long addr, int numpages); int set_memory_x(unsigned long addr, int numpages); int set_memory_nx(unsigned long addr, int numpages); @@ -48,10 +50,12 @@ int set_memory_4k(unsigned long addr, int numpages); int set_memory_array_uc(unsigned long *addr, int addrinarray); int set_memory_array_wc(unsigned long *addr, int addrinarray); +int set_memory_array_wt(unsigned long *addr, int addrinarray); int set_memory_array_wb(unsigned long *addr, int addrinarray); int set_pages_array_uc(struct page **pages, int addrinarray); int set_pages_array_wc(struct page **pages, int addrinarray); +int set_pages_array_wt(struct page **pages, int addrinarray); int set_pages_array_wb(struct page **pages, int addrinarray); /* diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 1965fc8..9308527 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -1504,12 +1504,10 @@ EXPORT_SYMBOL(set_memory_uc); static int _set_memory_array(unsigned long *addr, int addrinarray, enum page_cache_mode new_type) { + enum page_cache_mode set_type; int i, j; int ret; - /* - * for now UC MINUS. see comments in ioremap_nocache() - */ for (i = 0; i < addrinarray; i++) { ret = reserve_memtype(__pa(addr[i]), __pa(addr[i]) + PAGE_SIZE, new_type, NULL); @@ -1517,9 +1515,12 @@ static int _set_memory_array(unsigned long *addr, int addrinarray, goto out_free; } + /* If WC, set to UC- first and then WC */ + set_type = (new_type == _PAGE_CACHE_MODE_WC) ? + _PAGE_CACHE_MODE_UC_MINUS : new_type; + ret = change_page_attr_set(addr, addrinarray, - cachemode2pgprot(_PAGE_CACHE_MODE_UC_MINUS), - 1); + cachemode2pgprot(set_type), 1); if (!ret && new_type == _PAGE_CACHE_MODE_WC) ret = change_page_attr_set_clr(addr, addrinarray, @@ -1551,6 +1552,12 @@ int set_memory_array_wc(unsigned long *addr, int addrinarray) } EXPORT_SYMBOL(set_memory_array_wc); +int set_memory_array_wt(unsigned long *addr, int addrinarray) +{ + return _set_memory_array(addr, addrinarray, _PAGE_CACHE_MODE_WT); +} +EXPORT_SYMBOL(set_memory_array_wt); + int _set_memory_wc(unsigned long addr, int numpages) { int ret; @@ -1591,6 +1598,34 @@ out_err: } EXPORT_SYMBOL(set_memory_wc); +int _set_memory_wt(unsigned long addr, int numpages) +{ + return change_page_attr_set(&addr, numpages, + cachemode2pgprot(_PAGE_CACHE_MODE_WT), 0); +} + +int set_memory_wt(unsigned long addr, int numpages) +{ + int ret; + + ret = reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE, + _PAGE_CACHE_MODE_WT, NULL); + if (ret) + goto out_err; + + ret = _set_memory_wt(addr, numpages); + if (ret) + goto out_free; + + return 0; + +out_free: + free_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE); +out_err: + return ret; +} +EXPORT_SYMBOL(set_memory_wt); + int _set_memory_wb(unsigned long addr, int numpages) { /* WB cache mode is hard wired to all cache attribute bits being 0 */ @@ -1683,6 +1718,7 @@ static int _set_pages_array(struct page **pages, int addrinarray, { unsigned long start; unsigned long end; + enum page_cache_mode set_type; int i; int free_idx; int ret; @@ -1696,8 +1732,12 @@ static int _set_pages_array(struct page **pages, int addrinarray, goto err_out; } + /* If WC, set to UC- first and then WC */ + set_type = (new_type == _PAGE_CACHE_MODE_WC) ? + _PAGE_CACHE_MODE_UC_MINUS : new_type; + ret = cpa_set_pages_array(pages, addrinarray, - cachemode2pgprot(_PAGE_CACHE_MODE_UC_MINUS)); + cachemode2pgprot(set_type)); if (!ret && new_type == _PAGE_CACHE_MODE_WC) ret = change_page_attr_set_clr(NULL, addrinarray, cachemode2pgprot( @@ -1731,6 +1771,12 @@ int set_pages_array_wc(struct page **pages, int addrinarray) } EXPORT_SYMBOL(set_pages_array_wc); +int set_pages_array_wt(struct page **pages, int addrinarray) +{ + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); +} +EXPORT_SYMBOL(set_pages_array_wt); + int set_pages_wb(struct page *page, int numpages) { unsigned long addr = (unsigned long)page_address(page); -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f177.google.com (mail-wi0-f177.google.com [209.85.212.177]) by kanga.kvack.org (Postfix) with ESMTP id 5BF846B0032 for ; Wed, 25 Feb 2015 02:22:35 -0500 (EST) Received: by mail-wi0-f177.google.com with SMTP id bs8so2769362wib.4 for ; Tue, 24 Feb 2015 23:22:34 -0800 (PST) Received: from mail-wg0-x235.google.com (mail-wg0-x235.google.com. [2a00:1450:400c:c00::235]) by mx.google.com with ESMTPS id c6si71576354wje.211.2015.02.24.23.22.33 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 23:22:33 -0800 (PST) Received: by wggy19 with SMTP id y19so1703834wgg.13 for ; Tue, 24 Feb 2015 23:22:33 -0800 (PST) Date: Wed, 25 Feb 2015 08:22:28 +0100 From: Ingo Molnar Subject: Re: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT Message-ID: <20150225072228.GA13061@gmail.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> <1424823301-30927-8-git-send-email-toshi.kani@hp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1424823301-30927-8-git-send-email-toshi.kani@hp.com> Sender: owner-linux-mm@kvack.org List-ID: To: Toshi Kani Cc: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com * Toshi Kani wrote: > +int set_pages_array_wt(struct page **pages, int addrinarray) > +{ > + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); > +} > +EXPORT_SYMBOL(set_pages_array_wt); So by default we make new APIs EXPORT_SYMBOL_GPL(): we don't want proprietary modules mucking around with new code PAT interfaces, we only want modules we can analyze and fix in detail. Thanks, Ingo -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f48.google.com (mail-pa0-f48.google.com [209.85.220.48]) by kanga.kvack.org (Postfix) with ESMTP id 5DB5C6B0032 for ; Wed, 25 Feb 2015 10:20:36 -0500 (EST) Received: by padfb1 with SMTP id fb1so5919890pad.8 for ; Wed, 25 Feb 2015 07:20:36 -0800 (PST) Received: from g2t2354.austin.hp.com (g2t2354.austin.hp.com. [15.217.128.53]) by mx.google.com with ESMTPS id xu8si6731667pbc.86.2015.02.25.07.20.35 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Feb 2015 07:20:35 -0800 (PST) Message-ID: <1424877601.17007.108.camel@misato.fc.hp.com> Subject: Re: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT From: Toshi Kani Date: Wed, 25 Feb 2015 08:20:01 -0700 In-Reply-To: <20150225072228.GA13061@gmail.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> <1424823301-30927-8-git-send-email-toshi.kani@hp.com> <20150225072228.GA13061@gmail.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Ingo Molnar Cc: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com On Wed, 2015-02-25 at 08:22 +0100, Ingo Molnar wrote: > * Toshi Kani wrote: > > > +int set_pages_array_wt(struct page **pages, int addrinarray) > > +{ > > + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); > > +} > > +EXPORT_SYMBOL(set_pages_array_wt); > > So by default we make new APIs EXPORT_SYMBOL_GPL(): we > don't want proprietary modules mucking around with new code > PAT interfaces, we only want modules we can analyze and fix > in detail. Right. I have one question for this case. This set_pages_array_wt() extends the set_pages_array_xx() family, which are all exported with EXPORT_SYMBOL() today. In this case, should we keep them exported in the consistent manner, or should we still use GPL when adding a new one? Thanks, -Toshi -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-we0-f169.google.com (mail-we0-f169.google.com [74.125.82.169]) by kanga.kvack.org (Postfix) with ESMTP id 19F6F6B0032 for ; Thu, 26 Feb 2015 06:31:01 -0500 (EST) Received: by wevm14 with SMTP id m14so9653472wev.13 for ; Thu, 26 Feb 2015 03:31:00 -0800 (PST) Received: from mail-wg0-x230.google.com (mail-wg0-x230.google.com. [2a00:1450:400c:c00::230]) by mx.google.com with ESMTPS id bp6si723641wjb.180.2015.02.26.03.30.58 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Feb 2015 03:30:59 -0800 (PST) Received: by wggy19 with SMTP id y19so9716920wgg.10 for ; Thu, 26 Feb 2015 03:30:58 -0800 (PST) Date: Thu, 26 Feb 2015 12:30:54 +0100 From: Ingo Molnar Subject: Re: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT Message-ID: <20150226113054.GA4191@gmail.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> <1424823301-30927-8-git-send-email-toshi.kani@hp.com> <20150225072228.GA13061@gmail.com> <1424877601.17007.108.camel@misato.fc.hp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1424877601.17007.108.camel@misato.fc.hp.com> Sender: owner-linux-mm@kvack.org List-ID: To: Toshi Kani Cc: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com * Toshi Kani wrote: > On Wed, 2015-02-25 at 08:22 +0100, Ingo Molnar wrote: > > * Toshi Kani wrote: > > > > > +int set_pages_array_wt(struct page **pages, int addrinarray) > > > +{ > > > + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); > > > +} > > > +EXPORT_SYMBOL(set_pages_array_wt); > > > > So by default we make new APIs EXPORT_SYMBOL_GPL(): we > > don't want proprietary modules mucking around with new code > > PAT interfaces, we only want modules we can analyze and fix > > in detail. > > Right. I have one question for this case. This > set_pages_array_wt() extends the set_pages_array_xx() > family, which are all exported with EXPORT_SYMBOL() > today. In this case, should we keep them exported in the > consistent manner, or should we still use GPL when adding > a new one? Still keep it GPL, it's a new API that old modules obviously don't use. Thanks, Ingo -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-f181.google.com (mail-ob0-f181.google.com [209.85.214.181]) by kanga.kvack.org (Postfix) with ESMTP id B38336B006E for ; Thu, 26 Feb 2015 09:45:28 -0500 (EST) Received: by mail-ob0-f181.google.com with SMTP id vb8so11201648obc.12 for ; Thu, 26 Feb 2015 06:45:28 -0800 (PST) Received: from g4t3427.houston.hp.com (g4t3427.houston.hp.com. [15.201.208.55]) by mx.google.com with ESMTPS id fm4si547753obb.52.2015.02.26.06.45.27 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Feb 2015 06:45:27 -0800 (PST) Message-ID: <1424961893.17007.139.camel@misato.fc.hp.com> Subject: Re: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT From: Toshi Kani Date: Thu, 26 Feb 2015 07:44:53 -0700 In-Reply-To: <20150226113054.GA4191@gmail.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> <1424823301-30927-8-git-send-email-toshi.kani@hp.com> <20150225072228.GA13061@gmail.com> <1424877601.17007.108.camel@misato.fc.hp.com> <20150226113054.GA4191@gmail.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Ingo Molnar Cc: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com On Thu, 2015-02-26 at 12:30 +0100, Ingo Molnar wrote: > * Toshi Kani wrote: > > > On Wed, 2015-02-25 at 08:22 +0100, Ingo Molnar wrote: > > > * Toshi Kani wrote: > > > > > > > +int set_pages_array_wt(struct page **pages, int addrinarray) > > > > +{ > > > > + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); > > > > +} > > > > +EXPORT_SYMBOL(set_pages_array_wt); > > > > > > So by default we make new APIs EXPORT_SYMBOL_GPL(): we > > > don't want proprietary modules mucking around with new code > > > PAT interfaces, we only want modules we can analyze and fix > > > in detail. > > > > Right. I have one question for this case. This > > set_pages_array_wt() extends the set_pages_array_xx() > > family, which are all exported with EXPORT_SYMBOL() > > today. In this case, should we keep them exported in the > > consistent manner, or should we still use GPL when adding > > a new one? > > Still keep it GPL, it's a new API that old modules > obviously don't use. Got it. Thanks for the clarification. -Toshi -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f43.google.com (mail-oi0-f43.google.com [209.85.218.43]) by kanga.kvack.org (Postfix) with ESMTP id 840F16B0072 for ; Wed, 4 Mar 2015 13:27:54 -0500 (EST) Received: by oiba3 with SMTP id a3so7246934oib.7 for ; Wed, 04 Mar 2015 10:27:54 -0800 (PST) Received: from g1t5425.austin.hp.com (g1t5425.austin.hp.com. [15.216.225.55]) by mx.google.com with ESMTPS id qc10si2575790oeb.52.2015.03.04.10.27.53 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Mar 2015 10:27:53 -0800 (PST) Message-ID: <1425493634.17007.248.camel@misato.fc.hp.com> Subject: Re: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT From: Toshi Kani Date: Wed, 04 Mar 2015 11:27:14 -0700 In-Reply-To: <1424961893.17007.139.camel@misato.fc.hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> <1424823301-30927-8-git-send-email-toshi.kani@hp.com> <20150225072228.GA13061@gmail.com> <1424877601.17007.108.camel@misato.fc.hp.com> <20150226113054.GA4191@gmail.com> <1424961893.17007.139.camel@misato.fc.hp.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Ingo Molnar Cc: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com On Thu, 2015-02-26 at 07:44 -0700, Toshi Kani wrote: > On Thu, 2015-02-26 at 12:30 +0100, Ingo Molnar wrote: > > * Toshi Kani wrote: > > > > > On Wed, 2015-02-25 at 08:22 +0100, Ingo Molnar wrote: > > > > * Toshi Kani wrote: > > > > > > > > > +int set_pages_array_wt(struct page **pages, int addrinarray) > > > > > +{ > > > > > + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); > > > > > +} > > > > > +EXPORT_SYMBOL(set_pages_array_wt); > > > > > > > > So by default we make new APIs EXPORT_SYMBOL_GPL(): we > > > > don't want proprietary modules mucking around with new code > > > > PAT interfaces, we only want modules we can analyze and fix > > > > in detail. > > > > > > Right. I have one question for this case. This > > > set_pages_array_wt() extends the set_pages_array_xx() > > > family, which are all exported with EXPORT_SYMBOL() > > > today. In this case, should we keep them exported in the > > > consistent manner, or should we still use GPL when adding > > > a new one? > > > > Still keep it GPL, it's a new API that old modules > > obviously don't use. > > Got it. Thanks for the clarification. Since this is a minor change and there is no other comment at this point, I've submitted the updated patch 7/7 alone with the following subject for saving your mail box. :-) [PATCH v8-UPDATE 7/7] x86, mm: Add set_memory_wt() for WT Thanks, -Toshi -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752761AbbBYAPg (ORCPT ); Tue, 24 Feb 2015 19:15:36 -0500 Received: from g4t3426.houston.hp.com ([15.201.208.54]:34503 "EHLO g4t3426.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751222AbbBYAPf (ORCPT ); Tue, 24 Feb 2015 19:15:35 -0500 From: Toshi Kani To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com Subject: [PATCH v8 0/7] Support Write-Through mapping on x86 Date: Tue, 24 Feb 2015 17:14:54 -0700 Message-Id: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> X-Mailer: git-send-email 1.9.3 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patchset adds support of Write-Through (WT) mapping on x86. The study below shows that using WT mapping may be useful for non-volatile memory. http://www.hpl.hp.com/techreports/2012/HPL-2012-236.pdf All new/modified interfaces have been tested. v8: - Rebased to 4.0-rc1 and resolved conflicts with 9d34cfdf4 in patch 5/7. v7: - Rebased to 3.19-rc3 as Juergen's patchset for the PAT management has been accepted. v6: - Dropped the patch moving [set|get]_page_memtype() to pat.c since the tip branch already has this change. - Fixed an issue when CONFIG_X86_PAT is not defined. v5: - Clarified comment of why using slot 7. (Andy Lutomirski, Thomas Gleixner) - Moved [set|get]_page_memtype() to pat.c. (Thomas Gleixner) - Removed BUG() from set_page_memtype(). (Thomas Gleixner) v4: - Added set_memory_wt() by adding WT support of regular memory. v3: - Dropped the set_memory_wt() patch. (Andy Lutomirski) - Refactored the !pat_enabled handling. (H. Peter Anvin, Andy Lutomirski) - Added the picture of PTE encoding. (Konrad Rzeszutek Wilk) v2: - Changed WT to use slot 7 of the PAT MSR. (H. Peter Anvin, Andy Lutomirski) - Changed to have conservative checks to exclude all Pentium 2, 3, M, and 4 families. (Ingo Molnar, Henrique de Moraes Holschuh, Andy Lutomirski) - Updated documentation to cover WT interfaces and usages. (Andy Lutomirski, Yigal Korman) --- Toshi Kani (7): 1/7 x86, mm, pat: Set WT to PA7 slot of PAT MSR 2/7 x86, mm, pat: Change reserve_memtype() to handle WT 3/7 x86, mm, asm-gen: Add ioremap_wt() for WT 4/7 x86, mm, pat: Add pgprot_writethrough() for WT 5/7 x86, mm, pat: Refactor !pat_enable handling 6/7 x86, mm, asm: Add WT support to set_page_memtype() 7/7 x86, mm: Add set_memory_wt() for WT --- Documentation/x86/pat.txt | 13 ++- arch/x86/include/asm/cacheflush.h | 6 +- arch/x86/include/asm/io.h | 2 + arch/x86/include/asm/pgtable_types.h | 3 + arch/x86/mm/init.c | 6 +- arch/x86/mm/iomap_32.c | 12 +-- arch/x86/mm/ioremap.c | 26 ++++- arch/x86/mm/pageattr.c | 61 +++++++++-- arch/x86/mm/pat.c | 194 ++++++++++++++++++++++++----------- include/asm-generic/io.h | 9 ++ include/asm-generic/iomap.h | 4 + include/asm-generic/pgtable.h | 4 + 12 files changed, 251 insertions(+), 89 deletions(-) From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752950AbbBYAPk (ORCPT ); Tue, 24 Feb 2015 19:15:40 -0500 Received: from g4t3426.houston.hp.com ([15.201.208.54]:34530 "EHLO g4t3426.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752808AbbBYAPi (ORCPT ); Tue, 24 Feb 2015 19:15:38 -0500 From: Toshi Kani To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani Subject: [PATCH v8 1/7] x86, mm, pat: Set WT to PA7 slot of PAT MSR Date: Tue, 24 Feb 2015 17:14:55 -0700 Message-Id: <1424823301-30927-2-git-send-email-toshi.kani@hp.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch sets WT to the PA7 slot in the PAT MSR when the processor is not affected by the PAT errata. The PA7 slot is chosen to improve robustness in the presence of errata that might cause the high PAT bit to be ignored. This way a buggy PA7 slot access will hit the PA3 slot, which is UC, so at worst we lose performance without causing a correctness issue. The following Intel processors are affected by the PAT errata. errata cpuid ---------------------------------------------------- Pentium 2, A52 family 0x6, model 0x5 Pentium 3, E27 family 0x6, model 0x7, 0x8 Pentium 3 Xenon, G26 family 0x6, model 0x7, 0x8, 0xa Pentium M, Y26 family 0x6, model 0x9 Pentium M 90nm, X9 family 0x6, model 0xd Pentium 4, N46 family 0xf, model 0x0 Instead of making sharp boundary checks, this patch makes conservative checks to exclude all Pentium 2, 3, M and 4 family processors. For such processors, _PAGE_CACHE_MODE_WT is redirected to UC- per the default setup in __cachemode2pte_tbl[]. Signed-off-by: Toshi Kani Reviewed-by: Juergen Gross --- arch/x86/mm/pat.c | 71 ++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 56 insertions(+), 15 deletions(-) diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c index 7ac6869..76b56bc 100644 --- a/arch/x86/mm/pat.c +++ b/arch/x86/mm/pat.c @@ -197,6 +197,7 @@ void pat_init(void) { u64 pat; bool boot_cpu = !boot_pat_state; + struct cpuinfo_x86 *c = &boot_cpu_data; if (!pat_enabled) return; @@ -217,21 +218,61 @@ void pat_init(void) } } - /* Set PWT to Write-Combining. All other bits stay the same */ - /* - * PTE encoding used in Linux: - * PAT - * |PCD - * ||PWT - * ||| - * 000 WB _PAGE_CACHE_WB - * 001 WC _PAGE_CACHE_WC - * 010 UC- _PAGE_CACHE_UC_MINUS - * 011 UC _PAGE_CACHE_UC - * PAT bit unused - */ - pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) | - PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, UC); + if ((c->x86_vendor == X86_VENDOR_INTEL) && + (((c->x86 == 0x6) && (c->x86_model <= 0xd)) || + ((c->x86 == 0xf) && (c->x86_model <= 0x6)))) { + /* + * PAT support with the lower four entries. Intel Pentium 2, + * 3, M, and 4 are affected by PAT errata, which makes the + * upper four entries unusable. We do not use the upper four + * entries for all the affected processor families for safe. + * + * PTE encoding used in Linux: + * PAT + * |PCD + * ||PWT PAT + * ||| slot + * 000 0 WB : _PAGE_CACHE_MODE_WB + * 001 1 WC : _PAGE_CACHE_MODE_WC + * 010 2 UC-: _PAGE_CACHE_MODE_UC_MINUS + * 011 3 UC : _PAGE_CACHE_MODE_UC + * PAT bit unused + * + * NOTE: When WT or WP is used, it is redirected to UC- per + * the default setup in __cachemode2pte_tbl[]. + */ + pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) | + PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, UC); + } else { + /* + * PAT full support. We put WT in slot 7 to improve + * robustness in the presence of errata that might cause + * the high PAT bit to be ignored. This way a buggy slot 7 + * access will hit slot 3, and slot 3 is UC, so at worst + * we lose performance without causing a correctness issue. + * Pentium 4 erratum N46 is an example of such an erratum, + * although we try not to use PAT at all on affected CPUs. + * + * PTE encoding used in Linux: + * PAT + * |PCD + * ||PWT PAT + * ||| slot + * 000 0 WB : _PAGE_CACHE_MODE_WB + * 001 1 WC : _PAGE_CACHE_MODE_WC + * 010 2 UC-: _PAGE_CACHE_MODE_UC_MINUS + * 011 3 UC : _PAGE_CACHE_MODE_UC + * 100 4 WB : Reserved + * 101 5 WC : Reserved + * 110 6 UC-: Reserved + * 111 7 WT : _PAGE_CACHE_MODE_WT + * + * The reserved slots are unused, but mapped to their + * corresponding types in the presence of PAT errata. + */ + pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) | + PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, WT); + } /* Boot CPU check */ if (!boot_pat_state) { From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753151AbbBYAPm (ORCPT ); Tue, 24 Feb 2015 19:15:42 -0500 Received: from g4t3427.houston.hp.com ([15.201.208.55]:40599 "EHLO g4t3427.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752966AbbBYAPl (ORCPT ); Tue, 24 Feb 2015 19:15:41 -0500 From: Toshi Kani To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani Subject: [PATCH v8 2/7] x86, mm, pat: Change reserve_memtype() to handle WT Date: Tue, 24 Feb 2015 17:14:56 -0700 Message-Id: <1424823301-30927-3-git-send-email-toshi.kani@hp.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch changes reserve_memtype() to handle the WT cache mode with PAT. When PAT is not enabled, WB and UC- are the only types supported. When a target range is in RAM, reserve_ram_pages_type() verifies the requested type. In this case, WT and WP requests fail with -EINVAL and UC gets redirected to UC- since set_page_memtype() is limited to handle three types, WB, WC and UC-. Signed-off-by: Toshi Kani Reviewed-by: Konrad Rzeszutek Wilk --- arch/x86/mm/pat.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c index 76b56bc..b4db0d8 100644 --- a/arch/x86/mm/pat.c +++ b/arch/x86/mm/pat.c @@ -365,6 +365,8 @@ static int pat_pagerange_is_ram(resource_size_t start, resource_size_t end) /* * For RAM pages, we use page flags to mark the pages with appropriate type. + * The page flags are limited to three types, WB, WC and UC-. + * WT and WP requests fail with -EINVAL, and UC gets redirected to UC-. * Here we do two pass: * - Find the memtype of all the pages in the range, look for any conflicts * - In case of no conflicts, set the new memtype for pages in the range @@ -376,6 +378,13 @@ static int reserve_ram_pages_type(u64 start, u64 end, struct page *page; u64 pfn; + if ((req_type == _PAGE_CACHE_MODE_WT) || + (req_type == _PAGE_CACHE_MODE_WP)) { + if (new_type) + *new_type = _PAGE_CACHE_MODE_UC_MINUS; + return -EINVAL; + } + if (req_type == _PAGE_CACHE_MODE_UC) { /* We do not support strong UC */ WARN_ON_ONCE(1); @@ -425,6 +434,7 @@ static int free_ram_pages_type(u64 start, u64 end) * - _PAGE_CACHE_MODE_WC * - _PAGE_CACHE_MODE_UC_MINUS * - _PAGE_CACHE_MODE_UC + * - _PAGE_CACHE_MODE_WT * * If new_type is NULL, function will return an error if it cannot reserve the * region with req_type. If new_type is non-NULL, function will return @@ -442,12 +452,12 @@ int reserve_memtype(u64 start, u64 end, enum page_cache_mode req_type, BUG_ON(start >= end); /* end is exclusive */ if (!pat_enabled) { - /* This is identical to page table setting without PAT */ + /* WB and UC- are the only types supported without PAT */ if (new_type) { - if (req_type == _PAGE_CACHE_MODE_WC) - *new_type = _PAGE_CACHE_MODE_UC_MINUS; + if (req_type == _PAGE_CACHE_MODE_WB) + *new_type = _PAGE_CACHE_MODE_WB; else - *new_type = req_type; + *new_type = _PAGE_CACHE_MODE_UC_MINUS; } return 0; } From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753252AbbBYAPq (ORCPT ); Tue, 24 Feb 2015 19:15:46 -0500 Received: from g4t3425.houston.hp.com ([15.201.208.53]:46535 "EHLO g4t3425.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752808AbbBYAPn (ORCPT ); Tue, 24 Feb 2015 19:15:43 -0500 From: Toshi Kani To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani Subject: [PATCH v8 3/7] x86, mm, asm-gen: Add ioremap_wt() for WT Date: Tue, 24 Feb 2015 17:14:57 -0700 Message-Id: <1424823301-30927-4-git-send-email-toshi.kani@hp.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch adds ioremap_wt() for creating WT mapping on x86. It follows the same model as ioremap_wc() for multi-architecture support. ARCH_HAS_IOREMAP_WT is defined in the x86 version of io.h to indicate that ioremap_wt() is implemented on x86. Also update the PAT documentation file to cover ioremap_wt(). Signed-off-by: Toshi Kani Reviewed-by: Konrad Rzeszutek Wilk --- Documentation/x86/pat.txt | 4 +++- arch/x86/include/asm/io.h | 2 ++ arch/x86/mm/ioremap.c | 24 ++++++++++++++++++++++++ include/asm-generic/io.h | 9 +++++++++ include/asm-generic/iomap.h | 4 ++++ 5 files changed, 42 insertions(+), 1 deletion(-) diff --git a/Documentation/x86/pat.txt b/Documentation/x86/pat.txt index cf08c9f..be7b8c2 100644 --- a/Documentation/x86/pat.txt +++ b/Documentation/x86/pat.txt @@ -12,7 +12,7 @@ virtual addresses. PAT allows for different types of memory attributes. The most commonly used ones that will be supported at this time are Write-back, Uncached, -Write-combined and Uncached Minus. +Write-combined, Write-through and Uncached Minus. PAT APIs @@ -38,6 +38,8 @@ ioremap_nocache | -- | UC- | UC- | | | | | ioremap_wc | -- | -- | WC | | | | | +ioremap_wt | -- | -- | WT | + | | | | set_memory_uc | UC- | -- | -- | set_memory_wb | | | | | | | | diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h index 34a5b93..81942ef 100644 --- a/arch/x86/include/asm/io.h +++ b/arch/x86/include/asm/io.h @@ -35,6 +35,7 @@ */ #define ARCH_HAS_IOREMAP_WC +#define ARCH_HAS_IOREMAP_WT #include #include @@ -320,6 +321,7 @@ extern void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr); extern int ioremap_change_attr(unsigned long vaddr, unsigned long size, enum page_cache_mode pcm); extern void __iomem *ioremap_wc(resource_size_t offset, unsigned long size); +extern void __iomem *ioremap_wt(resource_size_t offset, unsigned long size); extern bool is_early_ioremap_ptep(pte_t *ptep); diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index fdf617c..4f794b2 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -167,6 +167,10 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr, prot = __pgprot(pgprot_val(prot) | cachemode2protval(_PAGE_CACHE_MODE_WC)); break; + case _PAGE_CACHE_MODE_WT: + prot = __pgprot(pgprot_val(prot) | + cachemode2protval(_PAGE_CACHE_MODE_WT)); + break; case _PAGE_CACHE_MODE_WB: break; } @@ -261,6 +265,26 @@ void __iomem *ioremap_wc(resource_size_t phys_addr, unsigned long size) } EXPORT_SYMBOL(ioremap_wc); +/** + * ioremap_wt - map memory into CPU space write through + * @phys_addr: bus address of the memory + * @size: size of the resource to map + * + * This version of ioremap ensures that the memory is marked write through. + * Write through stores data into memory while keeping the cache up-to-date. + * + * Must be freed with iounmap. + */ +void __iomem *ioremap_wt(resource_size_t phys_addr, unsigned long size) +{ + if (pat_enabled) + return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WT, + __builtin_return_address(0)); + else + return ioremap_nocache(phys_addr, size); +} +EXPORT_SYMBOL(ioremap_wt); + void __iomem *ioremap_cache(resource_size_t phys_addr, unsigned long size) { return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WB, diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h index 9db0423..bae62dc 100644 --- a/include/asm-generic/io.h +++ b/include/asm-generic/io.h @@ -777,8 +777,17 @@ static inline void __iomem *ioremap_wc(phys_addr_t offset, size_t size) } #endif +#ifndef ioremap_wt +#define ioremap_wt ioremap_wt +static inline void __iomem *ioremap_wt(phys_addr_t offset, size_t size) +{ + return ioremap_nocache(offset, size); +} +#endif + #ifndef iounmap #define iounmap iounmap + static inline void iounmap(void __iomem *addr) { } diff --git a/include/asm-generic/iomap.h b/include/asm-generic/iomap.h index 1b41011..d8f8622 100644 --- a/include/asm-generic/iomap.h +++ b/include/asm-generic/iomap.h @@ -66,6 +66,10 @@ extern void ioport_unmap(void __iomem *); #define ioremap_wc ioremap_nocache #endif +#ifndef ARCH_HAS_IOREMAP_WT +#define ioremap_wt ioremap_nocache +#endif + #ifdef CONFIG_PCI /* Destroy a virtual mapping cookie for a PCI BAR (memory or IO) */ struct pci_dev; From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753169AbbBYAQ6 (ORCPT ); Tue, 24 Feb 2015 19:16:58 -0500 Received: from g4t3425.houston.hp.com ([15.201.208.53]:46553 "EHLO g4t3425.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752966AbbBYAPo (ORCPT ); Tue, 24 Feb 2015 19:15:44 -0500 From: Toshi Kani To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani Subject: [PATCH v8 4/7] x86, mm, pat: Add pgprot_writethrough() for WT Date: Tue, 24 Feb 2015 17:14:58 -0700 Message-Id: <1424823301-30927-5-git-send-email-toshi.kani@hp.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch adds pgprot_writethrough() for setting WT to a given pgprot_t. Signed-off-by: Toshi Kani Reviewed-by: Konrad Rzeszutek Wilk --- arch/x86/include/asm/pgtable_types.h | 3 +++ arch/x86/mm/pat.c | 10 ++++++++++ include/asm-generic/pgtable.h | 4 ++++ 3 files changed, 17 insertions(+) diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h index 8c7c108..e962f06 100644 --- a/arch/x86/include/asm/pgtable_types.h +++ b/arch/x86/include/asm/pgtable_types.h @@ -367,6 +367,9 @@ extern int nx_enabled; #define pgprot_writecombine pgprot_writecombine extern pgprot_t pgprot_writecombine(pgprot_t prot); +#define pgprot_writethrough pgprot_writethrough +extern pgprot_t pgprot_writethrough(pgprot_t prot); + /* Indicate that x86 has its own track and untrack pfn vma functions */ #define __HAVE_PFNMAP_TRACKING diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c index b4db0d8..d3f7058 100644 --- a/arch/x86/mm/pat.c +++ b/arch/x86/mm/pat.c @@ -972,6 +972,16 @@ pgprot_t pgprot_writecombine(pgprot_t prot) } EXPORT_SYMBOL_GPL(pgprot_writecombine); +pgprot_t pgprot_writethrough(pgprot_t prot) +{ + if (pat_enabled) + return __pgprot(pgprot_val(prot) | + cachemode2protval(_PAGE_CACHE_MODE_WT)); + else + return pgprot_noncached(prot); +} +EXPORT_SYMBOL_GPL(pgprot_writethrough); + #if defined(CONFIG_DEBUG_FS) && defined(CONFIG_X86_PAT) static struct memtype *memtype_get_idx(loff_t pos) diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 4d46085..d1e59a4 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -256,6 +256,10 @@ static inline int pmd_same(pmd_t pmd_a, pmd_t pmd_b) #define pgprot_writecombine pgprot_noncached #endif +#ifndef pgprot_writethrough +#define pgprot_writethrough pgprot_noncached +#endif + #ifndef pgprot_device #define pgprot_device pgprot_noncached #endif From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752416AbbBYAPs (ORCPT ); Tue, 24 Feb 2015 19:15:48 -0500 Received: from g4t3425.houston.hp.com ([15.201.208.53]:46574 "EHLO g4t3425.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753247AbbBYAPq (ORCPT ); Tue, 24 Feb 2015 19:15:46 -0500 From: Toshi Kani To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani Subject: [PATCH v8 5/7] x86, mm, pat: Refactor !pat_enabled handling Date: Tue, 24 Feb 2015 17:14:59 -0700 Message-Id: <1424823301-30927-6-git-send-email-toshi.kani@hp.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch refactors the !pat_enabled handling code and integrates it into the PAT abstraction code. The PAT table is emulated by corresponding to the two cache attribute bits, PWT (Write Through) and PCD (Cache Disable). The emulated PAT table is the same as the BIOS default setup when the system has PAT but the "nopat" boot option is specified. The emulated PAT table is also used when MSR_IA32_CR_PAT returns 0 (9d34cfdf4). Signed-off-by: Toshi Kani Reviewed-by: Juergen Gross --- arch/x86/mm/init.c | 6 ++- arch/x86/mm/iomap_32.c | 12 +++---- arch/x86/mm/ioremap.c | 10 +----- arch/x86/mm/pageattr.c | 3 -- arch/x86/mm/pat.c | 85 ++++++++++++++++++++++++++++-------------------- 5 files changed, 61 insertions(+), 55 deletions(-) diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index a110efc..98b9518 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -37,7 +37,7 @@ */ uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { [_PAGE_CACHE_MODE_WB] = 0, - [_PAGE_CACHE_MODE_WC] = _PAGE_PWT, + [_PAGE_CACHE_MODE_WC] = _PAGE_PCD, [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD, [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT, [_PAGE_CACHE_MODE_WT] = _PAGE_PCD, @@ -46,11 +46,11 @@ uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { EXPORT_SYMBOL(__cachemode2pte_tbl); uint8_t __pte2cachemode_tbl[8] = { [__pte2cm_idx(0)] = _PAGE_CACHE_MODE_WB, - [__pte2cm_idx(_PAGE_PWT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx(_PAGE_PWT)] = _PAGE_CACHE_MODE_UC_MINUS, [__pte2cm_idx(_PAGE_PCD)] = _PAGE_CACHE_MODE_UC_MINUS, [__pte2cm_idx(_PAGE_PWT | _PAGE_PCD)] = _PAGE_CACHE_MODE_UC, [__pte2cm_idx(_PAGE_PAT)] = _PAGE_CACHE_MODE_WB, - [__pte2cm_idx(_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx(_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS, [__pte2cm_idx(_PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS, [__pte2cm_idx(_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC, }; diff --git a/arch/x86/mm/iomap_32.c b/arch/x86/mm/iomap_32.c index 9ca35fc..2c51a2b 100644 --- a/arch/x86/mm/iomap_32.c +++ b/arch/x86/mm/iomap_32.c @@ -77,13 +77,13 @@ void __iomem * iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot) { /* - * For non-PAT systems, promote PAGE_KERNEL_WC to PAGE_KERNEL_UC_MINUS. - * PAGE_KERNEL_WC maps to PWT, which translates to uncached if the - * MTRR is UC or WC. UC_MINUS gets the real intention, of the - * user, which is "WC if the MTRR is WC, UC if you can't do that." + * For non-PAT systems, translate non-WB request to UC- just in + * case the caller set the PWT bit to prot directly without using + * pgprot_writecombine(). UC- translates to uncached if the MTRR + * is UC or WC. UC- gets the real intention, of the user, which is + * "WC if the MTRR is WC, UC if you can't do that." */ - if (!pat_enabled && pgprot_val(prot) == - (__PAGE_KERNEL | cachemode2protval(_PAGE_CACHE_MODE_WC))) + if (!pat_enabled && pgprot2cachemode(prot) != _PAGE_CACHE_MODE_WB) prot = __pgprot(__PAGE_KERNEL | cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS)); diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index 4f794b2..f73db48 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -257,11 +257,8 @@ EXPORT_SYMBOL(ioremap_nocache); */ void __iomem *ioremap_wc(resource_size_t phys_addr, unsigned long size) { - if (pat_enabled) - return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WC, + return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WC, __builtin_return_address(0)); - else - return ioremap_nocache(phys_addr, size); } EXPORT_SYMBOL(ioremap_wc); @@ -277,11 +274,8 @@ EXPORT_SYMBOL(ioremap_wc); */ void __iomem *ioremap_wt(resource_size_t phys_addr, unsigned long size) { - if (pat_enabled) - return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WT, + return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WT, __builtin_return_address(0)); - else - return ioremap_nocache(phys_addr, size); } EXPORT_SYMBOL(ioremap_wt); diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 536ea2f..1965fc8 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -1573,9 +1573,6 @@ int set_memory_wc(unsigned long addr, int numpages) { int ret; - if (!pat_enabled) - return set_memory_uc(addr, numpages); - ret = reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE, _PAGE_CACHE_MODE_WC, NULL); if (ret) diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c index d3f7058..db3cc89 100644 --- a/arch/x86/mm/pat.c +++ b/arch/x86/mm/pat.c @@ -181,7 +181,11 @@ void pat_init_cache_modes(void) char pat_msg[33]; u64 pat; - rdmsrl(MSR_IA32_CR_PAT, pat); + if (pat_enabled) + rdmsrl(MSR_IA32_CR_PAT, pat); + else + pat = boot_pat_state; + pat_msg[32] = 0; for (i = 7; i >= 0; i--) { cache = pat_get_cache_mode((pat >> (i * 8)) & 7, @@ -199,28 +203,58 @@ void pat_init(void) bool boot_cpu = !boot_pat_state; struct cpuinfo_x86 *c = &boot_cpu_data; - if (!pat_enabled) - return; - if (!cpu_has_pat) { if (!boot_pat_state) { pat_disable("PAT not supported by CPU."); - return; - } else { + } else if (pat_enabled) { /* * If this happens we are on a secondary CPU, but * switched to PAT on the boot CPU. We have no way to * undo PAT. */ - printk(KERN_ERR "PAT enabled, " + pr_err("PAT enabled, " "but not supported by secondary CPU\n"); BUG(); } } - if ((c->x86_vendor == X86_VENDOR_INTEL) && - (((c->x86 == 0x6) && (c->x86_model <= 0xd)) || - ((c->x86 == 0xf) && (c->x86_model <= 0x6)))) { + /* Boot CPU check */ + if (pat_enabled && !boot_pat_state) { + rdmsrl(MSR_IA32_CR_PAT, boot_pat_state); + if (!boot_pat_state) + pat_disable("PAT read returns always zero, disabled."); + + } + + if (!pat_enabled) { + /* + * No PAT. Emulate the PAT table that corresponds to the two + * cache bits, PWT (Write Through) and PCD (Cache Disable). + * This setup is the same as the BIOS default setup when the + * system has PAT but the "nopat" boot option is specified. + * This emulated PAT table is also used when MSR_IA32_CR_PAT + * returns 0. + * + * PTE encoding used in Linux: + * PCD + * |PWT PAT + * || slot + * 00 0 WB : _PAGE_CACHE_MODE_WB + * 01 1 WT : _PAGE_CACHE_MODE_WT + * 10 2 UC-: _PAGE_CACHE_MODE_UC_MINUS + * 11 3 UC : _PAGE_CACHE_MODE_UC + * + * NOTE: When WC or WP is used, it is redirected to UC- per + * the default setup in __cachemode2pte_tbl[]. + */ + pat = PAT(0, WB) | PAT(1, WT) | PAT(2, UC_MINUS) | PAT(3, UC) | + PAT(4, WB) | PAT(5, WT) | PAT(6, UC_MINUS) | PAT(7, UC); + if (!boot_pat_state) + boot_pat_state = pat; + + } else if ((c->x86_vendor == X86_VENDOR_INTEL) && + (((c->x86 == 0x6) && (c->x86_model <= 0xd)) || + ((c->x86 == 0xf) && (c->x86_model <= 0x6)))) { /* * PAT support with the lower four entries. Intel Pentium 2, * 3, M, and 4 are affected by PAT errata, which makes the @@ -274,16 +308,8 @@ void pat_init(void) PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, WT); } - /* Boot CPU check */ - if (!boot_pat_state) { - rdmsrl(MSR_IA32_CR_PAT, boot_pat_state); - if (!boot_pat_state) { - pat_disable("PAT read returns always zero, disabled."); - return; - } - } - - wrmsrl(MSR_IA32_CR_PAT, pat); + if (pat_enabled) + wrmsrl(MSR_IA32_CR_PAT, pat); if (boot_cpu) pat_init_cache_modes(); @@ -452,13 +478,8 @@ int reserve_memtype(u64 start, u64 end, enum page_cache_mode req_type, BUG_ON(start >= end); /* end is exclusive */ if (!pat_enabled) { - /* WB and UC- are the only types supported without PAT */ - if (new_type) { - if (req_type == _PAGE_CACHE_MODE_WB) - *new_type = _PAGE_CACHE_MODE_WB; - else - *new_type = _PAGE_CACHE_MODE_UC_MINUS; - } + if (new_type) + *new_type = req_type; return 0; } @@ -964,21 +985,15 @@ void untrack_pfn(struct vm_area_struct *vma, unsigned long pfn, pgprot_t pgprot_writecombine(pgprot_t prot) { - if (pat_enabled) - return __pgprot(pgprot_val(prot) | + return __pgprot(pgprot_val(prot) | cachemode2protval(_PAGE_CACHE_MODE_WC)); - else - return pgprot_noncached(prot); } EXPORT_SYMBOL_GPL(pgprot_writecombine); pgprot_t pgprot_writethrough(pgprot_t prot) { - if (pat_enabled) - return __pgprot(pgprot_val(prot) | + return __pgprot(pgprot_val(prot) | cachemode2protval(_PAGE_CACHE_MODE_WT)); - else - return pgprot_noncached(prot); } EXPORT_SYMBOL_GPL(pgprot_writethrough); From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753325AbbBYAPu (ORCPT ); Tue, 24 Feb 2015 19:15:50 -0500 Received: from g4t3425.houston.hp.com ([15.201.208.53]:46612 "EHLO g4t3425.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753286AbbBYAPs (ORCPT ); Tue, 24 Feb 2015 19:15:48 -0500 From: Toshi Kani To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani Subject: [PATCH v8 6/7] x86, mm, asm: Add WT support to set_page_memtype() Date: Tue, 24 Feb 2015 17:15:00 -0700 Message-Id: <1424823301-30927-7-git-send-email-toshi.kani@hp.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org As set_memory_wb() calls set_page_memtype() with -1, _PGMT_DEFAULT is solely used for tracking the WB type. _PGMT_WB is defined but unused. Hence, this patch renames _PGMT_DEFAULT to _PGMT_WB to clarify its usage, and releases the slot used by _PGMT_WB before. As a result, set_memory_wb() is changed to call set_page_memtype() with _PGMT_WB, and get_page_memtype() returns _PAGE_CACHE_MODE_WB for _PGMT_WB. This patch then defines _PGMT_WT to the released slot. This enables set_page_memtype() to track the WT type. Signed-off-by: Toshi Kani --- arch/x86/mm/pat.c | 48 +++++++++++++++++++++--------------------------- 1 file changed, 21 insertions(+), 27 deletions(-) diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c index db3cc89..cfb14ac 100644 --- a/arch/x86/mm/pat.c +++ b/arch/x86/mm/pat.c @@ -69,18 +69,18 @@ static u64 __read_mostly boot_pat_state; #ifdef CONFIG_X86_PAT /* - * X86 PAT uses page flags WC and Uncached together to keep track of - * memory type of pages that have backing page struct. X86 PAT supports 3 - * different memory types, _PAGE_CACHE_MODE_WB, _PAGE_CACHE_MODE_WC and - * _PAGE_CACHE_MODE_UC_MINUS and fourth state where page's memory type has not - * been changed from its default (value of -1 used to denote this). + * X86 PAT uses page flags arch_1 and uncached together to keep track of + * memory type of pages that have backing page struct. X86 PAT supports 4 + * different memory types, _PAGE_CACHE_MODE_WT, _PAGE_CACHE_MODE_WC, + * _PAGE_CACHE_MODE_UC_MINUS and _PAGE_CACHE_MODE_WB where page's memory + * type has not been changed from its default. * Note we do not support _PAGE_CACHE_MODE_UC here. */ -#define _PGMT_DEFAULT 0 +#define _PGMT_WB 0 /* default */ #define _PGMT_WC (1UL << PG_arch_1) #define _PGMT_UC_MINUS (1UL << PG_uncached) -#define _PGMT_WB (1UL << PG_uncached | 1UL << PG_arch_1) +#define _PGMT_WT (1UL << PG_uncached | 1UL << PG_arch_1) #define _PGMT_MASK (1UL << PG_uncached | 1UL << PG_arch_1) #define _PGMT_CLEAR_MASK (~_PGMT_MASK) @@ -88,14 +88,14 @@ static inline enum page_cache_mode get_page_memtype(struct page *pg) { unsigned long pg_flags = pg->flags & _PGMT_MASK; - if (pg_flags == _PGMT_DEFAULT) - return -1; + if (pg_flags == _PGMT_WB) + return _PAGE_CACHE_MODE_WB; else if (pg_flags == _PGMT_WC) return _PAGE_CACHE_MODE_WC; else if (pg_flags == _PGMT_UC_MINUS) return _PAGE_CACHE_MODE_UC_MINUS; else - return _PAGE_CACHE_MODE_WB; + return _PAGE_CACHE_MODE_WT; } static inline void set_page_memtype(struct page *pg, @@ -112,11 +112,12 @@ static inline void set_page_memtype(struct page *pg, case _PAGE_CACHE_MODE_UC_MINUS: memtype_flags = _PGMT_UC_MINUS; break; - case _PAGE_CACHE_MODE_WB: - memtype_flags = _PGMT_WB; + case _PAGE_CACHE_MODE_WT: + memtype_flags = _PGMT_WT; break; + case _PAGE_CACHE_MODE_WB: default: - memtype_flags = _PGMT_DEFAULT; + memtype_flags = _PGMT_WB; /* default */ break; } @@ -391,8 +392,9 @@ static int pat_pagerange_is_ram(resource_size_t start, resource_size_t end) /* * For RAM pages, we use page flags to mark the pages with appropriate type. - * The page flags are limited to three types, WB, WC and UC-. - * WT and WP requests fail with -EINVAL, and UC gets redirected to UC-. + * The page flags are limited to four types, WB (default), WC, WT and UC-. + * WP request fails with -EINVAL, and UC gets redirected to UC-. + * A new memtype can only be set to the default memtype WB. * Here we do two pass: * - Find the memtype of all the pages in the range, look for any conflicts * - In case of no conflicts, set the new memtype for pages in the range @@ -404,8 +406,7 @@ static int reserve_ram_pages_type(u64 start, u64 end, struct page *page; u64 pfn; - if ((req_type == _PAGE_CACHE_MODE_WT) || - (req_type == _PAGE_CACHE_MODE_WP)) { + if (req_type == _PAGE_CACHE_MODE_WP) { if (new_type) *new_type = _PAGE_CACHE_MODE_UC_MINUS; return -EINVAL; @@ -422,7 +423,7 @@ static int reserve_ram_pages_type(u64 start, u64 end, page = pfn_to_page(pfn); type = get_page_memtype(page); - if (type != -1) { + if (type != _PAGE_CACHE_MODE_WB) { pr_info("reserve_ram_pages_type failed [mem %#010Lx-%#010Lx], track 0x%x, req 0x%x\n", start, end - 1, type, req_type); if (new_type) @@ -449,7 +450,7 @@ static int free_ram_pages_type(u64 start, u64 end) for (pfn = (start >> PAGE_SHIFT); pfn < (end >> PAGE_SHIFT); ++pfn) { page = pfn_to_page(pfn); - set_page_memtype(page, -1); + set_page_memtype(page, _PAGE_CACHE_MODE_WB); } return 0; } @@ -589,7 +590,7 @@ int free_memtype(u64 start, u64 end) * Only to be called when PAT is enabled * * Returns _PAGE_CACHE_MODE_WB, _PAGE_CACHE_MODE_WC, _PAGE_CACHE_MODE_UC_MINUS - * or _PAGE_CACHE_MODE_UC + * or _PAGE_CACHE_MODE_WT. */ static enum page_cache_mode lookup_memtype(u64 paddr) { @@ -603,13 +604,6 @@ static enum page_cache_mode lookup_memtype(u64 paddr) struct page *page; page = pfn_to_page(paddr >> PAGE_SHIFT); rettype = get_page_memtype(page); - /* - * -1 from get_page_memtype() implies RAM page is in its - * default state and not reserved, and hence of type WB - */ - if (rettype == -1) - rettype = _PAGE_CACHE_MODE_WB; - return rettype; } From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753356AbbBYAPz (ORCPT ); Tue, 24 Feb 2015 19:15:55 -0500 Received: from g4t3425.houston.hp.com ([15.201.208.53]:46647 "EHLO g4t3425.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753315AbbBYAPu (ORCPT ); Tue, 24 Feb 2015 19:15:50 -0500 From: Toshi Kani To: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com, Toshi Kani Subject: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT Date: Tue, 24 Feb 2015 17:15:01 -0700 Message-Id: <1424823301-30927-8-git-send-email-toshi.kani@hp.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch adds set_memory_wt(), set_memory_array_wt() and set_pages_array_wt() for setting specified range(s) of the regular memory to WT. Signed-off-by: Toshi Kani --- Documentation/x86/pat.txt | 9 ++++-- arch/x86/include/asm/cacheflush.h | 6 +++- arch/x86/mm/pageattr.c | 58 +++++++++++++++++++++++++++++++++---- 3 files changed, 63 insertions(+), 10 deletions(-) diff --git a/Documentation/x86/pat.txt b/Documentation/x86/pat.txt index be7b8c2..bf4339c 100644 --- a/Documentation/x86/pat.txt +++ b/Documentation/x86/pat.txt @@ -46,6 +46,9 @@ set_memory_uc | UC- | -- | -- | set_memory_wc | WC | -- | -- | set_memory_wb | | | | | | | | +set_memory_wt | WT | -- | -- | + set_memory_wb | | | | + | | | | pci sysfs resource | -- | -- | UC- | | | | | pci sysfs resource_wc | -- | -- | WC | @@ -117,8 +120,8 @@ can be more restrictive, in case of any existing aliasing for that address. For example: If there is an existing uncached mapping, a new ioremap_wc can return uncached mapping in place of write-combine requested. -set_memory_[uc|wc] and set_memory_wb should be used in pairs, where driver will -first make a region uc or wc and switch it back to wb after use. +set_memory_[uc|wc|wt] and set_memory_wb should be used in pairs, where driver +will first make a region uc, wc or wt and switch it back to wb after use. Over time writes to /proc/mtrr will be deprecated in favor of using PAT based interfaces. Users writing to /proc/mtrr are suggested to use above interfaces. @@ -126,7 +129,7 @@ interfaces. Users writing to /proc/mtrr are suggested to use above interfaces. Drivers should use ioremap_[uc|wc] to access PCI BARs with [uc|wc] access types. -Drivers should use set_memory_[uc|wc] to set access type for RAM ranges. +Drivers should use set_memory_[uc|wc|wt] to set access type for RAM ranges. PAT debugging diff --git a/arch/x86/include/asm/cacheflush.h b/arch/x86/include/asm/cacheflush.h index 47c8e32..b6f7457 100644 --- a/arch/x86/include/asm/cacheflush.h +++ b/arch/x86/include/asm/cacheflush.h @@ -8,7 +8,7 @@ /* * The set_memory_* API can be used to change various attributes of a virtual * address range. The attributes include: - * Cachability : UnCached, WriteCombining, WriteBack + * Cachability : UnCached, WriteCombining, WriteThrough, WriteBack * Executability : eXeutable, NoteXecutable * Read/Write : ReadOnly, ReadWrite * Presence : NotPresent @@ -35,9 +35,11 @@ int _set_memory_uc(unsigned long addr, int numpages); int _set_memory_wc(unsigned long addr, int numpages); +int _set_memory_wt(unsigned long addr, int numpages); int _set_memory_wb(unsigned long addr, int numpages); int set_memory_uc(unsigned long addr, int numpages); int set_memory_wc(unsigned long addr, int numpages); +int set_memory_wt(unsigned long addr, int numpages); int set_memory_wb(unsigned long addr, int numpages); int set_memory_x(unsigned long addr, int numpages); int set_memory_nx(unsigned long addr, int numpages); @@ -48,10 +50,12 @@ int set_memory_4k(unsigned long addr, int numpages); int set_memory_array_uc(unsigned long *addr, int addrinarray); int set_memory_array_wc(unsigned long *addr, int addrinarray); +int set_memory_array_wt(unsigned long *addr, int addrinarray); int set_memory_array_wb(unsigned long *addr, int addrinarray); int set_pages_array_uc(struct page **pages, int addrinarray); int set_pages_array_wc(struct page **pages, int addrinarray); +int set_pages_array_wt(struct page **pages, int addrinarray); int set_pages_array_wb(struct page **pages, int addrinarray); /* diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 1965fc8..9308527 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -1504,12 +1504,10 @@ EXPORT_SYMBOL(set_memory_uc); static int _set_memory_array(unsigned long *addr, int addrinarray, enum page_cache_mode new_type) { + enum page_cache_mode set_type; int i, j; int ret; - /* - * for now UC MINUS. see comments in ioremap_nocache() - */ for (i = 0; i < addrinarray; i++) { ret = reserve_memtype(__pa(addr[i]), __pa(addr[i]) + PAGE_SIZE, new_type, NULL); @@ -1517,9 +1515,12 @@ static int _set_memory_array(unsigned long *addr, int addrinarray, goto out_free; } + /* If WC, set to UC- first and then WC */ + set_type = (new_type == _PAGE_CACHE_MODE_WC) ? + _PAGE_CACHE_MODE_UC_MINUS : new_type; + ret = change_page_attr_set(addr, addrinarray, - cachemode2pgprot(_PAGE_CACHE_MODE_UC_MINUS), - 1); + cachemode2pgprot(set_type), 1); if (!ret && new_type == _PAGE_CACHE_MODE_WC) ret = change_page_attr_set_clr(addr, addrinarray, @@ -1551,6 +1552,12 @@ int set_memory_array_wc(unsigned long *addr, int addrinarray) } EXPORT_SYMBOL(set_memory_array_wc); +int set_memory_array_wt(unsigned long *addr, int addrinarray) +{ + return _set_memory_array(addr, addrinarray, _PAGE_CACHE_MODE_WT); +} +EXPORT_SYMBOL(set_memory_array_wt); + int _set_memory_wc(unsigned long addr, int numpages) { int ret; @@ -1591,6 +1598,34 @@ out_err: } EXPORT_SYMBOL(set_memory_wc); +int _set_memory_wt(unsigned long addr, int numpages) +{ + return change_page_attr_set(&addr, numpages, + cachemode2pgprot(_PAGE_CACHE_MODE_WT), 0); +} + +int set_memory_wt(unsigned long addr, int numpages) +{ + int ret; + + ret = reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE, + _PAGE_CACHE_MODE_WT, NULL); + if (ret) + goto out_err; + + ret = _set_memory_wt(addr, numpages); + if (ret) + goto out_free; + + return 0; + +out_free: + free_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE); +out_err: + return ret; +} +EXPORT_SYMBOL(set_memory_wt); + int _set_memory_wb(unsigned long addr, int numpages) { /* WB cache mode is hard wired to all cache attribute bits being 0 */ @@ -1683,6 +1718,7 @@ static int _set_pages_array(struct page **pages, int addrinarray, { unsigned long start; unsigned long end; + enum page_cache_mode set_type; int i; int free_idx; int ret; @@ -1696,8 +1732,12 @@ static int _set_pages_array(struct page **pages, int addrinarray, goto err_out; } + /* If WC, set to UC- first and then WC */ + set_type = (new_type == _PAGE_CACHE_MODE_WC) ? + _PAGE_CACHE_MODE_UC_MINUS : new_type; + ret = cpa_set_pages_array(pages, addrinarray, - cachemode2pgprot(_PAGE_CACHE_MODE_UC_MINUS)); + cachemode2pgprot(set_type)); if (!ret && new_type == _PAGE_CACHE_MODE_WC) ret = change_page_attr_set_clr(NULL, addrinarray, cachemode2pgprot( @@ -1731,6 +1771,12 @@ int set_pages_array_wc(struct page **pages, int addrinarray) } EXPORT_SYMBOL(set_pages_array_wc); +int set_pages_array_wt(struct page **pages, int addrinarray) +{ + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); +} +EXPORT_SYMBOL(set_pages_array_wt); + int set_pages_wb(struct page *page, int numpages) { unsigned long addr = (unsigned long)page_address(page); From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752851AbbBYHWg (ORCPT ); Wed, 25 Feb 2015 02:22:36 -0500 Received: from mail-wg0-f54.google.com ([74.125.82.54]:35310 "EHLO mail-wg0-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752775AbbBYHWe (ORCPT ); Wed, 25 Feb 2015 02:22:34 -0500 Date: Wed, 25 Feb 2015 08:22:28 +0100 From: Ingo Molnar To: Toshi Kani Cc: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com Subject: Re: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT Message-ID: <20150225072228.GA13061@gmail.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> <1424823301-30927-8-git-send-email-toshi.kani@hp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1424823301-30927-8-git-send-email-toshi.kani@hp.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Toshi Kani wrote: > +int set_pages_array_wt(struct page **pages, int addrinarray) > +{ > + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); > +} > +EXPORT_SYMBOL(set_pages_array_wt); So by default we make new APIs EXPORT_SYMBOL_GPL(): we don't want proprietary modules mucking around with new code PAT interfaces, we only want modules we can analyze and fix in detail. Thanks, Ingo From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753558AbbBYPUh (ORCPT ); Wed, 25 Feb 2015 10:20:37 -0500 Received: from g2t2354.austin.hp.com ([15.217.128.53]:42260 "EHLO g2t2354.austin.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753350AbbBYPUd (ORCPT ); Wed, 25 Feb 2015 10:20:33 -0500 Message-ID: <1424877601.17007.108.camel@misato.fc.hp.com> Subject: Re: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT From: Toshi Kani To: Ingo Molnar Cc: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com Date: Wed, 25 Feb 2015 08:20:01 -0700 In-Reply-To: <20150225072228.GA13061@gmail.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> <1424823301-30927-8-git-send-email-toshi.kani@hp.com> <20150225072228.GA13061@gmail.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4 (3.10.4-4.fc20) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2015-02-25 at 08:22 +0100, Ingo Molnar wrote: > * Toshi Kani wrote: > > > +int set_pages_array_wt(struct page **pages, int addrinarray) > > +{ > > + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); > > +} > > +EXPORT_SYMBOL(set_pages_array_wt); > > So by default we make new APIs EXPORT_SYMBOL_GPL(): we > don't want proprietary modules mucking around with new code > PAT interfaces, we only want modules we can analyze and fix > in detail. Right. I have one question for this case. This set_pages_array_wt() extends the set_pages_array_xx() family, which are all exported with EXPORT_SYMBOL() today. In this case, should we keep them exported in the consistent manner, or should we still use GPL when adding a new one? Thanks, -Toshi From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753688AbbBZLbB (ORCPT ); Thu, 26 Feb 2015 06:31:01 -0500 Received: from mail-wg0-f47.google.com ([74.125.82.47]:34305 "EHLO mail-wg0-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753471AbbBZLa7 (ORCPT ); Thu, 26 Feb 2015 06:30:59 -0500 Date: Thu, 26 Feb 2015 12:30:54 +0100 From: Ingo Molnar To: Toshi Kani Cc: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com Subject: Re: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT Message-ID: <20150226113054.GA4191@gmail.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> <1424823301-30927-8-git-send-email-toshi.kani@hp.com> <20150225072228.GA13061@gmail.com> <1424877601.17007.108.camel@misato.fc.hp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1424877601.17007.108.camel@misato.fc.hp.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Toshi Kani wrote: > On Wed, 2015-02-25 at 08:22 +0100, Ingo Molnar wrote: > > * Toshi Kani wrote: > > > > > +int set_pages_array_wt(struct page **pages, int addrinarray) > > > +{ > > > + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); > > > +} > > > +EXPORT_SYMBOL(set_pages_array_wt); > > > > So by default we make new APIs EXPORT_SYMBOL_GPL(): we > > don't want proprietary modules mucking around with new code > > PAT interfaces, we only want modules we can analyze and fix > > in detail. > > Right. I have one question for this case. This > set_pages_array_wt() extends the set_pages_array_xx() > family, which are all exported with EXPORT_SYMBOL() > today. In this case, should we keep them exported in the > consistent manner, or should we still use GPL when adding > a new one? Still keep it GPL, it's a new API that old modules obviously don't use. Thanks, Ingo From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932416AbbBZOpc (ORCPT ); Thu, 26 Feb 2015 09:45:32 -0500 Received: from g4t3427.houston.hp.com ([15.201.208.55]:43306 "EHLO g4t3427.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932234AbbBZOp1 (ORCPT ); Thu, 26 Feb 2015 09:45:27 -0500 Message-ID: <1424961893.17007.139.camel@misato.fc.hp.com> Subject: Re: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT From: Toshi Kani To: Ingo Molnar Cc: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com Date: Thu, 26 Feb 2015 07:44:53 -0700 In-Reply-To: <20150226113054.GA4191@gmail.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> <1424823301-30927-8-git-send-email-toshi.kani@hp.com> <20150225072228.GA13061@gmail.com> <1424877601.17007.108.camel@misato.fc.hp.com> <20150226113054.GA4191@gmail.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4 (3.10.4-4.fc20) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2015-02-26 at 12:30 +0100, Ingo Molnar wrote: > * Toshi Kani wrote: > > > On Wed, 2015-02-25 at 08:22 +0100, Ingo Molnar wrote: > > > * Toshi Kani wrote: > > > > > > > +int set_pages_array_wt(struct page **pages, int addrinarray) > > > > +{ > > > > + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); > > > > +} > > > > +EXPORT_SYMBOL(set_pages_array_wt); > > > > > > So by default we make new APIs EXPORT_SYMBOL_GPL(): we > > > don't want proprietary modules mucking around with new code > > > PAT interfaces, we only want modules we can analyze and fix > > > in detail. > > > > Right. I have one question for this case. This > > set_pages_array_wt() extends the set_pages_array_xx() > > family, which are all exported with EXPORT_SYMBOL() > > today. In this case, should we keep them exported in the > > consistent manner, or should we still use GPL when adding > > a new one? > > Still keep it GPL, it's a new API that old modules > obviously don't use. Got it. Thanks for the clarification. -Toshi From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932523AbbCDS1z (ORCPT ); Wed, 4 Mar 2015 13:27:55 -0500 Received: from g1t5425.austin.hp.com ([15.216.225.55]:43647 "EHLO g1t5425.austin.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758301AbbCDS1x (ORCPT ); Wed, 4 Mar 2015 13:27:53 -0500 X-Greylist: delayed 88971 seconds by postgrey-1.27 at vger.kernel.org; Wed, 04 Mar 2015 13:27:53 EST Message-ID: <1425493634.17007.248.camel@misato.fc.hp.com> Subject: Re: [PATCH v8 7/7] x86, mm: Add set_memory_wt() for WT From: Toshi Kani To: Ingo Molnar Cc: hpa@zytor.com, tglx@linutronix.de, mingo@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jgross@suse.com, stefan.bader@canonical.com, luto@amacapital.net, hmh@hmh.eng.br, yigal@plexistor.com, konrad.wilk@oracle.com, Elliott@hp.com Date: Wed, 04 Mar 2015 11:27:14 -0700 In-Reply-To: <1424961893.17007.139.camel@misato.fc.hp.com> References: <1424823301-30927-1-git-send-email-toshi.kani@hp.com> <1424823301-30927-8-git-send-email-toshi.kani@hp.com> <20150225072228.GA13061@gmail.com> <1424877601.17007.108.camel@misato.fc.hp.com> <20150226113054.GA4191@gmail.com> <1424961893.17007.139.camel@misato.fc.hp.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4 (3.10.4-4.fc20) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2015-02-26 at 07:44 -0700, Toshi Kani wrote: > On Thu, 2015-02-26 at 12:30 +0100, Ingo Molnar wrote: > > * Toshi Kani wrote: > > > > > On Wed, 2015-02-25 at 08:22 +0100, Ingo Molnar wrote: > > > > * Toshi Kani wrote: > > > > > > > > > +int set_pages_array_wt(struct page **pages, int addrinarray) > > > > > +{ > > > > > + return _set_pages_array(pages, addrinarray, _PAGE_CACHE_MODE_WT); > > > > > +} > > > > > +EXPORT_SYMBOL(set_pages_array_wt); > > > > > > > > So by default we make new APIs EXPORT_SYMBOL_GPL(): we > > > > don't want proprietary modules mucking around with new code > > > > PAT interfaces, we only want modules we can analyze and fix > > > > in detail. > > > > > > Right. I have one question for this case. This > > > set_pages_array_wt() extends the set_pages_array_xx() > > > family, which are all exported with EXPORT_SYMBOL() > > > today. In this case, should we keep them exported in the > > > consistent manner, or should we still use GPL when adding > > > a new one? > > > > Still keep it GPL, it's a new API that old modules > > obviously don't use. > > Got it. Thanks for the clarification. Since this is a minor change and there is no other comment at this point, I've submitted the updated patch 7/7 alone with the following subject for saving your mail box. :-) [PATCH v8-UPDATE 7/7] x86, mm: Add set_memory_wt() for WT Thanks, -Toshi