From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Message-ID: <1434749332.11808.113.camel@misato.fc.hp.com> Subject: Re: [PATCH v4 5/6] arch: introduce memremap_cache() and memremap_wt() From: Toshi Kani Date: Fri, 19 Jun 2015 15:28:52 -0600 In-Reply-To: <20150611211941.10271.10513.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211941.10271.10513.stgit@dwillia2-desk3.amr.corp.intel.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: Dan Williams Cc: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org, jgross@suse.com, x86@kernel.org, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, Andy Shevchenko , geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de List-ID: On Thu, 2015-06-11 at 17:19 -0400, Dan Williams wrote: > Existing users of ioremap_cache() are mapping memory that is known in > advance to not have i/o side effects. These users are forced to cast > away the __iomem annotation, or otherwise neglect to fix the sparse > errors thrown when dereferencing pointers to this memory. Provide > memremap_*() as a non __iomem annotated ioremap_*(). > > The ARCH_HAS_MEMREMAP kconfig symbol is introduced for archs to assert > that it is safe to recast / reuse the return value from ioremap as a > normal pointer to memory. In other words, archs that mandate specific > accessors for __iomem are not memremap() capable and drivers that care, > like pmem, can add a dependency to disable themselves on these archs. : > +#ifdef CONFIG_ARCH_HAS_MEMREMAP > +/* > + * memremap() is "ioremap" for cases where it is known that the resource > + * being mapped does not have i/o side effects and the __iomem > + * annotation is not applicable. > + */ > +static bool memremap_valid(resource_size_t offset, size_t size) > +{ > + if (region_is_ram(offset, size) != 0) { I noticed that region_is_ram() is buggy and always returns -1. I will submit the fix shortly. Thanks, -Toshi > + WARN_ONCE(1, "memremap attempted on ram %pa size: %zu\n", > + &offset, size); > + return false; > + } > + return true; > +} -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Subject: [PATCH v4 5/6] arch: introduce memremap_cache() and memremap_wt() From: Dan Williams Date: Thu, 11 Jun 2015 17:19:41 -0400 Message-ID: <20150611211941.10271.10513.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, Andy Shevchenko , geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de List-ID: Existing users of ioremap_cache() are mapping memory that is known in advance to not have i/o side effects. These users are forced to cast away the __iomem annotation, or otherwise neglect to fix the sparse errors thrown when dereferencing pointers to this memory. Provide memremap_*() as a non __iomem annotated ioremap_*(). The ARCH_HAS_MEMREMAP kconfig symbol is introduced for archs to assert that it is safe to recast / reuse the return value from ioremap as a normal pointer to memory. In other words, archs that mandate specific accessors for __iomem are not memremap() capable and drivers that care, like pmem, can add a dependency to disable themselves on these archs. Cc: Arnd Bergmann Acked-by: Andy Shevchenko Signed-off-by: Dan Williams --- arch/arm/Kconfig | 1 + arch/arm64/Kconfig | 1 + arch/arm64/kernel/efi.c | 4 ++- arch/arm64/kernel/smp_spin_table.c | 10 ++++---- arch/frv/Kconfig | 1 + arch/m68k/Kconfig | 1 + arch/metag/Kconfig | 1 + arch/mips/Kconfig | 1 + arch/powerpc/Kconfig | 1 + arch/x86/Kconfig | 1 + arch/x86/kernel/crash_dump_64.c | 6 ++--- arch/x86/kernel/kdebugfs.c | 8 +++---- arch/x86/kernel/ksysfs.c | 28 ++++++++++++----------- arch/x86/mm/ioremap.c | 10 +++----- arch/xtensa/Kconfig | 1 + drivers/acpi/apei/einj.c | 8 +++---- drivers/acpi/apei/erst.c | 4 ++- drivers/block/Kconfig | 1 + drivers/block/pmem.c | 7 +++--- drivers/firmware/google/memconsole.c | 4 ++- include/linux/device.h | 5 ++++ include/linux/io.h | 4 +++ kernel/resource.c | 41 +++++++++++++++++++++++++++++++++- lib/Kconfig | 5 +++- 24 files changed, 107 insertions(+), 47 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 45df48ba0b12..397426f8ca37 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -3,6 +3,7 @@ config ARM default y select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE select ARCH_HAS_ELF_RANDOMIZE + select ARCH_HAS_MEMREMAP select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST select ARCH_HAVE_CUSTOM_GPIO_H select ARCH_HAS_GCOV_PROFILE_ALL diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7796af4b1d6f..f07a9a5af61e 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -5,6 +5,7 @@ config ARM64 select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE select ARCH_HAS_ELF_RANDOMIZE select ARCH_HAS_GCOV_PROFILE_ALL + select ARCH_HAS_MEMREMAP select ARCH_HAS_SG_CHAIN select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST select ARCH_USE_CMPXCHG_LOCKREF diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c index ab21e0d58278..b672ef33f08b 100644 --- a/arch/arm64/kernel/efi.c +++ b/arch/arm64/kernel/efi.c @@ -289,7 +289,7 @@ static int __init arm64_enable_runtime_services(void) pr_info("Remapping and enabling EFI services.\n"); mapsize = memmap.map_end - memmap.map; - memmap.map = (__force void *)ioremap_cache((phys_addr_t)memmap.phys_map, + memmap.map = memremap_cache((phys_addr_t)memmap.phys_map, mapsize); if (!memmap.map) { pr_err("Failed to remap EFI memory map\n"); @@ -298,7 +298,7 @@ static int __init arm64_enable_runtime_services(void) memmap.map_end = memmap.map + mapsize; efi.memmap = &memmap; - efi.systab = (__force void *)ioremap_cache(efi_system_table, + efi.systab = memremap_cache(efi_system_table, sizeof(efi_system_table_t)); if (!efi.systab) { pr_err("Failed to remap EFI System Table\n"); diff --git a/arch/arm64/kernel/smp_spin_table.c b/arch/arm64/kernel/smp_spin_table.c index 14944e5b28da..893c8586e20f 100644 --- a/arch/arm64/kernel/smp_spin_table.c +++ b/arch/arm64/kernel/smp_spin_table.c @@ -67,18 +67,18 @@ static int smp_spin_table_cpu_init(struct device_node *dn, unsigned int cpu) static int smp_spin_table_cpu_prepare(unsigned int cpu) { - __le64 __iomem *release_addr; + __le64 *release_addr; if (!cpu_release_addr[cpu]) return -ENODEV; /* * The cpu-release-addr may or may not be inside the linear mapping. - * As ioremap_cache will either give us a new mapping or reuse the + * As memremap_cache will either give us a new mapping or reuse the * existing linear mapping, we can use it to cover both cases. In * either case the memory will be MT_NORMAL. */ - release_addr = ioremap_cache(cpu_release_addr[cpu], + release_addr = memremap_cache(cpu_release_addr[cpu], sizeof(*release_addr)); if (!release_addr) return -ENOMEM; @@ -91,7 +91,7 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) * the boot protocol. */ writeq_relaxed(__pa(secondary_holding_pen), release_addr); - __flush_dcache_area((__force void *)release_addr, + __flush_dcache_area(release_addr, sizeof(*release_addr)); /* @@ -99,7 +99,7 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) */ sev(); - iounmap(release_addr); + memunmap(release_addr); return 0; } diff --git a/arch/frv/Kconfig b/arch/frv/Kconfig index 34aa19352dc1..2373bf183527 100644 --- a/arch/frv/Kconfig +++ b/arch/frv/Kconfig @@ -14,6 +14,7 @@ config FRV select OLD_SIGSUSPEND3 select OLD_SIGACTION select HAVE_DEBUG_STACKOVERFLOW + select ARCH_HAS_MEMREMAP config ZONE_DMA bool diff --git a/arch/m68k/Kconfig b/arch/m68k/Kconfig index 2dd8f63bfbbb..831b1be8c43d 100644 --- a/arch/m68k/Kconfig +++ b/arch/m68k/Kconfig @@ -23,6 +23,7 @@ config M68K select MODULES_USE_ELF_RELA select OLD_SIGSUSPEND3 select OLD_SIGACTION + select ARCH_HAS_MEMREMAP config RWSEM_GENERIC_SPINLOCK bool diff --git a/arch/metag/Kconfig b/arch/metag/Kconfig index 0b389a81c43a..5669fe3eb807 100644 --- a/arch/metag/Kconfig +++ b/arch/metag/Kconfig @@ -24,6 +24,7 @@ config METAG select HAVE_PERF_EVENTS select HAVE_SYSCALL_TRACEPOINTS select HAVE_UNDERSCORE_SYMBOL_PREFIX + select ARCH_HAS_MEMREMAP select IRQ_DOMAIN select MODULES_USE_ELF_RELA select OF diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig index f5016656494f..9ee35e615c0d 100644 --- a/arch/mips/Kconfig +++ b/arch/mips/Kconfig @@ -58,6 +58,7 @@ config MIPS select SYSCTL_EXCEPTION_TRACE select HAVE_VIRT_CPU_ACCOUNTING_GEN select HAVE_IRQ_TIME_ACCOUNTING + select ARCH_HAS_MEMREMAP menu "Machine selection" diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 190cc48abc0c..73c1f8b1f022 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -153,6 +153,7 @@ config PPC select NO_BOOTMEM select HAVE_GENERIC_RCU_GUP select HAVE_PERF_EVENTS_NMI if PPC64 + select ARCH_HAS_MEMREMAP config GENERIC_CSUM def_bool CPU_LITTLE_ENDIAN diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 228aa35d7e89..f16caf7eac27 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -27,6 +27,7 @@ config X86 select ARCH_HAS_ELF_RANDOMIZE select ARCH_HAS_FAST_MULTIPLIER select ARCH_HAS_GCOV_PROFILE_ALL + select ARCH_HAS_MEMREMAP select ARCH_HAS_SG_CHAIN select ARCH_HAVE_NMI_SAFE_CMPXCHG select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI diff --git a/arch/x86/kernel/crash_dump_64.c b/arch/x86/kernel/crash_dump_64.c index afa64adb75ee..8e04011665fd 100644 --- a/arch/x86/kernel/crash_dump_64.c +++ b/arch/x86/kernel/crash_dump_64.c @@ -31,19 +31,19 @@ ssize_t copy_oldmem_page(unsigned long pfn, char *buf, if (!csize) return 0; - vaddr = ioremap_cache(pfn << PAGE_SHIFT, PAGE_SIZE); + vaddr = memremap_cache(pfn << PAGE_SHIFT, PAGE_SIZE); if (!vaddr) return -ENOMEM; if (userbuf) { if (copy_to_user(buf, vaddr + offset, csize)) { - iounmap(vaddr); + memunmap(vaddr); return -EFAULT; } } else memcpy(buf, vaddr + offset, csize); set_iounmap_nonlazy(); - iounmap(vaddr); + memunmap(vaddr); return csize; } diff --git a/arch/x86/kernel/kdebugfs.c b/arch/x86/kernel/kdebugfs.c index dc1404bf8e4b..731b10e2814f 100644 --- a/arch/x86/kernel/kdebugfs.c +++ b/arch/x86/kernel/kdebugfs.c @@ -49,7 +49,7 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf, pa = node->paddr + sizeof(struct setup_data) + pos; pg = pfn_to_page((pa + count - 1) >> PAGE_SHIFT); if (PageHighMem(pg)) { - p = ioremap_cache(pa, count); + p = memremap_cache(pa, count); if (!p) return -ENXIO; } else @@ -58,7 +58,7 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf, remain = copy_to_user(user_buf, p, count); if (PageHighMem(pg)) - iounmap(p); + memunmap(p); if (remain) return -EFAULT; @@ -128,7 +128,7 @@ static int __init create_setup_data_nodes(struct dentry *parent) pg = pfn_to_page((pa_data+sizeof(*data)-1) >> PAGE_SHIFT); if (PageHighMem(pg)) { - data = ioremap_cache(pa_data, sizeof(*data)); + data = memremap_cache(pa_data, sizeof(*data)); if (!data) { kfree(node); error = -ENXIO; @@ -144,7 +144,7 @@ static int __init create_setup_data_nodes(struct dentry *parent) pa_data = data->next; if (PageHighMem(pg)) - iounmap(data); + memunmap(data); if (error) goto err_dir; no++; diff --git a/arch/x86/kernel/ksysfs.c b/arch/x86/kernel/ksysfs.c index c2bedaea11f7..2fbc62886eae 100644 --- a/arch/x86/kernel/ksysfs.c +++ b/arch/x86/kernel/ksysfs.c @@ -16,8 +16,8 @@ #include #include #include +#include -#include #include static ssize_t version_show(struct kobject *kobj, @@ -79,12 +79,12 @@ static int get_setup_data_paddr(int nr, u64 *paddr) *paddr = pa_data; return 0; } - data = ioremap_cache(pa_data, sizeof(*data)); + data = memremap_cache(pa_data, sizeof(*data)); if (!data) return -ENOMEM; pa_data = data->next; - iounmap(data); + memunmap(data); i++; } return -EINVAL; @@ -97,17 +97,17 @@ static int __init get_setup_data_size(int nr, size_t *size) u64 pa_data = boot_params.hdr.setup_data; while (pa_data) { - data = ioremap_cache(pa_data, sizeof(*data)); + data = memremap_cache(pa_data, sizeof(*data)); if (!data) return -ENOMEM; if (nr == i) { *size = data->len; - iounmap(data); + memunmap(data); return 0; } pa_data = data->next; - iounmap(data); + memunmap(data); i++; } return -EINVAL; @@ -127,12 +127,12 @@ static ssize_t type_show(struct kobject *kobj, ret = get_setup_data_paddr(nr, &paddr); if (ret) return ret; - data = ioremap_cache(paddr, sizeof(*data)); + data = memremap_cache(paddr, sizeof(*data)); if (!data) return -ENOMEM; ret = sprintf(buf, "0x%x\n", data->type); - iounmap(data); + memunmap(data); return ret; } @@ -154,7 +154,7 @@ static ssize_t setup_data_data_read(struct file *fp, ret = get_setup_data_paddr(nr, &paddr); if (ret) return ret; - data = ioremap_cache(paddr, sizeof(*data)); + data = memremap_cache(paddr, sizeof(*data)); if (!data) return -ENOMEM; @@ -170,15 +170,15 @@ static ssize_t setup_data_data_read(struct file *fp, goto out; ret = count; - p = ioremap_cache(paddr + sizeof(*data), data->len); + p = memremap_cache(paddr + sizeof(*data), data->len); if (!p) { ret = -ENOMEM; goto out; } memcpy(buf, p + off, count); - iounmap(p); + memunmap(p); out: - iounmap(data); + memunmap(data); return ret; } @@ -250,13 +250,13 @@ static int __init get_setup_data_total_num(u64 pa_data, int *nr) *nr = 0; while (pa_data) { *nr += 1; - data = ioremap_cache(pa_data, sizeof(*data)); + data = memremap_cache(pa_data, sizeof(*data)); if (!data) { ret = -ENOMEM; goto out; } pa_data = data->next; - iounmap(data); + memunmap(data); } out: diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index cc5ccc415cc0..7f087cb793fa 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -414,12 +414,10 @@ void *xlate_dev_mem_ptr(phys_addr_t phys) if (page_is_ram(start >> PAGE_SHIFT)) return __va(phys); - vaddr = ioremap_cache(start, PAGE_SIZE); - /* Only add the offset on success and return NULL if the ioremap() failed: */ + vaddr = memremap_cache(start, PAGE_SIZE); if (vaddr) - vaddr += offset; - - return vaddr; + return vaddr + offset; + return NULL; } void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr) @@ -427,7 +425,7 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr) if (page_is_ram(phys >> PAGE_SHIFT)) return; - iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK)); + memunmap((void *)((unsigned long)addr & PAGE_MASK)); } static pte_t bm_pte[PAGE_SIZE/sizeof(pte_t)] __page_aligned_bss; diff --git a/arch/xtensa/Kconfig b/arch/xtensa/Kconfig index 87be10e8b57a..e601faf87cee 100644 --- a/arch/xtensa/Kconfig +++ b/arch/xtensa/Kconfig @@ -3,6 +3,7 @@ config ZONE_DMA config XTENSA def_bool y + select ARCH_HAS_MEMREMAP select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_IPC_PARSE_VERSION select ARCH_WANT_OPTIONAL_GPIOLIB diff --git a/drivers/acpi/apei/einj.c b/drivers/acpi/apei/einj.c index a095d4f858da..2ec9006cfb6c 100644 --- a/drivers/acpi/apei/einj.c +++ b/drivers/acpi/apei/einj.c @@ -318,7 +318,7 @@ static int __einj_error_trigger(u64 trigger_paddr, u32 type, sizeof(*trigger_tab) - 1); goto out; } - trigger_tab = ioremap_cache(trigger_paddr, sizeof(*trigger_tab)); + trigger_tab = memremap_cache(trigger_paddr, sizeof(*trigger_tab)); if (!trigger_tab) { pr_err(EINJ_PFX "Failed to map trigger table!\n"); goto out_rel_header; @@ -346,8 +346,8 @@ static int __einj_error_trigger(u64 trigger_paddr, u32 type, (unsigned long long)trigger_paddr + table_size - 1); goto out_rel_header; } - iounmap(trigger_tab); - trigger_tab = ioremap_cache(trigger_paddr, table_size); + memunmap(trigger_tab); + trigger_tab = memremap_cache(trigger_paddr, table_size); if (!trigger_tab) { pr_err(EINJ_PFX "Failed to map trigger table!\n"); goto out_rel_entry; @@ -409,7 +409,7 @@ out_rel_header: release_mem_region(trigger_paddr, sizeof(*trigger_tab)); out: if (trigger_tab) - iounmap(trigger_tab); + memunmap(trigger_tab); return rc; } diff --git a/drivers/acpi/apei/erst.c b/drivers/acpi/apei/erst.c index 3670bbab57a3..6b95066da51d 100644 --- a/drivers/acpi/apei/erst.c +++ b/drivers/acpi/apei/erst.c @@ -77,7 +77,7 @@ static struct acpi_table_erst *erst_tab; static struct erst_erange { u64 base; u64 size; - void __iomem *vaddr; + void *vaddr; u32 attr; } erst_erange; @@ -1185,7 +1185,7 @@ static int __init erst_init(void) goto err_unmap_reg; } rc = -ENOMEM; - erst_erange.vaddr = ioremap_cache(erst_erange.base, + erst_erange.vaddr = memremap_cache(erst_erange.base, erst_erange.size); if (!erst_erange.vaddr) goto err_release_erange; diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig index eb1fed5bd516..98418fc330ae 100644 --- a/drivers/block/Kconfig +++ b/drivers/block/Kconfig @@ -406,6 +406,7 @@ config BLK_DEV_RAM_DAX config BLK_DEV_PMEM tristate "Persistent memory block device support" + depends on ARCH_HAS_MEMREMAP help Saying Y here will allow you to use a contiguous range of reserved memory as one or more persistent block devices. diff --git a/drivers/block/pmem.c b/drivers/block/pmem.c index 095dfaadcaa5..b00b97314b57 100644 --- a/drivers/block/pmem.c +++ b/drivers/block/pmem.c @@ -23,6 +23,7 @@ #include #include #include +#include #define PMEM_MINORS 16 @@ -143,7 +144,7 @@ static struct pmem_device *pmem_alloc(struct device *dev, struct resource *res) * of the CPU caches in case of a crash. */ err = -ENOMEM; - pmem->virt_addr = ioremap_wt(pmem->phys_addr, pmem->size); + pmem->virt_addr = memremap_wt(pmem->phys_addr, pmem->size); if (!pmem->virt_addr) goto out_release_region; @@ -179,7 +180,7 @@ static struct pmem_device *pmem_alloc(struct device *dev, struct resource *res) out_free_queue: blk_cleanup_queue(pmem->pmem_queue); out_unmap: - iounmap(pmem->virt_addr); + memunmap(pmem->virt_addr); out_release_region: release_mem_region(pmem->phys_addr, pmem->size); out_free_dev: @@ -193,7 +194,7 @@ static void pmem_free(struct pmem_device *pmem) del_gendisk(pmem->pmem_disk); put_disk(pmem->pmem_disk); blk_cleanup_queue(pmem->pmem_queue); - iounmap(pmem->virt_addr); + memunmap(pmem->virt_addr); release_mem_region(pmem->phys_addr, pmem->size); kfree(pmem); } diff --git a/drivers/firmware/google/memconsole.c b/drivers/firmware/google/memconsole.c index 2f569aaed4c7..877433dc8297 100644 --- a/drivers/firmware/google/memconsole.c +++ b/drivers/firmware/google/memconsole.c @@ -52,14 +52,14 @@ static ssize_t memconsole_read(struct file *filp, struct kobject *kobp, char *memconsole; ssize_t ret; - memconsole = ioremap_cache(memconsole_baseaddr, memconsole_length); + memconsole = memremap_cache(memconsole_baseaddr, memconsole_length); if (!memconsole) { pr_err("memconsole: ioremap_cache failed\n"); return -ENOMEM; } ret = memory_read_from_buffer(buf, count, &pos, memconsole, memconsole_length); - iounmap(memconsole); + memunmap(memconsole); return ret; } diff --git a/include/linux/device.h b/include/linux/device.h index 6558af90c8fe..518f49c5d596 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -638,6 +638,11 @@ extern void devm_free_pages(struct device *dev, unsigned long addr); void __iomem *devm_ioremap_resource(struct device *dev, struct resource *res); +static inline void *devm_memremap_resource(struct device *dev, struct resource *res) +{ + return (void __force *) devm_ioremap_resource(dev, res); +} + /* allows to add/remove a custom action to devres stack */ int devm_add_action(struct device *dev, void (*action)(void *), void *data); void devm_remove_action(struct device *dev, void (*action)(void *), void *data); diff --git a/include/linux/io.h b/include/linux/io.h index 4cd8996cfea0..23ef87d311d5 100644 --- a/include/linux/io.h +++ b/include/linux/io.h @@ -123,4 +123,8 @@ static inline int arch_phys_wc_index(int handle) #endif #endif +extern void *memremap_cache(resource_size_t offset, size_t size); +extern void *memremap_wt(resource_size_t offset, size_t size); +extern void memunmap(void *addr); + #endif /* _LINUX_IO_H */ diff --git a/kernel/resource.c b/kernel/resource.c index 90552aab5f2d..2f8aca09da52 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -23,7 +23,7 @@ #include #include #include -#include +#include struct resource ioport_resource = { @@ -528,6 +528,45 @@ int region_is_ram(resource_size_t start, unsigned long size) return ret; } +#ifdef CONFIG_ARCH_HAS_MEMREMAP +/* + * memremap() is "ioremap" for cases where it is known that the resource + * being mapped does not have i/o side effects and the __iomem + * annotation is not applicable. + */ +static bool memremap_valid(resource_size_t offset, size_t size) +{ + if (region_is_ram(offset, size) != 0) { + WARN_ONCE(1, "memremap attempted on ram %pa size: %zu\n", + &offset, size); + return false; + } + return true; +} + +void *memremap_cache(resource_size_t offset, size_t size) +{ + if (!memremap_valid(offset, size)) + return NULL; + return (void __force *) ioremap_cache(offset, size); +} +EXPORT_SYMBOL(memremap_cache); + +void *memremap_wt(resource_size_t offset, size_t size) +{ + if (!memremap_valid(offset, size)) + return NULL; + return (void __force *) ioremap_wt(offset, size); +} +EXPORT_SYMBOL(memremap_wt); + +void memunmap(void *addr) +{ + iounmap((void __iomem __force *) addr); +} +EXPORT_SYMBOL(memunmap); +#endif /* CONFIG_ARCH_HAS_MEMREMAP */ + void __weak arch_remove_reservations(struct resource *avail) { } diff --git a/lib/Kconfig b/lib/Kconfig index 601965a948e8..bc7bc0278921 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -520,6 +520,9 @@ source "lib/fonts/Kconfig" # config ARCH_HAS_SG_CHAIN - def_bool n + bool + +config ARCH_HAS_MEMREMAP + bool endmenu -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Subject: [-tip PATCH v4 0/6] pmem api, generic ioremap_cache, and memremap From: Dan Williams Date: Thu, 11 Jun 2015 17:19:12 -0400 Message-ID: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, Andy Shevchenko , geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, kbuild test robot , hch@lst.de List-ID: The pmem api is responsible for shepherding data out to persistent media. The pmem driver uses this api, when available, to assert that data is durable by the time bio_endio() is invoked. When an architecture or cpu can not make persistence guarantees the driver warns and falls back to "best effort" implementation. Changes since v3 [1]: Rebased on tip/master now that Toshi's ioremap_wt() patches have landed in -tip. The primary change was reflowing the patches against the newly alphabetized 'select' options under config X86. [1]: https://lists.01.org/pipermail/linux-nvdimm/2015-June/001081.html --- Dan Williams (5): arch: unify ioremap prototypes and macro aliases cleanup IORESOURCE_CACHEABLE vs ioremap() arch/*/asm/io.h: add ioremap_cache() to all architectures devm: fix ioremap_cache() usage arch: introduce memremap_cache() and memremap_wt() Ross Zwisler (1): arch, x86: pmem api for ensuring durability of persistent memory updates arch/arc/include/asm/io.h | 1 arch/arm/Kconfig | 1 arch/arm/include/asm/io.h | 2 + arch/arm/mach-clps711x/board-cdb89712.c | 2 - arch/arm64/Kconfig | 1 arch/arm64/include/asm/io.h | 3 + arch/arm64/kernel/efi.c | 4 + arch/arm64/kernel/smp_spin_table.c | 10 ++- arch/avr32/include/asm/io.h | 1 arch/cris/include/asm/io.h | 8 +- arch/cris/mm/ioremap.c | 6 +- arch/frv/Kconfig | 1 arch/frv/include/asm/io.h | 6 ++ arch/ia64/include/asm/io.h | 9 +-- arch/ia64/mm/ioremap.c | 4 + arch/m32r/include/asm/io.h | 1 arch/m68k/Kconfig | 1 arch/m68k/include/asm/io_mm.h | 7 ++ arch/m68k/include/asm/io_no.h | 5 ++ arch/metag/Kconfig | 1 arch/metag/include/asm/io.h | 5 ++ arch/microblaze/include/asm/io.h | 1 arch/mips/Kconfig | 1 arch/mips/include/asm/io.h | 17 ++++- arch/mn10300/include/asm/io.h | 1 arch/nios2/include/asm/io.h | 1 arch/powerpc/Kconfig | 1 arch/powerpc/include/asm/io.h | 2 - arch/powerpc/kernel/pci_of_scan.c | 2 - arch/s390/include/asm/io.h | 1 arch/sparc/include/asm/io_32.h | 1 arch/sparc/include/asm/io_64.h | 9 ++- arch/sparc/kernel/pci.c | 3 - arch/tile/include/asm/io.h | 1 arch/x86/Kconfig | 2 + arch/x86/include/asm/cacheflush.h | 36 +++++++++++ arch/x86/include/asm/io.h | 7 ++ arch/x86/kernel/crash_dump_64.c | 6 +- arch/x86/kernel/kdebugfs.c | 8 +- arch/x86/kernel/ksysfs.c | 28 ++++----- arch/x86/mm/ioremap.c | 10 +-- arch/xtensa/Kconfig | 1 arch/xtensa/include/asm/io.h | 3 + drivers/acpi/apei/einj.c | 8 +- drivers/acpi/apei/erst.c | 4 + drivers/block/Kconfig | 1 drivers/block/pmem.c | 76 +++++++++++++++++++++-- drivers/firmware/google/memconsole.c | 4 + drivers/pci/probe.c | 3 - drivers/pnp/manager.c | 2 - drivers/scsi/aic94xx/aic94xx_init.c | 7 -- drivers/scsi/arcmsr/arcmsr_hba.c | 5 -- drivers/scsi/mvsas/mv_init.c | 15 +---- drivers/video/fbdev/ocfb.c | 1 include/asm-generic/io.h | 8 ++ include/asm-generic/iomap.h | 4 + include/linux/compiler.h | 2 + include/linux/device.h | 5 ++ include/linux/io.h | 6 ++ include/linux/pmem.h | 102 +++++++++++++++++++++++++++++++ kernel/resource.c | 41 ++++++++++++ lib/Kconfig | 8 ++ lib/devres.c | 48 ++++++--------- lib/pci_iomap.c | 7 +- 64 files changed, 440 insertions(+), 138 deletions(-) create mode 100644 include/linux/pmem.h -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: MIME-Version: 1.0 In-Reply-To: References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> <20150617113121.GC9246@lst.de> From: Andy Lutomirski Date: Wed, 17 Jun 2015 08:08:49 -0700 Message-ID: Subject: Re: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates Content-Type: text/plain; charset=UTF-8 Sender: owner-linux-mm@kvack.org To: Dan Williams Cc: Christoph Hellwig , Arnd Bergmann , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Thomas Gleixner , Ross Zwisler , Andrew Morton , Juergen Gross , X86 ML , "Kani, Toshimitsu" , "linux-nvdimm@lists.01.org" , Benjamin Herrenschmidt , Luis Rodriguez , Konrad Rzeszutek Wilk , "linux-kernel@vger.kernel.org" , Stefan Bader , "linux-mm@kvack.org" , Geert Uytterhoeven , Ralf Baechle , Henrique de Moraes Holschuh , Michael Ellerman , Tejun Heo , Paul Mackerras List-ID: On Wed, Jun 17, 2015 at 7:54 AM, Dan Williams wrote: > On Wed, Jun 17, 2015 at 4:31 AM, Christoph Hellwig wrote: >> This mess with arch_ methods and an ops vecor is almost unreadable. >> >> What's the problem with having something like: >> >> pmem_foo() >> { >> if (arch_has_pmem) // or sync_pmem >> arch_pmem_foo(); >> generic_pmem_foo(); >> } >> >> This adds a branch at runtime, but that shoudn't really be any slower >> than an indirect call on architectures that matter. > > No doubt it's premature optimization, but it bothered me that we'll > end up calling cpuid perhaps multiple times every i/o. If it's just a > readability concern I could wrap it in helpers. Getting it upstream > is my primary concern at this point so I have no strong attachment to > the indirect calls if that's all that is preventing an ack. A cpuid per i/o would be a killer, but the cpufeature code is way smarter than that. You want static_cpu_has, though -- it's even faster, since it gets patched at boot time. --Andy -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: MIME-Version: 1.0 In-Reply-To: <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> From: Andy Lutomirski Date: Wed, 17 Jun 2015 08:07:38 -0700 Message-ID: Subject: Re: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates Content-Type: text/plain; charset=UTF-8 Sender: owner-linux-mm@kvack.org To: Dan Williams Cc: Arnd Bergmann , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Thomas Gleixner , Ross Zwisler , Andrew Morton , Juergen Gross , X86 ML , Toshi Kani , linux-nvdimm , Benjamin Herrenschmidt , Luis Rodriguez , Konrad Rzeszutek Wilk , "linux-kernel@vger.kernel.org" , Stefan Bader , "linux-mm@kvack.org" , Geert Uytterhoeven , Ralf Baechle , Henrique de Moraes Holschuh , Michael Ellerman , Tejun Heo , Paul Mackerras , Christoph Hellwig List-ID: On Thu, Jun 11, 2015 at 2:19 PM, Dan Williams wrote: > From: Ross Zwisler > > Based on an original patch by Ross Zwisler [1]. > > Writes to persistent memory have the potential to be posted to cpu > cache, cpu write buffers, and platform write buffers (memory controller) > before being committed to persistent media. Provide apis, > memcpy_to_pmem(), sync_pmem(), and memremap_pmem(), to write data to > pmem and assert that it is durable in PMEM (a persistent linear address > range). A '__pmem' attribute is added so sparse can track proper usage > of pointers to pmem. > > [1]: https://lists.01.org/pipermail/linux-nvdimm/2015-May/000932.html > > Cc: Thomas Gleixner > Cc: Ingo Molnar > Cc: "H. Peter Anvin" > Signed-off-by: Ross Zwisler > [djbw: various reworks] > Signed-off-by: Dan Williams > --- > arch/x86/Kconfig | 1 > arch/x86/include/asm/cacheflush.h | 36 +++++++++++++ > arch/x86/include/asm/io.h | 6 ++ > drivers/block/pmem.c | 75 +++++++++++++++++++++++++-- > include/linux/compiler.h | 2 + > include/linux/pmem.h | 102 +++++++++++++++++++++++++++++++++++++ > lib/Kconfig | 3 + > 7 files changed, 218 insertions(+), 7 deletions(-) > create mode 100644 include/linux/pmem.h > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig > index f16caf7eac27..5dfb8f31ac48 100644 > --- a/arch/x86/Kconfig > +++ b/arch/x86/Kconfig > @@ -28,6 +28,7 @@ config X86 > select ARCH_HAS_FAST_MULTIPLIER > select ARCH_HAS_GCOV_PROFILE_ALL > select ARCH_HAS_MEMREMAP > + select ARCH_HAS_PMEM_API > select ARCH_HAS_SG_CHAIN > select ARCH_HAVE_NMI_SAFE_CMPXCHG > select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI > diff --git a/arch/x86/include/asm/cacheflush.h b/arch/x86/include/asm/cacheflush.h > index b6f7457d12e4..4d896487382c 100644 > --- a/arch/x86/include/asm/cacheflush.h > +++ b/arch/x86/include/asm/cacheflush.h > @@ -4,6 +4,7 @@ > /* Caches aren't brain-dead on the intel. */ > #include > #include > +#include > > /* > * The set_memory_* API can be used to change various attributes of a virtual > @@ -108,4 +109,39 @@ static inline int rodata_test(void) > } > #endif > > +#ifdef ARCH_HAS_NOCACHE_UACCESS > +static inline void arch_memcpy_to_pmem(void __pmem *dst, const void *src, size_t n) > +{ > + /* > + * We are copying between two kernel buffers, if > + * __copy_from_user_inatomic_nocache() returns an error (page > + * fault) we would have already taken an unhandled fault before > + * the BUG_ON. The BUG_ON is simply here to satisfy > + * __must_check and allow reuse of the common non-temporal store > + * implementation for memcpy_to_pmem(). > + */ > + BUG_ON(__copy_from_user_inatomic_nocache((void __force *) dst, > + (void __user *) src, n)); Ick. If we take a fault, we will lose the debugging information we would otherwise have gotten unless we get lucky and get a usable CR2 value in the oops. > +} > + > +static inline void arch_sync_pmem(void) > +{ > + wmb(); > + pcommit_sfence(); > +} This function is non-intuitive to me. It's really "arch-specific sync pmem after one or more copies using arch_memcpy_to_pmem". If normal stores or memcpy to non-WC memory is used instead, then it's insufficient if the memory is WB and it's unnecessarily slow if the memory is WT or UC (the first sfence isn't needed). I would change the name and add documentation. I'd also add a comment about the wmb() being an SFENCE to flush pending non-temporal writes. --Andy -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: MIME-Version: 1.0 In-Reply-To: <20150617113121.GC9246@lst.de> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> <20150617113121.GC9246@lst.de> Date: Wed, 17 Jun 2015 07:54:14 -0700 Message-ID: Subject: Re: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates From: Dan Williams Content-Type: text/plain; charset=UTF-8 Sender: owner-linux-mm@kvack.org To: Christoph Hellwig Cc: Arnd Bergmann , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Thomas Gleixner , Ross Zwisler , Andrew Morton , Juergen Gross , X86 ML , "Kani, Toshimitsu" , "linux-nvdimm@lists.01.org" , Benjamin Herrenschmidt , Luis Rodriguez , Konrad Rzeszutek Wilk , "linux-kernel@vger.kernel.org" , Stefan Bader , Andy Lutomirski , linux-mm@kvack.org, Geert Uytterhoeven , Ralf Baechle , Henrique de Moraes Holschuh , mpe@ellerman.id.au, Tejun Heo , Paul Mackerras List-ID: On Wed, Jun 17, 2015 at 4:31 AM, Christoph Hellwig wrote: > This mess with arch_ methods and an ops vecor is almost unreadable. > > What's the problem with having something like: > > pmem_foo() > { > if (arch_has_pmem) // or sync_pmem > arch_pmem_foo(); > generic_pmem_foo(); > } > > This adds a branch at runtime, but that shoudn't really be any slower > than an indirect call on architectures that matter. No doubt it's premature optimization, but it bothered me that we'll end up calling cpuid perhaps multiple times every i/o. If it's just a readability concern I could wrap it in helpers. Getting it upstream is my primary concern at this point so I have no strong attachment to the indirect calls if that's all that is preventing an ack. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Wed, 17 Jun 2015 13:31:21 +0200 From: Christoph Hellwig Subject: Re: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates Message-ID: <20150617113121.GC9246@lst.de> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> Sender: owner-linux-mm@kvack.org To: Dan Williams Cc: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org, jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@lists.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de List-ID: This mess with arch_ methods and an ops vecor is almost unreadable. What's the problem with having something like: pmem_foo() { if (arch_has_pmem) // or sync_pmem arch_pmem_foo(); generic_pmem_foo(); } This adds a branch at runtime, but that shoudn't really be any slower than an indirect call on architectures that matter. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Subject: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates From: Dan Williams Date: Thu, 11 Jun 2015 17:19:47 -0400 Message-ID: <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@lists.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de List-ID: From: Ross Zwisler Based on an original patch by Ross Zwisler [1]. Writes to persistent memory have the potential to be posted to cpu cache, cpu write buffers, and platform write buffers (memory controller) before being committed to persistent media. Provide apis, memcpy_to_pmem(), sync_pmem(), and memremap_pmem(), to write data to pmem and assert that it is durable in PMEM (a persistent linear address range). A '__pmem' attribute is added so sparse can track proper usage of pointers to pmem. [1]: https://lists.01.org/pipermail/linux-nvdimm/2015-May/000932.html Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Signed-off-by: Ross Zwisler [djbw: various reworks] Signed-off-by: Dan Williams --- arch/x86/Kconfig | 1 arch/x86/include/asm/cacheflush.h | 36 +++++++++++++ arch/x86/include/asm/io.h | 6 ++ drivers/block/pmem.c | 75 +++++++++++++++++++++++++-- include/linux/compiler.h | 2 + include/linux/pmem.h | 102 +++++++++++++++++++++++++++++++++++++ lib/Kconfig | 3 + 7 files changed, 218 insertions(+), 7 deletions(-) create mode 100644 include/linux/pmem.h diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f16caf7eac27..5dfb8f31ac48 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -28,6 +28,7 @@ config X86 select ARCH_HAS_FAST_MULTIPLIER select ARCH_HAS_GCOV_PROFILE_ALL select ARCH_HAS_MEMREMAP + select ARCH_HAS_PMEM_API select ARCH_HAS_SG_CHAIN select ARCH_HAVE_NMI_SAFE_CMPXCHG select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI diff --git a/arch/x86/include/asm/cacheflush.h b/arch/x86/include/asm/cacheflush.h index b6f7457d12e4..4d896487382c 100644 --- a/arch/x86/include/asm/cacheflush.h +++ b/arch/x86/include/asm/cacheflush.h @@ -4,6 +4,7 @@ /* Caches aren't brain-dead on the intel. */ #include #include +#include /* * The set_memory_* API can be used to change various attributes of a virtual @@ -108,4 +109,39 @@ static inline int rodata_test(void) } #endif +#ifdef ARCH_HAS_NOCACHE_UACCESS +static inline void arch_memcpy_to_pmem(void __pmem *dst, const void *src, size_t n) +{ + /* + * We are copying between two kernel buffers, if + * __copy_from_user_inatomic_nocache() returns an error (page + * fault) we would have already taken an unhandled fault before + * the BUG_ON. The BUG_ON is simply here to satisfy + * __must_check and allow reuse of the common non-temporal store + * implementation for memcpy_to_pmem(). + */ + BUG_ON(__copy_from_user_inatomic_nocache((void __force *) dst, + (void __user *) src, n)); +} + +static inline void arch_sync_pmem(void) +{ + wmb(); + pcommit_sfence(); +} + +static inline bool __arch_has_sync_pmem(void) +{ + return boot_cpu_has(X86_FEATURE_PCOMMIT); +} +#else /* ARCH_HAS_NOCACHE_UACCESS i.e. ARCH=um */ +extern void arch_memcpy_to_pmem(void __pmem *dst, const void *src, size_t n); +extern void arch_sync_pmem(void); + +static inline bool __arch_has_sync_pmem(void) +{ + return false; +} +#endif + #endif /* _ASM_X86_CACHEFLUSH_H */ diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h index e9d6691ec4c5..0a494ac22a8e 100644 --- a/arch/x86/include/asm/io.h +++ b/arch/x86/include/asm/io.h @@ -249,6 +249,12 @@ static inline void flush_write_buffers(void) #endif } +static inline void __pmem *arch_memremap_pmem(resource_size_t offset, + unsigned long size) +{ + return (void __force __pmem *) ioremap_cache(offset, size); +} + #endif /* __KERNEL__ */ extern void native_io_delay(void); diff --git a/drivers/block/pmem.c b/drivers/block/pmem.c index b00b97314b57..81090f61b8b1 100644 --- a/drivers/block/pmem.c +++ b/drivers/block/pmem.c @@ -23,23 +23,79 @@ #include #include #include +#include #include #define PMEM_MINORS 16 +struct pmem_ops { + void __pmem *(*remap)(resource_size_t offset, unsigned long size); + void (*copy)(void __pmem *dst, const void *src, size_t size); + void (*sync)(void); +}; + struct pmem_device { struct request_queue *pmem_queue; struct gendisk *pmem_disk; /* One contiguous memory region per device */ phys_addr_t phys_addr; - void *virt_addr; + void __pmem *virt_addr; size_t size; + struct pmem_ops ops; }; static int pmem_major; static atomic_t pmem_index; +static void default_sync_pmem(void) +{ + wmb(); +} + +static void default_memcpy_to_pmem(void __pmem *dst, const void *src, size_t size) +{ + memcpy((void __force *) dst, src, size); +} + +static void __pmem *default_memremap_pmem(resource_size_t offset, unsigned long size) +{ + return (void __pmem *)memremap_wt(offset, size); +} + +static void pmem_ops_default_init(struct pmem_device *pmem) +{ + /* + * These defaults seek to offer decent performance and minimize + * the window between i/o completion and writes being durable on + * media. However, it is undefined / architecture specific + * whether default_memcpy_to_pmem + default_pmem_sync is + * sufficient for making data durable relative to i/o + * completion. + */ + pmem->ops.remap = default_memremap_pmem; + pmem->ops.copy = default_memcpy_to_pmem; + pmem->ops.sync = default_sync_pmem; +} + +static bool pmem_ops_init(struct pmem_device *pmem) +{ + if (IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API) && + arch_has_sync_pmem()) { + /* + * This arch + cpu guarantees that bio_endio() == data + * durable on media. + */ + pmem->ops.remap = memremap_pmem; + pmem->ops.copy = memcpy_to_pmem; + pmem->ops.sync = sync_pmem; + return true; + } + + pmem_ops_default_init(pmem); + return false; +} + static void pmem_do_bvec(struct pmem_device *pmem, struct page *page, unsigned int len, unsigned int off, int rw, sector_t sector) @@ -48,11 +104,11 @@ static void pmem_do_bvec(struct pmem_device *pmem, struct page *page, size_t pmem_off = sector << 9; if (rw == READ) { - memcpy(mem + off, pmem->virt_addr + pmem_off, len); + memcpy_from_pmem(mem + off, pmem->virt_addr + pmem_off, len); flush_dcache_page(page); } else { flush_dcache_page(page); - memcpy(pmem->virt_addr + pmem_off, mem + off, len); + pmem->ops.copy(pmem->virt_addr + pmem_off, mem + off, len); } kunmap_atomic(mem); @@ -83,6 +139,8 @@ static void pmem_make_request(struct request_queue *q, struct bio *bio) sector += bvec.bv_len >> 9; } + if (rw) + pmem->ops.sync(); out: bio_endio(bio, err); } @@ -107,7 +165,8 @@ static long pmem_direct_access(struct block_device *bdev, sector_t sector, if (!pmem) return -ENODEV; - *kaddr = pmem->virt_addr + offset; + /* FIXME convert DAX to comprehend that this mapping has a lifetime */ + *kaddr = (void __force *) pmem->virt_addr + offset; *pfn = (pmem->phys_addr + offset) >> PAGE_SHIFT; return pmem->size - offset; @@ -132,6 +191,8 @@ static struct pmem_device *pmem_alloc(struct device *dev, struct resource *res) pmem->phys_addr = res->start; pmem->size = resource_size(res); + if (!pmem_ops_init(pmem)) + dev_warn(dev, "unable to guarantee persistence of writes\n"); err = -EINVAL; if (!request_mem_region(pmem->phys_addr, pmem->size, "pmem")) { @@ -144,7 +205,7 @@ static struct pmem_device *pmem_alloc(struct device *dev, struct resource *res) * of the CPU caches in case of a crash. */ err = -ENOMEM; - pmem->virt_addr = memremap_wt(pmem->phys_addr, pmem->size); + pmem->virt_addr = pmem->ops.remap(pmem->phys_addr, pmem->size); if (!pmem->virt_addr) goto out_release_region; @@ -180,7 +241,7 @@ static struct pmem_device *pmem_alloc(struct device *dev, struct resource *res) out_free_queue: blk_cleanup_queue(pmem->pmem_queue); out_unmap: - memunmap(pmem->virt_addr); + memunmap_pmem(pmem->virt_addr); out_release_region: release_mem_region(pmem->phys_addr, pmem->size); out_free_dev: @@ -194,7 +255,7 @@ static void pmem_free(struct pmem_device *pmem) del_gendisk(pmem->pmem_disk); put_disk(pmem->pmem_disk); blk_cleanup_queue(pmem->pmem_queue); - memunmap(pmem->virt_addr); + memunmap_pmem(pmem->virt_addr); release_mem_region(pmem->phys_addr, pmem->size); kfree(pmem); } diff --git a/include/linux/compiler.h b/include/linux/compiler.h index 05be2352fef8..26fc8bc77f85 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -21,6 +21,7 @@ # define __rcu __attribute__((noderef, address_space(4))) #else # define __rcu +# define __pmem __attribute__((noderef, address_space(5))) #endif extern void __chk_user_ptr(const volatile void __user *); extern void __chk_io_ptr(const volatile void __iomem *); @@ -42,6 +43,7 @@ extern void __chk_io_ptr(const volatile void __iomem *); # define __cond_lock(x,c) (c) # define __percpu # define __rcu +# define __pmem #endif /* Indirect macros required for expanded argument pasting, eg. __LINE__. */ diff --git a/include/linux/pmem.h b/include/linux/pmem.h new file mode 100644 index 000000000000..0fad4ad714cc --- /dev/null +++ b/include/linux/pmem.h @@ -0,0 +1,102 @@ +/* + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of version 2 of the GNU General Public License as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + */ +#ifndef __PMEM_H__ +#define __PMEM_H__ + +#include +#include + +/* + * Architectures that define ARCH_HAS_PMEM_API must provide + * implementations for arch_memremap_pmem(), arch_memcpy_to_pmem(), + * arch_sync_pmem(), and __arch_has_sync_pmem(). + */ + +#ifdef CONFIG_ARCH_HAS_PMEM_API +/** + * memremap_pmem - map physical persistent memory for pmem api + * @offset: physical address of persistent memory + * @size: size of the mapping + * + * Establish a mapping of the architecture specific memory type expected + * by memcpy_to_pmem() and sync_pmem(). For example, it may be + * the case that an uncacheable or writethrough mapping is sufficient, + * or a writeback mapping provided memcpy_to_pmem() and + * sync_pmem() arrange for the data to be written through the + * cache to persistent media. + */ +static inline void __pmem *memremap_pmem(resource_size_t offset, unsigned long size) +{ + return arch_memremap_pmem(offset, size); +} + +/** + * memcpy_to_pmem - copy data to persistent memory + * @dst: destination buffer for the copy + * @src: source buffer for the copy + * @n: length of the copy in bytes + * + * Perform a memory copy that results in the destination of the copy + * being effectively evicted from, or never written to, the processor + * cache hierarchy after the copy completes. After memcpy_to_pmem() + * data may still reside in cpu or platform buffers, so this operation + * must be followed by a sync_pmem(). + */ +static inline void memcpy_to_pmem(void __pmem *dst, const void *src, size_t n) +{ + arch_memcpy_to_pmem(dst, src, n); +} + +/** + * sync_pmem - synchronize writes to persistent memory + * + * After a series of memcpy_to_pmem() operations this drains data from + * cpu write buffers and any platform (memory controller) buffers to + * ensure that written data is durable on persistent memory media. + */ +static inline void sync_pmem(void) +{ + arch_sync_pmem(); +} + +/** + * arch_has_sync_pmem - true if sync_pmem() ensures durability + * + * For a given cpu implementation within an architecture it is possible + * that sync_pmem() resolves to a nop. In the case this returns + * false, pmem api users are unable to ensure durabilty and may want to + * fall back to a different data consistency model, or otherwise notify + * the user. + */ +static inline bool arch_has_sync_pmem(void) +{ + return __arch_has_sync_pmem(); +} +#else +/* undefined symbols */ +extern void __pmem *memremap_pmem(resource_size_t offet, unsigned long size); +extern void memcpy_to_pmem(void __pmem *dst, const void *src, size_t n); +extern void sync_pmem(void); +extern bool arch_has_sync_pmem(void); +#endif /* CONFIG_ARCH_HAS_PMEM_API */ + +static inline void memcpy_from_pmem(void *dst, void __pmem const *src, size_t size) +{ + memcpy(dst, (void __force const *) src, size); +} + +static inline void memunmap_pmem(void __pmem *addr) +{ + memunmap((void __force *) addr); +} +#endif /* __PMEM_H__ */ diff --git a/lib/Kconfig b/lib/Kconfig index bc7bc0278921..0d28cc560c6b 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -525,4 +525,7 @@ config ARCH_HAS_SG_CHAIN config ARCH_HAS_MEMREMAP bool +config ARCH_HAS_PMEM_API + bool + endmenu -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Message-ID: <1434562513.11808.100.camel@misato.fc.hp.com> Subject: Re: [PATCH v4 1/6] arch: unify ioremap prototypes and macro aliases From: Toshi Kani Date: Wed, 17 Jun 2015 11:35:13 -0600 In-Reply-To: <20150611211918.10271.74243.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211918.10271.74243.stgit@dwillia2-desk3.amr.corp.intel.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: Dan Williams Cc: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org, jgross@suse.com, x86@kernel.org, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, kbuild test robot , hch@lst.de List-ID: On Thu, 2015-06-11 at 17:19 -0400, Dan Williams wrote: > Some archs define the first parameter to ioremap() as unsigned long, > while the balance define it as resource_size_t. Unify on > resource_size_t to enable passing ioremap function pointers. Also, some > archs use function-like macros for defining ioremap aliases, but > asm-generic/iomap.h expects object-like macros, unify on the latter. > : > diff --git a/arch/ia64/include/asm/io.h b/arch/ia64/include/asm/io.h > index 80a7e34be009..8588ef767a44 100644 > --- a/arch/ia64/include/asm/io.h > +++ b/arch/ia64/include/asm/io.h > @@ -424,8 +424,8 @@ __writeq (unsigned long val, volatile void __iomem *addr) > > # ifdef __KERNEL__ > > -extern void __iomem * ioremap(unsigned long offset, unsigned long size); > -extern void __iomem * ioremap_nocache (unsigned long offset, unsigned long size); > +extern void __iomem * ioremap(resource_size_t offset, unsigned long size); > +extern void __iomem * ioremap_nocache (resource_size_t offset, unsigned long size); > extern void iounmap (volatile void __iomem *addr); > extern void __iomem * early_ioremap (unsigned long phys_addr, unsigned long size); > #define early_memremap(phys_addr, size) early_ioremap(phys_addr, size) This ia64 io.h also defines ioremap_cache(). Should this be also changed to resource_size_t? static inline void __iomem * ioremap_cache (unsigned long phys_addr, unsigned long size) { return ioremap(phys_addr, size); } -Toshi -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Wed, 17 Jun 2015 13:14:12 +0200 From: Christoph Hellwig Subject: Re: [PATCH v4 1/6] arch: unify ioremap prototypes and macro aliases Message-ID: <20150617111412.GA9079@lst.de> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211918.10271.74243.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150611211918.10271.74243.stgit@dwillia2-desk3.amr.corp.intel.com> Sender: owner-linux-mm@kvack.org To: Dan Williams Cc: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org, jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, kbuild test robot , hch@lst.de List-ID: On Thu, Jun 11, 2015 at 05:19:18PM -0400, Dan Williams wrote: > Some archs define the first parameter to ioremap() as unsigned long, > while the balance define it as resource_size_t. Unify on > resource_size_t to enable passing ioremap function pointers. Also, some > archs use function-like macros for defining ioremap aliases, but > asm-generic/iomap.h expects object-like macros, unify on the latter. Not urgent: but maybe this is a big hint that we should not define the prototypes on a per-architecture basis but in a common header file. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Subject: [PATCH v4 1/6] arch: unify ioremap prototypes and macro aliases From: Dan Williams Date: Thu, 11 Jun 2015 17:19:18 -0400 Message-ID: <20150611211918.10271.74243.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, kbuild test robot , hch@lst.de List-ID: Some archs define the first parameter to ioremap() as unsigned long, while the balance define it as resource_size_t. Unify on resource_size_t to enable passing ioremap function pointers. Also, some archs use function-like macros for defining ioremap aliases, but asm-generic/iomap.h expects object-like macros, unify on the latter. Reported-by: kbuild test robot Signed-off-by: Dan Williams --- arch/cris/include/asm/io.h | 8 ++++---- arch/cris/mm/ioremap.c | 6 +++--- arch/ia64/include/asm/io.h | 4 ++-- arch/ia64/mm/ioremap.c | 4 ++-- arch/powerpc/include/asm/io.h | 2 +- arch/sparc/include/asm/io_64.h | 8 ++++---- 6 files changed, 16 insertions(+), 16 deletions(-) diff --git a/arch/cris/include/asm/io.h b/arch/cris/include/asm/io.h index 752a3f45df60..2e4ee658fb04 100644 --- a/arch/cris/include/asm/io.h +++ b/arch/cris/include/asm/io.h @@ -34,17 +34,17 @@ static inline void * phys_to_virt(unsigned long address) return __va(address); } -extern void __iomem * __ioremap(unsigned long offset, unsigned long size, unsigned long flags); -extern void __iomem * __ioremap_prot(unsigned long phys_addr, unsigned long size, pgprot_t prot); +extern void __iomem * __ioremap(resource_size_t offset, unsigned long size, unsigned long flags); +extern void __iomem * __ioremap_prot(resource_size_t phys_addr, unsigned long size, pgprot_t prot); -static inline void __iomem * ioremap (unsigned long offset, unsigned long size) +static inline void __iomem * ioremap (resource_size_t offset, unsigned long size) { return __ioremap(offset, size, 0); } extern void iounmap(volatile void * __iomem addr); -extern void __iomem * ioremap_nocache(unsigned long offset, unsigned long size); +extern void __iomem * ioremap_nocache(resource_size_t offset, unsigned long size); /* * IO bus memory addresses are also 1:1 with the physical address diff --git a/arch/cris/mm/ioremap.c b/arch/cris/mm/ioremap.c index 80fdb995a8ce..51ae80432eb5 100644 --- a/arch/cris/mm/ioremap.c +++ b/arch/cris/mm/ioremap.c @@ -27,7 +27,7 @@ * have to convert them into an offset in a page-aligned mapping, but the * caller shouldn't need to know that small detail. */ -void __iomem * __ioremap_prot(unsigned long phys_addr, unsigned long size, pgprot_t prot) +void __iomem * __ioremap_prot(resource_size_t phys_addr, unsigned long size, pgprot_t prot) { void __iomem * addr; struct vm_struct * area; @@ -60,7 +60,7 @@ void __iomem * __ioremap_prot(unsigned long phys_addr, unsigned long size, pgpro return (void __iomem *) (offset + (char __iomem *)addr); } -void __iomem * __ioremap(unsigned long phys_addr, unsigned long size, unsigned long flags) +void __iomem * __ioremap(resource_size_t phys_addr, unsigned long size, unsigned long flags) { return __ioremap_prot(phys_addr, size, __pgprot(_PAGE_PRESENT | __READABLE | @@ -76,7 +76,7 @@ void __iomem * __ioremap(unsigned long phys_addr, unsigned long size, unsigned l * Must be freed with iounmap. */ -void __iomem *ioremap_nocache(unsigned long phys_addr, unsigned long size) +void __iomem *ioremap_nocache(resource_size_t phys_addr, unsigned long size) { return __ioremap(phys_addr | MEM_NON_CACHEABLE, size, 0); } diff --git a/arch/ia64/include/asm/io.h b/arch/ia64/include/asm/io.h index 80a7e34be009..8588ef767a44 100644 --- a/arch/ia64/include/asm/io.h +++ b/arch/ia64/include/asm/io.h @@ -424,8 +424,8 @@ __writeq (unsigned long val, volatile void __iomem *addr) # ifdef __KERNEL__ -extern void __iomem * ioremap(unsigned long offset, unsigned long size); -extern void __iomem * ioremap_nocache (unsigned long offset, unsigned long size); +extern void __iomem * ioremap(resource_size_t offset, unsigned long size); +extern void __iomem * ioremap_nocache (resource_size_t offset, unsigned long size); extern void iounmap (volatile void __iomem *addr); extern void __iomem * early_ioremap (unsigned long phys_addr, unsigned long size); #define early_memremap(phys_addr, size) early_ioremap(phys_addr, size) diff --git a/arch/ia64/mm/ioremap.c b/arch/ia64/mm/ioremap.c index 43964cde6214..205d71445f06 100644 --- a/arch/ia64/mm/ioremap.c +++ b/arch/ia64/mm/ioremap.c @@ -32,7 +32,7 @@ early_ioremap (unsigned long phys_addr, unsigned long size) } void __iomem * -ioremap (unsigned long phys_addr, unsigned long size) +ioremap (resource_size_t phys_addr, unsigned long size) { void __iomem *addr; struct vm_struct *area; @@ -102,7 +102,7 @@ ioremap (unsigned long phys_addr, unsigned long size) EXPORT_SYMBOL(ioremap); void __iomem * -ioremap_nocache (unsigned long phys_addr, unsigned long size) +ioremap_nocache (resource_size_t phys_addr, unsigned long size) { if (kern_mem_attribute(phys_addr, size) & EFI_MEMORY_WB) return NULL; diff --git a/arch/powerpc/include/asm/io.h b/arch/powerpc/include/asm/io.h index a8d2ef30d473..eaadc99b652b 100644 --- a/arch/powerpc/include/asm/io.h +++ b/arch/powerpc/include/asm/io.h @@ -720,7 +720,7 @@ extern void __iomem *ioremap(phys_addr_t address, unsigned long size); extern void __iomem *ioremap_prot(phys_addr_t address, unsigned long size, unsigned long flags); extern void __iomem *ioremap_wc(phys_addr_t address, unsigned long size); -#define ioremap_nocache(addr, size) ioremap((addr), (size)) +#define ioremap_nocache ioremap extern void iounmap(volatile void __iomem *addr); diff --git a/arch/sparc/include/asm/io_64.h b/arch/sparc/include/asm/io_64.h index c32fa3f752c8..b99ae1fac174 100644 --- a/arch/sparc/include/asm/io_64.h +++ b/arch/sparc/include/asm/io_64.h @@ -395,14 +395,14 @@ static inline void memcpy_toio(volatile void __iomem *dst, const void *src, /* On sparc64 we have the whole physical IO address space accessible * using physically addressed loads and stores, so this does nothing. */ -static inline void __iomem *ioremap(unsigned long offset, unsigned long size) +static inline void __iomem *ioremap(resource_size_t offset, unsigned long size) { return (void __iomem *)offset; } -#define ioremap_nocache(X,Y) ioremap((X),(Y)) -#define ioremap_wc(X,Y) ioremap((X),(Y)) -#define ioremap_wt(X,Y) ioremap((X),(Y)) +#define ioremap_nocache ioremap +#define ioremap_wc ioremap +#define ioremap_wt ioremap static inline void iounmap(volatile void __iomem *addr) { -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Subject: [PATCH v4 4/6] devm: fix ioremap_cache() usage From: Dan Williams Date: Thu, 11 Jun 2015 17:19:36 -0400 Message-ID: <20150611211936.10271.3169.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@lists.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de List-ID: Provide devm_ioremap_cache() and fix up devm_ioremap_resource() to actually provide cacheable mappings. On archs that implement ioremap_cache() devm_ioremap_resource() is always silently falling back to uncached when IORESOURCE_CACHEABLE is specified. Cc: Toshi Kani Cc: Arnd Bergmann Signed-off-by: Dan Williams --- include/linux/io.h | 2 ++ lib/devres.c | 53 +++++++++++++++++++++++++--------------------------- 2 files changed, 27 insertions(+), 28 deletions(-) diff --git a/include/linux/io.h b/include/linux/io.h index fb5a99800e77..4cd8996cfea0 100644 --- a/include/linux/io.h +++ b/include/linux/io.h @@ -71,6 +71,8 @@ static inline void devm_ioport_unmap(struct device *dev, void __iomem *addr) void __iomem *devm_ioremap(struct device *dev, resource_size_t offset, resource_size_t size); +void __iomem *devm_ioremap_cache(struct device *dev, resource_size_t offset, + resource_size_t size); void __iomem *devm_ioremap_nocache(struct device *dev, resource_size_t offset, resource_size_t size); void __iomem *devm_ioremap_wc(struct device *dev, resource_size_t offset, diff --git a/lib/devres.c b/lib/devres.c index f4001d90d24d..c8e75cdaf816 100644 --- a/lib/devres.c +++ b/lib/devres.c @@ -14,6 +14,8 @@ static int devm_ioremap_match(struct device *dev, void *res, void *match_data) return *(void **)res == match_data; } +typedef void __iomem *(*ioremap_fn)(resource_size_t offset, unsigned long size); + /** * devm_ioremap - Managed ioremap() * @dev: Generic device to remap IO address for @@ -22,8 +24,9 @@ static int devm_ioremap_match(struct device *dev, void *res, void *match_data) * * Managed ioremap(). Map is automatically unmapped on driver detach. */ -void __iomem *devm_ioremap(struct device *dev, resource_size_t offset, - resource_size_t size) +static void __iomem *devm_ioremap_type(struct device *dev, + resource_size_t offset, resource_size_t size, + ioremap_fn ioremap_type) { void __iomem **ptr, *addr; @@ -31,7 +34,7 @@ void __iomem *devm_ioremap(struct device *dev, resource_size_t offset, if (!ptr) return NULL; - addr = ioremap(offset, size); + addr = ioremap_type(offset, size); if (addr) { *ptr = addr; devres_add(dev, ptr); @@ -40,34 +43,25 @@ void __iomem *devm_ioremap(struct device *dev, resource_size_t offset, return addr; } + +void __iomem *devm_ioremap(struct device *dev, resource_size_t offset, + resource_size_t size) +{ + return devm_ioremap_type(dev, offset, size, ioremap); +} EXPORT_SYMBOL(devm_ioremap); -/** - * devm_ioremap_nocache - Managed ioremap_nocache() - * @dev: Generic device to remap IO address for - * @offset: BUS offset to map - * @size: Size of map - * - * Managed ioremap_nocache(). Map is automatically unmapped on driver - * detach. - */ +void __iomem *devm_ioremap_cache(struct device *dev, resource_size_t offset, + resource_size_t size) +{ + return devm_ioremap_type(dev, offset, size, ioremap_cache); +} +EXPORT_SYMBOL(devm_ioremap_cache); + void __iomem *devm_ioremap_nocache(struct device *dev, resource_size_t offset, resource_size_t size) { - void __iomem **ptr, *addr; - - ptr = devres_alloc(devm_ioremap_release, sizeof(*ptr), GFP_KERNEL); - if (!ptr) - return NULL; - - addr = ioremap_nocache(offset, size); - if (addr) { - *ptr = addr; - devres_add(dev, ptr); - } else - devres_free(ptr); - - return addr; + return devm_ioremap_type(dev, offset, size, ioremap_nocache); } EXPORT_SYMBOL(devm_ioremap_nocache); @@ -153,8 +147,11 @@ void __iomem *devm_ioremap_resource(struct device *dev, struct resource *res) return IOMEM_ERR_PTR(-EBUSY); } - /* FIXME: add devm_ioremap_cache support */ - dest_ptr = devm_ioremap(dev, res->start, size); + if (res->flags & IORESOURCE_CACHEABLE) + dest_ptr = devm_ioremap_cache(dev, res->start, size); + else + dest_ptr = devm_ioremap_nocache(dev, res->start, size); + if (!dest_ptr) { dev_err(dev, "ioremap failed for resource %pR\n", res); devm_release_mem_region(dev, res->start, size); -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Wed, 17 Jun 2015 13:27:59 +0200 From: Christoph Hellwig Subject: Re: [PATCH v4 3/6] arch/*/asm/io.h: add ioremap_cache() to all architectures Message-ID: <20150617112759.GB9246@lst.de> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211930.10271.9100.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150611211930.10271.9100.stgit@dwillia2-desk3.amr.corp.intel.com> Sender: owner-linux-mm@kvack.org To: Dan Williams Cc: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org, jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@lists.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de List-ID: Ok, this convinces me that we really should have a common defintion for ioremap() and friends first. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Subject: [PATCH v4 3/6] arch/*/asm/io.h: add ioremap_cache() to all architectures From: Dan Williams Date: Thu, 11 Jun 2015 17:19:30 -0400 Message-ID: <20150611211930.10271.9100.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@lists.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de List-ID: Similar to ioremap_wc() let architecture implementations optionally provide ioremap_cache(). As is, current ioremap_cache() users have architecture dependencies that prevent them from compiling on archs without ioremap_cache(). In some cases the architectures that have a cached ioremap() capability have an identifier other than "ioremap_cache". Allow drivers to compile with ioremap_cache() support and fallback to a safe / uncached ioremap otherwise. Cc: Toshi Kani Signed-off-by: Dan Williams --- arch/arc/include/asm/io.h | 1 + arch/arm/include/asm/io.h | 2 ++ arch/arm64/include/asm/io.h | 3 +++ arch/avr32/include/asm/io.h | 1 + arch/frv/include/asm/io.h | 6 ++++++ arch/ia64/include/asm/io.h | 5 ----- arch/m32r/include/asm/io.h | 1 + arch/m68k/include/asm/io_mm.h | 7 +++++++ arch/m68k/include/asm/io_no.h | 5 +++++ arch/metag/include/asm/io.h | 5 +++++ arch/microblaze/include/asm/io.h | 1 + arch/mips/include/asm/io.h | 17 +++++++++++++---- arch/mn10300/include/asm/io.h | 1 + arch/nios2/include/asm/io.h | 1 + arch/s390/include/asm/io.h | 1 + arch/sparc/include/asm/io_32.h | 1 + arch/sparc/include/asm/io_64.h | 1 + arch/tile/include/asm/io.h | 1 + arch/x86/include/asm/io.h | 1 + arch/xtensa/include/asm/io.h | 3 +++ include/asm-generic/io.h | 8 ++++++++ include/asm-generic/iomap.h | 4 ++++ 22 files changed, 67 insertions(+), 9 deletions(-) diff --git a/arch/arc/include/asm/io.h b/arch/arc/include/asm/io.h index 7cc4ced5dbf4..6b6f5a47acec 100644 --- a/arch/arc/include/asm/io.h +++ b/arch/arc/include/asm/io.h @@ -19,6 +19,7 @@ extern void __iomem *ioremap_prot(phys_addr_t offset, unsigned long size, extern void iounmap(const void __iomem *addr); #define ioremap_nocache(phy, sz) ioremap(phy, sz) +#define ioremap_cache(phy, sz) ioremap(phy, sz) #define ioremap_wc(phy, sz) ioremap(phy, sz) #define ioremap_wt(phy, sz) ioremap(phy, sz) diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h index 1b7677d1e5e1..5e2c5cbdffdc 100644 --- a/arch/arm/include/asm/io.h +++ b/arch/arm/include/asm/io.h @@ -23,6 +23,8 @@ #ifdef __KERNEL__ +#define ARCH_HAS_IOREMAP_CACHE + #include #include #include diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h index 7116d3973058..6a8836c9d993 100644 --- a/arch/arm64/include/asm/io.h +++ b/arch/arm64/include/asm/io.h @@ -21,6 +21,8 @@ #ifdef __KERNEL__ +#define ARCH_HAS_IOREMAP_CACHE + #include #include @@ -171,6 +173,7 @@ extern void __iomem *ioremap_cache(phys_addr_t phys_addr, size_t size); #define ioremap_nocache(addr, size) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE)) #define ioremap_wc(addr, size) __ioremap((addr), (size), __pgprot(PROT_NORMAL_NC)) #define ioremap_wt(addr, size) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE)) +#define ioremap_cache ioremap_cache #define iounmap __iounmap /* diff --git a/arch/avr32/include/asm/io.h b/arch/avr32/include/asm/io.h index e998ff5d8e1a..c6994d880dbd 100644 --- a/arch/avr32/include/asm/io.h +++ b/arch/avr32/include/asm/io.h @@ -297,6 +297,7 @@ extern void __iounmap(void __iomem *addr); #define ioremap_wc ioremap_nocache #define ioremap_wt ioremap_nocache +#define ioremap_cache ioremap_nocache #define cached(addr) P1SEGADDR(addr) #define uncached(addr) P2SEGADDR(addr) diff --git a/arch/frv/include/asm/io.h b/arch/frv/include/asm/io.h index a31b63ec4930..cd841f852af3 100644 --- a/arch/frv/include/asm/io.h +++ b/arch/frv/include/asm/io.h @@ -18,6 +18,7 @@ #ifdef __KERNEL__ #define ARCH_HAS_IOREMAP_WT +#define ARCH_HAS_IOREMAP_CACHE #include #include @@ -277,6 +278,11 @@ static inline void __iomem *ioremap_fullcache(unsigned long physaddr, unsigned l return __ioremap(physaddr, size, IOMAP_FULL_CACHING); } +static inline void __iomem *ioremap_cache(unsigned long physaddr, unsigned long size) +{ + return __ioremap(physaddr, size, IOMAP_FULL_CACHING); +} + #define ioremap_wc ioremap_nocache extern void iounmap(void volatile __iomem *addr); diff --git a/arch/ia64/include/asm/io.h b/arch/ia64/include/asm/io.h index 8588ef767a44..cba265c30c3c 100644 --- a/arch/ia64/include/asm/io.h +++ b/arch/ia64/include/asm/io.h @@ -431,11 +431,6 @@ extern void __iomem * early_ioremap (unsigned long phys_addr, unsigned long size #define early_memremap(phys_addr, size) early_ioremap(phys_addr, size) extern void early_iounmap (volatile void __iomem *addr, unsigned long size); #define early_memunmap(addr, size) early_iounmap(addr, size) -static inline void __iomem * ioremap_cache (unsigned long phys_addr, unsigned long size) -{ - return ioremap(phys_addr, size); -} - /* * String version of IO memory access ops: diff --git a/arch/m32r/include/asm/io.h b/arch/m32r/include/asm/io.h index 0c3f25ee3381..f3eceeac25c8 100644 --- a/arch/m32r/include/asm/io.h +++ b/arch/m32r/include/asm/io.h @@ -67,6 +67,7 @@ static inline void __iomem *ioremap(unsigned long offset, unsigned long size) extern void iounmap(volatile void __iomem *addr); #define ioremap_nocache(off,size) ioremap(off,size) +#define ioremap_cache ioremap_nocache #define ioremap_wc ioremap_nocache #define ioremap_wt ioremap_nocache diff --git a/arch/m68k/include/asm/io_mm.h b/arch/m68k/include/asm/io_mm.h index 618c85d3c786..aaf1009f2f94 100644 --- a/arch/m68k/include/asm/io_mm.h +++ b/arch/m68k/include/asm/io_mm.h @@ -21,6 +21,7 @@ #ifdef __KERNEL__ #define ARCH_HAS_IOREMAP_WT +#define ARCH_HAS_IOREMAP_CACHE #include #include @@ -478,6 +479,12 @@ static inline void __iomem *ioremap_fullcache(unsigned long physaddr, return __ioremap(physaddr, size, IOMAP_FULL_CACHING); } +static inline void __iomem *ioremap_cache(unsigned long physaddr, + unsigned long size) +{ + return __ioremap(physaddr, size, IOMAP_FULL_CACHING); +} + static inline void memset_io(volatile void __iomem *addr, unsigned char val, int count) { __builtin_memset((void __force *) addr, val, count); diff --git a/arch/m68k/include/asm/io_no.h b/arch/m68k/include/asm/io_no.h index ad7bd40e6742..020483566b73 100644 --- a/arch/m68k/include/asm/io_no.h +++ b/arch/m68k/include/asm/io_no.h @@ -4,6 +4,7 @@ #ifdef __KERNEL__ #define ARCH_HAS_IOREMAP_WT +#define ARCH_HAS_IOREMAP_CACHE #include #include @@ -163,6 +164,10 @@ static inline void *ioremap_fullcache(unsigned long physaddr, unsigned long size { return __ioremap(physaddr, size, IOMAP_FULL_CACHING); } +static inline void *ioremap_cache(unsigned long physaddr, unsigned long size) +{ + return __ioremap(physaddr, size, IOMAP_FULL_CACHING); +} #define iounmap(addr) do { } while(0) diff --git a/arch/metag/include/asm/io.h b/arch/metag/include/asm/io.h index 9890f21eadbe..d9b2873e3ca8 100644 --- a/arch/metag/include/asm/io.h +++ b/arch/metag/include/asm/io.h @@ -1,6 +1,8 @@ #ifndef _ASM_METAG_IO_H #define _ASM_METAG_IO_H +#define ARCH_HAS_IOREMAP_CACHE + #include #include @@ -157,6 +159,9 @@ extern void __iounmap(void __iomem *addr); #define ioremap_cached(offset, size) \ __ioremap((offset), (size), _PAGE_CACHEABLE) +#define ioremap_cache(offset, size) \ + __ioremap((offset), (size), _PAGE_CACHEABLE) + #define ioremap_wc(offset, size) \ __ioremap((offset), (size), _PAGE_WR_COMBINE) diff --git a/arch/microblaze/include/asm/io.h b/arch/microblaze/include/asm/io.h index 39b6315db82e..986cc0c9e67f 100644 --- a/arch/microblaze/include/asm/io.h +++ b/arch/microblaze/include/asm/io.h @@ -43,6 +43,7 @@ extern void __iomem *ioremap(phys_addr_t address, unsigned long size); #define ioremap_fullcache(addr, size) ioremap((addr), (size)) #define ioremap_wc(addr, size) ioremap((addr), (size)) #define ioremap_wt(addr, size) ioremap((addr), (size)) +#define ioremap_cache(addr, size) ioremap((addr), (size)) #endif /* CONFIG_MMU */ diff --git a/arch/mips/include/asm/io.h b/arch/mips/include/asm/io.h index 9e777cd42b67..6d4c3ae146a5 100644 --- a/arch/mips/include/asm/io.h +++ b/arch/mips/include/asm/io.h @@ -12,6 +12,8 @@ #ifndef _ASM_IO_H #define _ASM_IO_H +#define ARCH_HAS_IOREMAP_CACHE + #include #include #include @@ -232,8 +234,10 @@ static inline void __iomem * __ioremap_mode(phys_addr_t offset, unsigned long si * address is not guaranteed to be usable directly as a virtual * address. */ -#define ioremap(offset, size) \ - __ioremap_mode((offset), (size), _CACHE_UNCACHED) +static inline void __iomem *ioremap(resource_size_t offset, unsigned long size) +{ + return __ioremap_mode(offset, size, _CACHE_UNCACHED); +} /* * ioremap_nocache - map bus memory into CPU space @@ -254,8 +258,7 @@ static inline void __iomem * __ioremap_mode(phys_addr_t offset, unsigned long si * It's useful if some control registers are in such an area and * write combining or read caching is not desirable: */ -#define ioremap_nocache(offset, size) \ - __ioremap_mode((offset), (size), _CACHE_UNCACHED) +#define ioremap_nocache ioremap /* * ioremap_cachable - map bus memory into CPU space @@ -272,8 +275,14 @@ static inline void __iomem * __ioremap_mode(phys_addr_t offset, unsigned long si * the CPU. Also enables full write-combining. Useful for some * memory-like regions on I/O busses. */ +extern unsigned long _page_cachable_default; #define ioremap_cachable(offset, size) \ __ioremap_mode((offset), (size), _page_cachable_default) +static inline void __iomem *ioremap_cache(resource_size_t offset, + unsigned long size) +{ + return ioremap_cachable(offset, size); +} /* * These two are MIPS specific ioremap variant. ioremap_cacheable_cow diff --git a/arch/mn10300/include/asm/io.h b/arch/mn10300/include/asm/io.h index 07c5b4a3903b..dcab414f40df 100644 --- a/arch/mn10300/include/asm/io.h +++ b/arch/mn10300/include/asm/io.h @@ -283,6 +283,7 @@ static inline void __iomem *ioremap_nocache(unsigned long offset, unsigned long #define ioremap_wc ioremap_nocache #define ioremap_wt ioremap_nocache +#define ioremap_cache ioremap_nocache static inline void iounmap(void __iomem *addr) { diff --git a/arch/nios2/include/asm/io.h b/arch/nios2/include/asm/io.h index c5a62da22cd2..367e2ea7663a 100644 --- a/arch/nios2/include/asm/io.h +++ b/arch/nios2/include/asm/io.h @@ -47,6 +47,7 @@ static inline void iounmap(void __iomem *addr) #define ioremap_wc ioremap_nocache #define ioremap_wt ioremap_nocache +#define ioremap_cache ioremap_nocache /* Pages to physical address... */ #define page_to_phys(page) virt_to_phys(page_to_virt(page)) diff --git a/arch/s390/include/asm/io.h b/arch/s390/include/asm/io.h index cb5fdf3a78fc..6824c3daa2a1 100644 --- a/arch/s390/include/asm/io.h +++ b/arch/s390/include/asm/io.h @@ -30,6 +30,7 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr); #define ioremap_nocache(addr, size) ioremap(addr, size) #define ioremap_wc ioremap_nocache #define ioremap_wt ioremap_nocache +#define ioremap_cache ioremap_nocache static inline void __iomem *ioremap(unsigned long offset, unsigned long size) { diff --git a/arch/sparc/include/asm/io_32.h b/arch/sparc/include/asm/io_32.h index 57f26c398dc9..b9a734caf57d 100644 --- a/arch/sparc/include/asm/io_32.h +++ b/arch/sparc/include/asm/io_32.h @@ -128,6 +128,7 @@ static inline void sbus_memcpy_toio(volatile void __iomem *dst, */ void __iomem *ioremap(unsigned long offset, unsigned long size); #define ioremap_nocache(X,Y) ioremap((X),(Y)) +#define ioremap_cache(X,Y) ioremap((X),(Y)) #define ioremap_wc(X,Y) ioremap((X),(Y)) #define ioremap_wt(X,Y) ioremap((X),(Y)) void iounmap(volatile void __iomem *addr); diff --git a/arch/sparc/include/asm/io_64.h b/arch/sparc/include/asm/io_64.h index b99ae1fac174..61f5d04da892 100644 --- a/arch/sparc/include/asm/io_64.h +++ b/arch/sparc/include/asm/io_64.h @@ -401,6 +401,7 @@ static inline void __iomem *ioremap(resource_size_t offset, unsigned long size) } #define ioremap_nocache ioremap +#define ioremap_cache ioremap #define ioremap_wc ioremap #define ioremap_wt ioremap diff --git a/arch/tile/include/asm/io.h b/arch/tile/include/asm/io.h index dc61de15c1f9..fe853a135e25 100644 --- a/arch/tile/include/asm/io.h +++ b/arch/tile/include/asm/io.h @@ -53,6 +53,7 @@ extern void iounmap(volatile void __iomem *addr); #endif #define ioremap_nocache(physaddr, size) ioremap(physaddr, size) +#define ioremap_cache(physaddr, size) ioremap(physaddr, size) #define ioremap_wc(physaddr, size) ioremap(physaddr, size) #define ioremap_wt(physaddr, size) ioremap(physaddr, size) #define ioremap_fullcache(physaddr, size) ioremap(physaddr, size) diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h index 83ec9b1d77cc..e9d6691ec4c5 100644 --- a/arch/x86/include/asm/io.h +++ b/arch/x86/include/asm/io.h @@ -36,6 +36,7 @@ #define ARCH_HAS_IOREMAP_WC #define ARCH_HAS_IOREMAP_WT +#define ARCH_HAS_IOREMAP_CACHE #include #include diff --git a/arch/xtensa/include/asm/io.h b/arch/xtensa/include/asm/io.h index c39bb6e61911..f91a8a99aa29 100644 --- a/arch/xtensa/include/asm/io.h +++ b/arch/xtensa/include/asm/io.h @@ -12,6 +12,9 @@ #define _XTENSA_IO_H #ifdef __KERNEL__ + +#define ARCH_HAS_IOREMAP_CACHE + #include #include #include diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h index f56094cfdeff..a0665dfcab47 100644 --- a/include/asm-generic/io.h +++ b/include/asm-generic/io.h @@ -793,6 +793,14 @@ static inline void __iomem *ioremap_wt(phys_addr_t offset, size_t size) } #endif +#ifndef ioremap_cache +#define ioremap_cache ioremap_cache +static inline void __iomem *ioremap_cache(phys_addr_t offset, size_t size) +{ + return ioremap_nocache(offset, size); +} +#endif + #ifndef iounmap #define iounmap iounmap diff --git a/include/asm-generic/iomap.h b/include/asm-generic/iomap.h index d8f8622fa044..f0f30464cecd 100644 --- a/include/asm-generic/iomap.h +++ b/include/asm-generic/iomap.h @@ -70,6 +70,10 @@ extern void ioport_unmap(void __iomem *); #define ioremap_wt ioremap_nocache #endif +#ifndef ARCH_HAS_IOREMAP_CACHE +#define ioremap_cache ioremap_nocache +#endif + #ifdef CONFIG_PCI /* Destroy a virtual mapping cookie for a PCI BAR (memory or IO) */ struct pci_dev; -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Subject: [PATCH v4 2/6] cleanup IORESOURCE_CACHEABLE vs ioremap() From: Dan Williams Date: Thu, 11 Jun 2015 17:19:24 -0400 Message-ID: <20150611211924.10271.2950.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@lists.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de List-ID: Quoting Arnd: I was thinking the opposite approach and basically removing all uses of IORESOURCE_CACHEABLE from the kernel. There are only a handful of them.and we can probably replace them all with hardcoded ioremap_cached() calls in the cases they are actually useful. All existing usages of IORESOURCE_CACHEABLE call ioremap() instead of ioremap_nocache() if the resource is cacheable, however ioremap() is uncached by default. Clearly none of the existing usages care about the cacheability, so let's clean that up before introducing generic ioremap_cache() support across architectures. Suggested-by: Arnd Bergmann Signed-off-by: Dan Williams --- arch/arm/mach-clps711x/board-cdb89712.c | 2 +- arch/powerpc/kernel/pci_of_scan.c | 2 +- arch/sparc/kernel/pci.c | 3 +-- drivers/pci/probe.c | 3 +-- drivers/pnp/manager.c | 2 -- drivers/scsi/aic94xx/aic94xx_init.c | 7 +------ drivers/scsi/arcmsr/arcmsr_hba.c | 5 +---- drivers/scsi/mvsas/mv_init.c | 15 ++++----------- drivers/video/fbdev/ocfb.c | 1 - lib/devres.c | 7 ++----- lib/pci_iomap.c | 7 ++----- 11 files changed, 14 insertions(+), 40 deletions(-) diff --git a/arch/arm/mach-clps711x/board-cdb89712.c b/arch/arm/mach-clps711x/board-cdb89712.c index 1ec378c334e5..972abdb10028 100644 --- a/arch/arm/mach-clps711x/board-cdb89712.c +++ b/arch/arm/mach-clps711x/board-cdb89712.c @@ -95,7 +95,7 @@ static struct physmap_flash_data cdb89712_bootrom_pdata __initdata = { static struct resource cdb89712_bootrom_resources[] __initdata = { DEFINE_RES_NAMED(CS7_PHYS_BASE, SZ_128, "BOOTROM", IORESOURCE_MEM | - IORESOURCE_CACHEABLE | IORESOURCE_READONLY), + IORESOURCE_READONLY), }; static struct platform_device cdb89712_bootrom_pdev __initdata = { diff --git a/arch/powerpc/kernel/pci_of_scan.c b/arch/powerpc/kernel/pci_of_scan.c index 42e02a2d570b..d4726addff0b 100644 --- a/arch/powerpc/kernel/pci_of_scan.c +++ b/arch/powerpc/kernel/pci_of_scan.c @@ -102,7 +102,7 @@ static void of_pci_parse_addrs(struct device_node *node, struct pci_dev *dev) res = &dev->resource[(i - PCI_BASE_ADDRESS_0) >> 2]; } else if (i == dev->rom_base_reg) { res = &dev->resource[PCI_ROM_RESOURCE]; - flags |= IORESOURCE_READONLY | IORESOURCE_CACHEABLE; + flags |= IORESOURCE_READONLY; } else { printk(KERN_ERR "PCI: bad cfg reg num 0x%x\n", i); continue; diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c index c928bc64b4ba..04da147e0712 100644 --- a/arch/sparc/kernel/pci.c +++ b/arch/sparc/kernel/pci.c @@ -231,8 +231,7 @@ static void pci_parse_of_addrs(struct platform_device *op, res = &dev->resource[(i - PCI_BASE_ADDRESS_0) >> 2]; } else if (i == dev->rom_base_reg) { res = &dev->resource[PCI_ROM_RESOURCE]; - flags |= IORESOURCE_READONLY | IORESOURCE_CACHEABLE - | IORESOURCE_SIZEALIGN; + flags |= IORESOURCE_READONLY | IORESOURCE_SIZEALIGN; } else { printk(KERN_ERR "PCI: bad cfg reg num 0x%x\n", i); continue; diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 6675a7a1b9fc..bbc7f8f86051 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -326,8 +326,7 @@ static void pci_read_bases(struct pci_dev *dev, unsigned int howmany, int rom) struct resource *res = &dev->resource[PCI_ROM_RESOURCE]; dev->rom_base_reg = rom; res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | - IORESOURCE_READONLY | IORESOURCE_CACHEABLE | - IORESOURCE_SIZEALIGN; + IORESOURCE_READONLY | IORESOURCE_SIZEALIGN; __pci_read_base(dev, pci_bar_mem32, res, rom); } } diff --git a/drivers/pnp/manager.c b/drivers/pnp/manager.c index 9357aa779048..7ad3295752ef 100644 --- a/drivers/pnp/manager.c +++ b/drivers/pnp/manager.c @@ -97,8 +97,6 @@ static int pnp_assign_mem(struct pnp_dev *dev, struct pnp_mem *rule, int idx) /* ??? rule->flags restricted to 8 bits, all tests bogus ??? */ if (!(rule->flags & IORESOURCE_MEM_WRITEABLE)) res->flags |= IORESOURCE_READONLY; - if (rule->flags & IORESOURCE_MEM_CACHEABLE) - res->flags |= IORESOURCE_CACHEABLE; if (rule->flags & IORESOURCE_MEM_RANGELENGTH) res->flags |= IORESOURCE_RANGELENGTH; if (rule->flags & IORESOURCE_MEM_SHADOWABLE) diff --git a/drivers/scsi/aic94xx/aic94xx_init.c b/drivers/scsi/aic94xx/aic94xx_init.c index 02a2512b76a8..1058a7b1e334 100644 --- a/drivers/scsi/aic94xx/aic94xx_init.c +++ b/drivers/scsi/aic94xx/aic94xx_init.c @@ -101,12 +101,7 @@ static int asd_map_memio(struct asd_ha_struct *asd_ha) pci_name(asd_ha->pcidev)); goto Err; } - if (io_handle->flags & IORESOURCE_CACHEABLE) - io_handle->addr = ioremap(io_handle->start, - io_handle->len); - else - io_handle->addr = ioremap_nocache(io_handle->start, - io_handle->len); + io_handle->addr = ioremap(io_handle->start, io_handle->len); if (!io_handle->addr) { asd_printk("couldn't map MBAR%d of %s\n", i==0?0:1, pci_name(asd_ha->pcidev)); diff --git a/drivers/scsi/arcmsr/arcmsr_hba.c b/drivers/scsi/arcmsr/arcmsr_hba.c index 914c39f9f388..e4f77cad9fd8 100644 --- a/drivers/scsi/arcmsr/arcmsr_hba.c +++ b/drivers/scsi/arcmsr/arcmsr_hba.c @@ -259,10 +259,7 @@ static bool arcmsr_remap_pciregion(struct AdapterControlBlock *acb) addr = (unsigned long)pci_resource_start(pdev, 0); range = pci_resource_len(pdev, 0); flags = pci_resource_flags(pdev, 0); - if (flags & IORESOURCE_CACHEABLE) - mem_base0 = ioremap(addr, range); - else - mem_base0 = ioremap_nocache(addr, range); + mem_base0 = ioremap(addr, range); if (!mem_base0) { pr_notice("arcmsr%d: memory mapping region fail\n", acb->host->host_no); diff --git a/drivers/scsi/mvsas/mv_init.c b/drivers/scsi/mvsas/mv_init.c index 53030b0e8015..c01ef5f538b1 100644 --- a/drivers/scsi/mvsas/mv_init.c +++ b/drivers/scsi/mvsas/mv_init.c @@ -325,13 +325,9 @@ int mvs_ioremap(struct mvs_info *mvi, int bar, int bar_ex) goto err_out; res_flag_ex = pci_resource_flags(pdev, bar_ex); - if (res_flag_ex & IORESOURCE_MEM) { - if (res_flag_ex & IORESOURCE_CACHEABLE) - mvi->regs_ex = ioremap(res_start, res_len); - else - mvi->regs_ex = ioremap_nocache(res_start, - res_len); - } else + if (res_flag_ex & IORESOURCE_MEM) + mvi->regs_ex = ioremap(res_start, res_len); + else mvi->regs_ex = (void *)res_start; if (!mvi->regs_ex) goto err_out; @@ -343,10 +339,7 @@ int mvs_ioremap(struct mvs_info *mvi, int bar, int bar_ex) goto err_out; res_flag = pci_resource_flags(pdev, bar); - if (res_flag & IORESOURCE_CACHEABLE) - mvi->regs = ioremap(res_start, res_len); - else - mvi->regs = ioremap_nocache(res_start, res_len); + mvi->regs = ioremap(res_start, res_len); if (!mvi->regs) { if (mvi->regs_ex && (res_flag_ex & IORESOURCE_MEM)) diff --git a/drivers/video/fbdev/ocfb.c b/drivers/video/fbdev/ocfb.c index de9819660ca0..c9293aea8ec3 100644 --- a/drivers/video/fbdev/ocfb.c +++ b/drivers/video/fbdev/ocfb.c @@ -325,7 +325,6 @@ static int ocfb_probe(struct platform_device *pdev) dev_err(&pdev->dev, "I/O resource request failed\n"); return -ENXIO; } - res->flags &= ~IORESOURCE_CACHEABLE; fbdev->regs = devm_ioremap_resource(&pdev->dev, res); if (IS_ERR(fbdev->regs)) return PTR_ERR(fbdev->regs); diff --git a/lib/devres.c b/lib/devres.c index fbe2aac522e6..f4001d90d24d 100644 --- a/lib/devres.c +++ b/lib/devres.c @@ -153,11 +153,8 @@ void __iomem *devm_ioremap_resource(struct device *dev, struct resource *res) return IOMEM_ERR_PTR(-EBUSY); } - if (res->flags & IORESOURCE_CACHEABLE) - dest_ptr = devm_ioremap(dev, res->start, size); - else - dest_ptr = devm_ioremap_nocache(dev, res->start, size); - + /* FIXME: add devm_ioremap_cache support */ + dest_ptr = devm_ioremap(dev, res->start, size); if (!dest_ptr) { dev_err(dev, "ioremap failed for resource %pR\n", res); devm_release_mem_region(dev, res->start, size); diff --git a/lib/pci_iomap.c b/lib/pci_iomap.c index bcce5f149310..e1930dbab2da 100644 --- a/lib/pci_iomap.c +++ b/lib/pci_iomap.c @@ -41,11 +41,8 @@ void __iomem *pci_iomap_range(struct pci_dev *dev, len = maxlen; if (flags & IORESOURCE_IO) return __pci_ioport_map(dev, start, len); - if (flags & IORESOURCE_MEM) { - if (flags & IORESOURCE_CACHEABLE) - return ioremap(start, len); - return ioremap_nocache(start, len); - } + if (flags & IORESOURCE_MEM) + return ioremap(start, len); /* What? */ return NULL; } -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wg0-f46.google.com (mail-wg0-f46.google.com [74.125.82.46]) by kanga.kvack.org (Postfix) with ESMTP id E9FE56B0072 for ; Wed, 17 Jun 2015 11:16:36 -0400 (EDT) Received: by wgv5 with SMTP id 5so39854036wgv.1 for ; Wed, 17 Jun 2015 08:16:36 -0700 (PDT) Received: from Galois.linutronix.de (Galois.linutronix.de. [2001:470:1f0b:db:abcd:42:0:1]) by mx.google.com with ESMTPS id m6si9677796wif.81.2015.06.17.08.16.35 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Wed, 17 Jun 2015 08:16:35 -0700 (PDT) Date: Wed, 17 Jun 2015 17:15:57 +0200 (CEST) From: Thomas Gleixner Subject: Re: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates In-Reply-To: Message-ID: References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Andy Lutomirski Cc: Dan Williams , Arnd Bergmann , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Ross Zwisler , Andrew Morton , Juergen Gross , X86 ML , Toshi Kani , linux-nvdimm , Benjamin Herrenschmidt , Luis Rodriguez , Konrad Rzeszutek Wilk , "linux-kernel@vger.kernel.org" , Stefan Bader , "linux-mm@kvack.org" , Geert Uytterhoeven , Ralf Baechle , Henrique de Moraes Holschuh , Michael Ellerman , Tejun Heo , Paul Mackerras , Christoph Hellwig On Wed, 17 Jun 2015, Andy Lutomirski wrote: > On Thu, Jun 11, 2015 at 2:19 PM, Dan Williams wrote: > > +static inline void arch_sync_pmem(void) > > +{ > > + wmb(); > > + pcommit_sfence(); > > +} > > This function is non-intuitive to me. It's really "arch-specific sync > pmem after one or more copies using arch_memcpy_to_pmem". If normal > stores or memcpy to non-WC memory is used instead, then it's > insufficient if the memory is WB and it's unnecessarily slow if the > memory is WT or UC (the first sfence isn't needed). > > I would change the name and add documentation. I'd also add a comment > about the wmb() being an SFENCE to flush pending non-temporal writes. Not "I'd also add ...". Documentation of memory barriers are mandatory. Thanks, tglx -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753279AbbFKVV7 (ORCPT ); Thu, 11 Jun 2015 17:21:59 -0400 Received: from mga11.intel.com ([192.55.52.93]:52029 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752340AbbFKVV4 (ORCPT ); Thu, 11 Jun 2015 17:21:56 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,597,1427785200"; d="scan'208";a="709467526" Subject: [-tip PATCH v4 0/6] pmem api, generic ioremap_cache, and memremap From: Dan Williams To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@ml01.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, Andy Shevchenko , geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, kbuild test robot , hch@lst.de Date: Thu, 11 Jun 2015 17:19:12 -0400 Message-ID: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-8-g92dd MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The pmem api is responsible for shepherding data out to persistent media. The pmem driver uses this api, when available, to assert that data is durable by the time bio_endio() is invoked. When an architecture or cpu can not make persistence guarantees the driver warns and falls back to "best effort" implementation. Changes since v3 [1]: Rebased on tip/master now that Toshi's ioremap_wt() patches have landed in -tip. The primary change was reflowing the patches against the newly alphabetized 'select' options under config X86. [1]: https://lists.01.org/pipermail/linux-nvdimm/2015-June/001081.html --- Dan Williams (5): arch: unify ioremap prototypes and macro aliases cleanup IORESOURCE_CACHEABLE vs ioremap() arch/*/asm/io.h: add ioremap_cache() to all architectures devm: fix ioremap_cache() usage arch: introduce memremap_cache() and memremap_wt() Ross Zwisler (1): arch, x86: pmem api for ensuring durability of persistent memory updates arch/arc/include/asm/io.h | 1 arch/arm/Kconfig | 1 arch/arm/include/asm/io.h | 2 + arch/arm/mach-clps711x/board-cdb89712.c | 2 - arch/arm64/Kconfig | 1 arch/arm64/include/asm/io.h | 3 + arch/arm64/kernel/efi.c | 4 + arch/arm64/kernel/smp_spin_table.c | 10 ++- arch/avr32/include/asm/io.h | 1 arch/cris/include/asm/io.h | 8 +- arch/cris/mm/ioremap.c | 6 +- arch/frv/Kconfig | 1 arch/frv/include/asm/io.h | 6 ++ arch/ia64/include/asm/io.h | 9 +-- arch/ia64/mm/ioremap.c | 4 + arch/m32r/include/asm/io.h | 1 arch/m68k/Kconfig | 1 arch/m68k/include/asm/io_mm.h | 7 ++ arch/m68k/include/asm/io_no.h | 5 ++ arch/metag/Kconfig | 1 arch/metag/include/asm/io.h | 5 ++ arch/microblaze/include/asm/io.h | 1 arch/mips/Kconfig | 1 arch/mips/include/asm/io.h | 17 ++++- arch/mn10300/include/asm/io.h | 1 arch/nios2/include/asm/io.h | 1 arch/powerpc/Kconfig | 1 arch/powerpc/include/asm/io.h | 2 - arch/powerpc/kernel/pci_of_scan.c | 2 - arch/s390/include/asm/io.h | 1 arch/sparc/include/asm/io_32.h | 1 arch/sparc/include/asm/io_64.h | 9 ++- arch/sparc/kernel/pci.c | 3 - arch/tile/include/asm/io.h | 1 arch/x86/Kconfig | 2 + arch/x86/include/asm/cacheflush.h | 36 +++++++++++ arch/x86/include/asm/io.h | 7 ++ arch/x86/kernel/crash_dump_64.c | 6 +- arch/x86/kernel/kdebugfs.c | 8 +- arch/x86/kernel/ksysfs.c | 28 ++++----- arch/x86/mm/ioremap.c | 10 +-- arch/xtensa/Kconfig | 1 arch/xtensa/include/asm/io.h | 3 + drivers/acpi/apei/einj.c | 8 +- drivers/acpi/apei/erst.c | 4 + drivers/block/Kconfig | 1 drivers/block/pmem.c | 76 +++++++++++++++++++++-- drivers/firmware/google/memconsole.c | 4 + drivers/pci/probe.c | 3 - drivers/pnp/manager.c | 2 - drivers/scsi/aic94xx/aic94xx_init.c | 7 -- drivers/scsi/arcmsr/arcmsr_hba.c | 5 -- drivers/scsi/mvsas/mv_init.c | 15 +---- drivers/video/fbdev/ocfb.c | 1 include/asm-generic/io.h | 8 ++ include/asm-generic/iomap.h | 4 + include/linux/compiler.h | 2 + include/linux/device.h | 5 ++ include/linux/io.h | 6 ++ include/linux/pmem.h | 102 +++++++++++++++++++++++++++++++ kernel/resource.c | 41 ++++++++++++ lib/Kconfig | 8 ++ lib/devres.c | 48 ++++++--------- lib/pci_iomap.c | 7 +- 64 files changed, 440 insertions(+), 138 deletions(-) create mode 100644 include/linux/pmem.h From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754445AbbFKVWF (ORCPT ); Thu, 11 Jun 2015 17:22:05 -0400 Received: from mga01.intel.com ([192.55.52.88]:33346 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751224AbbFKVWA (ORCPT ); Thu, 11 Jun 2015 17:22:00 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,597,1427785200"; d="scan'208";a="725628825" Subject: [PATCH v4 1/6] arch: unify ioremap prototypes and macro aliases From: Dan Williams To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@ml01.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, kbuild test robot , hch@lst.de Date: Thu, 11 Jun 2015 17:19:18 -0400 Message-ID: <20150611211918.10271.74243.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-8-g92dd MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Some archs define the first parameter to ioremap() as unsigned long, while the balance define it as resource_size_t. Unify on resource_size_t to enable passing ioremap function pointers. Also, some archs use function-like macros for defining ioremap aliases, but asm-generic/iomap.h expects object-like macros, unify on the latter. Reported-by: kbuild test robot Signed-off-by: Dan Williams --- arch/cris/include/asm/io.h | 8 ++++---- arch/cris/mm/ioremap.c | 6 +++--- arch/ia64/include/asm/io.h | 4 ++-- arch/ia64/mm/ioremap.c | 4 ++-- arch/powerpc/include/asm/io.h | 2 +- arch/sparc/include/asm/io_64.h | 8 ++++---- 6 files changed, 16 insertions(+), 16 deletions(-) diff --git a/arch/cris/include/asm/io.h b/arch/cris/include/asm/io.h index 752a3f45df60..2e4ee658fb04 100644 --- a/arch/cris/include/asm/io.h +++ b/arch/cris/include/asm/io.h @@ -34,17 +34,17 @@ static inline void * phys_to_virt(unsigned long address) return __va(address); } -extern void __iomem * __ioremap(unsigned long offset, unsigned long size, unsigned long flags); -extern void __iomem * __ioremap_prot(unsigned long phys_addr, unsigned long size, pgprot_t prot); +extern void __iomem * __ioremap(resource_size_t offset, unsigned long size, unsigned long flags); +extern void __iomem * __ioremap_prot(resource_size_t phys_addr, unsigned long size, pgprot_t prot); -static inline void __iomem * ioremap (unsigned long offset, unsigned long size) +static inline void __iomem * ioremap (resource_size_t offset, unsigned long size) { return __ioremap(offset, size, 0); } extern void iounmap(volatile void * __iomem addr); -extern void __iomem * ioremap_nocache(unsigned long offset, unsigned long size); +extern void __iomem * ioremap_nocache(resource_size_t offset, unsigned long size); /* * IO bus memory addresses are also 1:1 with the physical address diff --git a/arch/cris/mm/ioremap.c b/arch/cris/mm/ioremap.c index 80fdb995a8ce..51ae80432eb5 100644 --- a/arch/cris/mm/ioremap.c +++ b/arch/cris/mm/ioremap.c @@ -27,7 +27,7 @@ * have to convert them into an offset in a page-aligned mapping, but the * caller shouldn't need to know that small detail. */ -void __iomem * __ioremap_prot(unsigned long phys_addr, unsigned long size, pgprot_t prot) +void __iomem * __ioremap_prot(resource_size_t phys_addr, unsigned long size, pgprot_t prot) { void __iomem * addr; struct vm_struct * area; @@ -60,7 +60,7 @@ void __iomem * __ioremap_prot(unsigned long phys_addr, unsigned long size, pgpro return (void __iomem *) (offset + (char __iomem *)addr); } -void __iomem * __ioremap(unsigned long phys_addr, unsigned long size, unsigned long flags) +void __iomem * __ioremap(resource_size_t phys_addr, unsigned long size, unsigned long flags) { return __ioremap_prot(phys_addr, size, __pgprot(_PAGE_PRESENT | __READABLE | @@ -76,7 +76,7 @@ void __iomem * __ioremap(unsigned long phys_addr, unsigned long size, unsigned l * Must be freed with iounmap. */ -void __iomem *ioremap_nocache(unsigned long phys_addr, unsigned long size) +void __iomem *ioremap_nocache(resource_size_t phys_addr, unsigned long size) { return __ioremap(phys_addr | MEM_NON_CACHEABLE, size, 0); } diff --git a/arch/ia64/include/asm/io.h b/arch/ia64/include/asm/io.h index 80a7e34be009..8588ef767a44 100644 --- a/arch/ia64/include/asm/io.h +++ b/arch/ia64/include/asm/io.h @@ -424,8 +424,8 @@ __writeq (unsigned long val, volatile void __iomem *addr) # ifdef __KERNEL__ -extern void __iomem * ioremap(unsigned long offset, unsigned long size); -extern void __iomem * ioremap_nocache (unsigned long offset, unsigned long size); +extern void __iomem * ioremap(resource_size_t offset, unsigned long size); +extern void __iomem * ioremap_nocache (resource_size_t offset, unsigned long size); extern void iounmap (volatile void __iomem *addr); extern void __iomem * early_ioremap (unsigned long phys_addr, unsigned long size); #define early_memremap(phys_addr, size) early_ioremap(phys_addr, size) diff --git a/arch/ia64/mm/ioremap.c b/arch/ia64/mm/ioremap.c index 43964cde6214..205d71445f06 100644 --- a/arch/ia64/mm/ioremap.c +++ b/arch/ia64/mm/ioremap.c @@ -32,7 +32,7 @@ early_ioremap (unsigned long phys_addr, unsigned long size) } void __iomem * -ioremap (unsigned long phys_addr, unsigned long size) +ioremap (resource_size_t phys_addr, unsigned long size) { void __iomem *addr; struct vm_struct *area; @@ -102,7 +102,7 @@ ioremap (unsigned long phys_addr, unsigned long size) EXPORT_SYMBOL(ioremap); void __iomem * -ioremap_nocache (unsigned long phys_addr, unsigned long size) +ioremap_nocache (resource_size_t phys_addr, unsigned long size) { if (kern_mem_attribute(phys_addr, size) & EFI_MEMORY_WB) return NULL; diff --git a/arch/powerpc/include/asm/io.h b/arch/powerpc/include/asm/io.h index a8d2ef30d473..eaadc99b652b 100644 --- a/arch/powerpc/include/asm/io.h +++ b/arch/powerpc/include/asm/io.h @@ -720,7 +720,7 @@ extern void __iomem *ioremap(phys_addr_t address, unsigned long size); extern void __iomem *ioremap_prot(phys_addr_t address, unsigned long size, unsigned long flags); extern void __iomem *ioremap_wc(phys_addr_t address, unsigned long size); -#define ioremap_nocache(addr, size) ioremap((addr), (size)) +#define ioremap_nocache ioremap extern void iounmap(volatile void __iomem *addr); diff --git a/arch/sparc/include/asm/io_64.h b/arch/sparc/include/asm/io_64.h index c32fa3f752c8..b99ae1fac174 100644 --- a/arch/sparc/include/asm/io_64.h +++ b/arch/sparc/include/asm/io_64.h @@ -395,14 +395,14 @@ static inline void memcpy_toio(volatile void __iomem *dst, const void *src, /* On sparc64 we have the whole physical IO address space accessible * using physically addressed loads and stores, so this does nothing. */ -static inline void __iomem *ioremap(unsigned long offset, unsigned long size) +static inline void __iomem *ioremap(resource_size_t offset, unsigned long size) { return (void __iomem *)offset; } -#define ioremap_nocache(X,Y) ioremap((X),(Y)) -#define ioremap_wc(X,Y) ioremap((X),(Y)) -#define ioremap_wt(X,Y) ioremap((X),(Y)) +#define ioremap_nocache ioremap +#define ioremap_wc ioremap +#define ioremap_wt ioremap static inline void iounmap(volatile void __iomem *addr) { From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754924AbbFKVWK (ORCPT ); Thu, 11 Jun 2015 17:22:10 -0400 Received: from mga14.intel.com ([192.55.52.115]:25084 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751224AbbFKVWG (ORCPT ); Thu, 11 Jun 2015 17:22:06 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,597,1427785200"; d="scan'208";a="741745734" Subject: [PATCH v4 2/6] cleanup IORESOURCE_CACHEABLE vs ioremap() From: Dan Williams To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@ml01.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de Date: Thu, 11 Jun 2015 17:19:24 -0400 Message-ID: <20150611211924.10271.2950.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-8-g92dd MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Quoting Arnd: I was thinking the opposite approach and basically removing all uses of IORESOURCE_CACHEABLE from the kernel. There are only a handful of them.and we can probably replace them all with hardcoded ioremap_cached() calls in the cases they are actually useful. All existing usages of IORESOURCE_CACHEABLE call ioremap() instead of ioremap_nocache() if the resource is cacheable, however ioremap() is uncached by default. Clearly none of the existing usages care about the cacheability, so let's clean that up before introducing generic ioremap_cache() support across architectures. Suggested-by: Arnd Bergmann Signed-off-by: Dan Williams --- arch/arm/mach-clps711x/board-cdb89712.c | 2 +- arch/powerpc/kernel/pci_of_scan.c | 2 +- arch/sparc/kernel/pci.c | 3 +-- drivers/pci/probe.c | 3 +-- drivers/pnp/manager.c | 2 -- drivers/scsi/aic94xx/aic94xx_init.c | 7 +------ drivers/scsi/arcmsr/arcmsr_hba.c | 5 +---- drivers/scsi/mvsas/mv_init.c | 15 ++++----------- drivers/video/fbdev/ocfb.c | 1 - lib/devres.c | 7 ++----- lib/pci_iomap.c | 7 ++----- 11 files changed, 14 insertions(+), 40 deletions(-) diff --git a/arch/arm/mach-clps711x/board-cdb89712.c b/arch/arm/mach-clps711x/board-cdb89712.c index 1ec378c334e5..972abdb10028 100644 --- a/arch/arm/mach-clps711x/board-cdb89712.c +++ b/arch/arm/mach-clps711x/board-cdb89712.c @@ -95,7 +95,7 @@ static struct physmap_flash_data cdb89712_bootrom_pdata __initdata = { static struct resource cdb89712_bootrom_resources[] __initdata = { DEFINE_RES_NAMED(CS7_PHYS_BASE, SZ_128, "BOOTROM", IORESOURCE_MEM | - IORESOURCE_CACHEABLE | IORESOURCE_READONLY), + IORESOURCE_READONLY), }; static struct platform_device cdb89712_bootrom_pdev __initdata = { diff --git a/arch/powerpc/kernel/pci_of_scan.c b/arch/powerpc/kernel/pci_of_scan.c index 42e02a2d570b..d4726addff0b 100644 --- a/arch/powerpc/kernel/pci_of_scan.c +++ b/arch/powerpc/kernel/pci_of_scan.c @@ -102,7 +102,7 @@ static void of_pci_parse_addrs(struct device_node *node, struct pci_dev *dev) res = &dev->resource[(i - PCI_BASE_ADDRESS_0) >> 2]; } else if (i == dev->rom_base_reg) { res = &dev->resource[PCI_ROM_RESOURCE]; - flags |= IORESOURCE_READONLY | IORESOURCE_CACHEABLE; + flags |= IORESOURCE_READONLY; } else { printk(KERN_ERR "PCI: bad cfg reg num 0x%x\n", i); continue; diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c index c928bc64b4ba..04da147e0712 100644 --- a/arch/sparc/kernel/pci.c +++ b/arch/sparc/kernel/pci.c @@ -231,8 +231,7 @@ static void pci_parse_of_addrs(struct platform_device *op, res = &dev->resource[(i - PCI_BASE_ADDRESS_0) >> 2]; } else if (i == dev->rom_base_reg) { res = &dev->resource[PCI_ROM_RESOURCE]; - flags |= IORESOURCE_READONLY | IORESOURCE_CACHEABLE - | IORESOURCE_SIZEALIGN; + flags |= IORESOURCE_READONLY | IORESOURCE_SIZEALIGN; } else { printk(KERN_ERR "PCI: bad cfg reg num 0x%x\n", i); continue; diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 6675a7a1b9fc..bbc7f8f86051 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -326,8 +326,7 @@ static void pci_read_bases(struct pci_dev *dev, unsigned int howmany, int rom) struct resource *res = &dev->resource[PCI_ROM_RESOURCE]; dev->rom_base_reg = rom; res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | - IORESOURCE_READONLY | IORESOURCE_CACHEABLE | - IORESOURCE_SIZEALIGN; + IORESOURCE_READONLY | IORESOURCE_SIZEALIGN; __pci_read_base(dev, pci_bar_mem32, res, rom); } } diff --git a/drivers/pnp/manager.c b/drivers/pnp/manager.c index 9357aa779048..7ad3295752ef 100644 --- a/drivers/pnp/manager.c +++ b/drivers/pnp/manager.c @@ -97,8 +97,6 @@ static int pnp_assign_mem(struct pnp_dev *dev, struct pnp_mem *rule, int idx) /* ??? rule->flags restricted to 8 bits, all tests bogus ??? */ if (!(rule->flags & IORESOURCE_MEM_WRITEABLE)) res->flags |= IORESOURCE_READONLY; - if (rule->flags & IORESOURCE_MEM_CACHEABLE) - res->flags |= IORESOURCE_CACHEABLE; if (rule->flags & IORESOURCE_MEM_RANGELENGTH) res->flags |= IORESOURCE_RANGELENGTH; if (rule->flags & IORESOURCE_MEM_SHADOWABLE) diff --git a/drivers/scsi/aic94xx/aic94xx_init.c b/drivers/scsi/aic94xx/aic94xx_init.c index 02a2512b76a8..1058a7b1e334 100644 --- a/drivers/scsi/aic94xx/aic94xx_init.c +++ b/drivers/scsi/aic94xx/aic94xx_init.c @@ -101,12 +101,7 @@ static int asd_map_memio(struct asd_ha_struct *asd_ha) pci_name(asd_ha->pcidev)); goto Err; } - if (io_handle->flags & IORESOURCE_CACHEABLE) - io_handle->addr = ioremap(io_handle->start, - io_handle->len); - else - io_handle->addr = ioremap_nocache(io_handle->start, - io_handle->len); + io_handle->addr = ioremap(io_handle->start, io_handle->len); if (!io_handle->addr) { asd_printk("couldn't map MBAR%d of %s\n", i==0?0:1, pci_name(asd_ha->pcidev)); diff --git a/drivers/scsi/arcmsr/arcmsr_hba.c b/drivers/scsi/arcmsr/arcmsr_hba.c index 914c39f9f388..e4f77cad9fd8 100644 --- a/drivers/scsi/arcmsr/arcmsr_hba.c +++ b/drivers/scsi/arcmsr/arcmsr_hba.c @@ -259,10 +259,7 @@ static bool arcmsr_remap_pciregion(struct AdapterControlBlock *acb) addr = (unsigned long)pci_resource_start(pdev, 0); range = pci_resource_len(pdev, 0); flags = pci_resource_flags(pdev, 0); - if (flags & IORESOURCE_CACHEABLE) - mem_base0 = ioremap(addr, range); - else - mem_base0 = ioremap_nocache(addr, range); + mem_base0 = ioremap(addr, range); if (!mem_base0) { pr_notice("arcmsr%d: memory mapping region fail\n", acb->host->host_no); diff --git a/drivers/scsi/mvsas/mv_init.c b/drivers/scsi/mvsas/mv_init.c index 53030b0e8015..c01ef5f538b1 100644 --- a/drivers/scsi/mvsas/mv_init.c +++ b/drivers/scsi/mvsas/mv_init.c @@ -325,13 +325,9 @@ int mvs_ioremap(struct mvs_info *mvi, int bar, int bar_ex) goto err_out; res_flag_ex = pci_resource_flags(pdev, bar_ex); - if (res_flag_ex & IORESOURCE_MEM) { - if (res_flag_ex & IORESOURCE_CACHEABLE) - mvi->regs_ex = ioremap(res_start, res_len); - else - mvi->regs_ex = ioremap_nocache(res_start, - res_len); - } else + if (res_flag_ex & IORESOURCE_MEM) + mvi->regs_ex = ioremap(res_start, res_len); + else mvi->regs_ex = (void *)res_start; if (!mvi->regs_ex) goto err_out; @@ -343,10 +339,7 @@ int mvs_ioremap(struct mvs_info *mvi, int bar, int bar_ex) goto err_out; res_flag = pci_resource_flags(pdev, bar); - if (res_flag & IORESOURCE_CACHEABLE) - mvi->regs = ioremap(res_start, res_len); - else - mvi->regs = ioremap_nocache(res_start, res_len); + mvi->regs = ioremap(res_start, res_len); if (!mvi->regs) { if (mvi->regs_ex && (res_flag_ex & IORESOURCE_MEM)) diff --git a/drivers/video/fbdev/ocfb.c b/drivers/video/fbdev/ocfb.c index de9819660ca0..c9293aea8ec3 100644 --- a/drivers/video/fbdev/ocfb.c +++ b/drivers/video/fbdev/ocfb.c @@ -325,7 +325,6 @@ static int ocfb_probe(struct platform_device *pdev) dev_err(&pdev->dev, "I/O resource request failed\n"); return -ENXIO; } - res->flags &= ~IORESOURCE_CACHEABLE; fbdev->regs = devm_ioremap_resource(&pdev->dev, res); if (IS_ERR(fbdev->regs)) return PTR_ERR(fbdev->regs); diff --git a/lib/devres.c b/lib/devres.c index fbe2aac522e6..f4001d90d24d 100644 --- a/lib/devres.c +++ b/lib/devres.c @@ -153,11 +153,8 @@ void __iomem *devm_ioremap_resource(struct device *dev, struct resource *res) return IOMEM_ERR_PTR(-EBUSY); } - if (res->flags & IORESOURCE_CACHEABLE) - dest_ptr = devm_ioremap(dev, res->start, size); - else - dest_ptr = devm_ioremap_nocache(dev, res->start, size); - + /* FIXME: add devm_ioremap_cache support */ + dest_ptr = devm_ioremap(dev, res->start, size); if (!dest_ptr) { dev_err(dev, "ioremap failed for resource %pR\n", res); devm_release_mem_region(dev, res->start, size); diff --git a/lib/pci_iomap.c b/lib/pci_iomap.c index bcce5f149310..e1930dbab2da 100644 --- a/lib/pci_iomap.c +++ b/lib/pci_iomap.c @@ -41,11 +41,8 @@ void __iomem *pci_iomap_range(struct pci_dev *dev, len = maxlen; if (flags & IORESOURCE_IO) return __pci_ioport_map(dev, start, len); - if (flags & IORESOURCE_MEM) { - if (flags & IORESOURCE_CACHEABLE) - return ioremap(start, len); - return ioremap_nocache(start, len); - } + if (flags & IORESOURCE_MEM) + return ioremap(start, len); /* What? */ return NULL; } From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755243AbbFKVWT (ORCPT ); Thu, 11 Jun 2015 17:22:19 -0400 Received: from mga02.intel.com ([134.134.136.20]:30731 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754933AbbFKVWL (ORCPT ); Thu, 11 Jun 2015 17:22:11 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,597,1427785200"; d="scan'208";a="506894907" Subject: [PATCH v4 3/6] arch/*/asm/io.h: add ioremap_cache() to all architectures From: Dan Williams To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@ml01.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de Date: Thu, 11 Jun 2015 17:19:30 -0400 Message-ID: <20150611211930.10271.9100.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-8-g92dd MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Similar to ioremap_wc() let architecture implementations optionally provide ioremap_cache(). As is, current ioremap_cache() users have architecture dependencies that prevent them from compiling on archs without ioremap_cache(). In some cases the architectures that have a cached ioremap() capability have an identifier other than "ioremap_cache". Allow drivers to compile with ioremap_cache() support and fallback to a safe / uncached ioremap otherwise. Cc: Toshi Kani Signed-off-by: Dan Williams --- arch/arc/include/asm/io.h | 1 + arch/arm/include/asm/io.h | 2 ++ arch/arm64/include/asm/io.h | 3 +++ arch/avr32/include/asm/io.h | 1 + arch/frv/include/asm/io.h | 6 ++++++ arch/ia64/include/asm/io.h | 5 ----- arch/m32r/include/asm/io.h | 1 + arch/m68k/include/asm/io_mm.h | 7 +++++++ arch/m68k/include/asm/io_no.h | 5 +++++ arch/metag/include/asm/io.h | 5 +++++ arch/microblaze/include/asm/io.h | 1 + arch/mips/include/asm/io.h | 17 +++++++++++++---- arch/mn10300/include/asm/io.h | 1 + arch/nios2/include/asm/io.h | 1 + arch/s390/include/asm/io.h | 1 + arch/sparc/include/asm/io_32.h | 1 + arch/sparc/include/asm/io_64.h | 1 + arch/tile/include/asm/io.h | 1 + arch/x86/include/asm/io.h | 1 + arch/xtensa/include/asm/io.h | 3 +++ include/asm-generic/io.h | 8 ++++++++ include/asm-generic/iomap.h | 4 ++++ 22 files changed, 67 insertions(+), 9 deletions(-) diff --git a/arch/arc/include/asm/io.h b/arch/arc/include/asm/io.h index 7cc4ced5dbf4..6b6f5a47acec 100644 --- a/arch/arc/include/asm/io.h +++ b/arch/arc/include/asm/io.h @@ -19,6 +19,7 @@ extern void __iomem *ioremap_prot(phys_addr_t offset, unsigned long size, extern void iounmap(const void __iomem *addr); #define ioremap_nocache(phy, sz) ioremap(phy, sz) +#define ioremap_cache(phy, sz) ioremap(phy, sz) #define ioremap_wc(phy, sz) ioremap(phy, sz) #define ioremap_wt(phy, sz) ioremap(phy, sz) diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h index 1b7677d1e5e1..5e2c5cbdffdc 100644 --- a/arch/arm/include/asm/io.h +++ b/arch/arm/include/asm/io.h @@ -23,6 +23,8 @@ #ifdef __KERNEL__ +#define ARCH_HAS_IOREMAP_CACHE + #include #include #include diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h index 7116d3973058..6a8836c9d993 100644 --- a/arch/arm64/include/asm/io.h +++ b/arch/arm64/include/asm/io.h @@ -21,6 +21,8 @@ #ifdef __KERNEL__ +#define ARCH_HAS_IOREMAP_CACHE + #include #include @@ -171,6 +173,7 @@ extern void __iomem *ioremap_cache(phys_addr_t phys_addr, size_t size); #define ioremap_nocache(addr, size) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE)) #define ioremap_wc(addr, size) __ioremap((addr), (size), __pgprot(PROT_NORMAL_NC)) #define ioremap_wt(addr, size) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE)) +#define ioremap_cache ioremap_cache #define iounmap __iounmap /* diff --git a/arch/avr32/include/asm/io.h b/arch/avr32/include/asm/io.h index e998ff5d8e1a..c6994d880dbd 100644 --- a/arch/avr32/include/asm/io.h +++ b/arch/avr32/include/asm/io.h @@ -297,6 +297,7 @@ extern void __iounmap(void __iomem *addr); #define ioremap_wc ioremap_nocache #define ioremap_wt ioremap_nocache +#define ioremap_cache ioremap_nocache #define cached(addr) P1SEGADDR(addr) #define uncached(addr) P2SEGADDR(addr) diff --git a/arch/frv/include/asm/io.h b/arch/frv/include/asm/io.h index a31b63ec4930..cd841f852af3 100644 --- a/arch/frv/include/asm/io.h +++ b/arch/frv/include/asm/io.h @@ -18,6 +18,7 @@ #ifdef __KERNEL__ #define ARCH_HAS_IOREMAP_WT +#define ARCH_HAS_IOREMAP_CACHE #include #include @@ -277,6 +278,11 @@ static inline void __iomem *ioremap_fullcache(unsigned long physaddr, unsigned l return __ioremap(physaddr, size, IOMAP_FULL_CACHING); } +static inline void __iomem *ioremap_cache(unsigned long physaddr, unsigned long size) +{ + return __ioremap(physaddr, size, IOMAP_FULL_CACHING); +} + #define ioremap_wc ioremap_nocache extern void iounmap(void volatile __iomem *addr); diff --git a/arch/ia64/include/asm/io.h b/arch/ia64/include/asm/io.h index 8588ef767a44..cba265c30c3c 100644 --- a/arch/ia64/include/asm/io.h +++ b/arch/ia64/include/asm/io.h @@ -431,11 +431,6 @@ extern void __iomem * early_ioremap (unsigned long phys_addr, unsigned long size #define early_memremap(phys_addr, size) early_ioremap(phys_addr, size) extern void early_iounmap (volatile void __iomem *addr, unsigned long size); #define early_memunmap(addr, size) early_iounmap(addr, size) -static inline void __iomem * ioremap_cache (unsigned long phys_addr, unsigned long size) -{ - return ioremap(phys_addr, size); -} - /* * String version of IO memory access ops: diff --git a/arch/m32r/include/asm/io.h b/arch/m32r/include/asm/io.h index 0c3f25ee3381..f3eceeac25c8 100644 --- a/arch/m32r/include/asm/io.h +++ b/arch/m32r/include/asm/io.h @@ -67,6 +67,7 @@ static inline void __iomem *ioremap(unsigned long offset, unsigned long size) extern void iounmap(volatile void __iomem *addr); #define ioremap_nocache(off,size) ioremap(off,size) +#define ioremap_cache ioremap_nocache #define ioremap_wc ioremap_nocache #define ioremap_wt ioremap_nocache diff --git a/arch/m68k/include/asm/io_mm.h b/arch/m68k/include/asm/io_mm.h index 618c85d3c786..aaf1009f2f94 100644 --- a/arch/m68k/include/asm/io_mm.h +++ b/arch/m68k/include/asm/io_mm.h @@ -21,6 +21,7 @@ #ifdef __KERNEL__ #define ARCH_HAS_IOREMAP_WT +#define ARCH_HAS_IOREMAP_CACHE #include #include @@ -478,6 +479,12 @@ static inline void __iomem *ioremap_fullcache(unsigned long physaddr, return __ioremap(physaddr, size, IOMAP_FULL_CACHING); } +static inline void __iomem *ioremap_cache(unsigned long physaddr, + unsigned long size) +{ + return __ioremap(physaddr, size, IOMAP_FULL_CACHING); +} + static inline void memset_io(volatile void __iomem *addr, unsigned char val, int count) { __builtin_memset((void __force *) addr, val, count); diff --git a/arch/m68k/include/asm/io_no.h b/arch/m68k/include/asm/io_no.h index ad7bd40e6742..020483566b73 100644 --- a/arch/m68k/include/asm/io_no.h +++ b/arch/m68k/include/asm/io_no.h @@ -4,6 +4,7 @@ #ifdef __KERNEL__ #define ARCH_HAS_IOREMAP_WT +#define ARCH_HAS_IOREMAP_CACHE #include #include @@ -163,6 +164,10 @@ static inline void *ioremap_fullcache(unsigned long physaddr, unsigned long size { return __ioremap(physaddr, size, IOMAP_FULL_CACHING); } +static inline void *ioremap_cache(unsigned long physaddr, unsigned long size) +{ + return __ioremap(physaddr, size, IOMAP_FULL_CACHING); +} #define iounmap(addr) do { } while(0) diff --git a/arch/metag/include/asm/io.h b/arch/metag/include/asm/io.h index 9890f21eadbe..d9b2873e3ca8 100644 --- a/arch/metag/include/asm/io.h +++ b/arch/metag/include/asm/io.h @@ -1,6 +1,8 @@ #ifndef _ASM_METAG_IO_H #define _ASM_METAG_IO_H +#define ARCH_HAS_IOREMAP_CACHE + #include #include @@ -157,6 +159,9 @@ extern void __iounmap(void __iomem *addr); #define ioremap_cached(offset, size) \ __ioremap((offset), (size), _PAGE_CACHEABLE) +#define ioremap_cache(offset, size) \ + __ioremap((offset), (size), _PAGE_CACHEABLE) + #define ioremap_wc(offset, size) \ __ioremap((offset), (size), _PAGE_WR_COMBINE) diff --git a/arch/microblaze/include/asm/io.h b/arch/microblaze/include/asm/io.h index 39b6315db82e..986cc0c9e67f 100644 --- a/arch/microblaze/include/asm/io.h +++ b/arch/microblaze/include/asm/io.h @@ -43,6 +43,7 @@ extern void __iomem *ioremap(phys_addr_t address, unsigned long size); #define ioremap_fullcache(addr, size) ioremap((addr), (size)) #define ioremap_wc(addr, size) ioremap((addr), (size)) #define ioremap_wt(addr, size) ioremap((addr), (size)) +#define ioremap_cache(addr, size) ioremap((addr), (size)) #endif /* CONFIG_MMU */ diff --git a/arch/mips/include/asm/io.h b/arch/mips/include/asm/io.h index 9e777cd42b67..6d4c3ae146a5 100644 --- a/arch/mips/include/asm/io.h +++ b/arch/mips/include/asm/io.h @@ -12,6 +12,8 @@ #ifndef _ASM_IO_H #define _ASM_IO_H +#define ARCH_HAS_IOREMAP_CACHE + #include #include #include @@ -232,8 +234,10 @@ static inline void __iomem * __ioremap_mode(phys_addr_t offset, unsigned long si * address is not guaranteed to be usable directly as a virtual * address. */ -#define ioremap(offset, size) \ - __ioremap_mode((offset), (size), _CACHE_UNCACHED) +static inline void __iomem *ioremap(resource_size_t offset, unsigned long size) +{ + return __ioremap_mode(offset, size, _CACHE_UNCACHED); +} /* * ioremap_nocache - map bus memory into CPU space @@ -254,8 +258,7 @@ static inline void __iomem * __ioremap_mode(phys_addr_t offset, unsigned long si * It's useful if some control registers are in such an area and * write combining or read caching is not desirable: */ -#define ioremap_nocache(offset, size) \ - __ioremap_mode((offset), (size), _CACHE_UNCACHED) +#define ioremap_nocache ioremap /* * ioremap_cachable - map bus memory into CPU space @@ -272,8 +275,14 @@ static inline void __iomem * __ioremap_mode(phys_addr_t offset, unsigned long si * the CPU. Also enables full write-combining. Useful for some * memory-like regions on I/O busses. */ +extern unsigned long _page_cachable_default; #define ioremap_cachable(offset, size) \ __ioremap_mode((offset), (size), _page_cachable_default) +static inline void __iomem *ioremap_cache(resource_size_t offset, + unsigned long size) +{ + return ioremap_cachable(offset, size); +} /* * These two are MIPS specific ioremap variant. ioremap_cacheable_cow diff --git a/arch/mn10300/include/asm/io.h b/arch/mn10300/include/asm/io.h index 07c5b4a3903b..dcab414f40df 100644 --- a/arch/mn10300/include/asm/io.h +++ b/arch/mn10300/include/asm/io.h @@ -283,6 +283,7 @@ static inline void __iomem *ioremap_nocache(unsigned long offset, unsigned long #define ioremap_wc ioremap_nocache #define ioremap_wt ioremap_nocache +#define ioremap_cache ioremap_nocache static inline void iounmap(void __iomem *addr) { diff --git a/arch/nios2/include/asm/io.h b/arch/nios2/include/asm/io.h index c5a62da22cd2..367e2ea7663a 100644 --- a/arch/nios2/include/asm/io.h +++ b/arch/nios2/include/asm/io.h @@ -47,6 +47,7 @@ static inline void iounmap(void __iomem *addr) #define ioremap_wc ioremap_nocache #define ioremap_wt ioremap_nocache +#define ioremap_cache ioremap_nocache /* Pages to physical address... */ #define page_to_phys(page) virt_to_phys(page_to_virt(page)) diff --git a/arch/s390/include/asm/io.h b/arch/s390/include/asm/io.h index cb5fdf3a78fc..6824c3daa2a1 100644 --- a/arch/s390/include/asm/io.h +++ b/arch/s390/include/asm/io.h @@ -30,6 +30,7 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr); #define ioremap_nocache(addr, size) ioremap(addr, size) #define ioremap_wc ioremap_nocache #define ioremap_wt ioremap_nocache +#define ioremap_cache ioremap_nocache static inline void __iomem *ioremap(unsigned long offset, unsigned long size) { diff --git a/arch/sparc/include/asm/io_32.h b/arch/sparc/include/asm/io_32.h index 57f26c398dc9..b9a734caf57d 100644 --- a/arch/sparc/include/asm/io_32.h +++ b/arch/sparc/include/asm/io_32.h @@ -128,6 +128,7 @@ static inline void sbus_memcpy_toio(volatile void __iomem *dst, */ void __iomem *ioremap(unsigned long offset, unsigned long size); #define ioremap_nocache(X,Y) ioremap((X),(Y)) +#define ioremap_cache(X,Y) ioremap((X),(Y)) #define ioremap_wc(X,Y) ioremap((X),(Y)) #define ioremap_wt(X,Y) ioremap((X),(Y)) void iounmap(volatile void __iomem *addr); diff --git a/arch/sparc/include/asm/io_64.h b/arch/sparc/include/asm/io_64.h index b99ae1fac174..61f5d04da892 100644 --- a/arch/sparc/include/asm/io_64.h +++ b/arch/sparc/include/asm/io_64.h @@ -401,6 +401,7 @@ static inline void __iomem *ioremap(resource_size_t offset, unsigned long size) } #define ioremap_nocache ioremap +#define ioremap_cache ioremap #define ioremap_wc ioremap #define ioremap_wt ioremap diff --git a/arch/tile/include/asm/io.h b/arch/tile/include/asm/io.h index dc61de15c1f9..fe853a135e25 100644 --- a/arch/tile/include/asm/io.h +++ b/arch/tile/include/asm/io.h @@ -53,6 +53,7 @@ extern void iounmap(volatile void __iomem *addr); #endif #define ioremap_nocache(physaddr, size) ioremap(physaddr, size) +#define ioremap_cache(physaddr, size) ioremap(physaddr, size) #define ioremap_wc(physaddr, size) ioremap(physaddr, size) #define ioremap_wt(physaddr, size) ioremap(physaddr, size) #define ioremap_fullcache(physaddr, size) ioremap(physaddr, size) diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h index 83ec9b1d77cc..e9d6691ec4c5 100644 --- a/arch/x86/include/asm/io.h +++ b/arch/x86/include/asm/io.h @@ -36,6 +36,7 @@ #define ARCH_HAS_IOREMAP_WC #define ARCH_HAS_IOREMAP_WT +#define ARCH_HAS_IOREMAP_CACHE #include #include diff --git a/arch/xtensa/include/asm/io.h b/arch/xtensa/include/asm/io.h index c39bb6e61911..f91a8a99aa29 100644 --- a/arch/xtensa/include/asm/io.h +++ b/arch/xtensa/include/asm/io.h @@ -12,6 +12,9 @@ #define _XTENSA_IO_H #ifdef __KERNEL__ + +#define ARCH_HAS_IOREMAP_CACHE + #include #include #include diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h index f56094cfdeff..a0665dfcab47 100644 --- a/include/asm-generic/io.h +++ b/include/asm-generic/io.h @@ -793,6 +793,14 @@ static inline void __iomem *ioremap_wt(phys_addr_t offset, size_t size) } #endif +#ifndef ioremap_cache +#define ioremap_cache ioremap_cache +static inline void __iomem *ioremap_cache(phys_addr_t offset, size_t size) +{ + return ioremap_nocache(offset, size); +} +#endif + #ifndef iounmap #define iounmap iounmap diff --git a/include/asm-generic/iomap.h b/include/asm-generic/iomap.h index d8f8622fa044..f0f30464cecd 100644 --- a/include/asm-generic/iomap.h +++ b/include/asm-generic/iomap.h @@ -70,6 +70,10 @@ extern void ioport_unmap(void __iomem *); #define ioremap_wt ioremap_nocache #endif +#ifndef ARCH_HAS_IOREMAP_CACHE +#define ioremap_cache ioremap_nocache +#endif + #ifdef CONFIG_PCI /* Destroy a virtual mapping cookie for a PCI BAR (memory or IO) */ struct pci_dev; From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755391AbbFKVXF (ORCPT ); Thu, 11 Jun 2015 17:23:05 -0400 Received: from mga14.intel.com ([192.55.52.115]:34115 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755179AbbFKVWS (ORCPT ); Thu, 11 Jun 2015 17:22:18 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,597,1427785200"; d="scan'208";a="709467720" Subject: [PATCH v4 4/6] devm: fix ioremap_cache() usage From: Dan Williams To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@ml01.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de Date: Thu, 11 Jun 2015 17:19:36 -0400 Message-ID: <20150611211936.10271.3169.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-8-g92dd MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Provide devm_ioremap_cache() and fix up devm_ioremap_resource() to actually provide cacheable mappings. On archs that implement ioremap_cache() devm_ioremap_resource() is always silently falling back to uncached when IORESOURCE_CACHEABLE is specified. Cc: Toshi Kani Cc: Arnd Bergmann Signed-off-by: Dan Williams --- include/linux/io.h | 2 ++ lib/devres.c | 53 +++++++++++++++++++++++++--------------------------- 2 files changed, 27 insertions(+), 28 deletions(-) diff --git a/include/linux/io.h b/include/linux/io.h index fb5a99800e77..4cd8996cfea0 100644 --- a/include/linux/io.h +++ b/include/linux/io.h @@ -71,6 +71,8 @@ static inline void devm_ioport_unmap(struct device *dev, void __iomem *addr) void __iomem *devm_ioremap(struct device *dev, resource_size_t offset, resource_size_t size); +void __iomem *devm_ioremap_cache(struct device *dev, resource_size_t offset, + resource_size_t size); void __iomem *devm_ioremap_nocache(struct device *dev, resource_size_t offset, resource_size_t size); void __iomem *devm_ioremap_wc(struct device *dev, resource_size_t offset, diff --git a/lib/devres.c b/lib/devres.c index f4001d90d24d..c8e75cdaf816 100644 --- a/lib/devres.c +++ b/lib/devres.c @@ -14,6 +14,8 @@ static int devm_ioremap_match(struct device *dev, void *res, void *match_data) return *(void **)res == match_data; } +typedef void __iomem *(*ioremap_fn)(resource_size_t offset, unsigned long size); + /** * devm_ioremap - Managed ioremap() * @dev: Generic device to remap IO address for @@ -22,8 +24,9 @@ static int devm_ioremap_match(struct device *dev, void *res, void *match_data) * * Managed ioremap(). Map is automatically unmapped on driver detach. */ -void __iomem *devm_ioremap(struct device *dev, resource_size_t offset, - resource_size_t size) +static void __iomem *devm_ioremap_type(struct device *dev, + resource_size_t offset, resource_size_t size, + ioremap_fn ioremap_type) { void __iomem **ptr, *addr; @@ -31,7 +34,7 @@ void __iomem *devm_ioremap(struct device *dev, resource_size_t offset, if (!ptr) return NULL; - addr = ioremap(offset, size); + addr = ioremap_type(offset, size); if (addr) { *ptr = addr; devres_add(dev, ptr); @@ -40,34 +43,25 @@ void __iomem *devm_ioremap(struct device *dev, resource_size_t offset, return addr; } + +void __iomem *devm_ioremap(struct device *dev, resource_size_t offset, + resource_size_t size) +{ + return devm_ioremap_type(dev, offset, size, ioremap); +} EXPORT_SYMBOL(devm_ioremap); -/** - * devm_ioremap_nocache - Managed ioremap_nocache() - * @dev: Generic device to remap IO address for - * @offset: BUS offset to map - * @size: Size of map - * - * Managed ioremap_nocache(). Map is automatically unmapped on driver - * detach. - */ +void __iomem *devm_ioremap_cache(struct device *dev, resource_size_t offset, + resource_size_t size) +{ + return devm_ioremap_type(dev, offset, size, ioremap_cache); +} +EXPORT_SYMBOL(devm_ioremap_cache); + void __iomem *devm_ioremap_nocache(struct device *dev, resource_size_t offset, resource_size_t size) { - void __iomem **ptr, *addr; - - ptr = devres_alloc(devm_ioremap_release, sizeof(*ptr), GFP_KERNEL); - if (!ptr) - return NULL; - - addr = ioremap_nocache(offset, size); - if (addr) { - *ptr = addr; - devres_add(dev, ptr); - } else - devres_free(ptr); - - return addr; + return devm_ioremap_type(dev, offset, size, ioremap_nocache); } EXPORT_SYMBOL(devm_ioremap_nocache); @@ -153,8 +147,11 @@ void __iomem *devm_ioremap_resource(struct device *dev, struct resource *res) return IOMEM_ERR_PTR(-EBUSY); } - /* FIXME: add devm_ioremap_cache support */ - dest_ptr = devm_ioremap(dev, res->start, size); + if (res->flags & IORESOURCE_CACHEABLE) + dest_ptr = devm_ioremap_cache(dev, res->start, size); + else + dest_ptr = devm_ioremap_nocache(dev, res->start, size); + if (!dest_ptr) { dev_err(dev, "ioremap failed for resource %pR\n", res); devm_release_mem_region(dev, res->start, size); From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755303AbbFKVW1 (ORCPT ); Thu, 11 Jun 2015 17:22:27 -0400 Received: from mga09.intel.com ([134.134.136.24]:62514 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755097AbbFKVWX (ORCPT ); Thu, 11 Jun 2015 17:22:23 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,597,1427785200"; d="scan'208";a="741745863" Subject: [PATCH v4 5/6] arch: introduce memremap_cache() and memremap_wt() From: Dan Williams To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@ml01.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, Andy Shevchenko , geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de Date: Thu, 11 Jun 2015 17:19:41 -0400 Message-ID: <20150611211941.10271.10513.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-8-g92dd MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Existing users of ioremap_cache() are mapping memory that is known in advance to not have i/o side effects. These users are forced to cast away the __iomem annotation, or otherwise neglect to fix the sparse errors thrown when dereferencing pointers to this memory. Provide memremap_*() as a non __iomem annotated ioremap_*(). The ARCH_HAS_MEMREMAP kconfig symbol is introduced for archs to assert that it is safe to recast / reuse the return value from ioremap as a normal pointer to memory. In other words, archs that mandate specific accessors for __iomem are not memremap() capable and drivers that care, like pmem, can add a dependency to disable themselves on these archs. Cc: Arnd Bergmann Acked-by: Andy Shevchenko Signed-off-by: Dan Williams --- arch/arm/Kconfig | 1 + arch/arm64/Kconfig | 1 + arch/arm64/kernel/efi.c | 4 ++- arch/arm64/kernel/smp_spin_table.c | 10 ++++---- arch/frv/Kconfig | 1 + arch/m68k/Kconfig | 1 + arch/metag/Kconfig | 1 + arch/mips/Kconfig | 1 + arch/powerpc/Kconfig | 1 + arch/x86/Kconfig | 1 + arch/x86/kernel/crash_dump_64.c | 6 ++--- arch/x86/kernel/kdebugfs.c | 8 +++---- arch/x86/kernel/ksysfs.c | 28 ++++++++++++----------- arch/x86/mm/ioremap.c | 10 +++----- arch/xtensa/Kconfig | 1 + drivers/acpi/apei/einj.c | 8 +++---- drivers/acpi/apei/erst.c | 4 ++- drivers/block/Kconfig | 1 + drivers/block/pmem.c | 7 +++--- drivers/firmware/google/memconsole.c | 4 ++- include/linux/device.h | 5 ++++ include/linux/io.h | 4 +++ kernel/resource.c | 41 +++++++++++++++++++++++++++++++++- lib/Kconfig | 5 +++- 24 files changed, 107 insertions(+), 47 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 45df48ba0b12..397426f8ca37 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -3,6 +3,7 @@ config ARM default y select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE select ARCH_HAS_ELF_RANDOMIZE + select ARCH_HAS_MEMREMAP select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST select ARCH_HAVE_CUSTOM_GPIO_H select ARCH_HAS_GCOV_PROFILE_ALL diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7796af4b1d6f..f07a9a5af61e 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -5,6 +5,7 @@ config ARM64 select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE select ARCH_HAS_ELF_RANDOMIZE select ARCH_HAS_GCOV_PROFILE_ALL + select ARCH_HAS_MEMREMAP select ARCH_HAS_SG_CHAIN select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST select ARCH_USE_CMPXCHG_LOCKREF diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c index ab21e0d58278..b672ef33f08b 100644 --- a/arch/arm64/kernel/efi.c +++ b/arch/arm64/kernel/efi.c @@ -289,7 +289,7 @@ static int __init arm64_enable_runtime_services(void) pr_info("Remapping and enabling EFI services.\n"); mapsize = memmap.map_end - memmap.map; - memmap.map = (__force void *)ioremap_cache((phys_addr_t)memmap.phys_map, + memmap.map = memremap_cache((phys_addr_t)memmap.phys_map, mapsize); if (!memmap.map) { pr_err("Failed to remap EFI memory map\n"); @@ -298,7 +298,7 @@ static int __init arm64_enable_runtime_services(void) memmap.map_end = memmap.map + mapsize; efi.memmap = &memmap; - efi.systab = (__force void *)ioremap_cache(efi_system_table, + efi.systab = memremap_cache(efi_system_table, sizeof(efi_system_table_t)); if (!efi.systab) { pr_err("Failed to remap EFI System Table\n"); diff --git a/arch/arm64/kernel/smp_spin_table.c b/arch/arm64/kernel/smp_spin_table.c index 14944e5b28da..893c8586e20f 100644 --- a/arch/arm64/kernel/smp_spin_table.c +++ b/arch/arm64/kernel/smp_spin_table.c @@ -67,18 +67,18 @@ static int smp_spin_table_cpu_init(struct device_node *dn, unsigned int cpu) static int smp_spin_table_cpu_prepare(unsigned int cpu) { - __le64 __iomem *release_addr; + __le64 *release_addr; if (!cpu_release_addr[cpu]) return -ENODEV; /* * The cpu-release-addr may or may not be inside the linear mapping. - * As ioremap_cache will either give us a new mapping or reuse the + * As memremap_cache will either give us a new mapping or reuse the * existing linear mapping, we can use it to cover both cases. In * either case the memory will be MT_NORMAL. */ - release_addr = ioremap_cache(cpu_release_addr[cpu], + release_addr = memremap_cache(cpu_release_addr[cpu], sizeof(*release_addr)); if (!release_addr) return -ENOMEM; @@ -91,7 +91,7 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) * the boot protocol. */ writeq_relaxed(__pa(secondary_holding_pen), release_addr); - __flush_dcache_area((__force void *)release_addr, + __flush_dcache_area(release_addr, sizeof(*release_addr)); /* @@ -99,7 +99,7 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) */ sev(); - iounmap(release_addr); + memunmap(release_addr); return 0; } diff --git a/arch/frv/Kconfig b/arch/frv/Kconfig index 34aa19352dc1..2373bf183527 100644 --- a/arch/frv/Kconfig +++ b/arch/frv/Kconfig @@ -14,6 +14,7 @@ config FRV select OLD_SIGSUSPEND3 select OLD_SIGACTION select HAVE_DEBUG_STACKOVERFLOW + select ARCH_HAS_MEMREMAP config ZONE_DMA bool diff --git a/arch/m68k/Kconfig b/arch/m68k/Kconfig index 2dd8f63bfbbb..831b1be8c43d 100644 --- a/arch/m68k/Kconfig +++ b/arch/m68k/Kconfig @@ -23,6 +23,7 @@ config M68K select MODULES_USE_ELF_RELA select OLD_SIGSUSPEND3 select OLD_SIGACTION + select ARCH_HAS_MEMREMAP config RWSEM_GENERIC_SPINLOCK bool diff --git a/arch/metag/Kconfig b/arch/metag/Kconfig index 0b389a81c43a..5669fe3eb807 100644 --- a/arch/metag/Kconfig +++ b/arch/metag/Kconfig @@ -24,6 +24,7 @@ config METAG select HAVE_PERF_EVENTS select HAVE_SYSCALL_TRACEPOINTS select HAVE_UNDERSCORE_SYMBOL_PREFIX + select ARCH_HAS_MEMREMAP select IRQ_DOMAIN select MODULES_USE_ELF_RELA select OF diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig index f5016656494f..9ee35e615c0d 100644 --- a/arch/mips/Kconfig +++ b/arch/mips/Kconfig @@ -58,6 +58,7 @@ config MIPS select SYSCTL_EXCEPTION_TRACE select HAVE_VIRT_CPU_ACCOUNTING_GEN select HAVE_IRQ_TIME_ACCOUNTING + select ARCH_HAS_MEMREMAP menu "Machine selection" diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 190cc48abc0c..73c1f8b1f022 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -153,6 +153,7 @@ config PPC select NO_BOOTMEM select HAVE_GENERIC_RCU_GUP select HAVE_PERF_EVENTS_NMI if PPC64 + select ARCH_HAS_MEMREMAP config GENERIC_CSUM def_bool CPU_LITTLE_ENDIAN diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 228aa35d7e89..f16caf7eac27 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -27,6 +27,7 @@ config X86 select ARCH_HAS_ELF_RANDOMIZE select ARCH_HAS_FAST_MULTIPLIER select ARCH_HAS_GCOV_PROFILE_ALL + select ARCH_HAS_MEMREMAP select ARCH_HAS_SG_CHAIN select ARCH_HAVE_NMI_SAFE_CMPXCHG select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI diff --git a/arch/x86/kernel/crash_dump_64.c b/arch/x86/kernel/crash_dump_64.c index afa64adb75ee..8e04011665fd 100644 --- a/arch/x86/kernel/crash_dump_64.c +++ b/arch/x86/kernel/crash_dump_64.c @@ -31,19 +31,19 @@ ssize_t copy_oldmem_page(unsigned long pfn, char *buf, if (!csize) return 0; - vaddr = ioremap_cache(pfn << PAGE_SHIFT, PAGE_SIZE); + vaddr = memremap_cache(pfn << PAGE_SHIFT, PAGE_SIZE); if (!vaddr) return -ENOMEM; if (userbuf) { if (copy_to_user(buf, vaddr + offset, csize)) { - iounmap(vaddr); + memunmap(vaddr); return -EFAULT; } } else memcpy(buf, vaddr + offset, csize); set_iounmap_nonlazy(); - iounmap(vaddr); + memunmap(vaddr); return csize; } diff --git a/arch/x86/kernel/kdebugfs.c b/arch/x86/kernel/kdebugfs.c index dc1404bf8e4b..731b10e2814f 100644 --- a/arch/x86/kernel/kdebugfs.c +++ b/arch/x86/kernel/kdebugfs.c @@ -49,7 +49,7 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf, pa = node->paddr + sizeof(struct setup_data) + pos; pg = pfn_to_page((pa + count - 1) >> PAGE_SHIFT); if (PageHighMem(pg)) { - p = ioremap_cache(pa, count); + p = memremap_cache(pa, count); if (!p) return -ENXIO; } else @@ -58,7 +58,7 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf, remain = copy_to_user(user_buf, p, count); if (PageHighMem(pg)) - iounmap(p); + memunmap(p); if (remain) return -EFAULT; @@ -128,7 +128,7 @@ static int __init create_setup_data_nodes(struct dentry *parent) pg = pfn_to_page((pa_data+sizeof(*data)-1) >> PAGE_SHIFT); if (PageHighMem(pg)) { - data = ioremap_cache(pa_data, sizeof(*data)); + data = memremap_cache(pa_data, sizeof(*data)); if (!data) { kfree(node); error = -ENXIO; @@ -144,7 +144,7 @@ static int __init create_setup_data_nodes(struct dentry *parent) pa_data = data->next; if (PageHighMem(pg)) - iounmap(data); + memunmap(data); if (error) goto err_dir; no++; diff --git a/arch/x86/kernel/ksysfs.c b/arch/x86/kernel/ksysfs.c index c2bedaea11f7..2fbc62886eae 100644 --- a/arch/x86/kernel/ksysfs.c +++ b/arch/x86/kernel/ksysfs.c @@ -16,8 +16,8 @@ #include #include #include +#include -#include #include static ssize_t version_show(struct kobject *kobj, @@ -79,12 +79,12 @@ static int get_setup_data_paddr(int nr, u64 *paddr) *paddr = pa_data; return 0; } - data = ioremap_cache(pa_data, sizeof(*data)); + data = memremap_cache(pa_data, sizeof(*data)); if (!data) return -ENOMEM; pa_data = data->next; - iounmap(data); + memunmap(data); i++; } return -EINVAL; @@ -97,17 +97,17 @@ static int __init get_setup_data_size(int nr, size_t *size) u64 pa_data = boot_params.hdr.setup_data; while (pa_data) { - data = ioremap_cache(pa_data, sizeof(*data)); + data = memremap_cache(pa_data, sizeof(*data)); if (!data) return -ENOMEM; if (nr == i) { *size = data->len; - iounmap(data); + memunmap(data); return 0; } pa_data = data->next; - iounmap(data); + memunmap(data); i++; } return -EINVAL; @@ -127,12 +127,12 @@ static ssize_t type_show(struct kobject *kobj, ret = get_setup_data_paddr(nr, &paddr); if (ret) return ret; - data = ioremap_cache(paddr, sizeof(*data)); + data = memremap_cache(paddr, sizeof(*data)); if (!data) return -ENOMEM; ret = sprintf(buf, "0x%x\n", data->type); - iounmap(data); + memunmap(data); return ret; } @@ -154,7 +154,7 @@ static ssize_t setup_data_data_read(struct file *fp, ret = get_setup_data_paddr(nr, &paddr); if (ret) return ret; - data = ioremap_cache(paddr, sizeof(*data)); + data = memremap_cache(paddr, sizeof(*data)); if (!data) return -ENOMEM; @@ -170,15 +170,15 @@ static ssize_t setup_data_data_read(struct file *fp, goto out; ret = count; - p = ioremap_cache(paddr + sizeof(*data), data->len); + p = memremap_cache(paddr + sizeof(*data), data->len); if (!p) { ret = -ENOMEM; goto out; } memcpy(buf, p + off, count); - iounmap(p); + memunmap(p); out: - iounmap(data); + memunmap(data); return ret; } @@ -250,13 +250,13 @@ static int __init get_setup_data_total_num(u64 pa_data, int *nr) *nr = 0; while (pa_data) { *nr += 1; - data = ioremap_cache(pa_data, sizeof(*data)); + data = memremap_cache(pa_data, sizeof(*data)); if (!data) { ret = -ENOMEM; goto out; } pa_data = data->next; - iounmap(data); + memunmap(data); } out: diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index cc5ccc415cc0..7f087cb793fa 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -414,12 +414,10 @@ void *xlate_dev_mem_ptr(phys_addr_t phys) if (page_is_ram(start >> PAGE_SHIFT)) return __va(phys); - vaddr = ioremap_cache(start, PAGE_SIZE); - /* Only add the offset on success and return NULL if the ioremap() failed: */ + vaddr = memremap_cache(start, PAGE_SIZE); if (vaddr) - vaddr += offset; - - return vaddr; + return vaddr + offset; + return NULL; } void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr) @@ -427,7 +425,7 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr) if (page_is_ram(phys >> PAGE_SHIFT)) return; - iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK)); + memunmap((void *)((unsigned long)addr & PAGE_MASK)); } static pte_t bm_pte[PAGE_SIZE/sizeof(pte_t)] __page_aligned_bss; diff --git a/arch/xtensa/Kconfig b/arch/xtensa/Kconfig index 87be10e8b57a..e601faf87cee 100644 --- a/arch/xtensa/Kconfig +++ b/arch/xtensa/Kconfig @@ -3,6 +3,7 @@ config ZONE_DMA config XTENSA def_bool y + select ARCH_HAS_MEMREMAP select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_IPC_PARSE_VERSION select ARCH_WANT_OPTIONAL_GPIOLIB diff --git a/drivers/acpi/apei/einj.c b/drivers/acpi/apei/einj.c index a095d4f858da..2ec9006cfb6c 100644 --- a/drivers/acpi/apei/einj.c +++ b/drivers/acpi/apei/einj.c @@ -318,7 +318,7 @@ static int __einj_error_trigger(u64 trigger_paddr, u32 type, sizeof(*trigger_tab) - 1); goto out; } - trigger_tab = ioremap_cache(trigger_paddr, sizeof(*trigger_tab)); + trigger_tab = memremap_cache(trigger_paddr, sizeof(*trigger_tab)); if (!trigger_tab) { pr_err(EINJ_PFX "Failed to map trigger table!\n"); goto out_rel_header; @@ -346,8 +346,8 @@ static int __einj_error_trigger(u64 trigger_paddr, u32 type, (unsigned long long)trigger_paddr + table_size - 1); goto out_rel_header; } - iounmap(trigger_tab); - trigger_tab = ioremap_cache(trigger_paddr, table_size); + memunmap(trigger_tab); + trigger_tab = memremap_cache(trigger_paddr, table_size); if (!trigger_tab) { pr_err(EINJ_PFX "Failed to map trigger table!\n"); goto out_rel_entry; @@ -409,7 +409,7 @@ out_rel_header: release_mem_region(trigger_paddr, sizeof(*trigger_tab)); out: if (trigger_tab) - iounmap(trigger_tab); + memunmap(trigger_tab); return rc; } diff --git a/drivers/acpi/apei/erst.c b/drivers/acpi/apei/erst.c index 3670bbab57a3..6b95066da51d 100644 --- a/drivers/acpi/apei/erst.c +++ b/drivers/acpi/apei/erst.c @@ -77,7 +77,7 @@ static struct acpi_table_erst *erst_tab; static struct erst_erange { u64 base; u64 size; - void __iomem *vaddr; + void *vaddr; u32 attr; } erst_erange; @@ -1185,7 +1185,7 @@ static int __init erst_init(void) goto err_unmap_reg; } rc = -ENOMEM; - erst_erange.vaddr = ioremap_cache(erst_erange.base, + erst_erange.vaddr = memremap_cache(erst_erange.base, erst_erange.size); if (!erst_erange.vaddr) goto err_release_erange; diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig index eb1fed5bd516..98418fc330ae 100644 --- a/drivers/block/Kconfig +++ b/drivers/block/Kconfig @@ -406,6 +406,7 @@ config BLK_DEV_RAM_DAX config BLK_DEV_PMEM tristate "Persistent memory block device support" + depends on ARCH_HAS_MEMREMAP help Saying Y here will allow you to use a contiguous range of reserved memory as one or more persistent block devices. diff --git a/drivers/block/pmem.c b/drivers/block/pmem.c index 095dfaadcaa5..b00b97314b57 100644 --- a/drivers/block/pmem.c +++ b/drivers/block/pmem.c @@ -23,6 +23,7 @@ #include #include #include +#include #define PMEM_MINORS 16 @@ -143,7 +144,7 @@ static struct pmem_device *pmem_alloc(struct device *dev, struct resource *res) * of the CPU caches in case of a crash. */ err = -ENOMEM; - pmem->virt_addr = ioremap_wt(pmem->phys_addr, pmem->size); + pmem->virt_addr = memremap_wt(pmem->phys_addr, pmem->size); if (!pmem->virt_addr) goto out_release_region; @@ -179,7 +180,7 @@ static struct pmem_device *pmem_alloc(struct device *dev, struct resource *res) out_free_queue: blk_cleanup_queue(pmem->pmem_queue); out_unmap: - iounmap(pmem->virt_addr); + memunmap(pmem->virt_addr); out_release_region: release_mem_region(pmem->phys_addr, pmem->size); out_free_dev: @@ -193,7 +194,7 @@ static void pmem_free(struct pmem_device *pmem) del_gendisk(pmem->pmem_disk); put_disk(pmem->pmem_disk); blk_cleanup_queue(pmem->pmem_queue); - iounmap(pmem->virt_addr); + memunmap(pmem->virt_addr); release_mem_region(pmem->phys_addr, pmem->size); kfree(pmem); } diff --git a/drivers/firmware/google/memconsole.c b/drivers/firmware/google/memconsole.c index 2f569aaed4c7..877433dc8297 100644 --- a/drivers/firmware/google/memconsole.c +++ b/drivers/firmware/google/memconsole.c @@ -52,14 +52,14 @@ static ssize_t memconsole_read(struct file *filp, struct kobject *kobp, char *memconsole; ssize_t ret; - memconsole = ioremap_cache(memconsole_baseaddr, memconsole_length); + memconsole = memremap_cache(memconsole_baseaddr, memconsole_length); if (!memconsole) { pr_err("memconsole: ioremap_cache failed\n"); return -ENOMEM; } ret = memory_read_from_buffer(buf, count, &pos, memconsole, memconsole_length); - iounmap(memconsole); + memunmap(memconsole); return ret; } diff --git a/include/linux/device.h b/include/linux/device.h index 6558af90c8fe..518f49c5d596 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -638,6 +638,11 @@ extern void devm_free_pages(struct device *dev, unsigned long addr); void __iomem *devm_ioremap_resource(struct device *dev, struct resource *res); +static inline void *devm_memremap_resource(struct device *dev, struct resource *res) +{ + return (void __force *) devm_ioremap_resource(dev, res); +} + /* allows to add/remove a custom action to devres stack */ int devm_add_action(struct device *dev, void (*action)(void *), void *data); void devm_remove_action(struct device *dev, void (*action)(void *), void *data); diff --git a/include/linux/io.h b/include/linux/io.h index 4cd8996cfea0..23ef87d311d5 100644 --- a/include/linux/io.h +++ b/include/linux/io.h @@ -123,4 +123,8 @@ static inline int arch_phys_wc_index(int handle) #endif #endif +extern void *memremap_cache(resource_size_t offset, size_t size); +extern void *memremap_wt(resource_size_t offset, size_t size); +extern void memunmap(void *addr); + #endif /* _LINUX_IO_H */ diff --git a/kernel/resource.c b/kernel/resource.c index 90552aab5f2d..2f8aca09da52 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -23,7 +23,7 @@ #include #include #include -#include +#include struct resource ioport_resource = { @@ -528,6 +528,45 @@ int region_is_ram(resource_size_t start, unsigned long size) return ret; } +#ifdef CONFIG_ARCH_HAS_MEMREMAP +/* + * memremap() is "ioremap" for cases where it is known that the resource + * being mapped does not have i/o side effects and the __iomem + * annotation is not applicable. + */ +static bool memremap_valid(resource_size_t offset, size_t size) +{ + if (region_is_ram(offset, size) != 0) { + WARN_ONCE(1, "memremap attempted on ram %pa size: %zu\n", + &offset, size); + return false; + } + return true; +} + +void *memremap_cache(resource_size_t offset, size_t size) +{ + if (!memremap_valid(offset, size)) + return NULL; + return (void __force *) ioremap_cache(offset, size); +} +EXPORT_SYMBOL(memremap_cache); + +void *memremap_wt(resource_size_t offset, size_t size) +{ + if (!memremap_valid(offset, size)) + return NULL; + return (void __force *) ioremap_wt(offset, size); +} +EXPORT_SYMBOL(memremap_wt); + +void memunmap(void *addr) +{ + iounmap((void __iomem __force *) addr); +} +EXPORT_SYMBOL(memunmap); +#endif /* CONFIG_ARCH_HAS_MEMREMAP */ + void __weak arch_remove_reservations(struct resource *avail) { } diff --git a/lib/Kconfig b/lib/Kconfig index 601965a948e8..bc7bc0278921 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -520,6 +520,9 @@ source "lib/fonts/Kconfig" # config ARCH_HAS_SG_CHAIN - def_bool n + bool + +config ARCH_HAS_MEMREMAP + bool endmenu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755356AbbFKVWe (ORCPT ); Thu, 11 Jun 2015 17:22:34 -0400 Received: from mga09.intel.com ([134.134.136.24]:62514 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754933AbbFKVW3 (ORCPT ); Thu, 11 Jun 2015 17:22:29 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,597,1427785200"; d="scan'208";a="725629091" Subject: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates From: Dan Williams To: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org Cc: jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@ml01.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de Date: Thu, 11 Jun 2015 17:19:47 -0400 Message-ID: <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-8-g92dd MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Ross Zwisler Based on an original patch by Ross Zwisler [1]. Writes to persistent memory have the potential to be posted to cpu cache, cpu write buffers, and platform write buffers (memory controller) before being committed to persistent media. Provide apis, memcpy_to_pmem(), sync_pmem(), and memremap_pmem(), to write data to pmem and assert that it is durable in PMEM (a persistent linear address range). A '__pmem' attribute is added so sparse can track proper usage of pointers to pmem. [1]: https://lists.01.org/pipermail/linux-nvdimm/2015-May/000932.html Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Signed-off-by: Ross Zwisler [djbw: various reworks] Signed-off-by: Dan Williams --- arch/x86/Kconfig | 1 arch/x86/include/asm/cacheflush.h | 36 +++++++++++++ arch/x86/include/asm/io.h | 6 ++ drivers/block/pmem.c | 75 +++++++++++++++++++++++++-- include/linux/compiler.h | 2 + include/linux/pmem.h | 102 +++++++++++++++++++++++++++++++++++++ lib/Kconfig | 3 + 7 files changed, 218 insertions(+), 7 deletions(-) create mode 100644 include/linux/pmem.h diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f16caf7eac27..5dfb8f31ac48 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -28,6 +28,7 @@ config X86 select ARCH_HAS_FAST_MULTIPLIER select ARCH_HAS_GCOV_PROFILE_ALL select ARCH_HAS_MEMREMAP + select ARCH_HAS_PMEM_API select ARCH_HAS_SG_CHAIN select ARCH_HAVE_NMI_SAFE_CMPXCHG select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI diff --git a/arch/x86/include/asm/cacheflush.h b/arch/x86/include/asm/cacheflush.h index b6f7457d12e4..4d896487382c 100644 --- a/arch/x86/include/asm/cacheflush.h +++ b/arch/x86/include/asm/cacheflush.h @@ -4,6 +4,7 @@ /* Caches aren't brain-dead on the intel. */ #include #include +#include /* * The set_memory_* API can be used to change various attributes of a virtual @@ -108,4 +109,39 @@ static inline int rodata_test(void) } #endif +#ifdef ARCH_HAS_NOCACHE_UACCESS +static inline void arch_memcpy_to_pmem(void __pmem *dst, const void *src, size_t n) +{ + /* + * We are copying between two kernel buffers, if + * __copy_from_user_inatomic_nocache() returns an error (page + * fault) we would have already taken an unhandled fault before + * the BUG_ON. The BUG_ON is simply here to satisfy + * __must_check and allow reuse of the common non-temporal store + * implementation for memcpy_to_pmem(). + */ + BUG_ON(__copy_from_user_inatomic_nocache((void __force *) dst, + (void __user *) src, n)); +} + +static inline void arch_sync_pmem(void) +{ + wmb(); + pcommit_sfence(); +} + +static inline bool __arch_has_sync_pmem(void) +{ + return boot_cpu_has(X86_FEATURE_PCOMMIT); +} +#else /* ARCH_HAS_NOCACHE_UACCESS i.e. ARCH=um */ +extern void arch_memcpy_to_pmem(void __pmem *dst, const void *src, size_t n); +extern void arch_sync_pmem(void); + +static inline bool __arch_has_sync_pmem(void) +{ + return false; +} +#endif + #endif /* _ASM_X86_CACHEFLUSH_H */ diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h index e9d6691ec4c5..0a494ac22a8e 100644 --- a/arch/x86/include/asm/io.h +++ b/arch/x86/include/asm/io.h @@ -249,6 +249,12 @@ static inline void flush_write_buffers(void) #endif } +static inline void __pmem *arch_memremap_pmem(resource_size_t offset, + unsigned long size) +{ + return (void __force __pmem *) ioremap_cache(offset, size); +} + #endif /* __KERNEL__ */ extern void native_io_delay(void); diff --git a/drivers/block/pmem.c b/drivers/block/pmem.c index b00b97314b57..81090f61b8b1 100644 --- a/drivers/block/pmem.c +++ b/drivers/block/pmem.c @@ -23,23 +23,79 @@ #include #include #include +#include #include #define PMEM_MINORS 16 +struct pmem_ops { + void __pmem *(*remap)(resource_size_t offset, unsigned long size); + void (*copy)(void __pmem *dst, const void *src, size_t size); + void (*sync)(void); +}; + struct pmem_device { struct request_queue *pmem_queue; struct gendisk *pmem_disk; /* One contiguous memory region per device */ phys_addr_t phys_addr; - void *virt_addr; + void __pmem *virt_addr; size_t size; + struct pmem_ops ops; }; static int pmem_major; static atomic_t pmem_index; +static void default_sync_pmem(void) +{ + wmb(); +} + +static void default_memcpy_to_pmem(void __pmem *dst, const void *src, size_t size) +{ + memcpy((void __force *) dst, src, size); +} + +static void __pmem *default_memremap_pmem(resource_size_t offset, unsigned long size) +{ + return (void __pmem *)memremap_wt(offset, size); +} + +static void pmem_ops_default_init(struct pmem_device *pmem) +{ + /* + * These defaults seek to offer decent performance and minimize + * the window between i/o completion and writes being durable on + * media. However, it is undefined / architecture specific + * whether default_memcpy_to_pmem + default_pmem_sync is + * sufficient for making data durable relative to i/o + * completion. + */ + pmem->ops.remap = default_memremap_pmem; + pmem->ops.copy = default_memcpy_to_pmem; + pmem->ops.sync = default_sync_pmem; +} + +static bool pmem_ops_init(struct pmem_device *pmem) +{ + if (IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API) && + arch_has_sync_pmem()) { + /* + * This arch + cpu guarantees that bio_endio() == data + * durable on media. + */ + pmem->ops.remap = memremap_pmem; + pmem->ops.copy = memcpy_to_pmem; + pmem->ops.sync = sync_pmem; + return true; + } + + pmem_ops_default_init(pmem); + return false; +} + static void pmem_do_bvec(struct pmem_device *pmem, struct page *page, unsigned int len, unsigned int off, int rw, sector_t sector) @@ -48,11 +104,11 @@ static void pmem_do_bvec(struct pmem_device *pmem, struct page *page, size_t pmem_off = sector << 9; if (rw == READ) { - memcpy(mem + off, pmem->virt_addr + pmem_off, len); + memcpy_from_pmem(mem + off, pmem->virt_addr + pmem_off, len); flush_dcache_page(page); } else { flush_dcache_page(page); - memcpy(pmem->virt_addr + pmem_off, mem + off, len); + pmem->ops.copy(pmem->virt_addr + pmem_off, mem + off, len); } kunmap_atomic(mem); @@ -83,6 +139,8 @@ static void pmem_make_request(struct request_queue *q, struct bio *bio) sector += bvec.bv_len >> 9; } + if (rw) + pmem->ops.sync(); out: bio_endio(bio, err); } @@ -107,7 +165,8 @@ static long pmem_direct_access(struct block_device *bdev, sector_t sector, if (!pmem) return -ENODEV; - *kaddr = pmem->virt_addr + offset; + /* FIXME convert DAX to comprehend that this mapping has a lifetime */ + *kaddr = (void __force *) pmem->virt_addr + offset; *pfn = (pmem->phys_addr + offset) >> PAGE_SHIFT; return pmem->size - offset; @@ -132,6 +191,8 @@ static struct pmem_device *pmem_alloc(struct device *dev, struct resource *res) pmem->phys_addr = res->start; pmem->size = resource_size(res); + if (!pmem_ops_init(pmem)) + dev_warn(dev, "unable to guarantee persistence of writes\n"); err = -EINVAL; if (!request_mem_region(pmem->phys_addr, pmem->size, "pmem")) { @@ -144,7 +205,7 @@ static struct pmem_device *pmem_alloc(struct device *dev, struct resource *res) * of the CPU caches in case of a crash. */ err = -ENOMEM; - pmem->virt_addr = memremap_wt(pmem->phys_addr, pmem->size); + pmem->virt_addr = pmem->ops.remap(pmem->phys_addr, pmem->size); if (!pmem->virt_addr) goto out_release_region; @@ -180,7 +241,7 @@ static struct pmem_device *pmem_alloc(struct device *dev, struct resource *res) out_free_queue: blk_cleanup_queue(pmem->pmem_queue); out_unmap: - memunmap(pmem->virt_addr); + memunmap_pmem(pmem->virt_addr); out_release_region: release_mem_region(pmem->phys_addr, pmem->size); out_free_dev: @@ -194,7 +255,7 @@ static void pmem_free(struct pmem_device *pmem) del_gendisk(pmem->pmem_disk); put_disk(pmem->pmem_disk); blk_cleanup_queue(pmem->pmem_queue); - memunmap(pmem->virt_addr); + memunmap_pmem(pmem->virt_addr); release_mem_region(pmem->phys_addr, pmem->size); kfree(pmem); } diff --git a/include/linux/compiler.h b/include/linux/compiler.h index 05be2352fef8..26fc8bc77f85 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -21,6 +21,7 @@ # define __rcu __attribute__((noderef, address_space(4))) #else # define __rcu +# define __pmem __attribute__((noderef, address_space(5))) #endif extern void __chk_user_ptr(const volatile void __user *); extern void __chk_io_ptr(const volatile void __iomem *); @@ -42,6 +43,7 @@ extern void __chk_io_ptr(const volatile void __iomem *); # define __cond_lock(x,c) (c) # define __percpu # define __rcu +# define __pmem #endif /* Indirect macros required for expanded argument pasting, eg. __LINE__. */ diff --git a/include/linux/pmem.h b/include/linux/pmem.h new file mode 100644 index 000000000000..0fad4ad714cc --- /dev/null +++ b/include/linux/pmem.h @@ -0,0 +1,102 @@ +/* + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of version 2 of the GNU General Public License as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + */ +#ifndef __PMEM_H__ +#define __PMEM_H__ + +#include +#include + +/* + * Architectures that define ARCH_HAS_PMEM_API must provide + * implementations for arch_memremap_pmem(), arch_memcpy_to_pmem(), + * arch_sync_pmem(), and __arch_has_sync_pmem(). + */ + +#ifdef CONFIG_ARCH_HAS_PMEM_API +/** + * memremap_pmem - map physical persistent memory for pmem api + * @offset: physical address of persistent memory + * @size: size of the mapping + * + * Establish a mapping of the architecture specific memory type expected + * by memcpy_to_pmem() and sync_pmem(). For example, it may be + * the case that an uncacheable or writethrough mapping is sufficient, + * or a writeback mapping provided memcpy_to_pmem() and + * sync_pmem() arrange for the data to be written through the + * cache to persistent media. + */ +static inline void __pmem *memremap_pmem(resource_size_t offset, unsigned long size) +{ + return arch_memremap_pmem(offset, size); +} + +/** + * memcpy_to_pmem - copy data to persistent memory + * @dst: destination buffer for the copy + * @src: source buffer for the copy + * @n: length of the copy in bytes + * + * Perform a memory copy that results in the destination of the copy + * being effectively evicted from, or never written to, the processor + * cache hierarchy after the copy completes. After memcpy_to_pmem() + * data may still reside in cpu or platform buffers, so this operation + * must be followed by a sync_pmem(). + */ +static inline void memcpy_to_pmem(void __pmem *dst, const void *src, size_t n) +{ + arch_memcpy_to_pmem(dst, src, n); +} + +/** + * sync_pmem - synchronize writes to persistent memory + * + * After a series of memcpy_to_pmem() operations this drains data from + * cpu write buffers and any platform (memory controller) buffers to + * ensure that written data is durable on persistent memory media. + */ +static inline void sync_pmem(void) +{ + arch_sync_pmem(); +} + +/** + * arch_has_sync_pmem - true if sync_pmem() ensures durability + * + * For a given cpu implementation within an architecture it is possible + * that sync_pmem() resolves to a nop. In the case this returns + * false, pmem api users are unable to ensure durabilty and may want to + * fall back to a different data consistency model, or otherwise notify + * the user. + */ +static inline bool arch_has_sync_pmem(void) +{ + return __arch_has_sync_pmem(); +} +#else +/* undefined symbols */ +extern void __pmem *memremap_pmem(resource_size_t offet, unsigned long size); +extern void memcpy_to_pmem(void __pmem *dst, const void *src, size_t n); +extern void sync_pmem(void); +extern bool arch_has_sync_pmem(void); +#endif /* CONFIG_ARCH_HAS_PMEM_API */ + +static inline void memcpy_from_pmem(void *dst, void __pmem const *src, size_t size) +{ + memcpy(dst, (void __force const *) src, size); +} + +static inline void memunmap_pmem(void __pmem *addr) +{ + memunmap((void __force *) addr); +} +#endif /* __PMEM_H__ */ diff --git a/lib/Kconfig b/lib/Kconfig index bc7bc0278921..0d28cc560c6b 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -525,4 +525,7 @@ config ARCH_HAS_SG_CHAIN config ARCH_HAS_MEMREMAP bool +config ARCH_HAS_PMEM_API + bool + endmenu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753623AbbFQLOW (ORCPT ); Wed, 17 Jun 2015 07:14:22 -0400 Received: from verein.lst.de ([213.95.11.211]:50893 "EHLO newverein.lst.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750859AbbFQLOO (ORCPT ); Wed, 17 Jun 2015 07:14:14 -0400 Date: Wed, 17 Jun 2015 13:14:12 +0200 From: Christoph Hellwig To: Dan Williams Cc: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org, jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@ml01.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, kbuild test robot , hch@lst.de Subject: Re: [PATCH v4 1/6] arch: unify ioremap prototypes and macro aliases Message-ID: <20150617111412.GA9079@lst.de> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211918.10271.74243.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150611211918.10271.74243.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: Mutt/1.5.17 (2007-11-01) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jun 11, 2015 at 05:19:18PM -0400, Dan Williams wrote: > Some archs define the first parameter to ioremap() as unsigned long, > while the balance define it as resource_size_t. Unify on > resource_size_t to enable passing ioremap function pointers. Also, some > archs use function-like macros for defining ioremap aliases, but > asm-generic/iomap.h expects object-like macros, unify on the latter. Not urgent: but maybe this is a big hint that we should not define the prototypes on a per-architecture basis but in a common header file. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756554AbbFQL2Y (ORCPT ); Wed, 17 Jun 2015 07:28:24 -0400 Received: from verein.lst.de ([213.95.11.211]:50985 "EHLO newverein.lst.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754479AbbFQL2B (ORCPT ); Wed, 17 Jun 2015 07:28:01 -0400 Date: Wed, 17 Jun 2015 13:27:59 +0200 From: Christoph Hellwig To: Dan Williams Cc: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org, jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@ml01.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de Subject: Re: [PATCH v4 3/6] arch/*/asm/io.h: add ioremap_cache() to all architectures Message-ID: <20150617112759.GB9246@lst.de> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211930.10271.9100.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150611211930.10271.9100.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: Mutt/1.5.17 (2007-11-01) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Ok, this convinces me that we really should have a common defintion for ioremap() and friends first. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755134AbbFQLbZ (ORCPT ); Wed, 17 Jun 2015 07:31:25 -0400 Received: from verein.lst.de ([213.95.11.211]:51023 "EHLO newverein.lst.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753630AbbFQLbX (ORCPT ); Wed, 17 Jun 2015 07:31:23 -0400 Date: Wed, 17 Jun 2015 13:31:21 +0200 From: Christoph Hellwig To: Dan Williams Cc: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org, jgross@suse.com, x86@kernel.org, toshi.kani@hp.com, linux-nvdimm@ml01.01.org, benh@kernel.crashing.org, mcgrof@suse.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de Subject: Re: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates Message-ID: <20150617113121.GC9246@lst.de> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: Mutt/1.5.17 (2007-11-01) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This mess with arch_ methods and an ops vecor is almost unreadable. What's the problem with having something like: pmem_foo() { if (arch_has_pmem) // or sync_pmem arch_pmem_foo(); generic_pmem_foo(); } This adds a branch at runtime, but that shoudn't really be any slower than an indirect call on architectures that matter. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756938AbbFQOyT (ORCPT ); Wed, 17 Jun 2015 10:54:19 -0400 Received: from mail-wi0-f173.google.com ([209.85.212.173]:34773 "EHLO mail-wi0-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756687AbbFQOyQ (ORCPT ); Wed, 17 Jun 2015 10:54:16 -0400 MIME-Version: 1.0 In-Reply-To: <20150617113121.GC9246@lst.de> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> <20150617113121.GC9246@lst.de> Date: Wed, 17 Jun 2015 07:54:14 -0700 Message-ID: Subject: Re: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates From: Dan Williams To: Christoph Hellwig Cc: Arnd Bergmann , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Thomas Gleixner , Ross Zwisler , Andrew Morton , Juergen Gross , X86 ML , "Kani, Toshimitsu" , "linux-nvdimm@lists.01.org" , Benjamin Herrenschmidt , Luis Rodriguez , Konrad Rzeszutek Wilk , "linux-kernel@vger.kernel.org" , Stefan Bader , Andy Lutomirski , linux-mm@kvack.org, Geert Uytterhoeven , Ralf Baechle , Henrique de Moraes Holschuh , mpe@ellerman.id.au, Tejun Heo , Paul Mackerras Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jun 17, 2015 at 4:31 AM, Christoph Hellwig wrote: > This mess with arch_ methods and an ops vecor is almost unreadable. > > What's the problem with having something like: > > pmem_foo() > { > if (arch_has_pmem) // or sync_pmem > arch_pmem_foo(); > generic_pmem_foo(); > } > > This adds a branch at runtime, but that shoudn't really be any slower > than an indirect call on architectures that matter. No doubt it's premature optimization, but it bothered me that we'll end up calling cpuid perhaps multiple times every i/o. If it's just a readability concern I could wrap it in helpers. Getting it upstream is my primary concern at this point so I have no strong attachment to the indirect calls if that's all that is preventing an ack. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755460AbbFQPIF (ORCPT ); Wed, 17 Jun 2015 11:08:05 -0400 Received: from mail-lb0-f170.google.com ([209.85.217.170]:34861 "EHLO mail-lb0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753812AbbFQPIB (ORCPT ); Wed, 17 Jun 2015 11:08:01 -0400 MIME-Version: 1.0 In-Reply-To: <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> From: Andy Lutomirski Date: Wed, 17 Jun 2015 08:07:38 -0700 Message-ID: Subject: Re: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates To: Dan Williams Cc: Arnd Bergmann , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Thomas Gleixner , Ross Zwisler , Andrew Morton , Juergen Gross , X86 ML , Toshi Kani , linux-nvdimm , Benjamin Herrenschmidt , Luis Rodriguez , Konrad Rzeszutek Wilk , "linux-kernel@vger.kernel.org" , Stefan Bader , "linux-mm@kvack.org" , Geert Uytterhoeven , Ralf Baechle , Henrique de Moraes Holschuh , Michael Ellerman , Tejun Heo , Paul Mackerras , Christoph Hellwig Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jun 11, 2015 at 2:19 PM, Dan Williams wrote: > From: Ross Zwisler > > Based on an original patch by Ross Zwisler [1]. > > Writes to persistent memory have the potential to be posted to cpu > cache, cpu write buffers, and platform write buffers (memory controller) > before being committed to persistent media. Provide apis, > memcpy_to_pmem(), sync_pmem(), and memremap_pmem(), to write data to > pmem and assert that it is durable in PMEM (a persistent linear address > range). A '__pmem' attribute is added so sparse can track proper usage > of pointers to pmem. > > [1]: https://lists.01.org/pipermail/linux-nvdimm/2015-May/000932.html > > Cc: Thomas Gleixner > Cc: Ingo Molnar > Cc: "H. Peter Anvin" > Signed-off-by: Ross Zwisler > [djbw: various reworks] > Signed-off-by: Dan Williams > --- > arch/x86/Kconfig | 1 > arch/x86/include/asm/cacheflush.h | 36 +++++++++++++ > arch/x86/include/asm/io.h | 6 ++ > drivers/block/pmem.c | 75 +++++++++++++++++++++++++-- > include/linux/compiler.h | 2 + > include/linux/pmem.h | 102 +++++++++++++++++++++++++++++++++++++ > lib/Kconfig | 3 + > 7 files changed, 218 insertions(+), 7 deletions(-) > create mode 100644 include/linux/pmem.h > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig > index f16caf7eac27..5dfb8f31ac48 100644 > --- a/arch/x86/Kconfig > +++ b/arch/x86/Kconfig > @@ -28,6 +28,7 @@ config X86 > select ARCH_HAS_FAST_MULTIPLIER > select ARCH_HAS_GCOV_PROFILE_ALL > select ARCH_HAS_MEMREMAP > + select ARCH_HAS_PMEM_API > select ARCH_HAS_SG_CHAIN > select ARCH_HAVE_NMI_SAFE_CMPXCHG > select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI > diff --git a/arch/x86/include/asm/cacheflush.h b/arch/x86/include/asm/cacheflush.h > index b6f7457d12e4..4d896487382c 100644 > --- a/arch/x86/include/asm/cacheflush.h > +++ b/arch/x86/include/asm/cacheflush.h > @@ -4,6 +4,7 @@ > /* Caches aren't brain-dead on the intel. */ > #include > #include > +#include > > /* > * The set_memory_* API can be used to change various attributes of a virtual > @@ -108,4 +109,39 @@ static inline int rodata_test(void) > } > #endif > > +#ifdef ARCH_HAS_NOCACHE_UACCESS > +static inline void arch_memcpy_to_pmem(void __pmem *dst, const void *src, size_t n) > +{ > + /* > + * We are copying between two kernel buffers, if > + * __copy_from_user_inatomic_nocache() returns an error (page > + * fault) we would have already taken an unhandled fault before > + * the BUG_ON. The BUG_ON is simply here to satisfy > + * __must_check and allow reuse of the common non-temporal store > + * implementation for memcpy_to_pmem(). > + */ > + BUG_ON(__copy_from_user_inatomic_nocache((void __force *) dst, > + (void __user *) src, n)); Ick. If we take a fault, we will lose the debugging information we would otherwise have gotten unless we get lucky and get a usable CR2 value in the oops. > +} > + > +static inline void arch_sync_pmem(void) > +{ > + wmb(); > + pcommit_sfence(); > +} This function is non-intuitive to me. It's really "arch-specific sync pmem after one or more copies using arch_memcpy_to_pmem". If normal stores or memcpy to non-WC memory is used instead, then it's insufficient if the memory is WB and it's unnecessarily slow if the memory is WT or UC (the first sfence isn't needed). I would change the name and add documentation. I'd also add a comment about the wmb() being an SFENCE to flush pending non-temporal writes. --Andy From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932342AbbFQPJP (ORCPT ); Wed, 17 Jun 2015 11:09:15 -0400 Received: from mail-la0-f53.google.com ([209.85.215.53]:36705 "EHLO mail-la0-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932296AbbFQPJM (ORCPT ); Wed, 17 Jun 2015 11:09:12 -0400 MIME-Version: 1.0 In-Reply-To: References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> <20150617113121.GC9246@lst.de> From: Andy Lutomirski Date: Wed, 17 Jun 2015 08:08:49 -0700 Message-ID: Subject: Re: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates To: Dan Williams Cc: Christoph Hellwig , Arnd Bergmann , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Thomas Gleixner , Ross Zwisler , Andrew Morton , Juergen Gross , X86 ML , "Kani, Toshimitsu" , "linux-nvdimm@lists.01.org" , Benjamin Herrenschmidt , Luis Rodriguez , Konrad Rzeszutek Wilk , "linux-kernel@vger.kernel.org" , Stefan Bader , "linux-mm@kvack.org" , Geert Uytterhoeven , Ralf Baechle , Henrique de Moraes Holschuh , Michael Ellerman , Tejun Heo , Paul Mackerras Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jun 17, 2015 at 7:54 AM, Dan Williams wrote: > On Wed, Jun 17, 2015 at 4:31 AM, Christoph Hellwig wrote: >> This mess with arch_ methods and an ops vecor is almost unreadable. >> >> What's the problem with having something like: >> >> pmem_foo() >> { >> if (arch_has_pmem) // or sync_pmem >> arch_pmem_foo(); >> generic_pmem_foo(); >> } >> >> This adds a branch at runtime, but that shoudn't really be any slower >> than an indirect call on architectures that matter. > > No doubt it's premature optimization, but it bothered me that we'll > end up calling cpuid perhaps multiple times every i/o. If it's just a > readability concern I could wrap it in helpers. Getting it upstream > is my primary concern at this point so I have no strong attachment to > the indirect calls if that's all that is preventing an ack. A cpuid per i/o would be a killer, but the cpufeature code is way smarter than that. You want static_cpu_has, though -- it's even faster, since it gets patched at boot time. --Andy From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757397AbbFQPQf (ORCPT ); Wed, 17 Jun 2015 11:16:35 -0400 Received: from www.linutronix.de ([62.245.132.108]:49725 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755216AbbFQPQd (ORCPT ); Wed, 17 Jun 2015 11:16:33 -0400 Date: Wed, 17 Jun 2015 17:15:57 +0200 (CEST) From: Thomas Gleixner To: Andy Lutomirski cc: Dan Williams , Arnd Bergmann , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Ross Zwisler , Andrew Morton , Juergen Gross , X86 ML , Toshi Kani , linux-nvdimm , Benjamin Herrenschmidt , Luis Rodriguez , Konrad Rzeszutek Wilk , "linux-kernel@vger.kernel.org" , Stefan Bader , "linux-mm@kvack.org" , Geert Uytterhoeven , Ralf Baechle , Henrique de Moraes Holschuh , Michael Ellerman , Tejun Heo , Paul Mackerras , Christoph Hellwig Subject: Re: [PATCH v4 6/6] arch, x86: pmem api for ensuring durability of persistent memory updates In-Reply-To: Message-ID: References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211947.10271.80768.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: Alpine 2.11 (DEB 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Linutronix-Spam-Score: -1.0 X-Linutronix-Spam-Level: - X-Linutronix-Spam-Status: No , -1.0 points, 5.0 required, ALL_TRUSTED=-1,SHORTCIRCUIT=-0.0001,URIBL_BLOCKED=0.001 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 17 Jun 2015, Andy Lutomirski wrote: > On Thu, Jun 11, 2015 at 2:19 PM, Dan Williams wrote: > > +static inline void arch_sync_pmem(void) > > +{ > > + wmb(); > > + pcommit_sfence(); > > +} > > This function is non-intuitive to me. It's really "arch-specific sync > pmem after one or more copies using arch_memcpy_to_pmem". If normal > stores or memcpy to non-WC memory is used instead, then it's > insufficient if the memory is WB and it's unnecessarily slow if the > memory is WT or UC (the first sfence isn't needed). > > I would change the name and add documentation. I'd also add a comment > about the wmb() being an SFENCE to flush pending non-temporal writes. Not "I'd also add ...". Documentation of memory barriers are mandatory. Thanks, tglx From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757072AbbFQRfq (ORCPT ); Wed, 17 Jun 2015 13:35:46 -0400 Received: from g4t3427.houston.hp.com ([15.201.208.55]:42013 "EHLO g4t3427.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753447AbbFQRfi (ORCPT ); Wed, 17 Jun 2015 13:35:38 -0400 Message-ID: <1434562513.11808.100.camel@misato.fc.hp.com> Subject: Re: [PATCH v4 1/6] arch: unify ioremap prototypes and macro aliases From: Toshi Kani To: Dan Williams Cc: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org, jgross@suse.com, x86@kernel.org, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@ml01.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, kbuild test robot , hch@lst.de Date: Wed, 17 Jun 2015 11:35:13 -0600 In-Reply-To: <20150611211918.10271.74243.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211918.10271.74243.stgit@dwillia2-desk3.amr.corp.intel.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4 (3.10.4-4.fc20) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2015-06-11 at 17:19 -0400, Dan Williams wrote: > Some archs define the first parameter to ioremap() as unsigned long, > while the balance define it as resource_size_t. Unify on > resource_size_t to enable passing ioremap function pointers. Also, some > archs use function-like macros for defining ioremap aliases, but > asm-generic/iomap.h expects object-like macros, unify on the latter. > : > diff --git a/arch/ia64/include/asm/io.h b/arch/ia64/include/asm/io.h > index 80a7e34be009..8588ef767a44 100644 > --- a/arch/ia64/include/asm/io.h > +++ b/arch/ia64/include/asm/io.h > @@ -424,8 +424,8 @@ __writeq (unsigned long val, volatile void __iomem *addr) > > # ifdef __KERNEL__ > > -extern void __iomem * ioremap(unsigned long offset, unsigned long size); > -extern void __iomem * ioremap_nocache (unsigned long offset, unsigned long size); > +extern void __iomem * ioremap(resource_size_t offset, unsigned long size); > +extern void __iomem * ioremap_nocache (resource_size_t offset, unsigned long size); > extern void iounmap (volatile void __iomem *addr); > extern void __iomem * early_ioremap (unsigned long phys_addr, unsigned long size); > #define early_memremap(phys_addr, size) early_ioremap(phys_addr, size) This ia64 io.h also defines ioremap_cache(). Should this be also changed to resource_size_t? static inline void __iomem * ioremap_cache (unsigned long phys_addr, unsigned long size) { return ioremap(phys_addr, size); } -Toshi From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756562AbbFSV3c (ORCPT ); Fri, 19 Jun 2015 17:29:32 -0400 Received: from g4t3426.houston.hp.com ([15.201.208.54]:52498 "EHLO g4t3426.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753555AbbFSV3Y (ORCPT ); Fri, 19 Jun 2015 17:29:24 -0400 Message-ID: <1434749332.11808.113.camel@misato.fc.hp.com> Subject: Re: [PATCH v4 5/6] arch: introduce memremap_cache() and memremap_wt() From: Toshi Kani To: Dan Williams Cc: arnd@arndb.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, tglx@linutronix.de, ross.zwisler@linux.intel.com, akpm@linux-foundation.org, jgross@suse.com, x86@kernel.org, konrad.wilk@oracle.com, benh@kernel.crashing.org, mcgrof@suse.com, linux-nvdimm@ml01.01.org, linux-kernel@vger.kernel.org, stefan.bader@canonical.com, luto@amacapital.net, linux-mm@kvack.org, Andy Shevchenko , geert@linux-m68k.org, ralf@linux-mips.org, hmh@hmh.eng.br, mpe@ellerman.id.au, tj@kernel.org, paulus@samba.org, hch@lst.de Date: Fri, 19 Jun 2015 15:28:52 -0600 In-Reply-To: <20150611211941.10271.10513.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150611211354.10271.57950.stgit@dwillia2-desk3.amr.corp.intel.com> <20150611211941.10271.10513.stgit@dwillia2-desk3.amr.corp.intel.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4 (3.10.4-4.fc20) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2015-06-11 at 17:19 -0400, Dan Williams wrote: > Existing users of ioremap_cache() are mapping memory that is known in > advance to not have i/o side effects. These users are forced to cast > away the __iomem annotation, or otherwise neglect to fix the sparse > errors thrown when dereferencing pointers to this memory. Provide > memremap_*() as a non __iomem annotated ioremap_*(). > > The ARCH_HAS_MEMREMAP kconfig symbol is introduced for archs to assert > that it is safe to recast / reuse the return value from ioremap as a > normal pointer to memory. In other words, archs that mandate specific > accessors for __iomem are not memremap() capable and drivers that care, > like pmem, can add a dependency to disable themselves on these archs. : > +#ifdef CONFIG_ARCH_HAS_MEMREMAP > +/* > + * memremap() is "ioremap" for cases where it is known that the resource > + * being mapped does not have i/o side effects and the __iomem > + * annotation is not applicable. > + */ > +static bool memremap_valid(resource_size_t offset, size_t size) > +{ > + if (region_is_ram(offset, size) != 0) { I noticed that region_is_ram() is buggy and always returns -1. I will submit the fix shortly. Thanks, -Toshi > + WARN_ONCE(1, "memremap attempted on ram %pa size: %zu\n", > + &offset, size); > + return false; > + } > + return true; > +} -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in Please read the FAQ at http://www.tux.org/lkml/