[PATCH v6 0/3] Add PMEM support for RISC-V

[PATCH v6 0/3] Add PMEM support for RISC-V

[Anup Patel]

The Linux NVDIMM PEM drivers require arch support to map and access the
persistent memory device. This series adds RISC-V PMEM support using
recently added Svpbmt and Zicbom support.

First two patches are fixes and remaining two patches add the required
PMEM support for Linux RISC-V.

These patches can also be found in riscv_pmem_v6 branch at:
https://github.com/avpatel/linux.git

Changes since v5:
 - Rebased on Linux-6.1-rc5
 - Directly add pmem.o in obj-y of PATCH3
 - Dropped PATCH1 since this is already merged.

Changes since v4:
 - Simplify PATCH2 by implementing RISC-V specific arch_memremap_wb()

Changes since v3:
 - Pickup correct version of Drew's patch as PATCH1

Changes since v2:
 - Rebased on Linux-6.1-rc1
 - Replaced PATCH1 with the patch proposed by Drew

Changes since v1:
 - Fix error reported by test bot
   https://lore.kernel.org/all/202208272028.IwrNZ0Ur-lkp@intel.com/

Anup Patel (3):
  RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
  RISC-V: Implement arch specific PMEM APIs
  RISC-V: Enable PMEM drivers

 arch/riscv/Kconfig           |  1 +
 arch/riscv/configs/defconfig |  1 +
 arch/riscv/include/asm/io.h  |  5 +++++
 arch/riscv/mm/Makefile       |  1 +
 arch/riscv/mm/pmem.c         | 21 +++++++++++++++++++++
 5 files changed, 29 insertions(+)
 create mode 100644 arch/riscv/mm/pmem.c

-- 
2.34.1

[PATCH v6 1/3] RISC-V: Fix MEMREMAP_WB for systems with Svpbmt

[Anup Patel]

Currently, the memremap() called with MEMREMAP_WB maps memory using
the generic ioremap() function which breaks on system with Svpbmt
because memory mapped using _PAGE_IOREMAP page attributes is treated
as strongly-ordered non-cacheable IO memory.

To address this, we implement RISC-V specific arch_memremap_wb()
which maps memory using _PAGE_KERNEL page attributes resulting in
write-back cacheable mapping on systems with Svpbmt.

Fixes: ff689fd21cb1 ("riscv: add RISC-V Svpbmt extension support")
Co-developed-by: Mayuresh Chitale <mchitale@ventanamicro.com>
Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
---
 arch/riscv/include/asm/io.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/riscv/include/asm/io.h b/arch/riscv/include/asm/io.h
index 92080a227937..42497d487a17 100644
--- a/arch/riscv/include/asm/io.h
+++ b/arch/riscv/include/asm/io.h
@@ -135,4 +135,9 @@ __io_writes_outs(outs, u64, q, __io_pbr(), __io_paw())
 
 #include <asm-generic/io.h>
 
+#ifdef CONFIG_MMU
+#define arch_memremap_wb(addr, size)	\
+	((__force void *)ioremap_prot((addr), (size), _PAGE_KERNEL))
+#endif
+
 #endif /* _ASM_RISCV_IO_H */
-- 
2.34.1

[PATCH v6 2/3] RISC-V: Implement arch specific PMEM APIs

[Anup Patel]

The NVDIMM PMEM driver expects arch specific APIs for cache maintenance
and if arch does not provide these APIs then NVDIMM PMEM driver will
always use MEMREMAP_WT to map persistent memory which in-turn maps as
UC memory type defined by the RISC-V Svpbmt specification.

Now that the Svpbmt and Zicbom support is available in RISC-V kernel,
we implement PMEM APIs using ALT_CMO_OP() macros so that the NVDIMM
PMEM driver can use MEMREMAP_WB to map persistent memory.

Co-developed-by: Mayuresh Chitale <mchitale@ventanamicro.com>
Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
---
 arch/riscv/Kconfig     |  1 +
 arch/riscv/mm/Makefile |  1 +
 arch/riscv/mm/pmem.c   | 21 +++++++++++++++++++++
 3 files changed, 23 insertions(+)
 create mode 100644 arch/riscv/mm/pmem.c

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index fa78595a6089..c0e22648bd16 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -25,6 +25,7 @@ config RISCV
 	select ARCH_HAS_GIGANTIC_PAGE
 	select ARCH_HAS_KCOV
 	select ARCH_HAS_MMIOWB
+	select ARCH_HAS_PMEM_API
 	select ARCH_HAS_PTE_SPECIAL
 	select ARCH_HAS_SET_DIRECT_MAP if MMU
 	select ARCH_HAS_SET_MEMORY if MMU
diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile
index d76aabf4b94d..b4f35da889bf 100644
--- a/arch/riscv/mm/Makefile
+++ b/arch/riscv/mm/Makefile
@@ -13,6 +13,7 @@ obj-y += extable.o
 obj-$(CONFIG_MMU) += fault.o pageattr.o
 obj-y += cacheflush.o
 obj-y += context.o
+obj-y += pmem.o
 
 ifeq ($(CONFIG_MMU),y)
 obj-$(CONFIG_SMP) += tlbflush.o
diff --git a/arch/riscv/mm/pmem.c b/arch/riscv/mm/pmem.c
new file mode 100644
index 000000000000..089df92ae876
--- /dev/null
+++ b/arch/riscv/mm/pmem.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2022 Ventana Micro Systems Inc.
+ */
+
+#include <linux/export.h>
+#include <linux/libnvdimm.h>
+
+#include <asm/cacheflush.h>
+
+void arch_wb_cache_pmem(void *addr, size_t size)
+{
+	ALT_CMO_OP(clean, addr, size, riscv_cbom_block_size);
+}
+EXPORT_SYMBOL_GPL(arch_wb_cache_pmem);
+
+void arch_invalidate_pmem(void *addr, size_t size)
+{
+	ALT_CMO_OP(inval, addr, size, riscv_cbom_block_size);
+}
+EXPORT_SYMBOL_GPL(arch_invalidate_pmem);
-- 
2.34.1

[PATCH v6 3/3] RISC-V: Enable PMEM drivers

[Anup Patel]

We now have PMEM arch support available in RISC-V kernel so let us
enable relevant drivers in defconfig.

Signed-off-by: Anup Patel <apatel@ventanamicro.com>
---
 arch/riscv/configs/defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/riscv/configs/defconfig b/arch/riscv/configs/defconfig
index 05fd5fcf24f9..462da9f7410d 100644
--- a/arch/riscv/configs/defconfig
+++ b/arch/riscv/configs/defconfig
@@ -159,6 +159,7 @@ CONFIG_VIRTIO_MMIO=y
 CONFIG_RPMSG_CHAR=y
 CONFIG_RPMSG_CTRL=y
 CONFIG_RPMSG_VIRTIO=y
+CONFIG_LIBNVDIMM=y
 CONFIG_EXT4_FS=y
 CONFIG_EXT4_FS_POSIX_ACL=y
 CONFIG_EXT4_FS_SECURITY=y
-- 
2.34.1

Re: [PATCH v6 0/3] Add PMEM support for RISC-V

[Conor Dooley]

Hey Anup,

Looks like patches 2 and 3 are failing for the nommu defconfigs:
https://patchwork.kernel.org/project/linux-riscv/list/?series=694995
Unfortunately, if the build fails NIPA doesn't appear to save a log.

In case it's relevant, should be gcc-11 & bintutils 2.37.

Thanks,
Conor.

On Mon, Nov 14, 2022 at 02:35:33PM +0530, Anup Patel wrote:
> The Linux NVDIMM PEM drivers require arch support to map and access the
> persistent memory device. This series adds RISC-V PMEM support using
> recently added Svpbmt and Zicbom support.
> 
> First two patches are fixes and remaining two patches add the required
> PMEM support for Linux RISC-V.
> 
> These patches can also be found in riscv_pmem_v6 branch at:
> https://github.com/avpatel/linux.git
> 
> Changes since v5:
>  - Rebased on Linux-6.1-rc5
>  - Directly add pmem.o in obj-y of PATCH3
>  - Dropped PATCH1 since this is already merged.
> 
> Changes since v4:
>  - Simplify PATCH2 by implementing RISC-V specific arch_memremap_wb()
> 
> Changes since v3:
>  - Pickup correct version of Drew's patch as PATCH1
> 
> Changes since v2:
>  - Rebased on Linux-6.1-rc1
>  - Replaced PATCH1 with the patch proposed by Drew
> 
> Changes since v1:
>  - Fix error reported by test bot
>    https://lore.kernel.org/all/202208272028.IwrNZ0Ur-lkp@intel.com/
> 
> Anup Patel (3):
>   RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
>   RISC-V: Implement arch specific PMEM APIs
>   RISC-V: Enable PMEM drivers
> 
>  arch/riscv/Kconfig           |  1 +
>  arch/riscv/configs/defconfig |  1 +
>  arch/riscv/include/asm/io.h  |  5 +++++
>  arch/riscv/mm/Makefile       |  1 +
>  arch/riscv/mm/pmem.c         | 21 +++++++++++++++++++++
>  5 files changed, 29 insertions(+)
>  create mode 100644 arch/riscv/mm/pmem.c
> 
> -- 
> 2.34.1
> 

Re: [PATCH v6 0/3] Add PMEM support for RISC-V

[Anup Patel]

On Mon, Nov 14, 2022 at 10:11 PM Conor Dooley <conor@kernel.org> wrote:
>
> Hey Anup,
>
> Looks like patches 2 and 3 are failing for the nommu defconfigs:
> https://patchwork.kernel.org/project/linux-riscv/list/?series=694995
> Unfortunately, if the build fails NIPA doesn't appear to save a log.
>
> In case it's relevant, should be gcc-11 & bintutils 2.37.

I tried nommu defconfigs with gcc-12.1.0 and binutils-2.39 and I did
not see any failure.

Regards,
Anup

>
> Thanks,
> Conor.
>
> On Mon, Nov 14, 2022 at 02:35:33PM +0530, Anup Patel wrote:
> > The Linux NVDIMM PEM drivers require arch support to map and access the
> > persistent memory device. This series adds RISC-V PMEM support using
> > recently added Svpbmt and Zicbom support.
> >
> > First two patches are fixes and remaining two patches add the required
> > PMEM support for Linux RISC-V.
> >
> > These patches can also be found in riscv_pmem_v6 branch at:
> > https://github.com/avpatel/linux.git
> >
> > Changes since v5:
> >  - Rebased on Linux-6.1-rc5
> >  - Directly add pmem.o in obj-y of PATCH3
> >  - Dropped PATCH1 since this is already merged.
> >
> > Changes since v4:
> >  - Simplify PATCH2 by implementing RISC-V specific arch_memremap_wb()
> >
> > Changes since v3:
> >  - Pickup correct version of Drew's patch as PATCH1
> >
> > Changes since v2:
> >  - Rebased on Linux-6.1-rc1
> >  - Replaced PATCH1 with the patch proposed by Drew
> >
> > Changes since v1:
> >  - Fix error reported by test bot
> >    https://lore.kernel.org/all/202208272028.IwrNZ0Ur-lkp@intel.com/
> >
> > Anup Patel (3):
> >   RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
> >   RISC-V: Implement arch specific PMEM APIs
> >   RISC-V: Enable PMEM drivers
> >
> >  arch/riscv/Kconfig           |  1 +
> >  arch/riscv/configs/defconfig |  1 +
> >  arch/riscv/include/asm/io.h  |  5 +++++
> >  arch/riscv/mm/Makefile       |  1 +
> >  arch/riscv/mm/pmem.c         | 21 +++++++++++++++++++++
> >  5 files changed, 29 insertions(+)
> >  create mode 100644 arch/riscv/mm/pmem.c
> >
> > --
> > 2.34.1
> >

Re: [PATCH v6 0/3] Add PMEM support for RISC-V

[Conor Dooley]

On Sat, Nov 19, 2022 at 06:11:08PM +0530, Anup Patel wrote:
> On Mon, Nov 14, 2022 at 10:11 PM Conor Dooley <conor@kernel.org> wrote:
> >
> > Hey Anup,
> >
> > Looks like patches 2 and 3 are failing for the nommu defconfigs:
> > https://patchwork.kernel.org/project/linux-riscv/list/?series=694995
> > Unfortunately, if the build fails NIPA doesn't appear to save a log.
> >
> > In case it's relevant, should be gcc-11 & bintutils 2.37.
> 
> I tried nommu defconfigs with gcc-12.1.0 and binutils-2.39 and I did
> not see any failure.

In the interim, I got my new machine set up properly and have gcc-11
locally now (11.1.0) with binutils 2.37. I just gave the series a try
and I cannot see the failure either but I am not using tuxmake.

Apologies for the noise Anup!
FWIW:
Acked-by: Conor Dooley <conor.dooley@microchip.com>

> 
> Regards,
> Anup
> 
> >
> > Thanks,
> > Conor.
> >
> > On Mon, Nov 14, 2022 at 02:35:33PM +0530, Anup Patel wrote:
> > > The Linux NVDIMM PEM drivers require arch support to map and access the
> > > persistent memory device. This series adds RISC-V PMEM support using
> > > recently added Svpbmt and Zicbom support.
> > >
> > > First two patches are fixes and remaining two patches add the required
> > > PMEM support for Linux RISC-V.
> > >
> > > These patches can also be found in riscv_pmem_v6 branch at:
> > > https://github.com/avpatel/linux.git
> > >
> > > Changes since v5:
> > >  - Rebased on Linux-6.1-rc5
> > >  - Directly add pmem.o in obj-y of PATCH3
> > >  - Dropped PATCH1 since this is already merged.
> > >
> > > Changes since v4:
> > >  - Simplify PATCH2 by implementing RISC-V specific arch_memremap_wb()
> > >
> > > Changes since v3:
> > >  - Pickup correct version of Drew's patch as PATCH1
> > >
> > > Changes since v2:
> > >  - Rebased on Linux-6.1-rc1
> > >  - Replaced PATCH1 with the patch proposed by Drew
> > >
> > > Changes since v1:
> > >  - Fix error reported by test bot
> > >    https://lore.kernel.org/all/202208272028.IwrNZ0Ur-lkp@intel.com/
> > >
> > > Anup Patel (3):
> > >   RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
> > >   RISC-V: Implement arch specific PMEM APIs
> > >   RISC-V: Enable PMEM drivers
> > >
> > >  arch/riscv/Kconfig           |  1 +
> > >  arch/riscv/configs/defconfig |  1 +
> > >  arch/riscv/include/asm/io.h  |  5 +++++
> > >  arch/riscv/mm/Makefile       |  1 +
> > >  arch/riscv/mm/pmem.c         | 21 +++++++++++++++++++++
> > >  5 files changed, 29 insertions(+)
> > >  create mode 100644 arch/riscv/mm/pmem.c
> > >
> > > --
> > > 2.34.1
> > >

Re: [PATCH v6 0/3] Add PMEM support for RISC-V

[Anup Patel]

Hi Palmer,

On Mon, Nov 14, 2022 at 2:36 PM Anup Patel <apatel@ventanamicro.com> wrote:
>
> The Linux NVDIMM PEM drivers require arch support to map and access the
> persistent memory device. This series adds RISC-V PMEM support using
> recently added Svpbmt and Zicbom support.
>
> First two patches are fixes and remaining two patches add the required
> PMEM support for Linux RISC-V.
>
> These patches can also be found in riscv_pmem_v6 branch at:
> https://github.com/avpatel/linux.git
>
> Changes since v5:
>  - Rebased on Linux-6.1-rc5
>  - Directly add pmem.o in obj-y of PATCH3
>  - Dropped PATCH1 since this is already merged.
>
> Changes since v4:
>  - Simplify PATCH2 by implementing RISC-V specific arch_memremap_wb()
>
> Changes since v3:
>  - Pickup correct version of Drew's patch as PATCH1
>
> Changes since v2:
>  - Rebased on Linux-6.1-rc1
>  - Replaced PATCH1 with the patch proposed by Drew
>
> Changes since v1:
>  - Fix error reported by test bot
>    https://lore.kernel.org/all/202208272028.IwrNZ0Ur-lkp@intel.com/
>
> Anup Patel (3):
>   RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
>   RISC-V: Implement arch specific PMEM APIs
>   RISC-V: Enable PMEM drivers

Can this series be considered for Linux-6.2 ?

Regards,
Anup

>
>  arch/riscv/Kconfig           |  1 +
>  arch/riscv/configs/defconfig |  1 +
>  arch/riscv/include/asm/io.h  |  5 +++++
>  arch/riscv/mm/Makefile       |  1 +
>  arch/riscv/mm/pmem.c         | 21 +++++++++++++++++++++
>  5 files changed, 29 insertions(+)
>  create mode 100644 arch/riscv/mm/pmem.c
>
> --
> 2.34.1
>

Re: [PATCH v6 0/3] Add PMEM support for RISC-V

[Palmer Dabbelt]

On Mon, 14 Nov 2022 14:35:33 +0530, Anup Patel wrote:
> The Linux NVDIMM PEM drivers require arch support to map and access the
> persistent memory device. This series adds RISC-V PMEM support using
> recently added Svpbmt and Zicbom support.
> 
> First two patches are fixes and remaining two patches add the required
> PMEM support for Linux RISC-V.
> 
> [...]

Applied, thanks!

[1/3] RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
      https://git.kernel.org/palmer/c/b91676fc16cd
[2/3] RISC-V: Implement arch specific PMEM APIs
      https://git.kernel.org/palmer/c/a49ab905a1fc
[3/3] RISC-V: Enable PMEM drivers
      https://git.kernel.org/palmer/c/497bcbe3ce04

Best regards,
-- 
Palmer Dabbelt <palmer@rivosinc.com>

Re: [PATCH v6 0/3] Add PMEM support for RISC-V

[patchwork-bot+linux-riscv]

Hello:

This series was applied to riscv/linux.git (for-next)
by Palmer Dabbelt <palmer@rivosinc.com>:

On Mon, 14 Nov 2022 14:35:33 +0530 you wrote:
> The Linux NVDIMM PEM drivers require arch support to map and access the
> persistent memory device. This series adds RISC-V PMEM support using
> recently added Svpbmt and Zicbom support.
> 
> First two patches are fixes and remaining two patches add the required
> PMEM support for Linux RISC-V.
> 
> [...]

Here is the summary with links:
  - [v6,1/3] RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
    https://git.kernel.org/riscv/c/b91676fc16cd
  - [v6,2/3] RISC-V: Implement arch specific PMEM APIs
    https://git.kernel.org/riscv/c/a49ab905a1fc
  - [v6,3/3] RISC-V: Enable PMEM drivers
    https://git.kernel.org/riscv/c/497bcbe3ce04

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html