* [PATCH 0/8] Simplefs: group and simplify linux fs code
From: Emanuele Giuseppe Esposito @ 2020-04-14 12:42 UTC (permalink / raw)
To: linux-nfs
Cc: Song Liu, linux-usb, bpf, Rafael J. Wysocki, David Airlie,
Heiko Carstens, Alexei Starovoitov, dri-devel, J. Bruce Fields,
Joseph Qi, Hugh Dickins, Paul Mackerras, John Johansen, netdev,
linux-s390, Christoph Hellwig, Andrew Donnellan,
Emanuele Giuseppe Esposito, Matthew Garrett, linux-efi,
Arnd Bergmann, Daniel Borkmann, Christian Borntraeger, linux-rdma,
Mark Fasheh, Anton Vorontsov, John Fastabend, James Morris,
Ard Biesheuvel, Jason Gunthorpe, Doug Ledford, oprofile-list,
Yonghong Song, Ian Kent, Andrii Nakryiko, Alexey Dobriyan,
Serge E. Hallyn, Robert Richter, Thomas Zimmermann, Vasily Gorbik,
Tony Luck, Kees Cook, James E.J. Bottomley, autofs,
Maarten Lankhorst, Uma Krishnan, Maxime Ripard, linux-fsdevel,
Manoj N. Kumar, Alexander Viro, Jakub Kicinski, KP Singh,
Trond Myklebust, Matthew R. Ochs, David S. Miller, Felipe Balbi,
Mike Marciniszyn, Iurii Zaikin, linux-scsi, Martin K. Petersen,
linux-mm, Greg Kroah-Hartman, Dennis Dalessandro, Miklos Szeredi,
linux-security-module, linux-kernel, Anna Schumaker,
Luis Chamberlain, Chuck Lever, Jeremy Kerr, Daniel Vetter,
Colin Cross, Frederic Barrat, Paolo Bonzini, Andrew Morton,
Mike Kravetz, linuxppc-dev, Martin KaFai Lau, ocfs2-devel,
Joel Becker
This series of patches introduce wrappers for functions,
arguments simplification in functions calls and most importantly
groups duplicated code in a single header, simplefs, to avoid redundancy
in the linux fs, especially debugfs and tracefs.
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Emanuele Giuseppe Esposito (8):
apparmor: just use vfs_kern_mount to make .null
fs: extract simple_pin/release_fs to separate files
fs: wrap simple_pin_fs/simple_release_fs arguments in a struct
fs: introduce simple_new_inode
simplefs: add alloc_anon_inode wrapper
simplefs: add file creation functions
debugfs: switch to simplefs inode creation API
tracefs: switch to simplefs inode creation API
arch/powerpc/platforms/cell/spufs/inode.c | 4 +-
arch/s390/hypfs/inode.c | 4 +-
drivers/gpu/drm/Kconfig | 1 +
drivers/gpu/drm/drm_drv.c | 13 +-
drivers/infiniband/hw/qib/qib_fs.c | 6 +-
drivers/misc/cxl/Kconfig | 1 +
drivers/misc/cxl/api.c | 14 +-
drivers/misc/ibmasm/ibmasmfs.c | 8 +-
drivers/misc/ocxl/Kconfig | 1 +
drivers/oprofile/oprofilefs.c | 8 +-
drivers/scsi/cxlflash/ocxl_hw.c | 15 +-
drivers/usb/gadget/function/f_fs.c | 8 +-
fs/Kconfig | 3 +
fs/Kconfig.binfmt | 1 +
fs/Makefile | 1 +
fs/autofs/inode.c | 4 +-
fs/binfmt_misc.c | 27 +--
fs/configfs/Kconfig | 1 +
fs/configfs/mount.c | 12 +-
fs/debugfs/inode.c | 171 +++----------------
fs/efivarfs/inode.c | 4 +-
fs/fuse/control.c | 4 +-
fs/hugetlbfs/inode.c | 8 +-
fs/libfs.c | 48 ++----
fs/ocfs2/dlmfs/dlmfs.c | 8 +-
fs/proc/base.c | 4 +-
fs/proc/proc_sysctl.c | 5 +-
fs/pstore/inode.c | 14 +-
fs/ramfs/inode.c | 4 +-
fs/simplefs.c | 194 ++++++++++++++++++++++
fs/tracefs/inode.c | 108 ++----------
include/linux/fs.h | 3 +-
include/linux/simplefs.h | 36 ++++
ipc/mqueue.c | 4 +-
kernel/bpf/inode.c | 7 +-
lib/Kconfig.debug | 16 +-
mm/shmem.c | 4 +-
net/sunrpc/rpc_pipe.c | 4 +-
security/Kconfig | 1 +
security/apparmor/apparmorfs.c | 48 +++---
security/inode.c | 17 +-
41 files changed, 385 insertions(+), 459 deletions(-)
create mode 100644 fs/simplefs.c
create mode 100644 include/linux/simplefs.h
--
2.25.2
^ permalink raw reply
* Re: [PATCH] vhost: do not enable VHOST_MENU by default
From: Thomas Bogendoerfer @ 2020-04-14 11:52 UTC (permalink / raw)
To: Jason Wang
Cc: linux-s390, Vasily Gorbik, kvm, mst, netdev, Heiko Carstens,
linux-kernel, linux-mips, Christian Borntraeger, geert,
Paul Mackerras, virtualization, linuxppc-dev
In-Reply-To: <20200414024438.19103-1-jasowang@redhat.com>
On Tue, Apr 14, 2020 at 10:44:38AM +0800, Jason Wang wrote:
> We try to keep the defconfig untouched after decoupling CONFIG_VHOST
> out of CONFIG_VIRTUALIZATION in commit 20c384f1ea1a
> ("vhost: refine vhost and vringh kconfig") by enabling VHOST_MENU by
> default. Then the defconfigs can keep enabling CONFIG_VHOST_NET
> without the caring of CONFIG_VHOST.
>
> But this will leave a "CONFIG_VHOST_MENU=y" in all defconfigs and even
> for the ones that doesn't want vhost. So it actually shifts the
> burdens to the maintainers of all other to add "CONFIG_VHOST_MENU is
> not set". So this patch tries to enable CONFIG_VHOST explicitly in
> defconfigs that enables CONFIG_VHOST_NET and CONFIG_VHOST_VSOCK.
>
> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
> Cc: Vasily Gorbik <gor@linux.ibm.com>
> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
> arch/mips/configs/malta_kvm_defconfig | 1 +
Acked-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Thomas.
--
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea. [ RFC1925, 2.3 ]
^ permalink raw reply
* Re: [PATCH 1/2] mm, treewide: Rename kzfree() to kfree_sensitive()
From: David Sterba @ 2020-04-14 12:48 UTC (permalink / raw)
To: Waiman Long
Cc: linux-btrfs, Jarkko Sakkinen, virtualization, David Howells,
linux-mm, linux-sctp, keyrings, kasan-dev, samba-technical,
linux-stm32, devel, linux-s390, linux-scsi, x86, James Morris,
Matthew Wilcox, cocci, linux-wpan, intel-wired-lan,
David Rientjes, Serge E. Hallyn, linux-pm, ecryptfs, linux-nfs,
linux-fscrypt, linux-mediatek, linux-amlogic, linux-integrity,
linux-arm-kernel, linux-cifs, Linus Torvalds, linux-wireless,
linux-kernel, linux-bluetooth, linux-security-module,
target-devel, tipc-discussion, linux-crypto, netdev, Joe Perches,
Andrew Morton, linuxppc-dev, wireguard, linux-ppp
In-Reply-To: <20200413211550.8307-2-longman@redhat.com>
On Mon, Apr 13, 2020 at 05:15:49PM -0400, Waiman Long wrote:
> fs/btrfs/ioctl.c | 2 +-
> diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
> index 40b729dce91c..eab3f8510426 100644
> --- a/fs/btrfs/ioctl.c
> +++ b/fs/btrfs/ioctl.c
> @@ -2691,7 +2691,7 @@ static int btrfs_ioctl_get_subvol_info(struct file *file, void __user *argp)
> btrfs_put_root(root);
> out_free:
> btrfs_free_path(path);
> - kzfree(subvol_info);
> + kfree_sensitive(subvol_info);
This is not in a sensitive context so please switch it to plain kfree.
With that you have my acked-by. Thanks.
^ permalink raw reply
* Re: [PATCH 3/4] dma-mapping: add a dma_ops_bypass flag to struct device
From: Greg Kroah-Hartman @ 2020-04-14 12:47 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Alexey Kardashevskiy, linuxppc-dev, Joerg Roedel, linux-kernel,
iommu, Robin Murphy, Lu Baolu
In-Reply-To: <20200414122506.438134-4-hch@lst.de>
On Tue, Apr 14, 2020 at 02:25:05PM +0200, Christoph Hellwig wrote:
> Several IOMMU drivers have a bypass mode where they can use a direct
> mapping if the devices DMA mask is large enough. Add generic support
> to the core dma-mapping code to do that to switch those drivers to
> a common solution.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> include/linux/device.h | 6 ++++
> kernel/dma/mapping.c | 70 +++++++++++++++++++++++++++++-------------
> 2 files changed, 55 insertions(+), 21 deletions(-)
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
^ permalink raw reply
* Re: [PATCH v2 4/4] mm/vmalloc: Hugepage vmalloc mappings
From: Christophe Leroy @ 2020-04-14 12:28 UTC (permalink / raw)
To: Matthew Wilcox, Nicholas Piggin
Cc: linux-arch, linuxppc-dev, Catalin Marinas, x86, linux-kernel,
linux-mm, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
Thomas Gleixner, Will Deacon, linux-arm-kernel
In-Reply-To: <20200413134106.GN21484@bombadil.infradead.org>
Le 13/04/2020 à 15:41, Matthew Wilcox a écrit :
> On Mon, Apr 13, 2020 at 10:53:03PM +1000, Nicholas Piggin wrote:
>> +static int vmap_pages_range_noflush(unsigned long start, unsigned long end,
>> + pgprot_t prot, struct page **pages,
>> + unsigned int page_shift)
>> +{
>> + if (page_shift == PAGE_SIZE) {
>
> ... I think you meant 'page_shift == PAGE_SHIFT'
>
> Overall I like this series, although it's a bit biased towards CPUs
> which have page sizes which match PMD/PUD sizes. It doesn't offer the
> possibility of using 64kB page sizes on ARM, for example. But it's a
> step in the right direction.
>
I was going to ask more or less the same question, I would have liked to
use 512kB hugepages on powerpc 8xx.
Even the 8M hugepages (still on the 8xx), can they be used as well,
taking into account that two PGD entries have to point to the same 8M page ?
I sent out a series which tends to make the management of 512k and 8M
pages closer to what Linux expects, in order to use them inside kernel,
for Linear mappings and Kasan mappings for the moment. See
https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=164620
It would be nice if we could amplify it a use it for ioremaps and
vmallocs as well.
Christophe
^ permalink raw reply
* [PATCH 4/4] powerpc: use the generic dma_ops_bypass mode
From: Christoph Hellwig @ 2020-04-14 12:25 UTC (permalink / raw)
To: iommu, Alexey Kardashevskiy
Cc: Greg Kroah-Hartman, Joerg Roedel, Robin Murphy, linux-kernel,
linuxppc-dev, Lu Baolu
In-Reply-To: <20200414122506.438134-1-hch@lst.de>
Use the DMA API bypass mechanism for direct window mappings. This uses
common code and speed up the direct mapping case by avoiding indirect
calls just when not using dma ops at all. It also fixes a problem where
the sync_* methods were using the bypass check for DMA allocations, but
those are part of the streaming ops.
Note that this patch loses the DMA_ATTR_WEAK_ORDERING override, which
has never been well defined, as is only used by a few drivers, which
IIRC never showed up in the typical Cell blade setups that are affected
by the ordering workaround.
Fixes: efd176a04bef ("powerpc/pseries/dma: Allow SWIOTLB")
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/powerpc/include/asm/device.h | 5 --
arch/powerpc/kernel/dma-iommu.c | 90 ++++---------------------------
2 files changed, 9 insertions(+), 86 deletions(-)
diff --git a/arch/powerpc/include/asm/device.h b/arch/powerpc/include/asm/device.h
index 266542769e4b..452402215e12 100644
--- a/arch/powerpc/include/asm/device.h
+++ b/arch/powerpc/include/asm/device.h
@@ -18,11 +18,6 @@ struct iommu_table;
* drivers/macintosh/macio_asic.c
*/
struct dev_archdata {
- /*
- * Set to %true if the dma_iommu_ops are requested to use a direct
- * window instead of dynamically mapping memory.
- */
- bool iommu_bypass : 1;
/*
* These two used to be a union. However, with the hybrid ops we need
* both so here we store both a DMA offset for direct mappings and
diff --git a/arch/powerpc/kernel/dma-iommu.c b/arch/powerpc/kernel/dma-iommu.c
index e486d1d78de2..569fecd7b5b2 100644
--- a/arch/powerpc/kernel/dma-iommu.c
+++ b/arch/powerpc/kernel/dma-iommu.c
@@ -14,23 +14,6 @@
* Generic iommu implementation
*/
-/*
- * The coherent mask may be smaller than the real mask, check if we can
- * really use a direct window.
- */
-static inline bool dma_iommu_alloc_bypass(struct device *dev)
-{
- return dev->archdata.iommu_bypass && !iommu_fixed_is_weak &&
- dma_direct_supported(dev, dev->coherent_dma_mask);
-}
-
-static inline bool dma_iommu_map_bypass(struct device *dev,
- unsigned long attrs)
-{
- return dev->archdata.iommu_bypass &&
- (!iommu_fixed_is_weak || (attrs & DMA_ATTR_WEAK_ORDERING));
-}
-
/* Allocates a contiguous real buffer and creates mappings over it.
* Returns the virtual address of the buffer and sets dma_handle
* to the dma address (mapping) of the first page.
@@ -39,8 +22,6 @@ static void *dma_iommu_alloc_coherent(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t flag,
unsigned long attrs)
{
- if (dma_iommu_alloc_bypass(dev))
- return dma_direct_alloc(dev, size, dma_handle, flag, attrs);
return iommu_alloc_coherent(dev, get_iommu_table_base(dev), size,
dma_handle, dev->coherent_dma_mask, flag,
dev_to_node(dev));
@@ -50,11 +31,7 @@ static void dma_iommu_free_coherent(struct device *dev, size_t size,
void *vaddr, dma_addr_t dma_handle,
unsigned long attrs)
{
- if (dma_iommu_alloc_bypass(dev))
- dma_direct_free(dev, size, vaddr, dma_handle, attrs);
- else
- iommu_free_coherent(get_iommu_table_base(dev), size, vaddr,
- dma_handle);
+ iommu_free_coherent(get_iommu_table_base(dev), size, vaddr, dma_handle);
}
/* Creates TCEs for a user provided buffer. The user buffer must be
@@ -67,9 +44,6 @@ static dma_addr_t dma_iommu_map_page(struct device *dev, struct page *page,
enum dma_data_direction direction,
unsigned long attrs)
{
- if (dma_iommu_map_bypass(dev, attrs))
- return dma_direct_map_page(dev, page, offset, size, direction,
- attrs);
return iommu_map_page(dev, get_iommu_table_base(dev), page, offset,
size, dma_get_mask(dev), direction, attrs);
}
@@ -79,11 +53,8 @@ static void dma_iommu_unmap_page(struct device *dev, dma_addr_t dma_handle,
size_t size, enum dma_data_direction direction,
unsigned long attrs)
{
- if (!dma_iommu_map_bypass(dev, attrs))
- iommu_unmap_page(get_iommu_table_base(dev), dma_handle, size,
- direction, attrs);
- else
- dma_direct_unmap_page(dev, dma_handle, size, direction, attrs);
+ iommu_unmap_page(get_iommu_table_base(dev), dma_handle, size, direction,
+ attrs);
}
@@ -91,8 +62,6 @@ static int dma_iommu_map_sg(struct device *dev, struct scatterlist *sglist,
int nelems, enum dma_data_direction direction,
unsigned long attrs)
{
- if (dma_iommu_map_bypass(dev, attrs))
- return dma_direct_map_sg(dev, sglist, nelems, direction, attrs);
return ppc_iommu_map_sg(dev, get_iommu_table_base(dev), sglist, nelems,
dma_get_mask(dev), direction, attrs);
}
@@ -101,11 +70,8 @@ static void dma_iommu_unmap_sg(struct device *dev, struct scatterlist *sglist,
int nelems, enum dma_data_direction direction,
unsigned long attrs)
{
- if (!dma_iommu_map_bypass(dev, attrs))
- ppc_iommu_unmap_sg(get_iommu_table_base(dev), sglist, nelems,
+ ppc_iommu_unmap_sg(get_iommu_table_base(dev), sglist, nelems,
direction, attrs);
- else
- dma_direct_unmap_sg(dev, sglist, nelems, direction, attrs);
}
static bool dma_iommu_bypass_supported(struct device *dev, u64 mask)
@@ -113,8 +79,9 @@ static bool dma_iommu_bypass_supported(struct device *dev, u64 mask)
struct pci_dev *pdev = to_pci_dev(dev);
struct pci_controller *phb = pci_bus_to_host(pdev->bus);
- return phb->controller_ops.iommu_bypass_supported &&
- phb->controller_ops.iommu_bypass_supported(pdev, mask);
+ if (iommu_fixed_is_weak || !phb->controller_ops.iommu_bypass_supported)
+ return false;
+ return phb->controller_ops.iommu_bypass_supported(pdev, mask);
}
/* We support DMA to/from any memory page via the iommu */
@@ -123,7 +90,7 @@ int dma_iommu_dma_supported(struct device *dev, u64 mask)
struct iommu_table *tbl = get_iommu_table_base(dev);
if (dev_is_pci(dev) && dma_iommu_bypass_supported(dev, mask)) {
- dev->archdata.iommu_bypass = true;
+ dev->dma_ops_bypass = true;
dev_dbg(dev, "iommu: 64-bit OK, using fixed ops\n");
return 1;
}
@@ -141,7 +108,7 @@ int dma_iommu_dma_supported(struct device *dev, u64 mask)
}
dev_dbg(dev, "iommu: not 64-bit, using default ops\n");
- dev->archdata.iommu_bypass = false;
+ dev->dma_ops_bypass = false;
return 1;
}
@@ -153,47 +120,12 @@ u64 dma_iommu_get_required_mask(struct device *dev)
if (!tbl)
return 0;
- if (dev_is_pci(dev)) {
- u64 bypass_mask = dma_direct_get_required_mask(dev);
-
- if (dma_iommu_bypass_supported(dev, bypass_mask))
- return bypass_mask;
- }
-
mask = 1ULL < (fls_long(tbl->it_offset + tbl->it_size) - 1);
mask += mask - 1;
return mask;
}
-static void dma_iommu_sync_for_cpu(struct device *dev, dma_addr_t addr,
- size_t size, enum dma_data_direction dir)
-{
- if (dma_iommu_alloc_bypass(dev))
- dma_direct_sync_single_for_cpu(dev, addr, size, dir);
-}
-
-static void dma_iommu_sync_for_device(struct device *dev, dma_addr_t addr,
- size_t sz, enum dma_data_direction dir)
-{
- if (dma_iommu_alloc_bypass(dev))
- dma_direct_sync_single_for_device(dev, addr, sz, dir);
-}
-
-extern void dma_iommu_sync_sg_for_cpu(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir)
-{
- if (dma_iommu_alloc_bypass(dev))
- dma_direct_sync_sg_for_cpu(dev, sgl, nents, dir);
-}
-
-extern void dma_iommu_sync_sg_for_device(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir)
-{
- if (dma_iommu_alloc_bypass(dev))
- dma_direct_sync_sg_for_device(dev, sgl, nents, dir);
-}
-
const struct dma_map_ops dma_iommu_ops = {
.alloc = dma_iommu_alloc_coherent,
.free = dma_iommu_free_coherent,
@@ -203,10 +135,6 @@ const struct dma_map_ops dma_iommu_ops = {
.map_page = dma_iommu_map_page,
.unmap_page = dma_iommu_unmap_page,
.get_required_mask = dma_iommu_get_required_mask,
- .sync_single_for_cpu = dma_iommu_sync_for_cpu,
- .sync_single_for_device = dma_iommu_sync_for_device,
- .sync_sg_for_cpu = dma_iommu_sync_sg_for_cpu,
- .sync_sg_for_device = dma_iommu_sync_sg_for_device,
.mmap = dma_common_mmap,
.get_sgtable = dma_common_get_sgtable,
};
--
2.25.1
^ permalink raw reply related
* [PATCH 3/4] dma-mapping: add a dma_ops_bypass flag to struct device
From: Christoph Hellwig @ 2020-04-14 12:25 UTC (permalink / raw)
To: iommu, Alexey Kardashevskiy
Cc: Greg Kroah-Hartman, Joerg Roedel, Robin Murphy, linux-kernel,
linuxppc-dev, Lu Baolu
In-Reply-To: <20200414122506.438134-1-hch@lst.de>
Several IOMMU drivers have a bypass mode where they can use a direct
mapping if the devices DMA mask is large enough. Add generic support
to the core dma-mapping code to do that to switch those drivers to
a common solution.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
include/linux/device.h | 6 ++++
kernel/dma/mapping.c | 70 +++++++++++++++++++++++++++++-------------
2 files changed, 55 insertions(+), 21 deletions(-)
diff --git a/include/linux/device.h b/include/linux/device.h
index ac8e37cd716a..143c2e0edb19 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -523,6 +523,11 @@ struct dev_links_info {
* sync_state() callback.
* @dma_coherent: this particular device is dma coherent, even if the
* architecture supports non-coherent devices.
+ * @dma_ops_bypass: If set to %true then the dma_ops are bypassed for the
+ * streaming DMA operations (->map_* / ->unmap_* / ->sync_*),
+ * and optionall (if the coherent mask is large enough) also
+ * for dma allocations. This flag is managed by the dma ops
+ * instance from ->dma_supported.
*
* At the lowest level, every device in a Linux system is represented by an
* instance of struct device. The device structure contains the information
@@ -622,6 +627,7 @@ struct device {
defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
bool dma_coherent:1;
#endif
+ bool dma_ops_bypass : 1;
};
static inline struct device *kobj_to_dev(struct kobject *kobj)
diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c
index 8e4155f9ee69..d776fb9f5adb 100644
--- a/kernel/dma/mapping.c
+++ b/kernel/dma/mapping.c
@@ -105,9 +105,33 @@ void *dmam_alloc_attrs(struct device *dev, size_t size, dma_addr_t *dma_handle,
}
EXPORT_SYMBOL(dmam_alloc_attrs);
-static inline bool dma_is_direct(const struct dma_map_ops *ops)
+/*
+ * Check if the devices uses a direct mapping for streaming DMA operations.
+ * This allows IOMMU drivers to set a bypass mode if the DMA mask is large
+ * enough.
+ */
+static inline bool dma_alloc_direct(struct device *dev,
+ const struct dma_map_ops *ops)
{
- return likely(!ops);
+ if (likely(!ops))
+ return true;
+ if (!dev->dma_ops_bypass)
+ return false;
+
+ return min_not_zero(dev->coherent_dma_mask, dev->bus_dma_limit) >=
+ dma_direct_get_required_mask(dev);
+}
+
+static inline bool dma_map_direct(struct device *dev,
+ const struct dma_map_ops *ops)
+{
+ if (likely(!ops))
+ return true;
+ if (!dev->dma_ops_bypass)
+ return false;
+
+ return min_not_zero(*dev->dma_mask, dev->bus_dma_limit) >=
+ dma_direct_get_required_mask(dev);
}
dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page,
@@ -118,7 +142,7 @@ dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page,
dma_addr_t addr;
BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
+ if (dma_map_direct(dev, ops))
addr = dma_direct_map_page(dev, page, offset, size, dir, attrs);
else
addr = ops->map_page(dev, page, offset, size, dir, attrs);
@@ -134,7 +158,7 @@ void dma_unmap_page_attrs(struct device *dev, dma_addr_t addr, size_t size,
const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
+ if (dma_map_direct(dev, ops))
dma_direct_unmap_page(dev, addr, size, dir, attrs);
else if (ops->unmap_page)
ops->unmap_page(dev, addr, size, dir, attrs);
@@ -153,7 +177,7 @@ int dma_map_sg_attrs(struct device *dev, struct scatterlist *sg, int nents,
int ents;
BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
+ if (dma_map_direct(dev, ops))
ents = dma_direct_map_sg(dev, sg, nents, dir, attrs);
else
ents = ops->map_sg(dev, sg, nents, dir, attrs);
@@ -172,7 +196,7 @@ void dma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg,
BUG_ON(!valid_dma_direction(dir));
debug_dma_unmap_sg(dev, sg, nents, dir);
- if (dma_is_direct(ops))
+ if (dma_map_direct(dev, ops))
dma_direct_unmap_sg(dev, sg, nents, dir, attrs);
else if (ops->unmap_sg)
ops->unmap_sg(dev, sg, nents, dir, attrs);
@@ -191,7 +215,7 @@ dma_addr_t dma_map_resource(struct device *dev, phys_addr_t phys_addr,
if (WARN_ON_ONCE(pfn_valid(PHYS_PFN(phys_addr))))
return DMA_MAPPING_ERROR;
- if (dma_is_direct(ops))
+ if (dma_map_direct(dev, ops))
addr = dma_direct_map_resource(dev, phys_addr, size, dir, attrs);
else if (ops->map_resource)
addr = ops->map_resource(dev, phys_addr, size, dir, attrs);
@@ -207,7 +231,7 @@ void dma_unmap_resource(struct device *dev, dma_addr_t addr, size_t size,
const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
- if (!dma_is_direct(ops) && ops->unmap_resource)
+ if (!dma_map_direct(dev, ops) && ops->unmap_resource)
ops->unmap_resource(dev, addr, size, dir, attrs);
debug_dma_unmap_resource(dev, addr, size, dir);
}
@@ -219,7 +243,7 @@ void dma_sync_single_for_cpu(struct device *dev, dma_addr_t addr, size_t size,
const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
+ if (dma_map_direct(dev, ops))
dma_direct_sync_single_for_cpu(dev, addr, size, dir);
else if (ops->sync_single_for_cpu)
ops->sync_single_for_cpu(dev, addr, size, dir);
@@ -233,7 +257,7 @@ void dma_sync_single_for_device(struct device *dev, dma_addr_t addr,
const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
+ if (dma_map_direct(dev, ops))
dma_direct_sync_single_for_device(dev, addr, size, dir);
else if (ops->sync_single_for_device)
ops->sync_single_for_device(dev, addr, size, dir);
@@ -247,7 +271,7 @@ void dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
+ if (dma_map_direct(dev, ops))
dma_direct_sync_sg_for_cpu(dev, sg, nelems, dir);
else if (ops->sync_sg_for_cpu)
ops->sync_sg_for_cpu(dev, sg, nelems, dir);
@@ -261,7 +285,7 @@ void dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
+ if (dma_map_direct(dev, ops))
dma_direct_sync_sg_for_device(dev, sg, nelems, dir);
else if (ops->sync_sg_for_device)
ops->sync_sg_for_device(dev, sg, nelems, dir);
@@ -302,7 +326,7 @@ int dma_get_sgtable_attrs(struct device *dev, struct sg_table *sgt,
{
const struct dma_map_ops *ops = get_dma_ops(dev);
- if (dma_is_direct(ops))
+ if (dma_alloc_direct(dev, ops))
return dma_direct_get_sgtable(dev, sgt, cpu_addr, dma_addr,
size, attrs);
if (!ops->get_sgtable)
@@ -372,7 +396,7 @@ bool dma_can_mmap(struct device *dev)
{
const struct dma_map_ops *ops = get_dma_ops(dev);
- if (dma_is_direct(ops))
+ if (dma_alloc_direct(dev, ops))
return dma_direct_can_mmap(dev);
return ops->mmap != NULL;
}
@@ -397,7 +421,7 @@ int dma_mmap_attrs(struct device *dev, struct vm_area_struct *vma,
{
const struct dma_map_ops *ops = get_dma_ops(dev);
- if (dma_is_direct(ops))
+ if (dma_alloc_direct(dev, ops))
return dma_direct_mmap(dev, vma, cpu_addr, dma_addr, size,
attrs);
if (!ops->mmap)
@@ -410,7 +434,7 @@ u64 dma_get_required_mask(struct device *dev)
{
const struct dma_map_ops *ops = get_dma_ops(dev);
- if (dma_is_direct(ops))
+ if (dma_alloc_direct(dev, ops))
return dma_direct_get_required_mask(dev);
if (ops->get_required_mask)
return ops->get_required_mask(dev);
@@ -441,7 +465,7 @@ void *dma_alloc_attrs(struct device *dev, size_t size, dma_addr_t *dma_handle,
/* let the implementation decide on the zone to allocate from: */
flag &= ~(__GFP_DMA | __GFP_DMA32 | __GFP_HIGHMEM);
- if (dma_is_direct(ops))
+ if (dma_alloc_direct(dev, ops))
cpu_addr = dma_direct_alloc(dev, size, dma_handle, flag, attrs);
else if (ops->alloc)
cpu_addr = ops->alloc(dev, size, dma_handle, flag, attrs);
@@ -473,7 +497,7 @@ void dma_free_attrs(struct device *dev, size_t size, void *cpu_addr,
return;
debug_dma_free_coherent(dev, size, cpu_addr, dma_handle);
- if (dma_is_direct(ops))
+ if (dma_alloc_direct(dev, ops))
dma_direct_free(dev, size, cpu_addr, dma_handle, attrs);
else if (ops->free)
ops->free(dev, size, cpu_addr, dma_handle, attrs);
@@ -484,7 +508,11 @@ int dma_supported(struct device *dev, u64 mask)
{
const struct dma_map_ops *ops = get_dma_ops(dev);
- if (dma_is_direct(ops))
+ /*
+ * ->dma_supported sets the bypass flag, so we must always call
+ * into the method here unless the device is truly direct mapped.
+ */
+ if (!ops)
return dma_direct_supported(dev, mask);
if (!ops->dma_supported)
return 1;
@@ -540,7 +568,7 @@ void dma_cache_sync(struct device *dev, void *vaddr, size_t size,
BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
+ if (dma_alloc_direct(dev, ops))
arch_dma_cache_sync(dev, vaddr, size, dir);
else if (ops->cache_sync)
ops->cache_sync(dev, vaddr, size, dir);
@@ -552,7 +580,7 @@ size_t dma_max_mapping_size(struct device *dev)
const struct dma_map_ops *ops = get_dma_ops(dev);
size_t size = SIZE_MAX;
- if (dma_is_direct(ops))
+ if (dma_map_direct(dev, ops))
size = dma_direct_max_mapping_size(dev);
else if (ops && ops->max_mapping_size)
size = ops->max_mapping_size(dev);
--
2.25.1
^ permalink raw reply related
* [PATCH 2/4] dma-mapping: inline the fast path dma-direct calls
From: Christoph Hellwig @ 2020-04-14 12:25 UTC (permalink / raw)
To: iommu, Alexey Kardashevskiy
Cc: Greg Kroah-Hartman, Joerg Roedel, Robin Murphy, linux-kernel,
linuxppc-dev, Lu Baolu
In-Reply-To: <20200414122506.438134-1-hch@lst.de>
Inline the single page map/unmap/sync dma-direct calls into the now
out of line generic wrappers. This restores the behavior of a single
function call that we had before moving the generic calls out of line.
Besides the dma-mapping callers there are just a few callers in IOMMU
drivers that have a bypass mode, and more of those are going to be
switched to the generic bypass soon.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
include/linux/dma-direct.h | 88 ++++++++++++++++++++++++++++----------
kernel/dma/direct.c | 65 ----------------------------
2 files changed, 65 insertions(+), 88 deletions(-)
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index da689ad5fffd..d653070d515b 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -3,8 +3,10 @@
#define _LINUX_DMA_DIRECT_H 1
#include <linux/dma-mapping.h>
+#include <linux/dma-noncoherent.h>
#include <linux/memblock.h> /* for min_low_pfn */
#include <linux/mem_encrypt.h>
+#include <linux/swiotlb.h>
extern unsigned int zone_dma_bits;
@@ -85,25 +87,17 @@ int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,
void *cpu_addr, dma_addr_t dma_addr, size_t size,
unsigned long attrs);
int dma_direct_supported(struct device *dev, u64 mask);
-dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
- unsigned long offset, size_t size, enum dma_data_direction dir,
- unsigned long attrs);
int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
enum dma_data_direction dir, unsigned long attrs);
dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
size_t size, enum dma_data_direction dir, unsigned long attrs);
+size_t dma_direct_max_mapping_size(struct device *dev);
#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
defined(CONFIG_SWIOTLB)
-void dma_direct_sync_single_for_device(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir);
-void dma_direct_sync_sg_for_device(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir);
+void dma_direct_sync_sg_for_device(struct device *dev, struct scatterlist *sgl,
+ int nents, enum dma_data_direction dir);
#else
-static inline void dma_direct_sync_single_for_device(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir)
-{
-}
static inline void dma_direct_sync_sg_for_device(struct device *dev,
struct scatterlist *sgl, int nents, enum dma_data_direction dir)
{
@@ -113,34 +107,82 @@ static inline void dma_direct_sync_sg_for_device(struct device *dev,
#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL) || \
defined(CONFIG_SWIOTLB)
-void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
- size_t size, enum dma_data_direction dir, unsigned long attrs);
void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
int nents, enum dma_data_direction dir, unsigned long attrs);
-void dma_direct_sync_single_for_cpu(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir);
void dma_direct_sync_sg_for_cpu(struct device *dev,
struct scatterlist *sgl, int nents, enum dma_data_direction dir);
#else
-static inline void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
- size_t size, enum dma_data_direction dir, unsigned long attrs)
-{
-}
static inline void dma_direct_unmap_sg(struct device *dev,
struct scatterlist *sgl, int nents, enum dma_data_direction dir,
unsigned long attrs)
{
}
+static inline void dma_direct_sync_sg_for_cpu(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir)
+{
+}
+#endif
+
+static inline void dma_direct_sync_single_for_device(struct device *dev,
+ dma_addr_t addr, size_t size, enum dma_data_direction dir)
+{
+ phys_addr_t paddr = dma_to_phys(dev, addr);
+
+ if (unlikely(is_swiotlb_buffer(paddr)))
+ swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_DEVICE);
+
+ if (!dev_is_dma_coherent(dev))
+ arch_sync_dma_for_device(paddr, size, dir);
+}
+
static inline void dma_direct_sync_single_for_cpu(struct device *dev,
dma_addr_t addr, size_t size, enum dma_data_direction dir)
{
+ phys_addr_t paddr = dma_to_phys(dev, addr);
+
+ if (!dev_is_dma_coherent(dev)) {
+ arch_sync_dma_for_cpu(paddr, size, dir);
+ arch_sync_dma_for_cpu_all();
+ }
+
+ if (unlikely(is_swiotlb_buffer(paddr)))
+ swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_CPU);
}
-static inline void dma_direct_sync_sg_for_cpu(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir)
+
+static inline dma_addr_t dma_direct_map_page(struct device *dev,
+ struct page *page, unsigned long offset, size_t size,
+ enum dma_data_direction dir, unsigned long attrs)
{
+ phys_addr_t phys = page_to_phys(page) + offset;
+ dma_addr_t dma_addr = phys_to_dma(dev, phys);
+
+ if (unlikely(swiotlb_force == SWIOTLB_FORCE))
+ return swiotlb_map(dev, phys, size, dir, attrs);
+
+ if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
+ if (swiotlb_force != SWIOTLB_NO_FORCE)
+ return swiotlb_map(dev, phys, size, dir, attrs);
+
+ dev_WARN_ONCE(dev, 1,
+ "DMA addr %pad+%zu overflow (mask %llx, bus limit %llx).\n",
+ &dma_addr, size, *dev->dma_mask, dev->bus_dma_limit);
+ return DMA_MAPPING_ERROR;
+ }
+
+ if (!dev_is_dma_coherent(dev) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ arch_sync_dma_for_device(phys, size, dir);
+ return dma_addr;
}
-#endif
-size_t dma_direct_max_mapping_size(struct device *dev);
+static inline void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
+ size_t size, enum dma_data_direction dir, unsigned long attrs)
+{
+ phys_addr_t phys = dma_to_phys(dev, addr);
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ dma_direct_sync_single_for_cpu(dev, addr, size, dir);
+
+ if (unlikely(is_swiotlb_buffer(phys)))
+ swiotlb_tbl_unmap_single(dev, phys, size, size, dir, attrs);
+}
#endif /* _LINUX_DMA_DIRECT_H */
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index f1a9099a4b5b..0a320f9b837f 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -10,11 +10,9 @@
#include <linux/dma-direct.h>
#include <linux/scatterlist.h>
#include <linux/dma-contiguous.h>
-#include <linux/dma-noncoherent.h>
#include <linux/pfn.h>
#include <linux/vmalloc.h>
#include <linux/set_memory.h>
-#include <linux/swiotlb.h>
/*
* Most architectures use ZONE_DMA for the first 16 Megabytes, but some use it
@@ -249,18 +247,6 @@ void dma_direct_free(struct device *dev, size_t size,
#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
defined(CONFIG_SWIOTLB)
-void dma_direct_sync_single_for_device(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir)
-{
- phys_addr_t paddr = dma_to_phys(dev, addr);
-
- if (unlikely(is_swiotlb_buffer(paddr)))
- swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_DEVICE);
-
- if (!dev_is_dma_coherent(dev))
- arch_sync_dma_for_device(paddr, size, dir);
-}
-
void dma_direct_sync_sg_for_device(struct device *dev,
struct scatterlist *sgl, int nents, enum dma_data_direction dir)
{
@@ -284,20 +270,6 @@ void dma_direct_sync_sg_for_device(struct device *dev,
#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL) || \
defined(CONFIG_SWIOTLB)
-void dma_direct_sync_single_for_cpu(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir)
-{
- phys_addr_t paddr = dma_to_phys(dev, addr);
-
- if (!dev_is_dma_coherent(dev)) {
- arch_sync_dma_for_cpu(paddr, size, dir);
- arch_sync_dma_for_cpu_all();
- }
-
- if (unlikely(is_swiotlb_buffer(paddr)))
- swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_CPU);
-}
-
void dma_direct_sync_sg_for_cpu(struct device *dev,
struct scatterlist *sgl, int nents, enum dma_data_direction dir)
{
@@ -319,18 +291,6 @@ void dma_direct_sync_sg_for_cpu(struct device *dev,
arch_sync_dma_for_cpu_all();
}
-void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
- size_t size, enum dma_data_direction dir, unsigned long attrs)
-{
- phys_addr_t phys = dma_to_phys(dev, addr);
-
- if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
- dma_direct_sync_single_for_cpu(dev, addr, size, dir);
-
- if (unlikely(is_swiotlb_buffer(phys)))
- swiotlb_tbl_unmap_single(dev, phys, size, size, dir, attrs);
-}
-
void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
int nents, enum dma_data_direction dir, unsigned long attrs)
{
@@ -343,31 +303,6 @@ void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
}
#endif
-dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
- unsigned long offset, size_t size, enum dma_data_direction dir,
- unsigned long attrs)
-{
- phys_addr_t phys = page_to_phys(page) + offset;
- dma_addr_t dma_addr = phys_to_dma(dev, phys);
-
- if (unlikely(swiotlb_force == SWIOTLB_FORCE))
- return swiotlb_map(dev, phys, size, dir, attrs);
-
- if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
- if (swiotlb_force != SWIOTLB_NO_FORCE)
- return swiotlb_map(dev, phys, size, dir, attrs);
-
- dev_WARN_ONCE(dev, 1,
- "DMA addr %pad+%zu overflow (mask %llx, bus limit %llx).\n",
- &dma_addr, size, *dev->dma_mask, dev->bus_dma_limit);
- return DMA_MAPPING_ERROR;
- }
-
- if (!dev_is_dma_coherent(dev) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
- arch_sync_dma_for_device(phys, size, dir);
- return dma_addr;
-}
-
int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
enum dma_data_direction dir, unsigned long attrs)
{
--
2.25.1
^ permalink raw reply related
* [PATCH 1/4] dma-mapping: move the remaining DMA API calls out of line
From: Christoph Hellwig @ 2020-04-14 12:25 UTC (permalink / raw)
To: iommu, Alexey Kardashevskiy
Cc: Greg Kroah-Hartman, Joerg Roedel, Robin Murphy, linux-kernel,
linuxppc-dev, Lu Baolu
In-Reply-To: <20200414122506.438134-1-hch@lst.de>
For a long time the DMA API has been implemented inline in dma-mapping.h,
but the function bodies can be quite large. Move them all out of line.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
include/linux/dma-direct.h | 58 +++++++++
include/linux/dma-mapping.h | 247 ++++--------------------------------
kernel/dma/direct.c | 9 --
kernel/dma/mapping.c | 164 ++++++++++++++++++++++++
4 files changed, 244 insertions(+), 234 deletions(-)
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index 24b8684aa21d..da689ad5fffd 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -85,4 +85,62 @@ int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,
void *cpu_addr, dma_addr_t dma_addr, size_t size,
unsigned long attrs);
int dma_direct_supported(struct device *dev, u64 mask);
+dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
+ unsigned long offset, size_t size, enum dma_data_direction dir,
+ unsigned long attrs);
+int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
+ enum dma_data_direction dir, unsigned long attrs);
+dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
+ size_t size, enum dma_data_direction dir, unsigned long attrs);
+
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+ defined(CONFIG_SWIOTLB)
+void dma_direct_sync_single_for_device(struct device *dev,
+ dma_addr_t addr, size_t size, enum dma_data_direction dir);
+void dma_direct_sync_sg_for_device(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir);
+#else
+static inline void dma_direct_sync_single_for_device(struct device *dev,
+ dma_addr_t addr, size_t size, enum dma_data_direction dir)
+{
+}
+static inline void dma_direct_sync_sg_for_device(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir)
+{
+}
+#endif
+
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+ defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL) || \
+ defined(CONFIG_SWIOTLB)
+void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
+ size_t size, enum dma_data_direction dir, unsigned long attrs);
+void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
+ int nents, enum dma_data_direction dir, unsigned long attrs);
+void dma_direct_sync_single_for_cpu(struct device *dev,
+ dma_addr_t addr, size_t size, enum dma_data_direction dir);
+void dma_direct_sync_sg_for_cpu(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir);
+#else
+static inline void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
+ size_t size, enum dma_data_direction dir, unsigned long attrs)
+{
+}
+static inline void dma_direct_unmap_sg(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir,
+ unsigned long attrs)
+{
+}
+static inline void dma_direct_sync_single_for_cpu(struct device *dev,
+ dma_addr_t addr, size_t size, enum dma_data_direction dir)
+{
+}
+static inline void dma_direct_sync_sg_for_cpu(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir)
+{
+}
+#endif
+
+size_t dma_direct_max_mapping_size(struct device *dev);
+
#endif /* _LINUX_DMA_DIRECT_H */
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 330ad58fbf4d..793ad775cd54 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -188,73 +188,6 @@ static inline int dma_mmap_from_global_coherent(struct vm_area_struct *vma,
}
#endif /* CONFIG_DMA_DECLARE_COHERENT */
-static inline bool dma_is_direct(const struct dma_map_ops *ops)
-{
- return likely(!ops);
-}
-
-/*
- * All the dma_direct_* declarations are here just for the indirect call bypass,
- * and must not be used directly drivers!
- */
-dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
- unsigned long offset, size_t size, enum dma_data_direction dir,
- unsigned long attrs);
-int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
- enum dma_data_direction dir, unsigned long attrs);
-dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
- size_t size, enum dma_data_direction dir, unsigned long attrs);
-
-#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
- defined(CONFIG_SWIOTLB)
-void dma_direct_sync_single_for_device(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir);
-void dma_direct_sync_sg_for_device(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir);
-#else
-static inline void dma_direct_sync_single_for_device(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir)
-{
-}
-static inline void dma_direct_sync_sg_for_device(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir)
-{
-}
-#endif
-
-#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
- defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL) || \
- defined(CONFIG_SWIOTLB)
-void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
- size_t size, enum dma_data_direction dir, unsigned long attrs);
-void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
- int nents, enum dma_data_direction dir, unsigned long attrs);
-void dma_direct_sync_single_for_cpu(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir);
-void dma_direct_sync_sg_for_cpu(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir);
-#else
-static inline void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
- size_t size, enum dma_data_direction dir, unsigned long attrs)
-{
-}
-static inline void dma_direct_unmap_sg(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir,
- unsigned long attrs)
-{
-}
-static inline void dma_direct_sync_single_for_cpu(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir)
-{
-}
-static inline void dma_direct_sync_sg_for_cpu(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir)
-{
-}
-#endif
-
-size_t dma_direct_max_mapping_size(struct device *dev);
-
#ifdef CONFIG_HAS_DMA
#include <asm/dma-mapping.h>
@@ -271,164 +204,6 @@ static inline void set_dma_ops(struct device *dev,
dev->dma_ops = dma_ops;
}
-static inline dma_addr_t dma_map_page_attrs(struct device *dev,
- struct page *page, size_t offset, size_t size,
- enum dma_data_direction dir, unsigned long attrs)
-{
- const struct dma_map_ops *ops = get_dma_ops(dev);
- dma_addr_t addr;
-
- BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
- addr = dma_direct_map_page(dev, page, offset, size, dir, attrs);
- else
- addr = ops->map_page(dev, page, offset, size, dir, attrs);
- debug_dma_map_page(dev, page, offset, size, dir, addr);
-
- return addr;
-}
-
-static inline void dma_unmap_page_attrs(struct device *dev, dma_addr_t addr,
- size_t size, enum dma_data_direction dir, unsigned long attrs)
-{
- const struct dma_map_ops *ops = get_dma_ops(dev);
-
- BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
- dma_direct_unmap_page(dev, addr, size, dir, attrs);
- else if (ops->unmap_page)
- ops->unmap_page(dev, addr, size, dir, attrs);
- debug_dma_unmap_page(dev, addr, size, dir);
-}
-
-/*
- * dma_maps_sg_attrs returns 0 on error and > 0 on success.
- * It should never return a value < 0.
- */
-static inline int dma_map_sg_attrs(struct device *dev, struct scatterlist *sg,
- int nents, enum dma_data_direction dir,
- unsigned long attrs)
-{
- const struct dma_map_ops *ops = get_dma_ops(dev);
- int ents;
-
- BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
- ents = dma_direct_map_sg(dev, sg, nents, dir, attrs);
- else
- ents = ops->map_sg(dev, sg, nents, dir, attrs);
- BUG_ON(ents < 0);
- debug_dma_map_sg(dev, sg, nents, ents, dir);
-
- return ents;
-}
-
-static inline void dma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg,
- int nents, enum dma_data_direction dir,
- unsigned long attrs)
-{
- const struct dma_map_ops *ops = get_dma_ops(dev);
-
- BUG_ON(!valid_dma_direction(dir));
- debug_dma_unmap_sg(dev, sg, nents, dir);
- if (dma_is_direct(ops))
- dma_direct_unmap_sg(dev, sg, nents, dir, attrs);
- else if (ops->unmap_sg)
- ops->unmap_sg(dev, sg, nents, dir, attrs);
-}
-
-static inline dma_addr_t dma_map_resource(struct device *dev,
- phys_addr_t phys_addr,
- size_t size,
- enum dma_data_direction dir,
- unsigned long attrs)
-{
- const struct dma_map_ops *ops = get_dma_ops(dev);
- dma_addr_t addr = DMA_MAPPING_ERROR;
-
- BUG_ON(!valid_dma_direction(dir));
-
- /* Don't allow RAM to be mapped */
- if (WARN_ON_ONCE(pfn_valid(PHYS_PFN(phys_addr))))
- return DMA_MAPPING_ERROR;
-
- if (dma_is_direct(ops))
- addr = dma_direct_map_resource(dev, phys_addr, size, dir, attrs);
- else if (ops->map_resource)
- addr = ops->map_resource(dev, phys_addr, size, dir, attrs);
-
- debug_dma_map_resource(dev, phys_addr, size, dir, addr);
- return addr;
-}
-
-static inline void dma_unmap_resource(struct device *dev, dma_addr_t addr,
- size_t size, enum dma_data_direction dir,
- unsigned long attrs)
-{
- const struct dma_map_ops *ops = get_dma_ops(dev);
-
- BUG_ON(!valid_dma_direction(dir));
- if (!dma_is_direct(ops) && ops->unmap_resource)
- ops->unmap_resource(dev, addr, size, dir, attrs);
- debug_dma_unmap_resource(dev, addr, size, dir);
-}
-
-static inline void dma_sync_single_for_cpu(struct device *dev, dma_addr_t addr,
- size_t size,
- enum dma_data_direction dir)
-{
- const struct dma_map_ops *ops = get_dma_ops(dev);
-
- BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
- dma_direct_sync_single_for_cpu(dev, addr, size, dir);
- else if (ops->sync_single_for_cpu)
- ops->sync_single_for_cpu(dev, addr, size, dir);
- debug_dma_sync_single_for_cpu(dev, addr, size, dir);
-}
-
-static inline void dma_sync_single_for_device(struct device *dev,
- dma_addr_t addr, size_t size,
- enum dma_data_direction dir)
-{
- const struct dma_map_ops *ops = get_dma_ops(dev);
-
- BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
- dma_direct_sync_single_for_device(dev, addr, size, dir);
- else if (ops->sync_single_for_device)
- ops->sync_single_for_device(dev, addr, size, dir);
- debug_dma_sync_single_for_device(dev, addr, size, dir);
-}
-
-static inline void
-dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
- int nelems, enum dma_data_direction dir)
-{
- const struct dma_map_ops *ops = get_dma_ops(dev);
-
- BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
- dma_direct_sync_sg_for_cpu(dev, sg, nelems, dir);
- else if (ops->sync_sg_for_cpu)
- ops->sync_sg_for_cpu(dev, sg, nelems, dir);
- debug_dma_sync_sg_for_cpu(dev, sg, nelems, dir);
-}
-
-static inline void
-dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
- int nelems, enum dma_data_direction dir)
-{
- const struct dma_map_ops *ops = get_dma_ops(dev);
-
- BUG_ON(!valid_dma_direction(dir));
- if (dma_is_direct(ops))
- dma_direct_sync_sg_for_device(dev, sg, nelems, dir);
- else if (ops->sync_sg_for_device)
- ops->sync_sg_for_device(dev, sg, nelems, dir);
- debug_dma_sync_sg_for_device(dev, sg, nelems, dir);
-
-}
static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
{
@@ -439,6 +214,28 @@ static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
return 0;
}
+dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page,
+ size_t offset, size_t size, enum dma_data_direction dir,
+ unsigned long attrs);
+void dma_unmap_page_attrs(struct device *dev, dma_addr_t addr, size_t size,
+ enum dma_data_direction dir, unsigned long attrs);
+int dma_map_sg_attrs(struct device *dev, struct scatterlist *sg, int nents,
+ enum dma_data_direction dir, unsigned long attrs);
+void dma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg,
+ int nents, enum dma_data_direction dir,
+ unsigned long attrs);
+dma_addr_t dma_map_resource(struct device *dev, phys_addr_t phys_addr,
+ size_t size, enum dma_data_direction dir, unsigned long attrs);
+void dma_unmap_resource(struct device *dev, dma_addr_t addr, size_t size,
+ enum dma_data_direction dir, unsigned long attrs);
+void dma_sync_single_for_cpu(struct device *dev, dma_addr_t addr, size_t size,
+ enum dma_data_direction dir);
+void dma_sync_single_for_device(struct device *dev, dma_addr_t addr,
+ size_t size, enum dma_data_direction dir);
+void dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
+ int nelems, enum dma_data_direction dir);
+void dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
+ int nelems, enum dma_data_direction dir);
void *dma_alloc_attrs(struct device *dev, size_t size, dma_addr_t *dma_handle,
gfp_t flag, unsigned long attrs);
void dma_free_attrs(struct device *dev, size_t size, void *cpu_addr,
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 8f4bbdaf965e..f1a9099a4b5b 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -260,7 +260,6 @@ void dma_direct_sync_single_for_device(struct device *dev,
if (!dev_is_dma_coherent(dev))
arch_sync_dma_for_device(paddr, size, dir);
}
-EXPORT_SYMBOL(dma_direct_sync_single_for_device);
void dma_direct_sync_sg_for_device(struct device *dev,
struct scatterlist *sgl, int nents, enum dma_data_direction dir)
@@ -280,7 +279,6 @@ void dma_direct_sync_sg_for_device(struct device *dev,
dir);
}
}
-EXPORT_SYMBOL(dma_direct_sync_sg_for_device);
#endif
#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
@@ -299,7 +297,6 @@ void dma_direct_sync_single_for_cpu(struct device *dev,
if (unlikely(is_swiotlb_buffer(paddr)))
swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_CPU);
}
-EXPORT_SYMBOL(dma_direct_sync_single_for_cpu);
void dma_direct_sync_sg_for_cpu(struct device *dev,
struct scatterlist *sgl, int nents, enum dma_data_direction dir)
@@ -321,7 +318,6 @@ void dma_direct_sync_sg_for_cpu(struct device *dev,
if (!dev_is_dma_coherent(dev))
arch_sync_dma_for_cpu_all();
}
-EXPORT_SYMBOL(dma_direct_sync_sg_for_cpu);
void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
size_t size, enum dma_data_direction dir, unsigned long attrs)
@@ -334,7 +330,6 @@ void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
if (unlikely(is_swiotlb_buffer(phys)))
swiotlb_tbl_unmap_single(dev, phys, size, size, dir, attrs);
}
-EXPORT_SYMBOL(dma_direct_unmap_page);
void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
int nents, enum dma_data_direction dir, unsigned long attrs)
@@ -346,7 +341,6 @@ void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
dma_direct_unmap_page(dev, sg->dma_address, sg_dma_len(sg), dir,
attrs);
}
-EXPORT_SYMBOL(dma_direct_unmap_sg);
#endif
dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
@@ -373,7 +367,6 @@ dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
arch_sync_dma_for_device(phys, size, dir);
return dma_addr;
}
-EXPORT_SYMBOL(dma_direct_map_page);
int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
enum dma_data_direction dir, unsigned long attrs)
@@ -395,7 +388,6 @@ int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
dma_direct_unmap_sg(dev, sgl, i, dir, attrs | DMA_ATTR_SKIP_CPU_SYNC);
return 0;
}
-EXPORT_SYMBOL(dma_direct_map_sg);
dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
size_t size, enum dma_data_direction dir, unsigned long attrs)
@@ -412,7 +404,6 @@ dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
return dma_addr;
}
-EXPORT_SYMBOL(dma_direct_map_resource);
int dma_direct_get_sgtable(struct device *dev, struct sg_table *sgt,
void *cpu_addr, dma_addr_t dma_addr, size_t size,
diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c
index 98e3d873792e..8e4155f9ee69 100644
--- a/kernel/dma/mapping.c
+++ b/kernel/dma/mapping.c
@@ -105,6 +105,170 @@ void *dmam_alloc_attrs(struct device *dev, size_t size, dma_addr_t *dma_handle,
}
EXPORT_SYMBOL(dmam_alloc_attrs);
+static inline bool dma_is_direct(const struct dma_map_ops *ops)
+{
+ return likely(!ops);
+}
+
+dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page,
+ size_t offset, size_t size, enum dma_data_direction dir,
+ unsigned long attrs)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+ dma_addr_t addr;
+
+ BUG_ON(!valid_dma_direction(dir));
+ if (dma_is_direct(ops))
+ addr = dma_direct_map_page(dev, page, offset, size, dir, attrs);
+ else
+ addr = ops->map_page(dev, page, offset, size, dir, attrs);
+ debug_dma_map_page(dev, page, offset, size, dir, addr);
+
+ return addr;
+}
+EXPORT_SYMBOL(dma_map_page_attrs);
+
+void dma_unmap_page_attrs(struct device *dev, dma_addr_t addr, size_t size,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+
+ BUG_ON(!valid_dma_direction(dir));
+ if (dma_is_direct(ops))
+ dma_direct_unmap_page(dev, addr, size, dir, attrs);
+ else if (ops->unmap_page)
+ ops->unmap_page(dev, addr, size, dir, attrs);
+ debug_dma_unmap_page(dev, addr, size, dir);
+}
+EXPORT_SYMBOL(dma_unmap_page_attrs);
+
+/*
+ * dma_maps_sg_attrs returns 0 on error and > 0 on success.
+ * It should never return a value < 0.
+ */
+int dma_map_sg_attrs(struct device *dev, struct scatterlist *sg, int nents,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+ int ents;
+
+ BUG_ON(!valid_dma_direction(dir));
+ if (dma_is_direct(ops))
+ ents = dma_direct_map_sg(dev, sg, nents, dir, attrs);
+ else
+ ents = ops->map_sg(dev, sg, nents, dir, attrs);
+ BUG_ON(ents < 0);
+ debug_dma_map_sg(dev, sg, nents, ents, dir);
+
+ return ents;
+}
+EXPORT_SYMBOL(dma_map_sg_attrs);
+
+void dma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg,
+ int nents, enum dma_data_direction dir,
+ unsigned long attrs)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+
+ BUG_ON(!valid_dma_direction(dir));
+ debug_dma_unmap_sg(dev, sg, nents, dir);
+ if (dma_is_direct(ops))
+ dma_direct_unmap_sg(dev, sg, nents, dir, attrs);
+ else if (ops->unmap_sg)
+ ops->unmap_sg(dev, sg, nents, dir, attrs);
+}
+EXPORT_SYMBOL(dma_unmap_sg_attrs);
+
+dma_addr_t dma_map_resource(struct device *dev, phys_addr_t phys_addr,
+ size_t size, enum dma_data_direction dir, unsigned long attrs)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+ dma_addr_t addr = DMA_MAPPING_ERROR;
+
+ BUG_ON(!valid_dma_direction(dir));
+
+ /* Don't allow RAM to be mapped */
+ if (WARN_ON_ONCE(pfn_valid(PHYS_PFN(phys_addr))))
+ return DMA_MAPPING_ERROR;
+
+ if (dma_is_direct(ops))
+ addr = dma_direct_map_resource(dev, phys_addr, size, dir, attrs);
+ else if (ops->map_resource)
+ addr = ops->map_resource(dev, phys_addr, size, dir, attrs);
+
+ debug_dma_map_resource(dev, phys_addr, size, dir, addr);
+ return addr;
+}
+EXPORT_SYMBOL(dma_map_resource);
+
+void dma_unmap_resource(struct device *dev, dma_addr_t addr, size_t size,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+
+ BUG_ON(!valid_dma_direction(dir));
+ if (!dma_is_direct(ops) && ops->unmap_resource)
+ ops->unmap_resource(dev, addr, size, dir, attrs);
+ debug_dma_unmap_resource(dev, addr, size, dir);
+}
+EXPORT_SYMBOL(dma_unmap_resource);
+
+void dma_sync_single_for_cpu(struct device *dev, dma_addr_t addr, size_t size,
+ enum dma_data_direction dir)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+
+ BUG_ON(!valid_dma_direction(dir));
+ if (dma_is_direct(ops))
+ dma_direct_sync_single_for_cpu(dev, addr, size, dir);
+ else if (ops->sync_single_for_cpu)
+ ops->sync_single_for_cpu(dev, addr, size, dir);
+ debug_dma_sync_single_for_cpu(dev, addr, size, dir);
+}
+EXPORT_SYMBOL(dma_sync_single_for_cpu);
+
+void dma_sync_single_for_device(struct device *dev, dma_addr_t addr,
+ size_t size, enum dma_data_direction dir)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+
+ BUG_ON(!valid_dma_direction(dir));
+ if (dma_is_direct(ops))
+ dma_direct_sync_single_for_device(dev, addr, size, dir);
+ else if (ops->sync_single_for_device)
+ ops->sync_single_for_device(dev, addr, size, dir);
+ debug_dma_sync_single_for_device(dev, addr, size, dir);
+}
+EXPORT_SYMBOL(dma_sync_single_for_device);
+
+void dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
+ int nelems, enum dma_data_direction dir)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+
+ BUG_ON(!valid_dma_direction(dir));
+ if (dma_is_direct(ops))
+ dma_direct_sync_sg_for_cpu(dev, sg, nelems, dir);
+ else if (ops->sync_sg_for_cpu)
+ ops->sync_sg_for_cpu(dev, sg, nelems, dir);
+ debug_dma_sync_sg_for_cpu(dev, sg, nelems, dir);
+}
+EXPORT_SYMBOL(dma_sync_sg_for_cpu);
+
+void dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
+ int nelems, enum dma_data_direction dir)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+
+ BUG_ON(!valid_dma_direction(dir));
+ if (dma_is_direct(ops))
+ dma_direct_sync_sg_for_device(dev, sg, nelems, dir);
+ else if (ops->sync_sg_for_device)
+ ops->sync_sg_for_device(dev, sg, nelems, dir);
+ debug_dma_sync_sg_for_device(dev, sg, nelems, dir);
+}
+EXPORT_SYMBOL(dma_sync_sg_for_device);
+
/*
* Create scatter-list for the already allocated DMA buffer.
*/
--
2.25.1
^ permalink raw reply related
* generic DMA bypass flag v3
From: Christoph Hellwig @ 2020-04-14 12:25 UTC (permalink / raw)
To: iommu, Alexey Kardashevskiy
Cc: Greg Kroah-Hartman, Joerg Roedel, Robin Murphy, linux-kernel,
linuxppc-dev, Lu Baolu
Hi all,
I've recently beeing chatting with Lu about using dma-iommu and
per-device DMA ops in the intel IOMMU driver, and one missing feature
in dma-iommu is a bypass mode where the direct mapping is used even
when an iommu is attached to improve performance. The powerpc
code already has a similar mode, so I'd like to move it to the core
DMA mapping code. As part of that I noticed that the current
powerpc code has a little bug in that it used the wrong check in the
dma_sync_* routines to see if the direct mapping code is used.
These two patches just add the generic code and move powerpc over,
the intel IOMMU bits will require a separate discussion.
The x86 AMD Gart code also has a bypass mode, but it is a lot
strange, so I'm not going to touch it for now.
Changes since v2:
- move the dma mapping helpers out of line
- check for possible direct mappings using the dma mask
Changes since v1:
- rebased to the current dma-mapping-for-next tree
^ permalink raw reply
* Re: [PATCH v2 0/4] huge vmalloc mappings
From: Nicholas Piggin @ 2020-04-14 12:23 UTC (permalink / raw)
To: David Rientjes
Cc: linux-arch, Will Deacon, Catalin Marinas, x86, linux-kernel,
linux-mm, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
Thomas Gleixner, linuxppc-dev, linux-arm-kernel
In-Reply-To: <alpine.DEB.2.21.2004131727150.260270@chino.kir.corp.google.com>
Excerpts from David Rientjes's message of April 14, 2020 10:27 am:
> On Mon, 13 Apr 2020, Nicholas Piggin wrote:
>
>> We can get a significant win with larger mappings for some of the big
>> global hashes.
>>
>> Since RFC, relevant architectures have added p?d_leaf accessors so no
>> real arch changes required, and I changed it not to allocate huge
>> mappings for modules and a bunch of other fixes.
>>
>
> Hi Nicholas,
>
> Any performance numbers to share besides the git diff in the last patch in
> the series? I'm wondering if anything from mmtests or lkp-tests makes
> sense to try?
Hey, no I don't have any other tests I've run. Some of the networking
hashes do make use of it as well though, and might see a few % in
the right kind of workload. There's probably a bunch of other stuff
where it could help a little bit, looking through the tree, I just don't
have anything specific.
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH v6 2/3] powerpc/powernv: Introduce support and parsing for self-save API
From: Pratik Sampat @ 2020-04-14 12:18 UTC (permalink / raw)
To: ego; +Cc: pratik.r.sampat, linuxram, linux-kernel, linuxppc-dev, oohall,
skiboot
In-Reply-To: <20200414074702.GF24277@in.ibm.com>
Hello Gautham,
[..snip..]
>> +
>> + if (curr_spr.supported_mode & FIRMWARE_SELF_SAVE) {
>> + rc = opal_slw_self_save_reg(pir,
>> + curr_spr.spr);
>> + if (rc != 0)
>> + return rc;
>> + switch (curr_spr.spr) {
>> + case SPRN_LPCR:
>> + is_lpcr_self_save = true;
> Could you consider converting is_lpcr_self_save and is_ptcr_self_save
> into static_keys ? For reference see commit
> 14c73bd344da("powerpc/vcpu: Assume dedicated processors as
> non-preempt")
>
Sure, using static keys is cleaner.
I'll also address the other nits mentioned in the e-mail earlier.
Thanks for the review.
Pratik
^ permalink raw reply
* Re: [PATCH v2 4/4] mm/vmalloc: Hugepage vmalloc mappings
From: Nicholas Piggin @ 2020-04-14 12:13 UTC (permalink / raw)
To: Christoph Hellwig
Cc: linux-arch, Will Deacon, Catalin Marinas, x86, linux-kernel,
linux-mm, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
Thomas Gleixner, linuxppc-dev, linux-arm-kernel
In-Reply-To: <20200414072316.GA5503@infradead.org>
Excerpts from Christoph Hellwig's message of April 14, 2020 5:23 pm:
> On Mon, Apr 13, 2020 at 10:53:03PM +1000, Nicholas Piggin wrote:
>> For platforms that define HAVE_ARCH_HUGE_VMAP and support PMD vmap mappings,
>> have vmalloc attempt to allocate PMD-sized pages first, before falling back
>> to small pages. Allocations which use something other than PAGE_KERNEL
>> protections are not permitted to use huge pages yet, not all callers expect
>> this (e.g., module allocations vs strict module rwx).
>>
>> This gives a 6x reduction in dTLB misses for a `git diff` (of linux), from
>> 45600 to 6500 and a 2.2% reduction in cycles on a 2-node POWER9.
>>
>> This can result in more internal fragmentation and memory overhead for a
>> given allocation. It can also cause greater NUMA unbalance on hashdist
>> allocations.
>>
>> There may be other callers that expect small pages under vmalloc but use
>> PAGE_KERNEL, I'm not sure if it's feasible to catch them all. An
>> alternative would be a new function or flag which enables large mappings,
>> and use that in callers.
>
> Why do we even use vmalloc in this case rather than just doing a huge
> page allocation?
Which case? Usually the answer would be because you don't want to use
contiguous physical memory and/or you don't want to use the linear
mapping.
> What callers are you intersted in?
The dentry and inode caches for this test, obviously.
Lots of other things could possibly benefit though, other system
hashes like networking, but lot of other vmalloc callers that might
benefit right away, some others could use some work to batch up
allocation sizes to benefit.
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH v6 0/3] powerpc/powernv: Introduce interface for self-restore support
From: Pratik Sampat @ 2020-04-14 12:10 UTC (permalink / raw)
To: ego; +Cc: pratik.r.sampat, linuxram, linux-kernel, linuxppc-dev, oohall,
skiboot
In-Reply-To: <20200414071149.GD24277@in.ibm.com>
Hello Gautham
On 14/04/20 12:41 pm, Gautham R Shenoy wrote:
> Hello Pratik,
>
> On Thu, Mar 26, 2020 at 12:40:31PM +0530, Pratik Rajesh Sampat wrote:
>> v5: https://lkml.org/lkml/2020/3/17/944
>> Changelog
>> v5-->v6
>> 1. Updated background, motivation and illuminated potential design
>> choices
>> 2. Re-organization of patch-set
>> a. Split introducing preference for optimization from 1/1 to patch 3/3
>> b. Merge introducing self-save API and parsing the device-tree
>> c. Introduce a supported mode called KERNEL_SAVE_RESTORE which
>> outlines and makes kernel supported SPRs for save-restore more
>> explicit
>>
> [..snip..]
>
>> Presenting the design choices in front of us:
>>
>> Design-Choice 1:
>> ----------------
>> A simple implementation is to just replace self-restore calls with
>> self-save as it is direct super-set.
>>
>> Pros:
>> A simple design, quick to implement
>>
>>
>> Cons:
>> * Breaks backward compatibility. Self-restore has historically been
>> supported in the firmware and an old firmware running on an new
>> kernel will be incompatible and deep stop states will be cut.
>> * Furthermore, critical SPRs which need to be restored
>> before 0x100 vector like HID0 are not supported by self-save.
>>
>> Design-Choice 2:
>> ----------------
>> Advertise both self-restore and self-save from OPAL including the set
>> of registers that each support. The kernel can then choose which API
>> to go with.
>> For the sake of simplicity, in case both modes are supported for an
>> SPR by default self-save would be called for it.
>>
>> Pros:
>> * Backwards compatible
>>
>> Cons:
>> Overhead in parsing device tree with the SPR list
>>
>> Possible optimization with Approach2:
>> -------------------------------------
>> There are SPRs whose values don't tend to change over time and invoking
>> self-save on them, where the values are gotten each time may turn out to
>> be inefficient. In that case calling a self-restore where passing the
>> value makes more sense as, if the value is same, the memory location
>> is not updated.
>> SPRs that dont change are as follows:
>> SPRN_HSPRG0,
>> SPRN_LPCR,
>> SPRN_PTCR,
>> SPRN_HMEER,
>> SPRN_HID0,
> We can just pick self-save wherever available and fallback to
> self-restore when self-save support is not avaiable for any SPR.
> The optimization that you mention here can be revisited if the
> additional latency due to self-save becomes observable (Note that both
> stop4 and stop5 have wakeup latency between 200-500us).
Yes, Patch2 introduces picking self-save when available and using
self-restore as fallback.
Sure. The optimization of preference does certainly add an additional
layer of complexity. As you mentioned, in the case the latency really
begins to hurt us we can revisit this discussion.
Thanks
Pratik
>> The values of PSSCR and MSR change at runtime and hence, the kernel
>> cannot determine during boot time what their values will be before
>> entering a particular deep-stop state.
>>
>> Therefore, a preference based interface is introduced for choosing
>> between self-save or self-restore between for each SPR.
>> The per-SPR preference is only a refinement of
>> approach 2 purely for performance reasons. It can be dropped if the
>> complexity is not deemed worth the returns.
>>
>> Patches Organization
>> ====================
>> Design Choice 2 has been chosen as an implementation to demonstrate in
>> the patch series.
>>
>> Patch1:
>> Devises an interface which lists all the interested SPRs, along with
>> highlighting the support of mode.
>> It is an isomorphic patch to replicate the functionality of the older
>> self-restore firmware for the new interface
>>
>> Patch2:
>> Introduces the self-save API and leverages upon the struct interface to
>> add another supported mode in the mix of saving and restoring. It also
>> enforces that in case both modes are supported self-save is chosen over
>> self-restore
>>
>> The commit also parses the device-tree and populate support for
>> self-save and self-restore in the supported mask
>>
>> Patch3:
>> Introduce an optimization to allow preference to choose between one more
>> over the one when both both modes are supported. This optimization can
>> allow for better performance for the SPRs that don't change in value and
>> hence self-restore is a better alternative, and in cases when it is
>> known for values to change self-save is more convenient.
>>
>> Pratik Rajesh Sampat (3):
>> powerpc/powernv: Introduce interface for self-restore support
>> powerpc/powernv: Introduce support and parsing for self-save API
>> powerpc/powernv: Preference optimization for SPRs with constant values
>>
>> .../bindings/powerpc/opal/power-mgt.txt | 18 +
>> arch/powerpc/include/asm/opal-api.h | 3 +-
>> arch/powerpc/include/asm/opal.h | 1 +
>> arch/powerpc/platforms/powernv/idle.c | 385 +++++++++++++++---
>> arch/powerpc/platforms/powernv/opal-call.c | 1 +
>> 5 files changed, 351 insertions(+), 57 deletions(-)
>>
>> --
>> 2.17.1
>>
^ permalink raw reply
* [PATCH 3/3] mm/hugetlb: Introduce HAVE_ARCH_CLEAR_HUGEPAGE_FLAGS
From: Anshuman Khandual @ 2020-04-14 11:44 UTC (permalink / raw)
To: linux-mm
Cc: Rich Felker, linux-ia64, linux-sh, Catalin Marinas,
Heiko Carstens, linux-kernel, James E.J. Bottomley,
Paul Mackerras, H. Peter Anvin, sparclinux, linux-riscv,
Will Deacon, linux-arch, linux-s390, Yoshinori Sato, Helge Deller,
x86, Russell King, Christian Borntraeger, Ingo Molnar, Fenghua Yu,
Vasily Gorbik, Anshuman Khandual, Thomas Bogendoerfer,
Borislav Petkov, Paul Walmsley, Thomas Gleixner, linux-arm-kernel,
Tony Luck, linux-parisc, linux-mips, Palmer Dabbelt, linuxppc-dev,
David S. Miller, Mike Kravetz
In-Reply-To: <1586864670-21799-1-git-send-email-anshuman.khandual@arm.com>
There are multiple similar definitions for arch_clear_hugepage_flags() on
various platforms. This introduces HAVE_ARCH_CLEAR_HUGEPAGE_FLAGS for those
platforms that need to define their own arch_clear_hugepage_flags() while
also providing a generic fallback definition for others to use. This help
reduce code duplication.
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
Cc: Helge Deller <deller@gmx.de>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Rich Felker <dalias@libc.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: x86@kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-ia64@vger.kernel.org
Cc: linux-mips@vger.kernel.org
Cc: linux-parisc@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-riscv@lists.infradead.org
Cc: linux-s390@vger.kernel.org
Cc: linux-sh@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Cc: linux-mm@kvack.org
Cc: linux-arch@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/arm/include/asm/hugetlb.h | 1 +
arch/arm64/include/asm/hugetlb.h | 1 +
arch/ia64/include/asm/hugetlb.h | 4 ----
arch/mips/include/asm/hugetlb.h | 4 ----
arch/parisc/include/asm/hugetlb.h | 4 ----
arch/powerpc/include/asm/hugetlb.h | 4 ----
arch/riscv/include/asm/hugetlb.h | 4 ----
arch/s390/include/asm/hugetlb.h | 1 +
arch/sh/include/asm/hugetlb.h | 1 +
arch/sparc/include/asm/hugetlb.h | 4 ----
arch/x86/include/asm/hugetlb.h | 4 ----
include/linux/hugetlb.h | 4 ++++
12 files changed, 8 insertions(+), 28 deletions(-)
diff --git a/arch/arm/include/asm/hugetlb.h b/arch/arm/include/asm/hugetlb.h
index 9ecd516d1ff7..7107c1e6f020 100644
--- a/arch/arm/include/asm/hugetlb.h
+++ b/arch/arm/include/asm/hugetlb.h
@@ -14,6 +14,7 @@
#include <asm/hugetlb-3level.h>
#include <asm-generic/hugetlb.h>
+#define HAVE_ARCH_CLEAR_HUGEPAGE_FLAGS
static inline void arch_clear_hugepage_flags(struct page *page)
{
clear_bit(PG_dcache_clean, &page->flags);
diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h
index 8f58e052697a..5369cf26a05d 100644
--- a/arch/arm64/include/asm/hugetlb.h
+++ b/arch/arm64/include/asm/hugetlb.h
@@ -17,6 +17,7 @@
extern bool arch_hugetlb_migration_supported(struct hstate *h);
#endif
+#define HAVE_ARCH_CLEAR_HUGEPAGE_FLAGS
static inline void arch_clear_hugepage_flags(struct page *page)
{
clear_bit(PG_dcache_clean, &page->flags);
diff --git a/arch/ia64/include/asm/hugetlb.h b/arch/ia64/include/asm/hugetlb.h
index f17c1e228045..2ecff18a32e5 100644
--- a/arch/ia64/include/asm/hugetlb.h
+++ b/arch/ia64/include/asm/hugetlb.h
@@ -28,10 +28,6 @@ static inline void huge_ptep_clear_flush(struct vm_area_struct *vma,
{
}
-static inline void arch_clear_hugepage_flags(struct page *page)
-{
-}
-
#include <asm-generic/hugetlb.h>
#endif /* _ASM_IA64_HUGETLB_H */
diff --git a/arch/mips/include/asm/hugetlb.h b/arch/mips/include/asm/hugetlb.h
index 8b201e281f67..10e3be870df7 100644
--- a/arch/mips/include/asm/hugetlb.h
+++ b/arch/mips/include/asm/hugetlb.h
@@ -75,10 +75,6 @@ static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma,
return changed;
}
-static inline void arch_clear_hugepage_flags(struct page *page)
-{
-}
-
#include <asm-generic/hugetlb.h>
#endif /* __ASM_HUGETLB_H */
diff --git a/arch/parisc/include/asm/hugetlb.h b/arch/parisc/include/asm/hugetlb.h
index 411d9d867baa..a69cf9efb0c1 100644
--- a/arch/parisc/include/asm/hugetlb.h
+++ b/arch/parisc/include/asm/hugetlb.h
@@ -42,10 +42,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma,
unsigned long addr, pte_t *ptep,
pte_t pte, int dirty);
-static inline void arch_clear_hugepage_flags(struct page *page)
-{
-}
-
#include <asm-generic/hugetlb.h>
#endif /* _ASM_PARISC64_HUGETLB_H */
diff --git a/arch/powerpc/include/asm/hugetlb.h b/arch/powerpc/include/asm/hugetlb.h
index 9dd50e1e4fe5..d259e056e6cf 100644
--- a/arch/powerpc/include/asm/hugetlb.h
+++ b/arch/powerpc/include/asm/hugetlb.h
@@ -61,10 +61,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma,
unsigned long addr, pte_t *ptep,
pte_t pte, int dirty);
-static inline void arch_clear_hugepage_flags(struct page *page)
-{
-}
-
#include <asm-generic/hugetlb.h>
#else /* ! CONFIG_HUGETLB_PAGE */
diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h
index 866f6ae6467c..a5c2ca1d1cd8 100644
--- a/arch/riscv/include/asm/hugetlb.h
+++ b/arch/riscv/include/asm/hugetlb.h
@@ -5,8 +5,4 @@
#include <asm-generic/hugetlb.h>
#include <asm/page.h>
-static inline void arch_clear_hugepage_flags(struct page *page)
-{
-}
-
#endif /* _ASM_RISCV_HUGETLB_H */
diff --git a/arch/s390/include/asm/hugetlb.h b/arch/s390/include/asm/hugetlb.h
index 7d27ea96ec2f..9f067a66609b 100644
--- a/arch/s390/include/asm/hugetlb.h
+++ b/arch/s390/include/asm/hugetlb.h
@@ -35,6 +35,7 @@ static inline int prepare_hugepage_range(struct file *file,
return 0;
}
+#define HAVE_ARCH_CLEAR_HUGEPAGE_FLAGS
static inline void arch_clear_hugepage_flags(struct page *page)
{
clear_bit(PG_arch_1, &page->flags);
diff --git a/arch/sh/include/asm/hugetlb.h b/arch/sh/include/asm/hugetlb.h
index 536ad2cb8aa4..869b5a947f07 100644
--- a/arch/sh/include/asm/hugetlb.h
+++ b/arch/sh/include/asm/hugetlb.h
@@ -26,6 +26,7 @@ static inline void huge_ptep_clear_flush(struct vm_area_struct *vma,
{
}
+#define HAVE_ARCH_CLEAR_HUGEPAGE_FLAGS
static inline void arch_clear_hugepage_flags(struct page *page)
{
clear_bit(PG_dcache_clean, &page->flags);
diff --git a/arch/sparc/include/asm/hugetlb.h b/arch/sparc/include/asm/hugetlb.h
index a056fe1119f5..53838a173f62 100644
--- a/arch/sparc/include/asm/hugetlb.h
+++ b/arch/sparc/include/asm/hugetlb.h
@@ -47,10 +47,6 @@ static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma,
return changed;
}
-static inline void arch_clear_hugepage_flags(struct page *page)
-{
-}
-
#define __HAVE_ARCH_HUGETLB_FREE_PGD_RANGE
void hugetlb_free_pgd_range(struct mmu_gather *tlb, unsigned long addr,
unsigned long end, unsigned long floor,
diff --git a/arch/x86/include/asm/hugetlb.h b/arch/x86/include/asm/hugetlb.h
index cc98f79074d0..1721b1aadeb1 100644
--- a/arch/x86/include/asm/hugetlb.h
+++ b/arch/x86/include/asm/hugetlb.h
@@ -7,8 +7,4 @@
#define hugepages_supported() boot_cpu_has(X86_FEATURE_PSE)
-static inline void arch_clear_hugepage_flags(struct page *page)
-{
-}
-
#endif /* _ASM_X86_HUGETLB_H */
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 2342e5a8d1dd..359cfa1b6f54 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -544,6 +544,10 @@ static inline int is_hugepage_only_range(struct mm_struct *mm,
}
#endif
+#ifndef HAVE_ARCH_CLEAR_HUGEPAGE_FLAGS
+static inline void arch_clear_hugepage_flags(struct page *page) { }
+#endif
+
#ifndef arch_make_huge_pte
static inline pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma,
struct page *page, int writable)
--
2.20.1
^ permalink raw reply related
* [PATCH 2/3] mm/hugetlb: Introduce HAVE_ARCH_IS_HUGEPAGE_ONLY_RANGE
From: Anshuman Khandual @ 2020-04-14 11:44 UTC (permalink / raw)
To: linux-mm
Cc: Rich Felker, linux-ia64, linux-sh, Catalin Marinas,
Heiko Carstens, linux-kernel, James E.J. Bottomley,
Paul Mackerras, H. Peter Anvin, sparclinux, linux-riscv,
Will Deacon, linux-arch, linux-s390, Yoshinori Sato, Helge Deller,
x86, Russell King, Christian Borntraeger, Ingo Molnar, Fenghua Yu,
Vasily Gorbik, Anshuman Khandual, Thomas Bogendoerfer,
Borislav Petkov, Paul Walmsley, Thomas Gleixner, linux-arm-kernel,
Tony Luck, linux-parisc, linux-mips, Palmer Dabbelt, linuxppc-dev,
David S. Miller, Mike Kravetz
In-Reply-To: <1586864670-21799-1-git-send-email-anshuman.khandual@arm.com>
There are multiple similar definitions for is_hugepage_only_range() across
platforms. This introduces HAVE_ARCH_IS_HUGEPAGE_ONLY_RANGE for platforms
that need to define their own is_hugepage_only_range() but otherwise there
is a generic fallback definition for others to use. This help reduce code
duplication.
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
Cc: Helge Deller <deller@gmx.de>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Rich Felker <dalias@libc.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: x86@kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-ia64@vger.kernel.org
Cc: linux-mips@vger.kernel.org
Cc: linux-parisc@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-riscv@lists.infradead.org
Cc: linux-s390@vger.kernel.org
Cc: linux-sh@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Cc: linux-mm@kvack.org
Cc: linux-arch@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
arch/arm/include/asm/hugetlb.h | 6 ------
arch/arm64/include/asm/hugetlb.h | 6 ------
arch/ia64/include/asm/hugetlb.h | 1 +
arch/mips/include/asm/hugetlb.h | 7 -------
arch/parisc/include/asm/hugetlb.h | 6 ------
arch/powerpc/include/asm/hugetlb.h | 1 +
arch/riscv/include/asm/hugetlb.h | 6 ------
arch/s390/include/asm/hugetlb.h | 7 -------
arch/sh/include/asm/hugetlb.h | 6 ------
arch/sparc/include/asm/hugetlb.h | 6 ------
arch/x86/include/asm/hugetlb.h | 6 ------
include/linux/hugetlb.h | 8 ++++++++
12 files changed, 10 insertions(+), 56 deletions(-)
diff --git a/arch/arm/include/asm/hugetlb.h b/arch/arm/include/asm/hugetlb.h
index 318dcf5921ab..9ecd516d1ff7 100644
--- a/arch/arm/include/asm/hugetlb.h
+++ b/arch/arm/include/asm/hugetlb.h
@@ -14,12 +14,6 @@
#include <asm/hugetlb-3level.h>
#include <asm-generic/hugetlb.h>
-static inline int is_hugepage_only_range(struct mm_struct *mm,
- unsigned long addr, unsigned long len)
-{
- return 0;
-}
-
static inline void arch_clear_hugepage_flags(struct page *page)
{
clear_bit(PG_dcache_clean, &page->flags);
diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h
index b88878ddc88b..8f58e052697a 100644
--- a/arch/arm64/include/asm/hugetlb.h
+++ b/arch/arm64/include/asm/hugetlb.h
@@ -17,12 +17,6 @@
extern bool arch_hugetlb_migration_supported(struct hstate *h);
#endif
-static inline int is_hugepage_only_range(struct mm_struct *mm,
- unsigned long addr, unsigned long len)
-{
- return 0;
-}
-
static inline void arch_clear_hugepage_flags(struct page *page)
{
clear_bit(PG_dcache_clean, &page->flags);
diff --git a/arch/ia64/include/asm/hugetlb.h b/arch/ia64/include/asm/hugetlb.h
index 36cc0396b214..f17c1e228045 100644
--- a/arch/ia64/include/asm/hugetlb.h
+++ b/arch/ia64/include/asm/hugetlb.h
@@ -13,6 +13,7 @@ void hugetlb_free_pgd_range(struct mmu_gather *tlb, unsigned long addr,
int prepare_hugepage_range(struct file *file,
unsigned long addr, unsigned long len);
+#define HAVE_ARCH_IS_HUGEPAGE_ONLY_RANGE
static inline int is_hugepage_only_range(struct mm_struct *mm,
unsigned long addr,
unsigned long len)
diff --git a/arch/mips/include/asm/hugetlb.h b/arch/mips/include/asm/hugetlb.h
index 425bb6fc3bda..8b201e281f67 100644
--- a/arch/mips/include/asm/hugetlb.h
+++ b/arch/mips/include/asm/hugetlb.h
@@ -11,13 +11,6 @@
#include <asm/page.h>
-static inline int is_hugepage_only_range(struct mm_struct *mm,
- unsigned long addr,
- unsigned long len)
-{
- return 0;
-}
-
#define __HAVE_ARCH_PREPARE_HUGEPAGE_RANGE
static inline int prepare_hugepage_range(struct file *file,
unsigned long addr,
diff --git a/arch/parisc/include/asm/hugetlb.h b/arch/parisc/include/asm/hugetlb.h
index 7cb595dcb7d7..411d9d867baa 100644
--- a/arch/parisc/include/asm/hugetlb.h
+++ b/arch/parisc/include/asm/hugetlb.h
@@ -12,12 +12,6 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr,
pte_t *ptep);
-static inline int is_hugepage_only_range(struct mm_struct *mm,
- unsigned long addr,
- unsigned long len) {
- return 0;
-}
-
/*
* If the arch doesn't supply something else, assume that hugepage
* size aligned regions are ok without further preparation.
diff --git a/arch/powerpc/include/asm/hugetlb.h b/arch/powerpc/include/asm/hugetlb.h
index bd6504c28c2f..9dd50e1e4fe5 100644
--- a/arch/powerpc/include/asm/hugetlb.h
+++ b/arch/powerpc/include/asm/hugetlb.h
@@ -22,6 +22,7 @@ void flush_dcache_icache_hugepage(struct page *page);
int slice_is_hugepage_only_range(struct mm_struct *mm, unsigned long addr,
unsigned long len);
+#define HAVE_ARCH_IS_HUGEPAGE_ONLY_RANGE
static inline int is_hugepage_only_range(struct mm_struct *mm,
unsigned long addr,
unsigned long len)
diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h
index 728a5db66597..866f6ae6467c 100644
--- a/arch/riscv/include/asm/hugetlb.h
+++ b/arch/riscv/include/asm/hugetlb.h
@@ -5,12 +5,6 @@
#include <asm-generic/hugetlb.h>
#include <asm/page.h>
-static inline int is_hugepage_only_range(struct mm_struct *mm,
- unsigned long addr,
- unsigned long len) {
- return 0;
-}
-
static inline void arch_clear_hugepage_flags(struct page *page)
{
}
diff --git a/arch/s390/include/asm/hugetlb.h b/arch/s390/include/asm/hugetlb.h
index de8f0bf5f238..7d27ea96ec2f 100644
--- a/arch/s390/include/asm/hugetlb.h
+++ b/arch/s390/include/asm/hugetlb.h
@@ -21,13 +21,6 @@ pte_t huge_ptep_get(pte_t *ptep);
pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
unsigned long addr, pte_t *ptep);
-static inline bool is_hugepage_only_range(struct mm_struct *mm,
- unsigned long addr,
- unsigned long len)
-{
- return false;
-}
-
/*
* If the arch doesn't supply something else, assume that hugepage
* size aligned regions are ok without further preparation.
diff --git a/arch/sh/include/asm/hugetlb.h b/arch/sh/include/asm/hugetlb.h
index 6f025fe18146..536ad2cb8aa4 100644
--- a/arch/sh/include/asm/hugetlb.h
+++ b/arch/sh/include/asm/hugetlb.h
@@ -5,12 +5,6 @@
#include <asm/cacheflush.h>
#include <asm/page.h>
-static inline int is_hugepage_only_range(struct mm_struct *mm,
- unsigned long addr,
- unsigned long len) {
- return 0;
-}
-
/*
* If the arch doesn't supply something else, assume that hugepage
* size aligned regions are ok without further preparation.
diff --git a/arch/sparc/include/asm/hugetlb.h b/arch/sparc/include/asm/hugetlb.h
index 3963f80d1cb3..a056fe1119f5 100644
--- a/arch/sparc/include/asm/hugetlb.h
+++ b/arch/sparc/include/asm/hugetlb.h
@@ -20,12 +20,6 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr,
pte_t *ptep);
-static inline int is_hugepage_only_range(struct mm_struct *mm,
- unsigned long addr,
- unsigned long len) {
- return 0;
-}
-
#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH
static inline void huge_ptep_clear_flush(struct vm_area_struct *vma,
unsigned long addr, pte_t *ptep)
diff --git a/arch/x86/include/asm/hugetlb.h b/arch/x86/include/asm/hugetlb.h
index f65cfb48cfdd..cc98f79074d0 100644
--- a/arch/x86/include/asm/hugetlb.h
+++ b/arch/x86/include/asm/hugetlb.h
@@ -7,12 +7,6 @@
#define hugepages_supported() boot_cpu_has(X86_FEATURE_PSE)
-static inline int is_hugepage_only_range(struct mm_struct *mm,
- unsigned long addr,
- unsigned long len) {
- return 0;
-}
-
static inline void arch_clear_hugepage_flags(struct page *page)
{
}
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 1e897e4168ac..2342e5a8d1dd 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -536,6 +536,14 @@ static inline unsigned int blocks_per_huge_page(struct hstate *h)
#include <asm/hugetlb.h>
+#ifndef HAVE_ARCH_IS_HUGEPAGE_ONLY_RANGE
+static inline int is_hugepage_only_range(struct mm_struct *mm,
+ unsigned long addr, unsigned long len)
+{
+ return 0;
+}
+#endif
+
#ifndef arch_make_huge_pte
static inline pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma,
struct page *page, int writable)
--
2.20.1
^ permalink raw reply related
* [PATCH 0/3] mm/hugetlb: Add some new arch enabling macros
From: Anshuman Khandual @ 2020-04-14 11:44 UTC (permalink / raw)
To: linux-mm
Cc: Rich Felker, linux-ia64, linux-sh, Catalin Marinas,
Heiko Carstens, linux-kernel, James E.J. Bottomley,
Paul Mackerras, H. Peter Anvin, sparclinux, linux-riscv,
Will Deacon, linux-arch, linux-s390, Yoshinori Sato, Helge Deller,
x86, Russell King, Christian Borntraeger, Ingo Molnar, Fenghua Yu,
Vasily Gorbik, Anshuman Khandual, Thomas Bogendoerfer,
Borislav Petkov, Paul Walmsley, Thomas Gleixner, linux-arm-kernel,
Tony Luck, linux-parisc, linux-mips, Palmer Dabbelt, linuxppc-dev,
David S. Miller, Mike Kravetz
This series adds two macros which platforms can enable to override generic
fallback definitions. Before that it drops __HAVE_ARCH_HUGE_PTEP_GET from
arm64 platform.
1. HAVE_ARCH_IS_HUGEPAGE_ONLY_RANGE - is_hugepage_only_range()
2. HAVE_ARCH_CLEAR_HUGEPAGE_FLAGS - arch_clear_hugepage_flags()
This has been boot tested on arm64 and x86 platforms but built tested on
some more platforms including the changed ones here. This series applies
on v5.7-rc1. After this arm (32 bit) remains the sole platform defining
it's own huge_ptep_get() via __HAVE_ARCH_HUGE_PTEP_GET.
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
Cc: Helge Deller <deller@gmx.de>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Rich Felker <dalias@libc.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: x86@kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-ia64@vger.kernel.org
Cc: linux-mips@vger.kernel.org
Cc: linux-parisc@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-riscv@lists.infradead.org
Cc: linux-s390@vger.kernel.org
Cc: linux-sh@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Cc: linux-mm@kvack.org
Cc: linux-arch@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Anshuman Khandual (3):
arm64/mm: Drop __HAVE_ARCH_HUGE_PTEP_GET
mm/hugetlb: Introduce HAVE_ARCH_IS_HUGEPAGE_ONLY_RANGE
mm/hugetlb: Introduce HAVE_ARCH_CLEAR_HUGEPAGE_FLAGS
arch/arm/include/asm/hugetlb.h | 7 +------
arch/arm64/include/asm/hugetlb.h | 13 +------------
arch/ia64/include/asm/hugetlb.h | 5 +----
arch/mips/include/asm/hugetlb.h | 11 -----------
arch/parisc/include/asm/hugetlb.h | 10 ----------
arch/powerpc/include/asm/hugetlb.h | 5 +----
arch/riscv/include/asm/hugetlb.h | 10 ----------
arch/s390/include/asm/hugetlb.h | 8 +-------
arch/sh/include/asm/hugetlb.h | 7 +------
arch/sparc/include/asm/hugetlb.h | 10 ----------
arch/x86/include/asm/hugetlb.h | 10 ----------
include/linux/hugetlb.h | 12 ++++++++++++
12 files changed, 18 insertions(+), 90 deletions(-)
--
2.20.1
^ permalink raw reply
* Re: [PATCH v2 4/4] mm/vmalloc: Hugepage vmalloc mappings
From: Nicholas Piggin @ 2020-04-14 11:39 UTC (permalink / raw)
To: Matthew Wilcox
Cc: linux-arch, Will Deacon, Catalin Marinas, x86, linux-kernel,
linux-mm, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
Thomas Gleixner, linuxppc-dev, linux-arm-kernel
In-Reply-To: <20200413134106.GN21484@bombadil.infradead.org>
Excerpts from Matthew Wilcox's message of April 13, 2020 11:41 pm:
> On Mon, Apr 13, 2020 at 10:53:03PM +1000, Nicholas Piggin wrote:
>> +static int vmap_pages_range_noflush(unsigned long start, unsigned long end,
>> + pgprot_t prot, struct page **pages,
>> + unsigned int page_shift)
>> +{
>> + if (page_shift == PAGE_SIZE) {
>
> ... I think you meant 'page_shift == PAGE_SHIFT'
Thanks, good catch. I obviously didn't test the fallback path (the
other path works for small pages, it just goes one at a time).
> Overall I like this series, although it's a bit biased towards CPUs
> which have page sizes which match PMD/PUD sizes. It doesn't offer the
> possibility of using 64kB page sizes on ARM, for example.
No, it's just an incremental step on existing huge vmap stuff in
tree, so such a thing would be out of scope.
> But it's a
> step in the right direction.
>
I don't know about moving kernel maps away from a generic Linux page
table format. I quite like moving to it and making it as generic as
possible.
On the other hand, I also would like to make some arch-specific
allowances for certain special cases that may not fit within the
standard page table format, but it might be a much more specific and
limited interface than the general vmalloc stuff.
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH v2 1/4] mm/vmalloc: fix vmalloc_to_page for huge vmap mappings
From: Nicholas Piggin @ 2020-04-14 11:31 UTC (permalink / raw)
To: Matthew Wilcox
Cc: linux-arch, Will Deacon, Catalin Marinas, x86, linux-kernel,
linux-mm, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
Thomas Gleixner, linuxppc-dev, linux-arm-kernel
In-Reply-To: <20200413133444.GM21484@bombadil.infradead.org>
Excerpts from Matthew Wilcox's message of April 13, 2020 11:34 pm:
> On Mon, Apr 13, 2020 at 10:53:00PM +1000, Nicholas Piggin wrote:
>> vmalloc_to_page returns NULL for addresses mapped by larger pages[*].
>> Whether or not a vmap is huge depends on the architecture details,
>> alignments, boot options, etc., which the caller can not be expected
>> to know. Therefore HUGE_VMAP is a regression for vmalloc_to_page.
>>
>> This change teaches vmalloc_to_page about larger pages, and returns
>> the struct page that corresponds to the offset within the large page.
>> This makes the API agnostic to mapping implementation details.
>
> I'm trying to get us away from returning tail pages from various
> functions. How much of a pain would it be to return the head page
> instead of the tail page?
Well, this is a fix for the interface for HUGE_VMAP stuff so it
doesn't really make sense to change the implementation here. If you
want to change or make a different API that would be a later patch, no?
> Obviously the implementation gets simpler,
> but can the callers cope? I've been focusing on the page cache, so I
> haven't been looking at the vmalloc side of things at all.
Well callers that operate on ioremap today (and vmalloc tomorrow) won't
cope, because they're expecting a base page. If you wanted to change it
I suspect the way to go would be introduce a new function and move
everyone over individually.
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH v6 4/4] Advertise the self-save and self-restore attributes in the device tree
From: Gautham R Shenoy @ 2020-04-14 7:08 UTC (permalink / raw)
To: Pratik Rajesh Sampat
Cc: ego, pratik.r.sampat, linuxram, linux-kernel, linuxppc-dev,
oohall, skiboot
In-Reply-To: <20200326070917.12744-5-psampat@linux.ibm.com>
Hello Pratik,
On Thu, Mar 26, 2020 at 12:39:17PM +0530, Pratik Rajesh Sampat wrote:
> Support for self save and self restore interface is advertised in the
> device tree, along with the list of SPRs it supports for each.
>
> The Special Purpose Register identification is encoded in a 2048 bitmask
> structure, where each bit signifies the identification key of that SPR
> which is consistent with that of the POWER architecture set for that
> register.
>
> Signed-off-by: Pratik Rajesh Sampat <psampat@linux.ibm.com>
> ---
> .../ibm,opal/power-mgt/self-restore.rst | 27 ++++
> .../ibm,opal/power-mgt/self-save.rst | 27 ++++
> hw/slw.c | 116 ++++++++++++++++++
> include/skiboot.h | 1 +
> 4 files changed, 171 insertions(+)
> create mode 100644 doc/device-tree/ibm,opal/power-mgt/self-restore.rst
> create mode 100644 doc/device-tree/ibm,opal/power-mgt/self-save.rst
>
> diff --git a/doc/device-tree/ibm,opal/power-mgt/self-restore.rst b/doc/device-tree/ibm,opal/power-mgt/self-restore.rst
> new file mode 100644
> index 00000000..2a2269f7
> --- /dev/null
> +++ b/doc/device-tree/ibm,opal/power-mgt/self-restore.rst
> @@ -0,0 +1,27 @@
> +ibm,opal/power-mgt/self-restore device tree entries
> +===================================================
> +
> +This node exports the bitmask representing the special purpose registers that
> +the self-restore API currently supports.
> +
> +Example:
> +
> +.. code-block:: dts
> +
> + self-restore {
> + sprn-bitmask = <0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x42010000 0x0 0x0
> + 0x20000 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
> + 0x0 0x0 0x100000 0x900000 0x0 0x0 0x530000 0x0 0x0 0x0
> + 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
> + 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
> + 0x0 0x10000>;
> + phandle = <0x1c7>;
> + };
> +
> +sprn-bitmask
> +------------
> +
> +This property is a bitmask of of all the existing SPRs and if the SPR is
> +supported, the corresponding bit of the SPR number is set to 1.
> +The representation of the bits are left-right, i.e the MSB of the first
> +doubleword represants the 0th bit.
> diff --git a/doc/device-tree/ibm,opal/power-mgt/self-save.rst b/doc/device-tree/ibm,opal/power-mgt/self-save.rst
> new file mode 100644
> index 00000000..c367720e
> --- /dev/null
> +++ b/doc/device-tree/ibm,opal/power-mgt/self-save.rst
> @@ -0,0 +1,27 @@
> +ibm,opal/power-mgt/self-save device tree entries
> +===================================================
> +
> +This node exports the bitmask representing the special purpose registers that
> +the self-save API currently supports.
> +
> +Example:
> +
> +.. code-block:: dts
> +
> + self-save {
> + sprn-bitmask = <0x0 0x0 0x0 0x0 0x100000 0x0 0x0 0x0 0x42010000 0x0 0x0
> + 0x20000 0x0 0x0 0x0 0x10000 0x0 0x0 0x0 0x0 0x0 0x0 0x0
> + 0x0 0x0 0x0 0x100000 0x840000 0x0 0x0 0x0 0x0 0x0 0x0
> + 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
> + 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
> + 0x0 0x10000>;
> + phandle = <0x1c8>;
> + };
> +
> +sprn-bitmask
> +------------
> +
> +This property is a bitmask of of all the existing SPRs and if the SPR is
> +supported, the corresponding bit of the SPR number is set to 1.
> +The representation of the bits are left-right, i.e the MSB of the first
> +doubleword represants the 0th bit.
> diff --git a/hw/slw.c b/hw/slw.c
> index 6a09cc2c..9d1fe2c5 100644
> --- a/hw/slw.c
> +++ b/hw/slw.c
> @@ -29,6 +29,7 @@
> #include <sbe_xip_image.h>
>
> static uint32_t slw_saved_reset[0x100];
> +#define SPR_BITMAP_LENGTH 2048
>
> static bool slw_current_le = false;
>
> @@ -750,6 +751,119 @@ static void slw_late_init_p9(struct proc_chip *chip)
> }
> }
>
> +/* Add device tree properties to determine self-save | restore */
> +void add_cpu_self_save_restore_properties(void)
> +{
> + struct dt_node *self_restore, *self_save, *power_mgt;
> + uint64_t *self_save_mask, *self_restore_mask;
> + bool self_save_supported = true;
> + uint64_t compVector = -1;
> + struct proc_chip *chip;
> + int i, rc;
> +
> + const uint64_t self_restore_regs[] = {
> + P8_SPR_HRMOR,
> + P8_SPR_HMEER,
> + P8_SPR_PMICR,
> + P8_SPR_PMCR,
> + P8_SPR_HID0,
> + P8_SPR_HID1,
> + P8_SPR_HID4,
> + P8_SPR_HID5,
> + P8_SPR_HSPRG0,
> + P8_SPR_LPCR,
> + P8_MSR_MSR
> + };
> +
> + const uint64_t self_save_regs[] = {
> + P9_STOP_SPR_DAWR,
> + P9_STOP_SPR_HSPRG0,
> + P9_STOP_SPR_LDBAR,
> + P9_STOP_SPR_LPCR,
> + P9_STOP_SPR_PSSCR,
> + P9_STOP_SPR_MSR,
> + P9_STOP_SPR_HRMOR,
> + P9_STOP_SPR_HMEER,
> + P9_STOP_SPR_PMCR,
> + P9_STOP_SPR_PTCR
> + };
> +
> + chip = next_chip(NULL);
> + assert(chip);
> + rc = proc_stop_api_discover_capability((void *) chip->homer_base,
> + &compVector);
> + if (rc == STOP_SAVE_ARG_INVALID_IMG) {
> + prlog(PR_DEBUG, "HOMER BASE INVALID\n");
> + return;
> + } else if (rc == STOP_SAVE_API_IMG_INCOMPATIBLE) {
> + prlog(PR_DEBUG, "STOP API running incompatible versions\n");
> + if ((compVector & SELF_RESTORE_VER_MISMATCH) == 0) {
> + prlog(PR_DEBUG, "Self-save API unsupported\n");
> + self_save_supported = false;
> + }
> + }
> +
> + power_mgt = dt_find_by_path(dt_root, "/ibm,opal/power-mgt");
> + if (!power_mgt) {
> + prerror("OCC: dt node /ibm,opal/power-mgt not found\n");
The prerror/warning in this function should be prefixed with "SLW"
instead of "OCC" here. This function has nothing to do with OCC.
> + return;
> + }
> +
> + self_restore = dt_new(power_mgt, "self-restore");
> + if (!self_restore) {
> + prerror("OCC: Failed to create self restore node");
Ditto.
> + return;
> + }
> +
> + self_restore_mask = zalloc(SPR_BITMAP_LENGTH / 8);
> + if (!self_restore_mask)
> + return;
> +
> + for (i = 0; i < ARRAY_SIZE(self_restore_regs); i++) {
> + int bitmask_idx = self_restore_regs[i] / 64;
> + uint64_t bitmask_pos = self_restore_regs[i] % 64;
> + self_restore_mask[bitmask_idx] |= 1ul << bitmask_pos;
> + }
> +
> + for (i = 0; i < (SPR_BITMAP_LENGTH / 64); i++) {
> + self_restore_mask[i] = cpu_to_be64(self_restore_mask[i]);
> + }
> +
> + dt_add_property(self_restore, "sprn-bitmask", self_restore_mask,
> + SPR_BITMAP_LENGTH / 8);
> + dt_add_property_string(self_restore, "compatible",
> + "ibm,opal-self-restore");
> + free(self_restore_mask);
> +
> + if (proc_gen != proc_gen_p9 || !self_save_supported)
We need a
prlog(PR_INFO, "SLW: self-save not supported on this platform");
?
> + return;
> +
> + self_save = dt_new(power_mgt, "self-save");
> + if (!self_save) {
> + prerror("OCC: Failed to create self save node");
> + return;
> + }
> +
> + self_save_mask = zalloc(SPR_BITMAP_LENGTH / 8);
> + if (!self_save_mask)
> + return;
> +
> + for (i = 0; i < ARRAY_SIZE(self_save_regs); i++) {
> + int bitmask_idx = self_save_regs[i] / 64;
> + uint64_t bitmask_pos = self_save_regs[i] % 64;
> + self_save_mask[bitmask_idx] |= 1ul << bitmask_pos;
> + }
> +
> + for (i = 0; i < (SPR_BITMAP_LENGTH / 64); i++) {
> + self_save_mask[i] = cpu_to_be64(self_save_mask[i]);
> + }
> +
> + dt_add_property(self_save, "sprn-bitmask", self_save_mask,
> + SPR_BITMAP_LENGTH / 8);
> + dt_add_property_string(self_save, "compatible", "ibm,opal-self-save");
> + free(self_save_mask);
> +}
> +
> /* Add device tree properties to describe idle states */
> void add_cpu_idle_state_properties(void)
> {
> @@ -1563,4 +1677,6 @@ void slw_init(void)
> }
> }
> add_cpu_idle_state_properties();
> + if (has_deep_states)
> + add_cpu_self_save_restore_properties();
> }
> diff --git a/include/skiboot.h b/include/skiboot.h
> index 9ced240e..d3631dea 100644
> --- a/include/skiboot.h
> +++ b/include/skiboot.h
> @@ -209,6 +209,7 @@ extern void early_uart_init(void);
> extern void homer_init(void);
> extern void slw_init(void);
> extern void add_cpu_idle_state_properties(void);
> +extern void add_cpu_self_save_restore_properties(void);
> extern void lpc_rtc_init(void);
Apart from these minor quibbles,
Reviewed-by: Gautham R. Shenoy <ego@linux.vnet.ibm.com>
>
> /* flash support */
> --
> 2.24.1
>
^ permalink raw reply
* Re: Build regressions/improvements in v5.7-rc1
From: Krzysztof Kozlowski @ 2020-04-14 11:19 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, virtualization, Geert Uytterhoeven, linux-kernel,
netdev
In-Reply-To: <20200414110627.GA1373@kozik-lap>
On Tue, Apr 14, 2020 at 01:06:27PM +0200, Krzysztof Kozlowski wrote:
> On Tue, Apr 14, 2020 at 08:23:32PM +1000, Michael Ellerman wrote:
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable-4k.h: error: expected ')' before '!=' token: => 58:40
> > >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable-4k.h: error: expected ')' before '==' token: => 57:37
> > >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable-4k.h: error: expected identifier or '(' before '!' token: => 56:25
> > >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable.h: error: expected ')' before '!=' token: => 163:40
> > >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable.h: error: expected ')' before '==' token: => 333:50
> > >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable.h: error: expected ')' before '^' token: => 333:36
> > >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable.h: error: expected identifier or '(' before '!' token: => 146:27, 144:24, 160:25, 161:24, 143:25
> > >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable.h: error: expected identifier or '(' before 'struct': => 77:21
> > >> + /kisskb/src/arch/powerpc/include/asm/nohash/pgtable.h: error: redefinition of 'pgd_huge': => 291:19
> > >> + /kisskb/src/arch/powerpc/include/asm/nohash/pte-book3e.h: error: redefinition of 'pte_mkprivileged': => 108:26
> > >> + /kisskb/src/arch/powerpc/include/asm/nohash/pte-book3e.h: error: redefinition of 'pte_mkuser': => 115:20
> > >> + /kisskb/src/arch/powerpc/kvm/book3s_64_vio_hv.c: error: 'struct kvm_arch' has no member named 'spapr_tce_tables': => 68:46, 68:2
> > >
> > > ppc64_book3e_allmodconfig
> >
> > Caused by:
> >
> > e93a1695d7fb ("iommu: Enable compile testing for some of drivers")
> >
> > Which did:
> >
> > config SPAPR_TCE_IOMMU
> > bool "sPAPR TCE IOMMU Support"
> > - depends on PPC_POWERNV || PPC_PSERIES
> > + depends on PPC_POWERNV || PPC_PSERIES || (PPC && COMPILE_TEST)
> >
> >
> > Which is just ... not right, the dependencies on the correct platform
> > are important, otherwise the build breaks.
>
> The SPAPR_TCE_IOMMU should compile fine. The actual trouble here is
> that KVM_BOOK3S_64 selects SPAPR_TCE_IOMMU which is a user-visible
> symbol. This is generally discouraged because of exactly this error -
> select ignores any dependencies.
>
> I can revert the COMPILE_TEST for SPAPR_TCE_IOMMU or change
> select->depends in KVM_BOOK3S_64. I think the latter is the proper
> change here.
Eh, not really, it looks more complex because there are more
dependencies (the book3s_64_vio_hv is pulled by KVM Makefile if
SPAPR_TCE_IOMMU is set).
I guess the revert of this part makes most sense.
Best regards,
Krzysztof
^ permalink raw reply
* [PATCH] target/ppc: Fix mtmsr(d) L=1 variant that loses interrupts
From: Nicholas Piggin @ 2020-04-14 11:11 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-stable, Nicholas Piggin, Cédric Le Goater, qemu-ppc,
Nathan Chancellor, linuxppc-dev, David Gibson
If mtmsr L=1 sets MSR[EE] while there is a maskable exception pending,
it does not cause an interrupt. This causes the test case to hang:
https://lists.gnu.org/archive/html/qemu-ppc/2019-10/msg00826.html
More recently, Linux reduced the occurance of operations (e.g., rfi)
which stop translation and allow pending interrupts to be processed.
This started causing hangs in Linux boot in long-running kernel tests,
running with '-d int' shows the decrementer stops firing despite DEC
wrapping and MSR[EE]=1.
https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-April/208301.html
The cause is the broken mtmsr L=1 behaviour, which is contrary to the
architecture. From Power ISA v3.0B, p.977, Move To Machine State Register,
Programming Note states:
If MSR[EE]=0 and an External, Decrementer, or Performance Monitor
exception is pending, executing an mtmsrd instruction that sets
MSR[EE] to 1 will cause the interrupt to occur before the next
instruction is executed, if no higher priority exception exists
Fix this by handling L=1 exactly the same way as L=0, modulo the MSR
bits altered.
The confusion arises from L=0 being "context synchronizing" whereas L=1
is "execution synchronizing", which is a weaker semantic. However this
is not a relaxation of the requirement that these exceptions cause
interrupts when MSR[EE]=1 (e.g., when mtmsr executes to completion as
TCG is doing here), rather it specifies how a pipelined processor can
have multiple instructions in flight where one may influence how another
behaves.
Cc: qemu-stable@nongnu.org
Reported-by: Anton Blanchard <anton@ozlabs.org>
Reported-by: Nathan Chancellor <natechancellor@gmail.com>
Tested-by: Nathan Chancellor <natechancellor@gmail.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
Thanks very much to Nathan for reporting and testing it, I added his
Tested-by tag despite a more polished patch, as the the basics are
still the same (and still fixes his test case here).
This bug possibly goes back to early v2.04 / mtmsrd L=1 support around
2007, and the code has been changed several times since then so may
require some backporting.
32-bit / mtmsr untested at the moment, I don't have an environment
handy.
target/ppc/translate.c | 46 +++++++++++++++++++++++++-----------------
1 file changed, 27 insertions(+), 19 deletions(-)
diff --git a/target/ppc/translate.c b/target/ppc/translate.c
index b207fb5386..9959259dba 100644
--- a/target/ppc/translate.c
+++ b/target/ppc/translate.c
@@ -4361,30 +4361,34 @@ static void gen_mtmsrd(DisasContext *ctx)
CHK_SV;
#if !defined(CONFIG_USER_ONLY)
+ if (tb_cflags(ctx->base.tb) & CF_USE_ICOUNT) {
+ gen_io_start();
+ }
if (ctx->opcode & 0x00010000) {
- /* Special form that does not need any synchronisation */
+ /* L=1 form only updates EE and RI */
TCGv t0 = tcg_temp_new();
+ TCGv t1 = tcg_temp_new();
tcg_gen_andi_tl(t0, cpu_gpr[rS(ctx->opcode)],
(1 << MSR_RI) | (1 << MSR_EE));
- tcg_gen_andi_tl(cpu_msr, cpu_msr,
+ tcg_gen_andi_tl(t1, cpu_msr,
~(target_ulong)((1 << MSR_RI) | (1 << MSR_EE)));
- tcg_gen_or_tl(cpu_msr, cpu_msr, t0);
+ tcg_gen_or_tl(t1, t1, t0);
+
+ gen_helper_store_msr(cpu_env, t1);
tcg_temp_free(t0);
+ tcg_temp_free(t1);
+
} else {
/*
* XXX: we need to update nip before the store if we enter
* power saving mode, we will exit the loop directly from
* ppc_store_msr
*/
- if (tb_cflags(ctx->base.tb) & CF_USE_ICOUNT) {
- gen_io_start();
- }
gen_update_nip(ctx, ctx->base.pc_next);
gen_helper_store_msr(cpu_env, cpu_gpr[rS(ctx->opcode)]);
- /* Must stop the translation as machine state (may have) changed */
- /* Note that mtmsr is not always defined as context-synchronizing */
- gen_stop_exception(ctx);
}
+ /* Must stop the translation as machine state (may have) changed */
+ gen_stop_exception(ctx);
#endif /* !defined(CONFIG_USER_ONLY) */
}
#endif /* defined(TARGET_PPC64) */
@@ -4394,15 +4398,23 @@ static void gen_mtmsr(DisasContext *ctx)
CHK_SV;
#if !defined(CONFIG_USER_ONLY)
- if (ctx->opcode & 0x00010000) {
- /* Special form that does not need any synchronisation */
+ if (tb_cflags(ctx->base.tb) & CF_USE_ICOUNT) {
+ gen_io_start();
+ }
+ if (ctx->opcode & 0x00010000) {
+ /* L=1 form only updates EE and RI */
TCGv t0 = tcg_temp_new();
+ TCGv t1 = tcg_temp_new();
tcg_gen_andi_tl(t0, cpu_gpr[rS(ctx->opcode)],
(1 << MSR_RI) | (1 << MSR_EE));
- tcg_gen_andi_tl(cpu_msr, cpu_msr,
+ tcg_gen_andi_tl(t1, cpu_msr,
~(target_ulong)((1 << MSR_RI) | (1 << MSR_EE)));
- tcg_gen_or_tl(cpu_msr, cpu_msr, t0);
+ tcg_gen_or_tl(t1, t1, t0);
+
+ gen_helper_store_msr(cpu_env, t1);
tcg_temp_free(t0);
+ tcg_temp_free(t1);
+
} else {
TCGv msr = tcg_temp_new();
@@ -4411,9 +4423,6 @@ static void gen_mtmsr(DisasContext *ctx)
* power saving mode, we will exit the loop directly from
* ppc_store_msr
*/
- if (tb_cflags(ctx->base.tb) & CF_USE_ICOUNT) {
- gen_io_start();
- }
gen_update_nip(ctx, ctx->base.pc_next);
#if defined(TARGET_PPC64)
tcg_gen_deposit_tl(msr, cpu_msr, cpu_gpr[rS(ctx->opcode)], 0, 32);
@@ -4422,10 +4431,9 @@ static void gen_mtmsr(DisasContext *ctx)
#endif
gen_helper_store_msr(cpu_env, msr);
tcg_temp_free(msr);
- /* Must stop the translation as machine state (may have) changed */
- /* Note that mtmsr is not always defined as context-synchronizing */
- gen_stop_exception(ctx);
}
+ /* Must stop the translation as machine state (may have) changed */
+ gen_stop_exception(ctx);
#endif
}
--
2.23.0
^ permalink raw reply related
* Re: Build regressions/improvements in v5.7-rc1
From: Krzysztof Kozlowski @ 2020-04-14 11:06 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, virtualization, Geert Uytterhoeven, linux-kernel,
netdev
In-Reply-To: <877dyijrh7.fsf@mpe.ellerman.id.au>
On Tue, Apr 14, 2020 at 08:23:32PM +1000, Michael Ellerman wrote:
> Geert Uytterhoeven <geert@linux-m68k.org> writes:
> > On Mon, 13 Apr 2020, Geert Uytterhoeven wrote:
> >> Below is the list of build error/warning regressions/improvements in
> >> v5.7-rc1[1] compared to v5.6[2].
> >>
> >> Summarized:
> >> - build errors: +132/-3
> >> - build warnings: +257/-79
> >>
> >> Happy fixing! ;-)
> >>
> >> Thanks to the linux-next team for providing the build service.
> >>
> >> [1] http://kisskb.ellerman.id.au/kisskb/branch/linus/head/8f3d9f354286745c751374f5f1fcafee6b3f3136/ (all 239 configs)
> >> [2] http://kisskb.ellerman.id.au/kisskb/branch/linus/head/7111951b8d4973bda27ff663f2cf18b663d15b48/ (all 239 configs)
> >>
> >>
> >> *** ERRORS ***
> >>
> >> 132 error regressions:
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/hash.h: error: implicit declaration of function 'pte_raw' [-Werror=implicit-function-declaration]: => 192:2
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/hash.h: error: implicit declaration of function 'pte_raw'; did you mean 'pte_read'? [-Werror=implicit-function-declaration]: => 192:8
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/mmu-hash.h: error: 'SLICE_NUM_HIGH' undeclared here (not in a function): => 698:2, 698:30
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/mmu-hash.h: error: 'struct mmu_psize_def' has no member named 'ap': => 207:28
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/mmu-hash.h: error: 'struct mmu_psize_def' has no member named 'avpnm': => 337:57
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/mmu-hash.h: error: 'struct mmu_psize_def' has no member named 'penc': => 411:49
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/mmu-hash.h: error: 'struct mmu_psize_def' has no member named 'penc'; did you mean 'enc'?: => 411:50
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/mmu-hash.h: error: 'struct mmu_psize_def' has no member named 'sllp': => 218:32, 219:26
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/mmu-hash.h: error: redefinition of 'mmu_psize_to_shift': => 195:28
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/mmu-hash.h: error: redefinition of 'shift_to_mmu_psize': => 185:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable-4k.h: error: implicit declaration of function 'pgd_raw' [-Werror=implicit-function-declaration]: => 35:3
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable-4k.h: error: implicit declaration of function 'pgd_raw'; did you mean 'pgd_val'? [-Werror=implicit-function-declaration]: => 35:13
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable-4k.h: error: implicit declaration of function 'pud_raw' [-Werror=implicit-function-declaration]: => 25:3
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable-4k.h: error: implicit declaration of function 'pud_raw'; did you mean 'pud_val'? [-Werror=implicit-function-declaration]: => 25:13
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable-4k.h: error: redefinition of 'hugepd_ok': => 44:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable-4k.h: error: redefinition of 'pgd_huge': => 29:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable-4k.h: error: redefinition of 'pmd_huge': => 9:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable-4k.h: error: redefinition of 'pud_huge': => 19:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: expected ')' before '!=' token: => 964:19, 921:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: expected ')' before '==' token: => 979:19, 801:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: expected ')' before '^' token: => 801:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: expected ')' before '__vmalloc_end': => 291:21, 274:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: expected identifier or '(' before '!' token: => 916:19, 904:19, 939:19, 959:19, 868:19, 873:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: expected identifier or '(' before 'struct': => 291:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: implicit declaration of function '__pgd_raw' [-Werror=implicit-function-declaration]: => 976:2
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: implicit declaration of function '__pgd_raw'; did you mean '__fdget_raw'? [-Werror=implicit-function-declaration]: => 976:9
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: implicit declaration of function '__pmd_raw' [-Werror=implicit-function-declaration]: => 1075:9, 1075:2
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: implicit declaration of function '__pte_raw' [-Werror=implicit-function-declaration]: => 552:2, 552:9
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: implicit declaration of function '__pud_raw' [-Werror=implicit-function-declaration]: => 935:9, 935:2
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: incompatible types when returning type 'int' but 'pgd_t' was expected: => 976:2
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: incompatible types when returning type 'int' but 'pgd_t' {aka 'struct <anonymous>'} was expected: => 976:9
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: incompatible types when returning type 'int' but 'pmd_t' was expected: => 1075:2
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: incompatible types when returning type 'int' but 'pmd_t' {aka 'struct <anonymous>'} was expected: => 1075:9
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: incompatible types when returning type 'int' but 'pte_t' was expected: => 675:2, 552:2, 971:2, 670:2, 1070:2, 647:2, 652:2, 642:2, 695:2, 660:2, 632:2, 665:2, 690:2, 627:2, 714:2, 930:2, 685:2, 637:2
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: incompatible types when returning type 'int' but 'pte_t' {aka 'struct <anonymous>'} was expected: => 1070:9, 695:9, 627:9, 685:9, 642:9, 930:9, 647:9, 690:9, 665:9, 660:9, 675:9, 670:9, 714:9, 552:9, 971:9, 637:9, 632:9, 652:9
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: incompatible types when returning type 'int' but 'pud_t' was expected: => 935:2
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: incompatible types when returning type 'int' but 'pud_t' {aka 'struct <anonymous>'} was expected: => 935:9
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of '__ptep_set_access_flags': => 789:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of '__ptep_test_and_clear_young': => 371:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of '__set_pte_at': => 815:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'huge_ptep_set_wrprotect': => 437:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pfn_pte': => 609:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pgd_clear': => 954:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pgd_is_leaf': => 1373:21, 1375:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pgd_pte': => 969:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pmd_clear': => 863:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pmd_is_leaf': => 1359:21, 1361:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pmd_pte': => 1068:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_access_permitted': => 587:20, 586:30
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_ci': => 853:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_clear': => 474:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_devmap': => 704:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_dirty': => 480:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_exec': => 495:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_exprotect': => 630:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_hw_valid': => 567:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_mkclean': => 635:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_mkdirty': => 663:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_mkexec': => 645:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_mkhuge': => 678:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_mkold': => 640:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_mkprivileged': => 688:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_mkpte': => 650:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_mkspecial': => 673:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_mkuser': => 693:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_mkwrite': => 655:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_mkyoung': => 668:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_modify': => 711:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_none': => 808:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_pfn': => 617:29
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_pgd': => 974:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_present': => 556:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_pud': => 933:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_read': => 421:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_special': => 490:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_unmap': => 1022:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_update': => 353:29
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_user': => 581:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_write': => 416:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_wrprotect': => 623:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pte_young': => 485:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'ptep_get_and_clear': => 451:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'ptep_get_and_clear_full': => 459:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'ptep_set_wrprotect': => 427:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pud_clear': => 911:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pud_is_leaf': => 1366:21, 1368:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: redefinition of 'pud_pte': => 928:21
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: static declaration of 'map_kernel_page' follows non-static declaration: => 1037:19
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: static declaration of 'vmemmap_create_mapping' follows non-static declaration: => 1049:29
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/pgtable.h: error: static declaration of 'vmemmap_remove_mapping' follows non-static declaration: => 1059:20
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/radix.h: error: implicit declaration of function 'pmd_raw' [-Werror=implicit-function-declaration]: => 221:2
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/radix.h: error: implicit declaration of function 'pmd_raw'; did you mean 'pmd_val'? [-Werror=implicit-function-declaration]: => 221:11
> >> + /kisskb/src/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h: error: 'struct mmu_psize_def' has no member named 'ap': => 11:30
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: 'struct kvm_vcpu_arch' has no member named 'book3s': => 316:19
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: 'struct kvm_vcpu_arch' has no member named 'fault_dar': => 396:19
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: 'struct kvm_vcpu_arch' has no member named 'fault_dar'; did you mean 'fault_dear'?: => 396:20
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_get_cr': => 343:19
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_get_ctr': => 363:21
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_get_fault_dar': => 394:21
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_get_gpr': => 333:21
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_get_lr': => 373:21
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_get_pc': => 383:21
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_get_xer': => 353:21
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_need_byteswap': => 389:20
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_set_cr': => 338:20
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_set_ctr': => 358:20
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_set_gpr': => 328:20
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_set_lr': => 368:20
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_set_pc': => 378:20
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_set_xer': => 348:20
> >> + /kisskb/src/arch/powerpc/include/asm/kvm_book3s.h: error: redefinition of 'kvmppc_supports_magic_page': => 405:20
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable-4k.h: error: expected ')' before '!=' token: => 58:40
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable-4k.h: error: expected ')' before '==' token: => 57:37
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable-4k.h: error: expected identifier or '(' before '!' token: => 56:25
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable.h: error: expected ')' before '!=' token: => 163:40
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable.h: error: expected ')' before '==' token: => 333:50
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable.h: error: expected ')' before '^' token: => 333:36
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable.h: error: expected identifier or '(' before '!' token: => 146:27, 144:24, 160:25, 161:24, 143:25
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/64/pgtable.h: error: expected identifier or '(' before 'struct': => 77:21
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/pgtable.h: error: redefinition of 'pgd_huge': => 291:19
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/pte-book3e.h: error: redefinition of 'pte_mkprivileged': => 108:26
> >> + /kisskb/src/arch/powerpc/include/asm/nohash/pte-book3e.h: error: redefinition of 'pte_mkuser': => 115:20
> >> + /kisskb/src/arch/powerpc/kvm/book3s_64_vio_hv.c: error: 'struct kvm_arch' has no member named 'spapr_tce_tables': => 68:46, 68:2
> >
> > ppc64_book3e_allmodconfig
>
> Caused by:
>
> e93a1695d7fb ("iommu: Enable compile testing for some of drivers")
>
> Which did:
>
> config SPAPR_TCE_IOMMU
> bool "sPAPR TCE IOMMU Support"
> - depends on PPC_POWERNV || PPC_PSERIES
> + depends on PPC_POWERNV || PPC_PSERIES || (PPC && COMPILE_TEST)
>
>
> Which is just ... not right, the dependencies on the correct platform
> are important, otherwise the build breaks.
The SPAPR_TCE_IOMMU should compile fine. The actual trouble here is
that KVM_BOOK3S_64 selects SPAPR_TCE_IOMMU which is a user-visible
symbol. This is generally discouraged because of exactly this error -
select ignores any dependencies.
I can revert the COMPILE_TEST for SPAPR_TCE_IOMMU or change
select->depends in KVM_BOOK3S_64. I think the latter is the proper
change here.
Best regards,
Krzysztof
^ permalink raw reply
* Re: [PATCH v6 2/3] powerpc/powernv: Introduce support and parsing for self-save API
From: Gautham R Shenoy @ 2020-04-14 7:47 UTC (permalink / raw)
To: Pratik Rajesh Sampat
Cc: ego, pratik.r.sampat, linuxram, linux-kernel, linuxppc-dev,
oohall, skiboot
In-Reply-To: <20200326071034.12838-3-psampat@linux.ibm.com>
On Thu, Mar 26, 2020 at 12:40:33PM +0530, Pratik Rajesh Sampat wrote:
> This commit introduces and leverages the Self save API. The difference
> between self-save and self-restore is that the value to be saved for the
> SPR does not need to be passed to the call.
>
> Add the new Self Save OPAL API call in the list of OPAL calls.
> Implement the self saving of the SPRs based on the support populated.
> This commit imposes the self-save over self-restore in case both are
> supported for a particular SPR.
s/imposes/prefers.
>
> Along with support for self-save, kernel supported save restore is also
> populated in the list. This property is only populated for those SPRs
> which encapsulate support from the kernel and hav ethe possibility to
^^^^^^^^^
have the
> garner support from a firmware mode too.
What you mean here is that some of the SPRs which are currently being
saved and restored by the kernel can be leverage the firmware
self-save support when available.
In particular, this is *required* for PTCR in an Ultravisor mode, as
the Hypervisor Kernel is not allowed to restore PTCR while running in
an Ultravisor environment.
>
> In addition, the commit also parses the device tree for nodes self-save,
> self-restore and populate support for the preferred SPRs based on what
> was advertised by the device tree.
>
> In the case a SPR is supported by the firmware self-save, self-restore
> and kernel save restore then the preference of execution is also in the
> same order as above.
>
> Signed-off-by: Pratik Rajesh Sampat <psampat@linux.ibm.com>
Some more comments below.
> ---
> .../bindings/powerpc/opal/power-mgt.txt | 18 +++
> arch/powerpc/include/asm/opal-api.h | 3 +-
> arch/powerpc/include/asm/opal.h | 1 +
> arch/powerpc/platforms/powernv/idle.c | 131 +++++++++++++++++-
> arch/powerpc/platforms/powernv/opal-call.c | 1 +
> 5 files changed, 146 insertions(+), 8 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt b/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
> index 9d619e955576..5fb03c6d7de9 100644
> --- a/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
> +++ b/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
> @@ -116,3 +116,21 @@ otherwise. The length of all the property arrays must be the same.
> which of the fields of the PMICR are set in the corresponding
> entries in ibm,cpu-idle-state-pmicr. This is an optional
> property on POWER8 and is absent on POWER9.
> +
> +- self-restore:
> + Array of unsigned 64-bit values containing a property for sprn-mask
> + with each bit indicating the index of the supported SPR for the
> + functionality. This is an optional property for both Power8 and Power9
> +
> +- self-save:
> + Array of unsigned 64-bit values containing a property for sprn-mask
> + with each bit indicating the index of the supported SPR for the
> + functionality. This is an optional property for both Power8 and Power9
> +
> +Example of arrangement of self-restore and self-save arrays:
> +For instance if PSSCR is supported, the value is 0x357 = 855.
> +Since the array is of 64 bit values, the index of the array is determined by
> +855 / 64 = 13th element. Within that index, the bit number is determined by
> +855 % 64 = 23rd bit.
> +This means that if the 23rd bit in array[13] is set, then that SPR is supported
> +by the corresponding self-save or self-restore API.
> diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h
> index c1f25a760eb1..1b6e1a68d431 100644
> --- a/arch/powerpc/include/asm/opal-api.h
> +++ b/arch/powerpc/include/asm/opal-api.h
> @@ -214,7 +214,8 @@
> #define OPAL_SECVAR_GET 176
> #define OPAL_SECVAR_GET_NEXT 177
> #define OPAL_SECVAR_ENQUEUE_UPDATE 178
> -#define OPAL_LAST 178
> +#define OPAL_SLW_SELF_SAVE_REG 181
> +#define OPAL_LAST 181
>
> #define QUIESCE_HOLD 1 /* Spin all calls at entry */
> #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
> diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
> index 9986ac34b8e2..a370b0e8d899 100644
> --- a/arch/powerpc/include/asm/opal.h
> +++ b/arch/powerpc/include/asm/opal.h
> @@ -204,6 +204,7 @@ int64_t opal_handle_hmi2(__be64 *out_flags);
> int64_t opal_register_dump_region(uint32_t id, uint64_t start, uint64_t end);
> int64_t opal_unregister_dump_region(uint32_t id);
> int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
> +int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
> int64_t opal_config_cpu_idle_state(uint64_t state, uint64_t flag);
> int64_t opal_pci_set_phb_cxl_mode(uint64_t phb_id, uint64_t mode, uint64_t pe_number);
> int64_t opal_pci_get_pbcq_tunnel_bar(uint64_t phb_id, uint64_t *addr);
> diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c
> index 858ceb86394d..e77b31621081 100644
> --- a/arch/powerpc/platforms/powernv/idle.c
> +++ b/arch/powerpc/platforms/powernv/idle.c
> @@ -35,13 +35,20 @@
> /*
> * Type of support for each SPR
> * FIRMWARE_RESTORE: firmware restoration supported: calls self-restore OPAL API
> + * FIRMWARE_SELF_SAVE: firmware save and restore: calls self-save OPAL API
> + * KERNEL_SAVE_RESTORE: kernel handles the saving and restoring of SPR
> */
> #define UNSUPPORTED 0x0
> #define FIRMWARE_RESTORE 0x1
> +#define FIRMWARE_SELF_SAVE 0x2
> +#define KERNEL_SAVE_RESTORE 0x4
>
> static u32 supported_cpuidle_states;
> struct pnv_idle_states_t *pnv_idle_states;
> int nr_pnv_idle_states;
> +/* Caching the lpcr & ptcr support to use later */
> +static bool is_lpcr_self_save;
> +static bool is_ptcr_self_save;
>
> struct preferred_sprs {
> u64 spr;
> @@ -51,6 +58,10 @@ struct preferred_sprs {
> /*
> * Supported mode: Default support. Can be overwritten during system
> * initialization
> + * Note: SPRs with support for KERNEL_SAVE_RESTORE in this list are only those
> + * which have a possibility of support from another firmware mode (i.e self-save
> + * or self-restore)
> + * SPRs with exclusive kernel save support are implicit.
> */
> struct preferred_sprs preferred_sprs[] = {
> {
> @@ -61,6 +72,10 @@ struct preferred_sprs preferred_sprs[] = {
> .spr = SPRN_LPCR,
> .supported_mode = FIRMWARE_RESTORE,
> },
> + {
> + .spr = SPRN_PTCR,
> + .supported_mode = KERNEL_SAVE_RESTORE,
> + },
> {
> .spr = SPRN_HMEER,
> .supported_mode = FIRMWARE_RESTORE,
> @@ -219,11 +234,33 @@ static int pnv_self_save_restore_sprs(void)
> curr_spr.spr == SPRN_HID4 ||
> curr_spr.spr == SPRN_HID5))
> continue;
> - if (curr_spr.supported_mode & FIRMWARE_RESTORE) {
> +
> + if (curr_spr.supported_mode & FIRMWARE_SELF_SAVE) {
> + rc = opal_slw_self_save_reg(pir,
> + curr_spr.spr);
> + if (rc != 0)
> + return rc;
> + switch (curr_spr.spr) {
> + case SPRN_LPCR:
> + is_lpcr_self_save = true;
Could you consider converting is_lpcr_self_save and is_ptcr_self_save
into static_keys ? For reference see commit
14c73bd344da("powerpc/vcpu: Assume dedicated processors as
non-preempt")
> + break;
> + case SPRN_PTCR:
> + is_ptcr_self_save = true;
> + break;
> + }
> + } else if (curr_spr.supported_mode & FIRMWARE_RESTORE) {
> rc = pnv_self_restore_sprs(pir, cpu,
> curr_spr.spr);
> if (rc != 0)
> return rc;
> + } else {
> + if (curr_spr.supported_mode & KERNEL_SAVE_RESTORE ||
> + (cpu_has_feature(CPU_FTR_ARCH_300) &&
> + (curr_spr.spr == SPRN_HID1 ||
> + curr_spr.spr == SPRN_HID4 ||
> + curr_spr.spr == SPRN_HID5)))
> + continue;
> + return OPAL_UNSUPPORTED;
> }
> }
> }
> @@ -762,7 +799,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)
> mmcr0 = mfspr(SPRN_MMCR0);
> }
> if ((psscr & PSSCR_RL_MASK) >= pnv_first_spr_loss_level) {
> - sprs.lpcr = mfspr(SPRN_LPCR);
> + if (!is_lpcr_self_save)
> + sprs.lpcr = mfspr(SPRN_LPCR);
> sprs.hfscr = mfspr(SPRN_HFSCR);
> sprs.fscr = mfspr(SPRN_FSCR);
> sprs.pid = mfspr(SPRN_PID);
> @@ -776,7 +814,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)
> sprs.mmcr1 = mfspr(SPRN_MMCR1);
> sprs.mmcr2 = mfspr(SPRN_MMCR2);
>
> - sprs.ptcr = mfspr(SPRN_PTCR);
> + if (!is_ptcr_self_save)
> + sprs.ptcr = mfspr(SPRN_PTCR);
> sprs.rpr = mfspr(SPRN_RPR);
> sprs.tscr = mfspr(SPRN_TSCR);
> if (!firmware_has_feature(FW_FEATURE_ULTRAVISOR))
> @@ -860,7 +899,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)
> goto core_woken;
>
> /* Per-core SPRs */
> - mtspr(SPRN_PTCR, sprs.ptcr);
> + if (!is_ptcr_self_save)
> + mtspr(SPRN_PTCR, sprs.ptcr);
> mtspr(SPRN_RPR, sprs.rpr);
> mtspr(SPRN_TSCR, sprs.tscr);
>
> @@ -881,7 +921,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)
> atomic_unlock_and_stop_thread_idle();
>
> /* Per-thread SPRs */
> - mtspr(SPRN_LPCR, sprs.lpcr);
> + if (!is_lpcr_self_save)
> + mtspr(SPRN_LPCR, sprs.lpcr);
> mtspr(SPRN_HFSCR, sprs.hfscr);
> mtspr(SPRN_FSCR, sprs.fscr);
> mtspr(SPRN_PID, sprs.pid);
> @@ -1060,8 +1101,10 @@ void pnv_program_cpu_hotplug_lpcr(unsigned int cpu, u64 lpcr_val)
> * Program the LPCR via stop-api only if the deepest stop state
> * can lose hypervisor context.
> */
> - if (supported_cpuidle_states & OPAL_PM_LOSE_FULL_CONTEXT)
> - opal_slw_set_reg(pir, SPRN_LPCR, lpcr_val);
> + if (supported_cpuidle_states & OPAL_PM_LOSE_FULL_CONTEXT) {
> + if (!is_lpcr_self_save)
> + opal_slw_set_reg(pir, SPRN_LPCR, lpcr_val);
> + }
> }
>
> /*
> @@ -1316,6 +1359,77 @@ static void __init pnv_probe_idle_states(void)
> supported_cpuidle_states |= pnv_idle_states[i].flags;
> }
>
> +/*
> + * Extracts and populates the self save or restore capabilities
> + * passed from the device tree node
Could you please document what @np and @support refer to here ?
> + */
> +static int extract_save_restore_state_dt(struct device_node *np, u32 support)
> +{
> + int nr_sprns = 0, i, bitmask_index;
> + u64 *temp_u64;
> + u64 bit_pos;
> +
> + nr_sprns = of_property_count_u64_elems(np, "sprn-bitmask");
> + if (nr_sprns <= 0)
> + return -EINVAL;
> + temp_u64 = kcalloc(nr_sprns, sizeof(u64), GFP_KERNEL);
> + if (of_property_read_u64_array(np, "sprn-bitmask",
> + temp_u64, nr_sprns)) {
> + pr_warn("cpuidle-powernv: failed to find registers in DT\n");
> + kfree(temp_u64);
> + return -EINVAL;
> + }
> + /*
> + * Populate acknowledgment of support for the sprs in the global vector
> + * gotten by the registers supplied by the firmware.
> + * The registers are in a bitmask, bit index within
> + * that specifies the SPR
> + */
> + for (i = 0; i < nr_preferred_sprs; i++) {
> + bitmask_index = BIT_ULL_WORD(preferred_sprs[i].spr);
> + bit_pos = BIT_ULL_MASK(preferred_sprs[i].spr);
> + if ((temp_u64[bitmask_index] & bit_pos) == 0) {
> + preferred_sprs[i].supported_mode &= ~support;
> + continue;
> + }
> + preferred_sprs[i].supported_mode |= support;
> + }
> +
> + kfree(temp_u64);
> + return 0;
> +}
> +
> +static int pnv_parse_deepstate_dt(void)
> +{
> + struct device_node *np;
> + int rc = 0, i;
> +
> + /*
> + * Self restore register population
> + * In the case the node is not found, the support for self-restore for
> + * already populated SPRs is *not* cut. This is because self-restore
> + * assumes legacy support. In an event, self-restore is actually not
> + * supported then the call to the firmware fails and deep stop states
> + * will be cut.
> + */
> + np = of_find_compatible_node(NULL, NULL, "ibm,opal-self-restore");
> + if (np) {
> + rc = extract_save_restore_state_dt(np, FIRMWARE_RESTORE);
> + if (rc != 0)
> + return rc;
> + }
> + /* Self save register population */
> + np = of_find_compatible_node(NULL, NULL, "ibm,opal-self-save");
> + if (!np) {
> + for (i = 0; i < nr_preferred_sprs; i++)
> + preferred_sprs[i].supported_mode &= ~FIRMWARE_SELF_SAVE;
> + } else {
> + rc = extract_save_restore_state_dt(np, FIRMWARE_SELF_SAVE);
> + }
> + of_node_put(np);
> + return rc;
> +}
> +
> /*
> * This function parses device-tree and populates all the information
> * into pnv_idle_states structure. It also sets up nr_pnv_idle_states
> @@ -1464,6 +1578,9 @@ static int __init pnv_init_idle_states(void)
> return rc;
> pnv_probe_idle_states();
>
> + rc = pnv_parse_deepstate_dt();
> + if (rc)
> + return rc;
> if (!cpu_has_feature(CPU_FTR_ARCH_300)) {
> if (!(supported_cpuidle_states & OPAL_PM_SLEEP_ENABLED_ER1)) {
> power7_fastsleep_workaround_entry = false;
> diff --git a/arch/powerpc/platforms/powernv/opal-call.c b/arch/powerpc/platforms/powernv/opal-call.c
> index 5cd0f52d258f..11e0ceb90de0 100644
> --- a/arch/powerpc/platforms/powernv/opal-call.c
> +++ b/arch/powerpc/platforms/powernv/opal-call.c
> @@ -223,6 +223,7 @@ OPAL_CALL(opal_handle_hmi, OPAL_HANDLE_HMI);
> OPAL_CALL(opal_handle_hmi2, OPAL_HANDLE_HMI2);
> OPAL_CALL(opal_config_cpu_idle_state, OPAL_CONFIG_CPU_IDLE_STATE);
> OPAL_CALL(opal_slw_set_reg, OPAL_SLW_SET_REG);
> +OPAL_CALL(opal_slw_self_save_reg, OPAL_SLW_SELF_SAVE_REG);
> OPAL_CALL(opal_register_dump_region, OPAL_REGISTER_DUMP_REGION);
> OPAL_CALL(opal_unregister_dump_region, OPAL_UNREGISTER_DUMP_REGION);
> OPAL_CALL(opal_pci_set_phb_cxl_mode, OPAL_PCI_SET_PHB_CAPI_MODE);
> --
> 2.17.1
>
^ permalink raw reply
* Re: [PATCH v6 1/3] powerpc/powernv: Introduce interface for self-restore support
From: Gautham R Shenoy @ 2020-04-14 7:16 UTC (permalink / raw)
To: Pratik Rajesh Sampat
Cc: ego, pratik.r.sampat, linuxram, linux-kernel, linuxppc-dev,
oohall, skiboot
In-Reply-To: <20200326071034.12838-2-psampat@linux.ibm.com>
On Thu, Mar 26, 2020 at 12:40:32PM +0530, Pratik Rajesh Sampat wrote:
> Introduces an interface that helps determine support for the
> self-restore API. The commit is isomorphic to the original interface of
> declaring SPRs to self-restore.
>
> Signed-off-by: Pratik Rajesh Sampat <psampat@linux.ibm.com>
This patch looks good to me.
Reviewed-by: Gautham R. Shenoy <ego@linux.vnet.ibm.com>
> ---
> arch/powerpc/platforms/powernv/idle.c | 200 +++++++++++++++++++-------
> 1 file changed, 152 insertions(+), 48 deletions(-)
>
> diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c
> index 78599bca66c2..858ceb86394d 100644
> --- a/arch/powerpc/platforms/powernv/idle.c
> +++ b/arch/powerpc/platforms/powernv/idle.c
> @@ -32,10 +32,67 @@
> #define P9_STOP_SPR_MSR 2000
> #define P9_STOP_SPR_PSSCR 855
>
> +/*
> + * Type of support for each SPR
> + * FIRMWARE_RESTORE: firmware restoration supported: calls self-restore OPAL API
> + */
> +#define UNSUPPORTED 0x0
> +#define FIRMWARE_RESTORE 0x1
> +
> static u32 supported_cpuidle_states;
> struct pnv_idle_states_t *pnv_idle_states;
> int nr_pnv_idle_states;
>
> +struct preferred_sprs {
> + u64 spr;
> + u32 supported_mode;
> +};
> +
> +/*
> + * Supported mode: Default support. Can be overwritten during system
> + * initialization
> + */
> +struct preferred_sprs preferred_sprs[] = {
> + {
> + .spr = SPRN_HSPRG0,
> + .supported_mode = FIRMWARE_RESTORE,
> + },
> + {
> + .spr = SPRN_LPCR,
> + .supported_mode = FIRMWARE_RESTORE,
> + },
> + {
> + .spr = SPRN_HMEER,
> + .supported_mode = FIRMWARE_RESTORE,
> + },
> + {
> + .spr = SPRN_HID0,
> + .supported_mode = FIRMWARE_RESTORE,
> + },
> + {
> + .spr = P9_STOP_SPR_MSR,
> + .supported_mode = FIRMWARE_RESTORE,
> + },
> + {
> + .spr = P9_STOP_SPR_PSSCR,
> + .supported_mode = FIRMWARE_RESTORE,
> + },
> + {
> + .spr = SPRN_HID1,
> + .supported_mode = FIRMWARE_RESTORE,
> + },
> + {
> + .spr = SPRN_HID4,
> + .supported_mode = FIRMWARE_RESTORE,
> + },
> + {
> + .spr = SPRN_HID5,
> + .supported_mode = FIRMWARE_RESTORE,
> + }
> +};
> +
> +const int nr_preferred_sprs = ARRAY_SIZE(preferred_sprs);
> +
> /*
> * The default stop state that will be used by ppc_md.power_save
> * function on platforms that support stop instruction.
> @@ -61,78 +118,125 @@ static bool deepest_stop_found;
>
> static unsigned long power7_offline_type;
>
> -static int pnv_save_sprs_for_deep_states(void)
> +static int pnv_self_restore_sprs(u64 pir, int cpu, u64 spr)
> {
> - int cpu;
> + u64 reg_val;
> int rc;
>
> - /*
> - * hid0, hid1, hid4, hid5, hmeer and lpcr values are symmetric across
> - * all cpus at boot. Get these reg values of current cpu and use the
> - * same across all cpus.
> - */
> - uint64_t lpcr_val = mfspr(SPRN_LPCR);
> - uint64_t hid0_val = mfspr(SPRN_HID0);
> - uint64_t hid1_val = mfspr(SPRN_HID1);
> - uint64_t hid4_val = mfspr(SPRN_HID4);
> - uint64_t hid5_val = mfspr(SPRN_HID5);
> - uint64_t hmeer_val = mfspr(SPRN_HMEER);
> - uint64_t msr_val = MSR_IDLE;
> - uint64_t psscr_val = pnv_deepest_stop_psscr_val;
> -
> - for_each_present_cpu(cpu) {
> - uint64_t pir = get_hard_smp_processor_id(cpu);
> - uint64_t hsprg0_val = (uint64_t)paca_ptrs[cpu];
> -
> - rc = opal_slw_set_reg(pir, SPRN_HSPRG0, hsprg0_val);
> + switch (spr) {
> + case SPRN_HSPRG0:
> + reg_val = (uint64_t)paca_ptrs[cpu];
> + rc = opal_slw_set_reg(pir, SPRN_HSPRG0, reg_val);
> if (rc != 0)
> return rc;
> -
> - rc = opal_slw_set_reg(pir, SPRN_LPCR, lpcr_val);
> + break;
> + case SPRN_LPCR:
> + reg_val = mfspr(SPRN_LPCR);
> + rc = opal_slw_set_reg(pir, SPRN_LPCR, reg_val);
> if (rc != 0)
> return rc;
> -
> + break;
> + case P9_STOP_SPR_MSR:
> + reg_val = MSR_IDLE;
> if (cpu_has_feature(CPU_FTR_ARCH_300)) {
> - rc = opal_slw_set_reg(pir, P9_STOP_SPR_MSR, msr_val);
> + rc = opal_slw_set_reg(pir, P9_STOP_SPR_MSR, reg_val);
> if (rc)
> return rc;
> -
> - rc = opal_slw_set_reg(pir,
> - P9_STOP_SPR_PSSCR, psscr_val);
> -
> + }
> + break;
> + case P9_STOP_SPR_PSSCR:
> + reg_val = pnv_deepest_stop_psscr_val;
> + if (cpu_has_feature(CPU_FTR_ARCH_300)) {
> + rc = opal_slw_set_reg(pir, P9_STOP_SPR_PSSCR, reg_val);
> if (rc)
> return rc;
> }
> -
> - /* HIDs are per core registers */
> + break;
> + case SPRN_HMEER:
> + reg_val = mfspr(SPRN_HMEER);
> if (cpu_thread_in_core(cpu) == 0) {
> -
> - rc = opal_slw_set_reg(pir, SPRN_HMEER, hmeer_val);
> - if (rc != 0)
> + rc = opal_slw_set_reg(pir, SPRN_HMEER, reg_val);
> + if (rc)
> return rc;
> -
> - rc = opal_slw_set_reg(pir, SPRN_HID0, hid0_val);
> - if (rc != 0)
> + }
> + break;
> + case SPRN_HID0:
> + reg_val = mfspr(SPRN_HID0);
> + if (cpu_thread_in_core(cpu) == 0) {
> + rc = opal_slw_set_reg(pir, SPRN_HID0, reg_val);
> + if (rc)
> return rc;
> + }
> + break;
> + case SPRN_HID1:
> + reg_val = mfspr(SPRN_HID1);
> + if (!cpu_has_feature(CPU_FTR_ARCH_300) &&
> + cpu_thread_in_core(cpu) == 0) {
> + rc = opal_slw_set_reg(pir, SPRN_HID1, reg_val);
> + if (rc)
> + return rc;
> + }
> + break;
> + case SPRN_HID4:
> + reg_val = mfspr(SPRN_HID4);
> + if (!cpu_has_feature(CPU_FTR_ARCH_300) &&
> + cpu_thread_in_core(cpu) == 0) {
> + rc = opal_slw_set_reg(pir, SPRN_HID4, reg_val);
> + if (rc)
> + return rc;
> + }
> + break;
> + case SPRN_HID5:
> + reg_val = mfspr(SPRN_HID5);
> + if (!cpu_has_feature(CPU_FTR_ARCH_300) &&
> + cpu_thread_in_core(cpu) == 0) {
> + rc = opal_slw_set_reg(pir, SPRN_HID5, reg_val);
> + if (rc)
> + return rc;
> + }
> + break;
> + default:
> + return -EINVAL;
> + }
> + return 0;
> +}
>
> - /* Only p8 needs to set extra HID regiters */
> - if (!cpu_has_feature(CPU_FTR_ARCH_300)) {
> -
> - rc = opal_slw_set_reg(pir, SPRN_HID1, hid1_val);
> - if (rc != 0)
> - return rc;
> -
> - rc = opal_slw_set_reg(pir, SPRN_HID4, hid4_val);
> - if (rc != 0)
> - return rc;
> +static int pnv_self_save_restore_sprs(void)
> +{
> + int rc, index, cpu;
> + u64 pir;
> + struct preferred_sprs curr_spr;
>
> - rc = opal_slw_set_reg(pir, SPRN_HID5, hid5_val);
> + for_each_present_cpu(cpu) {
> + pir = get_hard_smp_processor_id(cpu);
> + for (index = 0; index < nr_preferred_sprs; index++) {
> + curr_spr = preferred_sprs[index];
> + /* HIDs are per core register */
> + if (cpu_thread_in_core(cpu) != 0 &&
> + (curr_spr.spr == SPRN_HMEER ||
> + curr_spr.spr == SPRN_HID0 ||
> + curr_spr.spr == SPRN_HID1 ||
> + curr_spr.spr == SPRN_HID4 ||
> + curr_spr.spr == SPRN_HID5))
> + continue;
> + if (curr_spr.supported_mode & FIRMWARE_RESTORE) {
> + rc = pnv_self_restore_sprs(pir, cpu,
> + curr_spr.spr);
> if (rc != 0)
> return rc;
> }
> }
> }
> + return 0;
> +}
>
> +static int pnv_save_sprs_for_deep_states(void)
> +{
> + int rc;
> +
> + rc = pnv_self_save_restore_sprs();
> + if (rc != 0)
> + return rc;
> return 0;
> }
>
> --
> 2.17.1
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox