From: David Gibson <david@gibson.dropbear.id.au>
To: Alexey Kardashevskiy <aik@ozlabs.ru>
Cc: "Alex Williamson" <alex.williamson@redhat.com>,
"Jose Ricardo Ziviani" <joserz@linux.ibm.com>,
"Sam Bobroff" <sbobroff@linux.ibm.com>,
"Alistair Popple" <alistair@popple.id.au>,
linuxppc-dev@lists.ozlabs.org, kvm-ppc@vger.kernel.org,
"Piotr Jaroszynski" <pjaroszynski@nvidia.com>,
"Oliver O'Halloran" <oohall@gmail.com>,
"Andrew Donnellan" <andrew.donnellan@au1.ibm.com>,
"Leonardo Augusto Guimarães Garcia" <lagarcia@br.ibm.com>,
"Reza Arbab" <arbab@linux.ibm.com>
Subject: Re: [PATCH kernel v3 16/22] powerpc/powernv: Add purge cache OPAL call
Date: Mon, 19 Nov 2018 00:21:13 +0000 [thread overview]
Message-ID: <20181119002113.GC4878@umbus> (raw)
In-Reply-To: <20181113082823.2440-17-aik@ozlabs.ru>
[-- Attachment #1: Type: text/plain, Size: 3298 bytes --]
On Tue, Nov 13, 2018 at 07:28:17PM +1100, Alexey Kardashevskiy wrote:
> Flushing caches using the dcbf instruction takes quite some time if we
> need to flush gigabytes (16GB takes more than 15s); OPAL just added
> a big hammer to flush all caches.
>
> This adds opal_purge_cache() which will be used later to flush caches
> for coherent GPU memory which might suddenly become unavailable if a GPU
> is reset and NVLink is not (re)trained.
>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> ---
> arch/powerpc/include/asm/opal-api.h | 3 ++-
> arch/powerpc/include/asm/opal.h | 1 +
> arch/powerpc/platforms/powernv/opal.c | 1 +
> arch/powerpc/platforms/powernv/opal-wrappers.S | 1 +
> 4 files changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h
> index 870fb7b..55bc640 100644
> --- a/arch/powerpc/include/asm/opal-api.h
> +++ b/arch/powerpc/include/asm/opal-api.h
> @@ -210,7 +210,8 @@
> #define OPAL_PCI_GET_PBCQ_TUNNEL_BAR 164
> #define OPAL_PCI_SET_PBCQ_TUNNEL_BAR 165
> #define OPAL_NX_COPROC_INIT 167
> -#define OPAL_LAST 167
> +#define OPAL_CLEAR_CACHE 170
> +#define OPAL_LAST 170
>
> #define QUIESCE_HOLD 1 /* Spin all calls at entry */
> #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
> diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
> index ff38664..7db576e 100644
> --- a/arch/powerpc/include/asm/opal.h
> +++ b/arch/powerpc/include/asm/opal.h
> @@ -294,6 +294,7 @@ int opal_set_power_shift_ratio(u32 handle, int token, u32 psr);
> int opal_sensor_group_clear(u32 group_hndl, int token);
> int opal_sensor_group_enable(u32 group_hndl, int token, bool enable);
> int opal_nx_coproc_init(uint32_t chip_id, uint32_t ct);
> +int opal_purge_cache(void);
>
> s64 opal_signal_system_reset(s32 cpu);
> s64 opal_quiesce(u64 shutdown_type, s32 cpu);
> diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
> index beed86f..44ce824 100644
> --- a/arch/powerpc/platforms/powernv/opal.c
> +++ b/arch/powerpc/platforms/powernv/opal.c
> @@ -1113,3 +1113,4 @@ EXPORT_SYMBOL_GPL(opal_int_eoi);
> EXPORT_SYMBOL_GPL(opal_error_code);
> /* Export the below symbol for NX compression */
> EXPORT_SYMBOL(opal_nx_coproc_init);
> +EXPORT_SYMBOL(opal_purge_cache);
> diff --git a/arch/powerpc/platforms/powernv/opal-wrappers.S b/arch/powerpc/platforms/powernv/opal-wrappers.S
> index 2515282..5b886a6 100644
> --- a/arch/powerpc/platforms/powernv/opal-wrappers.S
> +++ b/arch/powerpc/platforms/powernv/opal-wrappers.S
> @@ -331,3 +331,4 @@ OPAL_CALL(opal_pci_set_pbcq_tunnel_bar, OPAL_PCI_SET_PBCQ_TUNNEL_BAR);
> OPAL_CALL(opal_sensor_read_u64, OPAL_SENSOR_READ_U64);
> OPAL_CALL(opal_sensor_group_enable, OPAL_SENSOR_GROUP_ENABLE);
> OPAL_CALL(opal_nx_coproc_init, OPAL_NX_COPROC_INIT);
> +OPAL_CALL(opal_purge_cache, OPAL_CLEAR_CACHE);
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
WARNING: multiple messages have this Message-ID (diff)
From: David Gibson <david@gibson.dropbear.id.au>
To: Alexey Kardashevskiy <aik@ozlabs.ru>
Cc: "Alex Williamson" <alex.williamson@redhat.com>,
"Jose Ricardo Ziviani" <joserz@linux.ibm.com>,
"Sam Bobroff" <sbobroff@linux.ibm.com>,
"Alistair Popple" <alistair@popple.id.au>,
linuxppc-dev@lists.ozlabs.org, kvm-ppc@vger.kernel.org,
"Piotr Jaroszynski" <pjaroszynski@nvidia.com>,
"Oliver O'Halloran" <oohall@gmail.com>,
"Andrew Donnellan" <andrew.donnellan@au1.ibm.com>,
"Leonardo Augusto Guimarães Garcia" <lagarcia@br.ibm.com>,
"Reza Arbab" <arbab@linux.ibm.com>
Subject: Re: [PATCH kernel v3 16/22] powerpc/powernv: Add purge cache OPAL call
Date: Mon, 19 Nov 2018 11:21:13 +1100 [thread overview]
Message-ID: <20181119002113.GC4878@umbus> (raw)
In-Reply-To: <20181113082823.2440-17-aik@ozlabs.ru>
[-- Attachment #1: Type: text/plain, Size: 3298 bytes --]
On Tue, Nov 13, 2018 at 07:28:17PM +1100, Alexey Kardashevskiy wrote:
> Flushing caches using the dcbf instruction takes quite some time if we
> need to flush gigabytes (16GB takes more than 15s); OPAL just added
> a big hammer to flush all caches.
>
> This adds opal_purge_cache() which will be used later to flush caches
> for coherent GPU memory which might suddenly become unavailable if a GPU
> is reset and NVLink is not (re)trained.
>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> ---
> arch/powerpc/include/asm/opal-api.h | 3 ++-
> arch/powerpc/include/asm/opal.h | 1 +
> arch/powerpc/platforms/powernv/opal.c | 1 +
> arch/powerpc/platforms/powernv/opal-wrappers.S | 1 +
> 4 files changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h
> index 870fb7b..55bc640 100644
> --- a/arch/powerpc/include/asm/opal-api.h
> +++ b/arch/powerpc/include/asm/opal-api.h
> @@ -210,7 +210,8 @@
> #define OPAL_PCI_GET_PBCQ_TUNNEL_BAR 164
> #define OPAL_PCI_SET_PBCQ_TUNNEL_BAR 165
> #define OPAL_NX_COPROC_INIT 167
> -#define OPAL_LAST 167
> +#define OPAL_CLEAR_CACHE 170
> +#define OPAL_LAST 170
>
> #define QUIESCE_HOLD 1 /* Spin all calls at entry */
> #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
> diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
> index ff38664..7db576e 100644
> --- a/arch/powerpc/include/asm/opal.h
> +++ b/arch/powerpc/include/asm/opal.h
> @@ -294,6 +294,7 @@ int opal_set_power_shift_ratio(u32 handle, int token, u32 psr);
> int opal_sensor_group_clear(u32 group_hndl, int token);
> int opal_sensor_group_enable(u32 group_hndl, int token, bool enable);
> int opal_nx_coproc_init(uint32_t chip_id, uint32_t ct);
> +int opal_purge_cache(void);
>
> s64 opal_signal_system_reset(s32 cpu);
> s64 opal_quiesce(u64 shutdown_type, s32 cpu);
> diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
> index beed86f..44ce824 100644
> --- a/arch/powerpc/platforms/powernv/opal.c
> +++ b/arch/powerpc/platforms/powernv/opal.c
> @@ -1113,3 +1113,4 @@ EXPORT_SYMBOL_GPL(opal_int_eoi);
> EXPORT_SYMBOL_GPL(opal_error_code);
> /* Export the below symbol for NX compression */
> EXPORT_SYMBOL(opal_nx_coproc_init);
> +EXPORT_SYMBOL(opal_purge_cache);
> diff --git a/arch/powerpc/platforms/powernv/opal-wrappers.S b/arch/powerpc/platforms/powernv/opal-wrappers.S
> index 2515282..5b886a6 100644
> --- a/arch/powerpc/platforms/powernv/opal-wrappers.S
> +++ b/arch/powerpc/platforms/powernv/opal-wrappers.S
> @@ -331,3 +331,4 @@ OPAL_CALL(opal_pci_set_pbcq_tunnel_bar, OPAL_PCI_SET_PBCQ_TUNNEL_BAR);
> OPAL_CALL(opal_sensor_read_u64, OPAL_SENSOR_READ_U64);
> OPAL_CALL(opal_sensor_group_enable, OPAL_SENSOR_GROUP_ENABLE);
> OPAL_CALL(opal_nx_coproc_init, OPAL_NX_COPROC_INIT);
> +OPAL_CALL(opal_purge_cache, OPAL_CLEAR_CACHE);
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2018-11-19 0:21 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-13 8:28 [PATCH kernel v3 00/22] powerpc/powernv/npu, vfio: NVIDIA V100 + P9 passthrough Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 01/22] powerpc/ioda/npu: Call skiboot's hot reset hook when disabling NPU2 Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 02/22] powerpc/mm/iommu/vfio_spapr_tce: Change mm_iommu_get to reference a region Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-15 5:32 ` [PATCH kernel v3 02/22] powerpc/mm/iommu/vfio_spapr_tce: Change mm_iommu_get to reference a regi David Gibson
2018-11-15 5:32 ` [PATCH kernel v3 02/22] powerpc/mm/iommu/vfio_spapr_tce: Change mm_iommu_get to reference a region David Gibson
2018-11-13 8:28 ` [PATCH kernel v3 03/22] powerpc/mm/iommu: Make mm_iommu_new() fail on existing regions Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-15 5:38 ` David Gibson
2018-11-15 5:38 ` David Gibson
2018-11-13 8:28 ` [PATCH kernel v3 04/22] powerpc/vfio/iommu/kvm: Do not pin device memory Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-16 3:11 ` David Gibson
2018-11-16 3:11 ` David Gibson
2018-11-13 8:28 ` [PATCH kernel v3 05/22] powerpc/powernv/npu: Add helper to access struct npu for NPU device Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-14 3:42 ` Alistair Popple
2018-11-14 3:42 ` Alistair Popple
2018-11-13 8:28 ` [PATCH kernel v3 06/22] powerpc/powernv: Detach npu struct from pnv_phb Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-14 4:28 ` Alistair Popple
2018-11-14 4:28 ` Alistair Popple
2018-11-19 7:18 ` Alexey Kardashevskiy
2018-11-19 7:18 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 07/22] powerpc/powernv/npu: Move OPAL calls away from context manipulation Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-14 4:57 ` Alistair Popple
2018-11-14 4:57 ` Alistair Popple
2018-11-13 8:28 ` [PATCH kernel v3 08/22] powerpc/pseries/iommu: Allow dynamic window to start from zero Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 09/22] powerpc/pseries/iommu: Force default DMA window removal Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-16 4:54 ` David Gibson
2018-11-16 4:54 ` David Gibson
2018-11-19 7:28 ` Alexey Kardashevskiy
2018-11-19 7:28 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 10/22] powerpc/pseries/iommu: Use memory@ nodes in max RAM address calculation Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-16 5:23 ` David Gibson
2018-11-16 5:23 ` David Gibson
2018-11-19 7:43 ` Alexey Kardashevskiy
2018-11-19 7:43 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 11/22] powerpc/pseries/npu: Enable platform support Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-16 5:25 ` David Gibson
2018-11-16 5:25 ` David Gibson
2018-11-19 7:50 ` Alexey Kardashevskiy
2018-11-19 7:50 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 12/22] powerpc/pseries: Remove IOMMU API support for non-LPAR systems Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 13/22] powerpc/powernv/pseries: Rework device adding to IOMMU groups Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 14/22] powerpc/iommu_api: Move IOMMU groups setup to a single place Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-19 0:15 ` David Gibson
2018-11-19 0:15 ` David Gibson
2018-11-13 8:28 ` [PATCH kernel v3 15/22] powerpc/powernv: Reference iommu_table while it is linked to a group Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-19 0:20 ` David Gibson
2018-11-19 0:20 ` David Gibson
2018-11-13 8:28 ` [PATCH kernel v3 16/22] powerpc/powernv: Add purge cache OPAL call Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-19 0:21 ` David Gibson [this message]
2018-11-19 0:21 ` David Gibson
2018-11-13 8:28 ` [PATCH kernel v3 17/22] powerpc/powernv/npu: Convert NPU IOMMU helpers to iommu_table_group_ops Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-19 0:24 ` David Gibson
2018-11-19 0:24 ` David Gibson
2018-11-13 8:28 ` [PATCH kernel v3 18/22] powerpc/powernv/npu: Add compound IOMMU groups Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-19 1:12 ` David Gibson
2018-11-19 1:12 ` David Gibson
2018-11-19 2:29 ` Alexey Kardashevskiy
2018-11-19 2:29 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 19/22] powerpc/powernv/npu: Add release_ownership hook Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 20/22] vfio_pci: Allow mapping extra regions Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 21/22] vfio_pci: Allow regions to add own capabilities Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
2018-11-13 8:28 ` [PATCH kernel v3 22/22] vfio_pci: Add NVIDIA GV100GL [Tesla V100 SXM2] [10de:1db1] subdriver Alexey Kardashevskiy
2018-11-13 8:28 ` Alexey Kardashevskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181119002113.GC4878@umbus \
--to=david@gibson.dropbear.id.au \
--cc=aik@ozlabs.ru \
--cc=alex.williamson@redhat.com \
--cc=alistair@popple.id.au \
--cc=andrew.donnellan@au1.ibm.com \
--cc=arbab@linux.ibm.com \
--cc=joserz@linux.ibm.com \
--cc=kvm-ppc@vger.kernel.org \
--cc=lagarcia@br.ibm.com \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=oohall@gmail.com \
--cc=pjaroszynski@nvidia.com \
--cc=sbobroff@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.