* [PATCH 2/2] arm64: Branch predictor hardening for Cavium ThunderX2
From: Jayachandran C @ 2018-01-08 17:19 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180108164651.GQ25869@arm.com>
On Mon, Jan 08, 2018 at 04:46:52PM +0000, Will Deacon wrote:
> On Sun, Jan 07, 2018 at 10:53:36PM -0800, Jayachandran C wrote:
> > Use PSCI based mitigation for speculative execution attacks targeting
> > the branch predictor. The approach is similar to the one used for
> > Cortex-A CPUs, but in case of ThunderX2 we add another SMC call to
> > test if the firmware supports the capability.
> >
> > If the secure firmware has been updated with the mitigation code to
> > invalidate the branch target buffer, we use the PSCI version call to
> > invoke it.
> >
> > Signed-off-by: Jayachandran C <jnair@caviumnetworks.com>
> > ---
> > arch/arm64/kernel/cpu_errata.c | 38 ++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 38 insertions(+)
> >
> > diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
> > index cb0fb37..abceb5d 100644
> > --- a/arch/arm64/kernel/cpu_errata.c
> > +++ b/arch/arm64/kernel/cpu_errata.c
> > @@ -124,6 +124,7 @@ static void install_bp_hardening_cb(const struct arm64_cpu_capabilities *entry,
> > __install_bp_hardening_cb(fn, hyp_vecs_start, hyp_vecs_end);
> > }
> >
> > +#include <linux/arm-smccc.h>
> > #include <linux/psci.h>
> >
> > static int enable_psci_bp_hardening(void *data)
> > @@ -138,6 +139,33 @@ static int enable_psci_bp_hardening(void *data)
> >
> > return 0;
> > }
> > +
> > +#define CAVIUM_TX2_SIP_SMC_CALL 0xC200FF00
> > +#define CAVIUM_TX2_BTB_HARDEN_CAP 0xB0A0
> > +
> > +static int enable_tx2_psci_bp_hardening(void *data)
> > +{
> > + const struct arm64_cpu_capabilities *entry = data;
> > + struct arm_smccc_res res;
> > +
> > + if (!entry->matches(entry, SCOPE_LOCAL_CPU))
> > + return;
> > +
> > + arm_smccc_smc(CAVIUM_TX2_SIP_SMC_CALL, CAVIUM_TX2_BTB_HARDEN_CAP, 0, 0, 0, 0, 0, 0, &res);
>
> One thing to be aware of here is that if somebody configures qemu to emulate
> a TX2, this may actually disappear into EL3 and not return. You're better
> off sticking with PSCI GET_VERSION in terms of portability, but it's your
> call -- I'd expect you to deal with any breakage reports on the list due
> to the SMC above. Fair?
I don't like having a custom SMC here either. But Overloading PSCI get version
is the problem as I wrote earlier - there is no way to check if the firmware
implements BTB hardening with overloading. There is a good chance that users
with old firmware will just fail without any warning.
Is there a reason for overloading PSCI get version? Allocating a new standard
SMC number would make checking for existance and usage much simpler.
I did not quite understand the possible issue in qemu, unimplemented SMC calls
are expected to return an error code. What am I missing here?
>
> > + if (res.a0 != 0) {
> > + pr_warn("Error: CONFIG_HARDEN_BRANCH_PREDICTOR enabled, but firmware does not support it\n");
> > + return 0;
> > + }
>
> Please don't print this here; see below.
>
> > + if (res.a1 == 1 && psci_ops.get_version) {
> > + pr_info("CPU%d: Branch predictor hardening enabled\n", smp_processor_id());
>
> If you want to print a message, please put it in the capability structure
> .desc field.
Thanks, will fix this in the next rev.
JC.
^ permalink raw reply
* [PATCH v6 11/16] firmware: arm_sdei: Add support for CPU and system power states
From: Lorenzo Pieralisi @ 2018-01-08 17:22 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180108153818.22743-12-james.morse@arm.com>
On Mon, Jan 08, 2018 at 03:38:13PM +0000, James Morse wrote:
[...]
> +/* When entering idle, mask/unmask events for this cpu */
Just mentioning (since I know you know), this notifier is called also
through syscore_ops so it is not just idle (CPUidle or Suspend-to-idle).
> +static int sdei_pm_notifier(struct notifier_block *nb, unsigned long action,
> + void *data)
> +{
> + int rv;
> +
> + switch (action) {
> + case CPU_PM_ENTER:
> + rv = sdei_mask_local_cpu();
> + break;
> + case CPU_PM_EXIT:
You should handle CPU_PM_ENTER_FAILED here for correctness,
in case the notifier chain fails.
Other than that:
Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> + rv = sdei_unmask_local_cpu();
> + break;
> + default:
> + return NOTIFY_DONE;
> + }
> +
> + if (rv)
> + return notifier_from_errno(rv);
> +
> + return NOTIFY_OK;
> +}
> +
> +static struct notifier_block sdei_pm_nb = {
> + .notifier_call = sdei_pm_notifier,
> +};
> +
> +static int sdei_device_suspend(struct device *dev)
> +{
> + on_each_cpu(_ipi_mask_cpu, NULL, true);
> +
> + return 0;
> +}
> +
> +static int sdei_device_resume(struct device *dev)
> +{
> + on_each_cpu(_ipi_unmask_cpu, NULL, true);
> +
> + return 0;
> +}
> +
> +/*
> + * We need all events to be reregistered when we resume from hibernate.
> + *
> + * The sequence is freeze->thaw. Reboot. freeze->restore. We unregister
> + * events during freeze, then re-register and re-enable them during thaw
> + * and restore.
> + */
> +static int sdei_device_freeze(struct device *dev)
> +{
> + int err;
> +
> + cpuhp_remove_state(CPUHP_AP_ARM_SDEI_STARTING);
> +
> + err = sdei_unregister_shared();
> + if (err)
> + return err;
> +
> + return 0;
> +}
> +
> +static int sdei_device_thaw(struct device *dev)
> +{
> + int err;
> +
> + /* re-register shared events */
> + err = sdei_reregister_shared();
> + if (err) {
> + pr_warn("Failed to re-register shared events...\n");
> + sdei_mark_interface_broken();
> + return err;
> + }
> +
> + err = cpuhp_setup_state(CPUHP_AP_ARM_SDEI_STARTING, "SDEI",
> + &sdei_cpuhp_up, &sdei_cpuhp_down);
> + if (err)
> + pr_warn("Failed to re-register CPU hotplug notifier...\n");
> +
> + return err;
> +}
> +
> +static int sdei_device_restore(struct device *dev)
> +{
> + int err;
> +
> + err = sdei_platform_reset();
> + if (err)
> + return err;
> +
> + return sdei_device_thaw(dev);
> +}
> +
> +static const struct dev_pm_ops sdei_pm_ops = {
> + .suspend = sdei_device_suspend,
> + .resume = sdei_device_resume,
> + .freeze = sdei_device_freeze,
> + .thaw = sdei_device_thaw,
> + .restore = sdei_device_restore,
> +};
> +
> +/*
> + * Mask all CPUs and unregister all events on panic, reboot or kexec.
> + */
> +static int sdei_reboot_notifier(struct notifier_block *nb, unsigned long action,
> + void *data)
> +{
> + /*
> + * We are going to reset the interface, after this there is no point
> + * doing work when we take CPUs offline.
> + */
> + cpuhp_remove_state(CPUHP_AP_ARM_SDEI_STARTING);
> +
> + sdei_platform_reset();
> +
> + return NOTIFY_OK;
> +}
> +
> +static struct notifier_block sdei_reboot_nb = {
> + .notifier_call = sdei_reboot_notifier,
> +};
> +
> static void sdei_smccc_smc(unsigned long function_id,
> unsigned long arg0, unsigned long arg1,
> unsigned long arg2, unsigned long arg3,
> @@ -547,9 +772,36 @@ static int sdei_probe(struct platform_device *pdev)
> return 0;
> }
>
> - on_each_cpu(&_ipi_unmask_cpu, NULL, false);
> + err = cpu_pm_register_notifier(&sdei_pm_nb);
> + if (err) {
> + pr_warn("Failed to register CPU PM notifier...\n");
> + goto error;
> + }
> +
> + err = register_reboot_notifier(&sdei_reboot_nb);
> + if (err) {
> + pr_warn("Failed to register reboot notifier...\n");
> + goto remove_cpupm;
> + }
> +
> + err = cpuhp_setup_state(CPUHP_AP_ARM_SDEI_STARTING, "SDEI",
> + &sdei_cpuhp_up, &sdei_cpuhp_down);
> + if (err) {
> + pr_warn("Failed to register CPU hotplug notifier...\n");
> + goto remove_reboot;
> + }
>
> return 0;
> +
> +remove_reboot:
> + unregister_reboot_notifier(&sdei_reboot_nb);
> +
> +remove_cpupm:
> + cpu_pm_unregister_notifier(&sdei_pm_nb);
> +
> +error:
> + sdei_mark_interface_broken();
> + return err;
> }
>
> static const struct of_device_id sdei_of_match[] = {
> @@ -560,6 +812,7 @@ static const struct of_device_id sdei_of_match[] = {
> static struct platform_driver sdei_driver = {
> .driver = {
> .name = "sdei",
> + .pm = &sdei_pm_ops,
> .of_match_table = sdei_of_match,
> },
> .probe = sdei_probe,
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index 201ab7267986..87b505a48a94 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -109,6 +109,7 @@ enum cpuhp_state {
> CPUHP_AP_PERF_XTENSA_STARTING,
> CPUHP_AP_PERF_METAG_STARTING,
> CPUHP_AP_MIPS_OP_LOONGSON3_STARTING,
> + CPUHP_AP_ARM_SDEI_STARTING,
> CPUHP_AP_ARM_VFP_STARTING,
> CPUHP_AP_ARM64_DEBUG_MONITORS_STARTING,
> CPUHP_AP_PERF_ARM_HW_BREAKPOINT_STARTING,
> --
> 2.15.0
>
^ permalink raw reply
* [PATCH] arm64: Implement branch predictor hardening for Falkor
From: Shanker Donthineni @ 2018-01-08 17:22 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180108092849.alfqtsmaf2dqakkr@kamzik.brq.redhat.com>
Hi Andrew,
On 01/08/2018 03:28 AM, Andrew Jones wrote:
> Hi Shanker,
>
> On Fri, Jan 05, 2018 at 02:28:59PM -0600, Shanker Donthineni wrote:
> ...
>> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
>> index cb0fb37..daf53a5 100644
>> --- a/arch/arm64/kernel/cpu_errata.c
>> +++ b/arch/arm64/kernel/cpu_errata.c
>> @@ -54,6 +54,8 @@ static int cpu_enable_trap_ctr_access(void *__unused)
>>
>> #ifdef CONFIG_KVM
>> extern char __psci_hyp_bp_inval_start[], __psci_hyp_bp_inval_end[];
>> +extern char __qcom_hyp_sanitize_link_stack_start[];
>> +extern char __qcom_hyp_sanitize_link_stack_end[];
>>
>> static void __copy_hyp_vect_bpi(int slot, const char *hyp_vecs_start,
>> const char *hyp_vecs_end)
>> @@ -96,8 +98,10 @@ static void __install_bp_hardening_cb(bp_hardening_cb_t fn,
>> spin_unlock(&bp_lock);
>> }
>> #else
>> -#define __psci_hyp_bp_inval_start NULL
>> -#define __psci_hyp_bp_inval_end NULL
>> +#define __psci_hyp_bp_inval_start NULL
>> +#define __psci_hyp_bp_inval_end NULL
>> +#define __qcom_hyp_sanitize_link_stack_start NULL
>> +#define __qcom_hyp_sanitize_link_stack_start NULL
> ^^ copy+paste error here
Thanks for catching typo, I'll fix in v2 patch.
>
> Thanks,
> drew
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
--
Shanker Donthineni
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
^ permalink raw reply
* [PATCH 2/2] arm64: Branch predictor hardening for Cavium ThunderX2
From: Will Deacon @ 2018-01-08 17:23 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180108171943.GA180149@jc-sabre>
On Mon, Jan 08, 2018 at 09:19:43AM -0800, Jayachandran C wrote:
> On Mon, Jan 08, 2018 at 04:46:52PM +0000, Will Deacon wrote:
> > On Sun, Jan 07, 2018 at 10:53:36PM -0800, Jayachandran C wrote:
> > > Use PSCI based mitigation for speculative execution attacks targeting
> > > the branch predictor. The approach is similar to the one used for
> > > Cortex-A CPUs, but in case of ThunderX2 we add another SMC call to
> > > test if the firmware supports the capability.
> > >
> > > If the secure firmware has been updated with the mitigation code to
> > > invalidate the branch target buffer, we use the PSCI version call to
> > > invoke it.
> > >
> > > Signed-off-by: Jayachandran C <jnair@caviumnetworks.com>
> > > ---
> > > arch/arm64/kernel/cpu_errata.c | 38 ++++++++++++++++++++++++++++++++++++++
> > > 1 file changed, 38 insertions(+)
> > >
> > > diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
> > > index cb0fb37..abceb5d 100644
> > > --- a/arch/arm64/kernel/cpu_errata.c
> > > +++ b/arch/arm64/kernel/cpu_errata.c
> > > @@ -124,6 +124,7 @@ static void install_bp_hardening_cb(const struct arm64_cpu_capabilities *entry,
> > > __install_bp_hardening_cb(fn, hyp_vecs_start, hyp_vecs_end);
> > > }
> > >
> > > +#include <linux/arm-smccc.h>
> > > #include <linux/psci.h>
> > >
> > > static int enable_psci_bp_hardening(void *data)
> > > @@ -138,6 +139,33 @@ static int enable_psci_bp_hardening(void *data)
> > >
> > > return 0;
> > > }
> > > +
> > > +#define CAVIUM_TX2_SIP_SMC_CALL 0xC200FF00
> > > +#define CAVIUM_TX2_BTB_HARDEN_CAP 0xB0A0
> > > +
> > > +static int enable_tx2_psci_bp_hardening(void *data)
> > > +{
> > > + const struct arm64_cpu_capabilities *entry = data;
> > > + struct arm_smccc_res res;
> > > +
> > > + if (!entry->matches(entry, SCOPE_LOCAL_CPU))
> > > + return;
> > > +
> > > + arm_smccc_smc(CAVIUM_TX2_SIP_SMC_CALL, CAVIUM_TX2_BTB_HARDEN_CAP, 0, 0, 0, 0, 0, 0, &res);
> >
> > One thing to be aware of here is that if somebody configures qemu to emulate
> > a TX2, this may actually disappear into EL3 and not return. You're better
> > off sticking with PSCI GET_VERSION in terms of portability, but it's your
> > call -- I'd expect you to deal with any breakage reports on the list due
> > to the SMC above. Fair?
>
> I don't like having a custom SMC here either. But Overloading PSCI get version
> is the problem as I wrote earlier - there is no way to check if the firmware
> implements BTB hardening with overloading. There is a good chance that users
> with old firmware will just fail without any warning.
That's true, but there is precedent for this elsewhere. For example, CPU
errata that require a firmware change are often not probable. Also, your SMC
call won't always work (see the qemu comment below). Note that I'm not
saying I won't take this code, just that you need to be aware of what
you're doing.
> Is there a reason for overloading PSCI get version? Allocating a new standard
> SMC number would make checking for existance and usage much simpler.
PSCI get version is what we have today. We're working on extending PSCI to
allocate a new standard SMC number, but we need something that can be used
with existing firmware too and standardisation doesn't happen overnight.
> I did not quite understand the possible issue in qemu, unimplemented SMC calls
> are expected to return an error code. What am I missing here?
Qemu will inject them into EL3, and there might not be anything there.
Will
^ permalink raw reply
* [PATCH 0/3] ARM branch predictor hardening
From: Tony Lindgren @ 2018-01-08 17:24 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <0d7ac442-1112-33e9-1f1f-cd600c3e326d@arm.com>
* Marc Zyngier <marc.zyngier@arm.com> [180108 17:05]:
> Hi Tony,
>
> On 08/01/18 16:54, Tony Lindgren wrote:
> > * Marc Zyngier <marc.zyngier@arm.com> [180106 04:14]:
> >> This small series implements some basic BP hardening by invalidating
> >> the BTB on CPUs that are known to be susceptible to aliasing attacks.
> >>
> >> These patches are closely modelled against what we do on arm64,
> >> although simpler as we can rely on an architected instruction to
> >> perform the invalidation.
> >>
> >> The first patch reuses the Cortex-A8 BTB invalidation in switch_mm and
> >> generalises it to be used on all affected CPUs. The second perform the
> >> same invalidation on fatal signal delivery. The last one nukes it on
> >> guest exit, and results in some major surgery (kudos to Dimitris
> >> Papastamos who came up with the magic vector decoding sequence).
> >
> > So if a Cortex-A8 has bootloder set the IBE bit, and kernel has
> > ARM_ERRATA_430973 enabled, is Cortex-A8 already hardened then?
>
> Almost. See the extra BTB invalidation in fault.c.
OK thanks. So for Cortex-A8, only patch "[PATCH 2/3] arm: Invalidate
BTB on fatal signal for Cortex A8, A9, A12, A15 and A17" is needed
if I read that right.
Regards,
Tony
^ permalink raw reply
* [PATCH v2 1/4] dmaengine: xilinx_dma: populate dma caps properly
From: Appana Durga Kedareswara Rao @ 2018-01-08 17:25 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180108170655.GJ18649@localhost>
Hi,
<Snip>
>> >> + xdev->common.dst_addr_widths = BIT(addr_width / 8);
>> >> + xdev->common.src_addr_widths = BIT(addr_width / 8);
>> >
>> >Do you not support trf of 1byte, 2 bytes, or 4 bytes wide transfers?
>> >What is value of addr_width here typically? Usually controllers can
>> >support different widths and this is a surprise that you support only
>> >one value
>>
>> Controller supports address width of 32 and 64.
>
>Then this should have both 32 and 64 values here
Address width is configurable parameter at the h/w level.
Since this IP is a soft IP user can create a design with either
32-bit or 64-bit address configuration.
Currently we are reading this configuration through device-tree (xlnx, addr-width property)
https://git.kernel.org/pub/scm/linux/kernel/git/vkoul/slave-dma.git/tree/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt#n19
Based on the h/w configuration setting the dst_addr_widths/src_addr_widths variables in this patch.
Please let me know if you are still not clear with my explanation will explain in detail...
Regards,
Kedar.
>
>> addr_width typical values are 32-bit or 64-bit .
>> Here addr_width is device-tree parameter...
>> my understanding of src_addr_widths/dst_addr_widths is, it is a bit
>> mask of the address with in bytes that DMA supports, please correct if my
>understanding is wrong.
>>
>> Regards,
>> Kedar.
>>
>> >
>> >--
>> >~Vinod
>
>--
>~Vinod
^ permalink raw reply
* mt76: fix memcpy to potential null pointer on failed allocation
From: Kalle Valo @ 2018-01-08 17:28 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20171214101322.10178-1-colin.king@canonical.com>
Colin Ian King <colin.king@canonical.com> wrote:
> From: Colin Ian King <colin.king@canonical.com>
>
> Currently if the allocation of skb fails and returns NULL then the
> call to skb_put will cause a null pointer dereference. Fix this by
> checking for a null skb and returning NULL. Note that calls to
> function mt76x2_mcu_msg_alloc don't directly check the null return
> but instead pass the NULL pointer to mt76x2_mcu_msg_send which
> checks for the NULL and returns ENOMEM in this case.
>
> Detected by CoverityScan, CID#1462624 ("Dereference null return value")
>
> Fixes: 7bc04215a66b ("mt76: add driver code for MT76x2e")
> Signed-off-by: Colin Ian King <colin.king@canonical.com>
> Acked-by: Felix Fietkau <nbd@nbd.name>
Patch applied to wireless-drivers-next.git, thanks.
364bea50dbea mt76: fix memcpy to potential null pointer on failed allocation
--
https://patchwork.kernel.org/patch/10111747/
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches
^ permalink raw reply
* [PATCH 0/3] ARM branch predictor hardening
From: Marc Zyngier @ 2018-01-08 17:30 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180108172435.GS3875@atomide.com>
On 08/01/18 17:24, Tony Lindgren wrote:
> * Marc Zyngier <marc.zyngier@arm.com> [180108 17:05]:
>> Hi Tony,
>>
>> On 08/01/18 16:54, Tony Lindgren wrote:
>>> * Marc Zyngier <marc.zyngier@arm.com> [180106 04:14]:
>>>> This small series implements some basic BP hardening by invalidating
>>>> the BTB on CPUs that are known to be susceptible to aliasing attacks.
>>>>
>>>> These patches are closely modelled against what we do on arm64,
>>>> although simpler as we can rely on an architected instruction to
>>>> perform the invalidation.
>>>>
>>>> The first patch reuses the Cortex-A8 BTB invalidation in switch_mm and
>>>> generalises it to be used on all affected CPUs. The second perform the
>>>> same invalidation on fatal signal delivery. The last one nukes it on
>>>> guest exit, and results in some major surgery (kudos to Dimitris
>>>> Papastamos who came up with the magic vector decoding sequence).
>>>
>>> So if a Cortex-A8 has bootloder set the IBE bit, and kernel has
>>> ARM_ERRATA_430973 enabled, is Cortex-A8 already hardened then?
>>
>> Almost. See the extra BTB invalidation in fault.c.
>
> OK thanks. So for Cortex-A8, only patch "[PATCH 2/3] arm: Invalidate
> BTB on fatal signal for Cortex A8, A9, A12, A15 and A17" is needed
> if I read that right.
That's the idea. Although I'm about to change that patch as it's been
pointed out to me that I'm doing that in a preemptible context, which
makes it a bit dodgy.
I should have something to post later today, once I'm done with fixing
the Thumb2 build (which completely breaks on patch #3)...
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply
* [PATCH v3 00/13] arm64 kpti hardening and variant 2 workarounds
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
Hi all,
This is version three of the patches previously posted here:
v1: http://lists.infradead.org/pipermail/linux-arm-kernel/2018-January/551838.html
v2: http://lists.infradead.org/pipermail/linux-arm-kernel/2018-January/552085.html
Changes since v2:
* Fix typo in comment
* Include Falkor hardening from Shanker
* Add ThunderX2 MIDRs (subsequent patches under review)
* Avoid applying hardening from preemtible context
* Fix stack offsets in hyp SMC call
Cheers,
Will
--->8
Jayachandran C (1):
arm64: cputype: Add MIDR values for Cavium ThunderX2 CPUs
Marc Zyngier (3):
arm64: Move post_ttbr_update_workaround to C code
arm64: KVM: Use per-CPU vector when BP hardening is enabled
arm64: KVM: Make PSCI_VERSION a fast path
Shanker Donthineni (1):
arm64: Implement branch predictor hardening for Falkor
Will Deacon (8):
arm64: use RET instruction for exiting the trampoline
arm64: Kconfig: Reword UNMAP_KERNEL_AT_EL0 kconfig entry
arm64: Take into account ID_AA64PFR0_EL1.CSV3
arm64: cpufeature: Pass capability structure to ->enable callback
drivers/firmware: Expose psci_get_version through psci_ops structure
arm64: Add skeleton to harden the branch predictor against aliasing
attacks
arm64: cputype: Add missing MIDR values for Cortex-A72 and Cortex-A75
arm64: Implement branch predictor hardening for affected Cortex-A CPUs
arch/arm/include/asm/kvm_mmu.h | 10 +++
arch/arm64/Kconfig | 30 +++++--
arch/arm64/include/asm/assembler.h | 13 ---
arch/arm64/include/asm/cpucaps.h | 4 +-
arch/arm64/include/asm/cputype.h | 7 ++
arch/arm64/include/asm/kvm_asm.h | 2 +
arch/arm64/include/asm/kvm_mmu.h | 38 +++++++++
arch/arm64/include/asm/mmu.h | 37 +++++++++
arch/arm64/include/asm/sysreg.h | 2 +
arch/arm64/kernel/Makefile | 4 +
arch/arm64/kernel/bpi.S | 87 ++++++++++++++++++++
arch/arm64/kernel/cpu_errata.c | 161 +++++++++++++++++++++++++++++++++++++
arch/arm64/kernel/cpufeature.c | 13 ++-
arch/arm64/kernel/entry.S | 19 ++++-
arch/arm64/kvm/hyp/entry.S | 12 +++
arch/arm64/kvm/hyp/switch.c | 25 +++++-
arch/arm64/mm/context.c | 11 +++
arch/arm64/mm/fault.c | 17 ++++
arch/arm64/mm/proc.S | 3 +-
drivers/firmware/psci.c | 2 +
include/linux/psci.h | 1 +
virt/kvm/arm/arm.c | 8 +-
22 files changed, 474 insertions(+), 32 deletions(-)
create mode 100644 arch/arm64/kernel/bpi.S
--
2.1.4
^ permalink raw reply
* [PATCH v3 01/13] arm64: use RET instruction for exiting the trampoline
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
Speculation attacks against the entry trampoline can potentially resteer
the speculative instruction stream through the indirect branch and into
arbitrary gadgets within the kernel.
This patch defends against these attacks by forcing a misprediction
through the return stack: a dummy BL instruction loads an entry into
the stack, so that the predicted program flow of the subsequent RET
instruction is to a branch-to-self instruction which is finally resolved
as a branch to the kernel vectors with speculation suppressed.
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm64/kernel/entry.S | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 031392ee5f47..6ceed4877daf 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -1029,6 +1029,14 @@ alternative_else_nop_endif
.if \regsize == 64
msr tpidrro_el0, x30 // Restored in kernel_ventry
.endif
+ /*
+ * Defend against branch aliasing attacks by pushing a dummy
+ * entry onto the return stack and using a RET instruction to
+ * enter the full-fat kernel vectors.
+ */
+ bl 2f
+ b .
+2:
tramp_map_kernel x30
#ifdef CONFIG_RANDOMIZE_BASE
adr x30, tramp_vectors + PAGE_SIZE
@@ -1041,7 +1049,7 @@ alternative_insn isb, nop, ARM64_WORKAROUND_QCOM_FALKOR_E1003
msr vbar_el1, x30
add x30, x30, #(1b - tramp_vectors)
isb
- br x30
+ ret
.endm
.macro tramp_exit, regsize = 64
--
2.1.4
^ permalink raw reply related
* [PATCH v3 02/13] arm64: Kconfig: Reword UNMAP_KERNEL_AT_EL0 kconfig entry
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
Although CONFIG_UNMAP_KERNEL_AT_EL0 does make KASLR more robust, it's
actually more useful as a mitigation against speculation attacks that
can leak arbitrary kernel data to userspace through speculation.
Reword the Kconfig help message to reflect this, and make the option
depend on EXPERT so that it is on by default for the majority of users.
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm64/Kconfig | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 3af1657fcac3..efaaa3a66b95 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -834,15 +834,14 @@ config FORCE_MAX_ZONEORDER
4M allocations matching the default size used by generic code.
config UNMAP_KERNEL_AT_EL0
- bool "Unmap kernel when running in userspace (aka \"KAISER\")"
+ bool "Unmap kernel when running in userspace (aka \"KAISER\")" if EXPERT
default y
help
- Some attacks against KASLR make use of the timing difference between
- a permission fault which could arise from a page table entry that is
- present in the TLB, and a translation fault which always requires a
- page table walk. This option defends against these attacks by unmapping
- the kernel whilst running in userspace, therefore forcing translation
- faults for all of kernel space.
+ Speculation attacks against some high-performance processors can
+ be used to bypass MMU permission checks and leak kernel data to
+ userspace. This can be defended against by unmapping the kernel
+ when running in userspace, mapping it back in on exception entry
+ via a trampoline page in the vector table.
If unsure, say Y.
--
2.1.4
^ permalink raw reply related
* [PATCH v3 03/13] arm64: Take into account ID_AA64PFR0_EL1.CSV3
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
For non-KASLR kernels where the KPTI behaviour has not been overridden
on the command line we can use ID_AA64PFR0_EL1.CSV3 to determine whether
or not we should unmap the kernel whilst running at EL0.
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/kernel/cpufeature.c | 8 +++++++-
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 08cc88574659..ae519bbd3f9e 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -437,6 +437,7 @@
#define ID_AA64ISAR1_DPB_SHIFT 0
/* id_aa64pfr0 */
+#define ID_AA64PFR0_CSV3_SHIFT 60
#define ID_AA64PFR0_SVE_SHIFT 32
#define ID_AA64PFR0_GIC_SHIFT 24
#define ID_AA64PFR0_ASIMD_SHIFT 20
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9f0545dfe497..d723fc071f39 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -145,6 +145,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar1[] = {
};
static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SVE_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_GIC_SHIFT, 4, 0),
S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI),
@@ -851,6 +852,8 @@ static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */
static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
int __unused)
{
+ u64 pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
+
/* Forced on command line? */
if (__kpti_forced) {
pr_info_once("kernel page table isolation forced %s by command line option\n",
@@ -862,7 +865,9 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
if (IS_ENABLED(CONFIG_RANDOMIZE_BASE))
return true;
- return false;
+ /* Defer to CPU feature registers */
+ return !cpuid_feature_extract_unsigned_field(pfr0,
+ ID_AA64PFR0_CSV3_SHIFT);
}
static int __init parse_kpti(char *str)
@@ -967,6 +972,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
},
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
{
+ .desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
.def_scope = SCOPE_SYSTEM,
.matches = unmap_kernel_at_el0,
--
2.1.4
^ permalink raw reply related
* [PATCH v3 04/13] arm64: cpufeature: Pass capability structure to ->enable callback
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
In order to invoke the CPU capability ->matches callback from the ->enable
callback for applying local-CPU workarounds, we need a handle on the
capability structure.
This patch passes a pointer to the capability structure to the ->enable
callback.
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm64/kernel/cpufeature.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index d723fc071f39..55712ab4e3bf 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1152,7 +1152,7 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
* uses an IPI, giving us a PSTATE that disappears when
* we return.
*/
- stop_machine(caps->enable, NULL, cpu_online_mask);
+ stop_machine(caps->enable, (void *)caps, cpu_online_mask);
}
}
}
@@ -1195,7 +1195,7 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps)
cpu_die_early();
}
if (caps->enable)
- caps->enable(NULL);
+ caps->enable((void *)caps);
}
}
--
2.1.4
^ permalink raw reply related
* [PATCH v3 05/13] drivers/firmware: Expose psci_get_version through psci_ops structure
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
Entry into recent versions of ARM Trusted Firmware will invalidate the CPU
branch predictor state in order to protect against aliasing attacks.
This patch exposes the PSCI "VERSION" function via psci_ops, so that it
can be invoked outside of the PSCI driver where necessary.
Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
drivers/firmware/psci.c | 2 ++
include/linux/psci.h | 1 +
2 files changed, 3 insertions(+)
diff --git a/drivers/firmware/psci.c b/drivers/firmware/psci.c
index d687ca3d5049..8b25d31e8401 100644
--- a/drivers/firmware/psci.c
+++ b/drivers/firmware/psci.c
@@ -496,6 +496,8 @@ static void __init psci_init_migrate(void)
static void __init psci_0_2_set_functions(void)
{
pr_info("Using standard PSCI v0.2 function IDs\n");
+ psci_ops.get_version = psci_get_version;
+
psci_function_id[PSCI_FN_CPU_SUSPEND] =
PSCI_FN_NATIVE(0_2, CPU_SUSPEND);
psci_ops.cpu_suspend = psci_cpu_suspend;
diff --git a/include/linux/psci.h b/include/linux/psci.h
index bdea1cb5e1db..6306ab10af18 100644
--- a/include/linux/psci.h
+++ b/include/linux/psci.h
@@ -26,6 +26,7 @@ int psci_cpu_init_idle(unsigned int cpu);
int psci_cpu_suspend_enter(unsigned long index);
struct psci_operations {
+ u32 (*get_version)(void);
int (*cpu_suspend)(u32 state, unsigned long entry_point);
int (*cpu_off)(u32 state);
int (*cpu_on)(unsigned long cpuid, unsigned long entry_point);
--
2.1.4
^ permalink raw reply related
* [PATCH v3 06/13] arm64: Move post_ttbr_update_workaround to C code
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
From: Marc Zyngier <marc.zyngier@arm.com>
We will soon need to invoke a CPU-specific function pointer after changing
page tables, so move post_ttbr_update_workaround out into C code to make
this possible.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/assembler.h | 13 -------------
arch/arm64/kernel/entry.S | 2 +-
arch/arm64/mm/context.c | 9 +++++++++
arch/arm64/mm/proc.S | 3 +--
4 files changed, 11 insertions(+), 16 deletions(-)
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index c45bc94f15d0..cee60ce0da52 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -476,17 +476,4 @@ alternative_endif
mrs \rd, sp_el0
.endm
-/*
- * Errata workaround post TTBRx_EL1 update.
- */
- .macro post_ttbr_update_workaround
-#ifdef CONFIG_CAVIUM_ERRATUM_27456
-alternative_if ARM64_WORKAROUND_CAVIUM_27456
- ic iallu
- dsb nsh
- isb
-alternative_else_nop_endif
-#endif
- .endm
-
#endif /* __ASM_ASSEMBLER_H */
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 6ceed4877daf..80b539845da6 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -277,7 +277,7 @@ alternative_else_nop_endif
* Cavium erratum 27456 (broadcast TLBI instructions may cause I-cache
* corruption).
*/
- post_ttbr_update_workaround
+ bl post_ttbr_update_workaround
.endif
1:
.if \el != 0
diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c
index 1cb3bc92ae5c..5f7097d0cd12 100644
--- a/arch/arm64/mm/context.c
+++ b/arch/arm64/mm/context.c
@@ -239,6 +239,15 @@ void check_and_switch_context(struct mm_struct *mm, unsigned int cpu)
cpu_switch_mm(mm->pgd, mm);
}
+/* Errata workaround post TTBRx_EL1 update. */
+asmlinkage void post_ttbr_update_workaround(void)
+{
+ asm(ALTERNATIVE("nop; nop; nop",
+ "ic iallu; dsb nsh; isb",
+ ARM64_WORKAROUND_CAVIUM_27456,
+ CONFIG_CAVIUM_ERRATUM_27456));
+}
+
static int asids_init(void)
{
asid_bits = get_cpu_asid_bits();
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 3146dc96f05b..6affb68a9a14 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -145,8 +145,7 @@ ENTRY(cpu_do_switch_mm)
isb
msr ttbr0_el1, x0 // now update TTBR0
isb
- post_ttbr_update_workaround
- ret
+ b post_ttbr_update_workaround // Back to C code...
ENDPROC(cpu_do_switch_mm)
.pushsection ".idmap.text", "ax"
--
2.1.4
^ permalink raw reply related
* [PATCH v3 07/13] arm64: Add skeleton to harden the branch predictor against aliasing attacks
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
Aliasing attacks against CPU branch predictors can allow an attacker to
redirect speculative control flow on some CPUs and potentially divulge
information from one context to another.
This patch adds initial skeleton code behind a new Kconfig option to
enable implementation-specific mitigations against these attacks for
CPUs that are affected.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm64/Kconfig | 17 +++++++++
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/include/asm/mmu.h | 37 ++++++++++++++++++++
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/kernel/Makefile | 4 +++
arch/arm64/kernel/bpi.S | 55 +++++++++++++++++++++++++++++
arch/arm64/kernel/cpu_errata.c | 74 ++++++++++++++++++++++++++++++++++++++++
arch/arm64/kernel/cpufeature.c | 1 +
arch/arm64/kernel/entry.S | 7 ++--
arch/arm64/mm/context.c | 2 ++
arch/arm64/mm/fault.c | 17 +++++++++
11 files changed, 215 insertions(+), 3 deletions(-)
create mode 100644 arch/arm64/kernel/bpi.S
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index efaaa3a66b95..cea44b95187c 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -845,6 +845,23 @@ config UNMAP_KERNEL_AT_EL0
If unsure, say Y.
+config HARDEN_BRANCH_PREDICTOR
+ bool "Harden the branch predictor against aliasing attacks" if EXPERT
+ default y
+ help
+ Speculation attacks against some high-performance processors rely on
+ being able to manipulate the branch predictor for a victim context by
+ executing aliasing branches in the attacker context. Such attacks
+ can be partially mitigated against by clearing internal branch
+ predictor state and limiting the prediction logic in some situations.
+
+ This config option will take CPU-specific actions to harden the
+ branch predictor against aliasing attacks and may rely on specific
+ instruction sequences or control bits being set by the system
+ firmware.
+
+ If unsure, say Y.
+
menuconfig ARMV8_DEPRECATED
bool "Emulate deprecated/obsolete ARMv8 instructions"
depends on COMPAT
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index b4537ffd1018..51616e77fe6b 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -42,7 +42,8 @@
#define ARM64_HAS_DCPOP 21
#define ARM64_SVE 22
#define ARM64_UNMAP_KERNEL_AT_EL0 23
+#define ARM64_HARDEN_BRANCH_PREDICTOR 24
-#define ARM64_NCAPS 24
+#define ARM64_NCAPS 25
#endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h
index 6f7bdb89817f..6dd83d75b82a 100644
--- a/arch/arm64/include/asm/mmu.h
+++ b/arch/arm64/include/asm/mmu.h
@@ -41,6 +41,43 @@ static inline bool arm64_kernel_unmapped_at_el0(void)
cpus_have_const_cap(ARM64_UNMAP_KERNEL_AT_EL0);
}
+typedef void (*bp_hardening_cb_t)(void);
+
+struct bp_hardening_data {
+ int hyp_vectors_slot;
+ bp_hardening_cb_t fn;
+};
+
+#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
+extern char __bp_harden_hyp_vecs_start[], __bp_harden_hyp_vecs_end[];
+
+DECLARE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data);
+
+static inline struct bp_hardening_data *arm64_get_bp_hardening_data(void)
+{
+ return this_cpu_ptr(&bp_hardening_data);
+}
+
+static inline void arm64_apply_bp_hardening(void)
+{
+ struct bp_hardening_data *d;
+
+ if (!cpus_have_const_cap(ARM64_HARDEN_BRANCH_PREDICTOR))
+ return;
+
+ d = arm64_get_bp_hardening_data();
+ if (d->fn)
+ d->fn();
+}
+#else
+static inline struct bp_hardening_data *arm64_get_bp_hardening_data(void)
+{
+ return NULL;
+}
+
+static inline void arm64_apply_bp_hardening(void) { }
+#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
+
extern void paging_init(void);
extern void bootmem_init(void);
extern void __iomem *early_io_map(phys_addr_t phys, unsigned long virt);
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index ae519bbd3f9e..871744973ece 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -438,6 +438,7 @@
/* id_aa64pfr0 */
#define ID_AA64PFR0_CSV3_SHIFT 60
+#define ID_AA64PFR0_CSV2_SHIFT 56
#define ID_AA64PFR0_SVE_SHIFT 32
#define ID_AA64PFR0_GIC_SHIFT 24
#define ID_AA64PFR0_ASIMD_SHIFT 20
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 067baace74a0..0c760db04858 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -53,6 +53,10 @@ arm64-obj-$(CONFIG_ARM64_RELOC_TEST) += arm64-reloc-test.o
arm64-reloc-test-y := reloc_test_core.o reloc_test_syms.o
arm64-obj-$(CONFIG_CRASH_DUMP) += crash_dump.o
+ifeq ($(CONFIG_KVM),y)
+arm64-obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o
+endif
+
obj-y += $(arm64-obj-y) vdso/ probes/
obj-m += $(arm64-obj-m)
head-y := head.o
diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S
new file mode 100644
index 000000000000..06a931eb2673
--- /dev/null
+++ b/arch/arm64/kernel/bpi.S
@@ -0,0 +1,55 @@
+/*
+ * Contains CPU specific branch predictor invalidation sequences
+ *
+ * Copyright (C) 2018 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+
+.macro ventry target
+ .rept 31
+ nop
+ .endr
+ b \target
+.endm
+
+.macro vectors target
+ ventry \target + 0x000
+ ventry \target + 0x080
+ ventry \target + 0x100
+ ventry \target + 0x180
+
+ ventry \target + 0x200
+ ventry \target + 0x280
+ ventry \target + 0x300
+ ventry \target + 0x380
+
+ ventry \target + 0x400
+ ventry \target + 0x480
+ ventry \target + 0x500
+ ventry \target + 0x580
+
+ ventry \target + 0x600
+ ventry \target + 0x680
+ ventry \target + 0x700
+ ventry \target + 0x780
+.endm
+
+ .align 11
+ENTRY(__bp_harden_hyp_vecs_start)
+ .rept 4
+ vectors __kvm_hyp_vector
+ .endr
+ENTRY(__bp_harden_hyp_vecs_end)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 0e27f86ee709..16ea5c6f314e 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -46,6 +46,80 @@ static int cpu_enable_trap_ctr_access(void *__unused)
return 0;
}
+#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
+#include <asm/mmu_context.h>
+#include <asm/cacheflush.h>
+
+DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data);
+
+#ifdef CONFIG_KVM
+static void __copy_hyp_vect_bpi(int slot, const char *hyp_vecs_start,
+ const char *hyp_vecs_end)
+{
+ void *dst = lm_alias(__bp_harden_hyp_vecs_start + slot * SZ_2K);
+ int i;
+
+ for (i = 0; i < SZ_2K; i += 0x80)
+ memcpy(dst + i, hyp_vecs_start, hyp_vecs_end - hyp_vecs_start);
+
+ flush_icache_range((uintptr_t)dst, (uintptr_t)dst + SZ_2K);
+}
+
+static void __install_bp_hardening_cb(bp_hardening_cb_t fn,
+ const char *hyp_vecs_start,
+ const char *hyp_vecs_end)
+{
+ static int last_slot = -1;
+ static DEFINE_SPINLOCK(bp_lock);
+ int cpu, slot = -1;
+
+ spin_lock(&bp_lock);
+ for_each_possible_cpu(cpu) {
+ if (per_cpu(bp_hardening_data.fn, cpu) == fn) {
+ slot = per_cpu(bp_hardening_data.hyp_vectors_slot, cpu);
+ break;
+ }
+ }
+
+ if (slot == -1) {
+ last_slot++;
+ BUG_ON(((__bp_harden_hyp_vecs_end - __bp_harden_hyp_vecs_start)
+ / SZ_2K) <= last_slot);
+ slot = last_slot;
+ __copy_hyp_vect_bpi(slot, hyp_vecs_start, hyp_vecs_end);
+ }
+
+ __this_cpu_write(bp_hardening_data.hyp_vectors_slot, slot);
+ __this_cpu_write(bp_hardening_data.fn, fn);
+ spin_unlock(&bp_lock);
+}
+#else
+static void __install_bp_hardening_cb(bp_hardening_cb_t fn,
+ const char *hyp_vecs_start,
+ const char *hyp_vecs_end)
+{
+ __this_cpu_write(bp_hardening_data.fn, fn);
+}
+#endif /* CONFIG_KVM */
+
+static void install_bp_hardening_cb(const struct arm64_cpu_capabilities *entry,
+ bp_hardening_cb_t fn,
+ const char *hyp_vecs_start,
+ const char *hyp_vecs_end)
+{
+ u64 pfr0;
+
+ if (!entry->matches(entry, SCOPE_LOCAL_CPU))
+ return;
+
+ pfr0 = read_cpuid(ID_AA64PFR0_EL1);
+ if (cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_CSV2_SHIFT))
+ return;
+
+ __install_bp_hardening_cb(fn, hyp_vecs_start, hyp_vecs_end);
+}
+#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
+
#define MIDR_RANGE(model, min, max) \
.def_scope = SCOPE_LOCAL_CPU, \
.matches = is_affected_midr_range, \
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 55712ab4e3bf..9d4d82c11528 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -146,6 +146,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar1[] = {
static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV2_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SVE_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_GIC_SHIFT, 4, 0),
S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI),
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 80b539845da6..07a7d4db8ec4 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -721,12 +721,15 @@ el0_ia:
* Instruction abort handling
*/
mrs x26, far_el1
- enable_daif
+ enable_da_f
+#ifdef CONFIG_TRACE_IRQFLAGS
+ bl trace_hardirqs_off
+#endif
ct_user_exit
mov x0, x26
mov x1, x25
mov x2, sp
- bl do_mem_abort
+ bl do_el0_ia_bp_hardening
b ret_to_user
el0_fpsimd_acc:
/*
diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c
index 5f7097d0cd12..d99b36555a16 100644
--- a/arch/arm64/mm/context.c
+++ b/arch/arm64/mm/context.c
@@ -246,6 +246,8 @@ asmlinkage void post_ttbr_update_workaround(void)
"ic iallu; dsb nsh; isb",
ARM64_WORKAROUND_CAVIUM_27456,
CONFIG_CAVIUM_ERRATUM_27456));
+
+ arm64_apply_bp_hardening();
}
static int asids_init(void)
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 22168cd0dde7..0e671ddf4855 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -708,6 +708,23 @@ asmlinkage void __exception do_mem_abort(unsigned long addr, unsigned int esr,
arm64_notify_die("", regs, &info, esr);
}
+asmlinkage void __exception do_el0_ia_bp_hardening(unsigned long addr,
+ unsigned int esr,
+ struct pt_regs *regs)
+{
+ /*
+ * We've taken an instruction abort from userspace and not yet
+ * re-enabled IRQs. If the address is a kernel address, apply
+ * BP hardening prior to enabling IRQs and pre-emption.
+ */
+ if (addr > TASK_SIZE)
+ arm64_apply_bp_hardening();
+
+ local_irq_enable();
+ do_mem_abort(addr, esr, regs);
+}
+
+
asmlinkage void __exception do_sp_pc_abort(unsigned long addr,
unsigned int esr,
struct pt_regs *regs)
--
2.1.4
^ permalink raw reply related
* [PATCH v3 08/13] arm64: KVM: Use per-CPU vector when BP hardening is enabled
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
From: Marc Zyngier <marc.zyngier@arm.com>
Now that we have per-CPU vectors, let's plug then in the KVM/arm64 code.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm/include/asm/kvm_mmu.h | 10 ++++++++++
arch/arm64/include/asm/kvm_mmu.h | 38 ++++++++++++++++++++++++++++++++++++++
arch/arm64/kvm/hyp/switch.c | 2 +-
virt/kvm/arm/arm.c | 8 +++++++-
4 files changed, 56 insertions(+), 2 deletions(-)
diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index fa6f2174276b..eb46fc81a440 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -221,6 +221,16 @@ static inline unsigned int kvm_get_vmid_bits(void)
return 8;
}
+static inline void *kvm_get_hyp_vector(void)
+{
+ return kvm_ksym_ref(__kvm_hyp_vector);
+}
+
+static inline int kvm_map_vectors(void)
+{
+ return 0;
+}
+
#endif /* !__ASSEMBLY__ */
#endif /* __ARM_KVM_MMU_H__ */
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 672c8684d5c2..2d6d4bd9de52 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -309,5 +309,43 @@ static inline unsigned int kvm_get_vmid_bits(void)
return (cpuid_feature_extract_unsigned_field(reg, ID_AA64MMFR1_VMIDBITS_SHIFT) == 2) ? 16 : 8;
}
+#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
+#include <asm/mmu.h>
+
+static inline void *kvm_get_hyp_vector(void)
+{
+ struct bp_hardening_data *data = arm64_get_bp_hardening_data();
+ void *vect = kvm_ksym_ref(__kvm_hyp_vector);
+
+ if (data->fn) {
+ vect = __bp_harden_hyp_vecs_start +
+ data->hyp_vectors_slot * SZ_2K;
+
+ if (!has_vhe())
+ vect = lm_alias(vect);
+ }
+
+ return vect;
+}
+
+static inline int kvm_map_vectors(void)
+{
+ return create_hyp_mappings(kvm_ksym_ref(__bp_harden_hyp_vecs_start),
+ kvm_ksym_ref(__bp_harden_hyp_vecs_end),
+ PAGE_HYP_EXEC);
+}
+
+#else
+static inline void *kvm_get_hyp_vector(void)
+{
+ return kvm_ksym_ref(__kvm_hyp_vector);
+}
+
+static inline int kvm_map_vectors(void)
+{
+ return 0;
+}
+#endif
+
#endif /* __ASSEMBLY__ */
#endif /* __ARM64_KVM_MMU_H__ */
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index f7c651f3a8c0..8d4f3c9d6dc4 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -52,7 +52,7 @@ static void __hyp_text __activate_traps_vhe(void)
val &= ~(CPACR_EL1_FPEN | CPACR_EL1_ZEN);
write_sysreg(val, cpacr_el1);
- write_sysreg(__kvm_hyp_vector, vbar_el1);
+ write_sysreg(kvm_get_hyp_vector(), vbar_el1);
}
static void __hyp_text __activate_traps_nvhe(void)
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 6b60c98a6e22..1c9fdb6db124 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -1158,7 +1158,7 @@ static void cpu_init_hyp_mode(void *dummy)
pgd_ptr = kvm_mmu_get_httbr();
stack_page = __this_cpu_read(kvm_arm_hyp_stack_page);
hyp_stack_ptr = stack_page + PAGE_SIZE;
- vector_ptr = (unsigned long)kvm_ksym_ref(__kvm_hyp_vector);
+ vector_ptr = (unsigned long)kvm_get_hyp_vector();
__cpu_init_hyp_mode(pgd_ptr, hyp_stack_ptr, vector_ptr);
__cpu_init_stage2();
@@ -1403,6 +1403,12 @@ static int init_hyp_mode(void)
goto out_err;
}
+ err = kvm_map_vectors();
+ if (err) {
+ kvm_err("Cannot map vectors\n");
+ goto out_err;
+ }
+
/*
* Map the Hyp stack pages
*/
--
2.1.4
^ permalink raw reply related
* [PATCH v3 09/13] arm64: KVM: Make PSCI_VERSION a fast path
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
From: Marc Zyngier <marc.zyngier@arm.com>
For those CPUs that require PSCI to perform a BP invalidation,
going all the way to the PSCI code for not much is a waste of
precious cycles. Let's terminate that call as early as possible.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm64/kvm/hyp/switch.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 8d4f3c9d6dc4..4d273f6d0e69 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -17,6 +17,7 @@
#include <linux/types.h>
#include <linux/jump_label.h>
+#include <uapi/linux/psci.h>
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
@@ -341,6 +342,18 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
goto again;
+ if (exit_code == ARM_EXCEPTION_TRAP &&
+ (kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC64 ||
+ kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC32) &&
+ vcpu_get_reg(vcpu, 0) == PSCI_0_2_FN_PSCI_VERSION) {
+ u64 val = PSCI_RET_NOT_SUPPORTED;
+ if (test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features))
+ val = 2;
+
+ vcpu_set_reg(vcpu, 0, val);
+ goto again;
+ }
+
if (static_branch_unlikely(&vgic_v2_cpuif_trap) &&
exit_code == ARM_EXCEPTION_TRAP) {
bool valid;
--
2.1.4
^ permalink raw reply related
* [PATCH v3 10/13] arm64: cputype: Add missing MIDR values for Cortex-A72 and Cortex-A75
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
Hook up MIDR values for the Cortex-A72 and Cortex-A75 CPUs, since they
will soon need MIDR matches for hardening the branch predictor.
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/cputype.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 235e77d98261..84385b94e70b 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -79,8 +79,10 @@
#define ARM_CPU_PART_AEM_V8 0xD0F
#define ARM_CPU_PART_FOUNDATION 0xD00
#define ARM_CPU_PART_CORTEX_A57 0xD07
+#define ARM_CPU_PART_CORTEX_A72 0xD08
#define ARM_CPU_PART_CORTEX_A53 0xD03
#define ARM_CPU_PART_CORTEX_A73 0xD09
+#define ARM_CPU_PART_CORTEX_A75 0xD0A
#define APM_CPU_PART_POTENZA 0x000
@@ -94,7 +96,9 @@
#define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
#define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)
+#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73)
+#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
--
2.1.4
^ permalink raw reply related
* [PATCH v3 11/13] arm64: Implement branch predictor hardening for affected Cortex-A CPUs
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
Cortex-A57, A72, A73 and A75 are susceptible to branch predictor aliasing
and can theoretically be attacked by malicious code.
This patch implements a PSCI-based mitigation for these CPUs when available.
The call into firmware will invalidate the branch predictor state, preventing
any malicious entries from affecting other victim contexts.
Co-developed-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm64/kernel/bpi.S | 24 ++++++++++++++++++++++++
arch/arm64/kernel/cpu_errata.c | 42 ++++++++++++++++++++++++++++++++++++++++++
2 files changed, 66 insertions(+)
diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S
index 06a931eb2673..dec95bd82e31 100644
--- a/arch/arm64/kernel/bpi.S
+++ b/arch/arm64/kernel/bpi.S
@@ -53,3 +53,27 @@ ENTRY(__bp_harden_hyp_vecs_start)
vectors __kvm_hyp_vector
.endr
ENTRY(__bp_harden_hyp_vecs_end)
+ENTRY(__psci_hyp_bp_inval_start)
+ sub sp, sp, #(8 * 18)
+ stp x16, x17, [sp, #(16 * 0)]
+ stp x14, x15, [sp, #(16 * 1)]
+ stp x12, x13, [sp, #(16 * 2)]
+ stp x10, x11, [sp, #(16 * 3)]
+ stp x8, x9, [sp, #(16 * 4)]
+ stp x6, x7, [sp, #(16 * 5)]
+ stp x4, x5, [sp, #(16 * 6)]
+ stp x2, x3, [sp, #(16 * 7)]
+ stp x0, x1, [sp, #(16 * 8)]
+ mov x0, #0x84000000
+ smc #0
+ ldp x16, x17, [sp, #(16 * 0)]
+ ldp x14, x15, [sp, #(16 * 1)]
+ ldp x12, x13, [sp, #(16 * 2)]
+ ldp x10, x11, [sp, #(16 * 3)]
+ ldp x8, x9, [sp, #(16 * 4)]
+ ldp x6, x7, [sp, #(16 * 5)]
+ ldp x4, x5, [sp, #(16 * 6)]
+ ldp x2, x3, [sp, #(16 * 7)]
+ ldp x0, x1, [sp, #(16 * 8)]
+ add sp, sp, #(8 * 18)
+ENTRY(__psci_hyp_bp_inval_end)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 16ea5c6f314e..cb0fb3796bb8 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -53,6 +53,8 @@ static int cpu_enable_trap_ctr_access(void *__unused)
DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data);
#ifdef CONFIG_KVM
+extern char __psci_hyp_bp_inval_start[], __psci_hyp_bp_inval_end[];
+
static void __copy_hyp_vect_bpi(int slot, const char *hyp_vecs_start,
const char *hyp_vecs_end)
{
@@ -94,6 +96,9 @@ static void __install_bp_hardening_cb(bp_hardening_cb_t fn,
spin_unlock(&bp_lock);
}
#else
+#define __psci_hyp_bp_inval_start NULL
+#define __psci_hyp_bp_inval_end NULL
+
static void __install_bp_hardening_cb(bp_hardening_cb_t fn,
const char *hyp_vecs_start,
const char *hyp_vecs_end)
@@ -118,6 +123,21 @@ static void install_bp_hardening_cb(const struct arm64_cpu_capabilities *entry,
__install_bp_hardening_cb(fn, hyp_vecs_start, hyp_vecs_end);
}
+
+#include <linux/psci.h>
+
+static int enable_psci_bp_hardening(void *data)
+{
+ const struct arm64_cpu_capabilities *entry = data;
+
+ if (psci_ops.get_version)
+ install_bp_hardening_cb(entry,
+ (bp_hardening_cb_t)psci_ops.get_version,
+ __psci_hyp_bp_inval_start,
+ __psci_hyp_bp_inval_end);
+
+ return 0;
+}
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
#define MIDR_RANGE(model, min, max) \
@@ -261,6 +281,28 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
},
#endif
+#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
+ {
+ .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
+ .enable = enable_psci_bp_hardening,
+ },
+ {
+ .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ .enable = enable_psci_bp_hardening,
+ },
+ {
+ .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ .enable = enable_psci_bp_hardening,
+ },
+ {
+ .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
+ .enable = enable_psci_bp_hardening,
+ },
+#endif
{
}
};
--
2.1.4
^ permalink raw reply related
* [PATCH v3 12/13] arm64: Implement branch predictor hardening for Falkor
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
From: Shanker Donthineni <shankerd@codeaurora.org>
Falkor is susceptible to branch predictor aliasing and can
theoretically be attacked by malicious code. This patch
implements a mitigation for these attacks, preventing any
malicious entries from affecting other victim contexts.
Signed-off-by: Shanker Donthineni <shankerd@codeaurora.org>
[will: fix label name when !CONFIG_KVM]
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/cpucaps.h | 3 ++-
arch/arm64/include/asm/kvm_asm.h | 2 ++
arch/arm64/kernel/bpi.S | 8 +++++++
arch/arm64/kernel/cpu_errata.c | 49 ++++++++++++++++++++++++++++++++++++++--
arch/arm64/kvm/hyp/entry.S | 12 ++++++++++
arch/arm64/kvm/hyp/switch.c | 10 ++++++++
6 files changed, 81 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index 51616e77fe6b..7049b4802587 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -43,7 +43,8 @@
#define ARM64_SVE 22
#define ARM64_UNMAP_KERNEL_AT_EL0 23
#define ARM64_HARDEN_BRANCH_PREDICTOR 24
+#define ARM64_HARDEN_BP_POST_GUEST_EXIT 25
-#define ARM64_NCAPS 25
+#define ARM64_NCAPS 26
#endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index ab4d0a926043..24961b732e65 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -68,6 +68,8 @@ extern u32 __kvm_get_mdcr_el2(void);
extern u32 __init_stage2_translation(void);
+extern void __qcom_hyp_sanitize_btac_predictors(void);
+
#endif
#endif /* __ARM_KVM_ASM_H__ */
diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S
index dec95bd82e31..76225c2611ea 100644
--- a/arch/arm64/kernel/bpi.S
+++ b/arch/arm64/kernel/bpi.S
@@ -77,3 +77,11 @@ ENTRY(__psci_hyp_bp_inval_start)
ldp x0, x1, [sp, #(16 * 8)]
add sp, sp, #(8 * 18)
ENTRY(__psci_hyp_bp_inval_end)
+
+ENTRY(__qcom_hyp_sanitize_link_stack_start)
+ stp x29, x30, [sp, #-16]!
+ .rept 16
+ bl . + 4
+ .endr
+ ldp x29, x30, [sp], #16
+ENTRY(__qcom_hyp_sanitize_link_stack_end)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index cb0fb3796bb8..7b4efde087fc 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -54,6 +54,8 @@ DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data);
#ifdef CONFIG_KVM
extern char __psci_hyp_bp_inval_start[], __psci_hyp_bp_inval_end[];
+extern char __qcom_hyp_sanitize_link_stack_start[];
+extern char __qcom_hyp_sanitize_link_stack_end[];
static void __copy_hyp_vect_bpi(int slot, const char *hyp_vecs_start,
const char *hyp_vecs_end)
@@ -96,8 +98,10 @@ static void __install_bp_hardening_cb(bp_hardening_cb_t fn,
spin_unlock(&bp_lock);
}
#else
-#define __psci_hyp_bp_inval_start NULL
-#define __psci_hyp_bp_inval_end NULL
+#define __psci_hyp_bp_inval_start NULL
+#define __psci_hyp_bp_inval_end NULL
+#define __qcom_hyp_sanitize_link_stack_start NULL
+#define __qcom_hyp_sanitize_link_stack_end NULL
static void __install_bp_hardening_cb(bp_hardening_cb_t fn,
const char *hyp_vecs_start,
@@ -138,6 +142,29 @@ static int enable_psci_bp_hardening(void *data)
return 0;
}
+
+static void qcom_link_stack_sanitization(void)
+{
+ u64 tmp;
+
+ asm volatile("mov %0, x30 \n"
+ ".rept 16 \n"
+ "bl . + 4 \n"
+ ".endr \n"
+ "mov x30, %0 \n"
+ : "=&r" (tmp));
+}
+
+static int qcom_enable_link_stack_sanitization(void *data)
+{
+ const struct arm64_cpu_capabilities *entry = data;
+
+ install_bp_hardening_cb(entry, qcom_link_stack_sanitization,
+ __qcom_hyp_sanitize_link_stack_start,
+ __qcom_hyp_sanitize_link_stack_end);
+
+ return 0;
+}
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
#define MIDR_RANGE(model, min, max) \
@@ -302,6 +329,24 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
.enable = enable_psci_bp_hardening,
},
+ {
+ .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
+ MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ .enable = qcom_enable_link_stack_sanitization,
+ },
+ {
+ .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
+ MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
+ .enable = qcom_enable_link_stack_sanitization,
+ },
+ {
+ .capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
+ MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ },
+ {
+ .capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
+ MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
+ },
#endif
{
}
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index 12ee62d6d410..9c45c6af1f58 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -196,3 +196,15 @@ alternative_endif
eret
ENDPROC(__fpsimd_guest_restore)
+
+ENTRY(__qcom_hyp_sanitize_btac_predictors)
+ /**
+ * Call SMC64 with Silicon provider serviceID 23<<8 (0xc2001700)
+ * 0xC2000000-0xC200FFFF: assigned to SiP Service Calls
+ * b15-b0: contains SiP functionID
+ */
+ movz x0, #0x1700
+ movk x0, #0xc200, lsl #16
+ smc #0
+ ret
+ENDPROC(__qcom_hyp_sanitize_btac_predictors)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 4d273f6d0e69..7e373791fad1 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -406,6 +406,16 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
/* 0 falls through to be handled out of EL2 */
}
+ if (cpus_have_const_cap(ARM64_HARDEN_BP_POST_GUEST_EXIT)) {
+ u32 midr = read_cpuid_id();
+
+ /* Apply BTAC predictors mitigation to all Falkor chips */
+ if (((midr & MIDR_CPU_MODEL_MASK) == MIDR_QCOM_FALKOR) ||
+ ((midr & MIDR_CPU_MODEL_MASK) == MIDR_QCOM_FALKOR_V1)) {
+ __qcom_hyp_sanitize_btac_predictors();
+ }
+ }
+
fp_enabled = __fpsimd_enabled();
__sysreg_save_guest_state(guest_ctxt);
--
2.1.4
^ permalink raw reply related
* [PATCH v3 13/13] arm64: cputype: Add MIDR values for Cavium ThunderX2 CPUs
From: Will Deacon @ 2018-01-08 17:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515432758-26440-1-git-send-email-will.deacon@arm.com>
From: Jayachandran C <jnair@caviumnetworks.com>
Add the older Broadcom ID as well as the new Cavium ID for ThunderX2
CPUs.
Signed-off-by: Jayachandran C <jnair@caviumnetworks.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/cputype.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 84385b94e70b..cce5735a677c 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -89,6 +89,7 @@
#define CAVIUM_CPU_PART_THUNDERX 0x0A1
#define CAVIUM_CPU_PART_THUNDERX_81XX 0x0A2
#define CAVIUM_CPU_PART_THUNDERX_83XX 0x0A3
+#define CAVIUM_CPU_PART_THUNDERX2 0x0AF
#define BRCM_CPU_PART_VULCAN 0x516
@@ -102,6 +103,8 @@
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
+#define MIDR_CAVIUM_THUNDERX2 MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX2)
+#define MIDR_BRCM_VULCAN MIDR_CPU_MODEL(ARM_CPU_IMP_BRCM, BRCM_CPU_PART_VULCAN)
#define MIDR_QCOM_FALKOR_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_FALKOR_V1)
#ifndef __ASSEMBLY__
--
2.1.4
^ permalink raw reply related
* [v2,03/11] arm64: Take into account ID_AA64PFR0_EL1.CSV3
From: Jayachandran C @ 2018-01-08 17:40 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <9bc1f137-d78c-e46e-e1bc-f49160d5f289@arm.com>
On Mon, Jan 08, 2018 at 09:20:09AM +0000, Marc Zyngier wrote:
> On 08/01/18 07:24, Jayachandran C wrote:
> > On Fri, Jan 05, 2018 at 01:12:33PM +0000, Will Deacon wrote:
> >> For non-KASLR kernels where the KPTI behaviour has not been overridden
> >> on the command line we can use ID_AA64PFR0_EL1.CSV3 to determine whether
> >> or not we should unmap the kernel whilst running at EL0.
> >>
> >> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> >> Signed-off-by: Will Deacon <will.deacon@arm.com>
> >> ---
> >> arch/arm64/include/asm/sysreg.h | 1 +
> >> arch/arm64/kernel/cpufeature.c | 8 +++++++-
> >> 2 files changed, 8 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> >> index 08cc88574659..ae519bbd3f9e 100644
> >> --- a/arch/arm64/include/asm/sysreg.h
> >> +++ b/arch/arm64/include/asm/sysreg.h
> >> @@ -437,6 +437,7 @@
> >> #define ID_AA64ISAR1_DPB_SHIFT 0
> >>
> >> /* id_aa64pfr0 */
> >> +#define ID_AA64PFR0_CSV3_SHIFT 60
> >> #define ID_AA64PFR0_SVE_SHIFT 32
> >> #define ID_AA64PFR0_GIC_SHIFT 24
> >> #define ID_AA64PFR0_ASIMD_SHIFT 20
> >> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> >> index 9f0545dfe497..d723fc071f39 100644
> >> --- a/arch/arm64/kernel/cpufeature.c
> >> +++ b/arch/arm64/kernel/cpufeature.c
> >> @@ -145,6 +145,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar1[] = {
> >> };
> >>
> >> static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
> >> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0),
> >> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SVE_SHIFT, 4, 0),
> >> ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_GIC_SHIFT, 4, 0),
> >> S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI),
> >> @@ -851,6 +852,8 @@ static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */
> >> static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
> >> int __unused)
> >> {
> >> + u64 pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
> >> +
> >> /* Forced on command line? */
> >> if (__kpti_forced) {
> >> pr_info_once("kernel page table isolation forced %s by command line option\n",
> >> @@ -862,7 +865,9 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
> >> if (IS_ENABLED(CONFIG_RANDOMIZE_BASE))
> >> return true;
> >>
> >> - return false;
> >> + /* Defer to CPU feature registers */
> >> + return !cpuid_feature_extract_unsigned_field(pfr0,
> >> + ID_AA64PFR0_CSV3_SHIFT);
> >
> > If I read this correctly, this enables KPTI on all processors without the CSV3
> > set (which seems to be a future capability).
> >
> > Turning on KPTI has a small but significant overhead, so I think we should turn
> > it off on processors that are not vulnerable to CVE-2017-5754. Can we add something
> > like this:
> >
> > --->8
> > diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> > index 19ed09b..202b037 100644
> > --- a/arch/arm64/kernel/cpufeature.c
> > +++ b/arch/arm64/kernel/cpufeature.c
> > @@ -862,6 +862,13 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
> > return __kpti_forced > 0;
> > }
> >
> > + /* Don't force KPTI for CPUs that are not vulnerable */
> > + switch (read_cpuid_id() & MIDR_CPU_MODEL_MASK) {
> > + case MIDR_CAVIUM_THUNDERX2:
> > + case MIDR_BRCM_VULCAN:
> > + return false;
> > + }
> > +
> > /* Useful for KASLR robustness */
> > if (IS_ENABLED(CONFIG_RANDOMIZE_BASE))
> > return true;
> >
>
> KPTI is also an improvement for KASLR. Why would you deprive a user of
> the choice to further secure their system?
The user has a choice with kpti= at the kernel command line, so we are
not depriving the user of a choice. KASLR is expected to be enabled by
distributions, and KPTI will be enabled by default as well.
On systems that are not vulnerable to variant 3, this is an unnecessary
overhead.
JC
^ permalink raw reply
* [PATCH 00/10] perf tools: Add support for CoreSight trace decoding
From: Mathieu Poirier @ 2018-01-08 17:45 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20171230005137.GE600@leoy-linaro>
Good day Leo,
On 29 December 2017 at 17:51, Leo Yan <leo.yan@linaro.org> wrote:
> Hi Mathieu,
>
> On Fri, Dec 15, 2017 at 09:44:49AM -0700, Mathieu Poirier wrote:
>> This patchset adds support for per-thread CoreSight trace decoding from the
>> "perf report" interface. It is largely modelled on what has been done for
>> intelPT traces and currently targets the ETMv4 architecture. Support for
>> cpu-wide scenarios and ETMv3/PTMv1.1 will follow shortly.
>>
>> The trace decoding support is done using the Open CoreSight Decoding
>> Library (openCSD), a stand alone open source project available here [1].
>> Integration of the openCSD library with the perf tools follow what has
>> been done for other support libraries. If the library has been installed
>> on a system the build scripts will include support for CoreSight trace
>> decoding:
>>
>> ... zlib: [ on ]
>> ... lzma: [ OFF ]
>> ... get_cpuid: [ on ]
>> ... bpf: [ on ]
>> ... libopencsd: [ on ] <------
>>
>> Instructions on how to build and install the openCSD library are provided
>> in the HOWTO.md of the project repository. We elected to keep the decoder
>> library independent of the kernel tree as it is also used outside of the
>> perf toolset and various non-linux projects.
>>
>> The work applies cleanly to [2] and proper functionning of the feature
>> depends on this patch [3].
>
> With latest perf code, it reports another error when analyse perf
> data: "0x3e0 [0x50]: failed to process type: 1".
>
> After roughly analysis, I found this is caused by one dummy event (in
> the binary from offset 0xf8 to offset 0x178). Because this event type
> is not set for 'PERF_SAMPLE_TIME', so the function
> perf_evsel__parse_sample_timestamp() checks the event has not set
> 'PERF_SAMPLE_TIME' then directly bail out with error.
This patch should fix the problem: https://patchwork.kernel.org/patch/10121515/
>
> 000000f0: 0800 0000 0000 0000 0100 0000 7000 0000 ............p...
> 00000100: 0900 0000 0000 0000 0100 0000 0000 0000 ................
> 00000110: 0300 0100 0000 0000 0400 0000 0000 0000 ................
> 00000120: 6133 8401 0000 0000 0000 0000 0000 0000 a3..............
> 00000130: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000140: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000150: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000160: 0000 0000 0000 0000 7000 0000 0000 0000 ........p.......
> 00000170: 0800 0000 0000 0000 4600 0000 0000 6802 ........F.....h.
>
> You could check the perf binary from [1]. Please note, this perf data
> I capatured from kernel 4.14-rc6, so is it might be compatible issue
> between 4.14-rc6 and 4.15?
>
> [1] http://people.linaro.org/~leo.yan/binaries/perf_4.15_r4/perf.data
>
> Thanks,
> Leo Yan
>
>> Review and comments would be greatly appreciated.
>>
>> Regards,
>> Mathieu
>>
>> [1]. https://github.com/Linaro/OpenCSD
>> [2]. git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git perf/core
>> [3]. https://lkml.org/lkml/2017/12/14/612
>>
>> Mathieu Poirier (8):
>> perf tools: Integrating the CoreSight decoding library
>> perf tools: Add initial entry point for decoder CoreSight traces
>> perf tools: Add decoder mechanic to support dumping trace data
>> perf tools: Add support for decoding CoreSight trace data
>> perf tools: Add functionality to communicate with the openCSD decoder
>> pert tools: Add queue management functionality
>> perf tools: Add full support for CoreSight trace decoding
>> perf tools: Add mechanic to synthesise CoreSight trace packets
>>
>> Tor Jeremiassen (2):
>> perf tools: Add processing of coresight metadata
>> MAINTAINERS: Adding entry for CoreSight trace decoding
>>
>> MAINTAINERS | 3 +-
>> tools/build/Makefile.feature | 6 +-
>> tools/build/feature/Makefile | 6 +-
>> tools/build/feature/test-all.c | 5 +
>> tools/build/feature/test-libopencsd.c | 8 +
>> tools/perf/Makefile.config | 13 +
>> tools/perf/util/Build | 6 +
>> tools/perf/util/auxtrace.c | 2 +
>> tools/perf/util/cs-etm-decoder/Build | 1 +
>> tools/perf/util/cs-etm-decoder/cs-etm-decoder.c | 513 ++++++++++++
>> tools/perf/util/cs-etm-decoder/cs-etm-decoder.h | 105 +++
>> tools/perf/util/cs-etm.c | 1023 +++++++++++++++++++++++
>> tools/perf/util/cs-etm.h | 18 +
>> 13 files changed, 1705 insertions(+), 4 deletions(-)
>> create mode 100644 tools/build/feature/test-libopencsd.c
>> create mode 100644 tools/perf/util/cs-etm-decoder/Build
>> create mode 100644 tools/perf/util/cs-etm-decoder/cs-etm-decoder.c
>> create mode 100644 tools/perf/util/cs-etm-decoder/cs-etm-decoder.h
>> create mode 100644 tools/perf/util/cs-etm.c
>>
>> --
>> 2.7.4
>>
^ permalink raw reply
* [PATCH 1/3] arm: Add BTB invalidation on switch_mm for Cortex-A9, A12, A15 and A17
From: Robin Murphy @ 2018-01-08 17:50 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <f48d11af-32b7-1c20-abaa-1efdf24616d0@arm.com>
Hi Marc,
On 08/01/18 14:46, Marc Zyngier wrote:
> On 08/01/18 14:29, Andre Przywara wrote:
>> Hi,
>>
>> On 06/01/18 12:09, Marc Zyngier wrote:
>>> In order to avoid aliasing attacks against the branch predictor,
>>> some implementations require to invalidate the BTB when switching
>>> from one user context to another.
>>>
>>> For this, we reuse the existing implementation for Cortex-A8, and
>>> apply it to A9, A12, A15, and A17.
>>>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>> arch/arm/mm/proc-v7-2level.S | 4 ++--
>>> arch/arm/mm/proc-v7-3level.S | 6 ++++++
>>> arch/arm/mm/proc-v7.S | 32 ++++++++++++++++----------------
>>> 3 files changed, 24 insertions(+), 18 deletions(-)
>>>
>>> diff --git a/arch/arm/mm/proc-v7-2level.S b/arch/arm/mm/proc-v7-2level.S
>>> index c6141a5435c3..0422e58b74e8 100644
>>> --- a/arch/arm/mm/proc-v7-2level.S
>>> +++ b/arch/arm/mm/proc-v7-2level.S
>>> @@ -41,7 +41,7 @@
>>> * even on Cortex-A8 revisions not affected by 430973.
>>> * If IBE is not set, the flush BTAC/BTB won't do anything.
>>> */
>>> -ENTRY(cpu_ca8_switch_mm)
>>> +ENTRY(cpu_v7_btbinv_switch_mm)
>>> #ifdef CONFIG_MMU
>>> mov r2, #0
>>> mcr p15, 0, r2, c7, c5, 6 @ flush BTAC/BTB
>>> @@ -66,7 +66,7 @@ ENTRY(cpu_v7_switch_mm)
>>> #endif
>>> bx lr
>>> ENDPROC(cpu_v7_switch_mm)
>>> -ENDPROC(cpu_ca8_switch_mm)
>>> +ENDPROC(cpu_v7_btbinv_switch_mm)
>>>
>>> /*
>>> * cpu_v7_set_pte_ext(ptep, pte)
>>> diff --git a/arch/arm/mm/proc-v7-3level.S b/arch/arm/mm/proc-v7-3level.S
>>> index 7d16bbc4102b..934272e1fa08 100644
>>> --- a/arch/arm/mm/proc-v7-3level.S
>>> +++ b/arch/arm/mm/proc-v7-3level.S
>>> @@ -54,6 +54,11 @@
>>> * Set the translation table base pointer to be pgd_phys (physical address of
>>> * the new TTB).
>>> */
>>> +ENTRY(cpu_v7_btbinv_switch_mm)
>>> +#ifdef CONFIG_MMU
>>> + mov r2, #0
>>> + mcr p15, 0, r2, c7, c5, 6 @ flush BTAC/BTB
>>
>> r2 is already used below: this is the second argument to the function.
>> The first argument is a phys_addr_t, so it takes both r0 and r1 already.
>> That's why we need to use r3 above.
>> This fixes a crash on the Midway for me.
>
> I suppose you have DRAM above 4GB. Lucky you! I'll fix that in v2.
Why even zero a register at all? Since BPIALL ignores the value of Rt
(DDI0406C.c, p1744) it shouldn't matter which register you use nor what
it contains.
Robin.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox