* [PATCH v7 08/24] ASoC: qdsp6: q6core: Add q6core driver
From: Banajit Goswami @ 2018-05-04 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180501120820.11016-9-srinivas.kandagatla@linaro.org>
On 5/1/2018 5:08 AM, Srinivas Kandagatla wrote:
> This patch adds support to core apr service, which is used to query
> status of other static and dynamic services on the dsp.
>
> Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
> Reviewed-and-tested-by: Rohit kumar <rohitkr@codeaurora.org>
> ---
> sound/soc/qcom/Kconfig | 4 +
> sound/soc/qcom/qdsp6/Makefile | 1 +
> sound/soc/qcom/qdsp6/q6core.c | 380 ++++++++++++++++++++++++++++++++++++++++++
> sound/soc/qcom/qdsp6/q6core.h | 15 ++
> 4 files changed, 400 insertions(+)
> create mode 100644 sound/soc/qcom/qdsp6/q6core.c
> create mode 100644 sound/soc/qcom/qdsp6/q6core.h
>
> diff --git a/sound/soc/qcom/Kconfig b/sound/soc/qcom/Kconfig
> index b44a9fcd7ed3..37ee0d958145 100644
> --- a/sound/soc/qcom/Kconfig
> +++ b/sound/soc/qcom/Kconfig
> @@ -44,10 +44,14 @@ config SND_SOC_APQ8016_SBC
> config SND_SOC_QDSP6_COMMON
> tristate
>
> +config SND_SOC_QDSP6_CORE
> + tristate
> +
> config SND_SOC_QDSP6
> tristate "SoC ALSA audio driver for QDSP6"
> depends on QCOM_APR && HAS_DMA
> select SND_SOC_QDSP6_COMMON
> + select SND_SOC_QDSP6_CORE
> help
> To add support for MSM QDSP6 Soc Audio.
> This will enable sound soc platform specific
> diff --git a/sound/soc/qcom/qdsp6/Makefile b/sound/soc/qcom/qdsp6/Makefile
> index accebdb49306..03b8e89c9731 100644
> --- a/sound/soc/qcom/qdsp6/Makefile
> +++ b/sound/soc/qcom/qdsp6/Makefile
> @@ -1 +1,2 @@
> obj-$(CONFIG_SND_SOC_QDSP6_COMMON) += q6dsp-common.o
> +obj-$(CONFIG_SND_SOC_QDSP6_CORE) += q6core.o
> diff --git a/sound/soc/qcom/qdsp6/q6core.c b/sound/soc/qcom/qdsp6/q6core.c
> new file mode 100644
> index 000000000000..701aa3f50a6a
> --- /dev/null
> +++ b/sound/soc/qcom/qdsp6/q6core.c
> @@ -0,0 +1,380 @@
> +// SPDX-License-Identifier: GPL-2.0
> +// Copyright (c) 2011-2017, The Linux Foundation. All rights reserved.
> +// Copyright (c) 2018, Linaro Limited
> +
> +#include <linux/slab.h>
> +#include <linux/wait.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/sched.h>
> +#include <linux/of.h>
> +#include <linux/of_platform.h>
> +#include <linux/jiffies.h>
> +#include <linux/wait.h>
> +#include <linux/soc/qcom/apr.h>
> +#include "q6core.h"
> +#include "q6dsp-errno.h"
> +
> +#define ADSP_STATE_READY_TIMEOUT_MS 3000
> +#define Q6_READY_TIMEOUT_MS 100
> +#define AVCS_CMD_ADSP_EVENT_GET_STATE 0x0001290C
> +#define AVCS_CMDRSP_ADSP_EVENT_GET_STATE 0x0001290D
> +#define AVCS_GET_VERSIONS 0x00012905
> +#define AVCS_GET_VERSIONS_RSP 0x00012906
> +#define AVCS_CMD_GET_FWK_VERSION 0x001292c
> +#define AVCS_CMDRSP_GET_FWK_VERSION 0x001292d
> +
> +struct avcs_svc_info {
<snip>
> +};
> +
> +static struct q6core *g_core;
> +
> +static int q6core_callback(struct apr_device *adev, struct apr_resp_pkt *data)
> +{
> + struct q6core *core = dev_get_drvdata(&adev->dev);
> + struct aprv2_ibasic_rsp_result_t *result;
> + struct apr_hdr *hdr = &data->hdr;
> +
> + result = data->payload;
> + switch (hdr->opcode) {
> + case APR_BASIC_RSP_RESULT:{
> + result = data->payload;
> + switch (result->opcode) {
> + case AVCS_GET_VERSIONS:
> + if (result->status == ADSP_EUNSUPPORTED)
> + core->get_version_supported = false;
> + core->resp_received = true;
> + break;
> + case AVCS_CMD_GET_FWK_VERSION:
> + if (result->status == ADSP_EUNSUPPORTED)
> + core->fwk_version_supported = false;
> + core->resp_received = true;
> + break;
> + case AVCS_CMD_ADSP_EVENT_GET_STATE:
> + if (result->status == ADSP_EUNSUPPORTED)
> + core->get_state_supported = false;
> + core->resp_received = true;
> + break;
> + }
> + break;
> + }
> + case AVCS_CMDRSP_GET_FWK_VERSION: {
> + struct avcs_cmdrsp_get_fwk_version *fwk;
> + int bytes;
> +
> + fwk = data->payload;
> + core->fwk_version_supported = true;
> + bytes = sizeof(*fwk) + fwk->num_services *
> + sizeof(fwk->svc_api_info[0]);
> +
> + core->fwk_version = kzalloc(bytes, GFP_ATOMIC);
> + if (!core->fwk_version)
> + return -ENOMEM;
When the above allocation fails,? core->fwk_version_supported will be
still true, and q6core_get_fwk_versions() will return 0 (timeout as
core->resp_received will not be set to true). This can cause a NULL
pointer dereference inside the if() loop pointed below (added comment).
Please move the line to set core->fwk_version_supported flag to after
memset() to copy fwk version info.
> +
> + memcpy(core->fwk_version, data->payload, bytes);
> +
> + core->resp_received = true;
> +
> + break;
> + }
> + case AVCS_GET_VERSIONS_RSP: {
> + struct avcs_cmdrsp_get_version *v;
> + int len;
> +
> + v = data->payload;
> + core->get_version_supported = true;
> +
<snip>
> + }
> +
> + return rc;
> +}
> +
> +static bool __q6core_is_adsp_ready(struct q6core *core)
> +{
> + struct apr_device *adev = core->adev;
> + struct apr_pkt pkt;
> + int rc;
> +
> + core->get_state_supported = false;
> +
> + pkt.hdr.hdr_field = APR_HDR_FIELD(APR_MSG_TYPE_SEQ_CMD,
> + APR_HDR_LEN(APR_HDR_SIZE), APR_PKT_VER);
> + pkt.hdr.pkt_size = APR_HDR_SIZE;
> + pkt.hdr.opcode = AVCS_CMD_ADSP_EVENT_GET_STATE;
> +
> + rc = apr_send_pkt(adev, &pkt);
> + if (rc < 0)
> + return false;
> +
> + rc = wait_event_timeout(core->wait, (core->resp_received),
> + msecs_to_jiffies(Q6_READY_TIMEOUT_MS));
> + if (rc > 0 && core->resp_received) {
> + core->resp_received = false;
> +
> + if (core->avcs_state == 0x1)
The AVCS state can be different non-zero value then 0x1.
A better way to handle this can be check for (core->avcs_state > 0) for
success, and then return the "core->avcs_state" to the caller.
> + return true;
> + }
> +
> + /* assume that the adsp is up if we not support this command */
> + if (!core->get_state_supported)
> + return true;
> +
> + return false;
> +}
> +
> +/**
> + * q6core_get_svc_api_info() - Get version number of a service.
> + *
> + * @svc_id: service id of the service.
> + * @info: Valid struct pointer to fill svc api information.
> + *
> + * Return: zero on success and error code on failure or unsupported
> + */
> +int q6core_get_svc_api_info(int svc_id, struct q6core_svc_api_info *ainfo)
> +{
> + int i;
> + int ret = -ENOTSUPP;
> +
> + if (!g_core || !ainfo)
> + return 0;
> +
> + mutex_lock(&g_core->lock);
> + if (!g_core->is_version_requested) {
> + if (q6core_get_fwk_versions(g_core) == -ENOTSUPP)
> + q6core_get_svc_versions(g_core);
> + g_core->is_version_requested = true;
> + }
> +
> + if (g_core->fwk_version_supported) {
> + for (i = 0; i < g_core->fwk_version->num_services; i++) {
..NULL pointer dereference here.
> + struct avcs_svc_api_info *info;
> +
> + info = &g_core->fwk_version->svc_api_info[i];
> + if (svc_id != info->service_id)
> + continue;
> +
> + ainfo->api_version = info->api_version;
> + ainfo->api_branch_version = info->api_branch_version;
> + ret = 0;
> + break;
> + }
> + } else if (g_core->get_version_supported) {
> + for (i = 0; i < g_core->svc_version->num_services; i++) {
Similar issue of NULL pointer dereference is also present for
g_core->get_version_supported flag.
> + struct avcs_svc_info *info;
> +
> + info = &g_core->svc_version->svc_api_info[i];
> + if (svc_id != info->service_id)
> + continue;
> +
> + ainfo->api_version = info->version;
> + ainfo->api_branch_version = 0;
> + ret = 0;
> + break;
<snip>
> + init_waitqueue_head(&g_core->wait);
> + return 0;
> +}
> +
> +static int q6core_exit(struct apr_device *adev)
> +{
> + struct q6core *core = dev_get_drvdata(&adev->dev);
> +
> + if (core->fwk_version_supported)
> + kfree(core->fwk_version);
> + if (core->get_version_supported)
> + kfree(core->svc_version);
> +
> + kfree(core);
> + g_core = NULL;
This assignment can be before kfree() to avoid any possible issue in
using g_core, after the pointer is freed.
--
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
^ permalink raw reply
* [PATCH v6 3/6] kernel/reboot.c: export pm_power_off_prepare
From: Oleksij Rempel @ 2018-05-04 18:50 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180305102524.5905-4-o.rempel@pengutronix.de>
Hallo Andrew,
I need your ACK or NACK for this patch.
This function is used to configure external PMIC to interpret
signal which will be triggered by pm_power_off as power off.
Since same signal can be used for stand by, I linked PMIC configuration
with pm_power_off_prepare to avoid possible conflicts.
On Mon, Mar 05, 2018 at 11:25:20AM +0100, Oleksij Rempel wrote:
> Export pm_power_off_prepare. It is needed to implement power off on
> Freescale/NXP iMX6 based boards with external power management
> integrated circuit (PMIC).
>
> Signed-off-by: Oleksij Rempel <o.rempel@pengutronix.de>
> ---
> kernel/reboot.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/kernel/reboot.c b/kernel/reboot.c
> index e4ced883d8de..350be6baa60d 100644
> --- a/kernel/reboot.c
> +++ b/kernel/reboot.c
> @@ -49,6 +49,7 @@ int reboot_force;
> */
>
> void (*pm_power_off_prepare)(void);
> +EXPORT_SYMBOL(pm_power_off_prepare);
>
> /**
> * emergency_restart - reboot the system
> --
> 2.16.1
>
>
--
Pengutronix e.K. | |
Industrial Linux Solutions | http://www.pengutronix.de/ |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/1430215f/attachment.sig>
^ permalink raw reply
* [PATCH v4 2/2] ThunderX2: Add Cavium ThunderX2 SoC UNCORE PMU driver
From: Ganapatrao Kulkarni @ 2018-05-04 18:46 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180426105938.y6unpt36lisb7kbr@lakrids.cambridge.arm.com>
Hi Mark,
On Thu, Apr 26, 2018 at 4:29 PM, Mark Rutland <mark.rutland@arm.com> wrote:
> Hi,
>
> On Wed, Apr 25, 2018 at 02:30:47PM +0530, Ganapatrao Kulkarni wrote:
>> +
>> +/* L3c and DMC has 16 and 8 channels per socket respectively.
>> + * Each Channel supports UNCORE PMU device and consists of
>> + * 4 independent programmable counters. Counters are 32 bit
>> + * and does not support overflow interrupt, they needs to be
>> + * sampled before overflow(i.e, at every 2 seconds).
>> + */
>> +
>> +#define UNCORE_MAX_COUNTERS 4
>> +#define UNCORE_L3_MAX_TILES 16
>> +#define UNCORE_DMC_MAX_CHANNELS 8
>> +
>> +#define UNCORE_HRTIMER_INTERVAL (2 * NSEC_PER_SEC)
>
> How was a period of two seconds chosen?
It has been suggested from hw team to sample before 3 to 4 seconds.
>
> What's the maximum clock speed for the L3C and DMC?
L3C at 1.5GHz and DMC at 1.2GHz
>
> Given that all channels compete for access to the muxed register
> interface, I suspect we need to try more often than once every 2
> seconds...
2 seconds seems to be sufficient. So far testing looks good.
>
> [...]
>
>> +struct active_timer {
>> + struct perf_event *event;
>> + struct hrtimer hrtimer;
>> +};
>> +
>> +/*
>> + * pmu on each socket has 2 uncore devices(dmc and l3),
>> + * each uncore device has up to 16 channels, each channel can sample
>> + * events independently with counters up to 4.
>> + *
>> + * struct thunderx2_pmu_uncore_channel created per channel.
>> + * struct thunderx2_pmu_uncore_dev per uncore device.
>> + */
>> +struct thunderx2_pmu_uncore_channel {
>> + struct thunderx2_pmu_uncore_dev *uncore_dev;
>> + struct pmu pmu;
>
> Can we put the pmu first in the struct, please?
ok
>
>> + int counter;
>
> AFAICT, this counter field is never used.
hmm ok, will remove.
>
>> + int channel;
>> + DECLARE_BITMAP(counter_mask, UNCORE_MAX_COUNTERS);
>> + struct active_timer *active_timers;
>
> You should only need a single timer per channel, rather than one per
> event.
>
> I think you can get rid of the active_timer structure, and have:
>
> struct perf_event *events[UNCORE_MAX_COUNTERS];
> struct hrtimer timer;
>
thanks, will change as suggested.
>> + /* to sync counter alloc/release */
>> + raw_spinlock_t lock;
>> +};
>> +
>> +struct thunderx2_pmu_uncore_dev {
>> + char *name;
>> + struct device *dev;
>> + enum thunderx2_uncore_type type;
>> + unsigned long base;
>
> This should be:
>
> void __iomem *base;
ok
>
> [...]
>
>> +static ssize_t cpumask_show(struct device *dev,
>> + struct device_attribute *attr, char *buf)
>> +{
>> + struct cpumask cpu_mask;
>> + struct thunderx2_pmu_uncore_channel *pmu_uncore =
>> + pmu_to_thunderx2_pmu_uncore(dev_get_drvdata(dev));
>> +
>> + /* Pick first online cpu from the node */
>> + cpumask_clear(&cpu_mask);
>> + cpumask_set_cpu(cpumask_first(
>> + cpumask_of_node(pmu_uncore->uncore_dev->node)),
>> + &cpu_mask);
>> +
>> + return cpumap_print_to_pagebuf(true, buf, &cpu_mask);
>> +}
>
> AFAICT cpumask_of_node() returns a mask that can include offline CPUs.
>
> Regardless, I don't think that you can keep track of the management CPU
> this way. Please keep track of the CPU the PMU should be managed by,
> either with a cpumask or int embedded within the PMU structure.
thanks, will add hotplug callbacks.
>
> At hotplug time, you'll need to update the management CPU, calling
> perf_pmu_migrate_context() when it is offlined.
ok
>
> [...]
>
>> +static int alloc_counter(struct thunderx2_pmu_uncore_channel *pmu_uncore)
>> +{
>> + int counter;
>> +
>> + raw_spin_lock(&pmu_uncore->lock);
>> + counter = find_first_zero_bit(pmu_uncore->counter_mask,
>> + pmu_uncore->uncore_dev->max_counters);
>> + if (counter == pmu_uncore->uncore_dev->max_counters) {
>> + raw_spin_unlock(&pmu_uncore->lock);
>> + return -ENOSPC;
>> + }
>> + set_bit(counter, pmu_uncore->counter_mask);
>> + raw_spin_unlock(&pmu_uncore->lock);
>> + return counter;
>> +}
>> +
>> +static void free_counter(struct thunderx2_pmu_uncore_channel *pmu_uncore,
>> + int counter)
>> +{
>> + raw_spin_lock(&pmu_uncore->lock);
>> + clear_bit(counter, pmu_uncore->counter_mask);
>> + raw_spin_unlock(&pmu_uncore->lock);
>> +}
>
> I don't believe that locking is required in either of these, as the perf
> core serializes pmu::add() and pmu::del(), where these get called.
thanks, it seems to be not required.
>
>> +
>> +/*
>> + * DMC and L3 counter interface is muxed across all channels.
>> + * hence we need to select the channel before accessing counter
>> + * data/control registers.
>
> Are there separate interfaces for all-dmc-channels and all-l3c-channels?
separate interface for DMC and L3c.
channels within DMC/L3C are muxed.
>
> ... or is a single interface used by all-dmc-and-l3c-channels?
>
>> + *
>> + * L3 Tile and DMC channel selection is through SMC call
>> + * SMC call arguments,
>> + * x0 = THUNDERX2_SMC_CALL_ID (Vendor SMC call Id)
>> + * x1 = THUNDERX2_SMC_SET_CHANNEL (Id to set DMC/L3C channel)
>> + * x2 = Node id
>
> How do we map Linux node IDs to the firmware's view of node IDs?
>
> I don't believe the two are necessarily the same -- Linux's node IDs are
> a Linux-specific construct.
both are same, it is numa node id from ACPI/firmware.
>
> It would be much nicer if we could pass something based on the MPIDR,
> which is a known HW construct, or if this implicitly affected the
> current node.
IMO, node id is sufficient.
>
> It would be vastly more sane for this to not be muxed at all. :/
i am helpless due to crappy hw design!
>
>> + * x3 = DMC(1)/L3C(0)
>> + * x4 = channel Id
>> + */
>> +static void uncore_select_channel(struct perf_event *event)
>> +{
>> + struct arm_smccc_res res;
>> + struct thunderx2_pmu_uncore_channel *pmu_uncore =
>> + pmu_to_thunderx2_pmu_uncore(event->pmu);
>> +
>> + arm_smccc_smc(THUNDERX2_SMC_CALL_ID, THUNDERX2_SMC_SET_CHANNEL,
>> + pmu_uncore->uncore_dev->node,
>> + pmu_uncore->uncore_dev->type,
>> + pmu_uncore->channel, 0, 0, 0, &res);
>> +
>> +}
>
> Why aren't we checking the return value of the SMC call?
thanks, i will add.
>
> The muxing and SMC sound quite scary. :/
i too agree!, however i have no other option since the muxing register
is a secure register.
>
>> +
>> +static void uncore_start_event_l3c(struct perf_event *event, int flags)
>> +{
>> + u32 val;
>> + struct hw_perf_event *hwc = &event->hw;
>> +
>> + /* event id encoded in bits [07:03] */
>> + val = GET_EVENTID(event) << 3;
>> + reg_writel(val, hwc->config_base);
>> +
>> + if (flags & PERF_EF_RELOAD) {
>> + u64 prev_raw_count =
>> + local64_read(&event->hw.prev_count);
>> + reg_writel(prev_raw_count, hwc->event_base);
>> + }
>> + local64_set(&event->hw.prev_count,
>> + reg_readl(hwc->event_base));
>
> It would be simpler to ignore PERF_EF_RELOAD, and always reprogram the
> prev_count and HW to zero.
ok, will change
>
>> +}
>> +
>> +static void uncore_start_event_dmc(struct perf_event *event, int flags)
>> +{
>> + u32 val, event_shift = 8;
>> + struct hw_perf_event *hwc = &event->hw;
>> +
>> + /* enable and start counters and read current value in prev_count */
>> + val = reg_readl(hwc->config_base);
>> +
>> + /* 8 bits for each counter,
>> + * bits [05:01] of a counter to set event type.
>> + */
>> + reg_writel((val & ~(0x1f << (((GET_COUNTERID(event)) *
>> + event_shift) + 1))) |
>> + (GET_EVENTID(event) <<
>> + (((GET_COUNTERID(event)) * event_shift) + 1)),
>> + hwc->config_base);
>
> This would be far more legible if val were constructed before the
> reg_writel(), especially if there were a helper for the event field
> shifting, e.g.
>
> #define DMC_EVENT_CFG(idx, val) ((val) << (((idx) * 8) + 1))
>
> int id = GET_COUNTERID(event);
> int event = GET_EVENTID(event);
>
> val = reg_readl(hwc->config_base);
> val &= ~DMC_EVENT_CFG(id, 0x1f);
> val |= DMCC_EVENT_CFG(id, event);
> reg_writel(val, hwc->config_base));
>
> What are bits 7:6 and 1 used for?
not used/reserved bits.
>
>> +
>> + if (flags & PERF_EF_RELOAD) {
>> + u64 prev_raw_count =
>> + local64_read(&event->hw.prev_count);
>> + reg_writel(prev_raw_count, hwc->event_base);
>> + }
>> + local64_set(&event->hw.prev_count,
>> + reg_readl(hwc->event_base));
>
>
> As with the L3C events, it would be simpler to always reprogram the
> prev_count and HW to zero.
ok
>
>> +}
>> +
>> +static void uncore_stop_event_l3c(struct perf_event *event)
>> +{
>> + reg_writel(0, event->hw.config_base);
>> +}
>> +
>> +static void uncore_stop_event_dmc(struct perf_event *event)
>> +{
>> + u32 val, event_shift = 8;
>> + struct hw_perf_event *hwc = &event->hw;
>> +
>> + val = reg_readl(hwc->config_base);
>> + reg_writel((val & ~(0xff << ((GET_COUNTERID(event)) * event_shift))),
>> + hwc->config_base);
>
>
> This could be simplified with the helper proposed above.
ok
>
>> +}
>> +
>> +static void init_cntr_base_l3c(struct perf_event *event,
>> + struct thunderx2_pmu_uncore_dev *uncore_dev)
>> +{
>> +
>> + struct hw_perf_event *hwc = &event->hw;
>> +
>> + /* counter ctrl/data reg offset at 8 */
>
> Offset 8, or stride 8?
stride 8
>
> What does the register layout look like?
>
>> + hwc->config_base = uncore_dev->base
>> + + L3C_COUNTER_CTL + (8 * GET_COUNTERID(event));
>> + hwc->event_base = uncore_dev->base
>> + + L3C_COUNTER_DATA + (8 * GET_COUNTERID(event));
>> +}
>> +
>> +static void init_cntr_base_dmc(struct perf_event *event,
>> + struct thunderx2_pmu_uncore_dev *uncore_dev)
>> +{
>> +
>> + struct hw_perf_event *hwc = &event->hw;
>> +
>> + hwc->config_base = uncore_dev->base
>> + + DMC_COUNTER_CTL;
>> + /* counter data reg offset at 0xc */
>
> A stride of 0xc seems unusual.
>
> What does the register layout look like?
the offsets are at 0x640, 0x64c, 0x658, 0x664,
>
>> + hwc->event_base = uncore_dev->base
>> + + DMC_COUNTER_DATA + (0xc * GET_COUNTERID(event));
>> +}
>> +
>> +static void thunderx2_uncore_update(struct perf_event *event)
>> +{
>> + s64 prev, new = 0;
>> + u64 delta;
>> + struct hw_perf_event *hwc = &event->hw;
>> + struct thunderx2_pmu_uncore_channel *pmu_uncore;
>> + enum thunderx2_uncore_type type;
>> +
>> + pmu_uncore = pmu_to_thunderx2_pmu_uncore(event->pmu);
>> + type = pmu_uncore->uncore_dev->type;
>
> AFAICT this variable is not used below.
thanks.
>
>> +
>> + if (pmu_uncore->uncore_dev->select_channel)
>> + pmu_uncore->uncore_dev->select_channel(event);
>
> This should always be non-NULL, right?
yes, unwanted check at the moment, will remove.
>
> [...]
>
>> +static bool thunderx2_uncore_validate_event_group(struct perf_event *event)
>> +{
>> + struct pmu *pmu = event->pmu;
>> + struct perf_event *leader = event->group_leader;
>> + struct perf_event *sibling;
>> + int counters = 0;
>> +
>> + if (leader->pmu != event->pmu && !is_software_event(leader))
>> + return false;
>> +
>> + counters++;
>
> I don't think this is right when event != leader and the leader is a SW
> event. In that case, the leader doesn't take a HW counter.
I think this check to avoid the grouping of multiple hw PMUs ,
however it is allowed to group sw events along with hw PMUs.
>
>> +
>> + for_each_sibling_event(sibling, event->group_leader) {
>> + if (is_software_event(sibling))
>> + continue;
>> + if (sibling->pmu != pmu)
>> + return false;
>> + counters++;
>> + }
>> +
>> + /*
>> + * If the group requires more counters than the HW has,
>> + * it cannot ever be scheduled.
>> + */
>> + return counters <= UNCORE_MAX_COUNTERS;
>> +}
>
> [...]
>
>> +static int thunderx2_uncore_event_init(struct perf_event *event)
>> +{
>> + struct hw_perf_event *hwc = &event->hw;
>> + struct thunderx2_pmu_uncore_channel *pmu_uncore;
>
>> + pmu_uncore = pmu_to_thunderx2_pmu_uncore(event->pmu);
>> +
>> + if (!pmu_uncore)
>> + return -ENODEV;
>
> This cannot happen, given pmu_to_thunderx2_pmu_uncore() is a wrapper
> around container_of().
thanks, unnecessary check!
>
>> +
>> + /* Pick first online cpu from the node */
>> + event->cpu = cpumask_first(
>> + cpumask_of_node(pmu_uncore->uncore_dev->node));
>
> I don't believe this is safe. You must keep track of which CPU is
> managing the PMU, with hotplug callbacks.
ok, will add callbacks.
>
> [...]
>
>> +static void thunderx2_uncore_start(struct perf_event *event, int flags)
>> +{
>> + struct hw_perf_event *hwc = &event->hw;
>> + struct thunderx2_pmu_uncore_channel *pmu_uncore;
>> + struct thunderx2_pmu_uncore_dev *uncore_dev;
>> + unsigned long irqflags;
>> + struct active_timer *active_timer;
>> +
>> + hwc->state = 0;
>> + pmu_uncore = pmu_to_thunderx2_pmu_uncore(event->pmu);
>> + uncore_dev = pmu_uncore->uncore_dev;
>> +
>> + raw_spin_lock_irqsave(&uncore_dev->lock, irqflags);
>> +
>> + if (uncore_dev->select_channel)
>> + uncore_dev->select_channel(event);
>> + uncore_dev->start_event(event, flags);
>> + raw_spin_unlock_irqrestore(&uncore_dev->lock, irqflags);
>> +
>> + perf_event_update_userpage(event);
>> + active_timer = &pmu_uncore->active_timers[GET_COUNTERID(event)];
>> + active_timer->event = event;
>> +
>> + hrtimer_start(&active_timer->hrtimer,
>> + ns_to_ktime(uncore_dev->hrtimer_interval),
>> + HRTIMER_MODE_REL_PINNED);
>> +}
>
> Please use a single hrtimer, and update *all* of the events when it
> fires.
sure
>
> I *think* that can be done in the pmu::pmu_enable() and
> pmu::pmu_disable() callbacks.
ok
>
> Are there control bits to enable/disable all counters, or can that only
> be done through the event configuration registers?
only through config register.
>
>> +static void thunderx2_uncore_stop(struct perf_event *event, int flags)
>> +{
>> + struct hw_perf_event *hwc = &event->hw;
>> + struct thunderx2_pmu_uncore_channel *pmu_uncore;
>> + struct thunderx2_pmu_uncore_dev *uncore_dev;
>> + unsigned long irqflags;
>> +
>> + if (hwc->state & PERF_HES_UPTODATE)
>> + return;
>> +
>> + pmu_uncore = pmu_to_thunderx2_pmu_uncore(event->pmu);
>> + uncore_dev = pmu_uncore->uncore_dev;
>> +
>> + raw_spin_lock_irqsave(&uncore_dev->lock, irqflags);
>> +
>> + if (uncore_dev->select_channel)
>> + uncore_dev->select_channel(event);
>
> AFAICT this cannot be NULL.
ok.
>
> [...]
>
>> +static int thunderx2_pmu_uncore_register(
>> + struct thunderx2_pmu_uncore_channel *pmu_uncore)
>> +{
>> + struct device *dev = pmu_uncore->uncore_dev->dev;
>> + char *name = pmu_uncore->uncore_dev->name;
>> + int channel = pmu_uncore->channel;
>> +
>> + /* Perf event registration */
>> + pmu_uncore->pmu = (struct pmu) {
>> + .attr_groups = pmu_uncore->uncore_dev->attr_groups,
>> + .task_ctx_nr = perf_invalid_context,
>> + .event_init = thunderx2_uncore_event_init,
>> + .add = thunderx2_uncore_add,
>> + .del = thunderx2_uncore_del,
>> + .start = thunderx2_uncore_start,
>> + .stop = thunderx2_uncore_stop,
>> + .read = thunderx2_uncore_read,
>> + };
>> +
>> + pmu_uncore->pmu.name = devm_kasprintf(dev, GFP_KERNEL,
>> + "%s_%d", name, channel);
>
> Does the channel idx take the NUMA node into account?
name already has node id suffix.
>
> [...]
>
>> +static int thunderx2_pmu_uncore_add(struct thunderx2_pmu_uncore_dev *uncore_dev,
>> + int channel)
>> +{
>
>> + /* we can run up to (max_counters * max_channels) events
>> + * simultaneously, allocate hrtimers per channel.
>> + */
>> + pmu_uncore->active_timers = devm_kzalloc(uncore_dev->dev,
>> + sizeof(struct active_timer) * uncore_dev->max_counters,
>> + GFP_KERNEL);
>
> Please just fold a single hrtimer into the thunderx2_pmu_uncore_channel
> structure, and you can get rid of this allocation...
sure, i will rewrite code to have timer per channel and all active
counters are updated when timer fires.
>
>> +
>> + for (counter = 0; counter < uncore_dev->max_counters; counter++) {
>> + hrtimer_init(&pmu_uncore->active_timers[counter].hrtimer,
>> + CLOCK_MONOTONIC,
>> + HRTIMER_MODE_REL);
>> + pmu_uncore->active_timers[counter].hrtimer.function =
>> + thunderx2_uncore_hrtimer_callback;
>> + }
>
> ... and simplify this initialization.
ok
>
> [...]
>
>> +static struct thunderx2_pmu_uncore_dev *init_pmu_uncore_dev(
>> + struct device *dev, acpi_handle handle,
>> + struct acpi_device *adev, u32 type)
>> +{
>> + struct thunderx2_pmu_uncore_dev *uncore_dev;
>> + unsigned long base;
>
>> + base = (unsigned long)devm_ioremap_resource(dev, &res);
>> + if (IS_ERR((void *)base)) {
>> + dev_err(dev, "PMU type %d: Fail to map resource\n", type);
>> + return NULL;
>> + }
>
> Please treat this as a void __iomem *base.
ok
>
> [...]
>
>> +static int thunderx2_uncore_probe(struct platform_device *pdev)
>> +{
>> + struct device *dev = &pdev->dev;
>> + struct arm_smccc_res res;
>> +
>> + set_dev_node(dev, acpi_get_node(ACPI_HANDLE(dev)));
>> +
>> + /* Make sure firmware supports DMC/L3C set channel smc call */
>> + arm_smccc_smc(THUNDERX2_SMC_CALL_ID, THUNDERX2_SMC_SET_CHANNEL,
>> + dev_to_node(dev), 0, 0, 0, 0, 0, &res);
>> + if (res.a0) {
>> + dev_err(dev, "No Firmware support for PMU UNCORE(%d)\n",
>> + dev_to_node(dev));
>> + return -ENODEV;
>> + }
>
> Please re-use the uncore_select_channel() wrapper rather than
> open-coding this.
ok.
>
> Which FW supports this interface?
it is through vendor specific ATF runtime services.
>
> Thanks,
> Mark.
thanks,
Ganapat
^ permalink raw reply
* [PATCH v8 0/6] optimize memblock_next_valid_pfn and early_pfn_valid on arm and arm64
From: Daniel Vacek @ 2018-05-04 18:33 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <23b14717-0f4a-10f2-5118-7cb8445fbdab@oracle.com>
On Fri, May 4, 2018 at 6:53 PM, Pavel Tatashin
<pasha.tatashin@oracle.com> wrote:
>> I'm wondering, ain't simple enabling of config
>> DEFERRED_STRUCT_PAGE_INIT provide even better speed-up? If that is the
>> case then it seems like this series is not needed at all, right?
>> I am not sure why is this config optional. It looks like it could be
>> enabled by default or even unconditionally considering that with
>> commit c9e97a1997fb ("mm: initialize pages on demand during boot") the
>> deferred code is statically disabled after all the pages are
>> initialized.
>
> Hi Daniel,
>
> Currently, deferred struct pages are initialized in parallel only on NUMA machines. I would like to make a change to use all the available CPUs even on a single socket systems, but that is not there yet. So, I believe Jia's performance improvements are still relevant.
Ahaa, I thought it also works on UP or single node systems. I didn't
study the code closely. Sorry about the noise. And thank you, Pavel.
You're right.
--nX
> Thank you,
> Pavel
^ permalink raw reply
* [PATCH 1/6] locking/atomic, asm-generic: instrument ordering variants
From: Peter Zijlstra @ 2018-05-04 18:24 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180504180909.dnhfflibjwywnm4l@lakrids.cambridge.arm.com>
On Fri, May 04, 2018 at 07:09:09PM +0100, Mark Rutland wrote:
> On Fri, May 04, 2018 at 08:01:05PM +0200, Peter Zijlstra wrote:
> > On Fri, May 04, 2018 at 06:39:32PM +0100, Mark Rutland wrote:
> > > include/asm-generic/atomic-instrumented.h | 1195 ++++++++++++++++++++++++-----
> > > 1 file changed, 1008 insertions(+), 187 deletions(-)
> >
> > Is there really no way to either generate or further macro compress this?
>
> I can definitely macro compress this somewhat, but the bulk of the
> repetition will be the ifdeffery, which can't be macro'd away IIUC.
Right, much like what we already have in linux/atomic.h I suspect,
having to duplicating that isn't brilliant either.
> Generating this with a script is possible -- do we do anything like that
> elsewhere?
There's include/generated/ in your build directory. But nothing on this
scale I think.
^ permalink raw reply
* [PATCH 1/6] locking/atomic, asm-generic: instrument ordering variants
From: Mark Rutland @ 2018-05-04 18:09 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180504180105.GS12217@hirez.programming.kicks-ass.net>
On Fri, May 04, 2018 at 08:01:05PM +0200, Peter Zijlstra wrote:
> On Fri, May 04, 2018 at 06:39:32PM +0100, Mark Rutland wrote:
> > Currently <asm-generic/atomic-instrumented.h> only instruments the fully
> > ordered variants of atomic functions, ignoring the {relaxed,acquire,release}
> > ordering variants.
> >
> > This patch reworks the header to instrument all ordering variants of the atomic
> > functions, so that architectures implementing these are instrumented
> > appropriately.
> >
> > To minimise repetition, a macro is used to generate each variant from a common
> > template. The {full,relaxed,acquire,release} order variants respectively are
> > then built using this template, where the architecture provides an
> > implementation.
> > include/asm-generic/atomic-instrumented.h | 1195 ++++++++++++++++++++++++-----
> > 1 file changed, 1008 insertions(+), 187 deletions(-)
>
> Is there really no way to either generate or further macro compress this?
I can definitely macro compress this somewhat, but the bulk of the
repetition will be the ifdeffery, which can't be macro'd away IIUC.
Generating this with a script is possible -- do we do anything like that
elsewhere?
> This is stupid repetitive, we just got rid of all that endless copy
> paste crap in atomic implementations and now we're going back to that.
>
> Adding or changing atomic bits becomes horrifically painful because of this.
Sure thing; mangling it to its current state was a pain enough.
Thanks,
Mark.
^ permalink raw reply
* [PATCH 1/4] arm/arm64: smccc/psci: add arm_smccc_get_conduit()
From: Mark Rutland @ 2018-05-04 18:02 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <d04f8eff-06e3-2080-e03e-cd35b7b2c382@arm.com>
On Fri, May 04, 2018 at 06:54:14PM +0100, Robin Murphy wrote:
> On 03/05/18 18:03, Mark Rutland wrote:
> > We can kill off the PSCI_CONDUIT_* definitions once we've migrated users
> > over to the nerw interface.
>
> new
Whoops; fixed locally.
[...]
> > +/**
> > + * arm_smccc_get_conduit()
> > + *
> > + * Returns the conduit to be used for SMCCCv1.1 or later.
> > + *
> > + * When SMCCCv1.1 is not present, returns SMCCC_CONDUIT_NONE.
> > + */
> > +enum arm_smccc_conduit arm_smccc_get_conduit(void);
>
> Given that implicit condition, can we save some confusion by naming this
> arm_smccc_1_1_conduit(), in line with the actual SMCCCv1.1 calls?
Sure; done.
Thanks,
Mark.
^ permalink raw reply
* [PATCH 1/6] locking/atomic, asm-generic: instrument ordering variants
From: Peter Zijlstra @ 2018-05-04 18:01 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180504173937.25300-2-mark.rutland@arm.com>
On Fri, May 04, 2018 at 06:39:32PM +0100, Mark Rutland wrote:
> Currently <asm-generic/atomic-instrumented.h> only instruments the fully
> ordered variants of atomic functions, ignoring the {relaxed,acquire,release}
> ordering variants.
>
> This patch reworks the header to instrument all ordering variants of the atomic
> functions, so that architectures implementing these are instrumented
> appropriately.
>
> To minimise repetition, a macro is used to generate each variant from a common
> template. The {full,relaxed,acquire,release} order variants respectively are
> then built using this template, where the architecture provides an
> implementation.
>
> To stick to an 80 column limit while keeping the templates legible, the return
> type and function name of each template are split over two lines. For
> consistency, this is done even when not strictly necessary.
>
> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
> Cc: Boqun Feng <boqun.feng@gmail.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Will Deacon <will.deacon@arm.com>
> ---
> include/asm-generic/atomic-instrumented.h | 1195 ++++++++++++++++++++++++-----
> 1 file changed, 1008 insertions(+), 187 deletions(-)
Is there really no way to either generate or further macro compress this?
This is stupid repetitive, we just got rid of all that endless copy
paste crap in atomic implementations and now we're going back to that.
Adding or changing atomic bits becomes horrifically painful because of this.
^ permalink raw reply
* [PATCH 1/4] arm/arm64: smccc/psci: add arm_smccc_get_conduit()
From: Robin Murphy @ 2018-05-04 17:54 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180503170330.5591-2-mark.rutland@arm.com>
Hi Mark,
On 03/05/18 18:03, Mark Rutland wrote:
> SMCCC callers are currently amassing a collection of enums for the SMCCC
> conduit, and are having to dig into the PSCI driver's internals in order
> to figure out what to do.
>
> Let's clean this up, with common SMCCC_CONDUIT_* definitions, and an
> arm_smccc_get_conduit() helper that abstracts the PSCI driver's internal
> state.
>
> We can kill off the PSCI_CONDUIT_* definitions once we've migrated users
> over to the nerw interface.
new
> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> ---
> drivers/firmware/psci.c | 15 +++++++++++++++
> include/linux/arm-smccc.h | 16 ++++++++++++++++
> 2 files changed, 31 insertions(+)
>
> diff --git a/drivers/firmware/psci.c b/drivers/firmware/psci.c
> index c80ec1d03274..310882185ca4 100644
> --- a/drivers/firmware/psci.c
> +++ b/drivers/firmware/psci.c
> @@ -64,6 +64,21 @@ struct psci_operations psci_ops = {
> .smccc_version = SMCCC_VERSION_1_0,
> };
>
> +enum arm_smccc_conduit arm_smccc_get_conduit(void)
> +{
> + if (psci_ops.smccc_version < SMCCC_VERSION_1_1)
> + return SMCCC_CONDUIT_NONE;
> +
> + switch (psci_ops.conduit) {
> + case PSCI_CONDUIT_SMC:
> + return SMCCC_CONDUIT_SMC;
> + case PSCI_CONDUIT_HVC:
> + return SMCCC_CONDUIT_HVC;
> + default:
> + return SMCCC_CONDUIT_NONE;
> + }
> +}
> +
> typedef unsigned long (psci_fn)(unsigned long, unsigned long,
> unsigned long, unsigned long);
> static psci_fn *invoke_psci_fn;
> diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h
> index a031897fca76..193e9d8a1ac2 100644
> --- a/include/linux/arm-smccc.h
> +++ b/include/linux/arm-smccc.h
> @@ -84,6 +84,22 @@
>
> #include <linux/linkage.h>
> #include <linux/types.h>
> +
> +enum arm_smccc_conduit {
> + SMCCC_CONDUIT_NONE,
> + SMCCC_CONDUIT_SMC,
> + SMCCC_CONDUIT_HVC,
> +};
> +
> +/**
> + * arm_smccc_get_conduit()
> + *
> + * Returns the conduit to be used for SMCCCv1.1 or later.
> + *
> + * When SMCCCv1.1 is not present, returns SMCCC_CONDUIT_NONE.
> + */
> +enum arm_smccc_conduit arm_smccc_get_conduit(void);
Given that implicit condition, can we save some confusion by naming this
arm_smccc_1_1_conduit(), in line with the actual SMCCCv1.1 calls?
Robin.
> +
> /**
> * struct arm_smccc_res - Result from SMC/HVC call
> * @a0-a3 result values from registers 0 to 3
>
^ permalink raw reply
* [PATCH 2/4] phy: rockchip-typec: support variable phy config value
From: kbuild test robot @ 2018-05-04 17:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1525421338-1021-2-git-send-email-hl@rock-chips.com>
Hi Lin,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on rockchip/for-next]
[also build test WARNING on v4.17-rc3 next-20180504]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Lin-Huang/drm-rockchip-add-transfer-function-for-cdn-dp/20180504-235745
base: https://git.kernel.org/pub/scm/linux/kernel/git/mmind/linux-rockchip.git for-next
config: arm64-defconfig (attached as .config)
compiler: aarch64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=arm64
All warnings (new ones prefixed by >>):
drivers/phy/rockchip/phy-rockchip-typec.c: In function 'rockchip_typec_phy_probe':
>> drivers/phy/rockchip/phy-rockchip-typec.c:1227:21: warning: assignment discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers]
tcphy->port_cfgs = &phy_cfgs[index];
^
drivers/phy/rockchip/phy-rockchip-typec.c: In function 'tcphy_cfg_dp_pll.isra.2':
>> drivers/phy/rockchip/phy-rockchip-typec.c:512:6: warning: 'clk_ctrl' may be used uninitialized in this function [-Wmaybe-uninitialized]
u32 clk_ctrl;
^~~~~~~~
>> drivers/phy/rockchip/phy-rockchip-typec.c:511:18: warning: 'phy_cfg' may be used uninitialized in this function [-Wmaybe-uninitialized]
struct phy_reg *phy_cfg;
^~~~~~~
vim +/const +1227 drivers/phy/rockchip/phy-rockchip-typec.c
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1194
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1195 static int rockchip_typec_phy_probe(struct platform_device *pdev)
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1196 {
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1197 struct device *dev = &pdev->dev;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1198 struct device_node *np = dev->of_node;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1199 struct device_node *child_np;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1200 struct rockchip_typec_phy *tcphy;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1201 struct phy_provider *phy_provider;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1202 struct resource *res;
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1203 const struct rockchip_usb3phy_port_cfg *phy_cfgs;
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1204 const struct of_device_id *match;
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1205 int index, ret;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1206
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1207 tcphy = devm_kzalloc(dev, sizeof(*tcphy), GFP_KERNEL);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1208 if (!tcphy)
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1209 return -ENOMEM;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1210
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1211 match = of_match_device(dev->driver->of_match_table, dev);
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1212 if (!match || !match->data) {
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1213 dev_err(dev, "phy configs are not assigned!\n");
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1214 return -EINVAL;
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1215 }
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1216
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1217 res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1218 tcphy->base = devm_ioremap_resource(dev, res);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1219 if (IS_ERR(tcphy->base))
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1220 return PTR_ERR(tcphy->base);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1221
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1222 phy_cfgs = match->data;
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1223 /* find out a proper config which can be matched with dt. */
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1224 index = 0;
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1225 while (phy_cfgs[index].reg) {
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1226 if (phy_cfgs[index].reg == res->start) {
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 @1227 tcphy->port_cfgs = &phy_cfgs[index];
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1228 break;
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1229 }
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1230
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1231 ++index;
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1232 }
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1233
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1234 if (!tcphy->port_cfgs) {
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1235 dev_err(dev, "no phy-config can be matched with %s node\n",
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1236 np->name);
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1237 return -EINVAL;
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1238 }
0fbc47d9 drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-02-16 1239
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1240 ret = tcphy_parse_dt(tcphy, dev);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1241 if (ret)
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1242 return ret;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1243
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1244 tcphy->dev = dev;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1245 platform_set_drvdata(pdev, tcphy);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1246 mutex_init(&tcphy->lock);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1247
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1248 typec_phy_pre_init(tcphy);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1249
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1250 tcphy->extcon = extcon_get_edev_by_phandle(dev, 0);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1251 if (IS_ERR(tcphy->extcon)) {
ec1fcd7b drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-03-01 1252 if (PTR_ERR(tcphy->extcon) == -ENODEV) {
ec1fcd7b drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-03-01 1253 tcphy->extcon = NULL;
ec1fcd7b drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-03-01 1254 } else {
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1255 if (PTR_ERR(tcphy->extcon) != -EPROBE_DEFER)
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1256 dev_err(dev, "Invalid or missing extcon\n");
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1257 return PTR_ERR(tcphy->extcon);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1258 }
ec1fcd7b drivers/phy/rockchip/phy-rockchip-typec.c Enric Balletbo i Serra 2018-03-01 1259 }
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1260
cdb051be drivers/phy/rockchip/phy-rockchip-typec.c Lin Huang 2018-05-04 1261 tcphy->typec_phy_config = type_c_dp_phy_config;
2a4d5962 drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-09-07 1262 pm_runtime_enable(dev);
2a4d5962 drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-09-07 1263
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1264 for_each_available_child_of_node(np, child_np) {
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1265 struct phy *phy;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1266
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1267 if (!of_node_cmp(child_np->name, "dp-port"))
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1268 phy = devm_phy_create(dev, child_np,
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1269 &rockchip_dp_phy_ops);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1270 else if (!of_node_cmp(child_np->name, "usb3-port"))
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1271 phy = devm_phy_create(dev, child_np,
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1272 &rockchip_usb3_phy_ops);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1273 else
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1274 continue;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1275
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1276 if (IS_ERR(phy)) {
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1277 dev_err(dev, "failed to create phy: %s\n",
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1278 child_np->name);
3cb0ab6e drivers/phy/rockchip/phy-rockchip-typec.c Chris Zhong 2016-09-08 1279 pm_runtime_disable(dev);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1280 return PTR_ERR(phy);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1281 }
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1282
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1283 phy_set_drvdata(phy, tcphy);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1284 }
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1285
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1286 phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1287 if (IS_ERR(phy_provider)) {
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1288 dev_err(dev, "Failed to register phy provider\n");
3cb0ab6e drivers/phy/rockchip/phy-rockchip-typec.c Chris Zhong 2016-09-08 1289 pm_runtime_disable(dev);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1290 return PTR_ERR(phy_provider);
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1291 }
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1292
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1293 return 0;
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1294 }
e96be45c drivers/phy/phy-rockchip-typec.c Chris Zhong 2016-08-23 1295
:::::: The code at line 1227 was first introduced by commit
:::::: 0fbc47d9e426a934668dbfeb0db26da6f0b7f35c phy: rockchip-typec: deprecate some DT properties for various register fields.
:::::: TO: Enric Balletbo i Serra <enric.balletbo@collabora.com>
:::::: CC: Kishon Vijay Abraham I <kishon@ti.com>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: .config.gz
Type: application/gzip
Size: 38203 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180505/e57a34fa/attachment-0001.gz>
^ permalink raw reply
* [PATCH v7 02/24] soc: qcom: Add APR bus driver
From: Bjorn Andersson @ 2018-05-04 17:45 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180501120820.11016-3-srinivas.kandagatla@linaro.org>
On Tue 01 May 05:07 PDT 2018, Srinivas Kandagatla wrote:
> This patch adds support to APR bus (Asynchronous Packet Router) driver.
> APR driver is made as a bus driver so that the apr devices can added removed
> more dynamically depending on the state of the services on the dsp.
> APR is used for communication between application processor and QDSP to
> use services on QDSP like Audio and others.
>
> Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
> Reviewed-and-tested-by: Rohit kumar <rohitkr@codeaurora.org>
> Acked-by: Andy Gross <andy.gross@linaro.org>
Reviewed-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Regards,
Bjorn
> ---
> drivers/soc/qcom/Kconfig | 9 +
> drivers/soc/qcom/Makefile | 1 +
> drivers/soc/qcom/apr.c | 378 ++++++++++++++++++++++++++++++++++++++++
> include/linux/mod_devicetable.h | 11 ++
> include/linux/soc/qcom/apr.h | 128 ++++++++++++++
> 5 files changed, 527 insertions(+)
> create mode 100644 drivers/soc/qcom/apr.c
> create mode 100644 include/linux/soc/qcom/apr.h
>
> diff --git a/drivers/soc/qcom/Kconfig b/drivers/soc/qcom/Kconfig
> index 5c4535b545cc..d053f2634c67 100644
> --- a/drivers/soc/qcom/Kconfig
> +++ b/drivers/soc/qcom/Kconfig
> @@ -108,4 +108,13 @@ config QCOM_WCNSS_CTRL
> Client driver for the WCNSS_CTRL SMD channel, used to download nv
> firmware to a newly booted WCNSS chip.
>
> +config QCOM_APR
> + tristate "Qualcomm APR Bus (Asynchronous Packet Router)"
> + depends on ARCH_QCOM
> + depends on RPMSG
> + help
> + Enable APR IPC protocol support between
> + application processor and QDSP6. APR is
> + used by audio driver to configure QDSP6
> + ASM, ADM and AFE modules.
> endmenu
> diff --git a/drivers/soc/qcom/Makefile b/drivers/soc/qcom/Makefile
> index dcebf2814e6d..39de5dee55d9 100644
> --- a/drivers/soc/qcom/Makefile
> +++ b/drivers/soc/qcom/Makefile
> @@ -12,3 +12,4 @@ obj-$(CONFIG_QCOM_SMEM_STATE) += smem_state.o
> obj-$(CONFIG_QCOM_SMP2P) += smp2p.o
> obj-$(CONFIG_QCOM_SMSM) += smsm.o
> obj-$(CONFIG_QCOM_WCNSS_CTRL) += wcnss_ctrl.o
> +obj-$(CONFIG_QCOM_APR) += apr.o
> diff --git a/drivers/soc/qcom/apr.c b/drivers/soc/qcom/apr.c
> new file mode 100644
> index 000000000000..97f3622da535
> --- /dev/null
> +++ b/drivers/soc/qcom/apr.c
> @@ -0,0 +1,378 @@
> +// SPDX-License-Identifier: GPL-2.0
> +// Copyright (c) 2011-2017, The Linux Foundation. All rights reserved.
> +// Copyright (c) 2018, Linaro Limited
> +
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/device.h>
> +#include <linux/spinlock.h>
> +#include <linux/idr.h>
> +#include <linux/slab.h>
> +#include <linux/of_device.h>
> +#include <linux/soc/qcom/apr.h>
> +#include <linux/rpmsg.h>
> +#include <linux/of.h>
> +
> +struct apr {
> + struct rpmsg_endpoint *ch;
> + struct device *dev;
> + spinlock_t svcs_lock;
> + struct idr svcs_idr;
> + int dest_domain_id;
> +};
> +
> +/**
> + * apr_send_pkt() - Send a apr message from apr device
> + *
> + * @adev: Pointer to previously registered apr device.
> + * @pkt: Pointer to apr packet to send
> + *
> + * Return: Will be an negative on packet size on success.
> + */
> +int apr_send_pkt(struct apr_device *adev, struct apr_pkt *pkt)
> +{
> + struct apr *apr = dev_get_drvdata(adev->dev.parent);
> + struct apr_hdr *hdr;
> + unsigned long flags;
> + int ret;
> +
> + spin_lock_irqsave(&adev->lock, flags);
> +
> + hdr = &pkt->hdr;
> + hdr->src_domain = APR_DOMAIN_APPS;
> + hdr->src_svc = adev->svc_id;
> + hdr->dest_domain = adev->domain_id;
> + hdr->dest_svc = adev->svc_id;
> +
> + ret = rpmsg_trysend(apr->ch, pkt, hdr->pkt_size);
> + spin_unlock_irqrestore(&adev->lock, flags);
> +
> + return ret ? ret : hdr->pkt_size;
> +}
> +EXPORT_SYMBOL_GPL(apr_send_pkt);
> +
> +static void apr_dev_release(struct device *dev)
> +{
> + struct apr_device *adev = to_apr_device(dev);
> +
> + kfree(adev);
> +}
> +
> +static int apr_callback(struct rpmsg_device *rpdev, void *buf,
> + int len, void *priv, u32 addr)
> +{
> + struct apr *apr = dev_get_drvdata(&rpdev->dev);
> + uint16_t hdr_size, msg_type, ver, svc_id;
> + struct apr_device *svc = NULL;
> + struct apr_driver *adrv = NULL;
> + struct apr_resp_pkt resp;
> + struct apr_hdr *hdr;
> + unsigned long flags;
> +
> + if (len <= APR_HDR_SIZE) {
> + dev_err(apr->dev, "APR: Improper apr pkt received:%p %d\n",
> + buf, len);
> + return -EINVAL;
> + }
> +
> + hdr = buf;
> + ver = APR_HDR_FIELD_VER(hdr->hdr_field);
> + if (ver > APR_PKT_VER + 1)
> + return -EINVAL;
> +
> + hdr_size = APR_HDR_FIELD_SIZE_BYTES(hdr->hdr_field);
> + if (hdr_size < APR_HDR_SIZE) {
> + dev_err(apr->dev, "APR: Wrong hdr size:%d\n", hdr_size);
> + return -EINVAL;
> + }
> +
> + if (hdr->pkt_size < APR_HDR_SIZE || hdr->pkt_size != len) {
> + dev_err(apr->dev, "APR: Wrong paket size\n");
> + return -EINVAL;
> + }
> +
> + msg_type = APR_HDR_FIELD_MT(hdr->hdr_field);
> + if (msg_type >= APR_MSG_TYPE_MAX && msg_type != APR_BASIC_RSP_RESULT) {
> + dev_err(apr->dev, "APR: Wrong message type: %d\n", msg_type);
> + return -EINVAL;
> + }
> +
> + if (hdr->src_domain >= APR_DOMAIN_MAX ||
> + hdr->dest_domain >= APR_DOMAIN_MAX ||
> + hdr->src_svc >= APR_SVC_MAX ||
> + hdr->dest_svc >= APR_SVC_MAX) {
> + dev_err(apr->dev, "APR: Wrong APR header\n");
> + return -EINVAL;
> + }
> +
> + svc_id = hdr->dest_svc;
> + spin_lock_irqsave(&apr->svcs_lock, flags);
> + svc = idr_find(&apr->svcs_idr, svc_id);
> + if (svc && svc->dev.driver)
> + adrv = to_apr_driver(svc->dev.driver);
> + spin_unlock_irqrestore(&apr->svcs_lock, flags);
> +
> + if (!adrv) {
> + dev_err(apr->dev, "APR: service is not registered\n");
> + return -EINVAL;
> + }
> +
> + resp.hdr = *hdr;
> + resp.payload_size = hdr->pkt_size - hdr_size;
> +
> + /*
> + * NOTE: hdr_size is not same as APR_HDR_SIZE as remote can include
> + * optional headers in to apr_hdr which should be ignored
> + */
> + if (resp.payload_size > 0)
> + resp.payload = buf + hdr_size;
> +
> + adrv->callback(svc, &resp);
> +
> + return 0;
> +}
> +
> +static int apr_device_match(struct device *dev, struct device_driver *drv)
> +{
> + struct apr_device *adev = to_apr_device(dev);
> + struct apr_driver *adrv = to_apr_driver(drv);
> + const struct apr_device_id *id = adrv->id_table;
> +
> + /* Attempt an OF style match first */
> + if (of_driver_match_device(dev, drv))
> + return 1;
> +
> + if (!id)
> + return 0;
> +
> + while (id->domain_id != 0 || id->svc_id != 0) {
> + if (id->domain_id == adev->domain_id &&
> + id->svc_id == adev->svc_id)
> + return 1;
> + id++;
> + }
> +
> + return 0;
> +}
> +
> +static int apr_device_probe(struct device *dev)
> +{
> + struct apr_device *adev = to_apr_device(dev);
> + struct apr_driver *adrv = to_apr_driver(dev->driver);
> +
> + return adrv->probe(adev);
> +}
> +
> +static int apr_device_remove(struct device *dev)
> +{
> + struct apr_device *adev = to_apr_device(dev);
> + struct apr_driver *adrv;
> + struct apr *apr = dev_get_drvdata(adev->dev.parent);
> +
> + if (dev->driver) {
> + adrv = to_apr_driver(dev->driver);
> + if (adrv->remove)
> + adrv->remove(adev);
> + spin_lock(&apr->svcs_lock);
> + idr_remove(&apr->svcs_idr, adev->svc_id);
> + spin_unlock(&apr->svcs_lock);
> + }
> +
> + return 0;
> +}
> +
> +static int apr_uevent(struct device *dev, struct kobj_uevent_env *env)
> +{
> + struct apr_device *adev = to_apr_device(dev);
> + int ret;
> +
> + ret = of_device_uevent_modalias(dev, env);
> + if (ret != -ENODEV)
> + return ret;
> +
> + return add_uevent_var(env, "MODALIAS=apr:%s", adev->name);
> +}
> +
> +struct bus_type aprbus = {
> + .name = "aprbus",
> + .match = apr_device_match,
> + .probe = apr_device_probe,
> + .uevent = apr_uevent,
> + .remove = apr_device_remove,
> +};
> +EXPORT_SYMBOL_GPL(aprbus);
> +
> +static int apr_add_device(struct device *dev, struct device_node *np,
> + const struct apr_device_id *id)
> +{
> + struct apr *apr = dev_get_drvdata(dev);
> + struct apr_device *adev = NULL;
> + int ret;
> +
> + adev = kzalloc(sizeof(*adev), GFP_KERNEL);
> + if (!adev)
> + return -ENOMEM;
> +
> + spin_lock_init(&adev->lock);
> +
> + adev->svc_id = id->svc_id;
> + adev->domain_id = id->domain_id;
> + adev->version = id->svc_version;
> + if (np)
> + strncpy(adev->name, np->name, APR_NAME_SIZE);
> + else
> + strncpy(adev->name, id->name, APR_NAME_SIZE);
> +
> + dev_set_name(&adev->dev, "aprsvc:%s:%x:%x", adev->name,
> + id->domain_id, id->svc_id);
> +
> + adev->dev.bus = &aprbus;
> + adev->dev.parent = dev;
> + adev->dev.of_node = np;
> + adev->dev.release = apr_dev_release;
> + adev->dev.driver = NULL;
> +
> + spin_lock(&apr->svcs_lock);
> + idr_alloc(&apr->svcs_idr, adev, id->svc_id,
> + id->svc_id + 1, GFP_ATOMIC);
> + spin_unlock(&apr->svcs_lock);
> +
> + dev_info(dev, "Adding APR dev: %s\n", dev_name(&adev->dev));
> +
> + ret = device_register(&adev->dev);
> + if (ret) {
> + dev_err(dev, "device_register failed: %d\n", ret);
> + put_device(&adev->dev);
> + }
> +
> + return ret;
> +}
> +
> +static void of_register_apr_devices(struct device *dev)
> +{
> + struct apr *apr = dev_get_drvdata(dev);
> + struct device_node *node;
> +
> + for_each_child_of_node(dev->of_node, node) {
> + struct apr_device_id id = { {0} };
> +
> + if (of_property_read_u32(node, "reg", &id.svc_id))
> + continue;
> +
> + id.domain_id = apr->dest_domain_id;
> +
> + if (apr_add_device(dev, node, &id))
> + dev_err(dev, "Failed to add apr %d svc\n", id.svc_id);
> + }
> +}
> +
> +static int apr_probe(struct rpmsg_device *rpdev)
> +{
> + struct device *dev = &rpdev->dev;
> + struct apr *apr;
> + int ret;
> +
> + apr = devm_kzalloc(dev, sizeof(*apr), GFP_KERNEL);
> + if (!apr)
> + return -ENOMEM;
> +
> + ret = of_property_read_u32(dev->of_node, "reg", &apr->dest_domain_id);
> + if (ret) {
> + dev_err(dev, "APR Domain ID not specified in DT\n");
> + return ret;
> + }
> +
> + dev_set_drvdata(dev, apr);
> + apr->ch = rpdev->ept;
> + apr->dev = dev;
> + spin_lock_init(&apr->svcs_lock);
> + idr_init(&apr->svcs_idr);
> + of_register_apr_devices(dev);
> +
> + return 0;
> +}
> +
> +static int apr_remove_device(struct device *dev, void *null)
> +{
> + struct apr_device *adev = to_apr_device(dev);
> +
> + device_unregister(&adev->dev);
> +
> + return 0;
> +}
> +
> +static void apr_remove(struct rpmsg_device *rpdev)
> +{
> + device_for_each_child(&rpdev->dev, NULL, apr_remove_device);
> +}
> +
> +/*
> + * __apr_driver_register() - Client driver registration with aprbus
> + *
> + * @drv:Client driver to be associated with client-device.
> + * @owner: owning module/driver
> + *
> + * This API will register the client driver with the aprbus
> + * It is called from the driver's module-init function.
> + */
> +int __apr_driver_register(struct apr_driver *drv, struct module *owner)
> +{
> + drv->driver.bus = &aprbus;
> + drv->driver.owner = owner;
> +
> + return driver_register(&drv->driver);
> +}
> +EXPORT_SYMBOL_GPL(__apr_driver_register);
> +
> +/*
> + * apr_driver_unregister() - Undo effect of apr_driver_register
> + *
> + * @drv: Client driver to be unregistered
> + */
> +void apr_driver_unregister(struct apr_driver *drv)
> +{
> + driver_unregister(&drv->driver);
> +}
> +EXPORT_SYMBOL_GPL(apr_driver_unregister);
> +
> +static const struct of_device_id apr_of_match[] = {
> + { .compatible = "qcom,apr"},
> + { .compatible = "qcom,apr-v2"},
> + {}
> +};
> +MODULE_DEVICE_TABLE(of, apr_of_match);
> +
> +static struct rpmsg_driver apr_driver = {
> + .probe = apr_probe,
> + .remove = apr_remove,
> + .callback = apr_callback,
> + .drv = {
> + .name = "qcom,apr",
> + .of_match_table = apr_of_match,
> + },
> +};
> +
> +static int __init apr_init(void)
> +{
> + int ret;
> +
> + ret = bus_register(&aprbus);
> + if (!ret)
> + ret = register_rpmsg_driver(&apr_driver);
> + else
> + bus_unregister(&aprbus);
> +
> + return ret;
> +}
> +
> +static void __exit apr_exit(void)
> +{
> + bus_unregister(&aprbus);
> + unregister_rpmsg_driver(&apr_driver);
> +}
> +
> +subsys_initcall(apr_init);
> +module_exit(apr_exit);
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Qualcomm APR Bus");
> diff --git a/include/linux/mod_devicetable.h b/include/linux/mod_devicetable.h
> index 7d361be2e24f..2014bd19f28e 100644
> --- a/include/linux/mod_devicetable.h
> +++ b/include/linux/mod_devicetable.h
> @@ -471,6 +471,17 @@ struct slim_device_id {
> kernel_ulong_t driver_data;
> };
>
> +#define APR_NAME_SIZE 32
> +#define APR_MODULE_PREFIX "apr:"
> +
> +struct apr_device_id {
> + char name[APR_NAME_SIZE];
> + __u32 domain_id;
> + __u32 svc_id;
> + __u32 svc_version;
> + kernel_ulong_t driver_data; /* Data private to the driver */
> +};
> +
> #define SPMI_NAME_SIZE 32
> #define SPMI_MODULE_PREFIX "spmi:"
>
> diff --git a/include/linux/soc/qcom/apr.h b/include/linux/soc/qcom/apr.h
> new file mode 100644
> index 000000000000..c5d52e2cb275
> --- /dev/null
> +++ b/include/linux/soc/qcom/apr.h
> @@ -0,0 +1,128 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#ifndef __QCOM_APR_H_
> +#define __QCOM_APR_H_
> +
> +#include <linux/spinlock.h>
> +#include <linux/device.h>
> +#include <linux/mod_devicetable.h>
> +#include <dt-bindings/soc/qcom,apr.h>
> +
> +extern struct bus_type aprbus;
> +
> +#define APR_HDR_LEN(hdr_len) ((hdr_len)/4)
> +
> +/*
> + * HEADER field
> + * version:0:3
> + * header_size : 4:7
> + * message_type : 8:9
> + * reserved: 10:15
> + */
> +#define APR_HDR_FIELD(msg_type, hdr_len, ver)\
> + (((msg_type & 0x3) << 8) | ((hdr_len & 0xF) << 4) | (ver & 0xF))
> +
> +#define APR_HDR_SIZE sizeof(struct apr_hdr)
> +#define APR_SEQ_CMD_HDR_FIELD APR_HDR_FIELD(APR_MSG_TYPE_SEQ_CMD, \
> + APR_HDR_LEN(APR_HDR_SIZE), \
> + APR_PKT_VER)
> +/* Version */
> +#define APR_PKT_VER 0x0
> +
> +/* Command and Response Types */
> +#define APR_MSG_TYPE_EVENT 0x0
> +#define APR_MSG_TYPE_CMD_RSP 0x1
> +#define APR_MSG_TYPE_SEQ_CMD 0x2
> +#define APR_MSG_TYPE_NSEQ_CMD 0x3
> +#define APR_MSG_TYPE_MAX 0x04
> +
> +/* APR Basic Response Message */
> +#define APR_BASIC_RSP_RESULT 0x000110E8
> +#define APR_RSP_ACCEPTED 0x000100BE
> +
> +struct aprv2_ibasic_rsp_result_t {
> + uint32_t opcode;
> + uint32_t status;
> +};
> +
> +/* hdr field Ver [0:3], Size [4:7], Message type [8:10] */
> +#define APR_HDR_FIELD_VER(h) (h & 0x000F)
> +#define APR_HDR_FIELD_SIZE(h) ((h & 0x00F0) >> 4)
> +#define APR_HDR_FIELD_SIZE_BYTES(h) (((h & 0x00F0) >> 4) * 4)
> +#define APR_HDR_FIELD_MT(h) ((h & 0x0300) >> 8)
> +
> +struct apr_hdr {
> + uint16_t hdr_field;
> + uint16_t pkt_size;
> + uint8_t src_svc;
> + uint8_t src_domain;
> + uint16_t src_port;
> + uint8_t dest_svc;
> + uint8_t dest_domain;
> + uint16_t dest_port;
> + uint32_t token;
> + uint32_t opcode;
> +} __packed;
> +
> +struct apr_pkt {
> + struct apr_hdr hdr;
> + uint8_t payload[];
> +};
> +
> +struct apr_resp_pkt {
> + struct apr_hdr hdr;
> + void *payload;
> + int payload_size;
> +};
> +
> +/* Bits 0 to 15 -- Minor version, Bits 16 to 31 -- Major version */
> +#define APR_SVC_MAJOR_VERSION(v) ((v >> 16) & 0xFF)
> +#define APR_SVC_MINOR_VERSION(v) (v & 0xFF)
> +
> +struct apr_device {
> + struct device dev;
> + uint16_t svc_id;
> + uint16_t domain_id;
> + uint32_t version;
> + char name[APR_NAME_SIZE];
> + spinlock_t lock;
> + struct list_head node;
> +};
> +
> +#define to_apr_device(d) container_of(d, struct apr_device, dev)
> +
> +struct apr_driver {
> + int (*probe)(struct apr_device *sl);
> + int (*remove)(struct apr_device *sl);
> + int (*callback)(struct apr_device *a,
> + struct apr_resp_pkt *d);
> + struct device_driver driver;
> + const struct apr_device_id *id_table;
> +};
> +
> +#define to_apr_driver(d) container_of(d, struct apr_driver, driver)
> +
> +/*
> + * use a macro to avoid include chaining to get THIS_MODULE
> + */
> +#define apr_driver_register(drv) __apr_driver_register(drv, THIS_MODULE)
> +
> +int __apr_driver_register(struct apr_driver *drv, struct module *owner);
> +void apr_driver_unregister(struct apr_driver *drv);
> +
> +/**
> + * module_apr_driver() - Helper macro for registering a aprbus driver
> + * @__aprbus_driver: aprbus_driver struct
> + *
> + * Helper macro for aprbus drivers which do not do anything special in
> + * module init/exit. This eliminates a lot of boilerplate. Each module
> + * may only use this macro once, and calling it replaces module_init()
> + * and module_exit()
> + */
> +#define module_apr_driver(__apr_driver) \
> + module_driver(__apr_driver, apr_driver_register, \
> + apr_driver_unregister)
> +
> +int apr_send_pkt(struct apr_device *adev, struct apr_pkt *pkt);
> +
> +#endif /* __QCOM_APR_H_ */
> --
> 2.16.2
>
^ permalink raw reply
* [PATCH 6/6] arm64: instrument smp_{load_acquire,store_release}
From: Mark Rutland @ 2018-05-04 17:39 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180504173937.25300-1-mark.rutland@arm.com>
Our __smp_store_release() and __smp_load_acquire() macros use inline
assembly, which is opaque to kasan. This means that kasan can't catch
erroneous use of these.
This patch adds kasan instrumentation to both.
It might be better to turn these into __arch_* variants, as we do for
the atomics, but this works for the time being.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/barrier.h | 22 ++++++++++++++--------
1 file changed, 14 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h
index f11518af96a9..1a9c601619e5 100644
--- a/arch/arm64/include/asm/barrier.h
+++ b/arch/arm64/include/asm/barrier.h
@@ -20,6 +20,8 @@
#ifndef __ASSEMBLY__
+#include <linux/kasan-checks.h>
+
#define __nops(n) ".rept " #n "\nnop\n.endr\n"
#define nops(n) asm volatile(__nops(n))
@@ -68,31 +70,33 @@ static inline unsigned long array_index_mask_nospec(unsigned long idx,
#define __smp_store_release(p, v) \
do { \
+ typeof(p) __p = (p); \
union { typeof(*p) __val; char __c[1]; } __u = \
{ .__val = (__force typeof(*p)) (v) }; \
compiletime_assert_atomic_type(*p); \
+ kasan_check_write(__p, sizeof(*__p)); \
switch (sizeof(*p)) { \
case 1: \
asm volatile ("stlrb %w1, %0" \
- : "=Q" (*p) \
+ : "=Q" (*__p) \
: "r" (*(__u8 *)__u.__c) \
: "memory"); \
break; \
case 2: \
asm volatile ("stlrh %w1, %0" \
- : "=Q" (*p) \
+ : "=Q" (*__p) \
: "r" (*(__u16 *)__u.__c) \
: "memory"); \
break; \
case 4: \
asm volatile ("stlr %w1, %0" \
- : "=Q" (*p) \
+ : "=Q" (*__p) \
: "r" (*(__u32 *)__u.__c) \
: "memory"); \
break; \
case 8: \
asm volatile ("stlr %1, %0" \
- : "=Q" (*p) \
+ : "=Q" (*__p) \
: "r" (*(__u64 *)__u.__c) \
: "memory"); \
break; \
@@ -102,27 +106,29 @@ do { \
#define __smp_load_acquire(p) \
({ \
union { typeof(*p) __val; char __c[1]; } __u; \
+ typeof(p) __p = (p); \
compiletime_assert_atomic_type(*p); \
+ kasan_check_read(__p, sizeof(*__p)); \
switch (sizeof(*p)) { \
case 1: \
asm volatile ("ldarb %w0, %1" \
: "=r" (*(__u8 *)__u.__c) \
- : "Q" (*p) : "memory"); \
+ : "Q" (*__p) : "memory"); \
break; \
case 2: \
asm volatile ("ldarh %w0, %1" \
: "=r" (*(__u16 *)__u.__c) \
- : "Q" (*p) : "memory"); \
+ : "Q" (*__p) : "memory"); \
break; \
case 4: \
asm volatile ("ldar %w0, %1" \
: "=r" (*(__u32 *)__u.__c) \
- : "Q" (*p) : "memory"); \
+ : "Q" (*__p) : "memory"); \
break; \
case 8: \
asm volatile ("ldar %0, %1" \
: "=r" (*(__u64 *)__u.__c) \
- : "Q" (*p) : "memory"); \
+ : "Q" (*__p) : "memory"); \
break; \
} \
__u.__val; \
--
2.11.0
^ permalink raw reply related
* [PATCH 5/6] arm64: use instrumented atomics
From: Mark Rutland @ 2018-05-04 17:39 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180504173937.25300-1-mark.rutland@arm.com>
As our atomics are written in inline assembly, they don't get
instrumented when we enable KASAN, and thus we can miss when they are
used on erroneous memory locations.
As with x86, let's use atomic-instrumented.h to give arm64 instrumented
atomics. This requires that we add an arch_ prefix to our atomic names,
but other than naming, no changes are made to the atomics themselves.
Due to include dependencies, we must move our definition of sync_cmpxchg
into <asm/cmpxchg.h>, but this is not harmful.
There should be no functional change as a result of this patch when
CONFIG_KASAN is not selected.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/atomic.h | 299 +++++++++++++++++-----------------
arch/arm64/include/asm/atomic_ll_sc.h | 28 ++--
arch/arm64/include/asm/atomic_lse.h | 43 ++---
arch/arm64/include/asm/cmpxchg.h | 25 +--
arch/arm64/include/asm/sync_bitops.h | 1 -
5 files changed, 202 insertions(+), 194 deletions(-)
diff --git a/arch/arm64/include/asm/atomic.h b/arch/arm64/include/asm/atomic.h
index c0235e0ff849..aefdce33f81a 100644
--- a/arch/arm64/include/asm/atomic.h
+++ b/arch/arm64/include/asm/atomic.h
@@ -53,158 +53,161 @@
#define ATOMIC_INIT(i) { (i) }
-#define atomic_read(v) READ_ONCE((v)->counter)
-#define atomic_set(v, i) WRITE_ONCE(((v)->counter), (i))
-
-#define atomic_add_return_relaxed atomic_add_return_relaxed
-#define atomic_add_return_acquire atomic_add_return_acquire
-#define atomic_add_return_release atomic_add_return_release
-#define atomic_add_return atomic_add_return
-
-#define atomic_inc_return_relaxed(v) atomic_add_return_relaxed(1, (v))
-#define atomic_inc_return_acquire(v) atomic_add_return_acquire(1, (v))
-#define atomic_inc_return_release(v) atomic_add_return_release(1, (v))
-#define atomic_inc_return(v) atomic_add_return(1, (v))
-
-#define atomic_sub_return_relaxed atomic_sub_return_relaxed
-#define atomic_sub_return_acquire atomic_sub_return_acquire
-#define atomic_sub_return_release atomic_sub_return_release
-#define atomic_sub_return atomic_sub_return
-
-#define atomic_dec_return_relaxed(v) atomic_sub_return_relaxed(1, (v))
-#define atomic_dec_return_acquire(v) atomic_sub_return_acquire(1, (v))
-#define atomic_dec_return_release(v) atomic_sub_return_release(1, (v))
-#define atomic_dec_return(v) atomic_sub_return(1, (v))
-
-#define atomic_fetch_add_relaxed atomic_fetch_add_relaxed
-#define atomic_fetch_add_acquire atomic_fetch_add_acquire
-#define atomic_fetch_add_release atomic_fetch_add_release
-#define atomic_fetch_add atomic_fetch_add
-
-#define atomic_fetch_sub_relaxed atomic_fetch_sub_relaxed
-#define atomic_fetch_sub_acquire atomic_fetch_sub_acquire
-#define atomic_fetch_sub_release atomic_fetch_sub_release
-#define atomic_fetch_sub atomic_fetch_sub
-
-#define atomic_fetch_and_relaxed atomic_fetch_and_relaxed
-#define atomic_fetch_and_acquire atomic_fetch_and_acquire
-#define atomic_fetch_and_release atomic_fetch_and_release
-#define atomic_fetch_and atomic_fetch_and
-
-#define atomic_fetch_andnot_relaxed atomic_fetch_andnot_relaxed
-#define atomic_fetch_andnot_acquire atomic_fetch_andnot_acquire
-#define atomic_fetch_andnot_release atomic_fetch_andnot_release
-#define atomic_fetch_andnot atomic_fetch_andnot
-
-#define atomic_fetch_or_relaxed atomic_fetch_or_relaxed
-#define atomic_fetch_or_acquire atomic_fetch_or_acquire
-#define atomic_fetch_or_release atomic_fetch_or_release
-#define atomic_fetch_or atomic_fetch_or
-
-#define atomic_fetch_xor_relaxed atomic_fetch_xor_relaxed
-#define atomic_fetch_xor_acquire atomic_fetch_xor_acquire
-#define atomic_fetch_xor_release atomic_fetch_xor_release
-#define atomic_fetch_xor atomic_fetch_xor
-
-#define atomic_xchg_relaxed(v, new) xchg_relaxed(&((v)->counter), (new))
-#define atomic_xchg_acquire(v, new) xchg_acquire(&((v)->counter), (new))
-#define atomic_xchg_release(v, new) xchg_release(&((v)->counter), (new))
-#define atomic_xchg(v, new) xchg(&((v)->counter), (new))
-
-#define atomic_cmpxchg_relaxed(v, old, new) \
- cmpxchg_relaxed(&((v)->counter), (old), (new))
-#define atomic_cmpxchg_acquire(v, old, new) \
- cmpxchg_acquire(&((v)->counter), (old), (new))
-#define atomic_cmpxchg_release(v, old, new) \
- cmpxchg_release(&((v)->counter), (old), (new))
-#define atomic_cmpxchg(v, old, new) cmpxchg(&((v)->counter), (old), (new))
-
-#define atomic_inc(v) atomic_add(1, (v))
-#define atomic_dec(v) atomic_sub(1, (v))
-#define atomic_inc_and_test(v) (atomic_inc_return(v) == 0)
-#define atomic_dec_and_test(v) (atomic_dec_return(v) == 0)
-#define atomic_sub_and_test(i, v) (atomic_sub_return((i), (v)) == 0)
-#define atomic_add_negative(i, v) (atomic_add_return((i), (v)) < 0)
-#define __atomic_add_unless(v, a, u) ___atomic_add_unless(v, a, u,)
-#define atomic_andnot atomic_andnot
+#define arch_atomic_read(v) READ_ONCE((v)->counter)
+#define arch_atomic_set(v, i) WRITE_ONCE(((v)->counter), (i))
+
+#define arch_atomic_add_return_relaxed arch_atomic_add_return_relaxed
+#define arch_atomic_add_return_acquire arch_atomic_add_return_acquire
+#define arch_atomic_add_return_release arch_atomic_add_return_release
+#define arch_atomic_add_return arch_atomic_add_return
+
+#define arch_atomic_inc_return_relaxed(v) arch_atomic_add_return_relaxed(1, (v))
+#define arch_atomic_inc_return_acquire(v) arch_atomic_add_return_acquire(1, (v))
+#define arch_atomic_inc_return_release(v) arch_atomic_add_return_release(1, (v))
+#define arch_atomic_inc_return(v) arch_atomic_add_return(1, (v))
+
+#define arch_atomic_sub_return_relaxed arch_atomic_sub_return_relaxed
+#define arch_atomic_sub_return_acquire arch_atomic_sub_return_acquire
+#define arch_atomic_sub_return_release arch_atomic_sub_return_release
+#define arch_atomic_sub_return arch_atomic_sub_return
+
+#define arch_atomic_dec_return_relaxed(v) arch_atomic_sub_return_relaxed(1, (v))
+#define arch_atomic_dec_return_acquire(v) arch_atomic_sub_return_acquire(1, (v))
+#define arch_atomic_dec_return_release(v) arch_atomic_sub_return_release(1, (v))
+#define arch_atomic_dec_return(v) arch_atomic_sub_return(1, (v))
+
+#define arch_atomic_fetch_add_relaxed arch_atomic_fetch_add_relaxed
+#define arch_atomic_fetch_add_acquire arch_atomic_fetch_add_acquire
+#define arch_atomic_fetch_add_release arch_atomic_fetch_add_release
+#define arch_atomic_fetch_add arch_atomic_fetch_add
+
+#define arch_atomic_fetch_sub_relaxed arch_atomic_fetch_sub_relaxed
+#define arch_atomic_fetch_sub_acquire arch_atomic_fetch_sub_acquire
+#define arch_atomic_fetch_sub_release arch_atomic_fetch_sub_release
+#define arch_atomic_fetch_sub arch_atomic_fetch_sub
+
+#define arch_atomic_fetch_and_relaxed arch_atomic_fetch_and_relaxed
+#define arch_atomic_fetch_and_acquire arch_atomic_fetch_and_acquire
+#define arch_atomic_fetch_and_release arch_atomic_fetch_and_release
+#define arch_atomic_fetch_and arch_atomic_fetch_and
+
+#define arch_atomic_fetch_andnot_relaxed arch_atomic_fetch_andnot_relaxed
+#define arch_atomic_fetch_andnot_acquire arch_atomic_fetch_andnot_acquire
+#define arch_atomic_fetch_andnot_release arch_atomic_fetch_andnot_release
+#define arch_atomic_fetch_andnot arch_atomic_fetch_andnot
+
+#define arch_atomic_fetch_or_relaxed arch_atomic_fetch_or_relaxed
+#define arch_atomic_fetch_or_acquire arch_atomic_fetch_or_acquire
+#define arch_atomic_fetch_or_release arch_atomic_fetch_or_release
+#define arch_atomic_fetch_or arch_atomic_fetch_or
+
+#define arch_atomic_fetch_xor_relaxed arch_atomic_fetch_xor_relaxed
+#define arch_atomic_fetch_xor_acquire arch_atomic_fetch_xor_acquire
+#define arch_atomic_fetch_xor_release arch_atomic_fetch_xor_release
+#define arch_atomic_fetch_xor arch_atomic_fetch_xor
+
+#define arch_atomic_xchg_relaxed(v, new) xchg_relaxed(&((v)->counter), (new))
+#define arch_atomic_xchg_acquire(v, new) xchg_acquire(&((v)->counter), (new))
+#define arch_atomic_xchg_release(v, new) xchg_release(&((v)->counter), (new))
+#define arch_atomic_xchg(v, new) xchg(&((v)->counter), (new))
+
+#define arch_atomic_cmpxchg_relaxed(v, old, new) \
+ arch_cmpxchg_relaxed(&((v)->counter), (old), (new))
+#define arch_atomic_cmpxchg_acquire(v, old, new) \
+ arch_cmpxchg_acquire(&((v)->counter), (old), (new))
+#define arch_atomic_cmpxchg_release(v, old, new) \
+ arch_cmpxchg_release(&((v)->counter), (old), (new))
+#define arch_atomic_cmpxchg(v, old, new) \
+ arch_cmpxchg(&((v)->counter), (old), (new))
+
+#define arch_atomic_inc(v) arch_atomic_add(1, (v))
+#define arch_atomic_dec(v) arch_atomic_sub(1, (v))
+#define arch_atomic_inc_and_test(v) (arch_atomic_inc_return(v) == 0)
+#define arch_atomic_dec_and_test(v) (arch_atomic_dec_return(v) == 0)
+#define arch_atomic_sub_and_test(i, v) (arch_atomic_sub_return((i), (v)) == 0)
+#define arch_atomic_add_negative(i, v) (arch_atomic_add_return((i), (v)) < 0)
+#define __arch_atomic_add_unless(v, a, u) ___atomic_add_unless(v, a, u,)
+#define arch_atomic_andnot arch_atomic_andnot
/*
* 64-bit atomic operations.
*/
-#define ATOMIC64_INIT ATOMIC_INIT
-#define atomic64_read atomic_read
-#define atomic64_set atomic_set
-
-#define atomic64_add_return_relaxed atomic64_add_return_relaxed
-#define atomic64_add_return_acquire atomic64_add_return_acquire
-#define atomic64_add_return_release atomic64_add_return_release
-#define atomic64_add_return atomic64_add_return
-
-#define atomic64_inc_return_relaxed(v) atomic64_add_return_relaxed(1, (v))
-#define atomic64_inc_return_acquire(v) atomic64_add_return_acquire(1, (v))
-#define atomic64_inc_return_release(v) atomic64_add_return_release(1, (v))
-#define atomic64_inc_return(v) atomic64_add_return(1, (v))
-
-#define atomic64_sub_return_relaxed atomic64_sub_return_relaxed
-#define atomic64_sub_return_acquire atomic64_sub_return_acquire
-#define atomic64_sub_return_release atomic64_sub_return_release
-#define atomic64_sub_return atomic64_sub_return
-
-#define atomic64_dec_return_relaxed(v) atomic64_sub_return_relaxed(1, (v))
-#define atomic64_dec_return_acquire(v) atomic64_sub_return_acquire(1, (v))
-#define atomic64_dec_return_release(v) atomic64_sub_return_release(1, (v))
-#define atomic64_dec_return(v) atomic64_sub_return(1, (v))
-
-#define atomic64_fetch_add_relaxed atomic64_fetch_add_relaxed
-#define atomic64_fetch_add_acquire atomic64_fetch_add_acquire
-#define atomic64_fetch_add_release atomic64_fetch_add_release
-#define atomic64_fetch_add atomic64_fetch_add
-
-#define atomic64_fetch_sub_relaxed atomic64_fetch_sub_relaxed
-#define atomic64_fetch_sub_acquire atomic64_fetch_sub_acquire
-#define atomic64_fetch_sub_release atomic64_fetch_sub_release
-#define atomic64_fetch_sub atomic64_fetch_sub
-
-#define atomic64_fetch_and_relaxed atomic64_fetch_and_relaxed
-#define atomic64_fetch_and_acquire atomic64_fetch_and_acquire
-#define atomic64_fetch_and_release atomic64_fetch_and_release
-#define atomic64_fetch_and atomic64_fetch_and
-
-#define atomic64_fetch_andnot_relaxed atomic64_fetch_andnot_relaxed
-#define atomic64_fetch_andnot_acquire atomic64_fetch_andnot_acquire
-#define atomic64_fetch_andnot_release atomic64_fetch_andnot_release
-#define atomic64_fetch_andnot atomic64_fetch_andnot
-
-#define atomic64_fetch_or_relaxed atomic64_fetch_or_relaxed
-#define atomic64_fetch_or_acquire atomic64_fetch_or_acquire
-#define atomic64_fetch_or_release atomic64_fetch_or_release
-#define atomic64_fetch_or atomic64_fetch_or
-
-#define atomic64_fetch_xor_relaxed atomic64_fetch_xor_relaxed
-#define atomic64_fetch_xor_acquire atomic64_fetch_xor_acquire
-#define atomic64_fetch_xor_release atomic64_fetch_xor_release
-#define atomic64_fetch_xor atomic64_fetch_xor
-
-#define atomic64_xchg_relaxed atomic_xchg_relaxed
-#define atomic64_xchg_acquire atomic_xchg_acquire
-#define atomic64_xchg_release atomic_xchg_release
-#define atomic64_xchg atomic_xchg
-
-#define atomic64_cmpxchg_relaxed atomic_cmpxchg_relaxed
-#define atomic64_cmpxchg_acquire atomic_cmpxchg_acquire
-#define atomic64_cmpxchg_release atomic_cmpxchg_release
-#define atomic64_cmpxchg atomic_cmpxchg
-
-#define atomic64_inc(v) atomic64_add(1, (v))
-#define atomic64_dec(v) atomic64_sub(1, (v))
-#define atomic64_inc_and_test(v) (atomic64_inc_return(v) == 0)
-#define atomic64_dec_and_test(v) (atomic64_dec_return(v) == 0)
-#define atomic64_sub_and_test(i, v) (atomic64_sub_return((i), (v)) == 0)
-#define atomic64_add_negative(i, v) (atomic64_add_return((i), (v)) < 0)
-#define atomic64_add_unless(v, a, u) (___atomic_add_unless(v, a, u, 64) != u)
-#define atomic64_andnot atomic64_andnot
-
-#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
+#define ATOMIC64_INIT ATOMIC_INIT
+#define arch_atomic64_read arch_atomic_read
+#define arch_atomic64_set arch_atomic_set
+
+#define arch_atomic64_add_return_relaxed arch_atomic64_add_return_relaxed
+#define arch_atomic64_add_return_acquire arch_atomic64_add_return_acquire
+#define arch_atomic64_add_return_release arch_atomic64_add_return_release
+#define arch_atomic64_add_return arch_atomic64_add_return
+
+#define arch_atomic64_inc_return_relaxed(v) arch_atomic64_add_return_relaxed(1, (v))
+#define arch_atomic64_inc_return_acquire(v) arch_atomic64_add_return_acquire(1, (v))
+#define arch_atomic64_inc_return_release(v) arch_atomic64_add_return_release(1, (v))
+#define arch_atomic64_inc_return(v) arch_atomic64_add_return(1, (v))
+
+#define arch_atomic64_sub_return_relaxed arch_atomic64_sub_return_relaxed
+#define arch_atomic64_sub_return_acquire arch_atomic64_sub_return_acquire
+#define arch_atomic64_sub_return_release arch_atomic64_sub_return_release
+#define arch_atomic64_sub_return arch_atomic64_sub_return
+
+#define arch_atomic64_dec_return_relaxed(v) arch_atomic64_sub_return_relaxed(1, (v))
+#define arch_atomic64_dec_return_acquire(v) arch_atomic64_sub_return_acquire(1, (v))
+#define arch_atomic64_dec_return_release(v) arch_atomic64_sub_return_release(1, (v))
+#define arch_atomic64_dec_return(v) arch_atomic64_sub_return(1, (v))
+
+#define arch_atomic64_fetch_add_relaxed arch_atomic64_fetch_add_relaxed
+#define arch_atomic64_fetch_add_acquire arch_atomic64_fetch_add_acquire
+#define arch_atomic64_fetch_add_release arch_atomic64_fetch_add_release
+#define arch_atomic64_fetch_add arch_atomic64_fetch_add
+
+#define arch_atomic64_fetch_sub_relaxed arch_atomic64_fetch_sub_relaxed
+#define arch_atomic64_fetch_sub_acquire arch_atomic64_fetch_sub_acquire
+#define arch_atomic64_fetch_sub_release arch_atomic64_fetch_sub_release
+#define arch_atomic64_fetch_sub arch_atomic64_fetch_sub
+
+#define arch_atomic64_fetch_and_relaxed arch_atomic64_fetch_and_relaxed
+#define arch_atomic64_fetch_and_acquire arch_atomic64_fetch_and_acquire
+#define arch_atomic64_fetch_and_release arch_atomic64_fetch_and_release
+#define arch_atomic64_fetch_and arch_atomic64_fetch_and
+
+#define arch_atomic64_fetch_andnot_relaxed arch_atomic64_fetch_andnot_relaxed
+#define arch_atomic64_fetch_andnot_acquire arch_atomic64_fetch_andnot_acquire
+#define arch_atomic64_fetch_andnot_release arch_atomic64_fetch_andnot_release
+#define arch_atomic64_fetch_andnot arch_atomic64_fetch_andnot
+
+#define arch_atomic64_fetch_or_relaxed arch_atomic64_fetch_or_relaxed
+#define arch_atomic64_fetch_or_acquire arch_atomic64_fetch_or_acquire
+#define arch_atomic64_fetch_or_release arch_atomic64_fetch_or_release
+#define arch_atomic64_fetch_or arch_atomic64_fetch_or
+
+#define arch_atomic64_fetch_xor_relaxed arch_atomic64_fetch_xor_relaxed
+#define arch_atomic64_fetch_xor_acquire arch_atomic64_fetch_xor_acquire
+#define arch_atomic64_fetch_xor_release arch_atomic64_fetch_xor_release
+#define arch_atomic64_fetch_xor arch_atomic64_fetch_xor
+
+#define arch_atomic64_xchg_relaxed arch_atomic_xchg_relaxed
+#define arch_atomic64_xchg_acquire arch_atomic_xchg_acquire
+#define arch_atomic64_xchg_release arch_atomic_xchg_release
+#define arch_atomic64_xchg arch_atomic_xchg
+
+#define arch_atomic64_cmpxchg_relaxed arch_atomic_cmpxchg_relaxed
+#define arch_atomic64_cmpxchg_acquire arch_atomic_cmpxchg_acquire
+#define arch_atomic64_cmpxchg_release arch_atomic_cmpxchg_release
+#define arch_atomic64_cmpxchg arch_atomic_cmpxchg
+
+#define arch_atomic64_inc(v) arch_atomic64_add(1, (v))
+#define arch_atomic64_dec(v) arch_atomic64_sub(1, (v))
+#define arch_atomic64_inc_and_test(v) (arch_atomic64_inc_return(v) == 0)
+#define arch_atomic64_dec_and_test(v) (arch_atomic64_dec_return(v) == 0)
+#define arch_atomic64_sub_and_test(i, v) (arch_atomic64_sub_return((i), (v)) == 0)
+#define arch_atomic64_add_negative(i, v) (arch_atomic64_add_return((i), (v)) < 0)
+#define arch_atomic64_add_unless(v, a, u) (___atomic_add_unless(v, a, u, 64) != u)
+#define arch_atomic64_andnot arch_atomic64_andnot
+
+#define arch_atomic64_inc_not_zero(v) arch_atomic64_add_unless((v), 1, 0)
+
+#include <asm-generic/atomic-instrumented.h>
#endif
#endif
diff --git a/arch/arm64/include/asm/atomic_ll_sc.h b/arch/arm64/include/asm/atomic_ll_sc.h
index 3175f4982682..c28d5a824104 100644
--- a/arch/arm64/include/asm/atomic_ll_sc.h
+++ b/arch/arm64/include/asm/atomic_ll_sc.h
@@ -39,7 +39,7 @@
#define ATOMIC_OP(op, asm_op) \
__LL_SC_INLINE void \
-__LL_SC_PREFIX(atomic_##op(int i, atomic_t *v)) \
+__LL_SC_PREFIX(arch_atomic_##op(int i, atomic_t *v)) \
{ \
unsigned long tmp; \
int result; \
@@ -53,11 +53,11 @@ __LL_SC_PREFIX(atomic_##op(int i, atomic_t *v)) \
: "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \
: "Ir" (i)); \
} \
-__LL_SC_EXPORT(atomic_##op);
+__LL_SC_EXPORT(arch_atomic_##op);
#define ATOMIC_OP_RETURN(name, mb, acq, rel, cl, op, asm_op) \
__LL_SC_INLINE int \
-__LL_SC_PREFIX(atomic_##op##_return##name(int i, atomic_t *v)) \
+__LL_SC_PREFIX(arch_atomic_##op##_return##name(int i, atomic_t *v)) \
{ \
unsigned long tmp; \
int result; \
@@ -75,11 +75,11 @@ __LL_SC_PREFIX(atomic_##op##_return##name(int i, atomic_t *v)) \
\
return result; \
} \
-__LL_SC_EXPORT(atomic_##op##_return##name);
+__LL_SC_EXPORT(arch_atomic_##op##_return##name);
#define ATOMIC_FETCH_OP(name, mb, acq, rel, cl, op, asm_op) \
__LL_SC_INLINE int \
-__LL_SC_PREFIX(atomic_fetch_##op##name(int i, atomic_t *v)) \
+__LL_SC_PREFIX(arch_atomic_fetch_##op##name(int i, atomic_t *v)) \
{ \
unsigned long tmp; \
int val, result; \
@@ -97,7 +97,7 @@ __LL_SC_PREFIX(atomic_fetch_##op##name(int i, atomic_t *v)) \
\
return result; \
} \
-__LL_SC_EXPORT(atomic_fetch_##op##name);
+__LL_SC_EXPORT(arch_atomic_fetch_##op##name);
#define ATOMIC_OPS(...) \
ATOMIC_OP(__VA_ARGS__) \
@@ -133,7 +133,7 @@ ATOMIC_OPS(xor, eor)
#define ATOMIC64_OP(op, asm_op) \
__LL_SC_INLINE void \
-__LL_SC_PREFIX(atomic64_##op(long i, atomic64_t *v)) \
+__LL_SC_PREFIX(arch_atomic64_##op(long i, atomic64_t *v)) \
{ \
long result; \
unsigned long tmp; \
@@ -147,11 +147,11 @@ __LL_SC_PREFIX(atomic64_##op(long i, atomic64_t *v)) \
: "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \
: "Ir" (i)); \
} \
-__LL_SC_EXPORT(atomic64_##op);
+__LL_SC_EXPORT(arch_atomic64_##op);
#define ATOMIC64_OP_RETURN(name, mb, acq, rel, cl, op, asm_op) \
__LL_SC_INLINE long \
-__LL_SC_PREFIX(atomic64_##op##_return##name(long i, atomic64_t *v)) \
+__LL_SC_PREFIX(arch_atomic64_##op##_return##name(long i, atomic64_t *v))\
{ \
long result; \
unsigned long tmp; \
@@ -169,11 +169,11 @@ __LL_SC_PREFIX(atomic64_##op##_return##name(long i, atomic64_t *v)) \
\
return result; \
} \
-__LL_SC_EXPORT(atomic64_##op##_return##name);
+__LL_SC_EXPORT(arch_atomic64_##op##_return##name);
#define ATOMIC64_FETCH_OP(name, mb, acq, rel, cl, op, asm_op) \
__LL_SC_INLINE long \
-__LL_SC_PREFIX(atomic64_fetch_##op##name(long i, atomic64_t *v)) \
+__LL_SC_PREFIX(arch_atomic64_fetch_##op##name(long i, atomic64_t *v)) \
{ \
long result, val; \
unsigned long tmp; \
@@ -191,7 +191,7 @@ __LL_SC_PREFIX(atomic64_fetch_##op##name(long i, atomic64_t *v)) \
\
return result; \
} \
-__LL_SC_EXPORT(atomic64_fetch_##op##name);
+__LL_SC_EXPORT(arch_atomic64_fetch_##op##name);
#define ATOMIC64_OPS(...) \
ATOMIC64_OP(__VA_ARGS__) \
@@ -226,7 +226,7 @@ ATOMIC64_OPS(xor, eor)
#undef ATOMIC64_OP
__LL_SC_INLINE long
-__LL_SC_PREFIX(atomic64_dec_if_positive(atomic64_t *v))
+__LL_SC_PREFIX(arch_atomic64_dec_if_positive(atomic64_t *v))
{
long result;
unsigned long tmp;
@@ -246,7 +246,7 @@ __LL_SC_PREFIX(atomic64_dec_if_positive(atomic64_t *v))
return result;
}
-__LL_SC_EXPORT(atomic64_dec_if_positive);
+__LL_SC_EXPORT(arch_atomic64_dec_if_positive);
#define __CMPXCHG_CASE(w, sz, name, mb, acq, rel, cl) \
__LL_SC_INLINE unsigned long \
diff --git a/arch/arm64/include/asm/atomic_lse.h b/arch/arm64/include/asm/atomic_lse.h
index 9ef0797380cb..9a071f71c521 100644
--- a/arch/arm64/include/asm/atomic_lse.h
+++ b/arch/arm64/include/asm/atomic_lse.h
@@ -25,9 +25,9 @@
#error "please don't include this file directly"
#endif
-#define __LL_SC_ATOMIC(op) __LL_SC_CALL(atomic_##op)
+#define __LL_SC_ATOMIC(op) __LL_SC_CALL(arch_atomic_##op)
#define ATOMIC_OP(op, asm_op) \
-static inline void atomic_##op(int i, atomic_t *v) \
+static inline void arch_atomic_##op(int i, atomic_t *v) \
{ \
register int w0 asm ("w0") = i; \
register atomic_t *x1 asm ("x1") = v; \
@@ -47,7 +47,7 @@ ATOMIC_OP(add, stadd)
#undef ATOMIC_OP
#define ATOMIC_FETCH_OP(name, mb, op, asm_op, cl...) \
-static inline int atomic_fetch_##op##name(int i, atomic_t *v) \
+static inline int arch_atomic_fetch_##op##name(int i, atomic_t *v) \
{ \
register int w0 asm ("w0") = i; \
register atomic_t *x1 asm ("x1") = v; \
@@ -79,7 +79,7 @@ ATOMIC_FETCH_OPS(add, ldadd)
#undef ATOMIC_FETCH_OPS
#define ATOMIC_OP_ADD_RETURN(name, mb, cl...) \
-static inline int atomic_add_return##name(int i, atomic_t *v) \
+static inline int arch_atomic_add_return##name(int i, atomic_t *v) \
{ \
register int w0 asm ("w0") = i; \
register atomic_t *x1 asm ("x1") = v; \
@@ -105,7 +105,7 @@ ATOMIC_OP_ADD_RETURN( , al, "memory")
#undef ATOMIC_OP_ADD_RETURN
-static inline void atomic_and(int i, atomic_t *v)
+static inline void arch_atomic_and(int i, atomic_t *v)
{
register int w0 asm ("w0") = i;
register atomic_t *x1 asm ("x1") = v;
@@ -123,7 +123,7 @@ static inline void atomic_and(int i, atomic_t *v)
}
#define ATOMIC_FETCH_OP_AND(name, mb, cl...) \
-static inline int atomic_fetch_and##name(int i, atomic_t *v) \
+static inline int arch_atomic_fetch_and##name(int i, atomic_t *v) \
{ \
register int w0 asm ("w0") = i; \
register atomic_t *x1 asm ("x1") = v; \
@@ -149,7 +149,7 @@ ATOMIC_FETCH_OP_AND( , al, "memory")
#undef ATOMIC_FETCH_OP_AND
-static inline void atomic_sub(int i, atomic_t *v)
+static inline void arch_atomic_sub(int i, atomic_t *v)
{
register int w0 asm ("w0") = i;
register atomic_t *x1 asm ("x1") = v;
@@ -167,7 +167,7 @@ static inline void atomic_sub(int i, atomic_t *v)
}
#define ATOMIC_OP_SUB_RETURN(name, mb, cl...) \
-static inline int atomic_sub_return##name(int i, atomic_t *v) \
+static inline int arch_atomic_sub_return##name(int i, atomic_t *v) \
{ \
register int w0 asm ("w0") = i; \
register atomic_t *x1 asm ("x1") = v; \
@@ -195,7 +195,7 @@ ATOMIC_OP_SUB_RETURN( , al, "memory")
#undef ATOMIC_OP_SUB_RETURN
#define ATOMIC_FETCH_OP_SUB(name, mb, cl...) \
-static inline int atomic_fetch_sub##name(int i, atomic_t *v) \
+static inline int arch_atomic_fetch_sub##name(int i, atomic_t *v) \
{ \
register int w0 asm ("w0") = i; \
register atomic_t *x1 asm ("x1") = v; \
@@ -222,9 +222,9 @@ ATOMIC_FETCH_OP_SUB( , al, "memory")
#undef ATOMIC_FETCH_OP_SUB
#undef __LL_SC_ATOMIC
-#define __LL_SC_ATOMIC64(op) __LL_SC_CALL(atomic64_##op)
+#define __LL_SC_ATOMIC64(op) __LL_SC_CALL(arch_atomic64_##op)
#define ATOMIC64_OP(op, asm_op) \
-static inline void atomic64_##op(long i, atomic64_t *v) \
+static inline void arch_atomic64_##op(long i, atomic64_t *v) \
{ \
register long x0 asm ("x0") = i; \
register atomic64_t *x1 asm ("x1") = v; \
@@ -244,7 +244,8 @@ ATOMIC64_OP(add, stadd)
#undef ATOMIC64_OP
#define ATOMIC64_FETCH_OP(name, mb, op, asm_op, cl...) \
-static inline long atomic64_fetch_##op##name(long i, atomic64_t *v) \
+static inline long \
+arch_atomic64_fetch_##op##name(long i, atomic64_t *v) \
{ \
register long x0 asm ("x0") = i; \
register atomic64_t *x1 asm ("x1") = v; \
@@ -276,7 +277,8 @@ ATOMIC64_FETCH_OPS(add, ldadd)
#undef ATOMIC64_FETCH_OPS
#define ATOMIC64_OP_ADD_RETURN(name, mb, cl...) \
-static inline long atomic64_add_return##name(long i, atomic64_t *v) \
+static inline long \
+arch_atomic64_add_return##name(long i, atomic64_t *v) \
{ \
register long x0 asm ("x0") = i; \
register atomic64_t *x1 asm ("x1") = v; \
@@ -302,7 +304,7 @@ ATOMIC64_OP_ADD_RETURN( , al, "memory")
#undef ATOMIC64_OP_ADD_RETURN
-static inline void atomic64_and(long i, atomic64_t *v)
+static inline void arch_atomic64_and(long i, atomic64_t *v)
{
register long x0 asm ("x0") = i;
register atomic64_t *x1 asm ("x1") = v;
@@ -320,7 +322,8 @@ static inline void atomic64_and(long i, atomic64_t *v)
}
#define ATOMIC64_FETCH_OP_AND(name, mb, cl...) \
-static inline long atomic64_fetch_and##name(long i, atomic64_t *v) \
+static inline long \
+arch_atomic64_fetch_and##name(long i, atomic64_t *v) \
{ \
register long x0 asm ("x0") = i; \
register atomic64_t *x1 asm ("x1") = v; \
@@ -346,7 +349,7 @@ ATOMIC64_FETCH_OP_AND( , al, "memory")
#undef ATOMIC64_FETCH_OP_AND
-static inline void atomic64_sub(long i, atomic64_t *v)
+static inline void arch_atomic64_sub(long i, atomic64_t *v)
{
register long x0 asm ("x0") = i;
register atomic64_t *x1 asm ("x1") = v;
@@ -364,7 +367,8 @@ static inline void atomic64_sub(long i, atomic64_t *v)
}
#define ATOMIC64_OP_SUB_RETURN(name, mb, cl...) \
-static inline long atomic64_sub_return##name(long i, atomic64_t *v) \
+static inline long \
+arch_atomic64_sub_return##name(long i, atomic64_t *v) \
{ \
register long x0 asm ("x0") = i; \
register atomic64_t *x1 asm ("x1") = v; \
@@ -392,7 +396,8 @@ ATOMIC64_OP_SUB_RETURN( , al, "memory")
#undef ATOMIC64_OP_SUB_RETURN
#define ATOMIC64_FETCH_OP_SUB(name, mb, cl...) \
-static inline long atomic64_fetch_sub##name(long i, atomic64_t *v) \
+static inline long \
+arch_atomic64_fetch_sub##name(long i, atomic64_t *v) \
{ \
register long x0 asm ("x0") = i; \
register atomic64_t *x1 asm ("x1") = v; \
@@ -418,7 +423,7 @@ ATOMIC64_FETCH_OP_SUB( , al, "memory")
#undef ATOMIC64_FETCH_OP_SUB
-static inline long atomic64_dec_if_positive(atomic64_t *v)
+static inline long arch_atomic64_dec_if_positive(atomic64_t *v)
{
register long x0 asm ("x0") = (long)v;
diff --git a/arch/arm64/include/asm/cmpxchg.h b/arch/arm64/include/asm/cmpxchg.h
index 4f5fd2a36e6e..0f470ffd2d59 100644
--- a/arch/arm64/include/asm/cmpxchg.h
+++ b/arch/arm64/include/asm/cmpxchg.h
@@ -154,18 +154,19 @@ __CMPXCHG_GEN(_mb)
})
/* cmpxchg */
-#define cmpxchg_relaxed(...) __cmpxchg_wrapper( , __VA_ARGS__)
-#define cmpxchg_acquire(...) __cmpxchg_wrapper(_acq, __VA_ARGS__)
-#define cmpxchg_release(...) __cmpxchg_wrapper(_rel, __VA_ARGS__)
-#define cmpxchg(...) __cmpxchg_wrapper( _mb, __VA_ARGS__)
-#define cmpxchg_local cmpxchg_relaxed
+#define arch_cmpxchg_relaxed(...) __cmpxchg_wrapper( , __VA_ARGS__)
+#define arch_cmpxchg_acquire(...) __cmpxchg_wrapper(_acq, __VA_ARGS__)
+#define arch_cmpxchg_release(...) __cmpxchg_wrapper(_rel, __VA_ARGS__)
+#define arch_cmpxchg(...) __cmpxchg_wrapper( _mb, __VA_ARGS__)
+#define arch_cmpxchg_local arch_cmpxchg_relaxed
+#define arch_sync_cmpxchg arch_cmpxchg
/* cmpxchg64 */
-#define cmpxchg64_relaxed cmpxchg_relaxed
-#define cmpxchg64_acquire cmpxchg_acquire
-#define cmpxchg64_release cmpxchg_release
-#define cmpxchg64 cmpxchg
-#define cmpxchg64_local cmpxchg_local
+#define arch_cmpxchg64_relaxed arch_cmpxchg_relaxed
+#define arch_cmpxchg64_acquire arch_cmpxchg_acquire
+#define arch_cmpxchg64_release arch_cmpxchg_release
+#define arch_cmpxchg64 arch_cmpxchg
+#define arch_cmpxchg64_local arch_cmpxchg_local
/* cmpxchg_double */
#define system_has_cmpxchg_double() 1
@@ -177,7 +178,7 @@ __CMPXCHG_GEN(_mb)
VM_BUG_ON((unsigned long *)(ptr2) - (unsigned long *)(ptr1) != 1); \
})
-#define cmpxchg_double(ptr1, ptr2, o1, o2, n1, n2) \
+#define arch_cmpxchg_double(ptr1, ptr2, o1, o2, n1, n2) \
({\
int __ret;\
__cmpxchg_double_check(ptr1, ptr2); \
@@ -187,7 +188,7 @@ __CMPXCHG_GEN(_mb)
__ret; \
})
-#define cmpxchg_double_local(ptr1, ptr2, o1, o2, n1, n2) \
+#define arch_cmpxchg_double_local(ptr1, ptr2, o1, o2, n1, n2) \
({\
int __ret;\
__cmpxchg_double_check(ptr1, ptr2); \
diff --git a/arch/arm64/include/asm/sync_bitops.h b/arch/arm64/include/asm/sync_bitops.h
index 24ed8f445b8b..e42de14627f2 100644
--- a/arch/arm64/include/asm/sync_bitops.h
+++ b/arch/arm64/include/asm/sync_bitops.h
@@ -22,6 +22,5 @@
#define sync_test_and_clear_bit(nr, p) test_and_clear_bit(nr, p)
#define sync_test_and_change_bit(nr, p) test_and_change_bit(nr, p)
#define sync_test_bit(nr, addr) test_bit(nr, addr)
-#define sync_cmpxchg cmpxchg
#endif
--
2.11.0
^ permalink raw reply related
* [PATCH 4/6] arm64: fix assembly constraints for cmpxchg
From: Mark Rutland @ 2018-05-04 17:39 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180504173937.25300-1-mark.rutland@arm.com>
Our LL/SC cmpxchg assembly uses "Lr" as the constraint for old, which
allows either an integer constant suitable for a 64-bit logical
oepration, or a register.
However, this assembly is also used for 32-bit cases (where we
explicitly add a 'w' prefix to the output format), where the set of
valid immediates differ, and we should use a 'Kr' constraint.
In some cases, this can result in build failures, when GCC selects an
immediate which is valid for a 64-bit logical operation, but we try to
assemble a 32-bit logical operation:
[mark at lakrids:~/src/linux]% uselinaro 17.05 make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- net/sunrpc/auth_gss/svcauth_gss.o
CHK include/config/kernel.release
CHK include/generated/uapi/linux/version.h
CHK include/generated/utsrelease.h
CHK include/generated/bounds.h
CHK include/generated/timeconst.h
CHK include/generated/asm-offsets.h
CALL scripts/checksyscalls.sh
CHK scripts/mod/devicetable-offsets.h
CC net/sunrpc/auth_gss/svcauth_gss.o
/tmp/ccj04KVh.s: Assembler messages:
/tmp/ccj04KVh.s:325: Error: immediate out of range at operand 3 -- `eor w2,w1,4294967295'
scripts/Makefile.build:324: recipe for target 'net/sunrpc/auth_gss/svcauth_gss.o' failed
make[1]: *** [net/sunrpc/auth_gss/svcauth_gss.o] Error 1
Makefile:1704: recipe for target 'net/sunrpc/auth_gss/svcauth_gss.o' failed
make: *** [net/sunrpc/auth_gss/svcauth_gss.o] Error 2
Note that today we largely avoid the specific failure above because GCC
happens to already have the value in a register, and in most cases uses
that rather than generating the immediate. The following code added to
an arbitrary file will cause the same failure:
unsigned int test_cmpxchg(unsigned int *l)
{
return cmpxchg(l, -1, 0);
}
While it would seem that we could conditionally use the 'K' constraint,
this seems to be handled erroneously by GCC (at least versions 6.3 and
7.1), with the same immediates being used, despite not being permitted
for 32-bit logical operations.
Thus we must avoid the use of an immediate in order to prevent failures
as above.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/atomic_ll_sc.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/atomic_ll_sc.h b/arch/arm64/include/asm/atomic_ll_sc.h
index f5a2d09afb38..3175f4982682 100644
--- a/arch/arm64/include/asm/atomic_ll_sc.h
+++ b/arch/arm64/include/asm/atomic_ll_sc.h
@@ -267,7 +267,7 @@ __LL_SC_PREFIX(__cmpxchg_case_##name(volatile void *ptr, \
"2:" \
: [tmp] "=&r" (tmp), [oldval] "=&r" (oldval), \
[v] "+Q" (*(unsigned long *)ptr) \
- : [old] "Lr" (old), [new] "r" (new) \
+ : [old] "r" (old), [new] "r" (new) \
: cl); \
\
return oldval; \
--
2.11.0
^ permalink raw reply related
* [PATCH 3/6] arm64: use <linux/atomic.h> for cmpxchg
From: Mark Rutland @ 2018-05-04 17:39 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180504173937.25300-1-mark.rutland@arm.com>
Currently a number of arm64-specific files include <asm/cmpxchg.h> for
the definition of the cmpxchg helpers. This works fine today, but won't
when we switch over to instrumented atomics, and as noted in
Documentation/core-api/atomic_ops.rst:
If someone wants to use xchg(), cmpxchg() and their variants,
linux/atomic.h should be included rather than asm/cmpxchg.h, unless
the code is in arch/* and can take care of itself.
... so let's switch to <linux/atomic.h> for these definitions.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/pgtable.h | 2 +-
arch/arm64/include/asm/sync_bitops.h | 2 +-
arch/arm64/mm/fault.c | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 7c4c8f318ba9..c797c0fbbce2 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -39,8 +39,8 @@
#ifndef __ASSEMBLY__
-#include <asm/cmpxchg.h>
#include <asm/fixmap.h>
+#include <linux/atomic.h>
#include <linux/mmdebug.h>
#include <linux/mm_types.h>
#include <linux/sched.h>
diff --git a/arch/arm64/include/asm/sync_bitops.h b/arch/arm64/include/asm/sync_bitops.h
index eee31a9f72a5..24ed8f445b8b 100644
--- a/arch/arm64/include/asm/sync_bitops.h
+++ b/arch/arm64/include/asm/sync_bitops.h
@@ -3,7 +3,7 @@
#define __ASM_SYNC_BITOPS_H__
#include <asm/bitops.h>
-#include <asm/cmpxchg.h>
+#include <linux/atomic.h>
/* sync_bitops functions are equivalent to the SMP implementation of the
* original functions, independently from CONFIG_SMP being defined.
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 4165485e8b6e..bfbc695e2ea2 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -18,6 +18,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <linux/atomic.h>
#include <linux/extable.h>
#include <linux/signal.h>
#include <linux/mm.h>
@@ -34,7 +35,6 @@
#include <linux/hugetlb.h>
#include <asm/bug.h>
-#include <asm/cmpxchg.h>
#include <asm/cpufeature.h>
#include <asm/exception.h>
#include <asm/debug-monitors.h>
--
2.11.0
^ permalink raw reply related
* [PATCH 2/6] locking/atomic, asm-generic: instrument atomic*andnot*()
From: Mark Rutland @ 2018-05-04 17:39 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180504173937.25300-1-mark.rutland@arm.com>
We don't currently define instrumentation wrappers for the various forms
of atomic*andnot*(), as these aren't implemented directly by x86.
So that we can instrument architectures which provide these, let's
define wrappers for all the variants of these atomics.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Will Deacon <will.deacon@arm.com>
---
include/asm-generic/atomic-instrumented.h | 112 ++++++++++++++++++++++++++++++
1 file changed, 112 insertions(+)
diff --git a/include/asm-generic/atomic-instrumented.h b/include/asm-generic/atomic-instrumented.h
index 26f0e3098442..b1920f0f64ab 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -498,6 +498,62 @@ INSTR_ATOMIC64_AND(_release)
#define atomic64_and_release atomic64_and_release
#endif
+#define INSTR_ATOMIC_ANDNOT(order) \
+static __always_inline void \
+atomic_andnot##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic_andnot##order(i, v); \
+}
+
+#ifdef arch_atomic_andnot
+INSTR_ATOMIC_ANDNOT()
+#define atomic_andnot atomic_andnot
+#endif
+
+#ifdef arch_atomic_andnot_relaxed
+INSTR_ATOMIC_ANDNOT(_relaxed)
+#define atomic_andnot_relaxed atomic_andnot_relaxed
+#endif
+
+#ifdef arch_atomic_andnot_acquire
+INSTR_ATOMIC_ANDNOT(_acquire)
+#define atomic_andnot_acquire atomic_andnot_acquire
+#endif
+
+#ifdef arch_atomic_andnot_release
+INSTR_ATOMIC_ANDNOT(_release)
+#define atomic_andnot_release atomic_andnot_release
+#endif
+
+#define INSTR_ATOMIC64_ANDNOT(order) \
+static __always_inline void \
+atomic64_andnot##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic64_andnot##order(i, v); \
+}
+
+#ifdef arch_atomic64_andnot
+INSTR_ATOMIC64_ANDNOT()
+#define atomic64_andnot atomic64_andnot
+#endif
+
+#ifdef arch_atomic64_andnot_relaxed
+INSTR_ATOMIC64_ANDNOT(_relaxed)
+#define atomic64_andnot_relaxed atomic64_andnot_relaxed
+#endif
+
+#ifdef arch_atomic64_andnot_acquire
+INSTR_ATOMIC64_ANDNOT(_acquire)
+#define atomic64_andnot_acquire atomic64_andnot_acquire
+#endif
+
+#ifdef arch_atomic64_andnot_release
+INSTR_ATOMIC64_ANDNOT(_release)
+#define atomic64_andnot_release atomic64_andnot_release
+#endif
+
#define INSTR_ATOMIC_OR(order) \
static __always_inline void \
atomic_or##order(int i, atomic_t *v) \
@@ -984,6 +1040,62 @@ INSTR_ATOMIC64_FETCH_AND(_release)
#define atomic64_fetch_and_release atomic64_fetch_and_release
#endif
+#define INSTR_ATOMIC_FETCH_ANDNOT(order) \
+static __always_inline int \
+atomic_fetch_andnot##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_fetch_andnot##order(i, v); \
+}
+
+#ifdef arch_atomic_fetch_andnot
+INSTR_ATOMIC_FETCH_ANDNOT()
+#define atomic_fetch_andnot atomic_fetch_andnot
+#endif
+
+#ifdef arch_atomic_fetch_andnot_relaxed
+INSTR_ATOMIC_FETCH_ANDNOT(_relaxed)
+#define atomic_fetch_andnot_relaxed atomic_fetch_andnot_relaxed
+#endif
+
+#ifdef arch_atomic_fetch_andnot_acquire
+INSTR_ATOMIC_FETCH_ANDNOT(_acquire)
+#define atomic_fetch_andnot_acquire atomic_fetch_andnot_acquire
+#endif
+
+#ifdef arch_atomic_fetch_andnot_release
+INSTR_ATOMIC_FETCH_ANDNOT(_release)
+#define atomic_fetch_andnot_release atomic_fetch_andnot_release
+#endif
+
+#define INSTR_ATOMIC64_FETCH_ANDNOT(order) \
+static __always_inline s64 \
+atomic64_fetch_andnot##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_fetch_andnot##order(i, v); \
+}
+
+#ifdef arch_atomic64_fetch_andnot
+INSTR_ATOMIC64_FETCH_ANDNOT()
+#define atomic64_fetch_andnot atomic64_fetch_andnot
+#endif
+
+#ifdef arch_atomic64_fetch_andnot_relaxed
+INSTR_ATOMIC64_FETCH_ANDNOT(_relaxed)
+#define atomic64_fetch_andnot_relaxed atomic64_fetch_andnot_relaxed
+#endif
+
+#ifdef arch_atomic64_fetch_andnot_acquire
+INSTR_ATOMIC64_FETCH_ANDNOT(_acquire)
+#define atomic64_fetch_andnot_acquire atomic64_fetch_andnot_acquire
+#endif
+
+#ifdef arch_atomic64_fetch_andnot_release
+INSTR_ATOMIC64_FETCH_ANDNOT(_release)
+#define atomic64_fetch_andnot_release atomic64_fetch_andnot_release
+#endif
+
#define INSTR_ATOMIC_FETCH_OR(order) \
static __always_inline int \
atomic_fetch_or##order(int i, atomic_t *v) \
--
2.11.0
^ permalink raw reply related
* [PATCH 1/6] locking/atomic, asm-generic: instrument ordering variants
From: Mark Rutland @ 2018-05-04 17:39 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180504173937.25300-1-mark.rutland@arm.com>
Currently <asm-generic/atomic-instrumented.h> only instruments the fully
ordered variants of atomic functions, ignoring the {relaxed,acquire,release}
ordering variants.
This patch reworks the header to instrument all ordering variants of the atomic
functions, so that architectures implementing these are instrumented
appropriately.
To minimise repetition, a macro is used to generate each variant from a common
template. The {full,relaxed,acquire,release} order variants respectively are
then built using this template, where the architecture provides an
implementation.
To stick to an 80 column limit while keeping the templates legible, the return
type and function name of each template are split over two lines. For
consistency, this is done even when not strictly necessary.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Will Deacon <will.deacon@arm.com>
---
include/asm-generic/atomic-instrumented.h | 1195 ++++++++++++++++++++++++-----
1 file changed, 1008 insertions(+), 187 deletions(-)
diff --git a/include/asm-generic/atomic-instrumented.h b/include/asm-generic/atomic-instrumented.h
index ec07f23678ea..26f0e3098442 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -40,171 +40,664 @@ static __always_inline void atomic64_set(atomic64_t *v, s64 i)
arch_atomic64_set(v, i);
}
-static __always_inline int atomic_xchg(atomic_t *v, int i)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic_xchg(v, i);
+#define INSTR_ATOMIC_XCHG(order) \
+static __always_inline int \
+atomic_xchg##order(atomic_t *v, int i) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_xchg##order(v, i); \
}
-static __always_inline s64 atomic64_xchg(atomic64_t *v, s64 i)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_xchg(v, i);
+INSTR_ATOMIC_XCHG()
+
+#ifdef arch_atomic_xchg_relaxed
+INSTR_ATOMIC_XCHG(_relaxed)
+#define atomic_xchg_relaxed atomic_xchg_relaxed
+#endif
+
+#ifdef arch_atomic_xchg_acquire
+INSTR_ATOMIC_XCHG(_acquire)
+#define atomic_xchg_acquire atomic_xchg_acquire
+#endif
+
+#ifdef arch_atomic_xchg_release
+INSTR_ATOMIC_XCHG(_release)
+#define atomic_xchg_release atomic_xchg_release
+#endif
+
+#define INSTR_ATOMIC64_XCHG(order) \
+static __always_inline s64 \
+atomic64_xchg##order(atomic64_t *v, s64 i) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_xchg##order(v, i); \
}
-static __always_inline int atomic_cmpxchg(atomic_t *v, int old, int new)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic_cmpxchg(v, old, new);
+INSTR_ATOMIC64_XCHG()
+
+#ifdef arch_atomic64_xchg_relaxed
+INSTR_ATOMIC64_XCHG(_relaxed)
+#define atomic64_xchg_relaxed atomic64_xchg_relaxed
+#endif
+
+#ifdef arch_atomic64_xchg_acquire
+INSTR_ATOMIC64_XCHG(_acquire)
+#define atomic64_xchg_acquire atomic64_xchg_acquire
+#endif
+
+#ifdef arch_atomic64_xchg_release
+INSTR_ATOMIC64_XCHG(_release)
+#define atomic64_xchg_release atomic64_xchg_release
+#endif
+
+#define INSTR_ATOMIC_CMPXCHG(order) \
+static __always_inline int \
+atomic_cmpxchg##order(atomic_t *v, int old, int new) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_cmpxchg##order(v, old, new); \
}
-static __always_inline s64 atomic64_cmpxchg(atomic64_t *v, s64 old, s64 new)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_cmpxchg(v, old, new);
+INSTR_ATOMIC_CMPXCHG()
+
+#ifdef arch_atomic_cmpxchg_relaxed
+INSTR_ATOMIC_CMPXCHG(_relaxed)
+#define atomic_cmpxchg_relaxed atomic_cmpxchg_relaxed
+#endif
+
+#ifdef arch_atomic_cmpxchg_acquire
+INSTR_ATOMIC_CMPXCHG(_acquire)
+#define atomic_cmpxchg_acquire atomic_cmpxchg_acquire
+#endif
+
+#ifdef arch_atomic_cmpxchg_release
+INSTR_ATOMIC_CMPXCHG(_release)
+#define atomic_cmpxchg_release atomic_cmpxchg_release
+#endif
+
+#define INSTR_ATOMIC64_CMPXCHG(order) \
+static __always_inline s64 \
+atomic64_cmpxchg##order(atomic64_t *v, s64 old, s64 new) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_cmpxchg##order(v, old, new); \
+}
+
+INSTR_ATOMIC64_CMPXCHG()
+
+#ifdef arch_atomic64_cmpxchg_relaxed
+INSTR_ATOMIC64_CMPXCHG(_relaxed)
+#define atomic64_cmpxchg_relaxed atomic64_cmpxchg_relaxed
+#endif
+
+#ifdef arch_atomic64_cmpxchg_acquire
+INSTR_ATOMIC64_CMPXCHG(_acquire)
+#define atomic64_cmpxchg_acquire atomic64_cmpxchg_acquire
+#endif
+
+#ifdef arch_atomic64_cmpxchg_release
+INSTR_ATOMIC64_CMPXCHG(_release)
+#define atomic64_cmpxchg_release atomic64_cmpxchg_release
+#endif
+
+#define INSTR_ATOMIC_TRY_CMPXCHG(order) \
+static __always_inline bool \
+atomic_try_cmpxchg##order(atomic_t *v, int *old, int new) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ kasan_check_read(old, sizeof(*old)); \
+ return arch_atomic_try_cmpxchg##order(v, old, new); \
}
#ifdef arch_atomic_try_cmpxchg
+INSTR_ATOMIC_TRY_CMPXCHG()
#define atomic_try_cmpxchg atomic_try_cmpxchg
-static __always_inline bool atomic_try_cmpxchg(atomic_t *v, int *old, int new)
-{
- kasan_check_write(v, sizeof(*v));
- kasan_check_read(old, sizeof(*old));
- return arch_atomic_try_cmpxchg(v, old, new);
-}
#endif
-#ifdef arch_atomic64_try_cmpxchg
-#define atomic64_try_cmpxchg atomic64_try_cmpxchg
-static __always_inline bool atomic64_try_cmpxchg(atomic64_t *v, s64 *old, s64 new)
-{
- kasan_check_write(v, sizeof(*v));
- kasan_check_read(old, sizeof(*old));
- return arch_atomic64_try_cmpxchg(v, old, new);
+#ifdef arch_atomic_try_cmpxchg_relaxed
+INSTR_ATOMIC_TRY_CMPXCHG(_relaxed)
+#define atomic_try_cmpxchg_relaxed atomic_try_cmpxchg_relaxed
+#endif
+
+#ifdef arch_atomic_try_cmpxchg_acquire
+INSTR_ATOMIC_TRY_CMPXCHG(_acquire)
+#define atomic_try_cmpxchg_acquire atomic_try_cmpxchg_acquire
+#endif
+
+#ifdef arch_atomic_try_cmpxchg_release
+INSTR_ATOMIC_TRY_CMPXCHG(_release)
+#define atomic_try_cmpxchg_release atomic_try_cmpxchg_release
+#endif
+
+#define INSTR_ATOMIC64_TRY_CMPXCHG(order) \
+static __always_inline bool \
+atomic64_try_cmpxchg##order(atomic64_t *v, s64 *old, s64 new) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ kasan_check_read(old, sizeof(*old)); \
+ return arch_atomic64_try_cmpxchg##order(v, old, new); \
}
+
+#ifdef arch_atomic64_try_cmpxchg
+INSTR_ATOMIC64_TRY_CMPXCHG()
+#define atomic_try_cmpxchg atomic_try_cmpxchg
#endif
-static __always_inline int __atomic_add_unless(atomic_t *v, int a, int u)
-{
- kasan_check_write(v, sizeof(*v));
- return __arch_atomic_add_unless(v, a, u);
+#ifdef arch_atomic64_try_cmpxchg_relaxed
+INSTR_ATOMIC64_TRY_CMPXCHG(_relaxed)
+#define atomic_try_cmpxchg_relaxed atomic_try_cmpxchg_relaxed
+#endif
+
+#ifdef arch_atomic64_try_cmpxchg_acquire
+INSTR_ATOMIC64_TRY_CMPXCHG(_acquire)
+#define atomic_try_cmpxchg_acquire atomic_try_cmpxchg_acquire
+#endif
+
+#ifdef arch_atomic64_try_cmpxchg_release
+INSTR_ATOMIC64_TRY_CMPXCHG(_release)
+#define atomic_try_cmpxchg_release atomic_try_cmpxchg_release
+#endif
+
+#define __INSTR_ATOMIC_ADD_UNLESS(order) \
+static __always_inline int \
+__atomic_add_unless##order(atomic_t *v, int a, int u) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return __arch_atomic_add_unless##order(v, a, u); \
}
+__INSTR_ATOMIC_ADD_UNLESS()
-static __always_inline bool atomic64_add_unless(atomic64_t *v, s64 a, s64 u)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_add_unless(v, a, u);
+#ifdef __arch_atomic_add_unless_relaxed
+__INSTR_ATOMIC_ADD_UNLESS(_relaxed)
+#define __atomic_add_unless_relaxed __atomic_add_unless_relaxed
+#endif
+
+#ifdef __arch_atomic_add_unless_acquire
+__INSTR_ATOMIC_ADD_UNLESS(_acquire)
+#define __atomic_add_unless_acquire __atomic_add_unless_acquire
+#endif
+
+#ifdef __arch_atomic_add_unless_release
+__INSTR_ATOMIC_ADD_UNLESS(_release)
+#define __atomic_add_unless_release __atomic_add_unless_release
+#endif
+
+#define INSTR_ATOMIC64_ADD_UNLESS(order) \
+static __always_inline bool \
+atomic64_add_unless##order(atomic64_t *v, s64 a, s64 u) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_add_unless##order(v, a, u); \
}
-static __always_inline void atomic_inc(atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic_inc(v);
+INSTR_ATOMIC64_ADD_UNLESS()
+
+#ifdef arch_atomic64_add_unless_relaxed
+INSTR_ATOMIC64_ADD_UNLESS(_relaxed)
+#define atomic64_add_unless_relaxed atomic64_add_unless_relaxed
+#endif
+
+#ifdef arch_atomic64_add_unless_acquire
+INSTR_ATOMIC64_ADD_UNLESS(_acquire)
+#define atomic64_add_unless_acquire atomic64_add_unless_acquire
+#endif
+
+#ifdef arch_atomic64_add_unless_release
+INSTR_ATOMIC64_ADD_UNLESS(_release)
+#define atomic64_add_unless_release atomic64_add_unless_release
+#endif
+
+#define INSTR_ATOMIC_INC(order) \
+static __always_inline void \
+atomic_inc##order(atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic_inc##order(v); \
}
-static __always_inline void atomic64_inc(atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic64_inc(v);
+INSTR_ATOMIC_INC()
+
+#ifdef arch_atomic_inc_relaxed
+INSTR_ATOMIC_INC(_relaxed)
+#define atomic_inc_relaxed atomic_inc_relaxed
+#endif
+
+#ifdef arch_atomic_inc_acquire
+INSTR_ATOMIC_INC(_acquire)
+#define atomic_inc_acquire atomic_inc_acquire
+#endif
+
+#ifdef arch_atomic_inc_release
+INSTR_ATOMIC_INC(_release)
+#define atomic_inc_release atomic_inc_release
+#endif
+
+#define INSTR_ATOMIC64_INC(order) \
+static __always_inline void \
+atomic64_inc##order(atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic64_inc##order(v); \
}
-static __always_inline void atomic_dec(atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic_dec(v);
+INSTR_ATOMIC64_INC()
+
+#ifdef arch_atomic64_inc_relaxed
+INSTR_ATOMIC64_INC(_relaxed)
+#define atomic64_inc_relaxed atomic64_inc_relaxed
+#endif
+
+#ifdef arch_atomic64_inc_acquire
+INSTR_ATOMIC64_INC(_acquire)
+#define atomic64_inc_acquire atomic64_inc_acquire
+#endif
+
+#ifdef arch_atomic64_inc_release
+INSTR_ATOMIC64_INC(_release)
+#define atomic64_inc_release atomic64_inc_release
+#endif
+
+#define INSTR_ATOMIC_DEC(order) \
+static __always_inline void \
+atomic_dec##order(atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic_dec##order(v); \
}
-static __always_inline void atomic64_dec(atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic64_dec(v);
+INSTR_ATOMIC_DEC()
+
+#ifdef arch_atomic_dec_relaxed
+INSTR_ATOMIC_DEC(_relaxed)
+#define atomic_dec_relaxed atomic_dec_relaxed
+#endif
+
+#ifdef arch_atomic_dec_acquire
+INSTR_ATOMIC_DEC(_acquire)
+#define atomic_dec_acquire atomic_dec_acquire
+#endif
+
+#ifdef arch_atomic_dec_release
+INSTR_ATOMIC_DEC(_release)
+#define atomic_dec_release atomic_dec_release
+#endif
+
+#define INSTR_ATOMIC64_DEC(order) \
+static __always_inline void \
+atomic64_dec##order(atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic64_dec##order(v); \
}
-static __always_inline void atomic_add(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic_add(i, v);
+INSTR_ATOMIC64_DEC()
+
+#ifdef arch_atomic64_dec_relaxed
+INSTR_ATOMIC64_DEC(_relaxed)
+#define atomic64_dec_relaxed atomic64_dec_relaxed
+#endif
+
+#ifdef arch_atomic64_dec_acquire
+INSTR_ATOMIC64_DEC(_acquire)
+#define atomic64_dec_acquire atomic64_dec_acquire
+#endif
+
+#ifdef arch_atomic64_dec_release
+INSTR_ATOMIC64_DEC(_release)
+#define atomic64_dec_release atomic64_dec_release
+#endif
+
+#define INSTR_ATOMIC_ADD(order) \
+static __always_inline void \
+atomic_add##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic_add##order(i, v); \
}
-static __always_inline void atomic64_add(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic64_add(i, v);
+INSTR_ATOMIC_ADD()
+
+#ifdef arch_atomic_add_relaxed
+INSTR_ATOMIC_ADD(_relaxed)
+#define atomic_add_relaxed atomic_add_relaxed
+#endif
+
+#ifdef arch_atomic_add_acquire
+INSTR_ATOMIC_ADD(_acquire)
+#define atomic_add_acquire atomic_add_acquire
+#endif
+
+#ifdef arch_atomic_add_release
+INSTR_ATOMIC_ADD(_release)
+#define atomic_add_release atomic_add_release
+#endif
+
+#define INSTR_ATOMIC64_ADD(order) \
+static __always_inline void \
+atomic64_add##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic64_add##order(i, v); \
}
-static __always_inline void atomic_sub(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic_sub(i, v);
+INSTR_ATOMIC64_ADD()
+
+#ifdef arch_atomic64_add_relaxed
+INSTR_ATOMIC64_ADD(_relaxed)
+#define atomic64_add_relaxed atomic64_add_relaxed
+#endif
+
+#ifdef arch_atomic64_add_acquire
+INSTR_ATOMIC64_ADD(_acquire)
+#define atomic64_add_acquire atomic64_add_acquire
+#endif
+
+#ifdef arch_atomic64_add_release
+INSTR_ATOMIC64_ADD(_release)
+#define atomic64_add_release atomic64_add_release
+#endif
+
+#define INSTR_ATOMIC_SUB(order) \
+static __always_inline void \
+atomic_sub##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic_sub##order(i, v); \
}
-static __always_inline void atomic64_sub(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic64_sub(i, v);
+INSTR_ATOMIC_SUB()
+
+#ifdef arch_atomic_sub_relaxed
+INSTR_ATOMIC_SUB(_relaxed)
+#define atomic_sub_relaxed atomic_sub_relaxed
+#endif
+
+#ifdef arch_atomic_sub_acquire
+INSTR_ATOMIC_SUB(_acquire)
+#define atomic_sub_acquire atomic_sub_acquire
+#endif
+
+#ifdef arch_atomic_sub_release
+INSTR_ATOMIC_SUB(_release)
+#define atomic_sub_release atomic_sub_release
+#endif
+
+#define INSTR_ATOMIC64_SUB(order) \
+static __always_inline void \
+atomic64_sub##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic64_sub##order(i, v); \
}
-static __always_inline void atomic_and(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic_and(i, v);
+INSTR_ATOMIC64_SUB()
+
+#ifdef arch_atomic64_sub_relaxed
+INSTR_ATOMIC64_SUB(_relaxed)
+#define atomic64_sub_relaxed atomic64_sub_relaxed
+#endif
+
+#ifdef arch_atomic64_sub_acquire
+INSTR_ATOMIC64_SUB(_acquire)
+#define atomic64_sub_acquire atomic64_sub_acquire
+#endif
+
+#ifdef arch_atomic64_sub_release
+INSTR_ATOMIC64_SUB(_release)
+#define atomic64_sub_release atomic64_sub_release
+#endif
+
+#define INSTR_ATOMIC_AND(order) \
+static __always_inline void \
+atomic_and##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic_and##order(i, v); \
}
-static __always_inline void atomic64_and(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic64_and(i, v);
+INSTR_ATOMIC_AND()
+
+#ifdef arch_atomic_and_relaxed
+INSTR_ATOMIC_AND(_relaxed)
+#define atomic_and_relaxed atomic_and_relaxed
+#endif
+
+#ifdef arch_atomic_and_acquire
+INSTR_ATOMIC_AND(_acquire)
+#define atomic_and_acquire atomic_and_acquire
+#endif
+
+#ifdef arch_atomic_and_release
+INSTR_ATOMIC_AND(_release)
+#define atomic_and_release atomic_and_release
+#endif
+
+#define INSTR_ATOMIC64_AND(order) \
+static __always_inline void \
+atomic64_and##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic64_and##order(i, v); \
}
-static __always_inline void atomic_or(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic_or(i, v);
+INSTR_ATOMIC64_AND()
+
+#ifdef arch_atomic64_and_relaxed
+INSTR_ATOMIC64_AND(_relaxed)
+#define atomic64_and_relaxed atomic64_and_relaxed
+#endif
+
+#ifdef arch_atomic64_and_acquire
+INSTR_ATOMIC64_AND(_acquire)
+#define atomic64_and_acquire atomic64_and_acquire
+#endif
+
+#ifdef arch_atomic64_and_release
+INSTR_ATOMIC64_AND(_release)
+#define atomic64_and_release atomic64_and_release
+#endif
+
+#define INSTR_ATOMIC_OR(order) \
+static __always_inline void \
+atomic_or##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic_or##order(i, v); \
}
-static __always_inline void atomic64_or(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic64_or(i, v);
+INSTR_ATOMIC_OR()
+
+#ifdef arch_atomic_or_relaxed
+INSTR_ATOMIC_OR(_relaxed)
+#define atomic_or_relaxed atomic_or_relaxed
+#endif
+
+#ifdef arch_atomic_or_acquire
+INSTR_ATOMIC_OR(_acquire)
+#define atomic_or_acquire atomic_or_acquire
+#endif
+
+#ifdef arch_atomic_or_release
+INSTR_ATOMIC_OR(_release)
+#define atomic_or_release atomic_or_release
+#endif
+
+#define INSTR_ATOMIC64_OR(order) \
+static __always_inline void \
+atomic64_or##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic64_or##order(i, v); \
}
-static __always_inline void atomic_xor(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic_xor(i, v);
+INSTR_ATOMIC64_OR()
+
+#ifdef arch_atomic64_or_relaxed
+INSTR_ATOMIC64_OR(_relaxed)
+#define atomic64_or_relaxed atomic64_or_relaxed
+#endif
+
+#ifdef arch_atomic64_or_acquire
+INSTR_ATOMIC64_OR(_acquire)
+#define atomic64_or_acquire atomic64_or_acquire
+#endif
+
+#ifdef arch_atomic64_or_release
+INSTR_ATOMIC64_OR(_release)
+#define atomic64_or_release atomic64_or_release
+#endif
+
+#define INSTR_ATOMIC_XOR(order) \
+static __always_inline void \
+atomic_xor##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic_xor##order(i, v); \
}
-static __always_inline void atomic64_xor(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- arch_atomic64_xor(i, v);
+INSTR_ATOMIC_XOR()
+
+#ifdef arch_atomic_xor_relaxed
+INSTR_ATOMIC_XOR(_relaxed)
+#define atomic_xor_relaxed atomic_xor_relaxed
+#endif
+
+#ifdef arch_atomic_xor_acquire
+INSTR_ATOMIC_XOR(_acquire)
+#define atomic_xor_acquire atomic_xor_acquire
+#endif
+
+#ifdef arch_atomic_xor_release
+INSTR_ATOMIC_XOR(_release)
+#define atomic_xor_release atomic_xor_release
+#endif
+
+#define INSTR_ATOMIC64_XOR(order) \
+static __always_inline void \
+atomic64_xor##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ arch_atomic64_xor##order(i, v); \
}
-static __always_inline int atomic_inc_return(atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic_inc_return(v);
+INSTR_ATOMIC64_XOR()
+
+#ifdef arch_atomic64_xor_relaxed
+INSTR_ATOMIC64_XOR(_relaxed)
+#define atomic64_xor_relaxed atomic64_xor_relaxed
+#endif
+
+#ifdef arch_atomic64_xor_acquire
+INSTR_ATOMIC64_XOR(_acquire)
+#define atomic64_xor_acquire atomic64_xor_acquire
+#endif
+
+#ifdef arch_atomic64_xor_release
+INSTR_ATOMIC64_XOR(_release)
+#define atomic64_xor_release atomic64_xor_release
+#endif
+
+#define INSTR_ATOMIC_INC_RETURN(order) \
+static __always_inline int \
+atomic_inc_return##order(atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_inc_return##order(v); \
}
-static __always_inline s64 atomic64_inc_return(atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_inc_return(v);
+INSTR_ATOMIC_INC_RETURN()
+
+#ifdef arch_atomic_inc_return_relaxed
+INSTR_ATOMIC_INC_RETURN(_relaxed)
+#define atomic_inc_return_relaxed atomic_inc_return_relaxed
+#endif
+
+#ifdef arch_atomic_inc_return_acquire
+INSTR_ATOMIC_INC_RETURN(_acquire)
+#define atomic_inc_return_acquire atomic_inc_return_acquire
+#endif
+
+#ifdef arch_atomic_inc_return_release
+INSTR_ATOMIC_INC_RETURN(_release)
+#define atomic_inc_return_release atomic_inc_return_release
+#endif
+
+#define INSTR_ATOMIC64_INC_RETURN(order) \
+static __always_inline s64 \
+atomic64_inc_return##order(atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_inc_return##order(v); \
}
-static __always_inline int atomic_dec_return(atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic_dec_return(v);
+INSTR_ATOMIC64_INC_RETURN()
+
+#ifdef arch_atomic64_inc_return_relaxed
+INSTR_ATOMIC64_INC_RETURN(_relaxed)
+#define atomic64_inc_return_relaxed atomic64_inc_return_relaxed
+#endif
+
+#ifdef arch_atomic64_inc_return_acquire
+INSTR_ATOMIC64_INC_RETURN(_acquire)
+#define atomic64_inc_return_acquire atomic64_inc_return_acquire
+#endif
+
+#ifdef arch_atomic64_inc_return_release
+INSTR_ATOMIC64_INC_RETURN(_release)
+#define atomic64_inc_return_release atomic64_inc_return_release
+#endif
+
+#define INSTR_ATOMIC_DEC_RETURN(order) \
+static __always_inline int \
+atomic_dec_return##order(atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_dec_return##order(v); \
}
-static __always_inline s64 atomic64_dec_return(atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_dec_return(v);
+INSTR_ATOMIC_DEC_RETURN()
+
+#ifdef arch_atomic_dec_return_relaxed
+INSTR_ATOMIC_DEC_RETURN(_relaxed)
+#define atomic_dec_return_relaxed atomic_dec_return_relaxed
+#endif
+
+#ifdef arch_atomic_dec_return_acquire
+INSTR_ATOMIC_DEC_RETURN(_acquire)
+#define atomic_dec_return_acquire atomic_dec_return_acquire
+#endif
+
+#ifdef arch_atomic_dec_return_release
+INSTR_ATOMIC_DEC_RETURN(_release)
+#define atomic_dec_return_release atomic_dec_return_release
+#endif
+
+#define INSTR_ATOMIC64_DEC_RETURN(order) \
+static __always_inline s64 \
+atomic64_dec_return##order(atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_dec_return##order(v); \
}
+INSTR_ATOMIC64_DEC_RETURN()
+
+#ifdef arch_atomic64_dec_return_relaxed
+INSTR_ATOMIC64_DEC_RETURN(_relaxed)
+#define atomic64_dec_return_relaxed atomic64_dec_return_relaxed
+#endif
+
+#ifdef arch_atomic64_dec_return_acquire
+INSTR_ATOMIC64_DEC_RETURN(_acquire)
+#define atomic64_dec_return_acquire atomic64_dec_return_acquire
+#endif
+
+#ifdef arch_atomic64_dec_return_release
+INSTR_ATOMIC64_DEC_RETURN(_release)
+#define atomic64_dec_return_release atomic64_dec_return_release
+#endif
+
static __always_inline s64 atomic64_inc_not_zero(atomic64_t *v)
{
kasan_check_write(v, sizeof(*v));
@@ -241,90 +734,356 @@ static __always_inline bool atomic64_inc_and_test(atomic64_t *v)
return arch_atomic64_inc_and_test(v);
}
-static __always_inline int atomic_add_return(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic_add_return(i, v);
+#define INSTR_ATOMIC_ADD_RETURN(order) \
+static __always_inline int \
+atomic_add_return##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_add_return##order(i, v); \
}
-static __always_inline s64 atomic64_add_return(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_add_return(i, v);
+INSTR_ATOMIC_ADD_RETURN()
+
+#ifdef arch_atomic_add_return_relaxed
+INSTR_ATOMIC_ADD_RETURN(_relaxed)
+#define atomic_add_return_relaxed atomic_add_return_relaxed
+#endif
+
+#ifdef arch_atomic_add_return_acquire
+INSTR_ATOMIC_ADD_RETURN(_acquire)
+#define atomic_add_return_acquire atomic_add_return_acquire
+#endif
+
+#ifdef arch_atomic_add_return_release
+INSTR_ATOMIC_ADD_RETURN(_release)
+#define atomic_add_return_release atomic_add_return_release
+#endif
+
+#define INSTR_ATOMIC64_ADD_RETURN(order) \
+static __always_inline s64 \
+atomic64_add_return##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_add_return##order(i, v); \
}
-static __always_inline int atomic_sub_return(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic_sub_return(i, v);
+INSTR_ATOMIC64_ADD_RETURN()
+
+#ifdef arch_atomic64_add_return_relaxed
+INSTR_ATOMIC64_ADD_RETURN(_relaxed)
+#define atomic64_add_return_relaxed atomic64_add_return_relaxed
+#endif
+
+#ifdef arch_atomic64_add_return_acquire
+INSTR_ATOMIC64_ADD_RETURN(_acquire)
+#define atomic64_add_return_acquire atomic64_add_return_acquire
+#endif
+
+#ifdef arch_atomic64_add_return_release
+INSTR_ATOMIC64_ADD_RETURN(_release)
+#define atomic64_add_return_release atomic64_add_return_release
+#endif
+
+#define INSTR_ATOMIC_SUB_RETURN(order) \
+static __always_inline int \
+atomic_sub_return##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_sub_return##order(i, v); \
}
-static __always_inline s64 atomic64_sub_return(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_sub_return(i, v);
+INSTR_ATOMIC_SUB_RETURN()
+
+#ifdef arch_atomic_sub_return_relaxed
+INSTR_ATOMIC_SUB_RETURN(_relaxed)
+#define atomic_sub_return_relaxed atomic_sub_return_relaxed
+#endif
+
+#ifdef arch_atomic_sub_return_acquire
+INSTR_ATOMIC_SUB_RETURN(_acquire)
+#define atomic_sub_return_acquire atomic_sub_return_acquire
+#endif
+
+#ifdef arch_atomic_sub_return_release
+INSTR_ATOMIC_SUB_RETURN(_release)
+#define atomic_sub_return_release atomic_sub_return_release
+#endif
+
+#define INSTR_ATOMIC64_SUB_RETURN(order) \
+static __always_inline s64 \
+atomic64_sub_return##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_sub_return##order(i, v); \
}
-static __always_inline int atomic_fetch_add(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic_fetch_add(i, v);
+INSTR_ATOMIC64_SUB_RETURN()
+
+#ifdef arch_atomic64_sub_return_relaxed
+INSTR_ATOMIC64_SUB_RETURN(_relaxed)
+#define atomic64_sub_return_relaxed atomic64_sub_return_relaxed
+#endif
+
+#ifdef arch_atomic64_sub_return_acquire
+INSTR_ATOMIC64_SUB_RETURN(_acquire)
+#define atomic64_sub_return_acquire atomic64_sub_return_acquire
+#endif
+
+#ifdef arch_atomic64_sub_return_release
+INSTR_ATOMIC64_SUB_RETURN(_release)
+#define atomic64_sub_return_release atomic64_sub_return_release
+#endif
+
+#define INSTR_ATOMIC_FETCH_ADD(order) \
+static __always_inline int \
+atomic_fetch_add##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_fetch_add##order(i, v); \
}
-static __always_inline s64 atomic64_fetch_add(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_fetch_add(i, v);
+INSTR_ATOMIC_FETCH_ADD()
+
+#ifdef arch_atomic_fetch_add_relaxed
+INSTR_ATOMIC_FETCH_ADD(_relaxed)
+#define atomic_fetch_add_relaxed atomic_fetch_add_relaxed
+#endif
+
+#ifdef arch_atomic_fetch_add_acquire
+INSTR_ATOMIC_FETCH_ADD(_acquire)
+#define atomic_fetch_add_acquire atomic_fetch_add_acquire
+#endif
+
+#ifdef arch_atomic_fetch_add_release
+INSTR_ATOMIC_FETCH_ADD(_release)
+#define atomic_fetch_add_release atomic_fetch_add_release
+#endif
+
+#define INSTR_ATOMIC64_FETCH_ADD(order) \
+static __always_inline s64 \
+atomic64_fetch_add##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_fetch_add##order(i, v); \
}
-static __always_inline int atomic_fetch_sub(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic_fetch_sub(i, v);
+INSTR_ATOMIC64_FETCH_ADD()
+
+#ifdef arch_atomic64_fetch_add_relaxed
+INSTR_ATOMIC64_FETCH_ADD(_relaxed)
+#define atomic64_fetch_add_relaxed atomic64_fetch_add_relaxed
+#endif
+
+#ifdef arch_atomic64_fetch_add_acquire
+INSTR_ATOMIC64_FETCH_ADD(_acquire)
+#define atomic64_fetch_add_acquire atomic64_fetch_add_acquire
+#endif
+
+#ifdef arch_atomic64_fetch_add_release
+INSTR_ATOMIC64_FETCH_ADD(_release)
+#define atomic64_fetch_add_release atomic64_fetch_add_release
+#endif
+
+#define INSTR_ATOMIC_FETCH_SUB(order) \
+static __always_inline int \
+atomic_fetch_sub##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_fetch_sub##order(i, v); \
}
-static __always_inline s64 atomic64_fetch_sub(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_fetch_sub(i, v);
+INSTR_ATOMIC_FETCH_SUB()
+
+#ifdef arch_atomic_fetch_sub_relaxed
+INSTR_ATOMIC_FETCH_SUB(_relaxed)
+#define atomic_fetch_sub_relaxed atomic_fetch_sub_relaxed
+#endif
+
+#ifdef arch_atomic_fetch_sub_acquire
+INSTR_ATOMIC_FETCH_SUB(_acquire)
+#define atomic_fetch_sub_acquire atomic_fetch_sub_acquire
+#endif
+
+#ifdef arch_atomic_fetch_sub_release
+INSTR_ATOMIC_FETCH_SUB(_release)
+#define atomic_fetch_sub_release atomic_fetch_sub_release
+#endif
+
+#define INSTR_ATOMIC64_FETCH_SUB(order) \
+static __always_inline s64 \
+atomic64_fetch_sub##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_fetch_sub##order(i, v); \
}
-static __always_inline int atomic_fetch_and(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic_fetch_and(i, v);
+INSTR_ATOMIC64_FETCH_SUB()
+
+#ifdef arch_atomic64_fetch_sub_relaxed
+INSTR_ATOMIC64_FETCH_SUB(_relaxed)
+#define atomic64_fetch_sub_relaxed atomic64_fetch_sub_relaxed
+#endif
+
+#ifdef arch_atomic64_fetch_sub_acquire
+INSTR_ATOMIC64_FETCH_SUB(_acquire)
+#define atomic64_fetch_sub_acquire atomic64_fetch_sub_acquire
+#endif
+
+#ifdef arch_atomic64_fetch_sub_release
+INSTR_ATOMIC64_FETCH_SUB(_release)
+#define atomic64_fetch_sub_release atomic64_fetch_sub_release
+#endif
+
+#define INSTR_ATOMIC_FETCH_AND(order) \
+static __always_inline int \
+atomic_fetch_and##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_fetch_and##order(i, v); \
}
-static __always_inline s64 atomic64_fetch_and(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_fetch_and(i, v);
+INSTR_ATOMIC_FETCH_AND()
+
+#ifdef arch_atomic_fetch_and_relaxed
+INSTR_ATOMIC_FETCH_AND(_relaxed)
+#define atomic_fetch_and_relaxed atomic_fetch_and_relaxed
+#endif
+
+#ifdef arch_atomic_fetch_and_acquire
+INSTR_ATOMIC_FETCH_AND(_acquire)
+#define atomic_fetch_and_acquire atomic_fetch_and_acquire
+#endif
+
+#ifdef arch_atomic_fetch_and_release
+INSTR_ATOMIC_FETCH_AND(_release)
+#define atomic_fetch_and_release atomic_fetch_and_release
+#endif
+
+#define INSTR_ATOMIC64_FETCH_AND(order) \
+static __always_inline s64 \
+atomic64_fetch_and##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_fetch_and##order(i, v); \
}
-static __always_inline int atomic_fetch_or(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic_fetch_or(i, v);
+INSTR_ATOMIC64_FETCH_AND()
+
+#ifdef arch_atomic64_fetch_and_relaxed
+INSTR_ATOMIC64_FETCH_AND(_relaxed)
+#define atomic64_fetch_and_relaxed atomic64_fetch_and_relaxed
+#endif
+
+#ifdef arch_atomic64_fetch_and_acquire
+INSTR_ATOMIC64_FETCH_AND(_acquire)
+#define atomic64_fetch_and_acquire atomic64_fetch_and_acquire
+#endif
+
+#ifdef arch_atomic64_fetch_and_release
+INSTR_ATOMIC64_FETCH_AND(_release)
+#define atomic64_fetch_and_release atomic64_fetch_and_release
+#endif
+
+#define INSTR_ATOMIC_FETCH_OR(order) \
+static __always_inline int \
+atomic_fetch_or##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_fetch_or##order(i, v); \
}
-static __always_inline s64 atomic64_fetch_or(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_fetch_or(i, v);
+INSTR_ATOMIC_FETCH_OR()
+
+#ifdef arch_atomic_fetch_or_relaxed
+INSTR_ATOMIC_FETCH_OR(_relaxed)
+#define atomic_fetch_or_relaxed atomic_fetch_or_relaxed
+#endif
+
+#ifdef arch_atomic_fetch_or_acquire
+INSTR_ATOMIC_FETCH_OR(_acquire)
+#define atomic_fetch_or_acquire atomic_fetch_or_acquire
+#endif
+
+#ifdef arch_atomic_fetch_or_release
+INSTR_ATOMIC_FETCH_OR(_release)
+#define atomic_fetch_or_release atomic_fetch_or_release
+#endif
+
+#define INSTR_ATOMIC64_FETCH_OR(order) \
+static __always_inline s64 \
+atomic64_fetch_or##order(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_fetch_or##order(i, v); \
}
-static __always_inline int atomic_fetch_xor(int i, atomic_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic_fetch_xor(i, v);
+INSTR_ATOMIC64_FETCH_OR()
+
+#ifdef arch_atomic64_fetch_or_relaxed
+INSTR_ATOMIC64_FETCH_OR(_relaxed)
+#define atomic64_fetch_or_relaxed atomic64_fetch_or_relaxed
+#endif
+
+#ifdef arch_atomic64_fetch_or_acquire
+INSTR_ATOMIC64_FETCH_OR(_acquire)
+#define atomic64_fetch_or_acquire atomic64_fetch_or_acquire
+#endif
+
+#ifdef arch_atomic64_fetch_or_release
+INSTR_ATOMIC64_FETCH_OR(_release)
+#define atomic64_fetch_or_release atomic64_fetch_or_release
+#endif
+
+#define INSTR_ATOMIC_FETCH_XOR(order) \
+static __always_inline int \
+atomic_fetch_xor##order(int i, atomic_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic_fetch_xor##order(i, v); \
}
-static __always_inline s64 atomic64_fetch_xor(s64 i, atomic64_t *v)
-{
- kasan_check_write(v, sizeof(*v));
- return arch_atomic64_fetch_xor(i, v);
+INSTR_ATOMIC_FETCH_XOR()
+
+#ifdef arch_atomic_fetch_xor_relaxed
+INSTR_ATOMIC_FETCH_XOR(_relaxed)
+#define atomic_fetch_xor_relaxed atomic_fetch_xor_relaxed
+#endif
+
+#ifdef arch_atomic_fetch_xor_acquire
+INSTR_ATOMIC_FETCH_XOR(_acquire)
+#define atomic_fetch_xor_acquire atomic_fetch_xor_acquire
+#endif
+
+#ifdef arch_atomic_fetch_xor_release
+INSTR_ATOMIC_FETCH_XOR(_release)
+#define atomic_fetch_xor_release atomic_fetch_xor_release
+#endif
+
+#define INSTR_ATOMIC64_FETCH_XOR(xorder) \
+static __always_inline s64 \
+atomic64_fetch_xor##xorder(s64 i, atomic64_t *v) \
+{ \
+ kasan_check_write(v, sizeof(*v)); \
+ return arch_atomic64_fetch_xor##xorder(i, v); \
}
+INSTR_ATOMIC64_FETCH_XOR()
+
+#ifdef arch_atomic64_fetch_xor_relaxed
+INSTR_ATOMIC64_FETCH_XOR(_relaxed)
+#define atomic64_fetch_xor_relaxed atomic64_fetch_xor_relaxed
+#endif
+
+#ifdef arch_atomic64_fetch_xor_acquire
+INSTR_ATOMIC64_FETCH_XOR(_acquire)
+#define atomic64_fetch_xor_acquire atomic64_fetch_xor_acquire
+#endif
+
+#ifdef arch_atomic64_fetch_xor_release
+INSTR_ATOMIC64_FETCH_XOR(_release)
+#define atomic64_fetch_xor_release atomic64_fetch_xor_release
+#endif
+
static __always_inline bool atomic_sub_and_test(int i, atomic_t *v)
{
kasan_check_write(v, sizeof(*v));
@@ -349,31 +1108,64 @@ static __always_inline bool atomic64_add_negative(s64 i, atomic64_t *v)
return arch_atomic64_add_negative(i, v);
}
-static __always_inline unsigned long
-cmpxchg_size(volatile void *ptr, unsigned long old, unsigned long new, int size)
-{
- kasan_check_write(ptr, size);
- switch (size) {
- case 1:
- return arch_cmpxchg((u8 *)ptr, (u8)old, (u8)new);
- case 2:
- return arch_cmpxchg((u16 *)ptr, (u16)old, (u16)new);
- case 4:
- return arch_cmpxchg((u32 *)ptr, (u32)old, (u32)new);
- case 8:
- BUILD_BUG_ON(sizeof(unsigned long) != 8);
- return arch_cmpxchg((u64 *)ptr, (u64)old, (u64)new);
- }
- BUILD_BUG();
- return 0;
+#define INSTR_CMPXCHG(order) \
+static __always_inline unsigned long \
+cmpxchg##order##_size(volatile void *ptr, unsigned long old, \
+ unsigned long new, int size) \
+{ \
+ kasan_check_write(ptr, size); \
+ switch (size) { \
+ case 1: \
+ return arch_cmpxchg##order((u8 *)ptr, (u8)old, (u8)new); \
+ case 2: \
+ return arch_cmpxchg##order((u16 *)ptr, (u16)old, (u16)new); \
+ case 4: \
+ return arch_cmpxchg##order((u32 *)ptr, (u32)old, (u32)new); \
+ case 8: \
+ BUILD_BUG_ON(sizeof(unsigned long) != 8); \
+ return arch_cmpxchg##order((u64 *)ptr, (u64)old, (u64)new); \
+ } \
+ BUILD_BUG(); \
+ return 0; \
}
+INSTR_CMPXCHG()
#define cmpxchg(ptr, old, new) \
({ \
((__typeof__(*(ptr)))cmpxchg_size((ptr), (unsigned long)(old), \
(unsigned long)(new), sizeof(*(ptr)))); \
})
+#ifdef arch_cmpxchg_relaxed
+INSTR_CMPXCHG(_relaxed)
+#define cmpxchg_relaxed(ptr, old, new) \
+({ \
+ ((__typeof__(*(ptr)))cmpxchg_relaxed_size((ptr), \
+ (unsigned long)(old), (unsigned long)(new), \
+ sizeof(*(ptr)))); \
+})
+#endif
+
+#ifdef arch_cmpxchg_acquire
+INSTR_CMPXCHG(_acquire)
+#define cmpxchg_acquire(ptr, old, new) \
+({ \
+ ((__typeof__(*(ptr)))cmpxchg_acquire_size((ptr), \
+ (unsigned long)(old), (unsigned long)(new), \
+ sizeof(*(ptr)))); \
+})
+#endif
+
+#ifdef arch_cmpxchg_release
+INSTR_CMPXCHG(_release)
+#define cmpxchg_release(ptr, old, new) \
+({ \
+ ((__typeof__(*(ptr)))cmpxchg_release_size((ptr), \
+ (unsigned long)(old), (unsigned long)(new), \
+ sizeof(*(ptr)))); \
+})
+#endif
+
static __always_inline unsigned long
sync_cmpxchg_size(volatile void *ptr, unsigned long old, unsigned long new,
int size)
@@ -428,19 +1220,48 @@ cmpxchg_local_size(volatile void *ptr, unsigned long old, unsigned long new,
sizeof(*(ptr)))); \
})
-static __always_inline u64
-cmpxchg64_size(volatile u64 *ptr, u64 old, u64 new)
-{
- kasan_check_write(ptr, sizeof(*ptr));
- return arch_cmpxchg64(ptr, old, new);
+#define INSTR_CMPXCHG64(order) \
+static __always_inline u64 \
+cmpxchg64##order##_size(volatile u64 *ptr, u64 old, u64 new) \
+{ \
+ kasan_check_write(ptr, sizeof(*ptr)); \
+ return arch_cmpxchg64##order(ptr, old, new); \
}
+INSTR_CMPXCHG64()
#define cmpxchg64(ptr, old, new) \
({ \
((__typeof__(*(ptr)))cmpxchg64_size((ptr), (u64)(old), \
(u64)(new))); \
})
+#ifdef arch_cmpxchg64_relaxed
+INSTR_CMPXCHG64(_relaxed)
+#define cmpxchg64_relaxed(ptr, old, new) \
+({ \
+ ((__typeof__(*(ptr)))cmpxchg64_relaxed_size((ptr), (u64)(old), \
+ (u64)(new))); \
+})
+#endif
+
+#ifdef arch_cmpxchg64_acquire
+INSTR_CMPXCHG64(_acquire)
+#define cmpxchg64_acquire(ptr, old, new) \
+({ \
+ ((__typeof__(*(ptr)))cmpxchg64_acquire_size((ptr), (u64)(old), \
+ (u64)(new))); \
+})
+#endif
+
+#ifdef arch_cmpxchg64_release
+INSTR_CMPXCHG64(_release)
+#define cmpxchg64_release(ptr, old, new) \
+({ \
+ ((__typeof__(*(ptr)))cmpxchg64_release_size((ptr), (u64)(old), \
+ (u64)(new))); \
+})
+#endif
+
static __always_inline u64
cmpxchg64_local_size(volatile u64 *ptr, u64 old, u64 new)
{
--
2.11.0
^ permalink raw reply related
* [PATCH 0/6] arm64: add instrumented atomics
From: Mark Rutland @ 2018-05-04 17:39 UTC (permalink / raw)
To: linux-arm-kernel
This series (based on v4.17-rc3) allows arm64's atomics to be
instrumented, which should make it easier to catch bugs where atomics
are used on erroneous memory locations.
The bulk of the diffstat is teaching the generic instrumentation about
the acquire/release/relaxed variants of each atomic, along with some
optional atomics which x86 doesn't implement directly.
To build an arm64 defonfig one additional patch [1] is required, which
fixes an include in the SUNRPC code. I've pushed the series, along with
that patch, to my arm64/atomic-instrumentation branch [2].
This has seen basic testing on a Juno R1 machine so far.
Thanks,
Mark.
[1] https://lkml.kernel.org/r/1489574142-20856-1-git-send-email-mark.rutland at arm.com
[2] git://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git arm64/atomic-instrumentation
Mark Rutland (6):
locking/atomic, asm-generic: instrument ordering variants
locking/atomic, asm-generic: instrument atomic*andnot*()
arm64: use <linux/atomic.h> for cmpxchg
arm64: fix assembly constraints for cmpxchg
arm64: use instrumented atomics
arm64: instrument smp_{load_acquire,store_release}
arch/arm64/include/asm/atomic.h | 299 +++----
arch/arm64/include/asm/atomic_ll_sc.h | 30 +-
arch/arm64/include/asm/atomic_lse.h | 43 +-
arch/arm64/include/asm/barrier.h | 22 +-
arch/arm64/include/asm/cmpxchg.h | 25 +-
arch/arm64/include/asm/pgtable.h | 2 +-
arch/arm64/include/asm/sync_bitops.h | 3 +-
arch/arm64/mm/fault.c | 2 +-
include/asm-generic/atomic-instrumented.h | 1305 +++++++++++++++++++++++++----
9 files changed, 1339 insertions(+), 392 deletions(-)
--
2.11.0
^ permalink raw reply
* [PATCH v2 13/27] coresight: Add generic TMC sg table framework
From: Mathieu Poirier @ 2018-05-04 17:35 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1525165857-11096-14-git-send-email-suzuki.poulose@arm.com>
On Tue, May 01, 2018 at 10:10:43AM +0100, Suzuki K Poulose wrote:
> This patch introduces a generic sg table data structure and
> associated operations. An SG table can be used to map a set
> of Data pages where the trace data could be stored by the TMC
> ETR. The information about the data pages could be stored in
> different formats, depending on the type of the underlying
> SG mechanism (e.g, TMC ETR SG vs Coresight CATU). The generic
> structure provides book keeping of the pages used for the data
> as well as the table contents. The table should be filled by
> the user of the infrastructure.
>
> A table can be created by specifying the number of data pages
> as well as the number of table pages required to hold the
> pointers, where the latter could be different for different
> types of tables. The pages are mapped in the appropriate dma
> data direction mode (i.e, DMA_TO_DEVICE for table pages
> and DMA_FROM_DEVICE for data pages). The framework can optionally
> accept a set of allocated data pages (e.g, perf ring buffer) and
> map them accordingly. The table and data pages are vmap'ed to allow
> easier access by the drivers. The framework also provides helpers to
> sync the data written to the pages with appropriate directions.
>
> This will be later used by the TMC ETR SG unit and CATU.
>
> Cc: Mathieu Poirier <matheiu.poirier@linaro.org>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
> drivers/hwtracing/coresight/coresight-tmc-etr.c | 284 ++++++++++++++++++++++++
> drivers/hwtracing/coresight/coresight-tmc.h | 50 +++++
> 2 files changed, 334 insertions(+)
>
> diff --git a/drivers/hwtracing/coresight/coresight-tmc-etr.c b/drivers/hwtracing/coresight/coresight-tmc-etr.c
> index 7af72d7..57a8fe1 100644
> --- a/drivers/hwtracing/coresight/coresight-tmc-etr.c
> +++ b/drivers/hwtracing/coresight/coresight-tmc-etr.c
> @@ -17,10 +17,294 @@
>
> #include <linux/coresight.h>
> #include <linux/dma-mapping.h>
> +#include <linux/slab.h>
> #include "coresight-catu.h"
> #include "coresight-priv.h"
> #include "coresight-tmc.h"
>
> +/*
> + * tmc_pages_get_offset: Go through all the pages in the tmc_pages
> + * and map the device address @addr to an offset within the virtual
> + * contiguous buffer.
> + */
> +static long
> +tmc_pages_get_offset(struct tmc_pages *tmc_pages, dma_addr_t addr)
> +{
> + int i;
> + dma_addr_t page_start;
> +
> + for (i = 0; i < tmc_pages->nr_pages; i++) {
> + page_start = tmc_pages->daddrs[i];
> + if (addr >= page_start && addr < (page_start + PAGE_SIZE))
> + return i * PAGE_SIZE + (addr - page_start);
> + }
> +
> + return -EINVAL;
> +}
> +
> +/*
> + * tmc_pages_free : Unmap and free the pages used by tmc_pages.
> + */
> +static void tmc_pages_free(struct tmc_pages *tmc_pages,
> + struct device *dev, enum dma_data_direction dir)
> +{
> + int i;
> +
> + for (i = 0; i < tmc_pages->nr_pages; i++) {
> + if (tmc_pages->daddrs && tmc_pages->daddrs[i])
> + dma_unmap_page(dev, tmc_pages->daddrs[i],
> + PAGE_SIZE, dir);
> + if (tmc_pages->pages && tmc_pages->pages[i])
> + __free_page(tmc_pages->pages[i]);
I think it's worth adding a comment saying that because of the page count, pages
given to the infracstructure (rather than allocated) won't be free'ed by
__free_page().
> + }
> +
> + kfree(tmc_pages->pages);
> + kfree(tmc_pages->daddrs);
> + tmc_pages->pages = NULL;
> + tmc_pages->daddrs = NULL;
> + tmc_pages->nr_pages = 0;
> +}
> +
> +/*
> + * tmc_pages_alloc : Allocate and map pages for a given @tmc_pages.
> + * If @pages is not NULL, the list of page virtual addresses are
> + * used as the data pages. The pages are then dma_map'ed for @dev
> + * with dma_direction @dir.
> + *
> + * Returns 0 upon success, else the error number.
> + */
> +static int tmc_pages_alloc(struct tmc_pages *tmc_pages,
> + struct device *dev, int node,
> + enum dma_data_direction dir, void **pages)
> +{
> + int i, nr_pages;
> + dma_addr_t paddr;
> + struct page *page;
> +
> + nr_pages = tmc_pages->nr_pages;
> + tmc_pages->daddrs = kcalloc(nr_pages, sizeof(*tmc_pages->daddrs),
> + GFP_KERNEL);
> + if (!tmc_pages->daddrs)
> + return -ENOMEM;
> + tmc_pages->pages = kcalloc(nr_pages, sizeof(*tmc_pages->pages),
> + GFP_KERNEL);
> + if (!tmc_pages->pages) {
> + kfree(tmc_pages->daddrs);
> + tmc_pages->daddrs = NULL;
> + return -ENOMEM;
> + }
> +
> + for (i = 0; i < nr_pages; i++) {
> + if (pages && pages[i]) {
> + page = virt_to_page(pages[i]);
> + get_page(page);
> + } else {
> + page = alloc_pages_node(node,
> + GFP_KERNEL | __GFP_ZERO, 0);
> + }
> + paddr = dma_map_page(dev, page, 0, PAGE_SIZE, dir);
> + if (dma_mapping_error(dev, paddr))
> + goto err;
> + tmc_pages->daddrs[i] = paddr;
> + tmc_pages->pages[i] = page;
> + }
> + return 0;
> +err:
> + tmc_pages_free(tmc_pages, dev, dir);
> + return -ENOMEM;
> +}
> +
> +static inline dma_addr_t tmc_sg_table_base_paddr(struct tmc_sg_table *sg_table)
> +{
> + if (WARN_ON(!sg_table->data_pages.pages[0]))
> + return 0;
> + return sg_table->table_daddr;
> +}
> +
> +static inline void *tmc_sg_table_base_vaddr(struct tmc_sg_table *sg_table)
> +{
> + if (WARN_ON(!sg_table->data_pages.pages[0]))
> + return NULL;
> + return sg_table->table_vaddr;
> +}
> +
> +static inline void *
> +tmc_sg_table_data_vaddr(struct tmc_sg_table *sg_table)
> +{
> + if (WARN_ON(!sg_table->data_pages.nr_pages))
> + return 0;
> + return sg_table->data_vaddr;
> +}
> +
> +static inline long
> +tmc_sg_get_data_page_offset(struct tmc_sg_table *sg_table, dma_addr_t addr)
> +{
> + return tmc_pages_get_offset(&sg_table->data_pages, addr);
> +}
> +
> +static inline void tmc_free_table_pages(struct tmc_sg_table *sg_table)
> +{
> + if (sg_table->table_vaddr)
> + vunmap(sg_table->table_vaddr);
> + tmc_pages_free(&sg_table->table_pages, sg_table->dev, DMA_TO_DEVICE);
> +}
> +
> +static void tmc_free_data_pages(struct tmc_sg_table *sg_table)
> +{
> + if (sg_table->data_vaddr)
> + vunmap(sg_table->data_vaddr);
> + tmc_pages_free(&sg_table->data_pages, sg_table->dev, DMA_FROM_DEVICE);
> +}
> +
> +void tmc_free_sg_table(struct tmc_sg_table *sg_table)
> +{
> + tmc_free_table_pages(sg_table);
> + tmc_free_data_pages(sg_table);
> +}
> +
> +/*
> + * Alloc pages for the table. Since this will be used by the device,
> + * allocate the pages closer to the device (i.e, dev_to_node(dev)
> + * rather than the CPU node).
> + */
> +static int tmc_alloc_table_pages(struct tmc_sg_table *sg_table)
> +{
> + int rc;
> + struct tmc_pages *table_pages = &sg_table->table_pages;
> +
> + rc = tmc_pages_alloc(table_pages, sg_table->dev,
> + dev_to_node(sg_table->dev),
> + DMA_TO_DEVICE, NULL);
> + if (rc)
> + return rc;
> + sg_table->table_vaddr = vmap(table_pages->pages,
> + table_pages->nr_pages,
> + VM_MAP,
> + PAGE_KERNEL);
> + if (!sg_table->table_vaddr)
> + rc = -ENOMEM;
> + else
> + sg_table->table_daddr = table_pages->daddrs[0];
> + return rc;
> +}
> +
> +static int tmc_alloc_data_pages(struct tmc_sg_table *sg_table, void **pages)
> +{
> + int rc;
> +
> + /* Allocate data pages on the node requested by the caller */
> + rc = tmc_pages_alloc(&sg_table->data_pages,
> + sg_table->dev, sg_table->node,
> + DMA_FROM_DEVICE, pages);
> + if (!rc) {
> + sg_table->data_vaddr = vmap(sg_table->data_pages.pages,
> + sg_table->data_pages.nr_pages,
> + VM_MAP,
> + PAGE_KERNEL);
Indentation.
> + if (!sg_table->data_vaddr)
> + rc = -ENOMEM;
> + }
> + return rc;
> +}
> +
> +/*
> + * tmc_alloc_sg_table: Allocate and setup dma pages for the TMC SG table
> + * and data buffers. TMC writes to the data buffers and reads from the SG
> + * Table pages.
> + *
> + * @dev - Device to which page should be DMA mapped.
> + * @node - Numa node for mem allocations
> + * @nr_tpages - Number of pages for the table entries.
> + * @nr_dpages - Number of pages for Data buffer.
> + * @pages - Optional list of virtual address of pages.
> + */
> +struct tmc_sg_table *tmc_alloc_sg_table(struct device *dev,
> + int node,
> + int nr_tpages,
> + int nr_dpages,
> + void **pages)
> +{
> + long rc;
> + struct tmc_sg_table *sg_table;
> +
> + sg_table = kzalloc(sizeof(*sg_table), GFP_KERNEL);
> + if (!sg_table)
> + return ERR_PTR(-ENOMEM);
> + sg_table->data_pages.nr_pages = nr_dpages;
> + sg_table->table_pages.nr_pages = nr_tpages;
> + sg_table->node = node;
> + sg_table->dev = dev;
> +
> + rc = tmc_alloc_data_pages(sg_table, pages);
> + if (!rc)
> + rc = tmc_alloc_table_pages(sg_table);
> + if (rc) {
> + tmc_free_sg_table(sg_table);
> + kfree(sg_table);
> + return ERR_PTR(rc);
> + }
> +
> + return sg_table;
> +}
> +
> +/*
> + * tmc_sg_table_sync_data_range: Sync the data buffer written
> + * by the device from @offset upto a @size bytes.
> + */
> +void tmc_sg_table_sync_data_range(struct tmc_sg_table *table,
> + u64 offset, u64 size)
> +{
> + int i, index, start;
> + int npages = DIV_ROUND_UP(size, PAGE_SIZE);
> + struct device *dev = table->dev;
> + struct tmc_pages *data = &table->data_pages;
> +
> + start = offset >> PAGE_SHIFT;
> + for (i = start; i < (start + npages); i++) {
> + index = i % data->nr_pages;
> + dma_sync_single_for_cpu(dev, data->daddrs[index],
> + PAGE_SIZE, DMA_FROM_DEVICE);
> + }
> +}
> +
> +/* tmc_sg_sync_table: Sync the page table */
> +void tmc_sg_table_sync_table(struct tmc_sg_table *sg_table)
> +{
> + int i;
> + struct device *dev = sg_table->dev;
> + struct tmc_pages *table_pages = &sg_table->table_pages;
> +
> + for (i = 0; i < table_pages->nr_pages; i++)
> + dma_sync_single_for_device(dev, table_pages->daddrs[i],
> + PAGE_SIZE, DMA_TO_DEVICE);
> +}
> +
> +/*
> + * tmc_sg_table_get_data: Get the buffer pointer for data @offset
> + * in the SG buffer. The @bufpp is updated to point to the buffer.
> + * Returns :
> + * the length of linear data available at @offset.
> + * or
> + * <= 0 if no data is available.
> + */
> +ssize_t tmc_sg_table_get_data(struct tmc_sg_table *sg_table,
> + u64 offset, size_t len, char **bufpp)
Indentation
> +{
> + size_t size;
> + int pg_idx = offset >> PAGE_SHIFT;
> + int pg_offset = offset & (PAGE_SIZE - 1);
> + struct tmc_pages *data_pages = &sg_table->data_pages;
> +
> + size = tmc_sg_table_buf_size(sg_table);
> + if (offset >= size)
> + return -EINVAL;
/* Make sure we don't go beyond the page array */
> + len = (len < (size - offset)) ? len : size - offset;
/* Respect page boundaries */
> + len = (len < (PAGE_SIZE - pg_offset)) ? len : (PAGE_SIZE - pg_offset);
> + if (len > 0)
> + *bufpp = page_address(data_pages->pages[pg_idx]) + pg_offset;
> + return len;
> +}
> +
> static inline void tmc_etr_enable_catu(struct tmc_drvdata *drvdata)
> {
> struct coresight_device *catu = tmc_etr_get_catu_device(drvdata);
> diff --git a/drivers/hwtracing/coresight/coresight-tmc.h b/drivers/hwtracing/coresight/coresight-tmc.h
> index 9cbc4d5..74d8f24 100644
> --- a/drivers/hwtracing/coresight/coresight-tmc.h
> +++ b/drivers/hwtracing/coresight/coresight-tmc.h
> @@ -19,6 +19,7 @@
> #define _CORESIGHT_TMC_H
>
> #include <linux/miscdevice.h>
> +#include <linux/dma-mapping.h>
Alphabetial order.
> #include "coresight-catu.h"
>
> #define TMC_RSZ 0x004
> @@ -172,6 +173,38 @@ struct tmc_drvdata {
> u32 etr_caps;
> };
>
> +/**
> + * struct tmc_pages - Collection of pages used for SG.
> + * @nr_pages: Number of pages in the list.
> + * @daddrs: Array of DMA'able page address.
> + * @pages: Array pages for the buffer.
> + */
> +struct tmc_pages {
> + int nr_pages;
> + dma_addr_t *daddrs;
> + struct page **pages;
> +};
> +
> +/*
> + * struct tmc_sg_table - Generic SG table for TMC
> + * @dev: Device for DMA allocations
> + * @table_vaddr: Contiguous Virtual address for PageTable
> + * @data_vaddr: Contiguous Virtual address for Data Buffer
> + * @table_daddr: DMA address of the PageTable base
> + * @node: Node for Page allocations
> + * @table_pages: List of pages & dma address for Table
> + * @data_pages: List of pages & dma address for Data
> + */
> +struct tmc_sg_table {
> + struct device *dev;
> + void *table_vaddr;
> + void *data_vaddr;
> + dma_addr_t table_daddr;
> + int node;
> + struct tmc_pages table_pages;
> + struct tmc_pages data_pages;
> +};
> +
> /* Generic functions */
> void tmc_wait_for_tmcready(struct tmc_drvdata *drvdata);
> void tmc_flush_and_stop(struct tmc_drvdata *drvdata);
> @@ -253,4 +286,21 @@ tmc_etr_get_catu_device(struct tmc_drvdata *drvdata)
> return NULL;
> }
>
> +struct tmc_sg_table *tmc_alloc_sg_table(struct device *dev,
> + int node,
> + int nr_tpages,
> + int nr_dpages,
> + void **pages);
> +void tmc_free_sg_table(struct tmc_sg_table *sg_table);
> +void tmc_sg_table_sync_table(struct tmc_sg_table *sg_table);
> +void tmc_sg_table_sync_data_range(struct tmc_sg_table *table,
> + u64 offset, u64 size);
> +ssize_t tmc_sg_table_get_data(struct tmc_sg_table *sg_table,
> + u64 offset, size_t len, char **bufpp);
> +static inline unsigned long
> +tmc_sg_table_buf_size(struct tmc_sg_table *sg_table)
> +{
> + return sg_table->data_pages.nr_pages << PAGE_SHIFT;
> +}
> +
> #endif
> --
> 2.7.4
>
^ permalink raw reply
* [PATCH v7 01/24] soc: qcom dt-bindings: Add APR bus bindings
From: Bjorn Andersson @ 2018-05-04 17:33 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180501120820.11016-2-srinivas.kandagatla@linaro.org>
On Tue 01 May 05:07 PDT 2018, Srinivas Kandagatla wrote:
> This patch add dt bindings for Qualcomm APR (Asynchronous Packet Router)
> bus driver. This bus is used for communicating with DSP which provides
> audio and various other services to cpu.
>
> Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
> Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Regards,
Bjorn
> ---
> .../devicetree/bindings/soc/qcom/qcom,apr.txt | 84 ++++++++++++++++++++++
> include/dt-bindings/soc/qcom,apr.h | 28 ++++++++
> 2 files changed, 112 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/soc/qcom/qcom,apr.txt
> create mode 100644 include/dt-bindings/soc/qcom,apr.h
>
> diff --git a/Documentation/devicetree/bindings/soc/qcom/qcom,apr.txt b/Documentation/devicetree/bindings/soc/qcom/qcom,apr.txt
> new file mode 100644
> index 000000000000..bcc612cc7423
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/soc/qcom/qcom,apr.txt
> @@ -0,0 +1,84 @@
> +Qualcomm APR (Asynchronous Packet Router) binding
> +
> +This binding describes the Qualcomm APR. APR is a IPC protocol for
> +communication between Application processor and QDSP. APR is mainly
> +used for audio/voice services on the QDSP.
> +
> +- compatible:
> + Usage: required
> + Value type: <stringlist>
> + Definition: must be "qcom,apr-v<VERSION-NUMBER>", example "qcom,apr-v2"
> +
> +- reg
> + Usage: required
> + Value type: <u32>
> + Definition: Destination processor ID.
> + Possible values are :
> + 1 - APR simulator
> + 2 - PC
> + 3 - MODEM
> + 4 - ADSP
> + 5 - APPS
> + 6 - MODEM2
> + 7 - APPS2
> +
> += APR SERVICES
> +Each subnode of the APR node represents service tied to this apr. The name
> +of the nodes are not important. The properties of these nodes are defined
> +by the individual bindings for the specific service
> +- All APR services MUST contain the following property:
> +
> +- reg
> + Usage: required
> + Value type: <u32>
> + Definition: APR Service ID
> + Possible values are :
> + 3 - DSP Core Service
> + 4 - Audio Front End Service.
> + 5 - Voice Stream Manager Service.
> + 6 - Voice processing manager.
> + 7 - Audio Stream Manager Service.
> + 8 - Audio Device Manager Service.
> + 9 - Multimode voice manager.
> + 10 - Core voice stream.
> + 11 - Core voice processor.
> + 12 - Ultrasound stream manager.
> + 13 - Listen stream manager.
> +
> += EXAMPLE
> +The following example represents a QDSP based sound card on a MSM8996 device
> +which uses apr as communication between Apps and QDSP.
> +
> + apr at 4 {
> + compatible = "qcom,apr-v2";
> + reg = <APR_DOMAIN_ADSP>;
> +
> + q6core at 3 {
> + compatible = "qcom,q6core";
> + reg = <APR_SVC_ADSP_CORE>;
> + };
> +
> + q6afe at 4 {
> + compatible = "qcom,q6afe";
> + reg = <APR_SVC_AFE>;
> +
> + dais {
> + #sound-dai-cells = <1>;
> + hdmi at 1 {
> + reg = <1>;
> + };
> + };
> + };
> +
> + q6asm at 7 {
> + compatible = "qcom,q6asm";
> + reg = <APR_SVC_ASM>;
> + ...
> + };
> +
> + q6adm at 8 {
> + compatible = "qcom,q6adm";
> + reg = <APR_SVC_ADM>;
> + ...
> + };
> + };
> diff --git a/include/dt-bindings/soc/qcom,apr.h b/include/dt-bindings/soc/qcom,apr.h
> new file mode 100644
> index 000000000000..006362400c0f
> --- /dev/null
> +++ b/include/dt-bindings/soc/qcom,apr.h
> @@ -0,0 +1,28 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __DT_BINDINGS_QCOM_APR_H
> +#define __DT_BINDINGS_QCOM_APR_H
> +
> +/* Domain IDs */
> +#define APR_DOMAIN_SIM 0x1
> +#define APR_DOMAIN_PC 0x2
> +#define APR_DOMAIN_MODEM 0x3
> +#define APR_DOMAIN_ADSP 0x4
> +#define APR_DOMAIN_APPS 0x5
> +#define APR_DOMAIN_MAX 0x6
> +
> +/* ADSP service IDs */
> +#define APR_SVC_ADSP_CORE 0x3
> +#define APR_SVC_AFE 0x4
> +#define APR_SVC_VSM 0x5
> +#define APR_SVC_VPM 0x6
> +#define APR_SVC_ASM 0x7
> +#define APR_SVC_ADM 0x8
> +#define APR_SVC_ADSP_MVM 0x09
> +#define APR_SVC_ADSP_CVS 0x0A
> +#define APR_SVC_ADSP_CVP 0x0B
> +#define APR_SVC_USM 0x0C
> +#define APR_SVC_LSM 0x0D
> +#define APR_SVC_VIDC 0x16
> +#define APR_SVC_MAX 0x17
> +
> +#endif /* __DT_BINDINGS_QCOM_APR_H */
> --
> 2.16.2
>
^ permalink raw reply
* [PATCH net-next v2 02/13] net: phy: sfp: handle non-wired SFP connectors
From: Antoine Tenart @ 2018-05-04 17:23 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <e338472c-1f26-e49b-9a2d-7942e6ed4288@gmail.com>
Hi Florian,
On Fri, May 04, 2018 at 10:04:48AM -0700, Florian Fainelli wrote:
> On 05/04/2018 06:56 AM, Antoine Tenart wrote:
> > SFP connectors can be solder on a board without having any of their pins
> > (LOS, i2c...) wired. In such cases the SFP link state cannot be guessed,
> > and the overall link status reporting is left to other layers.
> >
> > In order to achieve this, a new SFP_DEV status is added, named UNKNOWN.
> > This mode is set when it is not possible for the SFP code to get the
> > link status and as a result the link status is reported to be always UP
> > from the SFP point of view.
>
> Why represent the SFP in Device Tree then? Why not just declare this is
> a fixed link which would avoid having to introduce this "unknown" state.
The other solution would have been to represent this as a fixed-link.
But such a representation would report the link as being up all the
time, which is something we wanted to avoid as the GoP in PPv2 can
report some link status. This is achieved using SFP+phylink+PPv2.
And representing the SFP cage in the device tree, although it's a
"dummy" one, helps describing the hardware.
Thanks!
Antoine
--
Antoine T?nart, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply
* [PATCH 2/2] KVM: ARM: VGIC/ITS: Promote irq_lock() in update_affinity
From: Andre Przywara @ 2018-05-04 17:19 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180504171945.22118-1-andre.przywara@arm.com>
Apparently the development of update_affinity() overlapped with the
promotion of irq_lock to be _irqsave, so the patch didn't convert this
lock over. This will make lockdep complain.
Fix this disabling IRQs around the lock.
Cc: stable at vger.kernel.org
Fixes: 08c9fd042117 ("KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI")
Reported-by: Jan Glauber <jan.glauber@caviumnetworks.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
virt/kvm/arm/vgic/vgic-its.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 41abf92f2699..51a80b600632 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -350,10 +350,11 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
{
int ret = 0;
+ unsigned long flags;
- spin_lock(&irq->irq_lock);
+ spin_lock_irqsave(&irq->irq_lock, flags);
irq->target_vcpu = vcpu;
- spin_unlock(&irq->irq_lock);
+ spin_unlock_irqrestore(&irq->irq_lock, flags);
if (irq->hw) {
struct its_vlpi_map map;
--
2.14.1
^ permalink raw reply related
* [PATCH 1/2] KVM: ARM: Properly protect VGIC locks from IRQs
From: Andre Przywara @ 2018-05-04 17:19 UTC (permalink / raw)
To: linux-arm-kernel
As Jan reported [1], lockdep complains about the VGIC not being bullet
proof. This seems to be due to two issues:
- When commit 006df0f34930 ("KVM: arm/arm64: Support calling
vgic_update_irq_pending from irq context") promoted irq_lock and
ap_list_lock to _irqsave, we forgot two instances of irq_lock.
lockdeps seems to pick those up.
- If a lock is _irqsave, any other locks we take inside them should be
_irqsafe as well. So the lpi_list_lock needs to be promoted also.
This fixes both issues by simply making the remaining instances of those
locks _irqsave.
One irq_lock is addressed in a separate patch, to simplify backporting.
Cc: stable at vger.kernel.org
Fixes: 006df0f34930 ("KVM: arm/arm64: Support calling vgic_update_irq_pending from irq context")
Reported-by: Jan Glauber <jan.glauber@caviumnetworks.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2018-May/575718.html
---
virt/kvm/arm/vgic/vgic-debug.c | 5 +++--
virt/kvm/arm/vgic/vgic-its.c | 10 ++++++----
virt/kvm/arm/vgic/vgic.c | 22 ++++++++++++++--------
3 files changed, 23 insertions(+), 14 deletions(-)
diff --git a/virt/kvm/arm/vgic/vgic-debug.c b/virt/kvm/arm/vgic/vgic-debug.c
index 10b38178cff2..4ffc0b5e6105 100644
--- a/virt/kvm/arm/vgic/vgic-debug.c
+++ b/virt/kvm/arm/vgic/vgic-debug.c
@@ -211,6 +211,7 @@ static int vgic_debug_show(struct seq_file *s, void *v)
struct vgic_state_iter *iter = (struct vgic_state_iter *)v;
struct vgic_irq *irq;
struct kvm_vcpu *vcpu = NULL;
+ unsigned long flags;
if (iter->dist_id == 0) {
print_dist_state(s, &kvm->arch.vgic);
@@ -227,9 +228,9 @@ static int vgic_debug_show(struct seq_file *s, void *v)
irq = &kvm->arch.vgic.spis[iter->intid - VGIC_NR_PRIVATE_IRQS];
}
- spin_lock(&irq->irq_lock);
+ spin_lock_irqsave(&irq->irq_lock, flags);
print_irq_state(s, irq, vcpu);
- spin_unlock(&irq->irq_lock);
+ spin_unlock_irqrestore(&irq->irq_lock, flags);
return 0;
}
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index a8f07243aa9f..41abf92f2699 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -52,6 +52,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
{
struct vgic_dist *dist = &kvm->arch.vgic;
struct vgic_irq *irq = vgic_get_irq(kvm, NULL, intid), *oldirq;
+ unsigned long flags;
int ret;
/* In this case there is no put, since we keep the reference. */
@@ -71,7 +72,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
irq->intid = intid;
irq->target_vcpu = vcpu;
- spin_lock(&dist->lpi_list_lock);
+ spin_lock_irqsave(&dist->lpi_list_lock, flags);
/*
* There could be a race with another vgic_add_lpi(), so we need to
@@ -99,7 +100,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
dist->lpi_list_count++;
out_unlock:
- spin_unlock(&dist->lpi_list_lock);
+ spin_unlock_irqrestore(&dist->lpi_list_lock, flags);
/*
* We "cache" the configuration table entries in our struct vgic_irq's.
@@ -315,6 +316,7 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
{
struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
struct vgic_irq *irq;
+ unsigned long flags;
u32 *intids;
int irq_count, i = 0;
@@ -330,7 +332,7 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
if (!intids)
return -ENOMEM;
- spin_lock(&dist->lpi_list_lock);
+ spin_lock_irqsave(&dist->lpi_list_lock, flags);
list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) {
if (i == irq_count)
break;
@@ -339,7 +341,7 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
continue;
intids[i++] = irq->intid;
}
- spin_unlock(&dist->lpi_list_lock);
+ spin_unlock_irqrestore(&dist->lpi_list_lock, flags);
*intid_ptr = intids;
return i;
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index c62ecca24579..f7dc2347395b 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -43,9 +43,13 @@ struct vgic_global kvm_vgic_global_state __ro_after_init = {
* kvm->lock (mutex)
* its->cmd_lock (mutex)
* its->its_lock (mutex)
- * vgic_cpu->ap_list_lock
- * kvm->lpi_list_lock
- * vgic_irq->irq_lock
+ * vgic_cpu->ap_list_lock must be taken with IRQs disabled
+ * kvm->lpi_list_lock must be taken with IRQs disabled
+ * vgic_irq->irq_lock must be taken with IRQs disabled
+ *
+ * As the ap_list_lock might be taken from the timer interrupt handler,
+ * we have to disable IRQs before taking this lock and everything lower
+ * than it.
*
* If you need to take multiple locks, always take the upper lock first,
* then the lower ones, e.g. first take the its_lock, then the irq_lock.
@@ -72,8 +76,9 @@ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid)
{
struct vgic_dist *dist = &kvm->arch.vgic;
struct vgic_irq *irq = NULL;
+ unsigned long flags;
- spin_lock(&dist->lpi_list_lock);
+ spin_lock_irqsave(&dist->lpi_list_lock, flags);
list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) {
if (irq->intid != intid)
@@ -89,7 +94,7 @@ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid)
irq = NULL;
out_unlock:
- spin_unlock(&dist->lpi_list_lock);
+ spin_unlock_irqrestore(&dist->lpi_list_lock, flags);
return irq;
}
@@ -134,19 +139,20 @@ static void vgic_irq_release(struct kref *ref)
void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq)
{
struct vgic_dist *dist = &kvm->arch.vgic;
+ unsigned long flags;
if (irq->intid < VGIC_MIN_LPI)
return;
- spin_lock(&dist->lpi_list_lock);
+ spin_lock_irqsave(&dist->lpi_list_lock, flags);
if (!kref_put(&irq->refcount, vgic_irq_release)) {
- spin_unlock(&dist->lpi_list_lock);
+ spin_unlock_irqrestore(&dist->lpi_list_lock, flags);
return;
};
list_del(&irq->lpi_list);
dist->lpi_list_count--;
- spin_unlock(&dist->lpi_list_lock);
+ spin_unlock_irqrestore(&dist->lpi_list_lock, flags);
kfree(irq);
}
--
2.14.1
^ permalink raw reply related
* [PATCH net-next v2 01/13] net: phy: sfp: make the i2c-bus property really optional
From: Antoine Tenart @ 2018-05-04 17:19 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <fd91699d-c290-8821-b4a6-0789071cfba1@gmail.com>
Hi Florian,
On Fri, May 04, 2018 at 10:03:16AM -0700, Florian Fainelli wrote:
> On 05/04/2018 06:56 AM, Antoine Tenart wrote:
> >
> > static int sfp_read(struct sfp *sfp, bool a2, u8 addr, void *buf, size_t len)
> > {
> > + if (!sfp->read)
> > + return -EOPNOTSUPP;
>
> -ENODEV would be closer to the intended meaning IMHO, those this could
> be argue that this is yet another color to paint the bikeshed with.
I thought about -ENODEV as well, but ended up choosing -EOPNOTSUPP for
some reason. But I'm really fine with both solutions, it really depends
on if we want to return a callback isn't available from a s/w point of
view (-EOPNOTSUPP) or a h/w point of view (-ENODEV).
> > ret = sfp_read(sfp, false, 0, &id, sizeof(id));
> > + if (ret == -EOPNOTSUPP)
> > + return ret;
>
> Can you find a way such that only sfp_sm_mod_probe() needs to check
> whether the sfp read/write operations returned failure and then we just
> make sure the SFP state machine does not make any more progress? Having
> to check the sfp_read()/sfp_write() operations all over the place sounds
> error prone and won't scale in the future.
I tried doing this in this way (only having logic in the probe
function), but that wasn't as simple as this solution and it seemed
quite invasive as these read/write calls can be called from a few
functions but many code paths (as it's a state machine). So I choose the
easiest solution to maintain in the long run, as each future state
machine update could impact this.
Thanks!
Antoine
--
Antoine T?nart, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply
* [PATCH net-next v2 02/13] net: phy: sfp: handle non-wired SFP connectors
From: Andrew Lunn @ 2018-05-04 17:17 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <e338472c-1f26-e49b-9a2d-7942e6ed4288@gmail.com>
On Fri, May 04, 2018 at 10:04:48AM -0700, Florian Fainelli wrote:
> On 05/04/2018 06:56 AM, Antoine Tenart wrote:
> > SFP connectors can be solder on a board without having any of their pins
> > (LOS, i2c...) wired. In such cases the SFP link state cannot be guessed,
> > and the overall link status reporting is left to other layers.
> >
> > In order to achieve this, a new SFP_DEV status is added, named UNKNOWN.
> > This mode is set when it is not possible for the SFP code to get the
> > link status and as a result the link status is reported to be always UP
> > from the SFP point of view.
>
> Why represent the SFP in Device Tree then? Why not just declare this is
> a fixed link which would avoid having to introduce this "unknown" state.
Hi Antoine
I agree with Florian here.
It LOS was missing, but i2c worked, i could see some value in using
SFP, or order to be able to read the EEPROM. But if everything is
missing, fixed-link seems a better fit.
Andrew
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox