* [PATCH 0/2] IOMMU fault callback support
From: David Cohen @ 2011-02-15 13:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1297776033-8579-1-git-send-email-dacohen@gmail.com>
A missing prefix in the cover letter's subject. It's: OMAP: IOMMU:
Br,
David
On Tue, Feb 15, 2011 at 3:20 PM, David Cohen <dacohen@gmail.com> wrote:
> Hi,
>
> This patch set adds fault callback support to allow IOMMU users to debug or
> react when a fault happens.
> IOMMU faults might be very difficult to reproduce and then to figure out
> the source of the problem. Currently IOMMU driver prints not so useful
> debug message and does not notice user about such issue.
> With a fault callback, IOMMU user may debug much more useful information
> and/or react to go back to a valid state.
>
> Br,
>
> David
> ---
>
> David Cohen (2):
> ?OMAP2+: IOMMU: change OMAP2+ error message to dev_dbg()
> ?OMAP: IOMMU: add support to callback during fault handling
>
> ?arch/arm/mach-omap2/iommu2.c ? ? ? ? ? ?| ? 27 ++++++++++++++++----
> ?arch/arm/plat-omap/include/plat/iommu.h | ? 15 ++++++++++-
> ?arch/arm/plat-omap/iommu.c ? ? ? ? ? ? ?| ? 41 ++++++++++++++++++++++++++++---
> ?3 files changed, 72 insertions(+), 11 deletions(-)
>
> --
> 1.7.2.3
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-omap" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at ?http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* [PATCH] picoxcell_crypto: add support for the picoxcell crypto engines
From: Jamie Iles @ 2011-02-15 13:32 UTC (permalink / raw)
To: linux-arm-kernel
Picochip picoXcell devices have two crypto engines, one targeted
at IPSEC offload and the other at WCDMA layer 2 ciphering.
v2: - use symbolic constants for key sizes etc
- cleanup DDT generation
- use strict_strtoul() for stat irq threshold
- minor code cleanups
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
---
Thanks to Kim Phillips for some great review comments!
drivers/crypto/Kconfig | 17 +
drivers/crypto/Makefile | 2 +-
drivers/crypto/picoxcell_crypto.c | 1867 ++++++++++++++++++++++++++++++++
drivers/crypto/picoxcell_crypto_regs.h | 128 +++
4 files changed, 2013 insertions(+), 1 deletions(-)
create mode 100644 drivers/crypto/picoxcell_crypto.c
create mode 100644 drivers/crypto/picoxcell_crypto_regs.h
diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index eab2cf7..e541852 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig
@@ -252,4 +252,21 @@ config CRYPTO_DEV_OMAP_AES
OMAP processors have AES module accelerator. Select this if you
want to use the OMAP module for AES algorithms.
+config CRYPTO_DEV_PICOXCELL
+ tristate "Support for picoXcell IPSEC and Layer2 crypto engines"
+ depends on ARCH_PICOXCELL
+ select CRYPTO_AES
+ select CRYPTO_AUTHENC
+ select CRYPTO_ALGAPI
+ select CRYPTO_DES
+ select CRYPTO_CBC
+ select CRYPTO_ECB
+ select CRYPTO_SEQIV
+ help
+ This option enables support for the hardware offload engines in the
+ Picochip picoXcell SoC devices. Select this for IPSEC ESP offload
+ and for 3gpp Layer 2 ciphering support.
+
+ Saying m here will build a module named pipcoxcell_crypto.
+
endif # CRYPTO_HW
diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile
index 2566973..5203e34 100644
--- a/drivers/crypto/Makefile
+++ b/drivers/crypto/Makefile
@@ -10,4 +10,4 @@ obj-$(CONFIG_CRYPTO_DEV_IXP4XX) += ixp4xx_crypto.o
obj-$(CONFIG_CRYPTO_DEV_PPC4XX) += amcc/
obj-$(CONFIG_CRYPTO_DEV_OMAP_SHAM) += omap-sham.o
obj-$(CONFIG_CRYPTO_DEV_OMAP_AES) += omap-aes.o
-
+obj-$(CONFIG_CRYPTO_DEV_PICOXCELL) += picoxcell_crypto.o
diff --git a/drivers/crypto/picoxcell_crypto.c b/drivers/crypto/picoxcell_crypto.c
new file mode 100644
index 0000000..b092d0a
--- /dev/null
+++ b/drivers/crypto/picoxcell_crypto.c
@@ -0,0 +1,1867 @@
+/*
+ * Copyright (c) 2010-2011 Picochip Ltd., Jamie Iles
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+#include <crypto/aead.h>
+#include <crypto/aes.h>
+#include <crypto/algapi.h>
+#include <crypto/authenc.h>
+#include <crypto/des.h>
+#include <crypto/md5.h>
+#include <crypto/sha.h>
+#include <crypto/internal/skcipher.h>
+#include <linux/clk.h>
+#include <linux/crypto.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/dmapool.h>
+#include <linux/err.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm.h>
+#include <linux/rtnetlink.h>
+#include <linux/scatterlist.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/timer.h>
+
+#include "picoxcell_crypto_regs.h"
+
+/*
+ * The threshold for the number of entries in the CMD FIFO available before
+ * the CMD0_CNT interrupt is raised. Increasing this value will reduce the
+ * number of interrupts raised to the CPU.
+ */
+#define CMD0_IRQ_THRESHOLD 1
+
+/*
+ * The timeout period (in jiffies) for a PDU. When the the number of PDUs in
+ * flight is greater than the STAT_IRQ_THRESHOLD or 0 the timer is disabled.
+ * When there are packets in flight but lower than the threshold, we enable
+ * the timer and at expiry, attempt to remove any processed packets from the
+ * queue and if there are still packets left, schedule the timer again.
+ */
+#define PACKET_TIMEOUT 1
+
+/* The priority to register each algorithm with. */
+#define SPACC_CRYPTO_ALG_PRIORITY 10000
+
+#define SPACC_CRYPTO_KASUMI_F8_KEY_LEN 16
+#define SPACC_CRYPTO_IPSEC_CIPHER_PG_SZ 64
+#define SPACC_CRYPTO_IPSEC_HASH_PG_SZ 64
+#define SPACC_CRYPTO_IPSEC_MAX_CTXS 32
+#define SPACC_CRYPTO_IPSEC_FIFO_SZ 32
+#define SPACC_CRYPTO_L2_CIPHER_PG_SZ 64
+#define SPACC_CRYPTO_L2_HASH_PG_SZ 64
+#define SPACC_CRYPTO_L2_MAX_CTXS 128
+#define SPACC_CRYPTO_L2_FIFO_SZ 128
+
+#define MAX_DDT_LEN 16
+
+/* DDT format. This must match the hardware DDT format exactly. */
+struct spacc_ddt {
+ dma_addr_t p;
+ u32 len;
+};
+
+/*
+ * Asynchronous crypto request structure.
+ *
+ * This structure defines a request that is either queued for processing or
+ * being processed.
+ */
+struct spacc_req {
+ struct list_head list;
+ struct spacc_engine *engine;
+ struct crypto_async_request *req;
+ int result;
+ bool is_encrypt;
+ unsigned ctx_id;
+ dma_addr_t src_addr, dst_addr;
+ struct spacc_ddt *src_ddt, *dst_ddt;
+ void (*complete)(struct spacc_req *req);
+
+ /* AEAD specific bits. */
+ u8 *giv;
+ size_t giv_len;
+ dma_addr_t giv_pa;
+};
+
+struct spacc_engine {
+ void __iomem *regs;
+ struct list_head pending;
+ int next_ctx;
+ spinlock_t hw_lock;
+ int in_flight;
+ struct list_head completed;
+ struct list_head in_progress;
+ struct tasklet_struct complete;
+ unsigned long fifo_sz;
+ void __iomem *cipher_ctx_base;
+ void __iomem *hash_key_base;
+ struct spacc_alg *algs;
+ unsigned num_algs;
+ struct list_head registered_algs;
+ size_t cipher_pg_sz;
+ size_t hash_pg_sz;
+ const char *name;
+ struct clk *clk;
+ struct device *dev;
+ unsigned max_ctxs;
+ struct timer_list packet_timeout;
+ unsigned stat_irq_thresh;
+ struct dma_pool *req_pool;
+};
+
+/* Algorithm type mask. */
+#define SPACC_CRYPTO_ALG_MASK 0x7
+
+/* SPACC definition of a crypto algorithm. */
+struct spacc_alg {
+ unsigned long ctrl_default;
+ unsigned long type;
+ struct crypto_alg alg;
+ struct spacc_engine *engine;
+ struct list_head entry;
+ int key_offs;
+ int iv_offs;
+};
+
+/* Generic context structure for any algorithm type. */
+struct spacc_generic_ctx {
+ struct spacc_engine *engine;
+ int flags;
+ int key_offs;
+ int iv_offs;
+};
+
+/* Block cipher context. */
+struct spacc_ablk_ctx {
+ struct spacc_generic_ctx generic;
+ u8 key[AES_MAX_KEY_SIZE];
+ u8 key_len;
+ /*
+ * The fallback cipher. If the operation can't be done in hardware,
+ * fallback to a software version.
+ */
+ struct crypto_ablkcipher *sw_cipher;
+};
+
+/* AEAD cipher context. */
+struct spacc_aead_ctx {
+ struct spacc_generic_ctx generic;
+ u8 cipher_key[AES_MAX_KEY_SIZE];
+ u8 hash_ctx[SPACC_CRYPTO_IPSEC_HASH_PG_SZ];
+ u8 cipher_key_len;
+ u8 hash_key_len;
+ struct crypto_aead *sw_cipher;
+ size_t auth_size;
+ u8 salt[AES_BLOCK_SIZE];
+};
+
+static inline struct spacc_alg *to_spacc_alg(struct crypto_alg *alg)
+{
+ return alg ? container_of(alg, struct spacc_alg, alg) : NULL;
+}
+
+static inline int spacc_fifo_cmd_full(struct spacc_engine *engine)
+{
+ u32 fifo_stat = readl(engine->regs + SPA_FIFO_STAT_REG_OFFSET);
+
+ return fifo_stat & SPA_FIFO_CMD_FULL;
+}
+
+/*
+ * Given a cipher context, and a context number, get the base address of the
+ * context page.
+ *
+ * Returns the address of the context page where the key/context may
+ * be written.
+ */
+static inline void __iomem *spacc_ctx_page_addr(struct spacc_generic_ctx *ctx,
+ unsigned indx,
+ bool is_cipher_ctx)
+{
+ return is_cipher_ctx ? ctx->engine->cipher_ctx_base +
+ (indx * ctx->engine->cipher_pg_sz) :
+ ctx->engine->hash_key_base + (indx * ctx->engine->hash_pg_sz);
+}
+
+/* The context pages can only be written with 32-bit accesses. */
+static inline void memcpy_toio32(u32 __iomem *dst, const void *src,
+ unsigned count)
+{
+ const u32 *src32 = (const u32 *) src;
+
+ while (count--)
+ writel(*src32++, dst++);
+}
+
+static void spacc_cipher_write_ctx(struct spacc_generic_ctx *ctx,
+ void __iomem *page_addr, const u8 *key,
+ size_t key_len, const u8 *iv, size_t iv_len)
+{
+ void __iomem *key_ptr = page_addr + ctx->key_offs;
+ void __iomem *iv_ptr = page_addr + ctx->iv_offs;
+
+ memcpy_toio32(key_ptr, key, key_len / 4);
+ memcpy_toio32(iv_ptr, iv, iv_len / 4);
+}
+
+/*
+ * Load a context into the engines context memory.
+ *
+ * Returns the index of the context page where the context was loaded.
+ */
+static unsigned spacc_load_ctx(struct spacc_generic_ctx *ctx,
+ const u8 *ciph_key, size_t ciph_len,
+ const u8 *iv, size_t ivlen, const u8 *hash_key,
+ size_t hash_len)
+{
+ unsigned indx = ctx->engine->next_ctx++;
+ void __iomem *ciph_page_addr, *hash_page_addr;
+
+ ciph_page_addr = spacc_ctx_page_addr(ctx, indx, 1);
+ hash_page_addr = spacc_ctx_page_addr(ctx, indx, 0);
+
+ ctx->engine->next_ctx &= ctx->engine->fifo_sz - 1;
+ spacc_cipher_write_ctx(ctx, ciph_page_addr, ciph_key, ciph_len, iv,
+ ivlen);
+ writel(ciph_len | (indx << SPA_KEY_SZ_CTX_INDEX_OFFSET) |
+ (1 << SPA_KEY_SZ_CIPHER_OFFSET),
+ ctx->engine->regs + SPA_KEY_SZ_REG_OFFSET);
+
+ if (hash_key) {
+ memcpy_toio32(hash_page_addr, hash_key, hash_len / 4);
+ writel(hash_len | (indx << SPA_KEY_SZ_CTX_INDEX_OFFSET),
+ ctx->engine->regs + SPA_KEY_SZ_REG_OFFSET);
+ }
+
+ return indx;
+}
+
+/* Count the number of scatterlist entries in a scatterlist. */
+static int sg_count(struct scatterlist *sg_list, int nbytes)
+{
+ struct scatterlist *sg = sg_list;
+ int sg_nents = 0;
+
+ while (nbytes > 0) {
+ ++sg_nents;
+ nbytes -= sg->length;
+ sg = sg_next(sg);
+ }
+
+ return sg_nents;
+}
+
+static inline void ddt_set(struct spacc_ddt *ddt, dma_addr_t phys, size_t len)
+{
+ ddt->p = phys;
+ ddt->len = len;
+}
+
+/*
+ * Take a crypto request and scatterlists for the data and turn them into DDTs
+ * for passing to the crypto engines. This also DMA maps the data so that the
+ * crypto engines can DMA to/from them.
+ */
+static struct spacc_ddt *spacc_sg_to_ddt(struct spacc_engine *engine,
+ struct scatterlist *payload,
+ unsigned nbytes,
+ enum dma_data_direction dir,
+ dma_addr_t *ddt_phys)
+{
+ unsigned nents, mapped_ents;
+ struct scatterlist *cur;
+ struct spacc_ddt *ddt;
+ int i;
+
+ nents = sg_count(payload, nbytes);
+ mapped_ents = dma_map_sg(engine->dev, payload, nents, dir);
+
+ if (mapped_ents + 1 > MAX_DDT_LEN)
+ goto out;
+
+ ddt = dma_pool_alloc(engine->req_pool, GFP_ATOMIC, ddt_phys);
+ if (!ddt)
+ goto out;
+
+ for_each_sg(payload, cur, mapped_ents, i)
+ ddt_set(&ddt[i], sg_dma_address(cur), sg_dma_len(cur));
+ ddt_set(&ddt[mapped_ents], 0, 0);
+
+ return ddt;
+
+out:
+ dma_unmap_sg(engine->dev, payload, nents, dir);
+ return NULL;
+}
+
+static int spacc_aead_make_ddts(struct spacc_req *req, u8 *giv)
+{
+ struct aead_request *areq = container_of(req->req, struct aead_request,
+ base);
+ struct spacc_engine *engine = req->engine;
+ struct spacc_ddt *src_ddt, *dst_ddt;
+ unsigned ivsize = crypto_aead_ivsize(crypto_aead_reqtfm(areq));
+ unsigned nents = sg_count(areq->src, areq->cryptlen);
+ dma_addr_t iv_addr;
+ struct scatterlist *cur;
+ int i, dst_ents, src_ents, assoc_ents;
+ u8 *iv = giv ? giv : areq->iv;
+
+ src_ddt = dma_pool_alloc(engine->req_pool, GFP_ATOMIC, &req->src_addr);
+ if (!src_ddt)
+ return -ENOMEM;
+
+ dst_ddt = dma_pool_alloc(engine->req_pool, GFP_ATOMIC, &req->dst_addr);
+ if (!dst_ddt) {
+ dma_pool_free(engine->req_pool, src_ddt, req->src_addr);
+ return -ENOMEM;
+ }
+
+ req->src_ddt = src_ddt;
+ req->dst_ddt = dst_ddt;
+
+ assoc_ents = dma_map_sg(engine->dev, areq->assoc,
+ sg_count(areq->assoc, areq->assoclen), DMA_TO_DEVICE);
+ if (areq->src != areq->dst) {
+ src_ents = dma_map_sg(engine->dev, areq->src, nents,
+ DMA_TO_DEVICE);
+ dst_ents = dma_map_sg(engine->dev, areq->dst, nents,
+ DMA_FROM_DEVICE);
+ } else {
+ src_ents = dma_map_sg(engine->dev, areq->src, nents,
+ DMA_BIDIRECTIONAL);
+ dst_ents = 0;
+ }
+
+ /*
+ * Map the IV/GIV. For the GIV it needs to be bidirectional as it is
+ * formed by the crypto block and sent as the ESP IV for IPSEC.
+ */
+ iv_addr = dma_map_single(engine->dev, iv, ivsize,
+ giv ? DMA_BIDIRECTIONAL : DMA_TO_DEVICE);
+ req->giv_pa = iv_addr;
+
+ /*
+ * Map the associated data. For decryption we don't copy the
+ * associated data.
+ */
+ for_each_sg(areq->assoc, cur, assoc_ents, i) {
+ ddt_set(src_ddt++, sg_dma_address(cur), sg_dma_len(cur));
+ if (req->is_encrypt)
+ ddt_set(dst_ddt++, sg_dma_address(cur),
+ sg_dma_len(cur));
+ }
+ ddt_set(src_ddt++, iv_addr, ivsize);
+
+ if (giv || req->is_encrypt)
+ ddt_set(dst_ddt++, iv_addr, ivsize);
+
+ /*
+ * Now map in the payload for the source and destination and terminate
+ * with the NULL pointers.
+ */
+ for_each_sg(areq->src, cur, src_ents, i) {
+ ddt_set(src_ddt++, sg_dma_address(cur), sg_dma_len(cur));
+ if (areq->src == areq->dst)
+ ddt_set(dst_ddt++, sg_dma_address(cur),
+ sg_dma_len(cur));
+ }
+
+ for_each_sg(areq->dst, cur, dst_ents, i)
+ ddt_set(dst_ddt++, sg_dma_address(cur),
+ sg_dma_len(cur));
+
+ ddt_set(src_ddt, 0, 0);
+ ddt_set(dst_ddt, 0, 0);
+
+ return 0;
+}
+
+static void spacc_aead_free_ddts(struct spacc_req *req)
+{
+ struct aead_request *areq = container_of(req->req, struct aead_request,
+ base);
+ struct spacc_alg *alg = to_spacc_alg(req->req->tfm->__crt_alg);
+ struct spacc_ablk_ctx *aead_ctx = crypto_tfm_ctx(req->req->tfm);
+ struct spacc_engine *engine = aead_ctx->generic.engine;
+ unsigned ivsize = alg->alg.cra_aead.ivsize;
+ unsigned nents = sg_count(areq->src, areq->cryptlen);
+
+ if (areq->src != areq->dst) {
+ dma_unmap_sg(engine->dev, areq->src, nents, DMA_TO_DEVICE);
+ dma_unmap_sg(engine->dev, areq->dst,
+ sg_count(areq->dst, areq->cryptlen),
+ DMA_FROM_DEVICE);
+ } else
+ dma_unmap_sg(engine->dev, areq->src, nents, DMA_BIDIRECTIONAL);
+
+ dma_unmap_sg(engine->dev, areq->assoc,
+ sg_count(areq->assoc, areq->assoclen), DMA_TO_DEVICE);
+
+ dma_unmap_single(engine->dev, req->giv_pa, ivsize, DMA_BIDIRECTIONAL);
+
+ dma_pool_free(engine->req_pool, req->src_ddt, req->src_addr);
+ dma_pool_free(engine->req_pool, req->dst_ddt, req->dst_addr);
+}
+
+static void spacc_free_ddt(struct spacc_req *req, struct spacc_ddt *ddt,
+ dma_addr_t ddt_addr, struct scatterlist *payload,
+ unsigned nbytes, enum dma_data_direction dir)
+{
+ unsigned nents = sg_count(payload, nbytes);
+
+ dma_unmap_sg(req->engine->dev, payload, nents, dir);
+ dma_pool_free(req->engine->req_pool, ddt, ddt_addr);
+}
+
+/*
+ * Set key for a DES operation in an AEAD cipher. This also performs weak key
+ * checking if required.
+ */
+static int spacc_aead_des_setkey(struct crypto_aead *aead, const u8 *key,
+ unsigned int len)
+{
+ struct crypto_tfm *tfm = crypto_aead_tfm(aead);
+ struct spacc_aead_ctx *ctx = crypto_tfm_ctx(tfm);
+ u32 tmp[DES_EXPKEY_WORDS];
+
+ if (unlikely(!des_ekey(tmp, key)) &&
+ (crypto_aead_get_flags(aead)) & CRYPTO_TFM_REQ_WEAK_KEY) {
+ tfm->crt_flags |= CRYPTO_TFM_RES_WEAK_KEY;
+ return -EINVAL;
+ }
+
+ memcpy(ctx->cipher_key, key, len);
+ ctx->cipher_key_len = len;
+
+ return 0;
+}
+
+/* Set the key for the AES block cipher component of the AEAD transform. */
+static int spacc_aead_aes_setkey(struct crypto_aead *aead, const u8 *key,
+ unsigned int len)
+{
+ struct crypto_tfm *tfm = crypto_aead_tfm(aead);
+ struct spacc_aead_ctx *ctx = crypto_tfm_ctx(tfm);
+
+ /*
+ * IPSec engine only supports 128 and 256 bit AES keys. If we get a
+ * request for any other size (192 bits) then we need to do a software
+ * fallback.
+ */
+ if (len != AES_KEYSIZE_128 && len != AES_KEYSIZE_256) {
+ /*
+ * Set the fallback transform to use the same request flags as
+ * the hardware transform.
+ */
+ ctx->sw_cipher->base.crt_flags &= ~CRYPTO_TFM_REQ_MASK;
+ ctx->sw_cipher->base.crt_flags |=
+ tfm->crt_flags & CRYPTO_TFM_REQ_MASK;
+ return crypto_aead_setkey(ctx->sw_cipher, key, len);
+ }
+
+ memcpy(ctx->cipher_key, key, len);
+ ctx->cipher_key_len = len;
+
+ return 0;
+}
+
+static int spacc_aead_setkey(struct crypto_aead *tfm, const u8 *key,
+ unsigned int keylen)
+{
+ struct spacc_aead_ctx *ctx = crypto_aead_ctx(tfm);
+ struct spacc_alg *alg = to_spacc_alg(tfm->base.__crt_alg);
+ struct rtattr *rta = (void *)key;
+ struct crypto_authenc_key_param *param;
+ unsigned int authkeylen, enckeylen;
+ int err = -EINVAL;
+
+ if (!RTA_OK(rta, keylen))
+ goto badkey;
+
+ if (rta->rta_type != CRYPTO_AUTHENC_KEYA_PARAM)
+ goto badkey;
+
+ if (RTA_PAYLOAD(rta) < sizeof(*param))
+ goto badkey;
+
+ param = RTA_DATA(rta);
+ enckeylen = be32_to_cpu(param->enckeylen);
+
+ key += RTA_ALIGN(rta->rta_len);
+ keylen -= RTA_ALIGN(rta->rta_len);
+
+ if (keylen < enckeylen)
+ goto badkey;
+
+ authkeylen = keylen - enckeylen;
+
+ if (enckeylen > AES_MAX_KEY_SIZE)
+ goto badkey;
+
+ if ((alg->ctrl_default & SPACC_CRYPTO_ALG_MASK) ==
+ SPA_CTRL_CIPH_ALG_AES)
+ err = spacc_aead_aes_setkey(tfm, key + authkeylen, enckeylen);
+ else
+ err = spacc_aead_des_setkey(tfm, key + authkeylen, enckeylen);
+
+ if (err)
+ goto badkey;
+
+ memcpy(ctx->hash_ctx, key, authkeylen);
+ ctx->hash_key_len = authkeylen;
+
+ return 0;
+
+badkey:
+ crypto_aead_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
+ return -EINVAL;
+}
+
+static int spacc_aead_setauthsize(struct crypto_aead *tfm,
+ unsigned int authsize)
+{
+ struct spacc_aead_ctx *ctx = crypto_tfm_ctx(crypto_aead_tfm(tfm));
+
+ ctx->auth_size = authsize;
+
+ return 0;
+}
+
+/*
+ * Check if an AEAD request requires a fallback operation. Some requests can't
+ * be completed in hardware because the hardware may not support certain key
+ * sizes. In these cases we need to complete the request in software.
+ */
+static int spacc_aead_need_fallback(struct spacc_req *req)
+{
+ struct aead_request *aead_req;
+ struct crypto_tfm *tfm = req->req->tfm;
+ struct crypto_alg *alg = req->req->tfm->__crt_alg;
+ struct spacc_alg *spacc_alg = to_spacc_alg(alg);
+ struct spacc_aead_ctx *ctx = crypto_tfm_ctx(tfm);
+
+ aead_req = container_of(req->req, struct aead_request, base);
+ /*
+ * If we have a non-supported key-length, then we need to do a
+ * software fallback.
+ */
+ if ((spacc_alg->ctrl_default & SPACC_CRYPTO_ALG_MASK) ==
+ SPA_CTRL_CIPH_ALG_AES &&
+ ctx->cipher_key_len != AES_KEYSIZE_128 &&
+ ctx->cipher_key_len != AES_KEYSIZE_256)
+ return 1;
+
+ return 0;
+}
+
+static int spacc_aead_do_fallback(struct aead_request *req, unsigned alg_type,
+ bool is_encrypt)
+{
+ struct crypto_tfm *old_tfm = crypto_aead_tfm(crypto_aead_reqtfm(req));
+ struct spacc_aead_ctx *ctx = crypto_tfm_ctx(old_tfm);
+ int err;
+
+ if (ctx->sw_cipher) {
+ /*
+ * Change the request to use the software fallback transform,
+ * and once the ciphering has completed, put the old transform
+ * back into the request.
+ */
+ aead_request_set_tfm(req, ctx->sw_cipher);
+ err = is_encrypt ? crypto_aead_encrypt(req) :
+ crypto_aead_decrypt(req);
+ aead_request_set_tfm(req, __crypto_aead_cast(old_tfm));
+ } else
+ err = -EINVAL;
+
+ return err;
+}
+
+static void spacc_aead_complete(struct spacc_req *req)
+{
+ spacc_aead_free_ddts(req);
+ req->req->complete(req->req, req->result);
+}
+
+static int spacc_aead_submit(struct spacc_req *req)
+{
+ struct crypto_tfm *tfm = req->req->tfm;
+ struct spacc_aead_ctx *ctx = crypto_tfm_ctx(tfm);
+ struct crypto_alg *alg = req->req->tfm->__crt_alg;
+ struct spacc_alg *spacc_alg = to_spacc_alg(alg);
+ struct spacc_engine *engine = ctx->generic.engine;
+ u32 ctrl, proc_len, assoc_len;
+ struct aead_request *aead_req =
+ container_of(req->req, struct aead_request, base);
+
+ req->result = -EINPROGRESS;
+ req->ctx_id = spacc_load_ctx(&ctx->generic, ctx->cipher_key,
+ ctx->cipher_key_len, aead_req->iv, alg->cra_aead.ivsize,
+ ctx->hash_ctx, ctx->hash_key_len);
+
+ /* Set the source and destination DDT pointers. */
+ writel(req->src_addr, engine->regs + SPA_SRC_PTR_REG_OFFSET);
+ writel(req->dst_addr, engine->regs + SPA_DST_PTR_REG_OFFSET);
+ writel(0, engine->regs + SPA_OFFSET_REG_OFFSET);
+
+ assoc_len = aead_req->assoclen;
+ proc_len = aead_req->cryptlen + assoc_len;
+
+ /*
+ * If we aren't generating an IV, then we need to include the IV in the
+ * associated data so that it is included in the hash.
+ */
+ if (!req->giv) {
+ assoc_len += crypto_aead_ivsize(crypto_aead_reqtfm(aead_req));
+ proc_len += crypto_aead_ivsize(crypto_aead_reqtfm(aead_req));
+ } else
+ proc_len += req->giv_len;
+
+ /*
+ * If we are decrypting, we need to take the length of the ICV out of
+ * the processing length.
+ */
+ if (!req->is_encrypt)
+ proc_len -= ctx->auth_size;
+
+ writel(proc_len, engine->regs + SPA_PROC_LEN_REG_OFFSET);
+ writel(assoc_len, engine->regs + SPA_AAD_LEN_REG_OFFSET);
+ writel(ctx->auth_size, engine->regs + SPA_ICV_LEN_REG_OFFSET);
+ writel(0, engine->regs + SPA_ICV_OFFSET_REG_OFFSET);
+ writel(0, engine->regs + SPA_AUX_INFO_REG_OFFSET);
+
+ ctrl = spacc_alg->ctrl_default | (req->ctx_id << SPA_CTRL_CTX_IDX) |
+ (1 << SPA_CTRL_ICV_APPEND);
+ if (req->is_encrypt)
+ ctrl |= (1 << SPA_CTRL_ENCRYPT_IDX) | (1 << SPA_CTRL_AAD_COPY);
+ else
+ ctrl |= (1 << SPA_CTRL_KEY_EXP);
+
+ mod_timer(&engine->packet_timeout, jiffies + PACKET_TIMEOUT);
+
+ writel(ctrl, engine->regs + SPA_CTRL_REG_OFFSET);
+
+ return -EINPROGRESS;
+}
+
+/*
+ * Setup an AEAD request for processing. This will configure the engine, load
+ * the context and then start the packet processing.
+ *
+ * @giv Pointer to destination address for a generated IV. If the
+ * request does not need to generate an IV then this should be set to NULL.
+ */
+static int spacc_aead_setup(struct aead_request *req, u8 *giv,
+ unsigned alg_type, bool is_encrypt)
+{
+ struct crypto_alg *alg = req->base.tfm->__crt_alg;
+ struct spacc_engine *engine = to_spacc_alg(alg)->engine;
+ struct spacc_req *dev_req = aead_request_ctx(req);
+ int err = -EINPROGRESS;
+ unsigned long flags;
+ unsigned ivsize = crypto_aead_ivsize(crypto_aead_reqtfm(req));
+
+ dev_req->giv = giv;
+ dev_req->giv_len = ivsize;
+ dev_req->req = &req->base;
+ dev_req->is_encrypt = is_encrypt;
+ dev_req->result = -EBUSY;
+ dev_req->engine = engine;
+ dev_req->complete = spacc_aead_complete;
+
+ if (unlikely(spacc_aead_need_fallback(dev_req)))
+ return spacc_aead_do_fallback(req, alg_type, is_encrypt);
+
+ spacc_aead_make_ddts(dev_req, dev_req->giv);
+
+ err = -EINPROGRESS;
+ spin_lock_irqsave(&engine->hw_lock, flags);
+ if (unlikely(spacc_fifo_cmd_full(engine))) {
+ if (!(req->base.flags & CRYPTO_TFM_REQ_MAY_BACKLOG)) {
+ err = -EBUSY;
+ spin_unlock_irqrestore(&engine->hw_lock, flags);
+ goto out_free_ddts;
+ }
+ list_add_tail(&dev_req->list, &engine->pending);
+ } else {
+ ++engine->in_flight;
+ list_add_tail(&dev_req->list, &engine->in_progress);
+ spacc_aead_submit(dev_req);
+ }
+ spin_unlock_irqrestore(&engine->hw_lock, flags);
+
+ goto out;
+
+out_free_ddts:
+ spacc_aead_free_ddts(dev_req);
+out:
+ return err;
+}
+
+static int spacc_aead_encrypt(struct aead_request *req)
+{
+ struct crypto_aead *aead = crypto_aead_reqtfm(req);
+ struct crypto_tfm *tfm = crypto_aead_tfm(aead);
+ struct spacc_alg *alg = to_spacc_alg(tfm->__crt_alg);
+
+ return spacc_aead_setup(req, NULL, alg->type, 1);
+}
+
+static int spacc_aead_givencrypt(struct aead_givcrypt_request *req)
+{
+ struct crypto_aead *tfm = aead_givcrypt_reqtfm(req);
+ struct spacc_aead_ctx *ctx = crypto_aead_ctx(tfm);
+ size_t ivsize = crypto_aead_ivsize(tfm);
+ struct spacc_alg *alg = to_spacc_alg(tfm->base.__crt_alg);
+ unsigned len;
+ __be64 seq;
+
+ memcpy(req->areq.iv, ctx->salt, ivsize);
+ len = ivsize;
+ if (ivsize > sizeof(u64)) {
+ memset(req->giv, 0, ivsize - sizeof(u64));
+ len = sizeof(u64);
+ }
+ seq = cpu_to_be64(req->seq);
+ memcpy(req->giv + ivsize - len, &seq, len);
+
+ return spacc_aead_setup(&req->areq, req->giv, alg->type, 1);
+}
+
+static int spacc_aead_decrypt(struct aead_request *req)
+{
+ struct crypto_aead *aead = crypto_aead_reqtfm(req);
+ struct crypto_tfm *tfm = crypto_aead_tfm(aead);
+ struct spacc_alg *alg = to_spacc_alg(tfm->__crt_alg);
+
+ return spacc_aead_setup(req, NULL, alg->type, 0);
+}
+
+/*
+ * Initialise a new AEAD context. This is responsible for allocating the
+ * fallback cipher and initialising the context.
+ */
+static int spacc_aead_cra_init(struct crypto_tfm *tfm)
+{
+ struct spacc_aead_ctx *ctx = crypto_tfm_ctx(tfm);
+ struct crypto_alg *alg = tfm->__crt_alg;
+ struct spacc_alg *spacc_alg = to_spacc_alg(alg);
+ struct spacc_engine *engine = spacc_alg->engine;
+
+ ctx->generic.flags = spacc_alg->type;
+ ctx->generic.engine = engine;
+ ctx->sw_cipher = crypto_alloc_aead(alg->cra_name, 0,
+ CRYPTO_ALG_ASYNC |
+ CRYPTO_ALG_NEED_FALLBACK);
+ if (IS_ERR(ctx->sw_cipher)) {
+ dev_warn(engine->dev, "failed to allocate fallback for %s\n",
+ alg->cra_name);
+ ctx->sw_cipher = NULL;
+ }
+ ctx->generic.key_offs = spacc_alg->key_offs;
+ ctx->generic.iv_offs = spacc_alg->iv_offs;
+
+ get_random_bytes(ctx->salt, sizeof(ctx->salt));
+
+ tfm->crt_aead.reqsize = sizeof(struct spacc_req);
+
+ return 0;
+}
+
+/*
+ * Destructor for an AEAD context. This is called when the transform is freed
+ * and must free the fallback cipher.
+ */
+static void spacc_aead_cra_exit(struct crypto_tfm *tfm)
+{
+ struct spacc_aead_ctx *ctx = crypto_tfm_ctx(tfm);
+
+ if (ctx->sw_cipher)
+ crypto_free_aead(ctx->sw_cipher);
+ ctx->sw_cipher = NULL;
+}
+
+/*
+ * Set the DES key for a block cipher transform. This also performs weak key
+ * checking if the transform has requested it.
+ */
+static int spacc_des_setkey(struct crypto_ablkcipher *cipher, const u8 *key,
+ unsigned int len)
+{
+ struct crypto_tfm *tfm = crypto_ablkcipher_tfm(cipher);
+ struct spacc_ablk_ctx *ctx = crypto_tfm_ctx(tfm);
+ u32 tmp[DES_EXPKEY_WORDS];
+
+ if (len > DES3_EDE_KEY_SIZE) {
+ crypto_ablkcipher_set_flags(cipher, CRYPTO_TFM_RES_BAD_KEY_LEN);
+ return -EINVAL;
+ }
+
+ if (unlikely(!des_ekey(tmp, key)) &&
+ (crypto_ablkcipher_get_flags(cipher) & CRYPTO_TFM_REQ_WEAK_KEY)) {
+ tfm->crt_flags |= CRYPTO_TFM_RES_WEAK_KEY;
+ return -EINVAL;
+ }
+
+ memcpy(ctx->key, key, len);
+ ctx->key_len = len;
+
+ return 0;
+}
+
+/*
+ * Set the key for an AES block cipher. Some key lengths are not supported in
+ * hardware so this must also check whether a fallback is needed.
+ */
+static int spacc_aes_setkey(struct crypto_ablkcipher *cipher, const u8 *key,
+ unsigned int len)
+{
+ struct crypto_tfm *tfm = crypto_ablkcipher_tfm(cipher);
+ struct spacc_ablk_ctx *ctx = crypto_tfm_ctx(tfm);
+ int err = 0;
+
+ if (len > AES_MAX_KEY_SIZE) {
+ crypto_ablkcipher_set_flags(cipher, CRYPTO_TFM_RES_BAD_KEY_LEN);
+ return -EINVAL;
+ }
+
+ /*
+ * IPSec engine only supports 128 and 256 bit AES keys. If we get a
+ * request for any other size (192 bits) then we need to do a software
+ * fallback.
+ */
+ if ((len != AES_KEYSIZE_128 || len != AES_KEYSIZE_256) &&
+ ctx->sw_cipher) {
+ /*
+ * Set the fallback transform to use the same request flags as
+ * the hardware transform.
+ */
+ ctx->sw_cipher->base.crt_flags &= ~CRYPTO_TFM_REQ_MASK;
+ ctx->sw_cipher->base.crt_flags |=
+ cipher->base.crt_flags & CRYPTO_TFM_REQ_MASK;
+
+ err = crypto_ablkcipher_setkey(ctx->sw_cipher, key, len);
+ if (err)
+ goto sw_setkey_failed;
+ } else if ((len != AES_KEYSIZE_128 || len != AES_KEYSIZE_256) &&
+ !ctx->sw_cipher)
+ err = -EINVAL;
+
+ memcpy(ctx->key, key, len);
+ ctx->key_len = len;
+
+sw_setkey_failed:
+ if (err && ctx->sw_cipher) {
+ tfm->crt_flags &= ~CRYPTO_TFM_RES_MASK;
+ tfm->crt_flags |=
+ ctx->sw_cipher->base.crt_flags & CRYPTO_TFM_RES_MASK;
+ }
+
+ return err;
+}
+
+static int spacc_kasumi_f8_setkey(struct crypto_ablkcipher *cipher,
+ const u8 *key, unsigned int len)
+{
+ struct crypto_tfm *tfm = crypto_ablkcipher_tfm(cipher);
+ struct spacc_ablk_ctx *ctx = crypto_tfm_ctx(tfm);
+ int err = 0;
+
+ if (len > AES_MAX_KEY_SIZE) {
+ crypto_ablkcipher_set_flags(cipher, CRYPTO_TFM_RES_BAD_KEY_LEN);
+ err = -EINVAL;
+ goto out;
+ }
+
+ memcpy(ctx->key, key, len);
+ ctx->key_len = len;
+
+out:
+ return err;
+}
+
+static int spacc_ablk_need_fallback(struct spacc_req *req)
+{
+ struct spacc_ablk_ctx *ctx;
+ struct crypto_tfm *tfm = req->req->tfm;
+ struct crypto_alg *alg = req->req->tfm->__crt_alg;
+ struct spacc_alg *spacc_alg = to_spacc_alg(alg);
+
+ ctx = crypto_tfm_ctx(tfm);
+
+ return (spacc_alg->ctrl_default & SPACC_CRYPTO_ALG_MASK) ==
+ SPA_CTRL_CIPH_ALG_AES &&
+ ctx->key_len != AES_KEYSIZE_128 &&
+ ctx->key_len != AES_KEYSIZE_256;
+}
+
+static void spacc_ablk_complete(struct spacc_req *req)
+{
+ struct ablkcipher_request *ablk_req =
+ container_of(req->req, struct ablkcipher_request, base);
+
+ if (ablk_req->src != ablk_req->dst) {
+ spacc_free_ddt(req, req->src_ddt, req->src_addr, ablk_req->src,
+ ablk_req->nbytes, DMA_TO_DEVICE);
+ spacc_free_ddt(req, req->dst_ddt, req->dst_addr, ablk_req->dst,
+ ablk_req->nbytes, DMA_FROM_DEVICE);
+ } else
+ spacc_free_ddt(req, req->dst_ddt, req->dst_addr, ablk_req->dst,
+ ablk_req->nbytes, DMA_BIDIRECTIONAL);
+
+ req->req->complete(req->req, req->result);
+}
+
+static int spacc_ablk_submit(struct spacc_req *req)
+{
+ struct crypto_tfm *tfm = req->req->tfm;
+ struct spacc_ablk_ctx *ctx = crypto_tfm_ctx(tfm);
+ struct ablkcipher_request *ablk_req = ablkcipher_request_cast(req->req);
+ struct crypto_alg *alg = req->req->tfm->__crt_alg;
+ struct spacc_alg *spacc_alg = to_spacc_alg(alg);
+ struct spacc_engine *engine = ctx->generic.engine;
+ u32 ctrl;
+
+ req->ctx_id = spacc_load_ctx(&ctx->generic, ctx->key,
+ ctx->key_len, ablk_req->info, alg->cra_ablkcipher.ivsize,
+ NULL, 0);
+
+ writel(req->src_addr, engine->regs + SPA_SRC_PTR_REG_OFFSET);
+ writel(req->dst_addr, engine->regs + SPA_DST_PTR_REG_OFFSET);
+ writel(0, engine->regs + SPA_OFFSET_REG_OFFSET);
+
+ writel(ablk_req->nbytes, engine->regs + SPA_PROC_LEN_REG_OFFSET);
+ writel(0, engine->regs + SPA_ICV_OFFSET_REG_OFFSET);
+ writel(0, engine->regs + SPA_AUX_INFO_REG_OFFSET);
+ writel(0, engine->regs + SPA_AAD_LEN_REG_OFFSET);
+
+ ctrl = spacc_alg->ctrl_default | (req->ctx_id << SPA_CTRL_CTX_IDX) |
+ (req->is_encrypt ? (1 << SPA_CTRL_ENCRYPT_IDX) :
+ (1 << SPA_CTRL_KEY_EXP));
+
+ mod_timer(&engine->packet_timeout, jiffies + PACKET_TIMEOUT);
+
+ writel(ctrl, engine->regs + SPA_CTRL_REG_OFFSET);
+
+ return -EINPROGRESS;
+}
+
+static int spacc_ablk_do_fallback(struct ablkcipher_request *req,
+ unsigned alg_type, bool is_encrypt)
+{
+ struct crypto_tfm *old_tfm =
+ crypto_ablkcipher_tfm(crypto_ablkcipher_reqtfm(req));
+ struct spacc_ablk_ctx *ctx = crypto_tfm_ctx(old_tfm);
+ int err;
+
+ if (!ctx->sw_cipher)
+ return -EINVAL;
+
+ /*
+ * Change the request to use the software fallback transform, and once
+ * the ciphering has completed, put the old transform back into the
+ * request.
+ */
+ ablkcipher_request_set_tfm(req, ctx->sw_cipher);
+ err = is_encrypt ? crypto_ablkcipher_encrypt(req) :
+ crypto_ablkcipher_decrypt(req);
+ ablkcipher_request_set_tfm(req, __crypto_ablkcipher_cast(old_tfm));
+
+ return err;
+}
+
+static int spacc_ablk_setup(struct ablkcipher_request *req, unsigned alg_type,
+ bool is_encrypt)
+{
+ struct crypto_alg *alg = req->base.tfm->__crt_alg;
+ struct spacc_engine *engine = to_spacc_alg(alg)->engine;
+ struct spacc_req *dev_req = ablkcipher_request_ctx(req);
+ unsigned long flags;
+ int err = -ENOMEM;
+
+ dev_req->req = &req->base;
+ dev_req->is_encrypt = is_encrypt;
+ dev_req->engine = engine;
+ dev_req->complete = spacc_ablk_complete;
+ dev_req->result = -EINPROGRESS;
+
+ if (unlikely(spacc_ablk_need_fallback(dev_req)))
+ return spacc_ablk_do_fallback(req, alg_type, is_encrypt);
+
+ /*
+ * Create the DDT's for the engine. If we share the same source and
+ * destination then we can optimize by reusing the DDT's.
+ */
+ if (req->src != req->dst) {
+ dev_req->src_ddt = spacc_sg_to_ddt(engine, req->src,
+ req->nbytes, DMA_TO_DEVICE, &dev_req->src_addr);
+ if (!dev_req->src_ddt)
+ goto out;
+
+ dev_req->dst_ddt = spacc_sg_to_ddt(engine, req->dst,
+ req->nbytes, DMA_FROM_DEVICE, &dev_req->dst_addr);
+ if (!dev_req->dst_ddt)
+ goto out_free_src;
+ } else {
+ dev_req->dst_ddt = spacc_sg_to_ddt(engine, req->dst,
+ req->nbytes, DMA_BIDIRECTIONAL, &dev_req->dst_addr);
+ if (!dev_req->dst_ddt)
+ goto out;
+
+ dev_req->src_ddt = NULL;
+ dev_req->src_addr = dev_req->dst_addr;
+ }
+
+ err = -EINPROGRESS;
+ spin_lock_irqsave(&engine->hw_lock, flags);
+ /*
+ * Check if the engine will accept the operation now. If it won't then
+ * we either stick it on the end of a pending list if we can backlog,
+ * or bailout with an error if not.
+ */
+ if (unlikely(spacc_fifo_cmd_full(engine))) {
+ if (!(req->base.flags & CRYPTO_TFM_REQ_MAY_BACKLOG)) {
+ err = -EBUSY;
+ spin_unlock_irqrestore(&engine->hw_lock, flags);
+ goto out_free_ddts;
+ }
+ list_add_tail(&dev_req->list, &engine->pending);
+ } else {
+ ++engine->in_flight;
+ list_add_tail(&dev_req->list, &engine->in_progress);
+ spacc_ablk_submit(dev_req);
+ }
+ spin_unlock_irqrestore(&engine->hw_lock, flags);
+
+ goto out;
+
+out_free_ddts:
+ spacc_free_ddt(dev_req, dev_req->dst_ddt, dev_req->dst_addr, req->dst,
+ req->nbytes, req->src == req->dst ?
+ DMA_BIDIRECTIONAL : DMA_FROM_DEVICE);
+out_free_src:
+ if (req->src != req->dst)
+ spacc_free_ddt(dev_req, dev_req->src_ddt, dev_req->src_addr,
+ req->src, req->nbytes, DMA_TO_DEVICE);
+out:
+ return err;
+}
+
+static int spacc_ablk_cra_init(struct crypto_tfm *tfm)
+{
+ struct spacc_ablk_ctx *ctx = crypto_tfm_ctx(tfm);
+ struct crypto_alg *alg = tfm->__crt_alg;
+ struct spacc_alg *spacc_alg = to_spacc_alg(alg);
+ struct spacc_engine *engine = spacc_alg->engine;
+
+ ctx->generic.flags = spacc_alg->type;
+ ctx->generic.engine = engine;
+ if (alg->cra_flags & CRYPTO_ALG_NEED_FALLBACK) {
+ ctx->sw_cipher = crypto_alloc_ablkcipher(alg->cra_name, 0,
+ CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK);
+ if (IS_ERR(ctx->sw_cipher)) {
+ dev_warn(engine->dev, "failed to allocate fallback for %s\n",
+ alg->cra_name);
+ ctx->sw_cipher = NULL;
+ }
+ }
+ ctx->generic.key_offs = spacc_alg->key_offs;
+ ctx->generic.iv_offs = spacc_alg->iv_offs;
+
+ tfm->crt_ablkcipher.reqsize = sizeof(struct spacc_req);
+
+ return 0;
+}
+
+static void spacc_ablk_cra_exit(struct crypto_tfm *tfm)
+{
+ struct spacc_ablk_ctx *ctx = crypto_tfm_ctx(tfm);
+
+ if (ctx->sw_cipher)
+ crypto_free_ablkcipher(ctx->sw_cipher);
+ ctx->sw_cipher = NULL;
+}
+
+static int spacc_ablk_encrypt(struct ablkcipher_request *req)
+{
+ struct crypto_ablkcipher *cipher = crypto_ablkcipher_reqtfm(req);
+ struct crypto_tfm *tfm = crypto_ablkcipher_tfm(cipher);
+ struct spacc_alg *alg = to_spacc_alg(tfm->__crt_alg);
+
+ return spacc_ablk_setup(req, alg->type, 1);
+}
+
+static int spacc_ablk_decrypt(struct ablkcipher_request *req)
+{
+ struct crypto_ablkcipher *cipher = crypto_ablkcipher_reqtfm(req);
+ struct crypto_tfm *tfm = crypto_ablkcipher_tfm(cipher);
+ struct spacc_alg *alg = to_spacc_alg(tfm->__crt_alg);
+
+ return spacc_ablk_setup(req, alg->type, 0);
+}
+
+static inline int spacc_fifo_stat_empty(struct spacc_engine *engine)
+{
+ return readl(engine->regs + SPA_FIFO_STAT_REG_OFFSET) &
+ SPA_FIFO_STAT_EMPTY;
+}
+
+static void spacc_process_done(struct spacc_engine *engine)
+{
+ struct spacc_req *req;
+ unsigned long flags;
+
+ spin_lock_irqsave(&engine->hw_lock, flags);
+
+ while (!spacc_fifo_stat_empty(engine)) {
+ req = list_first_entry(&engine->in_progress, struct spacc_req,
+ list);
+ list_move_tail(&req->list, &engine->completed);
+
+ /* POP the status register. */
+ writel(~0, engine->regs + SPA_STAT_POP_REG_OFFSET);
+ req->result = (readl(engine->regs + SPA_STATUS_REG_OFFSET) &
+ SPA_STATUS_RES_CODE_MASK) >> SPA_STATUS_RES_CODE_OFFSET;
+
+ /*
+ * Convert the SPAcc error status into the standard POSIX error
+ * codes.
+ */
+ if (unlikely(req->result)) {
+ switch (req->result) {
+ case SPA_STATUS_ICV_FAIL:
+ req->result = -EBADMSG;
+ break;
+
+ case SPA_STATUS_MEMORY_ERROR:
+ dev_warn(engine->dev,
+ "memory error triggered\n");
+ req->result = -EFAULT;
+ break;
+
+ case SPA_STATUS_BLOCK_ERROR:
+ dev_warn(engine->dev,
+ "block error triggered\n");
+ req->result = -EIO;
+ break;
+ }
+ }
+ }
+
+ tasklet_schedule(&engine->complete);
+
+ spin_unlock_irqrestore(&engine->hw_lock, flags);
+}
+
+static irqreturn_t spacc_spacc_irq(int irq, void *dev)
+{
+ struct spacc_engine *engine = (struct spacc_engine *)dev;
+ u32 spacc_irq_stat = readl(engine->regs + SPA_IRQ_STAT_REG_OFFSET);
+
+ writel(spacc_irq_stat, engine->regs + SPA_IRQ_STAT_REG_OFFSET);
+ spacc_process_done(engine);
+
+ return IRQ_HANDLED;
+}
+
+static void spacc_packet_timeout(unsigned long data)
+{
+ struct spacc_engine *engine = (struct spacc_engine *)data;
+
+ spacc_process_done(engine);
+}
+
+static int spacc_req_submit(struct spacc_req *req)
+{
+ struct crypto_alg *alg = req->req->tfm->__crt_alg;
+
+ if (CRYPTO_ALG_TYPE_AEAD == (CRYPTO_ALG_TYPE_MASK & alg->cra_flags))
+ return spacc_aead_submit(req);
+ else
+ return spacc_ablk_submit(req);
+}
+
+static void spacc_spacc_complete(unsigned long data)
+{
+ struct spacc_engine *engine = (struct spacc_engine *)data;
+ struct spacc_req *req, *tmp;
+ unsigned long flags;
+ int num_removed = 0;
+ LIST_HEAD(completed);
+
+ spin_lock_irqsave(&engine->hw_lock, flags);
+ list_splice_init(&engine->completed, &completed);
+ spin_unlock_irqrestore(&engine->hw_lock, flags);
+
+ list_for_each_entry_safe(req, tmp, &completed, list) {
+ ++num_removed;
+ req->complete(req);
+ }
+
+ /* Try and fill the engine back up again. */
+ spin_lock_irqsave(&engine->hw_lock, flags);
+
+ engine->in_flight -= num_removed;
+
+ list_for_each_entry_safe(req, tmp, &engine->pending, list) {
+ if (spacc_fifo_cmd_full(engine))
+ break;
+
+ list_move_tail(&req->list, &engine->in_progress);
+ ++engine->in_flight;
+ req->result = spacc_req_submit(req);
+ }
+
+ if (engine->in_flight)
+ mod_timer(&engine->packet_timeout, jiffies + PACKET_TIMEOUT);
+
+ spin_unlock_irqrestore(&engine->hw_lock, flags);
+}
+
+#ifdef CONFIG_PM
+static int spacc_suspend(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct spacc_engine *engine = platform_get_drvdata(pdev);
+
+ /*
+ * We only support standby mode. All we have to do is gate the clock to
+ * the spacc. The hardware will preserve state until we turn it back
+ * on again.
+ */
+ clk_disable(engine->clk);
+
+ return 0;
+}
+
+static int spacc_resume(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct spacc_engine *engine = platform_get_drvdata(pdev);
+
+ return clk_enable(engine->clk);
+}
+
+static const struct dev_pm_ops spacc_pm_ops = {
+ .suspend = spacc_suspend,
+ .resume = spacc_resume,
+};
+#endif /* CONFIG_PM */
+
+static inline struct spacc_engine *spacc_dev_to_engine(struct device *dev)
+{
+ return dev ? platform_get_drvdata(to_platform_device(dev)) : NULL;
+}
+
+static ssize_t spacc_stat_irq_thresh_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct spacc_engine *engine = spacc_dev_to_engine(dev);
+
+ return snprintf(buf, PAGE_SIZE, "%u\n", engine->stat_irq_thresh);
+}
+
+static ssize_t spacc_stat_irq_thresh_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t len)
+{
+ struct spacc_engine *engine = spacc_dev_to_engine(dev);
+ unsigned long thresh;
+
+ if (strict_strtoul(buf, 0, &thresh))
+ return -EINVAL;
+
+ thresh = clamp(thresh, 1UL, engine->fifo_sz - 1);
+
+ engine->stat_irq_thresh = thresh;
+ writel(engine->stat_irq_thresh << SPA_IRQ_CTRL_STAT_CNT_OFFSET,
+ engine->regs + SPA_IRQ_CTRL_REG_OFFSET);
+
+ return len;
+}
+static DEVICE_ATTR(stat_irq_thresh, 0644, spacc_stat_irq_thresh_show,
+ spacc_stat_irq_thresh_store);
+
+static struct spacc_alg ipsec_engine_algs[] = {
+ {
+ .ctrl_default = SPA_CTRL_CIPH_ALG_AES | SPA_CTRL_CIPH_MODE_CBC,
+ .key_offs = 0,
+ .iv_offs = AES_MAX_KEY_SIZE,
+ .alg = {
+ .cra_name = "cbc(aes)",
+ .cra_driver_name = "cbc-aes-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER |
+ CRYPTO_ALG_ASYNC |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_ablk_ctx),
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_ablkcipher = {
+ .setkey = spacc_aes_setkey,
+ .encrypt = spacc_ablk_encrypt,
+ .decrypt = spacc_ablk_decrypt,
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ },
+ .cra_init = spacc_ablk_cra_init,
+ .cra_exit = spacc_ablk_cra_exit,
+ },
+ },
+ {
+ .key_offs = 0,
+ .iv_offs = AES_MAX_KEY_SIZE,
+ .ctrl_default = SPA_CTRL_CIPH_ALG_AES | SPA_CTRL_CIPH_MODE_ECB,
+ .alg = {
+ .cra_name = "ecb(aes)",
+ .cra_driver_name = "ecb-aes-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER |
+ CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_ablk_ctx),
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_ablkcipher = {
+ .setkey = spacc_aes_setkey,
+ .encrypt = spacc_ablk_encrypt,
+ .decrypt = spacc_ablk_decrypt,
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ },
+ .cra_init = spacc_ablk_cra_init,
+ .cra_exit = spacc_ablk_cra_exit,
+ },
+ },
+ {
+ .key_offs = DES_BLOCK_SIZE,
+ .iv_offs = 0,
+ .ctrl_default = SPA_CTRL_CIPH_ALG_DES | SPA_CTRL_CIPH_MODE_CBC,
+ .alg = {
+ .cra_name = "cbc(des)",
+ .cra_driver_name = "cbc-des-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = DES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_ablk_ctx),
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_ablkcipher = {
+ .setkey = spacc_des_setkey,
+ .encrypt = spacc_ablk_encrypt,
+ .decrypt = spacc_ablk_decrypt,
+ .min_keysize = DES_KEY_SIZE,
+ .max_keysize = DES_KEY_SIZE,
+ .ivsize = DES_BLOCK_SIZE,
+ },
+ .cra_init = spacc_ablk_cra_init,
+ .cra_exit = spacc_ablk_cra_exit,
+ },
+ },
+ {
+ .key_offs = DES_BLOCK_SIZE,
+ .iv_offs = 0,
+ .ctrl_default = SPA_CTRL_CIPH_ALG_DES | SPA_CTRL_CIPH_MODE_ECB,
+ .alg = {
+ .cra_name = "ecb(des)",
+ .cra_driver_name = "ecb-des-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = DES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_ablk_ctx),
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_ablkcipher = {
+ .setkey = spacc_des_setkey,
+ .encrypt = spacc_ablk_encrypt,
+ .decrypt = spacc_ablk_decrypt,
+ .min_keysize = DES_KEY_SIZE,
+ .max_keysize = DES_KEY_SIZE,
+ },
+ .cra_init = spacc_ablk_cra_init,
+ .cra_exit = spacc_ablk_cra_exit,
+ },
+ },
+ {
+ .key_offs = DES_BLOCK_SIZE,
+ .iv_offs = 0,
+ .ctrl_default = SPA_CTRL_CIPH_ALG_DES | SPA_CTRL_CIPH_MODE_CBC,
+ .alg = {
+ .cra_name = "cbc(des3_ede)",
+ .cra_driver_name = "cbc-des3-ede-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = DES3_EDE_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_ablk_ctx),
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_ablkcipher = {
+ .setkey = spacc_des_setkey,
+ .encrypt = spacc_ablk_encrypt,
+ .decrypt = spacc_ablk_decrypt,
+ .min_keysize = DES3_EDE_KEY_SIZE,
+ .max_keysize = DES3_EDE_KEY_SIZE,
+ .ivsize = DES3_EDE_BLOCK_SIZE,
+ },
+ .cra_init = spacc_ablk_cra_init,
+ .cra_exit = spacc_ablk_cra_exit,
+ },
+ },
+ {
+ .key_offs = DES_BLOCK_SIZE,
+ .iv_offs = 0,
+ .ctrl_default = SPA_CTRL_CIPH_ALG_DES | SPA_CTRL_CIPH_MODE_ECB,
+ .alg = {
+ .cra_name = "ecb(des3_ede)",
+ .cra_driver_name = "ecb-des3-ede-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = DES3_EDE_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_ablk_ctx),
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_ablkcipher = {
+ .setkey = spacc_des_setkey,
+ .encrypt = spacc_ablk_encrypt,
+ .decrypt = spacc_ablk_decrypt,
+ .min_keysize = DES3_EDE_KEY_SIZE,
+ .max_keysize = DES3_EDE_KEY_SIZE,
+ },
+ .cra_init = spacc_ablk_cra_init,
+ .cra_exit = spacc_ablk_cra_exit,
+ },
+ },
+ {
+ .ctrl_default = SPA_CTRL_CIPH_ALG_AES | SPA_CTRL_CIPH_MODE_CBC |
+ SPA_CTRL_HASH_ALG_SHA | SPA_CTRL_HASH_MODE_HMAC,
+ .key_offs = 0,
+ .iv_offs = AES_MAX_KEY_SIZE,
+ .alg = {
+ .cra_name = "authenc(hmac(sha1),cbc(aes))",
+ .cra_driver_name = "authenc-hmac-sha1-cbc-aes-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_aead_ctx),
+ .cra_type = &crypto_aead_type,
+ .cra_module = THIS_MODULE,
+ .cra_aead = {
+ .setkey = spacc_aead_setkey,
+ .setauthsize = spacc_aead_setauthsize,
+ .encrypt = spacc_aead_encrypt,
+ .decrypt = spacc_aead_decrypt,
+ .givencrypt = spacc_aead_givencrypt,
+ .ivsize = AES_BLOCK_SIZE,
+ .maxauthsize = SHA1_DIGEST_SIZE,
+ },
+ .cra_init = spacc_aead_cra_init,
+ .cra_exit = spacc_aead_cra_exit,
+ },
+ },
+ {
+ .ctrl_default = SPA_CTRL_CIPH_ALG_AES | SPA_CTRL_CIPH_MODE_CBC |
+ SPA_CTRL_HASH_ALG_SHA256 |
+ SPA_CTRL_HASH_MODE_HMAC,
+ .key_offs = 0,
+ .iv_offs = AES_MAX_KEY_SIZE,
+ .alg = {
+ .cra_name = "authenc(hmac(sha256),cbc(aes))",
+ .cra_driver_name = "authenc-hmac-sha256-cbc-aes-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_aead_ctx),
+ .cra_type = &crypto_aead_type,
+ .cra_module = THIS_MODULE,
+ .cra_aead = {
+ .setkey = spacc_aead_setkey,
+ .setauthsize = spacc_aead_setauthsize,
+ .encrypt = spacc_aead_encrypt,
+ .decrypt = spacc_aead_decrypt,
+ .givencrypt = spacc_aead_givencrypt,
+ .ivsize = AES_BLOCK_SIZE,
+ .maxauthsize = SHA256_DIGEST_SIZE,
+ },
+ .cra_init = spacc_aead_cra_init,
+ .cra_exit = spacc_aead_cra_exit,
+ },
+ },
+ {
+ .key_offs = 0,
+ .iv_offs = AES_MAX_KEY_SIZE,
+ .ctrl_default = SPA_CTRL_CIPH_ALG_AES | SPA_CTRL_CIPH_MODE_CBC |
+ SPA_CTRL_HASH_ALG_MD5 | SPA_CTRL_HASH_MODE_HMAC,
+ .alg = {
+ .cra_name = "authenc(hmac(md5),cbc(aes))",
+ .cra_driver_name = "authenc-hmac-md5-cbc-aes-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_aead_ctx),
+ .cra_type = &crypto_aead_type,
+ .cra_module = THIS_MODULE,
+ .cra_aead = {
+ .setkey = spacc_aead_setkey,
+ .setauthsize = spacc_aead_setauthsize,
+ .encrypt = spacc_aead_encrypt,
+ .decrypt = spacc_aead_decrypt,
+ .givencrypt = spacc_aead_givencrypt,
+ .ivsize = AES_BLOCK_SIZE,
+ .maxauthsize = MD5_DIGEST_SIZE,
+ },
+ .cra_init = spacc_aead_cra_init,
+ .cra_exit = spacc_aead_cra_exit,
+ },
+ },
+ {
+ .key_offs = DES_BLOCK_SIZE,
+ .iv_offs = 0,
+ .ctrl_default = SPA_CTRL_CIPH_ALG_DES | SPA_CTRL_CIPH_MODE_CBC |
+ SPA_CTRL_HASH_ALG_SHA | SPA_CTRL_HASH_MODE_HMAC,
+ .alg = {
+ .cra_name = "authenc(hmac(sha1),cbc(des3_ede))",
+ .cra_driver_name = "authenc-hmac-sha1-cbc-3des-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = DES3_EDE_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_aead_ctx),
+ .cra_type = &crypto_aead_type,
+ .cra_module = THIS_MODULE,
+ .cra_aead = {
+ .setkey = spacc_aead_setkey,
+ .setauthsize = spacc_aead_setauthsize,
+ .encrypt = spacc_aead_encrypt,
+ .decrypt = spacc_aead_decrypt,
+ .givencrypt = spacc_aead_givencrypt,
+ .ivsize = DES3_EDE_BLOCK_SIZE,
+ .maxauthsize = SHA1_DIGEST_SIZE,
+ },
+ .cra_init = spacc_aead_cra_init,
+ .cra_exit = spacc_aead_cra_exit,
+ },
+ },
+ {
+ .key_offs = DES_BLOCK_SIZE,
+ .iv_offs = 0,
+ .ctrl_default = SPA_CTRL_CIPH_ALG_AES | SPA_CTRL_CIPH_MODE_CBC |
+ SPA_CTRL_HASH_ALG_SHA256 |
+ SPA_CTRL_HASH_MODE_HMAC,
+ .alg = {
+ .cra_name = "authenc(hmac(sha256),cbc(des3_ede))",
+ .cra_driver_name = "authenc-hmac-sha256-cbc-3des-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = DES3_EDE_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_aead_ctx),
+ .cra_type = &crypto_aead_type,
+ .cra_module = THIS_MODULE,
+ .cra_aead = {
+ .setkey = spacc_aead_setkey,
+ .setauthsize = spacc_aead_setauthsize,
+ .encrypt = spacc_aead_encrypt,
+ .decrypt = spacc_aead_decrypt,
+ .givencrypt = spacc_aead_givencrypt,
+ .ivsize = DES3_EDE_BLOCK_SIZE,
+ .maxauthsize = SHA256_DIGEST_SIZE,
+ },
+ .cra_init = spacc_aead_cra_init,
+ .cra_exit = spacc_aead_cra_exit,
+ },
+ },
+ {
+ .key_offs = DES_BLOCK_SIZE,
+ .iv_offs = 0,
+ .ctrl_default = SPA_CTRL_CIPH_ALG_DES | SPA_CTRL_CIPH_MODE_CBC |
+ SPA_CTRL_HASH_ALG_MD5 | SPA_CTRL_HASH_MODE_HMAC,
+ .alg = {
+ .cra_name = "authenc(hmac(md5),cbc(des3_ede))",
+ .cra_driver_name = "authenc-hmac-md5-cbc-3des-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = DES3_EDE_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct spacc_aead_ctx),
+ .cra_type = &crypto_aead_type,
+ .cra_module = THIS_MODULE,
+ .cra_aead = {
+ .setkey = spacc_aead_setkey,
+ .setauthsize = spacc_aead_setauthsize,
+ .encrypt = spacc_aead_encrypt,
+ .decrypt = spacc_aead_decrypt,
+ .givencrypt = spacc_aead_givencrypt,
+ .ivsize = DES3_EDE_BLOCK_SIZE,
+ .maxauthsize = MD5_DIGEST_SIZE,
+ },
+ .cra_init = spacc_aead_cra_init,
+ .cra_exit = spacc_aead_cra_exit,
+ },
+ },
+};
+
+static struct spacc_alg l2_engine_algs[] = {
+ {
+ .key_offs = 0,
+ .iv_offs = SPACC_CRYPTO_KASUMI_F8_KEY_LEN,
+ .ctrl_default = SPA_CTRL_CIPH_ALG_KASUMI |
+ SPA_CTRL_CIPH_MODE_F8,
+ .alg = {
+ .cra_name = "f8(kasumi)",
+ .cra_driver_name = "f8-kasumi-picoxcell",
+ .cra_priority = SPACC_CRYPTO_ALG_PRIORITY,
+ .cra_flags = CRYPTO_ALG_TYPE_GIVCIPHER | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = 8,
+ .cra_ctxsize = sizeof(struct spacc_ablk_ctx),
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_ablkcipher = {
+ .setkey = spacc_kasumi_f8_setkey,
+ .encrypt = spacc_ablk_encrypt,
+ .decrypt = spacc_ablk_decrypt,
+ .min_keysize = 16,
+ .max_keysize = 16,
+ .ivsize = 8,
+ },
+ .cra_init = spacc_ablk_cra_init,
+ .cra_exit = spacc_ablk_cra_exit,
+ },
+ },
+};
+
+static int __devinit spacc_probe(struct platform_device *pdev,
+ unsigned max_ctxs, size_t cipher_pg_sz,
+ size_t hash_pg_sz, size_t fifo_sz,
+ struct spacc_alg *algs, size_t num_algs)
+{
+ int i, err, ret = -EINVAL;
+ struct resource *mem, *irq;
+ struct spacc_engine *engine = devm_kzalloc(&pdev->dev, sizeof(*engine),
+ GFP_KERNEL);
+ if (!engine)
+ return -ENOMEM;
+
+ engine->max_ctxs = max_ctxs;
+ engine->cipher_pg_sz = cipher_pg_sz;
+ engine->hash_pg_sz = hash_pg_sz;
+ engine->fifo_sz = fifo_sz;
+ engine->algs = algs;
+ engine->num_algs = num_algs;
+ engine->name = dev_name(&pdev->dev);
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+ if (!mem || !irq) {
+ dev_err(&pdev->dev, "no memory/irq resource for engine\n");
+ return -ENXIO;
+ }
+
+ if (!devm_request_mem_region(&pdev->dev, mem->start, resource_size(mem),
+ engine->name))
+ return -ENOMEM;
+
+ engine->regs = devm_ioremap(&pdev->dev, mem->start, resource_size(mem));
+ if (!engine->regs) {
+ dev_err(&pdev->dev, "memory map failed\n");
+ return -ENOMEM;
+ }
+
+ if (devm_request_irq(&pdev->dev, irq->start, spacc_spacc_irq, 0,
+ engine->name, engine)) {
+ dev_err(engine->dev, "failed to request IRQ\n");
+ return -EBUSY;
+ }
+
+ engine->dev = &pdev->dev;
+ engine->cipher_ctx_base = engine->regs + SPA_CIPH_KEY_BASE_REG_OFFSET;
+ engine->hash_key_base = engine->regs + SPA_HASH_KEY_BASE_REG_OFFSET;
+
+ engine->req_pool = dmam_pool_create(engine->name, engine->dev,
+ MAX_DDT_LEN * sizeof(struct spacc_ddt), 8, SZ_64K);
+ if (!engine->req_pool)
+ return -ENOMEM;
+
+ spin_lock_init(&engine->hw_lock);
+
+ engine->clk = clk_get(&pdev->dev, NULL);
+ if (IS_ERR(engine->clk)) {
+ dev_info(&pdev->dev, "clk unavailable\n");
+ device_remove_file(&pdev->dev, &dev_attr_stat_irq_thresh);
+ return PTR_ERR(engine->clk);
+ }
+
+ if (clk_enable(engine->clk)) {
+ dev_info(&pdev->dev, "unable to enable clk\n");
+ clk_put(engine->clk);
+ return -EIO;
+ }
+
+ err = device_create_file(&pdev->dev, &dev_attr_stat_irq_thresh);
+ if (err) {
+ clk_disable(engine->clk);
+ clk_put(engine->clk);
+ return err;
+ }
+
+
+ /*
+ * Use an IRQ threshold of 50% as a default. This seems to be a
+ * reasonable trade off of latency against throughput but can be
+ * changed at runtime.
+ */
+ engine->stat_irq_thresh = (engine->fifo_sz / 2);
+
+ /*
+ * Configure the interrupts. We only use the STAT_CNT interrupt as we
+ * only submit a new packet for processing when we complete another in
+ * the queue. This minimizes time spent in the interrupt handler.
+ */
+ writel(engine->stat_irq_thresh << SPA_IRQ_CTRL_STAT_CNT_OFFSET,
+ engine->regs + SPA_IRQ_CTRL_REG_OFFSET);
+ writel(SPA_IRQ_EN_STAT_EN | SPA_IRQ_EN_GLBL_EN,
+ engine->regs + SPA_IRQ_EN_REG_OFFSET);
+
+ setup_timer(&engine->packet_timeout, spacc_packet_timeout,
+ (unsigned long)engine);
+
+ INIT_LIST_HEAD(&engine->pending);
+ INIT_LIST_HEAD(&engine->completed);
+ INIT_LIST_HEAD(&engine->in_progress);
+ engine->in_flight = 0;
+ tasklet_init(&engine->complete, spacc_spacc_complete,
+ (unsigned long)engine);
+
+ platform_set_drvdata(pdev, engine);
+
+ INIT_LIST_HEAD(&engine->registered_algs);
+ for (i = 0; i < engine->num_algs; ++i) {
+ engine->algs[i].engine = engine;
+ err = crypto_register_alg(&engine->algs[i].alg);
+ if (!err) {
+ list_add_tail(&engine->algs[i].entry,
+ &engine->registered_algs);
+ ret = 0;
+ }
+ if (err)
+ dev_err(engine->dev, "failed to register alg \"%s\"\n",
+ engine->algs[i].alg.cra_name);
+ else
+ dev_dbg(engine->dev, "registered alg \"%s\"\n",
+ engine->algs[i].alg.cra_name);
+ }
+
+ return ret;
+}
+
+static int __devexit spacc_remove(struct platform_device *pdev)
+{
+ struct spacc_alg *alg, *next;
+ struct spacc_engine *engine = platform_get_drvdata(pdev);
+
+ del_timer_sync(&engine->packet_timeout);
+ device_remove_file(&pdev->dev, &dev_attr_stat_irq_thresh);
+
+ list_for_each_entry_safe(alg, next, &engine->registered_algs, entry) {
+ list_del(&alg->entry);
+ crypto_unregister_alg(&alg->alg);
+ }
+
+ clk_disable(engine->clk);
+ clk_put(engine->clk);
+
+ return 0;
+}
+
+static int __devinit ipsec_probe(struct platform_device *pdev)
+{
+ return spacc_probe(pdev, SPACC_CRYPTO_IPSEC_MAX_CTXS,
+ SPACC_CRYPTO_IPSEC_CIPHER_PG_SZ,
+ SPACC_CRYPTO_IPSEC_HASH_PG_SZ,
+ SPACC_CRYPTO_IPSEC_FIFO_SZ, ipsec_engine_algs,
+ ARRAY_SIZE(ipsec_engine_algs));
+}
+
+static struct platform_driver ipsec_driver = {
+ .probe = ipsec_probe,
+ .remove = __devexit_p(spacc_remove),
+ .driver = {
+ .name = "picoxcell-ipsec",
+#ifdef CONFIG_PM
+ .pm = &spacc_pm_ops,
+#endif /* CONFIG_PM */
+ },
+};
+
+static int __devinit l2_probe(struct platform_device *pdev)
+{
+ return spacc_probe(pdev, SPACC_CRYPTO_L2_MAX_CTXS,
+ SPACC_CRYPTO_L2_CIPHER_PG_SZ,
+ SPACC_CRYPTO_L2_HASH_PG_SZ, SPACC_CRYPTO_L2_FIFO_SZ,
+ l2_engine_algs, ARRAY_SIZE(l2_engine_algs));
+}
+
+static struct platform_driver l2_driver = {
+ .probe = l2_probe,
+ .remove = __devexit_p(spacc_remove),
+ .driver = {
+ .name = "picoxcell-l2",
+#ifdef CONFIG_PM
+ .pm = &spacc_pm_ops,
+#endif /* CONFIG_PM */
+ },
+};
+
+static int __init spacc_init(void)
+{
+ int ret = platform_driver_register(&ipsec_driver);
+ if (ret) {
+ pr_err("failed to register ipsec spacc driver");
+ goto out;
+ }
+
+ ret = platform_driver_register(&l2_driver);
+ if (ret) {
+ pr_err("failed to register l2 spacc driver");
+ goto l2_failed;
+ }
+
+ return 0;
+
+l2_failed:
+ platform_driver_unregister(&ipsec_driver);
+out:
+ return ret;
+}
+module_init(spacc_init);
+
+static void __exit spacc_exit(void)
+{
+ platform_driver_unregister(&ipsec_driver);
+ platform_driver_unregister(&l2_driver);
+}
+module_exit(spacc_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Jamie Iles");
diff --git a/drivers/crypto/picoxcell_crypto_regs.h b/drivers/crypto/picoxcell_crypto_regs.h
new file mode 100644
index 0000000..af93442
--- /dev/null
+++ b/drivers/crypto/picoxcell_crypto_regs.h
@@ -0,0 +1,128 @@
+/*
+ * Copyright (c) 2010 Picochip Ltd., Jamie Iles
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+#ifndef __PICOXCELL_CRYPTO_REGS_H__
+#define __PICOXCELL_CRYPTO_REGS_H__
+
+#define SPA_STATUS_OK 0
+#define SPA_STATUS_ICV_FAIL 1
+#define SPA_STATUS_MEMORY_ERROR 2
+#define SPA_STATUS_BLOCK_ERROR 3
+
+#define SPA_IRQ_CTRL_STAT_CNT_OFFSET 16
+#define SPA_IRQ_STAT_STAT_MASK (1 << 4)
+#define SPA_FIFO_STAT_STAT_OFFSET 16
+#define SPA_FIFO_STAT_STAT_CNT_MASK (0x3F << SPA_FIFO_STAT_STAT_OFFSET)
+#define SPA_STATUS_RES_CODE_OFFSET 24
+#define SPA_STATUS_RES_CODE_MASK (0x3 << SPA_STATUS_RES_CODE_OFFSET)
+#define SPA_KEY_SZ_CTX_INDEX_OFFSET 8
+#define SPA_KEY_SZ_CIPHER_OFFSET 31
+
+#define SPA_IRQ_EN_REG_OFFSET 0x00000000
+#define SPA_IRQ_STAT_REG_OFFSET 0x00000004
+#define SPA_IRQ_CTRL_REG_OFFSET 0x00000008
+#define SPA_FIFO_STAT_REG_OFFSET 0x0000000C
+#define SPA_SDMA_BRST_SZ_REG_OFFSET 0x00000010
+#define SPA_SRC_PTR_REG_OFFSET 0x00000020
+#define SPA_DST_PTR_REG_OFFSET 0x00000024
+#define SPA_OFFSET_REG_OFFSET 0x00000028
+#define SPA_AAD_LEN_REG_OFFSET 0x0000002C
+#define SPA_PROC_LEN_REG_OFFSET 0x00000030
+#define SPA_ICV_LEN_REG_OFFSET 0x00000034
+#define SPA_ICV_OFFSET_REG_OFFSET 0x00000038
+#define SPA_SW_CTRL_REG_OFFSET 0x0000003C
+#define SPA_CTRL_REG_OFFSET 0x00000040
+#define SPA_AUX_INFO_REG_OFFSET 0x0000004C
+#define SPA_STAT_POP_REG_OFFSET 0x00000050
+#define SPA_STATUS_REG_OFFSET 0x00000054
+#define SPA_KEY_SZ_REG_OFFSET 0x00000100
+#define SPA_CIPH_KEY_BASE_REG_OFFSET 0x00004000
+#define SPA_HASH_KEY_BASE_REG_OFFSET 0x00008000
+#define SPA_RC4_CTX_BASE_REG_OFFSET 0x00020000
+
+#define SPA_IRQ_EN_REG_RESET 0x00000000
+#define SPA_IRQ_CTRL_REG_RESET 0x00000000
+#define SPA_FIFO_STAT_REG_RESET 0x00000000
+#define SPA_SDMA_BRST_SZ_REG_RESET 0x00000000
+#define SPA_SRC_PTR_REG_RESET 0x00000000
+#define SPA_DST_PTR_REG_RESET 0x00000000
+#define SPA_OFFSET_REG_RESET 0x00000000
+#define SPA_AAD_LEN_REG_RESET 0x00000000
+#define SPA_PROC_LEN_REG_RESET 0x00000000
+#define SPA_ICV_LEN_REG_RESET 0x00000000
+#define SPA_ICV_OFFSET_REG_RESET 0x00000000
+#define SPA_SW_CTRL_REG_RESET 0x00000000
+#define SPA_CTRL_REG_RESET 0x00000000
+#define SPA_AUX_INFO_REG_RESET 0x00000000
+#define SPA_STAT_POP_REG_RESET 0x00000000
+#define SPA_STATUS_REG_RESET 0x00000000
+#define SPA_KEY_SZ_REG_RESET 0x00000000
+
+#define SPA_CTRL_HASH_ALG_IDX 4
+#define SPA_CTRL_CIPH_MODE_IDX 8
+#define SPA_CTRL_HASH_MODE_IDX 12
+#define SPA_CTRL_CTX_IDX 16
+#define SPA_CTRL_ENCRYPT_IDX 24
+#define SPA_CTRL_AAD_COPY 25
+#define SPA_CTRL_ICV_PT 26
+#define SPA_CTRL_ICV_ENC 27
+#define SPA_CTRL_ICV_APPEND 28
+#define SPA_CTRL_KEY_EXP 29
+
+#define SPA_KEY_SZ_CXT_IDX 8
+#define SPA_KEY_SZ_CIPHER_IDX 31
+
+#define SPA_IRQ_EN_CMD0_EN (1 << 0)
+#define SPA_IRQ_EN_STAT_EN (1 << 4)
+#define SPA_IRQ_EN_GLBL_EN (1 << 31)
+
+#define SPA_CTRL_CIPH_ALG_NULL 0x00
+#define SPA_CTRL_CIPH_ALG_DES 0x01
+#define SPA_CTRL_CIPH_ALG_AES 0x02
+#define SPA_CTRL_CIPH_ALG_RC4 0x03
+#define SPA_CTRL_CIPH_ALG_MULTI2 0x04
+#define SPA_CTRL_CIPH_ALG_KASUMI 0x05
+
+#define SPA_CTRL_HASH_ALG_NULL (0x00 << SPA_CTRL_HASH_ALG_IDX)
+#define SPA_CTRL_HASH_ALG_MD5 (0x01 << SPA_CTRL_HASH_ALG_IDX)
+#define SPA_CTRL_HASH_ALG_SHA (0x02 << SPA_CTRL_HASH_ALG_IDX)
+#define SPA_CTRL_HASH_ALG_SHA224 (0x03 << SPA_CTRL_HASH_ALG_IDX)
+#define SPA_CTRL_HASH_ALG_SHA256 (0x04 << SPA_CTRL_HASH_ALG_IDX)
+#define SPA_CTRL_HASH_ALG_SHA384 (0x05 << SPA_CTRL_HASH_ALG_IDX)
+#define SPA_CTRL_HASH_ALG_SHA512 (0x06 << SPA_CTRL_HASH_ALG_IDX)
+#define SPA_CTRL_HASH_ALG_AESMAC (0x07 << SPA_CTRL_HASH_ALG_IDX)
+#define SPA_CTRL_HASH_ALG_AESCMAC (0x08 << SPA_CTRL_HASH_ALG_IDX)
+#define SPA_CTRL_HASH_ALG_KASF9 (0x09 << SPA_CTRL_HASH_ALG_IDX)
+
+#define SPA_CTRL_CIPH_MODE_NULL (0x00 << SPA_CTRL_CIPH_MODE_IDX)
+#define SPA_CTRL_CIPH_MODE_ECB (0x00 << SPA_CTRL_CIPH_MODE_IDX)
+#define SPA_CTRL_CIPH_MODE_CBC (0x01 << SPA_CTRL_CIPH_MODE_IDX)
+#define SPA_CTRL_CIPH_MODE_CTR (0x02 << SPA_CTRL_CIPH_MODE_IDX)
+#define SPA_CTRL_CIPH_MODE_CCM (0x03 << SPA_CTRL_CIPH_MODE_IDX)
+#define SPA_CTRL_CIPH_MODE_GCM (0x05 << SPA_CTRL_CIPH_MODE_IDX)
+#define SPA_CTRL_CIPH_MODE_OFB (0x07 << SPA_CTRL_CIPH_MODE_IDX)
+#define SPA_CTRL_CIPH_MODE_CFB (0x08 << SPA_CTRL_CIPH_MODE_IDX)
+#define SPA_CTRL_CIPH_MODE_F8 (0x09 << SPA_CTRL_CIPH_MODE_IDX)
+
+#define SPA_CTRL_HASH_MODE_RAW (0x00 << SPA_CTRL_HASH_MODE_IDX)
+#define SPA_CTRL_HASH_MODE_SSLMAC (0x01 << SPA_CTRL_HASH_MODE_IDX)
+#define SPA_CTRL_HASH_MODE_HMAC (0x02 << SPA_CTRL_HASH_MODE_IDX)
+
+#define SPA_FIFO_STAT_EMPTY (1 << 31)
+#define SPA_FIFO_CMD_FULL (1 << 7)
+
+#endif /* __PICOXCELL_CRYPTO_REGS_H__ */
--
1.7.4
^ permalink raw reply related
* [PATCH 2/2] OMAP: IOMMU: add support to callback during fault handling
From: David Cohen @ 2011-02-15 13:20 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1297776033-8579-1-git-send-email-dacohen@gmail.com>
Add support to register a callback for IOMMU fault situations. Drivers using
IOMMU module might want to be informed when such errors happen in order to
debug it or react.
Signed-off-by: David Cohen <dacohen@gmail.com>
---
arch/arm/mach-omap2/iommu2.c | 21 +++++++++++++--
arch/arm/plat-omap/include/plat/iommu.h | 15 ++++++++++-
arch/arm/plat-omap/iommu.c | 41 ++++++++++++++++++++++++++++---
3 files changed, 69 insertions(+), 8 deletions(-)
diff --git a/arch/arm/mach-omap2/iommu2.c b/arch/arm/mach-omap2/iommu2.c
index 4244a07..559a066 100644
--- a/arch/arm/mach-omap2/iommu2.c
+++ b/arch/arm/mach-omap2/iommu2.c
@@ -143,10 +143,10 @@ static void omap2_iommu_set_twl(struct iommu *obj, bool on)
__iommu_set_twl(obj, false);
}
-static u32 omap2_iommu_fault_isr(struct iommu *obj, u32 *ra)
+static u32 omap2_iommu_fault_isr(struct iommu *obj, u32 *ra, u32 *iommu_errs)
{
int i;
- u32 stat, da;
+ u32 stat, da, errs;
const char *err_msg[] = {
"tlb miss",
"translation fault",
@@ -157,8 +157,10 @@ static u32 omap2_iommu_fault_isr(struct iommu *obj, u32 *ra)
stat = iommu_read_reg(obj, MMU_IRQSTATUS);
stat &= MMU_IRQ_MASK;
- if (!stat)
+ if (!stat) {
+ *iommu_errs = 0;
return 0;
+ }
da = iommu_read_reg(obj, MMU_FAULT_AD);
*ra = da;
@@ -171,6 +173,19 @@ static u32 omap2_iommu_fault_isr(struct iommu *obj, u32 *ra)
}
printk(KERN_DEBUG "\n");
+ errs = 0;
+ if (stat & MMU_IRQ_TLBMISS)
+ errs |= OMAP_IOMMU_ERR_TLB_MISS;
+ if (stat & MMU_IRQ_TRANSLATIONFAULT)
+ errs |= OMAP_IOMMU_ERR_TRANS_FAULT;
+ if (stat & MMU_IRQ_EMUMISS)
+ errs |= OMAP_IOMMU_ERR_EMU_MISS;
+ if (stat & MMU_IRQ_TABLEWALKFAULT)
+ errs |= OMAP_IOMMU_ERR_TBLWALK_FAULT;
+ if (stat & MMU_IRQ_MULTIHITFAULT)
+ errs |= OMAP_IOMMU_ERR_MULTIHIT_FAULT;
+ *iommu_errs = errs;
+
iommu_write_reg(obj, stat, MMU_IRQSTATUS);
return stat;
diff --git a/arch/arm/plat-omap/include/plat/iommu.h b/arch/arm/plat-omap/include/plat/iommu.h
index 19cbb5e..5a2475f 100644
--- a/arch/arm/plat-omap/include/plat/iommu.h
+++ b/arch/arm/plat-omap/include/plat/iommu.h
@@ -31,6 +31,7 @@ struct iommu {
struct clk *clk;
void __iomem *regbase;
struct device *dev;
+ void *fault_cb_priv;
unsigned int refcount;
struct mutex iommu_lock; /* global for this whole object */
@@ -48,6 +49,7 @@ struct iommu {
struct mutex mmap_lock; /* protect mmap */
int (*isr)(struct iommu *obj);
+ void (*fault_cb)(struct iommu *obj, u32 da, u32 iommu_errs, void *priv);
void *ctx; /* iommu context: registres saved area */
u32 da_start;
@@ -83,7 +85,7 @@ struct iommu_functions {
int (*enable)(struct iommu *obj);
void (*disable)(struct iommu *obj);
void (*set_twl)(struct iommu *obj, bool on);
- u32 (*fault_isr)(struct iommu *obj, u32 *ra);
+ u32 (*fault_isr)(struct iommu *obj, u32 *ra, u32 *iommu_errs);
void (*tlb_read_cr)(struct iommu *obj, struct cr_regs *cr);
void (*tlb_load_cr)(struct iommu *obj, struct cr_regs *cr);
@@ -109,6 +111,13 @@ struct iommu_platform_data {
u32 da_end;
};
+/* IOMMU errors */
+#define OMAP_IOMMU_ERR_TLB_MISS (1 << 0)
+#define OMAP_IOMMU_ERR_TRANS_FAULT (1 << 1)
+#define OMAP_IOMMU_ERR_EMU_MISS (1 << 2)
+#define OMAP_IOMMU_ERR_TBLWALK_FAULT (1 << 3)
+#define OMAP_IOMMU_ERR_MULTIHIT_FAULT (1 << 4)
+
#if defined(CONFIG_ARCH_OMAP1)
#error "iommu for this processor not implemented yet"
#else
@@ -161,6 +170,10 @@ extern size_t iopgtable_clear_entry(struct iommu *obj, u32 iova);
extern int iommu_set_da_range(struct iommu *obj, u32 start, u32 end);
extern struct iommu *iommu_get(const char *name);
extern void iommu_put(struct iommu *obj);
+extern int iommu_set_fault_callback(const char *name,
+ void (*fault_cb)(struct iommu *obj, u32 da,
+ u32 errs, void *priv),
+ void *fault_cb_priv);
extern void iommu_save_ctx(struct iommu *obj);
extern void iommu_restore_ctx(struct iommu *obj);
diff --git a/arch/arm/plat-omap/iommu.c b/arch/arm/plat-omap/iommu.c
index b1107c0..7f780ee 100644
--- a/arch/arm/plat-omap/iommu.c
+++ b/arch/arm/plat-omap/iommu.c
@@ -163,9 +163,9 @@ static u32 get_iopte_attr(struct iotlb_entry *e)
return arch_iommu->get_pte_attr(e);
}
-static u32 iommu_report_fault(struct iommu *obj, u32 *da)
+static u32 iommu_report_fault(struct iommu *obj, u32 *da, u32 *iommu_errs)
{
- return arch_iommu->fault_isr(obj, da);
+ return arch_iommu->fault_isr(obj, da, iommu_errs);
}
static void iotlb_lock_get(struct iommu *obj, struct iotlb_lock *l)
@@ -780,7 +780,7 @@ static void iopgtable_clear_entry_all(struct iommu *obj)
*/
static irqreturn_t iommu_fault_handler(int irq, void *data)
{
- u32 stat, da;
+ u32 stat, da, errs;
u32 *iopgd, *iopte;
int err = -EIO;
struct iommu *obj = data;
@@ -796,13 +796,19 @@ static irqreturn_t iommu_fault_handler(int irq, void *data)
return IRQ_HANDLED;
clk_enable(obj->clk);
- stat = iommu_report_fault(obj, &da);
+ stat = iommu_report_fault(obj, &da, &errs);
clk_disable(obj->clk);
if (!stat)
return IRQ_HANDLED;
iommu_disable(obj);
+ if (obj->fault_cb) {
+ obj->fault_cb(obj, da, errs, obj->fault_cb_priv);
+ /* No need to print error message as callback is called */
+ return IRQ_NONE;
+ }
+
iopgd = iopgd_offset(obj, da);
if (!iopgd_is_table(*iopgd)) {
@@ -917,6 +923,33 @@ void iommu_put(struct iommu *obj)
}
EXPORT_SYMBOL_GPL(iommu_put);
+int iommu_set_fault_callback(const char *name,
+ void (*fault_cb)(struct iommu *obj, u32 da,
+ u32 iommu_errs, void *priv),
+ void *fault_cb_priv)
+{
+ struct device *dev;
+ struct iommu *obj;
+
+ dev = driver_find_device(&omap_iommu_driver.driver, NULL, (void *)name,
+ device_match_by_alias);
+ if (!dev)
+ return -ENODEV;
+
+ obj = to_iommu(dev);
+ mutex_lock(&obj->iommu_lock);
+ if (obj->refcount != 0) {
+ mutex_unlock(&obj->iommu_lock);
+ return -EBUSY;
+ }
+ obj->fault_cb = fault_cb;
+ obj->fault_cb_priv = fault_cb_priv;
+ mutex_unlock(&obj->iommu_lock);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(iommu_set_fault_callback);
+
/*
* OMAP Device MMU(IOMMU) detection
*/
--
1.7.2.3
^ permalink raw reply related
* [PATCH 1/2] OMAP2+: IOMMU: change OMAP2+ error message to dev_dbg()
From: David Cohen @ 2011-02-15 13:20 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1297776033-8579-1-git-send-email-dacohen@gmail.com>
IOMMU upper layer is already printing error message. OMAP2+ specific
layer may print error message only for debug purpose.
Signed-off-by: David Cohen <dacohen@gmail.com>
---
arch/arm/mach-omap2/iommu2.c | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/arm/mach-omap2/iommu2.c b/arch/arm/mach-omap2/iommu2.c
index 14ee686..4244a07 100644
--- a/arch/arm/mach-omap2/iommu2.c
+++ b/arch/arm/mach-omap2/iommu2.c
@@ -163,13 +163,13 @@ static u32 omap2_iommu_fault_isr(struct iommu *obj, u32 *ra)
da = iommu_read_reg(obj, MMU_FAULT_AD);
*ra = da;
- dev_err(obj->dev, "%s:\tda:%08x ", __func__, da);
+ dev_dbg(obj->dev, "%s:\tda:%08x ", __func__, da);
for (i = 0; i < ARRAY_SIZE(err_msg); i++) {
if (stat & (1 << i))
- printk("%s ", err_msg[i]);
+ printk(KERN_DEBUG "%s ", err_msg[i]);
}
- printk("\n");
+ printk(KERN_DEBUG "\n");
iommu_write_reg(obj, stat, MMU_IRQSTATUS);
--
1.7.2.3
^ permalink raw reply related
* [PATCH 0/2] IOMMU fault callback support
From: David Cohen @ 2011-02-15 13:20 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
This patch set adds fault callback support to allow IOMMU users to debug or
react when a fault happens.
IOMMU faults might be very difficult to reproduce and then to figure out
the source of the problem. Currently IOMMU driver prints not so useful
debug message and does not notice user about such issue.
With a fault callback, IOMMU user may debug much more useful information
and/or react to go back to a valid state.
Br,
David
---
David Cohen (2):
OMAP2+: IOMMU: change OMAP2+ error message to dev_dbg()
OMAP: IOMMU: add support to callback during fault handling
arch/arm/mach-omap2/iommu2.c | 27 ++++++++++++++++----
arch/arm/plat-omap/include/plat/iommu.h | 15 ++++++++++-
arch/arm/plat-omap/iommu.c | 41 ++++++++++++++++++++++++++++---
3 files changed, 72 insertions(+), 11 deletions(-)
--
1.7.2.3
^ permalink raw reply
* [RFC] ARM: dma-mapping: outer cache is invalidated twice
From: Per Forlin @ 2011-02-15 13:14 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
My question concerns this patch
--------
commit 2ffe2da3e71652d4f4cae19539b5c78c2a239136
Author: Russell King <rmk+kernel@arm.linux.org.uk>
Date: Sat Oct 31 16:52:16 2009 +0000
ARMv6 and ARMv7 CPUs can perform speculative prefetching, which makes
DMA cache coherency handling slightly more interesting. Rather than
being able to rely upon the CPU not accessing the DMA buffer until DMA
has completed, we now must expect that the cache could be loaded with
possibly stale data from the DMA buffer.
Where DMA involves data being transferred to the device, we clean the
cache before handing it over for DMA, otherwise we invalidate the buffer
to get rid of potential writebacks. On DMA Completion, if data was
transferred from the device, we invalidate the buffer to get rid of
any stale speculative prefetches.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-By: Santosh Shilimkar <santosh.shilimkar@ti.com>
---------
file: arch/arm/mm/dma-mapping.c
> void ___dma_page_cpu_to_dev(struct page *page, unsigned long off,
> size_t size, enum dma_data_direction dir)
> {
> ...
> if (dir == DMA_FROM_DEVICE) {
> outer_inv_range(paddr, paddr + size);
> ...
> }
> EXPORT_SYMBOL(___dma_page_cpu_to_dev);
>
> void ___dma_page_dev_to_cpu(struct page *page, unsigned long off,
> size_t size, enum dma_data_direction dir)
> {
> ...
> if (dir != DMA_TO_DEVICE)
> outer_inv_range(paddr, paddr + size);
> ...
> }
outer_inv_range () is called twice for DMA_FROM_DEVICE.
The first time to "get rid of potential writebacks" and the second
time to "get rid of any stale speculative prefetches"
outer_inv_range() is a rather expensive operation. In the first case
isn't it enough to just call cache_sync()?
What about:
void ___dma_page_cpu_to_dev(struct page *page, unsigned long off,
size_t size, enum dma_data_direction dir)
{
...
if (dir == DMA_FROM_DEVICE) {
- outer_inv_range(paddr, paddr + size);
+ outer_sync();
...
}
/Per
^ permalink raw reply
* [PATCH v2] OMAP: IOMMU: add missing function declaration
From: Hiroshi DOYU @ 2011-02-15 13:09 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1297769473-24052-1-git-send-email-dacohen@gmail.com>
From: David Cohen <dacohen@gmail.com>
Subject: [PATCH v2] OMAP: IOMMU: add missing function declaration
Date: Tue, 15 Feb 2011 13:31:13 +0200
> Declaration of exported function 'iopgtable_lookup_entry' is missing from
> header file. Currently we have a sparse warning as it's not being used
> externally. Adding its declaration to avoid such warning and allow its usage
> in future.
>
> Signed-off-by: David Cohen <dacohen@gmail.com>
> ---
> arch/arm/plat-omap/include/plat/iommu.h | 2 ++
> 1 files changed, 2 insertions(+), 0 deletions(-)
Thanks. Tony, Please put this into your queue.
^ permalink raw reply
* [PATCH v5 0/3] add CNS3xxx AHCI support
From: Lin Mac @ 2011-02-15 13:02 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1294557970-654-1-git-send-email-mkl0301@gmail.com>
2011/1/9 <mkl0301@gmail.com>:
> From: Mac Lin <mkl0301@gmail.com>
>
> v5:
> ?- Rename ahci_platforms.h to ahci_platform.h
> ?- s/pltfm/platform/ in ahci_platform.c
>
> v4: http://www.spinics.net/lists/linux-ide/msg39852.html
> ?- v2 basically, which make SoC-specific changes into separate files
> ?- Rename module ahci_platform to ahci_platforms, into which link the ahci_platform and the SoC-specific changes
> ?- No rename of ahci_platform.c
> ?- Some cosmetic code changes since v2
>
> v3: http://www.spinics.net/lists/arm-kernel/msg109924.html
> ?- Make the SoC-specific changes into ahci_platform.c directly.
> ?- Some cosmetic code changes
> => ahci_platform.c is less readable due to SoC-specific changes and macro
>
> v2: http://www.spinics.net/lists/linux-ide/msg39766.html
> ?- Renaming ahci_platform.c to ahci_pltfm.c for linking SoC specific changes into ahci_platform, while not changing the module name.
> ?- Switch ahci_platform to module device table matching to add SoC specific support
> ?- Add CNS3xxx SoC specific AHCI support to ahci_cns3xxx.c
>
> v1: http://www.spinics.net/lists/arm-kernel/msg106236.html
> ?- Add CNS3xxx SoC specific AHCI support to arch/arm/mach-cns3xxx
> => The changes depends on libahci and libata, i.e. both of them need to be built into kernel
>
> This patchset is based on linux-2.6.37
>
ping?
Please kindly provide your opinion.
Best Regards,
Mac Lin
^ permalink raw reply
* [PATCHv5 0/3] Introduce the /proc/socinfo and use it to export OMAP data
From: Linus Walleij @ 2011-02-15 12:58 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1273587331-24604-1-git-send-email-eduardo.valentin@nokia.com>
2010/5/11 Eduardo Valentin <eduardo.valentin@nokia.com>:
> Here is the version 5 of the change to export OMAP data to userspace
> (name, revision, id code, production id and die id).
>
> Basically, this version is still attempting to create a new file under /proc.
> It is the /proc/socinfo, which should be used to export bits which are SoC specific
> (not CPU related, nor machine related).
>
> So, differences between previous version are:
> - merged patch 02/04 with 03/04 to avoid compilation breakages.
> - simplified the seq_file usage by using the single_open and single_release functions
> - exported a function to register a seq_operation .show callback
> - adapted the changes accordingly
>
> As usual, comments are welcome.
Eduardo, what has happened to this patchset?
Now we need something similar for arch/arm/mach-ux500 and I was sort of
hoping that this infrastructure would be in place already... wrong I was.
Do you want help in picking it up and try to polish it up?
Yours,
Linus Walleij
^ permalink raw reply
* [PATCH v2] ARM: S5PC210: add support for i2c PMICs on Universal_C210 board
From: Marek Szyprowski @ 2011-02-15 12:50 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20110215113504.GF4152@n2100.arm.linux.org.uk>
This patch adds basic definitions for MAX8952 & LP3974 (MAX8998
compatible) PMICs for UniversalC210 board. Power consumers for the
device drivers will be added later. These two PMICs occupy I2C5 bus.
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
---
arch/arm/mach-s5pv310/Kconfig | 2 +
arch/arm/mach-s5pv310/mach-universal_c210.c | 441 +++++++++++++++++++++++++++
2 files changed, 443 insertions(+), 0 deletions(-)
diff --git a/arch/arm/mach-s5pv310/Kconfig b/arch/arm/mach-s5pv310/Kconfig
index b2a9acc..a8b0425 100644
--- a/arch/arm/mach-s5pv310/Kconfig
+++ b/arch/arm/mach-s5pv310/Kconfig
@@ -102,7 +102,9 @@ config MACH_UNIVERSAL_C210
select S3C_DEV_HSMMC3
select S5PV310_SETUP_SDHCI
select S3C_DEV_I2C1
+ select S3C_DEV_I2C5
select S5PV310_SETUP_I2C1
+ select S5PV310_SETUP_I2C5
help
Machine support for Samsung Mobile Universal S5PC210 Reference
Board. S5PC210(MCP) is one of package option of S5PV310
diff --git a/arch/arm/mach-s5pv310/mach-universal_c210.c b/arch/arm/mach-s5pv310/mach-universal_c210.c
index 36bc3cf..88f704f 100644
--- a/arch/arm/mach-s5pv310/mach-universal_c210.c
+++ b/arch/arm/mach-s5pv310/mach-universal_c210.c
@@ -16,6 +16,7 @@
#include <linux/regulator/machine.h>
#include <linux/regulator/fixed.h>
#include <linux/mmc/host.h>
+#include <linux/gpio.h>
#include <asm/mach/arch.h>
#include <asm/mach-types.h>
@@ -24,10 +25,14 @@
#include <plat/s5pv310.h>
#include <plat/cpu.h>
#include <plat/devs.h>
+#include <plat/iic.h>
#include <plat/sdhci.h>
#include <mach/map.h>
+#include <linux/regulator/max8952.h>
+#include <linux/mfd/max8998.h>
+
/* Following are default values for UCON, ULCON and UFCON UART registers */
#define UNIVERSAL_UCON_DEFAULT (S3C2410_UCON_TXILEVEL | \
S3C2410_UCON_RXILEVEL | \
@@ -69,6 +74,438 @@ static struct s3c2410_uartcfg universal_uartcfgs[] __initdata = {
},
};
+static struct regulator_consumer_supply max8952_consumer =
+ REGULATOR_SUPPLY("vddarm", NULL);
+
+static struct max8952_platform_data universal_max8952_pdata __initdata = {
+ .gpio_vid0 = S5PV310_GPX0(3),
+ .gpio_vid1 = S5PV310_GPX0(4),
+ .gpio_en = -1, /* Not controllable, set "Always High" */
+ .default_mode = 0, /* vid0 = 0, vid1 = 0 */
+ .dvs_mode = { 48, 32, 28, 18 }, /* 1.25, 1.20, 1.05, 0.95V */
+ .sync_freq = 0, /* default: fastest */
+ .ramp_speed = 0, /* default: fastest */
+
+ .reg_data = {
+ .constraints = {
+ .name = "VARM_1.2V",
+ .min_uV = 770000,
+ .max_uV = 1400000,
+ .valid_ops_mask = REGULATOR_CHANGE_VOLTAGE,
+ .always_on = 1,
+ .boot_on = 1,
+ },
+ .num_consumer_supplies = 1,
+ .consumer_supplies = &max8952_consumer,
+ },
+};
+
+static struct regulator_consumer_supply lp3974_buck1_consumer =
+ REGULATOR_SUPPLY("vddint", NULL);
+
+static struct regulator_consumer_supply lp3974_buck2_consumer =
+ REGULATOR_SUPPLY("vddg3d", NULL);
+
+static struct regulator_init_data lp3974_buck1_data = {
+ .constraints = {
+ .name = "VINT_1.1V",
+ .min_uV = 750000,
+ .max_uV = 1500000,
+ .valid_ops_mask = REGULATOR_CHANGE_VOLTAGE |
+ REGULATOR_CHANGE_STATUS,
+ .boot_on = 1,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+ .num_consumer_supplies = 1,
+ .consumer_supplies = &lp3974_buck1_consumer,
+};
+
+static struct regulator_init_data lp3974_buck2_data = {
+ .constraints = {
+ .name = "VG3D_1.1V",
+ .min_uV = 750000,
+ .max_uV = 1500000,
+ .valid_ops_mask = REGULATOR_CHANGE_VOLTAGE |
+ REGULATOR_CHANGE_STATUS,
+ .boot_on = 1,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+ .num_consumer_supplies = 1,
+ .consumer_supplies = &lp3974_buck2_consumer,
+};
+
+static struct regulator_init_data lp3974_buck3_data = {
+ .constraints = {
+ .name = "VCC_1.8V",
+ .min_uV = 1800000,
+ .max_uV = 1800000,
+ .apply_uV = 1,
+ .always_on = 1,
+ .state_mem = {
+ .enabled = 1,
+ .disabled = 0,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_buck4_data = {
+ .constraints = {
+ .name = "VMEM_1.2V",
+ .min_uV = 1200000,
+ .max_uV = 1200000,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .apply_uV = 1,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo2_data = {
+ .constraints = {
+ .name = "VALIVE_1.2V",
+ .min_uV = 1200000,
+ .max_uV = 1200000,
+ .apply_uV = 1,
+ .always_on = 1,
+ .state_mem = {
+ .enabled = 1,
+ .disabled = 0,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo3_data = {
+ .constraints = {
+ .name = "VUSB+MIPI_1.1V",
+ .min_uV = 1100000,
+ .max_uV = 1100000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo4_data = {
+ .constraints = {
+ .name = "VADC_3.3V",
+ .min_uV = 3300000,
+ .max_uV = 3300000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo5_data = {
+ .constraints = {
+ .name = "VTF_2.8V",
+ .min_uV = 2800000,
+ .max_uV = 2800000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo6_data = {
+ .constraints = {
+ .name = "LDO6",
+ .min_uV = 2000000,
+ .max_uV = 2000000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .boot_on = 0,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo7_data = {
+ .constraints = {
+ .name = "VLCD+VMIPI_1.8V",
+ .min_uV = 1800000,
+ .max_uV = 1800000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo8_data = {
+ .constraints = {
+ .name = "VUSB+VDAC_3.3V",
+ .min_uV = 3300000,
+ .max_uV = 3300000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo9_data = {
+ .constraints = {
+ .name = "VCC_2.8V",
+ .min_uV = 2800000,
+ .max_uV = 2800000,
+ .apply_uV = 1,
+ .always_on = 1,
+ .state_mem = {
+ .enabled = 1,
+ .disabled = 0,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo10_data = {
+ .constraints = {
+ .name = "VPLL_1.1V",
+ .min_uV = 1100000,
+ .max_uV = 1100000,
+ .boot_on = 1,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo11_data = {
+ .constraints = {
+ .name = "CAM_AF_3.3V",
+ .min_uV = 3300000,
+ .max_uV = 3300000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo12_data = {
+ .constraints = {
+ .name = "PS_2.8V",
+ .min_uV = 2800000,
+ .max_uV = 2800000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo13_data = {
+ .constraints = {
+ .name = "VHIC_1.2V",
+ .min_uV = 1200000,
+ .max_uV = 1200000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo14_data = {
+ .constraints = {
+ .name = "CAM_I_HOST_1.8V",
+ .min_uV = 1800000,
+ .max_uV = 1800000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo15_data = {
+ .constraints = {
+ .name = "CAM_S_DIG+FM33_CORE_1.2V",
+ .min_uV = 1200000,
+ .max_uV = 1200000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo16_data = {
+ .constraints = {
+ .name = "CAM_S_ANA_2.8V",
+ .min_uV = 2800000,
+ .max_uV = 2800000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_ldo17_data = {
+ .constraints = {
+ .name = "VCC_3.0V_LCD",
+ .min_uV = 3000000,
+ .max_uV = 3000000,
+ .apply_uV = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .boot_on = 1,
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_32khz_ap_data = {
+ .constraints = {
+ .name = "32KHz AP",
+ .always_on = 1,
+ .state_mem = {
+ .enabled = 1,
+ .disabled = 0,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_32khz_cp_data = {
+ .constraints = {
+ .name = "32KHz CP",
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_vichg_data = {
+ .constraints = {
+ .name = "VICHG",
+ .state_mem = {
+ .enabled = 0,
+ .disabled = 1,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_esafeout1_data = {
+ .constraints = {
+ .name = "SAFEOUT1",
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 1,
+ .disabled = 0,
+ },
+ },
+};
+
+static struct regulator_init_data lp3974_esafeout2_data = {
+ .constraints = {
+ .name = "SAFEOUT2",
+ .boot_on = 1,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ .state_mem = {
+ .enabled = 1,
+ .disabled = 0,
+ },
+ },
+};
+
+static struct max8998_regulator_data lp3974_regulators[] = {
+ { MAX8998_LDO2, &lp3974_ldo2_data },
+ { MAX8998_LDO3, &lp3974_ldo3_data },
+ { MAX8998_LDO4, &lp3974_ldo4_data },
+ { MAX8998_LDO5, &lp3974_ldo5_data },
+ { MAX8998_LDO6, &lp3974_ldo6_data },
+ { MAX8998_LDO7, &lp3974_ldo7_data },
+ { MAX8998_LDO8, &lp3974_ldo8_data },
+ { MAX8998_LDO9, &lp3974_ldo9_data },
+ { MAX8998_LDO10, &lp3974_ldo10_data },
+ { MAX8998_LDO11, &lp3974_ldo11_data },
+ { MAX8998_LDO12, &lp3974_ldo12_data },
+ { MAX8998_LDO13, &lp3974_ldo13_data },
+ { MAX8998_LDO14, &lp3974_ldo14_data },
+ { MAX8998_LDO15, &lp3974_ldo15_data },
+ { MAX8998_LDO16, &lp3974_ldo16_data },
+ { MAX8998_LDO17, &lp3974_ldo17_data },
+ { MAX8998_BUCK1, &lp3974_buck1_data },
+ { MAX8998_BUCK2, &lp3974_buck2_data },
+ { MAX8998_BUCK3, &lp3974_buck3_data },
+ { MAX8998_BUCK4, &lp3974_buck4_data },
+ { MAX8998_EN32KHZ_AP, &lp3974_32khz_ap_data },
+ { MAX8998_EN32KHZ_CP, &lp3974_32khz_cp_data },
+ { MAX8998_ENVICHG, &lp3974_vichg_data },
+ { MAX8998_ESAFEOUT1, &lp3974_esafeout1_data },
+ { MAX8998_ESAFEOUT2, &lp3974_esafeout2_data },
+};
+
+static struct max8998_platform_data universal_lp3974_pdata = {
+ .num_regulators = ARRAY_SIZE(lp3974_regulators),
+ .regulators = lp3974_regulators,
+ .buck1_voltage1 = 1100000, /* INT */
+ .buck1_voltage2 = 1000000,
+ .buck1_voltage3 = 1100000,
+ .buck1_voltage4 = 1000000,
+ .buck1_set1 = S5PV310_GPX0(5),
+ .buck1_set2 = S5PV310_GPX0(6),
+ .buck2_voltage1 = 1200000, /* G3D */
+ .buck2_voltage2 = 1100000,
+ .buck1_default_idx = 0,
+ .buck2_set3 = S5PV310_GPE2(0),
+ .buck2_default_idx = 0,
+ .wakeup = true,
+};
+
+/* GPIO I2C 5 (PMIC) */
+static struct i2c_board_info i2c_devs5[] __initdata = {
+ {
+ I2C_BOARD_INFO("max8952", 0xC0 >> 1),
+ .platform_data = &universal_max8952_pdata,
+ }, {
+ I2C_BOARD_INFO("lp3974", 0xCC >> 1),
+ .platform_data = &universal_lp3974_pdata,
+ },
+};
+
+/* GPIO KEYS */
static struct gpio_keys_button universal_gpio_keys_tables[] = {
{
.code = KEY_VOLUMEUP,
@@ -206,6 +643,7 @@ static struct platform_device *universal_devices[] __initdata = {
/* Universal Devices */
&universal_gpio_keys,
+ &s3c_device_i2c5,
&s5p_device_onenand,
};
@@ -223,6 +661,9 @@ static void __init universal_machine_init(void)
i2c_register_board_info(0, i2c0_devs, ARRAY_SIZE(i2c0_devs));
i2c_register_board_info(1, i2c1_devs, ARRAY_SIZE(i2c1_devs));
+ s3c_i2c5_set_platdata(NULL);
+ i2c_register_board_info(5, i2c_devs5, ARRAY_SIZE(i2c_devs5));
+
/* Last */
platform_add_devices(universal_devices, ARRAY_SIZE(universal_devices));
}
--
1.7.1.569.g6f426
^ permalink raw reply related
* [PATCH v4 15/19] ARM: LPAE: use phys_addr_t instead of unsigned long for physical addresses
From: Catalin Marinas @ 2011-02-15 12:39 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20110215123522.GH4152@n2100.arm.linux.org.uk>
On Tue, 2011-02-15 at 12:35 +0000, Russell King - ARM Linux wrote:
> On Tue, Feb 15, 2011 at 11:52:22AM +0000, Will Deacon wrote:
> > Excellent! I've split the patch up into four distinct parts, as per your
> > suggestions. I've submitted these to your patch system (6670/1-6673/1)
> > alongside a fixed version of the printf format patch (6669/1) because
> > without that, you get a bunch of compiler warnings.
>
> Except 6669/1 still suffers from "%#08llx". For a value of one, that prints:
>
> 0x000001
>
> five zeros following the 0x rather than seven. The width in the format
> string includes the 0x prefix.
Ah, sorry, I only fixed one case and forgot about the rest (and
misleading Will).
--
Catalin
^ permalink raw reply
* [PATCH v4 15/19] ARM: LPAE: use phys_addr_t instead of unsigned long for physical addresses
From: Russell King - ARM Linux @ 2011-02-15 12:35 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1297770742.21448.2.camel@e102144-lin.cambridge.arm.com>
On Tue, Feb 15, 2011 at 11:52:22AM +0000, Will Deacon wrote:
> Excellent! I've split the patch up into four distinct parts, as per your
> suggestions. I've submitted these to your patch system (6670/1-6673/1)
> alongside a fixed version of the printf format patch (6669/1) because
> without that, you get a bunch of compiler warnings.
Except 6669/1 still suffers from "%#08llx". For a value of one, that prints:
0x000001
five zeros following the 0x rather than seven. The width in the format
string includes the 0x prefix.
^ permalink raw reply
* [RFC PATCH 2/2] ARMv7: Invalidate the TLB before freeing page tables
From: Catalin Marinas @ 2011-02-15 12:29 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20110215113242.GD4152@n2100.arm.linux.org.uk>
On Tue, 2011-02-15 at 11:32 +0000, Russell King - ARM Linux wrote:
> On Tue, Feb 15, 2011 at 11:02:28AM +0000, Catalin Marinas wrote:
> > On Tue, 2011-02-15 at 10:31 +0000, Russell King - ARM Linux wrote:
> > > We already have support for doing this, and Peter Zijlstra posted patches
> > > to convert ARM to use a generic implementation of the TLB shootdown code.
> > >
> > > http://marc.info/?l=linux-kernel&m=129604765010347&w=2
> > >
> > > Does this patch solve your problem?
> >
> > I don't think it does. Peter's patch moves the ARM TLB support to the
> > generic one which is a good clean-up, however it doesn't look like
> > anything is invalidating the TLB entry between pmd_clear() and
> > pte_free(), only after.
>
> Can we please stop thinking "ooh, lets use generic code, that's good" ?
> It's a rediculous attitude. The reason we have our own version is because
> the generic version is too heavy for our UP implementations and it doesn't
> do what we need to do there.
>
> I believe we should stick with our own version for UP, and switch over to
> the generic version for SMP.
Whatever we do, the issue with A15 is valid for both UP and SMP since we
can get an interrupt which allocates a page previously hosting a pte but
before the TLB was invalidated.
I'll follow up on your patch as well.
--
Catalin
^ permalink raw reply
* [RFC PATCH 2/2] ARMv7: Invalidate the TLB before freeing page tables
From: Russell King - ARM Linux @ 2011-02-15 12:14 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20110215113242.GD4152@n2100.arm.linux.org.uk>
On Tue, Feb 15, 2011 at 11:32:42AM +0000, Russell King - ARM Linux wrote:
> The point of TLB shootdown is that we unmap the entries from the page
> tables, then issue the TLB flushes, and then free the pages and page
> tables after that. All that Peter's patch tries to do is to get ARM to
> use the generic stuff.
As Peter's patch preserves the current behaviour, that's not sufficient.
So, let's do this our own way and delay pages and page table frees on
ARMv6 and v7. Untested.
Note that the generic code doesn't allow us to delay frees on UP as it
assumes that if there's no TLB entry, the CPU won't speculatively
prefetch. This seems to be where ARM differs from the rest of the
planet. Please confirm that this is indeed the case.
arch/arm/include/asm/tlb.h | 79 +++++++++++++++++++++++++++++++++++++-------
1 files changed, 67 insertions(+), 12 deletions(-)
diff --git a/arch/arm/include/asm/tlb.h b/arch/arm/include/asm/tlb.h
index f41a6f5..1ca3e16 100644
--- a/arch/arm/include/asm/tlb.h
+++ b/arch/arm/include/asm/tlb.h
@@ -30,6 +30,16 @@
#include <asm/pgalloc.h>
/*
+ * As v6 and v7 speculatively prefetch, which can drag new entries into the
+ * TLB, we need to delay freeing pages and page tables.
+ */
+#if defined(CONFIG_CPU_32v6) || defined(CONFIG_CPU_32v7)
+#define tlb_fast_mode(tlb) 0
+#else
+#define tlb_fast_mode(tlb) 1
+#endif
+
+/*
* TLB handling. This allows us to remove pages from the page
* tables, and efficiently handle the TLB issues.
*/
@@ -38,10 +48,42 @@ struct mmu_gather {
unsigned int fullmm;
unsigned long range_start;
unsigned long range_end;
+ unsigned int nr;
+ struct page *pages[FREE_PTE_NR];
};
DECLARE_PER_CPU(struct mmu_gather, mmu_gathers);
+static inline void tlb_flush(struct mmu_gather *tlb)
+{
+ if (tlb->fullmm)
+ flush_tlb_mm(tlb->mm);
+ else if (tlb->range_end > 0) {
+ flush_tlb_range(vma, tlb->range_start, tlb->range_end);
+ tlb->range_start = TASK_SIZE;
+ tlb->range_end = 0;
+ }
+}
+
+static inline void tlb_add_flush(struct mmu_gather *tlb, unsigned long addr)
+{
+ if (!tlb->fullmm) {
+ if (addr < tlb->range_start)
+ tlb->range_start = addr;
+ if (addr + PAGE_SIZE > tlb->range_end)
+ tlb->range_end = addr + PAGE_SIZE;
+ }
+}
+
+static inline void tlb_flush_mmu(struct mmu_gather *tlb)
+{
+ tlb_flush(tlb);
+ if (!tlb_fast_mode(tlb)) {
+ free_pages_and_swap_cache(tlb->pages, tlb->nr);
+ tlb->nr = 0;
+ }
+}
+
static inline struct mmu_gather *
tlb_gather_mmu(struct mm_struct *mm, unsigned int full_mm_flush)
{
@@ -49,6 +91,7 @@ tlb_gather_mmu(struct mm_struct *mm, unsigned int full_mm_flush)
tlb->mm = mm;
tlb->fullmm = full_mm_flush;
+ tlb->nr = 0;
return tlb;
}
@@ -56,8 +99,7 @@ tlb_gather_mmu(struct mm_struct *mm, unsigned int full_mm_flush)
static inline void
tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end)
{
- if (tlb->fullmm)
- flush_tlb_mm(tlb->mm);
+ tlb_flush_mmu(tlb);
/* keep the page table cache within bounds */
check_pgt_cache();
@@ -71,12 +113,7 @@ tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end)
static inline void
tlb_remove_tlb_entry(struct mmu_gather *tlb, pte_t *ptep, unsigned long addr)
{
- if (!tlb->fullmm) {
- if (addr < tlb->range_start)
- tlb->range_start = addr;
- if (addr + PAGE_SIZE > tlb->range_end)
- tlb->range_end = addr + PAGE_SIZE;
- }
+ tlb_add_flush(tlb, addr);
}
/*
@@ -97,12 +134,30 @@ tlb_start_vma(struct mmu_gather *tlb, struct vm_area_struct *vma)
static inline void
tlb_end_vma(struct mmu_gather *tlb, struct vm_area_struct *vma)
{
- if (!tlb->fullmm && tlb->range_end > 0)
- flush_tlb_range(vma, tlb->range_start, tlb->range_end);
+ if (!tlb->fullmm)
+ tlb_flush(tlb);
+}
+
+static inline void tlb_remove_page(struct mmu_gather *tlb, struct page *page)
+{
+ if (tlb_fast_mode(tlb)) {
+ free_page_and_swap_cache(page);
+ } else {
+ tlb->pages[tlb->nr++] = page;
+ if (tlb->nr >= FREE_PTE_NR)
+ tlb_flush_mmu(tlb);
+ }
+}
+
+static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte,
+ unsigned long addr)
+{
+ pgtable_page_dtor(pte);
+ tlb_add_flush(addr);
+ tlb_remove_page(tlb, pte);
}
-#define tlb_remove_page(tlb,page) free_page_and_swap_cache(page)
-#define pte_free_tlb(tlb, ptep, addr) pte_free((tlb)->mm, ptep)
+#define pte_free_tlb(tlb, ptep, addr) __pte_free_tlb(tlb, ptep, addr)
#define pmd_free_tlb(tlb, pmdp, addr) pmd_free((tlb)->mm, pmdp)
#define tlb_migrate_finish(mm) do { } while (0)
^ permalink raw reply related
* [PATCH] ARM: pmu: avoid setting IRQ affinity on UP systems
From: Will Deacon @ 2011-02-15 12:11 UTC (permalink / raw)
To: linux-arm-kernel
Now that we can execute a CONFIG_SMP kernel on a uniprocessor system,
extra care has to be taken in the PMU IRQ affinity setting code to
ensure that we don't always fail to initialise.
This patch changes the CPU PMU initialisation code so that when we
only have a single IRQ, whose affinity can not be changed at the
controller, we report success (0) rather than -EINVAL.
Cc: Jamie Iles <jamie@jamieiles.com>
Reported-by: Avik Sil <avik.sil@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
arch/arm/kernel/pmu.c | 21 ++++++++++++++-------
1 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/arch/arm/kernel/pmu.c b/arch/arm/kernel/pmu.c
index b8af96e..2b65d10 100644
--- a/arch/arm/kernel/pmu.c
+++ b/arch/arm/kernel/pmu.c
@@ -97,22 +97,29 @@ set_irq_affinity(int irq,
irq, cpu);
return err;
#else
- return 0;
+ return -EINVAL;
#endif
}
static int
init_cpu_pmu(void)
{
- int i, err = 0;
+ int i, irqs, err = 0;
struct platform_device *pdev = pmu_devices[ARM_PMU_DEVICE_CPU];
- if (!pdev) {
- err = -ENODEV;
- goto out;
- }
+ if (!pdev)
+ return -ENODEV;
+
+ irqs = pdev->num_resources;
+
+ /*
+ * If we have a single PMU interrupt that we can't shift, assume that
+ * we're running on a uniprocessor machine and continue.
+ */
+ if (irqs == 1 && !irq_can_set_affinity(platform_get_irq(pdev, 0)))
+ return 0;
- for (i = 0; i < pdev->num_resources; ++i) {
+ for (i = 0; i < irqs; ++i) {
err = set_irq_affinity(platform_get_irq(pdev, i), i);
if (err)
break;
--
1.7.0.4
^ permalink raw reply related
* [PATCH] ARM: PXA: Make PXA27x/PXA3xx overlay actually work
From: Vasily Khoruzhick @ 2011-02-15 11:58 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <AANLkTimR_6fYuJHuVC8TD5agjyCoUWiKQmEunhZHSQE5@mail.gmail.com>
On Tuesday 15 February 2011 13:51:06 Eric Miao wrote:
> >> > lcd_writel(fbi, FDADR0, fbi->fdadr[0]);
> >> > - lcd_writel(fbi, FDADR1, fbi->fdadr[1]);
> >> > + if (fbi->lccr0 & LCCR0_SDS)
> >> > + lcd_writel(fbi, FDADR1, fbi->fdadr[1]);
> >>
> >> My original intention was to simplify the code a bit by ignoring
> >> LCCR0_SDS, as FDADR1 would not take effect if not enabled even
> >> if it's being read/written.
> >
> > It leads to potential race condition when you try to reconfigure main
> > plane and overlay1 simultaneously.
>
> You are right on this.
>
> >> > +#ifdef CONFIG_FB_PXA_OVERLAY
> >> > + if (cpu_is_pxa27x())
> >> > + fbi->lccr0 |= LCCR0_OUC;
> >> > +#endif
> >> > +
> >>
> >> I seem to remember LCCR0_OUC is still valid on pxa3xx, did you
> >> do some test on pxa3xx as well?
> >
> > Sorry, I have no any pxa3xx boards.
>
> That's all right, I can give it a test later. The point is, why
> did you move the code here from pxafb_overlay_init()?
Because otherwise correct plane order (overlays on top) will be selected only
on next main plane reconfigure.
Regards
Vasily
^ permalink raw reply
* [PATCH v4 15/19] ARM: LPAE: use phys_addr_t instead of unsigned long for physical addresses
From: Will Deacon @ 2011-02-15 11:52 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20110212102803.GC15616@n2100.arm.linux.org.uk>
Hi Russell,
On Sat, 2011-02-12 at 10:28 +0000, Russell King - ARM Linux wrote:
> On Mon, Jan 24, 2011 at 05:55:57PM +0000, Catalin Marinas wrote:
> > arch/arm/include/asm/memory.h | 17 +++++++++--------
> > arch/arm/include/asm/outercache.h | 14 ++++++++------
> > arch/arm/include/asm/pgtable.h | 2 +-
> > arch/arm/include/asm/setup.h | 2 +-
> > arch/arm/kernel/setup.c | 5 +++--
> > arch/arm/mm/init.c | 6 +++---
> > arch/arm/mm/mmu.c | 7 ++++---
> > 7 files changed, 29 insertions(+), 24 deletions(-)
>
> If this is split up into four separate patches, we can probably sort out
> merging this for the upcoming window.
>
Excellent! I've split the patch up into four distinct parts, as per your
suggestions. I've submitted these to your patch system (6670/1-6673/1)
alongside a fixed version of the printf format patch (6669/1) because
without that, you get a bunch of compiler warnings.
Will
^ permalink raw reply
* [PATCH] ARM: PXA: Make PXA27x/PXA3xx overlay actually work
From: Eric Miao @ 2011-02-15 11:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <201102151041.21560.anarsoul@gmail.com>
>> > ? ? ? ?lcd_writel(fbi, FDADR0, fbi->fdadr[0]);
>> > - ? ? ? lcd_writel(fbi, FDADR1, fbi->fdadr[1]);
>> > + ? ? ? if (fbi->lccr0 & LCCR0_SDS)
>> > + ? ? ? ? ? ? ? lcd_writel(fbi, FDADR1, fbi->fdadr[1]);
>>
>> My original intention was to simplify the code a bit by ignoring
>> LCCR0_SDS, as FDADR1 would not take effect if not enabled even
>> if it's being read/written.
>
> It leads to potential race condition when you try to reconfigure main plane
> and overlay1 simultaneously.
>
You are right on this.
>> > +#ifdef CONFIG_FB_PXA_OVERLAY
>> > + ? ? ? if (cpu_is_pxa27x())
>> > + ? ? ? ? ? ? ? fbi->lccr0 |= LCCR0_OUC;
>> > +#endif
>> > +
>>
>> I seem to remember LCCR0_OUC is still valid on pxa3xx, did you
>> do some test on pxa3xx as well?
>
> Sorry, I have no any pxa3xx boards.
That's all right, I can give it a test later. The point is, why
did you move the code here from pxafb_overlay_init()?
^ permalink raw reply
* [PATCH] ARM: PXA: Make PXA27x/PXA3xx overlay actually work
From: Eric Miao @ 2011-02-15 11:43 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20110215094808.GB4152@n2100.arm.linux.org.uk>
On Tue, Feb 15, 2011 at 5:48 PM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Tue, Feb 15, 2011 at 03:35:44PM +0800, Eric Miao wrote:
>> > @@ -720,12 +726,10 @@ static int overlayfb_open(struct fb_info *info, int user)
>> > ? ? ? ?if (user == 0)
>> > ? ? ? ? ? ? ? ?return -ENODEV;
>> >
>> > - ? ? ? /* allow only one user at a time */
>> > - ? ? ? if (atomic_inc_and_test(&ofb->usage))
>> > - ? ? ? ? ? ? ? return -EBUSY;
>> > + ? ? ? if (ofb->usage++ == 0)
>> > + ? ? ? ? ? ? ? /* unblank the base framebuffer */
>> > + ? ? ? ? ? ? ? fb_blank(&ofb->fbi->fb, FB_BLANK_UNBLANK);
>>
>> The change above allows multiple user at a time? Then I guess
>> some other places need to be changed accordingly to avoid the
>> racing conditions.
>
> You can't prevent multiple users. ?Think threaded applications which
> share the same set of fds.
>
> Any driver which tries to do so by restricting the number of open()s is
> simply buggy.
>
OK, let's go ahead fix the racing conditions later.
^ permalink raw reply
* [PATCH] ARM: S5PC210: add support for i2c PMICs on Universal_C210 board
From: Russell King - ARM Linux @ 2011-02-15 11:35 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1297767555-13326-1-git-send-email-m.szyprowski@samsung.com>
On Tue, Feb 15, 2011 at 11:59:15AM +0100, Marek Szyprowski wrote:
> #include <mach/map.h>
> +#include <mach/gpio.h>
Need I say anything about this?
^ permalink raw reply
* [PATCH] ARM: errata: pl310 cache sync operation may be faulty
From: Russell King - ARM Linux @ 2011-02-15 11:34 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1297768683-30273-1-git-send-email-srinidhi.kasagar@stericsson.com>
On Tue, Feb 15, 2011 at 04:48:03PM +0530, srinidhi kasagar wrote:
> +#ifdef ARM_ERRATA_753970
> +#define L2X0_DUMMY_REG 0x740
> + /* write to an unmmapped register */
> + writel_relaxed(0, base + L2X0_DUMMY_REG);
> + cache_wait(base + L2X0_CACHE_SYNC, 1);
> +#else
> writel_relaxed(0, base + L2X0_CACHE_SYNC);
> cache_wait(base + L2X0_CACHE_SYNC, 1);
> +#endif
So why wrap cache_wait() up in that horrible ifdef as well - and why not
put the dummy register definition along side the other register definitions?
^ permalink raw reply
* [RFC PATCH 2/2] ARMv7: Invalidate the TLB before freeing page tables
From: Russell King - ARM Linux @ 2011-02-15 11:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1297767748.14691.15.camel@e102109-lin.cambridge.arm.com>
On Tue, Feb 15, 2011 at 11:02:28AM +0000, Catalin Marinas wrote:
> On Tue, 2011-02-15 at 10:31 +0000, Russell King - ARM Linux wrote:
> > We already have support for doing this, and Peter Zijlstra posted patches
> > to convert ARM to use a generic implementation of the TLB shootdown code.
> >
> > http://marc.info/?l=linux-kernel&m=129604765010347&w=2
> >
> > Does this patch solve your problem?
>
> I don't think it does. Peter's patch moves the ARM TLB support to the
> generic one which is a good clean-up, however it doesn't look like
> anything is invalidating the TLB entry between pmd_clear() and
> pte_free(), only after.
Can we please stop thinking "ooh, lets use generic code, that's good" ?
It's a rediculous attitude. The reason we have our own version is because
the generic version is too heavy for our UP implementations and it doesn't
do what we need to do there.
I believe we should stick with our own version for UP, and switch over to
the generic version for SMP.
> This is too late because we may speculatively
> get a global TLB entry (which isn't invalidated by the ASID TLB
> operations). So with Peter's patch we still have to implement
> __pte_free_tlb().
The point of TLB shootdown is that we unmap the entries from the page
tables, then issue the TLB flushes, and then free the pages and page
tables after that. All that Peter's patch tries to do is to get ARM to
use the generic stuff.
^ permalink raw reply
* [PATCH v2] OMAP: IOMMU: add missing function declaration
From: David Cohen @ 2011-02-15 11:31 UTC (permalink / raw)
To: linux-arm-kernel
Declaration of exported function 'iopgtable_lookup_entry' is missing from
header file. Currently we have a sparse warning as it's not being used
externally. Adding its declaration to avoid such warning and allow its usage
in future.
Signed-off-by: David Cohen <dacohen@gmail.com>
---
arch/arm/plat-omap/include/plat/iommu.h | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/arch/arm/plat-omap/include/plat/iommu.h b/arch/arm/plat-omap/include/plat/iommu.h
index 69230d6..19cbb5e 100644
--- a/arch/arm/plat-omap/include/plat/iommu.h
+++ b/arch/arm/plat-omap/include/plat/iommu.h
@@ -154,6 +154,8 @@ extern void flush_iotlb_range(struct iommu *obj, u32 start, u32 end);
extern void flush_iotlb_all(struct iommu *obj);
extern int iopgtable_store_entry(struct iommu *obj, struct iotlb_entry *e);
+extern void iopgtable_lookup_entry(struct iommu *obj, u32 da, u32 **ppgd,
+ u32 **ppte);
extern size_t iopgtable_clear_entry(struct iommu *obj, u32 iova);
extern int iommu_set_da_range(struct iommu *obj, u32 start, u32 end);
--
1.7.2.3
^ permalink raw reply related
* [PATCH] ARM: Improve the L2 cache performance when PL310 is used
From: Srinidhi Kasagar @ 2011-02-15 11:29 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <EAF47CD23C76F840A9E7FCE10091EFAB02C47A6ADF@dbde02.ent.ti.com>
Just curious to know, why the spinlock surrounding
l2x0_cache_sync still exists? I see that
Catalin's first version adds void lock for PL310
as they are atomic.
am I missing some discussion thread which I have
not noticed?
srinidhi
On Sat, Mar 13, 2010 at 7:04 PM, Shilimkar, Santosh
<santosh.shilimkar@ti.com> wrote:
>> -----Original Message-----
>> From: Catalin Marinas [mailto:catalin.marinas at arm.com]
>> Sent: Tuesday, March 09, 2010 3:45 PM
>> To: linux-arm-kernel at lists.infradead.org
>> Cc: Shilimkar, Santosh; Russell King
>> Subject: [PATCH] ARM: Improve the L2 cache performance when PL310 is used
>>
>> With this L2 cache controller, the cache maintenance by PA and sync
>> operations are atomic and do not require a "wait" loop or spinlocks.
>> This patch conditionally defines the cache_wait() function and locking
>> primitives (rather than duplicating the functions or file).
>>
>> Since L2x0 cache controllers do not work with ARMv7 CPUs, the patch
>> automatically enables CACHE_PL310 when CPU_V7 is defined.
>>
>> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Santosh Shilimkar <santosh.shilimkar@ti.com>
>> Cc: Russell King <rmk@arm.linux.org.uk>
>> ---
>>
>> We did some benchmarks and the performance benefit of this patch with a
>> PL310 cache controller is considerable.
>>
> I tested this patch and indeed it improves the performance by
> 12 % to 28 % for buffers ranging from 1 MB to 10 MB.
>
>> I also considered separate functions in the same file or a separate file
>> but this would mean having to move the TI's workaround to a new file as
>> well. Suggestions welcome.
>>
>>
>> ?arch/arm/mm/Kconfig ? ? ?| ? ?7 +++++
>> ?arch/arm/mm/cache-l2x0.c | ? 70 ++++++++++++++++++++++++++++++++--------------
>> ?2 files changed, 56 insertions(+), 21 deletions(-)
>>
>> diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig
>> index ef61c93..0a59071 100644
>> --- a/arch/arm/mm/Kconfig
>> +++ b/arch/arm/mm/Kconfig
>> @@ -781,6 +781,13 @@ config CACHE_L2X0
>> ? ? ? help
>> ? ? ? ? This option enables the L2x0 PrimeCell.
>>
>> +config CACHE_PL310
>> + ? ? bool
>> + ? ? depends on CACHE_L2X0
>> + ? ? default y if CPU_V7
>> + ? ? help
>> + ? ? ? This option enables support for the PL310 cache controller.
>> +
>> ?config CACHE_TAUROS2
>> ? ? ? bool "Enable the Tauros2 L2 cache controller"
>> ? ? ? depends on ARCH_DOVE
>> diff --git a/arch/arm/mm/cache-l2x0.c b/arch/arm/mm/cache-l2x0.c
>> index 9fc01b4..2a20d95 100644
>> --- a/arch/arm/mm/cache-l2x0.c
>> +++ b/arch/arm/mm/cache-l2x0.c
>> @@ -26,9 +26,21 @@
>> ?#define CACHE_LINE_SIZE ? ? ? ? ? ? ?32
>>
>> ?static void __iomem *l2x0_base;
>> -static DEFINE_SPINLOCK(l2x0_lock);
>> ?bool l2x0_disabled;
>>
>> +#ifdef CONFIG_CACHE_PL310
>> +static inline void cache_wait(void __iomem *reg, unsigned long mask)
>> +{
>> + ? ? /* cache operations are atomic */
>> +}
>> +
>> +#define l2x0_lock(lock, flags) ? ? ? ? ? ? ? ((void)(flags))
>> +#define l2x0_unlock(lock, flags) ? ? ((void)(flags))
>> +
>> +#define block_end(start, end) ? ? ? ? ? ? ? ?(end)
>> +
>> +#define L2CC_TYPE ? ? ? ? ? ? ? ? ? ?"PL310/L2C-310"
>> +#else
>> ?static inline void cache_wait(void __iomem *reg, unsigned long mask)
>> ?{
>> ? ? ? /* wait for the operation to complete */
>> @@ -36,6 +48,22 @@ static inline void cache_wait(void __iomem *reg, unsigned long mask)
>> ? ? ? ? ? ? ? ;
>> ?}
>>
>> +static DEFINE_SPINLOCK(l2x0_lock);
>> +#define l2x0_lock(lock, flags) ? ? ? ? ? ? ? spin_lock_irqsave(lock, flags)
>> +#define l2x0_unlock(lock, flags) ? ? spin_unlock_irqrestore(lock, flags)
>> +
>> +#define block_end(start, end) ? ? ? ? ? ? ? ?((start) + min((end) - (start), 4096UL))
>> +
>> +#define L2CC_TYPE ? ? ? ? ? ? ? ? ? ?"L2x0"
>> +#endif
>> +
>> +static inline void cache_wait_always(void __iomem *reg, unsigned long mask)
>> +{
>> + ? ? /* wait for the operation to complete */
>> + ? ? while (readl(reg) & mask)
>> + ? ? ? ? ? ? ;
>> +}
>> +
>> ?static inline void cache_sync(void)
>> ?{
>> ? ? ? void __iomem *base = l2x0_base;
>> @@ -99,11 +127,11 @@ static inline void l2x0_inv_all(void)
>> ? ? ? unsigned long flags;
>>
>> ? ? ? /* invalidate all ways */
>> - ? ? spin_lock_irqsave(&l2x0_lock, flags);
>> + ? ? l2x0_lock(&l2x0_lock, flags);
>> ? ? ? writel(0xff, l2x0_base + L2X0_INV_WAY);
>> - ? ? cache_wait(l2x0_base + L2X0_INV_WAY, 0xff);
>> + ? ? cache_wait_always(l2x0_base + L2X0_INV_WAY, 0xff);
>> ? ? ? cache_sync();
>> - ? ? spin_unlock_irqrestore(&l2x0_lock, flags);
>> + ? ? l2x0_unlock(&l2x0_lock, flags);
>> ?}
>>
>> ?static void l2x0_inv_range(unsigned long start, unsigned long end)
>> @@ -111,7 +139,7 @@ static void l2x0_inv_range(unsigned long start, unsigned long end)
>> ? ? ? void __iomem *base = l2x0_base;
>> ? ? ? unsigned long flags;
>>
>> - ? ? spin_lock_irqsave(&l2x0_lock, flags);
>> + ? ? l2x0_lock(&l2x0_lock, flags);
>> ? ? ? if (start & (CACHE_LINE_SIZE - 1)) {
>> ? ? ? ? ? ? ? start &= ~(CACHE_LINE_SIZE - 1);
>> ? ? ? ? ? ? ? debug_writel(0x03);
>> @@ -128,7 +156,7 @@ static void l2x0_inv_range(unsigned long start, unsigned long end)
>> ? ? ? }
>>
>> ? ? ? while (start < end) {
>> - ? ? ? ? ? ? unsigned long blk_end = start + min(end - start, 4096UL);
>> + ? ? ? ? ? ? unsigned long blk_end = block_end(start, end);
>>
>> ? ? ? ? ? ? ? while (start < blk_end) {
>> ? ? ? ? ? ? ? ? ? ? ? l2x0_inv_line(start);
>> @@ -136,13 +164,13 @@ static void l2x0_inv_range(unsigned long start, unsigned long end)
>> ? ? ? ? ? ? ? }
>>
>> ? ? ? ? ? ? ? if (blk_end < end) {
>> - ? ? ? ? ? ? ? ? ? ? spin_unlock_irqrestore(&l2x0_lock, flags);
>> - ? ? ? ? ? ? ? ? ? ? spin_lock_irqsave(&l2x0_lock, flags);
>> + ? ? ? ? ? ? ? ? ? ? l2x0_unlock(&l2x0_lock, flags);
>> + ? ? ? ? ? ? ? ? ? ? l2x0_lock(&l2x0_lock, flags);
>> ? ? ? ? ? ? ? }
>> ? ? ? }
>> ? ? ? cache_wait(base + L2X0_INV_LINE_PA, 1);
>> ? ? ? cache_sync();
>> - ? ? spin_unlock_irqrestore(&l2x0_lock, flags);
>> + ? ? l2x0_unlock(&l2x0_lock, flags);
>> ?}
>>
>> ?static void l2x0_clean_range(unsigned long start, unsigned long end)
>> @@ -150,10 +178,10 @@ static void l2x0_clean_range(unsigned long start, unsigned long end)
>> ? ? ? void __iomem *base = l2x0_base;
>> ? ? ? unsigned long flags;
>>
>> - ? ? spin_lock_irqsave(&l2x0_lock, flags);
>> + ? ? l2x0_lock(&l2x0_lock, flags);
>> ? ? ? start &= ~(CACHE_LINE_SIZE - 1);
>> ? ? ? while (start < end) {
>> - ? ? ? ? ? ? unsigned long blk_end = start + min(end - start, 4096UL);
>> + ? ? ? ? ? ? unsigned long blk_end = block_end(start, end);
>>
>> ? ? ? ? ? ? ? while (start < blk_end) {
>> ? ? ? ? ? ? ? ? ? ? ? l2x0_clean_line(start);
>> @@ -161,13 +189,13 @@ static void l2x0_clean_range(unsigned long start, unsigned long end)
>> ? ? ? ? ? ? ? }
>>
>> ? ? ? ? ? ? ? if (blk_end < end) {
>> - ? ? ? ? ? ? ? ? ? ? spin_unlock_irqrestore(&l2x0_lock, flags);
>> - ? ? ? ? ? ? ? ? ? ? spin_lock_irqsave(&l2x0_lock, flags);
>> + ? ? ? ? ? ? ? ? ? ? l2x0_unlock(&l2x0_lock, flags);
>> + ? ? ? ? ? ? ? ? ? ? l2x0_lock(&l2x0_lock, flags);
>> ? ? ? ? ? ? ? }
>> ? ? ? }
>> ? ? ? cache_wait(base + L2X0_CLEAN_LINE_PA, 1);
>> ? ? ? cache_sync();
>> - ? ? spin_unlock_irqrestore(&l2x0_lock, flags);
>> + ? ? l2x0_unlock(&l2x0_lock, flags);
>> ?}
>>
>> ?static void l2x0_flush_range(unsigned long start, unsigned long end)
>> @@ -175,10 +203,10 @@ static void l2x0_flush_range(unsigned long start, unsigned long end)
>> ? ? ? void __iomem *base = l2x0_base;
>> ? ? ? unsigned long flags;
>>
>> - ? ? spin_lock_irqsave(&l2x0_lock, flags);
>> + ? ? l2x0_lock(&l2x0_lock, flags);
>> ? ? ? start &= ~(CACHE_LINE_SIZE - 1);
>> ? ? ? while (start < end) {
>> - ? ? ? ? ? ? unsigned long blk_end = start + min(end - start, 4096UL);
>> + ? ? ? ? ? ? unsigned long blk_end = block_end(start, end);
>>
>> ? ? ? ? ? ? ? debug_writel(0x03);
>> ? ? ? ? ? ? ? while (start < blk_end) {
>> @@ -188,13 +216,13 @@ static void l2x0_flush_range(unsigned long start, unsigned long end)
>> ? ? ? ? ? ? ? debug_writel(0x00);
>>
>> ? ? ? ? ? ? ? if (blk_end < end) {
>> - ? ? ? ? ? ? ? ? ? ? spin_unlock_irqrestore(&l2x0_lock, flags);
>> - ? ? ? ? ? ? ? ? ? ? spin_lock_irqsave(&l2x0_lock, flags);
>> + ? ? ? ? ? ? ? ? ? ? l2x0_unlock(&l2x0_lock, flags);
>> + ? ? ? ? ? ? ? ? ? ? l2x0_lock(&l2x0_lock, flags);
>> ? ? ? ? ? ? ? }
>> ? ? ? }
>> ? ? ? cache_wait(base + L2X0_CLEAN_INV_LINE_PA, 1);
>> ? ? ? cache_sync();
>> - ? ? spin_unlock_irqrestore(&l2x0_lock, flags);
>> + ? ? l2x0_unlock(&l2x0_lock, flags);
>> ?}
>>
>> ?void __init l2x0_init(void __iomem *base, __u32 aux_val, __u32 aux_mask)
>> @@ -202,7 +230,7 @@ void __init l2x0_init(void __iomem *base, __u32 aux_val, __u32 aux_mask)
>> ? ? ? __u32 aux;
>>
>> ? ? ? if (l2x0_disabled) {
>> - ? ? ? ? ? ? printk(KERN_INFO "L2X0 cache controller disabled\n");
>> + ? ? ? ? ? ? pr_info(L2CC_TYPE " cache controller disabled\n");
>> ? ? ? ? ? ? ? return;
>> ? ? ? }
>>
>> @@ -232,7 +260,7 @@ void __init l2x0_init(void __iomem *base, __u32 aux_val, __u32 aux_mask)
>> ? ? ? outer_cache.clean_range = l2x0_clean_range;
>> ? ? ? outer_cache.flush_range = l2x0_flush_range;
>>
>> - ? ? printk(KERN_INFO "L2X0 cache controller enabled\n");
>> + ? ? pr_info(L2CC_TYPE " cache controller enabled\n");
>> ?}
>>
>> ?static int __init l2x0_disable(char *unused)
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
^ permalink raw reply
* [PATCH] ARM: errata: pl310 cache sync operation may be faulty
From: srinidhi kasagar @ 2011-02-15 11:18 UTC (permalink / raw)
To: linux-arm-kernel
The effect of cache sync operation is to drain the store
buffer and wait for all internal buffers to be empty. In
normal conditions, store buffer is able to merge the
normal memory writes within its 32-byte data buffers.
Due to this erratum present in r3p0, the effect of cache
sync operation on the store buffer still remains when
the operation completes. This means that the store buffer
is always asked to drain and this prevents it from merging
any further writes.
This can severely affect performance on the write traffic
esp. on Normal memory NC one.
The proposed workaround is to replace the normal offset of
cache sync operation(0x730) by another offset targeting an
unmapped PL310 register 0x740.
Signed-off-by: srinidhi kasagar <srinidhi.kasagar@stericsson.com>
Acked-by: Linus Walleij <linus.walleij@stericsson.com>
---
arch/arm/Kconfig | 15 +++++++++++++++
arch/arm/mm/cache-l2x0.c | 8 ++++++++
2 files changed, 23 insertions(+), 0 deletions(-)
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index d5eb308..f1946e4 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -1186,6 +1186,21 @@ config ARM_ERRATA_743622
visible impact on the overall performance or power consumption of the
processor.
+config ARM_ERRATA_753970
+ bool "ARM errata: cache sync operation may be faulty"
+ depends on CACHE_PL310
+ help
+ This option enables the workaround for the 753970 PL310 erratum.
+
+ Under some condition the effect of cache sync operation on
+ the store buffer still remains when the operation completes.
+ This means that the store buffer is always asked to drain and
+ this prevents it from merging any further writes. The workaround
+ is to replace the normal offset of cache sync operation (0x730)
+ by another offset targeting an unmapped PL310 register 0x740.
+ This has the same effect as the cache sync operation: store buffer
+ drain and waiting for all buffers empty.
+
endmenu
source "arch/arm/common/Kconfig"
diff --git a/arch/arm/mm/cache-l2x0.c b/arch/arm/mm/cache-l2x0.c
index 170c9bb..998d521 100644
--- a/arch/arm/mm/cache-l2x0.c
+++ b/arch/arm/mm/cache-l2x0.c
@@ -49,8 +49,16 @@ static inline void cache_wait(void __iomem *reg, unsigned long mask)
static inline void cache_sync(void)
{
void __iomem *base = l2x0_base;
+
+#ifdef ARM_ERRATA_753970
+#define L2X0_DUMMY_REG 0x740
+ /* write to an unmmapped register */
+ writel_relaxed(0, base + L2X0_DUMMY_REG);
+ cache_wait(base + L2X0_CACHE_SYNC, 1);
+#else
writel_relaxed(0, base + L2X0_CACHE_SYNC);
cache_wait(base + L2X0_CACHE_SYNC, 1);
+#endif
}
static inline void l2x0_clean_line(unsigned long addr)
--
1.7.0.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox