* Re: [patch] crypto: chcr - checking for IS_ERR() instead of NULL
From: Harsh Jain @ 2016-12-02 6:12 UTC (permalink / raw)
To: Dan Carpenter, Herbert Xu
Cc: David S. Miller, Jitendra Lulla, Atul Gupta, Hariprasad Shenai,
Wei Yongjun, linux-crypto, kernel-janitors
In-Reply-To: <d2933f1a-e15e-8c33-2905-3f8161b0db7d@chelsio.com>
On 02-12-2016 11:29, Harsh Jain wrote:
>
> On 02-12-2016 02:19, Dan Carpenter wrote:
>> The create_hash_wr() function never returns error pointers. It returns
>> NULL on error.
> Will fix the same, Thanks
It's a patch mail not bug reporting! Sorry for confusion. You have already fixed it. Thanks.
>> Fixes: 358961d1cd1e ("crypto: chcr - Added new structure chcr_wr")
>> Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
>>
>> diff --git a/drivers/crypto/chelsio/chcr_algo.c b/drivers/crypto/chelsio/chcr_algo.c
>> index 32361dd..2ed1e24 100644
>> --- a/drivers/crypto/chelsio/chcr_algo.c
>> +++ b/drivers/crypto/chelsio/chcr_algo.c
>> @@ -958,9 +958,8 @@ static int chcr_ahash_update(struct ahash_request *req)
>> req_ctx->result = 0;
>> req_ctx->data_len += params.sg_len + params.bfr_len;
>> skb = create_hash_wr(req, ¶ms);
>> -
>> - if (IS_ERR(skb))
>> - return PTR_ERR(skb);
>> + if (!skb)
>> + return -ENOMEM;
>>
>> if (remainder) {
>> u8 *temp;
>> @@ -1023,8 +1022,8 @@ static int chcr_ahash_final(struct ahash_request *req)
>> params.more = 0;
>> }
>> skb = create_hash_wr(req, ¶ms);
>> - if (IS_ERR(skb))
>> - return PTR_ERR(skb);
>> + if (!skb)
>> + return -ENOMEM;
>>
>> skb->dev = u_ctx->lldi.ports[0];
>> set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_channel_id);
>> @@ -1074,8 +1073,8 @@ static int chcr_ahash_finup(struct ahash_request *req)
>> }
>>
>> skb = create_hash_wr(req, ¶ms);
>> - if (IS_ERR(skb))
>> - return PTR_ERR(skb);
>> + if (!skb)
>> + return -ENOMEM;
>>
>> skb->dev = u_ctx->lldi.ports[0];
>> set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_channel_id);
>> @@ -1125,8 +1124,8 @@ static int chcr_ahash_digest(struct ahash_request *req)
>> }
>>
>> skb = create_hash_wr(req, ¶ms);
>> - if (IS_ERR(skb))
>> - return PTR_ERR(skb);
>> + if (!skb)
>> + return -ENOMEM;
>>
>> skb->dev = u_ctx->lldi.ports[0];
>> set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_channel_id);
^ permalink raw reply
* [PATCH 0/2] Add MediaTek crypto acclelrator driver
From: Ryder Lee @ 2016-12-02 3:26 UTC (permalink / raw)
To: Herbert Xu, David S. Miller, Matthias Brugger
Cc: devicetree, linux-mediatek, linux-kernel, linux-crypto,
linux-arm-kernel, Sean Wang, Ryder Lee
This adds support for the MediaTek hardware accelerator on
mt7623 SoC.
This driver currently implement:
- SHA1 and SHA2 family(HMAC) hash alogrithms.
- AES block cipher in CBC/ECB mode with 128/196/256 bits keys.
Ryder Lee (2):
Add crypto driver support for some MediaTek chips
crypto: mediatek - add DT bindings documentation
.../devicetree/bindings/crypto/mediatek-crypto.txt | 32 +
drivers/crypto/Kconfig | 17 +
drivers/crypto/Makefile | 1 +
drivers/crypto/mediatek/Makefile | 2 +
drivers/crypto/mediatek/mtk-aes.c | 734 +++++++++++
drivers/crypto/mediatek/mtk-platform.c | 575 ++++++++
drivers/crypto/mediatek/mtk-platform.h | 230 ++++
drivers/crypto/mediatek/mtk-regs.h | 194 +++
drivers/crypto/mediatek/mtk-sha.c | 1384 ++++++++++++++++++++
9 files changed, 3169 insertions(+)
create mode 100644 Documentation/devicetree/bindings/crypto/mediatek-crypto.txt
create mode 100644 drivers/crypto/mediatek/Makefile
create mode 100644 drivers/crypto/mediatek/mtk-aes.c
create mode 100644 drivers/crypto/mediatek/mtk-platform.c
create mode 100644 drivers/crypto/mediatek/mtk-platform.h
create mode 100644 drivers/crypto/mediatek/mtk-regs.h
create mode 100644 drivers/crypto/mediatek/mtk-sha.c
--
1.9.1
^ permalink raw reply
* [PATCH 2/2] crypto: mediatek - add DT bindings documentation
From: Ryder Lee @ 2016-12-02 3:26 UTC (permalink / raw)
To: Herbert Xu, David S. Miller, Matthias Brugger
Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-crypto-u79uwXL29TY76Z2rM5mHXA,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Sean Wang,
Ryder Lee
In-Reply-To: <1480649205-52695-1-git-send-email-ryder.lee-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
Add DT bindings documentation for the crypto driver
Signed-off-by: Ryder Lee <ryder.lee-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
.../devicetree/bindings/crypto/mediatek-crypto.txt | 32 ++++++++++++++++++++++
1 file changed, 32 insertions(+)
create mode 100644 Documentation/devicetree/bindings/crypto/mediatek-crypto.txt
diff --git a/Documentation/devicetree/bindings/crypto/mediatek-crypto.txt b/Documentation/devicetree/bindings/crypto/mediatek-crypto.txt
new file mode 100644
index 0000000..8b1db08
--- /dev/null
+++ b/Documentation/devicetree/bindings/crypto/mediatek-crypto.txt
@@ -0,0 +1,32 @@
+MediaTek cryptographic accelerators
+
+Required properties:
+- compatible: Should be "mediatek,mt7623-crypto"
+- reg: Address and length of the register set for the device
+- interrupts: Should contain the five crypto engines interrupts in numeric
+ order. These are global system and four descriptor rings.
+- clocks: the clock used by the core
+- clock-names: the names of the clock listed in the clocks property. These are
+ "ethif", "cryp"
+- power-domains: Must contain a reference to the PM domain.
+
+
+Optional properties:
+- interrupt-parent: Should be the phandle for the interrupt controller
+ that services interrupts for this device
+
+
+Example:
+ crypto: crypto@1b240000 {
+ compatible = "mediatek,mt7623-crypto";
+ reg = <0 0x1b240000 0 0x20000>;
+ interrupts = <GIC_SPI 82 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 83 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 84 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 91 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 97 IRQ_TYPE_LEVEL_LOW>;
+ clocks = <&topckgen CLK_TOP_ETHIF_SEL>,
+ <ðsys CLK_ETHSYS_CRYPTO>;
+ clock-names = "ethif","cryp";
+ power-domains = <&scpsys MT2701_POWER_DOMAIN_ETH>;
+ };
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 1/2] Add crypto driver support for some MediaTek chips
From: Ryder Lee @ 2016-12-02 3:26 UTC (permalink / raw)
To: Herbert Xu, David S. Miller, Matthias Brugger
Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-crypto-u79uwXL29TY76Z2rM5mHXA,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Sean Wang,
Ryder Lee
In-Reply-To: <1480649205-52695-1-git-send-email-ryder.lee-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
This adds support for the MediaTek hardware accelerator on
mt7623/mt2701/mt8521p SoC.
This driver currently implement:
- SHA1 and SHA2 family(HMAC) hash alogrithms.
- AES block cipher in CBC/ECB mode with 128/196/256 bits keys.
Signed-off-by: Ryder Lee <ryder.lee-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
drivers/crypto/Kconfig | 17 +
drivers/crypto/Makefile | 1 +
drivers/crypto/mediatek/Makefile | 2 +
drivers/crypto/mediatek/mtk-aes.c | 734 +++++++++++++++++
drivers/crypto/mediatek/mtk-platform.c | 575 +++++++++++++
drivers/crypto/mediatek/mtk-platform.h | 230 ++++++
drivers/crypto/mediatek/mtk-regs.h | 194 +++++
drivers/crypto/mediatek/mtk-sha.c | 1384 ++++++++++++++++++++++++++++++++
8 files changed, 3137 insertions(+)
create mode 100644 drivers/crypto/mediatek/Makefile
create mode 100644 drivers/crypto/mediatek/mtk-aes.c
create mode 100644 drivers/crypto/mediatek/mtk-platform.c
create mode 100644 drivers/crypto/mediatek/mtk-platform.h
create mode 100644 drivers/crypto/mediatek/mtk-regs.h
create mode 100644 drivers/crypto/mediatek/mtk-sha.c
diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index 4d2b81f..5d9c803 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig
@@ -553,6 +553,23 @@ config CRYPTO_DEV_ROCKCHIP
This driver interfaces with the hardware crypto accelerator.
Supporting cbc/ecb chainmode, and aes/des/des3_ede cipher mode.
+config CRYPTO_DEV_MEDIATEK
+ tristate "MediaTek's Cryptographic Engine driver"
+ depends on ARM && ARCH_MEDIATEK
+ select NEON
+ select KERNEL_MODE_NEON
+ select ARM_CRYPTO
+ select CRYPTO_AES
+ select CRYPTO_BLKCIPHER
+ select CRYPTO_SHA1_ARM_NEON
+ select CRYPTO_SHA256_ARM
+ select CRYPTO_SHA512_ARM
+ select CRYPTO_HMAC
+ help
+ This driver allows you to utilize the hardware crypto accelerator
+ which can be found on the MT7623 MT2701, MT8521p, etc ....
+ Select this if you want to use it for AES/SHA1/SHA2 algorithms.
+
source "drivers/crypto/chelsio/Kconfig"
endif # CRYPTO_HW
diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile
index ad7250f..272b51a 100644
--- a/drivers/crypto/Makefile
+++ b/drivers/crypto/Makefile
@@ -10,6 +10,7 @@ obj-$(CONFIG_CRYPTO_DEV_IMGTEC_HASH) += img-hash.o
obj-$(CONFIG_CRYPTO_DEV_IXP4XX) += ixp4xx_crypto.o
obj-$(CONFIG_CRYPTO_DEV_MV_CESA) += mv_cesa.o
obj-$(CONFIG_CRYPTO_DEV_MARVELL_CESA) += marvell/
+obj-$(CONFIG_CRYPTO_DEV_MEDIATEK) += mediatek/
obj-$(CONFIG_CRYPTO_DEV_MXS_DCP) += mxs-dcp.o
obj-$(CONFIG_CRYPTO_DEV_NIAGARA2) += n2_crypto.o
n2_crypto-y := n2_core.o n2_asm.o
diff --git a/drivers/crypto/mediatek/Makefile b/drivers/crypto/mediatek/Makefile
new file mode 100644
index 0000000..187be79
--- /dev/null
+++ b/drivers/crypto/mediatek/Makefile
@@ -0,0 +1,2 @@
+obj-$(CONFIG_CRYPTO_DEV_MEDIATEK) += mtk-crypto.o
+mtk-crypto-objs:= mtk-platform.o mtk-aes.o mtk-sha.o
diff --git a/drivers/crypto/mediatek/mtk-aes.c b/drivers/crypto/mediatek/mtk-aes.c
new file mode 100644
index 0000000..feb0e57
--- /dev/null
+++ b/drivers/crypto/mediatek/mtk-aes.c
@@ -0,0 +1,734 @@
+/*
+ * Cryptographic API.
+ *
+ * Support for MediaTek AES hardware accelerator.
+ *
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: Ryder Lee <ryder.lee-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * Some ideas are from atmel-aes.c drivers.
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/scatterlist.h>
+#include <crypto/scatterwalk.h>
+#include <crypto/algapi.h>
+#include <crypto/aes.h>
+#include "mtk-platform.h"
+#include "mtk-regs.h"
+
+#define AES_QUEUE_LENGTH 512
+#define AES_BUFFER_ORDER 2
+#define AES_BUFFER_SIZE ((PAGE_SIZE << AES_BUFFER_ORDER) \
+ & ~(AES_BLOCK_SIZE - 1))
+
+/* AES command token */
+#define AES_CT_SIZE_ECB 2
+#define AES_CT_SIZE_CBC 3
+#define AES_CT_CTRL_HDR 0x00220000
+#define AES_COMMAND0 0x05000000
+#define AES_COMMAND1 0x2d060000
+#define AES_COMMAND2 0xe4a63806
+
+/* AES transform information */
+#define AES_TFM_ECB (0x0 << 0)
+#define AES_TFM_CBC (0x1 << 0)
+#define AES_TFM_DECRYPT (0x5 << 0)
+#define AES_TFM_ENCRYPT (0x4 << 0)
+#define AES_TFM_SIZE(x) ((x) << 8)
+#define AES_TFM_128BITS (0xb << 16)
+#define AES_TFM_192BITS (0xd << 16)
+#define AES_TFM_256BITS (0xf << 16)
+#define AES_TFM_FULL_IV (0xf << 5)
+
+/* AES flags */
+#define AES_FLAGS_MODE_MSK GENMASK(2, 0)
+#define AES_FLAGS_ECB BIT(0)
+#define AES_FLAGS_CBC BIT(1)
+#define AES_FLAGS_ENCRYPT BIT(2)
+#define AES_FLAGS_BUSY BIT(3)
+
+/* AES command token */
+struct mtk_aes_ct {
+ u32 ct_ctrl0;
+ u32 ct_ctrl1;
+ u32 ct_ctrl2;
+};
+
+/* AES transform info */
+struct mtk_aes_tfm {
+ u32 tfm_ctrl0;
+ u32 tfm_ctrl1;
+
+ /* keys and IVs */
+ u8 state[AES_KEYSIZE_256 + AES_BLOCK_SIZE] __aligned(sizeof(u32));
+};
+
+struct mtk_aes_info {
+ struct mtk_aes_ct ct;
+ struct mtk_aes_tfm tfm;
+};
+
+struct mtk_aes_reqctx {
+ u64 mode;
+};
+
+struct mtk_aes_ctx {
+ struct mtk_cryp *cryp;
+ struct mtk_aes_info info;
+ u32 keylen;
+
+ unsigned long flags;
+};
+
+struct mtk_aes_drv {
+ struct list_head dev_list;
+ /* device list lock */
+ spinlock_t lock;
+};
+
+static struct mtk_aes_drv mtk_aes = {
+ .dev_list = LIST_HEAD_INIT(mtk_aes.dev_list),
+ .lock = __SPIN_LOCK_UNLOCKED(mtk_aes.lock),
+};
+
+static inline u32 mtk_aes_read(struct mtk_cryp *cryp, u32 offset)
+{
+ return readl_relaxed(cryp->base + offset);
+}
+
+static inline void mtk_aes_write(struct mtk_cryp *cryp,
+ u32 offset, u32 value)
+{
+ writel_relaxed(value, cryp->base + offset);
+}
+
+static struct mtk_cryp *mtk_aes_find_dev(struct mtk_aes_ctx *ctx)
+{
+ struct mtk_cryp *cryp = NULL;
+ struct mtk_cryp *tmp;
+
+ spin_lock_bh(&mtk_aes.lock);
+ if (!ctx->cryp) {
+ list_for_each_entry(tmp, &mtk_aes.dev_list, aes_list) {
+ cryp = tmp;
+ break;
+ }
+ ctx->cryp = cryp;
+ } else {
+ cryp = ctx->cryp;
+ }
+ spin_unlock_bh(&mtk_aes.lock);
+
+ return cryp;
+}
+
+static inline size_t mtk_aes_padlen(size_t len)
+{
+ len &= AES_BLOCK_SIZE - 1;
+ return len ? AES_BLOCK_SIZE - len : 0;
+}
+
+static bool mtk_aes_check_aligned(struct scatterlist *sg,
+ size_t len, struct mtk_aes_dma *dma)
+{
+ int nents;
+
+ if (!IS_ALIGNED(len, AES_BLOCK_SIZE))
+ return false;
+
+ for (nents = 0; sg; sg = sg_next(sg), ++nents) {
+ if (!IS_ALIGNED(sg->offset, sizeof(u32)))
+ return false;
+
+ if (len <= sg->length) {
+ if (!IS_ALIGNED(len, AES_BLOCK_SIZE))
+ return false;
+
+ dma->nents = nents + 1;
+ dma->remainder = sg->length - len;
+ sg->length = len;
+ return true;
+ }
+
+ if (!IS_ALIGNED(sg->length, AES_BLOCK_SIZE))
+ return false;
+
+ len -= sg->length;
+ }
+
+ return false;
+}
+
+static int mtk_aes_info_map(struct mtk_cryp *cryp,
+ struct mtk_aes *aes, size_t len)
+{
+ struct mtk_aes_ctx *ctx = crypto_ablkcipher_ctx(
+ crypto_ablkcipher_reqtfm(aes->req));
+ struct mtk_aes_info *info = aes->info;
+ struct mtk_aes_ct *ct = &info->ct;
+ struct mtk_aes_tfm *tfm = &info->tfm;
+ u32 keylen = ctx->keylen;
+
+ aes->ct_hdr = AES_CT_CTRL_HDR | len;
+ ct->ct_ctrl0 = AES_COMMAND0 | len;
+ ct->ct_ctrl1 = AES_COMMAND1;
+
+ if (aes->flags & AES_FLAGS_ENCRYPT)
+ tfm->tfm_ctrl0 = AES_TFM_ENCRYPT;
+ else
+ tfm->tfm_ctrl0 = AES_TFM_DECRYPT;
+
+ if (aes->flags & AES_FLAGS_CBC) {
+ aes->ct_size = AES_CT_SIZE_CBC;
+ ct->ct_ctrl2 = AES_COMMAND2;
+
+ tfm->tfm_ctrl0 |= AES_TFM_SIZE(WORD(keylen + AES_BLOCK_SIZE));
+ tfm->tfm_ctrl1 = AES_TFM_CBC;
+ tfm->tfm_ctrl1 |= AES_TFM_FULL_IV;
+
+ memcpy(tfm->state + keylen, aes->req->info, AES_BLOCK_SIZE);
+ } else if (aes->flags & AES_FLAGS_ECB) {
+ aes->ct_size = AES_CT_SIZE_ECB;
+ tfm->tfm_ctrl0 |= AES_TFM_SIZE(WORD(keylen));
+ tfm->tfm_ctrl1 = AES_TFM_ECB;
+ }
+
+ if (keylen == AES_KEYSIZE_128)
+ tfm->tfm_ctrl0 |= AES_TFM_128BITS;
+ else if (keylen == AES_KEYSIZE_256)
+ tfm->tfm_ctrl0 |= AES_TFM_256BITS;
+ else if (keylen == AES_KEYSIZE_192)
+ tfm->tfm_ctrl0 |= AES_TFM_192BITS;
+
+ aes->ct_dma = dma_map_single(cryp->dev, info, sizeof(*info),
+ DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(cryp->dev, aes->ct_dma))) {
+ dev_err(cryp->dev, "dma %d bytes error\n", sizeof(*info));
+ return -EINVAL;
+ }
+ aes->tfm_dma = aes->ct_dma + sizeof(*ct);
+
+ return 0;
+}
+
+static int mtk_aes_xmit(struct mtk_cryp *cryp, struct mtk_aes *aes)
+{
+ struct mtk_ring *ring = cryp->ring[aes->id];
+ struct mtk_desc *cmd = NULL, *res = NULL;
+ struct scatterlist *ssg, *dsg;
+ u32 len = aes->src.sg_len;
+ int nents;
+
+ for (nents = 0; nents < len; ++nents) {
+ ssg = &aes->src.sg[nents];
+ dsg = &aes->dst.sg[nents];
+
+ cmd = ring->cmd_base + ring->pos;
+ res = ring->res_base + ring->pos;
+
+ res->hdr = MTK_DESC_BUF_LEN(dsg->length);
+ res->buf = sg_dma_address(dsg);
+
+ cmd->hdr = MTK_DESC_BUF_LEN(ssg->length);
+ cmd->buf = sg_dma_address(ssg);
+
+ if (nents == 0) {
+ res->hdr |= MTK_DESC_FIRST;
+ cmd->hdr |= MTK_DESC_FIRST;
+ cmd->hdr |= MTK_DESC_CT_LEN(aes->ct_size);
+ cmd->ct = aes->ct_dma;
+ cmd->ct_hdr = aes->ct_hdr;
+ cmd->tfm = aes->tfm_dma;
+ }
+
+ if (++ring->pos == MTK_MAX_DESC_NUM)
+ ring->pos = 0;
+ }
+
+ cmd->hdr |= MTK_DESC_LAST;
+ res->hdr |= MTK_DESC_LAST;
+
+ /* make sure all descriptor are filled done */
+ wmb();
+ mtk_aes_write(cryp, RDR_PREP_COUNT(aes->id), MTK_DESC_CNT(len));
+ mtk_aes_write(cryp, CDR_PREP_COUNT(aes->id), MTK_DESC_CNT(len));
+
+ return -EINPROGRESS;
+}
+
+static inline void mtk_aes_restore_sg(const struct mtk_aes_dma *dma)
+{
+ struct scatterlist *sg = dma->sg;
+ int nents = dma->nents;
+
+ if (!dma->remainder)
+ return;
+
+ while (--nents > 0 && sg)
+ sg = sg_next(sg);
+
+ if (!sg)
+ return;
+
+ sg->length += dma->remainder;
+}
+
+static int mtk_aes_map(struct mtk_cryp *cryp, struct mtk_aes *aes)
+{
+ struct scatterlist *src = aes->req->src;
+ struct scatterlist *dst = aes->req->dst;
+ size_t len = aes->req->nbytes;
+ size_t padlen = 0;
+ bool src_aligned, dst_aligned;
+
+ aes->total = len;
+ aes->src.sg = src;
+ aes->dst.sg = dst;
+ aes->real_dst = dst;
+
+ src_aligned = mtk_aes_check_aligned(src, len, &aes->src);
+ if (src == dst)
+ dst_aligned = src_aligned;
+ else
+ dst_aligned = mtk_aes_check_aligned(dst, len, &aes->dst);
+
+ if (!src_aligned || !dst_aligned) {
+ padlen = mtk_aes_padlen(len);
+
+ if (len + padlen > AES_BUFFER_SIZE)
+ return -ENOMEM;
+
+ if (!src_aligned) {
+ sg_copy_to_buffer(src, sg_nents(src), aes->buf, len);
+ aes->src.sg = &aes->aligned_sg;
+ aes->src.nents = 1;
+ aes->src.remainder = 0;
+ }
+
+ if (!dst_aligned) {
+ aes->dst.sg = &aes->aligned_sg;
+ aes->dst.nents = 1;
+ aes->dst.remainder = 0;
+ }
+
+ sg_init_table(&aes->aligned_sg, 1);
+ sg_set_buf(&aes->aligned_sg, aes->buf, len + padlen);
+ }
+
+ if (aes->src.sg == aes->dst.sg) {
+ aes->src.sg_len = dma_map_sg(cryp->dev, aes->src.sg,
+ aes->src.nents, DMA_BIDIRECTIONAL);
+ aes->dst.sg_len = aes->src.sg_len;
+ if (unlikely(!aes->src.sg_len))
+ return -EFAULT;
+ } else {
+ aes->src.sg_len = dma_map_sg(cryp->dev, aes->src.sg,
+ aes->src.nents, DMA_TO_DEVICE);
+ if (unlikely(!aes->src.sg_len))
+ return -EFAULT;
+
+ aes->dst.sg_len = dma_map_sg(cryp->dev, aes->dst.sg,
+ aes->dst.nents, DMA_FROM_DEVICE);
+ if (unlikely(!aes->dst.sg_len)) {
+ dma_unmap_sg(cryp->dev, aes->src.sg,
+ aes->src.nents, DMA_TO_DEVICE);
+ return -EFAULT;
+ }
+ }
+
+ return mtk_aes_info_map(cryp, aes, len + padlen);
+}
+
+static int mtk_aes_handle_queue(struct mtk_cryp *cryp, u8 id,
+ struct ablkcipher_request *req)
+{
+ struct mtk_aes *aes = cryp->aes[id];
+ struct crypto_async_request *areq, *backlog;
+ struct mtk_aes_reqctx *rctx;
+ struct mtk_aes_ctx *ctx;
+ unsigned long flags;
+ int err, ret = 0;
+
+ spin_lock_irqsave(&aes->lock, flags);
+ if (req)
+ ret = ablkcipher_enqueue_request(&aes->queue, req);
+ if (aes->flags & AES_FLAGS_BUSY) {
+ spin_unlock_irqrestore(&aes->lock, flags);
+ return ret;
+ }
+ backlog = crypto_get_backlog(&aes->queue);
+ areq = crypto_dequeue_request(&aes->queue);
+ if (areq)
+ aes->flags |= AES_FLAGS_BUSY;
+ spin_unlock_irqrestore(&aes->lock, flags);
+
+ if (!areq)
+ return ret;
+
+ if (backlog)
+ backlog->complete(backlog, -EINPROGRESS);
+
+ req = ablkcipher_request_cast(areq);
+ ctx = crypto_ablkcipher_ctx(crypto_ablkcipher_reqtfm(req));
+ rctx = ablkcipher_request_ctx(req);
+ rctx->mode &= AES_FLAGS_MODE_MSK;
+ /* assign new request to device */
+ aes->req = req;
+ aes->info = &ctx->info;
+ aes->flags = (aes->flags & ~AES_FLAGS_MODE_MSK) | rctx->mode;
+
+ err = mtk_aes_map(cryp, aes);
+ if (err)
+ return err;
+
+ return mtk_aes_xmit(cryp, aes);
+}
+
+static void mtk_aes_unmap(struct mtk_cryp *cryp, struct mtk_aes *aes)
+{
+ dma_unmap_single(cryp->dev, aes->ct_dma,
+ sizeof(struct mtk_aes_info), DMA_TO_DEVICE);
+
+ if (aes->src.sg == aes->dst.sg) {
+ dma_unmap_sg(cryp->dev, aes->src.sg,
+ aes->src.nents, DMA_BIDIRECTIONAL);
+
+ if (aes->src.sg != &aes->aligned_sg)
+ mtk_aes_restore_sg(&aes->src);
+ } else {
+ dma_unmap_sg(cryp->dev, aes->dst.sg,
+ aes->dst.nents, DMA_FROM_DEVICE);
+
+ if (aes->dst.sg != &aes->aligned_sg)
+ mtk_aes_restore_sg(&aes->dst);
+
+ dma_unmap_sg(cryp->dev, aes->src.sg,
+ aes->src.nents, DMA_TO_DEVICE);
+
+ if (aes->src.sg != &aes->aligned_sg)
+ mtk_aes_restore_sg(&aes->src);
+ }
+
+ if (aes->dst.sg == &aes->aligned_sg)
+ sg_copy_from_buffer(aes->real_dst,
+ sg_nents(aes->real_dst),
+ aes->buf, aes->total);
+}
+
+static inline void mtk_aes_complete(struct mtk_cryp *cryp,
+ struct mtk_aes *aes)
+{
+ aes->flags &= ~AES_FLAGS_BUSY;
+
+ aes->req->base.complete(&aes->req->base, 0);
+
+ mtk_aes_handle_queue(cryp, aes->id, NULL);
+}
+
+static int mtk_aes_setkey(struct crypto_ablkcipher *tfm,
+ const u8 *key, u32 keylen)
+{
+ struct mtk_aes_ctx *ctx = crypto_ablkcipher_ctx(tfm);
+ u8 *state = ctx->info.tfm.state;
+
+ if (keylen != AES_KEYSIZE_128 &&
+ keylen != AES_KEYSIZE_192 &&
+ keylen != AES_KEYSIZE_256) {
+ crypto_ablkcipher_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
+ return -EINVAL;
+ }
+
+ ctx->keylen = keylen;
+ memcpy(state, key, keylen);
+
+ return 0;
+}
+
+static int mtk_aes_crypt(struct ablkcipher_request *req, u64 mode)
+{
+ struct mtk_aes_ctx *ctx = crypto_ablkcipher_ctx(
+ crypto_ablkcipher_reqtfm(req));
+ struct mtk_aes_reqctx *rctx = ablkcipher_request_ctx(req);
+
+ rctx->mode = mode;
+
+ return mtk_aes_handle_queue(ctx->cryp,
+ !(mode & AES_FLAGS_ENCRYPT), req);
+}
+
+static int mtk_ecb_encrypt(struct ablkcipher_request *req)
+{
+ return mtk_aes_crypt(req, AES_FLAGS_ENCRYPT | AES_FLAGS_ECB);
+}
+
+static int mtk_ecb_decrypt(struct ablkcipher_request *req)
+{
+ return mtk_aes_crypt(req, AES_FLAGS_ECB);
+}
+
+static int mtk_cbc_encrypt(struct ablkcipher_request *req)
+{
+ return mtk_aes_crypt(req, AES_FLAGS_ENCRYPT | AES_FLAGS_CBC);
+}
+
+static int mtk_cbc_decrypt(struct ablkcipher_request *req)
+{
+ return mtk_aes_crypt(req, AES_FLAGS_CBC);
+}
+
+static int mtk_aes_cra_init(struct crypto_tfm *tfm)
+{
+ struct mtk_aes_ctx *ctx = crypto_tfm_ctx(tfm);
+ struct mtk_cryp *cryp = NULL;
+
+ tfm->crt_ablkcipher.reqsize = sizeof(struct mtk_aes_reqctx);
+
+ cryp = mtk_aes_find_dev(ctx);
+ if (!cryp) {
+ pr_err("can't find crypto device\n");
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+static struct crypto_alg aes_algs[] = {
+{
+ .cra_name = "cbc(aes)",
+ .cra_driver_name = "cbc-aes-mtk",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER |
+ CRYPTO_ALG_ASYNC,
+ .cra_init = mtk_aes_cra_init,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_aes_ctx),
+ .cra_alignmask = 15,
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_u.ablkcipher = {
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .setkey = mtk_aes_setkey,
+ .encrypt = mtk_cbc_encrypt,
+ .decrypt = mtk_cbc_decrypt,
+ .ivsize = AES_BLOCK_SIZE,
+ }
+},
+{
+ .cra_name = "ecb(aes)",
+ .cra_driver_name = "ecb-aes-mtk",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER |
+ CRYPTO_ALG_ASYNC,
+ .cra_init = mtk_aes_cra_init,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_aes_ctx),
+ .cra_alignmask = 15,
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_u.ablkcipher = {
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .setkey = mtk_aes_setkey,
+ .encrypt = mtk_ecb_encrypt,
+ .decrypt = mtk_ecb_decrypt,
+ }
+},
+};
+
+static void mtk_aes_enc_task(unsigned long data)
+{
+ struct mtk_cryp *cryp = (struct mtk_cryp *)data;
+ struct mtk_aes *aes = cryp->aes[0];
+
+ mtk_aes_unmap(cryp, aes);
+ mtk_aes_complete(cryp, aes);
+}
+
+static void mtk_aes_dec_task(unsigned long data)
+{
+ struct mtk_cryp *cryp = (struct mtk_cryp *)data;
+ struct mtk_aes *aes = cryp->aes[1];
+
+ mtk_aes_unmap(cryp, aes);
+ mtk_aes_complete(cryp, aes);
+}
+
+static irqreturn_t mtk_aes_enc_irq(int irq, void *dev_id)
+{
+ struct mtk_cryp *cryp = (struct mtk_cryp *)dev_id;
+ struct mtk_aes *aes = cryp->aes[0];
+ u32 val = mtk_aes_read(cryp, RDR_STAT(RING0));
+
+ mtk_aes_write(cryp, RDR_STAT(RING0), val);
+
+ if (likely(AES_FLAGS_BUSY & aes->flags)) {
+ mtk_aes_write(cryp, RDR_PROC_COUNT(RING0), MTK_DESC_CNT_CLR);
+ mtk_aes_write(cryp, RDR_THRESH(RING0), MTK_RDR_THRESH_DEF);
+
+ tasklet_schedule(&aes->task);
+ } else {
+ dev_warn(cryp->dev, "AES interrupt when no active requests.\n");
+ }
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t mtk_aes_dec_irq(int irq, void *dev_id)
+{
+ struct mtk_cryp *cryp = (struct mtk_cryp *)dev_id;
+ struct mtk_aes *aes = cryp->aes[1];
+ u32 val = mtk_aes_read(cryp, RDR_STAT(RING1));
+
+ mtk_aes_write(cryp, RDR_STAT(RING1), val);
+
+ if (likely(AES_FLAGS_BUSY & aes->flags)) {
+ mtk_aes_write(cryp, RDR_PROC_COUNT(RING1), MTK_DESC_CNT_CLR);
+ mtk_aes_write(cryp, RDR_THRESH(RING1), MTK_RDR_THRESH_DEF);
+
+ tasklet_schedule(&aes->task);
+ } else {
+ dev_warn(cryp->dev, "AES interrupt when no active requests.\n");
+ }
+ return IRQ_HANDLED;
+}
+
+/* AES encryption record and decryption record */
+static int mtk_aes_record_init(struct mtk_cryp *cryp)
+{
+ struct mtk_aes **aes = cryp->aes;
+ int i, err = -ENOMEM;
+
+ for (i = 0; i < RECORD_NUM; i++) {
+ aes[i] = kzalloc(sizeof(**aes), GFP_KERNEL);
+ if (!aes[i])
+ goto err_cleanup;
+
+ aes[i]->buf = (void *)__get_free_pages(GFP_KERNEL,
+ AES_BUFFER_ORDER);
+ if (!aes[i]->buf)
+ goto err_cleanup;
+
+ aes[i]->id = i;
+
+ spin_lock_init(&aes[i]->lock);
+ crypto_init_queue(&aes[i]->queue, AES_QUEUE_LENGTH);
+ }
+
+ tasklet_init(&aes[0]->task, mtk_aes_enc_task, (unsigned long)cryp);
+ tasklet_init(&aes[1]->task, mtk_aes_dec_task, (unsigned long)cryp);
+
+ return 0;
+
+err_cleanup:
+ for (; i--; ) {
+ free_page((unsigned long)aes[i]->buf);
+ kfree(aes[i]);
+ }
+
+ return err;
+}
+
+static void mtk_aes_record_free(struct mtk_cryp *cryp)
+{
+ int i;
+
+ for (i = 0; i < RECORD_NUM; i++) {
+ tasklet_kill(&cryp->aes[i]->task);
+ free_page((unsigned long)cryp->aes[i]->buf);
+ kfree(cryp->aes[i]);
+ }
+}
+
+static void mtk_aes_unregister_algs(void)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(aes_algs); i++)
+ crypto_unregister_alg(&aes_algs[i]);
+}
+
+static int mtk_aes_register_algs(void)
+{
+ int err, i, j;
+
+ for (i = 0; i < ARRAY_SIZE(aes_algs); i++) {
+ err = crypto_register_alg(&aes_algs[i]);
+ if (err)
+ goto err_aes_algs;
+ }
+
+ return 0;
+
+err_aes_algs:
+ for (j = 0; j < i; j++)
+ crypto_unregister_alg(&aes_algs[j]);
+
+ return err;
+}
+
+int mtk_cipher_alg_register(struct mtk_cryp *cryp)
+{
+ int ret;
+
+ INIT_LIST_HEAD(&cryp->aes_list);
+
+ ret = mtk_aes_record_init(cryp);
+ if (ret)
+ goto err_record;
+
+ ret = devm_request_irq(cryp->dev, cryp->irq[RING0], mtk_aes_enc_irq,
+ IRQF_TRIGGER_LOW, "mtk-aes", cryp);
+ if (ret) {
+ dev_err(cryp->dev, "unable to request AES encryption irq.\n");
+ goto err_res;
+ }
+
+ ret = devm_request_irq(cryp->dev, cryp->irq[RING1], mtk_aes_dec_irq,
+ IRQF_TRIGGER_LOW, "mtk-aes", cryp);
+ if (ret) {
+ dev_err(cryp->dev, "unable to request AES decryption irq.\n");
+ goto err_res;
+ }
+
+ /* enable ring0 and ring1 interrupt for cipher */
+ mtk_aes_write(cryp, AIC_ENABLE_SET(RING0), MTK_IRQ_RDR0);
+ mtk_aes_write(cryp, AIC_ENABLE_SET(RING1), MTK_IRQ_RDR1);
+
+ spin_lock(&mtk_aes.lock);
+ list_add_tail(&cryp->aes_list, &mtk_aes.dev_list);
+ spin_unlock(&mtk_aes.lock);
+
+ ret = mtk_aes_register_algs();
+ if (ret)
+ goto err_algs;
+
+ return 0;
+
+err_algs:
+ spin_lock(&mtk_aes.lock);
+ list_del(&cryp->aes_list);
+ spin_unlock(&mtk_aes.lock);
+err_res:
+ mtk_aes_record_free(cryp);
+err_record:
+
+ dev_err(cryp->dev, "mtk-aes initialization failed.\n");
+ return ret;
+}
+EXPORT_SYMBOL(mtk_cipher_alg_register);
+
+void mtk_cipher_alg_release(struct mtk_cryp *cryp)
+{
+ spin_lock(&mtk_aes.lock);
+ list_del(&cryp->aes_list);
+ spin_unlock(&mtk_aes.lock);
+
+ mtk_aes_unregister_algs();
+ mtk_aes_record_free(cryp);
+}
+EXPORT_SYMBOL(mtk_cipher_alg_release);
diff --git a/drivers/crypto/mediatek/mtk-platform.c b/drivers/crypto/mediatek/mtk-platform.c
new file mode 100644
index 0000000..d3a465f
--- /dev/null
+++ b/drivers/crypto/mediatek/mtk-platform.c
@@ -0,0 +1,575 @@
+/*
+ * Support for MediaTek cryptographic accelerator.
+ *
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: Ryder Lee <ryder.lee-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/mfd/syscon.h>
+#include <linux/pm_runtime.h>
+#include <linux/clk.h>
+#include <linux/platform_device.h>
+#include "mtk-platform.h"
+#include "mtk-regs.h"
+
+#define MTK_BURST_SIZE(x, y) (((x) & ~0xf0) | ((y) << 4))
+#define MTK_DESC_SIZE_SET(x) ((x) << 0)
+#define MTK_DESC_OFFSET_SET(x) ((x) << 16)
+#define MTK_DFSE_RING_ID(x) (((x) >> 12) & 0xf)
+#define MTK_DSE_MIN_DATA(x) ((x) << 0)
+#define MTK_DSE_MAX_DATA(x) ((x) << 8)
+#define MTK_DFE_MIN_DATA(x) ((x) << 0)
+#define MTK_DFE_MAX_DATA(x) ((x) << 8)
+#define MTK_DFE_MIN_CTRL(x) ((x) << 16)
+#define MTK_DFE_MAX_CTRL(x) ((x) << 24)
+#define MTK_FETCH_SIZE_SET(x) ((x) << 0)
+#define MTK_FETCH_THRESH_SET(x) ((x) << 16)
+#define MTK_IN_BUF_MIN_THRESH(x) ((x) << 8)
+#define MTK_IN_BUF_MAX_THRESH(x) ((x) << 12)
+#define MTK_OUT_BUF_MIN_THRESH(x) ((x) << 0)
+#define MTK_OUT_BUF_MAX_THRESH(x) ((x) << 4)
+#define MTK_CMD_FIFO_SIZE(x) (((x) >> 8) & 0xf)
+#define MTK_RES_FIFO_SIZE(x) (((x) >> 12) & 0xf)
+#define MTK_HIA_DATA_WIDTH(x) (((x) >> 25) & 0x3)
+#define MTK_HIA_DMA_LENGTH(x) (((x) >> 20) & 0x1f)
+#define MTK_IN_TBUF_SIZE(x) (((x) >> 4) & 0xf)
+#define MTK_IN_DBUF_SIZE(x) (((x) >> 8) & 0xf)
+#define MTK_OUT_DBUF_SIZE(x) (((x) >> 16) & 0xf)
+#define MTK_AIC_INT_NUM(x) ((x) & 0x3f)
+#define MTK_AIC_VER_GET(x) ((x) & 0x0ff0ffff)
+#define MTK_PE_TOKEN_CTRL_DEF 0x00014004
+#define MTK_PE_INT_CTRL_DEF 0xc00f400f
+#define MTK_PRNG_CTRL_EN BIT(0)
+#define MTK_PRNG_CTRL_AUTO BIT(1)
+#define MTK_TOKEN_TIMEOUT_EN BIT(22)
+#define MTK_OVL_IRQ_EN BIT(25)
+#define MTK_ATP_PRESENT BIT(30)
+#define MTK_DFSE_THR_CTRL_EN BIT(30)
+#define MTK_DFSE_THR_CTRL_RESET BIT(31)
+#define MTK_HIA_SIGNATURE ((u16)0x35ca)
+#define MTK_CDR_STAT_CLR 0x1f
+#define MTK_RDR_STAT_CLR 0xff
+#define MTK_AIC_VER11 0x011036C9
+#define MTK_AIC_VER12 0x012036C9
+#define MTK_AIC_GLOBAL_CLR 0x7FF00000
+#define MTK_DFSE_IDLE 0xf
+
+/* Cryptographic engine capabilities */
+struct mtk_sys_cap {
+ /* Host interface adapter */
+ u32 hia_ver;
+ u32 hia_opt;
+
+ /* Packet engine */
+ u32 pkt_eng_opt;
+
+ /* Global hardware */
+ u32 hw_opt;
+};
+
+static void mtk_prng_reseed(struct mtk_cryp *cryp)
+{
+ /* 8 words to seed the PRNG to provide IVs */
+ void __iomem *base = cryp->base;
+ const u32 prng_key[8] = {0x48c24cfd, 0x6c07f742,
+ 0xaee75681, 0x0f27c239,
+ 0x79947198, 0xe2991275,
+ 0x21ac3c7c, 0xd008c4b4};
+
+ /* Disable PRNG and set PRNG manual mode */
+ writel(0, base + PE_PRNG_CTRL);
+
+ /* Write new seed data */
+ writel(prng_key[0], base + PE_PRNG_SEED_L);
+ writel(prng_key[1], base + PE_PRNG_SEED_H);
+
+ /* Write new key data */
+ writel(prng_key[2], base + PE_PRNG_KEY_0_L);
+ writel(prng_key[3], base + PE_PRNG_KEY_0_H);
+ writel(prng_key[4], base + PE_PRNG_KEY_1_L);
+ writel(prng_key[5], base + PE_PRNG_KEY_1_H);
+
+ /* Write new LFSR data */
+ writel(prng_key[6], base + PE_PRNG_LFSR_L);
+ writel(prng_key[7], base + PE_PRNG_LFSR_H);
+
+ /* Enable PRNG and set PRNG auto mode */
+ writel(MTK_PRNG_CTRL_EN | MTK_PRNG_CTRL_AUTO, base + PE_PRNG_CTRL);
+}
+
+static void mtk_desc_ring_link(struct mtk_cryp *cryp, u32 mask)
+{
+ /* Assign rings to DFE DSE thread and enable it */
+ writel(MTK_DFSE_THR_CTRL_EN | mask, cryp->base + DFE_THR_CTRL);
+ writel(MTK_DFSE_THR_CTRL_EN | mask, cryp->base + DSE_THR_CTRL);
+}
+
+static void mtk_dfe_dse_buf_setup(struct mtk_cryp *cryp,
+ struct mtk_sys_cap *cap)
+{
+ u32 width = MTK_HIA_DATA_WIDTH(cap->hia_opt) + 2;
+ u32 len = MTK_HIA_DMA_LENGTH(cap->hia_opt) - 1;
+ u32 ipbuf = min(MTK_IN_DBUF_SIZE(cap->hw_opt) + width, len);
+ u32 opbuf = min(MTK_OUT_DBUF_SIZE(cap->hw_opt) + width, len);
+ u32 itbuf = min(MTK_IN_TBUF_SIZE(cap->hw_opt) + width, len);
+ u32 val;
+
+ val = MTK_DFE_MIN_DATA(ipbuf - 1) | MTK_DFE_MAX_DATA(ipbuf) |
+ MTK_DFE_MIN_CTRL(itbuf - 1) | MTK_DFE_MAX_CTRL(itbuf);
+ writel(val, cryp->base + DFE_CFG);
+
+ val = MTK_DFE_MIN_DATA(opbuf - 1) | MTK_DFE_MAX_DATA(opbuf);
+ writel(val, cryp->base + DSE_CFG);
+
+ val = MTK_IN_BUF_MIN_THRESH(ipbuf - 1) | MTK_IN_BUF_MAX_THRESH(ipbuf);
+ writel(val, cryp->base + PE_IN_DBUF_THRESH);
+
+ val = MTK_IN_BUF_MIN_THRESH(itbuf - 1) | MTK_IN_BUF_MAX_THRESH(itbuf);
+ writel(val, cryp->base + PE_IN_TBUF_THRESH);
+
+ val = MTK_OUT_BUF_MIN_THRESH(opbuf - 1) | MTK_OUT_BUF_MAX_THRESH(opbuf);
+ writel(val, cryp->base + PE_OUT_DBUF_THRESH);
+
+ writel(0, cryp->base + PE_OUT_TBUF_THRESH);
+ writel(0, cryp->base + PE_OUT_BUF_CTRL);
+}
+
+static int mtk_dfe_dse_state_check(struct mtk_cryp *cryp)
+{
+ int ret = -EINVAL;
+ u32 val;
+
+ /* Check for completion of all DMA transfers */
+ val = readl(cryp->base + DFE_THR_STAT);
+ if (MTK_DFSE_RING_ID(val) == MTK_DFSE_IDLE) {
+ val = readl(cryp->base + DSE_THR_STAT);
+ if (MTK_DFSE_RING_ID(val) == MTK_DFSE_IDLE)
+ ret = 0;
+ }
+
+ if (!ret) {
+ /* Take DFE and DSE thread out of reset */
+ writel(0, cryp->base + DFE_THR_CTRL);
+ writel(0, cryp->base + DSE_THR_CTRL);
+ } else {
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
+static int mtk_dfe_dse_reset(struct mtk_cryp *cryp)
+{
+ int err;
+
+ /* Reset DSE DFE and correct system priorities for all rings. */
+ writel(MTK_DFSE_THR_CTRL_RESET, cryp->base + DFE_THR_CTRL);
+ writel(0, cryp->base + DFE_PRIO_0);
+ writel(0, cryp->base + DFE_PRIO_1);
+ writel(0, cryp->base + DFE_PRIO_2);
+ writel(0, cryp->base + DFE_PRIO_3);
+
+ writel(MTK_DFSE_THR_CTRL_RESET, cryp->base + DSE_THR_CTRL);
+ writel(0, cryp->base + DSE_PRIO_0);
+ writel(0, cryp->base + DSE_PRIO_1);
+ writel(0, cryp->base + DSE_PRIO_2);
+ writel(0, cryp->base + DSE_PRIO_3);
+
+ err = mtk_dfe_dse_state_check(cryp);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static void mtk_cmd_desc_ring_setup(struct mtk_cryp *cryp,
+ int i, struct mtk_sys_cap *cap)
+{
+ /* Full descriptor that fits FIFO minus one */
+ u32 count =
+ ((1 << MTK_CMD_FIFO_SIZE(cap->hia_opt)) / MTK_DESC_SIZE) - 1;
+ u32 size = count * MTK_DESC_OFFSET;
+ u32 thresh = count * MTK_DESC_SIZE;
+ u32 val;
+
+ /* Temporarily disable external triggering */
+ writel(0, cryp->base + CDR_CFG(i));
+
+ /* Clear CDR count */
+ writel(MTK_DESC_CNT_CLR, cryp->base + CDR_PREP_COUNT(i));
+ writel(MTK_DESC_CNT_CLR, cryp->base + CDR_PROC_COUNT(i));
+
+ writel(0, cryp->base + CDR_PREP_PNTR(i));
+ writel(0, cryp->base + CDR_PROC_PNTR(i));
+ writel(0, cryp->base + CDR_DMA_CFG(i));
+
+ /* Configure command ring host address space */
+ writel(0, cryp->base + CDR_BASE_ADDR_HI(i));
+ writel(cryp->ring[i]->cmd_dma, cryp->base + CDR_BASE_ADDR_LO(i));
+
+ writel(MTK_MAX_RING_SIZE, cryp->base + CDR_RING_SIZE(i));
+
+ /* Clear and disable all CDR interrupts */
+ writel(MTK_CDR_STAT_CLR, cryp->base + CDR_STAT(i));
+
+ /*
+ * Set command descriptor offset and enable additional
+ * token present in descriptor.
+ */
+ val = MTK_DESC_SIZE_SET(MTK_DESC_SIZE) |
+ MTK_DESC_OFFSET_SET(MTK_DESC_OFFSET) |
+ MTK_ATP_PRESENT;
+ writel(val, cryp->base + CDR_DESC_SIZE(i));
+
+ val = MTK_FETCH_SIZE_SET(size) | MTK_FETCH_THRESH_SET(thresh);
+ writel(val, cryp->base + CDR_CFG(i));
+}
+
+static void mtk_res_desc_ring_setup(struct mtk_cryp *cryp,
+ int i, struct mtk_sys_cap *cap)
+{
+ u32 rndup = 2;
+ u32 count = ((1 << MTK_RES_FIFO_SIZE(cap->hia_opt)) / rndup) - 1;
+ u32 size = count * MTK_DESC_OFFSET;
+ u32 thresh = count * rndup;
+ u32 val;
+
+ writel(0, cryp->base + RDR_CFG(i));
+
+ writel(MTK_DESC_CNT_CLR, cryp->base + RDR_PREP_COUNT(i));
+ writel(MTK_DESC_CNT_CLR, cryp->base + RDR_PROC_COUNT(i));
+
+ writel(0, cryp->base + RDR_PREP_PNTR(i));
+ writel(0, cryp->base + RDR_PROC_PNTR(i));
+ writel(0, cryp->base + RDR_DMA_CFG(i));
+
+ writel(0, cryp->base + RDR_BASE_ADDR_HI(i));
+ writel(cryp->ring[i]->res_dma, cryp->base + RDR_BASE_ADDR_LO(i));
+
+ writel(MTK_MAX_RING_SIZE, cryp->base + RDR_RING_SIZE(i));
+ writel(MTK_RDR_STAT_CLR, cryp->base + RDR_STAT(i));
+
+ /*
+ * RDR manager generates update interrupts on a per-completed-packet,
+ * and the rd_proc_thresh_irq interrupt is fired when proc_pkt_count
+ * for the RDR exceeds the number of packets.
+ */
+ writel(MTK_RDR_THRESH_DEF, cryp->base + RDR_THRESH(i));
+
+ /*
+ * Configure a threshold and time-out value for the processed
+ * result descriptors (or complete packets) that are written to
+ * the RDR.
+ */
+ val = MTK_DESC_SIZE_SET(MTK_DESC_SIZE) |
+ MTK_DESC_OFFSET_SET(MTK_DESC_OFFSET);
+ writel(val, cryp->base + RDR_DESC_SIZE(i));
+
+ /*
+ * Configure HIA fetch size and fetch threshold that are used to
+ * fetch blocks of multiple descriptors.
+ */
+ val = MTK_FETCH_SIZE_SET(size) |
+ MTK_FETCH_THRESH_SET(thresh) |
+ MTK_OVL_IRQ_EN;
+ writel(val, cryp->base + RDR_CFG(i));
+}
+
+static int mtk_packet_engine_setup(struct mtk_cryp *cryp)
+{
+ struct mtk_sys_cap cap;
+ int i, err;
+ u32 val;
+
+ cap.hia_ver = readl(cryp->base + HIA_VERSION);
+ cap.hia_opt = readl(cryp->base + HIA_OPTIONS);
+ cap.hw_opt = readl(cryp->base + EIP97_OPTIONS);
+
+ if (!(((u16)cap.hia_ver) == MTK_HIA_SIGNATURE))
+ return -EINVAL;
+
+ /* Configure endianness conversion method for master (DMA) interface */
+ writel(0, cryp->base + EIP97_MST_CTRL);
+
+ /* Set HIA burst size */
+ val = readl(cryp->base + HIA_MST_CTRL);
+ writel(MTK_BURST_SIZE(val, 5), cryp->base + HIA_MST_CTRL);
+
+ err = mtk_dfe_dse_reset(cryp);
+ if (err) {
+ dev_err(cryp->dev, "Failed to reset DFE and DSE.\n");
+ return err;
+ }
+
+ mtk_dfe_dse_buf_setup(cryp, &cap);
+
+ /* Enable the 4 rings for the packet engines. */
+ mtk_desc_ring_link(cryp, 0xf);
+
+ for (i = 0; i < RING_MAX; i++) {
+ mtk_cmd_desc_ring_setup(cryp, i, &cap);
+ mtk_res_desc_ring_setup(cryp, i, &cap);
+ }
+
+ val = MTK_PE_TOKEN_CTRL_DEF | MTK_TOKEN_TIMEOUT_EN;
+ writel(val, cryp->base + PE_TOKEN_CTRL_STAT);
+
+ /* Clear all pending interrupts */
+ writel(MTK_AIC_GLOBAL_CLR, cryp->base + AIC_G_ACK);
+ writel(MTK_PE_INT_CTRL_DEF, cryp->base + PE_INTERRUPT_CTRL_STAT);
+
+ return 0;
+}
+
+static int mtk_aic_cap_check(struct mtk_cryp *cryp, int hw)
+{
+ u32 val;
+
+ if (hw == RING_MAX)
+ val = readl(cryp->base + AIC_G_VERSION);
+ else
+ val = readl(cryp->base + AIC_VERSION(hw));
+
+ val = MTK_AIC_VER_GET(val);
+ if (val != MTK_AIC_VER11 && val != MTK_AIC_VER12)
+ return -ENXIO;
+
+ if (hw == RING_MAX)
+ val = readl(cryp->base + AIC_G_OPTIONS);
+ else
+ val = readl(cryp->base + AIC_OPTIONS(hw));
+
+ val = MTK_AIC_INT_NUM(val);
+ if (!val || val > 32)
+ return -ENXIO;
+
+ return 0;
+}
+
+static int mtk_aic_init(struct mtk_cryp *cryp, int hw)
+{
+ int err;
+
+ err = mtk_aic_cap_check(cryp, hw);
+ if (err)
+ return err;
+
+ /* Disable all interrupts and set initial configuration */
+ if (hw == RING_MAX) {
+ writel(0, cryp->base + AIC_G_ENABLE_CTRL);
+ writel(0, cryp->base + AIC_G_POL_CTRL);
+ writel(0, cryp->base + AIC_G_TYPE_CTRL);
+ writel(0, cryp->base + AIC_G_ENABLE_SET);
+ } else {
+ writel(0, cryp->base + AIC_ENABLE_CTRL(hw));
+ writel(0, cryp->base + AIC_POL_CTRL(hw));
+ writel(0, cryp->base + AIC_TYPE_CTRL(hw));
+ writel(0, cryp->base + AIC_ENABLE_SET(hw));
+ }
+
+ return 0;
+}
+
+static int mtk_accelerator_init(struct mtk_cryp *cryp)
+{
+ int i, err;
+
+ /* Initialize advanced interrupt controller(AIC) */
+ for (i = 0; i < 5; i++) {
+ err = mtk_aic_init(cryp, i);
+ if (err) {
+ dev_err(cryp->dev, "Failed to initialize AIC.\n");
+ return err;
+ }
+ }
+
+ /* Initialize packet engine */
+ err = mtk_packet_engine_setup(cryp);
+ if (err) {
+ dev_err(cryp->dev, "Failed to configure packet engine.\n");
+ return err;
+ }
+
+ mtk_prng_reseed(cryp);
+ return 0;
+}
+
+static void mtk_desc_dma_free(struct mtk_cryp *cryp)
+{
+ int i;
+
+ for (i = 0; i < RING_MAX; i++) {
+ dma_free_coherent(cryp->dev, MTK_MAX_RING_SIZE,
+ cryp->ring[i]->res_base,
+ cryp->ring[i]->res_dma);
+ dma_free_coherent(cryp->dev, MTK_MAX_RING_SIZE,
+ cryp->ring[i]->cmd_base,
+ cryp->ring[i]->cmd_dma);
+ kfree(cryp->ring[i]);
+ }
+}
+
+static int mtk_desc_ring_alloc(struct mtk_cryp *cryp)
+{
+ struct mtk_ring **ring = cryp->ring;
+ int i, err = ENOMEM;
+
+ for (i = 0; i < RING_MAX; i++) {
+ ring[i] = kzalloc(sizeof(**ring), GFP_KERNEL);
+ if (!ring[i])
+ goto err_cleanup;
+
+ ring[i]->cmd_base = dma_zalloc_coherent(cryp->dev,
+ MTK_MAX_RING_SIZE, &ring[i]->cmd_dma,
+ GFP_KERNEL);
+ if (!ring[i]->cmd_base)
+ goto err_cleanup;
+
+ ring[i]->res_base = dma_zalloc_coherent(cryp->dev,
+ MTK_MAX_RING_SIZE, &ring[i]->res_dma,
+ GFP_KERNEL);
+ if (!ring[i]->res_base)
+ goto err_cleanup;
+ }
+ return 0;
+
+err_cleanup:
+ for (; i--; ) {
+ dma_free_coherent(cryp->dev, MTK_MAX_RING_SIZE,
+ ring[i]->res_base, ring[i]->res_dma);
+ dma_free_coherent(cryp->dev, MTK_MAX_RING_SIZE,
+ ring[i]->cmd_base, ring[i]->cmd_dma);
+ kfree(ring[i]);
+ }
+ return err;
+}
+
+static int mtk_crypto_probe(struct platform_device *pdev)
+{
+ struct resource *res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ struct mtk_cryp *cryp;
+ int i, err;
+
+ cryp = devm_kzalloc(&pdev->dev, sizeof(*cryp), GFP_KERNEL);
+ if (!cryp)
+ return -ENOMEM;
+
+ cryp->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(cryp->base))
+ return PTR_ERR(cryp->base);
+
+ for (i = 0; i < 5; i++) {
+ cryp->irq[i] = platform_get_irq(pdev, i);
+ if (cryp->irq[i] < 0) {
+ dev_err(cryp->dev, "no IRQ:%d resource info\n", i);
+ return -ENXIO;
+ }
+ }
+
+ cryp->clk_ethif = devm_clk_get(&pdev->dev, "ethif");
+ cryp->clk_cryp = devm_clk_get(&pdev->dev, "cryp");
+ if (IS_ERR(cryp->clk_ethif) || IS_ERR(cryp->clk_cryp))
+ return -EPROBE_DEFER;
+
+ cryp->dev = &pdev->dev;
+ pm_runtime_enable(cryp->dev);
+ pm_runtime_get_sync(cryp->dev);
+
+ err = clk_prepare_enable(cryp->clk_ethif);
+ if (err)
+ goto err_clk_ethif;
+
+ err = clk_prepare_enable(cryp->clk_cryp);
+ if (err)
+ goto err_clk_cryp;
+
+ err = mtk_desc_ring_alloc(cryp);
+ if (err) {
+ dev_err(cryp->dev, "Unable to allocate descriptor rings.\n");
+ goto err_resource;
+ }
+
+ err = mtk_accelerator_init(cryp);
+ if (err) {
+ dev_err(cryp->dev, "Failed to initialize cryptographic engine.\n");
+ goto err_engine;
+ }
+
+ err = mtk_cipher_alg_register(cryp);
+ if (err) {
+ dev_err(cryp->dev, "Unable to register MTK-AES.\n");
+ goto err_cipher;
+ }
+
+ err = mtk_hash_alg_register(cryp);
+ if (err) {
+ dev_err(cryp->dev, "Unable to register MTK-SHA.\n");
+ goto err_hash;
+ }
+
+ platform_set_drvdata(pdev, cryp);
+ return 0;
+
+err_hash:
+ mtk_cipher_alg_release(cryp);
+err_cipher:
+ mtk_dfe_dse_reset(cryp);
+err_engine:
+ mtk_desc_dma_free(cryp);
+err_resource:
+ clk_disable_unprepare(cryp->clk_cryp);
+err_clk_cryp:
+ clk_disable_unprepare(cryp->clk_ethif);
+err_clk_ethif:
+ pm_runtime_put_sync(cryp->dev);
+ pm_runtime_disable(cryp->dev);
+
+ return err;
+}
+
+static int mtk_crypto_remove(struct platform_device *pdev)
+{
+ struct mtk_cryp *cryp = platform_get_drvdata(pdev);
+
+ mtk_hash_alg_release(cryp);
+ mtk_cipher_alg_release(cryp);
+ mtk_desc_dma_free(cryp);
+
+ clk_disable_unprepare(cryp->clk_cryp);
+ clk_disable_unprepare(cryp->clk_ethif);
+
+ pm_runtime_put_sync(cryp->dev);
+ pm_runtime_disable(cryp->dev);
+ platform_set_drvdata(pdev, NULL);
+
+ return 0;
+}
+
+const struct of_device_id of_crypto_id[] = {
+ { .compatible = "mediatek,mt7623-crypto" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, of_crypto_id);
+
+static struct platform_driver mtk_crypto_driver = {
+ .probe = mtk_crypto_probe,
+ .remove = mtk_crypto_remove,
+ .driver = {
+ .name = "mtk-crypto",
+ .owner = THIS_MODULE,
+ .of_match_table = of_crypto_id,
+ },
+};
+module_platform_driver(mtk_crypto_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Ryder Lee <ryder.lee-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>");
+MODULE_DESCRIPTION("Cryptographic accelerator driver for MediaTek SoC");
diff --git a/drivers/crypto/mediatek/mtk-platform.h b/drivers/crypto/mediatek/mtk-platform.h
new file mode 100644
index 0000000..5390ef4
--- /dev/null
+++ b/drivers/crypto/mediatek/mtk-platform.h
@@ -0,0 +1,230 @@
+/*
+ * Support for MediaTek cryptographic accelerator.
+ *
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: Ryder Lee <ryder.lee-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ */
+
+#ifndef __MTK_PLATFORM_H_
+#define __MTK_PLATFORM_H_
+
+#include <linux/crypto.h>
+#include <crypto/internal/hash.h>
+#include <linux/interrupt.h>
+
+#define MTK_RDR_THRESH_DEF 0x800001
+
+#define MTK_IRQ_RDR0 BIT(1)
+#define MTK_IRQ_RDR1 BIT(3)
+#define MTK_IRQ_RDR2 BIT(5)
+#define MTK_IRQ_RDR3 BIT(7)
+
+#define MTK_DESC_CNT_CLR BIT(31)
+#define MTK_DESC_LAST BIT(22)
+#define MTK_DESC_FIRST BIT(23)
+#define MTK_DESC_BUF_LEN(x) ((x) & 0x1ffff)
+#define MTK_DESC_CT_LEN(x) (((x) & 0xff) << 24)
+
+#define WORD(x) ((x) >> 2)
+
+/**
+ * Ring 0/1 are used by AES encrypt and decrypt.
+ * Ring 2/3 are used by SHA.
+ */
+enum {
+ RING0 = 0,
+ RING1,
+ RING2,
+ RING3,
+ RING_MAX,
+};
+
+#define RECORD_NUM (RING_MAX / 2)
+
+/**
+ * struct mtk_desc - DMA descriptor
+ * @hdr: the descriptor control header
+ * @buf: DMA address of input buffer
+ * @ct: the command token that control operation flow
+ * @ct_hdr: the command token control header
+ * @tag: the user-defined field
+ * @tfm: DMA address of transform state
+ * @bound: align descriptors offset boundary
+ *
+ * Structure passed to the crypto engine to describe the crypto
+ * operation to be executed.
+ */
+struct mtk_desc {
+ u32 hdr;
+ u32 buf;
+ u32 ct;
+ u32 ct_hdr;
+ u32 tag;
+ u32 tfm;
+ u32 bound[2];
+};
+
+/**
+ * struct mtk_ring - Descriptor ring
+ * @cmd_base: pointer to command descriptor ring base
+ * @cmd_dma: DMA address of command descriptor ring
+ * @res_base: pointer to result descriptor ring base
+ * @res_dma: DMA address of result descriptor ring
+ * @pos: current position in the ring
+ */
+struct mtk_ring {
+ struct mtk_desc *cmd_base;
+ dma_addr_t cmd_dma;
+ struct mtk_desc *res_base;
+ dma_addr_t res_dma;
+ u32 pos;
+};
+
+#define MTK_MAX_DESC_NUM 512
+#define MTK_DESC_OFFSET WORD(sizeof(struct mtk_desc))
+#define MTK_DESC_SIZE (MTK_DESC_OFFSET - 2)
+#define MTK_MAX_RING_SIZE ((sizeof(struct mtk_desc) * MTK_MAX_DESC_NUM))
+#define MTK_DESC_CNT(x) ((MTK_DESC_OFFSET * (x)) << 2)
+
+/**
+ * struct mtk_aes_dma - Structure that holds sg list info
+ * @sg: pointer to scatter-gather list
+ * @nents: number of entries in the sg list
+ * @remainder: remainder of sg list
+ * @sg_len: number of entries in the sg mapped list
+ */
+struct mtk_aes_dma {
+ struct scatterlist *sg;
+ int nents;
+ u32 remainder;
+ u32 sg_len;
+};
+
+/**
+ * struct mtk_aes - AES operation record
+ * @queue: crypto request queue
+ * @req: pointer to ablkcipher request
+ * @task: the tasklet is use in AES interrupt
+ * @src: the structure that holds source sg list info
+ * @dst: the structure that holds destination sg list info
+ * @aligned_sg: the scatter list is use to alignment
+ * @real_dst: pointer to the destination sg list
+ * @total: request buffer length
+ * @buf: pointer to page buffer
+ * @info: pointer to AES transform state and command token
+ * @ct_hdr: AES command token control field
+ * @ct_size: size of AES command token
+ * @ct_dma: DMA address of AES command token
+ * @tfm_dma: DMA address of AES transform state
+ * @id: record identification
+ * @flags: it's describing AES operation state
+ * @lock: the ablkcipher queue lock
+ *
+ * Structure used to record AES execution state
+ */
+struct mtk_aes {
+ struct crypto_queue queue;
+ struct ablkcipher_request *req;
+ struct tasklet_struct task;
+ struct mtk_aes_dma src;
+ struct mtk_aes_dma dst;
+
+ struct scatterlist aligned_sg;
+ struct scatterlist *real_dst;
+
+ size_t total;
+ void *buf;
+
+ void *info;
+ u32 ct_hdr;
+ u32 ct_size;
+ dma_addr_t ct_dma;
+ dma_addr_t tfm_dma;
+
+ u8 id;
+ unsigned long flags;
+ /* queue lock */
+ spinlock_t lock;
+};
+
+/**
+ * struct mtk_sha - SHA operation record
+ * @queue: crypto request queue
+ * @req: pointer to ahash request
+ * @task: the tasklet is use in SHA interrupt
+ * @info: pointer to SHA transform state and command token
+ * @ct_hdr: SHA command token control field
+ * @ct_size: size of SHA command token
+ * @ct_dma: DMA address of SHA command token
+ * @tfm_dma: DMA address of SHA transform state
+ * @id: record identification
+ * @flags: it's describing SHA operation state
+ * @lock: the ablkcipher queue lock
+ *
+ * Structure used to record SHA execution state.
+ */
+struct mtk_sha {
+ struct crypto_queue queue;
+ struct ahash_request *req;
+ struct tasklet_struct task;
+
+ void *info;
+ u32 ct_hdr;
+ u32 ct_size;
+ dma_addr_t ct_dma;
+ dma_addr_t tfm_dma;
+
+ u8 id;
+ unsigned long flags;
+ /* queue lock */
+ spinlock_t lock;
+};
+
+/**
+ * struct mtk_cryp - Cryptographic device
+ * @base: pointer to mapped register I/O base
+ * @dev: pointer to device
+ * @clk_ethif: pointer to ethif clock
+ * @clk_cryp: pointer to crypto clock
+ * @irq: global system and rings IRQ
+ * @ring: pointer to execution state of AES
+ * @aes: pointer to execution state of SHA
+ * @sha: each execution record map to a ring
+ * @aes_list: device list of AES
+ * @sha_list: device list of SHA
+ * @tmp: pointer to temporary buffer for internal use
+ * @tmp_dma: DMA address of temporary buffer
+ * @rec: it's used to select SHA record for tfm
+ *
+ * Structure storing cryptographic device information.
+ */
+struct mtk_cryp {
+ void __iomem *base;
+ struct device *dev;
+ struct clk *clk_ethif;
+ struct clk *clk_cryp;
+ int irq[5];
+
+ struct mtk_ring *ring[RING_MAX];
+ struct mtk_aes *aes[RECORD_NUM];
+ struct mtk_sha *sha[RECORD_NUM];
+
+ struct list_head aes_list;
+ struct list_head sha_list;
+
+ void *tmp;
+ dma_addr_t tmp_dma;
+ bool rec;
+};
+
+int mtk_cipher_alg_register(struct mtk_cryp *cryp);
+void mtk_cipher_alg_release(struct mtk_cryp *cryp);
+int mtk_hash_alg_register(struct mtk_cryp *cryp);
+void mtk_hash_alg_release(struct mtk_cryp *cryp);
+
+#endif /* __MTK_PLATFORM_H_ */
diff --git a/drivers/crypto/mediatek/mtk-regs.h b/drivers/crypto/mediatek/mtk-regs.h
new file mode 100644
index 0000000..94f4eb8
--- /dev/null
+++ b/drivers/crypto/mediatek/mtk-regs.h
@@ -0,0 +1,194 @@
+/*
+ * Support for MediaTek cryptographic accelerator.
+ *
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: Ryder Lee <ryder.lee-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ */
+
+#ifndef __MTK_REGS_H__
+#define __MTK_REGS_H__
+
+/* HIA, Command Descriptor Ring Manager */
+#define CDR_BASE_ADDR_LO(x) (0x0 + ((x) << 12))
+#define CDR_BASE_ADDR_HI(x) (0x4 + ((x) << 12))
+#define CDR_DATA_BASE_ADDR_LO(x) (0x8 + ((x) << 12))
+#define CDR_DATA_BASE_ADDR_HI(x) (0xC + ((x) << 12))
+#define CDR_ACD_BASE_ADDR_LO(x) (0x10 + ((x) << 12))
+#define CDR_ACD_BASE_ADDR_HI(x) (0x14 + ((x) << 12))
+#define CDR_RING_SIZE(x) (0x18 + ((x) << 12))
+#define CDR_DESC_SIZE(x) (0x1C + ((x) << 12))
+#define CDR_CFG(x) (0x20 + ((x) << 12))
+#define CDR_DMA_CFG(x) (0x24 + ((x) << 12))
+#define CDR_THRESH(x) (0x28 + ((x) << 12))
+#define CDR_PREP_COUNT(x) (0x2C + ((x) << 12))
+#define CDR_PROC_COUNT(x) (0x30 + ((x) << 12))
+#define CDR_PREP_PNTR(x) (0x34 + ((x) << 12))
+#define CDR_PROC_PNTR(x) (0x38 + ((x) << 12))
+#define CDR_STAT(x) (0x3C + ((x) << 12))
+
+/* HIA, Result Descriptor Ring Manager */
+#define RDR_BASE_ADDR_LO(x) (0x800 + ((x) << 12))
+#define RDR_BASE_ADDR_HI(x) (0x804 + ((x) << 12))
+#define RDR_DATA_BASE_ADDR_LO(x) (0x808 + ((x) << 12))
+#define RDR_DATA_BASE_ADDR_HI(x) (0x80C + ((x) << 12))
+#define RDR_ACD_BASE_ADDR_LO(x) (0x810 + ((x) << 12))
+#define RDR_ACD_BASE_ADDR_HI(x) (0x814 + ((x) << 12))
+#define RDR_RING_SIZE(x) (0x818 + ((x) << 12))
+#define RDR_DESC_SIZE(x) (0x81C + ((x) << 12))
+#define RDR_CFG(x) (0x820 + ((x) << 12))
+#define RDR_DMA_CFG(x) (0x824 + ((x) << 12))
+#define RDR_THRESH(x) (0x828 + ((x) << 12))
+#define RDR_PREP_COUNT(x) (0x82C + ((x) << 12))
+#define RDR_PROC_COUNT(x) (0x830 + ((x) << 12))
+#define RDR_PREP_PNTR(x) (0x834 + ((x) << 12))
+#define RDR_PROC_PNTR(x) (0x838 + ((x) << 12))
+#define RDR_STAT(x) (0x83C + ((x) << 12))
+
+/* HIA, Ring AIC */
+#define AIC_POL_CTRL(x) (0xE000 - ((x) << 12))
+#define AIC_TYPE_CTRL(x) (0xE004 - ((x) << 12))
+#define AIC_ENABLE_CTRL(x) (0xE008 - ((x) << 12))
+#define AIC_RAW_STAL(x) (0xE00C - ((x) << 12))
+#define AIC_ENABLE_SET(x) (0xE00C - ((x) << 12))
+#define AIC_ENABLED_STAT(x) (0xE010 - ((x) << 12))
+#define AIC_ACK(x) (0xE010 - ((x) << 12))
+#define AIC_ENABLE_CLR(x) (0xE014 - ((x) << 12))
+#define AIC_OPTIONS(x) (0xE018 - ((x) << 12))
+#define AIC_VERSION(x) (0xE01C - ((x) << 12))
+
+/* HIA, Global AIC */
+#define AIC_G_POL_CTRL 0xF800
+#define AIC_G_TYPE_CTRL 0xF804
+#define AIC_G_ENABLE_CTRL 0xF808
+#define AIC_G_RAW_STAT 0xF80C
+#define AIC_G_ENABLE_SET 0xF80C
+#define AIC_G_ENABLED_STAT 0xF810
+#define AIC_G_ACK 0xF810
+#define AIC_G_ENABLE_CLR 0xF814
+#define AIC_G_OPTIONS 0xF818
+#define AIC_G_VERSION 0xF81C
+
+/* HIA, Data Fetch Engine */
+#define DFE_CFG 0xF000
+#define DFE_PRIO_0 0xF010
+#define DFE_PRIO_1 0xF014
+#define DFE_PRIO_2 0xF018
+#define DFE_PRIO_3 0xF01C
+
+/* HIA, Data Fetch Engine access monitoring for CDR */
+#define DFE_RING_REGION_LO(x) (0xF080 + ((x) << 3))
+#define DFE_RING_REGION_HI(x) (0xF084 + ((x) << 3))
+
+/* HIA, Data Fetch Engine thread control and status for thread */
+#define DFE_THR_CTRL 0xF200
+#define DFE_THR_STAT 0xF204
+#define DFE_THR_DESC_CTRL 0xF208
+#define DFE_THR_DESC_DPTR_LO 0xF210
+#define DFE_THR_DESC_DPTR_HI 0xF214
+#define DFE_THR_DESC_ACDPTR_LO 0xF218
+#define DFE_THR_DESC_ACDPTR_HI 0xF21C
+
+/* HIA, Data Store Engine */
+#define DSE_CFG 0xF400
+#define DSE_PRIO_0 0xF410
+#define DSE_PRIO_1 0xF414
+#define DSE_PRIO_2 0xF418
+#define DSE_PRIO_3 0xF41C
+
+/* HIA, Data Store Engine access monitoring for RDR */
+#define DSE_RING_REGION_LO(x) (0xF480 + ((x) << 3))
+#define DSE_RING_REGION_HI(x) (0xF484 + ((x) << 3))
+
+/* HIA, Data Store Engine thread control and status for thread */
+#define DSE_THR_CTRL 0xF600
+#define DSE_THR_STAT 0xF604
+#define DSE_THR_DESC_CTRL 0xF608
+#define DSE_THR_DESC_DPTR_LO 0xF610
+#define DSE_THR_DESC_DPTR_HI 0xF614
+#define DSE_THR_DESC_S_DPTR_LO 0xF618
+#define DSE_THR_DESC_S_DPTR_HI 0xF61C
+#define DSE_THR_ERROR_STAT 0xF620
+
+/* HIA Global */
+#define HIA_MST_CTRL 0xFFF4
+#define HIA_OPTIONS 0xFFF8
+#define HIA_VERSION 0xFFFC
+
+/* Processing Engine Input Side, Processing Engine */
+#define PE_IN_DBUF_THRESH 0x10000
+#define PE_IN_TBUF_THRESH 0x10100
+
+/* Packet Engine Configuration / Status Registers */
+#define PE_TOKEN_CTRL_STAT 0x11000
+#define PE_FUNCTION_EN 0x11004
+#define PE_CONTEXT_CTRL 0x11008
+#define PE_INTERRUPT_CTRL_STAT 0x11010
+#define PE_CONTEXT_STAT 0x1100C
+#define PE_OUT_TRANS_CTRL_STAT 0x11018
+#define PE_OUT_BUF_CTRL 0x1101C
+
+/* Packet Engine PRNG Registers */
+#define PE_PRNG_STAT 0x11040
+#define PE_PRNG_CTRL 0x11044
+#define PE_PRNG_SEED_L 0x11048
+#define PE_PRNG_SEED_H 0x1104C
+#define PE_PRNG_KEY_0_L 0x11050
+#define PE_PRNG_KEY_0_H 0x11054
+#define PE_PRNG_KEY_1_L 0x11058
+#define PE_PRNG_KEY_1_H 0x1105C
+#define PE_PRNG_RES_0 0x11060
+#define PE_PRNG_RES_1 0x11064
+#define PE_PRNG_RES_2 0x11068
+#define PE_PRNG_RES_3 0x1106C
+#define PE_PRNG_LFSR_L 0x11070
+#define PE_PRNG_LFSR_H 0x11074
+
+/* Packet Engine AIC */
+#define PE_EIP96_AIC_POL_CTRL 0x113C0
+#define PE_EIP96_AIC_TYPE_CTRL 0x113C4
+#define PE_EIP96_AIC_ENABLE_CTRL 0x113C8
+#define PE_EIP96_AIC_RAW_STAT 0x113CC
+#define PE_EIP96_AIC_ENABLE_SET 0x113CC
+#define PE_EIP96_AIC_ENABLED_STAT 0x113D0
+#define PE_EIP96_AIC_ACK 0x113D0
+#define PE_EIP96_AIC_ENABLE_CLR 0x113D4
+#define PE_EIP96_AIC_OPTIONS 0x113D8
+#define PE_EIP96_AIC_VERSION 0x113DC
+
+/* Packet Engine Options & Version Registers */
+#define PE_EIP96_OPTIONS 0x113F8
+#define PE_EIP96_VERSION 0x113FC
+
+/* Processing Engine Output Side */
+#define PE_OUT_DBUF_THRESH 0x11C00
+#define PE_OUT_TBUF_THRESH 0x11D00
+
+/* Processing Engine Local AIC */
+#define PE_AIC_POL_CTRL 0x11F00
+#define PE_AIC_TYPE_CTRL 0x11F04
+#define PE_AIC_ENABLE_CTRL 0x11F08
+#define PE_AIC_RAW_STAT 0x11F0C
+#define PE_AIC_ENABLE_SET 0x11F0C
+#define PE_AIC_ENABLED_STAT 0x11F10
+#define PE_AIC_ENABLE_CLR 0x11F14
+#define PE_AIC_OPTIONS 0x11F18
+#define PE_AIC_VERSION 0x11F1C
+
+/* Processing Engine General Configuration and Version */
+#define PE_IN_FLIGHT 0x11FF0
+#define PE_OPTIONS 0x11FF8
+#define PE_VERSION 0x11FFC
+
+/* EIP-97 - Global */
+#define EIP97_CLOCK_STATE 0x1FFE4
+#define EIP97_FORCE_CLOCK_ON 0x1FFE8
+#define EIP97_FORCE_CLOCK_OFF 0x1FFEC
+#define EIP97_MST_CTRL 0x1FFF4
+#define EIP97_OPTIONS 0x1FFF8
+#define EIP97_VERSION 0x1FFFC
+#endif /* __MTK_REGS_H__ */
diff --git a/drivers/crypto/mediatek/mtk-sha.c b/drivers/crypto/mediatek/mtk-sha.c
new file mode 100644
index 0000000..45479bb
--- /dev/null
+++ b/drivers/crypto/mediatek/mtk-sha.c
@@ -0,0 +1,1384 @@
+/*
+ * Cryptographic API.
+ *
+ * Support for MediaTek SHA1/SHA2 hardware accelerator.
+ *
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: Ryder Lee <ryder.lee-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * Some ideas are from atmel-sha.c and omap-sham.c drivers.
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/scatterlist.h>
+#include <linux/crypto.h>
+#include <crypto/scatterwalk.h>
+#include <crypto/algapi.h>
+#include <crypto/sha.h>
+#include <crypto/internal/hash.h>
+#include "mtk-platform.h"
+#include "mtk-regs.h"
+
+#define SHA_ALIGN_MSK (sizeof(u32) - 1)
+#define SHA_QUEUE_SIZE 512
+#define SHA_TMP_STATE_SIZE 512
+
+#define SHA_DATA_LEN_MSK GENMASK(16, 0)
+#define SHA_BUFFER_LEN ((u32)PAGE_SIZE)
+
+#define SHA_OP_UPDATE 1
+#define SHA_OP_FINAL 2
+
+/* SHA command token */
+#define SHA_CT_SIZE 5
+#define SHA_CT_CTRL_HDR 0x02220000
+#define SHA_COMMAND0 0x03020000
+#define SHA_COMMAND1 0x21060000
+#define SHA_COMMAND2 0xe0e63802
+
+/* SHA transform information */
+#define SHA_TFM_HASH (0x2 << 0)
+#define SHA_TFM_DIG_TYPE (0x1 << 21)
+#define SHA_TFM_SIZE(x) ((x) << 8)
+#define SHA_TFM_START (0x1 << 4)
+#define SHA_TFM_CONTINUE (0x1 << 5)
+#define SHA_TFM_HASH_STORE (0x1 << 19)
+#define SHA_TFM_SHA1 (0x2 << 23)
+#define SHA_TFM_SHA256 (0x3 << 23)
+#define SHA_TFM_SHA224 (0x4 << 23)
+#define SHA_TFM_SHA512 (0x5 << 23)
+#define SHA_TFM_SHA384 (0x6 << 23)
+#define SHA_TFM_DIGEST(x) (((x) & 0xf) << 24)
+
+/* SHA flags */
+#define SHA_FLAGS_BUSY BIT(0)
+#define SHA_FLAGS_FINAL BIT(1)
+#define SHA_FLAGS_FINUP BIT(2)
+#define SHA_FLAGS_SG BIT(3)
+#define SHA_FLAGS_ALGO_MASK GENMASK(8, 4)
+#define SHA_FLAGS_SHA1 BIT(4)
+#define SHA_FLAGS_SHA224 BIT(5)
+#define SHA_FLAGS_SHA256 BIT(6)
+#define SHA_FLAGS_SHA384 BIT(7)
+#define SHA_FLAGS_SHA512 BIT(8)
+#define SHA_FLAGS_HMAC BIT(9)
+#define SHA_FLAGS_PAD BIT(10)
+
+/* SHA command token */
+struct mtk_sha_ct {
+ u32 tfm_ctrl0;
+ u32 tfm_ctrl1;
+ u32 ct_ctrl0;
+ u32 ct_ctrl1;
+ u32 ct_ctrl2;
+};
+
+/* SHA transform state */
+struct mtk_sha_tfm {
+ u32 tfm_ctrl0;
+ u32 tfm_ctrl1;
+ u8 digest[SHA512_DIGEST_SIZE] __aligned(sizeof(u32));
+};
+
+struct mtk_sha_info {
+ struct mtk_sha_ct ct;
+ struct mtk_sha_tfm tfm;
+};
+
+struct mtk_sha_reqctx {
+ struct mtk_sha_info info;
+ unsigned long flags;
+ unsigned long op;
+
+ u64 digcnt;
+ bool start;
+ size_t bufcnt;
+ dma_addr_t dma_addr;
+
+ /* walk state */
+ struct scatterlist *sg;
+ u32 offset; /* offset in current sg */
+ u32 total; /* total request */
+ size_t ds;
+ size_t bs;
+
+ u8 *buffer;
+};
+
+struct mtk_sha_hmac_ctx {
+ struct crypto_shash *shash;
+ u8 ipad[SHA512_BLOCK_SIZE] __aligned(sizeof(u32));
+ u8 opad[SHA512_BLOCK_SIZE] __aligned(sizeof(u32));
+};
+
+struct mtk_sha_ctx {
+ struct mtk_cryp *cryp;
+ unsigned long flags;
+ u8 id;
+ u8 buf[SHA_BUFFER_LEN] __aligned(sizeof(u32));
+
+ struct mtk_sha_hmac_ctx base[0];
+};
+
+struct mtk_sha_drv {
+ struct list_head dev_list;
+ /* device list lock */
+ spinlock_t lock;
+};
+
+static struct mtk_sha_drv mtk_sha = {
+ .dev_list = LIST_HEAD_INIT(mtk_sha.dev_list),
+ .lock = __SPIN_LOCK_UNLOCKED(mtk_sha.lock),
+};
+
+static int mtk_sha_handle_queue(struct mtk_cryp *cryp, u8 id,
+ struct ahash_request *req);
+
+static inline u32 mtk_sha_read(struct mtk_cryp *cryp, u32 offset)
+{
+ return readl_relaxed(cryp->base + offset);
+}
+
+static inline void mtk_sha_write(struct mtk_cryp *cryp,
+ u32 offset, u32 value)
+{
+ writel_relaxed(value, cryp->base + offset);
+}
+
+static struct mtk_cryp *mtk_sha_find_dev(struct mtk_sha_ctx *tctx)
+{
+ struct mtk_cryp *cryp = NULL;
+ struct mtk_cryp *tmp;
+
+ spin_lock_bh(&mtk_sha.lock);
+ if (!tctx->cryp) {
+ list_for_each_entry(tmp, &mtk_sha.dev_list, sha_list) {
+ cryp = tmp;
+ break;
+ }
+ tctx->cryp = cryp;
+ } else {
+ cryp = tctx->cryp;
+ }
+
+ /*
+ * Assign record id to tfm in round-robin fashion, and this
+ * help us to link to descriptor ring and record.
+ */
+ tctx->id = cryp->rec;
+ cryp->rec = !cryp->rec;
+
+ spin_unlock_bh(&mtk_sha.lock);
+
+ return cryp;
+}
+
+static int mtk_sha_append_sg(struct mtk_sha_reqctx *ctx)
+{
+ size_t count;
+
+ while ((ctx->bufcnt < SHA_BUFFER_LEN) && ctx->total) {
+ count = min(ctx->sg->length - ctx->offset, ctx->total);
+ count = min(count, SHA_BUFFER_LEN - ctx->bufcnt);
+
+ if (count <= 0) {
+ /*
+ * Check if count <= 0 because the buffer is full or
+ * because the sg length is 0. In the latest case,
+ * check if there is another sg in the list, a 0 length
+ * sg doesn't necessarily mean the end of the sg list.
+ */
+ if ((ctx->sg->length == 0) && !sg_is_last(ctx->sg)) {
+ ctx->sg = sg_next(ctx->sg);
+ continue;
+ } else {
+ break;
+ }
+ }
+
+ scatterwalk_map_and_copy(ctx->buffer + ctx->bufcnt, ctx->sg,
+ ctx->offset, count, 0);
+
+ ctx->bufcnt += count;
+ ctx->offset += count;
+ ctx->total -= count;
+
+ if (ctx->offset == ctx->sg->length) {
+ ctx->sg = sg_next(ctx->sg);
+ if (ctx->sg)
+ ctx->offset = 0;
+ else
+ ctx->total = 0;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * The purpose of this padding is to ensure that the padded message is a
+ * multiple of 512 bits (SHA1/SHA224/SHA256) or 1024 bits (SHA384/SHA512).
+ * The bit "1" is appended at the end of the message followed by
+ * "padlen-1" zero bits. Then a 64 bits block (SHA1/SHA224/SHA256) or
+ * 128 bits block (SHA384/SHA512) equals to the message length in bits
+ * is appended.
+ *
+ * For SHA1/SHA224/SHA256, padlen is calculated as followed:
+ * - if message length < 56 bytes then padlen = 56 - message length
+ * - else padlen = 64 + 56 - message length
+ *
+ * For SHA384/SHA512, padlen is calculated as followed:
+ * - if message length < 112 bytes then padlen = 112 - message length
+ * - else padlen = 128 + 112 - message length
+ */
+static void mtk_sha_fill_padding(struct mtk_sha_reqctx *ctx, u32 len)
+{
+ u32 index, padlen;
+ u64 bits[2];
+ u64 size = ctx->digcnt;
+
+ size += ctx->bufcnt;
+ size += len;
+
+ bits[1] = cpu_to_be64(size << 3);
+ bits[0] = cpu_to_be64(size >> 61);
+
+ if (ctx->flags & (SHA_FLAGS_SHA384 | SHA_FLAGS_SHA512)) {
+ index = ctx->bufcnt & 0x7f;
+ padlen = (index < 112) ? (112 - index) : ((128 + 112) - index);
+ *(ctx->buffer + ctx->bufcnt) = 0x80;
+ memset(ctx->buffer + ctx->bufcnt + 1, 0, padlen - 1);
+ memcpy(ctx->buffer + ctx->bufcnt + padlen, bits, 16);
+ ctx->bufcnt += padlen + 16;
+ ctx->flags |= SHA_FLAGS_PAD;
+ } else {
+ index = ctx->bufcnt & 0x3f;
+ padlen = (index < 56) ? (56 - index) : ((64 + 56) - index);
+ *(ctx->buffer + ctx->bufcnt) = 0x80;
+ memset(ctx->buffer + ctx->bufcnt + 1, 0, padlen - 1);
+ memcpy(ctx->buffer + ctx->bufcnt + padlen, &bits[1], 8);
+ ctx->bufcnt += padlen + 8;
+ ctx->flags |= SHA_FLAGS_PAD;
+ }
+}
+
+static void mtk_sha_info_init(struct mtk_sha *sha,
+ struct mtk_sha_reqctx *ctx)
+{
+ struct mtk_sha_info *info = sha->info;
+ struct mtk_sha_ct *ct = &info->ct;
+ struct mtk_sha_tfm *tfm = &info->tfm;
+
+ sha->ct_hdr = SHA_CT_CTRL_HDR;
+ sha->ct_size = SHA_CT_SIZE;
+
+ tfm->tfm_ctrl0 = SHA_TFM_HASH | SHA_TFM_DIG_TYPE |
+ SHA_TFM_SIZE(WORD(ctx->ds));
+
+ switch (ctx->flags & SHA_FLAGS_ALGO_MASK) {
+ case SHA_FLAGS_SHA1:
+ tfm->tfm_ctrl0 |= SHA_TFM_SHA1;
+ break;
+ case SHA_FLAGS_SHA224:
+ tfm->tfm_ctrl0 |= SHA_TFM_SHA224;
+ break;
+ case SHA_FLAGS_SHA256:
+ tfm->tfm_ctrl0 |= SHA_TFM_SHA256;
+ break;
+ case SHA_FLAGS_SHA384:
+ tfm->tfm_ctrl0 |= SHA_TFM_SHA384;
+ break;
+ case SHA_FLAGS_SHA512:
+ tfm->tfm_ctrl0 |= SHA_TFM_SHA512;
+ break;
+
+ default:
+ /* Should not happen... */
+ return;
+ }
+
+ tfm->tfm_ctrl1 = SHA_TFM_HASH_STORE;
+ ct->tfm_ctrl0 = tfm->tfm_ctrl0 | SHA_TFM_CONTINUE | SHA_TFM_START;
+ ct->tfm_ctrl1 = tfm->tfm_ctrl1;
+
+ ct->ct_ctrl0 = SHA_COMMAND0;
+ ct->ct_ctrl1 = SHA_COMMAND1;
+ ct->ct_ctrl2 = SHA_COMMAND2 | SHA_TFM_DIGEST(WORD(ctx->ds));
+}
+
+static int mtk_sha_info_map(struct mtk_cryp *cryp,
+ struct mtk_sha *sha, size_t len)
+{
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(sha->req);
+ struct mtk_sha_info *info = sha->info;
+ struct mtk_sha_ct *ct = &info->ct;
+
+ if (ctx->start)
+ ctx->start = false;
+ else
+ ct->tfm_ctrl0 &= ~SHA_TFM_START;
+
+ sha->ct_hdr = (sha->ct_hdr & ~SHA_DATA_LEN_MSK) | len;
+ ct->ct_ctrl0 = (ct->ct_ctrl0 & ~SHA_DATA_LEN_MSK) | len;
+
+ ctx->digcnt += len;
+
+ sha->ct_dma = dma_map_single(cryp->dev, info, sizeof(*info),
+ DMA_BIDIRECTIONAL);
+ if (unlikely(dma_mapping_error(cryp->dev, sha->ct_dma))) {
+ dev_err(cryp->dev, "dma %d bytes error\n", sizeof(*info));
+ return -EINVAL;
+ }
+ sha->tfm_dma = sha->ct_dma + sizeof(*ct);
+
+ return 0;
+}
+
+static int mtk_sha_finish_hmac(struct ahash_request *req)
+{
+ struct mtk_sha_ctx *tctx = crypto_tfm_ctx(req->base.tfm);
+ struct mtk_sha_hmac_ctx *bctx = tctx->base;
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(req);
+
+ SHASH_DESC_ON_STACK(shash, bctx->shash);
+
+ shash->tfm = bctx->shash;
+ shash->flags = 0; /* not CRYPTO_TFM_REQ_MAY_SLEEP */
+
+ return crypto_shash_init(shash) ?:
+ crypto_shash_update(shash, bctx->opad, ctx->bs) ?:
+ crypto_shash_finup(shash, req->result, ctx->ds, req->result);
+}
+
+static int mtk_sha_init(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct mtk_sha_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(req);
+
+ ctx->flags = 0;
+ ctx->ds = crypto_ahash_digestsize(tfm);
+
+ switch (ctx->ds) {
+ case SHA1_DIGEST_SIZE:
+ ctx->flags |= SHA_FLAGS_SHA1;
+ ctx->bs = SHA1_BLOCK_SIZE;
+ break;
+ case SHA224_DIGEST_SIZE:
+ ctx->flags |= SHA_FLAGS_SHA224;
+ ctx->bs = SHA224_BLOCK_SIZE;
+ break;
+ case SHA256_DIGEST_SIZE:
+ ctx->flags |= SHA_FLAGS_SHA256;
+ ctx->bs = SHA256_BLOCK_SIZE;
+ break;
+ case SHA384_DIGEST_SIZE:
+ ctx->flags |= SHA_FLAGS_SHA384;
+ ctx->bs = SHA384_BLOCK_SIZE;
+ break;
+ case SHA512_DIGEST_SIZE:
+ ctx->flags |= SHA_FLAGS_SHA512;
+ ctx->bs = SHA512_BLOCK_SIZE;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ ctx->bufcnt = 0;
+ ctx->digcnt = 0;
+ ctx->buffer = tctx->buf;
+ ctx->start = true;
+
+ if (tctx->flags & SHA_FLAGS_HMAC) {
+ struct mtk_sha_hmac_ctx *bctx = tctx->base;
+
+ memcpy(ctx->buffer, bctx->ipad, ctx->bs);
+ ctx->bufcnt = ctx->bs;
+ ctx->flags |= SHA_FLAGS_HMAC;
+ }
+
+ return 0;
+}
+
+static int mtk_sha_xmit(struct mtk_cryp *cryp, struct mtk_sha *sha,
+ dma_addr_t addr, size_t len)
+{
+ struct mtk_ring *ring = cryp->ring[sha->id];
+ struct mtk_desc *cmd = ring->cmd_base + ring->pos;
+ struct mtk_desc *res = ring->res_base + ring->pos;
+ int err;
+
+ err = mtk_sha_info_map(cryp, sha, len);
+ if (err)
+ return err;
+
+ res->hdr = MTK_DESC_FIRST | MTK_DESC_LAST |
+ MTK_DESC_BUF_LEN(len);
+
+ res->buf = cryp->tmp_dma;
+
+ cmd->hdr = MTK_DESC_FIRST | MTK_DESC_LAST |
+ MTK_DESC_BUF_LEN(len) |
+ MTK_DESC_CT_LEN(sha->ct_size);
+
+ cmd->buf = addr;
+ cmd->ct = sha->ct_dma;
+ cmd->ct_hdr = sha->ct_hdr;
+ cmd->tfm = sha->tfm_dma;
+
+ if (++ring->pos == MTK_MAX_DESC_NUM)
+ ring->pos = 0;
+
+ /* make sure all descriptor are filled done */
+ wmb();
+ mtk_sha_write(cryp, RDR_PREP_COUNT(sha->id), MTK_DESC_CNT(1));
+ mtk_sha_write(cryp, CDR_PREP_COUNT(sha->id), MTK_DESC_CNT(1));
+
+ return -EINPROGRESS;
+}
+
+static int mtk_sha_xmit2(struct mtk_cryp *cryp, struct mtk_sha *sha,
+ struct mtk_sha_reqctx *ctx, size_t len1, size_t len2)
+{
+ struct mtk_ring *ring = cryp->ring[sha->id];
+ struct mtk_desc *cmd = ring->cmd_base + ring->pos;
+ struct mtk_desc *res = ring->res_base + ring->pos;
+ int err;
+
+ err = mtk_sha_info_map(cryp, sha, len1 + len2);
+ if (err)
+ return err;
+
+ res->hdr = MTK_DESC_BUF_LEN(len1) | MTK_DESC_FIRST;
+ res->buf = cryp->tmp_dma;
+
+ cmd->hdr = MTK_DESC_BUF_LEN(len1) | MTK_DESC_FIRST |
+ MTK_DESC_CT_LEN(sha->ct_size);
+ cmd->buf = sg_dma_address(ctx->sg);
+ cmd->ct = sha->ct_dma;
+ cmd->ct_hdr = sha->ct_hdr;
+ cmd->tfm = sha->tfm_dma;
+
+ if (++ring->pos == MTK_MAX_DESC_NUM)
+ ring->pos = 0;
+
+ cmd = ring->cmd_base + ring->pos;
+ res = ring->res_base + ring->pos;
+
+ res->hdr = MTK_DESC_BUF_LEN(len2) | MTK_DESC_LAST;
+ res->buf = cryp->tmp_dma;
+
+ cmd->hdr = MTK_DESC_BUF_LEN(len2) | MTK_DESC_LAST;
+ cmd->buf = ctx->dma_addr;
+
+ if (++ring->pos == MTK_MAX_DESC_NUM)
+ ring->pos = 0;
+
+ /* make sure all descriptor are filled done */
+ wmb();
+ mtk_sha_write(cryp, RDR_PREP_COUNT(sha->id), MTK_DESC_CNT(2));
+ mtk_sha_write(cryp, CDR_PREP_COUNT(sha->id), MTK_DESC_CNT(2));
+
+ return -EINPROGRESS;
+}
+
+static int mtk_sha_dma_map(struct mtk_cryp *cryp, struct mtk_sha *sha,
+ struct mtk_sha_reqctx *ctx, size_t count)
+{
+ ctx->dma_addr = dma_map_single(cryp->dev, ctx->buffer,
+ SHA_BUFFER_LEN, DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(cryp->dev, ctx->dma_addr))) {
+ dev_err(cryp->dev, "dma map error\n");
+ return -EINVAL;
+ }
+
+ ctx->flags &= ~SHA_FLAGS_SG;
+
+ return mtk_sha_xmit(cryp, sha, ctx->dma_addr, count);
+}
+
+static int mtk_sha_update_slow(struct mtk_cryp *cryp, struct mtk_sha *sha)
+{
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(sha->req);
+ size_t count;
+ u32 final;
+
+ mtk_sha_append_sg(ctx);
+
+ final = (ctx->flags & SHA_FLAGS_FINUP) && !ctx->total;
+
+ dev_dbg(cryp->dev, "slow: bufcnt: %u\n", ctx->bufcnt);
+
+ if (final) {
+ sha->flags |= SHA_FLAGS_FINAL;
+ mtk_sha_fill_padding(ctx, 0);
+ }
+
+ if (final || (ctx->bufcnt == SHA_BUFFER_LEN && ctx->total)) {
+ count = ctx->bufcnt;
+ ctx->bufcnt = 0;
+
+ return mtk_sha_dma_map(cryp, sha, ctx, count);
+ }
+ return 0;
+}
+
+static int mtk_sha_update_req(struct mtk_cryp *cryp, struct mtk_sha *sha)
+{
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(sha->req);
+ u32 len, final, tail;
+ struct scatterlist *sg;
+
+ if (!ctx->total)
+ return 0;
+
+ if (ctx->bufcnt || ctx->offset)
+ return mtk_sha_update_slow(cryp, sha);
+
+ sg = ctx->sg;
+
+ if (!IS_ALIGNED(sg->offset, sizeof(u32)))
+ return mtk_sha_update_slow(cryp, sha);
+
+ if (!sg_is_last(sg) && !IS_ALIGNED(sg->length, ctx->bs))
+ /* size is not bs aligned */
+ return mtk_sha_update_slow(cryp, sha);
+
+ len = min(ctx->total, sg->length);
+
+ if (sg_is_last(sg)) {
+ if (!(ctx->flags & SHA_FLAGS_FINUP)) {
+ /* not last sg must be bs aligned */
+ tail = len & (ctx->bs - 1);
+ len -= tail;
+ }
+ }
+
+ ctx->total -= len;
+ ctx->offset = len; /* offset where to start slow */
+
+ final = (ctx->flags & SHA_FLAGS_FINUP) && !ctx->total;
+
+ /* Add padding */
+ if (final) {
+ size_t count;
+
+ tail = len & (ctx->bs - 1);
+ len -= tail;
+ ctx->total += tail;
+ ctx->offset = len; /* offset where to start slow */
+
+ sg = ctx->sg;
+ mtk_sha_append_sg(ctx);
+ mtk_sha_fill_padding(ctx, len);
+
+ ctx->dma_addr = dma_map_single(cryp->dev, ctx->buffer,
+ SHA_BUFFER_LEN, DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(cryp->dev, ctx->dma_addr))) {
+ dev_err(cryp->dev, "dma map bytes error\n");
+ return -EINVAL;
+ }
+
+ sha->flags |= SHA_FLAGS_FINAL;
+ count = ctx->bufcnt;
+ ctx->bufcnt = 0;
+
+ if (len == 0) {
+ ctx->flags &= ~SHA_FLAGS_SG;
+ return mtk_sha_xmit(cryp, sha, ctx->dma_addr, count);
+
+ } else {
+ ctx->sg = sg;
+ if (!dma_map_sg(cryp->dev, ctx->sg, 1, DMA_TO_DEVICE)) {
+ dev_err(cryp->dev, "dma_map_sg error\n");
+ return -EINVAL;
+ }
+
+ ctx->flags |= SHA_FLAGS_SG;
+ return mtk_sha_xmit2(cryp, sha, ctx, len, count);
+ }
+ }
+
+ if (!dma_map_sg(cryp->dev, ctx->sg, 1, DMA_TO_DEVICE)) {
+ dev_err(cryp->dev, "dma_map_sg error\n");
+ return -EINVAL;
+ }
+
+ ctx->flags |= SHA_FLAGS_SG;
+
+ return mtk_sha_xmit(cryp, sha, sg_dma_address(ctx->sg), len);
+}
+
+static int mtk_sha_final_req(struct mtk_cryp *cryp, struct mtk_sha *sha)
+{
+ struct ahash_request *req = sha->req;
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(req);
+ size_t count;
+
+ mtk_sha_fill_padding(ctx, 0);
+
+ sha->flags |= SHA_FLAGS_FINAL;
+ count = ctx->bufcnt;
+ ctx->bufcnt = 0;
+
+ return mtk_sha_dma_map(cryp, sha, ctx, count);
+}
+
+static int mtk_sha_finish(struct ahash_request *req)
+{
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(req);
+ u8 *digest = ctx->info.tfm.digest;
+
+ memcpy(req->result, digest, ctx->ds);
+
+ if (ctx->flags & SHA_FLAGS_HMAC)
+ return mtk_sha_finish_hmac(req);
+
+ return 0;
+}
+
+static void mtk_sha_finish_req(struct mtk_cryp *cryp,
+ struct mtk_sha *sha, int err)
+{
+ if (likely(!err && (SHA_FLAGS_FINAL & sha->flags)))
+ err = mtk_sha_finish(sha->req);
+
+ sha->flags &= ~(SHA_FLAGS_BUSY | SHA_FLAGS_FINAL);
+
+ sha->req->base.complete(&sha->req->base, err);
+ mtk_sha_handle_queue(cryp, sha->id - RING2, NULL);
+}
+
+static int mtk_sha_handle_queue(struct mtk_cryp *cryp, u8 id,
+ struct ahash_request *req)
+{
+ struct mtk_sha *sha = cryp->sha[id];
+ struct crypto_async_request *async_req, *backlog;
+ struct mtk_sha_reqctx *ctx;
+ unsigned long flags;
+ int err = 0, ret = 0;
+
+ spin_lock_irqsave(&sha->lock, flags);
+ if (req)
+ ret = ahash_enqueue_request(&sha->queue, req);
+
+ if (SHA_FLAGS_BUSY & sha->flags) {
+ spin_unlock_irqrestore(&sha->lock, flags);
+ return ret;
+ }
+
+ backlog = crypto_get_backlog(&sha->queue);
+ async_req = crypto_dequeue_request(&sha->queue);
+ if (async_req)
+ sha->flags |= SHA_FLAGS_BUSY;
+ spin_unlock_irqrestore(&sha->lock, flags);
+
+ if (!async_req)
+ return ret;
+
+ if (backlog)
+ backlog->complete(backlog, -EINPROGRESS);
+
+ req = ahash_request_cast(async_req);
+ ctx = ahash_request_ctx(req);
+
+ sha->req = req;
+ sha->info = &ctx->info;
+
+ mtk_sha_info_init(sha, ctx);
+
+ if (ctx->op == SHA_OP_UPDATE) {
+ err = mtk_sha_update_req(cryp, sha);
+ if (err != -EINPROGRESS && (ctx->flags & SHA_FLAGS_FINUP))
+ /* no final() after finup() */
+ err = mtk_sha_final_req(cryp, sha);
+ } else if (ctx->op == SHA_OP_FINAL) {
+ err = mtk_sha_final_req(cryp, sha);
+ }
+
+ if (unlikely(err != -EINPROGRESS))
+ /* task will not finish it, so do it here */
+ mtk_sha_finish_req(cryp, sha, err);
+
+ return ret;
+}
+
+static int mtk_sha_enqueue(struct ahash_request *req, u32 op)
+{
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(req);
+ struct mtk_sha_ctx *tctx = crypto_tfm_ctx(req->base.tfm);
+
+ ctx->op = op;
+
+ return mtk_sha_handle_queue(tctx->cryp, tctx->id, req);
+}
+
+static void mtk_sha_unmap(struct mtk_cryp *cryp, struct mtk_sha *sha)
+{
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(sha->req);
+
+ dma_unmap_single(cryp->dev, sha->ct_dma,
+ sizeof(struct mtk_sha_info), DMA_BIDIRECTIONAL);
+
+ if (ctx->flags & SHA_FLAGS_SG) {
+ dma_unmap_sg(cryp->dev, ctx->sg, 1, DMA_TO_DEVICE);
+ if (ctx->sg->length == ctx->offset) {
+ ctx->sg = sg_next(ctx->sg);
+ if (ctx->sg)
+ ctx->offset = 0;
+ }
+ if (ctx->flags & SHA_FLAGS_PAD) {
+ dma_unmap_single(cryp->dev, ctx->dma_addr,
+ SHA_BUFFER_LEN, DMA_TO_DEVICE);
+ }
+ } else
+ dma_unmap_single(cryp->dev, ctx->dma_addr,
+ SHA_BUFFER_LEN, DMA_TO_DEVICE);
+}
+
+static void mtk_sha_complete(struct mtk_cryp *cryp, struct mtk_sha *sha)
+{
+ int err = 0;
+
+ err = mtk_sha_update_req(cryp, sha);
+ if (err != -EINPROGRESS)
+ mtk_sha_finish_req(cryp, sha, err);
+}
+
+static int mtk_sha_update(struct ahash_request *req)
+{
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(req);
+
+ ctx->total = req->nbytes;
+ ctx->sg = req->src;
+ ctx->offset = 0;
+
+ if ((ctx->bufcnt + ctx->total < SHA_BUFFER_LEN) &&
+ !(ctx->flags & SHA_FLAGS_FINUP))
+ return mtk_sha_append_sg(ctx);
+
+ return mtk_sha_enqueue(req, SHA_OP_UPDATE);
+}
+
+static int mtk_sha_final(struct ahash_request *req)
+{
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(req);
+
+ ctx->flags |= SHA_FLAGS_FINUP;
+
+ if (ctx->flags & SHA_FLAGS_PAD)
+ return mtk_sha_finish(req);
+
+ return mtk_sha_enqueue(req, SHA_OP_FINAL);
+}
+
+static int mtk_sha_finup(struct ahash_request *req)
+{
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(req);
+ int err1, err2;
+
+ ctx->flags |= SHA_FLAGS_FINUP;
+
+ err1 = mtk_sha_update(req);
+ if (err1 == -EINPROGRESS || err1 == -EBUSY)
+ return err1;
+ /*
+ * final() has to be always called to cleanup resources
+ * even if update() failed
+ */
+ err2 = mtk_sha_final(req);
+
+ return err1 ?: err2;
+}
+
+static int mtk_sha_digest(struct ahash_request *req)
+{
+ return mtk_sha_init(req) ?: mtk_sha_finup(req);
+}
+
+static int mtk_sha_setkey(struct crypto_ahash *tfm,
+ const unsigned char *key, u32 keylen)
+{
+ struct mtk_sha_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct mtk_sha_hmac_ctx *bctx = tctx->base;
+ size_t bs = crypto_shash_blocksize(bctx->shash);
+ size_t ds = crypto_shash_digestsize(bctx->shash);
+ int err, i;
+
+ SHASH_DESC_ON_STACK(shash, bctx->shash);
+
+ shash->tfm = bctx->shash;
+ shash->flags = crypto_shash_get_flags(bctx->shash) &
+ CRYPTO_TFM_REQ_MAY_SLEEP;
+
+ if (keylen > bs) {
+ err = crypto_shash_digest(shash, key, keylen, bctx->ipad);
+ if (err)
+ return err;
+ keylen = ds;
+ } else {
+ memcpy(bctx->ipad, key, keylen);
+ }
+
+ memset(bctx->ipad + keylen, 0, bs - keylen);
+ memcpy(bctx->opad, bctx->ipad, bs);
+
+ for (i = 0; i < bs; i++) {
+ bctx->ipad[i] ^= 0x36;
+ bctx->opad[i] ^= 0x5c;
+ }
+
+ return err;
+}
+
+static int mtk_sha_export(struct ahash_request *req, void *out)
+{
+ const struct mtk_sha_reqctx *ctx = ahash_request_ctx(req);
+
+ memcpy(out, ctx, sizeof(*ctx));
+ return 0;
+}
+
+static int mtk_sha_import(struct ahash_request *req, const void *in)
+{
+ struct mtk_sha_reqctx *ctx = ahash_request_ctx(req);
+
+ memcpy(ctx, in, sizeof(*ctx));
+ return 0;
+}
+
+static int mtk_sha_cra_init_alg(struct crypto_tfm *tfm,
+ const char *alg_base)
+{
+ struct mtk_sha_ctx *tctx = crypto_tfm_ctx(tfm);
+ struct mtk_cryp *cryp = NULL;
+
+ cryp = mtk_sha_find_dev(tctx);
+ if (!cryp)
+ return -ENODEV;
+
+ crypto_ahash_set_reqsize(__crypto_ahash_cast(tfm),
+ sizeof(struct mtk_sha_reqctx));
+
+ if (alg_base) {
+ struct mtk_sha_hmac_ctx *bctx = tctx->base;
+
+ tctx->flags |= SHA_FLAGS_HMAC;
+ bctx->shash = crypto_alloc_shash(alg_base, 0,
+ CRYPTO_ALG_NEED_FALLBACK);
+ if (IS_ERR(bctx->shash)) {
+ pr_err("base driver %s could not be loaded.\n",
+ alg_base);
+
+ return PTR_ERR(bctx->shash);
+ }
+ }
+ return 0;
+}
+
+static int mtk_sha_cra_init(struct crypto_tfm *tfm)
+{
+ return mtk_sha_cra_init_alg(tfm, NULL);
+}
+
+static int mtk_sha_cra_sha1_init(struct crypto_tfm *tfm)
+{
+ return mtk_sha_cra_init_alg(tfm, "sha1");
+}
+
+static int mtk_sha_cra_sha224_init(struct crypto_tfm *tfm)
+{
+ return mtk_sha_cra_init_alg(tfm, "sha224");
+}
+
+static int mtk_sha_cra_sha256_init(struct crypto_tfm *tfm)
+{
+ return mtk_sha_cra_init_alg(tfm, "sha256");
+}
+
+static int mtk_sha_cra_sha384_init(struct crypto_tfm *tfm)
+{
+ return mtk_sha_cra_init_alg(tfm, "sha384");
+}
+
+static int mtk_sha_cra_sha512_init(struct crypto_tfm *tfm)
+{
+ return mtk_sha_cra_init_alg(tfm, "sha512");
+}
+
+static void mtk_sha_cra_exit(struct crypto_tfm *tfm)
+{
+ struct mtk_sha_ctx *tctx = crypto_tfm_ctx(tfm);
+
+ if (tctx->flags & SHA_FLAGS_HMAC) {
+ struct mtk_sha_hmac_ctx *bctx = tctx->base;
+
+ crypto_free_shash(bctx->shash);
+ }
+}
+
+static struct ahash_alg algs_sha1_sha224_sha256[] = {
+{
+ .init = mtk_sha_init,
+ .update = mtk_sha_update,
+ .final = mtk_sha_final,
+ .finup = mtk_sha_finup,
+ .digest = mtk_sha_digest,
+ .export = mtk_sha_export,
+ .import = mtk_sha_import,
+ .halg.digestsize = SHA1_DIGEST_SIZE,
+ .halg.statesize = sizeof(struct mtk_sha_reqctx),
+ .halg.base = {
+ .cra_name = "sha1",
+ .cra_driver_name = "mtk-sha1",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_blocksize = SHA1_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_sha_ctx),
+ .cra_alignmask = SHA_ALIGN_MSK,
+ .cra_module = THIS_MODULE,
+ .cra_init = mtk_sha_cra_init,
+ .cra_exit = mtk_sha_cra_exit,
+ }
+},
+{
+ .init = mtk_sha_init,
+ .update = mtk_sha_update,
+ .final = mtk_sha_final,
+ .finup = mtk_sha_finup,
+ .digest = mtk_sha_digest,
+ .export = mtk_sha_export,
+ .import = mtk_sha_import,
+ .halg.digestsize = SHA224_DIGEST_SIZE,
+ .halg.statesize = sizeof(struct mtk_sha_reqctx),
+ .halg.base = {
+ .cra_name = "sha224",
+ .cra_driver_name = "mtk-sha224",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_blocksize = SHA224_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_sha_ctx),
+ .cra_alignmask = SHA_ALIGN_MSK,
+ .cra_module = THIS_MODULE,
+ .cra_init = mtk_sha_cra_init,
+ .cra_exit = mtk_sha_cra_exit,
+ }
+},
+{
+ .init = mtk_sha_init,
+ .update = mtk_sha_update,
+ .final = mtk_sha_final,
+ .finup = mtk_sha_finup,
+ .digest = mtk_sha_digest,
+ .export = mtk_sha_export,
+ .import = mtk_sha_import,
+ .halg.digestsize = SHA256_DIGEST_SIZE,
+ .halg.statesize = sizeof(struct mtk_sha_reqctx),
+ .halg.base = {
+ .cra_name = "sha256",
+ .cra_driver_name = "mtk-sha256",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_blocksize = SHA256_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_sha_ctx),
+ .cra_alignmask = SHA_ALIGN_MSK,
+ .cra_module = THIS_MODULE,
+ .cra_init = mtk_sha_cra_init,
+ .cra_exit = mtk_sha_cra_exit,
+ }
+},
+{
+ .init = mtk_sha_init,
+ .update = mtk_sha_update,
+ .final = mtk_sha_final,
+ .finup = mtk_sha_finup,
+ .digest = mtk_sha_digest,
+ .export = mtk_sha_export,
+ .import = mtk_sha_import,
+ .setkey = mtk_sha_setkey,
+ .halg.digestsize = SHA1_DIGEST_SIZE,
+ .halg.statesize = sizeof(struct mtk_sha_reqctx),
+ .halg.base = {
+ .cra_name = "hmac(sha1)",
+ .cra_driver_name = "mtk-hmac-sha1",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_blocksize = SHA1_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_sha_ctx) +
+ sizeof(struct mtk_sha_hmac_ctx),
+ .cra_alignmask = SHA_ALIGN_MSK,
+ .cra_module = THIS_MODULE,
+ .cra_init = mtk_sha_cra_sha1_init,
+ .cra_exit = mtk_sha_cra_exit,
+ }
+},
+{
+ .init = mtk_sha_init,
+ .update = mtk_sha_update,
+ .final = mtk_sha_final,
+ .finup = mtk_sha_finup,
+ .digest = mtk_sha_digest,
+ .export = mtk_sha_export,
+ .import = mtk_sha_import,
+ .setkey = mtk_sha_setkey,
+ .halg.digestsize = SHA224_DIGEST_SIZE,
+ .halg.statesize = sizeof(struct mtk_sha_reqctx),
+ .halg.base = {
+ .cra_name = "hmac(sha224)",
+ .cra_driver_name = "mtk-hmac-sha224",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_blocksize = SHA224_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_sha_ctx) +
+ sizeof(struct mtk_sha_hmac_ctx),
+ .cra_alignmask = SHA_ALIGN_MSK,
+ .cra_module = THIS_MODULE,
+ .cra_init = mtk_sha_cra_sha224_init,
+ .cra_exit = mtk_sha_cra_exit,
+ }
+},
+{
+ .init = mtk_sha_init,
+ .update = mtk_sha_update,
+ .final = mtk_sha_final,
+ .finup = mtk_sha_finup,
+ .digest = mtk_sha_digest,
+ .export = mtk_sha_export,
+ .import = mtk_sha_import,
+ .setkey = mtk_sha_setkey,
+ .halg.digestsize = SHA256_DIGEST_SIZE,
+ .halg.statesize = sizeof(struct mtk_sha_reqctx),
+ .halg.base = {
+ .cra_name = "hmac(sha256)",
+ .cra_driver_name = "mtk-hmac-sha256",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_blocksize = SHA256_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_sha_ctx) +
+ sizeof(struct mtk_sha_hmac_ctx),
+ .cra_alignmask = SHA_ALIGN_MSK,
+ .cra_module = THIS_MODULE,
+ .cra_init = mtk_sha_cra_sha256_init,
+ .cra_exit = mtk_sha_cra_exit,
+ }
+},
+};
+
+static struct ahash_alg algs_sha384_sha512[] = {
+{
+ .init = mtk_sha_init,
+ .update = mtk_sha_update,
+ .final = mtk_sha_final,
+ .finup = mtk_sha_finup,
+ .digest = mtk_sha_digest,
+ .export = mtk_sha_export,
+ .import = mtk_sha_import,
+ .halg.digestsize = SHA384_DIGEST_SIZE,
+ .halg.statesize = sizeof(struct mtk_sha_reqctx),
+ .halg.base = {
+ .cra_name = "sha384",
+ .cra_driver_name = "mtk-sha384",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_blocksize = SHA384_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_sha_ctx),
+ .cra_alignmask = SHA_ALIGN_MSK,
+ .cra_module = THIS_MODULE,
+ .cra_init = mtk_sha_cra_init,
+ .cra_exit = mtk_sha_cra_exit,
+ }
+},
+{
+ .init = mtk_sha_init,
+ .update = mtk_sha_update,
+ .final = mtk_sha_final,
+ .finup = mtk_sha_finup,
+ .digest = mtk_sha_digest,
+ .export = mtk_sha_export,
+ .import = mtk_sha_import,
+ .halg.digestsize = SHA512_DIGEST_SIZE,
+ .halg.statesize = sizeof(struct mtk_sha_reqctx),
+ .halg.base = {
+ .cra_name = "sha512",
+ .cra_driver_name = "mtk-sha512",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_blocksize = SHA512_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_sha_ctx),
+ .cra_alignmask = SHA_ALIGN_MSK,
+ .cra_module = THIS_MODULE,
+ .cra_init = mtk_sha_cra_init,
+ .cra_exit = mtk_sha_cra_exit,
+ }
+},
+{
+ .init = mtk_sha_init,
+ .update = mtk_sha_update,
+ .final = mtk_sha_final,
+ .finup = mtk_sha_finup,
+ .digest = mtk_sha_digest,
+ .export = mtk_sha_export,
+ .import = mtk_sha_import,
+ .setkey = mtk_sha_setkey,
+ .halg.digestsize = SHA384_DIGEST_SIZE,
+ .halg.statesize = sizeof(struct mtk_sha_reqctx),
+ .halg.base = {
+ .cra_name = "hmac(sha384)",
+ .cra_driver_name = "mtk-hmac-sha384",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_blocksize = SHA384_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_sha_ctx) +
+ sizeof(struct mtk_sha_hmac_ctx),
+ .cra_alignmask = SHA_ALIGN_MSK,
+ .cra_module = THIS_MODULE,
+ .cra_init = mtk_sha_cra_sha384_init,
+ .cra_exit = mtk_sha_cra_exit,
+ }
+},
+{
+ .init = mtk_sha_init,
+ .update = mtk_sha_update,
+ .final = mtk_sha_final,
+ .finup = mtk_sha_finup,
+ .digest = mtk_sha_digest,
+ .export = mtk_sha_export,
+ .import = mtk_sha_import,
+ .setkey = mtk_sha_setkey,
+ .halg.digestsize = SHA512_DIGEST_SIZE,
+ .halg.statesize = sizeof(struct mtk_sha_reqctx),
+ .halg.base = {
+ .cra_name = "hmac(sha512)",
+ .cra_driver_name = "mtk-hmac-sha512",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_blocksize = SHA512_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct mtk_sha_ctx) +
+ sizeof(struct mtk_sha_hmac_ctx),
+ .cra_alignmask = SHA_ALIGN_MSK,
+ .cra_module = THIS_MODULE,
+ .cra_init = mtk_sha_cra_sha512_init,
+ .cra_exit = mtk_sha_cra_exit,
+ }
+},
+};
+
+static void mtk_sha_task0(unsigned long data)
+{
+ struct mtk_cryp *cryp = (struct mtk_cryp *)data;
+ struct mtk_sha *sha = cryp->sha[0];
+
+ mtk_sha_unmap(cryp, sha);
+ mtk_sha_complete(cryp, sha);
+}
+
+static void mtk_sha_task1(unsigned long data)
+{
+ struct mtk_cryp *cryp = (struct mtk_cryp *)data;
+ struct mtk_sha *sha = cryp->sha[1];
+
+ mtk_sha_unmap(cryp, sha);
+ mtk_sha_complete(cryp, sha);
+}
+
+static irqreturn_t mtk_sha_ring2_irq(int irq, void *dev_id)
+{
+ struct mtk_cryp *cryp = (struct mtk_cryp *)dev_id;
+ struct mtk_sha *sha = cryp->sha[0];
+ u32 val = mtk_sha_read(cryp, RDR_STAT(RING2));
+
+ mtk_sha_write(cryp, RDR_STAT(RING2), val);
+
+ if (likely((SHA_FLAGS_BUSY & sha->flags))) {
+ mtk_sha_write(cryp, RDR_PROC_COUNT(RING2), MTK_DESC_CNT_CLR);
+ mtk_sha_write(cryp, RDR_THRESH(RING2), MTK_RDR_THRESH_DEF);
+
+ tasklet_schedule(&sha->task);
+ } else {
+ dev_warn(cryp->dev, "AES interrupt when no active requests.\n");
+ }
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t mtk_sha_ring3_irq(int irq, void *dev_id)
+{
+ struct mtk_cryp *cryp = (struct mtk_cryp *)dev_id;
+ struct mtk_sha *sha = cryp->sha[1];
+ u32 val = mtk_sha_read(cryp, RDR_STAT(RING3));
+
+ mtk_sha_write(cryp, RDR_STAT(RING3), val);
+
+ if (likely((SHA_FLAGS_BUSY & sha->flags))) {
+ mtk_sha_write(cryp, RDR_PROC_COUNT(RING3), MTK_DESC_CNT_CLR);
+ mtk_sha_write(cryp, RDR_THRESH(RING3), MTK_RDR_THRESH_DEF);
+
+ tasklet_schedule(&sha->task);
+ } else {
+ dev_warn(cryp->dev, "AES interrupt when no active requests.\n");
+ }
+ return IRQ_HANDLED;
+}
+
+static int mtk_sha_record_init(struct mtk_cryp *cryp)
+{
+ struct mtk_sha **sha = cryp->sha;
+ int i, err = -ENOMEM;
+
+ for (i = 0; i < RECORD_NUM; i++) {
+ sha[i] = kzalloc(sizeof(**sha), GFP_KERNEL);
+ if (!sha[i])
+ goto err_cleanup;
+
+ sha[i]->id = i + RING2;
+
+ spin_lock_init(&sha[i]->lock);
+ crypto_init_queue(&sha[i]->queue, SHA_QUEUE_SIZE);
+ }
+
+ tasklet_init(&sha[0]->task, mtk_sha_task0, (unsigned long)cryp);
+ tasklet_init(&sha[1]->task, mtk_sha_task1, (unsigned long)cryp);
+
+ cryp->rec = 1;
+
+ return 0;
+
+err_cleanup:
+ for (; i--; )
+ kfree(sha[i]);
+ return err;
+}
+
+static void mtk_sha_record_free(struct mtk_cryp *cryp)
+{
+ int i;
+
+ for (i = 0; i < RECORD_NUM; i++) {
+ tasklet_kill(&cryp->sha[i]->task);
+ kfree(cryp->sha[i]);
+ }
+}
+
+static void mtk_sha_unregister_algs(void)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(algs_sha1_sha224_sha256); i++)
+ crypto_unregister_ahash(&algs_sha1_sha224_sha256[i]);
+
+ for (i = 0; i < ARRAY_SIZE(algs_sha384_sha512); i++)
+ crypto_unregister_ahash(&algs_sha384_sha512[i]);
+}
+
+static int mtk_sha_register_algs(void)
+{
+ int err, i;
+
+ for (i = 0; i < ARRAY_SIZE(algs_sha1_sha224_sha256); i++) {
+ err = crypto_register_ahash(&algs_sha1_sha224_sha256[i]);
+ if (err)
+ goto err_sha_224_256_algs;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(algs_sha384_sha512); i++) {
+ err = crypto_register_ahash(&algs_sha384_sha512[i]);
+ if (err)
+ goto err_sha_384_512_algs;
+ }
+
+ return 0;
+
+err_sha_384_512_algs:
+ for (; i--; )
+ crypto_unregister_ahash(&algs_sha384_sha512[i]);
+ i = ARRAY_SIZE(algs_sha1_sha224_sha256);
+err_sha_224_256_algs:
+ for (; i--; )
+ crypto_unregister_ahash(&algs_sha1_sha224_sha256[i]);
+
+ return err;
+}
+
+int mtk_hash_alg_register(struct mtk_cryp *cryp)
+{
+ int err;
+
+ INIT_LIST_HEAD(&cryp->sha_list);
+
+ err = mtk_sha_record_init(cryp);
+ if (err)
+ goto err_record;
+
+ err = devm_request_irq(cryp->dev, cryp->irq[RING2],
+ mtk_sha_ring2_irq, IRQF_TRIGGER_LOW,
+ "mtk-sha", cryp);
+ if (err) {
+ dev_err(cryp->dev, "unable to request sha irq0.\n");
+ goto err_res;
+ }
+
+ err = devm_request_irq(cryp->dev, cryp->irq[RING3],
+ mtk_sha_ring3_irq, IRQF_TRIGGER_LOW,
+ "mtk-sha", cryp);
+ if (err) {
+ dev_err(cryp->dev, "unable to request sha irq1.\n");
+ goto err_res;
+ }
+
+ /* enable ring2 and ring3 interrupt for hash */
+ mtk_sha_write(cryp, AIC_ENABLE_SET(RING2), MTK_IRQ_RDR2);
+ mtk_sha_write(cryp, AIC_ENABLE_SET(RING3), MTK_IRQ_RDR3);
+
+ cryp->tmp = dma_alloc_coherent(cryp->dev, SHA_TMP_STATE_SIZE,
+ &cryp->tmp_dma, GFP_KERNEL);
+ if (!cryp->tmp) {
+ dev_err(cryp->dev, "unable to allocate tmp buffer.\n");
+ err = -EINVAL;
+ goto err_res;
+ }
+
+ spin_lock(&mtk_sha.lock);
+ list_add_tail(&cryp->sha_list, &mtk_sha.dev_list);
+ spin_unlock(&mtk_sha.lock);
+
+ err = mtk_sha_register_algs();
+ if (err)
+ goto err_algs;
+
+ return 0;
+
+err_algs:
+ spin_lock(&mtk_sha.lock);
+ list_del(&cryp->sha_list);
+ spin_unlock(&mtk_sha.lock);
+ dma_free_coherent(cryp->dev, SHA_TMP_STATE_SIZE,
+ cryp->tmp, cryp->tmp_dma);
+err_res:
+ mtk_sha_record_free(cryp);
+err_record:
+
+ dev_err(cryp->dev, "mtk-sha initialization failed.\n");
+ return err;
+}
+EXPORT_SYMBOL(mtk_hash_alg_register);
+
+void mtk_hash_alg_release(struct mtk_cryp *cryp)
+{
+ spin_lock(&mtk_sha.lock);
+ list_del(&cryp->sha_list);
+ spin_unlock(&mtk_sha.lock);
+
+ mtk_sha_unregister_algs();
+ dma_free_coherent(cryp->dev, SHA_TMP_STATE_SIZE,
+ cryp->tmp, cryp->tmp_dma);
+ mtk_sha_record_free(cryp);
+}
+EXPORT_SYMBOL(mtk_hash_alg_release);
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: Crash in crypto mcryptd
From: Tim Chen @ 2016-12-02 1:47 UTC (permalink / raw)
To: Mikulas Patocka, Herbert Xu, David S. Miller
Cc: linux-crypto, dm-devel, Milan Broz, Megha Dey
In-Reply-To: <alpine.LRH.2.02.1612011819540.27565@file01.intranet.prod.int.rdu2.redhat.com>
On Thu, 2016-12-01 at 19:00 -0500, Mikulas Patocka wrote:
> Hi
>
> There is a bug in mcryptd initialization.
>
> This is a test module that tries various hash algorithms. When you load
> the module with "insmod test.ko 'alg=mcryptd(md5)'", the machine crashes.
I don't think your test setup is right. The mcryptd supports only multi-buffer
algorithm. I don't think there is such an implementation for md5.
Please refer to arch/x86/crypto/sha1-mb
multi-buffer implementation of sha1 to see the proper
setup and usage with mcryptd. You can also run tcrypt test to
exercise this code.
Tim
>
> Mikulas
>
>
> #include <linux/module.h>
> #include <linux/init.h>
> #include <crypto/hash.h>
>
> static char *alg = "md5";
>
> module_param_named(alg, alg, charp, 0444);
> MODULE_PARM_DESC(alg, "the algorith to test");
>
> static bool sync = true;
>
> module_param_named(sync, sync, bool, 0444);
> MODULE_PARM_DESC(alg, "sync flag");
>
> static int __init dump_init(void)
> {
> struct crypto_shash *h;
> char key[4];
> int r;
> printk("testing algorithm '%s'\n", alg);
> h = crypto_alloc_shash(alg, 0, sync ? CRYPTO_ALG_ASYNC : 0);
> if (IS_ERR(h)) {
> printk("error %d\n", (int)PTR_ERR(h));
> return PTR_ERR(h);
> }
> printk("setting key\n");
> r = crypto_shash_setkey(h, key, sizeof key);
> if (r)
> printk("setkey: %d\n", r);
> crypto_free_shash(h);
> printk("module loaded\n");
> return 0;
> }
>
> static void __exit dump_exit(void)
> {
> printk("dump exit\n");
> }
>
> module_init(dump_init)
> module_exit(dump_exit)
> MODULE_LICENSE("GPL");
>
>
> [898029.802035] BUG: unable to handle kernel NULL pointer dereference at
> (null)
> [898029.806060] IP: [<ffffffffa022014d>] md5_final+0xad/0x210 [md5]
> [898029.808156] PGD 11a5d8067 [898029.809051] PUD 11a491067
> PMD 0 [898029.810280]
> [898029.810904] Oops: 0002 [#1] PREEMPT SMP
> [898029.812239] Modules linked in: md5 testdump(O+) mcryptd uvesafb
> cfbfillrect cfbimgblt cn cfbcopyarea fbcon bitblit fbcon_rotate fbcon_ccw
> fbcon_ud fbcon_cw softcursor fb fbdev font ipv6 binfmt_misc mousedev
> af_packet psmouse pcspkr virtio_net virtio_balloon button ext4 crc16 jbd2
> mbcache dm_mod virtio_blk evdev virtio_pci virtio_ring virtio
> [898029.817178] CPU: 9 PID: 187 Comm: kworker/9:1 Tainted: G O
> 4.9.0-rc7+ #6
> [898029.818066] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> [898029.818732] Workqueue: crypto mcryptd_queue_worker [mcryptd]
> [898029.819394] task: ffff88011aa2bd80 task.stack: ffff880118480000
> [898029.820077] RIP: 0010:[<ffffffffa022014d>] [<ffffffffa022014d>]
> md5_final+0xad/0x210 [md5]
> [898029.821050] RSP: 0018:ffff880118483d48 EFLAGS: 00010286
> [898029.821661] RAX: 04b2008fd98c1dd4 RBX: ffff880119cd7f28 RCX:
> 00000000980980e9
> [898029.822464] RDX: 7e42f8ec980980e9 RSI: 00000000ef1c4f74 RDI:
> ffff880119cd7f30
> [898029.823293] RBP: ffff880118483d68 R08: 000000001b99d513 R09:
> 0000000000000000
> [898029.824117] R10: 0000000000000000 R11: 00000000b8b56373 R12:
> ffff880119cd7f18
> [898029.824944] R13: 0000000000000000 R14: ffff880119cd7f38 R15:
> ffffffffa01ee43c
> [898029.825776] FS: 0000000000000000(0000) GS:ffff88011fd20000(0000)
> knlGS:0000000000000000
> [898029.826712] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [898029.827376] CR2: 0000000000000000 CR3: 000000011a6c9000 CR4:
> 00000000000006a0
> [898029.828204] Stack:
> [898029.828452] ffff880119cd7f18 ffff88011fd3bb00 0000000000000000
> ffff880119cd7e00
> [898029.829351] ffff880118483da0 ffffffff8119f281 ffff880119cd7f18
> ffff88011fd3bb00
> [898029.830242] ffff88011fd3bae0 ffff880119cd7e00 ffffffffa01ee43c
> ffff880119cd7ec8
> [898029.831141] Call Trace:
> [898029.831460] [<ffffffff8119f281>] ? crypto_shash_final+0x31/0xb0
> [898029.832151] [<ffffffffa01ee43c>] ? mcryptd_queue_worker+0x1c/0x190
> [mcryptd]
> [898029.832980] [<ffffffff8119f743>] ? shash_ahash_finup+0x73/0x80
> [898029.833672] [<ffffffff81016a5f>] ? __switch_to+0x27f/0x460
> [898029.834305] [<ffffffffa01eeabf>] ? mcryptd_hash_digest+0x4f/0x80
> [mcryptd]
> [898029.835125] [<ffffffffa01ee467>] ? mcryptd_queue_worker+0x47/0x190
> [mcryptd]
> [898029.835963] [<ffffffff8105974f>] ? process_one_work+0x1bf/0x3f0
> [898029.836681] [<ffffffff810599c2>] ? worker_thread+0x42/0x4c0
> [898029.837362] [<ffffffff81059980>] ? process_one_work+0x3f0/0x3f0
> [898029.838045] [<ffffffff81059980>] ? process_one_work+0x3f0/0x3f0
> [898029.838739] [<ffffffff8105ea49>] ? kthread+0xb9/0xd0
> [898029.839318] [<ffffffff8105e990>] ? kthread_park+0x70/0x70
> [898029.839959] [<ffffffff8131b965>] ? ret_from_fork+0x25/0x30
> [898029.840594] Code: 14 c5 00 00 00 00 48 c1 e8 1d 41 89 44 24 5c 41 89
> 54 24 58 e8 45 ea 0e e1 49 8b 44 24 10 49 8b 54 24 18 48 8d 7b 08 48 83 e7
> f8 <49> 89 45 00 49 89 55 08 31 c0 49 c7 44 24 10 00 00 00 00 48 c7
> [898029.843633] RIP [<ffffffffa022014d>] md5_final+0xad/0x210 [md5]
> [898029.844354] RSP <ffff880118483d48>
> [898029.844769] CR2: 0000000000000000
> [898029.845166] ---[ end trace 2ecde0bf66717337 ]---
>
^ permalink raw reply
* Crash in crypto mcryptd
From: Mikulas Patocka @ 2016-12-02 0:00 UTC (permalink / raw)
To: Herbert Xu, David S. Miller, Tim Chen; +Cc: linux-crypto, dm-devel, Milan Broz
Hi
There is a bug in mcryptd initialization.
This is a test module that tries various hash algorithms. When you load
the module with "insmod test.ko 'alg=mcryptd(md5)'", the machine crashes.
Mikulas
#include <linux/module.h>
#include <linux/init.h>
#include <crypto/hash.h>
static char *alg = "md5";
module_param_named(alg, alg, charp, 0444);
MODULE_PARM_DESC(alg, "the algorith to test");
static bool sync = true;
module_param_named(sync, sync, bool, 0444);
MODULE_PARM_DESC(alg, "sync flag");
static int __init dump_init(void)
{
struct crypto_shash *h;
char key[4];
int r;
printk("testing algorithm '%s'\n", alg);
h = crypto_alloc_shash(alg, 0, sync ? CRYPTO_ALG_ASYNC : 0);
if (IS_ERR(h)) {
printk("error %d\n", (int)PTR_ERR(h));
return PTR_ERR(h);
}
printk("setting key\n");
r = crypto_shash_setkey(h, key, sizeof key);
if (r)
printk("setkey: %d\n", r);
crypto_free_shash(h);
printk("module loaded\n");
return 0;
}
static void __exit dump_exit(void)
{
printk("dump exit\n");
}
module_init(dump_init)
module_exit(dump_exit)
MODULE_LICENSE("GPL");
[898029.802035] BUG: unable to handle kernel NULL pointer dereference at
(null)
[898029.806060] IP: [<ffffffffa022014d>] md5_final+0xad/0x210 [md5]
[898029.808156] PGD 11a5d8067 [898029.809051] PUD 11a491067
PMD 0 [898029.810280]
[898029.810904] Oops: 0002 [#1] PREEMPT SMP
[898029.812239] Modules linked in: md5 testdump(O+) mcryptd uvesafb
cfbfillrect cfbimgblt cn cfbcopyarea fbcon bitblit fbcon_rotate fbcon_ccw
fbcon_ud fbcon_cw softcursor fb fbdev font ipv6 binfmt_misc mousedev
af_packet psmouse pcspkr virtio_net virtio_balloon button ext4 crc16 jbd2
mbcache dm_mod virtio_blk evdev virtio_pci virtio_ring virtio
[898029.817178] CPU: 9 PID: 187 Comm: kworker/9:1 Tainted: G O
4.9.0-rc7+ #6
[898029.818066] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[898029.818732] Workqueue: crypto mcryptd_queue_worker [mcryptd]
[898029.819394] task: ffff88011aa2bd80 task.stack: ffff880118480000
[898029.820077] RIP: 0010:[<ffffffffa022014d>] [<ffffffffa022014d>]
md5_final+0xad/0x210 [md5]
[898029.821050] RSP: 0018:ffff880118483d48 EFLAGS: 00010286
[898029.821661] RAX: 04b2008fd98c1dd4 RBX: ffff880119cd7f28 RCX:
00000000980980e9
[898029.822464] RDX: 7e42f8ec980980e9 RSI: 00000000ef1c4f74 RDI:
ffff880119cd7f30
[898029.823293] RBP: ffff880118483d68 R08: 000000001b99d513 R09:
0000000000000000
[898029.824117] R10: 0000000000000000 R11: 00000000b8b56373 R12:
ffff880119cd7f18
[898029.824944] R13: 0000000000000000 R14: ffff880119cd7f38 R15:
ffffffffa01ee43c
[898029.825776] FS: 0000000000000000(0000) GS:ffff88011fd20000(0000)
knlGS:0000000000000000
[898029.826712] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[898029.827376] CR2: 0000000000000000 CR3: 000000011a6c9000 CR4:
00000000000006a0
[898029.828204] Stack:
[898029.828452] ffff880119cd7f18 ffff88011fd3bb00 0000000000000000
ffff880119cd7e00
[898029.829351] ffff880118483da0 ffffffff8119f281 ffff880119cd7f18
ffff88011fd3bb00
[898029.830242] ffff88011fd3bae0 ffff880119cd7e00 ffffffffa01ee43c
ffff880119cd7ec8
[898029.831141] Call Trace:
[898029.831460] [<ffffffff8119f281>] ? crypto_shash_final+0x31/0xb0
[898029.832151] [<ffffffffa01ee43c>] ? mcryptd_queue_worker+0x1c/0x190
[mcryptd]
[898029.832980] [<ffffffff8119f743>] ? shash_ahash_finup+0x73/0x80
[898029.833672] [<ffffffff81016a5f>] ? __switch_to+0x27f/0x460
[898029.834305] [<ffffffffa01eeabf>] ? mcryptd_hash_digest+0x4f/0x80
[mcryptd]
[898029.835125] [<ffffffffa01ee467>] ? mcryptd_queue_worker+0x47/0x190
[mcryptd]
[898029.835963] [<ffffffff8105974f>] ? process_one_work+0x1bf/0x3f0
[898029.836681] [<ffffffff810599c2>] ? worker_thread+0x42/0x4c0
[898029.837362] [<ffffffff81059980>] ? process_one_work+0x3f0/0x3f0
[898029.838045] [<ffffffff81059980>] ? process_one_work+0x3f0/0x3f0
[898029.838739] [<ffffffff8105ea49>] ? kthread+0xb9/0xd0
[898029.839318] [<ffffffff8105e990>] ? kthread_park+0x70/0x70
[898029.839959] [<ffffffff8131b965>] ? ret_from_fork+0x25/0x30
[898029.840594] Code: 14 c5 00 00 00 00 48 c1 e8 1d 41 89 44 24 5c 41 89
54 24 58 e8 45 ea 0e e1 49 8b 44 24 10 49 8b 54 24 18 48 8d 7b 08 48 83 e7
f8 <49> 89 45 00 49 89 55 08 31 c0 49 c7 44 24 10 00 00 00 00 48 c7
[898029.843633] RIP [<ffffffffa022014d>] md5_final+0xad/0x210 [md5]
[898029.844354] RSP <ffff880118483d48>
[898029.844769] CR2: 0000000000000000
[898029.845166] ---[ end trace 2ecde0bf66717337 ]---
^ permalink raw reply
* Re: [PATCH 2/3] crypto: brcm: Add Broadcom SPU driver
From: kbuild test robot @ 2016-12-01 23:42 UTC (permalink / raw)
To: Rob Rice
Cc: kbuild-all-JC7UmRfGjtg, Herbert Xu, David S. Miller, Rob Herring,
Mark Rutland, linux-crypto-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Ray Jui, Scott Branden,
Jon Mason, bcm-kernel-feedback-list-dY08KVG/lbpWk0Htik3J/w,
Catalin Marinas, Will Deacon,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Steve Lin,
Rob Rice
In-Reply-To: <1480536453-24781-3-git-send-email-rob.rice-dY08KVG/lbpWk0Htik3J/w@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 1051 bytes --]
Hi Rob,
[auto build test ERROR on cryptodev/master]
[also build test ERROR on v4.9-rc7 next-20161201]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Rob-Rice/crypto-brcm-DT-documentation-for-Broadcom-SPU-driver/20161202-010038
base: https://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git master
config: arm64-defconfig (attached as .config)
compiler: aarch64-linux-gnu-gcc (Debian 6.1.1-9) 6.1.1 20160705
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=arm64
All errors (new ones prefixed by >>):
>> ERROR: "des_ekey" [drivers/crypto/bcm/bcm_crypto_spu.ko] undefined!
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 31957 bytes --]
^ permalink raw reply
* [patch] crypto: chcr - checking for IS_ERR() instead of NULL
From: Dan Carpenter @ 2016-12-01 20:49 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Harsh Jain, Jitendra Lulla, Atul Gupta,
Hariprasad Shenai, Wei Yongjun, linux-crypto, kernel-janitors
The create_hash_wr() function never returns error pointers. It returns
NULL on error.
Fixes: 358961d1cd1e ("crypto: chcr - Added new structure chcr_wr")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
diff --git a/drivers/crypto/chelsio/chcr_algo.c b/drivers/crypto/chelsio/chcr_algo.c
index 32361dd..2ed1e24 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -958,9 +958,8 @@ static int chcr_ahash_update(struct ahash_request *req)
req_ctx->result = 0;
req_ctx->data_len += params.sg_len + params.bfr_len;
skb = create_hash_wr(req, ¶ms);
-
- if (IS_ERR(skb))
- return PTR_ERR(skb);
+ if (!skb)
+ return -ENOMEM;
if (remainder) {
u8 *temp;
@@ -1023,8 +1022,8 @@ static int chcr_ahash_final(struct ahash_request *req)
params.more = 0;
}
skb = create_hash_wr(req, ¶ms);
- if (IS_ERR(skb))
- return PTR_ERR(skb);
+ if (!skb)
+ return -ENOMEM;
skb->dev = u_ctx->lldi.ports[0];
set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_channel_id);
@@ -1074,8 +1073,8 @@ static int chcr_ahash_finup(struct ahash_request *req)
}
skb = create_hash_wr(req, ¶ms);
- if (IS_ERR(skb))
- return PTR_ERR(skb);
+ if (!skb)
+ return -ENOMEM;
skb->dev = u_ctx->lldi.ports[0];
set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_channel_id);
@@ -1125,8 +1124,8 @@ static int chcr_ahash_digest(struct ahash_request *req)
}
skb = create_hash_wr(req, ¶ms);
- if (IS_ERR(skb))
- return PTR_ERR(skb);
+ if (!skb)
+ return -ENOMEM;
skb->dev = u_ctx->lldi.ports[0];
set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_channel_id);
^ permalink raw reply related
* [bug report] crypto: chcr - Add AEAD algos.
From: Dan Carpenter @ 2016-12-01 20:48 UTC (permalink / raw)
To: harsh; +Cc: linux-crypto
Hello Harsh Jain,
This is a semi-automatic email about new static checker warnings.
The patch 2debd3325e55: "crypto: chcr - Add AEAD algos." from Nov 29,
2016, leads to the following Smatch complaint:
drivers/crypto/chelsio/chcr_algo.c:2460 chcr_aead_op()
warn: variable dereferenced before check 'ctx' (see line 2457)
drivers/crypto/chelsio/chcr_algo.c:2460 chcr_aead_op()
warn: variable dereferenced before check 'ctx->dev' (see line 2457)
drivers/crypto/chelsio/chcr_algo.c
2456 struct chcr_context *ctx = crypto_aead_ctx(tfm);
2457 struct uld_ctx *u_ctx = ULD_CTX(ctx);
^^^^^^^^^^^^
Derefences both.
2458 struct sk_buff *skb;
2459
2460 if (ctx && !ctx->dev) {
^^^^^^^^^^^^^^^^
Check is too late.
2461 pr_err("chcr : %s : No crypto device.\n", __func__);
2462 return -ENXIO;
regards,
dan carpenter
^ permalink raw reply
* Re: [PATCH] crypto: marvell - Don't copy hash operation twice into the SRAM
From: Gregory CLEMENT @ 2016-12-01 16:27 UTC (permalink / raw)
To: Romain Perier
Cc: Boris Brezillon, Arnaud Ebalard, linux-crypto, Jason Cooper,
Andrew Lunn, Sebastian Hesselbarth, Thomas Petazzoni
In-Reply-To: <20161201162157.14563-1-romain.perier@free-electrons.com>
Hi Romain,
On jeu., déc. 01 2016, Romain Perier <romain.perier@free-electrons.com> wrote:
> No need to copy the template of an hash operation twice into the SRAM
> from the step function.
>
Does this patch fix a bug ot it is jsute a cleanup/improvement?
If it is a bug you should CC stable and add use the Fixes tag.
Gregory
> Signed-off-by: Romain Perier <romain.perier@free-electrons.com>
> ---
> drivers/crypto/marvell/hash.c | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/drivers/crypto/marvell/hash.c b/drivers/crypto/marvell/hash.c
> index 2a92605..fbbcbf8 100644
> --- a/drivers/crypto/marvell/hash.c
> +++ b/drivers/crypto/marvell/hash.c
> @@ -172,9 +172,6 @@ static void mv_cesa_ahash_std_step(struct ahash_request *req)
> for (i = 0; i < digsize / 4; i++)
> writel_relaxed(creq->state[i], engine->regs + CESA_IVDIG(i));
>
> - mv_cesa_adjust_op(engine, &creq->op_tmpl);
> - memcpy_toio(engine->sram, &creq->op_tmpl, sizeof(creq->op_tmpl));
> -
> if (creq->cache_ptr)
> memcpy_toio(engine->sram + CESA_SA_DATA_SRAM_OFFSET,
> creq->cache, creq->cache_ptr);
> --
> 2.9.3
>
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply
* [PATCH] crypto: marvell - Don't copy hash operation twice into the SRAM
From: Romain Perier @ 2016-12-01 16:21 UTC (permalink / raw)
To: Boris Brezillon, Arnaud Ebalard
Cc: linux-crypto, Jason Cooper, Andrew Lunn, Sebastian Hesselbarth,
Gregory Clement, Thomas Petazzoni
No need to copy the template of an hash operation twice into the SRAM
from the step function.
Signed-off-by: Romain Perier <romain.perier@free-electrons.com>
---
drivers/crypto/marvell/hash.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/drivers/crypto/marvell/hash.c b/drivers/crypto/marvell/hash.c
index 2a92605..fbbcbf8 100644
--- a/drivers/crypto/marvell/hash.c
+++ b/drivers/crypto/marvell/hash.c
@@ -172,9 +172,6 @@ static void mv_cesa_ahash_std_step(struct ahash_request *req)
for (i = 0; i < digsize / 4; i++)
writel_relaxed(creq->state[i], engine->regs + CESA_IVDIG(i));
- mv_cesa_adjust_op(engine, &creq->op_tmpl);
- memcpy_toio(engine->sram, &creq->op_tmpl, sizeof(creq->op_tmpl));
-
if (creq->cache_ptr)
memcpy_toio(engine->sram + CESA_SA_DATA_SRAM_OFFSET,
creq->cache, creq->cache_ptr);
--
2.9.3
^ permalink raw reply related
* Re: [PATCH] crypto: arm/aesbs - Select SIMD in Kconfig
From: Arnd Bergmann @ 2016-12-01 13:43 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Russell King, linux-crypto, linux-arm-kernel,
linux-kernel
In-Reply-To: <20161201125616.GB2249@gondor.apana.org.au>
On Thursday, December 1, 2016 8:56:16 PM CET Herbert Xu wrote:
> On Thu, Dec 01, 2016 at 12:47:59AM +0100, Arnd Bergmann wrote:
> > Commit 585b5fa63da9 ("crypto: arm/aes - Select SIMD in Kconfig") added
> > the dependency for CRYPTO_AES_ARM_CE, but missed the same change
> > for CRYPTO_AES_ARM_BS:
> >
> > arch/arm/crypto/aes-arm-bs.o: In function `aesbs_mod_init':
> > aesbs-glue.c:(.init.text+0x38): undefined reference to `simd_skcipher_create_compat'
> >
> > Fixes: 211f41af534a ("crypto: aesbs - Convert to skcipher")
> > Signed-off-by: Arnd Bergmann <arnd@arndb.de>
>
> Thanks Arnd. This should already be fixed by 6fdf436fd854.
Right. I have discarded my patch on today's linux-next and I
get no more link errors.
Arnd
^ permalink raw reply
* Re: [PATCH v2] crypto: AF_ALG - fix AEAD AIO handling of zero buffer
From: Herbert Xu @ 2016-12-01 13:22 UTC (permalink / raw)
To: Stephan Mueller; +Cc: linux-crypto
In-Reply-To: <3547426.Fep2bfj8cH@positron.chronox.de>
On Thu, Dec 01, 2016 at 08:22:37AM +0100, Stephan Mueller wrote:
> Hi Herbert,
>
> I split out the bug fix patch from the AD/tag formatting patch as they most likely will come after the next merge window.
>
> ---8<---
>
> Handle the case when the caller provided a zero buffer to
> sendmsg/sendpage. Such scenario is legal for AEAD ciphers when no
> plaintext / ciphertext and no AAD is provided and the caller only
> requests the generation of the tag value.
>
> Signed-off-by: Stephan Mueller <smueller@chronox.de>
Patch applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 1/4] crypto: api - fix comment typo
From: Herbert Xu @ 2016-12-01 13:21 UTC (permalink / raw)
To: Baruch Siach; +Cc: David S. Miller, linux-crypto
In-Reply-To: <900b42a57c35095988dcad0c1eb366da0fa71c05.1480511771.git.baruch@tkos.co.il>
On Wed, Nov 30, 2016 at 03:16:08PM +0200, Baruch Siach wrote:
> Signed-off-by: Baruch Siach <baruch@tkos.co.il>
All applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 1/1] crypto: set error code when kcalloc fails
From: Herbert Xu @ 2016-12-01 13:21 UTC (permalink / raw)
To: Pan Bian; +Cc: David S. Miller, linux-crypto, linux-kernel
In-Reply-To: <1480557883-18844-1-git-send-email-bianpan2016@163.com>
On Thu, Dec 01, 2016 at 10:04:43AM +0800, Pan Bian wrote:
> Fix bug https://bugzilla.kernel.org/show_bug.cgi?id=188521. In function
> skcipher_recvmsg_async(), variable err takes the return value, and its
> value should be negative on failures. Because variable err may be
> reassigned and checked before calling kcalloc(), its value may be 0
> (indicates no error) even if kcalloc() fails. This patch fixes the bug
> by explicitly assigning -ENOMEM to err when kcalloc() returns a NULL
> pointer.
>
> Signed-off-by: Pan Bian <bianpan2016@163.com>
Patch applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 1/2] crypto: caam: pass key buffers with typesafe pointers
From: Herbert Xu @ 2016-12-01 13:21 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Horia Geantă, Dan Douglass, David S. Miller, Catalin Vasile,
Tudor Ambarus, linux-crypto, linux-kernel
In-Reply-To: <20161130210234.1666417-1-arnd@arndb.de>
On Wed, Nov 30, 2016 at 10:01:59PM +0100, Arnd Bergmann wrote:
> The 'key' field is defined as a 'u64' and used for two different
> pieces of information: either to store a pointer or a dma_addr_t.
> The former leads to a build error on 32-bit machines:
>
> drivers/crypto/caam/caamalg_desc.c: In function 'cnstr_shdsc_aead_null_encap':
> drivers/crypto/caam/caamalg_desc.c:67:27: error: cast to pointer from integer of different size [-Werror=int-to-pointer-cast]
> drivers/crypto/caam/caamalg_desc.c: In function 'cnstr_shdsc_aead_null_decap':
> drivers/crypto/caam/caamalg_desc.c:143:27: error: cast to pointer from integer of different size [-Werror=int-to-pointer-cast]
>
> Using a union to provide correct types gets rid of the warnings
> and as well as a couple of redundant casts.
>
> Fixes: db57656b0072 ("crypto: caam - group algorithm related params")
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Both patches applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH] crypto: arm/aesbs - Select SIMD in Kconfig
From: Herbert Xu @ 2016-12-01 12:56 UTC (permalink / raw)
To: Arnd Bergmann
Cc: David S. Miller, Russell King, linux-crypto, linux-arm-kernel,
linux-kernel
In-Reply-To: <20161130234826.854879-1-arnd@arndb.de>
On Thu, Dec 01, 2016 at 12:47:59AM +0100, Arnd Bergmann wrote:
> Commit 585b5fa63da9 ("crypto: arm/aes - Select SIMD in Kconfig") added
> the dependency for CRYPTO_AES_ARM_CE, but missed the same change
> for CRYPTO_AES_ARM_BS:
>
> arch/arm/crypto/aes-arm-bs.o: In function `aesbs_mod_init':
> aesbs-glue.c:(.init.text+0x38): undefined reference to `simd_skcipher_create_compat'
>
> Fixes: 211f41af534a ("crypto: aesbs - Convert to skcipher")
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Thanks Arnd. This should already be fixed by 6fdf436fd854.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH] crypto: caam - fix key pointer size warning
From: Herbert Xu @ 2016-12-01 12:53 UTC (permalink / raw)
To: Horia Geantă; +Cc: David S. Miller, linux-crypto, Dan Douglass
In-Reply-To: <1480407769-15336-1-git-send-email-horia.geanta@nxp.com>
On Tue, Nov 29, 2016 at 10:22:49AM +0200, Horia Geantă wrote:
> When building on 32-bit, compiler issues [-Wint-to-pointer-cast] warnings:
>
> drivers/crypto/caam/caamalg_desc.c: In function 'cnstr_shdsc_aead_null_encap':
> drivers/crypto/caam/caamalg_desc.c:67:27: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
> append_key_as_imm(desc, (void *)adata->key, adata->keylen_pad,
> [...]
>
> Since in all these cases the u64 "key" member of alginfo struct holds
> a uintptr_t value, we can safely remove the warnings by explicitly casting
> to uinptr_t (before any other existing cast).
>
> Fixes: 8cea7b66b821 ("crypto: caam - refactor encryption descriptors generation")
> Reported-by: kbuild test robot <fengguang.wu@intel.com>
> Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
Thanks for the patch. I think Arnd's version is cleaner though
so I'll take that instead.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* [PATCH v5 1/1] crypto: add virtio-crypto driver
From: Gonglei @ 2016-12-01 12:39 UTC (permalink / raw)
To: linux-kernel, qemu-devel, virtio-dev, virtualization,
linux-crypto
Cc: luonengjun, mst, stefanha, weidong.huang, wu.wubin, xin.zeng,
claudio.fontana, herbert, pasic, davem, jianjay.zhou, hanweidong,
arei.gonglei, cornelia.huck, xuquan8, longpeng2, wanzongshun,
Gonglei
In-Reply-To: <1480595945-63656-1-git-send-email-arei.gonglei@huawei.com>
This patch introduces virtio-crypto driver for Linux Kernel.
The virtio crypto device is a virtual cryptography device
as well as a kind of virtual hardware accelerator for
virtual machines. The encryption anddecryption requests
are placed in the data queue and are ultimately handled by
thebackend crypto accelerators. The second queue is the
control queue used to create or destroy sessions for
symmetric algorithms and will control some advanced features
in the future. The virtio crypto device provides the following
cryptoservices: CIPHER, MAC, HASH, and AEAD.
For more information about virtio-crypto device, please see:
http://qemu-project.org/Features/VirtioCrypto
CC: Michael S. Tsirkin <mst@redhat.com>
CC: Cornelia Huck <cornelia.huck@de.ibm.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Herbert Xu <herbert@gondor.apana.org.au>
CC: Halil Pasic <pasic@linux.vnet.ibm.com>
CC: David S. Miller <davem@davemloft.net>
CC: Zeng Xin <xin.zeng@intel.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
---
MAINTAINERS | 9 +
drivers/crypto/Kconfig | 2 +
drivers/crypto/Makefile | 1 +
drivers/crypto/virtio/Kconfig | 10 +
drivers/crypto/virtio/Makefile | 5 +
drivers/crypto/virtio/virtio_crypto_algs.c | 537 +++++++++++++++++++++++++++
drivers/crypto/virtio/virtio_crypto_common.h | 122 ++++++
drivers/crypto/virtio/virtio_crypto_core.c | 464 +++++++++++++++++++++++
drivers/crypto/virtio/virtio_crypto_mgr.c | 264 +++++++++++++
include/uapi/linux/Kbuild | 1 +
include/uapi/linux/virtio_crypto.h | 450 ++++++++++++++++++++++
include/uapi/linux/virtio_ids.h | 1 +
12 files changed, 1866 insertions(+)
create mode 100644 drivers/crypto/virtio/Kconfig
create mode 100644 drivers/crypto/virtio/Makefile
create mode 100644 drivers/crypto/virtio/virtio_crypto_algs.c
create mode 100644 drivers/crypto/virtio/virtio_crypto_common.h
create mode 100644 drivers/crypto/virtio/virtio_crypto_core.c
create mode 100644 drivers/crypto/virtio/virtio_crypto_mgr.c
create mode 100644 include/uapi/linux/virtio_crypto.h
diff --git a/MAINTAINERS b/MAINTAINERS
index ad9b965..cccaaf0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12810,6 +12810,7 @@ F: drivers/net/virtio_net.c
F: drivers/block/virtio_blk.c
F: include/linux/virtio_*.h
F: include/uapi/linux/virtio_*.h
+F: drivers/crypto/virtio/
VIRTIO DRIVERS FOR S390
M: Christian Borntraeger <borntraeger@de.ibm.com>
@@ -12846,6 +12847,14 @@ S: Maintained
F: drivers/virtio/virtio_input.c
F: include/uapi/linux/virtio_input.h
+VIRTIO CRYPTO DRIVER
+M: Gonglei <arei.gonglei@huawei.com>
+L: virtualization@lists.linux-foundation.org
+L: linux-crypto@vger.kernel.org
+S: Maintained
+F: drivers/crypto/virtio/
+F: include/uapi/linux/virtio_crypto.h
+
VIA RHINE NETWORK DRIVER
S: Orphan
F: drivers/net/ethernet/via/via-rhine.c
diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index 4d2b81f..7956478 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig
@@ -555,4 +555,6 @@ config CRYPTO_DEV_ROCKCHIP
source "drivers/crypto/chelsio/Kconfig"
+source "drivers/crypto/virtio/Kconfig"
+
endif # CRYPTO_HW
diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile
index ad7250f..bc53cb8 100644
--- a/drivers/crypto/Makefile
+++ b/drivers/crypto/Makefile
@@ -32,3 +32,4 @@ obj-$(CONFIG_CRYPTO_DEV_VMX) += vmx/
obj-$(CONFIG_CRYPTO_DEV_SUN4I_SS) += sunxi-ss/
obj-$(CONFIG_CRYPTO_DEV_ROCKCHIP) += rockchip/
obj-$(CONFIG_CRYPTO_DEV_CHELSIO) += chelsio/
+obj-$(CONFIG_CRYPTO_DEV_VIRTIO) += virtio/
diff --git a/drivers/crypto/virtio/Kconfig b/drivers/crypto/virtio/Kconfig
new file mode 100644
index 0000000..d80f733
--- /dev/null
+++ b/drivers/crypto/virtio/Kconfig
@@ -0,0 +1,10 @@
+config CRYPTO_DEV_VIRTIO
+ tristate "VirtIO crypto driver"
+ depends on VIRTIO
+ select CRYPTO_AEAD
+ select CRYPTO_AUTHENC
+ select CRYPTO_BLKCIPHER
+ default m
+ help
+ This driver provides support for virtio crypto device. If you
+ choose 'M' here, this module will be called virtio_crypto.
diff --git a/drivers/crypto/virtio/Makefile b/drivers/crypto/virtio/Makefile
new file mode 100644
index 0000000..dd342c9
--- /dev/null
+++ b/drivers/crypto/virtio/Makefile
@@ -0,0 +1,5 @@
+obj-$(CONFIG_CRYPTO_DEV_VIRTIO) += virtio_crypto.o
+virtio_crypto-objs := \
+ virtio_crypto_algs.o \
+ virtio_crypto_mgr.o \
+ virtio_crypto_core.o
diff --git a/drivers/crypto/virtio/virtio_crypto_algs.c b/drivers/crypto/virtio/virtio_crypto_algs.c
new file mode 100644
index 0000000..5100056
--- /dev/null
+++ b/drivers/crypto/virtio/virtio_crypto_algs.c
@@ -0,0 +1,537 @@
+ /* Algorithms supported by virtio crypto device
+ *
+ * Authors: Gonglei <arei.gonglei@huawei.com>
+ *
+ * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/scatterlist.h>
+#include <crypto/algapi.h>
+#include <linux/err.h>
+#include <crypto/scatterwalk.h>
+#include <linux/atomic.h>
+
+#include <uapi/linux/virtio_crypto.h>
+#include "virtio_crypto_common.h"
+
+/*
+ * The algs_lock protects the below global virtio_crypto_active_devs
+ * and crypto algorithms registion.
+ */
+static DEFINE_MUTEX(algs_lock);
+static unsigned int virtio_crypto_active_devs;
+
+static u64 virtio_crypto_alg_sg_nents_length(struct scatterlist *sg)
+{
+ u64 total = 0;
+
+ for (total = 0; sg; sg = sg_next(sg))
+ total += sg->length;
+
+ return total;
+}
+
+static int
+virtio_crypto_alg_validate_key(int key_len, uint32_t *alg)
+{
+ switch (key_len) {
+ case AES_KEYSIZE_128:
+ case AES_KEYSIZE_192:
+ case AES_KEYSIZE_256:
+ *alg = VIRTIO_CRYPTO_CIPHER_AES_CBC;
+ break;
+ default:
+ pr_err("virtio_crypto: Unsupported key length: %d\n",
+ key_len);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static int virtio_crypto_alg_ablkcipher_init_session(
+ struct virtio_crypto_ablkcipher_ctx *ctx,
+ uint32_t alg, const uint8_t *key,
+ unsigned int keylen,
+ int encrypt)
+{
+ struct scatterlist outhdr, key_sg, inhdr, *sgs[3];
+ unsigned int tmp;
+ struct virtio_crypto *vcrypto = ctx->vcrypto;
+ int op = encrypt ? VIRTIO_CRYPTO_OP_ENCRYPT : VIRTIO_CRYPTO_OP_DECRYPT;
+ int err;
+ unsigned int num_out = 0, num_in = 0;
+
+ /*
+ * Avoid to do DMA from the stack, switch to using
+ * dynamically-allocated for the key
+ */
+ uint8_t *cipher_key = kmalloc(keylen, GFP_ATOMIC);
+
+ if (!cipher_key)
+ return -ENOMEM;
+
+ memcpy(cipher_key, key, keylen);
+
+ spin_lock(&vcrypto->ctrl_lock);
+ /* Pad ctrl header */
+ vcrypto->ctrl.header.opcode =
+ cpu_to_le32(VIRTIO_CRYPTO_CIPHER_CREATE_SESSION);
+ vcrypto->ctrl.header.algo = cpu_to_le32(alg);
+ /* Set the default dataqueue id to 0 */
+ vcrypto->ctrl.header.queue_id = 0;
+
+ vcrypto->input.status = cpu_to_le32(VIRTIO_CRYPTO_ERR);
+ /* Pad cipher's parameters */
+ vcrypto->ctrl.u.sym_create_session.op_type =
+ cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER);
+ vcrypto->ctrl.u.sym_create_session.u.cipher.para.algo =
+ vcrypto->ctrl.header.algo;
+ vcrypto->ctrl.u.sym_create_session.u.cipher.para.keylen =
+ cpu_to_le32(keylen);
+ vcrypto->ctrl.u.sym_create_session.u.cipher.para.op =
+ cpu_to_le32(op);
+
+ sg_init_one(&outhdr, &vcrypto->ctrl, sizeof(vcrypto->ctrl));
+ sgs[num_out++] = &outhdr;
+
+ /* Set key */
+ sg_init_one(&key_sg, cipher_key, keylen);
+ sgs[num_out++] = &key_sg;
+
+ /* Return status and session id back */
+ sg_init_one(&inhdr, &vcrypto->input, sizeof(vcrypto->input));
+ sgs[num_out + num_in++] = &inhdr;
+
+ err = virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out,
+ num_in, vcrypto, GFP_ATOMIC);
+ if (err < 0) {
+ spin_unlock(&vcrypto->ctrl_lock);
+ kzfree(cipher_key);
+ return err;
+ }
+ virtqueue_kick(vcrypto->ctrl_vq);
+
+ /*
+ * Trapping into the hypervisor, so the request should be
+ * handled immediately.
+ */
+ while (!virtqueue_get_buf(vcrypto->ctrl_vq, &tmp) &&
+ !virtqueue_is_broken(vcrypto->ctrl_vq))
+ cpu_relax();
+
+ if (le32_to_cpu(vcrypto->input.status) != VIRTIO_CRYPTO_OK) {
+ spin_unlock(&vcrypto->ctrl_lock);
+ pr_err("virtio_crypto: Create session failed status: %u\n",
+ le32_to_cpu(vcrypto->input.status));
+ kzfree(cipher_key);
+ return -EINVAL;
+ }
+
+ if (encrypt)
+ ctx->enc_sess_info.session_id =
+ le64_to_cpu(vcrypto->input.session_id);
+ else
+ ctx->dec_sess_info.session_id =
+ le64_to_cpu(vcrypto->input.session_id);
+
+ spin_unlock(&vcrypto->ctrl_lock);
+
+ kzfree(cipher_key);
+ return 0;
+}
+
+static int virtio_crypto_alg_ablkcipher_close_session(
+ struct virtio_crypto_ablkcipher_ctx *ctx,
+ int encrypt)
+{
+ struct scatterlist outhdr, status_sg, *sgs[2];
+ unsigned int tmp;
+ struct virtio_crypto_destroy_session_req *destroy_session;
+ struct virtio_crypto *vcrypto = ctx->vcrypto;
+ int err;
+ unsigned int num_out = 0, num_in = 0;
+
+ spin_lock(&vcrypto->ctrl_lock);
+ vcrypto->ctrl_status.status = VIRTIO_CRYPTO_ERR;
+ /* Pad ctrl header */
+ vcrypto->ctrl.header.opcode =
+ cpu_to_le32(VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION);
+ /* Set the default virtqueue id to 0 */
+ vcrypto->ctrl.header.queue_id = 0;
+
+ destroy_session = &vcrypto->ctrl.u.destroy_session;
+
+ if (encrypt)
+ destroy_session->session_id =
+ cpu_to_le64(ctx->enc_sess_info.session_id);
+ else
+ destroy_session->session_id =
+ cpu_to_le64(ctx->dec_sess_info.session_id);
+
+ sg_init_one(&outhdr, &vcrypto->ctrl, sizeof(vcrypto->ctrl));
+ sgs[num_out++] = &outhdr;
+
+ /* Return status and session id back */
+ sg_init_one(&status_sg, &vcrypto->ctrl_status.status,
+ sizeof(vcrypto->ctrl_status.status));
+ sgs[num_out + num_in++] = &status_sg;
+
+ err = virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out,
+ num_in, vcrypto, GFP_ATOMIC);
+ if (err < 0) {
+ spin_unlock(&vcrypto->ctrl_lock);
+ return err;
+ }
+ virtqueue_kick(vcrypto->ctrl_vq);
+
+ while (!virtqueue_get_buf(vcrypto->ctrl_vq, &tmp) &&
+ !virtqueue_is_broken(vcrypto->ctrl_vq))
+ cpu_relax();
+
+ if (vcrypto->ctrl_status.status != VIRTIO_CRYPTO_OK) {
+ spin_unlock(&vcrypto->ctrl_lock);
+ pr_err("virtio_crypto: Close session failed status: %u, session_id: 0x%llx\n",
+ vcrypto->ctrl_status.status,
+ destroy_session->session_id);
+
+ return -EINVAL;
+ }
+ spin_unlock(&vcrypto->ctrl_lock);
+
+ return 0;
+}
+
+static int virtio_crypto_alg_ablkcipher_init_sessions(
+ struct virtio_crypto_ablkcipher_ctx *ctx,
+ const uint8_t *key, unsigned int keylen)
+{
+ uint32_t alg;
+ int ret;
+ struct virtio_crypto *vcrypto = ctx->vcrypto;
+
+ if (keylen > vcrypto->max_cipher_key_len) {
+ pr_err("virtio_crypto: the key is too long\n");
+ goto bad_key;
+ }
+
+ if (virtio_crypto_alg_validate_key(keylen, &alg))
+ goto bad_key;
+
+ /* Create encryption session */
+ ret = virtio_crypto_alg_ablkcipher_init_session(ctx,
+ alg, key, keylen, 1);
+ if (ret)
+ return ret;
+ /* Create decryption session */
+ ret = virtio_crypto_alg_ablkcipher_init_session(ctx,
+ alg, key, keylen, 0);
+ if (ret) {
+ virtio_crypto_alg_ablkcipher_close_session(ctx, 1);
+ return ret;
+ }
+ return 0;
+
+bad_key:
+ crypto_tfm_set_flags(ctx->tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
+ return -EINVAL;
+}
+
+/* Note: kernel crypto API realization */
+static int virtio_crypto_ablkcipher_setkey(struct crypto_ablkcipher *tfm,
+ const uint8_t *key,
+ unsigned int keylen)
+{
+ struct virtio_crypto_ablkcipher_ctx *ctx = crypto_ablkcipher_ctx(tfm);
+ int ret;
+
+ if (!ctx->vcrypto) {
+ /* New key */
+ int node = virtio_crypto_get_current_node();
+ struct virtio_crypto *vcrypto =
+ virtcrypto_get_dev_node(node);
+ if (!vcrypto) {
+ pr_err("virtio_crypto: Could not find a virtio device in the system");
+ return -ENODEV;
+ }
+
+ ctx->vcrypto = vcrypto;
+ }
+
+ ret = virtio_crypto_alg_ablkcipher_init_sessions(ctx, key, keylen);
+ if (ret) {
+ virtcrypto_dev_put(ctx->vcrypto);
+ ctx->vcrypto = NULL;
+
+ return ret;
+ }
+
+ return 0;
+}
+
+static int
+__virtio_crypto_ablkcipher_do_req(struct virtio_crypto_request *vc_req,
+ struct ablkcipher_request *req,
+ struct data_queue *data_vq,
+ __u8 op)
+{
+ struct crypto_ablkcipher *tfm = crypto_ablkcipher_reqtfm(req);
+ unsigned int ivsize = crypto_ablkcipher_ivsize(tfm);
+ struct virtio_crypto_ablkcipher_ctx *ctx = vc_req->ablkcipher_ctx;
+ struct virtio_crypto *vcrypto = ctx->vcrypto;
+ struct virtio_crypto_op_data_req *req_data;
+ int src_nents, dst_nents;
+ int err;
+ unsigned long flags;
+ struct scatterlist outhdr, iv_sg, status_sg, **sgs;
+ int i;
+ u64 dst_len;
+ unsigned int num_out = 0, num_in = 0;
+ int sg_total;
+ uint8_t *iv;
+
+ src_nents = sg_nents_for_len(req->src, req->nbytes);
+ dst_nents = sg_nents(req->dst);
+
+ pr_debug("virtio_crypto: Number of sgs (src_nents: %d, dst_nents: %d)\n",
+ src_nents, dst_nents);
+
+ /* Why 3? outhdr + iv + inhdr */
+ sg_total = src_nents + dst_nents + 3;
+ sgs = kzalloc_node(sg_total * sizeof(*sgs), GFP_ATOMIC,
+ dev_to_node(&vcrypto->vdev->dev));
+ if (!sgs)
+ return -ENOMEM;
+
+ req_data = kzalloc_node(sizeof(*req_data), GFP_ATOMIC,
+ dev_to_node(&vcrypto->vdev->dev));
+ if (!req_data) {
+ kfree(sgs);
+ return -ENOMEM;
+ }
+
+ vc_req->req_data = req_data;
+ vc_req->type = VIRTIO_CRYPTO_SYM_OP_CIPHER;
+ /* Head of operation */
+ if (op) {
+ req_data->header.session_id =
+ cpu_to_le64(ctx->enc_sess_info.session_id);
+ req_data->header.opcode =
+ cpu_to_le32(VIRTIO_CRYPTO_CIPHER_ENCRYPT);
+ } else {
+ req_data->header.session_id =
+ cpu_to_le64(ctx->dec_sess_info.session_id);
+ req_data->header.opcode =
+ cpu_to_le32(VIRTIO_CRYPTO_CIPHER_DECRYPT);
+ }
+ req_data->u.sym_req.op_type = cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER);
+ req_data->u.sym_req.u.cipher.para.iv_len = cpu_to_le32(ivsize);
+ req_data->u.sym_req.u.cipher.para.src_data_len =
+ cpu_to_le32(req->nbytes);
+
+ dst_len = virtio_crypto_alg_sg_nents_length(req->dst);
+ if (unlikely(dst_len > U32_MAX)) {
+ pr_err("virtio_crypto: The dst_len is beyond U32_MAX\n");
+ err = -EINVAL;
+ goto free;
+ }
+
+ pr_debug("virtio_crypto: src_len: %u, dst_len: %llu\n",
+ req->nbytes, dst_len);
+
+ if (unlikely(req->nbytes + dst_len + ivsize +
+ sizeof(vc_req->status) > vcrypto->max_size)) {
+ pr_err("virtio_crypto: The length is too big\n");
+ err = -EINVAL;
+ goto free;
+ }
+
+ req_data->u.sym_req.u.cipher.para.dst_data_len =
+ cpu_to_le32((uint32_t)dst_len);
+
+ /* Outhdr */
+ sg_init_one(&outhdr, req_data, sizeof(*req_data));
+ sgs[num_out++] = &outhdr;
+
+ /* IV */
+
+ /*
+ * Avoid to do DMA from the stack, switch to using
+ * dynamically-allocated for the IV
+ */
+ iv = kzalloc_node(ivsize, GFP_ATOMIC,
+ dev_to_node(&vcrypto->vdev->dev));
+ if (!iv) {
+ err = -ENOMEM;
+ goto free;
+ }
+ memcpy(iv, req->info, ivsize);
+ sg_init_one(&iv_sg, iv, ivsize);
+ sgs[num_out++] = &iv_sg;
+ vc_req->iv = iv;
+
+ /* Source data */
+ for (i = 0; i < src_nents; i++)
+ sgs[num_out++] = &req->src[i];
+
+ /* Destination data */
+ for (i = 0; i < dst_nents; i++)
+ sgs[num_out + num_in++] = &req->dst[i];
+
+ /* Status */
+ sg_init_one(&status_sg, &vc_req->status, sizeof(vc_req->status));
+ sgs[num_out + num_in++] = &status_sg;
+
+ vc_req->sgs = sgs;
+
+ spin_lock_irqsave(&vcrypto->lock, flags);
+ err = virtqueue_add_sgs(data_vq->vq, sgs, num_out,
+ num_in, vc_req, GFP_ATOMIC);
+ spin_unlock_irqrestore(&vcrypto->lock, flags);
+ if (unlikely(err < 0))
+ goto free_iv;
+
+ return 0;
+
+free_iv:
+ kzfree(iv);
+free:
+ kzfree(req_data);
+ kfree(sgs);
+ return err;
+}
+
+static int virtio_crypto_ablkcipher_encrypt(struct ablkcipher_request *req)
+{
+ struct crypto_ablkcipher *atfm = crypto_ablkcipher_reqtfm(req);
+ struct virtio_crypto_ablkcipher_ctx *ctx = crypto_ablkcipher_ctx(atfm);
+ struct virtio_crypto_request *vc_req = ablkcipher_request_ctx(req);
+ struct virtio_crypto *vcrypto = ctx->vcrypto;
+ int ret;
+ /* Use the first data virtqueue as default */
+ struct data_queue *data_vq = &vcrypto->data_vq[0];
+
+ vc_req->ablkcipher_ctx = ctx;
+ vc_req->ablkcipher_req = req;
+ ret = __virtio_crypto_ablkcipher_do_req(vc_req, req, data_vq, 1);
+ if (ret < 0) {
+ pr_err("virtio_crypto: Encryption failed!\n");
+ return ret;
+ }
+ virtqueue_kick(data_vq->vq);
+
+ return -EINPROGRESS;
+}
+
+static int virtio_crypto_ablkcipher_decrypt(struct ablkcipher_request *req)
+{
+ struct crypto_ablkcipher *atfm = crypto_ablkcipher_reqtfm(req);
+ struct virtio_crypto_ablkcipher_ctx *ctx = crypto_ablkcipher_ctx(atfm);
+ struct virtio_crypto_request *vc_req = ablkcipher_request_ctx(req);
+ struct virtio_crypto *vcrypto = ctx->vcrypto;
+ int ret;
+ /* Use the first data virtqueue as default */
+ struct data_queue *data_vq = &vcrypto->data_vq[0];
+
+ vc_req->ablkcipher_ctx = ctx;
+ vc_req->ablkcipher_req = req;
+
+ ret = __virtio_crypto_ablkcipher_do_req(vc_req, req, data_vq, 0);
+ if (ret < 0) {
+ pr_err("virtio_crypto: Decryption failed!\n");
+ return ret;
+ }
+ virtqueue_kick(data_vq->vq);
+
+ return -EINPROGRESS;
+}
+
+static int virtio_crypto_ablkcipher_init(struct crypto_tfm *tfm)
+{
+ struct virtio_crypto_ablkcipher_ctx *ctx = crypto_tfm_ctx(tfm);
+
+ tfm->crt_ablkcipher.reqsize = sizeof(struct virtio_crypto_request);
+ ctx->tfm = tfm;
+
+ return 0;
+}
+
+static void virtio_crypto_ablkcipher_exit(struct crypto_tfm *tfm)
+{
+ struct virtio_crypto_ablkcipher_ctx *ctx = crypto_tfm_ctx(tfm);
+
+ if (!ctx->vcrypto)
+ return;
+
+ virtio_crypto_alg_ablkcipher_close_session(ctx, 1);
+ virtio_crypto_alg_ablkcipher_close_session(ctx, 0);
+ virtcrypto_dev_put(ctx->vcrypto);
+ ctx->vcrypto = NULL;
+}
+
+static struct crypto_alg virtio_crypto_algs[] = { {
+ .cra_name = "cbc(aes)",
+ .cra_driver_name = "virtio_crypto_aes_cbc",
+ .cra_priority = 4001,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER | CRYPTO_ALG_ASYNC,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct virtio_crypto_ablkcipher_ctx),
+ .cra_alignmask = 0,
+ .cra_module = THIS_MODULE,
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_init = virtio_crypto_ablkcipher_init,
+ .cra_exit = virtio_crypto_ablkcipher_exit,
+ .cra_u = {
+ .ablkcipher = {
+ .setkey = virtio_crypto_ablkcipher_setkey,
+ .decrypt = virtio_crypto_ablkcipher_decrypt,
+ .encrypt = virtio_crypto_ablkcipher_encrypt,
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ },
+ },
+} };
+
+int virtio_crypto_algs_register(void)
+{
+ int ret = 0;
+
+ mutex_lock(&algs_lock);
+ if (++virtio_crypto_active_devs != 1)
+ goto unlock;
+
+ ret = crypto_register_algs(virtio_crypto_algs,
+ ARRAY_SIZE(virtio_crypto_algs));
+ if (ret)
+ virtio_crypto_active_devs--;
+
+unlock:
+ mutex_unlock(&algs_lock);
+ return ret;
+}
+
+void virtio_crypto_algs_unregister(void)
+{
+ mutex_lock(&algs_lock);
+ if (--virtio_crypto_active_devs != 0)
+ goto unlock;
+
+ crypto_unregister_algs(virtio_crypto_algs,
+ ARRAY_SIZE(virtio_crypto_algs));
+
+unlock:
+ mutex_unlock(&algs_lock);
+}
diff --git a/drivers/crypto/virtio/virtio_crypto_common.h b/drivers/crypto/virtio/virtio_crypto_common.h
new file mode 100644
index 0000000..975404b
--- /dev/null
+++ b/drivers/crypto/virtio/virtio_crypto_common.h
@@ -0,0 +1,122 @@
+/* Common header for Virtio crypto device.
+ *
+ * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef _VIRTIO_CRYPTO_COMMON_H
+#define _VIRTIO_CRYPTO_COMMON_H
+
+#include <linux/virtio.h>
+#include <linux/crypto.h>
+#include <linux/spinlock.h>
+#include <crypto/aead.h>
+#include <crypto/aes.h>
+#include <crypto/authenc.h>
+
+
+/* Internal representation of a data virtqueue */
+struct data_queue {
+ /* Virtqueue associated with this send _queue */
+ struct virtqueue *vq;
+
+ /* Name of the tx queue: dataq.$index */
+ char name[32];
+};
+
+struct virtio_crypto {
+ struct virtio_device *vdev;
+ struct virtqueue *ctrl_vq;
+ struct data_queue *data_vq;
+
+ /* To protect the vq operations for the dataq */
+ spinlock_t lock;
+
+ /* To protect the vq operations for the controlq */
+ spinlock_t ctrl_lock;
+
+ /* Maximum of data queues supported by the device */
+ u32 max_data_queues;
+
+ /* Number of queue currently used by the driver */
+ u32 curr_queue;
+
+ /* Maximum length of cipher key */
+ u32 max_cipher_key_len;
+ /* Maximum length of authenticated key */
+ u32 max_auth_key_len;
+ /* Maximum size of per request */
+ u64 max_size;
+
+ /* Control VQ buffers: protected by the ctrl_lock */
+ struct virtio_crypto_op_ctrl_req ctrl;
+ struct virtio_crypto_session_input input;
+ struct virtio_crypto_inhdr ctrl_status;
+
+ unsigned long status;
+ atomic_t ref_count;
+ struct list_head list;
+ struct module *owner;
+ uint8_t dev_id;
+
+ /* Does the affinity hint is set for virtqueues? */
+ bool affinity_hint_set;
+};
+
+struct virtio_crypto_sym_session_info {
+ /* Backend session id, which come from the host side */
+ __u64 session_id;
+};
+
+struct virtio_crypto_ablkcipher_ctx {
+ struct virtio_crypto *vcrypto;
+ struct crypto_tfm *tfm;
+
+ struct virtio_crypto_sym_session_info enc_sess_info;
+ struct virtio_crypto_sym_session_info dec_sess_info;
+};
+
+struct virtio_crypto_request {
+ /* Cipher or aead */
+ uint32_t type;
+ uint8_t status;
+ struct virtio_crypto_ablkcipher_ctx *ablkcipher_ctx;
+ struct ablkcipher_request *ablkcipher_req;
+ struct virtio_crypto_op_data_req *req_data;
+ struct scatterlist **sgs;
+ uint8_t *iv;
+};
+
+int virtcrypto_devmgr_add_dev(struct virtio_crypto *vcrypto_dev);
+struct list_head *virtcrypto_devmgr_get_head(void);
+void virtcrypto_devmgr_rm_dev(struct virtio_crypto *vcrypto_dev);
+struct virtio_crypto *virtcrypto_devmgr_get_first(void);
+int virtcrypto_dev_in_use(struct virtio_crypto *vcrypto_dev);
+int virtcrypto_dev_get(struct virtio_crypto *vcrypto_dev);
+void virtcrypto_dev_put(struct virtio_crypto *vcrypto_dev);
+int virtcrypto_dev_started(struct virtio_crypto *vcrypto_dev);
+struct virtio_crypto *virtcrypto_get_dev_node(int node);
+int virtcrypto_dev_start(struct virtio_crypto *vcrypto);
+void virtcrypto_dev_stop(struct virtio_crypto *vcrypto);
+
+static inline int virtio_crypto_get_current_node(void)
+{
+ return topology_physical_package_id(smp_processor_id());
+}
+
+int virtio_crypto_algs_register(void);
+void virtio_crypto_algs_unregister(void);
+
+#endif /* _VIRTIO_CRYPTO_COMMON_H */
diff --git a/drivers/crypto/virtio/virtio_crypto_core.c b/drivers/crypto/virtio/virtio_crypto_core.c
new file mode 100644
index 0000000..286d829
--- /dev/null
+++ b/drivers/crypto/virtio/virtio_crypto_core.c
@@ -0,0 +1,464 @@
+ /* Driver for Virtio crypto device.
+ *
+ * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/virtio_config.h>
+#include <linux/cpu.h>
+
+#include <uapi/linux/virtio_crypto.h>
+#include "virtio_crypto_common.h"
+
+
+static void virtcrypto_dataq_callback(struct virtqueue *vq)
+{
+ struct virtio_crypto *vcrypto = vq->vdev->priv;
+ struct virtio_crypto_request *vc_req;
+ unsigned long flags;
+ unsigned int len;
+ struct ablkcipher_request *ablk_req;
+ int error;
+
+ spin_lock_irqsave(&vcrypto->lock, flags);
+ do {
+ virtqueue_disable_cb(vq);
+ while ((vc_req = virtqueue_get_buf(vq, &len)) != NULL) {
+ if (vc_req->type == VIRTIO_CRYPTO_SYM_OP_CIPHER) {
+ switch (vc_req->status) {
+ case VIRTIO_CRYPTO_OK:
+ error = 0;
+ break;
+ case VIRTIO_CRYPTO_INVSESS:
+ case VIRTIO_CRYPTO_ERR:
+ error = -EINVAL;
+ break;
+ case VIRTIO_CRYPTO_BADMSG:
+ error = -EBADMSG;
+ break;
+ default:
+ error = -EIO;
+ break;
+ }
+ ablk_req = vc_req->ablkcipher_req;
+ /* Finish the encrypt or decrypt process */
+ ablk_req->base.complete(&ablk_req->base, error);
+ }
+
+ kzfree(vc_req->iv);
+ kzfree(vc_req->req_data);
+ kfree(vc_req->sgs);
+ }
+ } while (!virtqueue_enable_cb(vq));
+ spin_unlock_irqrestore(&vcrypto->lock, flags);
+}
+
+static int virtcrypto_find_vqs(struct virtio_crypto *vi)
+{
+ vq_callback_t **callbacks;
+ struct virtqueue **vqs;
+ int ret = -ENOMEM;
+ int i, total_vqs;
+ const char **names;
+
+ /*
+ * We expect 1 data virtqueue, followed by
+ * possible N-1 data queues used in multiqueue mode,
+ * followed by control vq.
+ */
+ total_vqs = vi->max_data_queues + 1;
+
+ /* Allocate space for find_vqs parameters */
+ vqs = kcalloc(total_vqs, sizeof(*vqs), GFP_KERNEL);
+ if (!vqs)
+ goto err_vq;
+ callbacks = kcalloc(total_vqs, sizeof(*callbacks), GFP_KERNEL);
+ if (!callbacks)
+ goto err_callback;
+ names = kcalloc(total_vqs, sizeof(*names), GFP_KERNEL);
+ if (!names)
+ goto err_names;
+
+ /* Parameters for control virtqueue */
+ callbacks[total_vqs - 1] = NULL;
+ names[total_vqs - 1] = "controlq";
+
+ /* Allocate/initialize parameters for data virtqueues */
+ for (i = 0; i < vi->max_data_queues; i++) {
+ callbacks[i] = virtcrypto_dataq_callback;
+ snprintf(vi->data_vq[i].name, sizeof(vi->data_vq[i].name),
+ "dataq.%d", i);
+ names[i] = vi->data_vq[i].name;
+ }
+
+ ret = vi->vdev->config->find_vqs(vi->vdev, total_vqs, vqs, callbacks,
+ names);
+ if (ret)
+ goto err_find;
+
+ vi->ctrl_vq = vqs[total_vqs - 1];
+
+ for (i = 0; i < vi->max_data_queues; i++)
+ vi->data_vq[i].vq = vqs[i];
+
+ kfree(names);
+ kfree(callbacks);
+ kfree(vqs);
+
+ return 0;
+
+err_find:
+ kfree(names);
+err_names:
+ kfree(callbacks);
+err_callback:
+ kfree(vqs);
+err_vq:
+ return ret;
+}
+
+static int virtcrypto_alloc_queues(struct virtio_crypto *vi)
+{
+ vi->data_vq = kcalloc(vi->max_data_queues, sizeof(*vi->data_vq),
+ GFP_KERNEL);
+ if (!vi->data_vq)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void virtcrypto_clean_affinity(struct virtio_crypto *vi, long hcpu)
+{
+ int i;
+
+ if (vi->affinity_hint_set) {
+ for (i = 0; i < vi->max_data_queues; i++)
+ virtqueue_set_affinity(vi->data_vq[i].vq, -1);
+
+ vi->affinity_hint_set = false;
+ }
+}
+
+static void virtcrypto_set_affinity(struct virtio_crypto *vcrypto)
+{
+ int i = 0;
+ int cpu;
+
+ /*
+ * In single queue mode, we don't set the cpu affinity.
+ */
+ if (vcrypto->curr_queue == 1 || vcrypto->max_data_queues == 1) {
+ virtcrypto_clean_affinity(vcrypto, -1);
+ return;
+ }
+
+ /*
+ * In multiqueue mode, we let the queue to be private to one cpu
+ * by setting the affinity hint to eliminate the contention.
+ *
+ * TODO: adds cpu hotplug support by register cpu notifier.
+ *
+ */
+ for_each_online_cpu(cpu) {
+ virtqueue_set_affinity(vcrypto->data_vq[i].vq, cpu);
+ if (++i >= vcrypto->max_data_queues)
+ break;
+ }
+
+ vcrypto->affinity_hint_set = true;
+}
+
+static void virtcrypto_free_queues(struct virtio_crypto *vi)
+{
+ kfree(vi->data_vq);
+}
+
+static int virtcrypto_init_vqs(struct virtio_crypto *vi)
+{
+ int ret;
+
+ /* Allocate send & receive queues */
+ ret = virtcrypto_alloc_queues(vi);
+ if (ret)
+ goto err;
+
+ ret = virtcrypto_find_vqs(vi);
+ if (ret)
+ goto err_free;
+
+ get_online_cpus();
+ virtcrypto_set_affinity(vi);
+ put_online_cpus();
+
+ return 0;
+
+err_free:
+ virtcrypto_free_queues(vi);
+err:
+ return ret;
+}
+
+static int virtcrypto_update_status(struct virtio_crypto *vcrypto)
+{
+ u32 status;
+ int err;
+ unsigned long flags;
+
+ virtio_cread(vcrypto->vdev,
+ struct virtio_crypto_config, status, &status);
+
+ /*
+ * Unknown status bits would be a host error and the driver
+ * should consider the device to be broken.
+ */
+ if (status & (~VIRTIO_CRYPTO_S_HW_READY)) {
+ dev_warn(&vcrypto->vdev->dev,
+ "Unknown status bits: 0x%x\n", status);
+
+ spin_lock_irqsave(&vcrypto->lock, flags);
+ virtio_break_device(vcrypto->vdev);
+ spin_unlock_irqrestore(&vcrypto->lock, flags);
+ return -EPERM;
+ }
+
+ if (vcrypto->status == status)
+ return 0;
+
+ vcrypto->status = status;
+
+ if (vcrypto->status & VIRTIO_CRYPTO_S_HW_READY) {
+ err = virtcrypto_dev_start(vcrypto);
+ if (err) {
+ dev_err(&vcrypto->vdev->dev,
+ "Failed to start virtio crypto device.\n");
+
+ return -EPERM;
+ }
+ dev_info(&vcrypto->vdev->dev, "Accelerator is ready\n");
+ } else {
+ virtcrypto_dev_stop(vcrypto);
+ dev_info(&vcrypto->vdev->dev, "Accelerator is not ready\n");
+ }
+
+ return 0;
+}
+
+static void virtcrypto_del_vqs(struct virtio_crypto *vcrypto)
+{
+ struct virtio_device *vdev = vcrypto->vdev;
+
+ virtcrypto_clean_affinity(vcrypto, -1);
+
+ vdev->config->del_vqs(vdev);
+
+ virtcrypto_free_queues(vcrypto);
+}
+
+static int virtcrypto_probe(struct virtio_device *vdev)
+{
+ int err = -EFAULT;
+ struct virtio_crypto *vcrypto;
+ u32 max_data_queues = 0, max_cipher_key_len = 0;
+ u32 max_auth_key_len = 0;
+ u64 max_size = 0;
+
+ if (!virtio_has_feature(vdev, VIRTIO_F_VERSION_1))
+ return -ENODEV;
+
+ if (!vdev->config->get) {
+ dev_err(&vdev->dev, "%s failure: config access disabled\n",
+ __func__);
+ return -EINVAL;
+ }
+
+ if (num_possible_nodes() > 1 && dev_to_node(&vdev->dev) < 0) {
+ /*
+ * If the accelerator is connected to a node with no memory
+ * there is no point in using the accelerator since the remote
+ * memory transaction will be very slow.
+ */
+ dev_err(&vdev->dev, "Invalid NUMA configuration.\n");
+ return -EINVAL;
+ }
+
+ vcrypto = kzalloc_node(sizeof(*vcrypto), GFP_KERNEL,
+ dev_to_node(&vdev->dev));
+ if (!vcrypto)
+ return -ENOMEM;
+
+ virtio_cread(vdev, struct virtio_crypto_config,
+ max_dataqueues, &max_data_queues);
+ if (max_data_queues < 1)
+ max_data_queues = 1;
+
+ virtio_cread(vdev, struct virtio_crypto_config,
+ max_cipher_key_len, &max_cipher_key_len);
+ virtio_cread(vdev, struct virtio_crypto_config,
+ max_auth_key_len, &max_auth_key_len);
+ virtio_cread(vdev, struct virtio_crypto_config,
+ max_size, &max_size);
+
+ /* Add virtio crypto device to global table */
+ err = virtcrypto_devmgr_add_dev(vcrypto);
+ if (err) {
+ dev_err(&vdev->dev, "Failed to add new virtio crypto device.\n");
+ goto free;
+ }
+ vcrypto->owner = THIS_MODULE;
+ vcrypto = vdev->priv = vcrypto;
+ vcrypto->vdev = vdev;
+ spin_lock_init(&vcrypto->lock);
+ spin_lock_init(&vcrypto->ctrl_lock);
+
+ /* Use single data queue as default */
+ vcrypto->curr_queue = 1;
+ vcrypto->max_data_queues = max_data_queues;
+ vcrypto->max_cipher_key_len = max_cipher_key_len;
+ vcrypto->max_auth_key_len = max_auth_key_len;
+ vcrypto->max_size = max_size;
+
+ dev_info(&vdev->dev,
+ "max_queues: %u, max_cipher_key_len: %u, max_auth_key_len: %u, max_size 0x%llx\n",
+ vcrypto->max_data_queues,
+ vcrypto->max_cipher_key_len,
+ vcrypto->max_auth_key_len,
+ vcrypto->max_size);
+
+ err = virtcrypto_init_vqs(vcrypto);
+ if (err) {
+ dev_err(&vdev->dev, "Failed to initialize vqs.\n");
+ goto free_dev;
+ }
+ virtio_device_ready(vdev);
+
+ err = virtcrypto_update_status(vcrypto);
+ if (err)
+ goto free_vqs;
+
+ return 0;
+
+free_vqs:
+ vcrypto->vdev->config->reset(vdev);
+ virtcrypto_del_vqs(vcrypto);
+free_dev:
+ virtcrypto_devmgr_rm_dev(vcrypto);
+free:
+ kfree(vcrypto);
+ return err;
+}
+
+static void virtcrypto_free_unused_reqs(struct virtio_crypto *vcrypto)
+{
+ struct virtio_crypto_request *vc_req;
+ int i;
+ struct virtqueue *vq;
+
+ for (i = 0; i < vcrypto->max_data_queues; i++) {
+ vq = vcrypto->data_vq[i].vq;
+ while ((vc_req = virtqueue_detach_unused_buf(vq)) != NULL) {
+ kfree(vc_req->req_data);
+ kfree(vc_req->sgs);
+ }
+ }
+}
+
+static void virtcrypto_remove(struct virtio_device *vdev)
+{
+ struct virtio_crypto *vcrypto = vdev->priv;
+
+ dev_info(&vdev->dev, "Start virtcrypto_remove.\n");
+
+ if (virtcrypto_dev_started(vcrypto))
+ virtcrypto_dev_stop(vcrypto);
+ vdev->config->reset(vdev);
+ virtcrypto_free_unused_reqs(vcrypto);
+ virtcrypto_del_vqs(vcrypto);
+ virtcrypto_devmgr_rm_dev(vcrypto);
+ kfree(vcrypto);
+}
+
+static void virtcrypto_config_changed(struct virtio_device *vdev)
+{
+ struct virtio_crypto *vcrypto = vdev->priv;
+
+ virtcrypto_update_status(vcrypto);
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int virtcrypto_freeze(struct virtio_device *vdev)
+{
+ struct virtio_crypto *vcrypto = vdev->priv;
+
+ vdev->config->reset(vdev);
+ virtcrypto_free_unused_reqs(vcrypto);
+ if (virtcrypto_dev_started(vcrypto))
+ virtcrypto_dev_stop(vcrypto);
+
+ virtcrypto_del_vqs(vcrypto);
+ return 0;
+}
+
+static int virtcrypto_restore(struct virtio_device *vdev)
+{
+ struct virtio_crypto *vcrypto = vdev->priv;
+ int err;
+
+ err = virtcrypto_init_vqs(vcrypto);
+ if (err)
+ return err;
+
+ virtio_device_ready(vdev);
+ err = virtcrypto_dev_start(vcrypto);
+ if (err) {
+ dev_err(&vdev->dev, "Failed to start virtio crypto device.\n");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+#endif
+
+static unsigned int features[] = {
+ /* none */
+};
+
+static struct virtio_device_id id_table[] = {
+ { VIRTIO_ID_CRYPTO, VIRTIO_DEV_ANY_ID },
+ { 0 },
+};
+
+static struct virtio_driver virtio_crypto_driver = {
+ .driver.name = KBUILD_MODNAME,
+ .driver.owner = THIS_MODULE,
+ .feature_table = features,
+ .feature_table_size = ARRAY_SIZE(features),
+ .id_table = id_table,
+ .probe = virtcrypto_probe,
+ .remove = virtcrypto_remove,
+ .config_changed = virtcrypto_config_changed,
+#ifdef CONFIG_PM_SLEEP
+ .freeze = virtcrypto_freeze,
+ .restore = virtcrypto_restore,
+#endif
+};
+
+module_virtio_driver(virtio_crypto_driver);
+
+MODULE_DEVICE_TABLE(virtio, id_table);
+MODULE_DESCRIPTION("virtio crypto device driver");
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Gonglei <arei.gonglei@huawei.com>");
diff --git a/drivers/crypto/virtio/virtio_crypto_mgr.c b/drivers/crypto/virtio/virtio_crypto_mgr.c
new file mode 100644
index 0000000..a69ff71
--- /dev/null
+++ b/drivers/crypto/virtio/virtio_crypto_mgr.c
@@ -0,0 +1,264 @@
+ /* Management for virtio crypto devices (refer to adf_dev_mgr.c)
+ *
+ * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/mutex.h>
+#include <linux/list.h>
+#include <linux/module.h>
+
+#include <uapi/linux/virtio_crypto.h>
+#include "virtio_crypto_common.h"
+
+static LIST_HEAD(virtio_crypto_table);
+static uint32_t num_devices;
+
+/* The table_lock protects the above global list and num_devices */
+static DEFINE_MUTEX(table_lock);
+
+#define VIRTIO_CRYPTO_MAX_DEVICES 32
+
+
+/*
+ * virtcrypto_devmgr_add_dev() - Add vcrypto_dev to the acceleration
+ * framework.
+ * @vcrypto_dev: Pointer to virtio crypto device.
+ *
+ * Function adds virtio crypto device to the global list.
+ * To be used by virtio crypto device specific drivers.
+ *
+ * Return: 0 on success, error code othewise.
+ */
+int virtcrypto_devmgr_add_dev(struct virtio_crypto *vcrypto_dev)
+{
+ struct list_head *itr;
+
+ mutex_lock(&table_lock);
+ if (num_devices == VIRTIO_CRYPTO_MAX_DEVICES) {
+ pr_info("virtio_crypto: only support up to %d devices\n",
+ VIRTIO_CRYPTO_MAX_DEVICES);
+ mutex_unlock(&table_lock);
+ return -EFAULT;
+ }
+
+ list_for_each(itr, &virtio_crypto_table) {
+ struct virtio_crypto *ptr =
+ list_entry(itr, struct virtio_crypto, list);
+
+ if (ptr == vcrypto_dev) {
+ mutex_unlock(&table_lock);
+ return -EEXIST;
+ }
+ }
+ atomic_set(&vcrypto_dev->ref_count, 0);
+ list_add_tail(&vcrypto_dev->list, &virtio_crypto_table);
+ vcrypto_dev->dev_id = num_devices++;
+ mutex_unlock(&table_lock);
+ return 0;
+}
+
+struct list_head *virtcrypto_devmgr_get_head(void)
+{
+ return &virtio_crypto_table;
+}
+
+/*
+ * virtcrypto_devmgr_rm_dev() - Remove vcrypto_dev from the acceleration
+ * framework.
+ * @vcrypto_dev: Pointer to virtio crypto device.
+ *
+ * Function removes virtio crypto device from the acceleration framework.
+ * To be used by virtio crypto device specific drivers.
+ *
+ * Return: void
+ */
+void virtcrypto_devmgr_rm_dev(struct virtio_crypto *vcrypto_dev)
+{
+ mutex_lock(&table_lock);
+ list_del(&vcrypto_dev->list);
+ num_devices--;
+ mutex_unlock(&table_lock);
+}
+
+/*
+ * virtcrypto_devmgr_get_first()
+ *
+ * Function returns the first virtio crypto device from the acceleration
+ * framework.
+ *
+ * To be used by virtio crypto device specific drivers.
+ *
+ * Return: pointer to vcrypto_dev or NULL if not found.
+ */
+struct virtio_crypto *virtcrypto_devmgr_get_first(void)
+{
+ struct virtio_crypto *dev = NULL;
+
+ mutex_lock(&table_lock);
+ if (!list_empty(&virtio_crypto_table))
+ dev = list_first_entry(&virtio_crypto_table,
+ struct virtio_crypto,
+ list);
+ mutex_unlock(&table_lock);
+ return dev;
+}
+
+/*
+ * virtcrypto_dev_in_use() - Check whether vcrypto_dev is currently in use
+ * @vcrypto_dev: Pointer to virtio crypto device.
+ *
+ * To be used by virtio crypto device specific drivers.
+ *
+ * Return: 1 when device is in use, 0 otherwise.
+ */
+int virtcrypto_dev_in_use(struct virtio_crypto *vcrypto_dev)
+{
+ return atomic_read(&vcrypto_dev->ref_count) != 0;
+}
+
+/*
+ * virtcrypto_dev_get() - Increment vcrypto_dev reference count
+ * @vcrypto_dev: Pointer to virtio crypto device.
+ *
+ * Increment the vcrypto_dev refcount and if this is the first time
+ * incrementing it during this period the vcrypto_dev is in use,
+ * increment the module refcount too.
+ * To be used by virtio crypto device specific drivers.
+ *
+ * Return: 0 when successful, EFAULT when fail to bump module refcount
+ */
+int virtcrypto_dev_get(struct virtio_crypto *vcrypto_dev)
+{
+ if (atomic_add_return(1, &vcrypto_dev->ref_count) == 1)
+ if (!try_module_get(vcrypto_dev->owner))
+ return -EFAULT;
+ return 0;
+}
+
+/*
+ * virtcrypto_dev_put() - Decrement vcrypto_dev reference count
+ * @vcrypto_dev: Pointer to virtio crypto device.
+ *
+ * Decrement the vcrypto_dev refcount and if this is the last time
+ * decrementing it during this period the vcrypto_dev is in use,
+ * decrement the module refcount too.
+ * To be used by virtio crypto device specific drivers.
+ *
+ * Return: void
+ */
+void virtcrypto_dev_put(struct virtio_crypto *vcrypto_dev)
+{
+ if (atomic_sub_return(1, &vcrypto_dev->ref_count) == 0)
+ module_put(vcrypto_dev->owner);
+}
+
+/*
+ * virtcrypto_dev_started() - Check whether device has started
+ * @vcrypto_dev: Pointer to virtio crypto device.
+ *
+ * To be used by virtio crypto device specific drivers.
+ *
+ * Return: 1 when the device has started, 0 otherwise
+ */
+int virtcrypto_dev_started(struct virtio_crypto *vcrypto_dev)
+{
+ return (vcrypto_dev->status & VIRTIO_CRYPTO_S_HW_READY);
+}
+
+/*
+ * virtcrypto_get_dev_node() - Get vcrypto_dev on the node.
+ * @node: Node id the driver works.
+ *
+ * Function returns the virtio crypto device used fewest on the node.
+ *
+ * To be used by virtio crypto device specific drivers.
+ *
+ * Return: pointer to vcrypto_dev or NULL if not found.
+ */
+struct virtio_crypto *virtcrypto_get_dev_node(int node)
+{
+ struct virtio_crypto *vcrypto_dev = NULL, *tmp_dev;
+ unsigned long best = ~0;
+ unsigned long ctr;
+
+ mutex_lock(&table_lock);
+ list_for_each_entry(tmp_dev, virtcrypto_devmgr_get_head(), list) {
+
+ if ((node == dev_to_node(&tmp_dev->vdev->dev) ||
+ dev_to_node(&tmp_dev->vdev->dev) < 0) &&
+ virtcrypto_dev_started(tmp_dev)) {
+ ctr = atomic_read(&tmp_dev->ref_count);
+ if (best > ctr) {
+ vcrypto_dev = tmp_dev;
+ best = ctr;
+ }
+ }
+ }
+
+ if (!vcrypto_dev) {
+ pr_info("virtio_crypto: Could not find a device on node %d\n",
+ node);
+ /* Get any started device */
+ list_for_each_entry(tmp_dev,
+ virtcrypto_devmgr_get_head(), list) {
+ if (virtcrypto_dev_started(tmp_dev)) {
+ vcrypto_dev = tmp_dev;
+ break;
+ }
+ }
+ }
+ mutex_unlock(&table_lock);
+ if (!vcrypto_dev)
+ return NULL;
+
+ virtcrypto_dev_get(vcrypto_dev);
+ return vcrypto_dev;
+}
+
+/*
+ * virtcrypto_dev_start() - Start virtio crypto device
+ * @vcrypto: Pointer to virtio crypto device.
+ *
+ * Function notifies all the registered services that the virtio crypto device
+ * is ready to be used.
+ * To be used by virtio crypto device specific drivers.
+ *
+ * Return: 0 on success, EFAULT when fail to register algorithms
+ */
+int virtcrypto_dev_start(struct virtio_crypto *vcrypto)
+{
+ if (virtio_crypto_algs_register()) {
+ pr_err("virtio_crypto: Failed to register crypto algs\n");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
+/*
+ * virtcrypto_dev_stop() - Stop virtio crypto device
+ * @vcrypto: Pointer to virtio crypto device.
+ *
+ * Function notifies all the registered services that the virtio crypto device
+ * is ready to be used.
+ * To be used by virtio crypto device specific drivers.
+ *
+ * Return: void
+ */
+void virtcrypto_dev_stop(struct virtio_crypto *vcrypto)
+{
+ virtio_crypto_algs_unregister();
+}
diff --git a/include/uapi/linux/Kbuild b/include/uapi/linux/Kbuild
index cd2be1c..4bdb84c 100644
--- a/include/uapi/linux/Kbuild
+++ b/include/uapi/linux/Kbuild
@@ -460,6 +460,7 @@ header-y += virtio_rng.h
header-y += virtio_scsi.h
header-y += virtio_types.h
header-y += virtio_vsock.h
+header-y += virtio_crypto.h
header-y += vm_sockets.h
header-y += vt.h
header-y += vtpm_proxy.h
diff --git a/include/uapi/linux/virtio_crypto.h b/include/uapi/linux/virtio_crypto.h
new file mode 100644
index 0000000..50cdc8a
--- /dev/null
+++ b/include/uapi/linux/virtio_crypto.h
@@ -0,0 +1,450 @@
+#ifndef _VIRTIO_CRYPTO_H
+#define _VIRTIO_CRYPTO_H
+/* This header is BSD licensed so anyone can use the definitions to implement
+ * compatible drivers/servers.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of IBM nor the names of its contributors
+ * may be used to endorse or promote products derived from this software
+ * without specific prior written permission.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL IBM OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#include <linux/types.h>
+#include <linux/virtio_types.h>
+#include <linux/virtio_ids.h>
+#include <linux/virtio_config.h>
+
+
+#define VIRTIO_CRYPTO_SERVICE_CIPHER 0
+#define VIRTIO_CRYPTO_SERVICE_HASH 1
+#define VIRTIO_CRYPTO_SERVICE_MAC 2
+#define VIRTIO_CRYPTO_SERVICE_AEAD 3
+
+#define VIRTIO_CRYPTO_OPCODE(service, op) (((service) << 8) | (op))
+
+struct virtio_crypto_ctrl_header {
+#define VIRTIO_CRYPTO_CIPHER_CREATE_SESSION \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_CIPHER, 0x02)
+#define VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_CIPHER, 0x03)
+#define VIRTIO_CRYPTO_HASH_CREATE_SESSION \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_HASH, 0x02)
+#define VIRTIO_CRYPTO_HASH_DESTROY_SESSION \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_HASH, 0x03)
+#define VIRTIO_CRYPTO_MAC_CREATE_SESSION \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_MAC, 0x02)
+#define VIRTIO_CRYPTO_MAC_DESTROY_SESSION \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_MAC, 0x03)
+#define VIRTIO_CRYPTO_AEAD_CREATE_SESSION \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x02)
+#define VIRTIO_CRYPTO_AEAD_DESTROY_SESSION \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x03)
+ __le32 opcode;
+ __le32 algo;
+ __le32 flag;
+ /* data virtqueue id */
+ __le32 queue_id;
+};
+
+struct virtio_crypto_cipher_session_para {
+#define VIRTIO_CRYPTO_NO_CIPHER 0
+#define VIRTIO_CRYPTO_CIPHER_ARC4 1
+#define VIRTIO_CRYPTO_CIPHER_AES_ECB 2
+#define VIRTIO_CRYPTO_CIPHER_AES_CBC 3
+#define VIRTIO_CRYPTO_CIPHER_AES_CTR 4
+#define VIRTIO_CRYPTO_CIPHER_DES_ECB 5
+#define VIRTIO_CRYPTO_CIPHER_DES_CBC 6
+#define VIRTIO_CRYPTO_CIPHER_3DES_ECB 7
+#define VIRTIO_CRYPTO_CIPHER_3DES_CBC 8
+#define VIRTIO_CRYPTO_CIPHER_3DES_CTR 9
+#define VIRTIO_CRYPTO_CIPHER_KASUMI_F8 10
+#define VIRTIO_CRYPTO_CIPHER_SNOW3G_UEA2 11
+#define VIRTIO_CRYPTO_CIPHER_AES_F8 12
+#define VIRTIO_CRYPTO_CIPHER_AES_XTS 13
+#define VIRTIO_CRYPTO_CIPHER_ZUC_EEA3 14
+ __le32 algo;
+ /* length of key */
+ __le32 keylen;
+
+#define VIRTIO_CRYPTO_OP_ENCRYPT 1
+#define VIRTIO_CRYPTO_OP_DECRYPT 2
+ /* encrypt or decrypt */
+ __le32 op;
+ __le32 padding;
+};
+
+struct virtio_crypto_session_input {
+ /* Device-writable part */
+ __le64 session_id;
+ __le32 status;
+ __le32 padding;
+};
+
+struct virtio_crypto_cipher_session_req {
+ struct virtio_crypto_cipher_session_para para;
+ __u8 padding[32];
+};
+
+struct virtio_crypto_hash_session_para {
+#define VIRTIO_CRYPTO_NO_HASH 0
+#define VIRTIO_CRYPTO_HASH_MD5 1
+#define VIRTIO_CRYPTO_HASH_SHA1 2
+#define VIRTIO_CRYPTO_HASH_SHA_224 3
+#define VIRTIO_CRYPTO_HASH_SHA_256 4
+#define VIRTIO_CRYPTO_HASH_SHA_384 5
+#define VIRTIO_CRYPTO_HASH_SHA_512 6
+#define VIRTIO_CRYPTO_HASH_SHA3_224 7
+#define VIRTIO_CRYPTO_HASH_SHA3_256 8
+#define VIRTIO_CRYPTO_HASH_SHA3_384 9
+#define VIRTIO_CRYPTO_HASH_SHA3_512 10
+#define VIRTIO_CRYPTO_HASH_SHA3_SHAKE128 11
+#define VIRTIO_CRYPTO_HASH_SHA3_SHAKE256 12
+ __le32 algo;
+ /* hash result length */
+ __le32 hash_result_len;
+ __u8 padding[8];
+};
+
+struct virtio_crypto_hash_create_session_req {
+ struct virtio_crypto_hash_session_para para;
+ __u8 padding[40];
+};
+
+struct virtio_crypto_mac_session_para {
+#define VIRTIO_CRYPTO_NO_MAC 0
+#define VIRTIO_CRYPTO_MAC_HMAC_MD5 1
+#define VIRTIO_CRYPTO_MAC_HMAC_SHA1 2
+#define VIRTIO_CRYPTO_MAC_HMAC_SHA_224 3
+#define VIRTIO_CRYPTO_MAC_HMAC_SHA_256 4
+#define VIRTIO_CRYPTO_MAC_HMAC_SHA_384 5
+#define VIRTIO_CRYPTO_MAC_HMAC_SHA_512 6
+#define VIRTIO_CRYPTO_MAC_CMAC_3DES 25
+#define VIRTIO_CRYPTO_MAC_CMAC_AES 26
+#define VIRTIO_CRYPTO_MAC_KASUMI_F9 27
+#define VIRTIO_CRYPTO_MAC_SNOW3G_UIA2 28
+#define VIRTIO_CRYPTO_MAC_GMAC_AES 41
+#define VIRTIO_CRYPTO_MAC_GMAC_TWOFISH 42
+#define VIRTIO_CRYPTO_MAC_CBCMAC_AES 49
+#define VIRTIO_CRYPTO_MAC_CBCMAC_KASUMI_F9 50
+#define VIRTIO_CRYPTO_MAC_XCBC_AES 53
+ __le32 algo;
+ /* hash result length */
+ __le32 hash_result_len;
+ /* length of authenticated key */
+ __le32 auth_key_len;
+ __le32 padding;
+};
+
+struct virtio_crypto_mac_create_session_req {
+ struct virtio_crypto_mac_session_para para;
+ __u8 padding[40];
+};
+
+struct virtio_crypto_aead_session_para {
+#define VIRTIO_CRYPTO_NO_AEAD 0
+#define VIRTIO_CRYPTO_AEAD_GCM 1
+#define VIRTIO_CRYPTO_AEAD_CCM 2
+#define VIRTIO_CRYPTO_AEAD_CHACHA20_POLY1305 3
+ __le32 algo;
+ /* length of key */
+ __le32 key_len;
+ /* hash result length */
+ __le32 hash_result_len;
+ /* length of the additional authenticated data (AAD) in bytes */
+ __le32 aad_len;
+ /* encrypt or decrypt, See above VIRTIO_CRYPTO_OP_* */
+ __le32 op;
+ __le32 padding;
+};
+
+struct virtio_crypto_aead_create_session_req {
+ struct virtio_crypto_aead_session_para para;
+ __u8 padding[32];
+};
+
+struct virtio_crypto_alg_chain_session_para {
+#define VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER 1
+#define VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH 2
+ __le32 alg_chain_order;
+/* Plain hash */
+#define VIRTIO_CRYPTO_SYM_HASH_MODE_PLAIN 1
+/* Authenticated hash (mac) */
+#define VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH 2
+/* Nested hash */
+#define VIRTIO_CRYPTO_SYM_HASH_MODE_NESTED 3
+ __le32 hash_mode;
+ struct virtio_crypto_cipher_session_para cipher_param;
+ union {
+ struct virtio_crypto_hash_session_para hash_param;
+ struct virtio_crypto_mac_session_para mac_param;
+ __u8 padding[16];
+ } u;
+ /* length of the additional authenticated data (AAD) in bytes */
+ __le32 aad_len;
+ __le32 padding;
+};
+
+struct virtio_crypto_alg_chain_session_req {
+ struct virtio_crypto_alg_chain_session_para para;
+};
+
+struct virtio_crypto_sym_create_session_req {
+ union {
+ struct virtio_crypto_cipher_session_req cipher;
+ struct virtio_crypto_alg_chain_session_req chain;
+ __u8 padding[48];
+ } u;
+
+ /* Device-readable part */
+
+/* No operation */
+#define VIRTIO_CRYPTO_SYM_OP_NONE 0
+/* Cipher only operation on the data */
+#define VIRTIO_CRYPTO_SYM_OP_CIPHER 1
+/*
+ * Chain any cipher with any hash or mac operation. The order
+ * depends on the value of alg_chain_order param
+ */
+#define VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING 2
+ __le32 op_type;
+ __le32 padding;
+};
+
+struct virtio_crypto_destroy_session_req {
+ /* Device-readable part */
+ __le64 session_id;
+ __u8 padding[48];
+};
+
+/* The request of the control virtqueue's packet */
+struct virtio_crypto_op_ctrl_req {
+ struct virtio_crypto_ctrl_header header;
+
+ union {
+ struct virtio_crypto_sym_create_session_req
+ sym_create_session;
+ struct virtio_crypto_hash_create_session_req
+ hash_create_session;
+ struct virtio_crypto_mac_create_session_req
+ mac_create_session;
+ struct virtio_crypto_aead_create_session_req
+ aead_create_session;
+ struct virtio_crypto_destroy_session_req
+ destroy_session;
+ __u8 padding[56];
+ } u;
+};
+
+struct virtio_crypto_op_header {
+#define VIRTIO_CRYPTO_CIPHER_ENCRYPT \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_CIPHER, 0x00)
+#define VIRTIO_CRYPTO_CIPHER_DECRYPT \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_CIPHER, 0x01)
+#define VIRTIO_CRYPTO_HASH \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_HASH, 0x00)
+#define VIRTIO_CRYPTO_MAC \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_MAC, 0x00)
+#define VIRTIO_CRYPTO_AEAD_ENCRYPT \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x00)
+#define VIRTIO_CRYPTO_AEAD_DECRYPT \
+ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x01)
+ __le32 opcode;
+ /* algo should be service-specific algorithms */
+ __le32 algo;
+ /* session_id should be service-specific algorithms */
+ __le64 session_id;
+ /* control flag to control the request */
+ __le32 flag;
+ __le32 padding;
+};
+
+struct virtio_crypto_cipher_para {
+ /*
+ * Byte Length of valid IV/Counter
+ *
+ * For block ciphers in CBC or F8 mode, or for Kasumi in F8 mode, or for
+ * SNOW3G in UEA2 mode, this is the length of the IV (which
+ * must be the same as the block length of the cipher).
+ * For block ciphers in CTR mode, this is the length of the counter
+ * (which must be the same as the block length of the cipher).
+ * For AES-XTS, this is the 128bit tweak, i, from IEEE Std 1619-2007.
+ *
+ * The IV/Counter will be updated after every partial cryptographic
+ * operation.
+ */
+ __le32 iv_len;
+ /* length of source data */
+ __le32 src_data_len;
+ /* length of dst data */
+ __le32 dst_data_len;
+ __le32 padding;
+};
+
+struct virtio_crypto_hash_para {
+ /* length of source data */
+ __le32 src_data_len;
+ /* hash result length */
+ __le32 hash_result_len;
+};
+
+struct virtio_crypto_mac_para {
+ struct virtio_crypto_hash_para hash;
+};
+
+struct virtio_crypto_aead_para {
+ /*
+ * Byte Length of valid IV data pointed to by the below iv_addr
+ * parameter.
+ *
+ * For GCM mode, this is either 12 (for 96-bit IVs) or 16, in which
+ * case iv_addr points to J0.
+ * For CCM mode, this is the length of the nonce, which can be in the
+ * range 7 to 13 inclusive.
+ */
+ __le32 iv_len;
+ /* length of additional auth data */
+ __le32 aad_len;
+ /* length of source data */
+ __le32 src_data_len;
+ /* length of dst data */
+ __le32 dst_data_len;
+};
+
+struct virtio_crypto_cipher_data_req {
+ /* Device-readable part */
+ struct virtio_crypto_cipher_para para;
+ __u8 padding[24];
+};
+
+struct virtio_crypto_hash_data_req {
+ /* Device-readable part */
+ struct virtio_crypto_hash_para para;
+ __u8 padding[40];
+};
+
+struct virtio_crypto_mac_data_req {
+ /* Device-readable part */
+ struct virtio_crypto_mac_para para;
+ __u8 padding[40];
+};
+
+struct virtio_crypto_alg_chain_data_para {
+ __le32 iv_len;
+ /* Length of source data */
+ __le32 src_data_len;
+ /* Length of destination data */
+ __le32 dst_data_len;
+ /* Starting point for cipher processing in source data */
+ __le32 cipher_start_src_offset;
+ /* Length of the source data that the cipher will be computed on */
+ __le32 len_to_cipher;
+ /* Starting point for hash processing in source data */
+ __le32 hash_start_src_offset;
+ /* Length of the source data that the hash will be computed on */
+ __le32 len_to_hash;
+ /* Length of the additional auth data */
+ __le32 aad_len;
+ /* Length of the hash result */
+ __le32 hash_result_len;
+ __le32 reserved;
+};
+
+struct virtio_crypto_alg_chain_data_req {
+ /* Device-readable part */
+ struct virtio_crypto_alg_chain_data_para para;
+};
+
+struct virtio_crypto_sym_data_req {
+ union {
+ struct virtio_crypto_cipher_data_req cipher;
+ struct virtio_crypto_alg_chain_data_req chain;
+ __u8 padding[40];
+ } u;
+
+ /* See above VIRTIO_CRYPTO_SYM_OP_* */
+ __le32 op_type;
+ __le32 padding;
+};
+
+struct virtio_crypto_aead_data_req {
+ /* Device-readable part */
+ struct virtio_crypto_aead_para para;
+ __u8 padding[32];
+};
+
+/* The request of the data virtqueue's packet */
+struct virtio_crypto_op_data_req {
+ struct virtio_crypto_op_header header;
+
+ union {
+ struct virtio_crypto_sym_data_req sym_req;
+ struct virtio_crypto_hash_data_req hash_req;
+ struct virtio_crypto_mac_data_req mac_req;
+ struct virtio_crypto_aead_data_req aead_req;
+ __u8 padding[48];
+ } u;
+};
+
+#define VIRTIO_CRYPTO_OK 0
+#define VIRTIO_CRYPTO_ERR 1
+#define VIRTIO_CRYPTO_BADMSG 2
+#define VIRTIO_CRYPTO_NOTSUPP 3
+#define VIRTIO_CRYPTO_INVSESS 4 /* Invalid session id */
+
+/* The accelerator hardware is ready */
+#define VIRTIO_CRYPTO_S_HW_READY (1 << 0)
+
+struct virtio_crypto_config {
+ /* See VIRTIO_CRYPTO_OP_* above */
+ __u32 status;
+
+ /*
+ * Maximum number of data queue
+ */
+ __u32 max_dataqueues;
+
+ /*
+ * Specifies the services mask which the device support,
+ * see VIRTIO_CRYPTO_SERVICE_* above
+ */
+ __u32 crypto_services;
+
+ /* Detailed algorithms mask */
+ __u32 cipher_algo_l;
+ __u32 cipher_algo_h;
+ __u32 hash_algo;
+ __u32 mac_algo_l;
+ __u32 mac_algo_h;
+ __u32 aead_algo;
+ /* Maximum length of cipher key */
+ __u32 max_cipher_key_len;
+ /* Maximum length of authenticated key */
+ __u32 max_auth_key_len;
+ __u32 reserve;
+ /* Maximum size of each crypto request's content */
+ __u64 max_size;
+};
+
+struct virtio_crypto_inhdr {
+ /* See VIRTIO_CRYPTO_* above */
+ __u8 status;
+};
+#endif
diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h
index 3228d58..6d5c3b2 100644
--- a/include/uapi/linux/virtio_ids.h
+++ b/include/uapi/linux/virtio_ids.h
@@ -42,5 +42,6 @@
#define VIRTIO_ID_GPU 16 /* virtio GPU */
#define VIRTIO_ID_INPUT 18 /* virtio input */
#define VIRTIO_ID_VSOCK 19 /* virtio vsock transport */
+#define VIRTIO_ID_CRYPTO 20 /* virtio crypto */
#endif /* _LINUX_VIRTIO_IDS_H */
--
1.8.3.1
^ permalink raw reply related
* [PATCH v5 0/1] virtio-crypto: add Linux driver
From: Gonglei @ 2016-12-01 12:39 UTC (permalink / raw)
To: linux-kernel, qemu-devel, virtio-dev, virtualization,
linux-crypto
Cc: luonengjun, mst, stefanha, weidong.huang, wu.wubin, xin.zeng,
claudio.fontana, herbert, pasic, davem, jianjay.zhou, hanweidong,
arei.gonglei, cornelia.huck, xuquan8, longpeng2, wanzongshun,
Gonglei
v5:
- add comments for algs_lock and table_lock. [Stefan]
- use kzfree instead of kfree for key material security. [Stefan]
- drop unnecessary spin_lock for struct virtio_crypto_ablkcipher_ctx.
- dynamically allocated memory for iv in order to avoid to do DMA from
the stack memory in __virtio_crypto_ablkcipher_do_req().
- add logs for error path in virtio_crypto_alg_validate_key().
- add lock before calling virtio_break_device() in virtcrypto_update_status()
v4:
- rework unknow status bit handler by calling virtio_break_device(). [Cornelia]
- convert space to tab in Kconfig. [Stefan]
- rename virtio_crypto.c to virtio_crypto_core.c and then make the
moudle named virtio_crypto.ko for consistency. [Stefan]
- don't call virtcrypto_dev_stop() on failure path. [Stefan]
- don't add two empty lines. [Michael]
- fix possible race by add spin_lock in virtio_crypto_alg_ablkcipher_init_session() [Michael and Halil]
- drop virtcrypto_devmgr_get_first() calling in virtio_crypto_ablkcipher_setkey. [Michael]
- drop superfluous assigned value for virtio_crypto_algs[i].cra_flags
in virtio_crypto_algs_register(). [Stefan]
- decrease virtio_crypto_active_devs if calling crypto_register_algs() failed. [Stefan]
- fix some typos here and there. [Stefan]
- fix missing table_lock usage in virtio_crypto_mgr.c. [Stefan]
- drop confused comments in virtio_crypto_alg_ablkcipher_init_session()
for virtqueue_kick(). [Halil]
v3:
- set cpu affinity when data queues are not equal to the number of online cpus. [Michael]
- add TODO comments for cpu hotplug (changing the relationship of binding virtqueue and cpu)
- use __u32/64 in the config space since the virtio->get() doesn't support byte-swap yet. [Michael]
- drop the whole patch 1 of v2 because the above reason.
- add VERSION_1 check at the beginning of virtcrypto_probe()
- s/-1/EPERM/g in virtcrypto_update_status(), don't change err to EFAULT then. [Michael]
- add reset operation before delete the virtqueus. [Micheal]
- drop an unnecessiry spin_lock calling in virtcrypto_freeze(), avoid possible dead lock. [Micheal]
- redefine parameter alg's type in order to use a cast for it. [Michael]
- pad all structures to have the same size in one union, and add a member to
show the union's size in virtio_crypto.h. [Michael]
- update MAINTAINER file to add virtio-crypto stuff to Michael's entry so that
the corresponding patches can be CC'ed to Michael as well because the virtio-crypto
doesn't lay in driver/virtio directory.
The virtio crypto device is a virtual cryptography device
as well as a kind of virtual hardware accelerator for
virtual machines. The encryption anddecryption requests
are placed in the data queue and are ultimately handled by
thebackend crypto accelerators. The second queue is the
control queue used to create or destroy sessions for
symmetric algorithms and will control some advanced features
in the future. The virtio crypto device provides the following
cryptoservices: CIPHER, MAC, HASH, and AEAD.
For more information about virtio-crypto device, please see:
http://qemu-project.org/Features/VirtioCrypto
For better reviewing, pls see below explaination.
The patch mainly includes five files:
1) virtio_crypto.h is the header file for virtio-crypto device,
which is based on the virtio-crypto specification.
2) virtio_crypto_core.c is the entry of the driver module,
which is similar with other virtio devices, such as virtio-net,
virtio-input etc.
3) virtio_crypto_mgr.c is used to manage the virtio
crypto devices in the system. We support up to 32 virtio-crypto
devices currently. I use a global list to store the virtio crypto
devices which refer to Intel QAT driver. Meanwhile, the file
includs the functions of add/del/search/start/stop for virtio
crypto devices.
4) virtio_crypto_common.h is a private header file for virtio
crypto driver, includes structure definations, and function declarations.
5) virtio_crypto_algs.c is the realization of algs based on Linux Crypto Framwork,
which can register different crypto algorithms. Currently it's only support AES-CBC.
The Crypto guys can mainly focus on this file.
v2:
- stop doing DMA from the stack, CONFIG_VMAP_STACK=y [Salvatore]
- convert __virtio32/64 to __le32/64 in virtio_crypto.h
- remove VIRTIO_CRYPTO_S_STARTED based on the lastest virtio crypto spec.
- introduces the little edian functions for VIRTIO_1 devices in patch 1.
Gonglei (1):
crypto: add virtio-crypto driver
MAINTAINERS | 9 +
drivers/crypto/Kconfig | 2 +
drivers/crypto/Makefile | 1 +
drivers/crypto/virtio/Kconfig | 10 +
drivers/crypto/virtio/Makefile | 5 +
drivers/crypto/virtio/virtio_crypto_algs.c | 537 +++++++++++++++++++++++++++
drivers/crypto/virtio/virtio_crypto_common.h | 122 ++++++
drivers/crypto/virtio/virtio_crypto_core.c | 464 +++++++++++++++++++++++
drivers/crypto/virtio/virtio_crypto_mgr.c | 264 +++++++++++++
include/uapi/linux/Kbuild | 1 +
include/uapi/linux/virtio_crypto.h | 450 ++++++++++++++++++++++
include/uapi/linux/virtio_ids.h | 1 +
12 files changed, 1866 insertions(+)
create mode 100644 drivers/crypto/virtio/Kconfig
create mode 100644 drivers/crypto/virtio/Makefile
create mode 100644 drivers/crypto/virtio/virtio_crypto_algs.c
create mode 100644 drivers/crypto/virtio/virtio_crypto_common.h
create mode 100644 drivers/crypto/virtio/virtio_crypto_core.c
create mode 100644 drivers/crypto/virtio/virtio_crypto_mgr.c
create mode 100644 include/uapi/linux/virtio_crypto.h
--
1.8.3.1
^ permalink raw reply
* RE: [PATCH v4 1/1] crypto: add virtio-crypto driver
From: Gonglei (Arei) @ 2016-12-01 12:33 UTC (permalink / raw)
To: Stefan Hajnoczi, Andy Whitcroft
Cc: linux-kernel@vger.kernel.org, qemu-devel@nongnu.org,
virtio-dev@lists.oasis-open.org,
virtualization@lists.linux-foundation.org,
linux-crypto@vger.kernel.org, Luonengjun, mst@redhat.com,
Huangweidong (C), Wubin (H), xin.zeng@intel.com, Claudio Fontana,
herbert@gondor.apana.org.au, pasic@linux.vnet.ibm.com,
davem@davemloft.net, Zhoujian (jay, Euler)
In-Reply-To: <20161201115327.GB10441@stefanha-x1.localdomain>
>
> On Thu, Dec 01, 2016 at 02:27:19AM +0000, Gonglei (Arei) wrote:
> > > On Tue, Nov 29, 2016 at 08:48:14PM +0800, Gonglei wrote:
> > > > diff --git a/drivers/crypto/virtio/virtio_crypto_algs.c
> > > b/drivers/crypto/virtio/virtio_crypto_algs.c
> > > > new file mode 100644
> > > > index 0000000..08b077f
> > > > --- /dev/null
> > > > +++ b/drivers/crypto/virtio/virtio_crypto_algs.c
> > > > @@ -0,0 +1,518 @@
> > > > + /* Algorithms supported by virtio crypto device
> > > > + *
> > > > + * Authors: Gonglei <arei.gonglei@huawei.com>
> > > > + *
> > > > + * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
> > > > + *
> > > > + * This program is free software; you can redistribute it and/or modify
> > > > + * it under the terms of the GNU General Public License as published by
> > > > + * the Free Software Foundation; either version 2 of the License, or
> > > > + * (at your option) any later version.
> > > > + *
> > > > + * This program is distributed in the hope that it will be useful,
> > > > + * but WITHOUT ANY WARRANTY; without even the implied warranty
> of
> > > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
> the
> > > > + * GNU General Public License for more details.
> > > > + *
> > > > + * You should have received a copy of the GNU General Public License
> > > > + * along with this program; if not, see
> <http://www.gnu.org/licenses/>.
> > > > + */
> > > > +
> > > > +#include <linux/scatterlist.h>
> > > > +#include <crypto/algapi.h>
> > > > +#include <linux/err.h>
> > > > +#include <crypto/scatterwalk.h>
> > > > +#include <linux/atomic.h>
> > > > +
> > > > +#include <uapi/linux/virtio_crypto.h>
> > > > +#include "virtio_crypto_common.h"
> > > > +
> > > > +static DEFINE_MUTEX(algs_lock);
> > >
> > > Did you run checkpatch.pl? I think it encourages you to document what
> > > the lock protects.
> > >
> > Sure. Basically I run checkpatch.py each time. :)
> >
> > # ./scripts/checkpatch.pl 0001-crypto-add-virtio-crypto-driver.patch
> > total: 0 errors, 0 warnings, 1873 lines checked
> >
> > 0001-crypto-add-virtio-crypto-driver.patch has no obvious style problems and
> is ready for submission.
>
> Looks like a bug in checkpatch.pl:
>
> # check for spinlock_t definitions without a comment.
> if ($line =~ /^.\s*(struct\s+mutex|spinlock_t)\s+\S+;/ ||
> $line =~ /^.\s*(DEFINE_MUTEX)\s*\(/) {
> my $which = $1;
> if (!ctx_has_comment($first_line, $linenr)) {
> CHK("UNCOMMENTED_DEFINITION",
> "$1 definition without comment\n" . $herecurr);
> }
> }
>
> Since your mutex definition has the 'static' keyword in front of it
> checkpatch.pl misses it!
>
Anyway I added the comments. Thanks :)
Regards,
-Gonglei
> Andy: Is this checkpatch.pl behavior intentional?
>
> Stefan
^ permalink raw reply
* RE: [PATCH v4 1/1] crypto: add virtio-crypto driver
From: Gonglei (Arei) @ 2016-12-01 12:23 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: linux-kernel@vger.kernel.org, qemu-devel@nongnu.org,
virtio-dev@lists.oasis-open.org,
virtualization@lists.linux-foundation.org,
linux-crypto@vger.kernel.org, Luonengjun, mst@redhat.com,
Huangweidong (C), Wubin (H), xin.zeng@intel.com, Claudio Fontana,
herbert@gondor.apana.org.au, pasic@linux.vnet.ibm.com,
davem@davemloft.net, Zhoujian (jay, Euler)
In-Reply-To: <20161201115810.GC10441@stefanha-x1.localdomain>
>
> Subject: Re: [PATCH v4 1/1] crypto: add virtio-crypto driver
>
> On Thu, Dec 01, 2016 at 02:27:19AM +0000, Gonglei (Arei) wrote:
> > > On Tue, Nov 29, 2016 at 08:48:14PM +0800, Gonglei wrote:
> > > > +static int virtio_crypto_alg_ablkcipher_init_session(
> > > > + struct virtio_crypto_ablkcipher_ctx *ctx,
> > > > + uint32_t alg, const uint8_t *key,
> > > > + unsigned int keylen,
> > > > + int encrypt)
> > > > +{
> > > > + struct scatterlist outhdr, key_sg, inhdr, *sgs[3];
> > > > + unsigned int tmp;
> > > > + struct virtio_crypto *vcrypto = ctx->vcrypto;
> > > > + int op = encrypt ? VIRTIO_CRYPTO_OP_ENCRYPT :
> > > VIRTIO_CRYPTO_OP_DECRYPT;
> > > > + int err;
> > > > + unsigned int num_out = 0, num_in = 0;
> > > > +
> > > > + /*
> > > > + * Avoid to do DMA from the stack, switch to using
> > > > + * dynamically-allocated for the key
> > > > + */
> > > > + uint8_t *cipher_key = kmalloc(keylen, GFP_ATOMIC);
> > > > +
> > > > + if (!cipher_key)
> > > > + return -ENOMEM;
> > > > +
> > > > + memcpy(cipher_key, key, keylen);
> > >
> > > Are there any rules on handling key material in the kernel? This buffer
> > > is just kfreed later. Do you need to zero it out before freeing it?
> > >
> > Good questions. For kernel crypto core, each cipher request should be freed
> > by skcipher_request_free(): zeroize and free request data structure.
> >
> > I need to use kzfree() for key as well. I'll also check other stuffs. Thanks.
> >
> > > > +
> > > > + spin_lock(&vcrypto->ctrl_lock);
> > >
> > > The QAT accelerator driver doesn't spin while talking to the device in
> > > virtio_crypto_alg_ablkcipher_init_session(). I didn't find any other
> > > driver examples in the kernel tree, but this function seems like a
> > > weakness in the virtio-crypto device.
> > >
> > The control queues of virtio-net and virtio-console are also be locked
> > Please see:
> > __send_control_msg() in virtio_console.c and virtio-net's control queue
> > protected by rtnl lock.
> >
> > I didn't want to protect session creations but the virtqueue's operations
> > like what other virtio devices do.
> >
> > > While QEMU is servicing the create session command this vcpu is blocked.
> > > The QEMU global mutex is held so no other vcpu can enter QEMU and the
> > > QMP monitor is also blocked.
> > >
> > > This is a scalability and performance problem. Can you look at how QAT
> > > avoids this synchronous session setup?
> >
> > For QAT driver, the session creation is synchronous as well because it's a
> > plain software operation which can be completed ASAP.
>
> I'm mentioning the vmexit and wait for request completion in a spinlock
> because the same type of issue has been a performance bottleneck with
> virtio guest driver in the past.
>
> If there is a way to avoid spinning then that would be preferred. It's
> basically a known anti-pattern for virtio guest drivers.
>
Oh, sorry for my misunderstanding. ;)
> Could you initialize the session on the host side when the first
> asynchronous data is submitted for encryption/decryption instead of
> during init_session()?
>
I remember I discuss this problem with Alex two/three moths
ago. In some scenarios, its indeed a performance problem, such as each request has
different algorithms or keys in HTTP connections. It's performance will be better
if we just use one data virtqueue to pass session and data to the backend at one time.
But for the batch cipher operations with the same algorithm, the performance is poor,
Because we can't do batch operations for those requests. That's the great function
of session operations on the control virtqueue. Refer to the our clients requirements
and the existing QAT driver, the scenario is even more in NFV.
As your mention here, It's hard to do this IMO, because the backend can't know
the previous session belongs to which requests if we don't pass the session_id
to the backend.
Regards,
-Gonglei
^ permalink raw reply
* Re: [PATCH v4 1/1] crypto: add virtio-crypto driver
From: Stefan Hajnoczi @ 2016-12-01 11:58 UTC (permalink / raw)
To: Gonglei (Arei)
Cc: linux-kernel@vger.kernel.org, qemu-devel@nongnu.org,
virtio-dev@lists.oasis-open.org,
virtualization@lists.linux-foundation.org,
linux-crypto@vger.kernel.org, Luonengjun, mst@redhat.com,
Huangweidong (C), Wubin (H), xin.zeng@intel.com, Claudio Fontana,
herbert@gondor.apana.org.au, pasic@linux.vnet.ibm.com,
davem@davemloft.net, Zhoujian (jay, Euler)
In-Reply-To: <33183CC9F5247A488A2544077AF19020DA14EC02@DGGEMA505-MBX.china.huawei.com>
[-- Attachment #1: Type: text/plain, Size: 2939 bytes --]
On Thu, Dec 01, 2016 at 02:27:19AM +0000, Gonglei (Arei) wrote:
> > On Tue, Nov 29, 2016 at 08:48:14PM +0800, Gonglei wrote:
> > > +static int virtio_crypto_alg_ablkcipher_init_session(
> > > + struct virtio_crypto_ablkcipher_ctx *ctx,
> > > + uint32_t alg, const uint8_t *key,
> > > + unsigned int keylen,
> > > + int encrypt)
> > > +{
> > > + struct scatterlist outhdr, key_sg, inhdr, *sgs[3];
> > > + unsigned int tmp;
> > > + struct virtio_crypto *vcrypto = ctx->vcrypto;
> > > + int op = encrypt ? VIRTIO_CRYPTO_OP_ENCRYPT :
> > VIRTIO_CRYPTO_OP_DECRYPT;
> > > + int err;
> > > + unsigned int num_out = 0, num_in = 0;
> > > +
> > > + /*
> > > + * Avoid to do DMA from the stack, switch to using
> > > + * dynamically-allocated for the key
> > > + */
> > > + uint8_t *cipher_key = kmalloc(keylen, GFP_ATOMIC);
> > > +
> > > + if (!cipher_key)
> > > + return -ENOMEM;
> > > +
> > > + memcpy(cipher_key, key, keylen);
> >
> > Are there any rules on handling key material in the kernel? This buffer
> > is just kfreed later. Do you need to zero it out before freeing it?
> >
> Good questions. For kernel crypto core, each cipher request should be freed
> by skcipher_request_free(): zeroize and free request data structure.
>
> I need to use kzfree() for key as well. I'll also check other stuffs. Thanks.
>
> > > +
> > > + spin_lock(&vcrypto->ctrl_lock);
> >
> > The QAT accelerator driver doesn't spin while talking to the device in
> > virtio_crypto_alg_ablkcipher_init_session(). I didn't find any other
> > driver examples in the kernel tree, but this function seems like a
> > weakness in the virtio-crypto device.
> >
> The control queues of virtio-net and virtio-console are also be locked
> Please see:
> __send_control_msg() in virtio_console.c and virtio-net's control queue
> protected by rtnl lock.
>
> I didn't want to protect session creations but the virtqueue's operations
> like what other virtio devices do.
>
> > While QEMU is servicing the create session command this vcpu is blocked.
> > The QEMU global mutex is held so no other vcpu can enter QEMU and the
> > QMP monitor is also blocked.
> >
> > This is a scalability and performance problem. Can you look at how QAT
> > avoids this synchronous session setup?
>
> For QAT driver, the session creation is synchronous as well because it's a
> plain software operation which can be completed ASAP.
I'm mentioning the vmexit and wait for request completion in a spinlock
because the same type of issue has been a performance bottleneck with
virtio guest driver in the past.
If there is a way to avoid spinning then that would be preferred. It's
basically a known anti-pattern for virtio guest drivers.
Could you initialize the session on the host side when the first
asynchronous data is submitted for encryption/decryption instead of
during init_session()?
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]
^ permalink raw reply
* Re: [PATCH v4 1/1] crypto: add virtio-crypto driver
From: Stefan Hajnoczi @ 2016-12-01 11:53 UTC (permalink / raw)
To: Andy Whitcroft
Cc: linux-kernel@vger.kernel.org, qemu-devel@nongnu.org,
virtio-dev@lists.oasis-open.org,
virtualization@lists.linux-foundation.org,
linux-crypto@vger.kernel.org, Luonengjun, mst@redhat.com,
Huangweidong (C), Wubin (H), xin.zeng@intel.com, Claudio Fontana,
herbert@gondor.apana.org.au, pasic@linux.vnet.ibm.com,
davem@davemloft.net, Zhoujian (jay, Euler)
In-Reply-To: <33183CC9F5247A488A2544077AF19020DA14EC02@DGGEMA505-MBX.china.huawei.com>
[-- Attachment #1: Type: text/plain, Size: 2675 bytes --]
On Thu, Dec 01, 2016 at 02:27:19AM +0000, Gonglei (Arei) wrote:
> > On Tue, Nov 29, 2016 at 08:48:14PM +0800, Gonglei wrote:
> > > diff --git a/drivers/crypto/virtio/virtio_crypto_algs.c
> > b/drivers/crypto/virtio/virtio_crypto_algs.c
> > > new file mode 100644
> > > index 0000000..08b077f
> > > --- /dev/null
> > > +++ b/drivers/crypto/virtio/virtio_crypto_algs.c
> > > @@ -0,0 +1,518 @@
> > > + /* Algorithms supported by virtio crypto device
> > > + *
> > > + * Authors: Gonglei <arei.gonglei@huawei.com>
> > > + *
> > > + * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
> > > + *
> > > + * This program is free software; you can redistribute it and/or modify
> > > + * it under the terms of the GNU General Public License as published by
> > > + * the Free Software Foundation; either version 2 of the License, or
> > > + * (at your option) any later version.
> > > + *
> > > + * This program is distributed in the hope that it will be useful,
> > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > > + * GNU General Public License for more details.
> > > + *
> > > + * You should have received a copy of the GNU General Public License
> > > + * along with this program; if not, see <http://www.gnu.org/licenses/>.
> > > + */
> > > +
> > > +#include <linux/scatterlist.h>
> > > +#include <crypto/algapi.h>
> > > +#include <linux/err.h>
> > > +#include <crypto/scatterwalk.h>
> > > +#include <linux/atomic.h>
> > > +
> > > +#include <uapi/linux/virtio_crypto.h>
> > > +#include "virtio_crypto_common.h"
> > > +
> > > +static DEFINE_MUTEX(algs_lock);
> >
> > Did you run checkpatch.pl? I think it encourages you to document what
> > the lock protects.
> >
> Sure. Basically I run checkpatch.py each time. :)
>
> # ./scripts/checkpatch.pl 0001-crypto-add-virtio-crypto-driver.patch
> total: 0 errors, 0 warnings, 1873 lines checked
>
> 0001-crypto-add-virtio-crypto-driver.patch has no obvious style problems and is ready for submission.
Looks like a bug in checkpatch.pl:
# check for spinlock_t definitions without a comment.
if ($line =~ /^.\s*(struct\s+mutex|spinlock_t)\s+\S+;/ ||
$line =~ /^.\s*(DEFINE_MUTEX)\s*\(/) {
my $which = $1;
if (!ctx_has_comment($first_line, $linenr)) {
CHK("UNCOMMENTED_DEFINITION",
"$1 definition without comment\n" . $herecurr);
}
}
Since your mutex definition has the 'static' keyword in front of it
checkpatch.pl misses it!
Andy: Is this checkpatch.pl behavior intentional?
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]
^ permalink raw reply
* [PATCH v2] crypto: AF_ALG - fix AEAD AIO handling of zero buffer
From: Stephan Mueller @ 2016-12-01 7:22 UTC (permalink / raw)
To: herbert; +Cc: linux-crypto
Hi Herbert,
I split out the bug fix patch from the AD/tag formatting patch as they most likely will come after the next merge window.
---8<---
Handle the case when the caller provided a zero buffer to
sendmsg/sendpage. Such scenario is legal for AEAD ciphers when no
plaintext / ciphertext and no AAD is provided and the caller only
requests the generation of the tag value.
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
crypto/algif_aead.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/crypto/algif_aead.c b/crypto/algif_aead.c
index 80a0f1a..6e95137 100644
--- a/crypto/algif_aead.c
+++ b/crypto/algif_aead.c
@@ -448,12 +448,13 @@ static int aead_recvmsg_async(struct socket *sock, struct msghdr *msg,
used -= ctx->aead_assoclen + (ctx->enc ? as : 0);
/* take over all tx sgls from ctx */
- areq->tsgl = sock_kmalloc(sk, sizeof(*areq->tsgl) * sgl->cur,
+ areq->tsgl = sock_kmalloc(sk,
+ sizeof(*areq->tsgl) * max_t(u32, sgl->cur, 1),
GFP_KERNEL);
if (unlikely(!areq->tsgl))
goto free;
- sg_init_table(areq->tsgl, sgl->cur);
+ sg_init_table(areq->tsgl, max_t(u32, sgl->cur, 1));
for (i = 0; i < sgl->cur; i++)
sg_set_page(&areq->tsgl[i], sg_page(&sgl->sg[i]),
sgl->sg[i].length, sgl->sg[i].offset);
--
2.9.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox