* Fwd: [linux-next:master 7274/8070] undefined reference to `crypto_alloc_shash'
From: Stephan Müller @ 2017-04-09 20:09 UTC (permalink / raw)
To: herbert; +Cc: linux-crypto, David Howells
Hi Herbert,
I got the attached error report. This is due to CRYPTO_HASH2 is selected as m
whereas security/keys/dh.c is compiled as y. Note CRYPTO_HASH is selected as
y.
dh.c requires shash.
Kconfig for dh.c contains:
config KEY_DH_OPERATIONS
bool "Diffie-Hellman operations on retained keys"
depends on KEYS
select MPILIB
select CRYPTO_HASH
Shall I select CRYPTO_HASH2 instead or in addition to CRYPTO_HASH?
Thanks.
Ciao
Stephan
---------- Weitergeleitete Nachricht ----------
Betreff: [linux-next:master 7274/8070] undefined reference to
`crypto_alloc_shash'
Datum: Donnerstag, 6. April 2017, 10:03:30 CEST
Von: kbuild test robot <fengguang.wu@intel.com>
An: Stephan Mueller <smueller@chronox.de>
Kopie: kbuild-all@01.org, David Howells <dhowells@redhat.com>
tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
master
head: d203084e692ac2f656cf8c50449f8f90ca7875de
commit: f1c316a3ab9d24df6022682422fe897492f2c0c8 [7274/8070] KEYS: add
SP800-56A KDF support for DH
config: x86_64-randconfig-s0-04061437 (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
git checkout f1c316a3ab9d24df6022682422fe897492f2c0c8
# save the attached .config to linux build tree
make ARCH=x86_64
All errors (new ones prefixed by >>):
security/built-in.o: In function `__keyctl_dh_compute':
>> (.text+0xa4f7): undefined reference to `crypto_alloc_shash'
security/built-in.o: In function `__keyctl_dh_compute':
>> (.text+0xa7b5): undefined reference to `crypto_shash_update'
security/built-in.o: In function `__keyctl_dh_compute':
(.text+0xa7ed): undefined reference to `crypto_shash_update'
security/built-in.o: In function `__keyctl_dh_compute':
>> (.text+0xa832): undefined reference to `crypto_shash_final'
security/built-in.o: In function `__keyctl_dh_compute':
(.text+0xa881): undefined reference to `crypto_shash_final'
security/built-in.o: In function `__keyctl_dh_compute':
>> (.text+0xa9bd): undefined reference to `crypto_destroy_tfm'
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
-------------------------------------------------------------
^ permalink raw reply
* Re: crypto_memneq not backported to 3.10
From: Willy Tarreau @ 2017-04-09 13:25 UTC (permalink / raw)
To: Jason A. Donenfeld; +Cc: stable, LKML, Linux Crypto Mailing List
In-Reply-To: <CAHmME9qjWeskvkVJCNnY1Xm7uAu4=gnuSsxX5KMf3kt6kqF9ow@mail.gmail.com>
Hi Jason,
On Sun, Apr 09, 2017 at 02:59:53PM +0200, Jason A. Donenfeld wrote:
> Hey Willy,
>
> Linux 3.10 is inexplicably missing crypto_memneq, making all crypto
> mac comparisons use non constant-time comparisons. Bad news bears.
>
> 3.12 got these backported with
> d68e944a8fcb2c6212b38064771c9f5af7b0b92c,
> afe5a791d374e50a06ada7f4eda4e921e1b77996, and possibly others. I'd
> suggest following suit, since many people are relying on this kernel
> to do safe crypto.
Interesting. I remembered seeing some crypto_memneq stuff in the past,
and in fact there was one patch talking about this but trimmed down to
only affect other parts since crypto_memneq is indeed not part of 3.10.
I'll check if the 3.12 patches above can be safely backported, and I'll
have to re-apply the missing part of the one that was trimmed down
(commit 620c411 ("crypto: more robust crypto_memneq")).
Thanks!
Willy
^ permalink raw reply
* crypto_memneq not backported to 3.10
From: Jason A. Donenfeld @ 2017-04-09 12:59 UTC (permalink / raw)
To: Willy Tarreau, stable; +Cc: LKML, Linux Crypto Mailing List
Hey Willy,
Linux 3.10 is inexplicably missing crypto_memneq, making all crypto
mac comparisons use non constant-time comparisons. Bad news bears.
3.12 got these backported with
d68e944a8fcb2c6212b38064771c9f5af7b0b92c,
afe5a791d374e50a06ada7f4eda4e921e1b77996, and possibly others. I'd
suggest following suit, since many people are relying on this kernel
to do safe crypto.
Thanks,
Jason
^ permalink raw reply
* [PATCH v4 2/2] crypto: hw_random - Add new Exynos RNG driver
From: Krzysztof Kozlowski @ 2017-04-08 13:32 UTC (permalink / raw)
To: James Hogan, Matt Mackall, Herbert Xu, Kukjin Kim,
Krzysztof Kozlowski, Javier Martinez Canillas, David S. Miller,
Dave Airlie, David Airlie, Bernie Thompson,
Bartlomiej Zolnierkiewicz, Greg Kroah-Hartman,
PrasannaKumar Muralidharan, linux-kernel, linux-metag,
linux-crypto, linux-arm-kernel, linux-samsung-soc, dri-devel,
linux-fbdev
In-Reply-To: <20170408133245.30147-1-krzk@kernel.org>
Replace existing hw_ranndom/exynos-rng driver with a new, reworked one.
This is a driver for pseudo random number generator block which on
Exynos4 chipsets must be seeded with some value. On newer Exynos5420
chipsets it might seed itself from true random number generator block
but this is not implemented yet.
New driver is a complete rework to use the crypto ALGAPI instead of
hw_random API. Rationale for the change:
1. hw_random interface is for true RNG devices.
2. The old driver was seeding itself with jiffies which is not a
reliable source for randomness.
3. Device generates five random 32-bit numbers in each pass but old
driver was returning only one 32-bit number thus its performance was
reduced.
Compatibility with DeviceTree bindings is preserved.
New driver does not use runtime power management but manually enables
and disables the clock when needed. This is preferred approach because
using runtime PM just to toggle clock is huge overhead.
Another difference is reseeding itself with generated random data
periodically and during resuming from system suspend (previously driver
was re-seeding itself again with jiffies).
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: Stephan Müller <smueller@chronox.de>
Reviewed-by: PrasannaKumar Muralidharan <prasannatsmkumar@gmail.com>
---
MAINTAINERS | 8 +
drivers/char/hw_random/Kconfig | 14 --
drivers/char/hw_random/Makefile | 1 -
drivers/char/hw_random/exynos-rng.c | 231 ---------------------
drivers/crypto/Kconfig | 15 ++
drivers/crypto/Makefile | 1 +
drivers/crypto/exynos-rng.c | 389 ++++++++++++++++++++++++++++++++++++
7 files changed, 413 insertions(+), 246 deletions(-)
delete mode 100644 drivers/char/hw_random/exynos-rng.c
create mode 100644 drivers/crypto/exynos-rng.c
diff --git a/MAINTAINERS b/MAINTAINERS
index affecc6d59f4..371fda859d43 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -10977,6 +10977,14 @@ L: alsa-devel@alsa-project.org (moderated for non-subscribers)
S: Supported
F: sound/soc/samsung/
+SAMSUNG EXYNOS PSEUDO RANDOM NUMBER GENERATOR (RNG) DRIVER
+M: Krzysztof Kozlowski <krzk@kernel.org>
+L: linux-crypto@vger.kernel.org
+L: linux-samsung-soc@vger.kernel.org
+S: Maintained
+F: drivers/crypto/exynos-rng.c
+F: Documentation/devicetree/bindings/rng/samsung,exynos-rng4.txt
+
SAMSUNG FRAMEBUFFER DRIVER
M: Jingoo Han <jingoohan1@gmail.com>
L: linux-fbdev@vger.kernel.org
diff --git a/drivers/char/hw_random/Kconfig b/drivers/char/hw_random/Kconfig
index 0cafe08919c9..bdae802e7154 100644
--- a/drivers/char/hw_random/Kconfig
+++ b/drivers/char/hw_random/Kconfig
@@ -294,20 +294,6 @@ config HW_RANDOM_POWERNV
If unsure, say Y.
-config HW_RANDOM_EXYNOS
- tristate "EXYNOS HW random number generator support"
- depends on ARCH_EXYNOS || COMPILE_TEST
- depends on HAS_IOMEM
- default HW_RANDOM
- ---help---
- This driver provides kernel-side support for the Random Number
- Generator hardware found on EXYNOS SOCs.
-
- To compile this driver as a module, choose M here: the
- module will be called exynos-rng.
-
- If unsure, say Y.
-
config HW_RANDOM_TPM
tristate "TPM HW Random Number Generator support"
depends on TCG_TPM
diff --git a/drivers/char/hw_random/Makefile b/drivers/char/hw_random/Makefile
index 5f52b1e4e7be..6f1eecc2045c 100644
--- a/drivers/char/hw_random/Makefile
+++ b/drivers/char/hw_random/Makefile
@@ -24,7 +24,6 @@ obj-$(CONFIG_HW_RANDOM_OCTEON) += octeon-rng.o
obj-$(CONFIG_HW_RANDOM_NOMADIK) += nomadik-rng.o
obj-$(CONFIG_HW_RANDOM_PSERIES) += pseries-rng.o
obj-$(CONFIG_HW_RANDOM_POWERNV) += powernv-rng.o
-obj-$(CONFIG_HW_RANDOM_EXYNOS) += exynos-rng.o
obj-$(CONFIG_HW_RANDOM_HISI) += hisi-rng.o
obj-$(CONFIG_HW_RANDOM_TPM) += tpm-rng.o
obj-$(CONFIG_HW_RANDOM_BCM2835) += bcm2835-rng.o
diff --git a/drivers/char/hw_random/exynos-rng.c b/drivers/char/hw_random/exynos-rng.c
deleted file mode 100644
index 23d358553b21..000000000000
--- a/drivers/char/hw_random/exynos-rng.c
+++ /dev/null
@@ -1,231 +0,0 @@
-/*
- * exynos-rng.c - Random Number Generator driver for the exynos
- *
- * Copyright (C) 2012 Samsung Electronics
- * Jonghwa Lee <jonghwa3.lee@samsung.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation;
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#include <linux/hw_random.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/io.h>
-#include <linux/platform_device.h>
-#include <linux/clk.h>
-#include <linux/pm_runtime.h>
-#include <linux/err.h>
-
-#define EXYNOS_PRNG_STATUS_OFFSET 0x10
-#define EXYNOS_PRNG_SEED_OFFSET 0x140
-#define EXYNOS_PRNG_OUT1_OFFSET 0x160
-#define SEED_SETTING_DONE BIT(1)
-#define PRNG_START 0x18
-#define PRNG_DONE BIT(5)
-#define EXYNOS_AUTOSUSPEND_DELAY 100
-
-struct exynos_rng {
- struct device *dev;
- struct hwrng rng;
- void __iomem *mem;
- struct clk *clk;
-};
-
-static u32 exynos_rng_readl(struct exynos_rng *rng, u32 offset)
-{
- return readl_relaxed(rng->mem + offset);
-}
-
-static void exynos_rng_writel(struct exynos_rng *rng, u32 val, u32 offset)
-{
- writel_relaxed(val, rng->mem + offset);
-}
-
-static int exynos_rng_configure(struct exynos_rng *exynos_rng)
-{
- int i;
- int ret = 0;
-
- for (i = 0 ; i < 5 ; i++)
- exynos_rng_writel(exynos_rng, jiffies,
- EXYNOS_PRNG_SEED_OFFSET + 4*i);
-
- if (!(exynos_rng_readl(exynos_rng, EXYNOS_PRNG_STATUS_OFFSET)
- & SEED_SETTING_DONE))
- ret = -EIO;
-
- return ret;
-}
-
-static int exynos_init(struct hwrng *rng)
-{
- struct exynos_rng *exynos_rng = container_of(rng,
- struct exynos_rng, rng);
- int ret = 0;
-
- pm_runtime_get_sync(exynos_rng->dev);
- ret = exynos_rng_configure(exynos_rng);
- pm_runtime_mark_last_busy(exynos_rng->dev);
- pm_runtime_put_autosuspend(exynos_rng->dev);
-
- return ret;
-}
-
-static int exynos_read(struct hwrng *rng, void *buf,
- size_t max, bool wait)
-{
- struct exynos_rng *exynos_rng = container_of(rng,
- struct exynos_rng, rng);
- u32 *data = buf;
- int retry = 100;
- int ret = 4;
-
- pm_runtime_get_sync(exynos_rng->dev);
-
- exynos_rng_writel(exynos_rng, PRNG_START, 0);
-
- while (!(exynos_rng_readl(exynos_rng,
- EXYNOS_PRNG_STATUS_OFFSET) & PRNG_DONE) && --retry)
- cpu_relax();
- if (!retry) {
- ret = -ETIMEDOUT;
- goto out;
- }
-
- exynos_rng_writel(exynos_rng, PRNG_DONE, EXYNOS_PRNG_STATUS_OFFSET);
-
- *data = exynos_rng_readl(exynos_rng, EXYNOS_PRNG_OUT1_OFFSET);
-
-out:
- pm_runtime_mark_last_busy(exynos_rng->dev);
- pm_runtime_put_sync_autosuspend(exynos_rng->dev);
-
- return ret;
-}
-
-static int exynos_rng_probe(struct platform_device *pdev)
-{
- struct exynos_rng *exynos_rng;
- struct resource *res;
- int ret;
-
- exynos_rng = devm_kzalloc(&pdev->dev, sizeof(struct exynos_rng),
- GFP_KERNEL);
- if (!exynos_rng)
- return -ENOMEM;
-
- exynos_rng->dev = &pdev->dev;
- exynos_rng->rng.name = "exynos";
- exynos_rng->rng.init = exynos_init;
- exynos_rng->rng.read = exynos_read;
- exynos_rng->clk = devm_clk_get(&pdev->dev, "secss");
- if (IS_ERR(exynos_rng->clk)) {
- dev_err(&pdev->dev, "Couldn't get clock.\n");
- return -ENOENT;
- }
-
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- exynos_rng->mem = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(exynos_rng->mem))
- return PTR_ERR(exynos_rng->mem);
-
- platform_set_drvdata(pdev, exynos_rng);
-
- pm_runtime_set_autosuspend_delay(&pdev->dev, EXYNOS_AUTOSUSPEND_DELAY);
- pm_runtime_use_autosuspend(&pdev->dev);
- pm_runtime_enable(&pdev->dev);
-
- ret = devm_hwrng_register(&pdev->dev, &exynos_rng->rng);
- if (ret) {
- pm_runtime_dont_use_autosuspend(&pdev->dev);
- pm_runtime_disable(&pdev->dev);
- }
-
- return ret;
-}
-
-static int exynos_rng_remove(struct platform_device *pdev)
-{
- pm_runtime_dont_use_autosuspend(&pdev->dev);
- pm_runtime_disable(&pdev->dev);
-
- return 0;
-}
-
-static int __maybe_unused exynos_rng_runtime_suspend(struct device *dev)
-{
- struct platform_device *pdev = to_platform_device(dev);
- struct exynos_rng *exynos_rng = platform_get_drvdata(pdev);
-
- clk_disable_unprepare(exynos_rng->clk);
-
- return 0;
-}
-
-static int __maybe_unused exynos_rng_runtime_resume(struct device *dev)
-{
- struct platform_device *pdev = to_platform_device(dev);
- struct exynos_rng *exynos_rng = platform_get_drvdata(pdev);
-
- return clk_prepare_enable(exynos_rng->clk);
-}
-
-static int __maybe_unused exynos_rng_suspend(struct device *dev)
-{
- return pm_runtime_force_suspend(dev);
-}
-
-static int __maybe_unused exynos_rng_resume(struct device *dev)
-{
- struct platform_device *pdev = to_platform_device(dev);
- struct exynos_rng *exynos_rng = platform_get_drvdata(pdev);
- int ret;
-
- ret = pm_runtime_force_resume(dev);
- if (ret)
- return ret;
-
- return exynos_rng_configure(exynos_rng);
-}
-
-static const struct dev_pm_ops exynos_rng_pm_ops = {
- SET_SYSTEM_SLEEP_PM_OPS(exynos_rng_suspend, exynos_rng_resume)
- SET_RUNTIME_PM_OPS(exynos_rng_runtime_suspend,
- exynos_rng_runtime_resume, NULL)
-};
-
-static const struct of_device_id exynos_rng_dt_match[] = {
- {
- .compatible = "samsung,exynos4-rng",
- },
- { },
-};
-MODULE_DEVICE_TABLE(of, exynos_rng_dt_match);
-
-static struct platform_driver exynos_rng_driver = {
- .driver = {
- .name = "exynos-rng",
- .pm = &exynos_rng_pm_ops,
- .of_match_table = exynos_rng_dt_match,
- },
- .probe = exynos_rng_probe,
- .remove = exynos_rng_remove,
-};
-
-module_platform_driver(exynos_rng_driver);
-
-MODULE_DESCRIPTION("EXYNOS 4 H/W Random Number Generator driver");
-MODULE_AUTHOR("Jonghwa Lee <jonghwa3.lee@samsung.com>");
-MODULE_LICENSE("GPL");
diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index 1a60626937e4..7d1fdf6a751c 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig
@@ -388,6 +388,21 @@ config CRYPTO_DEV_MXC_SCC
This option enables support for the Security Controller (SCC)
found in Freescale i.MX25 chips.
+config CRYPTO_DEV_EXYNOS_RNG
+ tristate "EXYNOS HW pseudo random number generator support"
+ depends on ARCH_EXYNOS || COMPILE_TEST
+ depends on HAS_IOMEM
+ select CRYPTO_RNG
+ ---help---
+ This driver provides kernel-side support through the
+ cryptographic API for the pseudo random number generator hardware
+ found on Exynos SoCs.
+
+ To compile this driver as a module, choose M here: the
+ module will be called exynos-rng.
+
+ If unsure, say Y.
+
config CRYPTO_DEV_S5P
tristate "Support for Samsung S5PV210/Exynos crypto accelerator"
depends on ARCH_S5PV210 || ARCH_EXYNOS || COMPILE_TEST
diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile
index 41ca339e89d3..9603f1862b30 100644
--- a/drivers/crypto/Makefile
+++ b/drivers/crypto/Makefile
@@ -6,6 +6,7 @@ obj-$(CONFIG_CRYPTO_DEV_CAVIUM_ZIP) += cavium/
obj-$(CONFIG_CRYPTO_DEV_CCP) += ccp/
obj-$(CONFIG_CRYPTO_DEV_CHELSIO) += chelsio/
obj-$(CONFIG_CRYPTO_DEV_CPT) += cavium/cpt/
+obj-$(CONFIG_CRYPTO_DEV_EXYNOS_RNG) += exynos-rng.o
obj-$(CONFIG_CRYPTO_DEV_FSL_CAAM) += caam/
obj-$(CONFIG_CRYPTO_DEV_GEODE) += geode-aes.o
obj-$(CONFIG_CRYPTO_DEV_HIFN_795X) += hifn_795x.o
diff --git a/drivers/crypto/exynos-rng.c b/drivers/crypto/exynos-rng.c
new file mode 100644
index 000000000000..280d0abe2b7a
--- /dev/null
+++ b/drivers/crypto/exynos-rng.c
@@ -0,0 +1,389 @@
+/*
+ * exynos-rng.c - Random Number Generator driver for the Exynos
+ *
+ * Copyright (c) 2017 Krzysztof Kozlowski <krzk@kernel.org>
+ *
+ * Loosely based on old driver from drivers/char/hw_random/exynos-rng.c:
+ * Copyright (C) 2012 Samsung Electronics
+ * Jonghwa Lee <jonghwa3.lee@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation;
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/clk.h>
+#include <linux/crypto.h>
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+
+#include <crypto/internal/rng.h>
+
+#define EXYNOS_RNG_CONTROL 0x0
+#define EXYNOS_RNG_STATUS 0x10
+#define EXYNOS_RNG_SEED_BASE 0x140
+#define EXYNOS_RNG_SEED(n) (EXYNOS_RNG_SEED_BASE + (n * 0x4))
+#define EXYNOS_RNG_OUT_BASE 0x160
+#define EXYNOS_RNG_OUT(n) (EXYNOS_RNG_OUT_BASE + (n * 0x4))
+
+/* EXYNOS_RNG_CONTROL bit fields */
+#define EXYNOS_RNG_CONTROL_START 0x18
+/* EXYNOS_RNG_STATUS bit fields */
+#define EXYNOS_RNG_STATUS_SEED_SETTING_DONE BIT(1)
+#define EXYNOS_RNG_STATUS_RNG_DONE BIT(5)
+
+/* Five seed and output registers, each 4 bytes */
+#define EXYNOS_RNG_SEED_REGS 5
+#define EXYNOS_RNG_SEED_SIZE (EXYNOS_RNG_SEED_REGS * 4)
+
+/*
+ * Driver re-seeds itself with generated random numbers to increase
+ * the randomness.
+ *
+ * Time for next re-seed in ms.
+ */
+#define EXYNOS_RNG_RESEED_TIME 100
+/*
+ * In polling mode, do not wait infinitely for the engine to finish the work.
+ */
+#define EXYNOS_RNG_WAIT_RETRIES 100
+
+/* Context for crypto */
+struct exynos_rng_ctx {
+ struct exynos_rng_dev *rng;
+};
+
+/* Device associated memory */
+struct exynos_rng_dev {
+ struct device *dev;
+ void __iomem *mem;
+ struct clk *clk;
+ /* Generated numbers stored for seeding during resume */
+ u8 seed_save[EXYNOS_RNG_SEED_SIZE];
+ unsigned int seed_save_len;
+ /* Time of last seeding in jiffies */
+ unsigned long last_seeding;
+};
+
+static struct exynos_rng_dev *exynos_rng_dev;
+
+static u32 exynos_rng_readl(struct exynos_rng_dev *rng, u32 offset)
+{
+ return readl_relaxed(rng->mem + offset);
+}
+
+static void exynos_rng_writel(struct exynos_rng_dev *rng, u32 val, u32 offset)
+{
+ writel_relaxed(val, rng->mem + offset);
+}
+
+static int exynos_rng_set_seed(struct exynos_rng_dev *rng,
+ const u8 *seed, unsigned int slen)
+{
+ u32 val;
+ int i;
+
+ /* Round seed length because loop iterates over full register size */
+ slen = ALIGN_DOWN(slen, 4);
+
+ if (slen < EXYNOS_RNG_SEED_SIZE)
+ return -EINVAL;
+
+ for (i = 0; i < slen ; i += 4) {
+ unsigned int seed_reg = (i / 4) % EXYNOS_RNG_SEED_REGS;
+
+ val = seed[i] << 24;
+ val |= seed[i + 1] << 16;
+ val |= seed[i + 2] << 8;
+ val |= seed[i + 3] << 0;
+
+ exynos_rng_writel(rng, val, EXYNOS_RNG_SEED(seed_reg));
+ }
+
+ val = exynos_rng_readl(rng, EXYNOS_RNG_STATUS);
+ if (!(val & EXYNOS_RNG_STATUS_SEED_SETTING_DONE)) {
+ dev_warn(rng->dev, "Seed setting not finished\n");
+ return -EIO;
+ }
+
+ rng->last_seeding = jiffies;
+
+ return 0;
+}
+
+/*
+ * Read from output registers and put the data under 'dst' array,
+ * up to dlen bytes.
+ *
+ * Returns number of bytes actually stored in 'dst' (dlen
+ * or EXYNOS_RNG_SEED_SIZE).
+ */
+static unsigned int exynos_rng_copy_random(struct exynos_rng_dev *rng,
+ u8 *dst, unsigned int dlen)
+{
+ unsigned int cnt = 0;
+ int i, j;
+ u32 val;
+
+ for (j = 0; j < EXYNOS_RNG_SEED_REGS; j++) {
+ val = exynos_rng_readl(rng, EXYNOS_RNG_OUT(j));
+
+ for (i = 0; i < 4; i++) {
+ dst[cnt] = val & 0xff;
+ val >>= 8;
+ if (++cnt >= dlen)
+ return cnt;
+ }
+ }
+
+ return cnt;
+}
+
+/*
+ * Start the engine and poll for finish. Then read from output registers
+ * filling the 'dst' buffer up to 'dlen' bytes or up to size of generated
+ * random data (EXYNOS_RNG_SEED_SIZE).
+ *
+ * On success: return 0 and store number of read bytes under 'read' address.
+ * On error: return -ERRNO.
+ */
+static int exynos_rng_get_random(struct exynos_rng_dev *rng,
+ u8 *dst, unsigned int dlen,
+ unsigned int *read)
+{
+ int retry = EXYNOS_RNG_WAIT_RETRIES;
+
+ exynos_rng_writel(rng, EXYNOS_RNG_CONTROL_START,
+ EXYNOS_RNG_CONTROL);
+
+ while (!(exynos_rng_readl(rng,
+ EXYNOS_RNG_STATUS) & EXYNOS_RNG_STATUS_RNG_DONE) && --retry)
+ cpu_relax();
+
+ if (!retry)
+ return -ETIMEDOUT;
+
+ /* Clear status bit */
+ exynos_rng_writel(rng, EXYNOS_RNG_STATUS_RNG_DONE,
+ EXYNOS_RNG_STATUS);
+ *read = exynos_rng_copy_random(rng, dst, dlen);
+
+ return 0;
+}
+
+/* Re-seed itself from time to time */
+static void exynos_rng_reseed(struct exynos_rng_dev *rng)
+{
+ unsigned long next_seeding = rng->last_seeding + \
+ msecs_to_jiffies(EXYNOS_RNG_RESEED_TIME);
+ unsigned long now = jiffies;
+ unsigned int read = 0;
+ u8 seed[EXYNOS_RNG_SEED_SIZE];
+
+ if (time_before(now, next_seeding))
+ return;
+
+ if (exynos_rng_get_random(rng, seed, sizeof(seed), &read))
+ return;
+
+ exynos_rng_set_seed(rng, seed, read);
+}
+
+static int exynos_rng_generate(struct crypto_rng *tfm,
+ const u8 *src, unsigned int slen,
+ u8 *dst, unsigned int dlen)
+{
+ struct exynos_rng_ctx *ctx = crypto_rng_ctx(tfm);
+ struct exynos_rng_dev *rng = ctx->rng;
+ unsigned int read = 0;
+ int ret;
+
+ ret = clk_prepare_enable(rng->clk);
+ if (ret)
+ return ret;
+
+ do {
+ ret = exynos_rng_get_random(rng, dst, dlen, &read);
+ if (ret)
+ break;
+
+ dlen -= read;
+ dst += read;
+
+ exynos_rng_reseed(rng);
+ } while (dlen > 0);
+
+ clk_disable_unprepare(rng->clk);
+
+ return ret;
+}
+
+static int exynos_rng_seed(struct crypto_rng *tfm, const u8 *seed,
+ unsigned int slen)
+{
+ struct exynos_rng_ctx *ctx = crypto_rng_ctx(tfm);
+ struct exynos_rng_dev *rng = ctx->rng;
+ int ret;
+
+ ret = clk_prepare_enable(rng->clk);
+ if (ret)
+ return ret;
+
+ ret = exynos_rng_set_seed(ctx->rng, seed, slen);
+
+ clk_disable_unprepare(rng->clk);
+
+ return ret;
+}
+
+static int exynos_rng_kcapi_init(struct crypto_tfm *tfm)
+{
+ struct exynos_rng_ctx *ctx = crypto_tfm_ctx(tfm);
+
+ ctx->rng = exynos_rng_dev;
+
+ return 0;
+}
+
+static struct rng_alg exynos_rng_alg = {
+ .generate = exynos_rng_generate,
+ .seed = exynos_rng_seed,
+ .seedsize = EXYNOS_RNG_SEED_SIZE,
+ .base = {
+ .cra_name = "exynos_rng",
+ .cra_driver_name = "exynos_rng",
+ .cra_priority = 100,
+ .cra_ctxsize = sizeof(struct exynos_rng_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_init = exynos_rng_kcapi_init,
+ }
+};
+
+static int exynos_rng_probe(struct platform_device *pdev)
+{
+ struct exynos_rng_dev *rng;
+ struct resource *res;
+ int ret;
+
+ if (exynos_rng_dev)
+ return -EEXIST;
+
+ rng = devm_kzalloc(&pdev->dev, sizeof(*rng), GFP_KERNEL);
+ if (!rng)
+ return -ENOMEM;
+
+ rng->dev = &pdev->dev;
+ rng->clk = devm_clk_get(&pdev->dev, "secss");
+ if (IS_ERR(rng->clk)) {
+ dev_err(&pdev->dev, "Couldn't get clock.\n");
+ return PTR_ERR(rng->clk);
+ }
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ rng->mem = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(rng->mem))
+ return PTR_ERR(rng->mem);
+
+ platform_set_drvdata(pdev, rng);
+
+ exynos_rng_dev = rng;
+
+ ret = crypto_register_rng(&exynos_rng_alg);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "Couldn't register rng crypto alg: %d\n", ret);
+ exynos_rng_dev = NULL;
+ }
+
+ return ret;
+}
+
+static int exynos_rng_remove(struct platform_device *pdev)
+{
+ crypto_unregister_rng(&exynos_rng_alg);
+
+ exynos_rng_dev = NULL;
+
+ return 0;
+}
+
+static int __maybe_unused exynos_rng_suspend(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct exynos_rng_dev *rng = platform_get_drvdata(pdev);
+ int ret;
+
+ /* If we were never seeded then after resume it will be the same */
+ if (!rng->last_seeding)
+ return 0;
+
+ rng->seed_save_len = 0;
+ ret = clk_prepare_enable(rng->clk);
+ if (ret)
+ return ret;
+
+ /* Get new random numbers and store them for seeding on resume. */
+ exynos_rng_get_random(rng, rng->seed_save, sizeof(rng->seed_save),
+ &(rng->seed_save_len));
+ dev_dbg(rng->dev, "Stored %u bytes for seeding on system resume\n",
+ rng->seed_save_len);
+
+ clk_disable_unprepare(rng->clk);
+
+ return 0;
+}
+
+static int __maybe_unused exynos_rng_resume(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct exynos_rng_dev *rng = platform_get_drvdata(pdev);
+ int ret;
+
+ /* Never seeded so nothing to do */
+ if (!rng->last_seeding)
+ return 0;
+
+ ret = clk_prepare_enable(rng->clk);
+ if (ret)
+ return ret;
+
+ ret = exynos_rng_set_seed(rng, rng->seed_save, rng->seed_save_len);
+
+ clk_disable_unprepare(rng->clk);
+
+ return ret;
+}
+
+static SIMPLE_DEV_PM_OPS(exynos_rng_pm_ops, exynos_rng_suspend,
+ exynos_rng_resume);
+
+static const struct of_device_id exynos_rng_dt_match[] = {
+ {
+ .compatible = "samsung,exynos4-rng",
+ },
+ { },
+};
+MODULE_DEVICE_TABLE(of, exynos_rng_dt_match);
+
+static struct platform_driver exynos_rng_driver = {
+ .driver = {
+ .name = "exynos-rng",
+ .pm = &exynos_rng_pm_ops,
+ .of_match_table = exynos_rng_dt_match,
+ },
+ .probe = exynos_rng_probe,
+ .remove = exynos_rng_remove,
+};
+
+module_platform_driver(exynos_rng_driver);
+
+MODULE_DESCRIPTION("Exynos H/W Random Number Generator driver");
+MODULE_AUTHOR("Krzysztof Kozlowski <krzk@kernel.org>");
+MODULE_LICENSE("GPL");
--
2.9.3
^ permalink raw reply related
* [PATCH v4 1/2] linux/kernel.h: Add ALIGN_DOWN macro
From: Krzysztof Kozlowski @ 2017-04-08 13:32 UTC (permalink / raw)
To: James Hogan, Matt Mackall, Herbert Xu, Kukjin Kim,
Krzysztof Kozlowski, Javier Martinez Canillas, David S. Miller,
Dave Airlie, David Airlie, Bernie Thompson,
Bartlomiej Zolnierkiewicz, Greg Kroah-Hartman,
PrasannaKumar Muralidharan, linux-kernel, linux-metag,
linux-crypto, linux-arm-kernel, linux-samsung-soc, dri-devel,
linux-fbdev
In-Reply-To: <20170408133245.30147-1-krzk@kernel.org>
Few parts of kernel define their own macro for aligning down so provide
a common define for this, with the same usage and assumptions as existing
ALIGN.
Convert also three existing implementations to this one.
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
---
The metag change was not even compiled due to lack of cross compiler.
---
arch/metag/kernel/stacktrace.c | 2 --
drivers/gpu/drm/udl/udl_fb.c | 2 +-
include/linux/kernel.h | 1 +
include/video/udlfb.h | 2 +-
4 files changed, 3 insertions(+), 4 deletions(-)
diff --git a/arch/metag/kernel/stacktrace.c b/arch/metag/kernel/stacktrace.c
index 91ffc4b75c33..09d67b7f51ca 100644
--- a/arch/metag/kernel/stacktrace.c
+++ b/arch/metag/kernel/stacktrace.c
@@ -31,8 +31,6 @@ static void tbi_boing_init(void)
}
#endif
-#define ALIGN_DOWN(addr, size) ((addr)&(~((size)-1)))
-
/*
* Unwind the current stack frame and store the new register values in the
* structure passed as argument. Unwinding is equivalent to a function return,
diff --git a/drivers/gpu/drm/udl/udl_fb.c b/drivers/gpu/drm/udl/udl_fb.c
index d05abc69e305..4a6500362564 100644
--- a/drivers/gpu/drm/udl/udl_fb.c
+++ b/drivers/gpu/drm/udl/udl_fb.c
@@ -37,7 +37,7 @@ struct udl_fbdev {
};
#define DL_ALIGN_UP(x, a) ALIGN(x, a)
-#define DL_ALIGN_DOWN(x, a) ALIGN(x-(a-1), a)
+#define DL_ALIGN_DOWN(x, a) ALIGN_DOWN(x, a)
/** Read the red component (0..255) of a 32 bpp colour. */
#define DLO_RGB_GETRED(col) (uint8_t)((col) & 0xFF)
diff --git a/include/linux/kernel.h b/include/linux/kernel.h
index 4c26dc3a8295..3d9f8420f973 100644
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -47,6 +47,7 @@
/* @a is a power of 2 value */
#define ALIGN(x, a) __ALIGN_KERNEL((x), (a))
+#define ALIGN_DOWN(x, a) __ALIGN_KERNEL((x) - ((a) - 1), (a))
#define __ALIGN_MASK(x, mask) __ALIGN_KERNEL_MASK((x), (mask))
#define PTR_ALIGN(p, a) ((typeof(p))ALIGN((unsigned long)(p), (a)))
#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
diff --git a/include/video/udlfb.h b/include/video/udlfb.h
index f9466fa54ba4..3ea90aea5617 100644
--- a/include/video/udlfb.h
+++ b/include/video/udlfb.h
@@ -92,6 +92,6 @@ struct dlfb_data {
/* remove these once align.h patch is taken into kernel */
#define DL_ALIGN_UP(x, a) ALIGN(x, a)
-#define DL_ALIGN_DOWN(x, a) ALIGN(x-(a-1), a)
+#define DL_ALIGN_DOWN(x, a) ALIGN_DOWN(x, a)
#endif
--
2.9.3
^ permalink raw reply related
* [PATCH v4 0/2] crypto: hw_random - Add new Exynos RNG driver
From: Krzysztof Kozlowski @ 2017-04-08 13:32 UTC (permalink / raw)
To: James Hogan, Matt Mackall, Herbert Xu, Kukjin Kim,
Krzysztof Kozlowski, Javier Martinez Canillas, David S. Miller,
Dave Airlie, David Airlie, Bernie Thompson,
Bartlomiej Zolnierkiewicz, Greg Kroah-Hartman,
PrasannaKumar Muralidharan, linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-metag-u79uwXL29TY76Z2rM5mHXA,
linux-crypto-u79uwXL29TY76Z2rM5mHXA,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-samsung-soc-u79uwXL29TY76Z2rM5mHXA,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-fbdev-u79uwXL29TY76Z2rM5mHXA
Hi,
This is a follow up of my questions around exynos-rng [1].
Changes since v3:
=================
1. New patch: 1/2 for ALIGN_DOWN macro. The change in metag architecture
was not compiled. Please test it.
2. Dropped patches touching ARM defconfig as they are not changing and
they pollute this submission.
3. Utilize all seed provided by kcapi (suggested by Stephan Müller).
4. Drop dev->ctx (suggested by PrasannaKumar Muralidharan).
5. Remove any printks from set_seed callback as this might be a way
for unprivileged user to pollute the log (suggested by Stephan).
6. Minor cleanups: initialize 'read' variable in exynos_rng_reseed()
for readability (it is not strictly required).
7. Add review tags from Stephen and PrasannaKumar.
Changes since v2:
=================
1. Do not re-use random numbers for re-seed (neither for system resume
nor for periodic re-seed). Instead the driver will just generate new
random numbers (suggested by Stephan Müller).
Suspend path tested with suspend-to-freeze, not real suspend. Testing
on Trats2 would be welcomed.
Changes since v1:
=================
1. Re-work the code for seeding after system resume, following suggestions
and review by Stephan Müller.
2. Re-seed itself from time to time (every 100 ms), suggested by Stephan
Müller.
3. Use a define for retries (Bartlomiej Zolnierkiewicz).
4. Add some docs.
Description:
============
The existing exynos-rng has many issues. The most important one is that
it is a pseudo RNG device but uses hw_random interface which does not allow
proper seeding.
The RNG module on Exynos4 requires seeding. On newer SoCs (like Exynos5420)
it can seed itself from a true RNG. Converting the existing driver
to use TRNG would effectively drop support for Exynos4 and break
compatibility with existing users.
Instead I decided to convert it to crypto API. In the future I hope
to add support for seeding from TRNG module.
Tested with app [2].
Patches are independent. I will take the defconfig changes (2/3 and 3/3)
through samsung-soc tree.
Best regards,
Krzysztof
[1] https://www.spinics.net/lists/arm-kernel/msg569641.html
[2] https://www.spinics.net/lists/arm-kernel/msg571184.html
Krzysztof Kozlowski (2):
linux/kernel.h: Add ALIGN_DOWN macro
crypto: hw_random - Add new Exynos RNG driver
MAINTAINERS | 8 +
arch/metag/kernel/stacktrace.c | 2 -
drivers/char/hw_random/Kconfig | 14 --
drivers/char/hw_random/Makefile | 1 -
drivers/char/hw_random/exynos-rng.c | 231 ---------------------
drivers/crypto/Kconfig | 15 ++
drivers/crypto/Makefile | 1 +
drivers/crypto/exynos-rng.c | 389 ++++++++++++++++++++++++++++++++++++
drivers/gpu/drm/udl/udl_fb.c | 2 +-
include/linux/kernel.h | 1 +
include/video/udlfb.h | 2 +-
11 files changed, 416 insertions(+), 250 deletions(-)
delete mode 100644 drivers/char/hw_random/exynos-rng.c
create mode 100644 drivers/crypto/exynos-rng.c
--
2.9.3
^ permalink raw reply
* Re: XTS Crypto Not Found In /proc/crypto Even After Compiled for 4.10.1.
From: Krzysztof Kozlowski @ 2017-04-08 12:23 UTC (permalink / raw)
To: Herbert Xu
Cc: Nathan Royce, davem, linux-crypto, linux-kernel, Marek Szyprowski
In-Reply-To: <20170408020246.GA4815@gondor.apana.org.au>
On Sat, Apr 08, 2017 at 10:02:46AM +0800, Herbert Xu wrote:
> On Thu, Apr 06, 2017 at 05:54:14PM +0800, Herbert Xu wrote:
> > On Mon, Mar 13, 2017 at 07:06:01PM +0200, Krzysztof Kozlowski wrote:
> > >
> > > I bisected this to commit f1c131b45410 ("crypto: xts - Convert to
> > > skcipher"). The s5p-sss driver stays the same... but the xts changes and
> > > as a result we have a NULL pointer dereference (actually of value
> > > 00000004):
> > > [ 18.930195] Unable to handle kernel NULL pointer dereference at virtual address 00000004
> > > ...
> > > [ 18.972325] [<c0313c98>] (post_crypt) from [<c031408c>] (decrypt_done+0x4c/0x54)
> > > [ 18.972343] [<c031408c>] (decrypt_done) from [<c056309c>] (s5p_aes_interrupt+0x1bc/0x208)
> > > [ 18.972360] [<c056309c>] (s5p_aes_interrupt) from [<c0164930>] (irq_thread_fn+0x1c/0x54)
> > >
> > > Any hints?
> >
> > I haven't found any smoking guns, but the locking between the
> > tasklet and the IRQ routine looks suspect. First of all the
> > tasklet is modifying the dev structure without holding any locks.
>
> I think I see the problem. Could you please try this patch and
> let me know if it fixes the crash?
Yes, fixed! Thanks.
Tested on Odroid XU3 with following script:
https://github.com/krzk/tools/blob/master/tests/s5p-sss-cryptsetup.sh
Tested-by: Krzysztof Kozlowski <krzk@kernel.org>
Best regards,
Krzysztof
^ permalink raw reply
* 21741 linux-crypto
From: membership @ 2017-04-08 4:25 UTC (permalink / raw)
To: linux-crypto
[-- Attachment #1: 525.zip --]
[-- Type: application/zip, Size: 3572 bytes --]
^ permalink raw reply
* Re: XTS Crypto Not Found In /proc/crypto Even After Compiled for 4.10.1.
From: Herbert Xu @ 2017-04-08 2:02 UTC (permalink / raw)
To: Krzysztof Kozlowski
Cc: Nathan Royce, davem, linux-crypto, linux-kernel, Marek Szyprowski
In-Reply-To: <20170406095414.GA31658@gondor.apana.org.au>
On Thu, Apr 06, 2017 at 05:54:14PM +0800, Herbert Xu wrote:
> On Mon, Mar 13, 2017 at 07:06:01PM +0200, Krzysztof Kozlowski wrote:
> >
> > I bisected this to commit f1c131b45410 ("crypto: xts - Convert to
> > skcipher"). The s5p-sss driver stays the same... but the xts changes and
> > as a result we have a NULL pointer dereference (actually of value
> > 00000004):
> > [ 18.930195] Unable to handle kernel NULL pointer dereference at virtual address 00000004
> > ...
> > [ 18.972325] [<c0313c98>] (post_crypt) from [<c031408c>] (decrypt_done+0x4c/0x54)
> > [ 18.972343] [<c031408c>] (decrypt_done) from [<c056309c>] (s5p_aes_interrupt+0x1bc/0x208)
> > [ 18.972360] [<c056309c>] (s5p_aes_interrupt) from [<c0164930>] (irq_thread_fn+0x1c/0x54)
> >
> > Any hints?
>
> I haven't found any smoking guns, but the locking between the
> tasklet and the IRQ routine looks suspect. First of all the
> tasklet is modifying the dev structure without holding any locks.
I think I see the problem. Could you please try this patch and
let me know if it fixes the crash?
---8<---
Subject: crypto: xts - Fix use-after-free on EINPROGRESS
When we get an EINPROGRESS completion in xts, we will end up marking
the request as done and freeing it. This then blows up when the
request is really completed as we've already freed the memory.
Fixes: f1c131b45410 ("crypto: xts - Convert to skcipher")
Cc: <stable@vger.kernel.org>
Reported-by: Nathan Royce <nroycea+kernel@gmail.com>
Reported-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
diff --git a/crypto/xts.c b/crypto/xts.c
index e197e64..d86c11a 100644
--- a/crypto/xts.c
+++ b/crypto/xts.c
@@ -286,6 +286,13 @@ static void encrypt_done(struct crypto_async_request *areq, int err)
struct rctx *rctx;
rctx = skcipher_request_ctx(req);
+
+ if (err == -EINPROGRESS) {
+ if (rctx->left != req->cryptlen)
+ return;
+ goto out;
+ }
+
subreq = &rctx->subreq;
subreq->base.flags &= CRYPTO_TFM_REQ_MAY_BACKLOG;
@@ -293,6 +300,7 @@ static void encrypt_done(struct crypto_async_request *areq, int err)
if (rctx->left)
return;
+out:
skcipher_request_complete(req, err);
}
@@ -330,6 +338,13 @@ static void decrypt_done(struct crypto_async_request *areq, int err)
struct rctx *rctx;
rctx = skcipher_request_ctx(req);
+
+ if (err == -EINPROGRESS) {
+ if (rctx->left != req->cryptlen)
+ return;
+ goto out;
+ }
+
subreq = &rctx->subreq;
subreq->base.flags &= CRYPTO_TFM_REQ_MAY_BACKLOG;
@@ -337,6 +352,7 @@ static void decrypt_done(struct crypto_async_request *areq, int err)
if (rctx->left)
return;
+out:
skcipher_request_complete(req, err);
}
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply related
* Re: What should be the algo priority
From: Hamid Nassiby @ 2017-04-07 18:13 UTC (permalink / raw)
To: Harsh Jain; +Cc: Stephan Müller, linux-crypto
In-Reply-To: <CAFXBA=mHp_LB9tufr-aRGXkfoULGbMn5RxRV1ajSozx88Er0CQ@mail.gmail.com>
"authenc" and "hmac" are templates, not different implementations of a cipher.
Please take a look at:
https://kernel.readthedocs.io/en/sphinx-samples/crypto-API.html#terminology
On Thu, Apr 6, 2017 at 9:56 AM, Harsh Jain <harshjain.prof@gmail.com> wrote:
> On Tue, Apr 4, 2017 at 6:07 PM, Stephan Müller <smueller@chronox.de> wrote:
>> Am Dienstag, 4. April 2017, 09:53:17 CEST schrieb Harsh Jain:
>>
>> Hi Harsh,
>>
>>> Hi,
>>>
>>> Do we have any guidelines documented to decide what should be the
>>> algorithm priority. Specially for authenc implementation.Most of the
>>> drivers have fixed priority for all algos. Problem comes in when we
>>> have cbc(aes), hmac(sha1) and authenc(cbc(aes),hmac(sha1))
>>> implementation in driver. Base authenc driver gets more precedence
>>> because of higher priority(enc->base.cra_priority * 10 +
>>> auth_base->cra_priority;)
>>>
>>> What should be the priority of
>>> cbc(aes),
>>> hmac(sha1)
>>> authenc(cbc(aes),hmac(sha1))
>>
>> There is no general rule about the actual numbers. But commonly, the prios are
>> set such that the prios of C implementations < ASM implementations < hardware
>> accelerators. The idea is to give users the fastest implementation there is
>> for his particular system.
>
> It means cbc, hmac should have smaller(nearly 10 times less) priority
> than their authenc implementation otherwise request will not offload
> to driver because sw authenc priority is (aes * 10 + hmac).
>
>>
>> Ciao
>> Stephan
^ permalink raw reply
* RE: [7/7] crypto: caam/qi - add ablkcipher and authenc algorithms
From: Laurentiu Tudor @ 2017-04-07 14:00 UTC (permalink / raw)
To: Michael Ellerman, Horia Geantă, Herbert Xu, Scott Wood,
Roy Pledge
Cc: Claudiu Manoil, Cristian Stoica, Dan Douglass,
linux-arm-kernel@lists.infradead.org, Vakul Garg,
linuxppc-dev@lists.ozlabs.org, David S. Miller,
Alexandru Porosanu, linux-crypto@vger.kernel.org
In-Reply-To: <87inmggxp7.fsf@concordia.ellerman.id.au>
-----Original Message-----
From: Michael Ellerman [mailto:mpe@ellerman.id.au]
Sent: Friday, April 07, 2017 4:22 PM
To: Laurentiu Tudor <laurentiu.tudor@nxp.com>; Horia Geantă <horia.geanta@nxp.com>; Herbert Xu <herbert@gondor.apana.org.au>; Scott Wood <oss@buserror.net>; Roy Pledge <roy.pledge@nxp.com>
Cc: Claudiu Manoil <claudiu.manoil@nxp.com>; Cristian Stoica <cristian.stoica@nxp.com>; Dan Douglass <dan.douglass@nxp.com>; linux-arm-kernel@lists.infradead.org; Vakul Garg <vakul.garg@nxp.com>; linuxppc-dev@lists.ozlabs.org; David S. Miller <davem@davemloft.net>; Alexandru Porosanu <alexandru.porosanu@nxp.com>; linux-crypto@vger.kernel.org
Subject: Re: [7/7] crypto: caam/qi - add ablkcipher and authenc algorithms
Importance: High
Laurentiu Tudor <laurentiu.tudor@nxp.com> writes:
> On 04/05/2017 01:06 PM, Michael Ellerman wrote:
>> Laurentiu Tudor <laurentiu.tudor@nxp.com> writes:
>>
>>> Hi Michael,
>>>
>>> Just a couple of basic things to check:
>>> - was the dtb updated to the newest?
>>
>> Possibly not, it's an automated build/boot, I'll have to check what
>> it does with the dtb.
>>
>>> - is the qman node present? This should be easily visible in
>>> /proc/device-tree/soc@ffe000000/qman@318000.
>>
>> No it's not there.
>>
>> That's running linux-next with:
>>
>> CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API_QI=n
>>
>>
>> Does that mean I didn't update the device tree?
>
> I think so. Also, I just checked that the node is actually there by
> compiling p5020ds.dts and then decompiling the dtb.
> OK, I'll make sure I update the DTB.
>
> It will still be good if the code was a bit more robust about the qman being missing.
Totally agree. We should handle this error condition.
---
Thanks & Best Regards, Laurentiu
^ permalink raw reply
* Re: [7/7] crypto: caam/qi - add ablkcipher and authenc algorithms
From: Michael Ellerman @ 2017-04-07 13:21 UTC (permalink / raw)
To: Laurentiu Tudor, Horia Geantă, Herbert Xu, Scott Wood,
Roy Pledge
Cc: Claudiu Manoil, Cristian Stoica, Dan Douglass,
linux-arm-kernel@lists.infradead.org, Vakul Garg,
linuxppc-dev@lists.ozlabs.org, David S. Miller,
Alexandru Porosanu, linux-crypto@vger.kernel.org
In-Reply-To: <58E4D9C3.4070406@nxp.com>
Laurentiu Tudor <laurentiu.tudor@nxp.com> writes:
> On 04/05/2017 01:06 PM, Michael Ellerman wrote:
>> Laurentiu Tudor <laurentiu.tudor@nxp.com> writes:
>>
>>> Hi Michael,
>>>
>>> Just a couple of basic things to check:
>>> - was the dtb updated to the newest?
>>
>> Possibly not, it's an automated build/boot, I'll have to check what it
>> does with the dtb.
>>
>>> - is the qman node present? This should be easily visible in
>>> /proc/device-tree/soc@ffe000000/qman@318000.
>>
>> No it's not there.
>>
>> That's running linux-next with:
>>
>> CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API_QI=n
>>
>>
>> Does that mean I didn't update the device tree?
>
> I think so. Also, I just checked that the node is actually there by
> compiling p5020ds.dts and then decompiling the dtb.
OK, I'll make sure I update the DTB.
It will still be good if the code was a bit more robust about the qman
being missing.
cheers
^ permalink raw reply
* [RFC PATCH v5] crypto: Add IV generation algorithms
From: Binoy Jayan @ 2017-04-07 10:47 UTC (permalink / raw)
To: Oded, Ofir
Cc: Herbert Xu, David S. Miller, linux-crypto, Mark Brown,
Arnd Bergmann, linux-kernel, Alasdair Kergon, Mike Snitzer,
dm-devel, Shaohua Li, linux-raid, Rajendra, Milan Broz, Gilad,
Binoy Jayan
In-Reply-To: <1491562064-23591-1-git-send-email-binoy.jayan@linaro.org>
Currently, the iv generation algorithms are implemented in dm-crypt.c.
The goal is to move these algorithms from the dm layer to the kernel
crypto layer by implementing them as template ciphers so they can be
implemented in hardware for performance. As part of this patchset, the
iv-generation code is moved from the dm layer to the crypto layer and
adapt the dm-layer to send a whole 'bio' (as defined in the block layer)
at a time. Each bio contains an in memory representation of physically
contiguous disk blocks. The dm layer sets up a chained scatterlist of
these blocks split into physically contiguous segments in memory so that
DMA can be performed. Also, the key management code is moved from dm layer
to the cryto layer since the key selection for encrypting neighboring
sectors depend on the keycount.
Synchronous crypto requests to encrypt/decrypt a sector are processed
sequentially. Asynchronous requests if processed in parallel, are freed
in the async callback. The dm layer allocates space for iv. The hardware
implementations can choose to make use of this space to generate their IVs
sequentially or allocate it on their own.
Interface to the crypto layer - include/crypto/geniv.h
Signed-off-by: Binoy Jayan <binoy.jayan@linaro.org>
---
drivers/md/dm-crypt.c | 1916 ++++++++++++++++++++++++++++++++++--------------
include/crypto/geniv.h | 47 ++
2 files changed, 1424 insertions(+), 539 deletions(-)
create mode 100644 include/crypto/geniv.h
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 389a363..ce2bb80 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -32,170 +32,113 @@
#include <crypto/algapi.h>
#include <crypto/skcipher.h>
#include <keys/user-type.h>
-
#include <linux/device-mapper.h>
-
-#define DM_MSG_PREFIX "crypt"
-
-/*
- * context holding the current state of a multi-part conversion
- */
-struct convert_context {
- struct completion restart;
- struct bio *bio_in;
- struct bio *bio_out;
- struct bvec_iter iter_in;
- struct bvec_iter iter_out;
- sector_t cc_sector;
- atomic_t cc_pending;
- struct skcipher_request *req;
+#include <crypto/internal/skcipher.h>
+#include <linux/backing-dev.h>
+#include <linux/log2.h>
+#include <crypto/geniv.h>
+
+#define DM_MSG_PREFIX "crypt"
+#define MAX_SG_LIST (BIO_MAX_PAGES * 8)
+#define MIN_IOS 64
+#define LMK_SEED_SIZE 64 /* hash + 0 */
+#define TCW_WHITENING_SIZE 16
+
+struct geniv_ctx;
+struct geniv_req_ctx;
+
+/* Sub request for each of the skcipher_request's for a segment */
+struct geniv_subreq {
+ struct scatterlist src;
+ struct scatterlist dst;
+ struct geniv_req_ctx *rctx;
+ struct skcipher_request req CRYPTO_MINALIGN_ATTR;
};
-/*
- * per bio private data
- */
-struct dm_crypt_io {
- struct crypt_config *cc;
- struct bio *base_bio;
- struct work_struct work;
-
- struct convert_context ctx;
-
- atomic_t io_pending;
- int error;
- sector_t sector;
-
- struct rb_node rb_node;
-} CRYPTO_MINALIGN_ATTR;
-
-struct dm_crypt_request {
- struct convert_context *ctx;
- struct scatterlist sg_in;
- struct scatterlist sg_out;
+struct geniv_req_ctx {
+ struct geniv_subreq *subreq;
+ int is_write;
sector_t iv_sector;
+ unsigned int nents;
+ u8 *iv;
+ struct completion restart;
+ atomic_t req_pending;
+ struct skcipher_request *req;
};
-struct crypt_config;
-
struct crypt_iv_operations {
- int (*ctr)(struct crypt_config *cc, struct dm_target *ti,
- const char *opts);
- void (*dtr)(struct crypt_config *cc);
- int (*init)(struct crypt_config *cc);
- int (*wipe)(struct crypt_config *cc);
- int (*generator)(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq);
- int (*post)(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq);
+ int (*ctr)(struct geniv_ctx *ctx);
+ void (*dtr)(struct geniv_ctx *ctx);
+ int (*init)(struct geniv_ctx *ctx);
+ int (*wipe)(struct geniv_ctx *ctx);
+ int (*generator)(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq);
+ int (*post)(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq);
};
-struct iv_essiv_private {
+struct geniv_essiv_private {
struct crypto_ahash *hash_tfm;
u8 *salt;
};
-struct iv_benbi_private {
+struct geniv_benbi_private {
int shift;
};
-#define LMK_SEED_SIZE 64 /* hash + 0 */
-struct iv_lmk_private {
+struct geniv_lmk_private {
struct crypto_shash *hash_tfm;
u8 *seed;
};
-#define TCW_WHITENING_SIZE 16
-struct iv_tcw_private {
+struct geniv_tcw_private {
struct crypto_shash *crc32_tfm;
u8 *iv_seed;
u8 *whitening;
};
-/*
- * Crypt: maps a linear range of a block device
- * and encrypts / decrypts at the same time.
- */
-enum flags { DM_CRYPT_SUSPENDED, DM_CRYPT_KEY_VALID,
- DM_CRYPT_SAME_CPU, DM_CRYPT_NO_OFFLOAD };
-
-/*
- * The fields in here must be read only after initialization.
- */
-struct crypt_config {
- struct dm_dev *dev;
- sector_t start;
-
- /*
- * pool for per bio private data, crypto requests and
- * encryption requeusts/buffer pages
- */
- mempool_t *req_pool;
- mempool_t *page_pool;
- struct bio_set *bs;
- struct mutex bio_alloc_lock;
-
- struct workqueue_struct *io_queue;
- struct workqueue_struct *crypt_queue;
-
- struct task_struct *write_thread;
- wait_queue_head_t write_thread_wait;
- struct rb_root write_tree;
-
+struct geniv_ctx {
+ unsigned int tfms_count;
+ struct crypto_skcipher *child;
+ struct crypto_skcipher **tfms;
+ char *ivmode;
+ unsigned int iv_size;
+ char *algname;
+ char *ivopts;
char *cipher;
- char *cipher_string;
- char *key_string;
-
+ char *ciphermode;
const struct crypt_iv_operations *iv_gen_ops;
union {
- struct iv_essiv_private essiv;
- struct iv_benbi_private benbi;
- struct iv_lmk_private lmk;
- struct iv_tcw_private tcw;
+ struct geniv_essiv_private essiv;
+ struct geniv_benbi_private benbi;
+ struct geniv_lmk_private lmk;
+ struct geniv_tcw_private tcw;
} iv_gen_private;
- sector_t iv_offset;
- unsigned int iv_size;
-
- /* ESSIV: struct crypto_cipher *essiv_tfm */
void *iv_private;
- struct crypto_skcipher **tfms;
- unsigned tfms_count;
-
- /*
- * Layout of each crypto request:
- *
- * struct skcipher_request
- * context
- * padding
- * struct dm_crypt_request
- * padding
- * IV
- *
- * The padding is added so that dm_crypt_request and the IV are
- * correctly aligned.
- */
- unsigned int dmreq_start;
-
- unsigned int per_bio_data_size;
-
- unsigned long flags;
+ struct crypto_skcipher *tfm;
+ mempool_t *subreq_pool;
unsigned int key_size;
+ unsigned int key_extra_size;
unsigned int key_parts; /* independent parts in key buffer */
- unsigned int key_extra_size; /* additional keys length */
- u8 key[0];
+ enum setkey_op keyop;
+ char *msg;
+ u8 *key;
};
-#define MIN_IOS 64
-
-static void clone_init(struct dm_crypt_io *, struct bio *);
-static void kcryptd_queue_crypt(struct dm_crypt_io *io);
-static u8 *iv_of_dmreq(struct crypt_config *cc, struct dm_crypt_request *dmreq);
+static struct crypto_skcipher *any_tfm(struct geniv_ctx *ctx)
+{
+ return ctx->tfms[0];
+}
-/*
- * Use this to access cipher attributes that are the same for each CPU.
- */
-static struct crypto_skcipher *any_tfm(struct crypt_config *cc)
+static inline
+struct geniv_req_ctx *geniv_req_ctx(struct skcipher_request *req)
{
- return cc->tfms[0];
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ unsigned long align = crypto_skcipher_alignmask(tfm);
+
+ return (void *) PTR_ALIGN((u8 *) skcipher_request_ctx(req), align + 1);
}
/*
@@ -245,44 +188,50 @@ static struct crypto_skcipher *any_tfm(struct crypt_config *cc)
* http://article.gmane.org/gmane.linux.kernel.device-mapper.dm-crypt/454
*/
-static int crypt_iv_plain_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_plain_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq)
{
- memset(iv, 0, cc->iv_size);
- *(__le32 *)iv = cpu_to_le32(dmreq->iv_sector & 0xffffffff);
+ u8 *iv = rctx->iv;
+
+ memset(iv, 0, ctx->iv_size);
+ *(__le32 *)iv = cpu_to_le32(rctx->iv_sector & 0xffffffff);
return 0;
}
-static int crypt_iv_plain64_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_plain64_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq)
{
- memset(iv, 0, cc->iv_size);
- *(__le64 *)iv = cpu_to_le64(dmreq->iv_sector);
+ u8 *iv = rctx->iv;
+
+ memset(iv, 0, ctx->iv_size);
+ *(__le64 *)iv = cpu_to_le64(rctx->iv_sector);
return 0;
}
/* Initialise ESSIV - compute salt but no local memory allocations */
-static int crypt_iv_essiv_init(struct crypt_config *cc)
+static int crypt_iv_essiv_init(struct geniv_ctx *ctx)
{
- struct iv_essiv_private *essiv = &cc->iv_gen_private.essiv;
- AHASH_REQUEST_ON_STACK(req, essiv->hash_tfm);
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
struct scatterlist sg;
struct crypto_cipher *essiv_tfm;
int err;
+ AHASH_REQUEST_ON_STACK(req, essiv->hash_tfm);
- sg_init_one(&sg, cc->key, cc->key_size);
+ sg_init_one(&sg, ctx->key, ctx->key_size);
ahash_request_set_tfm(req, essiv->hash_tfm);
ahash_request_set_callback(req, CRYPTO_TFM_REQ_MAY_SLEEP, NULL, NULL);
- ahash_request_set_crypt(req, &sg, essiv->salt, cc->key_size);
+ ahash_request_set_crypt(req, &sg, essiv->salt, ctx->key_size);
err = crypto_ahash_digest(req);
ahash_request_zero(req);
if (err)
return err;
- essiv_tfm = cc->iv_private;
+ essiv_tfm = ctx->iv_private;
err = crypto_cipher_setkey(essiv_tfm, essiv->salt,
crypto_ahash_digestsize(essiv->hash_tfm));
@@ -293,16 +242,16 @@ static int crypt_iv_essiv_init(struct crypt_config *cc)
}
/* Wipe salt and reset key derived from volume key */
-static int crypt_iv_essiv_wipe(struct crypt_config *cc)
+static int crypt_iv_essiv_wipe(struct geniv_ctx *ctx)
{
- struct iv_essiv_private *essiv = &cc->iv_gen_private.essiv;
- unsigned salt_size = crypto_ahash_digestsize(essiv->hash_tfm);
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
+ unsigned int salt_size = crypto_ahash_digestsize(essiv->hash_tfm);
struct crypto_cipher *essiv_tfm;
int r, err = 0;
memset(essiv->salt, 0, salt_size);
- essiv_tfm = cc->iv_private;
+ essiv_tfm = ctx->iv_private;
r = crypto_cipher_setkey(essiv_tfm, essiv->salt, salt_size);
if (r)
err = r;
@@ -311,42 +260,40 @@ static int crypt_iv_essiv_wipe(struct crypt_config *cc)
}
/* Set up per cpu cipher state */
-static struct crypto_cipher *setup_essiv_cpu(struct crypt_config *cc,
- struct dm_target *ti,
- u8 *salt, unsigned saltsize)
+static struct crypto_cipher *setup_essiv_cpu(struct geniv_ctx *ctx,
+ u8 *salt, unsigned int saltsize)
{
struct crypto_cipher *essiv_tfm;
int err;
/* Setup the essiv_tfm with the given salt */
- essiv_tfm = crypto_alloc_cipher(cc->cipher, 0, CRYPTO_ALG_ASYNC);
+ essiv_tfm = crypto_alloc_cipher(ctx->cipher, 0, CRYPTO_ALG_ASYNC);
+
if (IS_ERR(essiv_tfm)) {
- ti->error = "Error allocating crypto tfm for ESSIV";
+ DMERR("Error allocating crypto tfm for ESSIV\n");
return essiv_tfm;
}
if (crypto_cipher_blocksize(essiv_tfm) !=
- crypto_skcipher_ivsize(any_tfm(cc))) {
- ti->error = "Block size of ESSIV cipher does "
- "not match IV size of block cipher";
+ crypto_skcipher_ivsize(any_tfm(ctx))) {
+ DMERR("Block size of ESSIV cipher does not match IV size of block cipher\n");
crypto_free_cipher(essiv_tfm);
return ERR_PTR(-EINVAL);
}
err = crypto_cipher_setkey(essiv_tfm, salt, saltsize);
if (err) {
- ti->error = "Failed to set key for ESSIV cipher";
+ DMERR("Failed to set key for ESSIV cipher\n");
crypto_free_cipher(essiv_tfm);
return ERR_PTR(err);
}
-
return essiv_tfm;
}
-static void crypt_iv_essiv_dtr(struct crypt_config *cc)
+static void crypt_iv_essiv_dtr(struct geniv_ctx *ctx)
{
struct crypto_cipher *essiv_tfm;
- struct iv_essiv_private *essiv = &cc->iv_gen_private.essiv;
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
crypto_free_ahash(essiv->hash_tfm);
essiv->hash_tfm = NULL;
@@ -354,52 +301,50 @@ static void crypt_iv_essiv_dtr(struct crypt_config *cc)
kzfree(essiv->salt);
essiv->salt = NULL;
- essiv_tfm = cc->iv_private;
+ essiv_tfm = ctx->iv_private;
if (essiv_tfm)
crypto_free_cipher(essiv_tfm);
- cc->iv_private = NULL;
+ ctx->iv_private = NULL;
}
-static int crypt_iv_essiv_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
+static int crypt_iv_essiv_ctr(struct geniv_ctx *ctx)
{
struct crypto_cipher *essiv_tfm = NULL;
struct crypto_ahash *hash_tfm = NULL;
u8 *salt = NULL;
int err;
- if (!opts) {
- ti->error = "Digest algorithm missing for ESSIV mode";
+ if (!ctx->ivopts) {
+ DMERR("Digest algorithm missing for ESSIV mode\n");
return -EINVAL;
}
/* Allocate hash algorithm */
- hash_tfm = crypto_alloc_ahash(opts, 0, CRYPTO_ALG_ASYNC);
+ hash_tfm = crypto_alloc_ahash(ctx->ivopts, 0, CRYPTO_ALG_ASYNC);
if (IS_ERR(hash_tfm)) {
- ti->error = "Error initializing ESSIV hash";
err = PTR_ERR(hash_tfm);
+ DMERR("Error initializing ESSIV hash. err=%d\n", err);
goto bad;
}
salt = kzalloc(crypto_ahash_digestsize(hash_tfm), GFP_KERNEL);
if (!salt) {
- ti->error = "Error kmallocing salt storage in ESSIV";
err = -ENOMEM;
goto bad;
}
- cc->iv_gen_private.essiv.salt = salt;
- cc->iv_gen_private.essiv.hash_tfm = hash_tfm;
+ ctx->iv_gen_private.essiv.salt = salt;
+ ctx->iv_gen_private.essiv.hash_tfm = hash_tfm;
- essiv_tfm = setup_essiv_cpu(cc, ti, salt,
+ essiv_tfm = setup_essiv_cpu(ctx, salt,
crypto_ahash_digestsize(hash_tfm));
if (IS_ERR(essiv_tfm)) {
- crypt_iv_essiv_dtr(cc);
+ crypt_iv_essiv_dtr(ctx);
return PTR_ERR(essiv_tfm);
}
- cc->iv_private = essiv_tfm;
+ ctx->iv_private = essiv_tfm;
return 0;
@@ -410,70 +355,73 @@ static int crypt_iv_essiv_ctr(struct crypt_config *cc, struct dm_target *ti,
return err;
}
-static int crypt_iv_essiv_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_essiv_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq)
{
- struct crypto_cipher *essiv_tfm = cc->iv_private;
+ u8 *iv = rctx->iv;
+ struct crypto_cipher *essiv_tfm = ctx->iv_private;
- memset(iv, 0, cc->iv_size);
- *(__le64 *)iv = cpu_to_le64(dmreq->iv_sector);
+ memset(iv, 0, ctx->iv_size);
+ *(__le64 *)iv = cpu_to_le64(rctx->iv_sector);
crypto_cipher_encrypt_one(essiv_tfm, iv, iv);
return 0;
}
-static int crypt_iv_benbi_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
+static int crypt_iv_benbi_ctr(struct geniv_ctx *ctx)
{
- unsigned bs = crypto_skcipher_blocksize(any_tfm(cc));
+ unsigned int bs = crypto_skcipher_blocksize(any_tfm(ctx));
int log = ilog2(bs);
/* we need to calculate how far we must shift the sector count
- * to get the cipher block count, we use this shift in _gen */
+ * to get the cipher block count, we use this shift in _gen
+ */
if (1 << log != bs) {
- ti->error = "cypher blocksize is not a power of 2";
+ DMERR("cypher blocksize is not a power of 2\n");
return -EINVAL;
}
if (log > 9) {
- ti->error = "cypher blocksize is > 512";
+ DMERR("cypher blocksize is > 512\n");
return -EINVAL;
}
- cc->iv_gen_private.benbi.shift = 9 - log;
+ ctx->iv_gen_private.benbi.shift = 9 - log;
return 0;
}
-static void crypt_iv_benbi_dtr(struct crypt_config *cc)
-{
-}
-
-static int crypt_iv_benbi_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_benbi_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq)
{
+ u8 *iv = rctx->iv;
__be64 val;
- memset(iv, 0, cc->iv_size - sizeof(u64)); /* rest is cleared below */
+ memset(iv, 0, ctx->iv_size - sizeof(u64)); /* rest is cleared below */
- val = cpu_to_be64(((u64)dmreq->iv_sector << cc->iv_gen_private.benbi.shift) + 1);
- put_unaligned(val, (__be64 *)(iv + cc->iv_size - sizeof(u64)));
+ val = cpu_to_be64(((u64) rctx->iv_sector <<
+ ctx->iv_gen_private.benbi.shift) + 1);
+ put_unaligned(val, (__be64 *)(iv + ctx->iv_size - sizeof(u64)));
return 0;
}
-static int crypt_iv_null_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_null_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq)
{
- memset(iv, 0, cc->iv_size);
+ u8 *iv = rctx->iv;
+ memset(iv, 0, ctx->iv_size);
return 0;
}
-static void crypt_iv_lmk_dtr(struct crypt_config *cc)
+static void crypt_iv_lmk_dtr(struct geniv_ctx *ctx)
{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
if (lmk->hash_tfm && !IS_ERR(lmk->hash_tfm))
crypto_free_shash(lmk->hash_tfm);
@@ -483,49 +431,49 @@ static void crypt_iv_lmk_dtr(struct crypt_config *cc)
lmk->seed = NULL;
}
-static int crypt_iv_lmk_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
+static int crypt_iv_lmk_ctr(struct geniv_ctx *ctx)
{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
lmk->hash_tfm = crypto_alloc_shash("md5", 0, 0);
if (IS_ERR(lmk->hash_tfm)) {
- ti->error = "Error initializing LMK hash";
+ DMERR("Error initializing LMK hash; err=%ld\n",
+ PTR_ERR(lmk->hash_tfm));
return PTR_ERR(lmk->hash_tfm);
}
/* No seed in LMK version 2 */
- if (cc->key_parts == cc->tfms_count) {
+ if (ctx->key_parts == ctx->tfms_count) {
lmk->seed = NULL;
return 0;
}
lmk->seed = kzalloc(LMK_SEED_SIZE, GFP_KERNEL);
if (!lmk->seed) {
- crypt_iv_lmk_dtr(cc);
- ti->error = "Error kmallocing seed storage in LMK";
+ crypt_iv_lmk_dtr(ctx);
+ DMERR("Error kmallocing seed storage in LMK\n");
return -ENOMEM;
}
return 0;
}
-static int crypt_iv_lmk_init(struct crypt_config *cc)
+static int crypt_iv_lmk_init(struct geniv_ctx *ctx)
{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
- int subkey_size = cc->key_size / cc->key_parts;
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
+ int subkey_size = ctx->key_size / ctx->key_parts;
/* LMK seed is on the position of LMK_KEYS + 1 key */
if (lmk->seed)
- memcpy(lmk->seed, cc->key + (cc->tfms_count * subkey_size),
+ memcpy(lmk->seed, ctx->key + (ctx->tfms_count * subkey_size),
crypto_shash_digestsize(lmk->hash_tfm));
return 0;
}
-static int crypt_iv_lmk_wipe(struct crypt_config *cc)
+static int crypt_iv_lmk_wipe(struct geniv_ctx *ctx)
{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
if (lmk->seed)
memset(lmk->seed, 0, LMK_SEED_SIZE);
@@ -533,15 +481,14 @@ static int crypt_iv_lmk_wipe(struct crypt_config *cc)
return 0;
}
-static int crypt_iv_lmk_one(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq,
- u8 *data)
+static int crypt_iv_lmk_one(struct geniv_ctx *ctx, u8 *iv,
+ struct geniv_req_ctx *rctx, u8 *data)
{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
- SHASH_DESC_ON_STACK(desc, lmk->hash_tfm);
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
struct md5_state md5state;
__le32 buf[4];
int i, r;
+ SHASH_DESC_ON_STACK(desc, lmk->hash_tfm);
desc->tfm = lmk->hash_tfm;
desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP;
@@ -562,8 +509,9 @@ static int crypt_iv_lmk_one(struct crypt_config *cc, u8 *iv,
return r;
/* Sector is cropped to 56 bits here */
- buf[0] = cpu_to_le32(dmreq->iv_sector & 0xFFFFFFFF);
- buf[1] = cpu_to_le32((((u64)dmreq->iv_sector >> 32) & 0x00FFFFFF) | 0x80000000);
+ buf[0] = cpu_to_le32(rctx->iv_sector & 0xFFFFFFFF);
+ buf[1] = cpu_to_le32((((u64)rctx->iv_sector >> 32) & 0x00FFFFFF)
+ | 0x80000000);
buf[2] = cpu_to_le32(4024);
buf[3] = 0;
r = crypto_shash_update(desc, (u8 *)buf, sizeof(buf));
@@ -577,50 +525,54 @@ static int crypt_iv_lmk_one(struct crypt_config *cc, u8 *iv,
for (i = 0; i < MD5_HASH_WORDS; i++)
__cpu_to_le32s(&md5state.hash[i]);
- memcpy(iv, &md5state.hash, cc->iv_size);
+ memcpy(iv, &md5state.hash, ctx->iv_size);
return 0;
}
-static int crypt_iv_lmk_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_lmk_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq)
{
u8 *src;
+ u8 *iv = rctx->iv;
int r = 0;
- if (bio_data_dir(dmreq->ctx->bio_in) == WRITE) {
- src = kmap_atomic(sg_page(&dmreq->sg_in));
- r = crypt_iv_lmk_one(cc, iv, dmreq, src + dmreq->sg_in.offset);
+ if (rctx->is_write) {
+ src = kmap_atomic(sg_page(&subreq->src));
+ r = crypt_iv_lmk_one(ctx, iv, rctx, src + subreq->src.offset);
kunmap_atomic(src);
} else
- memset(iv, 0, cc->iv_size);
+ memset(iv, 0, ctx->iv_size);
return r;
}
-static int crypt_iv_lmk_post(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_lmk_post(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq)
{
u8 *dst;
+ u8 *iv = rctx->iv;
int r;
- if (bio_data_dir(dmreq->ctx->bio_in) == WRITE)
+ if (rctx->is_write)
return 0;
- dst = kmap_atomic(sg_page(&dmreq->sg_out));
- r = crypt_iv_lmk_one(cc, iv, dmreq, dst + dmreq->sg_out.offset);
+ dst = kmap_atomic(sg_page(&subreq->dst));
+ r = crypt_iv_lmk_one(ctx, iv, rctx, dst + subreq->dst.offset);
/* Tweak the first block of plaintext sector */
if (!r)
- crypto_xor(dst + dmreq->sg_out.offset, iv, cc->iv_size);
+ crypto_xor(dst + subreq->dst.offset, iv, ctx->iv_size);
kunmap_atomic(dst);
return r;
}
-static void crypt_iv_tcw_dtr(struct crypt_config *cc)
+static void crypt_iv_tcw_dtr(struct geniv_ctx *ctx)
{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
kzfree(tcw->iv_seed);
tcw->iv_seed = NULL;
@@ -632,64 +584,65 @@ static void crypt_iv_tcw_dtr(struct crypt_config *cc)
tcw->crc32_tfm = NULL;
}
-static int crypt_iv_tcw_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
+static int crypt_iv_tcw_ctr(struct geniv_ctx *ctx)
{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
- if (cc->key_size <= (cc->iv_size + TCW_WHITENING_SIZE)) {
- ti->error = "Wrong key size for TCW";
+ if (ctx->key_size <= (ctx->iv_size + TCW_WHITENING_SIZE)) {
+ DMERR("Wrong key size (%d) for TCW. Choose a value > %d bytes\n",
+ ctx->key_size,
+ ctx->iv_size + TCW_WHITENING_SIZE);
return -EINVAL;
}
tcw->crc32_tfm = crypto_alloc_shash("crc32", 0, 0);
if (IS_ERR(tcw->crc32_tfm)) {
- ti->error = "Error initializing CRC32 in TCW";
+ DMERR("Error initializing CRC32 in TCW; err=%ld\n",
+ PTR_ERR(tcw->crc32_tfm));
return PTR_ERR(tcw->crc32_tfm);
}
- tcw->iv_seed = kzalloc(cc->iv_size, GFP_KERNEL);
+ tcw->iv_seed = kzalloc(ctx->iv_size, GFP_KERNEL);
tcw->whitening = kzalloc(TCW_WHITENING_SIZE, GFP_KERNEL);
if (!tcw->iv_seed || !tcw->whitening) {
- crypt_iv_tcw_dtr(cc);
- ti->error = "Error allocating seed storage in TCW";
+ crypt_iv_tcw_dtr(ctx);
+ DMERR("Error allocating seed storage in TCW\n");
return -ENOMEM;
}
return 0;
}
-static int crypt_iv_tcw_init(struct crypt_config *cc)
+static int crypt_iv_tcw_init(struct geniv_ctx *ctx)
{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
- int key_offset = cc->key_size - cc->iv_size - TCW_WHITENING_SIZE;
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+ int key_offset = ctx->key_size - ctx->iv_size - TCW_WHITENING_SIZE;
- memcpy(tcw->iv_seed, &cc->key[key_offset], cc->iv_size);
- memcpy(tcw->whitening, &cc->key[key_offset + cc->iv_size],
+ memcpy(tcw->iv_seed, &ctx->key[key_offset], ctx->iv_size);
+ memcpy(tcw->whitening, &ctx->key[key_offset + ctx->iv_size],
TCW_WHITENING_SIZE);
return 0;
}
-static int crypt_iv_tcw_wipe(struct crypt_config *cc)
+static int crypt_iv_tcw_wipe(struct geniv_ctx *ctx)
{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
- memset(tcw->iv_seed, 0, cc->iv_size);
+ memset(tcw->iv_seed, 0, ctx->iv_size);
memset(tcw->whitening, 0, TCW_WHITENING_SIZE);
return 0;
}
-static int crypt_iv_tcw_whitening(struct crypt_config *cc,
- struct dm_crypt_request *dmreq,
- u8 *data)
+static int crypt_iv_tcw_whitening(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx, u8 *data)
{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
- __le64 sector = cpu_to_le64(dmreq->iv_sector);
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+ __le64 sector = cpu_to_le64(rctx->iv_sector);
u8 buf[TCW_WHITENING_SIZE];
- SHASH_DESC_ON_STACK(desc, tcw->crc32_tfm);
int i, r;
+ SHASH_DESC_ON_STACK(desc, tcw->crc32_tfm);
/* xor whitening with sector number */
memcpy(buf, tcw->whitening, TCW_WHITENING_SIZE);
@@ -713,99 +666,1032 @@ static int crypt_iv_tcw_whitening(struct crypt_config *cc,
crypto_xor(&buf[0], &buf[12], 4);
crypto_xor(&buf[4], &buf[8], 4);
- /* apply whitening (8 bytes) to whole sector */
- for (i = 0; i < ((1 << SECTOR_SHIFT) / 8); i++)
- crypto_xor(data + i * 8, buf, 8);
-out:
- memzero_explicit(buf, sizeof(buf));
- return r;
-}
+ /* apply whitening (8 bytes) to whole sector */
+ for (i = 0; i < (SECTOR_SIZE / 8); i++)
+ crypto_xor(data + i * 8, buf, 8);
+out:
+ memzero_explicit(buf, sizeof(buf));
+ return r;
+}
+
+static int crypt_iv_tcw_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq)
+{
+ u8 *iv = rctx->iv;
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+ __le64 sector = cpu_to_le64(rctx->iv_sector);
+ u8 *src;
+ int r = 0;
+
+ /* Remove whitening from ciphertext */
+ if (!rctx->is_write) {
+ src = kmap_atomic(sg_page(&subreq->src));
+ r = crypt_iv_tcw_whitening(ctx, rctx,
+ src + subreq->src.offset);
+ kunmap_atomic(src);
+ }
+
+ /* Calculate IV */
+ memcpy(iv, tcw->iv_seed, ctx->iv_size);
+ crypto_xor(iv, (u8 *)§or, 8);
+ if (ctx->iv_size > 8)
+ crypto_xor(&iv[8], (u8 *)§or, ctx->iv_size - 8);
+
+ return r;
+}
+
+static int crypt_iv_tcw_post(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq)
+{
+ u8 *dst;
+ int r;
+
+ if (!rctx->is_write)
+ return 0;
+
+ /* Apply whitening on ciphertext */
+ dst = kmap_atomic(sg_page(&subreq->dst));
+ r = crypt_iv_tcw_whitening(ctx, rctx, dst + subreq->dst.offset);
+ kunmap_atomic(dst);
+
+ return r;
+}
+
+static const struct crypt_iv_operations crypt_iv_plain_ops = {
+ .generator = crypt_iv_plain_gen
+};
+
+static const struct crypt_iv_operations crypt_iv_plain64_ops = {
+ .generator = crypt_iv_plain64_gen
+};
+
+static const struct crypt_iv_operations crypt_iv_essiv_ops = {
+ .ctr = crypt_iv_essiv_ctr,
+ .dtr = crypt_iv_essiv_dtr,
+ .init = crypt_iv_essiv_init,
+ .wipe = crypt_iv_essiv_wipe,
+ .generator = crypt_iv_essiv_gen
+};
+
+static const struct crypt_iv_operations crypt_iv_benbi_ops = {
+ .ctr = crypt_iv_benbi_ctr,
+ .generator = crypt_iv_benbi_gen
+};
+
+static const struct crypt_iv_operations crypt_iv_null_ops = {
+ .generator = crypt_iv_null_gen
+};
+
+static const struct crypt_iv_operations crypt_iv_lmk_ops = {
+ .ctr = crypt_iv_lmk_ctr,
+ .dtr = crypt_iv_lmk_dtr,
+ .init = crypt_iv_lmk_init,
+ .wipe = crypt_iv_lmk_wipe,
+ .generator = crypt_iv_lmk_gen,
+ .post = crypt_iv_lmk_post
+};
+
+static const struct crypt_iv_operations crypt_iv_tcw_ops = {
+ .ctr = crypt_iv_tcw_ctr,
+ .dtr = crypt_iv_tcw_dtr,
+ .init = crypt_iv_tcw_init,
+ .wipe = crypt_iv_tcw_wipe,
+ .generator = crypt_iv_tcw_gen,
+ .post = crypt_iv_tcw_post
+};
+
+static int geniv_setkey_set(struct geniv_ctx *ctx)
+{
+ int ret = 0;
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->init)
+ ret = ctx->iv_gen_ops->init(ctx);
+ return ret;
+}
+
+static int geniv_setkey_wipe(struct geniv_ctx *ctx)
+{
+ int ret = 0;
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->wipe) {
+ ret = ctx->iv_gen_ops->wipe(ctx);
+ if (ret)
+ return ret;
+ }
+ return ret;
+}
+
+static int geniv_init_iv(struct geniv_ctx *ctx)
+{
+ int ret = -EINVAL;
+
+ DMDEBUG("IV Generation algorithm : %s\n", ctx->ivmode);
+
+ if (ctx->ivmode == NULL)
+ ctx->iv_gen_ops = NULL;
+ else if (strcmp(ctx->ivmode, "plain") == 0)
+ ctx->iv_gen_ops = &crypt_iv_plain_ops;
+ else if (strcmp(ctx->ivmode, "plain64") == 0)
+ ctx->iv_gen_ops = &crypt_iv_plain64_ops;
+ else if (strcmp(ctx->ivmode, "essiv") == 0)
+ ctx->iv_gen_ops = &crypt_iv_essiv_ops;
+ else if (strcmp(ctx->ivmode, "benbi") == 0)
+ ctx->iv_gen_ops = &crypt_iv_benbi_ops;
+ else if (strcmp(ctx->ivmode, "null") == 0)
+ ctx->iv_gen_ops = &crypt_iv_null_ops;
+ else if (strcmp(ctx->ivmode, "lmk") == 0)
+ ctx->iv_gen_ops = &crypt_iv_lmk_ops;
+ else if (strcmp(ctx->ivmode, "tcw") == 0) {
+ ctx->iv_gen_ops = &crypt_iv_tcw_ops;
+ ctx->key_parts += 2; /* IV + whitening */
+ ctx->key_extra_size = ctx->iv_size + TCW_WHITENING_SIZE;
+ } else {
+ ret = -EINVAL;
+ DMERR("Invalid IV mode %s\n", ctx->ivmode);
+ goto end;
+ }
+
+ /* Allocate IV */
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->ctr) {
+ ret = ctx->iv_gen_ops->ctr(ctx);
+ if (ret < 0) {
+ DMERR("Error creating IV for %s\n", ctx->ivmode);
+ goto end;
+ }
+ }
+
+ /* Initialize IV (set keys for ESSIV etc) */
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->init) {
+ ret = ctx->iv_gen_ops->init(ctx);
+ if (ret < 0)
+ DMERR("Error creating IV for %s\n", ctx->ivmode);
+ }
+ ret = 0;
+end:
+ return ret;
+}
+
+static void geniv_free_tfms(struct geniv_ctx *ctx)
+{
+ unsigned int i;
+
+ if (!ctx->tfms)
+ return;
+
+ for (i = 0; i < ctx->tfms_count; i++)
+ if (ctx->tfms[i] && !IS_ERR(ctx->tfms[i])) {
+ crypto_free_skcipher(ctx->tfms[i]);
+ ctx->tfms[i] = NULL;
+ }
+
+ kfree(ctx->tfms);
+ ctx->tfms = NULL;
+}
+
+/* Allocate memory for the underlying cipher algorithm. Ex: cbc(aes)
+ */
+
+static int geniv_alloc_tfms(struct crypto_skcipher *parent,
+ struct geniv_ctx *ctx)
+{
+ unsigned int i, reqsize, align;
+ int err = 0;
+
+ ctx->tfms = kcalloc(ctx->tfms_count, sizeof(struct crypto_skcipher *),
+ GFP_KERNEL);
+ if (!ctx->tfms) {
+ err = -ENOMEM;
+ goto end;
+ }
+
+ /* First instance is already allocated in geniv_init_tfm */
+ ctx->tfms[0] = ctx->child;
+ for (i = 1; i < ctx->tfms_count; i++) {
+ ctx->tfms[i] = crypto_alloc_skcipher(ctx->ciphermode, 0, 0);
+ if (IS_ERR(ctx->tfms[i])) {
+ err = PTR_ERR(ctx->tfms[i]);
+ geniv_free_tfms(ctx);
+ goto end;
+ }
+
+ /* Setup the current cipher's request structure */
+ align = crypto_skcipher_alignmask(parent);
+ align &= ~(crypto_tfm_ctx_alignment() - 1);
+ reqsize = align + sizeof(struct geniv_req_ctx) +
+ crypto_skcipher_reqsize(ctx->tfms[i]);
+ crypto_skcipher_set_reqsize(parent, reqsize);
+ }
+
+end:
+ return err;
+}
+
+/* Initialize the cipher's context with the key, ivmode and other parameters.
+ * Also allocate IV generation template ciphers and initialize them.
+ */
+
+static int geniv_setkey_init(struct crypto_skcipher *parent,
+ struct geniv_key_info *info)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(parent);
+ int ret = -ENOMEM;
+
+ ctx->iv_size = crypto_skcipher_ivsize(parent);
+ ctx->tfms_count = info->tfms_count;
+ ctx->key = info->key;
+ ctx->key_size = info->key_size;
+ ctx->key_parts = info->key_parts;
+ ctx->ivopts = info->ivopts;
+
+ ret = geniv_alloc_tfms(parent, ctx);
+ if (ret)
+ goto end;
+
+ ret = geniv_init_iv(ctx);
+
+end:
+ return ret;
+}
+
+static int geniv_setkey_tfms(struct crypto_skcipher *parent,
+ struct geniv_ctx *ctx,
+ struct geniv_key_info *info)
+{
+ unsigned int subkey_size;
+ int ret = 0, i;
+
+ /* Ignore extra keys (which are used for IV etc) */
+ subkey_size = (ctx->key_size - ctx->key_extra_size)
+ >> ilog2(ctx->tfms_count);
+
+ for (i = 0; i < ctx->tfms_count; i++) {
+ struct crypto_skcipher *child = ctx->tfms[i];
+ char *subkey = ctx->key + (subkey_size) * i;
+
+ crypto_skcipher_clear_flags(child, CRYPTO_TFM_REQ_MASK);
+ crypto_skcipher_set_flags(child,
+ crypto_skcipher_get_flags(parent) &
+ CRYPTO_TFM_REQ_MASK);
+ ret = crypto_skcipher_setkey(child, subkey, subkey_size);
+ if (ret) {
+ DMERR("Error setting key for tfms[%d]\n", i);
+ break;
+ }
+ crypto_skcipher_set_flags(parent,
+ crypto_skcipher_get_flags(child) &
+ CRYPTO_TFM_RES_MASK);
+ }
+
+ return ret;
+}
+
+static int geniv_setkey(struct crypto_skcipher *parent,
+ const u8 *key, unsigned int keylen)
+{
+ int err = 0;
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(parent);
+ struct geniv_key_info *info = (struct geniv_key_info *) key;
+
+ DMDEBUG("SETKEY Operation : %d\n", info->keyop);
+
+ switch (info->keyop) {
+ case SETKEY_OP_INIT:
+ err = geniv_setkey_init(parent, info);
+ break;
+ case SETKEY_OP_SET:
+ err = geniv_setkey_set(ctx);
+ break;
+ case SETKEY_OP_WIPE:
+ err = geniv_setkey_wipe(ctx);
+ break;
+ }
+
+ if (err)
+ goto end;
+
+ err = geniv_setkey_tfms(parent, ctx, info);
+
+end:
+ return err;
+}
+
+static void geniv_async_done(struct crypto_async_request *async_req, int error);
+
+static int geniv_alloc_subreq(struct skcipher_request *req,
+ struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx)
+{
+ int key_index, r = 0;
+ struct skcipher_request *sreq;
+
+ if (!rctx->subreq) {
+ rctx->subreq = mempool_alloc(ctx->subreq_pool, GFP_NOIO);
+ if (!rctx->subreq)
+ r = -ENOMEM;
+ }
+
+ sreq = &rctx->subreq->req;
+ rctx->subreq->rctx = rctx;
+
+ key_index = rctx->iv_sector & (ctx->tfms_count - 1);
+
+ skcipher_request_set_tfm(sreq, ctx->tfms[key_index]);
+ skcipher_request_set_callback(sreq, req->base.flags,
+ geniv_async_done, rctx->subreq);
+ return r;
+}
+
+/* Asynchronous IO completion callback for each sector in a segment. When all
+ * pending i/o are completed the parent cipher's async function is called.
+ */
+
+static void geniv_async_done(struct crypto_async_request *async_req, int error)
+{
+ struct geniv_subreq *subreq =
+ (struct geniv_subreq *) async_req->data;
+ struct geniv_req_ctx *rctx = subreq->rctx;
+ struct skcipher_request *req = rctx->req;
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+
+ /*
+ * A request from crypto driver backlog is going to be processed now,
+ * finish the completion and continue in crypt_convert().
+ * (Callback will be called for the second time for this request.)
+ */
+
+ if (error == -EINPROGRESS) {
+ complete(&rctx->restart);
+ return;
+ }
+
+ if (!error && ctx->iv_gen_ops && ctx->iv_gen_ops->post)
+ error = ctx->iv_gen_ops->post(ctx, rctx, subreq);
+
+ mempool_free(subreq, ctx->subreq_pool);
+
+ /* req_pending needs to be checked before req->base.complete is called
+ * as we need 'req_pending' to be equal to 1 to ensure all subrequests
+ * are processed.
+ */
+ if (!atomic_dec_and_test(&rctx->req_pending)) {
+ /* Call the parent cipher's completion function */
+ skcipher_request_complete(req, error);
+ }
+}
+
+static unsigned int geniv_get_sectors(struct scatterlist *sg1,
+ struct scatterlist *sg2,
+ unsigned int segments)
+{
+ unsigned int i, n1, n2, nents;
+
+ n1 = n2 = 0;
+ for (i = 0; i < segments ; i++) {
+ n1 += sg1[i].length >> SECTOR_SHIFT;
+ n1 += (sg1[i].length & ~SECTOR_MASK) ? 1 : 0;
+ }
+
+ for (i = 0; i < segments ; i++) {
+ n2 += sg2[i].length >> SECTOR_SHIFT;
+ n2 += (sg2[i].length & ~SECTOR_MASK) ? 1 : 0;
+ }
+
+ nents = n1 > n2 ? n1 : n2;
+ return nents;
+}
+
+/* Iterate scatterlist of segments to retrieve the 512-byte sectors so that
+ * unique IVs could be generated for each 512-byte sector. This split may not
+ * be necessary e.g. when these ciphers are modelled in hardware, where it can
+ * make use of the hardware's IV generation capabilities.
+ */
+
+static int geniv_iter_block(struct skcipher_request *req,
+ struct geniv_subreq *subreq,
+ struct geniv_req_ctx *rctx,
+ unsigned int *seg_no,
+ unsigned int *done)
+
+{
+ unsigned int srcoff, dstoff, len, rem;
+ struct scatterlist *src1, *dst1, *src2, *dst2;
+
+ if (unlikely(*seg_no >= rctx->nents))
+ return 0; /* done */
+
+ src1 = &req->src[*seg_no];
+ dst1 = &req->dst[*seg_no];
+ src2 = &subreq->src;
+ dst2 = &subreq->dst;
+
+ if (*done >= src1->length) {
+ (*seg_no)++;
+
+ if (*seg_no >= rctx->nents)
+ return 0; /* done */
+
+ src1 = &req->src[*seg_no];
+ dst1 = &req->dst[*seg_no];
+ *done = 0;
+ }
+
+ srcoff = src1->offset + *done;
+ dstoff = dst1->offset + *done;
+ rem = src1->length - *done;
+
+ len = rem > SECTOR_SIZE ? SECTOR_SIZE : rem;
+
+ DMDEBUG("segment:(%d/%u), srcoff:%d, dstoff:%d, done:%d, rem:%d\n",
+ *seg_no + 1, rctx->nents, srcoff, dstoff, *done, rem);
+
+ sg_init_table(src2, 1);
+ sg_set_page(src2, sg_page(src1), len, srcoff);
+ sg_init_table(dst2, 1);
+ sg_set_page(dst2, sg_page(dst1), len, dstoff);
+
+ *done += len;
+
+ return len; /* bytes returned */
+}
+
+/* Common encryt/decrypt function for geniv template cipher. Before the crypto
+ * operation, it splits the memory segments (in the scatterlist) into 512 byte
+ * sectors. The initialization vector(IV) used is based on a unique sector
+ * number which is generated here.
+ */
+static int geniv_crypt(struct skcipher_request *req, int encrypt)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ struct geniv_req_ctx *rctx = geniv_req_ctx(req);
+ struct geniv_req_info *rinfo = (struct geniv_req_info *) req->iv;
+ int i, bytes, cryptlen, ret = 0;
+ unsigned int sectors, segno = 0, done = 0;
+ char *str __maybe_unused = encrypt ? "encrypt" : "decrypt";
+
+ /* Instance of 'struct geniv_req_info' is stored in IV ptr */
+ rctx->is_write = encrypt;
+ rctx->iv_sector = rinfo->iv_sector;
+ rctx->nents = rinfo->nents;
+ rctx->iv = rinfo->iv;
+ rctx->req = req;
+ rctx->subreq = NULL;
+ cryptlen = req->cryptlen;
+
+ DMDEBUG("geniv:%s: starting sector=%d, #segments=%u\n", str,
+ (unsigned int) rctx->iv_sector, rctx->nents);
+
+ sectors = geniv_get_sectors(req->src, req->dst, rctx->nents);
+
+ init_completion(&rctx->restart);
+ atomic_set(&rctx->req_pending, 1);
+
+ for (i = 0; i < sectors; i++) {
+ struct geniv_subreq *subreq;
+
+ ret = geniv_alloc_subreq(req, ctx, rctx);
+ if (ret)
+ goto end;
+
+ subreq = rctx->subreq;
+ subreq->rctx = rctx;
+
+ atomic_inc(&rctx->req_pending);
+ bytes = geniv_iter_block(req, subreq, rctx, &segno, &done);
+
+ if (bytes == 0)
+ break;
+
+ cryptlen -= bytes;
+
+ if (ctx->iv_gen_ops)
+ ret = ctx->iv_gen_ops->generator(ctx, rctx, subreq);
+
+ if (ret < 0) {
+ DMERR("Error in generating IV ret: %d\n", ret);
+ goto end;
+ }
+
+ skcipher_request_set_crypt(&subreq->req, &subreq->src,
+ &subreq->dst, bytes, rctx->iv);
+
+ if (encrypt)
+ ret = crypto_skcipher_encrypt(&subreq->req);
+
+ else
+ ret = crypto_skcipher_decrypt(&subreq->req);
+
+ if (!ret && ctx->iv_gen_ops && ctx->iv_gen_ops->post)
+ ret = ctx->iv_gen_ops->post(ctx, rctx, subreq);
+
+ switch (ret) {
+ /*
+ * The request was queued by a crypto driver
+ * but the driver request queue is full, let's wait.
+ */
+ case -EBUSY:
+ wait_for_completion(&rctx->restart);
+ reinit_completion(&rctx->restart);
+ /* fall through */
+ /*
+ * The request is queued and processed asynchronously,
+ * completion function geniv_async_done() is called.
+ */
+ case -EINPROGRESS:
+ /* Marking this NULL lets the creation of a new sub-
+ * request when 'geniv_alloc_subreq' is called.
+ */
+ rctx->subreq = NULL;
+ rctx->iv_sector++;
+ cond_resched();
+ break;
+ /*
+ * The request was already processed (synchronously).
+ */
+ case 0:
+ atomic_dec(&rctx->req_pending);
+ rctx->iv_sector++;
+ cond_resched();
+ continue;
+
+ /* There was an error while processing the request. */
+ default:
+ atomic_dec(&rctx->req_pending);
+ return ret;
+ }
+
+ if (ret)
+ break;
+ }
+
+ if (rctx->subreq && atomic_read(&rctx->req_pending) == 1) {
+ DMDEBUG("geniv:%s: Freeing sub request\n", str);
+ mempool_free(rctx->subreq, ctx->subreq_pool);
+ }
+
+end:
+ return ret;
+}
+
+static int geniv_encrypt(struct skcipher_request *req)
+{
+ return geniv_crypt(req, 1);
+}
+
+static int geniv_decrypt(struct skcipher_request *req)
+{
+ return geniv_crypt(req, 0);
+}
+
+static int geniv_init_tfm(struct crypto_skcipher *tfm)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ unsigned int reqsize, align;
+ char *algname, *chainmode;
+ int psize, ret = 0;
+
+ algname = (char *) crypto_tfm_alg_name(crypto_skcipher_tfm(tfm));
+ ctx->ciphermode = kmalloc(CRYPTO_MAX_ALG_NAME, GFP_KERNEL);
+ if (!ctx->ciphermode) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ctx->algname = kmalloc(CRYPTO_MAX_ALG_NAME, GFP_KERNEL);
+ if (!ctx->algname) {
+ ret = -ENOMEM;
+ goto free_ciphermode;
+ }
+
+ strlcpy(ctx->algname, algname, CRYPTO_MAX_ALG_NAME);
+ algname = ctx->algname;
+
+ /* Parse the algorithm name 'ivmode(chainmode(cipher))' */
+ ctx->ivmode = strsep(&algname, "(");
+ chainmode = strsep(&algname, "(");
+ ctx->cipher = strsep(&algname, ")");
+
+ snprintf(ctx->ciphermode, CRYPTO_MAX_ALG_NAME, "%s(%s)",
+ chainmode, ctx->cipher);
+
+ DMDEBUG("ciphermode=%s, ivmode=%s\n", ctx->ciphermode, ctx->ivmode);
+
+ /*
+ * Usually the underlying cipher instances are spawned here, but since
+ * the value of tfms_count (which is equal to the key_count) is not
+ * known yet, create only one instance and delay the creation of the
+ * rest of the instances of the underlying cipher 'cbc(aes)' until
+ * the setkey operation is invoked.
+ * The first instance created i.e. ctx->child will later be assigned as
+ * the 1st element in the array ctx->tfms. Creation of atleast one
+ * instance of the cipher is necessary to be created here to uncover
+ * any errors earlier than during the setkey operation later where the
+ * remaining instances are created.
+ */
+ ctx->child = crypto_alloc_skcipher(ctx->ciphermode, 0, 0);
+ if (IS_ERR(ctx->child)) {
+ ret = PTR_ERR(ctx->child);
+ DMERR("Failed to create skcipher %s. err %d\n",
+ ctx->ciphermode, ret);
+ goto free_algname;
+ }
+
+ /* Setup the current cipher's request structure */
+ align = crypto_skcipher_alignmask(tfm);
+ align &= ~(crypto_tfm_ctx_alignment() - 1);
+ reqsize = align + sizeof(struct geniv_req_ctx)
+ + crypto_skcipher_reqsize(ctx->child);
+ crypto_skcipher_set_reqsize(tfm, reqsize);
+
+ /* create memory pool for sub-request structure */
+ psize = sizeof(struct geniv_subreq)
+ + crypto_skcipher_reqsize(ctx->child);
+ ctx->subreq_pool = mempool_create_kmalloc_pool(MIN_IOS, psize);
+ if (!ctx->subreq_pool) {
+ ret = -ENOMEM;
+ DMERR("Could not allocate crypt sub-request mempool\n");
+ goto free_skcipher;
+ }
+out:
+ return ret;
+
+free_skcipher:
+ crypto_free_skcipher(ctx->child);
+free_algname:
+ kfree(ctx->algname);
+free_ciphermode:
+ kfree(ctx->ciphermode);
+ goto out;
+}
+
+static void geniv_exit_tfm(struct crypto_skcipher *tfm)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->dtr)
+ ctx->iv_gen_ops->dtr(ctx);
+
+ mempool_destroy(ctx->subreq_pool);
+ geniv_free_tfms(ctx);
+ kfree(ctx->ciphermode);
+ kfree(ctx->algname);
+}
+
+static void geniv_free(struct skcipher_instance *inst)
+{
+ struct crypto_skcipher_spawn *spawn = skcipher_instance_ctx(inst);
+
+ crypto_drop_skcipher(spawn);
+ kfree(inst);
+}
+
+static int geniv_create(struct crypto_template *tmpl,
+ struct rtattr **tb, char *algname)
+{
+ struct crypto_attr_type *algt;
+ struct skcipher_instance *inst;
+ struct skcipher_alg *alg;
+ struct crypto_skcipher_spawn *spawn;
+ const char *cipher_name;
+ int err;
+
+ algt = crypto_get_attr_type(tb);
+
+ if (IS_ERR(algt))
+ return PTR_ERR(algt);
+
+ if ((algt->type ^ CRYPTO_ALG_TYPE_SKCIPHER) & algt->mask)
+ return -EINVAL;
+
+ cipher_name = crypto_attr_alg_name(tb[1]);
+
+ if (IS_ERR(cipher_name))
+ return PTR_ERR(cipher_name);
+
+ inst = kzalloc(sizeof(*inst) + sizeof(*spawn), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ spawn = skcipher_instance_ctx(inst);
+
+ crypto_set_skcipher_spawn(spawn, skcipher_crypto_instance(inst));
+ err = crypto_grab_skcipher(spawn, cipher_name, 0,
+ crypto_requires_sync(algt->type,
+ algt->mask));
+
+ if (err)
+ goto err_free_inst;
+
+ alg = crypto_spawn_skcipher_alg(spawn);
+
+ err = -EINVAL;
+
+ /* Only support blocks of size which is of a power of 2 */
+ if (!is_power_of_2(alg->base.cra_blocksize))
+ goto err_drop_spawn;
+
+ /* algname: essiv, base.cra_name: cbc(aes) */
+ err = -ENAMETOOLONG;
+ if (snprintf(inst->alg.base.cra_name, CRYPTO_MAX_ALG_NAME, "%s(%s)",
+ algname, alg->base.cra_name) >= CRYPTO_MAX_ALG_NAME)
+ goto err_drop_spawn;
+ if (snprintf(inst->alg.base.cra_driver_name, CRYPTO_MAX_ALG_NAME,
+ "%s(%s)", algname, alg->base.cra_driver_name) >=
+ CRYPTO_MAX_ALG_NAME)
+ goto err_drop_spawn;
+
+ inst->alg.base.cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER;
+ inst->alg.base.cra_priority = alg->base.cra_priority;
+ inst->alg.base.cra_blocksize = alg->base.cra_blocksize;
+ inst->alg.base.cra_alignmask = alg->base.cra_alignmask;
+ inst->alg.base.cra_flags = alg->base.cra_flags & CRYPTO_ALG_ASYNC;
+ inst->alg.ivsize = alg->base.cra_blocksize;
+ inst->alg.chunksize = crypto_skcipher_alg_chunksize(alg);
+ inst->alg.min_keysize = crypto_skcipher_alg_min_keysize(alg);
+ inst->alg.max_keysize = crypto_skcipher_alg_max_keysize(alg);
+
+ inst->alg.setkey = geniv_setkey;
+ inst->alg.encrypt = geniv_encrypt;
+ inst->alg.decrypt = geniv_decrypt;
+
+ inst->alg.base.cra_ctxsize = sizeof(struct geniv_ctx);
+
+ inst->alg.init = geniv_init_tfm;
+ inst->alg.exit = geniv_exit_tfm;
+
+ inst->free = geniv_free;
+
+ err = skcipher_register_instance(tmpl, inst);
+ if (err)
+ goto err_drop_spawn;
+
+out:
+ return err;
+
+err_drop_spawn:
+ crypto_drop_skcipher(spawn);
+err_free_inst:
+ kfree(inst);
+ goto out;
+}
+
+static int crypto_plain_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "plain");
+}
+
+static int crypto_plain64_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "plain64");
+}
+
+static int crypto_essiv_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "essiv");
+}
+
+static int crypto_benbi_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "benbi");
+}
+
+static int crypto_null_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "null");
+}
+
+static int crypto_lmk_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "lmk");
+}
+
+static int crypto_tcw_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "tcw");
+}
+
+static struct crypto_template crypto_plain_tmpl = {
+ .name = "plain",
+ .create = crypto_plain_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_plain64_tmpl = {
+ .name = "plain64",
+ .create = crypto_plain64_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_essiv_tmpl = {
+ .name = "essiv",
+ .create = crypto_essiv_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_benbi_tmpl = {
+ .name = "benbi",
+ .create = crypto_benbi_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_null_tmpl = {
+ .name = "null",
+ .create = crypto_null_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_lmk_tmpl = {
+ .name = "lmk",
+ .create = crypto_lmk_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_tcw_tmpl = {
+ .name = "tcw",
+ .create = crypto_tcw_create,
+ .module = THIS_MODULE,
+};
+
+static int __init geniv_register_algs(void)
+{
+ int err;
+
+ err = crypto_register_template(&crypto_plain_tmpl);
+ if (err)
+ goto out;
+
+ err = crypto_register_template(&crypto_plain64_tmpl);
+ if (err)
+ goto out_undo_plain;
+
+ err = crypto_register_template(&crypto_essiv_tmpl);
+ if (err)
+ goto out_undo_plain64;
+
+ err = crypto_register_template(&crypto_benbi_tmpl);
+ if (err)
+ goto out_undo_essiv;
+
+ err = crypto_register_template(&crypto_null_tmpl);
+ if (err)
+ goto out_undo_benbi;
+
+ err = crypto_register_template(&crypto_lmk_tmpl);
+ if (err)
+ goto out_undo_null;
+
+ err = crypto_register_template(&crypto_tcw_tmpl);
+ if (!err)
+ goto out;
+
+ crypto_unregister_template(&crypto_lmk_tmpl);
+out_undo_null:
+ crypto_unregister_template(&crypto_null_tmpl);
+out_undo_benbi:
+ crypto_unregister_template(&crypto_benbi_tmpl);
+out_undo_essiv:
+ crypto_unregister_template(&crypto_essiv_tmpl);
+out_undo_plain64:
+ crypto_unregister_template(&crypto_plain64_tmpl);
+out_undo_plain:
+ crypto_unregister_template(&crypto_plain_tmpl);
+out:
+ return err;
+}
+
+static void __exit geniv_deregister_algs(void)
+{
+ crypto_unregister_template(&crypto_plain_tmpl);
+ crypto_unregister_template(&crypto_plain64_tmpl);
+ crypto_unregister_template(&crypto_essiv_tmpl);
+ crypto_unregister_template(&crypto_benbi_tmpl);
+ crypto_unregister_template(&crypto_null_tmpl);
+ crypto_unregister_template(&crypto_lmk_tmpl);
+ crypto_unregister_template(&crypto_tcw_tmpl);
+}
+
+/* End of geniv template cipher algorithms */
+
+/*
+ * context holding the current state of a multi-part conversion
+ */
+struct convert_context {
+ struct completion restart;
+ struct bio *bio_in;
+ struct bio *bio_out;
+ struct bvec_iter iter_in;
+ struct bvec_iter iter_out;
+ sector_t cc_sector;
+ atomic_t cc_pending;
+ struct skcipher_request *req;
+};
+
+/*
+ * per bio private data
+ */
+struct dm_crypt_io {
+ struct crypt_config *cc;
+ struct bio *base_bio;
+ struct work_struct work;
+
+ struct convert_context ctx;
-static int crypt_iv_tcw_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
- __le64 sector = cpu_to_le64(dmreq->iv_sector);
- u8 *src;
- int r = 0;
+ atomic_t io_pending;
+ int error;
+ sector_t sector;
- /* Remove whitening from ciphertext */
- if (bio_data_dir(dmreq->ctx->bio_in) != WRITE) {
- src = kmap_atomic(sg_page(&dmreq->sg_in));
- r = crypt_iv_tcw_whitening(cc, dmreq, src + dmreq->sg_in.offset);
- kunmap_atomic(src);
- }
+ struct rb_node rb_node;
+} CRYPTO_MINALIGN_ATTR;
- /* Calculate IV */
- memcpy(iv, tcw->iv_seed, cc->iv_size);
- crypto_xor(iv, (u8 *)§or, 8);
- if (cc->iv_size > 8)
- crypto_xor(&iv[8], (u8 *)§or, cc->iv_size - 8);
+struct dm_crypt_request {
+ struct convert_context *ctx;
+ struct scatterlist *sg_in;
+ struct scatterlist *sg_out;
+ sector_t iv_sector;
+};
- return r;
-}
+struct crypt_config;
-static int crypt_iv_tcw_post(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- u8 *dst;
- int r;
+/*
+ * Crypt: maps a linear range of a block device
+ * and encrypts / decrypts at the same time.
+ */
+enum flags { DM_CRYPT_SUSPENDED, DM_CRYPT_KEY_VALID,
+ DM_CRYPT_SAME_CPU, DM_CRYPT_NO_OFFLOAD };
- if (bio_data_dir(dmreq->ctx->bio_in) != WRITE)
- return 0;
+/*
+ * The fields in here must be read only after initialization.
+ */
+struct crypt_config {
+ struct dm_dev *dev;
+ sector_t start;
- /* Apply whitening on ciphertext */
- dst = kmap_atomic(sg_page(&dmreq->sg_out));
- r = crypt_iv_tcw_whitening(cc, dmreq, dst + dmreq->sg_out.offset);
- kunmap_atomic(dst);
+ /*
+ * pool for per bio private data, crypto requests and
+ * encryption requeusts/buffer pages
+ */
+ mempool_t *req_pool;
+ mempool_t *page_pool;
+ struct bio_set *bs;
+ struct mutex bio_alloc_lock;
- return r;
-}
+ struct workqueue_struct *io_queue;
+ struct workqueue_struct *crypt_queue;
-static const struct crypt_iv_operations crypt_iv_plain_ops = {
- .generator = crypt_iv_plain_gen
-};
+ struct task_struct *write_thread;
+ wait_queue_head_t write_thread_wait;
+ struct rb_root write_tree;
-static const struct crypt_iv_operations crypt_iv_plain64_ops = {
- .generator = crypt_iv_plain64_gen
-};
+ char *cipher;
+ char *cipher_string;
+ char *key_string;
-static const struct crypt_iv_operations crypt_iv_essiv_ops = {
- .ctr = crypt_iv_essiv_ctr,
- .dtr = crypt_iv_essiv_dtr,
- .init = crypt_iv_essiv_init,
- .wipe = crypt_iv_essiv_wipe,
- .generator = crypt_iv_essiv_gen
-};
+ sector_t iv_offset;
+ unsigned int iv_size;
-static const struct crypt_iv_operations crypt_iv_benbi_ops = {
- .ctr = crypt_iv_benbi_ctr,
- .dtr = crypt_iv_benbi_dtr,
- .generator = crypt_iv_benbi_gen
-};
+ /* ESSIV: struct crypto_cipher *essiv_tfm */
+ void *iv_private;
+ struct crypto_skcipher *tfm;
+ unsigned int tfms_count;
-static const struct crypt_iv_operations crypt_iv_null_ops = {
- .generator = crypt_iv_null_gen
-};
+ /*
+ * Layout of each crypto request:
+ *
+ * struct skcipher_request
+ * context
+ * padding
+ * struct dm_crypt_request
+ * padding
+ * IV
+ *
+ * The padding is added so that dm_crypt_request and the IV are
+ * correctly aligned.
+ */
+ unsigned int dmreq_start;
-static const struct crypt_iv_operations crypt_iv_lmk_ops = {
- .ctr = crypt_iv_lmk_ctr,
- .dtr = crypt_iv_lmk_dtr,
- .init = crypt_iv_lmk_init,
- .wipe = crypt_iv_lmk_wipe,
- .generator = crypt_iv_lmk_gen,
- .post = crypt_iv_lmk_post
-};
+ unsigned int per_bio_data_size;
-static const struct crypt_iv_operations crypt_iv_tcw_ops = {
- .ctr = crypt_iv_tcw_ctr,
- .dtr = crypt_iv_tcw_dtr,
- .init = crypt_iv_tcw_init,
- .wipe = crypt_iv_tcw_wipe,
- .generator = crypt_iv_tcw_gen,
- .post = crypt_iv_tcw_post
+ unsigned long flags;
+ unsigned int key_size;
+ unsigned int key_parts; /* independent parts in key buffer */
+ unsigned int key_extra_size; /* additional keys length */
+ u8 key[0];
};
+static void clone_init(struct dm_crypt_io *, struct bio *);
+static void kcryptd_queue_crypt(struct dm_crypt_io *io);
+static u8 *iv_of_dmreq(struct crypt_config *cc, struct dm_crypt_request *dmreq);
+
static void crypt_convert_init(struct crypt_config *cc,
struct convert_context *ctx,
struct bio *bio_out, struct bio *bio_in,
@@ -837,53 +1723,7 @@ static u8 *iv_of_dmreq(struct crypt_config *cc,
struct dm_crypt_request *dmreq)
{
return (u8 *)ALIGN((unsigned long)(dmreq + 1),
- crypto_skcipher_alignmask(any_tfm(cc)) + 1);
-}
-
-static int crypt_convert_block(struct crypt_config *cc,
- struct convert_context *ctx,
- struct skcipher_request *req)
-{
- struct bio_vec bv_in = bio_iter_iovec(ctx->bio_in, ctx->iter_in);
- struct bio_vec bv_out = bio_iter_iovec(ctx->bio_out, ctx->iter_out);
- struct dm_crypt_request *dmreq;
- u8 *iv;
- int r;
-
- dmreq = dmreq_of_req(cc, req);
- iv = iv_of_dmreq(cc, dmreq);
-
- dmreq->iv_sector = ctx->cc_sector;
- dmreq->ctx = ctx;
- sg_init_table(&dmreq->sg_in, 1);
- sg_set_page(&dmreq->sg_in, bv_in.bv_page, 1 << SECTOR_SHIFT,
- bv_in.bv_offset);
-
- sg_init_table(&dmreq->sg_out, 1);
- sg_set_page(&dmreq->sg_out, bv_out.bv_page, 1 << SECTOR_SHIFT,
- bv_out.bv_offset);
-
- bio_advance_iter(ctx->bio_in, &ctx->iter_in, 1 << SECTOR_SHIFT);
- bio_advance_iter(ctx->bio_out, &ctx->iter_out, 1 << SECTOR_SHIFT);
-
- if (cc->iv_gen_ops) {
- r = cc->iv_gen_ops->generator(cc, iv, dmreq);
- if (r < 0)
- return r;
- }
-
- skcipher_request_set_crypt(req, &dmreq->sg_in, &dmreq->sg_out,
- 1 << SECTOR_SHIFT, iv);
-
- if (bio_data_dir(ctx->bio_in) == WRITE)
- r = crypto_skcipher_encrypt(req);
- else
- r = crypto_skcipher_decrypt(req);
-
- if (!r && cc->iv_gen_ops && cc->iv_gen_ops->post)
- r = cc->iv_gen_ops->post(cc, iv, dmreq);
-
- return r;
+ crypto_skcipher_alignmask(cc->tfm) + 1);
}
static void kcryptd_async_done(struct crypto_async_request *async_req,
@@ -892,12 +1732,10 @@ static void kcryptd_async_done(struct crypto_async_request *async_req,
static void crypt_alloc_req(struct crypt_config *cc,
struct convert_context *ctx)
{
- unsigned key_index = ctx->cc_sector & (cc->tfms_count - 1);
-
if (!ctx->req)
ctx->req = mempool_alloc(cc->req_pool, GFP_NOIO);
- skcipher_request_set_tfm(ctx->req, cc->tfms[key_index]);
+ skcipher_request_set_tfm(ctx->req, cc->tfm);
/*
* Use REQ_MAY_BACKLOG so a cipher driver internally backlogs
@@ -920,57 +1758,97 @@ static void crypt_free_req(struct crypt_config *cc,
/*
* Encrypt / decrypt data from one bio to another one (can be the same one)
*/
-static int crypt_convert(struct crypt_config *cc,
- struct convert_context *ctx)
+
+static int crypt_convert_bio(struct crypt_config *cc,
+ struct convert_context *ctx)
{
+ unsigned int cryptlen, n1, n2, nents, i = 0, bytes = 0;
+ struct skcipher_request *req;
+ struct dm_crypt_request *dmreq;
+ struct geniv_req_info rinfo;
+ struct bio_vec bv_in, bv_out;
int r;
+ u8 *iv;
atomic_set(&ctx->cc_pending, 1);
+ crypt_alloc_req(cc, ctx);
+
+ req = ctx->req;
+ dmreq = dmreq_of_req(cc, req);
+ iv = iv_of_dmreq(cc, dmreq);
- while (ctx->iter_in.bi_size && ctx->iter_out.bi_size) {
+ n1 = bio_segments(ctx->bio_in);
+ n2 = bio_segments(ctx->bio_out);
+ nents = n1 > n2 ? n1 : n2;
+ nents = nents > MAX_SG_LIST ? MAX_SG_LIST : nents;
+ cryptlen = ctx->iter_in.bi_size;
- crypt_alloc_req(cc, ctx);
+ DMDEBUG("dm-crypt:%s: segments:[in=%u, out=%u] bi_size=%u\n",
+ bio_data_dir(ctx->bio_in) == WRITE ? "write" : "read",
+ n1, n2, cryptlen);
- atomic_inc(&ctx->cc_pending);
+ dmreq->sg_in = kcalloc(nents, sizeof(struct scatterlist), GFP_KERNEL);
+ dmreq->sg_out = kcalloc(nents, sizeof(struct scatterlist), GFP_KERNEL);
+ if (!dmreq->sg_in || !dmreq->sg_out) {
+ DMERR("dm-crypt: Failed to allocate scatterlist\n");
+ r = -ENOMEM;
+ goto end;
+ }
+ dmreq->ctx = ctx;
- r = crypt_convert_block(cc, ctx, ctx->req);
+ sg_init_table(dmreq->sg_in, nents);
+ sg_init_table(dmreq->sg_out, nents);
- switch (r) {
- /*
- * The request was queued by a crypto driver
- * but the driver request queue is full, let's wait.
- */
- case -EBUSY:
- wait_for_completion(&ctx->restart);
- reinit_completion(&ctx->restart);
- /* fall through */
- /*
- * The request is queued and processed asynchronously,
- * completion function kcryptd_async_done() will be called.
- */
- case -EINPROGRESS:
- ctx->req = NULL;
- ctx->cc_sector++;
- continue;
- /*
- * The request was already processed (synchronously).
- */
- case 0:
- atomic_dec(&ctx->cc_pending);
- ctx->cc_sector++;
- cond_resched();
- continue;
+ while (ctx->iter_in.bi_size && ctx->iter_out.bi_size && i < nents) {
+ bv_in = bio_iter_iovec(ctx->bio_in, ctx->iter_in);
+ bv_out = bio_iter_iovec(ctx->bio_out, ctx->iter_out);
- /* There was an error while processing the request. */
- default:
- atomic_dec(&ctx->cc_pending);
- return r;
- }
+ sg_set_page(&dmreq->sg_in[i], bv_in.bv_page, bv_in.bv_len,
+ bv_in.bv_offset);
+ sg_set_page(&dmreq->sg_out[i], bv_out.bv_page, bv_out.bv_len,
+ bv_out.bv_offset);
+
+ bio_advance_iter(ctx->bio_in, &ctx->iter_in, bv_in.bv_len);
+ bio_advance_iter(ctx->bio_out, &ctx->iter_out, bv_out.bv_len);
+
+ bytes += bv_in.bv_len;
+ i++;
}
- return 0;
+ DMDEBUG("dm-crypt: Processed %u of %u bytes\n", bytes, cryptlen);
+
+ rinfo.iv_sector = ctx->cc_sector;
+ rinfo.nents = nents;
+ rinfo.iv = iv;
+
+ skcipher_request_set_crypt(req, dmreq->sg_in, dmreq->sg_out,
+ bytes, &rinfo);
+
+ if (bio_data_dir(ctx->bio_in) == WRITE)
+ r = crypto_skcipher_encrypt(req);
+ else
+ r = crypto_skcipher_decrypt(req);
+
+ switch (r) {
+ /* The request was queued so wait. */
+ case -EBUSY:
+ wait_for_completion(&ctx->restart);
+ reinit_completion(&ctx->restart);
+ /* fall through */
+ /*
+ * The request is queued and processed asynchronously,
+ * completion function kcryptd_async_done() is called.
+ */
+ case -EINPROGRESS:
+ ctx->req = NULL;
+ cond_resched();
+ break;
+ }
+end:
+ return r;
}
+
static void crypt_free_buffer_pages(struct crypt_config *cc, struct bio *clone);
/*
@@ -1070,11 +1948,17 @@ static void crypt_dec_pending(struct dm_crypt_io *io)
{
struct crypt_config *cc = io->cc;
struct bio *base_bio = io->base_bio;
+ struct dm_crypt_request *dmreq;
int error = io->error;
if (!atomic_dec_and_test(&io->io_pending))
return;
+ dmreq = dmreq_of_req(cc, io->ctx.req);
+ DMDEBUG("dm-crypt: Freeing scatterlists [sync]\n");
+ kfree(dmreq->sg_in);
+ kfree(dmreq->sg_out);
+
if (io->ctx.req)
crypt_free_req(cc, io->ctx.req, base_bio);
@@ -1313,7 +2197,7 @@ static void kcryptd_crypt_write_convert(struct dm_crypt_io *io)
sector += bio_sectors(clone);
crypt_inc_pending(io);
- r = crypt_convert(cc, &io->ctx);
+ r = crypt_convert_bio(cc, &io->ctx);
if (r)
io->error = -EIO;
crypt_finished = atomic_dec_and_test(&io->ctx.cc_pending);
@@ -1343,7 +2227,8 @@ static void kcryptd_crypt_read_convert(struct dm_crypt_io *io)
crypt_convert_init(cc, &io->ctx, io->base_bio, io->base_bio,
io->sector);
- r = crypt_convert(cc, &io->ctx);
+ r = crypt_convert_bio(cc, &io->ctx);
+
if (r < 0)
io->error = -EIO;
@@ -1371,12 +2256,13 @@ static void kcryptd_async_done(struct crypto_async_request *async_req,
return;
}
- if (!error && cc->iv_gen_ops && cc->iv_gen_ops->post)
- error = cc->iv_gen_ops->post(cc, iv_of_dmreq(cc, dmreq), dmreq);
-
if (error < 0)
io->error = -EIO;
+ DMDEBUG("dm-crypt: Freeing scatterlists and request struct [async]\n");
+ kfree(dmreq->sg_in);
+ kfree(dmreq->sg_out);
+
crypt_free_req(cc, req_of_dmreq(cc, dmreq), io->base_bio);
if (!atomic_dec_and_test(&ctx->cc_pending))
@@ -1430,62 +2316,38 @@ static int crypt_decode_key(u8 *key, char *hex, unsigned int size)
return 0;
}
-static void crypt_free_tfms(struct crypt_config *cc)
+static void crypt_free_tfm(struct crypt_config *cc)
{
- unsigned i;
-
- if (!cc->tfms)
+ if (!cc->tfm)
return;
- for (i = 0; i < cc->tfms_count; i++)
- if (cc->tfms[i] && !IS_ERR(cc->tfms[i])) {
- crypto_free_skcipher(cc->tfms[i]);
- cc->tfms[i] = NULL;
- }
+ if (cc->tfm && !IS_ERR(cc->tfm))
+ crypto_free_skcipher(cc->tfm);
- kfree(cc->tfms);
- cc->tfms = NULL;
+ cc->tfm = NULL;
}
-static int crypt_alloc_tfms(struct crypt_config *cc, char *ciphermode)
+static int crypt_alloc_tfm(struct crypt_config *cc, char *ciphermode)
{
- unsigned i;
int err;
- cc->tfms = kzalloc(cc->tfms_count * sizeof(struct crypto_skcipher *),
- GFP_KERNEL);
- if (!cc->tfms)
- return -ENOMEM;
-
- for (i = 0; i < cc->tfms_count; i++) {
- cc->tfms[i] = crypto_alloc_skcipher(ciphermode, 0, 0);
- if (IS_ERR(cc->tfms[i])) {
- err = PTR_ERR(cc->tfms[i]);
- crypt_free_tfms(cc);
- return err;
- }
+ cc->tfm = crypto_alloc_skcipher(ciphermode, 0, 0);
+ if (IS_ERR(cc->tfm)) {
+ err = PTR_ERR(cc->tfm);
+ crypt_free_tfm(cc);
+ return err;
}
return 0;
}
-static int crypt_setkey(struct crypt_config *cc)
+static inline int crypt_setkey(struct crypt_config *cc, enum setkey_op keyop,
+ char *ivopts)
{
- unsigned subkey_size;
- int err = 0, i, r;
-
- /* Ignore extra keys (which are used for IV etc) */
- subkey_size = (cc->key_size - cc->key_extra_size) >> ilog2(cc->tfms_count);
-
- for (i = 0; i < cc->tfms_count; i++) {
- r = crypto_skcipher_setkey(cc->tfms[i],
- cc->key + (i * subkey_size),
- subkey_size);
- if (r)
- err = r;
- }
+ DECLARE_GENIV_KEY(kinfo, keyop, cc->tfms_count, cc->key, cc->key_size,
+ cc->key_parts, ivopts);
- return err;
+ return crypto_skcipher_setkey(cc->tfm, (u8 *) &kinfo, sizeof(kinfo));
}
#ifdef CONFIG_KEYS
@@ -1498,7 +2360,9 @@ static bool contains_whitespace(const char *str)
return false;
}
-static int crypt_set_keyring_key(struct crypt_config *cc, const char *key_string)
+static int crypt_set_keyring_key(struct crypt_config *cc,
+ const char *key_string,
+ enum setkey_op keyop, char *ivopts)
{
char *new_key_string, *key_desc;
int ret;
@@ -1559,7 +2423,7 @@ static int crypt_set_keyring_key(struct crypt_config *cc, const char *key_string
/* clear the flag since following operations may invalidate previously valid key */
clear_bit(DM_CRYPT_KEY_VALID, &cc->flags);
- ret = crypt_setkey(cc);
+ ret = crypt_setkey(cc, keyop, ivopts);
/* wipe the kernel key payload copy in each case */
memset(cc->key, 0, cc->key_size * sizeof(u8));
@@ -1599,7 +2463,9 @@ static int get_key_size(char **key_string)
#else
-static int crypt_set_keyring_key(struct crypt_config *cc, const char *key_string)
+static int crypt_set_keyring_key(struct crypt_config *cc,
+ const char *key_string,
+ enum setkey_op keyop, char *ivopts)
{
return -EINVAL;
}
@@ -1611,7 +2477,8 @@ static int get_key_size(char **key_string)
#endif
-static int crypt_set_key(struct crypt_config *cc, char *key)
+static int crypt_set_key(struct crypt_config *cc, enum setkey_op keyop,
+ char *key, char *ivopts)
{
int r = -EINVAL;
int key_string_len = strlen(key);
@@ -1622,7 +2489,7 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
/* ':' means the key is in kernel keyring, short-circuit normal key processing */
if (key[0] == ':') {
- r = crypt_set_keyring_key(cc, key + 1);
+ r = crypt_set_keyring_key(cc, key + 1, keyop, ivopts);
goto out;
}
@@ -1636,7 +2503,7 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
if (cc->key_size && crypt_decode_key(cc->key, key, cc->key_size) < 0)
goto out;
- r = crypt_setkey(cc);
+ r = crypt_setkey(cc, keyop, ivopts);
if (!r)
set_bit(DM_CRYPT_KEY_VALID, &cc->flags);
@@ -1647,6 +2514,17 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
return r;
}
+static int crypt_init_key(struct dm_target *ti, char *key, char *ivopts)
+{
+ struct crypt_config *cc = ti->private;
+ int ret;
+
+ ret = crypt_set_key(cc, SETKEY_OP_INIT, key, ivopts);
+ if (ret < 0)
+ ti->error = "Error decoding and setting key";
+ return ret;
+}
+
static int crypt_wipe_key(struct crypt_config *cc)
{
clear_bit(DM_CRYPT_KEY_VALID, &cc->flags);
@@ -1654,7 +2532,7 @@ static int crypt_wipe_key(struct crypt_config *cc)
kzfree(cc->key_string);
cc->key_string = NULL;
- return crypt_setkey(cc);
+ return crypt_setkey(cc, SETKEY_OP_WIPE, NULL);
}
static void crypt_dtr(struct dm_target *ti)
@@ -1674,7 +2552,7 @@ static void crypt_dtr(struct dm_target *ti)
if (cc->crypt_queue)
destroy_workqueue(cc->crypt_queue);
- crypt_free_tfms(cc);
+ crypt_free_tfm(cc);
if (cc->bs)
bioset_free(cc->bs);
@@ -1682,9 +2560,6 @@ static void crypt_dtr(struct dm_target *ti)
mempool_destroy(cc->page_pool);
mempool_destroy(cc->req_pool);
- if (cc->iv_gen_ops && cc->iv_gen_ops->dtr)
- cc->iv_gen_ops->dtr(cc);
-
if (cc->dev)
dm_put_device(ti, cc->dev);
@@ -1762,22 +2637,30 @@ static int crypt_ctr_cipher(struct dm_target *ti,
if (!cipher_api)
goto bad_mem;
- ret = snprintf(cipher_api, CRYPTO_MAX_ALG_NAME,
- "%s(%s)", chainmode, cipher);
+create_cipher:
+ /* For those ciphers which do not support IVs,
+ * use the 'null' template cipher
+ */
+
+ if (!ivmode)
+ ivmode = "null";
+
+ ret = snprintf(cipher_api, CRYPTO_MAX_ALG_NAME, "%s(%s(%s))",
+ ivmode, chainmode, cipher);
if (ret < 0) {
kfree(cipher_api);
goto bad_mem;
}
/* Allocate cipher */
- ret = crypt_alloc_tfms(cc, cipher_api);
+ ret = crypt_alloc_tfm(cc, cipher_api);
if (ret < 0) {
ti->error = "Error allocating crypto tfm";
goto bad;
}
/* Initialize IV */
- cc->iv_size = crypto_skcipher_ivsize(any_tfm(cc));
+ cc->iv_size = crypto_skcipher_ivsize(cc->tfm);
if (cc->iv_size)
/* at least a 64 bit sector number should fit in our buffer */
cc->iv_size = max(cc->iv_size,
@@ -1785,23 +2668,10 @@ static int crypt_ctr_cipher(struct dm_target *ti,
else if (ivmode) {
DMWARN("Selected cipher does not support IVs");
ivmode = NULL;
+ goto create_cipher;
}
- /* Choose ivmode, see comments at iv code. */
- if (ivmode == NULL)
- cc->iv_gen_ops = NULL;
- else if (strcmp(ivmode, "plain") == 0)
- cc->iv_gen_ops = &crypt_iv_plain_ops;
- else if (strcmp(ivmode, "plain64") == 0)
- cc->iv_gen_ops = &crypt_iv_plain64_ops;
- else if (strcmp(ivmode, "essiv") == 0)
- cc->iv_gen_ops = &crypt_iv_essiv_ops;
- else if (strcmp(ivmode, "benbi") == 0)
- cc->iv_gen_ops = &crypt_iv_benbi_ops;
- else if (strcmp(ivmode, "null") == 0)
- cc->iv_gen_ops = &crypt_iv_null_ops;
- else if (strcmp(ivmode, "lmk") == 0) {
- cc->iv_gen_ops = &crypt_iv_lmk_ops;
+ if (strcmp(ivmode, "lmk") == 0) {
/*
* Version 2 and 3 is recognised according
* to length of provided multi-key string.
@@ -1813,39 +2683,14 @@ static int crypt_ctr_cipher(struct dm_target *ti,
cc->key_extra_size = cc->key_size / cc->key_parts;
}
} else if (strcmp(ivmode, "tcw") == 0) {
- cc->iv_gen_ops = &crypt_iv_tcw_ops;
cc->key_parts += 2; /* IV + whitening */
cc->key_extra_size = cc->iv_size + TCW_WHITENING_SIZE;
- } else {
- ret = -EINVAL;
- ti->error = "Invalid IV mode";
- goto bad;
}
/* Initialize and set key */
- ret = crypt_set_key(cc, key);
- if (ret < 0) {
- ti->error = "Error decoding and setting key";
+ ret = crypt_init_key(ti, key, ivopts);
+ if (ret < 0)
goto bad;
- }
-
- /* Allocate IV */
- if (cc->iv_gen_ops && cc->iv_gen_ops->ctr) {
- ret = cc->iv_gen_ops->ctr(cc, ti, ivopts);
- if (ret < 0) {
- ti->error = "Error creating IV";
- goto bad;
- }
- }
-
- /* Initialize IV (set keys for ESSIV etc) */
- if (cc->iv_gen_ops && cc->iv_gen_ops->init) {
- ret = cc->iv_gen_ops->init(cc);
- if (ret < 0) {
- ti->error = "Error initialising IV";
- goto bad;
- }
- }
ret = 0;
bad:
@@ -1901,20 +2746,20 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv)
goto bad;
cc->dmreq_start = sizeof(struct skcipher_request);
- cc->dmreq_start += crypto_skcipher_reqsize(any_tfm(cc));
+ cc->dmreq_start += crypto_skcipher_reqsize(cc->tfm);
cc->dmreq_start = ALIGN(cc->dmreq_start, __alignof__(struct dm_crypt_request));
- if (crypto_skcipher_alignmask(any_tfm(cc)) < CRYPTO_MINALIGN) {
+ if (crypto_skcipher_alignmask(cc->tfm) < CRYPTO_MINALIGN) {
/* Allocate the padding exactly */
iv_size_padding = -(cc->dmreq_start + sizeof(struct dm_crypt_request))
- & crypto_skcipher_alignmask(any_tfm(cc));
+ & crypto_skcipher_alignmask(cc->tfm);
} else {
/*
* If the cipher requires greater alignment than kmalloc
* alignment, we don't know the exact position of the
* initialization vector. We must assume worst case.
*/
- iv_size_padding = crypto_skcipher_alignmask(any_tfm(cc));
+ iv_size_padding = crypto_skcipher_alignmask(cc->tfm);
}
ret = -ENOMEM;
@@ -2072,8 +2917,9 @@ static int crypt_map(struct dm_target *ti, struct bio *bio)
if (bio_data_dir(io->base_bio) == READ) {
if (kcryptd_io_read(io, GFP_NOWAIT))
kcryptd_queue_read(io);
- } else
+ } else {
kcryptd_queue_crypt(io);
+ }
return DM_MAPIO_SUBMITTED;
}
@@ -2155,7 +3001,7 @@ static void crypt_resume(struct dm_target *ti)
static int crypt_message(struct dm_target *ti, unsigned argc, char **argv)
{
struct crypt_config *cc = ti->private;
- int key_size, ret = -EINVAL;
+ int key_size;
if (argc < 2)
goto error;
@@ -2173,19 +3019,9 @@ static int crypt_message(struct dm_target *ti, unsigned argc, char **argv)
return -EINVAL;
}
- ret = crypt_set_key(cc, argv[2]);
- if (ret)
- return ret;
- if (cc->iv_gen_ops && cc->iv_gen_ops->init)
- ret = cc->iv_gen_ops->init(cc);
- return ret;
+ return crypt_set_key(cc, SETKEY_OP_SET, argv[2], NULL);
}
if (argc == 2 && !strcasecmp(argv[1], "wipe")) {
- if (cc->iv_gen_ops && cc->iv_gen_ops->wipe) {
- ret = cc->iv_gen_ops->wipe(cc);
- if (ret)
- return ret;
- }
return crypt_wipe_key(cc);
}
}
@@ -2216,7 +3052,7 @@ static void crypt_io_hints(struct dm_target *ti, struct queue_limits *limits)
static struct target_type crypt_target = {
.name = "crypt",
- .version = {1, 15, 0},
+ .version = {1, 16, 0},
.module = THIS_MODULE,
.ctr = crypt_ctr,
.dtr = crypt_dtr,
@@ -2234,6 +3070,7 @@ static int __init dm_crypt_init(void)
{
int r;
+ geniv_register_algs();
r = dm_register_target(&crypt_target);
if (r < 0)
DMERR("register failed %d", r);
@@ -2244,6 +3081,7 @@ static int __init dm_crypt_init(void)
static void __exit dm_crypt_exit(void)
{
dm_unregister_target(&crypt_target);
+ geniv_deregister_algs();
}
module_init(dm_crypt_init);
diff --git a/include/crypto/geniv.h b/include/crypto/geniv.h
new file mode 100644
index 0000000..599ce62
--- /dev/null
+++ b/include/crypto/geniv.h
@@ -0,0 +1,47 @@
+/*
+ * geniv: common interface for IV generation algorithms
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ *
+ */
+#ifndef _CRYPTO_GENIV_
+#define _CRYPTO_GENIV_
+
+#define SECTOR_SIZE (1 << SECTOR_SHIFT)
+#define SECTOR_MASK (~((1 << SECTOR_SHIFT) - 1))
+
+enum setkey_op {
+ SETKEY_OP_INIT,
+ SETKEY_OP_SET,
+ SETKEY_OP_WIPE,
+};
+
+struct geniv_key_info {
+ enum setkey_op keyop;
+ unsigned int tfms_count;
+ u8 *key;
+ unsigned int key_size;
+ unsigned int key_parts;
+ char *ivopts;
+};
+
+#define DECLARE_GENIV_KEY(c, op, n, k, sz, kp, opts) \
+ struct geniv_key_info c = { \
+ .keyop = op, \
+ .tfms_count = n, \
+ .key = k, \
+ .key_size = sz, \
+ .key_parts = kp, \
+ .ivopts = opts, \
+ }
+
+struct geniv_req_info {
+ sector_t iv_sector;
+ unsigned int nents;
+ u8 *iv;
+};
+
+#endif
--
Binoy Jayan
^ permalink raw reply related
* [RFC PATCH v5] IV Generation algorithms for dm-crypt
From: Binoy Jayan @ 2017-04-07 10:47 UTC (permalink / raw)
To: Oded, Ofir
Cc: Herbert Xu, David S. Miller, linux-crypto, Mark Brown,
Arnd Bergmann, linux-kernel, Alasdair Kergon, Mike Snitzer,
dm-devel, Shaohua Li, linux-raid, Rajendra, Milan Broz, Gilad,
Binoy Jayan
===============================================================================
dm-crypt optimization for larger block sizes
===============================================================================
Currently, the iv generation algorithms are implemented in dm-crypt.c. The goal
is to move these algorithms from the dm layer to the kernel crypto layer by
implementing them as template ciphers so they can be used in relation with
algorithms like aes, and with multiple modes like cbc, ecb etc. As part of this
patchset, the iv-generation code is moved from the dm layer to the crypto layer
and adapt the dm-layer to send a whole 'bio' (as defined in the block layer)
at a time. Each bio contains the in memory representation of physically
contiguous disk blocks. Since the bio itself may not be contiguous in main
memory, the dm layer sets up a chained scatterlist of these blocks split into
physically contiguous segments in memory so that DMA can be performed.
One challenge in doing so is that the IVs are generated based on a 512-byte
sector number. This infact limits the block sizes to 512 bytes. But this should
not be a problem if a hardware with iv generation support is used. The geniv
itself splits the segments into sectors so it could choose the IV based on
sector number. But it could be modelled in hardware effectively by not
splitting up the segments in the bio.
Another challenge faced is that dm-crypt has an option to use multiple keys.
The key selection is done based on the sector number. If the whole bio is
encrypted / decrypted with the same key, the encrypted volumes will not be
compatible with the original dm-crypt [without the changes]. So, the key
selection code is moved to crypto layer so the neighboring sectors are
encrypted with a different key.
The dm layer allocates space for iv. The hardware drivers can choose to make
use of this space to generate their IVs sequentially or allocate it on their
own. This can be moved to crypto layer too. Postponing this decision until
the requirement to integrate milan's changes are clear.
Interface to the crypto layer - include/crypto/geniv.h
More information on test procedure can be found in v1.
-------------------------------------------------------------------------------
Peformance comparison [Tests on 1 GiB Volume] on db410c
Test script:
https://github.com/binoyjayan/utilities/blob/master/utils/dmtest
dmtest -d <block device> -o out.log -s 1024 -r 384 -f 768
-------------------------------------------------------------------------------
This includes tests done with dd, fio and bonnie++ with the original dm-crypt
and the proposed solution with algorithm 'essiv(cbc(aes-arm))' implemented
in software. The hardware is yet to be evaluated. These tests are to make sure
there is no drastic performance degradation on systems without hw crypto.
Tests with dd [direct i/o]
Sequential read -0.134 %
Sequential Write +0.091 %
Tests with fio [Aggregate bandwidth - aggrb]
Random Read +0.358 %
Random Write +0.010 %
Tests with bonnie++ [768 MiB File, 384 MiB Ram]
after mounting dm-crypt target as ext4
Sequential o/p [per-char] -2.876 %
Sequential o/p [per-blk] +0.992 %
Sequential o/p [re-write] +4.465 %
Sequential i/p [per-char] -0.453 %
Sequential i/p [per-blk] -0.740 %
Sequential create -0.255 %
Sequential delete +0.042 %
Random create -0.007 %
Random delete +0.454 %
NB: The '+' sign shows improvement and '-' shows degradation.
The tests were performed with minimal cpu load.
Tests with higher cpu load to be done
Revisions:
----------
v1: https://patchwork.kernel.org/patch/9439175
v2: https://patchwork.kernel.org/patch/9471923
v3: https://lkml.org/lkml/2017/1/18/170
v4: https://patchwork.kernel.org/patch/9559665
v4 --> v5
----------
1. Fix for the multiple instance issue in /proc/crypto
2. Few cosmetic changes including struct alignment
3. Simplified 'struct geniv_req_info'
v3 --> v4
----------
Fix for the bug reported by Gilad Ben-Yossef.
The element '__ctx' in 'struct skcipher_request req' overflowed into the
element 'struct scatterlist src' which immediately follows 'req' in
'struct geniv_subreq' and corrupted src.
v2 --> v3
----------
1. Moved iv algorithms in dm-crypt.c for control
2. Key management code moved from dm layer to cryto layer
so that cipher instance selection can be made depending on key_index
3. The revision v2 had scatterlist nodes created for every sector in the bio.
It is modified to create only once scatterlist node to reduce memory
foot print. Synchronous requests are processed sequentially. Asynchronous
requests are processed in parallel and is freed in the async callback.
4. Changed allocation for sub-requests using mempool
v1 --> v2
----------
1. dm-crypt changes to process larger block sizes (one segment in a bio)
2. Incorporated changes w.r.t. comments from Herbert.
Binoy Jayan (1):
crypto: Add IV generation algorithms
drivers/md/dm-crypt.c | 1916 ++++++++++++++++++++++++++++++++++--------------
include/crypto/geniv.h | 47 ++
2 files changed, 1424 insertions(+), 539 deletions(-)
create mode 100644 include/crypto/geniv.h
--
Binoy Jayan
^ permalink raw reply
* Re: hmac(crc32)
From: Herbert Xu @ 2017-04-07 6:14 UTC (permalink / raw)
To: Stephan Müller; +Cc: linux-crypto
In-Reply-To: <1891573.4Vd40q9uOn@positron.chronox.de>
On Wed, Mar 08, 2017 at 09:20:08AM +0100, Stephan Müller wrote:
> Hi Herbert,
>
> hmac(crc32) is defined in testmgr.c and tcrypt.c. Yet, when using that cipher,
> I get an ENOENT:
>
> alg: hash: Failed to load transform for hmac(crc32): -2
>
>
> Is there such a thing as hmac(crc32)?
Yes this is a bogus driver implementation that sneaked through.
We should remove bfin_crc along with the test vector.
Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: dm-crypt IV generation (summary)
From: Herbert Xu @ 2017-04-07 6:12 UTC (permalink / raw)
To: Ondrej Mosnacek
Cc: Milan Broz, linux-crypto, dm-devel, Mike Snitzer, Alasdair Kergon,
Mikulas Patocka, Binoy Jayan, Gilad Ben-Yossef
In-Reply-To: <CAFqZXNs0=6+PvpGq3weRwCJkUOfSn06i_fD6ZuvXo_0rms9zeA@mail.gmail.com>
On Fri, Mar 10, 2017 at 02:44:26PM +0100, Ondrej Mosnacek wrote:
>
> ISSUES:
> a) The 'keycount' parameter.
> In order to support multi-key modes from Loop-AES,
> dm-crypt accepts a keycount parameter which, if it != 1, causes
> consecutive sectors to be encrypted with a different key. This
> parameter can be specified with any of the cipher modes, which makes
> porting the whole scale of modes supported by dm-crypt to crypto API
> rather messy, since a lot of dm-crypt specific stuff needs to be moved
> into the crypto drivers.
Actually I think this one can probably easily handled in the crypto
layer. All we need is to add a multikey template that sits on top
of an underlying IV generator. The multikey instance can then accept
a key length that is a multiple of the underlying key length.
> b) New AEAD functionality; random IV generator.
> The soon-to-be-added AEAD functionality in dm-crypt
> introduces a new IV generator that generates IVs randomly and stores
> them as sector metadata. This means IV generation cannot be handled
> solely in the driver. Also, additional AEAD implementation of IV
> generators would be eventually needed.
Again I don't see the problem here. IV generators are supposed
to return the IV to the caller so that it can be transmitted.
For example, the IV generated in IPsec is explicitly transmitted.
Here we just need to store the IV.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 2/4] crypto: af_alg - Allow arbitrarily long algorithm names
From: Herbert Xu @ 2017-04-07 5:25 UTC (permalink / raw)
To: Alexander Sverdlin; +Cc: David S. Miller, linux-crypto, netdev
In-Reply-To: <32d123fa-5c5f-4db6-d333-24b71639b738@nokia.com>
On Thu, Apr 06, 2017 at 02:32:27PM +0200, Alexander Sverdlin wrote:
>
> > diff --git a/crypto/af_alg.c b/crypto/af_alg.c
> > index 690deca..3556d8e 100644
> > --- a/crypto/af_alg.c
> > +++ b/crypto/af_alg.c
> > @@ -160,11 +160,11 @@ static int alg_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len)
> > if (sock->state == SS_CONNECTED)
> > return -EINVAL;
> >
> > - if (addr_len != sizeof(*sa))
> > + if (addr_len < sizeof(*sa))
> > return -EINVAL;
> >
> > sa->salg_type[sizeof(sa->salg_type) - 1] = 0;
> > - sa->salg_name[sizeof(sa->salg_name) - 1] = 0;
> > + sa->salg_name[sizeof(sa->salg_name) + addr_len - sizeof(*sa) - 1] = 0;
> >
> > type = alg_get_type(sa->salg_type);
> > if (IS_ERR(type) && PTR_ERR(type) == -ENOENT) {
>
> Why should userspace ever extend the structure if salg_name is hardcoded to 64 in if_alg.h?
> This patch doesn't change the behavior at all, or am I missing something?
We cannot change the structure in the user-space API by increasing
its length because that would break backwards compatibility.
What this patch does is relax the size check so that user-space can
pass in a name that is longer than 64 bytes. This would fail on
older kernels with EINVAL. Of course, user-space must be modified
to allow for such longer names. That is beyond the scope of this
patch.
For names shorter than 64 bytes the existing API needs be respected
and user-space must pad it to exactly 64 bytes.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* [PATCH] padata: free correct variable
From: Jason A. Donenfeld @ 2017-04-07 0:33 UTC (permalink / raw)
To: linux-crypto, steffen.klassert, linux-kernel; +Cc: Jason A. Donenfeld, stable
The author meant to free the variable that was just allocated, instead
of the one that failed to be allocated, but made a simple typo. This
patch rectifies that.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Cc: stable@vger.kernel.org
---
kernel/padata.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/padata.c b/kernel/padata.c
index 05316c9f32da..1faa7d62d87d 100644
--- a/kernel/padata.c
+++ b/kernel/padata.c
@@ -353,7 +353,7 @@ static int padata_setup_cpumasks(struct parallel_data *pd,
cpumask_and(pd->cpumask.pcpu, pcpumask, cpu_online_mask);
if (!alloc_cpumask_var(&pd->cpumask.cbcpu, GFP_KERNEL)) {
- free_cpumask_var(pd->cpumask.cbcpu);
+ free_cpumask_var(pd->cpumask.pcpu);
return -ENOMEM;
}
--
2.12.2
^ permalink raw reply related
* Re: [PATCH 3/4] xfrm: Prepare for CRYPTO_MAX_ALG_NAME expansion
From: Steffen Klassert @ 2017-04-06 21:15 UTC (permalink / raw)
To: Herbert Xu; +Cc: Alexander Sverdlin, David S. Miller, linux-crypto, netdev
In-Reply-To: <E1cw2aQ-00085u-SY@gondobar>
On Thu, Apr 06, 2017 at 04:16:10PM +0800, Herbert Xu wrote:
> This patch fixes the xfrm_user code to use the actual array size
> rather than the hard-coded CRYPTO_MAX_ALG_NAME length. This is
> because the array size is fixed at 64 bytes while we want to increase
> the in-kernel CRYPTO_MAX_ALG_NAME value.
>
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Acked-by: Steffen Klassert <steffen.klassert@secunet.com>
^ permalink raw reply
* Re: [PATCH 0/4] crypto: CRYPTO_MAX_ALG_NAME is too low
From: Steffen Klassert @ 2017-04-06 21:14 UTC (permalink / raw)
To: David Miller; +Cc: herbert, alexander.sverdlin, linux-crypto, netdev
In-Reply-To: <20170406.135832.1914702009261438916.davem@davemloft.net>
On Thu, Apr 06, 2017 at 01:58:32PM -0700, David Miller wrote:
> From: Herbert Xu <herbert@gondor.apana.org.au>
> Date: Thu, 6 Apr 2017 16:15:09 +0800
>
> > As the final patch depends on all three it would be easiest if
> > we pushed the xfrm patch through the crypto tree. Steffen/David?
>
> No objections from me for this going through the crypto tree.
I have no objections too.
^ permalink raw reply
* Re: [PATCH 0/4] crypto: CRYPTO_MAX_ALG_NAME is too low
From: David Miller @ 2017-04-06 20:58 UTC (permalink / raw)
To: herbert; +Cc: alexander.sverdlin, linux-crypto, netdev
In-Reply-To: <20170406081509.GB30557@gondor.apana.org.au>
From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Thu, 6 Apr 2017 16:15:09 +0800
> As the final patch depends on all three it would be easiest if
> we pushed the xfrm patch through the crypto tree. Steffen/David?
No objections from me for this going through the crypto tree.
^ permalink raw reply
* Re: [PATCH 3/4] xfrm: Prepare for CRYPTO_MAX_ALG_NAME expansion
From: Alexander Sverdlin @ 2017-04-06 15:10 UTC (permalink / raw)
To: Herbert Xu, David S. Miller, linux-crypto, netdev
In-Reply-To: <E1cw2aQ-00085u-SY@gondobar>
On 06/04/17 10:16, Herbert Xu wrote:
> This patch fixes the xfrm_user code to use the actual array size
> rather than the hard-coded CRYPTO_MAX_ALG_NAME length. This is
> because the array size is fixed at 64 bytes while we want to increase
> the in-kernel CRYPTO_MAX_ALG_NAME value.
>
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Acked-by: Alexander Sverdlin <alexander.sverdlin@nokia.com>
Tested-by: Alexander Sverdlin <alexander.sverdlin@nokia.com>
> ---
>
> net/xfrm/xfrm_user.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c
> index 9705c27..96557cf 100644
> --- a/net/xfrm/xfrm_user.c
> +++ b/net/xfrm/xfrm_user.c
> @@ -55,7 +55,7 @@ static int verify_one_alg(struct nlattr **attrs, enum xfrm_attr_type_t type)
> return -EINVAL;
> }
>
> - algp->alg_name[CRYPTO_MAX_ALG_NAME - 1] = '\0';
> + algp->alg_name[sizeof(algp->alg_name) - 1] = '\0';
> return 0;
> }
>
> @@ -71,7 +71,7 @@ static int verify_auth_trunc(struct nlattr **attrs)
> if (nla_len(rt) < xfrm_alg_auth_len(algp))
> return -EINVAL;
>
> - algp->alg_name[CRYPTO_MAX_ALG_NAME - 1] = '\0';
> + algp->alg_name[sizeof(algp->alg_name) - 1] = '\0';
> return 0;
> }
>
> @@ -87,7 +87,7 @@ static int verify_aead(struct nlattr **attrs)
> if (nla_len(rt) < aead_len(algp))
> return -EINVAL;
>
> - algp->alg_name[CRYPTO_MAX_ALG_NAME - 1] = '\0';
> + algp->alg_name[sizeof(algp->alg_name) - 1] = '\0';
> return 0;
> }
>
>
--
Best regards,
Alexander Sverdlin.
^ permalink raw reply
* Re: [PATCH 4/4] crypto: api - Extend algorithm name limit to 128 bytes
From: Alexander Sverdlin @ 2017-04-06 15:11 UTC (permalink / raw)
To: Herbert Xu, David S. Miller, linux-crypto, netdev
In-Reply-To: <E1cw2aR-000868-VF@gondobar>
On 06/04/17 10:16, Herbert Xu wrote:
> With the new explicit IV generators, we may now exceed the 64-byte
> length limit on the algorithm name, e.g., with
>
> echainiv(authencesn(hmac(sha256-generic),cbc(des3_ede-generic)))
>
> This patch extends the length limit to 128 bytes.
>
> Reported-by: Alexander Sverdlin <alexander.sverdlin@nokia.com>
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Acked-by: Alexander Sverdlin <alexander.sverdlin@nokia.com>
Tested-by: Alexander Sverdlin <alexander.sverdlin@nokia.com>
> ---
>
> include/linux/crypto.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/include/linux/crypto.h b/include/linux/crypto.h
> index c0b0cf3..84da997 100644
> --- a/include/linux/crypto.h
> +++ b/include/linux/crypto.h
> @@ -123,7 +123,7 @@
> /*
> * Miscellaneous stuff.
> */
> -#define CRYPTO_MAX_ALG_NAME 64
> +#define CRYPTO_MAX_ALG_NAME 128
>
> /*
> * The macro CRYPTO_MINALIGN_ATTR (along with the void * type in the actual
>
--
Best regards,
Alexander Sverdlin.
^ permalink raw reply
* Re: [PATCH 1/4] crypto: user - Prepare for CRYPTO_MAX_ALG_NAME expansion
From: Alexander Sverdlin @ 2017-04-06 15:10 UTC (permalink / raw)
To: Herbert Xu, David S. Miller, linux-crypto, netdev
In-Reply-To: <E1cw2aO-00085E-NH@gondobar>
On 06/04/17 10:16, Herbert Xu wrote:
> This patch hard-codes CRYPTO_MAX_NAME in the user-space API to
> 64, which is the current value of CRYPTO_MAX_ALG_NAME. This patch
> also replaces all remaining occurences of CRYPTO_MAX_ALG_NAME
> in the user-space API with CRYPTO_MAX_NAME.
>
> This way the user-space API will not be modified when we raise
> the value of CRYPTO_MAX_ALG_NAME.
>
> Furthermore, the code has been updated to handle names longer than
> the user-space API. They will be truncated.
>
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Acked-by: Alexander Sverdlin <alexander.sverdlin@nokia.com>
Tested-by: Alexander Sverdlin <alexander.sverdlin@nokia.com>
> ---
>
> crypto/crypto_user.c | 18 +++++++++---------
> include/uapi/linux/cryptouser.h | 10 +++++-----
> 2 files changed, 14 insertions(+), 14 deletions(-)
>
> diff --git a/crypto/crypto_user.c b/crypto/crypto_user.c
> index a90404a..89acaab 100644
> --- a/crypto/crypto_user.c
> +++ b/crypto/crypto_user.c
> @@ -83,7 +83,7 @@ static int crypto_report_cipher(struct sk_buff *skb, struct crypto_alg *alg)
> {
> struct crypto_report_cipher rcipher;
>
> - strncpy(rcipher.type, "cipher", sizeof(rcipher.type));
> + strlcpy(rcipher.type, "cipher", sizeof(rcipher.type));
>
> rcipher.blocksize = alg->cra_blocksize;
> rcipher.min_keysize = alg->cra_cipher.cia_min_keysize;
> @@ -102,7 +102,7 @@ static int crypto_report_comp(struct sk_buff *skb, struct crypto_alg *alg)
> {
> struct crypto_report_comp rcomp;
>
> - strncpy(rcomp.type, "compression", sizeof(rcomp.type));
> + strlcpy(rcomp.type, "compression", sizeof(rcomp.type));
> if (nla_put(skb, CRYPTOCFGA_REPORT_COMPRESS,
> sizeof(struct crypto_report_comp), &rcomp))
> goto nla_put_failure;
> @@ -116,7 +116,7 @@ static int crypto_report_acomp(struct sk_buff *skb, struct crypto_alg *alg)
> {
> struct crypto_report_acomp racomp;
>
> - strncpy(racomp.type, "acomp", sizeof(racomp.type));
> + strlcpy(racomp.type, "acomp", sizeof(racomp.type));
>
> if (nla_put(skb, CRYPTOCFGA_REPORT_ACOMP,
> sizeof(struct crypto_report_acomp), &racomp))
> @@ -131,7 +131,7 @@ static int crypto_report_akcipher(struct sk_buff *skb, struct crypto_alg *alg)
> {
> struct crypto_report_akcipher rakcipher;
>
> - strncpy(rakcipher.type, "akcipher", sizeof(rakcipher.type));
> + strlcpy(rakcipher.type, "akcipher", sizeof(rakcipher.type));
>
> if (nla_put(skb, CRYPTOCFGA_REPORT_AKCIPHER,
> sizeof(struct crypto_report_akcipher), &rakcipher))
> @@ -146,7 +146,7 @@ static int crypto_report_kpp(struct sk_buff *skb, struct crypto_alg *alg)
> {
> struct crypto_report_kpp rkpp;
>
> - strncpy(rkpp.type, "kpp", sizeof(rkpp.type));
> + strlcpy(rkpp.type, "kpp", sizeof(rkpp.type));
>
> if (nla_put(skb, CRYPTOCFGA_REPORT_KPP,
> sizeof(struct crypto_report_kpp), &rkpp))
> @@ -160,10 +160,10 @@ static int crypto_report_kpp(struct sk_buff *skb, struct crypto_alg *alg)
> static int crypto_report_one(struct crypto_alg *alg,
> struct crypto_user_alg *ualg, struct sk_buff *skb)
> {
> - strncpy(ualg->cru_name, alg->cra_name, sizeof(ualg->cru_name));
> - strncpy(ualg->cru_driver_name, alg->cra_driver_name,
> + strlcpy(ualg->cru_name, alg->cra_name, sizeof(ualg->cru_name));
> + strlcpy(ualg->cru_driver_name, alg->cra_driver_name,
> sizeof(ualg->cru_driver_name));
> - strncpy(ualg->cru_module_name, module_name(alg->cra_module),
> + strlcpy(ualg->cru_module_name, module_name(alg->cra_module),
> sizeof(ualg->cru_module_name));
>
> ualg->cru_type = 0;
> @@ -176,7 +176,7 @@ static int crypto_report_one(struct crypto_alg *alg,
> if (alg->cra_flags & CRYPTO_ALG_LARVAL) {
> struct crypto_report_larval rl;
>
> - strncpy(rl.type, "larval", sizeof(rl.type));
> + strlcpy(rl.type, "larval", sizeof(rl.type));
> if (nla_put(skb, CRYPTOCFGA_REPORT_LARVAL,
> sizeof(struct crypto_report_larval), &rl))
> goto nla_put_failure;
> diff --git a/include/uapi/linux/cryptouser.h b/include/uapi/linux/cryptouser.h
> index 11d21fc..b4def5c 100644
> --- a/include/uapi/linux/cryptouser.h
> +++ b/include/uapi/linux/cryptouser.h
> @@ -31,7 +31,7 @@ enum {
> #define CRYPTO_MSG_MAX (__CRYPTO_MSG_MAX - 1)
> #define CRYPTO_NR_MSGTYPES (CRYPTO_MSG_MAX + 1 - CRYPTO_MSG_BASE)
>
> -#define CRYPTO_MAX_NAME CRYPTO_MAX_ALG_NAME
> +#define CRYPTO_MAX_NAME 64
>
> /* Netlink message attributes. */
> enum crypto_attr_type_t {
> @@ -53,9 +53,9 @@ enum crypto_attr_type_t {
> };
>
> struct crypto_user_alg {
> - char cru_name[CRYPTO_MAX_ALG_NAME];
> - char cru_driver_name[CRYPTO_MAX_ALG_NAME];
> - char cru_module_name[CRYPTO_MAX_ALG_NAME];
> + char cru_name[CRYPTO_MAX_NAME];
> + char cru_driver_name[CRYPTO_MAX_NAME];
> + char cru_module_name[CRYPTO_MAX_NAME];
> __u32 cru_type;
> __u32 cru_mask;
> __u32 cru_refcnt;
> @@ -73,7 +73,7 @@ struct crypto_report_hash {
> };
>
> struct crypto_report_cipher {
> - char type[CRYPTO_MAX_ALG_NAME];
> + char type[CRYPTO_MAX_NAME];
> unsigned int blocksize;
> unsigned int min_keysize;
> unsigned int max_keysize;
>
--
Best regards,
Alexander Sverdlin.
^ permalink raw reply
* Re: [PATCH 0/4] crypto: CRYPTO_MAX_ALG_NAME is too low
From: Alexander Sverdlin @ 2017-04-06 15:10 UTC (permalink / raw)
To: Herbert Xu; +Cc: David S. Miller, linux-crypto, netdev
In-Reply-To: <20170406081509.GB30557@gondor.apana.org.au>
Hi!
On 06/04/17 10:15, Herbert Xu wrote:
> On Thu, Mar 16, 2017 at 03:16:29PM +0100, Alexander Sverdlin wrote:
>> This is a regression caused by 856e3f4092
>> ("crypto: seqiv - Add support for new AEAD interface")
>>
>> As I've said above, I can offer one of the two solutions, which patch should I send?
>> Or do you see any better alternatives?
> Here is a series of patches which should fix the problem.
>
> The first three patches prepare the user-space interfaces to deal
> with longer names. The final patch extends it.
>
> Note that with crypto_user I haven't actually extended it to
> configure longer names. It'll only be able to configure names
> less than 64 bytes. However, it should be able to dump/read
> algorithms with longer names, albeit the name will be truncated
> to 64 bytes length.
>
> Steffen, when convenient could you look into extending the crypto
> user interface to handle longer names (preferably arbitraty length
> since netlink should be able to deal with that)?
>
> Likewise xfrm is still fixed to 64 bytes long. But this should
> be OK as the problematic case only arises with IV generators for
> now and we do not allow IV generators to be specified through xfrm.
>
> af_alg on the other hand now allows arbitrarily long names.
I'm not sure about patch 2 (as I've replied separately), but I've applied
and tested the whole series and it at least solves the original problem
with long algorithm name.
> As the final patch depends on all three it would be easiest if
> we pushed the xfrm patch through the crypto tree. Steffen/David?
--
Best regards,
Alexander Sverdlin.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox