[v2] USB: wusbcore: crypto: Remove VLA usage

[v2] USB: wusbcore: crypto: Remove VLA usage

[Gustavo A. R. Silva]

In preparation to enabling -Wvla, remove VLA and replace it
with dynamic memory allocation instead.

The use of stack Variable Length Arrays needs to be avoided, as they
can be a vector for stack exhaustion, which can be both a runtime bug
or a security flaw. Also, in general, as code evolves it is easy to
lose track of how big a VLA can get. Thus, we can end up having runtime
failures that are hard to debug.

Also, fixed as part of the directive to remove all VLAs from
the kernel: https://lkml.org/lkml/2018/3/7/621

Notice that in this particular case, an alternative to kzalloc is kcalloc,
in which case the code would look as follows instead:

iv = kcalloc(crypto_skcipher_ivsize(tfm_cbc), sizeof(*iv), GFP_KERNEL);

but if the data type of _iv_ never changes, or the type size is always one
byte, kzalloc is good enough.

Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com>
---
Changes in v2:
 - Fix a memory leak in previous patch.

 drivers/usb/wusbcore/crypto.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/usb/wusbcore/crypto.c b/drivers/usb/wusbcore/crypto.c
index 4c00be2d..aff50eb 100644
--- a/drivers/usb/wusbcore/crypto.c
+++ b/drivers/usb/wusbcore/crypto.c
@@ -202,7 +202,7 @@ static int wusb_ccm_mac(struct crypto_skcipher *tfm_cbc,
 	struct scatterlist sg[4], sg_dst;
 	void *dst_buf;
 	size_t dst_size;
-	u8 iv[crypto_skcipher_ivsize(tfm_cbc)];
+	u8 *iv;
 	size_t zero_padding;
 
 	/*
@@ -224,7 +224,9 @@ static int wusb_ccm_mac(struct crypto_skcipher *tfm_cbc,
 	if (!dst_buf)
 		goto error_dst_buf;
 
-	memset(iv, 0, sizeof(iv));
+	iv = kzalloc(crypto_skcipher_ivsize(tfm_cbc), GFP_KERNEL);
+	if (!iv)
+		goto error_iv;
 
 	/* Setup B0 */
 	scratch->b0.flags = 0x59;	/* Format B0 */
@@ -276,6 +278,8 @@ static int wusb_ccm_mac(struct crypto_skcipher *tfm_cbc,
 	bytewise_xor(mic, &scratch->ax, iv, 8);
 	result = 8;
 error_cbc_crypt:
+	kfree(iv);
+error_iv:
 	kfree(dst_buf);
 error_dst_buf:
 	return result;

[PATCH v2] USB: wusbcore: crypto: Remove VLA usage

[Gustavo A. R. Silva]

In preparation to enabling -Wvla, remove VLA and replace it
with dynamic memory allocation instead.

The use of stack Variable Length Arrays needs to be avoided, as they
can be a vector for stack exhaustion, which can be both a runtime bug
or a security flaw. Also, in general, as code evolves it is easy to
lose track of how big a VLA can get. Thus, we can end up having runtime
failures that are hard to debug.

Also, fixed as part of the directive to remove all VLAs from
the kernel: https://lkml.org/lkml/2018/3/7/621

Notice that in this particular case, an alternative to kzalloc is kcalloc,
in which case the code would look as follows instead:

iv = kcalloc(crypto_skcipher_ivsize(tfm_cbc), sizeof(*iv), GFP_KERNEL);

but if the data type of _iv_ never changes, or the type size is always one
byte, kzalloc is good enough.

Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com>
---
Changes in v2:
 - Fix a memory leak in previous patch.

 drivers/usb/wusbcore/crypto.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/usb/wusbcore/crypto.c b/drivers/usb/wusbcore/crypto.c
index 4c00be2d..aff50eb 100644
--- a/drivers/usb/wusbcore/crypto.c
+++ b/drivers/usb/wusbcore/crypto.c
@@ -202,7 +202,7 @@ static int wusb_ccm_mac(struct crypto_skcipher *tfm_cbc,
 	struct scatterlist sg[4], sg_dst;
 	void *dst_buf;
 	size_t dst_size;
-	u8 iv[crypto_skcipher_ivsize(tfm_cbc)];
+	u8 *iv;
 	size_t zero_padding;
 
 	/*
@@ -224,7 +224,9 @@ static int wusb_ccm_mac(struct crypto_skcipher *tfm_cbc,
 	if (!dst_buf)
 		goto error_dst_buf;
 
-	memset(iv, 0, sizeof(iv));
+	iv = kzalloc(crypto_skcipher_ivsize(tfm_cbc), GFP_KERNEL);
+	if (!iv)
+		goto error_iv;
 
 	/* Setup B0 */
 	scratch->b0.flags = 0x59;	/* Format B0 */
@@ -276,6 +278,8 @@ static int wusb_ccm_mac(struct crypto_skcipher *tfm_cbc,
 	bytewise_xor(mic, &scratch->ax, iv, 8);
 	result = 8;
 error_cbc_crypt:
+	kfree(iv);
+error_iv:
 	kfree(dst_buf);
 error_dst_buf:
 	return result;
-- 
2.7.4