* RE: [PATCH 1/1] crypto: If two strings are exact match, they must have same length.
From: Ming Ma (mingma) @ 2017-03-27 23:04 UTC (permalink / raw)
To: Herbert Xu
Cc: davem@davemloft.net, linux-crypto@vger.kernel.org,
linux-kernel@vger.kernel.org
In-Reply-To: <20170322030040.GB29214@gondor.apana.org.au>
Please ignore this patch, we have seen some issues in older verison of linux kernel. But it doesn't seem to be an issue in the latest kernel.
thanks
-----Original Message-----
From: linux-crypto-owner@vger.kernel.org [mailto:linux-crypto-owner@vger.kernel.org] On Behalf Of Herbert Xu
Sent: Tuesday, March 21, 2017 8:01 PM
To: Ming Ma (mingma) <mingma@micron.com>
Cc: davem@davemloft.net; linux-crypto@vger.kernel.org; linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/1] crypto: If two strings are exact match, they must have same length.
On Tue, Mar 21, 2017 at 04:40:40PM -0500, Ming Ma wrote:
> When both "crct10dif-pclmul" algorithm and "crct10dif-generic"
> algorithm exist in crypto_alg_list, "crct10dif-pclmul" should be
> selected, since it has higher priority than "crct10dif-generic".
> However, both algorithms have the same cra_name "crct10dif". If we use
> "crct10dif" to find a matched algorithm in crypto_alg_list, it's
> possible "crct10dif-generic" is selected, because the code calls
> strcmp to decide if two string are exact match, but doesn't check if two strings have the same length.
>
> exact = !strcmp(q->cra_driver_name, name);
>
> So ,if "crct10dif-generic" is in front of "crct10dif-pclmul" in
> crypto_alg_list, it will be picked as the matched algorithm, even if
> it has lower priority than "crct10dif-pclmul".
> Signed-off-by: Ming Ma <mingma@micron.com>
> ---
> crypto/api.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/crypto/api.c b/crypto/api.c index b16ce16..5b3d45a 100644
> --- a/crypto/api.c
> +++ b/crypto/api.c
> @@ -76,7 +76,8 @@ static struct crypto_alg *__crypto_alg_lookup(const char *name, u32 type,
> ((struct crypto_larval *)q)->mask != mask)
> continue;
>
> - exact = !strcmp(q->cra_driver_name, name);
> + exact = (strlen(name) == strlen(q->cra_driver_name)) &&
> + !strcmp(q->cra_driver_name, name);
> fuzzy = !strcmp(q->cra_name, name);
> if (!exact && !(fuzzy && q->cra_priority > best))
> continue;
This is bogus. Please describe how you reproduced the problem.
The priority matching should work.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au> Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 0/7] crypto: aes - allow generic AES to be omitted
From: Eric Biggers @ 2017-03-28 5:43 UTC (permalink / raw)
To: Ard Biesheuvel; +Cc: linux-crypto, herbert, nico
In-Reply-To: <1490554148-10953-1-git-send-email-ard.biesheuvel@linaro.org>
Hi Ard,
On Sun, Mar 26, 2017 at 07:49:01PM +0100, Ard Biesheuvel wrote:
> The generic AES driver uses 16 lookup tables of 1 KB each, and has
> encryption and decryption routines that are fully unrolled. Given how
> the dependencies between this code and other drivers are declared in
> Kconfig files, this code is always pulled into the core kernel, even
> if it is usually superseded at runtime by accelerated drivers that
> exist for many architectures.
>
> This leaves us with 25 KB of dead code in the kernel, which is negligible
> in typical environments, but which is actually a big deal for the IoT
> domain, where every kilobyte counts.
>
> For this reason, this series refactors the way the various AES
> implementations are wired up, to allow the generic version in
> crypto/aes_generic.c to be omitted from the build entirely.
>
> Patch #1 removes some bogus 'select CRYPTO_AES' statement.
>
> Patch #2 introduces CRYPTO_NEED_AES which can be selected by driver that
> require an AES cipher to be available, but don't care how it is implemented.
>
> Patches #3 and #4 make some preparatory changes that allow dependencies on
> crypto_aes_expand_key to be fulfilled by the new (and much smaller) fixed
> time AES driver. (#5)
>
> Patch #6 splits the generic AES driver into a core containing the precomputed
> sub/shift/mix tables and the key expansion routines on the one hand, and the
> encryption/decryption routines and the crypto API registration on the other.
>
> Patch #7 introduces the CRYPTO_HAVE_AES Kconfig symbol, and adds statements to
> various AES implementations that can fulfil the CRYPTO_NEED_AES dependencies
> added in patch #2. The introduced Kconfig logic allows CRYPTO_AES to be
> deselected even if AES dependencies exist, as long as one of these alternatives
> is selected.
Just a thought: how about renaming CRYPTO_AES to CRYPTO_AES_GENERIC, then
renaming what you called CRYPTO_NEED_AES to CRYPTO_AES? Then all the 'select
CRYPTO_AES' can remain as-is, instead of replacing them with the (in my opinion
uglier) 'select CRYPTO_NEED_AES'. And it should still work for people who have
CRYPTO_AES=y or CRYPTO_AES=m in their kernel config, since they'll still get at
least one AES implementation (though they may stop getting the generic one).
Also, in general I think we need better Kconfig help text. As proposed you can
now toggle simply "AES cipher algorithms", and nowhere in the help text is it
mentioned that that is only the generic implementation, which you don't need if
you have enabled some other implementation. Similarly for "Fixed time AES
cipher"; it perhaps should be mentioned that it's only useful if a fixed-time
implementation using special CPU instructions like AES-NI or ARMv8-CE isn't
usable.
- Eric
^ permalink raw reply
* Re: [PATCH] arm64: dts: ls1012a: add crypto node
From: Horia Geantă @ 2017-03-28 7:19 UTC (permalink / raw)
To: Shawn Guo
Cc: Mark Rutland, devicetree@vger.kernel.org, Herbert Xu,
Harninder Rai, Catalin Marinas, Bhaskar U, Will Deacon,
Dan Douglass, Rob Herring, linux-crypto@vger.kernel.org,
David S. Miller, linux-arm-kernel@lists.infradead.org
In-Reply-To: <20170324140320.GP30608@dragon>
On 3/24/2017 4:04 PM, Shawn Guo wrote:
> On Fri, Mar 24, 2017 at 08:29:17AM +0000, Horia Geantă wrote:
>> On 3/24/2017 9:35 AM, Shawn Guo wrote:
>>> On Fri, Mar 24, 2017 at 07:17:50AM +0000, Horia Geantă wrote:
>>>>>> + sec_mon: sec_mon@1e90000 {
>>>>>
>>>>> Hyphen is more preferred to be used in node name than underscore.
>>>>>
>>>> This would imply changing the
>>>> Documentation/devicetree/bindings/crypto/fsl-sec4.txt binding and
>>>> dealing with all the consequences, which IIUC is probably not worth.
>>>
>>> I do not care the bindings doc that much, since I'm not the maintainer
>>> of it. What are the consequences specifically, if we use a better node
>>> name in dts than bindings example?
>>>
>> Users relying on finding the sec_mon node will obviously stop working.
>> I don't see any in-kernel users, however there could be others I am not
>> aware of and DT bindings should provide for backwards compatibility.
>
> Okay, point taken. You can keep the node name as it is.
>
>> I could deprecate "sec_mon" in the bindings and suggest "sec-mon"
>> instead, while leaving all existing dts files as-is.
>> The risk is breaking LS1012A users relying on "sec_mon".
>
> For existing bindings, I do not care that much. But for new ones, I do
> hope that we recommend to use hyphen, as that's more idiomatic at least
> for Linux kernel.
>
>> I see that ePAPR:
>> -allows both for hyphen and underline in case of node names
>> -allows only for hyphen (i.e. forbids underline) in case of alias nodes
>>
>> In the first case, I understand there's an (undocumented?) agreement to
>> prefer hyphen over underline.
>
> Both are valid, but hyphen is more idiomatic for Linux kernel.
>
>> For the 2nd one, does this mean I should change alias names?
>
> This is something I see difference between specification and DTC.
>
> aliases {
> alias-name = &label_name;
> };
>
> label_name: node-name {
> ...
> };
>
> The spec says that only hyphen is valid for alias name, but DTC works
> happily with underscore too. From my experience with DTC playing, both
> hyphen and underscore are valid for alias and node name. But for label
> name, only underscore is valid. Using hyphen in label name will cause
> DTC to report syntax error.
>
Yes indeed, thanks for pointing it out.
For the sake of current patch, please clarify whether a v2 is needed.
IIUC:
-sec_mon node name could stay the same (existing binding)
-label names are ok, since underline is the only option allowed by DTC
-alias names are out-of-spec but accepted by DTC; if changing underline
to hyphen is requested, I will push out v2
Thanks,
Horia
^ permalink raw reply
* Re: [PATCH 0/7] crypto: aes - allow generic AES to be omitted
From: Ard Biesheuvel @ 2017-03-28 8:51 UTC (permalink / raw)
To: Eric Biggers; +Cc: linux-crypto@vger.kernel.org, Herbert Xu, nico@linaro.org
In-Reply-To: <20170328054349.GA1023@zzz>
On 28 March 2017 at 06:43, Eric Biggers <ebiggers3@gmail.com> wrote:
> Hi Ard,
>
> On Sun, Mar 26, 2017 at 07:49:01PM +0100, Ard Biesheuvel wrote:
>> The generic AES driver uses 16 lookup tables of 1 KB each, and has
>> encryption and decryption routines that are fully unrolled. Given how
>> the dependencies between this code and other drivers are declared in
>> Kconfig files, this code is always pulled into the core kernel, even
>> if it is usually superseded at runtime by accelerated drivers that
>> exist for many architectures.
>>
>> This leaves us with 25 KB of dead code in the kernel, which is negligible
>> in typical environments, but which is actually a big deal for the IoT
>> domain, where every kilobyte counts.
>>
>> For this reason, this series refactors the way the various AES
>> implementations are wired up, to allow the generic version in
>> crypto/aes_generic.c to be omitted from the build entirely.
>>
>> Patch #1 removes some bogus 'select CRYPTO_AES' statement.
>>
>> Patch #2 introduces CRYPTO_NEED_AES which can be selected by driver that
>> require an AES cipher to be available, but don't care how it is implemented.
>>
>> Patches #3 and #4 make some preparatory changes that allow dependencies on
>> crypto_aes_expand_key to be fulfilled by the new (and much smaller) fixed
>> time AES driver. (#5)
>>
>> Patch #6 splits the generic AES driver into a core containing the precomputed
>> sub/shift/mix tables and the key expansion routines on the one hand, and the
>> encryption/decryption routines and the crypto API registration on the other.
>>
>> Patch #7 introduces the CRYPTO_HAVE_AES Kconfig symbol, and adds statements to
>> various AES implementations that can fulfil the CRYPTO_NEED_AES dependencies
>> added in patch #2. The introduced Kconfig logic allows CRYPTO_AES to be
>> deselected even if AES dependencies exist, as long as one of these alternatives
>> is selected.
>
> Just a thought: how about renaming CRYPTO_AES to CRYPTO_AES_GENERIC, then
> renaming what you called CRYPTO_NEED_AES to CRYPTO_AES? Then all the 'select
> CRYPTO_AES' can remain as-is, instead of replacing them with the (in my opinion
> uglier) 'select CRYPTO_NEED_AES'. And it should still work for people who have
> CRYPTO_AES=y or CRYPTO_AES=m in their kernel config, since they'll still get at
> least one AES implementation (though they may stop getting the generic one).
>
> Also, in general I think we need better Kconfig help text. As proposed you can
> now toggle simply "AES cipher algorithms", and nowhere in the help text is it
> mentioned that that is only the generic implementation, which you don't need if
> you have enabled some other implementation. Similarly for "Fixed time AES
> cipher"; it perhaps should be mentioned that it's only useful if a fixed-time
> implementation using special CPU instructions like AES-NI or ARMv8-CE isn't
> usable.
>
Thanks for the feedback. I take it you are on board with the general idea then?
Re name change, those are good points. I will experiment with that.
I was a bit on the fence about modifying the x86 code more than
required, but actually, I think it makes sense for the AES-NI code to
use fixed-time AES as a fallback rather than the table-based x86 code,
given that the fallback is rarely used (only when executed in the
context of an interrupt taken from kernel code that is already using
the FPU) and falling back to a non-fixed time implementation loses
some guarantees that the AES-NI code gives.
^ permalink raw reply
* [PATCH 2/2] crypto: ccp - Mark driver as little-endian only
From: Arnd Bergmann @ 2017-03-28 9:58 UTC (permalink / raw)
To: Tom Lendacky, Gary Hook, Herbert Xu, David S. Miller
Cc: Arnd Bergmann, linux-crypto, linux-kernel
In-Reply-To: <20170328095814.3734615-1-arnd@arndb.de>
The driver causes a warning when built as big-endian:
drivers/crypto/ccp/ccp-dev-v5.c: In function 'ccp5_perform_des3':
include/uapi/linux/byteorder/big_endian.h:32:26: error: large integer implicitly truncated to unsigned type [-Werror=overflow]
#define __cpu_to_le32(x) ((__force __le32)__swab32((x)))
^
include/linux/byteorder/generic.h:87:21: note: in expansion of macro '__cpu_to_le32'
#define cpu_to_le32 __cpu_to_le32
^~~~~~~~~~~~~
drivers/crypto/ccp/ccp-dev-v5.c:436:28: note: in expansion of macro 'cpu_to_le32'
CCP5_CMD_KEY_MEM(&desc) = cpu_to_le32(CCP_MEMTYPE_SB);
The warning is correct, doing a 32-bit byte swap on a value that gets
assigned into a bit field cannot work, since we would only write zeroes
in this case, regardless of the input.
In fact, the use of bit fields in hardware defined data structures is
not portable to start with, so until all these bit fields get replaced
by something else, the driver cannot work on big-endian machines, and
I'm adding an annotation here to prevent it from being selected.
The CCPv3 code seems to not suffer from this problem, only v5 uses
bitfields.
Fixes: 4b394a232df7 ("crypto: ccp - Let a v5 CCP provide the same function as v3")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
drivers/crypto/ccp/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/crypto/ccp/Kconfig b/drivers/crypto/ccp/Kconfig
index 2238f77aa248..07af9ece84f9 100644
--- a/drivers/crypto/ccp/Kconfig
+++ b/drivers/crypto/ccp/Kconfig
@@ -1,6 +1,7 @@
config CRYPTO_DEV_CCP_DD
tristate "Cryptographic Coprocessor device driver"
depends on CRYPTO_DEV_CCP
+ depends on !CPU_BIG_ENDIAN || BROKEN
default m
select HW_RANDOM
select DMA_ENGINE
--
2.9.0
^ permalink raw reply related
* [PATCH 1/2] crypto: ccp - Reduce stack frame size with KASAN
From: Arnd Bergmann @ 2017-03-28 9:58 UTC (permalink / raw)
To: Tom Lendacky, Gary Hook, Herbert Xu, David S. Miller
Cc: Arnd Bergmann, linux-crypto, linux-kernel
The newly added AES GCM implementation uses one of the largest stack frames
in the kernel, around 1KB on normal 64-bit kernels, and 1.6KB when CONFIG_KASAN
is enabled:
drivers/crypto/ccp/ccp-ops.c: In function 'ccp_run_aes_gcm_cmd':
drivers/crypto/ccp/ccp-ops.c:851:1: error: the frame size of 1632 bytes is larger than 1536 bytes [-Werror=frame-larger-than=]
This is problematic for multiple reasons:
- The crypto functions are often used in deep call chains, e.g. behind
mm, fs and dm layers, making it more likely to run into an actual stack
overflow
- Using this much stack space is an indicator that the code is not
written to be as efficient as it could be.
- While this goes unnoticed at the moment in mainline with the frame size
warning being disabled when KASAN is in use, I would like to enable
the warning again, and the current code is slightly above my arbitrary
pick for a limit of 1536 bytes (I already did patches for every other
driver exceeding this).
A more drastic refactoring of the driver might be needed to reduce the
stack usage more substantially, but this patch is fairly simple and
at least addresses the third one of the problems I mentioned, reducing the
stack size by about 150 bytes and bringing it below the warning limit
I picked.
Fixes: 36cf515b9bbe ("crypto: ccp - Enable support for AES GCM on v5 CCPs")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
drivers/crypto/ccp/ccp-dev.h | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/crypto/ccp/ccp-dev.h b/drivers/crypto/ccp/ccp-dev.h
index 3a45c2af2fbd..c5ea0796a891 100644
--- a/drivers/crypto/ccp/ccp-dev.h
+++ b/drivers/crypto/ccp/ccp-dev.h
@@ -432,24 +432,24 @@ struct ccp_dma_info {
unsigned int offset;
unsigned int length;
enum dma_data_direction dir;
-};
+} __packed __aligned(4);
struct ccp_dm_workarea {
struct device *dev;
struct dma_pool *dma_pool;
- unsigned int length;
u8 *address;
struct ccp_dma_info dma;
+ unsigned int length;
};
struct ccp_sg_workarea {
struct scatterlist *sg;
int nents;
+ unsigned int dma_count;
struct scatterlist *dma_sg;
struct device *dma_dev;
- unsigned int dma_count;
enum dma_data_direction dma_dir;
unsigned int sg_used;
--
2.9.0
^ permalink raw reply related
* Re: [PATCH] arm64: dts: ls1012a: add crypto node
From: Shawn Guo @ 2017-03-28 10:41 UTC (permalink / raw)
To: Horia Geantă
Cc: Mark Rutland, devicetree@vger.kernel.org, Herbert Xu,
Harninder Rai, Catalin Marinas, Bhaskar U, Will Deacon,
Dan Douglass, Rob Herring, linux-crypto@vger.kernel.org,
David S. Miller, linux-arm-kernel@lists.infradead.org
In-Reply-To: <VI1PR0401MB259164F1B575677D65CB8F2E98320@VI1PR0401MB2591.eurprd04.prod.outlook.com>
On Tue, Mar 28, 2017 at 07:19:43AM +0000, Horia Geantă wrote:
> For the sake of current patch, please clarify whether a v2 is needed.
> IIUC:
> -sec_mon node name could stay the same (existing binding)
> -label names are ok, since underline is the only option allowed by DTC
> -alias names are out-of-spec but accepted by DTC; if changing underline
> to hyphen is requested, I will push out v2
All these are fine. But we agreed that the alias definitions can be
shared and should be moved to fsl-ls1012a.dtsi, right?
Shawn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply
* [PATCH v2] arm64: dts: ls1012a: add crypto node
From: Horia Geantă @ 2017-03-28 11:46 UTC (permalink / raw)
To: Shawn Guo, Rob Herring, Mark Rutland
Cc: devicetree, Herbert Xu, Harninder Rai, Catalin Marinas,
Bhaskar Upadhaya, Will Deacon, Dan Douglass, linux-crypto,
David S. Miller, linux-arm-kernel
In-Reply-To: <20170328104115.GT30608@dragon>
LS1012A has a SEC v5.4 security engine.
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
v2: move aliases from board specific files into the shared dtsi.
arch/arm64/boot/dts/freescale/fsl-ls1012a.dtsi | 100 ++++++++++++++++++++++++-
1 file changed, 99 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1012a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1012a.dtsi
index cffebb4b3df1..1c3493606cca 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1012a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1012a.dtsi
@@ -42,7 +42,7 @@
* OTHER DEALINGS IN THE SOFTWARE.
*/
-#include <dt-bindings/interrupt-controller/irq.h>
+#include <dt-bindings/interrupt-controller/arm-gic.h>
/ {
compatible = "fsl,ls1012a";
@@ -50,6 +50,15 @@
#address-cells = <2>;
#size-cells = <2>;
+ aliases {
+ crypto = &crypto;
+ rtic_a = &rtic_a;
+ rtic_b = &rtic_b;
+ rtic_c = &rtic_c;
+ rtic_d = &rtic_d;
+ sec_mon = &sec_mon;
+ };
+
cpus {
#address-cells = <1>;
#size-cells = <0>;
@@ -113,6 +122,95 @@
big-endian;
};
+ crypto: crypto@1700000 {
+ compatible = "fsl,sec-v5.4", "fsl,sec-v5.0",
+ "fsl,sec-v4.0";
+ fsl,sec-era = <8>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0x0 0x00 0x1700000 0x100000>;
+ reg = <0x00 0x1700000 0x0 0x100000>;
+ interrupts = <GIC_SPI 75 IRQ_TYPE_LEVEL_HIGH>;
+
+ sec_jr0: jr@10000 {
+ compatible = "fsl,sec-v5.4-job-ring",
+ "fsl,sec-v5.0-job-ring",
+ "fsl,sec-v4.0-job-ring";
+ reg = <0x10000 0x10000>;
+ interrupts = <GIC_SPI 71 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ sec_jr1: jr@20000 {
+ compatible = "fsl,sec-v5.4-job-ring",
+ "fsl,sec-v5.0-job-ring",
+ "fsl,sec-v4.0-job-ring";
+ reg = <0x20000 0x10000>;
+ interrupts = <GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ sec_jr2: jr@30000 {
+ compatible = "fsl,sec-v5.4-job-ring",
+ "fsl,sec-v5.0-job-ring",
+ "fsl,sec-v4.0-job-ring";
+ reg = <0x30000 0x10000>;
+ interrupts = <GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ sec_jr3: jr@40000 {
+ compatible = "fsl,sec-v5.4-job-ring",
+ "fsl,sec-v5.0-job-ring",
+ "fsl,sec-v4.0-job-ring";
+ reg = <0x40000 0x10000>;
+ interrupts = <GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ rtic@60000 {
+ compatible = "fsl,sec-v5.4-rtic",
+ "fsl,sec-v5.0-rtic",
+ "fsl,sec-v4.0-rtic";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ reg = <0x60000 0x100 0x60e00 0x18>;
+ ranges = <0x0 0x60100 0x500>;
+
+ rtic_a: rtic-a@0 {
+ compatible = "fsl,sec-v5.4-rtic-memory",
+ "fsl,sec-v5.0-rtic-memory",
+ "fsl,sec-v4.0-rtic-memory";
+ reg = <0x00 0x20 0x100 0x100>;
+ };
+
+ rtic_b: rtic-b@20 {
+ compatible = "fsl,sec-v5.4-rtic-memory",
+ "fsl,sec-v5.0-rtic-memory",
+ "fsl,sec-v4.0-rtic-memory";
+ reg = <0x20 0x20 0x200 0x100>;
+ };
+
+ rtic_c: rtic-c@40 {
+ compatible = "fsl,sec-v5.4-rtic-memory",
+ "fsl,sec-v5.0-rtic-memory",
+ "fsl,sec-v4.0-rtic-memory";
+ reg = <0x40 0x20 0x300 0x100>;
+ };
+
+ rtic_d: rtic-d@60 {
+ compatible = "fsl,sec-v5.4-rtic-memory",
+ "fsl,sec-v5.0-rtic-memory",
+ "fsl,sec-v4.0-rtic-memory";
+ reg = <0x60 0x20 0x400 0x100>;
+ };
+ };
+ };
+
+ sec_mon: sec_mon@1e90000 {
+ compatible = "fsl,sec-v5.4-mon", "fsl,sec-v5.0-mon",
+ "fsl,sec-v4.0-mon";
+ reg = <0x0 0x1e90000 0x0 0x10000>;
+ interrupts = <GIC_SPI 78 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 79 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
dcfg: dcfg@1ee0000 {
compatible = "fsl,ls1012a-dcfg",
"syscon";
--
2.12.0.264.gd6db3f216544
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* 8537 linux-crypto
From: agiva @ 2017-03-28 11:47 UTC (permalink / raw)
To: linux-crypto
[-- Attachment #1: 66697092.zip --]
[-- Type: application/zip, Size: 2639 bytes --]
^ permalink raw reply
* [PATCH 4.10 075/111] hwrng: amd - Revert managed API changes
From: Greg Kroah-Hartman @ 2017-03-28 12:31 UTC (permalink / raw)
To: linux-kernel
Cc: Greg Kroah-Hartman, stable, Prarit Bhargava, Matt Mackall,
Corentin LABBE, PrasannaKumar Muralidharan, Wei Yongjun,
linux-crypto, linux-geode, Herbert Xu
In-Reply-To: <20170328122915.640228468@linuxfoundation.org>
4.10-stable review patch. If anyone has any objections, please let me know.
------------------
From: Prarit Bhargava <prarit@redhat.com>
commit 69db7009318758769d625b023402161c750f7876 upstream.
After commit 31b2a73c9c5f ("hwrng: amd - Migrate to managed API"), the
amd-rng driver uses devres with pci_dev->dev to keep track of resources,
but does not actually register a PCI driver. This results in the
following issues:
1. The message
WARNING: CPU: 2 PID: 621 at drivers/base/dd.c:349 driver_probe_device+0x38c
is output when the i2c_amd756 driver loads and attempts to register a PCI
driver. The PCI & device subsystems assume that no resources have been
registered for the device, and the WARN_ON() triggers since amd-rng has
already do so.
2. The driver leaks memory because the driver does not attach to a
device. The driver only uses the PCI device as a reference. devm_*()
functions will release resources on driver detach, which the amd-rng
driver will never do. As a result,
3. The driver cannot be reloaded because there is always a use of the
ioport and region after the first load of the driver.
Revert the changes made by 31b2a73c9c5f ("hwrng: amd - Migrate to managed
API").
Signed-off-by: Prarit Bhargava <prarit@redhat.com>
Fixes: 31b2a73c9c5f ("hwrng: amd - Migrate to managed API").
Cc: Matt Mackall <mpm@selenic.com>
Cc: Corentin LABBE <clabbe.montjoie@gmail.com>
Cc: PrasannaKumar Muralidharan <prasannatsmkumar@gmail.com>
Cc: Wei Yongjun <weiyongjun1@huawei.com>
Cc: linux-crypto@vger.kernel.org
Cc: linux-geode@lists.infradead.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/char/hw_random/amd-rng.c | 42 +++++++++++++++++++++++++++++++--------
1 file changed, 34 insertions(+), 8 deletions(-)
--- a/drivers/char/hw_random/amd-rng.c
+++ b/drivers/char/hw_random/amd-rng.c
@@ -55,6 +55,7 @@ MODULE_DEVICE_TABLE(pci, pci_tbl);
struct amd768_priv {
void __iomem *iobase;
struct pci_dev *pcidev;
+ u32 pmbase;
};
static int amd_rng_read(struct hwrng *rng, void *buf, size_t max, bool wait)
@@ -148,33 +149,58 @@ found:
if (pmbase == 0)
return -EIO;
- priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
if (!priv)
return -ENOMEM;
- if (!devm_request_region(&pdev->dev, pmbase + PMBASE_OFFSET,
- PMBASE_SIZE, DRV_NAME)) {
+ if (!request_region(pmbase + PMBASE_OFFSET, PMBASE_SIZE, DRV_NAME)) {
dev_err(&pdev->dev, DRV_NAME " region 0x%x already in use!\n",
pmbase + 0xF0);
- return -EBUSY;
+ err = -EBUSY;
+ goto out;
}
- priv->iobase = devm_ioport_map(&pdev->dev, pmbase + PMBASE_OFFSET,
- PMBASE_SIZE);
+ priv->iobase = ioport_map(pmbase + PMBASE_OFFSET, PMBASE_SIZE);
if (!priv->iobase) {
pr_err(DRV_NAME "Cannot map ioport\n");
- return -ENOMEM;
+ err = -EINVAL;
+ goto err_iomap;
}
amd_rng.priv = (unsigned long)priv;
+ priv->pmbase = pmbase;
priv->pcidev = pdev;
pr_info(DRV_NAME " detected\n");
- return devm_hwrng_register(&pdev->dev, &amd_rng);
+ err = hwrng_register(&amd_rng);
+ if (err) {
+ pr_err(DRV_NAME " registering failed (%d)\n", err);
+ goto err_hwrng;
+ }
+ return 0;
+
+err_hwrng:
+ ioport_unmap(priv->iobase);
+err_iomap:
+ release_region(pmbase + PMBASE_OFFSET, PMBASE_SIZE);
+out:
+ kfree(priv);
+ return err;
}
static void __exit mod_exit(void)
{
+ struct amd768_priv *priv;
+
+ priv = (struct amd768_priv *)amd_rng.priv;
+
+ hwrng_unregister(&amd_rng);
+
+ ioport_unmap(priv->iobase);
+
+ release_region(priv->pmbase + PMBASE_OFFSET, PMBASE_SIZE);
+
+ kfree(priv);
}
module_init(mod_init);
^ permalink raw reply
* [PATCH 4.9 61/88] hwrng: amd - Revert managed API changes
From: Greg Kroah-Hartman @ 2017-03-28 12:31 UTC (permalink / raw)
To: linux-kernel
Cc: Greg Kroah-Hartman, stable, Prarit Bhargava, Matt Mackall,
Corentin LABBE, PrasannaKumar Muralidharan, Wei Yongjun,
linux-crypto, linux-geode, Herbert Xu
In-Reply-To: <20170328122748.656530096@linuxfoundation.org>
4.9-stable review patch. If anyone has any objections, please let me know.
------------------
From: Prarit Bhargava <prarit@redhat.com>
commit 69db7009318758769d625b023402161c750f7876 upstream.
After commit 31b2a73c9c5f ("hwrng: amd - Migrate to managed API"), the
amd-rng driver uses devres with pci_dev->dev to keep track of resources,
but does not actually register a PCI driver. This results in the
following issues:
1. The message
WARNING: CPU: 2 PID: 621 at drivers/base/dd.c:349 driver_probe_device+0x38c
is output when the i2c_amd756 driver loads and attempts to register a PCI
driver. The PCI & device subsystems assume that no resources have been
registered for the device, and the WARN_ON() triggers since amd-rng has
already do so.
2. The driver leaks memory because the driver does not attach to a
device. The driver only uses the PCI device as a reference. devm_*()
functions will release resources on driver detach, which the amd-rng
driver will never do. As a result,
3. The driver cannot be reloaded because there is always a use of the
ioport and region after the first load of the driver.
Revert the changes made by 31b2a73c9c5f ("hwrng: amd - Migrate to managed
API").
Signed-off-by: Prarit Bhargava <prarit@redhat.com>
Fixes: 31b2a73c9c5f ("hwrng: amd - Migrate to managed API").
Cc: Matt Mackall <mpm@selenic.com>
Cc: Corentin LABBE <clabbe.montjoie@gmail.com>
Cc: PrasannaKumar Muralidharan <prasannatsmkumar@gmail.com>
Cc: Wei Yongjun <weiyongjun1@huawei.com>
Cc: linux-crypto@vger.kernel.org
Cc: linux-geode@lists.infradead.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/char/hw_random/amd-rng.c | 42 +++++++++++++++++++++++++++++++--------
1 file changed, 34 insertions(+), 8 deletions(-)
--- a/drivers/char/hw_random/amd-rng.c
+++ b/drivers/char/hw_random/amd-rng.c
@@ -55,6 +55,7 @@ MODULE_DEVICE_TABLE(pci, pci_tbl);
struct amd768_priv {
void __iomem *iobase;
struct pci_dev *pcidev;
+ u32 pmbase;
};
static int amd_rng_read(struct hwrng *rng, void *buf, size_t max, bool wait)
@@ -148,33 +149,58 @@ found:
if (pmbase == 0)
return -EIO;
- priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
if (!priv)
return -ENOMEM;
- if (!devm_request_region(&pdev->dev, pmbase + PMBASE_OFFSET,
- PMBASE_SIZE, DRV_NAME)) {
+ if (!request_region(pmbase + PMBASE_OFFSET, PMBASE_SIZE, DRV_NAME)) {
dev_err(&pdev->dev, DRV_NAME " region 0x%x already in use!\n",
pmbase + 0xF0);
- return -EBUSY;
+ err = -EBUSY;
+ goto out;
}
- priv->iobase = devm_ioport_map(&pdev->dev, pmbase + PMBASE_OFFSET,
- PMBASE_SIZE);
+ priv->iobase = ioport_map(pmbase + PMBASE_OFFSET, PMBASE_SIZE);
if (!priv->iobase) {
pr_err(DRV_NAME "Cannot map ioport\n");
- return -ENOMEM;
+ err = -EINVAL;
+ goto err_iomap;
}
amd_rng.priv = (unsigned long)priv;
+ priv->pmbase = pmbase;
priv->pcidev = pdev;
pr_info(DRV_NAME " detected\n");
- return devm_hwrng_register(&pdev->dev, &amd_rng);
+ err = hwrng_register(&amd_rng);
+ if (err) {
+ pr_err(DRV_NAME " registering failed (%d)\n", err);
+ goto err_hwrng;
+ }
+ return 0;
+
+err_hwrng:
+ ioport_unmap(priv->iobase);
+err_iomap:
+ release_region(pmbase + PMBASE_OFFSET, PMBASE_SIZE);
+out:
+ kfree(priv);
+ return err;
}
static void __exit mod_exit(void)
{
+ struct amd768_priv *priv;
+
+ priv = (struct amd768_priv *)amd_rng.priv;
+
+ hwrng_unregister(&amd_rng);
+
+ ioport_unmap(priv->iobase);
+
+ release_region(priv->pmbase + PMBASE_OFFSET, PMBASE_SIZE);
+
+ kfree(priv);
}
module_init(mod_init);
^ permalink raw reply
* [PATCH 4.9 62/88] hwrng: geode - Revert managed API changes
From: Greg Kroah-Hartman @ 2017-03-28 12:31 UTC (permalink / raw)
To: linux-kernel
Cc: Greg Kroah-Hartman, stable, Prarit Bhargava, Matt Mackall,
Corentin LABBE, PrasannaKumar Muralidharan, Wei Yongjun,
linux-crypto, linux-geode, Herbert Xu
In-Reply-To: <20170328122748.656530096@linuxfoundation.org>
4.9-stable review patch. If anyone has any objections, please let me know.
------------------
From: Prarit Bhargava <prarit@redhat.com>
commit 8c75704ebcac2ffa31ee7bcc359baf701b52bf00 upstream.
After commit e9afc746299d ("hwrng: geode - Use linux/io.h instead of
asm/io.h") the geode-rng driver uses devres with pci_dev->dev to keep
track of resources, but does not actually register a PCI driver. This
results in the following issues:
1. The driver leaks memory because the driver does not attach to a
device. The driver only uses the PCI device as a reference. devm_*()
functions will release resources on driver detach, which the geode-rng
driver will never do. As a result,
2. The driver cannot be reloaded because there is always a use of the
ioport and region after the first load of the driver.
Revert the changes made by e9afc746299d ("hwrng: geode - Use linux/io.h
instead of asm/io.h").
Signed-off-by: Prarit Bhargava <prarit@redhat.com>
Fixes: 6e9b5e76882c ("hwrng: geode - Migrate to managed API")
Cc: Matt Mackall <mpm@selenic.com>
Cc: Corentin LABBE <clabbe.montjoie@gmail.com>
Cc: PrasannaKumar Muralidharan <prasannatsmkumar@gmail.com>
Cc: Wei Yongjun <weiyongjun1@huawei.com>
Cc: linux-crypto@vger.kernel.org
Cc: linux-geode@lists.infradead.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/char/hw_random/geode-rng.c | 50 +++++++++++++++++++++++++------------
1 file changed, 35 insertions(+), 15 deletions(-)
--- a/drivers/char/hw_random/geode-rng.c
+++ b/drivers/char/hw_random/geode-rng.c
@@ -31,6 +31,9 @@
#include <linux/module.h>
#include <linux/pci.h>
+
+#define PFX KBUILD_MODNAME ": "
+
#define GEODE_RNG_DATA_REG 0x50
#define GEODE_RNG_STATUS_REG 0x54
@@ -82,6 +85,7 @@ static struct hwrng geode_rng = {
static int __init mod_init(void)
{
+ int err = -ENODEV;
struct pci_dev *pdev = NULL;
const struct pci_device_id *ent;
void __iomem *mem;
@@ -89,27 +93,43 @@ static int __init mod_init(void)
for_each_pci_dev(pdev) {
ent = pci_match_id(pci_tbl, pdev);
- if (ent) {
- rng_base = pci_resource_start(pdev, 0);
- if (rng_base == 0)
- return -ENODEV;
-
- mem = devm_ioremap(&pdev->dev, rng_base, 0x58);
- if (!mem)
- return -ENOMEM;
- geode_rng.priv = (unsigned long)mem;
-
- pr_info("AMD Geode RNG detected\n");
- return devm_hwrng_register(&pdev->dev, &geode_rng);
- }
+ if (ent)
+ goto found;
}
-
/* Device not found. */
- return -ENODEV;
+ goto out;
+
+found:
+ rng_base = pci_resource_start(pdev, 0);
+ if (rng_base == 0)
+ goto out;
+ err = -ENOMEM;
+ mem = ioremap(rng_base, 0x58);
+ if (!mem)
+ goto out;
+ geode_rng.priv = (unsigned long)mem;
+
+ pr_info("AMD Geode RNG detected\n");
+ err = hwrng_register(&geode_rng);
+ if (err) {
+ pr_err(PFX "RNG registering failed (%d)\n",
+ err);
+ goto err_unmap;
+ }
+out:
+ return err;
+
+err_unmap:
+ iounmap(mem);
+ goto out;
}
static void __exit mod_exit(void)
{
+ void __iomem *mem = (void __iomem *)geode_rng.priv;
+
+ hwrng_unregister(&geode_rng);
+ iounmap(mem);
}
module_init(mod_init);
^ permalink raw reply
* [PATCH 4.10 076/111] hwrng: geode - Revert managed API changes
From: Greg Kroah-Hartman @ 2017-03-28 12:31 UTC (permalink / raw)
To: linux-kernel
Cc: Greg Kroah-Hartman, stable, Prarit Bhargava, Matt Mackall,
Corentin LABBE, PrasannaKumar Muralidharan, Wei Yongjun,
linux-crypto, linux-geode, Herbert Xu
In-Reply-To: <20170328122915.640228468@linuxfoundation.org>
4.10-stable review patch. If anyone has any objections, please let me know.
------------------
From: Prarit Bhargava <prarit@redhat.com>
commit 8c75704ebcac2ffa31ee7bcc359baf701b52bf00 upstream.
After commit e9afc746299d ("hwrng: geode - Use linux/io.h instead of
asm/io.h") the geode-rng driver uses devres with pci_dev->dev to keep
track of resources, but does not actually register a PCI driver. This
results in the following issues:
1. The driver leaks memory because the driver does not attach to a
device. The driver only uses the PCI device as a reference. devm_*()
functions will release resources on driver detach, which the geode-rng
driver will never do. As a result,
2. The driver cannot be reloaded because there is always a use of the
ioport and region after the first load of the driver.
Revert the changes made by e9afc746299d ("hwrng: geode - Use linux/io.h
instead of asm/io.h").
Signed-off-by: Prarit Bhargava <prarit@redhat.com>
Fixes: 6e9b5e76882c ("hwrng: geode - Migrate to managed API")
Cc: Matt Mackall <mpm@selenic.com>
Cc: Corentin LABBE <clabbe.montjoie@gmail.com>
Cc: PrasannaKumar Muralidharan <prasannatsmkumar@gmail.com>
Cc: Wei Yongjun <weiyongjun1@huawei.com>
Cc: linux-crypto@vger.kernel.org
Cc: linux-geode@lists.infradead.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/char/hw_random/geode-rng.c | 50 +++++++++++++++++++++++++------------
1 file changed, 35 insertions(+), 15 deletions(-)
--- a/drivers/char/hw_random/geode-rng.c
+++ b/drivers/char/hw_random/geode-rng.c
@@ -31,6 +31,9 @@
#include <linux/module.h>
#include <linux/pci.h>
+
+#define PFX KBUILD_MODNAME ": "
+
#define GEODE_RNG_DATA_REG 0x50
#define GEODE_RNG_STATUS_REG 0x54
@@ -82,6 +85,7 @@ static struct hwrng geode_rng = {
static int __init mod_init(void)
{
+ int err = -ENODEV;
struct pci_dev *pdev = NULL;
const struct pci_device_id *ent;
void __iomem *mem;
@@ -89,27 +93,43 @@ static int __init mod_init(void)
for_each_pci_dev(pdev) {
ent = pci_match_id(pci_tbl, pdev);
- if (ent) {
- rng_base = pci_resource_start(pdev, 0);
- if (rng_base == 0)
- return -ENODEV;
-
- mem = devm_ioremap(&pdev->dev, rng_base, 0x58);
- if (!mem)
- return -ENOMEM;
- geode_rng.priv = (unsigned long)mem;
-
- pr_info("AMD Geode RNG detected\n");
- return devm_hwrng_register(&pdev->dev, &geode_rng);
- }
+ if (ent)
+ goto found;
}
-
/* Device not found. */
- return -ENODEV;
+ goto out;
+
+found:
+ rng_base = pci_resource_start(pdev, 0);
+ if (rng_base == 0)
+ goto out;
+ err = -ENOMEM;
+ mem = ioremap(rng_base, 0x58);
+ if (!mem)
+ goto out;
+ geode_rng.priv = (unsigned long)mem;
+
+ pr_info("AMD Geode RNG detected\n");
+ err = hwrng_register(&geode_rng);
+ if (err) {
+ pr_err(PFX "RNG registering failed (%d)\n",
+ err);
+ goto err_unmap;
+ }
+out:
+ return err;
+
+err_unmap:
+ iounmap(mem);
+ goto out;
}
static void __exit mod_exit(void)
{
+ void __iomem *mem = (void __iomem *)geode_rng.priv;
+
+ hwrng_unregister(&geode_rng);
+ iounmap(mem);
}
module_init(mod_init);
^ permalink raw reply
* [RFC TLS Offload Support 00/15] cover letter
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
Overview
========
A kernel TLS Tx only socket option for TCP sockets.
Similarly to the kernel TLS socket(https://lwn.net/Articles/665602),
only symmetric crypto is done in the kernel, as well as TLS record framing.
The handshake remains in userspace, and the negotiated cipher keys/iv are provided to the TCP socket.
Today, userspace TLS must perform 2 passes over the data. First, it has to encrypt the data. Second, the data is copied to the TCP socket in the kernel.
Kernel TLS avoids one pass over the data by encrypting the data from userspace pages into kernelspace buffers.
Non application-data TLS records must be encrypted using the latest crypto state available in the kernel. It is possible to get the crypto context from the kernel and encrypt such recrods in user-space. But we choose to encrypt such TLS records in the kernel by setting the MSG_OOB flag and providing the record type with the data.
TLS Tx crypto offload is a new feature of network devices. It enables the kernel TLS socket to skip encryption and authentication operations on the transmit side of the data path, delegating those to the NIC. In turn, the NIC encrypts packets that belong to an offloaded TLS socket on the fly. The NIC does not modify any packet headers. It expects to receive fully framed TCP packets with TLS records as payload. The NIC replaces plaintext with ciphertext and fills the authentication tag. The NIC does not hold any state beyond the context needed to encrypt the next expected packet, i.e. expected TCP sequence number and crypto state.
There are 2 flows for TLS Tx offload, a fast path and a slow path.
Fast path: packet matches the expected TCP sequence number in the context.
Slow path: packet does not match the expected TCP sequence number in the context. For example: TCP retransmissions. For a packet in the slow path, we need to resynchronize the crypto context of the NIC by providing the TLS record data for that packet before it could be encrypted and transmitted by the NIC.
Motivation
==========
1) Performance: The CPU overhead of encryption in the data path is high, at least 4x for netperf over TLS between 2 machines connected back-to-back.
Our single stream performance tests show that using crypto offload for TLS sockets achieves the same throughput as plain TCP traffic while increasing CPU utilization by only
x1.4.
2) Flexibility: The protocol stack is implemented entirely on the host CPU.
Compared to solutions based on TCP offload, this approach offloads only encryption. Keeping memory management, congestion control, etc. in the host CPU.
Notes
=====
1) New paths:
o net/tls - TLS layer in kernel
o drivers/net/ethernet/mellanox/accelerator/* - NIC driver support, currently implemented as seperated modules.
In the future this code will go into the mlx5 driver. We attached to this patch only the module that integrated with TLS layer.
The complete NIC sample driver is available at https://github.com/Mellanox/tls-offload/tree/tx_rfc_v5
2) We implemented support for this API in OpenSSL 1.1.0, the code is available at https://github.com/Mellanox/tls-openssl/tree/master
3) TLS crypto offload was presented during netdevconf1.2, more details could be found in the presentation and paper:
https://netdevconf.org/1.2/session.html?boris-pismenny
4) These RFC patches are based on kernel 4.9-rc7.
Aviad Yehezkel (5):
tcp: export do_tcp_sendpages function
tcp: export tcp_rate_check_app_limited function
tcp: Add TLS socket options for TCP sockets
tls: tls offload support
mlx/tls: Enable MLX5_CORE_QP_SIM mode for tls
Dave Watson (2):
crypto: Add gcm template for rfc5288
crypto: rfc5288 aesni optimized intel routines
Ilya Lesokhin (8):
tcp: Add clean acked data hook
net: Add TLS offload netdevice and socket support
mlx/mlx5_core: Allow sending multiple packets
mlx/tls: Hardware interface
mlx/tls: Sysfs configuration interface Configure the driver/hardware
interface via sysfs.
mlx/tls: Add mlx_accel offload driver for TLS
mlx/tls: TLS offload driver Add the main module entrypoints and tie
the module into the build system
net/tls: Add software offload
MAINTAINERS | 14 +
arch/x86/crypto/aesni-intel_asm.S | 6 +
arch/x86/crypto/aesni-intel_avx-x86_64.S | 4 +
arch/x86/crypto/aesni-intel_glue.c | 105 ++-
crypto/gcm.c | 122 ++++
crypto/tcrypt.c | 14 +-
crypto/testmgr.c | 16 +
crypto/testmgr.h | 47 ++
drivers/net/ethernet/mellanox/Kconfig | 1 +
drivers/net/ethernet/mellanox/Makefile | 1 +
.../net/ethernet/mellanox/accelerator/tls/Kconfig | 11 +
.../net/ethernet/mellanox/accelerator/tls/Makefile | 4 +
.../net/ethernet/mellanox/accelerator/tls/tls.c | 658 +++++++++++++++++++
.../net/ethernet/mellanox/accelerator/tls/tls.h | 100 +++
.../ethernet/mellanox/accelerator/tls/tls_cmds.h | 112 ++++
.../net/ethernet/mellanox/accelerator/tls/tls_hw.c | 429 ++++++++++++
.../net/ethernet/mellanox/accelerator/tls/tls_hw.h | 49 ++
.../ethernet/mellanox/accelerator/tls/tls_main.c | 77 +++
.../ethernet/mellanox/accelerator/tls/tls_sysfs.c | 196 ++++++
.../ethernet/mellanox/accelerator/tls/tls_sysfs.h | 47 ++
drivers/net/ethernet/mellanox/mlx5/core/en_tx.c | 11 +-
include/linux/netdevice.h | 23 +
include/net/inet_connection_sock.h | 2 +
include/net/tcp.h | 2 +
include/net/tls.h | 228 +++++++
include/uapi/linux/Kbuild | 1 +
include/uapi/linux/tcp.h | 2 +
include/uapi/linux/tls.h | 84 +++
net/Kconfig | 1 +
net/Makefile | 1 +
net/ipv4/tcp.c | 37 +-
net/ipv4/tcp_input.c | 3 +
net/ipv4/tcp_rate.c | 1 +
net/tls/Kconfig | 12 +
net/tls/Makefile | 7 +
net/tls/tls_device.c | 594 +++++++++++++++++
net/tls/tls_main.c | 352 ++++++++++
net/tls/tls_sw.c | 729 +++++++++++++++++++++
38 files changed, 4078 insertions(+), 25 deletions(-)
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/Kconfig
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/Makefile
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls.c
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls.h
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls_cmds.h
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls_hw.c
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls_hw.h
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls_main.c
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.c
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.h
create mode 100644 include/net/tls.h
create mode 100644 include/uapi/linux/tls.h
create mode 100644 net/tls/Kconfig
create mode 100644 net/tls/Makefile
create mode 100644 net/tls/tls_device.c
create mode 100644 net/tls/tls_main.c
create mode 100644 net/tls/tls_sw.c
--
2.7.4
^ permalink raw reply
* [RFC TLS Offload Support 04/15] net: Add TLS offload netdevice and socket support
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
In-Reply-To: <1490707592-1430-1-git-send-email-aviadye@mellanox.com>
From: Ilya Lesokhin <ilyal@mellanox.com>
This patch add a new NDO to add and delete TLS contexts on netdevices.
Signed-off-by: Boris Pismenny <borisp@mellanox.com>
Signed-off-by: Ilya Lesokhin <ilyal@mellanox.com>
Signed-off-by: Aviad Yehezkel <aviadye@mellanox.com>
---
include/linux/netdevice.h | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 51f9336..ce4760c 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -844,6 +844,25 @@ struct xfrmdev_ops {
};
#endif
+#if IS_ENABLED(CONFIG_TLS)
+enum tls_offload_ctx_dir {
+ TLS_OFFLOAD_CTX_DIR_RX,
+ TLS_OFFLOAD_CTX_DIR_TX,
+};
+
+struct tls_crypto_info;
+struct tls_offload_context;
+
+struct tlsdev_ops {
+ int (*tls_dev_add)(struct net_device *netdev, struct sock *sk,
+ enum tls_offload_ctx_dir direction,
+ struct tls_crypto_info *crypto_info,
+ struct tls_offload_context **ctx);
+ void (*tls_dev_del)(struct net_device *netdev, struct sock *sk,
+ enum tls_offload_ctx_dir direction);
+};
+#endif
+
/*
* This structure defines the management hooks for network devices.
* The following hooks can be defined; unless noted otherwise, they are
@@ -1722,6 +1741,10 @@ struct net_device {
const struct xfrmdev_ops *xfrmdev_ops;
#endif
+#if IS_ENABLED(CONFIG_TLS)
+ const struct tlsdev_ops *tlsdev_ops;
+#endif
+
const struct header_ops *header_ops;
unsigned int flags;
--
2.7.4
^ permalink raw reply related
* [RFC TLS Offload Support 05/15] tcp: Add TLS socket options for TCP sockets
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
In-Reply-To: <1490707592-1430-1-git-send-email-aviadye@mellanox.com>
This patch adds TLS_TX and TLS_RX TCP socket options.
Setting these socket options will change the sk->sk_prot
operations of the TCP socket. The user is responsible to
prevent races between calls to the previous operations
and the new operations. After successful return, data
sent on this socket will be encapsulated in TLS.
Signed-off-by: Aviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: Boris Pismenny <borisp@mellanox.com>
Signed-off-by: Ilya Lesokhin <ilyal@mellanox.com>
---
include/uapi/linux/tcp.h | 2 ++
net/ipv4/tcp.c | 32 ++++++++++++++++++++++++++++++++
2 files changed, 34 insertions(+)
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index c53de26..f9f0e29 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -116,6 +116,8 @@ enum {
#define TCP_SAVE_SYN 27 /* Record SYN headers for new connections */
#define TCP_SAVED_SYN 28 /* Get SYN headers recorded for connection */
#define TCP_REPAIR_WINDOW 29 /* Get/set window parameters */
+#define TCP_TLS_TX 30
+#define TCP_TLS_RX 31
struct tcp_repair_opt {
__u32 opt_code;
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 302fee9..2d190e3 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -273,6 +273,7 @@
#include <net/icmp.h>
#include <net/inet_common.h>
#include <net/tcp.h>
+#include <net/tls.h>
#include <net/xfrm.h>
#include <net/ip.h>
#include <net/sock.h>
@@ -2676,6 +2677,21 @@ static int do_tcp_setsockopt(struct sock *sk, int level,
tp->notsent_lowat = val;
sk->sk_write_space(sk);
break;
+ case TCP_TLS_TX:
+ case TCP_TLS_RX: {
+ int (*fn)(struct sock *sk, int optname,
+ char __user *optval, unsigned int optlen);
+
+ fn = symbol_get(tls_sk_attach);
+ if (!fn) {
+ err = -EINVAL;
+ break;
+ }
+
+ err = fn(sk, optname, optval, optlen);
+ symbol_put(tls_sk_attach);
+ break;
+ }
default:
err = -ENOPROTOOPT;
break;
@@ -3064,6 +3080,22 @@ static int do_tcp_getsockopt(struct sock *sk, int level,
}
return 0;
}
+ case TCP_TLS_TX:
+ case TCP_TLS_RX: {
+ int err;
+ int (*fn)(struct sock *sk, int optname,
+ char __user *optval, int __user *optlen);
+
+ fn = symbol_get(tls_sk_query);
+ if (!fn) {
+ err = -EINVAL;
+ break;
+ }
+
+ err = fn(sk, optname, optval, optlen);
+ symbol_put(tls_sk_query);
+ return err;
+ }
default:
return -ENOPROTOOPT;
}
--
2.7.4
^ permalink raw reply related
* [RFC TLS Offload Support 06/15] tls: tls offload support
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
In-Reply-To: <1490707592-1430-1-git-send-email-aviadye@mellanox.com>
This patch introduces TX HW offload.
tls_main: contains generic logic that will be shared by both
SW and HW implementations.
tls_device: contains generic HW logic that is shared by all
HW offload implementations.
Signed-off-by: Boris Pismenny <borisp@mellanox.com>
Signed-off-by: Ilya Lesokhin <ilyal@mellanox.com>
Signed-off-by: Aviad Yehezkel <aviadye@mellanox.com>
---
MAINTAINERS | 13 +
include/net/tls.h | 184 ++++++++++++++
include/uapi/linux/Kbuild | 1 +
include/uapi/linux/tls.h | 84 +++++++
net/Kconfig | 1 +
net/Makefile | 1 +
net/tls/Kconfig | 12 +
net/tls/Makefile | 7 +
net/tls/tls_device.c | 594 ++++++++++++++++++++++++++++++++++++++++++++++
net/tls/tls_main.c | 348 +++++++++++++++++++++++++++
10 files changed, 1245 insertions(+)
create mode 100644 include/net/tls.h
create mode 100644 include/uapi/linux/tls.h
create mode 100644 net/tls/Kconfig
create mode 100644 net/tls/Makefile
create mode 100644 net/tls/tls_device.c
create mode 100644 net/tls/tls_main.c
diff --git a/MAINTAINERS b/MAINTAINERS
index b340ef6..e3b70c3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8486,6 +8486,19 @@ F: net/ipv6/
F: include/net/ip*
F: arch/x86/net/*
+NETWORKING [TLS]
+M: Ilya Lesokhin <ilyal@mellanox.com>
+M: Aviad Yehezkel <aviadye@mellanox.com>
+M: Boris Pismenny <borisp@mellanox.com>
+M: Haggai Eran <haggaie@mellanox.com>
+L: netdev@vger.kernel.org
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git
+S: Maintained
+F: net/tls/*
+F: include/uapi/linux/tls.h
+F: include/net/tls.h
+
NETWORKING [IPSEC]
M: Steffen Klassert <steffen.klassert@secunet.com>
M: Herbert Xu <herbert@gondor.apana.org.au>
diff --git a/include/net/tls.h b/include/net/tls.h
new file mode 100644
index 0000000..f7f0cde
--- /dev/null
+++ b/include/net/tls.h
@@ -0,0 +1,184 @@
+/* Copyright (c) 2016-2017, Mellanox Technologies All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * - Neither the name of the Mellanox Technologies nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior written
+ * permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE
+ */
+
+#ifndef _TLS_OFFLOAD_H
+#define _TLS_OFFLOAD_H
+
+#include <linux/types.h>
+
+#include <uapi/linux/tls.h>
+
+
+/* Maximum data size carried in a TLS record */
+#define TLS_MAX_PAYLOAD_SIZE ((size_t)1 << 14)
+
+#define TLS_HEADER_SIZE 5
+#define TLS_NONCE_OFFSET TLS_HEADER_SIZE
+
+#define TLS_CRYPTO_INFO_READY(info) ((info)->cipher_type)
+#define TLS_IS_STATE_HW(info) ((info)->state == TLS_STATE_HW)
+
+#define TLS_RECORD_TYPE_DATA 0x17
+
+
+struct tls_record_info {
+ struct list_head list;
+ u32 end_seq;
+ int len;
+ int num_frags;
+ skb_frag_t frags[MAX_SKB_FRAGS];
+};
+
+struct tls_offload_context {
+ struct list_head records_list;
+ struct tls_record_info *open_record;
+ struct tls_record_info *retransmit_hint;
+ u32 expectedSN;
+ spinlock_t lock; /* protects records list */
+};
+
+struct tls_context {
+ union {
+ struct tls_crypto_info crypto_send;
+ struct tls_crypto_info_aes_gcm_128 crypto_send_aes_gcm_128;
+ };
+
+ void *priv_ctx;
+
+ u16 prepand_size;
+ u16 tag_size;
+ u16 iv_size;
+ char *iv;
+
+ /* TODO: change sw code to use below fields and push_frags function */
+ skb_frag_t *pending_frags;
+ u16 num_pending_frags;
+ u16 pending_offset;
+
+ void (*sk_write_space)(struct sock *sk);
+ void (*sk_destruct)(struct sock *sk);
+};
+
+
+int tls_sk_query(struct sock *sk, int optname, char __user *optval,
+ int __user *optlen);
+int tls_sk_attach(struct sock *sk, int optname, char __user *optval,
+ unsigned int optlen);
+
+void tls_clear_device_offload(struct sock *sk, struct tls_context *ctx);
+int tls_set_device_offload(struct sock *sk, struct tls_context *ctx);
+int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size);
+int tls_device_sendpage(struct sock *sk, struct page *page,
+ int offset, size_t size, int flags);
+
+struct tls_record_info *tls_get_record(struct tls_offload_context *context,
+ u32 seq);
+
+void tls_sk_destruct(struct sock *sk, struct tls_context *ctx);
+void tls_icsk_clean_acked(struct sock *sk);
+
+void tls_device_sk_destruct(struct sock *sk);
+
+
+int tls_push_frags(struct sock *sk, struct tls_context *ctx,
+ skb_frag_t *frag, u16 num_frags, u16 first_offset,
+ int flags);
+int tls_push_paritial_record(struct sock *sk, struct tls_context *ctx,
+ int flags);
+
+static inline bool tls_is_pending_open_record(struct tls_context *ctx)
+{
+ return !!ctx->num_pending_frags;
+}
+
+static inline bool tls_is_sk_tx_device_offloaded(struct sock *sk)
+{
+ return smp_load_acquire(&sk->sk_destruct) ==
+ &tls_device_sk_destruct;
+}
+
+static inline void tls_err_abort(struct sock *sk)
+{
+ xchg(&sk->sk_err, -EBADMSG);
+ sk->sk_error_report(sk);
+}
+
+static inline void tls_increment_seqno(unsigned char *seq, struct sock *sk)
+{
+ int i;
+
+ for (i = 7; i >= 0; i--) {
+ ++seq[i];
+ if (seq[i] != 0)
+ break;
+ }
+
+ if (i == -1)
+ tls_err_abort(sk);
+}
+
+static inline void tls_fill_prepend(struct tls_context *ctx,
+ char *buf,
+ size_t plaintext_len,
+ unsigned char record_type)
+{
+ size_t pkt_len, iv_size = ctx->iv_size;
+
+ pkt_len = plaintext_len + iv_size + ctx->tag_size;
+
+ /* we cover nonce explicit here as well, so buf should be of
+ * size KTLS_DTLS_HEADER_SIZE + KTLS_DTLS_NONCE_EXPLICIT_SIZE
+ */
+ buf[0] = record_type;
+ buf[1] = TLS_VERSION_MINOR(ctx->crypto_send.version);
+ buf[2] = TLS_VERSION_MAJOR(ctx->crypto_send.version);
+ /* we can use IV for nonce explicit according to spec */
+ buf[3] = pkt_len >> 8;
+ buf[4] = pkt_len & 0xFF;
+ memcpy(buf + TLS_NONCE_OFFSET, ctx->iv, iv_size);
+}
+
+static inline struct tls_context *tls_get_ctx(const struct sock *sk)
+{
+ return sk->sk_user_data;
+}
+
+static inline struct tls_offload_context *tls_offload_ctx(
+ const struct tls_context *tls_ctx)
+{
+ return (struct tls_offload_context *)tls_ctx->priv_ctx;
+}
+
+
+#endif /* _TLS_OFFLOAD_H */
diff --git a/include/uapi/linux/Kbuild b/include/uapi/linux/Kbuild
index cd2be1c..96ae5ca 100644
--- a/include/uapi/linux/Kbuild
+++ b/include/uapi/linux/Kbuild
@@ -406,6 +406,7 @@ header-y += sysinfo.h
header-y += target_core_user.h
header-y += taskstats.h
header-y += tcp.h
+header-y += tls.h
header-y += tcp_metrics.h
header-y += telephony.h
header-y += termios.h
diff --git a/include/uapi/linux/tls.h b/include/uapi/linux/tls.h
new file mode 100644
index 0000000..464621b
--- /dev/null
+++ b/include/uapi/linux/tls.h
@@ -0,0 +1,84 @@
+/* Copyright (c) 2016-2017, Mellanox Technologies All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * - Neither the name of the Mellanox Technologies nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior written
+ * permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE
+ */
+
+#ifndef _UAPI_LINUX_TLS_H
+#define _UAPI_LINUX_TLS_H
+
+#include <linux/types.h>
+#include <asm/byteorder.h>
+#include <linux/socket.h>
+#include <linux/tcp.h>
+
+/* Supported versions */
+#define TLS_VERSION_MINOR(ver) ((ver) & 0xFF)
+#define TLS_VERSION_MAJOR(ver) (((ver) >> 8) & 0xFF)
+
+#define TLS_VERSION_NUMBER(id) ((((id##_VERSION_MAJOR) & 0xFF) << 8) | \
+ ((id##_VERSION_MINOR) & 0xFF))
+
+#define TLS_1_2_VERSION_MAJOR 0x3
+#define TLS_1_2_VERSION_MINOR 0x3
+#define TLS_1_2_VERSION TLS_VERSION_NUMBER(TLS_1_2)
+
+/* Supported ciphers */
+#define TLS_CIPHER_AES_GCM_128 51
+#define TLS_CIPHER_AES_GCM_128_IV_SIZE ((size_t)8)
+#define TLS_CIPHER_AES_GCM_128_KEY_SIZE ((size_t)16)
+#define TLS_CIPHER_AES_GCM_128_SALT_SIZE ((size_t)4)
+#define TLS_CIPHER_AES_GCM_128_TAG_SIZE ((size_t)16)
+
+struct tls_ctrlmsg {
+ unsigned char type;
+ unsigned char data[0];
+} __attribute__((packed));
+
+enum tls_state {
+ TLS_STATE_SW = 0x0,
+ TLS_STATE_HW = 0x1,
+};
+
+struct tls_crypto_info {
+ __u16 version;
+ __u16 cipher_type;
+ __u32 state;
+};
+
+struct tls_crypto_info_aes_gcm_128 {
+ struct tls_crypto_info info;
+ unsigned char iv[TLS_CIPHER_AES_GCM_128_IV_SIZE];
+ unsigned char key[TLS_CIPHER_AES_GCM_128_KEY_SIZE];
+ unsigned char salt[TLS_CIPHER_AES_GCM_128_SALT_SIZE];
+};
+
+#endif /* _UAPI_LINUX_TLS_H */
diff --git a/net/Kconfig b/net/Kconfig
index a100500..b50e899 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -55,6 +55,7 @@ menu "Networking options"
source "net/packet/Kconfig"
source "net/unix/Kconfig"
+source "net/tls/Kconfig"
source "net/xfrm/Kconfig"
source "net/iucv/Kconfig"
diff --git a/net/Makefile b/net/Makefile
index 4cafaa2..23da6df 100644
--- a/net/Makefile
+++ b/net/Makefile
@@ -15,6 +15,7 @@ obj-$(CONFIG_LLC) += llc/
obj-$(CONFIG_NET) += ethernet/ 802/ sched/ netlink/
obj-$(CONFIG_NETFILTER) += netfilter/
obj-$(CONFIG_INET) += ipv4/
+obj-$(CONFIG_TLS) += tls/
obj-$(CONFIG_XFRM) += xfrm/
obj-$(CONFIG_UNIX) += unix/
obj-$(CONFIG_NET) += ipv6/
diff --git a/net/tls/Kconfig b/net/tls/Kconfig
new file mode 100644
index 0000000..75bfb43
--- /dev/null
+++ b/net/tls/Kconfig
@@ -0,0 +1,12 @@
+#
+# TLS configuration
+#
+config TLS
+ tristate "Transport Layer Security support"
+ depends on NET
+ default m
+ ---help---
+ Enable kernel support for TLS protocol. This allows processing
+ of protocol in kernel as well as oflloading it to HW.
+
+ If unsure, say N.
diff --git a/net/tls/Makefile b/net/tls/Makefile
new file mode 100644
index 0000000..65e5677
--- /dev/null
+++ b/net/tls/Makefile
@@ -0,0 +1,7 @@
+#
+# Makefile for the TLS subsystem.
+#
+
+obj-$(CONFIG_TLS) += tls.o
+
+tls-y := tls_main.o tls_device.o
diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c
new file mode 100644
index 0000000..77a4a59
--- /dev/null
+++ b/net/tls/tls_device.c
@@ -0,0 +1,594 @@
+/* Copyright (c) 2016-2017, Mellanox Technologies All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * - Neither the name of the Mellanox Technologies nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior written
+ * permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE
+ */
+
+#include <linux/module.h>
+#include <net/tcp.h>
+#include <net/inet_common.h>
+#include <linux/highmem.h>
+#include <linux/netdevice.h>
+
+#include <net/tls.h>
+
+/* We assume that the socket is already connected */
+static struct net_device *get_netdev_for_sock(struct sock *sk)
+{
+ struct inet_sock *inet = inet_sk(sk);
+ struct net_device *netdev = NULL;
+
+ pr_info("Using output interface 0x%x\n", inet->cork.fl.flowi_oif);
+ netdev = dev_get_by_index(sock_net(sk), inet->cork.fl.flowi_oif);
+
+ return netdev;
+}
+
+static void detach_sock_from_netdev(struct sock *sk, struct tls_context *ctx)
+{
+ struct net_device *netdev;
+
+ netdev = get_netdev_for_sock(sk);
+ if (!netdev) {
+ pr_err("got offloaded socket with no netdev\n");
+ return;
+ }
+
+ if (!netdev->tlsdev_ops) {
+ pr_err("attach_sock_to_netdev: netdev %s with no TLS offload\n",
+ netdev->name);
+ return;
+ }
+
+ netdev->tlsdev_ops->tls_dev_del(netdev, sk, TLS_OFFLOAD_CTX_DIR_TX);
+ dev_put(netdev);
+}
+
+static int attach_sock_to_netdev(struct sock *sk, struct tls_context *ctx)
+{
+ struct net_device *netdev = get_netdev_for_sock(sk);
+ int rc = -EINVAL;
+
+ if (!netdev) {
+ pr_err("attach_sock_to_netdev: netdev not found\n");
+ goto out;
+ }
+
+ if (!netdev->tlsdev_ops) {
+ pr_err("attach_sock_to_netdev: netdev %s with no TLS offload\n",
+ netdev->name);
+ goto out;
+ }
+
+ rc = netdev->tlsdev_ops->tls_dev_add(
+ netdev,
+ sk,
+ TLS_OFFLOAD_CTX_DIR_TX,
+ &ctx->crypto_send,
+ (struct tls_offload_context **)(&ctx->priv_ctx));
+ if (rc) {
+ pr_err("The netdev has refused to offload this socket\n");
+ goto out;
+ }
+
+ sk->sk_bound_dev_if = netdev->ifindex;
+ sk_dst_reset(sk);
+
+ rc = 0;
+out:
+ dev_put(netdev);
+ return rc;
+}
+
+static void destroy_record(struct tls_record_info *record)
+{
+ skb_frag_t *frag;
+ int nr_frags = record->num_frags;
+
+ while (nr_frags > 0) {
+ frag = &record->frags[nr_frags - 1];
+ __skb_frag_unref(frag);
+ --nr_frags;
+ }
+ kfree(record);
+}
+
+static void delete_all_records(struct tls_offload_context *offload_ctx)
+{
+ struct tls_record_info *info, *temp;
+
+ list_for_each_entry_safe(info, temp, &offload_ctx->records_list, list) {
+ list_del(&info->list);
+ destroy_record(info);
+ }
+}
+
+void tls_clear_device_offload(struct sock *sk, struct tls_context *tls_ctx)
+{
+ struct tls_offload_context *ctx = tls_offload_ctx(tls_ctx);
+
+ if (!ctx)
+ return;
+
+ if (ctx->open_record)
+ destroy_record(ctx->open_record);
+
+ delete_all_records(ctx);
+ detach_sock_from_netdev(sk, tls_ctx);
+}
+
+void tls_icsk_clean_acked(struct sock *sk)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(sk);
+ struct tls_offload_context *ctx;
+ struct tcp_sock *tp = tcp_sk(sk);
+ struct tls_record_info *info, *temp;
+ unsigned long flags;
+
+ if (!tls_ctx)
+ return;
+
+ ctx = tls_offload_ctx(tls_ctx);
+
+ spin_lock_irqsave(&ctx->lock, flags);
+ info = ctx->retransmit_hint;
+ if (info && !before(tp->snd_una, info->end_seq)) {
+ ctx->retransmit_hint = NULL;
+ list_del(&info->list);
+ destroy_record(info);
+ }
+
+ list_for_each_entry_safe(info, temp, &ctx->records_list, list) {
+ if (before(tp->snd_una, info->end_seq))
+ break;
+ list_del(&info->list);
+
+ destroy_record(info);
+ }
+
+ spin_unlock_irqrestore(&ctx->lock, flags);
+}
+EXPORT_SYMBOL(tls_icsk_clean_acked);
+
+void tls_device_sk_destruct(struct sock *sk)
+{
+ struct tls_context *ctx = tls_get_ctx(sk);
+
+ tls_clear_device_offload(sk, ctx);
+ tls_sk_destruct(sk, ctx);
+}
+EXPORT_SYMBOL(tls_device_sk_destruct);
+
+static inline void tls_append_frag(struct tls_record_info *record,
+ struct page_frag *pfrag,
+ int size)
+{
+ skb_frag_t *frag;
+
+ frag = &record->frags[record->num_frags - 1];
+ if (frag->page.p == pfrag->page &&
+ frag->page_offset + frag->size == pfrag->offset) {
+ frag->size += size;
+ } else {
+ ++frag;
+ frag->page.p = pfrag->page;
+ frag->page_offset = pfrag->offset;
+ frag->size = size;
+ ++record->num_frags;
+ get_page(pfrag->page);
+ }
+
+ pfrag->offset += size;
+ record->len += size;
+}
+
+static inline int tls_push_record(struct sock *sk,
+ struct tls_context *ctx,
+ struct tls_offload_context *offload_ctx,
+ struct tls_record_info *record,
+ struct page_frag *pfrag,
+ int flags,
+ unsigned char record_type)
+{
+ skb_frag_t *frag;
+ struct tcp_sock *tp = tcp_sk(sk);
+ struct page_frag fallback_frag;
+ struct page_frag *tag_pfrag = pfrag;
+
+ /* fill prepand */
+ frag = &record->frags[0];
+ tls_fill_prepend(ctx,
+ skb_frag_address(frag),
+ record->len - ctx->prepand_size,
+ record_type);
+
+ if (unlikely(!skb_page_frag_refill(
+ ctx->tag_size,
+ pfrag, GFP_KERNEL))) {
+ /* HW doesn't care about the data in the tag
+ * so in case pfrag has no room
+ * for a tag and we can't allocate a new pfrag
+ * just use the page in the first frag
+ * rather then write a complicated fall back code.
+ */
+ tag_pfrag = &fallback_frag;
+ tag_pfrag->page = skb_frag_page(frag);
+ tag_pfrag->offset = 0;
+ }
+
+ tls_append_frag(record, tag_pfrag, ctx->tag_size);
+ record->end_seq = tp->write_seq + record->len;
+ spin_lock_irq(&offload_ctx->lock);
+ list_add_tail(&record->list, &offload_ctx->records_list);
+ spin_unlock_irq(&offload_ctx->lock);
+
+ offload_ctx->open_record = NULL;
+ tls_increment_seqno(ctx->iv, sk);
+
+ /* all ready, send */
+ return tls_push_frags(sk, ctx, record->frags,
+ record->num_frags, 0, flags);
+
+}
+
+static inline int tls_get_new_record(
+ struct tls_offload_context *offload_ctx,
+ struct page_frag *pfrag,
+ size_t prepand_size)
+{
+ skb_frag_t *frag;
+ struct tls_record_info *record;
+
+ /* TODO: do we want to use pfrag
+ * to store the record metadata?
+ * the lifetime of the data and
+ * metadata is the same and
+ * we can avoid kmalloc overhead.
+ */
+ record = kmalloc(sizeof(*record), GFP_KERNEL);
+ if (!record)
+ return -ENOMEM;
+
+ frag = &record->frags[0];
+ __skb_frag_set_page(frag, pfrag->page);
+ frag->page_offset = pfrag->offset;
+ skb_frag_size_set(frag, prepand_size);
+
+ get_page(pfrag->page);
+ pfrag->offset += prepand_size;
+
+ record->num_frags = 1;
+ record->len = prepand_size;
+ offload_ctx->open_record = record;
+ return 0;
+}
+
+static inline int tls_do_allocation(
+ struct sock *sk,
+ struct tls_offload_context *offload_ctx,
+ struct page_frag *pfrag,
+ size_t prepand_size)
+{
+ struct tls_record_info *record;
+
+ if (!sk_page_frag_refill(sk, pfrag))
+ return -ENOMEM;
+
+ record = offload_ctx->open_record;
+ if (!record) {
+ tls_get_new_record(offload_ctx, pfrag, prepand_size);
+ record = offload_ctx->open_record;
+ if (!record)
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static int tls_push_data(struct sock *sk,
+ struct iov_iter *msg_iter,
+ size_t size, int flags,
+ unsigned char record_type)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(sk);
+ struct tls_offload_context *ctx = tls_offload_ctx(tls_ctx);
+ struct tls_record_info *record = ctx->open_record;
+ struct page_frag *pfrag;
+ int copy, rc = 0;
+ size_t orig_size = size;
+ u32 max_open_record_len;
+ long timeo;
+ int more = flags & (MSG_SENDPAGE_NOTLAST | MSG_MORE);
+ int tls_push_record_flags = flags | MSG_SENDPAGE_NOTLAST;
+ bool last = false;
+
+ if (sk->sk_err)
+ return sk->sk_err;
+
+ /* Only one writer at a time is allowed */
+ if (sk->sk_write_pending)
+ return -EBUSY;
+ timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT);
+ pfrag = sk_page_frag(sk);
+
+ /* KTLS_TLS_HEADER_SIZE is not counted as part of the TLS record, and
+ * we need to leave room for an authentication tag.
+ */
+ max_open_record_len = TLS_MAX_PAYLOAD_SIZE
+ + TLS_HEADER_SIZE - tls_ctx->tag_size;
+
+ if (tls_is_pending_open_record(tls_ctx)) {
+ rc = tls_push_paritial_record(sk, tls_ctx, flags);
+ if (rc < 0)
+ return rc;
+ }
+
+ do {
+ if (tls_do_allocation(sk, ctx, pfrag,
+ tls_ctx->prepand_size)) {
+ rc = sk_stream_wait_memory(sk, &timeo);
+ if (!rc)
+ continue;
+
+ record = ctx->open_record;
+ if (!record)
+ break;
+handle_error:
+ if (record_type != TLS_RECORD_TYPE_DATA) {
+ /* avoid sending partial
+ * record with type !=
+ * application_data
+ */
+ size = orig_size;
+ destroy_record(record);
+ ctx->open_record = NULL;
+ } else if (record->len > tls_ctx->prepand_size) {
+ goto last_record;
+ }
+
+ break;
+ }
+
+ record = ctx->open_record;
+ copy = min_t(size_t, size, (pfrag->size - pfrag->offset));
+ copy = min_t(size_t, copy, (max_open_record_len - record->len));
+
+ if (copy_from_iter_nocache(
+ page_address(pfrag->page) + pfrag->offset,
+ copy, msg_iter) != copy) {
+ rc = -EFAULT;
+ goto handle_error;
+ }
+ tls_append_frag(record, pfrag, copy);
+
+ size -= copy;
+ if (!size) {
+last_record:
+ tls_push_record_flags = flags;
+ last = true;
+ }
+
+ if ((last && !more) ||
+ (record->len >= max_open_record_len) ||
+ (record->num_frags >= MAX_SKB_FRAGS - 1)) {
+ rc = tls_push_record(sk,
+ tls_ctx,
+ ctx,
+ record,
+ pfrag,
+ tls_push_record_flags,
+ record_type);
+ if (rc < 0)
+ break;
+ }
+ } while (!last);
+
+ if (orig_size - size > 0) {
+ rc = orig_size - size;
+ if (record_type != TLS_RECORD_TYPE_DATA)
+ rc++;
+ }
+
+ return rc;
+}
+
+static inline bool record_is_open(struct sock *sk)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(sk);
+ struct tls_offload_context *ctx = tls_offload_ctx(tls_ctx);
+ struct tls_record_info *record = ctx->open_record;
+
+ return record;
+}
+
+int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size)
+{
+ unsigned char record_type = TLS_RECORD_TYPE_DATA;
+ int rc = 0;
+
+ lock_sock(sk);
+
+ if (unlikely(msg->msg_flags & MSG_OOB)) {
+ if ((msg->msg_flags & MSG_MORE) || record_is_open(sk)) {
+ rc = -EINVAL;
+ goto out;
+ }
+
+ if (copy_from_iter(&record_type, 1, &msg->msg_iter) != 1) {
+ rc = -EFAULT;
+ goto out;
+ }
+
+ --size;
+ msg->msg_flags &= ~MSG_OOB;
+ }
+
+ rc = tls_push_data(sk, &msg->msg_iter, size,
+ msg->msg_flags,
+ record_type);
+
+out:
+ release_sock(sk);
+ return rc;
+}
+
+int tls_device_sendpage(struct sock *sk, struct page *page,
+ int offset, size_t size, int flags)
+{
+ struct iov_iter msg_iter;
+ struct kvec iov;
+ char *kaddr = kmap(page);
+ int rc = 0;
+
+ if (flags & MSG_SENDPAGE_NOTLAST)
+ flags |= MSG_MORE;
+
+ lock_sock(sk);
+
+ if (flags & MSG_OOB) {
+ rc = -ENOTSUPP;
+ goto out;
+ }
+
+ iov.iov_base = kaddr + offset;
+ iov.iov_len = size;
+ iov_iter_kvec(&msg_iter, WRITE | ITER_KVEC, &iov, 1, size);
+ rc = tls_push_data(sk, &msg_iter, size,
+ flags,
+ TLS_RECORD_TYPE_DATA);
+ kunmap(page);
+
+out:
+ release_sock(sk);
+ return rc;
+}
+
+struct tls_record_info *tls_get_record(struct tls_offload_context *context,
+ u32 seq)
+{
+ struct tls_record_info *info;
+
+ info = context->retransmit_hint;
+ if (!info ||
+ before(seq, info->end_seq - info->len))
+ info = list_first_entry(&context->records_list,
+ struct tls_record_info, list);
+
+ list_for_each_entry_from(info, &context->records_list, list) {
+ if (before(seq, info->end_seq)) {
+ if (!context->retransmit_hint ||
+ after(info->end_seq,
+ context->retransmit_hint->end_seq))
+ context->retransmit_hint = info;
+ return info;
+ }
+ }
+
+ return NULL;
+}
+EXPORT_SYMBOL(tls_get_record);
+
+int tls_set_device_offload(struct sock *sk, struct tls_context *ctx)
+{
+ struct tls_crypto_info *crypto_info;
+ struct tls_offload_context *offload_ctx;
+ struct tls_record_info *dummy_record;
+ u16 nonece_size, tag_size, iv_size;
+ char *iv;
+ int rc;
+
+ if (!ctx) {
+ rc = -EINVAL;
+ goto out;
+ }
+
+ if (ctx->priv_ctx) {
+ rc = -EEXIST;
+ goto out;
+ }
+
+ crypto_info = &ctx->crypto_send;
+ switch (crypto_info->cipher_type) {
+ case TLS_CIPHER_AES_GCM_128: {
+ nonece_size = TLS_CIPHER_AES_GCM_128_IV_SIZE;
+ tag_size = TLS_CIPHER_AES_GCM_128_TAG_SIZE;
+ iv_size = TLS_CIPHER_AES_GCM_128_IV_SIZE;
+ iv = ((struct tls_crypto_info_aes_gcm_128 *)crypto_info)->iv;
+ break;
+ }
+ default:
+ rc = -EINVAL;
+ goto out;
+ }
+
+ dummy_record = kmalloc(sizeof(*dummy_record), GFP_KERNEL);
+ if (!dummy_record) {
+ rc = -ENOMEM;
+ goto out;
+ }
+
+ rc = attach_sock_to_netdev(sk, ctx);
+ if (rc)
+ goto err_dummy_record;
+
+ ctx->prepand_size = TLS_HEADER_SIZE + nonece_size;
+ ctx->tag_size = tag_size;
+ ctx->iv_size = iv_size;
+ ctx->iv = kmalloc(iv_size, GFP_KERNEL);
+ if (!ctx->iv) {
+ rc = ENOMEM;
+ goto detach_sock;
+ }
+ memcpy(ctx->iv, iv, iv_size);
+
+ offload_ctx = ctx->priv_ctx;
+ dummy_record->end_seq = offload_ctx->expectedSN;
+ dummy_record->len = 0;
+ dummy_record->num_frags = 0;
+
+ INIT_LIST_HEAD(&offload_ctx->records_list);
+ list_add_tail(&dummy_record->list, &offload_ctx->records_list);
+ spin_lock_init(&offload_ctx->lock);
+
+ inet_csk(sk)->icsk_clean_acked = &tls_icsk_clean_acked;
+
+ /* After this line the tx_handler might access the offload context */
+ smp_store_release(&sk->sk_destruct,
+ &tls_device_sk_destruct);
+ goto out;
+
+detach_sock:
+ detach_sock_from_netdev(sk, ctx);
+err_dummy_record:
+ kfree(dummy_record);
+out:
+ return rc;
+}
diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
new file mode 100644
index 0000000..6a3df25
--- /dev/null
+++ b/net/tls/tls_main.c
@@ -0,0 +1,348 @@
+/* Copyright (c) 2016-2017, Mellanox Technologies All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * - Neither the name of the Mellanox Technologies nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior written
+ * permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE
+ */
+
+#include <linux/module.h>
+
+#include <net/tcp.h>
+#include <net/inet_common.h>
+#include <linux/highmem.h>
+#include <linux/netdevice.h>
+
+#include <net/tls.h>
+
+MODULE_AUTHOR("Mellanox Technologies");
+MODULE_DESCRIPTION("Transport Layer Security Support");
+MODULE_LICENSE("Dual BSD/GPL");
+
+static struct proto tls_device_prot;
+
+int tls_push_frags(struct sock *sk,
+ struct tls_context *ctx,
+ skb_frag_t *frag,
+ u16 num_frags,
+ u16 first_offset,
+ int flags)
+{
+ int sendpage_flags = flags | MSG_SENDPAGE_NOTLAST;
+ int ret = 0;
+ size_t size;
+ int offset = first_offset;
+
+ size = skb_frag_size(frag) - offset;
+ offset += frag->page_offset;
+
+ while (1) {
+ if (!--num_frags)
+ sendpage_flags = flags;
+
+ /* is sending application-limited? */
+ tcp_rate_check_app_limited(sk);
+retry:
+ ret = do_tcp_sendpages(sk,
+ skb_frag_page(frag),
+ offset,
+ size,
+ sendpage_flags);
+
+ if (ret != size) {
+ if (ret > 0) {
+ offset += ret;
+ size -= ret;
+ goto retry;
+ }
+
+ offset -= frag->page_offset;
+ ctx->pending_offset = offset;
+ ctx->pending_frags = frag;
+ ctx->num_pending_frags = num_frags + 1;
+ return ret;
+ }
+
+ if (!num_frags)
+ break;
+
+ frag++;
+ offset = frag->page_offset;
+ size = skb_frag_size(frag);
+ }
+
+ return 0;
+}
+
+int tls_push_paritial_record(struct sock *sk, struct tls_context *ctx,
+ int flags) {
+ skb_frag_t *frag = ctx->pending_frags;
+ u16 offset = ctx->pending_offset;
+ u16 num_frags = ctx->num_pending_frags;
+
+ ctx->num_pending_frags = 0;
+
+ return tls_push_frags(sk, ctx, frag,
+ num_frags, offset, flags);
+}
+
+static void tls_write_space(struct sock *sk)
+{
+ struct tls_context *ctx = tls_get_ctx(sk);
+
+ if (tls_is_pending_open_record(ctx)) {
+ gfp_t sk_allocation = sk->sk_allocation;
+ int rc;
+
+ sk->sk_allocation = GFP_ATOMIC;
+ rc = tls_push_paritial_record(sk, ctx,
+ MSG_DONTWAIT | MSG_NOSIGNAL);
+ sk->sk_allocation = sk_allocation;
+
+ if (rc < 0)
+ return;
+ }
+
+ ctx->sk_write_space(sk);
+}
+
+int tls_sk_query(struct sock *sk, int optname, char __user *optval,
+ int __user *optlen)
+{
+ int rc = 0;
+ struct tls_context *ctx = tls_get_ctx(sk);
+ struct tls_crypto_info *crypto_info;
+ int len;
+
+ if (get_user(len, optlen))
+ return -EFAULT;
+
+ if (!optval || (len < sizeof(*crypto_info))) {
+ rc = -EINVAL;
+ goto out;
+ }
+
+ if (!ctx) {
+ rc = -EBUSY;
+ goto out;
+ }
+
+ /* get user crypto info */
+ switch (optname) {
+ case TCP_TLS_TX: {
+ crypto_info = &ctx->crypto_send;
+ break;
+ }
+ case TCP_TLS_RX:
+ /* fallthru since for now we don't support */
+ default: {
+ rc = -ENOPROTOOPT;
+ goto out;
+ }
+ }
+
+ if (!TLS_CRYPTO_INFO_READY(crypto_info)) {
+ rc = -EBUSY;
+ goto out;
+ }
+
+ if (len == sizeof(crypto_info)) {
+ rc = copy_to_user(optval, crypto_info, sizeof(*crypto_info));
+ goto out;
+ }
+
+ switch (crypto_info->cipher_type) {
+ case TLS_CIPHER_AES_GCM_128: {
+ struct tls_crypto_info_aes_gcm_128 *crypto_info_aes_gcm_128 =
+ container_of(crypto_info,
+ struct tls_crypto_info_aes_gcm_128,
+ info);
+
+ if (len != sizeof(*crypto_info_aes_gcm_128)) {
+ rc = -EINVAL;
+ goto out;
+ }
+ if (TLS_IS_STATE_HW(crypto_info)) {
+ lock_sock(sk);
+ memcpy(crypto_info_aes_gcm_128->iv,
+ ctx->iv,
+ TLS_CIPHER_AES_GCM_128_IV_SIZE);
+ release_sock(sk);
+ }
+ rc = copy_to_user(optval,
+ crypto_info_aes_gcm_128,
+ sizeof(*crypto_info_aes_gcm_128));
+ break;
+ }
+ default:
+ rc = -EINVAL;
+ }
+
+out:
+ return rc;
+}
+EXPORT_SYMBOL(tls_sk_query);
+
+void tls_sk_destruct(struct sock *sk, struct tls_context *ctx)
+{
+ ctx->sk_destruct(sk);
+ kfree(ctx->iv);
+ kfree(ctx);
+ module_put(THIS_MODULE);
+}
+
+int tls_sk_attach(struct sock *sk, int optname, char __user *optval,
+ unsigned int optlen)
+{
+ int rc = 0;
+ struct tls_context *ctx = tls_get_ctx(sk);
+ struct tls_crypto_info *crypto_info;
+ bool allocated_tls_ctx = false;
+
+ if (!optval || (optlen < sizeof(*crypto_info))) {
+ rc = -EINVAL;
+ goto out;
+ }
+
+ /* allocate tls context */
+ if (!ctx) {
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ if (!ctx) {
+ rc = -ENOMEM;
+ goto out;
+ }
+ sk->sk_user_data = ctx;
+ allocated_tls_ctx = true;
+ }
+
+ /* get user crypto info */
+ switch (optname) {
+ case TCP_TLS_TX: {
+ crypto_info = &ctx->crypto_send;
+ break;
+ }
+ case TCP_TLS_RX:
+ /* fallthru since for now we don't support */
+ default: {
+ rc = -ENOPROTOOPT;
+ goto err_sk_user_data;
+ }
+ }
+
+ /* Currently we don't support set crypto info more than one time */
+ if (TLS_CRYPTO_INFO_READY(crypto_info)) {
+ rc = -EEXIST;
+ goto err_sk_user_data;
+ }
+
+ rc = copy_from_user(crypto_info, optval, sizeof(*crypto_info));
+ if (rc) {
+ rc = -EFAULT;
+ goto err_sk_user_data;
+ }
+
+ /* currently we support only HW offload */
+ if (!TLS_IS_STATE_HW(crypto_info)) {
+ rc = -ENOPROTOOPT;
+ goto err_crypto_info;
+ }
+
+ /* check version */
+ if (crypto_info->version != TLS_1_2_VERSION) {
+ rc = -ENOTSUPP;
+ goto err_crypto_info;
+ }
+
+ switch (crypto_info->cipher_type) {
+ case TLS_CIPHER_AES_GCM_128: {
+ if (optlen != sizeof(struct tls_crypto_info_aes_gcm_128)) {
+ rc = -EINVAL;
+ goto err_crypto_info;
+ }
+ rc = copy_from_user(crypto_info,
+ optval,
+ sizeof(struct tls_crypto_info_aes_gcm_128));
+ break;
+ }
+ default:
+ rc = -EINVAL;
+ goto err_crypto_info;
+ }
+
+ if (rc) {
+ rc = -EFAULT;
+ goto err_crypto_info;
+ }
+
+ ctx->sk_write_space = sk->sk_write_space;
+ ctx->sk_destruct = sk->sk_destruct;
+ sk->sk_write_space = tls_write_space;
+
+ if (TLS_IS_STATE_HW(crypto_info)) {
+ rc = tls_set_device_offload(sk, ctx);
+ if (rc)
+ goto err_crypto_info;
+ }
+
+ if (!try_module_get(THIS_MODULE)) {
+ rc = -EINVAL;
+ goto err_set_device_offload;
+ }
+
+ /* TODO: add protection */
+ sk->sk_prot = &tls_device_prot;
+ goto out;
+
+err_set_device_offload:
+ tls_clear_device_offload(sk, ctx);
+err_crypto_info:
+ memset(crypto_info, 0, sizeof(*crypto_info));
+err_sk_user_data:
+ if (allocated_tls_ctx)
+ kfree(ctx);
+out:
+ return rc;
+}
+EXPORT_SYMBOL(tls_sk_attach);
+
+static int __init tls_init(void)
+{
+ tls_device_prot = tcp_prot;
+ tls_device_prot.sendmsg = tls_device_sendmsg;
+ tls_device_prot.sendpage = tls_device_sendpage;
+
+ return 0;
+}
+
+static void __exit tls_exit(void)
+{
+}
+
+module_init(tls_init);
+module_exit(tls_exit);
--
2.7.4
^ permalink raw reply related
* [RFC TLS Offload Support 07/15] mlx/mlx5_core: Allow sending multiple packets
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
In-Reply-To: <1490707592-1430-1-git-send-email-aviadye@mellanox.com>
From: Ilya Lesokhin <ilyal@mellanox.com>
Modify mlx5e_xmit to xmit multiple packet chained
using skb->next
Signed-off-by: Ilya Lesokhin <ilyal@mellanox.com>
Signed-off-by: Aviad Yehezkel <aviadye@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en_tx.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
index e6ce509..f2d0cc0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
@@ -35,7 +35,7 @@
#include "en.h"
#define MLX5E_SQ_NOPS_ROOM MLX5_SEND_WQE_MAX_WQEBBS
-#define MLX5E_SQ_STOP_ROOM (MLX5_SEND_WQE_MAX_WQEBBS +\
+#define MLX5E_SQ_STOP_ROOM (2 * MLX5_SEND_WQE_MAX_WQEBBS +\
MLX5E_SQ_NOPS_ROOM)
void mlx5e_send_nop(struct mlx5e_sq *sq, bool notify_hw)
@@ -405,6 +405,8 @@ netdev_tx_t mlx5e_xmit(struct sk_buff *skb, struct net_device *dev)
struct mlx5e_sq *sq = NULL;
struct mlx5_accel_ops *accel_ops;
struct mlx5_swp_info swp_info = {0};
+ struct sk_buff *next;
+ int rc;
rcu_read_lock();
accel_ops = mlx5_accel_get(priv->mdev);
@@ -417,7 +419,12 @@ netdev_tx_t mlx5e_xmit(struct sk_buff *skb, struct net_device *dev)
sq = priv->txq_to_sq_map[skb_get_queue_mapping(skb)];
- return mlx5e_sq_xmit(sq, skb, &swp_info);
+ do {
+ next = skb->next;
+ rc = mlx5e_sq_xmit(sq, skb, &swp_info);
+ skb = next;
+ } while (next);
+ return rc;
}
bool mlx5e_poll_tx_cq(struct mlx5e_cq *cq, int napi_budget)
--
2.7.4
^ permalink raw reply related
* [RFC TLS Offload Support 09/15] mlx/tls: Sysfs configuration interface Configure the driver/hardware interface via sysfs.
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
In-Reply-To: <1490707592-1430-1-git-send-email-aviadye@mellanox.com>
From: Ilya Lesokhin <ilyal@mellanox.com>
Signed-off-by: Guy Shapiro <guysh@mellanox.com>
Signed-off-by: Ilya Lesokhin <ilyal@mellanox.com>
Signed-off-by: Matan Barak <matanb@mellanox.com>
Signed-off-by: Aviad Yehezkel <aviadye@mellanox.com>
---
.../ethernet/mellanox/accelerator/tls/tls_sysfs.c | 194 +++++++++++++++++++++
.../ethernet/mellanox/accelerator/tls/tls_sysfs.h | 45 +++++
2 files changed, 239 insertions(+)
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.c
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.h
diff --git a/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.c b/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.c
new file mode 100644
index 0000000..2860fc3
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.c
@@ -0,0 +1,194 @@
+/*
+ * Copyright (c) 2015-2017 Mellanox Technologies. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+
+#include <rdma/ib_verbs.h>
+
+#include "tls_sysfs.h"
+#include "tls_cmds.h"
+
+#ifdef MLX_TLS_SADB_RDMA
+struct mlx_tls_attribute {
+ struct attribute attr;
+ ssize_t (*show)(struct mlx_tls_dev *dev, char *buf);
+ ssize_t (*store)(struct mlx_tls_dev *dev, const char *buf,
+ size_t count);
+};
+
+#define MLX_TLS_ATTR(_name, _mode, _show, _store) \
+ struct mlx_tls_attribute mlx_tls_attr_##_name = { \
+ .attr = {.name = __stringify(_name), .mode = _mode}, \
+ .show = _show, \
+ .store = _store, \
+ }
+#define to_mlx_tls_dev(obj) \
+ container_of(kobj, struct mlx_tls_dev, kobj)
+#define to_mlx_tls_attr(_attr) \
+ container_of(attr, struct mlx_tls_attribute, attr)
+
+static ssize_t mlx_tls_attr_show(struct kobject *kobj, struct attribute *attr,
+ char *buf)
+{
+ struct mlx_tls_dev *dev = to_mlx_tls_dev(kobj);
+ struct mlx_tls_attribute *mlx_tls_attr = to_mlx_tls_attr(attr);
+ ssize_t ret = -EIO;
+
+ if (mlx_tls_attr->show)
+ ret = mlx_tls_attr->show(dev, buf);
+
+ return ret;
+}
+
+static ssize_t mlx_tls_attr_store(struct kobject *kobj, struct attribute *attr,
+ const char *buf, size_t count)
+{
+ struct mlx_tls_dev *dev = to_mlx_tls_dev(kobj);
+ struct mlx_tls_attribute *mlx_tls_attr = to_mlx_tls_attr(attr);
+ ssize_t ret = -EIO;
+
+ if (mlx_tls_attr->store)
+ ret = mlx_tls_attr->store(dev, buf, count);
+
+ return ret;
+}
+
+static ssize_t mlx_tls_sqpn_read(struct mlx_tls_dev *dev, char *buf)
+{
+ return sprintf(buf, "%u\n", dev->conn->qp->qp_num);
+}
+
+static ssize_t mlx_tls_sgid_read(struct mlx_tls_dev *dev, char *buf)
+{
+ union ib_gid *sgid = (union ib_gid *)&dev->conn->fpga_qpc.remote_ip;
+
+ return sprintf(buf, "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x\n",
+ be16_to_cpu(((__be16 *)sgid->raw)[0]),
+ be16_to_cpu(((__be16 *)sgid->raw)[1]),
+ be16_to_cpu(((__be16 *)sgid->raw)[2]),
+ be16_to_cpu(((__be16 *)sgid->raw)[3]),
+ be16_to_cpu(((__be16 *)sgid->raw)[4]),
+ be16_to_cpu(((__be16 *)sgid->raw)[5]),
+ be16_to_cpu(((__be16 *)sgid->raw)[6]),
+ be16_to_cpu(((__be16 *)sgid->raw)[7]));
+}
+
+static ssize_t mlx_tls_dqpn_read(struct mlx_tls_dev *dev, char *buf)
+{
+ return sprintf(buf, "%u\n", dev->conn->fpga_qpn);
+}
+
+static ssize_t mlx_tls_dqpn_write(struct mlx_tls_dev *dev, const char *buf,
+ size_t count)
+{
+ int tmp;
+
+ tmp = sscanf(buf, "%u\n", &dev->conn->fpga_qpn);
+ mlx_accel_core_connect(dev->conn);
+
+ return count;
+}
+
+static ssize_t mlx_tls_dgid_read(struct mlx_tls_dev *dev, char *buf)
+{
+ union ib_gid *dgid = (union ib_gid *)&dev->conn->fpga_qpc.fpga_ip;
+
+ return sprintf(buf, "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x\n",
+ be16_to_cpu(((__be16 *)dgid->raw)[0]),
+ be16_to_cpu(((__be16 *)dgid->raw)[1]),
+ be16_to_cpu(((__be16 *)dgid->raw)[2]),
+ be16_to_cpu(((__be16 *)dgid->raw)[3]),
+ be16_to_cpu(((__be16 *)dgid->raw)[4]),
+ be16_to_cpu(((__be16 *)dgid->raw)[5]),
+ be16_to_cpu(((__be16 *)dgid->raw)[6]),
+ be16_to_cpu(((__be16 *)dgid->raw)[7]));
+}
+
+static ssize_t mlx_tls_dgid_write(struct mlx_tls_dev *dev, const char *buf,
+ size_t count)
+{
+ union ib_gid *dgid = (union ib_gid *)&dev->conn->fpga_qpc.fpga_ip;
+ int i = 0;
+ int tmp;
+
+ tmp = sscanf(buf,
+ "%04hx:%04hx:%04hx:%04hx:%04hx:%04hx:%04hx:%04hx\n",
+ &(((__be16 *)dgid->raw)[0]),
+ &(((__be16 *)dgid->raw)[1]),
+ &(((__be16 *)dgid->raw)[2]),
+ &(((__be16 *)dgid->raw)[3]),
+ &(((__be16 *)dgid->raw)[4]),
+ &(((__be16 *)dgid->raw)[5]),
+ &(((__be16 *)dgid->raw)[6]),
+ &(((__be16 *)dgid->raw)[7]));
+
+ for (i = 0; i < 8; i++)
+ ((__be16 *)dgid->raw)[i] = cpu_to_be16(((u16 *)dgid->raw)[i]);
+
+ return count;
+}
+
+static void mlx_tls_dev_release(struct kobject *kobj)
+{
+}
+
+static MLX_TLS_ATTR(sqpn, 0444, mlx_tls_sqpn_read, NULL);
+static MLX_TLS_ATTR(sgid, 0444, mlx_tls_sgid_read, NULL);
+static MLX_TLS_ATTR(dqpn, 0666, mlx_tls_dqpn_read, mlx_tls_dqpn_write);
+static MLX_TLS_ATTR(dgid, 0666, mlx_tls_dgid_read, mlx_tls_dgid_write);
+
+struct attribute *mlx_tls_def_attrs[] = {
+ &mlx_tls_attr_sqpn.attr,
+ &mlx_tls_attr_sgid.attr,
+ &mlx_tls_attr_dqpn.attr,
+ &mlx_tls_attr_dgid.attr,
+ NULL,
+};
+
+const struct sysfs_ops mlx_tls_dev_sysfs_ops = {
+ .show = mlx_tls_attr_show,
+ .store = mlx_tls_attr_store,
+};
+
+static struct kobj_type mlx_tls_dev_type = {
+ .release = mlx_tls_dev_release,
+ .sysfs_ops = &mlx_tls_dev_sysfs_ops,
+ .default_attrs = mlx_tls_def_attrs,
+};
+
+int tls_sysfs_init_and_add(struct kobject *kobj, struct kobject *parent,
+ const char *fmt, char *arg)
+{
+ return kobject_init_and_add(kobj, &mlx_tls_dev_type,
+ parent,
+ fmt, arg);
+}
+#endif
diff --git a/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.h b/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.h
new file mode 100644
index 0000000..bfaa857
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.h
@@ -0,0 +1,45 @@
+/*
+ * Copyright (c) 2015-2017 Mellanox Technologies. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+#ifndef __TLS_SYSFS_H__
+#define __TLS_SYSFS_H__
+
+#include <linux/sysfs.h>
+
+#include "tls.h"
+
+#ifdef MLX_TLS_SADB_RDMA
+int tls_sysfs_init_and_add(struct kobject *kobj, struct kobject *parent,
+ const char *fmt, char *arg);
+#endif
+
+#endif /* __TLS_SYSFS_H__ */
--
2.7.4
^ permalink raw reply related
* [RFC TLS Offload Support 10/15] mlx/tls: Add mlx_accel offload driver for TLS
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
In-Reply-To: <1490707592-1430-1-git-send-email-aviadye@mellanox.com>
From: Ilya Lesokhin <ilyal@mellanox.com>
Implement the transmit and receive callbacks as well as the netdev
operations for adding and removing sockets.
Signed-off-by: Guy Shapiro <guysh@mellanox.com>
Signed-off-by: Ilya Lesokhin <ilyal@mellanox.com>
Signed-off-by: Matan Barak <matanb@mellanox.com>
Signed-off-by: Haggai Eran <haggaie@mellanox.com>
Signed-off-by: Aviad Yehezkel <aviadye@mellanox.com>
---
.../net/ethernet/mellanox/accelerator/tls/tls.c | 652 +++++++++++++++++++++
.../net/ethernet/mellanox/accelerator/tls/tls.h | 100 ++++
2 files changed, 752 insertions(+)
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls.c
create mode 100644 drivers/net/ethernet/mellanox/accelerator/tls/tls.h
diff --git a/drivers/net/ethernet/mellanox/accelerator/tls/tls.c b/drivers/net/ethernet/mellanox/accelerator/tls/tls.c
new file mode 100644
index 0000000..07a4b67
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/accelerator/tls/tls.c
@@ -0,0 +1,652 @@
+/*
+ * Copyright (c) 2015-2017 Mellanox Technologies. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+#include "tls.h"
+#include "tls_sysfs.h"
+#include "tls_hw.h"
+#include "tls_cmds.h"
+#include <linux/mlx5/driver.h>
+#include <linux/netdevice.h>
+
+static LIST_HEAD(mlx_tls_devs);
+static DEFINE_MUTEX(mlx_tls_mutex);
+
+/* Start of context identifiers range (inclusive) */
+#define SWID_START 5
+/* End of context identifiers range (exclusive) */
+#define SWID_END BIT(24)
+
+static netdev_features_t mlx_tls_feature_chk(struct sk_buff *skb,
+ struct net_device *netdev,
+ netdev_features_t features,
+ bool *done)
+{
+ return features;
+}
+
+int mlx_tls_get_count(struct net_device *netdev)
+{
+ return 0;
+}
+
+int mlx_tls_get_strings(struct net_device *netdev, uint8_t *data)
+{
+ return 0;
+}
+
+int mlx_tls_get_stats(struct net_device *netdev, u64 *data)
+{
+ return 0;
+}
+
+/* must hold mlx_tls_mutex to call this function */
+static struct mlx_tls_dev *find_mlx_tls_dev_by_netdev(
+ struct net_device *netdev)
+{
+ struct mlx_tls_dev *dev;
+
+ list_for_each_entry(dev, &mlx_tls_devs, accel_dev_list) {
+ if (dev->netdev == netdev)
+ return dev;
+ }
+
+ return NULL;
+}
+
+struct mlx_tls_offload_context *get_tls_context(struct sock *sk)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(sk);
+
+ return container_of(tls_offload_ctx(tls_ctx),
+ struct mlx_tls_offload_context,
+ context);
+}
+
+static int mlx_tls_add(struct net_device *netdev,
+ struct sock *sk,
+ enum tls_offload_ctx_dir direction,
+ struct tls_crypto_info *crypto_info,
+ struct tls_offload_context **ctx)
+{
+ struct tls_crypto_info_aes_gcm_128 *crypto_info_aes_gcm_128;
+ struct mlx_tls_offload_context *context;
+ struct mlx_tls_dev *dev;
+ int swid;
+ int ret;
+
+ pr_info("mlx_tls_add called\n");
+
+ if (direction == TLS_OFFLOAD_CTX_DIR_RX) {
+ pr_err("mlx_tls_add(): do not support recv\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ if (!crypto_info ||
+ crypto_info->cipher_type != TLS_CIPHER_AES_GCM_128) {
+ pr_err("mlx_tls_add(): support only aes_gcm_128\n");
+ ret = -EINVAL;
+ goto out;
+ }
+ crypto_info_aes_gcm_128 =
+ (struct tls_crypto_info_aes_gcm_128 *)crypto_info;
+
+ dev = mlx_tls_find_dev_by_netdev(netdev);
+ if (!dev) {
+ pr_err("mlx_tls_add(): tls dev not found\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ swid = ida_simple_get(&dev->swid_ida, SWID_START, SWID_END,
+ GFP_KERNEL);
+ if (swid < 0) {
+ pr_err("mlx_tls_add(): Failed to allocate swid\n");
+ ret = swid;
+ goto out;
+ }
+
+ context = kzalloc(sizeof(*context), GFP_KERNEL);
+ if (!context) {
+ ret = -ENOMEM;
+ goto release_swid;
+ }
+
+ context->swid = htonl(swid);
+ context->context.expectedSN = tcp_sk(sk)->write_seq;
+
+ ret = mlx_tls_hw_start_cmd(dev,
+ sk,
+ crypto_info_aes_gcm_128,
+ context);
+ if (ret)
+ goto relese_context;
+
+ try_module_get(THIS_MODULE);
+ *ctx = &context->context;
+out:
+ return ret;
+
+relese_context:
+ kfree(context);
+release_swid:
+ ida_simple_remove(&dev->swid_ida, swid);
+ return ret;
+}
+
+static void mlx_tls_del(struct net_device *netdev,
+ struct sock *sk,
+ enum tls_offload_ctx_dir direction)
+{
+ struct mlx_tls_offload_context *context = NULL;
+
+ if (direction == TLS_OFFLOAD_CTX_DIR_RX) {
+ pr_err("mlx_tls_del(): do not support recv\n");
+ return;
+ }
+
+ context = get_tls_context(sk);
+ if (context)
+ mlx_tls_hw_stop_cmd(netdev, context);
+ else
+ pr_err("delete non-offloaded context\n");
+}
+
+static const struct tlsdev_ops mlx_tls_ops = {
+ .tls_dev_add = mlx_tls_add,
+ .tls_dev_del = mlx_tls_del,
+};
+
+struct mlx_tls_dev *mlx_tls_find_dev_by_netdev(struct net_device *netdev)
+{
+ struct mlx_tls_dev *dev;
+
+ mutex_lock(&mlx_tls_mutex);
+ dev = find_mlx_tls_dev_by_netdev(netdev);
+ mutex_unlock(&mlx_tls_mutex);
+ return dev;
+}
+
+#define SYNDROME_OFFLOAD_REQUIRED 32
+#define SYNDROME_SYNC 33.
+#define SYNDROME_BYPASS 34
+
+#define MIN_BYPASS_RECORD_SIZE 29
+#define BYPASS_RECORD_PADDING_SIZE \
+ (MIN_BYPASS_RECORD_SIZE - TLS_HEADER_SIZE)
+
+#define MAX_BYPASS_SIZE ((1 << 15) - BYPASS_RECORD_PADDING_SIZE - 1)
+
+static void create_bypass_record(u8 *buf, u16 len)
+{
+ len += BYPASS_RECORD_PADDING_SIZE;
+ buf[0] = TLS_RECORD_TYPE_DATA;
+ buf[1] = TLS_1_2_VERSION_MAJOR;
+ buf[2] = TLS_1_2_VERSION_MINOR;
+ buf[3] = len >> 8;
+ buf[4] = len & 0xFF;
+ memset(buf + TLS_HEADER_SIZE, 0, BYPASS_RECORD_PADDING_SIZE);
+}
+
+struct sync_info {
+ s32 sync_len;
+ int nr_frags;
+ skb_frag_t frags[MAX_SKB_FRAGS];
+};
+
+static int get_sync_data(struct mlx_tls_offload_context *context,
+ u32 tcp_seq, struct sync_info *info)
+{
+ struct tls_record_info *record;
+ unsigned long flags;
+ int remaining;
+ s32 sync_size;
+ int ret = -EINVAL;
+ int i = 0;
+
+ spin_lock_irqsave(&context->context.lock, flags);
+ record = tls_get_record(&context->context, tcp_seq);
+
+ if (unlikely(!record)) {
+ pr_err("record not found for seq %u\n", tcp_seq);
+ goto out;
+ }
+
+ sync_size = tcp_seq - (record->end_seq - record->len);
+ info->sync_len = sync_size;
+ if (unlikely(sync_size < 0)) {
+ if (record->len != 0) {
+ pr_err("Invalid record for seq %u\n", tcp_seq);
+ goto out;
+ }
+ goto done;
+ }
+
+ remaining = sync_size;
+ while (remaining > 0) {
+ info->frags[i] = record->frags[i];
+ __skb_frag_ref(&info->frags[i]);
+ remaining -= skb_frag_size(&info->frags[i]);
+
+ if (remaining < 0) {
+ skb_frag_size_add(
+ &info->frags[i],
+ remaining);
+ }
+
+ i++;
+ }
+ info->nr_frags = i;
+done:
+ ret = 0;
+out:
+ spin_unlock_irqrestore(&context->context.lock, flags);
+ return ret;
+}
+
+static struct sk_buff *complete_sync_skb(
+ struct sk_buff *skb,
+ struct sk_buff *nskb,
+ u32 tcp_seq,
+ int headln,
+ unsigned char syndrome
+ )
+{
+ struct iphdr *iph;
+ struct tcphdr *th;
+ int mss;
+ struct pet *pet;
+ __be16 tcp_seq_low;
+
+ nskb->dev = skb->dev;
+ skb_reset_mac_header(nskb);
+ skb_set_network_header(nskb, skb_network_offset(skb));
+ skb_set_transport_header(nskb, skb_transport_offset(skb));
+ memcpy(nskb->data, skb->data, headln);
+
+ iph = ip_hdr(nskb);
+ iph->tot_len = htons(nskb->len - skb_network_offset(nskb));
+ th = tcp_hdr(nskb);
+ tcp_seq -= nskb->data_len;
+ th->seq = htonl(tcp_seq);
+ tcp_seq_low = htons(tcp_seq);
+
+ mss = nskb->dev->mtu - (headln - skb_network_offset(nskb));
+ skb_shinfo(nskb)->gso_size = 0;
+ if (nskb->data_len > mss) {
+ skb_shinfo(nskb)->gso_size = mss;
+ skb_shinfo(nskb)->gso_segs = DIV_ROUND_UP(nskb->data_len, mss);
+ }
+ skb_shinfo(nskb)->gso_type = skb_shinfo(skb)->gso_type;
+
+ nskb->queue_mapping = skb->queue_mapping;
+
+ pet = (struct pet *)(nskb->data + sizeof(struct ethhdr));
+ pet->syndrome = syndrome;
+ memcpy(pet->content.raw, &tcp_seq_low, sizeof(tcp_seq_low));
+
+ nskb->ip_summed = CHECKSUM_PARTIAL;
+ __skb_pull(nskb, skb_transport_offset(skb));
+ inet_csk(skb->sk)->icsk_af_ops->send_check(skb->sk, nskb);
+ __skb_push(nskb, skb_transport_offset(skb));
+
+ nskb->next = skb;
+ nskb->xmit_more = 1;
+ return nskb;
+}
+
+static void strip_pet(struct sk_buff *skb)
+{
+ struct ethhdr *old_eth;
+ struct ethhdr *new_eth;
+
+ old_eth = (struct ethhdr *)((skb->data) - sizeof(struct ethhdr));
+ new_eth = (struct ethhdr *)((skb_pull_inline(skb, sizeof(struct pet)))
+ - sizeof(struct ethhdr));
+ skb->mac_header += sizeof(struct pet);
+
+ memmove(new_eth, old_eth, 2 * ETH_ALEN);
+ /* Ethertype is already in its new place */
+}
+
+static struct sk_buff *handle_ooo(struct mlx_tls_offload_context *context,
+ struct sk_buff *skb)
+{
+ struct sync_info info;
+ u32 tcp_seq = ntohl(tcp_hdr(skb)->seq);
+ struct sk_buff *nskb;
+ int linear_len = 0;
+ int headln;
+ unsigned char syndrome = SYNDROME_SYNC;
+
+ if (get_sync_data(context, tcp_seq, &info)) {
+ dev_kfree_skb_any(skb);
+ return NULL;
+ }
+
+ headln = skb_transport_offset(skb) + tcp_hdrlen(skb);
+
+ if (unlikely(info.sync_len < 0)) {
+ if (-info.sync_len > MAX_BYPASS_SIZE) {
+ if (skb->len - headln > -info.sync_len) {
+ pr_err("Required bypass record is too big\n");
+ /* can fragment into two large SKBs in SW */
+ return NULL;
+ }
+ skb_push(skb, sizeof(struct ethhdr));
+ strip_pet(skb);
+ skb_pull(skb, sizeof(struct ethhdr));
+ return skb;
+ }
+
+ linear_len = MIN_BYPASS_RECORD_SIZE;
+ }
+
+ linear_len += headln;
+ nskb = alloc_skb(linear_len, GFP_ATOMIC);
+ if (unlikely(!nskb)) {
+ dev_kfree_skb_any(skb);
+ return NULL;
+ }
+
+ skb_put(nskb, linear_len);
+ syndrome = SYNDROME_SYNC;
+ if (likely(info.sync_len >= 0)) {
+ int i;
+
+ for (i = 0; i < info.nr_frags; i++)
+ skb_shinfo(nskb)->frags[i] = info.frags[i];
+
+ skb_shinfo(nskb)->nr_frags = info.nr_frags;
+ nskb->data_len = info.sync_len;
+ nskb->len += info.sync_len;
+ } else {
+ create_bypass_record(nskb->data + headln, -info.sync_len);
+ tcp_seq -= MIN_BYPASS_RECORD_SIZE;
+ syndrome = SYNDROME_BYPASS;
+ }
+
+ return complete_sync_skb(skb, nskb, tcp_seq, headln, syndrome);
+}
+
+static int insert_pet(struct sk_buff *skb)
+{
+ struct ethhdr *eth;
+ struct pet *pet;
+ struct mlx_tls_offload_context *context;
+
+ pr_debug("insert_pet started\n");
+ if (skb_cow_head(skb, sizeof(struct pet)))
+ return -ENOMEM;
+
+ eth = (struct ethhdr *)skb_push(skb, sizeof(struct pet));
+ skb->mac_header -= sizeof(struct pet);
+ pet = (struct pet *)(eth + 1);
+
+ memmove(skb->data, skb->data + sizeof(struct pet), 2 * ETH_ALEN);
+
+ eth->h_proto = cpu_to_be16(MLX5_METADATA_ETHER_TYPE);
+ pet->syndrome = SYNDROME_OFFLOAD_REQUIRED;
+
+ memset(pet->content.raw, 0, sizeof(pet->content.raw));
+ context = get_tls_context(skb->sk);
+ memcpy(pet->content.send.sid, &context->swid,
+ sizeof(pet->content.send.sid));
+
+ return 0;
+}
+
+static struct sk_buff *mlx_tls_tx_handler(struct sk_buff *skb,
+ struct mlx5_swp_info *swp_info)
+{
+ struct mlx_tls_offload_context *context;
+ int datalen;
+ u32 skb_seq;
+
+ pr_debug("mlx_tls_tx_handler started\n");
+
+ if (!skb->sk || !tls_is_sk_tx_device_offloaded(skb->sk))
+ goto out;
+
+ datalen = skb->len - (skb_transport_offset(skb) + tcp_hdrlen(skb));
+ if (!datalen)
+ goto out;
+
+ skb_seq = ntohl(tcp_hdr(skb)->seq);
+
+ context = get_tls_context(skb->sk);
+ pr_debug("mlx_tls_tx_handler: mapping: %u cpu %u size %u with swid %u expectedSN: %u actualSN: %u\n",
+ skb->queue_mapping, smp_processor_id(), skb->len,
+ ntohl(context->swid), context->context.expectedSN, skb_seq);
+
+ insert_pet(skb);
+
+ if (unlikely(context->context.expectedSN != skb_seq)) {
+ skb = handle_ooo(context, skb);
+ if (!skb)
+ goto out;
+
+ pr_info("Sending sync packet\n");
+
+ if (!skb->next)
+ goto out;
+ }
+ context->context.expectedSN = skb_seq + datalen;
+
+out:
+ return skb;
+}
+
+static struct sk_buff *mlx_tls_rx_handler(struct sk_buff *skb, u8 *rawpet,
+ u8 petlen)
+{
+ struct pet *pet = (struct pet *)rawpet;
+
+ if (petlen != sizeof(*pet))
+ goto out;
+
+ dev_dbg(&skb->dev->dev, ">> rx_handler %u bytes\n", skb->len);
+ dev_dbg(&skb->dev->dev, " RX PET: size %lu, etherType %04X, syndrome %02x\n",
+ sizeof(*pet), be16_to_cpu(pet->ethertype), pet->syndrome);
+
+ if (pet->syndrome != 48) {
+ dev_dbg(&skb->dev->dev, "unexpected pet syndrome %d\n",
+ pet->syndrome);
+ goto out;
+ }
+
+out:
+ return skb;
+}
+
+/* Must hold mlx_tls_mutex to call this function.
+ * Assumes that dev->core_ctx is destroyed be the caller
+ */
+static void mlx_tls_free(struct mlx_tls_dev *dev)
+{
+ list_del(&dev->accel_dev_list);
+#ifdef MLX_TLS_SADB_RDMA
+ kobject_put(&dev->kobj);
+#endif
+ dev_put(dev->netdev);
+ kfree(dev);
+}
+
+int mlx_tls_netdev_event(struct notifier_block *this, unsigned long event,
+ void *ptr)
+{
+ struct net_device *netdev = netdev_notifier_info_to_dev(ptr);
+ struct mlx_tls_dev *accel_dev = NULL;
+
+ if (!netdev)
+ goto out;
+
+ pr_debug("mlx_tls_netdev_event: %lu\n", event);
+
+ /* We are interested only in net devices going down */
+ if (event != NETDEV_UNREGISTER)
+ goto out;
+
+ /* Take down all connections using a netdev that is going down */
+ mutex_lock(&mlx_tls_mutex);
+ accel_dev = find_mlx_tls_dev_by_netdev(netdev);
+ if (!accel_dev) {
+ pr_debug("mlx_tls_netdev_event: Failed to find tls device for net device\n");
+ goto unlock;
+ }
+ mlx_tls_free(accel_dev);
+
+unlock:
+ mutex_unlock(&mlx_tls_mutex);
+out:
+ return NOTIFY_DONE;
+}
+
+static struct mlx5_accel_ops mlx_tls_client_ops = {
+ .rx_handler = mlx_tls_rx_handler,
+ .tx_handler = mlx_tls_tx_handler,
+ .feature_chk = mlx_tls_feature_chk,
+ .get_count = mlx_tls_get_count,
+ .get_strings = mlx_tls_get_strings,
+ .get_stats = mlx_tls_get_stats,
+ .mtu_extra = sizeof(struct pet),
+ .features = 0,
+};
+
+int mlx_tls_add_one(struct mlx_accel_core_device *accel_device)
+{
+ int ret = 0;
+ struct mlx_tls_dev *dev = NULL;
+ struct net_device *netdev = NULL;
+#ifdef MLX_TLS_SADB_RDMA
+ struct mlx_accel_core_conn_init_attr init_attr = {0};
+#endif
+ pr_debug("mlx_tls_add_one called for %s\n", accel_device->name);
+
+ dev = kzalloc(sizeof(*dev), GFP_KERNEL);
+ if (!dev) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ INIT_LIST_HEAD(&dev->accel_dev_list);
+ dev->accel_device = accel_device;
+ ida_init(&dev->swid_ida);
+
+#ifdef MLX_TLS_SADB_RDMA
+ init_attr.rx_size = 128;
+ init_attr.tx_size = 32;
+ init_attr.recv_cb = mlx_tls_hw_qp_recv_cb;
+ init_attr.cb_arg = dev;
+ dev->conn = mlx_accel_core_conn_create(accel_device, &init_attr);
+ if (IS_ERR(dev->conn)) {
+ ret = PTR_ERR(dev->conn);
+ pr_err("mlx_tls_add_one(): Got error while creating connection %d\n",
+ ret);
+ goto err_dev;
+ }
+#endif
+ netdev = accel_device->ib_dev->get_netdev(accel_device->ib_dev,
+ accel_device->port);
+ if (!netdev) {
+ pr_err("mlx_tls_add_one(): Failed to retrieve net device from ib device\n");
+ ret = -EINVAL;
+ goto err_conn;
+ }
+ dev->netdev = netdev;
+
+ ret = mlx_accel_core_client_ops_register(accel_device,
+ &mlx_tls_client_ops);
+ if (ret) {
+ pr_err("mlx_tls_add_one(): Failed to register client ops %d\n",
+ ret);
+ goto err_netdev;
+ }
+
+#ifdef MLX_TLS_SADB_RDMA
+ ret = tls_sysfs_init_and_add(&dev->kobj,
+ mlx_accel_core_kobj(dev->accel_device),
+ "%s",
+ "accel_dev");
+ if (ret) {
+ pr_err("mlx_tls_add_one(): Got error from kobject_init_and_add %d\n",
+ ret);
+ goto err_ops_register;
+ }
+#endif
+
+ mutex_lock(&mlx_tls_mutex);
+ list_add(&dev->accel_dev_list, &mlx_tls_devs);
+ mutex_unlock(&mlx_tls_mutex);
+
+ dev->netdev->tlsdev_ops = &mlx_tls_ops;
+ goto out;
+
+#ifdef MLX_TLS_SADB_RDMA
+err_ops_register:
+ mlx_accel_core_client_ops_unregister(accel_device);
+#endif
+err_netdev:
+ dev_put(netdev);
+err_conn:
+ mlx_accel_core_conn_destroy(dev->conn);
+#ifdef MLX_TLS_SADB_RDMA
+err_dev:
+#endif
+ kfree(dev);
+out:
+ return ret;
+}
+
+void mlx_tls_remove_one(struct mlx_accel_core_device *accel_device)
+{
+ struct mlx_tls_dev *dev;
+ struct net_device *netdev = NULL;
+
+ pr_debug("mlx_tls_remove_one called for %s\n", accel_device->name);
+
+ mutex_lock(&mlx_tls_mutex);
+
+ list_for_each_entry(dev, &mlx_tls_devs, accel_dev_list) {
+ if (dev->accel_device == accel_device) {
+ netdev = dev->netdev;
+ netdev->tlsdev_ops = NULL;
+ mlx_accel_core_client_ops_unregister(accel_device);
+#ifdef MLX_TLS_SADB_RDMA
+ mlx_accel_core_conn_destroy(dev->conn);
+#endif
+ mlx_tls_free(dev);
+ break;
+ }
+ }
+ mutex_unlock(&mlx_tls_mutex);
+}
diff --git a/drivers/net/ethernet/mellanox/accelerator/tls/tls.h b/drivers/net/ethernet/mellanox/accelerator/tls/tls.h
new file mode 100644
index 0000000..7c7539a
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/accelerator/tls/tls.h
@@ -0,0 +1,100 @@
+/*
+ * Copyright (c) 2015-2017 Mellanox Technologies. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+#ifndef __TLS_H__
+#define __TLS_H__
+
+#include <linux/types.h>
+#include <linux/kobject.h>
+#include <linux/kfifo.h>
+#include <linux/list.h>
+#include <linux/skbuff.h>
+#include <linux/hashtable.h>
+#include <net/sock.h>
+#include <net/inet_common.h>
+#include <net/tls.h>
+#include <linux/mlx5/accel_sdk.h>
+
+#include "tls_cmds.h"
+
+#define DRIVER_NAME "mlx_tls"
+#define DRIVER_VERSION "0.1"
+#define DRIVER_RELDATE "January 2016"
+
+#define MLX_TLS_DEVICE_NAME "mlx_tls"
+/* TODO: Consider moving this to include/uapi/linux/if_ether.h */
+
+struct send_pet_content {
+ /* The next field is meaningful only for sync packets with LSO
+ * enabled (by the syndrome field))
+ */
+ __be16 first_seq; /* LSBs of the first TCP seq in the packet */
+ unsigned char sid[3];
+} __packed;
+
+/*TODO: move this to HW/cmds header files when added*/
+struct pet {
+ unsigned char syndrome;
+ union {
+ unsigned char raw[5];
+ /* from host to FPGA */
+ struct send_pet_content send;
+ } __packed content;
+ /* packet type ID field */
+ __be16 ethertype;
+} __packed;
+
+struct mlx_tls_dev {
+ struct kobject kobj;
+ struct list_head accel_dev_list;
+ struct mlx_accel_core_device *accel_device;
+ struct mlx_accel_core_conn *conn;
+ struct net_device *netdev;
+ struct ida swid_ida;
+};
+
+struct mlx_tls_offload_context {
+ struct tls_offload_context context;
+ struct list_head tls_del_list;
+ struct net_device *netdev;
+ __be32 swid;
+};
+
+int mlx_tls_netdev_event(struct notifier_block *this,
+ unsigned long event, void *ptr);
+
+int mlx_tls_add_one(struct mlx_accel_core_device *accel_device);
+void mlx_tls_remove_one(struct mlx_accel_core_device *accel_device);
+
+struct mlx_tls_dev *mlx_tls_find_dev_by_netdev(struct net_device *netdev);
+
+#endif /* __TLS_H__ */
--
2.7.4
^ permalink raw reply related
* [RFC TLS Offload Support 12/15] mlx/tls: Enable MLX5_CORE_QP_SIM mode for tls
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
In-Reply-To: <1490707592-1430-1-git-send-email-aviadye@mellanox.com>
Signed-off-by: Aviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: Ilya Lesokhin <ilyal@mellanox.com>
---
drivers/net/ethernet/mellanox/accelerator/tls/tls.c | 6 ++++++
drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.c | 2 ++
drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.h | 2 ++
3 files changed, 10 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/accelerator/tls/tls.c b/drivers/net/ethernet/mellanox/accelerator/tls/tls.c
index 07a4b67..3560f784 100644
--- a/drivers/net/ethernet/mellanox/accelerator/tls/tls.c
+++ b/drivers/net/ethernet/mellanox/accelerator/tls/tls.c
@@ -494,9 +494,11 @@ static struct sk_buff *mlx_tls_rx_handler(struct sk_buff *skb, u8 *rawpet,
static void mlx_tls_free(struct mlx_tls_dev *dev)
{
list_del(&dev->accel_dev_list);
+#if IS_ENABLED(CONFIG_MLX5_CORE_FPGA_QP_SIM)
#ifdef MLX_TLS_SADB_RDMA
kobject_put(&dev->kobj);
#endif
+#endif
dev_put(dev->netdev);
kfree(dev);
}
@@ -592,6 +594,7 @@ int mlx_tls_add_one(struct mlx_accel_core_device *accel_device)
goto err_netdev;
}
+#if IS_ENABLED(CONFIG_MLX5_CORE_FPGA_QP_SIM)
#ifdef MLX_TLS_SADB_RDMA
ret = tls_sysfs_init_and_add(&dev->kobj,
mlx_accel_core_kobj(dev->accel_device),
@@ -603,6 +606,7 @@ int mlx_tls_add_one(struct mlx_accel_core_device *accel_device)
goto err_ops_register;
}
#endif
+#endif
mutex_lock(&mlx_tls_mutex);
list_add(&dev->accel_dev_list, &mlx_tls_devs);
@@ -611,10 +615,12 @@ int mlx_tls_add_one(struct mlx_accel_core_device *accel_device)
dev->netdev->tlsdev_ops = &mlx_tls_ops;
goto out;
+#if IS_ENABLED(CONFIG_MLX5_CORE_FPGA_QP_SIM)
#ifdef MLX_TLS_SADB_RDMA
err_ops_register:
mlx_accel_core_client_ops_unregister(accel_device);
#endif
+#endif
err_netdev:
dev_put(netdev);
err_conn:
diff --git a/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.c b/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.c
index 2860fc3..76ba784 100644
--- a/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.c
+++ b/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.c
@@ -36,6 +36,7 @@
#include "tls_sysfs.h"
#include "tls_cmds.h"
+#if IS_ENABLED(CONFIG_MLX5_CORE_FPGA_QP_SIM)
#ifdef MLX_TLS_SADB_RDMA
struct mlx_tls_attribute {
struct attribute attr;
@@ -192,3 +193,4 @@ int tls_sysfs_init_and_add(struct kobject *kobj, struct kobject *parent,
fmt, arg);
}
#endif
+#endif
diff --git a/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.h b/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.h
index bfaa857..d7c3185 100644
--- a/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.h
+++ b/drivers/net/ethernet/mellanox/accelerator/tls/tls_sysfs.h
@@ -37,9 +37,11 @@
#include "tls.h"
+#if IS_ENABLED(CONFIG_MLX5_CORE_FPGA_QP_SIM)
#ifdef MLX_TLS_SADB_RDMA
int tls_sysfs_init_and_add(struct kobject *kobj, struct kobject *parent,
const char *fmt, char *arg);
#endif
+#endif
#endif /* __TLS_SYSFS_H__ */
--
2.7.4
^ permalink raw reply related
* [RFC TLS Offload Support 13/15] crypto: Add gcm template for rfc5288
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
In-Reply-To: <1490707592-1430-1-git-send-email-aviadye@mellanox.com>
From: Dave Watson <davejwatson@fb.com>
AAD data length is 13 bytes, tag is 16.
Signed-off-by: Dave Watson <davejwatson@fb.com>
---
crypto/gcm.c | 122 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
crypto/tcrypt.c | 14 ++++---
crypto/testmgr.c | 16 ++++++++
crypto/testmgr.h | 47 +++++++++++++++++++++
4 files changed, 194 insertions(+), 5 deletions(-)
diff --git a/crypto/gcm.c b/crypto/gcm.c
index f624ac9..07c2805 100644
--- a/crypto/gcm.c
+++ b/crypto/gcm.c
@@ -1016,6 +1016,120 @@ static struct crypto_template crypto_rfc4106_tmpl = {
.module = THIS_MODULE,
};
+static int crypto_rfc5288_encrypt(struct aead_request *req)
+{
+ if (req->assoclen != 21)
+ return -EINVAL;
+
+ req = crypto_rfc4106_crypt(req);
+
+ return crypto_aead_encrypt(req);
+}
+
+static int crypto_rfc5288_decrypt(struct aead_request *req)
+{
+ if (req->assoclen != 21)
+ return -EINVAL;
+
+ req = crypto_rfc4106_crypt(req);
+
+ return crypto_aead_decrypt(req);
+}
+
+static int crypto_rfc5288_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ struct crypto_attr_type *algt;
+ struct aead_instance *inst;
+ struct crypto_aead_spawn *spawn;
+ struct aead_alg *alg;
+ const char *ccm_name;
+ int err;
+
+ algt = crypto_get_attr_type(tb);
+ if (IS_ERR(algt))
+ return PTR_ERR(algt);
+
+ if ((algt->type ^ CRYPTO_ALG_TYPE_AEAD) & algt->mask)
+ return -EINVAL;
+
+ ccm_name = crypto_attr_alg_name(tb[1]);
+ if (IS_ERR(ccm_name))
+ return PTR_ERR(ccm_name);
+
+ inst = kzalloc(sizeof(*inst) + sizeof(*spawn), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ spawn = aead_instance_ctx(inst);
+ crypto_set_aead_spawn(spawn, aead_crypto_instance(inst));
+ err = crypto_grab_aead(spawn, ccm_name, 0,
+ crypto_requires_sync(algt->type, algt->mask));
+ if (err)
+ goto out_free_inst;
+
+ alg = crypto_spawn_aead_alg(spawn);
+
+ err = -EINVAL;
+
+ /* Underlying IV size must be 12. */
+ if (crypto_aead_alg_ivsize(alg) != 12)
+ goto out_drop_alg;
+
+ /* Not a stream cipher? */
+ if (alg->base.cra_blocksize != 1)
+ goto out_drop_alg;
+
+ err = -ENAMETOOLONG;
+ if (snprintf(inst->alg.base.cra_name, CRYPTO_MAX_ALG_NAME,
+ "rfc5288(%s)", alg->base.cra_name) >=
+ CRYPTO_MAX_ALG_NAME ||
+ snprintf(inst->alg.base.cra_driver_name, CRYPTO_MAX_ALG_NAME,
+ "rfc5288(%s)", alg->base.cra_driver_name) >=
+ CRYPTO_MAX_ALG_NAME)
+ goto out_drop_alg;
+
+ inst->alg.base.cra_flags = alg->base.cra_flags & CRYPTO_ALG_ASYNC;
+ inst->alg.base.cra_priority = alg->base.cra_priority;
+ inst->alg.base.cra_blocksize = 1;
+ inst->alg.base.cra_alignmask = alg->base.cra_alignmask;
+
+ inst->alg.base.cra_ctxsize = sizeof(struct crypto_rfc4106_ctx);
+
+ inst->alg.ivsize = 8;
+ inst->alg.chunksize = crypto_aead_alg_chunksize(alg);
+ inst->alg.maxauthsize = crypto_aead_alg_maxauthsize(alg);
+
+ inst->alg.init = crypto_rfc4106_init_tfm;
+ inst->alg.exit = crypto_rfc4106_exit_tfm;
+
+ inst->alg.setkey = crypto_rfc4106_setkey;
+ inst->alg.setauthsize = crypto_rfc4106_setauthsize;
+ inst->alg.encrypt = crypto_rfc5288_encrypt;
+ inst->alg.decrypt = crypto_rfc5288_decrypt;
+
+ inst->free = crypto_rfc4106_free;
+
+ err = aead_register_instance(tmpl, inst);
+ if (err)
+ goto out_drop_alg;
+
+out:
+ return err;
+
+out_drop_alg:
+ crypto_drop_aead(spawn);
+out_free_inst:
+ kfree(inst);
+ goto out;
+}
+
+static struct crypto_template crypto_rfc5288_tmpl = {
+ .name = "rfc5288",
+ .create = crypto_rfc5288_create,
+ .module = THIS_MODULE,
+};
+
static int crypto_rfc4543_setkey(struct crypto_aead *parent, const u8 *key,
unsigned int keylen)
{
@@ -1284,8 +1398,14 @@ static int __init crypto_gcm_module_init(void)
if (err)
goto out_undo_rfc4106;
+ err = crypto_register_template(&crypto_rfc5288_tmpl);
+ if (err)
+ goto out_undo_rfc4543;
+
return 0;
+out_undo_rfc4543:
+ crypto_unregister_template(&crypto_rfc4543_tmpl);
out_undo_rfc4106:
crypto_unregister_template(&crypto_rfc4106_tmpl);
out_undo_gcm:
@@ -1302,6 +1422,7 @@ static void __exit crypto_gcm_module_exit(void)
kfree(gcm_zeroes);
crypto_unregister_template(&crypto_rfc4543_tmpl);
crypto_unregister_template(&crypto_rfc4106_tmpl);
+ crypto_unregister_template(&crypto_rfc5288_tmpl);
crypto_unregister_template(&crypto_gcm_tmpl);
crypto_unregister_template(&crypto_gcm_base_tmpl);
}
@@ -1315,4 +1436,5 @@ MODULE_AUTHOR("Mikko Herranen <mh1@iki.fi>");
MODULE_ALIAS_CRYPTO("gcm_base");
MODULE_ALIAS_CRYPTO("rfc4106");
MODULE_ALIAS_CRYPTO("rfc4543");
+MODULE_ALIAS_CRYPTO("rfc5288");
MODULE_ALIAS_CRYPTO("gcm");
diff --git a/crypto/tcrypt.c b/crypto/tcrypt.c
index ae22f05..22538a7 100644
--- a/crypto/tcrypt.c
+++ b/crypto/tcrypt.c
@@ -1338,26 +1338,30 @@ static int do_test(const char *alg, u32 type, u32 mask, int m)
break;
case 152:
- ret += tcrypt_test("rfc4543(gcm(aes))");
+ ret += tcrypt_test("rfc5288(gcm(aes))");
break;
case 153:
- ret += tcrypt_test("cmac(aes)");
+ ret += tcrypt_test("rfc4543(gcm(aes))");
break;
case 154:
- ret += tcrypt_test("cmac(des3_ede)");
+ ret += tcrypt_test("cmac(aes)");
break;
case 155:
- ret += tcrypt_test("authenc(hmac(sha1),cbc(aes))");
+ ret += tcrypt_test("cmac(des3_ede)");
break;
case 156:
- ret += tcrypt_test("authenc(hmac(md5),ecb(cipher_null))");
+ ret += tcrypt_test("authenc(hmac(sha1),cbc(aes))");
break;
case 157:
+ ret += tcrypt_test("authenc(hmac(md5),ecb(cipher_null))");
+ break;
+
+ case 158:
ret += tcrypt_test("authenc(hmac(sha1),ecb(cipher_null))");
break;
case 181:
diff --git a/crypto/testmgr.c b/crypto/testmgr.c
index 62dffa0..4cae414 100644
--- a/crypto/testmgr.c
+++ b/crypto/testmgr.c
@@ -3748,6 +3748,22 @@ static const struct alg_test_desc alg_test_descs[] = {
}
}
}, {
+ .alg = "rfc5288(gcm(aes))",
+ .test = alg_test_aead,
+ .fips_allowed = 1,
+ .suite = {
+ .aead = {
+ .enc = {
+ .vecs = aes_gcm_rfc5288_enc_tv_template,
+ .count = AES_GCM_5288_ENC_TEST_VECTORS
+ },
+ .dec = {
+ .vecs = aes_gcm_rfc5288_dec_tv_template,
+ .count = AES_GCM_5288_DEC_TEST_VECTORS
+ }
+ }
+ }
+ }, {
.alg = "rfc7539(chacha20,poly1305)",
.test = alg_test_aead,
.suite = {
diff --git a/crypto/testmgr.h b/crypto/testmgr.h
index e64a4ef..65d725a 100644
--- a/crypto/testmgr.h
+++ b/crypto/testmgr.h
@@ -15193,6 +15193,8 @@ static struct cipher_testvec cast6_xts_dec_tv_template[] = {
#define AES_GCM_DEC_TEST_VECTORS 8
#define AES_GCM_4106_ENC_TEST_VECTORS 23
#define AES_GCM_4106_DEC_TEST_VECTORS 23
+#define AES_GCM_5288_ENC_TEST_VECTORS 1
+#define AES_GCM_5288_DEC_TEST_VECTORS 1
#define AES_GCM_4543_ENC_TEST_VECTORS 1
#define AES_GCM_4543_DEC_TEST_VECTORS 2
#define AES_CCM_ENC_TEST_VECTORS 8
@@ -21932,6 +21934,7 @@ static struct aead_testvec aes_gcm_rfc4106_dec_tv_template[] = {
.assoc = "\x01\x01\x01\x01\x01\x01\x01\x01"
"\x00\x00\x00\x00\x00\x00\x00\x00",
.alen = 16,
+
.result = "\x01\x01\x01\x01\x01\x01\x01\x01"
"\x01\x01\x01\x01\x01\x01\x01\x01",
.rlen = 16,
@@ -22485,6 +22488,50 @@ static struct aead_testvec aes_gcm_rfc4106_dec_tv_template[] = {
}
};
+static struct aead_testvec aes_gcm_rfc5288_enc_tv_template[] = {
+ {
+ .key = "\x34\x19\x96\x6e\xc5\x8c\x17\x9c"
+ "\x56\x78\x5e\xbb\x30\x52\x21\x89"
+ "\xea\xbc\x6e\x50",
+ .klen = 20,
+ .iv = "\x00\x00\x00\x00\x00\x00\x00\x01"
+ "\x5f\x73\x65\x73",
+ .assoc = "\x00\x00\x00\x00\x00\x00\x00\x01"
+ "\x17\x03\x03\x00\x10\x00\x00\x00"
+ "\x00\x00\x00\x00\x00",
+ .alen = 21,
+ .input = zeroed_string,
+ .ilen = 16,
+ .result = "\xa5\x2b\x6c\x6e\x2d\x78\x6f\x80"
+ "\x0e\x65\x69\x70\x0a\xe8\x86\xed"
+ "\x6d\x38\x29\x1d\x35\x3f\x62\xcf"
+ "\x46\x9c\x19\x78\x00\x0d\x67\xaa",
+ .rlen = 32,
+ }
+};
+
+static struct aead_testvec aes_gcm_rfc5288_dec_tv_template[] = {
+ {
+ .key = "\x73\xf0\xfa\x44\x76\xf5\xd5\x17"
+ "\x00\x12\x42\x85\xcb\x4f\x92\x1f"
+ "\x7d\x63\x9f\xc6",
+ .klen = 20,
+ .iv = "\x00\x00\x00\x00\x00\x00\x00\x01"
+ "\x74\x61\x73\x6b",
+ .assoc = "\x00\x00\x00\x00\x00\x00\x00\x01"
+ "\x17\x03\x03\x00\x10\x00\x00\x00"
+ "\x00\x00\x00\x00\x00",
+ .alen = 21,
+ .input = "\x05\x56\x46\x23\x1c\x86\x5e\xd0"
+ "\x12\x37\x2a\xa3\x65\x8b\x8c\x90"
+ "\xab\xbd\xca\xda\xae\x6e\xc0\xb2"
+ "\x91\x1b\x9b\x34\xe3\xea\x86\x8f",
+ .ilen = 32,
+ .result = zeroed_string,
+ .rlen = 16,
+ },
+};
+
static struct aead_testvec aes_gcm_rfc4543_enc_tv_template[] = {
{ /* From draft-mcgrew-gcm-test-01 */
.key = "\x4c\x80\xcd\xef\xbb\x5d\x10\xda"
--
2.7.4
^ permalink raw reply related
* [RFC TLS Offload Support 14/15] crypto: rfc5288 aesni optimized intel routines
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
In-Reply-To: <1490707592-1430-1-git-send-email-aviadye@mellanox.com>
From: Dave Watson <davejwatson@fb.com>
The assembly routines require the AAD data to be padded out
to the nearest 4 bytes.
Copy the 13 byte tag to a spare assoc data area when necessary
Signed-off-by: Dave Watson <davejwatson@fb.com>
---
arch/x86/crypto/aesni-intel_asm.S | 6 ++
arch/x86/crypto/aesni-intel_avx-x86_64.S | 4 ++
arch/x86/crypto/aesni-intel_glue.c | 105 ++++++++++++++++++++++++++-----
3 files changed, 99 insertions(+), 16 deletions(-)
diff --git a/arch/x86/crypto/aesni-intel_asm.S b/arch/x86/crypto/aesni-intel_asm.S
index 383a6f8..4e80bb8 100644
--- a/arch/x86/crypto/aesni-intel_asm.S
+++ b/arch/x86/crypto/aesni-intel_asm.S
@@ -229,6 +229,9 @@ XMM2 XMM3 XMM4 XMMDst TMP6 TMP7 i i_seq operation
MOVADQ SHUF_MASK(%rip), %xmm14
mov arg7, %r10 # %r10 = AAD
mov arg8, %r12 # %r12 = aadLen
+ add $3, %r12
+ and $~3, %r12
+
mov %r12, %r11
pxor %xmm\i, %xmm\i
@@ -454,6 +457,9 @@ XMM2 XMM3 XMM4 XMMDst TMP6 TMP7 i i_seq operation
MOVADQ SHUF_MASK(%rip), %xmm14
mov arg7, %r10 # %r10 = AAD
mov arg8, %r12 # %r12 = aadLen
+ add $3, %r12
+ and $~3, %r12
+
mov %r12, %r11
pxor %xmm\i, %xmm\i
_get_AAD_loop\num_initial_blocks\operation:
diff --git a/arch/x86/crypto/aesni-intel_avx-x86_64.S b/arch/x86/crypto/aesni-intel_avx-x86_64.S
index 522ab68..0756e4a 100644
--- a/arch/x86/crypto/aesni-intel_avx-x86_64.S
+++ b/arch/x86/crypto/aesni-intel_avx-x86_64.S
@@ -360,6 +360,8 @@ VARIABLE_OFFSET = 16*8
mov arg6, %r10 # r10 = AAD
mov arg7, %r12 # r12 = aadLen
+ add $3, %r12
+ and $~3, %r12
mov %r12, %r11
@@ -1619,6 +1621,8 @@ ENDPROC(aesni_gcm_dec_avx_gen2)
mov arg6, %r10 # r10 = AAD
mov arg7, %r12 # r12 = aadLen
+ add $3, %r12
+ and $~3, %r12
mov %r12, %r11
diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index 4ff90a7..dcada94 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -957,6 +957,8 @@ static int helper_rfc4106_encrypt(struct aead_request *req)
{
u8 one_entry_in_sg = 0;
u8 *src, *dst, *assoc;
+ u8 *assocmem = NULL;
+
__be32 counter = cpu_to_be32(1);
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(tfm);
@@ -966,12 +968,8 @@ static int helper_rfc4106_encrypt(struct aead_request *req)
struct scatter_walk src_sg_walk;
struct scatter_walk dst_sg_walk = {};
unsigned int i;
-
- /* Assuming we are supporting rfc4106 64-bit extended */
- /* sequence numbers We need to have the AAD length equal */
- /* to 16 or 20 bytes */
- if (unlikely(req->assoclen != 16 && req->assoclen != 20))
- return -EINVAL;
+ unsigned int padded_assoclen = (req->assoclen + 3) & ~3;
+ u8 assocbuf[24];
/* IV below built */
for (i = 0; i < 4; i++)
@@ -996,7 +994,8 @@ static int helper_rfc4106_encrypt(struct aead_request *req)
} else {
/* Allocate memory for src, dst, assoc */
assoc = kmalloc(req->cryptlen + auth_tag_len + req->assoclen,
- GFP_ATOMIC);
+ GFP_ATOMIC);
+ assocmem = assoc;
if (unlikely(!assoc))
return -ENOMEM;
scatterwalk_map_and_copy(assoc, req->src, 0,
@@ -1005,6 +1004,14 @@ static int helper_rfc4106_encrypt(struct aead_request *req)
dst = src;
}
+ if (req->assoclen != padded_assoclen) {
+ scatterwalk_map_and_copy(assocbuf, req->src, 0,
+ req->assoclen, 0);
+ memset(assocbuf + req->assoclen, 0,
+ padded_assoclen - req->assoclen);
+ assoc = assocbuf;
+ }
+
kernel_fpu_begin();
aesni_gcm_enc_tfm(aes_ctx, dst, src, req->cryptlen, iv,
ctx->hash_subkey, assoc, req->assoclen - 8,
@@ -1025,7 +1032,7 @@ static int helper_rfc4106_encrypt(struct aead_request *req)
} else {
scatterwalk_map_and_copy(dst, req->dst, req->assoclen,
req->cryptlen + auth_tag_len, 1);
- kfree(assoc);
+ kfree(assocmem);
}
return 0;
}
@@ -1034,6 +1041,7 @@ static int helper_rfc4106_decrypt(struct aead_request *req)
{
u8 one_entry_in_sg = 0;
u8 *src, *dst, *assoc;
+ u8 *assocmem = NULL;
unsigned long tempCipherLen = 0;
__be32 counter = cpu_to_be32(1);
int retval = 0;
@@ -1043,16 +1051,11 @@ static int helper_rfc4106_decrypt(struct aead_request *req)
unsigned long auth_tag_len = crypto_aead_authsize(tfm);
u8 iv[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
u8 authTag[16];
+ u8 assocbuf[24];
struct scatter_walk src_sg_walk;
struct scatter_walk dst_sg_walk = {};
unsigned int i;
-
- if (unlikely(req->assoclen != 16 && req->assoclen != 20))
- return -EINVAL;
-
- /* Assuming we are supporting rfc4106 64-bit extended */
- /* sequence numbers We need to have the AAD length */
- /* equal to 16 or 20 bytes */
+ unsigned int padded_assoclen = (req->assoclen + 3) & ~3;
tempCipherLen = (unsigned long)(req->cryptlen - auth_tag_len);
/* IV below built */
@@ -1079,6 +1082,7 @@ static int helper_rfc4106_decrypt(struct aead_request *req)
} else {
/* Allocate memory for src, dst, assoc */
assoc = kmalloc(req->cryptlen + req->assoclen, GFP_ATOMIC);
+ assocmem = assoc;
if (!assoc)
return -ENOMEM;
scatterwalk_map_and_copy(assoc, req->src, 0,
@@ -1087,6 +1091,14 @@ static int helper_rfc4106_decrypt(struct aead_request *req)
dst = src;
}
+ if (req->assoclen != padded_assoclen) {
+ scatterwalk_map_and_copy(assocbuf, req->src,
+ 0, req->assoclen, 0);
+ memset(assocbuf + req->assoclen, 0,
+ padded_assoclen - req->assoclen);
+ assoc = assocbuf;
+ }
+
kernel_fpu_begin();
aesni_gcm_dec_tfm(aes_ctx, dst, src, tempCipherLen, iv,
ctx->hash_subkey, assoc, req->assoclen - 8,
@@ -1109,7 +1121,7 @@ static int helper_rfc4106_decrypt(struct aead_request *req)
} else {
scatterwalk_map_and_copy(dst, req->dst, req->assoclen,
tempCipherLen, 1);
- kfree(assoc);
+ kfree(assocmem);
}
return retval;
}
@@ -1151,6 +1163,12 @@ static int rfc4106_encrypt(struct aead_request *req)
aead_request_set_tfm(req, tfm);
+ /* Assuming we are supporting rfc4106 64-bit extended */
+ /* sequence numbers We need to have the AAD length */
+ /* equal to 16 or 20 bytes */
+ if (unlikely(req->assoclen != 16 && req->assoclen != 20))
+ return -EINVAL;
+
return crypto_aead_encrypt(req);
}
@@ -1167,6 +1185,43 @@ static int rfc4106_decrypt(struct aead_request *req)
aead_request_set_tfm(req, tfm);
+ /* Assuming we are supporting rfc4106 64-bit extended */
+ /* sequence numbers We need to have the AAD length */
+ /* equal to 16 or 20 bytes */
+ if (unlikely(req->assoclen != 16 && req->assoclen != 20))
+ return -EINVAL;
+
+ return crypto_aead_decrypt(req);
+}
+
+static int rfc5288_encrypt(struct aead_request *req)
+{
+ struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+ struct cryptd_aead **ctx = crypto_aead_ctx(tfm);
+ struct cryptd_aead *cryptd_tfm = *ctx;
+
+ if (unlikely(req->assoclen != 21))
+ return -EINVAL;
+
+ aead_request_set_tfm(req, irq_fpu_usable() ?
+ cryptd_aead_child(cryptd_tfm) :
+ &cryptd_tfm->base);
+
+ return crypto_aead_encrypt(req);
+}
+
+static int rfc5288_decrypt(struct aead_request *req)
+{
+ struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+ struct cryptd_aead **ctx = crypto_aead_ctx(tfm);
+ struct cryptd_aead *cryptd_tfm = *ctx;
+
+ if (unlikely(req->assoclen != 21))
+ return -EINVAL;
+
+ aead_request_set_tfm(req, irq_fpu_usable() ?
+ cryptd_aead_child(cryptd_tfm) :
+ &cryptd_tfm->base);
return crypto_aead_decrypt(req);
}
#endif
@@ -1506,6 +1561,24 @@ static struct aead_alg aesni_aead_algs[] = { {
.cra_blocksize = 1,
.cra_ctxsize = sizeof(struct aesni_rfc4106_gcm_sync_ctx),
.cra_module = THIS_MODULE,
+ },
+}, {
+ .init = rfc4106_init,
+ .exit = rfc4106_exit,
+ .setkey = rfc4106_set_key,
+ .setauthsize = rfc4106_set_authsize,
+ .encrypt = rfc5288_encrypt,
+ .decrypt = rfc5288_decrypt,
+ .ivsize = 8,
+ .maxauthsize = 16,
+ .base = {
+ .cra_name = "rfc5288(gcm(aes))",
+ .cra_driver_name = "rfc5288-gcm-aesni",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_blocksize = 1,
+ .cra_ctxsize = sizeof(struct cryptd_aead *),
+ .cra_module = THIS_MODULE,
},
} };
#else
--
2.7.4
^ permalink raw reply related
* [RFC TLS Offload Support 01/15] tcp: Add clean acked data hook
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
In-Reply-To: <1490707592-1430-1-git-send-email-aviadye@mellanox.com>
From: Ilya Lesokhin <ilyal@mellanox.com>
Called when a TCP segment is acknowledged.
Could be used by application protocols who hold additional
metadata associated with the stream data
This is required for TLS offloads to release metadata
for acknowledged TLS records.
Signed-off-by: Boris Pismenny <borisp@mellanox.com>
Signed-off-by: Ilya Lesokhin <ilyal@mellanox.com>
Signed-off-by: Aviad Yehezkel <aviadye@mellanox.com>
---
include/net/inet_connection_sock.h | 2 ++
net/ipv4/tcp_input.c | 3 +++
2 files changed, 5 insertions(+)
diff --git a/include/net/inet_connection_sock.h b/include/net/inet_connection_sock.h
index 146054c..0b0aceb 100644
--- a/include/net/inet_connection_sock.h
+++ b/include/net/inet_connection_sock.h
@@ -77,6 +77,7 @@ struct inet_connection_sock_af_ops {
* @icsk_pmtu_cookie Last pmtu seen by socket
* @icsk_ca_ops Pluggable congestion control hook
* @icsk_af_ops Operations which are AF_INET{4,6} specific
+ * @icsk_clean_acked Clean acked data hook
* @icsk_ca_state: Congestion control state
* @icsk_retransmits: Number of unrecovered [RTO] timeouts
* @icsk_pending: Scheduled timer event
@@ -99,6 +100,7 @@ struct inet_connection_sock {
__u32 icsk_pmtu_cookie;
const struct tcp_congestion_ops *icsk_ca_ops;
const struct inet_connection_sock_af_ops *icsk_af_ops;
+ void (*icsk_clean_acked)(struct sock *sk);
unsigned int (*icsk_sync_mss)(struct sock *sk, u32 pmtu);
__u8 icsk_ca_state:6,
icsk_ca_setsockopt:1,
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index fe668c1..c158bec 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -3667,6 +3667,9 @@ static int tcp_ack(struct sock *sk, const struct sk_buff *skb, int flag)
if (!prior_packets)
goto no_queue;
+ if (icsk->icsk_clean_acked)
+ icsk->icsk_clean_acked(sk);
+
/* See if we can take anything off of the retransmit queue. */
flag |= tcp_clean_rtx_queue(sk, prior_fackets, prior_snd_una, &acked,
&sack_state, &now);
--
2.7.4
^ permalink raw reply related
* [RFC TLS Offload Support 03/15] tcp: export tcp_rate_check_app_limited function
From: Aviad Yehezkel @ 2017-03-28 13:26 UTC (permalink / raw)
To: davem, aviadye, ilyal, borisp, davejwatson, netdev
Cc: matanb, liranl, haggaie, tom, herbert, nmav, fridolin.pokorny,
ilant, kliteyn, linux-crypto, saeedm, aviadye
In-Reply-To: <1490707592-1430-1-git-send-email-aviadye@mellanox.com>
We will use it via tls new code.
Signed-off-by: Aviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: Ilya Lesokhin <ilyal@mellanox.com>
Signed-off-by: Boris Pismenny <borisp@mellanox.com>
---
net/ipv4/tcp_rate.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/net/ipv4/tcp_rate.c b/net/ipv4/tcp_rate.c
index 9be1581..a226f76 100644
--- a/net/ipv4/tcp_rate.c
+++ b/net/ipv4/tcp_rate.c
@@ -184,3 +184,4 @@ void tcp_rate_check_app_limited(struct sock *sk)
tp->app_limited =
(tp->delivered + tcp_packets_in_flight(tp)) ? : 1;
}
+EXPORT_SYMBOL(tcp_rate_check_app_limited);
--
2.7.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox