* [RFC PATCH v2 13/32] KVM: SVM: Enable SEV by setting the SEV_ENABLE CPU feature
From: Brijesh Singh @ 2017-03-02 15:15 UTC (permalink / raw)
To: simon.guinot, linux-efi, brijesh.singh, kvm, rkrcmar, matt,
linux-pci, linus.walleij, gary.hook, linux-mm, paul.gortmaker,
hpa, cl, dan.j.williams, aarcange, sfr, andriy.shevchenko,
herbert, bhe, xemul, joro, x86, peterz, piotr.luc, mingo, msalter,
ross.zwisler, bp, dyoung, thomas.lendacky, jroedel, keescook,
arnd, <toshi
In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine>
From: Tom Lendacky <thomas.lendacky@amd.com>
Modify the SVM cpuid update function to indicate if Secure Encrypted
Virtualization (SEV) is active in the guest by setting the SEV KVM CPU
features bit. SEV is active if Secure Memory Encryption is enabled in
the host and the SEV_ENABLE bit of the VMCB is set.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/kvm/cpuid.c | 4 +++-
arch/x86/kvm/svm.c | 18 ++++++++++++++++++
2 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 1639de8..e0c40a8 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -601,7 +601,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
entry->edx = 0;
break;
case 0x80000000:
- entry->eax = min(entry->eax, 0x8000001a);
+ entry->eax = min(entry->eax, 0x8000001f);
break;
case 0x80000001:
entry->edx &= kvm_cpuid_8000_0001_edx_x86_features;
@@ -634,6 +634,8 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
break;
case 0x8000001d:
break;
+ case 0x8000001f:
+ break;
/*Add support for Centaur's CPUID instruction*/
case 0xC0000000:
/*Just support up to 0xC0000004 now*/
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 75b0645..36d61ff 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -46,6 +46,7 @@
#include <asm/irq_remapping.h>
#include <asm/virtext.h>
+#include <asm/mem_encrypt.h>
#include "trace.h"
#define __ex(x) __kvm_handle_fault_on_reboot(x)
@@ -5005,10 +5006,27 @@ static void svm_cpuid_update(struct kvm_vcpu *vcpu)
{
struct vcpu_svm *svm = to_svm(vcpu);
struct kvm_cpuid_entry2 *entry;
+ struct vmcb_control_area *ca = &svm->vmcb->control;
+ struct kvm_cpuid_entry2 *features, *sev_info;
/* Update nrips enabled cache */
svm->nrips_enabled = !!guest_cpuid_has_nrips(&svm->vcpu);
+ /* Check for Secure Encrypted Virtualization support */
+ features = kvm_find_cpuid_entry(vcpu, KVM_CPUID_FEATURES, 0);
+ if (!features)
+ return;
+
+ sev_info = kvm_find_cpuid_entry(vcpu, 0x8000001f, 0);
+ if (!sev_info)
+ return;
+
+ if (ca->nested_ctl & SVM_NESTED_CTL_SEV_ENABLE) {
+ features->eax |= (1 << KVM_FEATURE_SEV);
+ cpuid(0x8000001f, &sev_info->eax, &sev_info->ebx,
+ &sev_info->ecx, &sev_info->edx);
+ }
+
if (!kvm_vcpu_apicv_active(vcpu))
return;
^ permalink raw reply related
* [RFC PATCH v2 12/32] x86: Add early boot support when running with SEV active
From: Brijesh Singh @ 2017-03-02 15:14 UTC (permalink / raw)
To: simon.guinot, linux-efi, brijesh.singh, kvm, rkrcmar, matt,
linux-pci, linus.walleij, gary.hook, linux-mm, paul.gortmaker,
hpa, cl, dan.j.williams, aarcange, sfr, andriy.shevchenko,
herbert, bhe, xemul, joro, x86, peterz, piotr.luc, mingo, msalter,
ross.zwisler, bp, dyoung, thomas.lendacky, jroedel, keescook,
arnd, <toshi
In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine>
From: Tom Lendacky <thomas.lendacky@amd.com>
Early in the boot process, add checks to determine if the kernel is
running with Secure Encrypted Virtualization (SEV) active by issuing
a CPUID instruction.
During early compressed kernel booting, if SEV is active the pagetables are
updated so that data is accessed and decompressed with encryption.
During uncompressed kernel booting, if SEV is the memory encryption mask is
set and a flag is set to indicate that SEV is enabled.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/boot/compressed/Makefile | 2 +
arch/x86/boot/compressed/head_64.S | 16 +++++++
arch/x86/boot/compressed/mem_encrypt.S | 75 ++++++++++++++++++++++++++++++++
arch/x86/include/uapi/asm/hyperv.h | 4 ++
arch/x86/include/uapi/asm/kvm_para.h | 3 +
arch/x86/kernel/mem_encrypt_init.c | 24 ++++++++++
6 files changed, 124 insertions(+)
create mode 100644 arch/x86/boot/compressed/mem_encrypt.S
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index 44163e8..51f9cd0 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -72,6 +72,8 @@ vmlinux-objs-y := $(obj)/vmlinux.lds $(obj)/head_$(BITS).o $(obj)/misc.o \
$(obj)/string.o $(obj)/cmdline.o $(obj)/error.o \
$(obj)/piggy.o $(obj)/cpuflags.o
+vmlinux-objs-$(CONFIG_X86_64) += $(obj)/mem_encrypt.o
+
vmlinux-objs-$(CONFIG_EARLY_PRINTK) += $(obj)/early_serial_console.o
vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/kaslr.o
ifdef CONFIG_X86_64
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index d2ae1f8..625b5380 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -130,6 +130,19 @@ ENTRY(startup_32)
/*
* Build early 4G boot pagetable
*/
+ /*
+ * If SEV is active set the encryption mask in the page tables. This
+ * will insure that when the kernel is copied and decompressed it
+ * will be done so encrypted.
+ */
+ call sev_enabled
+ xorl %edx, %edx
+ testl %eax, %eax
+ jz 1f
+ subl $32, %eax /* Encryption bit is always above bit 31 */
+ bts %eax, %edx /* Set encryption mask for page tables */
+1:
+
/* Initialize Page tables to 0 */
leal pgtable(%ebx), %edi
xorl %eax, %eax
@@ -140,12 +153,14 @@ ENTRY(startup_32)
leal pgtable + 0(%ebx), %edi
leal 0x1007 (%edi), %eax
movl %eax, 0(%edi)
+ addl %edx, 4(%edi)
/* Build Level 3 */
leal pgtable + 0x1000(%ebx), %edi
leal 0x1007(%edi), %eax
movl $4, %ecx
1: movl %eax, 0x00(%edi)
+ addl %edx, 0x04(%edi)
addl $0x00001000, %eax
addl $8, %edi
decl %ecx
@@ -156,6 +171,7 @@ ENTRY(startup_32)
movl $0x00000183, %eax
movl $2048, %ecx
1: movl %eax, 0(%edi)
+ addl %edx, 4(%edi)
addl $0x00200000, %eax
addl $8, %edi
decl %ecx
diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S
new file mode 100644
index 0000000..8313c31
--- /dev/null
+++ b/arch/x86/boot/compressed/mem_encrypt.S
@@ -0,0 +1,75 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky@amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+
+#include <asm/processor-flags.h>
+#include <asm/msr.h>
+#include <asm/asm-offsets.h>
+#include <uapi/asm/kvm_para.h>
+
+ .text
+ .code32
+ENTRY(sev_enabled)
+ xor %eax, %eax
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+ push %ebx
+ push %ecx
+ push %edx
+
+ /* Check if running under a hypervisor */
+ movl $0x40000000, %eax
+ cpuid
+ cmpl $0x40000001, %eax
+ jb .Lno_sev
+
+ movl $0x40000001, %eax
+ cpuid
+ bt $KVM_FEATURE_SEV, %eax
+ jnc .Lno_sev
+
+ /*
+ * Check for memory encryption feature:
+ * CPUID Fn8000_001F[EAX] - Bit 0
+ */
+ movl $0x8000001f, %eax
+ cpuid
+ bt $0, %eax
+ jnc .Lno_sev
+
+ /*
+ * Get memory encryption information:
+ * CPUID Fn8000_001F[EBX] - Bits 5:0
+ * Pagetable bit position used to indicate encryption
+ */
+ movl %ebx, %eax
+ andl $0x3f, %eax
+ movl %eax, sev_enc_bit(%ebp)
+ jmp .Lsev_exit
+
+.Lno_sev:
+ xor %eax, %eax
+
+.Lsev_exit:
+ pop %edx
+ pop %ecx
+ pop %ebx
+
+#endif /* CONFIG_AMD_MEM_ENCRYPT */
+
+ ret
+ENDPROC(sev_enabled)
+
+ .bss
+sev_enc_bit:
+ .word 0
diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
index 9b1a918..8278161 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -3,6 +3,8 @@
#include <linux/types.h>
+#ifndef __ASSEMBLY__
+
/*
* The below CPUID leaves are present if VersionAndFeatures.HypervisorPresent
* is set by CPUID(HvCpuIdFunctionVersionAndFeatures).
@@ -363,4 +365,6 @@ struct hv_timer_message_payload {
#define HV_STIMER_AUTOENABLE (1ULL << 3)
#define HV_STIMER_SINT(config) (__u8)(((config) >> 16) & 0x0F)
+#endif /* __ASSEMBLY__ */
+
#endif
diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h
index bc2802f..e81b74a 100644
--- a/arch/x86/include/uapi/asm/kvm_para.h
+++ b/arch/x86/include/uapi/asm/kvm_para.h
@@ -26,6 +26,8 @@
#define KVM_FEATURE_PV_UNHALT 7
#define KVM_FEATURE_SEV 8
+#ifndef __ASSEMBLY__
+
/* The last 8 bits are used to indicate how to interpret the flags field
* in pvclock structure. If no bits are set, all flags are ignored.
*/
@@ -100,5 +102,6 @@ struct kvm_vcpu_pv_apf_data {
#define KVM_PV_EOI_ENABLED KVM_PV_EOI_MASK
#define KVM_PV_EOI_DISABLED 0x0
+#endif /* __ASSEMBLY__ */
#endif /* _UAPI_ASM_X86_KVM_PARA_H */
diff --git a/arch/x86/kernel/mem_encrypt_init.c b/arch/x86/kernel/mem_encrypt_init.c
index 35c5e3d..5d514e6 100644
--- a/arch/x86/kernel/mem_encrypt_init.c
+++ b/arch/x86/kernel/mem_encrypt_init.c
@@ -22,6 +22,7 @@
#include <asm/processor-flags.h>
#include <asm/msr.h>
#include <asm/cmdline.h>
+#include <asm/kvm_para.h>
static char sme_cmdline_arg_on[] __initdata = "mem_encrypt=on";
static char sme_cmdline_arg_off[] __initdata = "mem_encrypt=off";
@@ -232,6 +233,29 @@ unsigned long __init sme_enable(void *boot_data)
void *cmdline_arg;
u64 msr;
+ /* Check if running under a hypervisor */
+ eax = 0x40000000;
+ ecx = 0;
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+ if (eax > 0x40000000) {
+ eax = 0x40000001;
+ ecx = 0;
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+ if (!(eax & BIT(KVM_FEATURE_SEV)))
+ goto out;
+
+ eax = 0x8000001f;
+ ecx = 0;
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+ if (!(eax & 1))
+ goto out;
+
+ sme_me_mask = 1UL << (ebx & 0x3f);
+ sev_enabled = 1;
+
+ goto out;
+ }
+
/* Check for an AMD processor */
eax = 0;
ecx = 0;
^ permalink raw reply related
* [RFC PATCH v2 14/32] x86: mm: Provide support to use memblock when spliting large pages
From: Brijesh Singh @ 2017-03-02 15:15 UTC (permalink / raw)
To: simon.guinot, linux-efi, brijesh.singh, kvm, rkrcmar, matt,
linux-pci, linus.walleij, gary.hook, linux-mm, paul.gortmaker,
hpa, cl, dan.j.williams, aarcange, sfr, andriy.shevchenko,
herbert, bhe, xemul, joro, x86, peterz, piotr.luc, mingo, msalter,
ross.zwisler, bp, dyoung, thomas.lendacky, jroedel, keescook,
arnd, <toshi
In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine>
If kernel_maps_pages_in_pgd is called early in boot process to change the
memory attributes then it fails to allocate memory when spliting large
pages. The patch extends the cpa_data to provide the support to use
memblock_alloc when slab allocator is not available.
The feature will be used in Secure Encrypted Virtualization (SEV) mode,
where we may need to change the memory region attributes in early boot
process.
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
arch/x86/mm/pageattr.c | 51 ++++++++++++++++++++++++++++++++++++++++--------
1 file changed, 42 insertions(+), 9 deletions(-)
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index 46cc89d..9e4ab3b 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -14,6 +14,7 @@
#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/vmalloc.h>
+#include <linux/memblock.h>
#include <asm/e820/api.h>
#include <asm/processor.h>
@@ -37,6 +38,7 @@ struct cpa_data {
int flags;
unsigned long pfn;
unsigned force_split : 1;
+ unsigned force_memblock :1;
int curpage;
struct page **pages;
};
@@ -627,9 +629,8 @@ try_preserve_large_page(pte_t *kpte, unsigned long address,
static int
__split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
- struct page *base)
+ pte_t *pbase, unsigned long new_pfn)
{
- pte_t *pbase = (pte_t *)page_address(base);
unsigned long ref_pfn, pfn, pfninc = 1;
unsigned int i, level;
pte_t *tmp;
@@ -646,7 +647,7 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
return 1;
}
- paravirt_alloc_pte(&init_mm, page_to_pfn(base));
+ paravirt_alloc_pte(&init_mm, new_pfn);
switch (level) {
case PG_LEVEL_2M:
@@ -707,7 +708,8 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
* pagetable protections, the actual ptes set above control the
* primary protection behavior:
*/
- __set_pmd_pte(kpte, address, mk_pte(base, __pgprot(_KERNPG_TABLE)));
+ __set_pmd_pte(kpte, address,
+ native_make_pte((new_pfn << PAGE_SHIFT) + _KERNPG_TABLE));
/*
* Intel Atom errata AAH41 workaround.
@@ -723,21 +725,50 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
return 0;
}
+static pte_t *try_alloc_pte(struct cpa_data *cpa, unsigned long *pfn)
+{
+ unsigned long phys;
+ struct page *base;
+
+ if (cpa->force_memblock) {
+ phys = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+ if (!phys)
+ return NULL;
+ *pfn = phys >> PAGE_SHIFT;
+ return (pte_t *)__va(phys);
+ }
+
+ base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0);
+ if (!base)
+ return NULL;
+ *pfn = page_to_pfn(base);
+ return (pte_t *)page_address(base);
+}
+
+static void try_free_pte(struct cpa_data *cpa, pte_t *pte)
+{
+ if (cpa->force_memblock)
+ memblock_free(__pa(pte), PAGE_SIZE);
+ else
+ __free_page((struct page *)pte);
+}
+
static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
unsigned long address)
{
- struct page *base;
+ pte_t *new_pte;
+ unsigned long new_pfn;
if (!debug_pagealloc_enabled())
spin_unlock(&cpa_lock);
- base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0);
+ new_pte = try_alloc_pte(cpa, &new_pfn);
if (!debug_pagealloc_enabled())
spin_lock(&cpa_lock);
- if (!base)
+ if (!new_pte)
return -ENOMEM;
- if (__split_large_page(cpa, kpte, address, base))
- __free_page(base);
+ if (__split_large_page(cpa, kpte, address, new_pte, new_pfn))
+ try_free_pte(cpa, new_pte);
return 0;
}
@@ -2035,6 +2066,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address,
unsigned numpages, unsigned long page_flags)
{
int retval = -EINVAL;
+ int use_memblock = !slab_is_available();
struct cpa_data cpa = {
.vaddr = &address,
@@ -2044,6 +2076,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address,
.mask_set = __pgprot(0),
.mask_clr = __pgprot(0),
.flags = 0,
+ .force_memblock = use_memblock,
};
if (!(__supported_pte_mask & _PAGE_NX))
^ permalink raw reply related
* [RFC PATCH v2 10/32] x86: DMA support for SEV memory encryption
From: Brijesh Singh @ 2017-03-02 15:14 UTC (permalink / raw)
To: simon.guinot, linux-efi, brijesh.singh, kvm, rkrcmar, matt,
linux-pci, linus.walleij, gary.hook, linux-mm, paul.gortmaker,
hpa, cl, dan.j.williams, aarcange, sfr, andriy.shevchenko,
herbert, bhe, xemul, joro, x86, peterz, piotr.luc, mingo, msalter,
ross.zwisler, bp, dyoung, thomas.lendacky, jroedel, keescook,
arnd, <toshi
In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine>
From: Tom Lendacky <thomas.lendacky@amd.com>
DMA access to memory mapped as encrypted while SEV is active can not be
encrypted during device write or decrypted during device read. In order
for DMA to properly work when SEV is active, the swiotlb bounce buffers
must be used.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/mm/mem_encrypt.c | 77 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 77 insertions(+)
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 090419b..7df5f4c 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -197,8 +197,81 @@ void __init sme_early_init(void)
/* Update the protection map with memory encryption mask */
for (i = 0; i < ARRAY_SIZE(protection_map); i++)
protection_map[i] = pgprot_encrypted(protection_map[i]);
+
+ if (sev_active())
+ swiotlb_force = SWIOTLB_FORCE;
+}
+
+static void *sme_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
+ gfp_t gfp, unsigned long attrs)
+{
+ unsigned long dma_mask;
+ unsigned int order;
+ struct page *page;
+ void *vaddr = NULL;
+
+ dma_mask = dma_alloc_coherent_mask(dev, gfp);
+ order = get_order(size);
+
+ gfp &= ~__GFP_ZERO;
+
+ page = alloc_pages_node(dev_to_node(dev), gfp, order);
+ if (page) {
+ dma_addr_t addr;
+
+ /*
+ * Since we will be clearing the encryption bit, check the
+ * mask with it already cleared.
+ */
+ addr = phys_to_dma(dev, page_to_phys(page)) & ~sme_me_mask;
+ if ((addr + size) > dma_mask) {
+ __free_pages(page, get_order(size));
+ } else {
+ vaddr = page_address(page);
+ *dma_handle = addr;
+ }
+ }
+
+ if (!vaddr)
+ vaddr = swiotlb_alloc_coherent(dev, size, dma_handle, gfp);
+
+ if (!vaddr)
+ return NULL;
+
+ /* Clear the SME encryption bit for DMA use if not swiotlb area */
+ if (!is_swiotlb_buffer(dma_to_phys(dev, *dma_handle))) {
+ set_memory_decrypted((unsigned long)vaddr, 1 << order);
+ *dma_handle &= ~sme_me_mask;
+ }
+
+ return vaddr;
}
+static void sme_free(struct device *dev, size_t size, void *vaddr,
+ dma_addr_t dma_handle, unsigned long attrs)
+{
+ /* Set the SME encryption bit for re-use if not swiotlb area */
+ if (!is_swiotlb_buffer(dma_to_phys(dev, dma_handle)))
+ set_memory_encrypted((unsigned long)vaddr,
+ 1 << get_order(size));
+
+ swiotlb_free_coherent(dev, size, vaddr, dma_handle);
+}
+
+static struct dma_map_ops sme_dma_ops = {
+ .alloc = sme_alloc,
+ .free = sme_free,
+ .map_page = swiotlb_map_page,
+ .unmap_page = swiotlb_unmap_page,
+ .map_sg = swiotlb_map_sg_attrs,
+ .unmap_sg = swiotlb_unmap_sg_attrs,
+ .sync_single_for_cpu = swiotlb_sync_single_for_cpu,
+ .sync_single_for_device = swiotlb_sync_single_for_device,
+ .sync_sg_for_cpu = swiotlb_sync_sg_for_cpu,
+ .sync_sg_for_device = swiotlb_sync_sg_for_device,
+ .mapping_error = swiotlb_dma_mapping_error,
+};
+
/* Architecture __weak replacement functions */
void __init mem_encrypt_init(void)
{
@@ -208,6 +281,10 @@ void __init mem_encrypt_init(void)
/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
swiotlb_update_mem_attributes();
+ /* Use SEV DMA operations if SEV is active */
+ if (sev_active())
+ dma_ops = &sme_dma_ops;
+
pr_info("AMD Secure Memory Encryption (SME) active\n");
}
^ permalink raw reply related
* Re: [RFC PATCH v2 19/32] crypto: ccp: Introduce the AMD Secure Processor device
From: Mark Rutland @ 2017-03-02 17:39 UTC (permalink / raw)
To: Brijesh Singh
Cc: simon.guinot, linux-efi, kvm, rkrcmar, matt, linux-pci,
linus.walleij, gary.hook, linux-mm, paul.gortmaker, hpa, cl,
dan.j.williams, aarcange, sfr, andriy.shevchenko, herbert, bhe,
xemul, joro, x86, peterz, piotr.luc, mingo, msalter, ross.zwisler,
bp, dyoung, thomas.lendacky, jroedel, keescook, arnd, toshi.kani,
mathieu.desnoyers, luto, devel
In-Reply-To: <148846777589.2349.11698765767451886038.stgit@brijesh-build-machine>
On Thu, Mar 02, 2017 at 10:16:15AM -0500, Brijesh Singh wrote:
> The CCP device is part of the AMD Secure Processor. In order to expand the
> usage of the AMD Secure Processor, create a framework that allows functional
> components of the AMD Secure Processor to be initialized and handled
> appropriately.
>
> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
> ---
> drivers/crypto/Kconfig | 10 +
> drivers/crypto/ccp/Kconfig | 43 +++--
> drivers/crypto/ccp/Makefile | 8 -
> drivers/crypto/ccp/ccp-dev-v3.c | 86 +++++-----
> drivers/crypto/ccp/ccp-dev-v5.c | 73 ++++-----
> drivers/crypto/ccp/ccp-dev.c | 137 +++++++++-------
> drivers/crypto/ccp/ccp-dev.h | 35 ----
> drivers/crypto/ccp/sp-dev.c | 308 ++++++++++++++++++++++++++++++++++++
> drivers/crypto/ccp/sp-dev.h | 140 ++++++++++++++++
> drivers/crypto/ccp/sp-pci.c | 324 ++++++++++++++++++++++++++++++++++++++
> drivers/crypto/ccp/sp-platform.c | 268 +++++++++++++++++++++++++++++++
> include/linux/ccp.h | 3
> 12 files changed, 1240 insertions(+), 195 deletions(-)
> create mode 100644 drivers/crypto/ccp/sp-dev.c
> create mode 100644 drivers/crypto/ccp/sp-dev.h
> create mode 100644 drivers/crypto/ccp/sp-pci.c
> create mode 100644 drivers/crypto/ccp/sp-platform.c
> diff --git a/drivers/crypto/ccp/Makefile b/drivers/crypto/ccp/Makefile
> index 346ceb8..8127e18 100644
> --- a/drivers/crypto/ccp/Makefile
> +++ b/drivers/crypto/ccp/Makefile
> @@ -1,11 +1,11 @@
> -obj-$(CONFIG_CRYPTO_DEV_CCP_DD) += ccp.o
> -ccp-objs := ccp-dev.o \
> +obj-$(CONFIG_CRYPTO_DEV_SP_DD) += ccp.o
> +ccp-objs := sp-dev.o sp-platform.o
> +ccp-$(CONFIG_PCI) += sp-pci.o
> +ccp-$(CONFIG_CRYPTO_DEV_CCP) += ccp-dev.o \
> ccp-ops.o \
> ccp-dev-v3.o \
> ccp-dev-v5.o \
> - ccp-platform.o \
> ccp-dmaengine.o
It looks like ccp-platform.c has morphed into sp-platform.c (judging by
the compatible string and general shape of the code), and the original
ccp-platform.c is no longer built.
Shouldn't ccp-platform.c be deleted by this patch?
Thanks,
Mark.
^ permalink raw reply
* [RFC PATCH v2 15/32] x86: Add support for changing memory encryption attribute in early boot
From: Brijesh Singh @ 2017-03-02 15:15 UTC (permalink / raw)
To: simon.guinot, linux-efi, brijesh.singh, kvm, rkrcmar, matt,
linux-pci, linus.walleij, gary.hook, linux-mm, paul.gortmaker,
hpa, cl, dan.j.williams, aarcange, sfr, andriy.shevchenko,
herbert, bhe, xemul, joro, x86, peterz, piotr.luc, mingo, msalter,
ross.zwisler, bp, dyoung, thomas.lendacky, jroedel, keescook,
arnd, <toshi
In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine>
Some KVM-specific custom MSRs shares the guest physical address with
hypervisor. When SEV is active, the shared physical address must be mapped
with encryption attribute cleared so that both hypervsior and guest can
access the data.
Add APIs to change memory encryption attribute in early boot code.
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
arch/x86/include/asm/mem_encrypt.h | 15 +++++++++
arch/x86/mm/mem_encrypt.c | 63 ++++++++++++++++++++++++++++++++++++
2 files changed, 78 insertions(+)
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 9799835..95bbe4c 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -47,6 +47,9 @@ void __init sme_unmap_bootdata(char *real_mode_data);
void __init sme_early_init(void);
+int __init early_set_memory_decrypted(void *addr, unsigned long size);
+int __init early_set_memory_encrypted(void *addr, unsigned long size);
+
/* Architecture __weak replacement functions */
void __init mem_encrypt_init(void);
@@ -110,6 +113,18 @@ static inline void __init sme_early_init(void)
{
}
+static inline int __init early_set_memory_decrypted(void *addr,
+ unsigned long size)
+{
+ return 1;
+}
+
+static inline int __init early_set_memory_encrypted(void *addr,
+ unsigned long size)
+{
+ return 1;
+}
+
#define __sme_pa __pa
#define __sme_pa_nodebug __pa_nodebug
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 7df5f4c..567e0d8 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -15,6 +15,7 @@
#include <linux/mm.h>
#include <linux/dma-mapping.h>
#include <linux/swiotlb.h>
+#include <linux/mem_encrypt.h>
#include <asm/tlbflush.h>
#include <asm/fixmap.h>
@@ -258,6 +259,68 @@ static void sme_free(struct device *dev, size_t size, void *vaddr,
swiotlb_free_coherent(dev, size, vaddr, dma_handle);
}
+static unsigned long __init get_pte_flags(unsigned long address)
+{
+ int level;
+ pte_t *pte;
+ unsigned long flags = _KERNPG_TABLE_NOENC | _PAGE_ENC;
+
+ pte = lookup_address(address, &level);
+ if (!pte)
+ return flags;
+
+ switch (level) {
+ case PG_LEVEL_4K:
+ flags = pte_flags(*pte);
+ break;
+ case PG_LEVEL_2M:
+ flags = pmd_flags(*(pmd_t *)pte);
+ break;
+ case PG_LEVEL_1G:
+ flags = pud_flags(*(pud_t *)pte);
+ break;
+ default:
+ break;
+ }
+
+ return flags;
+}
+
+int __init early_set_memory_enc_dec(void *vaddr, unsigned long size,
+ unsigned long flags)
+{
+ unsigned long pfn, npages;
+ unsigned long addr = (unsigned long)vaddr & PAGE_MASK;
+
+ /* We are going to change the physical page attribute from C=1 to C=0.
+ * Flush the caches to ensure that all the data with C=1 is flushed to
+ * memory. Any caching of the vaddr after function returns will
+ * use C=0.
+ */
+ clflush_cache_range(vaddr, size);
+
+ npages = PAGE_ALIGN(size) >> PAGE_SHIFT;
+ pfn = slow_virt_to_phys((void *)addr) >> PAGE_SHIFT;
+
+ return kernel_map_pages_in_pgd(init_mm.pgd, pfn, addr, npages,
+ flags & ~sme_me_mask);
+
+}
+
+int __init early_set_memory_decrypted(void *vaddr, unsigned long size)
+{
+ unsigned long flags = get_pte_flags((unsigned long)vaddr);
+
+ return early_set_memory_enc_dec(vaddr, size, flags & ~sme_me_mask);
+}
+
+int __init early_set_memory_encrypted(void *vaddr, unsigned long size)
+{
+ unsigned long flags = get_pte_flags((unsigned long)vaddr);
+
+ return early_set_memory_enc_dec(vaddr, size, flags | _PAGE_ENC);
+}
+
static struct dma_map_ops sme_dma_ops = {
.alloc = sme_alloc,
.free = sme_free,
^ permalink raw reply related
* [RFC PATCH v2 00/32] x86: Secure Encrypted Virtualization (AMD)
From: Brijesh Singh @ 2017-03-02 15:12 UTC (permalink / raw)
To: simon.guinot, linux-efi, brijesh.singh, kvm, rkrcmar, matt,
linux-pci, linus.walleij, gary.hook, linux-mm, paul.gortmaker,
hpa, cl, dan.j.williams, aarcange, sfr, andriy.shevchenko,
herbert, bhe, xemul, joro, x86, peterz, piotr.luc, mingo, msalter,
ross.zwisler, bp, dyoung, thomas.lendacky, jroedel, keescook,
arnd, <toshi
This RFC series provides support for AMD's new Secure Encrypted Virtualization
(SEV) feature. This RFC is build upon Secure Memory Encryption (SME) RFCv4 [1].
SEV is an extension to the AMD-V architecture which supports running multiple
VMs under the control of a hypervisor. When enabled, SEV hardware tags all
code and data with its VM ASID which indicates which VM the data originated
from or is intended for. This tag is kept with the data at all times when
inside the SOC, and prevents that data from being used by anyone other than the
owner. While the tag protects VM data inside the SOC, AES with 128 bit
encryption protects data outside the SOC. When data leaves or enters the SOC,
it is encrypted/decrypted respectively by hardware with a key based on the
associated tag.
SEV guest VMs have the concept of private and shared memory. Private memory is
encrypted with the guest-specific key, while shared memory may be encrypted
with hypervisor key. Certain types of memory (namely instruction pages and
guest page tables) are always treated as private memory by the hardware.
For data memory, SEV guest VMs can choose which pages they would like to be
private. The choice is done using the standard CPU page tables using the C-bit,
and is fully controlled by the guest. Due to security reasons all the DMA
operations inside the guest must be performed on shared pages (C-bit clear).
Note that since C-bit is only controllable by the guest OS when it is operating
in 64-bit or 32-bit PAE mode, in all other modes the SEV hardware forces the
C-bit to a 1.
SEV is designed to protect guest VMs from a benign but vulnerable (i.e. not
fully malicious) hypervisor. In particular, it reduces the attack surface of
guest VMs and can prevent certain types of VM-escape bugs (e.g. hypervisor
read-anywhere) from being used to steal guest data.
The RFC series also expands crypto driver (ccp.ko) to include the support for
Platform Security Processor (PSP) which is used for communicating with SEV
firmware that runs within the AMD secure processor providing a secure key
management interfaces. The hypervisor uses this interface to encrypt the
bootstrap code and perform common activities such as launching, running,
snapshotting, migrating and debugging encrypted guest.
A new ioctl (KVM_MEMORY_ENCRYPT_OP) is introduced which can be used by Qemu to
issue SEV guest life cycle commands.
The RFC series also includes patches required in guest OS to enable SEV feature.
A guest OS can check SEV support by calling KVM_FEATURE cpuid instruction.
The patch breakdown:
* [1 - 17]: guest OS specific changes when SEV is active
* [18]: already queued in kvm upstream tree but was not in tip tree hence its
included so that build does not fail
* [19 - 21]: since CCP and PSP shares the same PCIe ID hence the patch expands
the CCP driver by creating a high level AMD Secure Processor (SP) framework
to allow integration of PSP device into ccp.ko.
* [22 - 32]: hypervisor changes to support memory encryption
The following links provide additional details:
AMD Memory Encryption whitepaper:
http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf
AMD64 Architecture Programmer's Manual:
http://support.amd.com/TechDocs/24593.pdf
SME is section 7.10
SEV is section 15.34
Secure Encrypted Virutualization Key Management:
http://support.amd.com/TechDocs/55766_SEV-KM API_Specification.pdf
KVM Forum Presentation:
http://www.linux-kvm.org/images/7/74/02x08A-Thomas_Lendacky-AMDs_Virtualizatoin_Memory_Encryption_Technology.pdf
[1] http://marc.info/?l=linux-kernel&m=148725974113693&w=2
---
Based on the feedbacks, we have started adding the SEV guest support in OVMF
BIOS. This series has been tested using EDK2/OVMF BIOS, the initial EDK2 patches
has been submmited on edk2 mailing list for discussion.
TODO:
- add support for migration commands
- update QEMU RFC's to SEV spec 0.14
- investigate virtio and vfio support for SEV guest
- investigate SMM support for SEV guest
- add support for nested virtualization
Changes since v1:
- update to newer SEV key management API spec (0.12 -> 0.14)
- expand the CCP driver and integrate the PSP interface support
- remove the usage of SEV ref_count and release the SEV FW resources in
kvm_x86_ops->vm_destroy
- acquire the kvm->lock before executing the SEV commands and release on exit.
- rename ioctl from KVM_SEV_ISSUE_CMD to KVM_MEMORY_ENCRYPT_OP
- extend KVM_MEMORY_ENCRYPT_OP ioctl to require file descriptor for the SEV
device. A program without access to /dev/sev will not be able to issue SEV
commands
- update vmcb on succesful LAUNCH_FINISH to indicate that SEV is active
- serveral fixes based on Paolo's review feedbacks
- add APIs to support sharing the guest physical address with hypervisor
- update kvm pvclock driver to use the shared buffer when SEV is active
- pin the SEV guest memory
Brijesh Singh (18):
x86: mm: Provide support to use memblock when spliting large pages
x86: Add support for changing memory encryption attribute in early boot
x86: kvm: Provide support to create Guest and HV shared per-CPU variables
x86: kvmclock: Clear encryption attribute when SEV is active
crypto: ccp: Introduce the AMD Secure Processor device
crypto: ccp: Add Platform Security Processor (PSP) interface support
crypto: ccp: Add Secure Encrypted Virtualization (SEV) interface support
kvm: svm: prepare to reserve asid for SEV guest
kvm: introduce KVM_MEMORY_ENCRYPT_OP ioctl
kvm: x86: prepare for SEV guest management API support
kvm: svm: Add support for SEV LAUNCH_START command
kvm: svm: Add support for SEV LAUNCH_UPDATE_DATA command
kvm: svm: Add support for SEV LAUNCH_FINISH command
kvm: svm: Add support for SEV GUEST_STATUS command
kvm: svm: Add support for SEV DEBUG_DECRYPT command
kvm: svm: Add support for SEV DEBUG_ENCRYPT command
kvm: svm: Add support for SEV LAUNCH_MEASURE command
x86: kvm: Pin the guest memory when SEV is active
Tom Lendacky (14):
x86: Add the Secure Encrypted Virtualization CPU feature
x86: Secure Encrypted Virtualization (SEV) support
KVM: SVM: prepare for new bit definition in nested_ctl
KVM: SVM: Add SEV feature definitions to KVM
x86: Use encrypted access of BOOT related data with SEV
x86/pci: Use memremap when walking setup data
x86/efi: Access EFI data as encrypted when SEV is active
x86: Use PAGE_KERNEL protection for ioremap of memory page
x86: Change early_ioremap to early_memremap for BOOT data
x86: DMA support for SEV memory encryption
x86: Unroll string I/O when SEV is active
x86: Add early boot support when running with SEV active
KVM: SVM: Enable SEV by setting the SEV_ENABLE CPU feature
kvm: svm: Use the hardware provided GPA instead of page walk
arch/x86/boot/compressed/Makefile | 2
arch/x86/boot/compressed/head_64.S | 16
arch/x86/boot/compressed/mem_encrypt.S | 75 ++
arch/x86/include/asm/cpufeatures.h | 1
arch/x86/include/asm/io.h | 26 +
arch/x86/include/asm/kvm_emulate.h | 1
arch/x86/include/asm/kvm_host.h | 19 +
arch/x86/include/asm/mem_encrypt.h | 29 +
arch/x86/include/asm/msr-index.h | 2
arch/x86/include/asm/svm.h | 3
arch/x86/include/uapi/asm/hyperv.h | 4
arch/x86/include/uapi/asm/kvm_para.h | 4
arch/x86/kernel/acpi/boot.c | 4
arch/x86/kernel/cpu/amd.c | 22 +
arch/x86/kernel/cpu/scattered.c | 1
arch/x86/kernel/kvm.c | 43 +
arch/x86/kernel/kvmclock.c | 65 ++
arch/x86/kernel/mem_encrypt_init.c | 24 +
arch/x86/kernel/mpparse.c | 10
arch/x86/kvm/cpuid.c | 4
arch/x86/kvm/emulate.c | 20 -
arch/x86/kvm/svm.c | 1051 ++++++++++++++++++++++++++++++++
arch/x86/kvm/x86.c | 60 ++
arch/x86/mm/ioremap.c | 44 +
arch/x86/mm/mem_encrypt.c | 143 ++++
arch/x86/mm/pageattr.c | 51 +-
arch/x86/pci/common.c | 4
arch/x86/platform/efi/efi_64.c | 15
drivers/crypto/Kconfig | 10
drivers/crypto/ccp/Kconfig | 55 +-
drivers/crypto/ccp/Makefile | 10
drivers/crypto/ccp/ccp-dev-v3.c | 86 +--
drivers/crypto/ccp/ccp-dev-v5.c | 73 +-
drivers/crypto/ccp/ccp-dev.c | 137 ++--
drivers/crypto/ccp/ccp-dev.h | 35 -
drivers/crypto/ccp/psp-dev.c | 211 ++++++
drivers/crypto/ccp/psp-dev.h | 102 +++
drivers/crypto/ccp/sev-dev.c | 348 +++++++++++
drivers/crypto/ccp/sev-dev.h | 67 ++
drivers/crypto/ccp/sev-ops.c | 324 ++++++++++
drivers/crypto/ccp/sp-dev.c | 324 ++++++++++
drivers/crypto/ccp/sp-dev.h | 172 +++++
drivers/crypto/ccp/sp-pci.c | 328 ++++++++++
drivers/crypto/ccp/sp-platform.c | 268 ++++++++
drivers/sfi/sfi_core.c | 6
include/asm-generic/vmlinux.lds.h | 3
include/linux/ccp.h | 3
include/linux/mem_encrypt.h | 6
include/linux/mm.h | 1
include/linux/percpu-defs.h | 9
include/linux/psp-sev.h | 672 ++++++++++++++++++++
include/uapi/linux/Kbuild | 1
include/uapi/linux/kvm.h | 100 +++
include/uapi/linux/psp-sev.h | 123 ++++
kernel/resource.c | 40 +
55 files changed, 4991 insertions(+), 266 deletions(-)
create mode 100644 arch/x86/boot/compressed/mem_encrypt.S
create mode 100644 drivers/crypto/ccp/psp-dev.c
create mode 100644 drivers/crypto/ccp/psp-dev.h
create mode 100644 drivers/crypto/ccp/sev-dev.c
create mode 100644 drivers/crypto/ccp/sev-dev.h
create mode 100644 drivers/crypto/ccp/sev-ops.c
create mode 100644 drivers/crypto/ccp/sp-dev.c
create mode 100644 drivers/crypto/ccp/sp-dev.h
create mode 100644 drivers/crypto/ccp/sp-pci.c
create mode 100644 drivers/crypto/ccp/sp-platform.c
create mode 100644 include/linux/psp-sev.h
create mode 100644 include/uapi/linux/psp-sev.h
^ permalink raw reply
* Re: [RFC PATCH v2 19/32] crypto: ccp: Introduce the AMD Secure Processor device
From: Brijesh Singh @ 2017-03-02 19:11 UTC (permalink / raw)
To: Mark Rutland
Cc: linux-efi, brijesh.singh, labbott, kvm, rkrcmar, matt, linux-pci,
linus.walleij, gary.hook, linux-mm, hpa, cl, tglx, aarcange, sfr,
mchehab, simon.guinot, bhe, xemul, joro, x86, peterz, piotr.luc,
mingo, msalter, ross.zwisler, bp, dyoung, thomas.lendacky,
jroedel, keescook, arnd, toshi.kani, mathieu.desnoyers, luto,
pbonzini, bhelgaas, dan.j.williams, andriy.shevchenko, akpm,
herbert
In-Reply-To: <20170302173936.GC11970@leverpostej>
Hi Mark,
On 03/02/2017 11:39 AM, Mark Rutland wrote:
> On Thu, Mar 02, 2017 at 10:16:15AM -0500, Brijesh Singh wrote:
>> The CCP device is part of the AMD Secure Processor. In order to expand the
>> usage of the AMD Secure Processor, create a framework that allows functional
>> components of the AMD Secure Processor to be initialized and handled
>> appropriately.
>>
>> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
>> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
>> ---
>> drivers/crypto/Kconfig | 10 +
>> drivers/crypto/ccp/Kconfig | 43 +++--
>> drivers/crypto/ccp/Makefile | 8 -
>> drivers/crypto/ccp/ccp-dev-v3.c | 86 +++++-----
>> drivers/crypto/ccp/ccp-dev-v5.c | 73 ++++-----
>> drivers/crypto/ccp/ccp-dev.c | 137 +++++++++-------
>> drivers/crypto/ccp/ccp-dev.h | 35 ----
>> drivers/crypto/ccp/sp-dev.c | 308 ++++++++++++++++++++++++++++++++++++
>> drivers/crypto/ccp/sp-dev.h | 140 ++++++++++++++++
>> drivers/crypto/ccp/sp-pci.c | 324 ++++++++++++++++++++++++++++++++++++++
>> drivers/crypto/ccp/sp-platform.c | 268 +++++++++++++++++++++++++++++++
>> include/linux/ccp.h | 3
>> 12 files changed, 1240 insertions(+), 195 deletions(-)
>> create mode 100644 drivers/crypto/ccp/sp-dev.c
>> create mode 100644 drivers/crypto/ccp/sp-dev.h
>> create mode 100644 drivers/crypto/ccp/sp-pci.c
>> create mode 100644 drivers/crypto/ccp/sp-platform.c
>
>> diff --git a/drivers/crypto/ccp/Makefile b/drivers/crypto/ccp/Makefile
>> index 346ceb8..8127e18 100644
>> --- a/drivers/crypto/ccp/Makefile
>> +++ b/drivers/crypto/ccp/Makefile
>> @@ -1,11 +1,11 @@
>> -obj-$(CONFIG_CRYPTO_DEV_CCP_DD) += ccp.o
>> -ccp-objs := ccp-dev.o \
>> +obj-$(CONFIG_CRYPTO_DEV_SP_DD) += ccp.o
>> +ccp-objs := sp-dev.o sp-platform.o
>> +ccp-$(CONFIG_PCI) += sp-pci.o
>> +ccp-$(CONFIG_CRYPTO_DEV_CCP) += ccp-dev.o \
>> ccp-ops.o \
>> ccp-dev-v3.o \
>> ccp-dev-v5.o \
>> - ccp-platform.o \
>> ccp-dmaengine.o
>
> It looks like ccp-platform.c has morphed into sp-platform.c (judging by
> the compatible string and general shape of the code), and the original
> ccp-platform.c is no longer built.
>
> Shouldn't ccp-platform.c be deleted by this patch?
>
Good catch. Both ccp-platform.c and ccp-pci.c should have been deleted
by this patch. I missed deleting it, will fix in next rev.
~ Brijesh
^ permalink raw reply
* [RFC PATCH v2 27/32] kvm: svm: Add support for SEV LAUNCH_FINISH command
From: Brijesh Singh @ 2017-03-02 15:17 UTC (permalink / raw)
To: simon.guinot, linux-efi, brijesh.singh, kvm, rkrcmar, matt,
linux-pci, linus.walleij, gary.hook, linux-mm, paul.gortmaker,
hpa, cl, dan.j.williams, aarcange, sfr, andriy.shevchenko,
herbert, bhe, xemul, joro, x86, peterz, piotr.luc, mingo, msalter,
ross.zwisler, bp, dyoung, thomas.lendacky, jroedel, keescook,
arnd, <toshi
In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine>
The command is used for finializing the SEV guest launch process.
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
arch/x86/kvm/svm.c | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 62c2b22..c108064 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -5921,6 +5921,38 @@ static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)
return ret;
}
+static int sev_launch_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+ int i, ret;
+ struct sev_data_launch_finish *data;
+ struct kvm_vcpu *vcpu;
+
+ if (!sev_guest(kvm))
+ return -EINVAL;
+
+ data = kzalloc(sizeof(*data), GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
+
+ /* launch finish */
+ data->handle = sev_get_handle(kvm);
+ ret = sev_issue_cmd(kvm, SEV_CMD_LAUNCH_FINISH, data, &argp->error);
+ if (ret)
+ goto err_1;
+
+ /* Iterate through each vcpus and set SEV KVM_SEV_FEATURE bit in
+ * KVM_CPUID_FEATURE to indicate that SEV is enabled on this vcpu
+ */
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ sev_init_vmcb(to_svm(vcpu));
+ svm_cpuid_update(vcpu);
+ }
+
+err_1:
+ kfree(data);
+ return ret;
+}
+
static int amd_memory_encryption_cmd(struct kvm *kvm, void __user *argp)
{
int r = -ENOTTY;
@@ -5940,6 +5972,10 @@ static int amd_memory_encryption_cmd(struct kvm *kvm, void __user *argp)
r = sev_launch_update_data(kvm, &sev_cmd);
break;
}
+ case KVM_SEV_LAUNCH_FINISH: {
+ r = sev_launch_finish(kvm, &sev_cmd);
+ break;
+ }
default:
break;
}
^ permalink raw reply related
* [RFC PATCH v2 17/32] x86: kvmclock: Clear encryption attribute when SEV is active
From: Brijesh Singh @ 2017-03-02 15:15 UTC (permalink / raw)
To: simon.guinot, linux-efi, brijesh.singh, kvm, rkrcmar, matt,
linux-pci, linus.walleij, gary.hook, linux-mm, paul.gortmaker,
hpa, cl, dan.j.williams, aarcange, sfr, andriy.shevchenko,
herbert, bhe, xemul, joro, x86, peterz, piotr.luc, mingo, msalter,
ross.zwisler, bp, dyoung, thomas.lendacky, jroedel, keescook,
arnd, <toshi
In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine>
The guest physical memory area holding the struct pvclock_wall_clock and
struct pvclock_vcpu_time_info are shared with the hypervisor. Hypervisor
periodically updates the contents of the memory. When SEV is active we must
clear the encryption attributes of the shared memory pages so that both
hypervisor and guest can access the data.
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
arch/x86/kernel/kvmclock.c | 65 ++++++++++++++++++++++++++++++++++++++------
1 file changed, 56 insertions(+), 9 deletions(-)
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 278de4f..3b38b3d 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -27,6 +27,7 @@
#include <linux/sched.h>
#include <linux/sched/clock.h>
+#include <asm/mem_encrypt.h>
#include <asm/x86_init.h>
#include <asm/reboot.h>
@@ -44,7 +45,7 @@ early_param("no-kvmclock", parse_no_kvmclock);
/* The hypervisor will put information about time periodically here */
static struct pvclock_vsyscall_time_info *hv_clock;
-static struct pvclock_wall_clock wall_clock;
+static struct pvclock_wall_clock *wall_clock;
struct pvclock_vsyscall_time_info *pvclock_pvti_cpu0_va(void)
{
@@ -62,15 +63,18 @@ static void kvm_get_wallclock(struct timespec *now)
int low, high;
int cpu;
- low = (int)__pa_symbol(&wall_clock);
- high = ((u64)__pa_symbol(&wall_clock) >> 32);
+ if (!wall_clock)
+ return;
+
+ low = (int)slow_virt_to_phys(wall_clock);
+ high = ((u64)slow_virt_to_phys(wall_clock) >> 32);
native_write_msr(msr_kvm_wall_clock, low, high);
cpu = get_cpu();
vcpu_time = &hv_clock[cpu].pvti;
- pvclock_read_wallclock(&wall_clock, vcpu_time, now);
+ pvclock_read_wallclock(wall_clock, vcpu_time, now);
put_cpu();
}
@@ -246,11 +250,40 @@ static void kvm_shutdown(void)
native_machine_shutdown();
}
+static phys_addr_t kvm_memblock_alloc(phys_addr_t size, phys_addr_t align)
+{
+ phys_addr_t mem;
+
+ mem = memblock_alloc(size, align);
+ if (!mem)
+ return 0;
+
+ /* When SEV is active clear the encryption attributes of the pages */
+ if (sev_active()) {
+ if (early_set_memory_decrypted(__va(mem), size))
+ goto e_free;
+ }
+
+ return mem;
+e_free:
+ memblock_free(mem, size);
+ return 0;
+}
+
+static void kvm_memblock_free(phys_addr_t addr, phys_addr_t size)
+{
+ /* When SEV is active restore the encryption attributes of the pages */
+ if (sev_active())
+ early_set_memory_encrypted(__va(addr), size);
+
+ memblock_free(addr, size);
+}
+
void __init kvmclock_init(void)
{
struct pvclock_vcpu_time_info *vcpu_time;
- unsigned long mem;
- int size, cpu;
+ unsigned long mem, mem_wall_clock;
+ int size, cpu, wall_clock_size;
u8 flags;
size = PAGE_ALIGN(sizeof(struct pvclock_vsyscall_time_info)*NR_CPUS);
@@ -267,15 +300,29 @@ void __init kvmclock_init(void)
printk(KERN_INFO "kvm-clock: Using msrs %x and %x",
msr_kvm_system_time, msr_kvm_wall_clock);
- mem = memblock_alloc(size, PAGE_SIZE);
- if (!mem)
+ wall_clock_size = PAGE_ALIGN(sizeof(struct pvclock_wall_clock));
+ mem_wall_clock = kvm_memblock_alloc(wall_clock_size, PAGE_SIZE);
+ if (!mem_wall_clock)
return;
+
+ wall_clock = __va(mem_wall_clock);
+ memset(wall_clock, 0, wall_clock_size);
+
+ mem = kvm_memblock_alloc(size, PAGE_SIZE);
+ if (!mem) {
+ kvm_memblock_free(mem_wall_clock, wall_clock_size);
+ wall_clock = NULL;
+ return;
+ }
+
hv_clock = __va(mem);
memset(hv_clock, 0, size);
if (kvm_register_clock("primary cpu clock")) {
hv_clock = NULL;
- memblock_free(mem, size);
+ kvm_memblock_free(mem, size);
+ kvm_memblock_free(mem_wall_clock, wall_clock_size);
+ wall_clock = NULL;
return;
}
^ permalink raw reply related
* [PATCH V2 1/3] crypto: ccp - Add SHA-2 384- and 512-bit support
From: Gary R Hook @ 2017-03-02 21:26 UTC (permalink / raw)
To: linux-crypto; +Cc: thomas.lendacky, gary.hook, herbert, davem
In-Reply-To: <20170302212331.30959.44149.stgit@taos>
Incorporate 384-bit and 512-bit hashing for a version 5 CCP
device
Signed-off-by: Gary R Hook <gary.hook@amd.com>
---
drivers/crypto/ccp/ccp-crypto-sha.c | 22 +++++++++++
drivers/crypto/ccp/ccp-crypto.h | 8 ++--
drivers/crypto/ccp/ccp-ops.c | 72 +++++++++++++++++++++++++++++++++++
include/linux/ccp.h | 2 +
4 files changed, 101 insertions(+), 3 deletions(-)
diff --git a/drivers/crypto/ccp/ccp-crypto-sha.c b/drivers/crypto/ccp/ccp-crypto-sha.c
index 84a652b..6b46eea 100644
--- a/drivers/crypto/ccp/ccp-crypto-sha.c
+++ b/drivers/crypto/ccp/ccp-crypto-sha.c
@@ -146,6 +146,12 @@ static int ccp_do_sha_update(struct ahash_request *req, unsigned int nbytes,
case CCP_SHA_TYPE_256:
rctx->cmd.u.sha.ctx_len = SHA256_DIGEST_SIZE;
break;
+ case CCP_SHA_TYPE_384:
+ rctx->cmd.u.sha.ctx_len = SHA384_DIGEST_SIZE;
+ break;
+ case CCP_SHA_TYPE_512:
+ rctx->cmd.u.sha.ctx_len = SHA512_DIGEST_SIZE;
+ break;
default:
/* Should never get here */
break;
@@ -393,6 +399,22 @@ struct ccp_sha_def {
.digest_size = SHA256_DIGEST_SIZE,
.block_size = SHA256_BLOCK_SIZE,
},
+ {
+ .version = CCP_VERSION(5, 0),
+ .name = "sha384",
+ .drv_name = "sha384-ccp",
+ .type = CCP_SHA_TYPE_384,
+ .digest_size = SHA384_DIGEST_SIZE,
+ .block_size = SHA384_BLOCK_SIZE,
+ },
+ {
+ .version = CCP_VERSION(5, 0),
+ .name = "sha512",
+ .drv_name = "sha512-ccp",
+ .type = CCP_SHA_TYPE_512,
+ .digest_size = SHA512_DIGEST_SIZE,
+ .block_size = SHA512_BLOCK_SIZE,
+ },
};
static int ccp_register_hmac_alg(struct list_head *head,
diff --git a/drivers/crypto/ccp/ccp-crypto.h b/drivers/crypto/ccp/ccp-crypto.h
index 8335b32..95cce27 100644
--- a/drivers/crypto/ccp/ccp-crypto.h
+++ b/drivers/crypto/ccp/ccp-crypto.h
@@ -137,9 +137,11 @@ struct ccp_aes_cmac_exp_ctx {
u8 buf[AES_BLOCK_SIZE];
};
-/***** SHA related defines *****/
-#define MAX_SHA_CONTEXT_SIZE SHA256_DIGEST_SIZE
-#define MAX_SHA_BLOCK_SIZE SHA256_BLOCK_SIZE
+/* SHA-related defines
+ * These values must be large enough to accommodate any variant
+ */
+#define MAX_SHA_CONTEXT_SIZE SHA512_DIGEST_SIZE
+#define MAX_SHA_BLOCK_SIZE SHA512_BLOCK_SIZE
struct ccp_sha_ctx {
struct scatterlist opad_sg;
diff --git a/drivers/crypto/ccp/ccp-ops.c b/drivers/crypto/ccp/ccp-ops.c
index efac3d5..213a752 100644
--- a/drivers/crypto/ccp/ccp-ops.c
+++ b/drivers/crypto/ccp/ccp-ops.c
@@ -41,6 +41,20 @@
cpu_to_be32(SHA256_H6), cpu_to_be32(SHA256_H7),
};
+static const __be64 ccp_sha384_init[SHA512_DIGEST_SIZE / sizeof(__be64)] = {
+ cpu_to_be64(SHA384_H0), cpu_to_be64(SHA384_H1),
+ cpu_to_be64(SHA384_H2), cpu_to_be64(SHA384_H3),
+ cpu_to_be64(SHA384_H4), cpu_to_be64(SHA384_H5),
+ cpu_to_be64(SHA384_H6), cpu_to_be64(SHA384_H7),
+};
+
+static const __be64 ccp_sha512_init[SHA512_DIGEST_SIZE / sizeof(__be64)] = {
+ cpu_to_be64(SHA512_H0), cpu_to_be64(SHA512_H1),
+ cpu_to_be64(SHA512_H2), cpu_to_be64(SHA512_H3),
+ cpu_to_be64(SHA512_H4), cpu_to_be64(SHA512_H5),
+ cpu_to_be64(SHA512_H6), cpu_to_be64(SHA512_H7),
+};
+
#define CCP_NEW_JOBID(ccp) ((ccp->vdata->version == CCP_VERSION(3, 0)) ? \
ccp_gen_jobid(ccp) : 0)
@@ -947,6 +961,18 @@ static int ccp_run_sha_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
return -EINVAL;
block_size = SHA256_BLOCK_SIZE;
break;
+ case CCP_SHA_TYPE_384:
+ if (cmd_q->ccp->vdata->version < CCP_VERSION(4, 0)
+ || sha->ctx_len < SHA384_DIGEST_SIZE)
+ return -EINVAL;
+ block_size = SHA384_BLOCK_SIZE;
+ break;
+ case CCP_SHA_TYPE_512:
+ if (cmd_q->ccp->vdata->version < CCP_VERSION(4, 0)
+ || sha->ctx_len < SHA512_DIGEST_SIZE)
+ return -EINVAL;
+ block_size = SHA512_BLOCK_SIZE;
+ break;
default:
return -EINVAL;
}
@@ -1034,6 +1060,21 @@ static int ccp_run_sha_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
sb_count = 1;
ooffset = ioffset = 0;
break;
+ case CCP_SHA_TYPE_384:
+ digest_size = SHA384_DIGEST_SIZE;
+ init = (void *) ccp_sha384_init;
+ ctx_size = SHA512_DIGEST_SIZE;
+ sb_count = 2;
+ ioffset = 0;
+ ooffset = 2 * CCP_SB_BYTES - SHA384_DIGEST_SIZE;
+ break;
+ case CCP_SHA_TYPE_512:
+ digest_size = SHA512_DIGEST_SIZE;
+ init = (void *) ccp_sha512_init;
+ ctx_size = SHA512_DIGEST_SIZE;
+ sb_count = 2;
+ ooffset = ioffset = 0;
+ break;
default:
ret = -EINVAL;
goto e_data;
@@ -1052,6 +1093,11 @@ static int ccp_run_sha_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
op.u.sha.type = sha->type;
op.u.sha.msg_bits = sha->msg_bits;
+ /* For SHA1/224/256 the context fits in a single (32-byte) SB entry;
+ * SHA384/512 require 2 adjacent SB slots, with the right half in the
+ * first slot, and the left half in the second. Each portion must then
+ * be in little endian format: use the 256-bit byte swap option.
+ */
ret = ccp_init_dm_workarea(&ctx, cmd_q, sb_count * CCP_SB_BYTES,
DMA_BIDIRECTIONAL);
if (ret)
@@ -1063,6 +1109,13 @@ static int ccp_run_sha_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
case CCP_SHA_TYPE_256:
memcpy(ctx.address + ioffset, init, ctx_size);
break;
+ case CCP_SHA_TYPE_384:
+ case CCP_SHA_TYPE_512:
+ memcpy(ctx.address + ctx_size / 2, init,
+ ctx_size / 2);
+ memcpy(ctx.address, init + ctx_size / 2,
+ ctx_size / 2);
+ break;
default:
ret = -EINVAL;
goto e_ctx;
@@ -1129,6 +1182,15 @@ static int ccp_run_sha_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
sha->ctx, 0,
digest_size);
break;
+ case CCP_SHA_TYPE_384:
+ case CCP_SHA_TYPE_512:
+ ccp_get_dm_area(&ctx, 0,
+ sha->ctx, LSB_ITEM_SIZE - ooffset,
+ LSB_ITEM_SIZE);
+ ccp_get_dm_area(&ctx, LSB_ITEM_SIZE + ooffset,
+ sha->ctx, 0,
+ LSB_ITEM_SIZE - ooffset);
+ break;
default:
ret = -EINVAL;
goto e_ctx;
@@ -1166,6 +1228,16 @@ static int ccp_run_sha_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
ctx.address + ooffset,
digest_size);
break;
+ case CCP_SHA_TYPE_384:
+ case CCP_SHA_TYPE_512:
+ memcpy(hmac_buf + block_size,
+ ctx.address + LSB_ITEM_SIZE + ooffset,
+ LSB_ITEM_SIZE);
+ memcpy(hmac_buf + block_size +
+ (LSB_ITEM_SIZE - ooffset),
+ ctx.address,
+ LSB_ITEM_SIZE);
+ break;
default:
ret = -EINVAL;
goto e_ctx;
diff --git a/include/linux/ccp.h b/include/linux/ccp.h
index c71dd8f..90a1fbe 100644
--- a/include/linux/ccp.h
+++ b/include/linux/ccp.h
@@ -249,6 +249,8 @@ enum ccp_sha_type {
CCP_SHA_TYPE_1 = 1,
CCP_SHA_TYPE_224,
CCP_SHA_TYPE_256,
+ CCP_SHA_TYPE_384,
+ CCP_SHA_TYPE_512,
CCP_SHA_TYPE__LAST,
};
^ permalink raw reply related
* [PATCH V2 2/3] crypto: ccp - Enable support for AES GCM on v5 CCPs
From: Gary R Hook @ 2017-03-02 21:26 UTC (permalink / raw)
To: linux-crypto; +Cc: thomas.lendacky, gary.hook, herbert, davem
In-Reply-To: <20170302212331.30959.44149.stgit@taos>
A version 5 device provides the primitive commands
required for AES GCM. This patch adds support for
en/decryption.
Signed-off-by: Gary R Hook <gary.hook@amd.com>
---
drivers/crypto/ccp/Makefile | 1
drivers/crypto/ccp/ccp-crypto-aes-galois.c | 257 ++++++++++++++++++++++++++++
drivers/crypto/ccp/ccp-crypto-main.c | 12 +
drivers/crypto/ccp/ccp-crypto.h | 14 ++
drivers/crypto/ccp/ccp-dev-v5.c | 2
drivers/crypto/ccp/ccp-dev.h | 1
drivers/crypto/ccp/ccp-ops.c | 252 +++++++++++++++++++++++++++
include/linux/ccp.h | 9 +
8 files changed, 548 insertions(+)
create mode 100644 drivers/crypto/ccp/ccp-crypto-aes-galois.c
diff --git a/drivers/crypto/ccp/Makefile b/drivers/crypto/ccp/Makefile
index 346ceb8..9ca1722 100644
--- a/drivers/crypto/ccp/Makefile
+++ b/drivers/crypto/ccp/Makefile
@@ -12,4 +12,5 @@ ccp-crypto-objs := ccp-crypto-main.o \
ccp-crypto-aes.o \
ccp-crypto-aes-cmac.o \
ccp-crypto-aes-xts.o \
+ ccp-crypto-aes-galois.o \
ccp-crypto-sha.o
diff --git a/drivers/crypto/ccp/ccp-crypto-aes-galois.c b/drivers/crypto/ccp/ccp-crypto-aes-galois.c
new file mode 100644
index 0000000..8bc18c9
--- /dev/null
+++ b/drivers/crypto/ccp/ccp-crypto-aes-galois.c
@@ -0,0 +1,257 @@
+/*
+ * AMD Cryptographic Coprocessor (CCP) AES GCM crypto API support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Gary R Hook <gary.hook@amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/delay.h>
+#include <linux/scatterlist.h>
+#include <linux/crypto.h>
+#include <crypto/internal/aead.h>
+#include <crypto/algapi.h>
+#include <crypto/aes.h>
+#include <crypto/ctr.h>
+#include <crypto/scatterwalk.h>
+#include <linux/delay.h>
+
+#include "ccp-crypto.h"
+
+#define AES_GCM_IVSIZE 12
+
+static int ccp_aes_gcm_complete(struct crypto_async_request *async_req, int ret)
+{
+ return ret;
+}
+
+static int ccp_aes_gcm_setkey(struct crypto_aead *tfm, const u8 *key,
+ unsigned int key_len)
+{
+ struct ccp_ctx *ctx = crypto_aead_ctx(tfm);
+
+ switch (key_len) {
+ case AES_KEYSIZE_128:
+ ctx->u.aes.type = CCP_AES_TYPE_128;
+ break;
+ case AES_KEYSIZE_192:
+ ctx->u.aes.type = CCP_AES_TYPE_192;
+ break;
+ case AES_KEYSIZE_256:
+ ctx->u.aes.type = CCP_AES_TYPE_256;
+ break;
+ default:
+ crypto_aead_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
+ return -EINVAL;
+ }
+
+ ctx->u.aes.mode = CCP_AES_MODE_GCM;
+ ctx->u.aes.key_len = key_len;
+
+ memcpy(ctx->u.aes.key, key, key_len);
+ sg_init_one(&ctx->u.aes.key_sg, ctx->u.aes.key, key_len);
+
+ return 0;
+}
+
+static int ccp_aes_gcm_setauthsize(struct crypto_aead *tfm,
+ unsigned int authsize)
+{
+ return 0;
+}
+
+static int ccp_aes_gcm_crypt(struct aead_request *req, bool encrypt)
+{
+ struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+ struct ccp_ctx *ctx = crypto_aead_ctx(tfm);
+ struct ccp_aes_req_ctx *rctx = aead_request_ctx(req);
+ struct scatterlist *iv_sg = NULL;
+ unsigned int iv_len = 0;
+ int i;
+ int ret = 0;
+
+ if (!ctx->u.aes.key_len)
+ return -EINVAL;
+
+ if (ctx->u.aes.mode != CCP_AES_MODE_GCM)
+ return -EINVAL;
+
+ if (!req->iv)
+ return -EINVAL;
+
+ /*
+ * 5 parts:
+ * plaintext/ciphertext input
+ * AAD
+ * key
+ * IV
+ * Destination+tag buffer
+ */
+
+ /* According to the way AES GCM has been implemented here,
+ * per RFC 4106 it seems, the provided IV is fixed at 12 bytes,
+ * occupies the beginning of the IV array. Write a 32-bit
+ * integer after that (bytes 13-16) with a value of "1".
+ */
+ memcpy(rctx->iv, req->iv, AES_GCM_IVSIZE);
+ for (i = 0; i < 3; i++)
+ rctx->iv[i + AES_GCM_IVSIZE] = 0;
+ rctx->iv[AES_BLOCK_SIZE - 1] = 1;
+
+ /* Set up a scatterlist for the IV */
+ iv_sg = &rctx->iv_sg;
+ iv_len = AES_BLOCK_SIZE;
+ sg_init_one(iv_sg, rctx->iv, iv_len);
+
+ /* The AAD + plaintext are concatenated in the src buffer */
+ memset(&rctx->cmd, 0, sizeof(rctx->cmd));
+ INIT_LIST_HEAD(&rctx->cmd.entry);
+ rctx->cmd.engine = CCP_ENGINE_AES;
+ rctx->cmd.u.aes.type = ctx->u.aes.type;
+ rctx->cmd.u.aes.mode = ctx->u.aes.mode;
+ rctx->cmd.u.aes.action =
+ (encrypt) ? CCP_AES_ACTION_ENCRYPT : CCP_AES_ACTION_DECRYPT;
+ rctx->cmd.u.aes.key = &ctx->u.aes.key_sg;
+ rctx->cmd.u.aes.key_len = ctx->u.aes.key_len;
+ rctx->cmd.u.aes.iv = iv_sg;
+ rctx->cmd.u.aes.iv_len = iv_len;
+ rctx->cmd.u.aes.src = req->src;
+ rctx->cmd.u.aes.src_len = req->cryptlen;
+ rctx->cmd.u.aes.aad_len = req->assoclen;
+
+ /* The cipher text + the tag are in the dst buffer */
+ rctx->cmd.u.aes.dst = req->dst;
+
+ ret = ccp_crypto_enqueue_request(&req->base, &rctx->cmd);
+
+ return ret;
+}
+
+static int ccp_aes_gcm_encrypt(struct aead_request *req)
+{
+ return ccp_aes_gcm_crypt(req, true);
+}
+
+static int ccp_aes_gcm_decrypt(struct aead_request *req)
+{
+ return ccp_aes_gcm_crypt(req, false);
+}
+
+static int ccp_aes_gcm_cra_init(struct crypto_aead *tfm)
+{
+ struct ccp_ctx *ctx = crypto_aead_ctx(tfm);
+
+ ctx->complete = ccp_aes_gcm_complete;
+ ctx->u.aes.key_len = 0;
+
+ crypto_aead_set_reqsize(tfm, sizeof(struct ccp_aes_req_ctx));
+
+ return 0;
+}
+
+static void ccp_aes_gcm_cra_exit(struct crypto_tfm *tfm)
+{
+}
+
+static struct aead_alg ccp_aes_gcm_defaults = {
+ .setkey = ccp_aes_gcm_setkey,
+ .setauthsize = ccp_aes_gcm_setauthsize,
+ .encrypt = ccp_aes_gcm_encrypt,
+ .decrypt = ccp_aes_gcm_decrypt,
+ .init = ccp_aes_gcm_cra_init,
+ .ivsize = AES_GCM_IVSIZE,
+ .maxauthsize = AES_BLOCK_SIZE,
+ .base = {
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER |
+ CRYPTO_ALG_ASYNC |
+ CRYPTO_ALG_KERN_DRIVER_ONLY |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct ccp_ctx),
+ .cra_priority = CCP_CRA_PRIORITY,
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_exit = ccp_aes_gcm_cra_exit,
+ .cra_module = THIS_MODULE,
+ },
+};
+
+struct ccp_aes_aead_def {
+ enum ccp_aes_mode mode;
+ unsigned int version;
+ const char *name;
+ const char *driver_name;
+ unsigned int blocksize;
+ unsigned int ivsize;
+ struct aead_alg *alg_defaults;
+};
+
+static struct ccp_aes_aead_def aes_aead_algs[] = {
+ {
+ .mode = CCP_AES_MODE_GHASH,
+ .version = CCP_VERSION(5, 0),
+ .name = "gcm(aes)",
+ .driver_name = "gcm-aes-ccp",
+ .blocksize = 1,
+ .ivsize = AES_BLOCK_SIZE,
+ .alg_defaults = &ccp_aes_gcm_defaults,
+ },
+};
+
+static int ccp_register_aes_aead(struct list_head *head,
+ const struct ccp_aes_aead_def *def)
+{
+ struct ccp_crypto_aead *ccp_aead;
+ struct aead_alg *alg;
+ int ret;
+
+ ccp_aead = kzalloc(sizeof(*ccp_aead), GFP_KERNEL);
+ if (!ccp_aead)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&ccp_aead->entry);
+
+ ccp_aead->mode = def->mode;
+
+ /* Copy the defaults and override as necessary */
+ alg = &ccp_aead->alg;
+ *alg = *def->alg_defaults;
+ snprintf(alg->base.cra_name, CRYPTO_MAX_ALG_NAME, "%s", def->name);
+ snprintf(alg->base.cra_driver_name, CRYPTO_MAX_ALG_NAME, "%s",
+ def->driver_name);
+ alg->base.cra_blocksize = def->blocksize;
+ alg->base.cra_ablkcipher.ivsize = def->ivsize;
+
+ ret = crypto_register_aead(alg);
+ if (ret) {
+ pr_err("%s ablkcipher algorithm registration error (%d)\n",
+ alg->base.cra_name, ret);
+ kfree(ccp_aead);
+ return ret;
+ }
+
+ list_add(&ccp_aead->entry, head);
+
+ return 0;
+}
+
+int ccp_register_aes_aeads(struct list_head *head)
+{
+ int i, ret;
+ unsigned int ccpversion = ccp_version();
+
+ for (i = 0; i < ARRAY_SIZE(aes_aead_algs); i++) {
+ if (aes_aead_algs[i].version > ccpversion)
+ continue;
+ ret = ccp_register_aes_aead(head, &aes_aead_algs[i]);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
diff --git a/drivers/crypto/ccp/ccp-crypto-main.c b/drivers/crypto/ccp/ccp-crypto-main.c
index e0380e5..a33b8d6 100644
--- a/drivers/crypto/ccp/ccp-crypto-main.c
+++ b/drivers/crypto/ccp/ccp-crypto-main.c
@@ -36,6 +36,7 @@
/* List heads for the supported algorithms */
static LIST_HEAD(hash_algs);
static LIST_HEAD(cipher_algs);
+static LIST_HEAD(aead_algs);
/* For any tfm, requests for that tfm must be returned on the order
* received. With multiple queues available, the CCP can process more
@@ -335,6 +336,10 @@ static int ccp_register_algs(void)
ret = ccp_register_aes_xts_algs(&cipher_algs);
if (ret)
return ret;
+
+ ret = ccp_register_aes_aeads(&aead_algs);
+ if (ret)
+ return ret;
}
if (!sha_disable) {
@@ -350,6 +355,7 @@ static void ccp_unregister_algs(void)
{
struct ccp_crypto_ahash_alg *ahash_alg, *ahash_tmp;
struct ccp_crypto_ablkcipher_alg *ablk_alg, *ablk_tmp;
+ struct ccp_crypto_aead *aead_alg, *aead_tmp;
list_for_each_entry_safe(ahash_alg, ahash_tmp, &hash_algs, entry) {
crypto_unregister_ahash(&ahash_alg->alg);
@@ -362,6 +368,12 @@ static void ccp_unregister_algs(void)
list_del(&ablk_alg->entry);
kfree(ablk_alg);
}
+
+ list_for_each_entry_safe(aead_alg, aead_tmp, &aead_algs, entry) {
+ crypto_unregister_aead(&aead_alg->alg);
+ list_del(&aead_alg->entry);
+ kfree(aead_alg);
+ }
}
static int ccp_crypto_init(void)
diff --git a/drivers/crypto/ccp/ccp-crypto.h b/drivers/crypto/ccp/ccp-crypto.h
index 95cce27..34da62d 100644
--- a/drivers/crypto/ccp/ccp-crypto.h
+++ b/drivers/crypto/ccp/ccp-crypto.h
@@ -19,6 +19,8 @@
#include <linux/ccp.h>
#include <crypto/algapi.h>
#include <crypto/aes.h>
+#include <crypto/internal/aead.h>
+#include <crypto/aead.h>
#include <crypto/ctr.h>
#include <crypto/hash.h>
#include <crypto/sha.h>
@@ -33,6 +35,14 @@ struct ccp_crypto_ablkcipher_alg {
struct crypto_alg alg;
};
+struct ccp_crypto_aead {
+ struct list_head entry;
+
+ u32 mode;
+
+ struct aead_alg alg;
+};
+
struct ccp_crypto_ahash_alg {
struct list_head entry;
@@ -95,6 +105,9 @@ struct ccp_aes_req_ctx {
struct scatterlist iv_sg;
u8 iv[AES_BLOCK_SIZE];
+ struct scatterlist tag_sg;
+ u8 tag[AES_BLOCK_SIZE];
+
/* Fields used for RFC3686 requests */
u8 *rfc3686_info;
u8 rfc3686_iv[AES_BLOCK_SIZE];
@@ -212,6 +225,7 @@ struct scatterlist *ccp_crypto_sg_table_add(struct sg_table *table,
int ccp_register_aes_algs(struct list_head *head);
int ccp_register_aes_cmac_algs(struct list_head *head);
int ccp_register_aes_xts_algs(struct list_head *head);
+int ccp_register_aes_aeads(struct list_head *head);
int ccp_register_sha_algs(struct list_head *head);
#endif
diff --git a/drivers/crypto/ccp/ccp-dev-v5.c b/drivers/crypto/ccp/ccp-dev-v5.c
index d9e1876..d44a738 100644
--- a/drivers/crypto/ccp/ccp-dev-v5.c
+++ b/drivers/crypto/ccp/ccp-dev-v5.c
@@ -289,6 +289,8 @@ static int ccp5_perform_aes(struct ccp_op *op)
CCP_AES_TYPE(&function) = op->u.aes.type;
if (op->u.aes.mode == CCP_AES_MODE_CFB)
CCP_AES_SIZE(&function) = 0x7f;
+ if ((op->u.aes.mode == CCP_AES_MODE_GCTR) && op->eom)
+ CCP_AES_SIZE(&function) = op->u.aes.size;
CCP5_CMD_FUNCTION(&desc) = function.raw;
diff --git a/drivers/crypto/ccp/ccp-dev.h b/drivers/crypto/ccp/ccp-dev.h
index 830f35e..8f313e6 100644
--- a/drivers/crypto/ccp/ccp-dev.h
+++ b/drivers/crypto/ccp/ccp-dev.h
@@ -466,6 +466,7 @@ struct ccp_aes_op {
enum ccp_aes_type type;
enum ccp_aes_mode mode;
enum ccp_aes_action action;
+ unsigned int size;
};
struct ccp_xts_aes_op {
diff --git a/drivers/crypto/ccp/ccp-ops.c b/drivers/crypto/ccp/ccp-ops.c
index 213a752..83c8cf7 100644
--- a/drivers/crypto/ccp/ccp-ops.c
+++ b/drivers/crypto/ccp/ccp-ops.c
@@ -600,6 +600,255 @@ static int ccp_run_aes_cmac_cmd(struct ccp_cmd_queue *cmd_q,
return ret;
}
+static int ccp_run_aes_gcm_cmd(struct ccp_cmd_queue *cmd_q,
+ struct ccp_cmd *cmd)
+{
+ struct ccp_aes_engine *aes = &cmd->u.aes;
+ struct ccp_dm_workarea key, ctx, final_wa, tag;
+ struct ccp_data src, dst;
+ struct ccp_data aad;
+ struct ccp_op op;
+
+ unsigned long long *final;
+ unsigned int dm_offset;
+ unsigned int ilen;
+ bool in_place = true; /* Default value */
+ int ret;
+
+ struct scatterlist *p_inp, sg_inp[2];
+ struct scatterlist *p_tag, sg_tag[2];
+ struct scatterlist *p_outp, sg_outp[2];
+ struct scatterlist *p_aad;
+
+ if (!aes->iv)
+ return -EINVAL;
+
+ if (!((aes->key_len == AES_KEYSIZE_128) ||
+ (aes->key_len == AES_KEYSIZE_192) ||
+ (aes->key_len == AES_KEYSIZE_256)))
+ return -EINVAL;
+
+ if (!aes->key) /* Gotta have a key SGL */
+ return -EINVAL;
+
+ /* First, decompose the source buffer into AAD & PT,
+ * and the destination buffer into AAD, CT & tag, or
+ * the input into CT & tag.
+ * It is expected that the input and output SGs will
+ * be valid, even if the AAD and input lengths are 0.
+ */
+ p_aad = aes->src;
+ p_inp = scatterwalk_ffwd(sg_inp, aes->src, aes->aad_len);
+ p_outp = scatterwalk_ffwd(sg_outp, aes->dst, aes->aad_len);
+ if (aes->action == CCP_AES_ACTION_ENCRYPT) {
+ ilen = aes->src_len;
+ p_tag = scatterwalk_ffwd(sg_tag, p_outp, ilen);
+ } else {
+ /* Input length for decryption includes tag */
+ ilen = aes->src_len - AES_BLOCK_SIZE;
+ p_tag = scatterwalk_ffwd(sg_tag, p_inp, ilen);
+ }
+
+ memset(&op, 0, sizeof(op));
+ op.cmd_q = cmd_q;
+ op.jobid = CCP_NEW_JOBID(cmd_q->ccp);
+ op.sb_key = cmd_q->sb_key; /* Pre-allocated */
+ op.sb_ctx = cmd_q->sb_ctx; /* Pre-allocated */
+ op.init = 1;
+ op.u.aes.type = aes->type;
+
+ /* Copy the key to the LSB */
+ ret = ccp_init_dm_workarea(&key, cmd_q,
+ CCP_AES_CTX_SB_COUNT * CCP_SB_BYTES,
+ DMA_TO_DEVICE);
+ if (ret)
+ return ret;
+
+ dm_offset = CCP_SB_BYTES - aes->key_len;
+ ccp_set_dm_area(&key, dm_offset, aes->key, 0, aes->key_len);
+ ret = ccp_copy_to_sb(cmd_q, &key, op.jobid, op.sb_key,
+ CCP_PASSTHRU_BYTESWAP_256BIT);
+ if (ret) {
+ cmd->engine_error = cmd_q->cmd_error;
+ goto e_key;
+ }
+
+ /* Copy the context (IV) to the LSB.
+ * There is an assumption here that the IV is 96 bits in length, plus
+ * a nonce of 32 bits. If no IV is present, use a zeroed buffer.
+ */
+ ret = ccp_init_dm_workarea(&ctx, cmd_q,
+ CCP_AES_CTX_SB_COUNT * CCP_SB_BYTES,
+ DMA_BIDIRECTIONAL);
+ if (ret)
+ goto e_key;
+
+ dm_offset = CCP_AES_CTX_SB_COUNT * CCP_SB_BYTES - aes->iv_len;
+ ccp_set_dm_area(&ctx, dm_offset, aes->iv, 0, aes->iv_len);
+
+ ret = ccp_copy_to_sb(cmd_q, &ctx, op.jobid, op.sb_ctx,
+ CCP_PASSTHRU_BYTESWAP_256BIT);
+ if (ret) {
+ cmd->engine_error = cmd_q->cmd_error;
+ goto e_ctx;
+ }
+
+ op.init = 1;
+ if (aes->aad_len > 0) {
+ /* Step 1: Run a GHASH over the Additional Authenticated Data */
+ ret = ccp_init_data(&aad, cmd_q, p_aad, aes->aad_len,
+ AES_BLOCK_SIZE,
+ DMA_TO_DEVICE);
+ if (ret)
+ goto e_ctx;
+
+ op.u.aes.mode = CCP_AES_MODE_GHASH;
+ op.u.aes.action = CCP_AES_GHASHAAD;
+
+ while (aad.sg_wa.bytes_left) {
+ ccp_prepare_data(&aad, NULL, &op, AES_BLOCK_SIZE, true);
+
+ ret = cmd_q->ccp->vdata->perform->aes(&op);
+ if (ret) {
+ cmd->engine_error = cmd_q->cmd_error;
+ goto e_aad;
+ }
+
+ ccp_process_data(&aad, NULL, &op);
+ op.init = 0;
+ }
+ }
+
+ op.u.aes.mode = CCP_AES_MODE_GCTR;
+ op.u.aes.action = aes->action;
+
+ if (ilen > 0) {
+ /* Step 2: Run a GCTR over the plaintext */
+ in_place = (sg_virt(p_inp) == sg_virt(p_outp)) ? true : false;
+
+ ret = ccp_init_data(&src, cmd_q, p_inp, ilen,
+ AES_BLOCK_SIZE,
+ in_place ? DMA_BIDIRECTIONAL
+ : DMA_TO_DEVICE);
+ if (ret)
+ goto e_ctx;
+
+ if (in_place) {
+ dst = src;
+ } else {
+ ret = ccp_init_data(&dst, cmd_q, p_outp, ilen,
+ AES_BLOCK_SIZE, DMA_FROM_DEVICE);
+ if (ret)
+ goto e_src;
+ }
+
+ op.soc = 0;
+ op.eom = 0;
+ op.init = 1;
+ while (src.sg_wa.bytes_left) {
+ ccp_prepare_data(&src, &dst, &op, AES_BLOCK_SIZE, true);
+ if (!src.sg_wa.bytes_left) {
+ unsigned int nbytes = aes->src_len
+ % AES_BLOCK_SIZE;
+
+ if (nbytes) {
+ op.eom = 1;
+ op.u.aes.size = (nbytes * 8) - 1;
+ }
+ }
+
+ ret = cmd_q->ccp->vdata->perform->aes(&op);
+ if (ret) {
+ cmd->engine_error = cmd_q->cmd_error;
+ goto e_dst;
+ }
+
+ ccp_process_data(&src, &dst, &op);
+ op.init = 0;
+ }
+ }
+
+ /* Step 3: Update the IV portion of the context with the original IV */
+ ret = ccp_copy_from_sb(cmd_q, &ctx, op.jobid, op.sb_ctx,
+ CCP_PASSTHRU_BYTESWAP_256BIT);
+ if (ret) {
+ cmd->engine_error = cmd_q->cmd_error;
+ goto e_dst;
+ }
+
+ ccp_set_dm_area(&ctx, dm_offset, aes->iv, 0, aes->iv_len);
+
+ ret = ccp_copy_to_sb(cmd_q, &ctx, op.jobid, op.sb_ctx,
+ CCP_PASSTHRU_BYTESWAP_256BIT);
+ if (ret) {
+ cmd->engine_error = cmd_q->cmd_error;
+ goto e_dst;
+ }
+
+ /* Step 4: Concatenate the lengths of the AAD and source, and
+ * hash that 16 byte buffer.
+ */
+ ret = ccp_init_dm_workarea(&final_wa, cmd_q, AES_BLOCK_SIZE,
+ DMA_BIDIRECTIONAL);
+ if (ret)
+ goto e_dst;
+ final = (unsigned long long *) final_wa.address;
+ final[0] = cpu_to_be64(aes->aad_len * 8);
+ final[1] = cpu_to_be64(ilen * 8);
+
+ op.u.aes.mode = CCP_AES_MODE_GHASH;
+ op.u.aes.action = CCP_AES_GHASHFINAL;
+ op.src.type = CCP_MEMTYPE_SYSTEM;
+ op.src.u.dma.address = final_wa.dma.address;
+ op.src.u.dma.length = AES_BLOCK_SIZE;
+ op.dst.type = CCP_MEMTYPE_SYSTEM;
+ op.dst.u.dma.address = final_wa.dma.address;
+ op.dst.u.dma.length = AES_BLOCK_SIZE;
+ op.eom = 1;
+ op.u.aes.size = 0;
+ ret = cmd_q->ccp->vdata->perform->aes(&op);
+ if (ret)
+ goto e_dst;
+
+ if (aes->action == CCP_AES_ACTION_ENCRYPT) {
+ /* Put the ciphered tag after the ciphertext. */
+ ccp_get_dm_area(&final_wa, 0, p_tag, 0, AES_BLOCK_SIZE);
+ } else {
+ /* Does this ciphered tag match the input? */
+ ret = ccp_init_dm_workarea(&tag, cmd_q, AES_BLOCK_SIZE,
+ DMA_BIDIRECTIONAL);
+ if (ret)
+ goto e_tag;
+ ccp_set_dm_area(&tag, 0, p_tag, 0, AES_BLOCK_SIZE);
+
+ ret = memcmp(tag.address, final_wa.address, AES_BLOCK_SIZE);
+ ccp_dm_free(&tag);
+ }
+
+e_tag:
+ ccp_dm_free(&final_wa);
+
+e_dst:
+ if (aes->src_len && !in_place)
+ ccp_free_data(&dst, cmd_q);
+
+e_src:
+ if (aes->src_len)
+ ccp_free_data(&src, cmd_q);
+
+e_aad:
+ if (aes->aad_len)
+ ccp_free_data(&aad, cmd_q);
+
+e_ctx:
+ ccp_dm_free(&ctx);
+
+e_key:
+ ccp_dm_free(&key);
+
+ return ret;
+}
+
static int ccp_run_aes_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
{
struct ccp_aes_engine *aes = &cmd->u.aes;
@@ -613,6 +862,9 @@ static int ccp_run_aes_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
if (aes->mode == CCP_AES_MODE_CMAC)
return ccp_run_aes_cmac_cmd(cmd_q, cmd);
+ if (aes->mode == CCP_AES_MODE_GCM)
+ return ccp_run_aes_gcm_cmd(cmd_q, cmd);
+
if (!((aes->key_len == AES_KEYSIZE_128) ||
(aes->key_len == AES_KEYSIZE_192) ||
(aes->key_len == AES_KEYSIZE_256)))
diff --git a/include/linux/ccp.h b/include/linux/ccp.h
index 90a1fbe..6f78295 100644
--- a/include/linux/ccp.h
+++ b/include/linux/ccp.h
@@ -123,6 +123,10 @@ enum ccp_aes_mode {
CCP_AES_MODE_CFB,
CCP_AES_MODE_CTR,
CCP_AES_MODE_CMAC,
+ CCP_AES_MODE_GHASH,
+ CCP_AES_MODE_GCTR,
+ CCP_AES_MODE_GCM,
+ CCP_AES_MODE_GMAC,
CCP_AES_MODE__LAST,
};
@@ -137,6 +141,9 @@ enum ccp_aes_action {
CCP_AES_ACTION_ENCRYPT,
CCP_AES_ACTION__LAST,
};
+/* Overloaded field */
+#define CCP_AES_GHASHAAD CCP_AES_ACTION_DECRYPT
+#define CCP_AES_GHASHFINAL CCP_AES_ACTION_ENCRYPT
/**
* struct ccp_aes_engine - CCP AES operation
@@ -181,6 +188,8 @@ struct ccp_aes_engine {
struct scatterlist *cmac_key; /* K1/K2 cmac key required for
* final cmac cmd */
u32 cmac_key_len; /* In bytes */
+
+ u32 aad_len; /* In bytes */
};
/***** XTS-AES engine *****/
^ permalink raw reply related
* [PATCH V2 3/3] crypto: ccp - Enable 3DES function on v5 CCPs
From: Gary R Hook @ 2017-03-02 21:27 UTC (permalink / raw)
To: linux-crypto; +Cc: thomas.lendacky, gary.hook, herbert, davem
In-Reply-To: <20170302212331.30959.44149.stgit@taos>
Wire up support for Triple DES in ECB mode.
Signed-off-by: Gary R Hook <gary.hook@amd.com>
---
drivers/crypto/ccp/Makefile | 1
drivers/crypto/ccp/ccp-crypto-des3.c | 254 ++++++++++++++++++++++++++++++++++
drivers/crypto/ccp/ccp-crypto-main.c | 10 +
drivers/crypto/ccp/ccp-crypto.h | 22 +++
drivers/crypto/ccp/ccp-dev-v3.c | 1
drivers/crypto/ccp/ccp-dev-v5.c | 54 +++++++
drivers/crypto/ccp/ccp-dev.h | 14 ++
drivers/crypto/ccp/ccp-ops.c | 198 +++++++++++++++++++++++++++
include/linux/ccp.h | 57 +++++++-
9 files changed, 608 insertions(+), 3 deletions(-)
create mode 100644 drivers/crypto/ccp/ccp-crypto-des3.c
diff --git a/drivers/crypto/ccp/Makefile b/drivers/crypto/ccp/Makefile
index 9ca1722..60919a3 100644
--- a/drivers/crypto/ccp/Makefile
+++ b/drivers/crypto/ccp/Makefile
@@ -13,4 +13,5 @@ ccp-crypto-objs := ccp-crypto-main.o \
ccp-crypto-aes-cmac.o \
ccp-crypto-aes-xts.o \
ccp-crypto-aes-galois.o \
+ ccp-crypto-des3.o \
ccp-crypto-sha.o
diff --git a/drivers/crypto/ccp/ccp-crypto-des3.c b/drivers/crypto/ccp/ccp-crypto-des3.c
new file mode 100644
index 0000000..5af7347
--- /dev/null
+++ b/drivers/crypto/ccp/ccp-crypto-des3.c
@@ -0,0 +1,254 @@
+/*
+ * AMD Cryptographic Coprocessor (CCP) DES3 crypto API support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Gary R Hook <ghook@amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/delay.h>
+#include <linux/scatterlist.h>
+#include <linux/crypto.h>
+#include <crypto/algapi.h>
+#include <crypto/scatterwalk.h>
+#include <crypto/des.h>
+
+#include "ccp-crypto.h"
+
+static int ccp_des3_complete(struct crypto_async_request *async_req, int ret)
+{
+ struct ablkcipher_request *req = ablkcipher_request_cast(async_req);
+ struct ccp_ctx *ctx = crypto_tfm_ctx(req->base.tfm);
+ struct ccp_des3_req_ctx *rctx = ablkcipher_request_ctx(req);
+
+ if (ret)
+ return ret;
+
+ if (ctx->u.des3.mode != CCP_DES3_MODE_ECB)
+ memcpy(req->info, rctx->iv, DES3_EDE_BLOCK_SIZE);
+
+ return 0;
+}
+
+static int ccp_des3_setkey(struct crypto_ablkcipher *tfm, const u8 *key,
+ unsigned int key_len)
+{
+ struct ccp_ctx *ctx = crypto_tfm_ctx(crypto_ablkcipher_tfm(tfm));
+ struct ccp_crypto_ablkcipher_alg *alg =
+ ccp_crypto_ablkcipher_alg(crypto_ablkcipher_tfm(tfm));
+ u32 *flags = &tfm->base.crt_flags;
+
+
+ /* From des_generic.c:
+ *
+ * RFC2451:
+ * If the first two or last two independent 64-bit keys are
+ * equal (k1 == k2 or k2 == k3), then the DES3 operation is simply the
+ * same as DES. Implementers MUST reject keys that exhibit this
+ * property.
+ */
+ const u32 *K = (const u32 *)key;
+
+ if (unlikely(!((K[0] ^ K[2]) | (K[1] ^ K[3])) ||
+ !((K[2] ^ K[4]) | (K[3] ^ K[5]))) &&
+ (*flags & CRYPTO_TFM_REQ_WEAK_KEY)) {
+ *flags |= CRYPTO_TFM_RES_WEAK_KEY;
+ return -EINVAL;
+ }
+
+ /* It's not clear that there is any support for a keysize of 112.
+ * If needed, the caller should make K1 == K3
+ */
+ ctx->u.des3.type = CCP_DES3_TYPE_168;
+ ctx->u.des3.mode = alg->mode;
+ ctx->u.des3.key_len = key_len;
+
+ memcpy(ctx->u.des3.key, key, key_len);
+ sg_init_one(&ctx->u.des3.key_sg, ctx->u.des3.key, key_len);
+
+ return 0;
+}
+
+static int ccp_des3_crypt(struct ablkcipher_request *req, bool encrypt)
+{
+ struct ccp_ctx *ctx = crypto_tfm_ctx(req->base.tfm);
+ struct ccp_des3_req_ctx *rctx = ablkcipher_request_ctx(req);
+ struct scatterlist *iv_sg = NULL;
+ unsigned int iv_len = 0;
+ int ret;
+
+ if (!ctx->u.des3.key_len)
+ return -EINVAL;
+
+ if (((ctx->u.des3.mode == CCP_DES3_MODE_ECB) ||
+ (ctx->u.des3.mode == CCP_DES3_MODE_CBC)) &&
+ (req->nbytes & (DES3_EDE_BLOCK_SIZE - 1)))
+ return -EINVAL;
+
+ if (ctx->u.des3.mode != CCP_DES3_MODE_ECB) {
+ if (!req->info)
+ return -EINVAL;
+
+ memcpy(rctx->iv, req->info, DES3_EDE_BLOCK_SIZE);
+ iv_sg = &rctx->iv_sg;
+ iv_len = DES3_EDE_BLOCK_SIZE;
+ sg_init_one(iv_sg, rctx->iv, iv_len);
+ }
+
+ memset(&rctx->cmd, 0, sizeof(rctx->cmd));
+ INIT_LIST_HEAD(&rctx->cmd.entry);
+ rctx->cmd.engine = CCP_ENGINE_DES3;
+ rctx->cmd.u.des3.type = ctx->u.des3.type;
+ rctx->cmd.u.des3.mode = ctx->u.des3.mode;
+ rctx->cmd.u.des3.action = (encrypt)
+ ? CCP_DES3_ACTION_ENCRYPT
+ : CCP_DES3_ACTION_DECRYPT;
+ rctx->cmd.u.des3.key = &ctx->u.des3.key_sg;
+ rctx->cmd.u.des3.key_len = ctx->u.des3.key_len;
+ rctx->cmd.u.des3.iv = iv_sg;
+ rctx->cmd.u.des3.iv_len = iv_len;
+ rctx->cmd.u.des3.src = req->src;
+ rctx->cmd.u.des3.src_len = req->nbytes;
+ rctx->cmd.u.des3.dst = req->dst;
+
+ ret = ccp_crypto_enqueue_request(&req->base, &rctx->cmd);
+
+ return ret;
+}
+
+static int ccp_des3_encrypt(struct ablkcipher_request *req)
+{
+ return ccp_des3_crypt(req, true);
+}
+
+static int ccp_des3_decrypt(struct ablkcipher_request *req)
+{
+ return ccp_des3_crypt(req, false);
+}
+
+static int ccp_des3_cra_init(struct crypto_tfm *tfm)
+{
+ struct ccp_ctx *ctx = crypto_tfm_ctx(tfm);
+
+ ctx->complete = ccp_des3_complete;
+ ctx->u.des3.key_len = 0;
+
+ tfm->crt_ablkcipher.reqsize = sizeof(struct ccp_des3_req_ctx);
+
+ return 0;
+}
+
+static void ccp_des3_cra_exit(struct crypto_tfm *tfm)
+{
+}
+
+static struct crypto_alg ccp_des3_defaults = {
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER |
+ CRYPTO_ALG_ASYNC |
+ CRYPTO_ALG_KERN_DRIVER_ONLY |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_blocksize = DES3_EDE_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct ccp_ctx),
+ .cra_priority = CCP_CRA_PRIORITY,
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_init = ccp_des3_cra_init,
+ .cra_exit = ccp_des3_cra_exit,
+ .cra_module = THIS_MODULE,
+ .cra_ablkcipher = {
+ .setkey = ccp_des3_setkey,
+ .encrypt = ccp_des3_encrypt,
+ .decrypt = ccp_des3_decrypt,
+ .min_keysize = DES3_EDE_KEY_SIZE,
+ .max_keysize = DES3_EDE_KEY_SIZE,
+ },
+};
+
+struct ccp_des3_def {
+ enum ccp_des3_mode mode;
+ unsigned int version;
+ const char *name;
+ const char *driver_name;
+ unsigned int blocksize;
+ unsigned int ivsize;
+ struct crypto_alg *alg_defaults;
+};
+
+static struct ccp_des3_def des3_algs[] = {
+ {
+ .mode = CCP_DES3_MODE_ECB,
+ .version = CCP_VERSION(5, 0),
+ .name = "ecb(des3_ede)",
+ .driver_name = "ecb-des3-ccp",
+ .blocksize = DES3_EDE_BLOCK_SIZE,
+ .ivsize = 0,
+ .alg_defaults = &ccp_des3_defaults,
+ },
+ {
+ .mode = CCP_DES3_MODE_CBC,
+ .version = CCP_VERSION(5, 0),
+ .name = "cbc(des3_ede)",
+ .driver_name = "cbc-des3-ccp",
+ .blocksize = DES3_EDE_BLOCK_SIZE,
+ .ivsize = DES3_EDE_BLOCK_SIZE,
+ .alg_defaults = &ccp_des3_defaults,
+ },
+};
+
+static int ccp_register_des3_alg(struct list_head *head,
+ const struct ccp_des3_def *def)
+{
+ struct ccp_crypto_ablkcipher_alg *ccp_alg;
+ struct crypto_alg *alg;
+ int ret;
+
+ ccp_alg = kzalloc(sizeof(*ccp_alg), GFP_KERNEL);
+ if (!ccp_alg)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&ccp_alg->entry);
+
+ ccp_alg->mode = def->mode;
+
+ /* Copy the defaults and override as necessary */
+ alg = &ccp_alg->alg;
+ *alg = *def->alg_defaults;
+ snprintf(alg->cra_name, CRYPTO_MAX_ALG_NAME, "%s", def->name);
+ snprintf(alg->cra_driver_name, CRYPTO_MAX_ALG_NAME, "%s",
+ def->driver_name);
+ alg->cra_blocksize = def->blocksize;
+ alg->cra_ablkcipher.ivsize = def->ivsize;
+
+ ret = crypto_register_alg(alg);
+ if (ret) {
+ pr_err("%s ablkcipher algorithm registration error (%d)\n",
+ alg->cra_name, ret);
+ kfree(ccp_alg);
+ return ret;
+ }
+
+ list_add(&ccp_alg->entry, head);
+
+ return 0;
+}
+
+int ccp_register_des3_algs(struct list_head *head)
+{
+ int i, ret;
+ unsigned int ccpversion = ccp_version();
+
+ for (i = 0; i < ARRAY_SIZE(des3_algs); i++) {
+ if (des3_algs[i].version > ccpversion)
+ continue;
+ ret = ccp_register_des3_alg(head, &des3_algs[i]);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
diff --git a/drivers/crypto/ccp/ccp-crypto-main.c b/drivers/crypto/ccp/ccp-crypto-main.c
index a33b8d6..8dccbdd 100644
--- a/drivers/crypto/ccp/ccp-crypto-main.c
+++ b/drivers/crypto/ccp/ccp-crypto-main.c
@@ -33,6 +33,10 @@
module_param(sha_disable, uint, 0444);
MODULE_PARM_DESC(sha_disable, "Disable use of SHA - any non-zero value");
+static unsigned int des3_disable;
+module_param(des3_disable, uint, 0444);
+MODULE_PARM_DESC(des3_disable, "Disable use of 3DES - any non-zero value");
+
/* List heads for the supported algorithms */
static LIST_HEAD(hash_algs);
static LIST_HEAD(cipher_algs);
@@ -342,6 +346,12 @@ static int ccp_register_algs(void)
return ret;
}
+ if (!des3_disable) {
+ ret = ccp_register_des3_algs(&cipher_algs);
+ if (ret)
+ return ret;
+ }
+
if (!sha_disable) {
ret = ccp_register_sha_algs(&hash_algs);
if (ret)
diff --git a/drivers/crypto/ccp/ccp-crypto.h b/drivers/crypto/ccp/ccp-crypto.h
index 34da62d..dd5bf15 100644
--- a/drivers/crypto/ccp/ccp-crypto.h
+++ b/drivers/crypto/ccp/ccp-crypto.h
@@ -25,6 +25,8 @@
#include <crypto/hash.h>
#include <crypto/sha.h>
+#define CCP_LOG_LEVEL KERN_INFO
+
#define CCP_CRA_PRIORITY 300
struct ccp_crypto_ablkcipher_alg {
@@ -150,6 +152,24 @@ struct ccp_aes_cmac_exp_ctx {
u8 buf[AES_BLOCK_SIZE];
};
+/***** 3DES related defines *****/
+struct ccp_des3_ctx {
+ enum ccp_engine engine;
+ enum ccp_des3_type type;
+ enum ccp_des3_mode mode;
+
+ struct scatterlist key_sg;
+ unsigned int key_len;
+ u8 key[AES_MAX_KEY_SIZE];
+};
+
+struct ccp_des3_req_ctx {
+ struct scatterlist iv_sg;
+ u8 iv[AES_BLOCK_SIZE];
+
+ struct ccp_cmd cmd;
+};
+
/* SHA-related defines
* These values must be large enough to accommodate any variant
*/
@@ -214,6 +234,7 @@ struct ccp_ctx {
union {
struct ccp_aes_ctx aes;
struct ccp_sha_ctx sha;
+ struct ccp_des3_ctx des3;
} u;
};
@@ -227,5 +248,6 @@ struct scatterlist *ccp_crypto_sg_table_add(struct sg_table *table,
int ccp_register_aes_xts_algs(struct list_head *head);
int ccp_register_aes_aeads(struct list_head *head);
int ccp_register_sha_algs(struct list_head *head);
+int ccp_register_des3_algs(struct list_head *head);
#endif
diff --git a/drivers/crypto/ccp/ccp-dev-v3.c b/drivers/crypto/ccp/ccp-dev-v3.c
index 7bc0998..a3689a6 100644
--- a/drivers/crypto/ccp/ccp-dev-v3.c
+++ b/drivers/crypto/ccp/ccp-dev-v3.c
@@ -553,6 +553,7 @@ static irqreturn_t ccp_irq_handler(int irq, void *data)
static const struct ccp_actions ccp3_actions = {
.aes = ccp_perform_aes,
.xts_aes = ccp_perform_xts_aes,
+ .des3 = NULL,
.sha = ccp_perform_sha,
.rsa = ccp_perform_rsa,
.passthru = ccp_perform_passthru,
diff --git a/drivers/crypto/ccp/ccp-dev-v5.c b/drivers/crypto/ccp/ccp-dev-v5.c
index d44a738..72a1777 100644
--- a/drivers/crypto/ccp/ccp-dev-v5.c
+++ b/drivers/crypto/ccp/ccp-dev-v5.c
@@ -108,6 +108,12 @@ static void ccp_lsb_free(struct ccp_cmd_queue *cmd_q, unsigned int start,
u16 type:2;
} aes_xts;
struct {
+ u16 size:7;
+ u16 encrypt:1;
+ u16 mode:5;
+ u16 type:2;
+ } des3;
+ struct {
u16 rsvd1:10;
u16 type:4;
u16 rsvd2:1;
@@ -139,6 +145,10 @@ static void ccp_lsb_free(struct ccp_cmd_queue *cmd_q, unsigned int start,
#define CCP_AES_TYPE(p) ((p)->aes.type)
#define CCP_XTS_SIZE(p) ((p)->aes_xts.size)
#define CCP_XTS_ENCRYPT(p) ((p)->aes_xts.encrypt)
+#define CCP_DES3_SIZE(p) ((p)->des3.size)
+#define CCP_DES3_ENCRYPT(p) ((p)->des3.encrypt)
+#define CCP_DES3_MODE(p) ((p)->des3.mode)
+#define CCP_DES3_TYPE(p) ((p)->des3.type)
#define CCP_SHA_TYPE(p) ((p)->sha.type)
#define CCP_RSA_SIZE(p) ((p)->rsa.size)
#define CCP_PT_BYTESWAP(p) ((p)->pt.byteswap)
@@ -391,6 +401,47 @@ static int ccp5_perform_sha(struct ccp_op *op)
return ccp5_do_cmd(&desc, op->cmd_q);
}
+static int ccp5_perform_des3(struct ccp_op *op)
+{
+ struct ccp5_desc desc;
+ union ccp_function function;
+ u32 key_addr = op->sb_key * LSB_ITEM_SIZE;
+
+ /* Zero out all the fields of the command desc */
+ memset(&desc, 0, sizeof(struct ccp5_desc));
+
+ CCP5_CMD_ENGINE(&desc) = CCP_ENGINE_DES3;
+
+ CCP5_CMD_SOC(&desc) = op->soc;
+ CCP5_CMD_IOC(&desc) = 1;
+ CCP5_CMD_INIT(&desc) = op->init;
+ CCP5_CMD_EOM(&desc) = op->eom;
+ CCP5_CMD_PROT(&desc) = 0;
+
+ function.raw = 0;
+ CCP_DES3_ENCRYPT(&function) = op->u.des3.action;
+ CCP_DES3_MODE(&function) = op->u.des3.mode;
+ CCP_DES3_TYPE(&function) = op->u.des3.type;
+ CCP5_CMD_FUNCTION(&desc) = cpu_to_le32(function.raw);
+
+ CCP5_CMD_LEN(&desc) = cpu_to_le32(op->src.u.dma.length);
+
+ CCP5_CMD_SRC_LO(&desc) = cpu_to_le32(ccp_addr_lo(&op->src.u.dma));
+ CCP5_CMD_SRC_HI(&desc) = cpu_to_le32(ccp_addr_hi(&op->src.u.dma));
+ CCP5_CMD_SRC_MEM(&desc) = cpu_to_le32(CCP_MEMTYPE_SYSTEM);
+
+ CCP5_CMD_DST_LO(&desc) = cpu_to_le32(ccp_addr_lo(&op->dst.u.dma));
+ CCP5_CMD_DST_HI(&desc) = cpu_to_le32(ccp_addr_hi(&op->dst.u.dma));
+ CCP5_CMD_DST_MEM(&desc) = cpu_to_le32(CCP_MEMTYPE_SYSTEM);
+
+ CCP5_CMD_KEY_LO(&desc) = cpu_to_le32(lower_32_bits(key_addr));
+ CCP5_CMD_KEY_HI(&desc) = 0;
+ CCP5_CMD_KEY_MEM(&desc) = cpu_to_le32(CCP_MEMTYPE_SB);
+ CCP5_CMD_LSB_ID(&desc) = cpu_to_le32(op->sb_ctx);
+
+ return ccp5_do_cmd(&desc, op->cmd_q);
+}
+
static int ccp5_perform_rsa(struct ccp_op *op)
{
struct ccp5_desc desc;
@@ -438,6 +489,7 @@ static int ccp5_perform_passthru(struct ccp_op *op)
struct ccp_dma_info *saddr = &op->src.u.dma;
struct ccp_dma_info *daddr = &op->dst.u.dma;
+
memset(&desc, 0, Q_DESC_SIZE);
CCP5_CMD_ENGINE(&desc) = CCP_ENGINE_PASSTHRU;
@@ -732,6 +784,7 @@ static int ccp5_init(struct ccp_device *ccp)
dev_dbg(dev, "queue #%u available\n", i);
}
+
if (ccp->cmd_q_count == 0) {
dev_notice(dev, "no command queues available\n");
ret = -EIO;
@@ -997,6 +1050,7 @@ static void ccp5other_config(struct ccp_device *ccp)
.aes = ccp5_perform_aes,
.xts_aes = ccp5_perform_xts_aes,
.sha = ccp5_perform_sha,
+ .des3 = ccp5_perform_des3,
.rsa = ccp5_perform_rsa,
.passthru = ccp5_perform_passthru,
.ecc = ccp5_perform_ecc,
diff --git a/drivers/crypto/ccp/ccp-dev.h b/drivers/crypto/ccp/ccp-dev.h
index 8f313e6..384a545 100644
--- a/drivers/crypto/ccp/ccp-dev.h
+++ b/drivers/crypto/ccp/ccp-dev.h
@@ -190,6 +190,9 @@
#define CCP_XTS_AES_KEY_SB_COUNT 1
#define CCP_XTS_AES_CTX_SB_COUNT 1
+#define CCP_DES3_KEY_SB_COUNT 1
+#define CCP_DES3_CTX_SB_COUNT 1
+
#define CCP_SHA_SB_COUNT 1
#define CCP_RSA_MAX_WIDTH 4096
@@ -474,6 +477,12 @@ struct ccp_xts_aes_op {
enum ccp_xts_aes_unit_size unit_size;
};
+struct ccp_des3_op {
+ enum ccp_des3_type type;
+ enum ccp_des3_mode mode;
+ enum ccp_des3_action action;
+};
+
struct ccp_sha_op {
enum ccp_sha_type type;
u64 msg_bits;
@@ -511,6 +520,7 @@ struct ccp_op {
union {
struct ccp_aes_op aes;
struct ccp_xts_aes_op xts;
+ struct ccp_des3_op des3;
struct ccp_sha_op sha;
struct ccp_rsa_op rsa;
struct ccp_passthru_op passthru;
@@ -619,13 +629,13 @@ struct ccp5_desc {
struct ccp_actions {
int (*aes)(struct ccp_op *);
int (*xts_aes)(struct ccp_op *);
+ int (*des3)(struct ccp_op *);
int (*sha)(struct ccp_op *);
int (*rsa)(struct ccp_op *);
int (*passthru)(struct ccp_op *);
int (*ecc)(struct ccp_op *);
u32 (*sballoc)(struct ccp_cmd_queue *, unsigned int);
- void (*sbfree)(struct ccp_cmd_queue *, unsigned int,
- unsigned int);
+ void (*sbfree)(struct ccp_cmd_queue *, unsigned int, unsigned int);
unsigned int (*get_free_slots)(struct ccp_cmd_queue *);
int (*init)(struct ccp_device *);
void (*destroy)(struct ccp_device *);
diff --git a/drivers/crypto/ccp/ccp-ops.c b/drivers/crypto/ccp/ccp-ops.c
index 83c8cf7..7ae7f14 100644
--- a/drivers/crypto/ccp/ccp-ops.c
+++ b/drivers/crypto/ccp/ccp-ops.c
@@ -16,6 +16,7 @@
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <crypto/scatterwalk.h>
+#include <crypto/des.h>
#include <linux/ccp.h>
#include "ccp-dev.h"
@@ -1183,6 +1184,200 @@ static int ccp_run_xts_aes_cmd(struct ccp_cmd_queue *cmd_q,
return ret;
}
+static int ccp_run_des3_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
+{
+ struct ccp_des3_engine *des3 = &cmd->u.des3;
+
+ struct ccp_dm_workarea key, ctx;
+ struct ccp_data src, dst;
+ struct ccp_op op;
+ unsigned int dm_offset;
+ unsigned int len_singlekey;
+ bool in_place = false;
+ int ret;
+
+ /* Error checks */
+ if (!cmd_q->ccp->vdata->perform->des3)
+ return -EINVAL;
+
+ if (des3->key_len != DES3_EDE_KEY_SIZE)
+ return -EINVAL;
+
+ if (((des3->mode == CCP_DES3_MODE_ECB) ||
+ (des3->mode == CCP_DES3_MODE_CBC)) &&
+ (des3->src_len & (DES3_EDE_BLOCK_SIZE - 1)))
+ return -EINVAL;
+
+ if (!des3->key || !des3->src || !des3->dst)
+ return -EINVAL;
+
+ if (des3->mode != CCP_DES3_MODE_ECB) {
+ if (des3->iv_len != DES3_EDE_BLOCK_SIZE)
+ return -EINVAL;
+
+ if (!des3->iv)
+ return -EINVAL;
+ }
+
+ ret = -EIO;
+ /* Zero out all the fields of the command desc */
+ memset(&op, 0, sizeof(op));
+
+ /* Set up the Function field */
+ op.cmd_q = cmd_q;
+ op.jobid = CCP_NEW_JOBID(cmd_q->ccp);
+ op.sb_key = cmd_q->sb_key;
+
+ op.init = (des3->mode == CCP_DES3_MODE_ECB) ? 0 : 1;
+ op.u.des3.type = des3->type;
+ op.u.des3.mode = des3->mode;
+ op.u.des3.action = des3->action;
+
+ /*
+ * All supported key sizes fit in a single (32-byte) KSB entry and
+ * (like AES) must be in little endian format. Use the 256-bit byte
+ * swap passthru option to convert from big endian to little endian.
+ */
+ ret = ccp_init_dm_workarea(&key, cmd_q,
+ CCP_DES3_KEY_SB_COUNT * CCP_SB_BYTES,
+ DMA_TO_DEVICE);
+ if (ret)
+ return ret;
+
+ /*
+ * The contents of the key triplet are in the reverse order of what
+ * is required by the engine. Copy the 3 pieces individually to put
+ * them where they belong.
+ */
+ dm_offset = CCP_SB_BYTES - des3->key_len; /* Basic offset */
+
+ len_singlekey = des3->key_len / 3;
+ ccp_set_dm_area(&key, dm_offset + 2 * len_singlekey,
+ des3->key, 0, len_singlekey);
+ ccp_set_dm_area(&key, dm_offset + len_singlekey,
+ des3->key, len_singlekey, len_singlekey);
+ ccp_set_dm_area(&key, dm_offset,
+ des3->key, 2 * len_singlekey, len_singlekey);
+
+ /* Copy the key to the SB */
+ ret = ccp_copy_to_sb(cmd_q, &key, op.jobid, op.sb_key,
+ CCP_PASSTHRU_BYTESWAP_256BIT);
+ if (ret) {
+ cmd->engine_error = cmd_q->cmd_error;
+ goto e_key;
+ }
+
+ /*
+ * The DES3 context fits in a single (32-byte) KSB entry and
+ * must be in little endian format. Use the 256-bit byte swap
+ * passthru option to convert from big endian to little endian.
+ */
+ if (des3->mode != CCP_DES3_MODE_ECB) {
+ u32 load_mode;
+
+ op.sb_ctx = cmd_q->sb_ctx;
+
+ ret = ccp_init_dm_workarea(&ctx, cmd_q,
+ CCP_DES3_CTX_SB_COUNT * CCP_SB_BYTES,
+ DMA_BIDIRECTIONAL);
+ if (ret)
+ goto e_key;
+
+ /* Load the context into the LSB */
+ dm_offset = CCP_SB_BYTES - des3->iv_len;
+ ccp_set_dm_area(&ctx, dm_offset, des3->iv, 0, des3->iv_len);
+
+ if (cmd_q->ccp->vdata->version == CCP_VERSION(3, 0))
+ load_mode = CCP_PASSTHRU_BYTESWAP_NOOP;
+ else
+ load_mode = CCP_PASSTHRU_BYTESWAP_256BIT;
+ ret = ccp_copy_to_sb(cmd_q, &ctx, op.jobid, op.sb_ctx,
+ load_mode);
+ if (ret) {
+ cmd->engine_error = cmd_q->cmd_error;
+ goto e_ctx;
+ }
+ }
+
+ /*
+ * Prepare the input and output data workareas. For in-place
+ * operations we need to set the dma direction to BIDIRECTIONAL
+ * and copy the src workarea to the dst workarea.
+ */
+ if (sg_virt(des3->src) == sg_virt(des3->dst))
+ in_place = true;
+
+ ret = ccp_init_data(&src, cmd_q, des3->src, des3->src_len,
+ DES3_EDE_BLOCK_SIZE,
+ in_place ? DMA_BIDIRECTIONAL : DMA_TO_DEVICE);
+ if (ret)
+ goto e_ctx;
+
+ if (in_place)
+ dst = src;
+ else {
+ ret = ccp_init_data(&dst, cmd_q, des3->dst, des3->src_len,
+ DES3_EDE_BLOCK_SIZE, DMA_FROM_DEVICE);
+ if (ret)
+ goto e_src;
+ }
+
+ /* Send data to the CCP DES3 engine */
+ while (src.sg_wa.bytes_left) {
+ ccp_prepare_data(&src, &dst, &op, DES3_EDE_BLOCK_SIZE, true);
+ if (!src.sg_wa.bytes_left) {
+ op.eom = 1;
+
+ /* Since we don't retrieve the context in ECB mode
+ * we have to wait for the operation to complete
+ * on the last piece of data
+ */
+ op.soc = 0;
+ }
+
+ ret = cmd_q->ccp->vdata->perform->des3(&op);
+ if (ret) {
+ cmd->engine_error = cmd_q->cmd_error;
+ goto e_dst;
+ }
+
+ ccp_process_data(&src, &dst, &op);
+ }
+
+ if (des3->mode != CCP_DES3_MODE_ECB) {
+ /* Retrieve the context and make BE */
+ ret = ccp_copy_from_sb(cmd_q, &ctx, op.jobid, op.sb_ctx,
+ CCP_PASSTHRU_BYTESWAP_256BIT);
+ if (ret) {
+ cmd->engine_error = cmd_q->cmd_error;
+ goto e_dst;
+ }
+
+ /* ...but we only need the last DES3_EDE_BLOCK_SIZE bytes */
+ if (cmd_q->ccp->vdata->version == CCP_VERSION(3, 0))
+ dm_offset = CCP_SB_BYTES - des3->iv_len;
+ else
+ dm_offset = 0;
+ ccp_get_dm_area(&ctx, dm_offset, des3->iv, 0,
+ DES3_EDE_BLOCK_SIZE);
+ }
+e_dst:
+ if (!in_place)
+ ccp_free_data(&dst, cmd_q);
+
+e_src:
+ ccp_free_data(&src, cmd_q);
+
+e_ctx:
+ if (des3->mode != CCP_DES3_MODE_ECB)
+ ccp_dm_free(&ctx);
+
+e_key:
+ ccp_dm_free(&key);
+
+ return ret;
+}
+
static int ccp_run_sha_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
{
struct ccp_sha_engine *sha = &cmd->u.sha;
@@ -2147,6 +2342,9 @@ int ccp_run_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
case CCP_ENGINE_XTS_AES_128:
ret = ccp_run_xts_aes_cmd(cmd_q, cmd);
break;
+ case CCP_ENGINE_DES3:
+ ret = ccp_run_des3_cmd(cmd_q, cmd);
+ break;
case CCP_ENGINE_SHA:
ret = ccp_run_sha_cmd(cmd_q, cmd);
break;
diff --git a/include/linux/ccp.h b/include/linux/ccp.h
index 6f78295..dbe8aa0 100644
--- a/include/linux/ccp.h
+++ b/include/linux/ccp.h
@@ -301,6 +301,60 @@ struct ccp_sha_engine {
* final sha cmd */
};
+/***** 3DES engine *****/
+enum ccp_des3_mode {
+ CCP_DES3_MODE_ECB = 0,
+ CCP_DES3_MODE_CBC,
+ CCP_DES3_MODE_CFB,
+ CCP_DES3_MODE__LAST,
+};
+
+enum ccp_des3_type {
+ CCP_DES3_TYPE_168 = 1,
+ CCP_DES3_TYPE__LAST,
+ };
+
+enum ccp_des3_action {
+ CCP_DES3_ACTION_DECRYPT = 0,
+ CCP_DES3_ACTION_ENCRYPT,
+ CCP_DES3_ACTION__LAST,
+};
+
+/**
+ * struct ccp_des3_engine - CCP SHA operation
+ * @type: Type of 3DES operation
+ * @mode: cipher mode
+ * @action: 3DES operation (decrypt/encrypt)
+ * @key: key to be used for this 3DES operation
+ * @key_len: length of key (in bytes)
+ * @iv: IV to be used for this AES operation
+ * @iv_len: length in bytes of iv
+ * @src: input data to be used for this operation
+ * @src_len: length of input data used for this operation (in bytes)
+ * @dst: output data produced by this operation
+ *
+ * Variables required to be set when calling ccp_enqueue_cmd():
+ * - type, mode, action, key, key_len, src, dst, src_len
+ * - iv, iv_len for any mode other than ECB
+ *
+ * The iv variable is used as both input and output. On completion of the
+ * 3DES operation the new IV overwrites the old IV.
+ */
+struct ccp_des3_engine {
+ enum ccp_des3_type type;
+ enum ccp_des3_mode mode;
+ enum ccp_des3_action action;
+
+ struct scatterlist *key;
+ u32 key_len; /* In bytes */
+
+ struct scatterlist *iv;
+ u32 iv_len; /* In bytes */
+
+ struct scatterlist *src, *dst;
+ u64 src_len; /* In bytes */
+};
+
/***** RSA engine *****/
/**
* struct ccp_rsa_engine - CCP RSA operation
@@ -550,7 +604,7 @@ struct ccp_ecc_engine {
enum ccp_engine {
CCP_ENGINE_AES = 0,
CCP_ENGINE_XTS_AES_128,
- CCP_ENGINE_RSVD1,
+ CCP_ENGINE_DES3,
CCP_ENGINE_SHA,
CCP_ENGINE_RSA,
CCP_ENGINE_PASSTHRU,
@@ -598,6 +652,7 @@ struct ccp_cmd {
union {
struct ccp_aes_engine aes;
struct ccp_xts_aes_engine xts;
+ struct ccp_des3_engine des3;
struct ccp_sha_engine sha;
struct ccp_rsa_engine rsa;
struct ccp_passthru_engine passthru;
^ permalink raw reply related
* [PATCH V2 0/3] Series short description
From: Gary R Hook @ 2017-03-02 21:26 UTC (permalink / raw)
To: linux-crypto; +Cc: thomas.lendacky, gary.hook, herbert, davem
The following series:
- Move verbose init messages to debug mode
- Update the queue pointers in the event of an error
- Simplify buffer management and eliminate an unused option
---
Gary R Hook (3):
crypto: ccp - Add SHA-2 384- and 512-bit support
crypto: ccp - Enable support for AES GCM on v5 CCPs
crypto: ccp - Enable 3DES function on v5 CCPs
drivers/crypto/ccp/Makefile | 2
drivers/crypto/ccp/ccp-crypto-aes-galois.c | 257 ++++++++++++++
drivers/crypto/ccp/ccp-crypto-des3.c | 254 ++++++++++++++
drivers/crypto/ccp/ccp-crypto-main.c | 22 +
drivers/crypto/ccp/ccp-crypto-sha.c | 22 +
drivers/crypto/ccp/ccp-crypto.h | 44 ++
drivers/crypto/ccp/ccp-dev-v3.c | 1
drivers/crypto/ccp/ccp-dev-v5.c | 56 +++
drivers/crypto/ccp/ccp-dev.h | 15 +
drivers/crypto/ccp/ccp-ops.c | 522 ++++++++++++++++++++++++++++
include/linux/ccp.h | 68 ++++
11 files changed, 1257 insertions(+), 6 deletions(-)
create mode 100644 drivers/crypto/ccp/ccp-crypto-aes-galois.c
create mode 100644 drivers/crypto/ccp/ccp-crypto-des3.c
^ permalink raw reply
* Re: [PATCH V2 0/3] Series short description
From: Gary R Hook @ 2017-03-02 21:29 UTC (permalink / raw)
To: Hook, Gary, linux-crypto@vger.kernel.org
Cc: Lendacky, Thomas, herbert@gondor.apana.org.au,
davem@davemloft.net
In-Reply-To: <20170302212331.30959.44149.stgit@taos>
On 03/02/2017 03:26 PM, Hook, Gary wrote:
> The following series:
> - Move verbose init messages to debug mode
> - Update the queue pointers in the event of an error
> - Simplify buffer management and eliminate an unused option
*sigh* That Subject line is supposed to read "Minor CCP improvements and
clean-up".
>
> ---
>
> Gary R Hook (3):
> crypto: ccp - Add SHA-2 384- and 512-bit support
> crypto: ccp - Enable support for AES GCM on v5 CCPs
> crypto: ccp - Enable 3DES function on v5 CCPs
>
>
> drivers/crypto/ccp/Makefile | 2
> drivers/crypto/ccp/ccp-crypto-aes-galois.c | 257 ++++++++++++++
> drivers/crypto/ccp/ccp-crypto-des3.c | 254 ++++++++++++++
> drivers/crypto/ccp/ccp-crypto-main.c | 22 +
> drivers/crypto/ccp/ccp-crypto-sha.c | 22 +
> drivers/crypto/ccp/ccp-crypto.h | 44 ++
> drivers/crypto/ccp/ccp-dev-v3.c | 1
> drivers/crypto/ccp/ccp-dev-v5.c | 56 +++
> drivers/crypto/ccp/ccp-dev.h | 15 +
> drivers/crypto/ccp/ccp-ops.c | 522
> ++++++++++++++++++++++++++++
> include/linux/ccp.h | 68 ++++
> 11 files changed, 1257 insertions(+), 6 deletions(-)
> create mode 100644 drivers/crypto/ccp/ccp-crypto-aes-galois.c
> create mode 100644 drivers/crypto/ccp/ccp-crypto-des3.c
>
> --
> Signature
--
This is my day job. Follow me at:
IG/Twitter/Facebook: @grhookphoto
IG/Twitter/Facebook: @grhphotographer
^ permalink raw reply
* Re: [PATCH] crypto: Add ECB dependency for XTS mode
From: Milan Broz @ 2017-03-03 6:25 UTC (permalink / raw)
To: stable; +Cc: linux-crypto, herbert
In-Reply-To: <20170223073826.19066-1-gmazyland@gmail.com>
Patch below should be backported to 4.10 stable
(only 4.10, older kernels are ok).
We have reports some systems fail to boot from LUKS now
(missing ecb module in initramdisk) ...
Upstream commit is 12cb3a1c4184f891d965d1f39f8cfcc9ef617647
Thanks,
Milan
On 02/23/2017 08:38 AM, Milan Broz wrote:
> Since the
> commit f1c131b45410a202eb45cc55980a7a9e4e4b4f40
> crypto: xts - Convert to skcipher
> the XTS mode is based on ECB, so the mode must select
> ECB otherwise it can fail to initialize.
>
> Signed-off-by: Milan Broz <gmazyland@gmail.com>
> ---
> crypto/Kconfig | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/crypto/Kconfig b/crypto/Kconfig
> index 160f08e721cc..9c245eb0dd83 100644
> --- a/crypto/Kconfig
> +++ b/crypto/Kconfig
> @@ -374,6 +374,7 @@ config CRYPTO_XTS
> select CRYPTO_BLKCIPHER
> select CRYPTO_MANAGER
> select CRYPTO_GF128MUL
> + select CRYPTO_ECB
> help
> XTS: IEEE1619/D16 narrow block cipher use with aes-xts-plain,
> key size 256, 384 or 512 bits. This implementation currently
>
^ permalink raw reply
* [PATCH] crypto: powerpc - Fix initialisation of crc32c context
From: Daniel Axtens @ 2017-03-03 6:56 UTC (permalink / raw)
To: linuxppc-dev, linux-crypto; +Cc: anton, Daniel Axtens
Turning on crypto self-tests on a POWER8 shows:
alg: hash: Test 1 failed for crc32c-vpmsum
00000000: ff ff ff ff
Comparing the code with the Intel CRC32c implementation on which
ours is based shows that we are doing an init with 0, not ~0
as CRC32c requires.
This probably wasn't caught because btrfs does its own weird
open-coded initialisation.
Initialise our internal context to ~0 on init.
This makes the self-tests pass, and btrfs continues to work.
Fixes: 6dd7a82cc54e ("crypto: powerpc - Add POWER8 optimised crc32c")
Cc: Anton Blanchard <anton@samba.org>
Cc: stable@vger.kernel.org
Signed-off-by: Daniel Axtens <dja@axtens.net>
---
arch/powerpc/crypto/crc32c-vpmsum_glue.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/crypto/crc32c-vpmsum_glue.c b/arch/powerpc/crypto/crc32c-vpmsum_glue.c
index 9fa046d56eba..411994551afc 100644
--- a/arch/powerpc/crypto/crc32c-vpmsum_glue.c
+++ b/arch/powerpc/crypto/crc32c-vpmsum_glue.c
@@ -52,7 +52,7 @@ static int crc32c_vpmsum_cra_init(struct crypto_tfm *tfm)
{
u32 *key = crypto_tfm_ctx(tfm);
- *key = 0;
+ *key = ~0;
return 0;
}
--
2.9.3
^ permalink raw reply related
* Re: XTS Crypto Not Found In /proc/crypto Even After Compiled for 4.10.1.
From: Herbert Xu @ 2017-03-03 4:02 UTC (permalink / raw)
To: Nathan Royce
Cc: davem, linux-crypto, linux-kernel, Krzysztof Kozlowski,
Marek Szyprowski
In-Reply-To: <CALaQ_hoZ6wh-H_NNoT80r6kthvFpL05zNWr7upVqispknXMEvg@mail.gmail.com>
On Thu, Mar 02, 2017 at 05:35:30PM -0600, Nathan Royce wrote:
> ARM ODroid XU4
>
> $ cat /proc/config.gz | gunzip | grep XTS
> CONFIG_CRYPTO_XTS=y
>
> $ grep xts /proc/crypto
> //4.9.13
> name : xts(aes)
> driver : xts(aes-generic)
> //4.10.1
> <blank>
> //cbc can be found though
>
> CRYPTTAB:
> cryptswap1 UUID=<sanitized> /dev/urandom
> swap,offset=2048,cipher=aes-xts-plain64:sha512,size=512,nofail
>
> FSTAB:
> /dev/mapper/cryptswap1 none swap sw 0 0
>
> Boot Log:
> [ 10.535985] ------------[ cut here ]------------
> [ 10.539252] WARNING: CPU: 0 PID: 0 at crypto/skcipher.c:430
> skcipher_walk_first+0x13c/0x14c
> [ 10.547542] Modules linked in: xor xor_neon aes_arm zlib_deflate
> dm_crypt raid6_pq nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
> ip_tables x_tables
> [ 10.561716] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.10.1-dirty #1
> [ 10.568049] Hardware name: SAMSUNG EXYNOS (Flattened Device Tree)
> [ 10.574171] [<c010eb54>] (unwind_backtrace) from [<c010b408>]
> (show_stack+0x10/0x14)
> [ 10.581893] [<c010b408>] (show_stack) from [<c036772c>]
> (dump_stack+0x84/0x98)
> [ 10.589073] [<c036772c>] (dump_stack) from [<c011bb20>]
> (__warn+0xe8/0x100)
> [ 10.595975] [<c011bb20>] (__warn) from [<c011bc30>]
> (warn_slowpath_null+0x20/0x28)
> [ 10.603546] [<c011bc30>] (warn_slowpath_null) from [<c0329a48>]
> (skcipher_walk_first+0x13c/0x14c)
> [ 10.612390] [<c0329a48>] (skcipher_walk_first) from [<c0329b30>]
> (skcipher_walk_virt+0x1c/0x38)
> [ 10.621056] [<c0329b30>] (skcipher_walk_virt) from [<c0330ed0>]
> (post_crypt+0x38/0x1c4)
> [ 10.629022] [<c0330ed0>] (post_crypt) from [<c0331470>]
> (decrypt_done+0x4c/0x54)
> [ 10.636389] [<c0331470>] (decrypt_done) from [<c05a03f0>]
> (s5p_aes_complete+0x70/0xfc)
> [ 10.644274] [<c05a03f0>] (s5p_aes_complete) from [<c05a05b0>]
> (s5p_aes_interrupt+0x134/0x1a0)
> [ 10.652771] [<c05a05b0>] (s5p_aes_interrupt) from [<c016dc3c>]
> (__handle_irq_event_percpu+0x9c/0x124)
This looks like a bug in the s5p driver. It's calling the completion
function straight from the IRQ handler, which is triggering the
sanity check in skcipher_walk_first.
The s5p driver needs to schedule a tasklet to call the completion
function.
Do you have crypto self-test enabled? If so it should've caught
this at run-time. Otherwise you can disable the s5p driver until
it's fixed.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH V2 2/3] crypto: ccp - Enable support for AES GCM on v5 CCPs
From: Stephan Müller @ 2017-03-03 7:15 UTC (permalink / raw)
To: Gary R Hook; +Cc: linux-crypto, thomas.lendacky, herbert, davem
In-Reply-To: <20170302212654.30959.31.stgit@taos>
Am Donnerstag, 2. März 2017, 22:26:54 CET schrieb Gary R Hook:
Hi Gary,
> A version 5 device provides the primitive commands
> required for AES GCM. This patch adds support for
> en/decryption.
>
> Signed-off-by: Gary R Hook <gary.hook@amd.com>
> ---
> drivers/crypto/ccp/Makefile | 1
> drivers/crypto/ccp/ccp-crypto-aes-galois.c | 257
> ++++++++++++++++++++++++++++ drivers/crypto/ccp/ccp-crypto-main.c |
> 12 +
> drivers/crypto/ccp/ccp-crypto.h | 14 ++
> drivers/crypto/ccp/ccp-dev-v5.c | 2
> drivers/crypto/ccp/ccp-dev.h | 1
> drivers/crypto/ccp/ccp-ops.c | 252
> +++++++++++++++++++++++++++ include/linux/ccp.h |
> 9 +
> 8 files changed, 548 insertions(+)
> create mode 100644 drivers/crypto/ccp/ccp-crypto-aes-galois.c
>
> diff --git a/drivers/crypto/ccp/Makefile b/drivers/crypto/ccp/Makefile
> index 346ceb8..9ca1722 100644
> --- a/drivers/crypto/ccp/Makefile
> +++ b/drivers/crypto/ccp/Makefile
> @@ -12,4 +12,5 @@ ccp-crypto-objs := ccp-crypto-main.o \
> ccp-crypto-aes.o \
> ccp-crypto-aes-cmac.o \
> ccp-crypto-aes-xts.o \
> + ccp-crypto-aes-galois.o \
> ccp-crypto-sha.o
> diff --git a/drivers/crypto/ccp/ccp-crypto-aes-galois.c
> b/drivers/crypto/ccp/ccp-crypto-aes-galois.c new file mode 100644
> index 0000000..8bc18c9
> --- /dev/null
> +++ b/drivers/crypto/ccp/ccp-crypto-aes-galois.c
> @@ -0,0 +1,257 @@
> +/*
> + * AMD Cryptographic Coprocessor (CCP) AES GCM crypto API support
> + *
> + * Copyright (C) 2016 Advanced Micro Devices, Inc.
> + *
> + * Author: Gary R Hook <gary.hook@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +#include <linux/module.h>
> +#include <linux/sched.h>
> +#include <linux/delay.h>
> +#include <linux/scatterlist.h>
> +#include <linux/crypto.h>
> +#include <crypto/internal/aead.h>
> +#include <crypto/algapi.h>
> +#include <crypto/aes.h>
> +#include <crypto/ctr.h>
> +#include <crypto/scatterwalk.h>
> +#include <linux/delay.h>
> +
> +#include "ccp-crypto.h"
> +
> +#define AES_GCM_IVSIZE 12
> +
> +static int ccp_aes_gcm_complete(struct crypto_async_request *async_req, int
> ret) +{
> + return ret;
> +}
> +
> +static int ccp_aes_gcm_setkey(struct crypto_aead *tfm, const u8 *key,
> + unsigned int key_len)
> +{
> + struct ccp_ctx *ctx = crypto_aead_ctx(tfm);
> +
> + switch (key_len) {
> + case AES_KEYSIZE_128:
> + ctx->u.aes.type = CCP_AES_TYPE_128;
> + break;
> + case AES_KEYSIZE_192:
> + ctx->u.aes.type = CCP_AES_TYPE_192;
> + break;
> + case AES_KEYSIZE_256:
> + ctx->u.aes.type = CCP_AES_TYPE_256;
> + break;
> + default:
> + crypto_aead_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
> + return -EINVAL;
> + }
> +
> + ctx->u.aes.mode = CCP_AES_MODE_GCM;
> + ctx->u.aes.key_len = key_len;
> +
> + memcpy(ctx->u.aes.key, key, key_len);
> + sg_init_one(&ctx->u.aes.key_sg, ctx->u.aes.key, key_len);
> +
> + return 0;
> +}
> +
> +static int ccp_aes_gcm_setauthsize(struct crypto_aead *tfm,
> + unsigned int authsize)
> +{
> + return 0;
> +}
> +
> +static int ccp_aes_gcm_crypt(struct aead_request *req, bool encrypt)
> +{
> + struct crypto_aead *tfm = crypto_aead_reqtfm(req);
> + struct ccp_ctx *ctx = crypto_aead_ctx(tfm);
> + struct ccp_aes_req_ctx *rctx = aead_request_ctx(req);
> + struct scatterlist *iv_sg = NULL;
> + unsigned int iv_len = 0;
> + int i;
> + int ret = 0;
> +
> + if (!ctx->u.aes.key_len)
> + return -EINVAL;
> +
> + if (ctx->u.aes.mode != CCP_AES_MODE_GCM)
> + return -EINVAL;
> +
> + if (!req->iv)
> + return -EINVAL;
> +
> + /*
> + * 5 parts:
> + * plaintext/ciphertext input
> + * AAD
> + * key
> + * IV
> + * Destination+tag buffer
> + */
> +
> + /* According to the way AES GCM has been implemented here,
> + * per RFC 4106 it seems, the provided IV is fixed at 12 bytes,
When you have that restriction, should the cipher be called rfc4106(gcm(aes))?
But then, the key is 4 bytes longer than a normal AES key as it contains the
leading 32 bits of the IV.
> + * occupies the beginning of the IV array. Write a 32-bit
> + * integer after that (bytes 13-16) with a value of "1".
> + */
> + memcpy(rctx->iv, req->iv, AES_GCM_IVSIZE);
> + for (i = 0; i < 3; i++)
> + rctx->iv[i + AES_GCM_IVSIZE] = 0;
> + rctx->iv[AES_BLOCK_SIZE - 1] = 1;
> +
> + /* Set up a scatterlist for the IV */
> + iv_sg = &rctx->iv_sg;
> + iv_len = AES_BLOCK_SIZE;
> + sg_init_one(iv_sg, rctx->iv, iv_len);
> +
> + /* The AAD + plaintext are concatenated in the src buffer */
> + memset(&rctx->cmd, 0, sizeof(rctx->cmd));
> + INIT_LIST_HEAD(&rctx->cmd.entry);
> + rctx->cmd.engine = CCP_ENGINE_AES;
> + rctx->cmd.u.aes.type = ctx->u.aes.type;
> + rctx->cmd.u.aes.mode = ctx->u.aes.mode;
> + rctx->cmd.u.aes.action =
> + (encrypt) ? CCP_AES_ACTION_ENCRYPT : CCP_AES_ACTION_DECRYPT;
Instead of this condition, why not changing the encrypt/decrypt function to
directly provide the enc/dec variables?
> + rctx->cmd.u.aes.key = &ctx->u.aes.key_sg;
> + rctx->cmd.u.aes.key_len = ctx->u.aes.key_len;
> + rctx->cmd.u.aes.iv = iv_sg;
> + rctx->cmd.u.aes.iv_len = iv_len;
> + rctx->cmd.u.aes.src = req->src;
> + rctx->cmd.u.aes.src_len = req->cryptlen;
> + rctx->cmd.u.aes.aad_len = req->assoclen;
Just to be on the safe side: is the implementation good when cryptlen or
assoclen is 0?
Ciao
Stephan
^ permalink raw reply
* Re: XTS Crypto Not Found In /proc/crypto Even After Compiled for 4.10.1.
From: Nathan Royce @ 2017-03-03 9:00 UTC (permalink / raw)
To: Herbert Xu
Cc: davem, linux-crypto, linux-kernel, Krzysztof Kozlowski,
Marek Szyprowski
In-Reply-To: <20170303040210.GA24737@gondor.apana.org.au>
OK, I went ahead and enabled self tests
"CRYPTO_MANAGER_DISABLE_TESTS=n", and my system was able to boot,
albeit with failures:
*****
Mar 02 23:14:38 server kernel: ---[ end trace 1c8a91f28cbcebf3 ]---
Mar 02 23:14:38 server kernel: alg: skcipher: encryption failed on
test 1 for xts(ecb-aes-s5p): ret=35
Mar 02 23:14:38 server kernel: device-mapper: table: 254:0: crypt:
Error allocating crypto tfm
Mar 02 23:14:38 server kernel: device-mapper: ioctl: error adding
target to table
Mar 02 23:14:39 server systemd-cryptsetup[234]: Failed to activate
with key file '/dev/urandom': Invalid argument
*****
(weird that it asked for the passphrase)
But I do question whether the root issue is related to s5p... Maybe
there is a correlation in the warning, but to me it looks like the
issue is something else.
In my OP, I noted that the xts crypto isn't even found in /proc/crypto
in 4.10. I'd think it would at least be listed, even if it isn't used.
CBC is listed in /proc/crypto with kernel 4.9.13 and 4.10.1 (cbc-aes-s5p)
XTS is listed in /proc/crypto with kernel 4.9.13 but NOT 4.10.1
I should also add that I didn't include other tainted messages since
they followed the messages I first posted.
I was assuming that when the first issue would work, the others would
follow suit. I just didn't want to inundate with possible junk.
I still have the log file if you think it would be helpful to post the rest.
PS: I also noticed the bounce from my first mail submission because I
didn't enable plain-text for the e-mail (marked as spam because the
email contained html). I rectified that for this reply.
On Thu, Mar 2, 2017 at 10:02 PM, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> On Thu, Mar 02, 2017 at 05:35:30PM -0600, Nathan Royce wrote:
>> ARM ODroid XU4
>>
>> $ cat /proc/config.gz | gunzip | grep XTS
>> CONFIG_CRYPTO_XTS=y
>>
>> $ grep xts /proc/crypto
>> //4.9.13
>> name : xts(aes)
>> driver : xts(aes-generic)
>> //4.10.1
>> <blank>
>> //cbc can be found though
>>
>> CRYPTTAB:
>> cryptswap1 UUID=<sanitized> /dev/urandom
>> swap,offset=2048,cipher=aes-xts-plain64:sha512,size=512,nofail
>>
>> FSTAB:
>> /dev/mapper/cryptswap1 none swap sw 0 0
>>
>> Boot Log:
>> [ 10.535985] ------------[ cut here ]------------
>> [ 10.539252] WARNING: CPU: 0 PID: 0 at crypto/skcipher.c:430
>> skcipher_walk_first+0x13c/0x14c
>> [ 10.547542] Modules linked in: xor xor_neon aes_arm zlib_deflate
>> dm_crypt raid6_pq nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
>> ip_tables x_tables
>> [ 10.561716] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.10.1-dirty #1
>> [ 10.568049] Hardware name: SAMSUNG EXYNOS (Flattened Device Tree)
>> [ 10.574171] [<c010eb54>] (unwind_backtrace) from [<c010b408>]
>> (show_stack+0x10/0x14)
>> [ 10.581893] [<c010b408>] (show_stack) from [<c036772c>]
>> (dump_stack+0x84/0x98)
>> [ 10.589073] [<c036772c>] (dump_stack) from [<c011bb20>]
>> (__warn+0xe8/0x100)
>> [ 10.595975] [<c011bb20>] (__warn) from [<c011bc30>]
>> (warn_slowpath_null+0x20/0x28)
>> [ 10.603546] [<c011bc30>] (warn_slowpath_null) from [<c0329a48>]
>> (skcipher_walk_first+0x13c/0x14c)
>> [ 10.612390] [<c0329a48>] (skcipher_walk_first) from [<c0329b30>]
>> (skcipher_walk_virt+0x1c/0x38)
>> [ 10.621056] [<c0329b30>] (skcipher_walk_virt) from [<c0330ed0>]
>> (post_crypt+0x38/0x1c4)
>> [ 10.629022] [<c0330ed0>] (post_crypt) from [<c0331470>]
>> (decrypt_done+0x4c/0x54)
>> [ 10.636389] [<c0331470>] (decrypt_done) from [<c05a03f0>]
>> (s5p_aes_complete+0x70/0xfc)
>> [ 10.644274] [<c05a03f0>] (s5p_aes_complete) from [<c05a05b0>]
>> (s5p_aes_interrupt+0x134/0x1a0)
>> [ 10.652771] [<c05a05b0>] (s5p_aes_interrupt) from [<c016dc3c>]
>> (__handle_irq_event_percpu+0x9c/0x124)
>
> This looks like a bug in the s5p driver. It's calling the completion
> function straight from the IRQ handler, which is triggering the
> sanity check in skcipher_walk_first.
>
> The s5p driver needs to schedule a tasklet to call the completion
> function.
>
> Do you have crypto self-test enabled? If so it should've caught
> this at run-time. Otherwise you can disable the s5p driver until
> it's fixed.
>
> Cheers,
> --
> Email: Herbert Xu <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: XTS Crypto Not Found In /proc/crypto Even After Compiled for 4.10.1.
From: Herbert Xu @ 2017-03-03 9:33 UTC (permalink / raw)
To: Nathan Royce
Cc: davem, linux-crypto, linux-kernel, Krzysztof Kozlowski,
Marek Szyprowski
In-Reply-To: <CALaQ_hruPpfO0ggwcQZ_j+WB9=ONDatLbe3sERviVd=w1iNhSw@mail.gmail.com>
On Fri, Mar 03, 2017 at 03:00:26AM -0600, Nathan Royce wrote:
> OK, I went ahead and enabled self tests
> "CRYPTO_MANAGER_DISABLE_TESTS=n", and my system was able to boot,
> albeit with failures:
> *****
> Mar 02 23:14:38 server kernel: ---[ end trace 1c8a91f28cbcebf3 ]---
> Mar 02 23:14:38 server kernel: alg: skcipher: encryption failed on
> test 1 for xts(ecb-aes-s5p): ret=35
> Mar 02 23:14:38 server kernel: device-mapper: table: 254:0: crypt:
> Error allocating crypto tfm
> Mar 02 23:14:38 server kernel: device-mapper: ioctl: error adding
> target to table
> Mar 02 23:14:39 server systemd-cryptsetup[234]: Failed to activate
> with key file '/dev/urandom': Invalid argument
> *****
> (weird that it asked for the passphrase)
>
> But I do question whether the root issue is related to s5p... Maybe
> there is a correlation in the warning, but to me it looks like the
> issue is something else.
I see. Do you have ECB enabled in your config? The new XTS requires
ECB to be present so that could be your problem.
There is already a patch on its way to stable to add the Kconfig
select on ECB.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: XTS Crypto Not Found In /proc/crypto Even After Compiled for 4.10.1.
From: Nathan Royce @ 2017-03-03 10:36 UTC (permalink / raw)
To: Herbert Xu
Cc: davem, linux-crypto, linux-kernel, Krzysztof Kozlowski,
Marek Szyprowski
I do have ECB selected as well:
DM_CRYPT=y
CRYPTO_ECB=y
CRYPTO_XTS=y
name : ecb(aes)
driver : ecb-aes-s5p
module : kernel
priority : 100
refcnt : 1
selftest : passed
internal : no
type : ablkcipher
async : yes
blocksize : 16
min keysize : 16
max keysize : 32
ivsize : 0
geniv : <default>
//still no "xts" can be found in the list
I saw this about the regression that sounds similar to my issue,
except even when I built-in dm_crypt (no initramfs. just diving
straight into system), it still fails:
http://www.mail-archive.com/linux-crypto@vger.kernel.org/msg23748.html
On Fri, Mar 3, 2017 at 3:33 AM, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> On Fri, Mar 03, 2017 at 03:00:26AM -0600, Nathan Royce wrote:
>> OK, I went ahead and enabled self tests
>> "CRYPTO_MANAGER_DISABLE_TESTS=n", and my system was able to boot,
>> albeit with failures:
>> *****
>> Mar 02 23:14:38 server kernel: ---[ end trace 1c8a91f28cbcebf3 ]---
>> Mar 02 23:14:38 server kernel: alg: skcipher: encryption failed on
>> test 1 for xts(ecb-aes-s5p): ret=35
>> Mar 02 23:14:38 server kernel: device-mapper: table: 254:0: crypt:
>> Error allocating crypto tfm
>> Mar 02 23:14:38 server kernel: device-mapper: ioctl: error adding
>> target to table
>> Mar 02 23:14:39 server systemd-cryptsetup[234]: Failed to activate
>> with key file '/dev/urandom': Invalid argument
>> *****
>> (weird that it asked for the passphrase)
>>
>> But I do question whether the root issue is related to s5p... Maybe
>> there is a correlation in the warning, but to me it looks like the
>> issue is something else.
>
> I see. Do you have ECB enabled in your config? The new XTS requires
> ECB to be present so that could be your problem.
>
> There is already a patch on its way to stable to add the Kconfig
> select on ECB.
>
> Cheers,
> --
> Email: Herbert Xu <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: XTS Crypto Not Found In /proc/crypto Even After Compiled for 4.10.1.
From: Herbert Xu @ 2017-03-03 12:04 UTC (permalink / raw)
To: Nathan Royce
Cc: davem, linux-crypto, linux-kernel, Krzysztof Kozlowski,
Marek Szyprowski
In-Reply-To: <CALaQ_hoafES-K1ds-XUPC6h4VnHX_ebym4LYjeuq1jHdbLYuDQ@mail.gmail.com>
On Fri, Mar 03, 2017 at 04:36:18AM -0600, Nathan Royce wrote:
> I do have ECB selected as well:
> DM_CRYPT=y
> CRYPTO_ECB=y
> CRYPTO_XTS=y
>
> name : ecb(aes)
> driver : ecb-aes-s5p
> module : kernel
> priority : 100
> refcnt : 1
> selftest : passed
> internal : no
> type : ablkcipher
> async : yes
> blocksize : 16
> min keysize : 16
> max keysize : 32
> ivsize : 0
> geniv : <default>
> //still no "xts" can be found in the list
Weird. So you can't find any instances of xts in /proc/crypto
at all? Even if the self-test fails it should still register an
entry there...
In any case, I think disabling the s5p driver should work at
least.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [RFC PATCH v2 19/32] crypto: ccp: Introduce the AMD Secure Processor device
From: Andy Shevchenko @ 2017-03-03 13:55 UTC (permalink / raw)
To: Brijesh Singh, Mark Rutland
Cc: simon.guinot, linux-efi, kvm, rkrcmar, matt, linux-pci,
linus.walleij, gary.hook, linux-mm, paul.gortmaker, hpa, cl,
dan.j.williams, aarcange, sfr, herbert, bhe, xemul, joro, x86,
peterz, piotr.luc, mingo, msalter, ross.zwisler, bp, dyoung,
thomas.lendacky, jroedel, keescook, arnd, toshi.kani,
mathieu.desnoyers, luto, devel, bhelgaas, tglx, mchehab,
iamjoonsoo.kim, labbott
In-Reply-To: <0db0055f-9208-524f-74aa-674894ee90d3@amd.com>
On Thu, 2017-03-02 at 13:11 -0600, Brijesh Singh wrote:
> Hi Mark,
>
> On 03/02/2017 11:39 AM, Mark Rutland wrote:
> > On Thu, Mar 02, 2017 at 10:16:15AM -0500, Brijesh Singh wrote:
> > >
> > > +ccp-$(CONFIG_CRYPTO_DEV_CCP) += ccp-dev.o \
> > > ccp-ops.o \
> > > ccp-dev-v3.o \
> > > ccp-dev-v5.o \
> > > - ccp-platform.o \
> > > ccp-dmaengine.o
> >
> > It looks like ccp-platform.c has morphed into sp-platform.c (judging
> > by
> > the compatible string and general shape of the code), and the
> > original
> > ccp-platform.c is no longer built.
> >
> > Shouldn't ccp-platform.c be deleted by this patch?
> >
>
> Good catch. Both ccp-platform.c and ccp-pci.c should have been
> deleted
> by this patch. I missed deleting it, will fix in next rev.
Don't forget to use -M -C when preparing / sending patches.
--
Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Intel Finland Oy
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: XTS Crypto Not Found In /proc/crypto Even After Compiled for 4.10.1.
From: Nathan Royce @ 2017-03-03 14:08 UTC (permalink / raw)
To: Herbert Xu
Cc: davem, linux-crypto, linux-kernel, Krzysztof Kozlowski,
Marek Szyprowski
In-Reply-To: <20170303120432.GA26540@gondor.apana.org.au>
Yup, when I disabled the s5p driver, xts DID show in the /proc/crypto list.
Heh, I was about to ask if it was something I should push towards
another maintainer for s5p stuff, but found you listed in that as
well.
If I am incorrect in that assumption, do let me know whom else I
should make aware of this issue.
Also let me know if you would like the rest of the kernel panic. Maybe
you already have enough to go on and don't need it.
Thanks for all that clarity.
On Fri, Mar 3, 2017 at 6:04 AM, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> On Fri, Mar 03, 2017 at 04:36:18AM -0600, Nathan Royce wrote:
>> I do have ECB selected as well:
>> DM_CRYPT=y
>> CRYPTO_ECB=y
>> CRYPTO_XTS=y
>>
>> name : ecb(aes)
>> driver : ecb-aes-s5p
>> module : kernel
>> priority : 100
>> refcnt : 1
>> selftest : passed
>> internal : no
>> type : ablkcipher
>> async : yes
>> blocksize : 16
>> min keysize : 16
>> max keysize : 32
>> ivsize : 0
>> geniv : <default>
>> //still no "xts" can be found in the list
>
> Weird. So you can't find any instances of xts in /proc/crypto
> at all? Even if the self-test fails it should still register an
> entry there...
>
> In any case, I think disabling the s5p driver should work at
> least.
>
> Cheers,
> --
> Email: Herbert Xu <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox