From: Evgeniy Polyakov <johnpol@2ka.mipt.ru>
To: netdev@oss.sgi.com
Cc: cryptoapi@lists.logix.cz
Subject: Asynchronous crypto layer.
Date: Fri, 29 Oct 2004 10:22:38 +0400 [thread overview]
Message-ID: <1099030958.4944.148.camel@uganda> (raw)
[-- Attachment #1.1: Type: text/plain, Size: 6248 bytes --]
I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
It support following features:
- multiple asynchronous crypto device queues
- crypto session routing
- crypto session binding
- modular load balancing
- crypto session batching genetically implemented by design
- crypto session priority
- different kinds of crypto operation(RNG, asymmetrical crypto, HMAC and
any other)
Some design notes:
acrypto has one main crypto session queue(double linked list, probably
it should be done like crypto_route or sk_buff queue), into which each
newly allocated session is inserted and this is a place where load
balancing searches it's food. When new session is being prepared for
insertion it calls load balancer's ->find_device() method, which should
return suitable device(current simple_lb load balancer returns device
with the lowest load(device has the least number of session in it's
queue)) if it exists. After crypto_device being returned acrypto creates
new crypto routing entry which points to returned device and adds it to
crypto session routing queue. Crypto session is being inserted into
device's queue according to it's priority and it is crypto device driver
that should process it's session list according to session's priority.
All insertion and deletion are guarded by appropriate locks, but
session_list traversing is not guarded in crypto_lb_thread() since
session can be removed _only_ from that function by design, so if crypto
device (atomically) marks session as completed and not being processed
and use list_for_each_safe() for traversing it's queue all should be OK.
Each crypto load balancer must implement 2 methods:
->rehash() and ->find_device() which will be called from any context and
under spinlock.
->rehash() method should be called to remix crypto sessions in device's
queues, for example if driver decides that it's device is broken it
marks itself as broken and load balancer(or scheduler if you like)
should remove all sessions from this queue to some other devices.
If session can not be completed scheduler must mark it as broken and
complete it(by calling first broke_session() and then complete_session()
and stop_process_session()). Consumer must check if operation was
successful(and therefore session is not broken).
->find_device() method should return appropriate crypto device.
For crypto session to be successfully allocated crypto consumer must
provide two structures - struct crypto_session_initializer
(hmm, why only one z?) and struct crypto_data.
struct crypto_session_initializer contains data needed to find
appropriate device, like type of operation, mode of operation, some
flags(for example SESSION_BINDED, which means that session must be bound
to specified in bdev field crypto device, it is useful for TCPA/TPM),
session priority and callback which will be called after all routing for
given session are finished.
struct crypto_data contains scatterlists for src, dst, key and iv.
It also has void *priv field and it's size which is allocated and may be
used by any crypto agent(for example VIA PadLock driver uses it to store
aes_ctx field, crypto_session can use this field to store some pointers
needed in ->callback()).
Actually callback will be called from queue_work, but I suppose it is
better to not assume calling context.
->callback() will be called after all crypto routing for given session
are done with the same parameters as were provided in initialisation
time(if session has only one routing callback will be called with
original parameters, but if it has several routes callback will be
called with parameters from the latest processed one). I believe crypto
callback should not know about crypto sessions, routings, device and so
on, proper restriction is always a good idea.
Crypto routing.
This feature allows the same session to be processed by several
devices/algorithms. For example if you need to encrypt data and then
sign it in TPM device you can create one route to encryption device and
then route it to TPM device. (Note: this feature must be discussed since
there is no time slice after session allocation, only in
crypto_device->data_ready() method and there are locking issues in
->callback() method).
Crypto device.
It can be either software emulator or hardware accelerator chip(like
HIFN 79*/83* or Via PadLock ACE/RNG, or even TPM device like each IBM
ThinkPad or some HP laptops have
(gentle hint: _they_ even have a _windows_ software for them :) )).
It can be registered with asynchronous crypto layer and must provide
some data for it:
->data_ready() method - it is called each time new session is added to
device's queue.
Array of struct crypto_capability and it's amount -
struct crypto_capability describes each operation given device can
handle, and has a maximum session queue length parameter.
Note: this structure can [be extended to] include "rate" parameter to
show absolute speed of given operation in some units, which therefore
can be used by scheduler(load balancer) for proper device selection.
Actually queue length can somehow reflects device's "speed".
Also attached:
- simple synchronous <-> asynchronous bridge for sha1 crypto provider.
It can be easily extended to handle whole number of synchronous software
implemented algorithms.
- driver for Via PadLock ACE - not tested, since my M10000 does not have
xcrypt module. This work is 98% by Michal Ludvig <mludvig suse.cz>, I
have just silently stolen it several days ago.
I've obtained bits of documentation for HIFN 795* so driver will be
created for it. I want to thank Wim Vandeputte <wim kd85.com> and
www.soekris.com for steps towards me in ordering and delivering process
to Russia.
Asynchronous crypto layer depends on kernel connector which was
presented in both linux-kernel@ and netdev@ several weeks ago.
Currently acrypto does not handle userspace session requests(for example
from OpenSSL) but I will implement it soon in form of char device and
it's ->mmap method.
Please review and comment, I am open for discussion.
Thank you.
--
Evgeniy Polyakov
Crash is better than data corruption. -- Art Grabowski
[-- Attachment #1.2: acrypto.h --]
[-- Type: text/x-chdr, Size: 7358 bytes --]
/*
* acrypto.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __ACRYPTO_H
#define __ACRYPTO_H
#define SCACHE_NAMELEN 32
struct crypto_session_initializer;
struct crypto_data;
typedef void (*crypto_callback_t)(struct crypto_session_initializer *, struct crypto_data *);
#if 0
#define SESSION_COMPLETED 0
#define SESSION_FINISHED 1
#define SESSION_STARTED 2
#define SESSION_PROCESSED 3
#define session_completed(s) (test_bit(SESSION_COMPLETED, (unsigned long *)&s->ci.flags))
#define complete_session(s) do { set_bit(SESSION_COMPLETED, (unsigned long *)&s->ci.flags); }while (0)
#define uncomplete_session(s) do { clear_bit(SESSION_COMPLETED, (unsigned long *)&s->ci.flags); } while (0)
#define session_started(s) (test_bit(SESSION_STARTED, (unsigned long *)&s->ci.flags))
#define start_session(s) do { set_bit(SESSION_STARTED, (unsigned long *)&s->ci.flags); } while (0)
#define unstart_session(s) do { clear_bit(SESSION_STARTED, (unsigned long *)&s->ci.flags); } while (0)
#define session_finished(s) (test_bit(SESSION_FINISHED, (unsigned long *)&s->ci.flags))
#define finish_session(s) do { set_bit(SESSION_FINISHED, (unsigned long *)&s->ci.flags); } while (0)
#define unfinish_session(s) do { clear_bit(SESSION_FINISHED, (unsigned long *)&s->ci.flags); } while (0)
#define session_is_processed(s) (test_bit(SESSION_PROCESSED, (unsigned long *)&s->ci.flags))
#define start_process_session(s) do { set_bit(SESSION_PROCESSED, (unsigned long *)&s->ci.flags); } while (0)
#define stop_process_session(s) do { clear_bit(SESSION_PROCESSED, (unsigned long *)&s->ci.flags); } while (0)
#else
#define SESSION_COMPLETED (1<<15)
#define SESSION_FINISHED (1<<14)
#define SESSION_STARTED (1<<13)
#define SESSION_PROCESSED (1<<12)
#define SESSION_BINDED (1<<11)
#define SESSION_BROKEN (1<<10)
#define session_completed(s) (s->ci.flags & SESSION_COMPLETED)
#define complete_session(s) do {s->ci.flags |= SESSION_COMPLETED;} while(0)
#define uncomplete_session(s) do {s->ci.flags &= ~SESSION_COMPLETED;} while (0)
#define session_finished(s) (s->ci.flags & SESSION_FINISHED)
#define finish_session(s) do {s->ci.flags |= SESSION_FINISHED;} while(0)
#define unfinish_session(s) do {s->ci.flags &= ~SESSION_FINISHED;} while (0)
#define session_started(s) (s->ci.flags & SESSION_STARTED)
#define start_session(s) do {s->ci.flags |= SESSION_STARTED;} while(0)
#define unstart_session(s) do {s->ci.flags &= ~SESSION_STARTED;} while (0)
#define session_is_processed(s) (s->ci.flags & SESSION_PROCESSED)
#define start_process_session(s) do {s->ci.flags |= SESSION_PROCESSED;} while(0)
#define stop_process_session(s) do {s->ci.flags &= ~SESSION_PROCESSED;} while (0)
#define session_binded(s) (s->ci.flags & SESSION_BINDED)
#define bind_session(s) do {s->ci.flags |= SESSION_BINDED;} while(0)
#define unbind_session(s) do {s->ci.flags &= ~SESSION_BINDED;} while (0)
#define sci_binded(ci) (ci->flags & SESSION_BINDED)
#define session_broken(s) (s->ci.flags & SESSION_BROKEN)
#define broke_session(s) do {s->ci.flags |= SESSION_BROKEN;} while(0)
#define unbroke_session(s) do {s->ci.flags &= ~SESSION_BROKEN;} while (0)
#endif
struct crypto_device_stat
{
__u64 scompleted;
__u64 sfinished;
__u64 sstarted;
__u64 kmem_failed;
};
#ifdef __KERNEL__
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/device.h>
#include <linux/workqueue.h>
#include <asm/scatterlist.h>
#define DEBUG
#ifdef DEBUG
#define dprintk(f, a...) printk(KERN_EMERG f, ##a)
#else
#define dprintk(f, a...)
#endif
#define CRYPTO_MAX_PRIV_SIZE 1024
#define DEVICE_BROKEN (1<<0)
#define device_broken(dev) (dev->flags & DEVICE_BROKEN)
#define broke_device(dev) do {dev->flags |= DEVICE_BROKEN;} while(0)
#define repair_device(dev) do {dev->flags &= ~DEVICE_BROKEN;} while(0)
struct crypto_capability
{
u16 operation;
u16 type;
u16 mode;
u16 qlen;
};
struct crypto_session_initializer
{
u16 operation;
u16 type;
u16 mode;
u16 priority;
u64 id;
u64 dev_id;
u32 flags;
u32 bdev;
crypto_callback_t callback;
};
struct crypto_data
{
int sg_src_num;
struct scatterlist sg_src;
struct scatterlist sg_dst;
struct scatterlist sg_key;
struct scatterlist sg_iv;
void *priv;
unsigned int priv_size;
};
struct crypto_device
{
char name[SCACHE_NAMELEN];
spinlock_t session_lock;
struct list_head session_list;
u64 sid;
spinlock_t lock;
atomic_t refcnt;
u32 flags;
u32 id;
struct list_head cdev_entry;
void (*data_ready)(struct crypto_device *);
struct device_driver *driver;
struct device device;
struct class_device class_device;
struct completion dev_released;
spinlock_t stat_lock;
struct crypto_device_stat stat;
struct crypto_capability *cap;
int cap_number;
};
struct crypto_route_head
{
struct crypto_route *next;
struct crypto_route *prev;
__u32 qlen;
spinlock_t lock;
};
struct crypto_route
{
struct crypto_route *next;
struct crypto_route *prev;
struct crypto_route_head *list;
struct crypto_device *dev;
struct crypto_session_initializer ci;
};
struct crypto_session
{
struct list_head dev_queue_entry;
struct list_head main_queue_entry;
struct crypto_session_initializer ci;
struct crypto_data data;
spinlock_t lock;
struct work_struct work;
struct crypto_route_head route_list;
};
struct crypto_session *crypto_session_alloc(struct crypto_session_initializer *, struct crypto_data *);
inline void crypto_session_dequeue_main(struct crypto_session *);
inline void __crypto_session_dequeue_main(struct crypto_session *);
void crypto_session_dequeue_route(struct crypto_session *);
inline void crypto_device_get(struct crypto_device *);
inline void crypto_device_put(struct crypto_device *);
inline struct crypto_device *crypto_device_get_name(char *);
int __crypto_device_add(struct crypto_device *);
int crypto_device_add(struct crypto_device *);
void __crypto_device_remove(struct crypto_device *);
void crypto_device_remove(struct crypto_device *);
int match_initializer(struct crypto_device *, struct crypto_session_initializer *);
inline void crypto_session_insert_main(struct crypto_device *dev, struct crypto_session *s);
inline void crypto_session_insert(struct crypto_device *dev, struct crypto_session *s);
#endif /* __KERNEL__ */
#endif /* __ACRYPTO_H */
[-- Attachment #1.3: consumer.c --]
[-- Type: text/x-csrc, Size: 5584 bytes --]
/*
* consumer.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/err.h>
#include <linux/mm.h>
#include <linux/completion.h>
#include "acrypto.h"
#include "crypto_def.h"
#include "via-padlock/padlock.h"
#define KEY_LENGTH 16
static char ckey[KEY_LENGTH];
static int key_length = sizeof(ckey);
static void ctest_callback(struct crypto_session_initializer *ci, struct crypto_data *data);
static struct crypto_session_initializer ci =
{
.operation = CRYPTO_OP_ENCRYPT,
.type = CRYPTO_TYPE_AES_128,
.mode = CRYPTO_MODE_ECB,
.priority = 4,
.callback = ctest_callback,
};
static struct crypto_data cdata;
#define CSESSION_MAX 1000
static struct crypto_session *s;
static int watermark;
static struct completion callback_completed;
static void ctest_callback(struct crypto_session_initializer *ci, struct crypto_data *data)
{
int i, off, size, ssize;
struct scatterlist *sg;
unsigned char *ptr;
watermark--;
dprintk("%s: session %llu [%llu]\n", __func__, ci->id, ci->dev_id);
dprintk("src=%s, len=%d.\n", (char *)page_address(data->sg_src.page), data->sg_src.length);
sg = &data->sg_key;
ptr = (unsigned char *)page_address(sg->page);
off = sg->offset;
size = sg->length;
printk("key[%d]=", size);
for (i=0; i<size; ++i)
printk("0x%02x, ", ptr[i+off]);
printk("\n");
sg = &data->sg_src;
ptr = (unsigned char *)page_address(sg->page);
off = sg->offset;
ssize = size = sg->length;
printk("src[%d]=", size);
for (i=0; i<size; ++i)
printk("0x%02x, ", ptr[i+off]);
printk("\n");
sg = &data->sg_dst;
ptr = (unsigned char *)page_address(sg->page);
off = sg->offset;
size = sg->length;
if (size == 0)
{
dprintk("size=5, setting to %d.\n", ssize);
size = ssize;
}
printk("dst[%d]=", size);
for (i=0; i<size; ++i)
printk("0x%02x, ", ptr[i+off]);
printk("\n");
complete(&callback_completed);
}
int ctimer_func(void *data, int size, int op)
{
u8 *ptr;
if (size > PAGE_SIZE)
size = PAGE_SIZE;
ptr = (u8 *)page_address(cdata.sg_src.page);
memcpy(ptr, data, size);
cdata.sg_src.length = size;
ci.operation = op;
s = crypto_session_alloc(&ci, &cdata);
if (s)
watermark++;
else
dprintk("allocation failed.\n");
return (s)?0:-EINVAL;
}
static int alloc_sg(struct scatterlist *sg, void *data, int size)
{
sg->offset = 0;
sg->page = alloc_pages(GFP_KERNEL, get_order(size));
if (!sg->page) {
printk(KERN_ERR "Failed to allocate page.\n");
return -ENOMEM;
}
if (data)
memcpy(sg->page, data, size);
sg->length = size;
return 0;
}
int consumer_init(void)
{
int err;
char str[] = "test message qwerty asdzxc\n";
int size;
init_completion(&callback_completed);
cdata.sg_src_num = 1;
err = alloc_sg(&cdata.sg_src, NULL, PAGE_SIZE);
if (err)
goto err_out_return;
err = alloc_sg(&cdata.sg_dst, NULL, PAGE_SIZE);
if (err)
goto err_out_src;
err = alloc_sg(&cdata.sg_key, ckey, key_length);
if (err)
goto err_out_dst;
err = alloc_sg(&cdata.sg_iv, NULL, PAGE_SIZE);
if (err)
goto err_out_key;
cdata.priv_size = sizeof(struct aes_ctx);
size = 32;
err = ctimer_func(str, size, CRYPTO_OP_ENCRYPT);
if (err)
goto err_out_iv;
wait_for_completion(&callback_completed);
init_completion(&callback_completed);
err = ctimer_func(page_address(cdata.sg_dst.page), cdata.sg_dst.length, CRYPTO_OP_DECRYPT);
if (err)
goto err_out_iv;
wait_for_completion(&callback_completed);
err_out_iv:
__free_pages(cdata.sg_iv.page, get_order(1));
err_out_key:
__free_pages(cdata.sg_key.page, get_order(cdata.sg_key.length));
err_out_dst:
__free_pages(cdata.sg_dst.page, get_order(1));
err_out_src:
__free_pages(cdata.sg_src.page, get_order(1));
err_out_return:
return -ENODEV;
}
void consumer_fini(void)
{
while(watermark)
{
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(HZ);
printk(KERN_INFO "Waiting for sessions to be freed: watermark=%d.\n",
watermark);
}
if (!cdata.sg_key.length)
printk("BUG: key length is 0 in %s.\n", __func__);
__free_pages(cdata.sg_key.page, get_order(key_length));
__free_pages(cdata.sg_dst.page, get_order(1));
__free_pages(cdata.sg_src.page, get_order(1));
printk(KERN_INFO "Test crypto module consumer is unloaded.\n");
}
module_init(consumer_init);
module_exit(consumer_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Test crypto module consumer.");
[-- Attachment #1.4: crypto_conn.c --]
[-- Type: text/x-csrc, Size: 5120 bytes --]
/*
* crypto_conn.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/vmalloc.h>
#include "acrypto.h"
#include "crypto_lb.h"
#include "../connector/connector.h"
#include "crypto_conn.h"
static struct cb_id crypto_conn_id = { 0xdead, 0x0000 };
static char crypto_conn_name[] = "crconn";
static void crypto_conn_callback(void *data)
{
struct cn_msg *msg, *reply;
struct crypto_conn_data *d, *cmd;
struct crypto_device *dev;
u32 sessions;
msg = (struct cn_msg *)data;
d = (struct crypto_conn_data *)msg->data;
if (msg->len < sizeof(*d))
{
dprintk(KERN_ERR "Wrong message to crypto connector: msg->len=%u < %u.\n",
msg->len, sizeof(*d));
return;
}
if (msg->len != sizeof(*d) + d->len)
{
dprintk(KERN_ERR "Wrong message to crypto connector: msg->len=%u != %u.\n",
msg->len, sizeof(*d) + d->len);
return;
}
dev = crypto_device_get_name(d->name);
if (!dev)
{
dprintk(KERN_INFO "Crypto device %s was not found.\n", d->name);
return;
}
switch (d->cmd)
{
case CRYPTO_CONN_READ_SESSIONS:
reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + sizeof(sessions), GFP_ATOMIC);
if (reply) {
memcpy(reply, msg, sizeof(*reply));
reply->len = sizeof(*cmd) + sizeof(sessions);
/*
* See protocol description in connector.c
*/
reply->ack++;
cmd = (struct crypto_conn_data *)(reply + 1);
memcpy(cmd, d, sizeof(*cmd));
cmd->len = sizeof(sessions);
sessions = atomic_read(&dev->refcnt);
memcpy(cmd+1, &sessions, sizeof(sessions));
cn_netlink_send(reply, 0);
kfree(reply);
} else
dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
sizeof(*msg) + sizeof(*cmd), d->cmd);
break;
case CRYPTO_CONN_DUMP_QUEUE:
reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + 1024*sizeof(struct crypto_session_initializer), GFP_ATOMIC);
if (reply) {
struct crypto_session *s;
struct crypto_session_initializer *ptr;
memcpy(reply, msg, sizeof(*reply));
/*
* See protocol description in connector.c
*/
reply->ack++;
cmd = (struct crypto_conn_data *)(reply + 1);
memcpy(cmd, d, sizeof(*cmd));
ptr = (struct crypto_session_initializer *)(cmd+1);
sessions = 0;
spin_lock_bh(&dev->session_lock);
list_for_each_entry(s, &dev->session_list, dev_queue_entry)
{
memcpy(ptr, &s->ci, sizeof(*ptr));
sessions++;
ptr++;
if (sessions >= 1024)
break;
}
spin_unlock_bh(&dev->session_lock);
cmd->len = sizeof(*ptr)*sessions;
reply->len = sizeof(*cmd) + cmd->len;
cn_netlink_send(reply, 0);
kfree(reply);
} else
dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
sizeof(*msg) + sizeof(*cmd), d->cmd);
break;
case CRYPTO_GET_STAT:
reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + sizeof(struct crypto_device_stat), GFP_ATOMIC);
if (reply) {
struct crypto_device_stat *ptr;
memcpy(reply, msg, sizeof(*reply));
reply->len = sizeof(*cmd) + sizeof(*ptr);
/*
* See protocol description in connector.c
*/
reply->ack++;
cmd = (struct crypto_conn_data *)(reply + 1);
memcpy(cmd, d, sizeof(*cmd));
cmd->len = sizeof(*ptr);
ptr = (struct crypto_device_stat *)(cmd+1);
memcpy(ptr, &dev->stat, sizeof(*ptr));
cn_netlink_send(reply, 0);
kfree(reply);
} else
dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
sizeof(*msg) + sizeof(*cmd), d->cmd);
break;
default:
dprintk(KERN_ERR "Wrong operation 0x%04x for crypto connector.\n",
d->cmd);
return;
}
crypto_device_put(dev);
}
int crypto_conn_init(void)
{
int err;
err = cn_add_callback(&crypto_conn_id, crypto_conn_name, crypto_conn_callback);
if (err)
return err;
dprintk(KERN_INFO "Crypto connector callback is registered.\n");
return 0;
}
void crypto_conn_fini(void)
{
cn_del_callback(&crypto_conn_id);
dprintk(KERN_INFO "Crypto connector callback is unregistered.\n");
}
[-- Attachment #1.5: crypto_conn.h --]
[-- Type: text/x-chdr, Size: 1261 bytes --]
/*
* crypto_conn.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __CRYPTO_CONN_H
#define __CRYPTO_CONN_H
#include "acrypto.h"
#define CRYPTO_CONN_READ_SESSIONS 0
#define CRYPTO_CONN_DUMP_QUEUE 1
#define CRYPTO_GET_STAT 2
struct crypto_conn_data
{
char name[SCACHE_NAMELEN];
__u16 cmd;
__u16 len;
__u8 data[0];
};
#ifdef __KERNEL__
int crypto_conn_init(void);
void crypto_conn_fini(void);
#endif /* __KERNEL__ */
#endif /* __CRYPTO_CONN_H */
[-- Attachment #1.6: crypto_def.h --]
[-- Type: text/x-chdr, Size: 1203 bytes --]
/*
* crypto_def.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __CRYPTO_DEF_H
#define __CRYPTO_DEF_H
#define CRYPTO_OP_DECRYPT 0
#define CRYPTO_OP_ENCRYPT 1
#define CRYPTO_OP_HMAC 2
#define CRYPTO_MODE_ECB 0
#define CRYPTO_MODE_CBC 1
#define CRYPTO_MODE_CFB 2
#define CRYPTO_MODE_OFB 3
#define CRYPTO_TYPE_AES_128 0
#define CRYPTO_TYPE_AES_192 1
#define CRYPTO_TYPE_AES_256 2
#endif /* __CRYPTO_DEF_H */
[-- Attachment #1.7: crypto_dev.c --]
[-- Type: text/x-csrc, Size: 10412 bytes --]
/*
* crypto_dev.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/device.h>
#include "acrypto.h"
static LIST_HEAD(cdev_list);
static spinlock_t cdev_lock = SPIN_LOCK_UNLOCKED;
static u32 cdev_ids;
struct list_head *crypto_device_list = &cdev_list;
spinlock_t *crypto_device_lock = &cdev_lock;
static int crypto_match(struct device *dev, struct device_driver *drv)
{
return 1;
}
static int crypto_probe(struct device *dev)
{
return -ENODEV;
}
static int crypto_remove(struct device *dev)
{
return 0;
}
static void crypto_release(struct device *dev)
{
struct crypto_device *d = container_of(dev, struct crypto_device, device);
complete(&d->dev_released);
}
static void crypto_class_release(struct class *class)
{
}
static void crypto_class_release_device(struct class_device *class_dev)
{
}
struct class crypto_class =
{
.name = "acrypto",
.class_release = crypto_class_release,
.release = crypto_class_release_device
};
struct bus_type crypto_bus_type =
{
.name = "acrypto",
.match = crypto_match
};
struct device_driver crypto_driver = {
.name = "crypto_driver",
.bus = &crypto_bus_type,
.probe = crypto_probe,
.remove = crypto_remove,
};
struct device crypto_dev = {
.parent = NULL,
.bus = &crypto_bus_type,
.bus_id = "Asynchronous crypto",
.driver = &crypto_driver,
.release = &crypto_release
};
static ssize_t sessions_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%d\n", atomic_read(&d->refcnt));
}
static ssize_t name_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%s\n", d->name);
}
static ssize_t devices_show(struct class_device *dev, char *buf)
{
struct crypto_device *d;
int off = 0;
spin_lock_bh(&cdev_lock);
list_for_each_entry(d, &cdev_list, cdev_entry)
{
off += sprintf(buf+off, "%s ", d->name);
}
spin_unlock_bh(&cdev_lock);
if (!off)
off = sprintf(buf, "No devices registered yet.");
off += sprintf(buf+off, "\n");
return off;
}
static ssize_t kmem_failed_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%llu\n", d->stat.kmem_failed);
}
static ssize_t sstarted_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%llu\n", d->stat.sstarted);
}
static ssize_t sfinished_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%llu\n", d->stat.sfinished);
}
static ssize_t scompleted_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%llu\n", d->stat.scompleted);
}
static CLASS_DEVICE_ATTR(sessions, 0444, sessions_show, NULL);
static CLASS_DEVICE_ATTR(name, 0444, name_show, NULL);
CLASS_DEVICE_ATTR(devices, 0444, devices_show, NULL);
static CLASS_DEVICE_ATTR(scompleted, 0444, scompleted_show, NULL);
static CLASS_DEVICE_ATTR(sstarted, 0444, sstarted_show, NULL);
static CLASS_DEVICE_ATTR(sfinished, 0444, sfinished_show, NULL);
static CLASS_DEVICE_ATTR(kmem_failed, 0444, kmem_failed_show, NULL);
static inline int compare_device(struct crypto_device *d1, struct crypto_device *d2)
{
if (!strncmp(d1->name, d2->name, sizeof(d1->name)))
return 1;
return 0;
}
static void create_device_attributes(struct crypto_device *dev)
{
class_device_create_file(&dev->class_device, &class_device_attr_sessions);
class_device_create_file(&dev->class_device, &class_device_attr_name);
class_device_create_file(&dev->class_device, &class_device_attr_scompleted);
class_device_create_file(&dev->class_device, &class_device_attr_sstarted);
class_device_create_file(&dev->class_device, &class_device_attr_sfinished);
class_device_create_file(&dev->class_device, &class_device_attr_kmem_failed);
}
static void remove_device_attributes(struct crypto_device *dev)
{
class_device_remove_file(&dev->class_device, &class_device_attr_sessions);
class_device_remove_file(&dev->class_device, &class_device_attr_name);
class_device_remove_file(&dev->class_device, &class_device_attr_scompleted);
class_device_remove_file(&dev->class_device, &class_device_attr_sstarted);
class_device_remove_file(&dev->class_device, &class_device_attr_sfinished);
class_device_remove_file(&dev->class_device, &class_device_attr_kmem_failed);
}
static int __match_initializer(struct crypto_capability *cap, struct crypto_session_initializer *ci)
{
if ( cap->operation == ci->operation &&
cap->type == ci->type &&
cap->mode == (ci->mode & 0x1fff))
return 1;
return 0;
}
int match_initializer(struct crypto_device *dev, struct crypto_session_initializer *ci)
{
int i;
for (i=0; i<dev->cap_number; ++i)
{
struct crypto_capability *cap = &dev->cap[i];
if (__match_initializer(cap, ci))
{
if (cap->qlen >= atomic_read(&dev->refcnt) + 1)
return 1;
}
}
return 0;
}
inline void crypto_device_get(struct crypto_device *dev)
{
atomic_inc(&dev->refcnt);
}
inline struct crypto_device *crypto_device_get_name(char *name)
{
struct crypto_device *dev;
int found = 0;
spin_lock_bh(&cdev_lock);
list_for_each_entry(dev, &cdev_list, cdev_entry)
{
if (!strcmp(dev->name, name))
{
found = 1;
crypto_device_get(dev);
break;
}
}
spin_unlock_bh(&cdev_lock);
if (!found)
return NULL;
return dev;
}
inline void crypto_device_put(struct crypto_device *dev)
{
atomic_dec(&dev->refcnt);
}
int __crypto_device_add(struct crypto_device *dev)
{
int err;
memset(&dev->stat, 0, sizeof(dev->stat));
spin_lock_init(&dev->stat_lock);
spin_lock_init(&dev->lock);
spin_lock_init(&dev->session_lock);
INIT_LIST_HEAD(&dev->session_list);
atomic_set(&dev->refcnt, 0);
dev->sid = 0;
dev->flags = 0;
init_completion(&dev->dev_released);
memcpy(&dev->device, &crypto_dev, sizeof(struct device));
dev->driver = &crypto_driver;
snprintf(dev->device.bus_id, sizeof(dev->device.bus_id), "%s", dev->name);
err = device_register(&dev->device);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto device %s: err=%d.\n",
dev->name, err);
return err;
}
snprintf(dev->class_device.class_id, sizeof(dev->class_device.class_id), "%s", dev->name);
dev->class_device.dev = &dev->device;
dev->class_device.class = &crypto_class;
err = class_device_register(&dev->class_device);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto class device %s: err=%d.\n",
dev->name, err);
device_unregister(&dev->device);
return err;
}
create_device_attributes(dev);
return 0;
}
void __crypto_device_remove(struct crypto_device *dev)
{
remove_device_attributes(dev);
class_device_unregister(&dev->class_device);
device_unregister(&dev->device);
}
int crypto_device_add(struct crypto_device *dev)
{
int err;
err = __crypto_device_add(dev);
if (err)
return err;
spin_lock_bh(&cdev_lock);
list_add(&dev->cdev_entry, &cdev_list);
dev->id = ++cdev_ids;
spin_unlock_bh(&cdev_lock);
dprintk(KERN_INFO "Crypto device %s was registered with ID=%x.\n", dev->name, dev->id);
return 0;
}
void crypto_device_remove(struct crypto_device *dev)
{
struct crypto_device *__dev, *n;
unsigned long flags;
__crypto_device_remove(dev);
spin_lock_irqsave(&cdev_lock, flags);
list_for_each_entry_safe(__dev, n, &cdev_list, cdev_entry)
{
if (compare_device(__dev, dev))
{
list_del(&__dev->cdev_entry);
spin_unlock_irqrestore(&cdev_lock, flags);
/*
* In test cases or when crypto device driver is not written correctly,
* it's ->data_ready() method will not be callen anymore
* after device is removed from crypto device list.
*
* For such cases we either should provide ->flush() call
* or properly write ->data_ready() method.
*/
while (atomic_read(&__dev->refcnt))
{
dprintk(KERN_INFO "Waiting for %s to become free: refcnt=%d.\n",
__dev->name, atomic_read(&dev->refcnt));
/*
* Hack zone: you need to write good ->data_ready()
* and crypto device driver itself.
*
* Driver shoud not buzz if it has pending sessions
* in it's queue and it was removed from global device list.
*
* Although I can workaround it here, for example by
* flushing the whole queue and drop all pending sessions.
*/
__dev->data_ready(__dev);
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(HZ);
}
dprintk(KERN_ERR "Crypto device %s was unregistered.\n", dev->name);
return;
}
}
spin_unlock_irqrestore(&cdev_lock, flags);
dprintk(KERN_ERR "Crypto device %s was not registered.\n", dev->name);
}
EXPORT_SYMBOL(crypto_device_add);
EXPORT_SYMBOL(crypto_device_remove);
EXPORT_SYMBOL(crypto_device_get);
EXPORT_SYMBOL(crypto_device_get_name);
EXPORT_SYMBOL(crypto_device_put);
EXPORT_SYMBOL(match_initializer);
[-- Attachment #1.8: crypto_lb.c --]
[-- Type: text/x-csrc, Size: 15618 bytes --]
/*
* crypto_lb.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/err.h>
#include "acrypto.h"
#include "crypto_lb.h"
#include "crypto_stat.h"
#include "crypto_route.h"
static LIST_HEAD(crypto_lb_list);
static spinlock_t crypto_lb_lock = SPIN_LOCK_UNLOCKED;
static int lb_num = 0;
static struct crypto_lb *current_lb, *default_lb;
static struct completion thread_exited;
static int need_exit;
static struct workqueue_struct *crypto_lb_queue;
static DECLARE_WAIT_QUEUE_HEAD(crypto_lb_wait_queue);
extern struct list_head *crypto_device_list;
extern spinlock_t *crypto_device_lock;
extern int force_lb_remove;
extern struct crypto_device main_crypto_device;
static inline int lb_is_current(struct crypto_lb *l)
{
return (l->crypto_device_list != NULL && l->crypto_device_lock != NULL);
}
static inline int lb_is_default(struct crypto_lb *l)
{
return (l == default_lb);
}
static inline void __lb_set_current(struct crypto_lb *l)
{
struct crypto_lb *c = current_lb;
if (c)
{
l->crypto_device_list = crypto_device_list;
l->crypto_device_lock = crypto_device_lock;
current_lb = l;
c->crypto_device_list = NULL;
c->crypto_device_lock = NULL;
}
else
{
l->crypto_device_list = crypto_device_list;
l->crypto_device_lock = crypto_device_lock;
current_lb = l;
}
}
static inline void lb_set_current(struct crypto_lb *l)
{
struct crypto_lb *c = current_lb;
if (c)
{
spin_lock_bh(&c->lock);
__lb_set_current(l);
spin_unlock_bh(&c->lock);
}
else
__lb_set_current(l);
}
static inline void __lb_set_default(struct crypto_lb *l)
{
default_lb = l;
}
static inline void lb_set_default(struct crypto_lb *l)
{
struct crypto_lb *c = default_lb;
if (c)
{
spin_lock_bh(&c->lock);
__lb_set_default(l);
spin_unlock_bh(&c->lock);
}
else
__lb_set_default(l);
}
static int crypto_lb_match(struct device *dev, struct device_driver *drv)
{
return 1;
}
static int crypto_lb_probe(struct device *dev)
{
return -ENODEV;
}
static int crypto_lb_remove(struct device *dev)
{
return 0;
}
static void crypto_lb_release(struct device *dev)
{
struct crypto_lb *d = container_of(dev, struct crypto_lb, device);
complete(&d->dev_released);
}
static void crypto_lb_class_release(struct class *class)
{
}
static void crypto_lb_class_release_device(struct class_device *class_dev)
{
}
struct class crypto_lb_class =
{
.name = "crypto_lb",
.class_release = crypto_lb_class_release,
.release = crypto_lb_class_release_device
};
struct bus_type crypto_lb_bus_type =
{
.name = "crypto_lb",
.match = crypto_lb_match
};
struct device_driver crypto_lb_driver = {
.name = "crypto_lb_driver",
.bus = &crypto_lb_bus_type,
.probe = crypto_lb_probe,
.remove = crypto_lb_remove,
};
struct device crypto_lb_dev = {
.parent = NULL,
.bus = &crypto_lb_bus_type,
.bus_id = "crypto load balancer",
.driver = &crypto_lb_driver,
.release = &crypto_lb_release
};
static ssize_t name_show(struct class_device *dev, char *buf)
{
struct crypto_lb *lb = container_of(dev, struct crypto_lb, class_device);
return sprintf(buf, "%s\n", lb->name);
}
static ssize_t current_show(struct class_device *dev, char *buf)
{
struct crypto_lb *lb;
int off = 0;
spin_lock_bh(&crypto_lb_lock);
list_for_each_entry(lb, &crypto_lb_list, lb_entry)
{
if (lb_is_current(lb))
off += sprintf(buf+off, "[");
if (lb_is_default(lb))
off += sprintf(buf+off, "(");
off += sprintf(buf+off, "%s", lb->name);
if (lb_is_default(lb))
off += sprintf(buf+off, ")");
if (lb_is_current(lb))
off += sprintf(buf+off, "]");
}
spin_unlock_bh(&crypto_lb_lock);
if (!off)
off = sprintf(buf, "No load balancers regitered yet.");
off += sprintf(buf+off, "\n");
return off;
}
static ssize_t current_store(struct class_device *dev, const char *buf, size_t count)
{
struct crypto_lb *lb;
spin_lock_bh(&crypto_lb_lock);
list_for_each_entry(lb, &crypto_lb_list, lb_entry)
{
if (count == strlen(lb->name) && !strcmp(buf, lb->name))
{
lb_set_current(lb);
lb_set_default(lb);
dprintk(KERN_INFO "Load balancer %s is set as current and default.\n",
lb->name);
break;
}
}
spin_unlock_bh(&crypto_lb_lock);
return count;
}
static CLASS_DEVICE_ATTR(name, 0444, name_show, NULL);
CLASS_DEVICE_ATTR(lbs, 0644, current_show, current_store);
static void create_device_attributes(struct crypto_lb *lb)
{
class_device_create_file(&lb->class_device, &class_device_attr_name);
}
static void remove_device_attributes(struct crypto_lb *lb)
{
class_device_remove_file(&lb->class_device, &class_device_attr_name);
}
static inline int compare_lb(struct crypto_lb *l1, struct crypto_lb *l2)
{
if (!strncmp(l1->name, l2->name, sizeof(l1->name)))
return 1;
return 0;
}
inline void crypto_lb_rehash(void)
{
if (!current_lb)
return;
spin_lock_bh(¤t_lb->lock);
current_lb->rehash(current_lb);
spin_unlock_bh(¤t_lb->lock);
wake_up_interruptible(&crypto_lb_wait_queue);
}
struct crypto_device *crypto_lb_find_device(struct crypto_session_initializer *ci)
{
struct crypto_device *dev;
if (!current_lb)
return NULL;
if (sci_binded(ci))
{
int found = 0;
spin_lock_bh(crypto_device_lock);
list_for_each_entry(dev, crypto_device_list, cdev_entry)
{
if (dev->id == ci->bdev)
{
found = 1;
break;
}
}
spin_unlock_bh(crypto_device_lock);
return (found)?dev:NULL;
}
spin_lock_bh(¤t_lb->lock);
current_lb->rehash(current_lb);
spin_lock(crypto_device_lock);
dev = current_lb->find_device(current_lb, ci);
if (dev)
crypto_device_get(dev);
spin_unlock(crypto_device_lock);
spin_unlock_bh(¤t_lb->lock);
wake_up_interruptible(&crypto_lb_wait_queue);
return dev;
}
static int __crypto_lb_register(struct crypto_lb *lb)
{
int err;
spin_lock_init(&lb->lock);
init_completion(&lb->dev_released);
memcpy(&lb->device, &crypto_lb_dev, sizeof(struct device));
lb->driver = &crypto_lb_driver;
snprintf(lb->device.bus_id, sizeof(lb->device.bus_id), "%s", lb->name);
err = device_register(&lb->device);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto load balancer device %s: err=%d.\n",
lb->name, err);
return err;
}
snprintf(lb->class_device.class_id, sizeof(lb->class_device.class_id), "%s", lb->name);
lb->class_device.dev = &lb->device;
lb->class_device.class = &crypto_lb_class;
err = class_device_register(&lb->class_device);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto load balancer class device %s: err=%d.\n",
lb->name, err);
device_unregister(&lb->device);
return err;
}
create_device_attributes(lb);
wake_up_interruptible(&crypto_lb_wait_queue);
return 0;
}
static void __crypto_lb_unregister(struct crypto_lb *lb)
{
wake_up_interruptible(&crypto_lb_wait_queue);
remove_device_attributes(lb);
class_device_unregister(&lb->class_device);
device_unregister(&lb->device);
}
int crypto_lb_register(struct crypto_lb *lb, int set_current, int set_default)
{
struct crypto_lb *__lb;
int err;
spin_lock_bh(&crypto_lb_lock);
list_for_each_entry(__lb, &crypto_lb_list, lb_entry)
{
if (unlikely(compare_lb(__lb, lb)))
{
spin_unlock_bh(&crypto_lb_lock);
dprintk(KERN_ERR "Crypto load balancer %s is already registered.\n", lb->name);
return -EINVAL;
}
}
list_add(&lb->lb_entry, &crypto_lb_list);
spin_unlock_bh(&crypto_lb_lock);
err = __crypto_lb_register(lb);
if (err)
{
spin_lock_bh(&crypto_lb_lock);
list_del(&lb->lb_entry);
spin_unlock_bh(&crypto_lb_lock);
return err;
}
if (!default_lb || set_default)
lb_set_default(lb);
if (!current_lb || set_current)
lb_set_current(lb);
dprintk(KERN_INFO "Crypto load balancer %s was registered and set to be [%s.%s].\n",
lb->name,
(lb_is_current(lb))?"current":"not current",
(lb_is_default(lb))?"default":"not default");
lb_num++;
return 0;
}
void crypto_lb_unregister(struct crypto_lb *lb)
{
struct crypto_lb *__lb, *n;
if (lb_num == 1)
{
dprintk(KERN_INFO "You are removing crypto load balancer %s which is current and default.\n"
"There is no other crypto load balancers. "
"Removing %s delayed untill new load balancer is registered.\n",
lb->name, (force_lb_remove)?"is not":"is");
while (lb_num == 1 && !force_lb_remove)
{
set_current_state(TASK_INTERRUPTIBLE);
schedule_timeout(HZ);
if (signal_pending(current))
flush_signals(current);
}
}
__crypto_lb_unregister(lb);
spin_lock_bh(&crypto_lb_lock);
list_for_each_entry_safe(__lb, n, &crypto_lb_list, lb_entry)
{
if (compare_lb(__lb, lb))
{
lb_num--;
list_del(&__lb->lb_entry);
dprintk(KERN_ERR "Crypto load balancer %s was unregistered.\n", lb->name);
}
else if (lb_num)
{
if (lb_is_default(lb))
lb_set_default(__lb);
if (lb_is_current(lb))
lb_set_current(default_lb);
}
}
spin_unlock_bh(&crypto_lb_lock);
}
static void crypto_lb_queue_wrapper(void *data)
{
struct crypto_device *dev = &main_crypto_device;
struct crypto_session *s = (struct crypto_session *)data;
dprintk(KERN_INFO "%s: Calling callback for session %llu [%llu] flags=%x, "
"op=%04u, type=%04x, mode=%04x, priority=%04x\n",
__func__,
s->ci.id, s->ci.dev_id, s->ci.flags,
s->ci.operation, s->ci.type, s->ci.mode, s->ci.priority);
/*
* Prevent races with crypto devices
* which may change flags of the sessions in theirs queues.
*/
spin_lock(&s->lock);
crypto_stat_finish_inc(s);
finish_session(s);
unstart_session(s);
spin_unlock(&s->lock);
s->ci.callback(&s->ci, &s->data);
if (session_finished(s))
{
crypto_session_dequeue_route(s);
kfree(s);
}
else
{
/*
* Special case: crypto consumer marks session as "not finished"
* in it's callback - it means that crypto consumer wants
* this session to be processed further,
* for example crypto consumer can add new route and then
* mark session as "not finished".
*/
uncomplete_session(s);
unstart_session(s);
crypto_session_insert_main(dev, s);
}
wake_up_interruptible(&crypto_lb_wait_queue);
}
static void crypto_lb_process_next_route(struct crypto_session *s)
{
struct crypto_route *rt;
struct crypto_device *dev;
rt = crypto_route_dequeue(s);
if (rt)
{
dev = rt->dev;
spin_lock_bh(&dev->session_lock);
list_del(&s->dev_queue_entry);
spin_unlock_bh(&dev->session_lock);
crypto_route_free(rt);
dev = crypto_route_get_current_device(s);
if (dev)
{
dprintk(KERN_INFO "%s: processing new route to %s.\n",
__func__, dev->name);
memcpy(&s->ci, &rt->ci, sizeof(s->ci));
crypto_session_insert(dev, s);
/*
* Reference to this device was already hold when
* new routing was added.
*/
crypto_device_put(dev);
}
}
}
int crypto_lb_thread(void *data)
{
struct crypto_session *s, *n;
struct crypto_device *dev = (struct crypto_device *)data;
daemonize("%s", dev->name);
allow_signal(SIGTERM);
while (!need_exit)
{
spin_lock_bh(&dev->session_lock);
list_for_each_entry_safe(s, n, &dev->session_list, main_queue_entry)
{
printk("session %llu [%llu]: flags=%x, route_num=%d, %s,%s,%s,%s.\n",
s->ci.id, s->ci.dev_id, s->ci.flags,
crypto_route_queue_len(s),
(session_completed(s))?"completed": "not completed",
(session_finished(s))?"finished": "not finished",
(session_started(s))?"started": "not started",
(session_is_processed(s))?"is being processed": "is not being processed");
if (!spin_trylock(&s->lock))
continue;
if (session_is_processed(s))
goto unlock;
if (session_started(s))
goto unlock;
if (session_completed(s))
{
uncomplete_session(s);
if (crypto_route_queue_len(s) > 1)
{
crypto_lb_process_next_route(s);
}
else
{
start_session(s);
crypto_stat_start_inc(s);
__crypto_session_dequeue_main(s);
spin_unlock(&s->lock);
INIT_WORK(&s->work, &crypto_lb_queue_wrapper, s);
queue_work(crypto_lb_queue, &s->work);
continue;
}
}
unlock:
spin_unlock(&s->lock);
}
spin_unlock_bh(&dev->session_lock);
interruptible_sleep_on_timeout(&crypto_lb_wait_queue, 1000);
}
flush_workqueue(crypto_lb_queue);
complete_and_exit(&thread_exited, 0);
}
int crypto_lb_init(void)
{
int err;
long pid;
err = bus_register(&crypto_lb_bus_type);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto load balancer bus: err=%d.\n", err);
goto err_out_exit;
}
err = driver_register(&crypto_lb_driver);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto load balancer driver: err=%d.\n", err);
goto err_out_bus_unregister;
}
crypto_lb_class.class_dev_attrs = &class_device_attr_lbs;
err = class_register(&crypto_lb_class);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto load balancer class: err=%d.\n", err);
goto err_out_driver_unregister;
}
crypto_lb_queue = create_workqueue("clbq");
if (!crypto_lb_queue)
{
dprintk(KERN_ERR "Failed to create crypto load balaner work queue.\n");
goto err_out_class_unregister;
}
init_completion(&thread_exited);
pid = kernel_thread(crypto_lb_thread, &main_crypto_device, CLONE_FS | CLONE_FILES);
if (IS_ERR((void *)pid))
{
dprintk(KERN_ERR "Failed to create kernel load balancing thread.\n");
goto err_out_destroy_workqueue;
}
return 0;
err_out_destroy_workqueue:
destroy_workqueue(crypto_lb_queue);
err_out_class_unregister:
class_unregister(&crypto_lb_class);
err_out_driver_unregister:
driver_unregister(&crypto_lb_driver);
err_out_bus_unregister:
bus_unregister(&crypto_lb_bus_type);
err_out_exit:
return err;
}
void crypto_lb_fini(void)
{
need_exit = 1;
wait_for_completion(&thread_exited);
destroy_workqueue(crypto_lb_queue);
class_unregister(&crypto_lb_class);
driver_unregister(&crypto_lb_driver);
bus_unregister(&crypto_lb_bus_type);
}
EXPORT_SYMBOL(crypto_lb_register);
EXPORT_SYMBOL(crypto_lb_unregister);
EXPORT_SYMBOL(crypto_lb_rehash);
EXPORT_SYMBOL(crypto_lb_find_device);
[-- Attachment #1.9: crypto_lb.h --]
[-- Type: text/x-chdr, Size: 1736 bytes --]
/*
* crypto_lb.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __CRYPTO_LB_H
#define __CRYPTO_LB_H
#include "acrypto.h"
#define CRYPTO_LB_NAMELEN 32
struct crypto_lb
{
struct list_head lb_entry;
char name[CRYPTO_LB_NAMELEN];
void (*rehash)(struct crypto_lb *);
struct crypto_device * (*find_device)(struct crypto_lb *, struct crypto_session_initializer *);
spinlock_t lock;
spinlock_t *crypto_device_lock;
struct list_head *crypto_device_list;
struct device_driver *driver;
struct device device;
struct class_device class_device;
struct completion dev_released;
};
int crypto_lb_register(struct crypto_lb *lb, int set_current, int set_default);
void crypto_lb_unregister(struct crypto_lb *lb);
inline void crypto_lb_rehash(void);
struct crypto_device *crypto_lb_find_device(struct crypto_session_initializer *ci);
int crypto_lb_init(void);
void crypto_lb_fini(void);
#endif /* __CRYPTO_LB_H */
[-- Attachment #1.10: crypto_main.c --]
[-- Type: text/x-csrc, Size: 8030 bytes --]
/*
* crypto_main.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include "acrypto.h"
#include "crypto_lb.h"
#include "crypto_conn.h"
#include "crypto_route.h"
int force_lb_remove;
module_param(force_lb_remove, int, 0);
struct crypto_device main_crypto_device;
extern struct bus_type crypto_bus_type;
extern struct device_driver crypto_driver;
extern struct class crypto_class;
extern struct device crypto_dev;
extern struct class_device_attribute class_device_attr_devices;
extern struct class_device_attribute class_device_attr_lbs;
static inline void dump_ci(struct crypto_session_initializer *ci)
{
dprintk("%llu [%llu] op=%04u, type=%04x, mode=%04x, priority=%04x",
ci->id, ci->dev_id,
ci->operation, ci->type, ci->mode, ci->priority);
}
static void __crypto_session_insert(struct crypto_device *dev, struct crypto_session *s)
{
struct crypto_session *__s;
if (unlikely(list_empty(&dev->session_list)))
{
list_add(&s->dev_queue_entry, &dev->session_list);
}
else
{
int inserted = 0;
list_for_each_entry(__s, &dev->session_list, dev_queue_entry)
{
if (__s->ci.priority < s->ci.priority)
{
list_add_tail(&s->dev_queue_entry, &__s->dev_queue_entry);
inserted = 1;
break;
}
}
if (!inserted)
list_add_tail(&s->dev_queue_entry, &dev->session_list);
}
dump_ci(&s->ci);
dprintk(" added to crypto device %s [%d].\n", dev->name, atomic_read(&dev->refcnt));
}
inline void crypto_session_insert_main(struct crypto_device *dev, struct crypto_session *s)
{
struct crypto_session *__s;
spin_lock_bh(&dev->session_lock);
crypto_device_get(dev);
if (unlikely(list_empty(&dev->session_list)))
{
list_add(&s->main_queue_entry, &dev->session_list);
}
else
{
int inserted = 0;
list_for_each_entry(__s, &dev->session_list, main_queue_entry)
{
if (__s->ci.priority < s->ci.priority)
{
list_add_tail(&s->main_queue_entry, &__s->main_queue_entry);
inserted = 1;
break;
}
}
if (!inserted)
list_add_tail(&s->main_queue_entry, &dev->session_list);
}
spin_unlock_bh(&dev->session_lock);
dump_ci(&s->ci);
dprintk(" added to main crypto device %s [%d].\n", dev->name, atomic_read(&dev->refcnt));
}
inline void crypto_session_insert(struct crypto_device *dev, struct crypto_session *s)
{
spin_lock_bh(&dev->session_lock);
__crypto_session_insert(dev, s);
spin_unlock_bh(&dev->session_lock);
if (dev->data_ready)
dev->data_ready(dev);
}
struct crypto_session *crypto_session_alloc(struct crypto_session_initializer *ci, struct crypto_data *d)
{
struct crypto_device *dev = &main_crypto_device;
struct crypto_device *ldev;
struct crypto_session *s;
int err;
if (d->priv_size > CRYPTO_MAX_PRIV_SIZE)
{
dprintk("priv_size %u is too big, maximum allowed %u.\n", d->priv_size, CRYPTO_MAX_PRIV_SIZE);
return NULL;
}
ldev = crypto_lb_find_device(ci);
if (!ldev)
{
dprintk("Cannot find suitable device.\n");
return NULL;
}
s = kmalloc(sizeof(*s) + d->priv_size, GFP_ATOMIC);
if (!s)
{
ldev->stat.kmem_failed++;
goto err_out_device_put;
}
memset(s, 0xAB, sizeof(*s));
crypto_route_head_init(&s->route_list);
spin_lock_init(&s->lock);
memcpy(&s->ci, ci, sizeof(s->ci));
memcpy(&s->data, d, sizeof(s->data));
if (d->priv_size)
{
s->data.priv = s+1;
if (d->priv)
memcpy(s->data.priv, d->priv, d->priv_size);
}
else
s->data.priv = NULL;
s->ci.id = dev->sid++;
s->ci.dev_id = ldev->sid++;
s->ci.flags = 0;
#if 1
err = crypto_route_add(ldev, s, ci);
if (err)
{
dprintk("Can not add route to device %s.\n", ldev->name);
goto err_out_session_free;
}
crypto_session_insert(ldev, s);
crypto_device_put(ldev);
#endif
crypto_session_insert_main(dev, s);
return s;
err_out_session_free:
crypto_device_put(ldev);
err_out_device_put:
kfree(s);
return NULL;
}
void crypto_session_dequeue_route(struct crypto_session *s)
{
struct crypto_route *rt;
struct crypto_device *dev;
BUG_ON(crypto_route_queue_len(s) > 1);
while ((rt = crypto_route_dequeue(s)))
{
dev = rt->dev;
dprintk(KERN_INFO "Removing route entry for device %s.\n", dev->name);
spin_lock_bh(&dev->session_lock);
list_del(&s->dev_queue_entry);
spin_unlock_bh(&dev->session_lock);
crypto_route_free(rt);
}
}
inline void __crypto_session_dequeue_main(struct crypto_session *s)
{
struct crypto_device *dev = &main_crypto_device;
list_del(&s->main_queue_entry);
crypto_device_put(dev);
}
inline void crypto_session_dequeue_main(struct crypto_session *s)
{
struct crypto_device *dev = &main_crypto_device;
spin_lock_bh(&dev->session_lock);
__crypto_session_dequeue_main(s);
spin_unlock_bh(&dev->session_lock);
}
int __devinit cmain_init(void)
{
struct crypto_device *dev = &main_crypto_device;
int err;
snprintf(dev->name, sizeof(dev->name), "crypto_sessions");
err = bus_register(&crypto_bus_type);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto bus: err=%d.\n", err);
return err;
}
err = driver_register(&crypto_driver);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto driver: err=%d.\n", err);
goto err_out_bus_unregister;
}
err = class_register(&crypto_class);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto class: err=%d.\n", err);
goto err_out_driver_unregister;
}
err = crypto_lb_init();
if (err)
goto err_out_class_unregister;
err = crypto_conn_init();
if (err)
goto err_out_crypto_lb_fini;
err = __crypto_device_add(dev);
if (err)
goto err_out_crypto_conn_fini;
err = class_device_create_file(&dev->class_device, &class_device_attr_devices);
if (err)
dprintk("Failed to create \"devices\" attribute: err=%d.\n", err);
err = class_device_create_file(&dev->class_device, &class_device_attr_lbs);
if (err)
dprintk("Failed to create \"lbs\" attribute: err=%d.\n", err);
return 0;
err_out_crypto_conn_fini:
crypto_conn_fini();
err_out_crypto_lb_fini:
crypto_lb_fini();
err_out_class_unregister:
class_unregister(&crypto_class);
err_out_driver_unregister:
driver_unregister(&crypto_driver);
err_out_bus_unregister:
bus_unregister(&crypto_bus_type);
return err;
}
void __devexit cmain_fini(void)
{
struct crypto_device *dev = &main_crypto_device;
class_device_remove_file(&dev->class_device, &class_device_attr_devices);
class_device_remove_file(&dev->class_device, &class_device_attr_lbs);
__crypto_device_remove(dev);
crypto_conn_fini();
crypto_lb_fini();
class_unregister(&crypto_class);
driver_unregister(&crypto_driver);
bus_unregister(&crypto_bus_type);
}
module_init(cmain_init);
module_exit(cmain_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Asynchronous crypto layer.");
EXPORT_SYMBOL(crypto_session_alloc);
[-- Attachment #1.11: crypto_route.h --]
[-- Type: text/x-chdr, Size: 5024 bytes --]
/*
* crypto_route.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __CRYPTO_ROUTE_H
#define __CRYPTO_ROUTE_H
#include <linux/types.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include "acrypto.h"
static inline struct crypto_route *crypto_route_alloc(struct crypto_device *dev, struct crypto_session_initializer *ci)
{
struct crypto_route *rt;
crypto_device_get(dev);
if (!match_initializer(dev, ci))
{
crypto_device_put(dev);
return NULL;
}
rt = kmalloc(sizeof(*rt), GFP_ATOMIC);
if (!rt)
{
crypto_device_put(dev);
return NULL;
}
memset(rt, 0, sizeof(*rt));
memcpy(&rt->ci, ci, sizeof(*ci));
rt->dev = dev;
return rt;
}
static inline void crypto_route_free(struct crypto_route *rt)
{
crypto_device_put(rt->dev);
rt->dev = NULL;
kfree(rt);
}
static inline void __crypto_route_del(struct crypto_route *rt, struct crypto_route_head *list)
{
struct crypto_route *next, *prev;
list->qlen--;
next = rt->next;
prev = rt->prev;
rt->next = rt->prev = NULL;
rt->list = NULL;
next->prev = prev;
prev->next = next;
}
static inline void crypto_route_del(struct crypto_route *rt)
{
struct crypto_route_head *list = rt->list;
if (list)
{
spin_lock_bh(&list->lock);
if (list == rt->list)
__crypto_route_del(rt, rt->list);
spin_unlock_bh(&list->lock);
crypto_route_free(rt);
}
}
static inline struct crypto_route *__crypto_route_dequeue(struct crypto_route_head *list)
{
struct crypto_route *next, *prev, *result;
prev = (struct crypto_route *) list;
next = prev->next;
result = NULL;
if (next != prev) {
result = next;
next = next->next;
list->qlen--;
next->prev = prev;
prev->next = next;
result->next = result->prev = NULL;
result->list = NULL;
}
return result;
}
static inline struct crypto_route *crypto_route_dequeue(struct crypto_session *s)
{
struct crypto_route *rt;
spin_lock_bh(&s->route_list.lock);
rt = __crypto_route_dequeue(&s->route_list);
spin_unlock_bh(&s->route_list.lock);
return rt;
}
static inline void __crypto_route_queue(struct crypto_route *rt, struct crypto_route_head *list)
{
struct crypto_route *prev, *next;
rt->list = list;
list->qlen++;
next = (struct crypto_route *)list;
prev = next->prev;
rt->next = next;
rt->prev = prev;
next->prev = prev->next = rt;
}
static inline void crypto_route_queue(struct crypto_route *rt, struct crypto_session *s)
{
spin_lock_bh(&s->route_list.lock);
__crypto_route_queue(rt, &s->route_list);
spin_unlock_bh(&s->route_list.lock);
}
static inline int crypto_route_add(struct crypto_device *dev, struct crypto_session *s, struct crypto_session_initializer *ci)
{
struct crypto_route *rt;
rt = crypto_route_alloc(dev, ci);
if (!rt)
return -ENOMEM;
crypto_route_queue(rt, s);
return 0;
}
static inline int crypto_route_queue_len(struct crypto_session *s)
{
return s->route_list.qlen;
}
static inline void crypto_route_head_init(struct crypto_route_head *list)
{
spin_lock_init(&list->lock);
list->prev = list->next = (struct crypto_route *)list;
list->qlen = 0;
}
static inline struct crypto_route *__crypto_route_current(struct crypto_route_head *list)
{
struct crypto_route *next, *prev, *result;
prev = (struct crypto_route *) list;
next = prev->next;
result = NULL;
if (next != prev)
result = next;
return result;
}
static inline struct crypto_route *crypto_route_current(struct crypto_session *s)
{
struct crypto_route_head *list;
struct crypto_route *rt = NULL;
list = &s->route_list;
if (list)
{
spin_lock_bh(&list->lock);
rt = __crypto_route_current(list);
spin_unlock_bh(&list->lock);
}
return rt;
}
static inline struct crypto_device *crypto_route_get_current_device(struct crypto_session *s)
{
struct crypto_route *rt = NULL;
struct crypto_device *dev = NULL;
struct crypto_route_head *list = &s->route_list;
spin_lock_bh(&list->lock);
rt = __crypto_route_current(list);
if (rt)
{
dev = rt->dev;
crypto_device_get(dev);
}
spin_unlock_bh(&list->lock);
return dev;
}
#endif /* __CRYPTO_ROUTE_H */
[-- Attachment #1.12: crypto_stat.c --]
[-- Type: text/x-csrc, Size: 2092 bytes --]
/*
* crypto_stat.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include "acrypto.h"
#include "crypto_route.h"
inline void crypto_stat_start_inc(struct crypto_session *s)
{
struct crypto_device *dev;
dev = crypto_route_get_current_device(s);
if (dev)
{
spin_lock_bh(&dev->stat_lock);
dev->stat.sstarted++;
spin_unlock_bh(&dev->stat_lock);
crypto_device_put(dev);
}
}
inline void crypto_stat_finish_inc(struct crypto_session *s)
{
struct crypto_device *dev;
dev = crypto_route_get_current_device(s);
if (dev)
{
spin_lock_bh(&dev->stat_lock);
dev->stat.sfinished++;
spin_unlock_bh(&dev->stat_lock);
crypto_device_put(dev);
}
}
inline void crypto_stat_complete_inc(struct crypto_session *s)
{
struct crypto_device *dev;
dev = crypto_route_get_current_device(s);
if (dev)
{
spin_lock_bh(&dev->stat_lock);
dev->stat.scompleted++;
spin_unlock_bh(&dev->stat_lock);
crypto_device_put(dev);
}
}
EXPORT_SYMBOL(crypto_stat_start_inc);
EXPORT_SYMBOL(crypto_stat_finish_inc);
EXPORT_SYMBOL(crypto_stat_complete_inc);
[-- Attachment #1.13: crypto_stat.h --]
[-- Type: text/x-chdr, Size: 1127 bytes --]
/*
* crypto_stat.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __CRYPTO_STAT_H
#define __CRYPTO_STAT_H
#include "acrypto.h"
inline void crypto_stat_start_inc(struct crypto_session *s);
inline void crypto_stat_finish_inc(struct crypto_session *s);
inline void crypto_stat_complete_inc(struct crypto_session *s);
#endif /* __CRYPTO_STAT_H */
[-- Attachment #1.14: Makefile --]
[-- Type: text/x-makefile, Size: 572 bytes --]
TOPDIR := $(PWD)
#KDIR := /lib/modules/$(shell uname -r)/build
KDIR := /usr/local/src/linux-2.6/linux-2.6
PWD := $(shell pwd)
#CC := /tmp/sparse-bk/cgcc
CC := gcc
obj-m += acrypto.o simple_lb.o
obj-m += consumer.o
#slow_consumer.o
obj-m += provider.o slow_provider.o sha1_provider.o
acrypto-objs := crypto_main.o crypto_lb.o crypto_dev.o crypto_conn.o crypto_stat.o
obj-m += via-padlock/
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
clean:
rm -f *.o *.ko *.mod.* .*.cmd *~ .*.d
rm -rf .tmp_versions
cd via-padlock && make clean
[-- Attachment #1.15: provider.c --]
[-- Type: text/x-csrc, Size: 2680 bytes --]
/*
* provider.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/err.h>
#include "acrypto.h"
#include "crypto_stat.h"
static int prov_data_ready_reentry;
static void prov_data_ready(struct crypto_device *dev);
static struct crypto_capability prov_caps[] = {
{0, 0, 0, 1000}, {1, 1, 1, 1000}, {2, 2, 2, 1000}};
static int prov_cap_number = sizeof(prov_caps)/sizeof(prov_caps[0]);
static struct crypto_device pdev =
{
.name = "test_provider",
.data_ready = prov_data_ready,
.cap = &prov_caps[0],
};
static void prov_data_ready(struct crypto_device *dev)
{
struct crypto_session *s, *n;
if (prov_data_ready_reentry)
return;
prov_data_ready_reentry++;
//spin_lock_bh(&dev->session_lock);
list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry)
{
if (!session_completed(s))
{
dprintk("Completing session %llu [%llu] in %s.\n",
s->ci.id, s->ci.dev_id, pdev.name);
crypto_stat_complete_inc(s);
complete_session(s);
}
}
//spin_unlock_bh(&dev->session_lock);
prov_data_ready_reentry--;
}
int prov_init(void)
{
int err;
pdev.cap_number = prov_cap_number;
err = crypto_device_add(&pdev);
if (err)
return err;
dprintk(KERN_INFO "Test crypto provider module %s is loaded.\n", pdev.name);
return 0;
}
void prov_fini(void)
{
crypto_device_remove(&pdev);
dprintk(KERN_INFO "Test crypto provider module %s is unloaded.\n", pdev.name);
}
module_init(prov_init);
module_exit(prov_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Test crypto module provider.");
[-- Attachment #1.16: sha1_provider.c --]
[-- Type: text/x-csrc, Size: 4307 bytes --]
/*
* sha1_provider.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/err.h>
#include <linux/crypto.h>
#include <linux/mm.h>
#include "acrypto.h"
#include "crypto_stat.h"
static void prov_data_ready(struct crypto_device *dev);
static struct crypto_capability prov_caps[] = {
{3, 3, 3, 100}
};
static int prov_cap_number = sizeof(prov_caps)/sizeof(prov_caps[0]);
static struct completion thread_exited;
static DECLARE_WAIT_QUEUE_HEAD(sha1_wait_queue);
static int need_exit;
static struct crypto_session *sp;
static struct crypto_tfm *tfm;
static struct crypto_device pdev =
{
.name = "sha1_provider",
.data_ready = prov_data_ready,
.cap = &prov_caps[0],
};
static void prov_data_ready(struct crypto_device *dev)
{
struct crypto_session *s, *n;
if (!spin_trylock_bh(&dev->session_lock))
return;
list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry)
{
if (!session_completed(s))
{
sp = s;
wake_up(&sha1_wait_queue);
break;
}
}
spin_unlock_bh(&dev->session_lock);
}
static int sha1_thread(void *data)
{
struct crypto_device *dev = (struct crypto_device *)data;
u8 *key, *out;
unsigned int keylen;
daemonize("%s", dev->name);
allow_signal(SIGTERM);
while (!need_exit)
{
interruptible_sleep_on(&sha1_wait_queue);
if (need_exit)
break;
if (!sp)
continue;
start_process_session(sp);
key = ((u8 *)page_address(sp->data.sg_key.page)) + sp->data.sg_key.offset;
keylen = sp->data.sg_key.length;
out = ((u8 *)page_address(sp->data.sg_dst.page)) + sp->data.sg_dst.offset;
crypto_hmac(tfm, key, &keylen, &sp->data.sg_src, sp->data.sg_src_num, out);
sp->data.sg_dst.length = 20;
dprintk("Completing session %llu [%llu] in %s.\n",
sp->ci.id, sp->ci.dev_id, pdev.name);
crypto_stat_complete_inc(sp);
complete_session(sp);
stop_process_session(sp);
prov_data_ready(dev);
sp = NULL;
}
complete_and_exit(&thread_exited, 0);
}
int prov_init(void)
{
int err, pid;
tfm = crypto_alloc_tfm("sha1", 0);
if (!tfm)
{
printk(KERN_ERR "Failed to allocate SHA1 tfm.\n");
return -EINVAL;
}
init_completion(&thread_exited);
pid = kernel_thread(sha1_thread, &pdev, CLONE_FS | CLONE_FILES);
if (IS_ERR((void *)pid))
{
err = -EINVAL;
dprintk(KERN_ERR "Failed to create kernel load balancing thread.\n");
goto err_out_free_tfm;
}
pdev.cap_number = prov_cap_number;
err = crypto_device_add(&pdev);
if (err)
goto err_out_remove_thread;
dprintk(KERN_INFO "Test crypto provider module %s is loaded.\n", pdev.name);
return 0;
err_out_remove_thread:
need_exit = 1;
wake_up(&sha1_wait_queue);
wait_for_completion(&thread_exited);
err_out_free_tfm:
crypto_free_tfm(tfm);
return err;
}
void prov_fini(void)
{
need_exit = 1;
wake_up(&sha1_wait_queue);
wait_for_completion(&thread_exited);
crypto_device_remove(&pdev);
crypto_free_tfm(tfm);
dprintk(KERN_INFO "Test crypto provider module %s is unloaded.\n", pdev.name);
}
module_init(prov_init);
module_exit(prov_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Test crypto module provider.");
[-- Attachment #1.17: simple_lb.c --]
[-- Type: text/x-csrc, Size: 2467 bytes --]
/*
* simple_lb.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include "crypto_lb.h"
static void simple_lb_rehash(struct crypto_lb *);
static struct crypto_device *simple_lb_find_device(struct crypto_lb *, struct crypto_session_initializer *);
struct crypto_lb simple_lb =
{
.name = "simple_lb",
.rehash = simple_lb_rehash,
.find_device = simple_lb_find_device
};
static void simple_lb_rehash(struct crypto_lb *lb)
{
BUG_ON(!lb->crypto_device_list);
BUG_ON(!lb->crypto_device_lock);
}
static struct crypto_device *simple_lb_find_device(struct crypto_lb *lb, struct crypto_session_initializer *ci)
{
struct crypto_device *dev, *__dev;
int min = 0x7ffffff;
__dev = NULL;
list_for_each_entry(dev, lb->crypto_device_list, cdev_entry)
{
if (device_broken(dev))
continue;
if (!match_initializer(dev, ci))
continue;
if (atomic_read(&dev->refcnt) < min)
{
min = atomic_read(&dev->refcnt);
__dev = dev;
}
}
return __dev;
}
int __devinit simple_lb_init(void)
{
dprintk(KERN_INFO "Registering simple crypto load balancer.\n");
return crypto_lb_register(&simple_lb, 1, 1);
}
void __devexit simple_lb_fini(void)
{
dprintk(KERN_INFO "Unregistering simple crypto load balancer.\n");
crypto_lb_unregister(&simple_lb);
}
module_init(simple_lb_init);
module_exit(simple_lb_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Simple crypto load balancer.");
[-- Attachment #1.18: Makefile --]
[-- Type: text/x-makefile, Size: 138 bytes --]
obj-m += padlock.o
padlock-objs := padlock-aes.o padlock-generic.o
clean:
rm -f *.o *.ko *.mod.* .*.cmd *~
rm -rf .tmp_versions
[-- Attachment #1.19: padlock-aes.c --]
[-- Type: text/x-csrc, Size: 16301 bytes --]
/*
* Cryptographic API.
*
* Support for VIA PadLock hardware crypto engine.
*
* Linux developers:
* Michal Ludvig <mludvig@suse.cz>
*
* Key expansion routine taken from crypto/aes.c
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* ---------------------------------------------------------------------------
* Copyright (c) 2002, Dr Brian Gladman <brg@gladman.me.uk>, Worcester, UK.
* All rights reserved.
*
* LICENSE TERMS
*
* The free distribution and use of this software in both source and binary
* form is allowed (with or without changes) provided that:
*
* 1. distributions of this source code include the above copyright
* notice, this list of conditions and the following disclaimer;
*
* 2. distributions in binary form include the above copyright
* notice, this list of conditions and the following disclaimer
* in the documentation and/or other associated materials;
*
* 3. the copyright holder's name is not used to endorse products
* built using this software without specific written permission.
*
* ALTERNATIVELY, provided that this notice is retained in full, this product
* may be distributed under the terms of the GNU General Public License (GPL),
* in which case the provisions of the GPL apply INSTEAD OF those given above.
*
* DISCLAIMER
*
* This software is provided 'as is' with no explicit or implied warranties
* in respect of its properties, including, but not limited to, correctness
* and/or fitness for purpose.
* ---------------------------------------------------------------------------
*/
#include <linux/module.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/crypto.h>
#include <asm/byteorder.h>
#include <linux/mm.h>
#include <asm/scatterlist.h>
#include "padlock.h"
#include "../crypto_def.h"
#include "../acrypto.h"
#include "../crypto_stat.h"
static inline int aes_hw_extkey_available (u8 key_len);
static inline
u32 generic_rotr32 (const u32 x, const unsigned bits)
{
const unsigned n = bits % 32;
return (x >> n) | (x << (32 - n));
}
static inline
u32 generic_rotl32 (const u32 x, const unsigned bits)
{
const unsigned n = bits % 32;
return (x << n) | (x >> (32 - n));
}
#define rotl generic_rotl32
#define rotr generic_rotr32
/*
* #define byte(x, nr) ((unsigned char)((x) >> (nr*8)))
*/
inline static u8
byte(const u32 x, const unsigned n)
{
return x >> (n << 3);
}
#define u32_in(x) le32_to_cpu(*(const u32 *)(x))
#define u32_out(to, from) (*(u32 *)(to) = cpu_to_le32(from))
static u8 pow_tab[256];
static u8 log_tab[256];
static u8 sbx_tab[256];
static u8 isb_tab[256];
static u32 rco_tab[10];
static u32 ft_tab[4][256];
static u32 it_tab[4][256];
static u32 fl_tab[4][256];
static u32 il_tab[4][256];
static inline u8
f_mult (u8 a, u8 b)
{
u8 aa = log_tab[a], cc = aa + log_tab[b];
return pow_tab[cc + (cc < aa ? 1 : 0)];
}
#define ff_mult(a,b) (a && b ? f_mult(a, b) : 0)
#define f_rn(bo, bi, n, k) \
bo[n] = ft_tab[0][byte(bi[n],0)] ^ \
ft_tab[1][byte(bi[(n + 1) & 3],1)] ^ \
ft_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
ft_tab[3][byte(bi[(n + 3) & 3],3)] ^ *(k + n)
#define i_rn(bo, bi, n, k) \
bo[n] = it_tab[0][byte(bi[n],0)] ^ \
it_tab[1][byte(bi[(n + 3) & 3],1)] ^ \
it_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
it_tab[3][byte(bi[(n + 1) & 3],3)] ^ *(k + n)
#define ls_box(x) \
( fl_tab[0][byte(x, 0)] ^ \
fl_tab[1][byte(x, 1)] ^ \
fl_tab[2][byte(x, 2)] ^ \
fl_tab[3][byte(x, 3)] )
#define f_rl(bo, bi, n, k) \
bo[n] = fl_tab[0][byte(bi[n],0)] ^ \
fl_tab[1][byte(bi[(n + 1) & 3],1)] ^ \
fl_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
fl_tab[3][byte(bi[(n + 3) & 3],3)] ^ *(k + n)
#define i_rl(bo, bi, n, k) \
bo[n] = il_tab[0][byte(bi[n],0)] ^ \
il_tab[1][byte(bi[(n + 3) & 3],1)] ^ \
il_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
il_tab[3][byte(bi[(n + 1) & 3],3)] ^ *(k + n)
static void
gen_tabs (void)
{
u32 i, t;
u8 p, q;
/* log and power tables for GF(2**8) finite field with
0x011b as modular polynomial - the simplest prmitive
root is 0x03, used here to generate the tables */
for (i = 0, p = 1; i < 256; ++i) {
pow_tab[i] = (u8) p;
log_tab[p] = (u8) i;
p ^= (p << 1) ^ (p & 0x80 ? 0x01b : 0);
}
log_tab[1] = 0;
for (i = 0, p = 1; i < 10; ++i) {
rco_tab[i] = p;
p = (p << 1) ^ (p & 0x80 ? 0x01b : 0);
}
for (i = 0; i < 256; ++i) {
p = (i ? pow_tab[255 - log_tab[i]] : 0);
q = ((p >> 7) | (p << 1)) ^ ((p >> 6) | (p << 2));
p ^= 0x63 ^ q ^ ((q >> 6) | (q << 2));
sbx_tab[i] = p;
isb_tab[p] = (u8) i;
}
for (i = 0; i < 256; ++i) {
p = sbx_tab[i];
t = p;
fl_tab[0][i] = t;
fl_tab[1][i] = rotl (t, 8);
fl_tab[2][i] = rotl (t, 16);
fl_tab[3][i] = rotl (t, 24);
t = ((u32) ff_mult (2, p)) |
((u32) p << 8) |
((u32) p << 16) | ((u32) ff_mult (3, p) << 24);
ft_tab[0][i] = t;
ft_tab[1][i] = rotl (t, 8);
ft_tab[2][i] = rotl (t, 16);
ft_tab[3][i] = rotl (t, 24);
p = isb_tab[i];
t = p;
il_tab[0][i] = t;
il_tab[1][i] = rotl (t, 8);
il_tab[2][i] = rotl (t, 16);
il_tab[3][i] = rotl (t, 24);
t = ((u32) ff_mult (14, p)) |
((u32) ff_mult (9, p) << 8) |
((u32) ff_mult (13, p) << 16) |
((u32) ff_mult (11, p) << 24);
it_tab[0][i] = t;
it_tab[1][i] = rotl (t, 8);
it_tab[2][i] = rotl (t, 16);
it_tab[3][i] = rotl (t, 24);
}
}
#define star_x(x) (((x) & 0x7f7f7f7f) << 1) ^ ((((x) & 0x80808080) >> 7) * 0x1b)
#define imix_col(y,x) \
u = star_x(x); \
v = star_x(u); \
w = star_x(v); \
t = w ^ (x); \
(y) = u ^ v ^ w; \
(y) ^= rotr(u ^ t, 8) ^ \
rotr(v ^ t, 16) ^ \
rotr(t,24)
/* initialise the key schedule from the user supplied key */
#define loop4(i) \
{ t = rotr(t, 8); t = ls_box(t) ^ rco_tab[i]; \
t ^= E_KEY[4 * i]; E_KEY[4 * i + 4] = t; \
t ^= E_KEY[4 * i + 1]; E_KEY[4 * i + 5] = t; \
t ^= E_KEY[4 * i + 2]; E_KEY[4 * i + 6] = t; \
t ^= E_KEY[4 * i + 3]; E_KEY[4 * i + 7] = t; \
}
#define loop6(i) \
{ t = rotr(t, 8); t = ls_box(t) ^ rco_tab[i]; \
t ^= E_KEY[6 * i]; E_KEY[6 * i + 6] = t; \
t ^= E_KEY[6 * i + 1]; E_KEY[6 * i + 7] = t; \
t ^= E_KEY[6 * i + 2]; E_KEY[6 * i + 8] = t; \
t ^= E_KEY[6 * i + 3]; E_KEY[6 * i + 9] = t; \
t ^= E_KEY[6 * i + 4]; E_KEY[6 * i + 10] = t; \
t ^= E_KEY[6 * i + 5]; E_KEY[6 * i + 11] = t; \
}
#define loop8(i) \
{ t = rotr(t, 8); ; t = ls_box(t) ^ rco_tab[i]; \
t ^= E_KEY[8 * i]; E_KEY[8 * i + 8] = t; \
t ^= E_KEY[8 * i + 1]; E_KEY[8 * i + 9] = t; \
t ^= E_KEY[8 * i + 2]; E_KEY[8 * i + 10] = t; \
t ^= E_KEY[8 * i + 3]; E_KEY[8 * i + 11] = t; \
t = E_KEY[8 * i + 4] ^ ls_box(t); \
E_KEY[8 * i + 12] = t; \
t ^= E_KEY[8 * i + 5]; E_KEY[8 * i + 13] = t; \
t ^= E_KEY[8 * i + 6]; E_KEY[8 * i + 14] = t; \
t ^= E_KEY[8 * i + 7]; E_KEY[8 * i + 15] = t; \
}
static int
aes_set_key(void *ctx_arg, const u8 *in_key, unsigned int key_len)
{
struct aes_ctx *ctx = ctx_arg;
u32 i, t, u, v, w;
u32 P[AES_EXTENDED_KEY_SIZE];
u32 rounds;
if (key_len != 16 && key_len != 24 && key_len != 32) {
return -EINVAL;
}
ctx->key_length = key_len;
ctx->E = ctx->e_data;
ctx->D = ctx->d_data;
/* Ensure 16-Bytes alignmentation of keys for VIA PadLock. */
if ((int)(ctx->e_data) & 0x0F)
ctx->E += 4 - (((int)(ctx->e_data) & 0x0F) / sizeof (ctx->e_data[0]));
if ((int)(ctx->d_data) & 0x0F)
ctx->D += 4 - (((int)(ctx->d_data) & 0x0F) / sizeof (ctx->d_data[0]));
E_KEY[0] = u32_in (in_key);
E_KEY[1] = u32_in (in_key + 4);
E_KEY[2] = u32_in (in_key + 8);
E_KEY[3] = u32_in (in_key + 12);
/* Don't generate extended keys if the hardware can do it. */
if (aes_hw_extkey_available(key_len))
return 0;
switch (key_len) {
case 16:
t = E_KEY[3];
for (i = 0; i < 10; ++i)
loop4 (i);
break;
case 24:
E_KEY[4] = u32_in (in_key + 16);
t = E_KEY[5] = u32_in (in_key + 20);
for (i = 0; i < 8; ++i)
loop6 (i);
break;
case 32:
E_KEY[4] = u32_in (in_key + 16);
E_KEY[5] = u32_in (in_key + 20);
E_KEY[6] = u32_in (in_key + 24);
t = E_KEY[7] = u32_in (in_key + 28);
for (i = 0; i < 7; ++i)
loop8 (i);
break;
}
D_KEY[0] = E_KEY[0];
D_KEY[1] = E_KEY[1];
D_KEY[2] = E_KEY[2];
D_KEY[3] = E_KEY[3];
for (i = 4; i < key_len + 24; ++i) {
imix_col (D_KEY[i], E_KEY[i]);
}
/* PadLock needs a different format of the decryption key. */
rounds = 10 + (key_len - 16) / 4;
for (i = 0; i < rounds; i++) {
P[((i + 1) * 4) + 0] = D_KEY[((rounds - i - 1) * 4) + 0];
P[((i + 1) * 4) + 1] = D_KEY[((rounds - i - 1) * 4) + 1];
P[((i + 1) * 4) + 2] = D_KEY[((rounds - i - 1) * 4) + 2];
P[((i + 1) * 4) + 3] = D_KEY[((rounds - i - 1) * 4) + 3];
}
P[0] = E_KEY[(rounds * 4) + 0];
P[1] = E_KEY[(rounds * 4) + 1];
P[2] = E_KEY[(rounds * 4) + 2];
P[3] = E_KEY[(rounds * 4) + 3];
memcpy(D_KEY, P, AES_EXTENDED_KEY_SIZE_B);
return 0;
}
/* Tells whether the ACE is capable to generate
the extended key for a given key_len. */
static inline int aes_hw_extkey_available(u8 key_len)
{
/* TODO: We should check the actual CPU model/stepping
as it's likely that the capability will be
added in the next CPU revisions. */
if (key_len == 16)
return 1;
return 0;
}
static void aes_padlock(void *ctx_arg, u8 *out_arg, const u8 *in_arg,
const u8 *iv_arg, size_t nbytes, int encdec,
int mode)
{
struct aes_ctx *ctx = ctx_arg;
char bigbuf[sizeof(union cword) + 16];
union cword *cword;
void *key;
if (((long)bigbuf) & 0x0F)
cword = (void*)(bigbuf + 16 - ((long)bigbuf & 0x0F));
else
cword = (void*)bigbuf;
/* Prepare Control word. */
memset (cword, 0, sizeof(union cword));
cword->b.encdec = !encdec; /* in the rest of cryptoapi ENC=1/DEC=0 */
cword->b.rounds = 10 + (ctx->key_length - 16) / 4;
cword->b.ksize = (ctx->key_length - 16) / 8;
/* Is the hardware capable to generate the extended key? */
if (!aes_hw_extkey_available(ctx->key_length))
cword->b.keygen = 1;
/* ctx->E starts with a plain key - if the hardware is capable
to generate the extended key itself we must supply
the plain key for both Encryption and Decryption. */
if (encdec == CRYPTO_OP_ENCRYPT || cword->b.keygen == 0)
key = ctx->E;
else
key = ctx->D;
padlock_aligner(out_arg, in_arg, iv_arg, key, cword,
nbytes, AES_BLOCK_SIZE, encdec, mode);
}
static void aes_padlock_ecb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
size_t nbytes, int encdec)
{
aes_padlock(ctx, dst, src, NULL, nbytes, encdec, CRYPTO_MODE_ECB);
}
static void aes_padlock_cbc(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
size_t nbytes, int encdec)
{
aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_CBC);
}
static void aes_padlock_cfb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
size_t nbytes, int encdec)
{
aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_CFB);
}
static void aes_padlock_ofb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
size_t nbytes, int encdec)
{
aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_OFB);
}
static struct crypto_capability padlock_caps[] =
{
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_OFB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_OFB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_OFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_OFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_OFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_OFB, 1000},
};
static int padlock_cap_number = sizeof(padlock_caps)/sizeof(padlock_caps[0]);
static void padlock_data_ready(struct crypto_device *dev);
static int padlock_data_ready_reentry;
static struct crypto_device padlock_device =
{
.name = "via-padlock",
.data_ready = padlock_data_ready,
.cap = &padlock_caps[0],
};
static void process_session(struct crypto_session *s)
{
int err;
u8 *key, *dst, *src, *iv;
size_t size, keylen;
key = ((u8 *)page_address(s->data.sg_key.page)) + s->data.sg_key.offset;
keylen = s->data.sg_key.length;
dst = ((u8 *)page_address(s->data.sg_dst.page)) + s->data.sg_dst.offset;
src = ((u8 *)page_address(s->data.sg_src.page)) + s->data.sg_src.offset;
size = s->data.sg_src.length;
iv = ((u8 *)page_address(s->data.sg_iv.page)) + s->data.sg_iv.offset;
err = aes_set_key(s->data.priv, key, keylen);
if (err)
return;
switch (s->ci.mode)
{
case CRYPTO_MODE_ECB:
aes_padlock_ecb(s->data.priv, dst, src, iv, size, s->ci.operation);
break;
case CRYPTO_MODE_CBC:
aes_padlock_cbc(s->data.priv, dst, src, iv, size, s->ci.operation);
break;
case CRYPTO_MODE_CFB:
aes_padlock_cfb(s->data.priv, dst, src, iv, size, s->ci.operation);
break;
case CRYPTO_MODE_OFB:
aes_padlock_ofb(s->data.priv, dst, src, iv, size, s->ci.operation);
break;
}
s->data.sg_dst.length = size;
return;
}
static void padlock_data_ready(struct crypto_device *dev)
{
struct crypto_session *s, *n;
if (padlock_data_ready_reentry)
return;
padlock_data_ready_reentry++;
list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry)
{
if (!session_completed(s))
{
start_process_session(s);
process_session(s);
crypto_stat_complete_inc(s);
complete_session(s);
stop_process_session(s);
}
}
padlock_data_ready_reentry--;
}
int padlock_init_aes(void)
{
u32 cpuid, edx;
u32 val = 0xC0000000;
cpuid = cpuid_eax(val);
edx = cpuid_edx(val);
printk("val=%x, cpuid=%x, edx=%x.\n", val, cpuid, edx);
if (cpuid >= val + 1)
{
printk("Board supports ACE.\n");
}
else
{
printk("Board does not support ACE.\n");
return -ENODEV;
}
printk(KERN_NOTICE "Using VIA PadLock ACE for AES algorithm (multiblock).\n");
padlock_device.cap_number = padlock_cap_number;
gen_tabs();
return crypto_device_add(&padlock_device);
}
void padlock_fini_aes(void)
{
crypto_device_remove(&padlock_device);
}
[-- Attachment #1.20: padlock-generic.c --]
[-- Type: text/x-csrc, Size: 5401 bytes --]
/*
* Cryptographic API.
*
* Support for VIA PadLock hardware crypto engine.
*
* Linux developers:
* Michal Ludvig <mludvig@suse.cz>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*/
#include <linux/module.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/crypto.h>
#include <asm/byteorder.h>
#include "padlock.h"
#include "../acrypto.h"
#include "../crypto_def.h"
#define PFX "padlock: "
typedef void (xcrypt_t)(u8 *input, u8 *output, u8 *key, u8 *iv,
void *control_word, u32 count);
static inline void padlock_xcrypt_ecb(u8 *input, u8 *output, u8 *key,
u8 *iv, void *control_word, u32 count)
{
asm volatile ("pushfl; popfl"); /* enforce key reload. */
asm volatile (".byte 0xf3,0x0f,0xa7,0xc8" /* rep xcryptecb */
: "=m"(*output), "+S"(input), "+D"(output)
: "d"(control_word), "b"(key), "c"(count));
}
static inline void padlock_xcrypt_cbc(u8 *input, u8 *output, u8 *key,
u8 *iv, void *control_word, u32 count)
{
asm volatile ("pushfl; popfl"); /* enforce key reload. */
asm volatile (".byte 0xf3,0x0f,0xa7,0xd0" /* rep xcryptcbc */
: "=m"(*output), "+S"(input), "+D"(output)
: "d"(control_word), "b"(key), "c"(count), "a"(iv));
}
static inline void padlock_xcrypt_cfb(u8 *input, u8 *output, u8 *key,
u8 *iv, void *control_word, u32 count)
{
asm volatile ("pushfl; popfl"); /* enforce key reload. */
asm volatile (".byte 0xf3,0x0f,0xa7,0xe0" /* rep xcryptcfb */
: "=m"(*output), "+S"(input), "+D"(output)
: "d"(control_word), "b"(key), "c"(count), "a"(iv));
}
static inline void padlock_xcrypt_ofb(u8 *input, u8 *output, u8 *key,
u8 *iv, void *control_word, u32 count)
{
asm volatile ("pushfl; popfl"); /* enforce key reload. */
asm volatile (".byte 0xf3,0x0f,0xa7,0xe8" /* rep xcryptofb */
: "=m"(*output), "+S"(input), "+D"(output)
: "d"(control_word), "b"(key), "c"(count), "a"(iv));
}
void *crypto_aligned_kmalloc(size_t size, int mode, size_t alignment, void **index)
{
char *ptr;
ptr = kmalloc(size + alignment, mode);
*index = ptr;
if (alignment > 1 && ((long)ptr & (alignment - 1))) {
ptr += alignment - ((long)ptr & (alignment - 1));
}
return ptr;
}
void padlock_aligner(u8 *out_arg, const u8 *in_arg, const u8 *iv_arg,
void *key, union cword *cword,
size_t nbytes, size_t blocksize,
int encdec, int mode)
{
/* Don't blindly modify this structure - the items must
fit on 16-Bytes boundaries! */
struct padlock_xcrypt_data {
u8 iv[blocksize]; /* Initialization vector */
};
u8 *in, *out, *iv;
void *index = NULL;
char bigbuf[sizeof(struct padlock_xcrypt_data) + 16];
struct padlock_xcrypt_data *data;
/* Place 'data' at the first 16-Bytes aligned address in 'bigbuf'. */
if (((long)bigbuf) & 0x0F)
data = (void*)(bigbuf + 16 - ((long)bigbuf & 0x0F));
else
data = (void*)bigbuf;
if (((long)in_arg) & 0x0F) {
in = crypto_aligned_kmalloc(nbytes, GFP_KERNEL, 16, &index);
memcpy(in, in_arg, nbytes);
}
else
in = (u8*)in_arg;
if (((long)out_arg) & 0x0F) {
if (index)
out = in; /* xcrypt can work "in place" */
else
out = crypto_aligned_kmalloc(nbytes, GFP_KERNEL, 16, &index);
}
else
out = out_arg;
/* Always make a local copy of IV - xcrypt may change it! */
iv = data->iv;
if (iv_arg)
memcpy(iv, iv_arg, blocksize);
dprintk("data=%p\n", data);
dprintk("in=%p\n", in);
dprintk("out=%p\n", out);
dprintk("iv=%p\n", iv);
dprintk("nbytes=%d, blocksize=%d.\n", nbytes, blocksize);
switch (mode) {
case CRYPTO_MODE_ECB:
padlock_xcrypt_ecb(in, out, key, iv, cword, nbytes/blocksize);
break;
case CRYPTO_MODE_CBC:
padlock_xcrypt_cbc(in, out, key, iv, cword, nbytes/blocksize);
break;
case CRYPTO_MODE_CFB:
padlock_xcrypt_cfb(in, out, key, iv, cword, nbytes/blocksize);
break;
case CRYPTO_MODE_OFB:
padlock_xcrypt_ofb(in, out, key, iv, cword, nbytes/blocksize);
break;
default:
BUG();
}
/* Copy the 16-Byte aligned output to the caller's buffer. */
if (out != out_arg)
memcpy(out_arg, out, nbytes);
if (index)
kfree(index);
}
static int __init padlock_init(void)
{
int ret = -ENOSYS;
#if 0
if (!cpu_has_xcrypt) {
printk(KERN_ERR PFX "VIA PadLock not detected.\n");
return -ENODEV;
}
if (!cpu_has_xcrypt_enabled) {
printk(KERN_ERR PFX "VIA PadLock detected, but not enabled. Hmm, strange...\n");
return -ENODEV;
}
#endif
if ((ret = padlock_init_aes())) {
printk(KERN_ERR PFX "VIA PadLock AES initialization failed.\n");
return ret;
}
if (ret == -ENOSYS)
printk(KERN_ERR PFX "Hmm, VIA PadLock was compiled without any algorithm.\n");
return ret;
}
static void __exit padlock_fini(void)
{
padlock_fini_aes();
}
module_init(padlock_init);
module_exit(padlock_fini);
MODULE_DESCRIPTION("VIA PadLock crypto engine support.");
MODULE_LICENSE("Dual BSD/GPL");
MODULE_AUTHOR("Michal Ludvig");
[-- Attachment #1.21: padlock.h --]
[-- Type: text/x-chdr, Size: 1504 bytes --]
/*
* Cryptographic API.
*
* Copyright (c) 2004 Michal Ludvig <mludvig@suse.cz>
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the Free
* Software Foundation; either version 2 of the License, or (at your option)
* any later version.
*
*/
#ifndef _CRYPTO_PADLOCK_H
#define _CRYPTO_PADLOCK_H
#define AES_MIN_KEY_SIZE 16 /* in u8 units */
#define AES_MAX_KEY_SIZE 32 /* ditto */
#define AES_BLOCK_SIZE 16 /* ditto */
#define AES_EXTENDED_KEY_SIZE 64 /* in u32 units */
#define AES_EXTENDED_KEY_SIZE_B (AES_EXTENDED_KEY_SIZE * sizeof(u32))
struct aes_ctx {
u32 e_data[AES_EXTENDED_KEY_SIZE+4];
u32 d_data[AES_EXTENDED_KEY_SIZE+4];
int key_length;
u32 *E;
u32 *D;
};
#define E_KEY ctx->E
#define D_KEY ctx->D
/* Control word. */
#if 1
union cword {
u32 cword[4];
struct {
int rounds:4;
int algo:3;
int keygen:1;
int interm:1;
int encdec:1;
int ksize:2;
} b;
};
#else
union cword {
u32 cword[4];
struct {
unsigned rounds:4,
algo:3,
keygen:1,
interm:1,
encdec:1,
ksize:2;
} b;
};
#endif
#define PFX "padlock: "
void padlock_aligner(u8 *out_arg, const u8 *in_arg, const u8 *iv_arg,
void *key, union cword *cword,
size_t nbytes, size_t blocksize,
int encdec, int mode);
int padlock_init_aes(void);
void padlock_fini_aes(void);
#endif /* _CRYPTO_PADLOCK_H */
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
next reply other threads:[~2004-10-29 6:22 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2004-10-29 6:22 Evgeniy Polyakov [this message]
2004-10-29 12:42 ` Asynchronous crypto layer jamal
2004-10-29 14:06 ` Evgeniy Polyakov
2004-10-29 14:03 ` Michal Ludvig
2004-10-29 14:36 ` Evgeniy Polyakov
2004-10-29 14:53 ` Michal Ludvig
2004-10-29 15:11 ` jamal
2004-10-29 15:27 ` Evgeniy Polyakov
2004-10-30 20:39 ` Eugene Surovegin
2004-10-30 21:17 ` Evgeniy Polyakov
2004-10-30 21:09 ` Eugene Surovegin
2004-10-30 21:46 ` Evgeniy Polyakov
2004-10-29 15:08 ` jamal
2004-10-29 15:37 ` Evgeniy Polyakov
2004-10-29 15:28 ` Michal Ludvig
2004-10-29 16:16 ` James Morris
2004-10-29 16:45 ` Evgeniy Polyakov
2004-10-29 20:00 ` Sam Leffler
2004-10-31 6:09 ` James Morris
2004-10-31 6:35 ` Sam Leffler
2004-10-30 20:35 ` Eugene Surovegin
2004-10-30 21:04 ` Evgeniy Polyakov
2004-10-30 20:56 ` Eugene Surovegin
2004-10-30 21:24 ` Evgeniy Polyakov
2004-10-30 23:41 ` jamal
2004-10-31 9:13 ` Evgeniy Polyakov
2004-10-31 10:46 ` Michal Ludvig
2004-10-31 15:03 ` jamal
2004-10-31 16:07 ` James Morris
2004-11-01 6:01 ` Evgeniy Polyakov
2004-10-31 15:36 ` Michael Richardson
2004-10-31 16:09 ` James Morris
2004-10-31 14:56 ` jamal
2004-11-01 5:12 ` Evgeniy Polyakov
2004-10-29 17:44 ` James Morris
2004-10-29 17:46 ` James Morris
2004-10-30 5:20 ` Evgeniy Polyakov
2004-10-31 5:43 ` James Morris
2004-10-30 5:19 ` Evgeniy Polyakov
2004-10-30 8:34 ` Evgeniy Polyakov
2004-10-30 8:36 ` Evgeniy Polyakov
2004-10-30 16:57 ` Michal Ludvig
2004-10-30 17:40 ` Evgeniy Polyakov
2004-10-30 20:17 ` Michal Ludvig
2004-10-30 20:56 ` Evgeniy Polyakov
2004-10-30 19:42 ` James Morris
2004-10-29 19:56 ` Sam Leffler
2004-10-30 6:16 ` Christoph Hellwig
2004-10-30 20:37 ` Evgeniy Polyakov
2004-10-31 16:05 ` James Morris
2004-11-01 5:58 ` Evgeniy Polyakov
2004-12-14 6:56 ` James Morris
2004-12-14 7:58 ` Evgeniy Polyakov
2004-11-02 15:24 ` Evgeniy Polyakov
2004-11-02 16:12 ` Evgeniy Polyakov
2004-12-14 7:23 ` James Morris
2004-12-14 8:07 ` Evgeniy Polyakov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1099030958.4944.148.camel@uganda \
--to=johnpol@2ka.mipt.ru \
--cc=cryptoapi@lists.logix.cz \
--cc=netdev@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).