* Asynchronous crypto layer.
@ 2004-10-29 6:22 Evgeniy Polyakov
2004-10-29 12:42 ` jamal
` (6 more replies)
0 siblings, 7 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-29 6:22 UTC (permalink / raw)
To: netdev; +Cc: cryptoapi
[-- Attachment #1.1: Type: text/plain, Size: 6248 bytes --]
I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
It support following features:
- multiple asynchronous crypto device queues
- crypto session routing
- crypto session binding
- modular load balancing
- crypto session batching genetically implemented by design
- crypto session priority
- different kinds of crypto operation(RNG, asymmetrical crypto, HMAC and
any other)
Some design notes:
acrypto has one main crypto session queue(double linked list, probably
it should be done like crypto_route or sk_buff queue), into which each
newly allocated session is inserted and this is a place where load
balancing searches it's food. When new session is being prepared for
insertion it calls load balancer's ->find_device() method, which should
return suitable device(current simple_lb load balancer returns device
with the lowest load(device has the least number of session in it's
queue)) if it exists. After crypto_device being returned acrypto creates
new crypto routing entry which points to returned device and adds it to
crypto session routing queue. Crypto session is being inserted into
device's queue according to it's priority and it is crypto device driver
that should process it's session list according to session's priority.
All insertion and deletion are guarded by appropriate locks, but
session_list traversing is not guarded in crypto_lb_thread() since
session can be removed _only_ from that function by design, so if crypto
device (atomically) marks session as completed and not being processed
and use list_for_each_safe() for traversing it's queue all should be OK.
Each crypto load balancer must implement 2 methods:
->rehash() and ->find_device() which will be called from any context and
under spinlock.
->rehash() method should be called to remix crypto sessions in device's
queues, for example if driver decides that it's device is broken it
marks itself as broken and load balancer(or scheduler if you like)
should remove all sessions from this queue to some other devices.
If session can not be completed scheduler must mark it as broken and
complete it(by calling first broke_session() and then complete_session()
and stop_process_session()). Consumer must check if operation was
successful(and therefore session is not broken).
->find_device() method should return appropriate crypto device.
For crypto session to be successfully allocated crypto consumer must
provide two structures - struct crypto_session_initializer
(hmm, why only one z?) and struct crypto_data.
struct crypto_session_initializer contains data needed to find
appropriate device, like type of operation, mode of operation, some
flags(for example SESSION_BINDED, which means that session must be bound
to specified in bdev field crypto device, it is useful for TCPA/TPM),
session priority and callback which will be called after all routing for
given session are finished.
struct crypto_data contains scatterlists for src, dst, key and iv.
It also has void *priv field and it's size which is allocated and may be
used by any crypto agent(for example VIA PadLock driver uses it to store
aes_ctx field, crypto_session can use this field to store some pointers
needed in ->callback()).
Actually callback will be called from queue_work, but I suppose it is
better to not assume calling context.
->callback() will be called after all crypto routing for given session
are done with the same parameters as were provided in initialisation
time(if session has only one routing callback will be called with
original parameters, but if it has several routes callback will be
called with parameters from the latest processed one). I believe crypto
callback should not know about crypto sessions, routings, device and so
on, proper restriction is always a good idea.
Crypto routing.
This feature allows the same session to be processed by several
devices/algorithms. For example if you need to encrypt data and then
sign it in TPM device you can create one route to encryption device and
then route it to TPM device. (Note: this feature must be discussed since
there is no time slice after session allocation, only in
crypto_device->data_ready() method and there are locking issues in
->callback() method).
Crypto device.
It can be either software emulator or hardware accelerator chip(like
HIFN 79*/83* or Via PadLock ACE/RNG, or even TPM device like each IBM
ThinkPad or some HP laptops have
(gentle hint: _they_ even have a _windows_ software for them :) )).
It can be registered with asynchronous crypto layer and must provide
some data for it:
->data_ready() method - it is called each time new session is added to
device's queue.
Array of struct crypto_capability and it's amount -
struct crypto_capability describes each operation given device can
handle, and has a maximum session queue length parameter.
Note: this structure can [be extended to] include "rate" parameter to
show absolute speed of given operation in some units, which therefore
can be used by scheduler(load balancer) for proper device selection.
Actually queue length can somehow reflects device's "speed".
Also attached:
- simple synchronous <-> asynchronous bridge for sha1 crypto provider.
It can be easily extended to handle whole number of synchronous software
implemented algorithms.
- driver for Via PadLock ACE - not tested, since my M10000 does not have
xcrypt module. This work is 98% by Michal Ludvig <mludvig suse.cz>, I
have just silently stolen it several days ago.
I've obtained bits of documentation for HIFN 795* so driver will be
created for it. I want to thank Wim Vandeputte <wim kd85.com> and
www.soekris.com for steps towards me in ordering and delivering process
to Russia.
Asynchronous crypto layer depends on kernel connector which was
presented in both linux-kernel@ and netdev@ several weeks ago.
Currently acrypto does not handle userspace session requests(for example
from OpenSSL) but I will implement it soon in form of char device and
it's ->mmap method.
Please review and comment, I am open for discussion.
Thank you.
--
Evgeniy Polyakov
Crash is better than data corruption. -- Art Grabowski
[-- Attachment #1.2: acrypto.h --]
[-- Type: text/x-chdr, Size: 7358 bytes --]
/*
* acrypto.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __ACRYPTO_H
#define __ACRYPTO_H
#define SCACHE_NAMELEN 32
struct crypto_session_initializer;
struct crypto_data;
typedef void (*crypto_callback_t)(struct crypto_session_initializer *, struct crypto_data *);
#if 0
#define SESSION_COMPLETED 0
#define SESSION_FINISHED 1
#define SESSION_STARTED 2
#define SESSION_PROCESSED 3
#define session_completed(s) (test_bit(SESSION_COMPLETED, (unsigned long *)&s->ci.flags))
#define complete_session(s) do { set_bit(SESSION_COMPLETED, (unsigned long *)&s->ci.flags); }while (0)
#define uncomplete_session(s) do { clear_bit(SESSION_COMPLETED, (unsigned long *)&s->ci.flags); } while (0)
#define session_started(s) (test_bit(SESSION_STARTED, (unsigned long *)&s->ci.flags))
#define start_session(s) do { set_bit(SESSION_STARTED, (unsigned long *)&s->ci.flags); } while (0)
#define unstart_session(s) do { clear_bit(SESSION_STARTED, (unsigned long *)&s->ci.flags); } while (0)
#define session_finished(s) (test_bit(SESSION_FINISHED, (unsigned long *)&s->ci.flags))
#define finish_session(s) do { set_bit(SESSION_FINISHED, (unsigned long *)&s->ci.flags); } while (0)
#define unfinish_session(s) do { clear_bit(SESSION_FINISHED, (unsigned long *)&s->ci.flags); } while (0)
#define session_is_processed(s) (test_bit(SESSION_PROCESSED, (unsigned long *)&s->ci.flags))
#define start_process_session(s) do { set_bit(SESSION_PROCESSED, (unsigned long *)&s->ci.flags); } while (0)
#define stop_process_session(s) do { clear_bit(SESSION_PROCESSED, (unsigned long *)&s->ci.flags); } while (0)
#else
#define SESSION_COMPLETED (1<<15)
#define SESSION_FINISHED (1<<14)
#define SESSION_STARTED (1<<13)
#define SESSION_PROCESSED (1<<12)
#define SESSION_BINDED (1<<11)
#define SESSION_BROKEN (1<<10)
#define session_completed(s) (s->ci.flags & SESSION_COMPLETED)
#define complete_session(s) do {s->ci.flags |= SESSION_COMPLETED;} while(0)
#define uncomplete_session(s) do {s->ci.flags &= ~SESSION_COMPLETED;} while (0)
#define session_finished(s) (s->ci.flags & SESSION_FINISHED)
#define finish_session(s) do {s->ci.flags |= SESSION_FINISHED;} while(0)
#define unfinish_session(s) do {s->ci.flags &= ~SESSION_FINISHED;} while (0)
#define session_started(s) (s->ci.flags & SESSION_STARTED)
#define start_session(s) do {s->ci.flags |= SESSION_STARTED;} while(0)
#define unstart_session(s) do {s->ci.flags &= ~SESSION_STARTED;} while (0)
#define session_is_processed(s) (s->ci.flags & SESSION_PROCESSED)
#define start_process_session(s) do {s->ci.flags |= SESSION_PROCESSED;} while(0)
#define stop_process_session(s) do {s->ci.flags &= ~SESSION_PROCESSED;} while (0)
#define session_binded(s) (s->ci.flags & SESSION_BINDED)
#define bind_session(s) do {s->ci.flags |= SESSION_BINDED;} while(0)
#define unbind_session(s) do {s->ci.flags &= ~SESSION_BINDED;} while (0)
#define sci_binded(ci) (ci->flags & SESSION_BINDED)
#define session_broken(s) (s->ci.flags & SESSION_BROKEN)
#define broke_session(s) do {s->ci.flags |= SESSION_BROKEN;} while(0)
#define unbroke_session(s) do {s->ci.flags &= ~SESSION_BROKEN;} while (0)
#endif
struct crypto_device_stat
{
__u64 scompleted;
__u64 sfinished;
__u64 sstarted;
__u64 kmem_failed;
};
#ifdef __KERNEL__
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/device.h>
#include <linux/workqueue.h>
#include <asm/scatterlist.h>
#define DEBUG
#ifdef DEBUG
#define dprintk(f, a...) printk(KERN_EMERG f, ##a)
#else
#define dprintk(f, a...)
#endif
#define CRYPTO_MAX_PRIV_SIZE 1024
#define DEVICE_BROKEN (1<<0)
#define device_broken(dev) (dev->flags & DEVICE_BROKEN)
#define broke_device(dev) do {dev->flags |= DEVICE_BROKEN;} while(0)
#define repair_device(dev) do {dev->flags &= ~DEVICE_BROKEN;} while(0)
struct crypto_capability
{
u16 operation;
u16 type;
u16 mode;
u16 qlen;
};
struct crypto_session_initializer
{
u16 operation;
u16 type;
u16 mode;
u16 priority;
u64 id;
u64 dev_id;
u32 flags;
u32 bdev;
crypto_callback_t callback;
};
struct crypto_data
{
int sg_src_num;
struct scatterlist sg_src;
struct scatterlist sg_dst;
struct scatterlist sg_key;
struct scatterlist sg_iv;
void *priv;
unsigned int priv_size;
};
struct crypto_device
{
char name[SCACHE_NAMELEN];
spinlock_t session_lock;
struct list_head session_list;
u64 sid;
spinlock_t lock;
atomic_t refcnt;
u32 flags;
u32 id;
struct list_head cdev_entry;
void (*data_ready)(struct crypto_device *);
struct device_driver *driver;
struct device device;
struct class_device class_device;
struct completion dev_released;
spinlock_t stat_lock;
struct crypto_device_stat stat;
struct crypto_capability *cap;
int cap_number;
};
struct crypto_route_head
{
struct crypto_route *next;
struct crypto_route *prev;
__u32 qlen;
spinlock_t lock;
};
struct crypto_route
{
struct crypto_route *next;
struct crypto_route *prev;
struct crypto_route_head *list;
struct crypto_device *dev;
struct crypto_session_initializer ci;
};
struct crypto_session
{
struct list_head dev_queue_entry;
struct list_head main_queue_entry;
struct crypto_session_initializer ci;
struct crypto_data data;
spinlock_t lock;
struct work_struct work;
struct crypto_route_head route_list;
};
struct crypto_session *crypto_session_alloc(struct crypto_session_initializer *, struct crypto_data *);
inline void crypto_session_dequeue_main(struct crypto_session *);
inline void __crypto_session_dequeue_main(struct crypto_session *);
void crypto_session_dequeue_route(struct crypto_session *);
inline void crypto_device_get(struct crypto_device *);
inline void crypto_device_put(struct crypto_device *);
inline struct crypto_device *crypto_device_get_name(char *);
int __crypto_device_add(struct crypto_device *);
int crypto_device_add(struct crypto_device *);
void __crypto_device_remove(struct crypto_device *);
void crypto_device_remove(struct crypto_device *);
int match_initializer(struct crypto_device *, struct crypto_session_initializer *);
inline void crypto_session_insert_main(struct crypto_device *dev, struct crypto_session *s);
inline void crypto_session_insert(struct crypto_device *dev, struct crypto_session *s);
#endif /* __KERNEL__ */
#endif /* __ACRYPTO_H */
[-- Attachment #1.3: consumer.c --]
[-- Type: text/x-csrc, Size: 5584 bytes --]
/*
* consumer.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/err.h>
#include <linux/mm.h>
#include <linux/completion.h>
#include "acrypto.h"
#include "crypto_def.h"
#include "via-padlock/padlock.h"
#define KEY_LENGTH 16
static char ckey[KEY_LENGTH];
static int key_length = sizeof(ckey);
static void ctest_callback(struct crypto_session_initializer *ci, struct crypto_data *data);
static struct crypto_session_initializer ci =
{
.operation = CRYPTO_OP_ENCRYPT,
.type = CRYPTO_TYPE_AES_128,
.mode = CRYPTO_MODE_ECB,
.priority = 4,
.callback = ctest_callback,
};
static struct crypto_data cdata;
#define CSESSION_MAX 1000
static struct crypto_session *s;
static int watermark;
static struct completion callback_completed;
static void ctest_callback(struct crypto_session_initializer *ci, struct crypto_data *data)
{
int i, off, size, ssize;
struct scatterlist *sg;
unsigned char *ptr;
watermark--;
dprintk("%s: session %llu [%llu]\n", __func__, ci->id, ci->dev_id);
dprintk("src=%s, len=%d.\n", (char *)page_address(data->sg_src.page), data->sg_src.length);
sg = &data->sg_key;
ptr = (unsigned char *)page_address(sg->page);
off = sg->offset;
size = sg->length;
printk("key[%d]=", size);
for (i=0; i<size; ++i)
printk("0x%02x, ", ptr[i+off]);
printk("\n");
sg = &data->sg_src;
ptr = (unsigned char *)page_address(sg->page);
off = sg->offset;
ssize = size = sg->length;
printk("src[%d]=", size);
for (i=0; i<size; ++i)
printk("0x%02x, ", ptr[i+off]);
printk("\n");
sg = &data->sg_dst;
ptr = (unsigned char *)page_address(sg->page);
off = sg->offset;
size = sg->length;
if (size == 0)
{
dprintk("size=5, setting to %d.\n", ssize);
size = ssize;
}
printk("dst[%d]=", size);
for (i=0; i<size; ++i)
printk("0x%02x, ", ptr[i+off]);
printk("\n");
complete(&callback_completed);
}
int ctimer_func(void *data, int size, int op)
{
u8 *ptr;
if (size > PAGE_SIZE)
size = PAGE_SIZE;
ptr = (u8 *)page_address(cdata.sg_src.page);
memcpy(ptr, data, size);
cdata.sg_src.length = size;
ci.operation = op;
s = crypto_session_alloc(&ci, &cdata);
if (s)
watermark++;
else
dprintk("allocation failed.\n");
return (s)?0:-EINVAL;
}
static int alloc_sg(struct scatterlist *sg, void *data, int size)
{
sg->offset = 0;
sg->page = alloc_pages(GFP_KERNEL, get_order(size));
if (!sg->page) {
printk(KERN_ERR "Failed to allocate page.\n");
return -ENOMEM;
}
if (data)
memcpy(sg->page, data, size);
sg->length = size;
return 0;
}
int consumer_init(void)
{
int err;
char str[] = "test message qwerty asdzxc\n";
int size;
init_completion(&callback_completed);
cdata.sg_src_num = 1;
err = alloc_sg(&cdata.sg_src, NULL, PAGE_SIZE);
if (err)
goto err_out_return;
err = alloc_sg(&cdata.sg_dst, NULL, PAGE_SIZE);
if (err)
goto err_out_src;
err = alloc_sg(&cdata.sg_key, ckey, key_length);
if (err)
goto err_out_dst;
err = alloc_sg(&cdata.sg_iv, NULL, PAGE_SIZE);
if (err)
goto err_out_key;
cdata.priv_size = sizeof(struct aes_ctx);
size = 32;
err = ctimer_func(str, size, CRYPTO_OP_ENCRYPT);
if (err)
goto err_out_iv;
wait_for_completion(&callback_completed);
init_completion(&callback_completed);
err = ctimer_func(page_address(cdata.sg_dst.page), cdata.sg_dst.length, CRYPTO_OP_DECRYPT);
if (err)
goto err_out_iv;
wait_for_completion(&callback_completed);
err_out_iv:
__free_pages(cdata.sg_iv.page, get_order(1));
err_out_key:
__free_pages(cdata.sg_key.page, get_order(cdata.sg_key.length));
err_out_dst:
__free_pages(cdata.sg_dst.page, get_order(1));
err_out_src:
__free_pages(cdata.sg_src.page, get_order(1));
err_out_return:
return -ENODEV;
}
void consumer_fini(void)
{
while(watermark)
{
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(HZ);
printk(KERN_INFO "Waiting for sessions to be freed: watermark=%d.\n",
watermark);
}
if (!cdata.sg_key.length)
printk("BUG: key length is 0 in %s.\n", __func__);
__free_pages(cdata.sg_key.page, get_order(key_length));
__free_pages(cdata.sg_dst.page, get_order(1));
__free_pages(cdata.sg_src.page, get_order(1));
printk(KERN_INFO "Test crypto module consumer is unloaded.\n");
}
module_init(consumer_init);
module_exit(consumer_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Test crypto module consumer.");
[-- Attachment #1.4: crypto_conn.c --]
[-- Type: text/x-csrc, Size: 5120 bytes --]
/*
* crypto_conn.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/vmalloc.h>
#include "acrypto.h"
#include "crypto_lb.h"
#include "../connector/connector.h"
#include "crypto_conn.h"
static struct cb_id crypto_conn_id = { 0xdead, 0x0000 };
static char crypto_conn_name[] = "crconn";
static void crypto_conn_callback(void *data)
{
struct cn_msg *msg, *reply;
struct crypto_conn_data *d, *cmd;
struct crypto_device *dev;
u32 sessions;
msg = (struct cn_msg *)data;
d = (struct crypto_conn_data *)msg->data;
if (msg->len < sizeof(*d))
{
dprintk(KERN_ERR "Wrong message to crypto connector: msg->len=%u < %u.\n",
msg->len, sizeof(*d));
return;
}
if (msg->len != sizeof(*d) + d->len)
{
dprintk(KERN_ERR "Wrong message to crypto connector: msg->len=%u != %u.\n",
msg->len, sizeof(*d) + d->len);
return;
}
dev = crypto_device_get_name(d->name);
if (!dev)
{
dprintk(KERN_INFO "Crypto device %s was not found.\n", d->name);
return;
}
switch (d->cmd)
{
case CRYPTO_CONN_READ_SESSIONS:
reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + sizeof(sessions), GFP_ATOMIC);
if (reply) {
memcpy(reply, msg, sizeof(*reply));
reply->len = sizeof(*cmd) + sizeof(sessions);
/*
* See protocol description in connector.c
*/
reply->ack++;
cmd = (struct crypto_conn_data *)(reply + 1);
memcpy(cmd, d, sizeof(*cmd));
cmd->len = sizeof(sessions);
sessions = atomic_read(&dev->refcnt);
memcpy(cmd+1, &sessions, sizeof(sessions));
cn_netlink_send(reply, 0);
kfree(reply);
} else
dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
sizeof(*msg) + sizeof(*cmd), d->cmd);
break;
case CRYPTO_CONN_DUMP_QUEUE:
reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + 1024*sizeof(struct crypto_session_initializer), GFP_ATOMIC);
if (reply) {
struct crypto_session *s;
struct crypto_session_initializer *ptr;
memcpy(reply, msg, sizeof(*reply));
/*
* See protocol description in connector.c
*/
reply->ack++;
cmd = (struct crypto_conn_data *)(reply + 1);
memcpy(cmd, d, sizeof(*cmd));
ptr = (struct crypto_session_initializer *)(cmd+1);
sessions = 0;
spin_lock_bh(&dev->session_lock);
list_for_each_entry(s, &dev->session_list, dev_queue_entry)
{
memcpy(ptr, &s->ci, sizeof(*ptr));
sessions++;
ptr++;
if (sessions >= 1024)
break;
}
spin_unlock_bh(&dev->session_lock);
cmd->len = sizeof(*ptr)*sessions;
reply->len = sizeof(*cmd) + cmd->len;
cn_netlink_send(reply, 0);
kfree(reply);
} else
dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
sizeof(*msg) + sizeof(*cmd), d->cmd);
break;
case CRYPTO_GET_STAT:
reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + sizeof(struct crypto_device_stat), GFP_ATOMIC);
if (reply) {
struct crypto_device_stat *ptr;
memcpy(reply, msg, sizeof(*reply));
reply->len = sizeof(*cmd) + sizeof(*ptr);
/*
* See protocol description in connector.c
*/
reply->ack++;
cmd = (struct crypto_conn_data *)(reply + 1);
memcpy(cmd, d, sizeof(*cmd));
cmd->len = sizeof(*ptr);
ptr = (struct crypto_device_stat *)(cmd+1);
memcpy(ptr, &dev->stat, sizeof(*ptr));
cn_netlink_send(reply, 0);
kfree(reply);
} else
dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
sizeof(*msg) + sizeof(*cmd), d->cmd);
break;
default:
dprintk(KERN_ERR "Wrong operation 0x%04x for crypto connector.\n",
d->cmd);
return;
}
crypto_device_put(dev);
}
int crypto_conn_init(void)
{
int err;
err = cn_add_callback(&crypto_conn_id, crypto_conn_name, crypto_conn_callback);
if (err)
return err;
dprintk(KERN_INFO "Crypto connector callback is registered.\n");
return 0;
}
void crypto_conn_fini(void)
{
cn_del_callback(&crypto_conn_id);
dprintk(KERN_INFO "Crypto connector callback is unregistered.\n");
}
[-- Attachment #1.5: crypto_conn.h --]
[-- Type: text/x-chdr, Size: 1261 bytes --]
/*
* crypto_conn.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __CRYPTO_CONN_H
#define __CRYPTO_CONN_H
#include "acrypto.h"
#define CRYPTO_CONN_READ_SESSIONS 0
#define CRYPTO_CONN_DUMP_QUEUE 1
#define CRYPTO_GET_STAT 2
struct crypto_conn_data
{
char name[SCACHE_NAMELEN];
__u16 cmd;
__u16 len;
__u8 data[0];
};
#ifdef __KERNEL__
int crypto_conn_init(void);
void crypto_conn_fini(void);
#endif /* __KERNEL__ */
#endif /* __CRYPTO_CONN_H */
[-- Attachment #1.6: crypto_def.h --]
[-- Type: text/x-chdr, Size: 1203 bytes --]
/*
* crypto_def.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __CRYPTO_DEF_H
#define __CRYPTO_DEF_H
#define CRYPTO_OP_DECRYPT 0
#define CRYPTO_OP_ENCRYPT 1
#define CRYPTO_OP_HMAC 2
#define CRYPTO_MODE_ECB 0
#define CRYPTO_MODE_CBC 1
#define CRYPTO_MODE_CFB 2
#define CRYPTO_MODE_OFB 3
#define CRYPTO_TYPE_AES_128 0
#define CRYPTO_TYPE_AES_192 1
#define CRYPTO_TYPE_AES_256 2
#endif /* __CRYPTO_DEF_H */
[-- Attachment #1.7: crypto_dev.c --]
[-- Type: text/x-csrc, Size: 10412 bytes --]
/*
* crypto_dev.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/device.h>
#include "acrypto.h"
static LIST_HEAD(cdev_list);
static spinlock_t cdev_lock = SPIN_LOCK_UNLOCKED;
static u32 cdev_ids;
struct list_head *crypto_device_list = &cdev_list;
spinlock_t *crypto_device_lock = &cdev_lock;
static int crypto_match(struct device *dev, struct device_driver *drv)
{
return 1;
}
static int crypto_probe(struct device *dev)
{
return -ENODEV;
}
static int crypto_remove(struct device *dev)
{
return 0;
}
static void crypto_release(struct device *dev)
{
struct crypto_device *d = container_of(dev, struct crypto_device, device);
complete(&d->dev_released);
}
static void crypto_class_release(struct class *class)
{
}
static void crypto_class_release_device(struct class_device *class_dev)
{
}
struct class crypto_class =
{
.name = "acrypto",
.class_release = crypto_class_release,
.release = crypto_class_release_device
};
struct bus_type crypto_bus_type =
{
.name = "acrypto",
.match = crypto_match
};
struct device_driver crypto_driver = {
.name = "crypto_driver",
.bus = &crypto_bus_type,
.probe = crypto_probe,
.remove = crypto_remove,
};
struct device crypto_dev = {
.parent = NULL,
.bus = &crypto_bus_type,
.bus_id = "Asynchronous crypto",
.driver = &crypto_driver,
.release = &crypto_release
};
static ssize_t sessions_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%d\n", atomic_read(&d->refcnt));
}
static ssize_t name_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%s\n", d->name);
}
static ssize_t devices_show(struct class_device *dev, char *buf)
{
struct crypto_device *d;
int off = 0;
spin_lock_bh(&cdev_lock);
list_for_each_entry(d, &cdev_list, cdev_entry)
{
off += sprintf(buf+off, "%s ", d->name);
}
spin_unlock_bh(&cdev_lock);
if (!off)
off = sprintf(buf, "No devices registered yet.");
off += sprintf(buf+off, "\n");
return off;
}
static ssize_t kmem_failed_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%llu\n", d->stat.kmem_failed);
}
static ssize_t sstarted_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%llu\n", d->stat.sstarted);
}
static ssize_t sfinished_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%llu\n", d->stat.sfinished);
}
static ssize_t scompleted_show(struct class_device *dev, char *buf)
{
struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
return sprintf(buf, "%llu\n", d->stat.scompleted);
}
static CLASS_DEVICE_ATTR(sessions, 0444, sessions_show, NULL);
static CLASS_DEVICE_ATTR(name, 0444, name_show, NULL);
CLASS_DEVICE_ATTR(devices, 0444, devices_show, NULL);
static CLASS_DEVICE_ATTR(scompleted, 0444, scompleted_show, NULL);
static CLASS_DEVICE_ATTR(sstarted, 0444, sstarted_show, NULL);
static CLASS_DEVICE_ATTR(sfinished, 0444, sfinished_show, NULL);
static CLASS_DEVICE_ATTR(kmem_failed, 0444, kmem_failed_show, NULL);
static inline int compare_device(struct crypto_device *d1, struct crypto_device *d2)
{
if (!strncmp(d1->name, d2->name, sizeof(d1->name)))
return 1;
return 0;
}
static void create_device_attributes(struct crypto_device *dev)
{
class_device_create_file(&dev->class_device, &class_device_attr_sessions);
class_device_create_file(&dev->class_device, &class_device_attr_name);
class_device_create_file(&dev->class_device, &class_device_attr_scompleted);
class_device_create_file(&dev->class_device, &class_device_attr_sstarted);
class_device_create_file(&dev->class_device, &class_device_attr_sfinished);
class_device_create_file(&dev->class_device, &class_device_attr_kmem_failed);
}
static void remove_device_attributes(struct crypto_device *dev)
{
class_device_remove_file(&dev->class_device, &class_device_attr_sessions);
class_device_remove_file(&dev->class_device, &class_device_attr_name);
class_device_remove_file(&dev->class_device, &class_device_attr_scompleted);
class_device_remove_file(&dev->class_device, &class_device_attr_sstarted);
class_device_remove_file(&dev->class_device, &class_device_attr_sfinished);
class_device_remove_file(&dev->class_device, &class_device_attr_kmem_failed);
}
static int __match_initializer(struct crypto_capability *cap, struct crypto_session_initializer *ci)
{
if ( cap->operation == ci->operation &&
cap->type == ci->type &&
cap->mode == (ci->mode & 0x1fff))
return 1;
return 0;
}
int match_initializer(struct crypto_device *dev, struct crypto_session_initializer *ci)
{
int i;
for (i=0; i<dev->cap_number; ++i)
{
struct crypto_capability *cap = &dev->cap[i];
if (__match_initializer(cap, ci))
{
if (cap->qlen >= atomic_read(&dev->refcnt) + 1)
return 1;
}
}
return 0;
}
inline void crypto_device_get(struct crypto_device *dev)
{
atomic_inc(&dev->refcnt);
}
inline struct crypto_device *crypto_device_get_name(char *name)
{
struct crypto_device *dev;
int found = 0;
spin_lock_bh(&cdev_lock);
list_for_each_entry(dev, &cdev_list, cdev_entry)
{
if (!strcmp(dev->name, name))
{
found = 1;
crypto_device_get(dev);
break;
}
}
spin_unlock_bh(&cdev_lock);
if (!found)
return NULL;
return dev;
}
inline void crypto_device_put(struct crypto_device *dev)
{
atomic_dec(&dev->refcnt);
}
int __crypto_device_add(struct crypto_device *dev)
{
int err;
memset(&dev->stat, 0, sizeof(dev->stat));
spin_lock_init(&dev->stat_lock);
spin_lock_init(&dev->lock);
spin_lock_init(&dev->session_lock);
INIT_LIST_HEAD(&dev->session_list);
atomic_set(&dev->refcnt, 0);
dev->sid = 0;
dev->flags = 0;
init_completion(&dev->dev_released);
memcpy(&dev->device, &crypto_dev, sizeof(struct device));
dev->driver = &crypto_driver;
snprintf(dev->device.bus_id, sizeof(dev->device.bus_id), "%s", dev->name);
err = device_register(&dev->device);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto device %s: err=%d.\n",
dev->name, err);
return err;
}
snprintf(dev->class_device.class_id, sizeof(dev->class_device.class_id), "%s", dev->name);
dev->class_device.dev = &dev->device;
dev->class_device.class = &crypto_class;
err = class_device_register(&dev->class_device);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto class device %s: err=%d.\n",
dev->name, err);
device_unregister(&dev->device);
return err;
}
create_device_attributes(dev);
return 0;
}
void __crypto_device_remove(struct crypto_device *dev)
{
remove_device_attributes(dev);
class_device_unregister(&dev->class_device);
device_unregister(&dev->device);
}
int crypto_device_add(struct crypto_device *dev)
{
int err;
err = __crypto_device_add(dev);
if (err)
return err;
spin_lock_bh(&cdev_lock);
list_add(&dev->cdev_entry, &cdev_list);
dev->id = ++cdev_ids;
spin_unlock_bh(&cdev_lock);
dprintk(KERN_INFO "Crypto device %s was registered with ID=%x.\n", dev->name, dev->id);
return 0;
}
void crypto_device_remove(struct crypto_device *dev)
{
struct crypto_device *__dev, *n;
unsigned long flags;
__crypto_device_remove(dev);
spin_lock_irqsave(&cdev_lock, flags);
list_for_each_entry_safe(__dev, n, &cdev_list, cdev_entry)
{
if (compare_device(__dev, dev))
{
list_del(&__dev->cdev_entry);
spin_unlock_irqrestore(&cdev_lock, flags);
/*
* In test cases or when crypto device driver is not written correctly,
* it's ->data_ready() method will not be callen anymore
* after device is removed from crypto device list.
*
* For such cases we either should provide ->flush() call
* or properly write ->data_ready() method.
*/
while (atomic_read(&__dev->refcnt))
{
dprintk(KERN_INFO "Waiting for %s to become free: refcnt=%d.\n",
__dev->name, atomic_read(&dev->refcnt));
/*
* Hack zone: you need to write good ->data_ready()
* and crypto device driver itself.
*
* Driver shoud not buzz if it has pending sessions
* in it's queue and it was removed from global device list.
*
* Although I can workaround it here, for example by
* flushing the whole queue and drop all pending sessions.
*/
__dev->data_ready(__dev);
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(HZ);
}
dprintk(KERN_ERR "Crypto device %s was unregistered.\n", dev->name);
return;
}
}
spin_unlock_irqrestore(&cdev_lock, flags);
dprintk(KERN_ERR "Crypto device %s was not registered.\n", dev->name);
}
EXPORT_SYMBOL(crypto_device_add);
EXPORT_SYMBOL(crypto_device_remove);
EXPORT_SYMBOL(crypto_device_get);
EXPORT_SYMBOL(crypto_device_get_name);
EXPORT_SYMBOL(crypto_device_put);
EXPORT_SYMBOL(match_initializer);
[-- Attachment #1.8: crypto_lb.c --]
[-- Type: text/x-csrc, Size: 15618 bytes --]
/*
* crypto_lb.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/err.h>
#include "acrypto.h"
#include "crypto_lb.h"
#include "crypto_stat.h"
#include "crypto_route.h"
static LIST_HEAD(crypto_lb_list);
static spinlock_t crypto_lb_lock = SPIN_LOCK_UNLOCKED;
static int lb_num = 0;
static struct crypto_lb *current_lb, *default_lb;
static struct completion thread_exited;
static int need_exit;
static struct workqueue_struct *crypto_lb_queue;
static DECLARE_WAIT_QUEUE_HEAD(crypto_lb_wait_queue);
extern struct list_head *crypto_device_list;
extern spinlock_t *crypto_device_lock;
extern int force_lb_remove;
extern struct crypto_device main_crypto_device;
static inline int lb_is_current(struct crypto_lb *l)
{
return (l->crypto_device_list != NULL && l->crypto_device_lock != NULL);
}
static inline int lb_is_default(struct crypto_lb *l)
{
return (l == default_lb);
}
static inline void __lb_set_current(struct crypto_lb *l)
{
struct crypto_lb *c = current_lb;
if (c)
{
l->crypto_device_list = crypto_device_list;
l->crypto_device_lock = crypto_device_lock;
current_lb = l;
c->crypto_device_list = NULL;
c->crypto_device_lock = NULL;
}
else
{
l->crypto_device_list = crypto_device_list;
l->crypto_device_lock = crypto_device_lock;
current_lb = l;
}
}
static inline void lb_set_current(struct crypto_lb *l)
{
struct crypto_lb *c = current_lb;
if (c)
{
spin_lock_bh(&c->lock);
__lb_set_current(l);
spin_unlock_bh(&c->lock);
}
else
__lb_set_current(l);
}
static inline void __lb_set_default(struct crypto_lb *l)
{
default_lb = l;
}
static inline void lb_set_default(struct crypto_lb *l)
{
struct crypto_lb *c = default_lb;
if (c)
{
spin_lock_bh(&c->lock);
__lb_set_default(l);
spin_unlock_bh(&c->lock);
}
else
__lb_set_default(l);
}
static int crypto_lb_match(struct device *dev, struct device_driver *drv)
{
return 1;
}
static int crypto_lb_probe(struct device *dev)
{
return -ENODEV;
}
static int crypto_lb_remove(struct device *dev)
{
return 0;
}
static void crypto_lb_release(struct device *dev)
{
struct crypto_lb *d = container_of(dev, struct crypto_lb, device);
complete(&d->dev_released);
}
static void crypto_lb_class_release(struct class *class)
{
}
static void crypto_lb_class_release_device(struct class_device *class_dev)
{
}
struct class crypto_lb_class =
{
.name = "crypto_lb",
.class_release = crypto_lb_class_release,
.release = crypto_lb_class_release_device
};
struct bus_type crypto_lb_bus_type =
{
.name = "crypto_lb",
.match = crypto_lb_match
};
struct device_driver crypto_lb_driver = {
.name = "crypto_lb_driver",
.bus = &crypto_lb_bus_type,
.probe = crypto_lb_probe,
.remove = crypto_lb_remove,
};
struct device crypto_lb_dev = {
.parent = NULL,
.bus = &crypto_lb_bus_type,
.bus_id = "crypto load balancer",
.driver = &crypto_lb_driver,
.release = &crypto_lb_release
};
static ssize_t name_show(struct class_device *dev, char *buf)
{
struct crypto_lb *lb = container_of(dev, struct crypto_lb, class_device);
return sprintf(buf, "%s\n", lb->name);
}
static ssize_t current_show(struct class_device *dev, char *buf)
{
struct crypto_lb *lb;
int off = 0;
spin_lock_bh(&crypto_lb_lock);
list_for_each_entry(lb, &crypto_lb_list, lb_entry)
{
if (lb_is_current(lb))
off += sprintf(buf+off, "[");
if (lb_is_default(lb))
off += sprintf(buf+off, "(");
off += sprintf(buf+off, "%s", lb->name);
if (lb_is_default(lb))
off += sprintf(buf+off, ")");
if (lb_is_current(lb))
off += sprintf(buf+off, "]");
}
spin_unlock_bh(&crypto_lb_lock);
if (!off)
off = sprintf(buf, "No load balancers regitered yet.");
off += sprintf(buf+off, "\n");
return off;
}
static ssize_t current_store(struct class_device *dev, const char *buf, size_t count)
{
struct crypto_lb *lb;
spin_lock_bh(&crypto_lb_lock);
list_for_each_entry(lb, &crypto_lb_list, lb_entry)
{
if (count == strlen(lb->name) && !strcmp(buf, lb->name))
{
lb_set_current(lb);
lb_set_default(lb);
dprintk(KERN_INFO "Load balancer %s is set as current and default.\n",
lb->name);
break;
}
}
spin_unlock_bh(&crypto_lb_lock);
return count;
}
static CLASS_DEVICE_ATTR(name, 0444, name_show, NULL);
CLASS_DEVICE_ATTR(lbs, 0644, current_show, current_store);
static void create_device_attributes(struct crypto_lb *lb)
{
class_device_create_file(&lb->class_device, &class_device_attr_name);
}
static void remove_device_attributes(struct crypto_lb *lb)
{
class_device_remove_file(&lb->class_device, &class_device_attr_name);
}
static inline int compare_lb(struct crypto_lb *l1, struct crypto_lb *l2)
{
if (!strncmp(l1->name, l2->name, sizeof(l1->name)))
return 1;
return 0;
}
inline void crypto_lb_rehash(void)
{
if (!current_lb)
return;
spin_lock_bh(¤t_lb->lock);
current_lb->rehash(current_lb);
spin_unlock_bh(¤t_lb->lock);
wake_up_interruptible(&crypto_lb_wait_queue);
}
struct crypto_device *crypto_lb_find_device(struct crypto_session_initializer *ci)
{
struct crypto_device *dev;
if (!current_lb)
return NULL;
if (sci_binded(ci))
{
int found = 0;
spin_lock_bh(crypto_device_lock);
list_for_each_entry(dev, crypto_device_list, cdev_entry)
{
if (dev->id == ci->bdev)
{
found = 1;
break;
}
}
spin_unlock_bh(crypto_device_lock);
return (found)?dev:NULL;
}
spin_lock_bh(¤t_lb->lock);
current_lb->rehash(current_lb);
spin_lock(crypto_device_lock);
dev = current_lb->find_device(current_lb, ci);
if (dev)
crypto_device_get(dev);
spin_unlock(crypto_device_lock);
spin_unlock_bh(¤t_lb->lock);
wake_up_interruptible(&crypto_lb_wait_queue);
return dev;
}
static int __crypto_lb_register(struct crypto_lb *lb)
{
int err;
spin_lock_init(&lb->lock);
init_completion(&lb->dev_released);
memcpy(&lb->device, &crypto_lb_dev, sizeof(struct device));
lb->driver = &crypto_lb_driver;
snprintf(lb->device.bus_id, sizeof(lb->device.bus_id), "%s", lb->name);
err = device_register(&lb->device);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto load balancer device %s: err=%d.\n",
lb->name, err);
return err;
}
snprintf(lb->class_device.class_id, sizeof(lb->class_device.class_id), "%s", lb->name);
lb->class_device.dev = &lb->device;
lb->class_device.class = &crypto_lb_class;
err = class_device_register(&lb->class_device);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto load balancer class device %s: err=%d.\n",
lb->name, err);
device_unregister(&lb->device);
return err;
}
create_device_attributes(lb);
wake_up_interruptible(&crypto_lb_wait_queue);
return 0;
}
static void __crypto_lb_unregister(struct crypto_lb *lb)
{
wake_up_interruptible(&crypto_lb_wait_queue);
remove_device_attributes(lb);
class_device_unregister(&lb->class_device);
device_unregister(&lb->device);
}
int crypto_lb_register(struct crypto_lb *lb, int set_current, int set_default)
{
struct crypto_lb *__lb;
int err;
spin_lock_bh(&crypto_lb_lock);
list_for_each_entry(__lb, &crypto_lb_list, lb_entry)
{
if (unlikely(compare_lb(__lb, lb)))
{
spin_unlock_bh(&crypto_lb_lock);
dprintk(KERN_ERR "Crypto load balancer %s is already registered.\n", lb->name);
return -EINVAL;
}
}
list_add(&lb->lb_entry, &crypto_lb_list);
spin_unlock_bh(&crypto_lb_lock);
err = __crypto_lb_register(lb);
if (err)
{
spin_lock_bh(&crypto_lb_lock);
list_del(&lb->lb_entry);
spin_unlock_bh(&crypto_lb_lock);
return err;
}
if (!default_lb || set_default)
lb_set_default(lb);
if (!current_lb || set_current)
lb_set_current(lb);
dprintk(KERN_INFO "Crypto load balancer %s was registered and set to be [%s.%s].\n",
lb->name,
(lb_is_current(lb))?"current":"not current",
(lb_is_default(lb))?"default":"not default");
lb_num++;
return 0;
}
void crypto_lb_unregister(struct crypto_lb *lb)
{
struct crypto_lb *__lb, *n;
if (lb_num == 1)
{
dprintk(KERN_INFO "You are removing crypto load balancer %s which is current and default.\n"
"There is no other crypto load balancers. "
"Removing %s delayed untill new load balancer is registered.\n",
lb->name, (force_lb_remove)?"is not":"is");
while (lb_num == 1 && !force_lb_remove)
{
set_current_state(TASK_INTERRUPTIBLE);
schedule_timeout(HZ);
if (signal_pending(current))
flush_signals(current);
}
}
__crypto_lb_unregister(lb);
spin_lock_bh(&crypto_lb_lock);
list_for_each_entry_safe(__lb, n, &crypto_lb_list, lb_entry)
{
if (compare_lb(__lb, lb))
{
lb_num--;
list_del(&__lb->lb_entry);
dprintk(KERN_ERR "Crypto load balancer %s was unregistered.\n", lb->name);
}
else if (lb_num)
{
if (lb_is_default(lb))
lb_set_default(__lb);
if (lb_is_current(lb))
lb_set_current(default_lb);
}
}
spin_unlock_bh(&crypto_lb_lock);
}
static void crypto_lb_queue_wrapper(void *data)
{
struct crypto_device *dev = &main_crypto_device;
struct crypto_session *s = (struct crypto_session *)data;
dprintk(KERN_INFO "%s: Calling callback for session %llu [%llu] flags=%x, "
"op=%04u, type=%04x, mode=%04x, priority=%04x\n",
__func__,
s->ci.id, s->ci.dev_id, s->ci.flags,
s->ci.operation, s->ci.type, s->ci.mode, s->ci.priority);
/*
* Prevent races with crypto devices
* which may change flags of the sessions in theirs queues.
*/
spin_lock(&s->lock);
crypto_stat_finish_inc(s);
finish_session(s);
unstart_session(s);
spin_unlock(&s->lock);
s->ci.callback(&s->ci, &s->data);
if (session_finished(s))
{
crypto_session_dequeue_route(s);
kfree(s);
}
else
{
/*
* Special case: crypto consumer marks session as "not finished"
* in it's callback - it means that crypto consumer wants
* this session to be processed further,
* for example crypto consumer can add new route and then
* mark session as "not finished".
*/
uncomplete_session(s);
unstart_session(s);
crypto_session_insert_main(dev, s);
}
wake_up_interruptible(&crypto_lb_wait_queue);
}
static void crypto_lb_process_next_route(struct crypto_session *s)
{
struct crypto_route *rt;
struct crypto_device *dev;
rt = crypto_route_dequeue(s);
if (rt)
{
dev = rt->dev;
spin_lock_bh(&dev->session_lock);
list_del(&s->dev_queue_entry);
spin_unlock_bh(&dev->session_lock);
crypto_route_free(rt);
dev = crypto_route_get_current_device(s);
if (dev)
{
dprintk(KERN_INFO "%s: processing new route to %s.\n",
__func__, dev->name);
memcpy(&s->ci, &rt->ci, sizeof(s->ci));
crypto_session_insert(dev, s);
/*
* Reference to this device was already hold when
* new routing was added.
*/
crypto_device_put(dev);
}
}
}
int crypto_lb_thread(void *data)
{
struct crypto_session *s, *n;
struct crypto_device *dev = (struct crypto_device *)data;
daemonize("%s", dev->name);
allow_signal(SIGTERM);
while (!need_exit)
{
spin_lock_bh(&dev->session_lock);
list_for_each_entry_safe(s, n, &dev->session_list, main_queue_entry)
{
printk("session %llu [%llu]: flags=%x, route_num=%d, %s,%s,%s,%s.\n",
s->ci.id, s->ci.dev_id, s->ci.flags,
crypto_route_queue_len(s),
(session_completed(s))?"completed": "not completed",
(session_finished(s))?"finished": "not finished",
(session_started(s))?"started": "not started",
(session_is_processed(s))?"is being processed": "is not being processed");
if (!spin_trylock(&s->lock))
continue;
if (session_is_processed(s))
goto unlock;
if (session_started(s))
goto unlock;
if (session_completed(s))
{
uncomplete_session(s);
if (crypto_route_queue_len(s) > 1)
{
crypto_lb_process_next_route(s);
}
else
{
start_session(s);
crypto_stat_start_inc(s);
__crypto_session_dequeue_main(s);
spin_unlock(&s->lock);
INIT_WORK(&s->work, &crypto_lb_queue_wrapper, s);
queue_work(crypto_lb_queue, &s->work);
continue;
}
}
unlock:
spin_unlock(&s->lock);
}
spin_unlock_bh(&dev->session_lock);
interruptible_sleep_on_timeout(&crypto_lb_wait_queue, 1000);
}
flush_workqueue(crypto_lb_queue);
complete_and_exit(&thread_exited, 0);
}
int crypto_lb_init(void)
{
int err;
long pid;
err = bus_register(&crypto_lb_bus_type);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto load balancer bus: err=%d.\n", err);
goto err_out_exit;
}
err = driver_register(&crypto_lb_driver);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto load balancer driver: err=%d.\n", err);
goto err_out_bus_unregister;
}
crypto_lb_class.class_dev_attrs = &class_device_attr_lbs;
err = class_register(&crypto_lb_class);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto load balancer class: err=%d.\n", err);
goto err_out_driver_unregister;
}
crypto_lb_queue = create_workqueue("clbq");
if (!crypto_lb_queue)
{
dprintk(KERN_ERR "Failed to create crypto load balaner work queue.\n");
goto err_out_class_unregister;
}
init_completion(&thread_exited);
pid = kernel_thread(crypto_lb_thread, &main_crypto_device, CLONE_FS | CLONE_FILES);
if (IS_ERR((void *)pid))
{
dprintk(KERN_ERR "Failed to create kernel load balancing thread.\n");
goto err_out_destroy_workqueue;
}
return 0;
err_out_destroy_workqueue:
destroy_workqueue(crypto_lb_queue);
err_out_class_unregister:
class_unregister(&crypto_lb_class);
err_out_driver_unregister:
driver_unregister(&crypto_lb_driver);
err_out_bus_unregister:
bus_unregister(&crypto_lb_bus_type);
err_out_exit:
return err;
}
void crypto_lb_fini(void)
{
need_exit = 1;
wait_for_completion(&thread_exited);
destroy_workqueue(crypto_lb_queue);
class_unregister(&crypto_lb_class);
driver_unregister(&crypto_lb_driver);
bus_unregister(&crypto_lb_bus_type);
}
EXPORT_SYMBOL(crypto_lb_register);
EXPORT_SYMBOL(crypto_lb_unregister);
EXPORT_SYMBOL(crypto_lb_rehash);
EXPORT_SYMBOL(crypto_lb_find_device);
[-- Attachment #1.9: crypto_lb.h --]
[-- Type: text/x-chdr, Size: 1736 bytes --]
/*
* crypto_lb.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __CRYPTO_LB_H
#define __CRYPTO_LB_H
#include "acrypto.h"
#define CRYPTO_LB_NAMELEN 32
struct crypto_lb
{
struct list_head lb_entry;
char name[CRYPTO_LB_NAMELEN];
void (*rehash)(struct crypto_lb *);
struct crypto_device * (*find_device)(struct crypto_lb *, struct crypto_session_initializer *);
spinlock_t lock;
spinlock_t *crypto_device_lock;
struct list_head *crypto_device_list;
struct device_driver *driver;
struct device device;
struct class_device class_device;
struct completion dev_released;
};
int crypto_lb_register(struct crypto_lb *lb, int set_current, int set_default);
void crypto_lb_unregister(struct crypto_lb *lb);
inline void crypto_lb_rehash(void);
struct crypto_device *crypto_lb_find_device(struct crypto_session_initializer *ci);
int crypto_lb_init(void);
void crypto_lb_fini(void);
#endif /* __CRYPTO_LB_H */
[-- Attachment #1.10: crypto_main.c --]
[-- Type: text/x-csrc, Size: 8030 bytes --]
/*
* crypto_main.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include "acrypto.h"
#include "crypto_lb.h"
#include "crypto_conn.h"
#include "crypto_route.h"
int force_lb_remove;
module_param(force_lb_remove, int, 0);
struct crypto_device main_crypto_device;
extern struct bus_type crypto_bus_type;
extern struct device_driver crypto_driver;
extern struct class crypto_class;
extern struct device crypto_dev;
extern struct class_device_attribute class_device_attr_devices;
extern struct class_device_attribute class_device_attr_lbs;
static inline void dump_ci(struct crypto_session_initializer *ci)
{
dprintk("%llu [%llu] op=%04u, type=%04x, mode=%04x, priority=%04x",
ci->id, ci->dev_id,
ci->operation, ci->type, ci->mode, ci->priority);
}
static void __crypto_session_insert(struct crypto_device *dev, struct crypto_session *s)
{
struct crypto_session *__s;
if (unlikely(list_empty(&dev->session_list)))
{
list_add(&s->dev_queue_entry, &dev->session_list);
}
else
{
int inserted = 0;
list_for_each_entry(__s, &dev->session_list, dev_queue_entry)
{
if (__s->ci.priority < s->ci.priority)
{
list_add_tail(&s->dev_queue_entry, &__s->dev_queue_entry);
inserted = 1;
break;
}
}
if (!inserted)
list_add_tail(&s->dev_queue_entry, &dev->session_list);
}
dump_ci(&s->ci);
dprintk(" added to crypto device %s [%d].\n", dev->name, atomic_read(&dev->refcnt));
}
inline void crypto_session_insert_main(struct crypto_device *dev, struct crypto_session *s)
{
struct crypto_session *__s;
spin_lock_bh(&dev->session_lock);
crypto_device_get(dev);
if (unlikely(list_empty(&dev->session_list)))
{
list_add(&s->main_queue_entry, &dev->session_list);
}
else
{
int inserted = 0;
list_for_each_entry(__s, &dev->session_list, main_queue_entry)
{
if (__s->ci.priority < s->ci.priority)
{
list_add_tail(&s->main_queue_entry, &__s->main_queue_entry);
inserted = 1;
break;
}
}
if (!inserted)
list_add_tail(&s->main_queue_entry, &dev->session_list);
}
spin_unlock_bh(&dev->session_lock);
dump_ci(&s->ci);
dprintk(" added to main crypto device %s [%d].\n", dev->name, atomic_read(&dev->refcnt));
}
inline void crypto_session_insert(struct crypto_device *dev, struct crypto_session *s)
{
spin_lock_bh(&dev->session_lock);
__crypto_session_insert(dev, s);
spin_unlock_bh(&dev->session_lock);
if (dev->data_ready)
dev->data_ready(dev);
}
struct crypto_session *crypto_session_alloc(struct crypto_session_initializer *ci, struct crypto_data *d)
{
struct crypto_device *dev = &main_crypto_device;
struct crypto_device *ldev;
struct crypto_session *s;
int err;
if (d->priv_size > CRYPTO_MAX_PRIV_SIZE)
{
dprintk("priv_size %u is too big, maximum allowed %u.\n", d->priv_size, CRYPTO_MAX_PRIV_SIZE);
return NULL;
}
ldev = crypto_lb_find_device(ci);
if (!ldev)
{
dprintk("Cannot find suitable device.\n");
return NULL;
}
s = kmalloc(sizeof(*s) + d->priv_size, GFP_ATOMIC);
if (!s)
{
ldev->stat.kmem_failed++;
goto err_out_device_put;
}
memset(s, 0xAB, sizeof(*s));
crypto_route_head_init(&s->route_list);
spin_lock_init(&s->lock);
memcpy(&s->ci, ci, sizeof(s->ci));
memcpy(&s->data, d, sizeof(s->data));
if (d->priv_size)
{
s->data.priv = s+1;
if (d->priv)
memcpy(s->data.priv, d->priv, d->priv_size);
}
else
s->data.priv = NULL;
s->ci.id = dev->sid++;
s->ci.dev_id = ldev->sid++;
s->ci.flags = 0;
#if 1
err = crypto_route_add(ldev, s, ci);
if (err)
{
dprintk("Can not add route to device %s.\n", ldev->name);
goto err_out_session_free;
}
crypto_session_insert(ldev, s);
crypto_device_put(ldev);
#endif
crypto_session_insert_main(dev, s);
return s;
err_out_session_free:
crypto_device_put(ldev);
err_out_device_put:
kfree(s);
return NULL;
}
void crypto_session_dequeue_route(struct crypto_session *s)
{
struct crypto_route *rt;
struct crypto_device *dev;
BUG_ON(crypto_route_queue_len(s) > 1);
while ((rt = crypto_route_dequeue(s)))
{
dev = rt->dev;
dprintk(KERN_INFO "Removing route entry for device %s.\n", dev->name);
spin_lock_bh(&dev->session_lock);
list_del(&s->dev_queue_entry);
spin_unlock_bh(&dev->session_lock);
crypto_route_free(rt);
}
}
inline void __crypto_session_dequeue_main(struct crypto_session *s)
{
struct crypto_device *dev = &main_crypto_device;
list_del(&s->main_queue_entry);
crypto_device_put(dev);
}
inline void crypto_session_dequeue_main(struct crypto_session *s)
{
struct crypto_device *dev = &main_crypto_device;
spin_lock_bh(&dev->session_lock);
__crypto_session_dequeue_main(s);
spin_unlock_bh(&dev->session_lock);
}
int __devinit cmain_init(void)
{
struct crypto_device *dev = &main_crypto_device;
int err;
snprintf(dev->name, sizeof(dev->name), "crypto_sessions");
err = bus_register(&crypto_bus_type);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto bus: err=%d.\n", err);
return err;
}
err = driver_register(&crypto_driver);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto driver: err=%d.\n", err);
goto err_out_bus_unregister;
}
err = class_register(&crypto_class);
if (err)
{
dprintk(KERN_ERR "Failed to register crypto class: err=%d.\n", err);
goto err_out_driver_unregister;
}
err = crypto_lb_init();
if (err)
goto err_out_class_unregister;
err = crypto_conn_init();
if (err)
goto err_out_crypto_lb_fini;
err = __crypto_device_add(dev);
if (err)
goto err_out_crypto_conn_fini;
err = class_device_create_file(&dev->class_device, &class_device_attr_devices);
if (err)
dprintk("Failed to create \"devices\" attribute: err=%d.\n", err);
err = class_device_create_file(&dev->class_device, &class_device_attr_lbs);
if (err)
dprintk("Failed to create \"lbs\" attribute: err=%d.\n", err);
return 0;
err_out_crypto_conn_fini:
crypto_conn_fini();
err_out_crypto_lb_fini:
crypto_lb_fini();
err_out_class_unregister:
class_unregister(&crypto_class);
err_out_driver_unregister:
driver_unregister(&crypto_driver);
err_out_bus_unregister:
bus_unregister(&crypto_bus_type);
return err;
}
void __devexit cmain_fini(void)
{
struct crypto_device *dev = &main_crypto_device;
class_device_remove_file(&dev->class_device, &class_device_attr_devices);
class_device_remove_file(&dev->class_device, &class_device_attr_lbs);
__crypto_device_remove(dev);
crypto_conn_fini();
crypto_lb_fini();
class_unregister(&crypto_class);
driver_unregister(&crypto_driver);
bus_unregister(&crypto_bus_type);
}
module_init(cmain_init);
module_exit(cmain_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Asynchronous crypto layer.");
EXPORT_SYMBOL(crypto_session_alloc);
[-- Attachment #1.11: crypto_route.h --]
[-- Type: text/x-chdr, Size: 5024 bytes --]
/*
* crypto_route.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __CRYPTO_ROUTE_H
#define __CRYPTO_ROUTE_H
#include <linux/types.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include "acrypto.h"
static inline struct crypto_route *crypto_route_alloc(struct crypto_device *dev, struct crypto_session_initializer *ci)
{
struct crypto_route *rt;
crypto_device_get(dev);
if (!match_initializer(dev, ci))
{
crypto_device_put(dev);
return NULL;
}
rt = kmalloc(sizeof(*rt), GFP_ATOMIC);
if (!rt)
{
crypto_device_put(dev);
return NULL;
}
memset(rt, 0, sizeof(*rt));
memcpy(&rt->ci, ci, sizeof(*ci));
rt->dev = dev;
return rt;
}
static inline void crypto_route_free(struct crypto_route *rt)
{
crypto_device_put(rt->dev);
rt->dev = NULL;
kfree(rt);
}
static inline void __crypto_route_del(struct crypto_route *rt, struct crypto_route_head *list)
{
struct crypto_route *next, *prev;
list->qlen--;
next = rt->next;
prev = rt->prev;
rt->next = rt->prev = NULL;
rt->list = NULL;
next->prev = prev;
prev->next = next;
}
static inline void crypto_route_del(struct crypto_route *rt)
{
struct crypto_route_head *list = rt->list;
if (list)
{
spin_lock_bh(&list->lock);
if (list == rt->list)
__crypto_route_del(rt, rt->list);
spin_unlock_bh(&list->lock);
crypto_route_free(rt);
}
}
static inline struct crypto_route *__crypto_route_dequeue(struct crypto_route_head *list)
{
struct crypto_route *next, *prev, *result;
prev = (struct crypto_route *) list;
next = prev->next;
result = NULL;
if (next != prev) {
result = next;
next = next->next;
list->qlen--;
next->prev = prev;
prev->next = next;
result->next = result->prev = NULL;
result->list = NULL;
}
return result;
}
static inline struct crypto_route *crypto_route_dequeue(struct crypto_session *s)
{
struct crypto_route *rt;
spin_lock_bh(&s->route_list.lock);
rt = __crypto_route_dequeue(&s->route_list);
spin_unlock_bh(&s->route_list.lock);
return rt;
}
static inline void __crypto_route_queue(struct crypto_route *rt, struct crypto_route_head *list)
{
struct crypto_route *prev, *next;
rt->list = list;
list->qlen++;
next = (struct crypto_route *)list;
prev = next->prev;
rt->next = next;
rt->prev = prev;
next->prev = prev->next = rt;
}
static inline void crypto_route_queue(struct crypto_route *rt, struct crypto_session *s)
{
spin_lock_bh(&s->route_list.lock);
__crypto_route_queue(rt, &s->route_list);
spin_unlock_bh(&s->route_list.lock);
}
static inline int crypto_route_add(struct crypto_device *dev, struct crypto_session *s, struct crypto_session_initializer *ci)
{
struct crypto_route *rt;
rt = crypto_route_alloc(dev, ci);
if (!rt)
return -ENOMEM;
crypto_route_queue(rt, s);
return 0;
}
static inline int crypto_route_queue_len(struct crypto_session *s)
{
return s->route_list.qlen;
}
static inline void crypto_route_head_init(struct crypto_route_head *list)
{
spin_lock_init(&list->lock);
list->prev = list->next = (struct crypto_route *)list;
list->qlen = 0;
}
static inline struct crypto_route *__crypto_route_current(struct crypto_route_head *list)
{
struct crypto_route *next, *prev, *result;
prev = (struct crypto_route *) list;
next = prev->next;
result = NULL;
if (next != prev)
result = next;
return result;
}
static inline struct crypto_route *crypto_route_current(struct crypto_session *s)
{
struct crypto_route_head *list;
struct crypto_route *rt = NULL;
list = &s->route_list;
if (list)
{
spin_lock_bh(&list->lock);
rt = __crypto_route_current(list);
spin_unlock_bh(&list->lock);
}
return rt;
}
static inline struct crypto_device *crypto_route_get_current_device(struct crypto_session *s)
{
struct crypto_route *rt = NULL;
struct crypto_device *dev = NULL;
struct crypto_route_head *list = &s->route_list;
spin_lock_bh(&list->lock);
rt = __crypto_route_current(list);
if (rt)
{
dev = rt->dev;
crypto_device_get(dev);
}
spin_unlock_bh(&list->lock);
return dev;
}
#endif /* __CRYPTO_ROUTE_H */
[-- Attachment #1.12: crypto_stat.c --]
[-- Type: text/x-csrc, Size: 2092 bytes --]
/*
* crypto_stat.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include "acrypto.h"
#include "crypto_route.h"
inline void crypto_stat_start_inc(struct crypto_session *s)
{
struct crypto_device *dev;
dev = crypto_route_get_current_device(s);
if (dev)
{
spin_lock_bh(&dev->stat_lock);
dev->stat.sstarted++;
spin_unlock_bh(&dev->stat_lock);
crypto_device_put(dev);
}
}
inline void crypto_stat_finish_inc(struct crypto_session *s)
{
struct crypto_device *dev;
dev = crypto_route_get_current_device(s);
if (dev)
{
spin_lock_bh(&dev->stat_lock);
dev->stat.sfinished++;
spin_unlock_bh(&dev->stat_lock);
crypto_device_put(dev);
}
}
inline void crypto_stat_complete_inc(struct crypto_session *s)
{
struct crypto_device *dev;
dev = crypto_route_get_current_device(s);
if (dev)
{
spin_lock_bh(&dev->stat_lock);
dev->stat.scompleted++;
spin_unlock_bh(&dev->stat_lock);
crypto_device_put(dev);
}
}
EXPORT_SYMBOL(crypto_stat_start_inc);
EXPORT_SYMBOL(crypto_stat_finish_inc);
EXPORT_SYMBOL(crypto_stat_complete_inc);
[-- Attachment #1.13: crypto_stat.h --]
[-- Type: text/x-chdr, Size: 1127 bytes --]
/*
* crypto_stat.h
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef __CRYPTO_STAT_H
#define __CRYPTO_STAT_H
#include "acrypto.h"
inline void crypto_stat_start_inc(struct crypto_session *s);
inline void crypto_stat_finish_inc(struct crypto_session *s);
inline void crypto_stat_complete_inc(struct crypto_session *s);
#endif /* __CRYPTO_STAT_H */
[-- Attachment #1.14: Makefile --]
[-- Type: text/x-makefile, Size: 572 bytes --]
TOPDIR := $(PWD)
#KDIR := /lib/modules/$(shell uname -r)/build
KDIR := /usr/local/src/linux-2.6/linux-2.6
PWD := $(shell pwd)
#CC := /tmp/sparse-bk/cgcc
CC := gcc
obj-m += acrypto.o simple_lb.o
obj-m += consumer.o
#slow_consumer.o
obj-m += provider.o slow_provider.o sha1_provider.o
acrypto-objs := crypto_main.o crypto_lb.o crypto_dev.o crypto_conn.o crypto_stat.o
obj-m += via-padlock/
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
clean:
rm -f *.o *.ko *.mod.* .*.cmd *~ .*.d
rm -rf .tmp_versions
cd via-padlock && make clean
[-- Attachment #1.15: provider.c --]
[-- Type: text/x-csrc, Size: 2680 bytes --]
/*
* provider.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/err.h>
#include "acrypto.h"
#include "crypto_stat.h"
static int prov_data_ready_reentry;
static void prov_data_ready(struct crypto_device *dev);
static struct crypto_capability prov_caps[] = {
{0, 0, 0, 1000}, {1, 1, 1, 1000}, {2, 2, 2, 1000}};
static int prov_cap_number = sizeof(prov_caps)/sizeof(prov_caps[0]);
static struct crypto_device pdev =
{
.name = "test_provider",
.data_ready = prov_data_ready,
.cap = &prov_caps[0],
};
static void prov_data_ready(struct crypto_device *dev)
{
struct crypto_session *s, *n;
if (prov_data_ready_reentry)
return;
prov_data_ready_reentry++;
//spin_lock_bh(&dev->session_lock);
list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry)
{
if (!session_completed(s))
{
dprintk("Completing session %llu [%llu] in %s.\n",
s->ci.id, s->ci.dev_id, pdev.name);
crypto_stat_complete_inc(s);
complete_session(s);
}
}
//spin_unlock_bh(&dev->session_lock);
prov_data_ready_reentry--;
}
int prov_init(void)
{
int err;
pdev.cap_number = prov_cap_number;
err = crypto_device_add(&pdev);
if (err)
return err;
dprintk(KERN_INFO "Test crypto provider module %s is loaded.\n", pdev.name);
return 0;
}
void prov_fini(void)
{
crypto_device_remove(&pdev);
dprintk(KERN_INFO "Test crypto provider module %s is unloaded.\n", pdev.name);
}
module_init(prov_init);
module_exit(prov_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Test crypto module provider.");
[-- Attachment #1.16: sha1_provider.c --]
[-- Type: text/x-csrc, Size: 4307 bytes --]
/*
* sha1_provider.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/err.h>
#include <linux/crypto.h>
#include <linux/mm.h>
#include "acrypto.h"
#include "crypto_stat.h"
static void prov_data_ready(struct crypto_device *dev);
static struct crypto_capability prov_caps[] = {
{3, 3, 3, 100}
};
static int prov_cap_number = sizeof(prov_caps)/sizeof(prov_caps[0]);
static struct completion thread_exited;
static DECLARE_WAIT_QUEUE_HEAD(sha1_wait_queue);
static int need_exit;
static struct crypto_session *sp;
static struct crypto_tfm *tfm;
static struct crypto_device pdev =
{
.name = "sha1_provider",
.data_ready = prov_data_ready,
.cap = &prov_caps[0],
};
static void prov_data_ready(struct crypto_device *dev)
{
struct crypto_session *s, *n;
if (!spin_trylock_bh(&dev->session_lock))
return;
list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry)
{
if (!session_completed(s))
{
sp = s;
wake_up(&sha1_wait_queue);
break;
}
}
spin_unlock_bh(&dev->session_lock);
}
static int sha1_thread(void *data)
{
struct crypto_device *dev = (struct crypto_device *)data;
u8 *key, *out;
unsigned int keylen;
daemonize("%s", dev->name);
allow_signal(SIGTERM);
while (!need_exit)
{
interruptible_sleep_on(&sha1_wait_queue);
if (need_exit)
break;
if (!sp)
continue;
start_process_session(sp);
key = ((u8 *)page_address(sp->data.sg_key.page)) + sp->data.sg_key.offset;
keylen = sp->data.sg_key.length;
out = ((u8 *)page_address(sp->data.sg_dst.page)) + sp->data.sg_dst.offset;
crypto_hmac(tfm, key, &keylen, &sp->data.sg_src, sp->data.sg_src_num, out);
sp->data.sg_dst.length = 20;
dprintk("Completing session %llu [%llu] in %s.\n",
sp->ci.id, sp->ci.dev_id, pdev.name);
crypto_stat_complete_inc(sp);
complete_session(sp);
stop_process_session(sp);
prov_data_ready(dev);
sp = NULL;
}
complete_and_exit(&thread_exited, 0);
}
int prov_init(void)
{
int err, pid;
tfm = crypto_alloc_tfm("sha1", 0);
if (!tfm)
{
printk(KERN_ERR "Failed to allocate SHA1 tfm.\n");
return -EINVAL;
}
init_completion(&thread_exited);
pid = kernel_thread(sha1_thread, &pdev, CLONE_FS | CLONE_FILES);
if (IS_ERR((void *)pid))
{
err = -EINVAL;
dprintk(KERN_ERR "Failed to create kernel load balancing thread.\n");
goto err_out_free_tfm;
}
pdev.cap_number = prov_cap_number;
err = crypto_device_add(&pdev);
if (err)
goto err_out_remove_thread;
dprintk(KERN_INFO "Test crypto provider module %s is loaded.\n", pdev.name);
return 0;
err_out_remove_thread:
need_exit = 1;
wake_up(&sha1_wait_queue);
wait_for_completion(&thread_exited);
err_out_free_tfm:
crypto_free_tfm(tfm);
return err;
}
void prov_fini(void)
{
need_exit = 1;
wake_up(&sha1_wait_queue);
wait_for_completion(&thread_exited);
crypto_device_remove(&pdev);
crypto_free_tfm(tfm);
dprintk(KERN_INFO "Test crypto provider module %s is unloaded.\n", pdev.name);
}
module_init(prov_init);
module_exit(prov_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Test crypto module provider.");
[-- Attachment #1.17: simple_lb.c --]
[-- Type: text/x-csrc, Size: 2467 bytes --]
/*
* simple_lb.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include "crypto_lb.h"
static void simple_lb_rehash(struct crypto_lb *);
static struct crypto_device *simple_lb_find_device(struct crypto_lb *, struct crypto_session_initializer *);
struct crypto_lb simple_lb =
{
.name = "simple_lb",
.rehash = simple_lb_rehash,
.find_device = simple_lb_find_device
};
static void simple_lb_rehash(struct crypto_lb *lb)
{
BUG_ON(!lb->crypto_device_list);
BUG_ON(!lb->crypto_device_lock);
}
static struct crypto_device *simple_lb_find_device(struct crypto_lb *lb, struct crypto_session_initializer *ci)
{
struct crypto_device *dev, *__dev;
int min = 0x7ffffff;
__dev = NULL;
list_for_each_entry(dev, lb->crypto_device_list, cdev_entry)
{
if (device_broken(dev))
continue;
if (!match_initializer(dev, ci))
continue;
if (atomic_read(&dev->refcnt) < min)
{
min = atomic_read(&dev->refcnt);
__dev = dev;
}
}
return __dev;
}
int __devinit simple_lb_init(void)
{
dprintk(KERN_INFO "Registering simple crypto load balancer.\n");
return crypto_lb_register(&simple_lb, 1, 1);
}
void __devexit simple_lb_fini(void)
{
dprintk(KERN_INFO "Unregistering simple crypto load balancer.\n");
crypto_lb_unregister(&simple_lb);
}
module_init(simple_lb_init);
module_exit(simple_lb_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Simple crypto load balancer.");
[-- Attachment #1.18: Makefile --]
[-- Type: text/x-makefile, Size: 138 bytes --]
obj-m += padlock.o
padlock-objs := padlock-aes.o padlock-generic.o
clean:
rm -f *.o *.ko *.mod.* .*.cmd *~
rm -rf .tmp_versions
[-- Attachment #1.19: padlock-aes.c --]
[-- Type: text/x-csrc, Size: 16301 bytes --]
/*
* Cryptographic API.
*
* Support for VIA PadLock hardware crypto engine.
*
* Linux developers:
* Michal Ludvig <mludvig@suse.cz>
*
* Key expansion routine taken from crypto/aes.c
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* ---------------------------------------------------------------------------
* Copyright (c) 2002, Dr Brian Gladman <brg@gladman.me.uk>, Worcester, UK.
* All rights reserved.
*
* LICENSE TERMS
*
* The free distribution and use of this software in both source and binary
* form is allowed (with or without changes) provided that:
*
* 1. distributions of this source code include the above copyright
* notice, this list of conditions and the following disclaimer;
*
* 2. distributions in binary form include the above copyright
* notice, this list of conditions and the following disclaimer
* in the documentation and/or other associated materials;
*
* 3. the copyright holder's name is not used to endorse products
* built using this software without specific written permission.
*
* ALTERNATIVELY, provided that this notice is retained in full, this product
* may be distributed under the terms of the GNU General Public License (GPL),
* in which case the provisions of the GPL apply INSTEAD OF those given above.
*
* DISCLAIMER
*
* This software is provided 'as is' with no explicit or implied warranties
* in respect of its properties, including, but not limited to, correctness
* and/or fitness for purpose.
* ---------------------------------------------------------------------------
*/
#include <linux/module.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/crypto.h>
#include <asm/byteorder.h>
#include <linux/mm.h>
#include <asm/scatterlist.h>
#include "padlock.h"
#include "../crypto_def.h"
#include "../acrypto.h"
#include "../crypto_stat.h"
static inline int aes_hw_extkey_available (u8 key_len);
static inline
u32 generic_rotr32 (const u32 x, const unsigned bits)
{
const unsigned n = bits % 32;
return (x >> n) | (x << (32 - n));
}
static inline
u32 generic_rotl32 (const u32 x, const unsigned bits)
{
const unsigned n = bits % 32;
return (x << n) | (x >> (32 - n));
}
#define rotl generic_rotl32
#define rotr generic_rotr32
/*
* #define byte(x, nr) ((unsigned char)((x) >> (nr*8)))
*/
inline static u8
byte(const u32 x, const unsigned n)
{
return x >> (n << 3);
}
#define u32_in(x) le32_to_cpu(*(const u32 *)(x))
#define u32_out(to, from) (*(u32 *)(to) = cpu_to_le32(from))
static u8 pow_tab[256];
static u8 log_tab[256];
static u8 sbx_tab[256];
static u8 isb_tab[256];
static u32 rco_tab[10];
static u32 ft_tab[4][256];
static u32 it_tab[4][256];
static u32 fl_tab[4][256];
static u32 il_tab[4][256];
static inline u8
f_mult (u8 a, u8 b)
{
u8 aa = log_tab[a], cc = aa + log_tab[b];
return pow_tab[cc + (cc < aa ? 1 : 0)];
}
#define ff_mult(a,b) (a && b ? f_mult(a, b) : 0)
#define f_rn(bo, bi, n, k) \
bo[n] = ft_tab[0][byte(bi[n],0)] ^ \
ft_tab[1][byte(bi[(n + 1) & 3],1)] ^ \
ft_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
ft_tab[3][byte(bi[(n + 3) & 3],3)] ^ *(k + n)
#define i_rn(bo, bi, n, k) \
bo[n] = it_tab[0][byte(bi[n],0)] ^ \
it_tab[1][byte(bi[(n + 3) & 3],1)] ^ \
it_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
it_tab[3][byte(bi[(n + 1) & 3],3)] ^ *(k + n)
#define ls_box(x) \
( fl_tab[0][byte(x, 0)] ^ \
fl_tab[1][byte(x, 1)] ^ \
fl_tab[2][byte(x, 2)] ^ \
fl_tab[3][byte(x, 3)] )
#define f_rl(bo, bi, n, k) \
bo[n] = fl_tab[0][byte(bi[n],0)] ^ \
fl_tab[1][byte(bi[(n + 1) & 3],1)] ^ \
fl_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
fl_tab[3][byte(bi[(n + 3) & 3],3)] ^ *(k + n)
#define i_rl(bo, bi, n, k) \
bo[n] = il_tab[0][byte(bi[n],0)] ^ \
il_tab[1][byte(bi[(n + 3) & 3],1)] ^ \
il_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
il_tab[3][byte(bi[(n + 1) & 3],3)] ^ *(k + n)
static void
gen_tabs (void)
{
u32 i, t;
u8 p, q;
/* log and power tables for GF(2**8) finite field with
0x011b as modular polynomial - the simplest prmitive
root is 0x03, used here to generate the tables */
for (i = 0, p = 1; i < 256; ++i) {
pow_tab[i] = (u8) p;
log_tab[p] = (u8) i;
p ^= (p << 1) ^ (p & 0x80 ? 0x01b : 0);
}
log_tab[1] = 0;
for (i = 0, p = 1; i < 10; ++i) {
rco_tab[i] = p;
p = (p << 1) ^ (p & 0x80 ? 0x01b : 0);
}
for (i = 0; i < 256; ++i) {
p = (i ? pow_tab[255 - log_tab[i]] : 0);
q = ((p >> 7) | (p << 1)) ^ ((p >> 6) | (p << 2));
p ^= 0x63 ^ q ^ ((q >> 6) | (q << 2));
sbx_tab[i] = p;
isb_tab[p] = (u8) i;
}
for (i = 0; i < 256; ++i) {
p = sbx_tab[i];
t = p;
fl_tab[0][i] = t;
fl_tab[1][i] = rotl (t, 8);
fl_tab[2][i] = rotl (t, 16);
fl_tab[3][i] = rotl (t, 24);
t = ((u32) ff_mult (2, p)) |
((u32) p << 8) |
((u32) p << 16) | ((u32) ff_mult (3, p) << 24);
ft_tab[0][i] = t;
ft_tab[1][i] = rotl (t, 8);
ft_tab[2][i] = rotl (t, 16);
ft_tab[3][i] = rotl (t, 24);
p = isb_tab[i];
t = p;
il_tab[0][i] = t;
il_tab[1][i] = rotl (t, 8);
il_tab[2][i] = rotl (t, 16);
il_tab[3][i] = rotl (t, 24);
t = ((u32) ff_mult (14, p)) |
((u32) ff_mult (9, p) << 8) |
((u32) ff_mult (13, p) << 16) |
((u32) ff_mult (11, p) << 24);
it_tab[0][i] = t;
it_tab[1][i] = rotl (t, 8);
it_tab[2][i] = rotl (t, 16);
it_tab[3][i] = rotl (t, 24);
}
}
#define star_x(x) (((x) & 0x7f7f7f7f) << 1) ^ ((((x) & 0x80808080) >> 7) * 0x1b)
#define imix_col(y,x) \
u = star_x(x); \
v = star_x(u); \
w = star_x(v); \
t = w ^ (x); \
(y) = u ^ v ^ w; \
(y) ^= rotr(u ^ t, 8) ^ \
rotr(v ^ t, 16) ^ \
rotr(t,24)
/* initialise the key schedule from the user supplied key */
#define loop4(i) \
{ t = rotr(t, 8); t = ls_box(t) ^ rco_tab[i]; \
t ^= E_KEY[4 * i]; E_KEY[4 * i + 4] = t; \
t ^= E_KEY[4 * i + 1]; E_KEY[4 * i + 5] = t; \
t ^= E_KEY[4 * i + 2]; E_KEY[4 * i + 6] = t; \
t ^= E_KEY[4 * i + 3]; E_KEY[4 * i + 7] = t; \
}
#define loop6(i) \
{ t = rotr(t, 8); t = ls_box(t) ^ rco_tab[i]; \
t ^= E_KEY[6 * i]; E_KEY[6 * i + 6] = t; \
t ^= E_KEY[6 * i + 1]; E_KEY[6 * i + 7] = t; \
t ^= E_KEY[6 * i + 2]; E_KEY[6 * i + 8] = t; \
t ^= E_KEY[6 * i + 3]; E_KEY[6 * i + 9] = t; \
t ^= E_KEY[6 * i + 4]; E_KEY[6 * i + 10] = t; \
t ^= E_KEY[6 * i + 5]; E_KEY[6 * i + 11] = t; \
}
#define loop8(i) \
{ t = rotr(t, 8); ; t = ls_box(t) ^ rco_tab[i]; \
t ^= E_KEY[8 * i]; E_KEY[8 * i + 8] = t; \
t ^= E_KEY[8 * i + 1]; E_KEY[8 * i + 9] = t; \
t ^= E_KEY[8 * i + 2]; E_KEY[8 * i + 10] = t; \
t ^= E_KEY[8 * i + 3]; E_KEY[8 * i + 11] = t; \
t = E_KEY[8 * i + 4] ^ ls_box(t); \
E_KEY[8 * i + 12] = t; \
t ^= E_KEY[8 * i + 5]; E_KEY[8 * i + 13] = t; \
t ^= E_KEY[8 * i + 6]; E_KEY[8 * i + 14] = t; \
t ^= E_KEY[8 * i + 7]; E_KEY[8 * i + 15] = t; \
}
static int
aes_set_key(void *ctx_arg, const u8 *in_key, unsigned int key_len)
{
struct aes_ctx *ctx = ctx_arg;
u32 i, t, u, v, w;
u32 P[AES_EXTENDED_KEY_SIZE];
u32 rounds;
if (key_len != 16 && key_len != 24 && key_len != 32) {
return -EINVAL;
}
ctx->key_length = key_len;
ctx->E = ctx->e_data;
ctx->D = ctx->d_data;
/* Ensure 16-Bytes alignmentation of keys for VIA PadLock. */
if ((int)(ctx->e_data) & 0x0F)
ctx->E += 4 - (((int)(ctx->e_data) & 0x0F) / sizeof (ctx->e_data[0]));
if ((int)(ctx->d_data) & 0x0F)
ctx->D += 4 - (((int)(ctx->d_data) & 0x0F) / sizeof (ctx->d_data[0]));
E_KEY[0] = u32_in (in_key);
E_KEY[1] = u32_in (in_key + 4);
E_KEY[2] = u32_in (in_key + 8);
E_KEY[3] = u32_in (in_key + 12);
/* Don't generate extended keys if the hardware can do it. */
if (aes_hw_extkey_available(key_len))
return 0;
switch (key_len) {
case 16:
t = E_KEY[3];
for (i = 0; i < 10; ++i)
loop4 (i);
break;
case 24:
E_KEY[4] = u32_in (in_key + 16);
t = E_KEY[5] = u32_in (in_key + 20);
for (i = 0; i < 8; ++i)
loop6 (i);
break;
case 32:
E_KEY[4] = u32_in (in_key + 16);
E_KEY[5] = u32_in (in_key + 20);
E_KEY[6] = u32_in (in_key + 24);
t = E_KEY[7] = u32_in (in_key + 28);
for (i = 0; i < 7; ++i)
loop8 (i);
break;
}
D_KEY[0] = E_KEY[0];
D_KEY[1] = E_KEY[1];
D_KEY[2] = E_KEY[2];
D_KEY[3] = E_KEY[3];
for (i = 4; i < key_len + 24; ++i) {
imix_col (D_KEY[i], E_KEY[i]);
}
/* PadLock needs a different format of the decryption key. */
rounds = 10 + (key_len - 16) / 4;
for (i = 0; i < rounds; i++) {
P[((i + 1) * 4) + 0] = D_KEY[((rounds - i - 1) * 4) + 0];
P[((i + 1) * 4) + 1] = D_KEY[((rounds - i - 1) * 4) + 1];
P[((i + 1) * 4) + 2] = D_KEY[((rounds - i - 1) * 4) + 2];
P[((i + 1) * 4) + 3] = D_KEY[((rounds - i - 1) * 4) + 3];
}
P[0] = E_KEY[(rounds * 4) + 0];
P[1] = E_KEY[(rounds * 4) + 1];
P[2] = E_KEY[(rounds * 4) + 2];
P[3] = E_KEY[(rounds * 4) + 3];
memcpy(D_KEY, P, AES_EXTENDED_KEY_SIZE_B);
return 0;
}
/* Tells whether the ACE is capable to generate
the extended key for a given key_len. */
static inline int aes_hw_extkey_available(u8 key_len)
{
/* TODO: We should check the actual CPU model/stepping
as it's likely that the capability will be
added in the next CPU revisions. */
if (key_len == 16)
return 1;
return 0;
}
static void aes_padlock(void *ctx_arg, u8 *out_arg, const u8 *in_arg,
const u8 *iv_arg, size_t nbytes, int encdec,
int mode)
{
struct aes_ctx *ctx = ctx_arg;
char bigbuf[sizeof(union cword) + 16];
union cword *cword;
void *key;
if (((long)bigbuf) & 0x0F)
cword = (void*)(bigbuf + 16 - ((long)bigbuf & 0x0F));
else
cword = (void*)bigbuf;
/* Prepare Control word. */
memset (cword, 0, sizeof(union cword));
cword->b.encdec = !encdec; /* in the rest of cryptoapi ENC=1/DEC=0 */
cword->b.rounds = 10 + (ctx->key_length - 16) / 4;
cword->b.ksize = (ctx->key_length - 16) / 8;
/* Is the hardware capable to generate the extended key? */
if (!aes_hw_extkey_available(ctx->key_length))
cword->b.keygen = 1;
/* ctx->E starts with a plain key - if the hardware is capable
to generate the extended key itself we must supply
the plain key for both Encryption and Decryption. */
if (encdec == CRYPTO_OP_ENCRYPT || cword->b.keygen == 0)
key = ctx->E;
else
key = ctx->D;
padlock_aligner(out_arg, in_arg, iv_arg, key, cword,
nbytes, AES_BLOCK_SIZE, encdec, mode);
}
static void aes_padlock_ecb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
size_t nbytes, int encdec)
{
aes_padlock(ctx, dst, src, NULL, nbytes, encdec, CRYPTO_MODE_ECB);
}
static void aes_padlock_cbc(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
size_t nbytes, int encdec)
{
aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_CBC);
}
static void aes_padlock_cfb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
size_t nbytes, int encdec)
{
aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_CFB);
}
static void aes_padlock_ofb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
size_t nbytes, int encdec)
{
aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_OFB);
}
static struct crypto_capability padlock_caps[] =
{
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_OFB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_OFB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_OFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_OFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_OFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CBC, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CFB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_OFB, 1000},
};
static int padlock_cap_number = sizeof(padlock_caps)/sizeof(padlock_caps[0]);
static void padlock_data_ready(struct crypto_device *dev);
static int padlock_data_ready_reentry;
static struct crypto_device padlock_device =
{
.name = "via-padlock",
.data_ready = padlock_data_ready,
.cap = &padlock_caps[0],
};
static void process_session(struct crypto_session *s)
{
int err;
u8 *key, *dst, *src, *iv;
size_t size, keylen;
key = ((u8 *)page_address(s->data.sg_key.page)) + s->data.sg_key.offset;
keylen = s->data.sg_key.length;
dst = ((u8 *)page_address(s->data.sg_dst.page)) + s->data.sg_dst.offset;
src = ((u8 *)page_address(s->data.sg_src.page)) + s->data.sg_src.offset;
size = s->data.sg_src.length;
iv = ((u8 *)page_address(s->data.sg_iv.page)) + s->data.sg_iv.offset;
err = aes_set_key(s->data.priv, key, keylen);
if (err)
return;
switch (s->ci.mode)
{
case CRYPTO_MODE_ECB:
aes_padlock_ecb(s->data.priv, dst, src, iv, size, s->ci.operation);
break;
case CRYPTO_MODE_CBC:
aes_padlock_cbc(s->data.priv, dst, src, iv, size, s->ci.operation);
break;
case CRYPTO_MODE_CFB:
aes_padlock_cfb(s->data.priv, dst, src, iv, size, s->ci.operation);
break;
case CRYPTO_MODE_OFB:
aes_padlock_ofb(s->data.priv, dst, src, iv, size, s->ci.operation);
break;
}
s->data.sg_dst.length = size;
return;
}
static void padlock_data_ready(struct crypto_device *dev)
{
struct crypto_session *s, *n;
if (padlock_data_ready_reentry)
return;
padlock_data_ready_reentry++;
list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry)
{
if (!session_completed(s))
{
start_process_session(s);
process_session(s);
crypto_stat_complete_inc(s);
complete_session(s);
stop_process_session(s);
}
}
padlock_data_ready_reentry--;
}
int padlock_init_aes(void)
{
u32 cpuid, edx;
u32 val = 0xC0000000;
cpuid = cpuid_eax(val);
edx = cpuid_edx(val);
printk("val=%x, cpuid=%x, edx=%x.\n", val, cpuid, edx);
if (cpuid >= val + 1)
{
printk("Board supports ACE.\n");
}
else
{
printk("Board does not support ACE.\n");
return -ENODEV;
}
printk(KERN_NOTICE "Using VIA PadLock ACE for AES algorithm (multiblock).\n");
padlock_device.cap_number = padlock_cap_number;
gen_tabs();
return crypto_device_add(&padlock_device);
}
void padlock_fini_aes(void)
{
crypto_device_remove(&padlock_device);
}
[-- Attachment #1.20: padlock-generic.c --]
[-- Type: text/x-csrc, Size: 5401 bytes --]
/*
* Cryptographic API.
*
* Support for VIA PadLock hardware crypto engine.
*
* Linux developers:
* Michal Ludvig <mludvig@suse.cz>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*/
#include <linux/module.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/crypto.h>
#include <asm/byteorder.h>
#include "padlock.h"
#include "../acrypto.h"
#include "../crypto_def.h"
#define PFX "padlock: "
typedef void (xcrypt_t)(u8 *input, u8 *output, u8 *key, u8 *iv,
void *control_word, u32 count);
static inline void padlock_xcrypt_ecb(u8 *input, u8 *output, u8 *key,
u8 *iv, void *control_word, u32 count)
{
asm volatile ("pushfl; popfl"); /* enforce key reload. */
asm volatile (".byte 0xf3,0x0f,0xa7,0xc8" /* rep xcryptecb */
: "=m"(*output), "+S"(input), "+D"(output)
: "d"(control_word), "b"(key), "c"(count));
}
static inline void padlock_xcrypt_cbc(u8 *input, u8 *output, u8 *key,
u8 *iv, void *control_word, u32 count)
{
asm volatile ("pushfl; popfl"); /* enforce key reload. */
asm volatile (".byte 0xf3,0x0f,0xa7,0xd0" /* rep xcryptcbc */
: "=m"(*output), "+S"(input), "+D"(output)
: "d"(control_word), "b"(key), "c"(count), "a"(iv));
}
static inline void padlock_xcrypt_cfb(u8 *input, u8 *output, u8 *key,
u8 *iv, void *control_word, u32 count)
{
asm volatile ("pushfl; popfl"); /* enforce key reload. */
asm volatile (".byte 0xf3,0x0f,0xa7,0xe0" /* rep xcryptcfb */
: "=m"(*output), "+S"(input), "+D"(output)
: "d"(control_word), "b"(key), "c"(count), "a"(iv));
}
static inline void padlock_xcrypt_ofb(u8 *input, u8 *output, u8 *key,
u8 *iv, void *control_word, u32 count)
{
asm volatile ("pushfl; popfl"); /* enforce key reload. */
asm volatile (".byte 0xf3,0x0f,0xa7,0xe8" /* rep xcryptofb */
: "=m"(*output), "+S"(input), "+D"(output)
: "d"(control_word), "b"(key), "c"(count), "a"(iv));
}
void *crypto_aligned_kmalloc(size_t size, int mode, size_t alignment, void **index)
{
char *ptr;
ptr = kmalloc(size + alignment, mode);
*index = ptr;
if (alignment > 1 && ((long)ptr & (alignment - 1))) {
ptr += alignment - ((long)ptr & (alignment - 1));
}
return ptr;
}
void padlock_aligner(u8 *out_arg, const u8 *in_arg, const u8 *iv_arg,
void *key, union cword *cword,
size_t nbytes, size_t blocksize,
int encdec, int mode)
{
/* Don't blindly modify this structure - the items must
fit on 16-Bytes boundaries! */
struct padlock_xcrypt_data {
u8 iv[blocksize]; /* Initialization vector */
};
u8 *in, *out, *iv;
void *index = NULL;
char bigbuf[sizeof(struct padlock_xcrypt_data) + 16];
struct padlock_xcrypt_data *data;
/* Place 'data' at the first 16-Bytes aligned address in 'bigbuf'. */
if (((long)bigbuf) & 0x0F)
data = (void*)(bigbuf + 16 - ((long)bigbuf & 0x0F));
else
data = (void*)bigbuf;
if (((long)in_arg) & 0x0F) {
in = crypto_aligned_kmalloc(nbytes, GFP_KERNEL, 16, &index);
memcpy(in, in_arg, nbytes);
}
else
in = (u8*)in_arg;
if (((long)out_arg) & 0x0F) {
if (index)
out = in; /* xcrypt can work "in place" */
else
out = crypto_aligned_kmalloc(nbytes, GFP_KERNEL, 16, &index);
}
else
out = out_arg;
/* Always make a local copy of IV - xcrypt may change it! */
iv = data->iv;
if (iv_arg)
memcpy(iv, iv_arg, blocksize);
dprintk("data=%p\n", data);
dprintk("in=%p\n", in);
dprintk("out=%p\n", out);
dprintk("iv=%p\n", iv);
dprintk("nbytes=%d, blocksize=%d.\n", nbytes, blocksize);
switch (mode) {
case CRYPTO_MODE_ECB:
padlock_xcrypt_ecb(in, out, key, iv, cword, nbytes/blocksize);
break;
case CRYPTO_MODE_CBC:
padlock_xcrypt_cbc(in, out, key, iv, cword, nbytes/blocksize);
break;
case CRYPTO_MODE_CFB:
padlock_xcrypt_cfb(in, out, key, iv, cword, nbytes/blocksize);
break;
case CRYPTO_MODE_OFB:
padlock_xcrypt_ofb(in, out, key, iv, cword, nbytes/blocksize);
break;
default:
BUG();
}
/* Copy the 16-Byte aligned output to the caller's buffer. */
if (out != out_arg)
memcpy(out_arg, out, nbytes);
if (index)
kfree(index);
}
static int __init padlock_init(void)
{
int ret = -ENOSYS;
#if 0
if (!cpu_has_xcrypt) {
printk(KERN_ERR PFX "VIA PadLock not detected.\n");
return -ENODEV;
}
if (!cpu_has_xcrypt_enabled) {
printk(KERN_ERR PFX "VIA PadLock detected, but not enabled. Hmm, strange...\n");
return -ENODEV;
}
#endif
if ((ret = padlock_init_aes())) {
printk(KERN_ERR PFX "VIA PadLock AES initialization failed.\n");
return ret;
}
if (ret == -ENOSYS)
printk(KERN_ERR PFX "Hmm, VIA PadLock was compiled without any algorithm.\n");
return ret;
}
static void __exit padlock_fini(void)
{
padlock_fini_aes();
}
module_init(padlock_init);
module_exit(padlock_fini);
MODULE_DESCRIPTION("VIA PadLock crypto engine support.");
MODULE_LICENSE("Dual BSD/GPL");
MODULE_AUTHOR("Michal Ludvig");
[-- Attachment #1.21: padlock.h --]
[-- Type: text/x-chdr, Size: 1504 bytes --]
/*
* Cryptographic API.
*
* Copyright (c) 2004 Michal Ludvig <mludvig@suse.cz>
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the Free
* Software Foundation; either version 2 of the License, or (at your option)
* any later version.
*
*/
#ifndef _CRYPTO_PADLOCK_H
#define _CRYPTO_PADLOCK_H
#define AES_MIN_KEY_SIZE 16 /* in u8 units */
#define AES_MAX_KEY_SIZE 32 /* ditto */
#define AES_BLOCK_SIZE 16 /* ditto */
#define AES_EXTENDED_KEY_SIZE 64 /* in u32 units */
#define AES_EXTENDED_KEY_SIZE_B (AES_EXTENDED_KEY_SIZE * sizeof(u32))
struct aes_ctx {
u32 e_data[AES_EXTENDED_KEY_SIZE+4];
u32 d_data[AES_EXTENDED_KEY_SIZE+4];
int key_length;
u32 *E;
u32 *D;
};
#define E_KEY ctx->E
#define D_KEY ctx->D
/* Control word. */
#if 1
union cword {
u32 cword[4];
struct {
int rounds:4;
int algo:3;
int keygen:1;
int interm:1;
int encdec:1;
int ksize:2;
} b;
};
#else
union cword {
u32 cword[4];
struct {
unsigned rounds:4,
algo:3,
keygen:1,
interm:1,
encdec:1,
ksize:2;
} b;
};
#endif
#define PFX "padlock: "
void padlock_aligner(u8 *out_arg, const u8 *in_arg, const u8 *iv_arg,
void *key, union cword *cword,
size_t nbytes, size_t blocksize,
int encdec, int mode);
int padlock_init_aes(void);
void padlock_fini_aes(void);
#endif /* _CRYPTO_PADLOCK_H */
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 6:22 Asynchronous crypto layer Evgeniy Polyakov
@ 2004-10-29 12:42 ` jamal
2004-10-29 14:06 ` Evgeniy Polyakov
2004-10-29 17:44 ` James Morris
` (5 subsequent siblings)
6 siblings, 1 reply; 57+ messages in thread
From: jamal @ 2004-10-29 12:42 UTC (permalink / raw)
To: johnpol; +Cc: netdev, cryptoapi
On Fri, 2004-10-29 at 02:22, Evgeniy Polyakov wrote:
> I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
> It support following features:
> - multiple asynchronous crypto device queues
> - crypto session routing
> - crypto session binding
> - modular load balancing
> - crypto session batching genetically implemented by design
> - crypto session priority
> - different kinds of crypto operation(RNG, asymmetrical crypto, HMAC and
> any other)
>
Very nice.
I am very curious if you see perfomance improvements over old scheme
in the case of a single crypto chip in a fast CPU.
It has been shown in the past that with a xeon in the range of 2Ghz the
context setup and the big lock (and lack of async) implied that
performance enhancement using a crypto chip was negligible.
IIRC, the only time it started showing anything useful was when a
compute intensive alg like 3DES was chewing packets >= 1000 bytes.
My suspicion is you will show it is better to use a crypto chip with
async;-> In the minimal you should show some improvement.
cheers,
jamal
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 14:06 ` Evgeniy Polyakov
@ 2004-10-29 14:03 ` Michal Ludvig
2004-10-29 14:36 ` Evgeniy Polyakov
2004-10-29 15:08 ` jamal
2004-10-30 20:35 ` Eugene Surovegin
2 siblings, 1 reply; 57+ messages in thread
From: Michal Ludvig @ 2004-10-29 14:03 UTC (permalink / raw)
To: johnpol; +Cc: hadi, netdev, cryptoapi
Evgeniy Polyakov told me that:
> On 29 Oct 2004 08:42:18 -0400
> jamal <hadi@cyberus.ca> wrote:
>
>>>I am very curious if you see perfomance improvements over old scheme
>>in the case of a single crypto chip in a fast CPU.
>>It has been shown in the past that with a xeon in the range of 2Ghz the
>>context setup and the big lock (and lack of async) implied that
>>performance enhancement using a crypto chip was negligible.
>>IIRC, the only time it started showing anything useful was when a
>>compute intensive alg like 3DES was chewing packets >= 1000 bytes.
>>
>>My suspicion is you will show it is better to use a crypto chip with
>>async;-> In the minimal you should show some improvement.
>
>
> If we have a hardware accelerator chip, than we _already_ have improvements
> with even the worst async crypto layer, since software and hardware
> will work in parrallel.
> I agree that multigigahertz box will beat my HIFN card,
> but I doubt it can beat 1gghz VIA.
I have a very preliminary driver for FastCrypt PCI board for 3DES at
http://www.logix.cz/michal/devel/fcrypt/
For now it works with some very ugly hacks in the current cryptoapi, but
I can give it a try with your acrypto and report the results.
I admit I haven't read your sources too deeply yet so excuse me a dumb
question - does acrypto replace or extend cryptoapi? Once I get it
running will it take over e.g. encryption for IPsec?
Michal Ludvig
--
* A mouse is a device used to point at the xterm you want to type in.
* Personal homepage - http://www.logix.cz/michal
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 12:42 ` jamal
@ 2004-10-29 14:06 ` Evgeniy Polyakov
2004-10-29 14:03 ` Michal Ludvig
` (2 more replies)
0 siblings, 3 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-29 14:06 UTC (permalink / raw)
To: hadi; +Cc: netdev, cryptoapi
On 29 Oct 2004 08:42:18 -0400
jamal <hadi@cyberus.ca> wrote:
> On Fri, 2004-10-29 at 02:22, Evgeniy Polyakov wrote:
> > I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
> > It support following features:
> > - multiple asynchronous crypto device queues
> > - crypto session routing
> > - crypto session binding
> > - modular load balancing
> > - crypto session batching genetically implemented by design
> > - crypto session priority
> > - different kinds of crypto operation(RNG, asymmetrical crypto, HMAC and
> > any other)
> >
>
> Very nice.
> I am very curious if you see perfomance improvements over old scheme
> in the case of a single crypto chip in a fast CPU.
> It has been shown in the past that with a xeon in the range of 2Ghz the
> context setup and the big lock (and lack of async) implied that
> performance enhancement using a crypto chip was negligible.
> IIRC, the only time it started showing anything useful was when a
> compute intensive alg like 3DES was chewing packets >= 1000 bytes.
>
> My suspicion is you will show it is better to use a crypto chip with
> async;-> In the minimal you should show some improvement.
If we have a hardware accelerator chip, than we _already_ have improvements
with even the worst async crypto layer, since software and hardware
will work in parrallel.
I agree that multigigahertz box will beat my HIFN card,
but I doubt it can beat 1gghz VIA.
And what about random numbers, async crypto, TPM?
They all requires some kind of generalization.
>
> cheers,
> jamal
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 14:03 ` Michal Ludvig
@ 2004-10-29 14:36 ` Evgeniy Polyakov
2004-10-29 14:53 ` Michal Ludvig
0 siblings, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-29 14:36 UTC (permalink / raw)
To: Michal Ludvig; +Cc: netdev, cryptoapi, hadi
On Fri, 29 Oct 2004 16:03:06 +0200
Michal Ludvig <michal@logix.cz> wrote:
> Evgeniy Polyakov told me that:
> > On 29 Oct 2004 08:42:18 -0400
> > jamal <hadi@cyberus.ca> wrote:
> >
> >>>I am very curious if you see perfomance improvements over old scheme
> >>in the case of a single crypto chip in a fast CPU.
> >>It has been shown in the past that with a xeon in the range of 2Ghz the
> >>context setup and the big lock (and lack of async) implied that
> >>performance enhancement using a crypto chip was negligible.
> >>IIRC, the only time it started showing anything useful was when a
> >>compute intensive alg like 3DES was chewing packets >= 1000 bytes.
> >>
> >>My suspicion is you will show it is better to use a crypto chip with
> >>async;-> In the minimal you should show some improvement.
> >
> >
> > If we have a hardware accelerator chip, than we _already_ have improvements
> > with even the worst async crypto layer, since software and hardware
> > will work in parrallel.
> > I agree that multigigahertz box will beat my HIFN card,
> > but I doubt it can beat 1gghz VIA.
>
> I have a very preliminary driver for FastCrypt PCI board for 3DES at
> http://www.logix.cz/michal/devel/fcrypt/
> For now it works with some very ugly hacks in the current cryptoapi, but
> I can give it a try with your acrypto and report the results.
It would be very appreciated>
> I admit I haven't read your sources too deeply yet so excuse me a dumb
> question - does acrypto replace or extend cryptoapi? Once I get it
> running will it take over e.g. encryption for IPsec?
They are both an addendum to each other,
any crypto layer can be "tunrned into compatibility mode" -
i.e. anyone can write bridges like attached sha1_provider.c (it is bridge
from async into sync mode). So I can not answer - but in _current_
implementation without any kind of bridges it is an extension.
If you compile this sources then you will still have old sync behaviour,
IPsec and any other old-style application should be rewritten(like
attached consumer.c) to use new asynchronous crypto layer features.
> Michal Ludvig
> --
> * A mouse is a device used to point at the xterm you want to type in.
> * Personal homepage - http://www.logix.cz/michal
> _______________________________________________
>
> Subscription: http://lists.logix.cz/mailman/listinfo/cryptoapi
> List archive: http://lists.logix.cz/pipermail/cryptoapi
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 14:36 ` Evgeniy Polyakov
@ 2004-10-29 14:53 ` Michal Ludvig
2004-10-29 15:11 ` jamal
2004-10-29 15:27 ` Evgeniy Polyakov
0 siblings, 2 replies; 57+ messages in thread
From: Michal Ludvig @ 2004-10-29 14:53 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: netdev, cryptoapi
On Fri, 29 Oct 2004, Evgeniy Polyakov wrote:
> > I have a very preliminary driver for FastCrypt PCI board for 3DES at
> > http://www.logix.cz/michal/devel/fcrypt/
> > For now it works with some very ugly hacks in the current cryptoapi, but
> > I can give it a try with your acrypto and report the results.
>
> It would be very appreciated>
Re Hifn - you may be interested in this Linux driver:
http://kernel.ebshome.net/hifn7955-2.6-4.diff
> > I admit I haven't read your sources too deeply yet so excuse me a dumb
> > question - does acrypto replace or extend cryptoapi? Once I get it
> > running will it take over e.g. encryption for IPsec?
>
> They are both an addendum to each other,
> any crypto layer can be "tunrned into compatibility mode" -
> i.e. anyone can write bridges like attached sha1_provider.c (it is bridge
> from async into sync mode). So I can not answer - but in _current_
> implementation without any kind of bridges it is an extension.
>
> If you compile this sources then you will still have old sync behaviour,
> IPsec and any other old-style application should be rewritten(like
> attached consumer.c) to use new asynchronous crypto layer features.
This could be a bit of problem, because AFAIK the IPsec code doesn't allow
sleeping while processing the packet. It will likely need a bigger
rewrite.
Anyway, have you got any tool to measure the cipher throughput without
IPsec/dm-crypt being updated for acrypto?
Michal Ludvig
--
* A mouse is a device used to point at the xterm you want to type in.
* Personal homepage - http://www.logix.cz/michal
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 14:06 ` Evgeniy Polyakov
2004-10-29 14:03 ` Michal Ludvig
@ 2004-10-29 15:08 ` jamal
2004-10-29 15:37 ` Evgeniy Polyakov
2004-10-29 16:16 ` James Morris
2004-10-30 20:35 ` Eugene Surovegin
2 siblings, 2 replies; 57+ messages in thread
From: jamal @ 2004-10-29 15:08 UTC (permalink / raw)
To: johnpol; +Cc: netdev, cryptoapi
On Fri, 2004-10-29 at 10:06, Evgeniy Polyakov wrote:
> If we have a hardware accelerator chip, than we _already_ have improvements
> with even the worst async crypto layer, since software and hardware
> will work in parrallel.
Thats what i am hoping - and theory points to it. Numbers, numbers
please ;->
> I agree that multigigahertz box will beat my HIFN card,
> but I doubt it can beat 1gghz VIA.
The VIA is an interesting one; but it would still be interesting to see
comparisons.
Actually I should say a good comparison will be with a dual opteron
>= 2Ghz which i have no doubt will smoke a xeon - all in s/ware.
> And what about random numbers, async crypto, TPM?
> They all requires some kind of generalization.
>
Similar deferal approach should do it.
If you are brave, you should probably experiment as well with
non-crypto stuff to defer things like gettimeofday computes.
(i.e ask the hardware for time, defer response and then call netif_rx
with a tag saying time has been computed).
cheers,
jamal
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 14:53 ` Michal Ludvig
@ 2004-10-29 15:11 ` jamal
2004-10-29 15:27 ` Evgeniy Polyakov
1 sibling, 0 replies; 57+ messages in thread
From: jamal @ 2004-10-29 15:11 UTC (permalink / raw)
To: Michal Ludvig; +Cc: Evgeniy Polyakov, netdev, cryptoapi
On Fri, 2004-10-29 at 10:53, Michal Ludvig wrote:
> On Fri, 29 Oct 2004, Evgeniy Polyakov wrote:
.
> >
> > If you compile this sources then you will still have old sync behaviour,
> > IPsec and any other old-style application should be rewritten(like
> > attached consumer.c) to use new asynchronous crypto layer features.
>
> This could be a bit of problem, because AFAIK the IPsec code doesn't allow
> sleeping while processing the packet. It will likely need a bigger
> rewrite.
>
I dont think its a big rewrite.
cheers,
jamal
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 14:53 ` Michal Ludvig
2004-10-29 15:11 ` jamal
@ 2004-10-29 15:27 ` Evgeniy Polyakov
2004-10-30 20:39 ` Eugene Surovegin
1 sibling, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-29 15:27 UTC (permalink / raw)
To: Michal Ludvig; +Cc: netdev, cryptoapi
On Fri, 29 Oct 2004 16:53:30 +0200 (CEST)
Michal Ludvig <michal@logix.cz> wrote:
> On Fri, 29 Oct 2004, Evgeniy Polyakov wrote:
>
> > > I have a very preliminary driver for FastCrypt PCI board for 3DES at
> > > http://www.logix.cz/michal/devel/fcrypt/
> > > For now it works with some very ugly hacks in the current cryptoapi, but
> > > I can give it a try with your acrypto and report the results.
> >
> > It would be very appreciated>
>
> Re Hifn - you may be interested in this Linux driver:
> http://kernel.ebshome.net/hifn7955-2.6-4.diff
Yes, I've seen this patch.
but it was designed to be usefull only in custom MX box, so it has some
design notes that I do not agree with.
> > > I admit I haven't read your sources too deeply yet so excuse me a dumb
> > > question - does acrypto replace or extend cryptoapi? Once I get it
> > > running will it take over e.g. encryption for IPsec?
> >
> > They are both an addendum to each other,
> > any crypto layer can be "tunrned into compatibility mode" -
> > i.e. anyone can write bridges like attached sha1_provider.c (it is bridge
> > from async into sync mode). So I can not answer - but in _current_
> > implementation without any kind of bridges it is an extension.
> >
> > If you compile this sources then you will still have old sync behaviour,
> > IPsec and any other old-style application should be rewritten(like
> > attached consumer.c) to use new asynchronous crypto layer features.
>
> This could be a bit of problem, because AFAIK the IPsec code doesn't allow
> sleeping while processing the packet. It will likely need a bigger
> rewrite.
It is design issue. In this schema IPsec can not win with acrypto.
but wireless stack, which currently under heavy development, can find acrypto
very effective.
> Anyway, have you got any tool to measure the cipher throughput without
> IPsec/dm-crypt being updated for acrypto?
Currently I'm porting from 2.4 my simple block device which was used as
communicator between host cpu and embedded pci based board.
Probably it can be used as simple measurement tool.
Or even I will write from scratch this weekend simple block device
like cryptoloop.
> Michal Ludvig
> --
> * A mouse is a device used to point at the xterm you want to type in.
> * Personal homepage - http://www.logix.cz/michal
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 15:37 ` Evgeniy Polyakov
@ 2004-10-29 15:28 ` Michal Ludvig
0 siblings, 0 replies; 57+ messages in thread
From: Michal Ludvig @ 2004-10-29 15:28 UTC (permalink / raw)
To: johnpol; +Cc: hadi, netdev, cryptoapi
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Evgeniy Polyakov told me that:
> On 29 Oct 2004 11:08:03 -0400 jamal <hadi@cyberus.ca> wrote:
>
>> On Fri, 2004-10-29 at 10:06, Evgeniy Polyakov wrote:
>>
>>> I agree that multigigahertz box will beat my HIFN card, but I
>>> doubt it can beat 1gghz VIA.
>>
>> The VIA is an interesting one; but it would still be interesting to
>> see comparisons. Actually I should say a good comparison will be
>> with a dual opteron >= 2Ghz which i have no doubt will smoke a xeon
>> - all in s/ware.
>
> He-he, dual Opteron? It costs as 5 Via boards :)
>
> ok, I will follow this plan: I will create simple block device like
> cryptoloop, it will be _really_ simple without any kind of
> portability and/or stability (I hope it will be one day, at least it
> took so for 2.4) and I will measure async vs sync.
Then I could give the numbers for dual (and quad-) Opteron as well ;-))
But make sure that your cryptodriver could take advantage of SMP!
> Then Michal can measure it's fcrypt driver also.
I'll have to port it to acrypto first. But I'll do so :-)
Michal Ludvig
- --
SUSE Labs mludvig@suse.cz
(+420) 296.542.396 http://www.suse.cz
Personal homepage http://www.logix.cz/michal
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFBgmGfDDolCcRbIhgRAvpjAJ4+YPJh3+UTtuBvuaU2lfHaHzhNeQCbBJo9
QGUpOlP/OifPWLmhPpnmayI=
=16lO
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 15:08 ` jamal
@ 2004-10-29 15:37 ` Evgeniy Polyakov
2004-10-29 15:28 ` Michal Ludvig
2004-10-29 16:16 ` James Morris
1 sibling, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-29 15:37 UTC (permalink / raw)
To: hadi; +Cc: netdev, cryptoapi
On 29 Oct 2004 11:08:03 -0400
jamal <hadi@cyberus.ca> wrote:
> On Fri, 2004-10-29 at 10:06, Evgeniy Polyakov wrote:
>
>
> > If we have a hardware accelerator chip, than we _already_ have improvements
> > with even the worst async crypto layer, since software and hardware
> > will work in parrallel.
>
> Thats what i am hoping - and theory points to it. Numbers, numbers
> please ;->
Michal already have numbers with his patches, that(I studied crypto-dev patch)
implements acrypto not from the scratch but as a addendum for syncronous crypto
layer, so this approach already has nitpicks.
> > I agree that multigigahertz box will beat my HIFN card,
> > but I doubt it can beat 1gghz VIA.
>
> The VIA is an interesting one; but it would still be interesting to see
> comparisons.
> Actually I should say a good comparison will be with a dual opteron
> >= 2Ghz which i have no doubt will smoke a xeon - all in s/ware.
He-he, dual Opteron? It costs as 5 Via boards :)
ok, I will follow this plan:
I will create simple block device like cryptoloop, it will be _really_ simple
without any kind of portability and/or stability (I hope it will be
one day, at least it took so for 2.4) and I will measure async vs sync.
Then Michal can measure it's fcrypt driver also.
> > And what about random numbers, async crypto, TPM?
> > They all requires some kind of generalization.
> >
>
> Similar deferal approach should do it.
> If you are brave, you should probably experiment as well with
> non-crypto stuff to defer things like gettimeofday computes.
> (i.e ask the hardware for time, defer response and then call netif_rx
> with a tag saying time has been computed).
gettimeofday is to fast to use in async schema - in such times
context switching already plays big role.
> cheers,
> jamal
>
>
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 15:08 ` jamal
2004-10-29 15:37 ` Evgeniy Polyakov
@ 2004-10-29 16:16 ` James Morris
2004-10-29 16:45 ` Evgeniy Polyakov
1 sibling, 1 reply; 57+ messages in thread
From: James Morris @ 2004-10-29 16:16 UTC (permalink / raw)
To: jamal; +Cc: johnpol, netdev, cryptoapi
On 29 Oct 2004, jamal wrote:
> On Fri, 2004-10-29 at 10:06, Evgeniy Polyakov wrote:
>
>
> > If we have a hardware accelerator chip, than we _already_ have improvements
> > with even the worst async crypto layer, since software and hardware
> > will work in parrallel.
>
> Thats what i am hoping - and theory points to it. Numbers, numbers
> please ;->
There's an OpenBSD paper which demonstrates the usefulness of crypto
hardware for reducing cpu contention. I can't get to their web site now,
but it's there somewhere under a security link.
- James
--
James Morris
<jmorris@redhat.com>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 16:16 ` James Morris
@ 2004-10-29 16:45 ` Evgeniy Polyakov
2004-10-29 20:00 ` Sam Leffler
0 siblings, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-29 16:45 UTC (permalink / raw)
To: James Morris; +Cc: jamal, netdev, cryptoapi
On Fri, 29 Oct 2004 12:16:35 -0400 (EDT)
James Morris <jmorris@redhat.com> wrote:
> On 29 Oct 2004, jamal wrote:
>
> > On Fri, 2004-10-29 at 10:06, Evgeniy Polyakov wrote:
> >
> >
> > > If we have a hardware accelerator chip, than we _already_ have improvements
> > > with even the worst async crypto layer, since software and hardware
> > > will work in parrallel.
> >
> > Thats what i am hoping - and theory points to it. Numbers, numbers
> > please ;->
>
> There's an OpenBSD paper which demonstrates the usefulness of crypto
> hardware for reducing cpu contention. I can't get to their web site now,
> but it's there somewhere under a security link.
Asynchronous crypto layer was written under impression of those paper.
OpenBSD layer is very good, but it has some nitpics some of which are really
not good, but I hope all of them are resolved in my driver.
You can find paper for example there: http://www.madchat.org/crypto/ocf.pdf
>
> - James
> --
> James Morris
> <jmorris@redhat.com>
>
>
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 6:22 Asynchronous crypto layer Evgeniy Polyakov
2004-10-29 12:42 ` jamal
@ 2004-10-29 17:44 ` James Morris
2004-10-29 17:46 ` James Morris
2004-10-30 5:19 ` Evgeniy Polyakov
2004-10-29 19:56 ` Sam Leffler
` (4 subsequent siblings)
6 siblings, 2 replies; 57+ messages in thread
From: James Morris @ 2004-10-29 17:44 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: netdev, cryptoapi
On Fri, 29 Oct 2004, Evgeniy Polyakov wrote:
> I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
Could you please resend as a patch against the latest bk snapshot?
i.e. output of 'diff -purN tree-a tree-b'
- James
--
James Morris
<jmorris@redhat.com>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 17:44 ` James Morris
@ 2004-10-29 17:46 ` James Morris
2004-10-30 5:20 ` Evgeniy Polyakov
2004-10-30 5:19 ` Evgeniy Polyakov
1 sibling, 1 reply; 57+ messages in thread
From: James Morris @ 2004-10-29 17:46 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: netdev, cryptoapi
On Fri, 29 Oct 2004, James Morris wrote:
> On Fri, 29 Oct 2004, Evgeniy Polyakov wrote:
>
> > I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
>
Also, please consider EXPORT_SYMBOL_GPL instead of EXPORT_SYMBOL.
- James
--
James Morris
<jmorris@redhat.com>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 6:22 Asynchronous crypto layer Evgeniy Polyakov
2004-10-29 12:42 ` jamal
2004-10-29 17:44 ` James Morris
@ 2004-10-29 19:56 ` Sam Leffler
2004-10-30 6:16 ` Christoph Hellwig
2004-10-30 20:37 ` Evgeniy Polyakov
` (3 subsequent siblings)
6 siblings, 1 reply; 57+ messages in thread
From: Sam Leffler @ 2004-10-29 19:56 UTC (permalink / raw)
To: johnpol; +Cc: netdev, cryptoapi
Evgeniy Polyakov wrote:
> I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
> It support following features:
> - multiple asynchronous crypto device queues
> - crypto session routing
> - crypto session binding
> - modular load balancing
> - crypto session batching genetically implemented by design
> - crypto session priority
> - different kinds of crypto operation(RNG, asymmetrical crypto, HMAC and
> any other)
It's really great to see this stuff come in to Linux. Hopefully you're
aware of existing functionality of this sort in each of the BSD systems?
It would be interesting to compare performance between systems.
I realize this suggestion will be met with the usual boo's and catcalls
but it'd also be nice if all systems could share a common user-level api
so, for example, you could just _use_ the support in openssl that's been
around for several years.
Sam
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 16:45 ` Evgeniy Polyakov
@ 2004-10-29 20:00 ` Sam Leffler
2004-10-31 6:09 ` James Morris
0 siblings, 1 reply; 57+ messages in thread
From: Sam Leffler @ 2004-10-29 20:00 UTC (permalink / raw)
To: johnpol; +Cc: James Morris, jamal, netdev, cryptoapi
Evgeniy Polyakov wrote:
> On Fri, 29 Oct 2004 12:16:35 -0400 (EDT)
> James Morris <jmorris@redhat.com> wrote:
>
>
>>On 29 Oct 2004, jamal wrote:
>>
>>
>>>On Fri, 2004-10-29 at 10:06, Evgeniy Polyakov wrote:
>>>
>>>
>>>
>>>>If we have a hardware accelerator chip, than we _already_ have improvements
>>>>with even the worst async crypto layer, since software and hardware
>>>>will work in parrallel.
>>>
>>>Thats what i am hoping - and theory points to it. Numbers, numbers
>>>please ;->
>>
>>There's an OpenBSD paper which demonstrates the usefulness of crypto
>>hardware for reducing cpu contention. I can't get to their web site now,
>>but it's there somewhere under a security link.
>
>
> Asynchronous crypto layer was written under impression of those paper.
> OpenBSD layer is very good, but it has some nitpics some of which are really
> not good, but I hope all of them are resolved in my driver.
>
> You can find paper for example there: http://www.madchat.org/crypto/ocf.pdf
And a paper I wrote explains how I fixed many of the problems with the
openbsd code in freebsd.
http://www.usenix.org/publications/library/proceedings/bsdcon03/tech/leffler_crypto/leffler_crpto_html/
In the same proceedings you'll also find a horrible paper I wrote about
some performance tuning I did in my IPsec implementation that uses the
crypto infrastructure.
http://www.usenix.org/publications/library/proceedings/bsdcon03/tech/full_papers/leffler_ipsec/leffler_ipsec_html/
Sam
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 17:44 ` James Morris
2004-10-29 17:46 ` James Morris
@ 2004-10-30 5:19 ` Evgeniy Polyakov
2004-10-30 8:34 ` Evgeniy Polyakov
2004-10-30 16:57 ` Michal Ludvig
1 sibling, 2 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-30 5:19 UTC (permalink / raw)
To: James Morris; +Cc: netdev, cryptoapi
[-- Attachment #1: Type: text/plain, Size: 848 bytes --]
On Fri, 29 Oct 2004 13:44:24 -0400 (EDT)
James Morris <jmorris@redhat.com> wrote:
> On Fri, 29 Oct 2004, Evgeniy Polyakov wrote:
>
> > I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
>
> Could you please resend as a patch against the latest bk snapshot?
>
> i.e. output of 'diff -purN tree-a tree-b'
>
Attached following patches:
Kconfig.connector.patch
Kconfig.crypto.patch
Makefile.connector.patch
Makefile.crypto.patch
acrypto.patch
connector.patch
Any other files are examples and do not included here.
>
> - James
> --
> James Morris
> <jmorris@redhat.com>
>
>
> _______________________________________________
>
> Subscription: http://lists.logix.cz/mailman/listinfo/cryptoapi
> List archive: http://lists.logix.cz/pipermail/cryptoapi
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
[-- Attachment #2: connector.patch --]
[-- Type: application/octet-stream, Size: 23619 bytes --]
diff -Nru /tmp/empty/Kconfig linux-2.6/drivers/connector/Kconfig
--- /tmp/empty/Kconfig 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/connector/Kconfig 2004-09-26 00:12:57.000000000 +0400
@@ -0,0 +1,13 @@
+menu "Connector - unified userspace <-> kernelspace linker"
+
+config CONNECTOR
+ tristate "Connector - unified userspace <-> kernelspace linker"
+ depends on NET
+ ---help---
+ This is unified userspace <-> kernelspace connector working on top
+ of the netlink socket protocol.
+
+ Connector support can also be built as a module. If so, the module
+ will be called connector.ko.
+
+endmenu
diff -Nru /tmp/empty/Makefile linux-2.6/drivers/connector/Makefile
--- /tmp/empty/Makefile 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/connector/Makefile 2004-09-26 00:12:57.000000000 +0400
@@ -0,0 +1,2 @@
+obj-$(CONFIG_CONNECTOR) += cn.o
+cn-objs := cn_queue.o connector.o
diff -Nru /tmp/empty/cn_queue.c linux-2.6/drivers/connector/cn_queue.c
--- /tmp/empty/cn_queue.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/connector/cn_queue.c 2004-09-26 21:50:12.000000000 +0400
@@ -0,0 +1,219 @@
+/*
+ * cn_queue.c
+ *
+ * 2004 Copyright (c) Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ * All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ *
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/list.h>
+#include <linux/workqueue.h>
+#include <linux/spinlock.h>
+#include <linux/slab.h>
+#include <linux/skbuff.h>
+#include <linux/suspend.h>
+
+#include "cn_queue.h"
+
+static void cn_queue_wrapper(void *data)
+{
+ struct cn_callback_entry *cbq = (struct cn_callback_entry *)data;
+
+ atomic_inc(&cbq->cb->refcnt);
+ cbq->cb->callback(cbq->cb->priv);
+ atomic_dec(&cbq->cb->refcnt);
+
+ cbq->destruct_data(cbq->ddata);
+}
+
+static struct cn_callback_entry *cn_queue_alloc_callback_entry(struct
+ cn_callback *cb)
+{
+ struct cn_callback_entry *cbq;
+
+ cbq = kmalloc(sizeof(*cbq), GFP_KERNEL);
+ if (!cbq) {
+ printk(KERN_ERR "Failed to create new callback queue.\n");
+ return NULL;
+ }
+
+ memset(cbq, 0, sizeof(*cbq));
+
+ cbq->cb = cb;
+
+ INIT_WORK(&cbq->work, &cn_queue_wrapper, cbq);
+
+ return cbq;
+}
+
+static void cn_queue_free_callback(struct cn_callback_entry *cbq)
+{
+ cancel_delayed_work(&cbq->work);
+
+ while (atomic_read(&cbq->cb->refcnt)) {
+ printk(KERN_INFO "Waiting for %s to become free: refcnt=%d.\n",
+ cbq->pdev->name, atomic_read(&cbq->cb->refcnt));
+ set_current_state(TASK_INTERRUPTIBLE);
+ schedule_timeout(HZ);
+
+ if (current->flags & PF_FREEZE)
+ refrigerator(PF_FREEZE);
+
+ if (signal_pending(current))
+ flush_signals(current);
+ }
+
+ kfree(cbq);
+}
+
+int cn_cb_equal(struct cb_id *i1, struct cb_id *i2)
+{
+ return ((i1->idx == i2->idx) && (i1->val == i2->val));
+}
+
+int cn_queue_add_callback(struct cn_queue_dev *dev, struct cn_callback *cb)
+{
+ struct cn_callback_entry *cbq, *n, *__cbq;
+ int found = 0;
+
+ cbq = cn_queue_alloc_callback_entry(cb);
+ if (!cbq)
+ return -ENOMEM;
+
+ atomic_inc(&dev->refcnt);
+ cbq->pdev = dev;
+
+ spin_lock(&dev->queue_lock);
+ list_for_each_entry_safe(__cbq, n, &dev->queue_list, callback_entry) {
+ if (cn_cb_equal(&__cbq->cb->id, &cb->id)) {
+ found = 1;
+ break;
+ }
+ }
+ if (!found) {
+ atomic_set(&cbq->cb->refcnt, 1);
+ list_add_tail(&cbq->callback_entry, &dev->queue_list);
+ }
+ spin_unlock(&dev->queue_lock);
+
+ if (found) {
+ atomic_dec(&dev->refcnt);
+ atomic_set(&cbq->cb->refcnt, 0);
+ cn_queue_free_callback(cbq);
+ return -EINVAL;
+ }
+
+ cbq->nls = dev->nls;
+ cbq->seq = 0;
+ //cbq->group = ++dev->netlink_groups;
+ cbq->group = cbq->cb->id.idx;
+
+ return 0;
+}
+
+void cn_queue_del_callback(struct cn_queue_dev *dev, struct cn_callback *cb)
+{
+ struct cn_callback_entry *cbq = NULL, *n;
+ int found = 0;
+
+ spin_lock(&dev->queue_lock);
+ list_for_each_entry_safe(cbq, n, &dev->queue_list, callback_entry) {
+ if (cn_cb_equal(&cbq->cb->id, &cb->id)) {
+ list_del(&cbq->callback_entry);
+ found = 1;
+ break;
+ }
+ }
+ spin_unlock(&dev->queue_lock);
+
+ if (found) {
+ atomic_dec(&cbq->cb->refcnt);
+ cn_queue_free_callback(cbq);
+ atomic_dec(&dev->refcnt);
+ }
+}
+
+struct cn_queue_dev *cn_queue_alloc_dev(char *name, struct sock *nls)
+{
+ struct cn_queue_dev *dev;
+
+ dev = kmalloc(sizeof(*dev), GFP_KERNEL);
+ if (!dev) {
+ printk(KERN_ERR "%s: Failed to allocte new struct cn_queue_dev.\n",
+ name);
+ return NULL;
+ }
+
+ memset(dev, 0, sizeof(*dev));
+
+ snprintf(dev->name, sizeof(dev->name), "%s", name);
+
+ atomic_set(&dev->refcnt, 0);
+ INIT_LIST_HEAD(&dev->queue_list);
+ spin_lock_init(&dev->queue_lock);
+
+ dev->nls = nls;
+ dev->netlink_groups = 0;
+
+ dev->cn_queue = create_workqueue(dev->name);
+ if (!dev->cn_queue) {
+ printk(KERN_ERR "Failed to create %s queue.\n", dev->name);
+ kfree(dev);
+ return NULL;
+ }
+
+ return dev;
+}
+
+void cn_queue_free_dev(struct cn_queue_dev *dev)
+{
+ struct cn_callback_entry *cbq, *n;
+
+ flush_workqueue(dev->cn_queue);
+ destroy_workqueue(dev->cn_queue);
+
+ spin_lock(&dev->queue_lock);
+ list_for_each_entry_safe(cbq, n, &dev->queue_list, callback_entry) {
+ list_del(&cbq->callback_entry);
+ atomic_dec(&cbq->cb->refcnt);
+ }
+ spin_unlock(&dev->queue_lock);
+
+ while (atomic_read(&dev->refcnt)) {
+ printk(KERN_INFO "Waiting for %s to become free: refcnt=%d.\n",
+ dev->name, atomic_read(&dev->refcnt));
+ set_current_state(TASK_INTERRUPTIBLE);
+ schedule_timeout(HZ);
+
+ if (current->flags & PF_FREEZE)
+ refrigerator(PF_FREEZE);
+
+ if (signal_pending(current))
+ flush_signals(current);
+ }
+
+ memset(dev, 0, sizeof(*dev));
+ kfree(dev);
+ dev = NULL;
+}
+
+EXPORT_SYMBOL(cn_queue_add_callback);
+EXPORT_SYMBOL(cn_queue_del_callback);
+EXPORT_SYMBOL(cn_queue_alloc_dev);
+EXPORT_SYMBOL(cn_queue_free_dev);
diff -Nru /tmp/empty/cn_queue.h linux-2.6/drivers/connector/cn_queue.h
--- /tmp/empty/cn_queue.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/connector/cn_queue.h 2004-09-26 00:14:16.000000000 +0400
@@ -0,0 +1,90 @@
+/*
+ * cn_queue.h
+ *
+ * 2004 Copyright (c) Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ * All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CN_QUEUE_H
+#define __CN_QUEUE_H
+
+#include <asm/types.h>
+
+struct cb_id
+{
+ __u32 idx;
+ __u32 val;
+};
+
+#ifdef __KERNEL__
+
+#include <asm/atomic.h>
+
+#include <linux/list.h>
+#include <linux/workqueue.h>
+
+#define CN_CBQ_NAMELEN 32
+
+struct cn_queue_dev
+{
+ atomic_t refcnt;
+ unsigned char name[CN_CBQ_NAMELEN];
+
+ struct workqueue_struct *cn_queue;
+
+ struct list_head queue_list;
+ spinlock_t queue_lock;
+
+ int netlink_groups;
+ struct sock *nls;
+};
+
+struct cn_callback
+{
+ unsigned char name[CN_CBQ_NAMELEN];
+
+ struct cb_id id;
+ void (* callback)(void *);
+ void *priv;
+
+ atomic_t refcnt;
+};
+
+struct cn_callback_entry
+{
+ struct list_head callback_entry;
+ struct cn_callback *cb;
+ struct work_struct work;
+ struct cn_queue_dev *pdev;
+
+ void (* destruct_data)(void *);
+ void *ddata;
+
+ int seq, group;
+ struct sock *nls;
+};
+
+int cn_queue_add_callback(struct cn_queue_dev *dev, struct cn_callback *cb);
+void cn_queue_del_callback(struct cn_queue_dev *dev, struct cn_callback *cb);
+
+struct cn_queue_dev *cn_queue_alloc_dev(char *name, struct sock *);
+void cn_queue_free_dev(struct cn_queue_dev *dev);
+
+int cn_cb_equal(struct cb_id *, struct cb_id *);
+
+#endif /* __KERNEL__ */
+#endif /* __CN_QUEUE_H */
diff -Nru /tmp/empty/connector.c linux-2.6/drivers/connector/connector.c
--- /tmp/empty/connector.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/connector/connector.c 2004-09-26 21:52:02.000000000 +0400
@@ -0,0 +1,498 @@
+/*
+ * connector.c
+ *
+ * 2004 Copyright (c) Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ * All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/list.h>
+#include <linux/skbuff.h>
+#include <linux/netlink.h>
+#include <linux/moduleparam.h>
+
+#include <net/sock.h>
+
+#include "../connector/connector.h"
+#include "../connector/cn_queue.h"
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
+MODULE_DESCRIPTION("Generic userspace <-> kernelspace connector.");
+
+static int unit = NETLINK_NFLOG;
+static u32 cn_idx = -1;
+static u32 cn_val = -1;
+
+module_param(unit, int, 0);
+module_param(cn_idx, uint, 0);
+module_param(cn_val, uint, 0);
+
+spinlock_t notify_lock = SPIN_LOCK_UNLOCKED;
+static LIST_HEAD(notify_list);
+
+static struct cn_dev cdev;
+
+/*
+ * msg->seq and msg->ack are used to determine message genealogy.
+ * When someone sends message it puts there locally unique sequence
+ * and random acknowledge numbers.
+ * Sequence number may be copied into nlmsghdr->nlmsg_seq too.
+ *
+ * Sequence number is incremented with each message to be sent.
+ *
+ * If we expect reply to our message,
+ * then sequence number in received message MUST be the same as in original message,
+ * and acknowledge number MUST be the same + 1.
+ *
+ * If we receive message and it's sequence number is not equal to one we are expecting,
+ * then it is new message.
+ * If we receive message and it's sequence number is the same as one we are expecting,
+ * but it's acknowledge is not equal acknowledge number in original message + 1,
+ * then it is new message.
+ *
+ */
+void cn_netlink_send(struct cn_msg *msg, u32 __groups)
+{
+ struct cn_callback_entry *n, *__cbq;
+ unsigned int size;
+ struct sk_buff *skb;
+ struct nlmsghdr *nlh;
+ struct cn_msg *data;
+ struct cn_dev *dev = &cdev;
+ u32 groups = 0;
+ int found = 0;
+
+ if (!__groups)
+ {
+ spin_lock(&dev->cbdev->queue_lock);
+ list_for_each_entry_safe(__cbq, n, &dev->cbdev->queue_list, callback_entry) {
+ if (cn_cb_equal(&__cbq->cb->id, &msg->id)) {
+ found = 1;
+ groups = __cbq->group;
+ }
+ }
+ spin_unlock(&dev->cbdev->queue_lock);
+
+ if (!found) {
+ printk(KERN_ERR "Failed to find multicast netlink group for callback[0x%x.0x%x]. seq=%u\n",
+ msg->id.idx, msg->id.val, msg->seq);
+ return;
+ }
+ }
+ else
+ groups = __groups;
+
+ size = NLMSG_SPACE(sizeof(*msg) + msg->len);
+
+ skb = alloc_skb(size, GFP_ATOMIC);
+ if (!skb) {
+ printk(KERN_ERR "Failed to allocate new skb with size=%u.\n", size);
+ return;
+ }
+
+ nlh = NLMSG_PUT(skb, 0, msg->seq, NLMSG_DONE, size - sizeof(*nlh));
+
+ data = (struct cn_msg *)NLMSG_DATA(nlh);
+
+ memcpy(data, msg, sizeof(*data) + msg->len);
+#if 0
+ printk("%s: len=%u, seq=%u, ack=%u, group=%u.\n",
+ __func__, msg->len, msg->seq, msg->ack, groups);
+#endif
+ NETLINK_CB(skb).dst_groups = groups;
+ netlink_broadcast(dev->nls, skb, 0, groups, GFP_ATOMIC);
+
+ return;
+
+ nlmsg_failure:
+ printk(KERN_ERR "Failed to send %u.%u\n", msg->seq, msg->ack);
+ kfree_skb(skb);
+ return;
+}
+
+static int cn_call_callback(struct cn_msg *msg, void (*destruct_data) (void *), void *data)
+{
+ struct cn_callback_entry *n, *__cbq;
+ struct cn_dev *dev = &cdev;
+ int found = 0;
+
+ spin_lock(&dev->cbdev->queue_lock);
+ list_for_each_entry_safe(__cbq, n, &dev->cbdev->queue_list, callback_entry) {
+ if (cn_cb_equal(&__cbq->cb->id, &msg->id)) {
+ __cbq->cb->priv = msg;
+
+ __cbq->ddata = data;
+ __cbq->destruct_data = destruct_data;
+
+ queue_work(dev->cbdev->cn_queue, &__cbq->work);
+ found = 1;
+ break;
+ }
+ }
+ spin_unlock(&dev->cbdev->queue_lock);
+
+ return found;
+}
+
+static int __cn_rx_skb(struct sk_buff *skb, struct nlmsghdr *nlh)
+{
+ u32 pid, uid, seq, group;
+ struct cn_msg *msg;
+
+ pid = NETLINK_CREDS(skb)->pid;
+ uid = NETLINK_CREDS(skb)->uid;
+ seq = nlh->nlmsg_seq;
+ group = NETLINK_CB((skb)).groups;
+ msg = (struct cn_msg *)NLMSG_DATA(nlh);
+
+ if (msg->len != nlh->nlmsg_len - sizeof(*msg) - sizeof(*nlh)) {
+ printk(KERN_ERR "skb does not have enough length: "
+ "requested msg->len=%u[%u], nlh->nlmsg_len=%u[%u], skb->len=%u[must be %u].\n",
+ msg->len, NLMSG_SPACE(msg->len),
+ nlh->nlmsg_len, nlh->nlmsg_len - sizeof(*nlh),
+ skb->len, msg->len + sizeof(*msg));
+ return -EINVAL;
+ }
+#if 0
+ printk(KERN_INFO "pid=%u, uid=%u, seq=%u, group=%u.\n",
+ pid, uid, seq, group);
+#endif
+ return cn_call_callback(msg, (void (*)(void *))kfree_skb, skb);
+}
+
+static void cn_rx_skb(struct sk_buff *__skb)
+{
+ struct nlmsghdr *nlh;
+ u32 len;
+ int err;
+ struct sk_buff *skb;
+
+ skb = skb_get(__skb);
+ if (!skb) {
+ printk(KERN_ERR "Failed to reference an skb.\n");
+ return;
+ }
+#if 0
+ printk(KERN_INFO
+ "skb: len=%u, data_len=%u, truesize=%u, proto=%u, cloned=%d, shared=%d.\n",
+ skb->len, skb->data_len, skb->truesize, skb->protocol,
+ skb_cloned(skb), skb_shared(skb));
+#endif
+ while (skb->len >= NLMSG_SPACE(0)) {
+ nlh = (struct nlmsghdr *)skb->data;
+ if (nlh->nlmsg_len < sizeof(struct cn_msg) ||
+ skb->len < nlh->nlmsg_len ||
+ nlh->nlmsg_len > CONNECTOR_MAX_MSG_SIZE) {
+ printk(KERN_INFO "nlmsg_len=%u, sizeof(*nlh)=%u\n",
+ nlh->nlmsg_len, sizeof(*nlh));
+ break;
+ }
+
+ len = NLMSG_ALIGN(nlh->nlmsg_len);
+ if (len > skb->len)
+ len = skb->len;
+
+ err = __cn_rx_skb(skb, nlh);
+ if (err) {
+#if 0
+ if (err < 0 && (nlh->nlmsg_flags & NLM_F_ACK))
+ netlink_ack(skb, nlh, -err);
+#endif
+ kfree_skb(skb);
+ break;
+ } else {
+#if 0
+ if (nlh->nlmsg_flags & NLM_F_ACK)
+ netlink_ack(skb, nlh, 0);
+#endif
+ kfree_skb(skb);
+ break;
+ }
+ skb_pull(skb, len);
+ }
+}
+
+static void cn_input(struct sock *sk, int len)
+{
+ struct sk_buff *skb;
+
+ while ((skb = skb_dequeue(&sk->sk_receive_queue)) != NULL)
+ cn_rx_skb(skb);
+}
+
+static void cn_notify(struct cb_id *id, u32 notify_event)
+{
+ struct cn_ctl_entry *ent;
+
+ spin_lock(¬ify_lock);
+ list_for_each_entry(ent, ¬ify_list, notify_entry) {
+ int i;
+ struct cn_notify_req *req;
+ struct cn_ctl_msg *ctl = ent->msg;
+ int a, b;
+
+ a = b = 0;
+
+ req = (struct cn_notify_req *)ctl->data;
+ for (i=0; i<ctl->idx_notify_num; ++i, ++req) {
+ if (id->idx >= req->first && id->idx < req->first + req->range) {
+ a = 1;
+ break;
+ }
+ }
+
+ for (i=0; i<ctl->val_notify_num; ++i, ++req) {
+ if (id->val >= req->first && id->val < req->first + req->range) {
+ b = 1;
+ break;
+ }
+ }
+
+ if (a && b) {
+ struct cn_msg m;
+
+ printk(KERN_INFO "Notifying group %x with event %u about %x.%x.\n",
+ ctl->group, notify_event,
+ id->idx, id->val);
+
+ memset(&m, 0, sizeof(m));
+ m.ack = notify_event;
+
+ memcpy(&m.id, id, sizeof(m.id));
+ cn_netlink_send(&m, ctl->group);
+ }
+ }
+ spin_unlock(¬ify_lock);
+}
+
+int cn_add_callback(struct cb_id *id, char *name, void (*callback) (void *))
+{
+ int err;
+ struct cn_dev *dev = &cdev;
+ struct cn_callback *cb;
+
+ cb = kmalloc(sizeof(*cb), GFP_KERNEL);
+ if (!cb) {
+ printk(KERN_INFO "%s: Failed to allocate new struct cn_callback.\n",
+ dev->cbdev->name);
+ return -ENOMEM;
+ }
+
+ memset(cb, 0, sizeof(*cb));
+
+ snprintf(cb->name, sizeof(cb->name), "%s", name);
+
+ memcpy(&cb->id, id, sizeof(cb->id));
+ cb->callback = callback;
+
+ atomic_set(&cb->refcnt, 0);
+
+ err = cn_queue_add_callback(dev->cbdev, cb);
+ if (err) {
+ kfree(cb);
+ return err;
+ }
+
+ cn_notify(id, 0);
+
+ return 0;
+}
+
+void cn_del_callback(struct cb_id *id)
+{
+ struct cn_dev *dev = &cdev;
+ struct cn_callback_entry *n, *__cbq;
+
+ list_for_each_entry_safe(__cbq, n, &dev->cbdev->queue_list, callback_entry) {
+ if (cn_cb_equal(&__cbq->cb->id, id)) {
+ cn_queue_del_callback(dev->cbdev, __cbq->cb);
+ cn_notify(id, 1);
+ break;
+ }
+ }
+}
+
+static int cn_ctl_msg_equals(struct cn_ctl_msg *m1, struct cn_ctl_msg *m2)
+{
+ int i;
+ struct cn_notify_req *req1, *req2;
+
+ if (m1->idx_notify_num != m2->idx_notify_num)
+ return 0;
+
+ if (m1->val_notify_num != m2->val_notify_num)
+ return 0;
+
+ if (m1->len != m2->len)
+ return 0;
+
+ if ((m1->idx_notify_num + m1->val_notify_num)*sizeof(*req1) != m1->len) {
+ printk(KERN_ERR "Notify entry[idx_num=%x, val_num=%x, len=%u] contains garbage. Removing.\n",
+ m1->idx_notify_num, m1->val_notify_num, m1->len);
+ return 1;
+ }
+
+ req1 = (struct cn_notify_req *)m1->data;
+ req2 = (struct cn_notify_req *)m2->data;
+
+ for (i=0; i<m1->idx_notify_num; ++i) {
+ if (memcmp(req1, req2, sizeof(*req1)))
+ return 0;
+
+ req1++;
+ req2++;
+ }
+
+ for (i=0; i<m1->val_notify_num; ++i) {
+ if (memcmp(req1, req2, sizeof(*req1)))
+ return 0;
+
+ req1++;
+ req2++;
+ }
+
+ return 1;
+}
+
+static void cn_callback(void * data)
+{
+ struct cn_msg *msg = (struct cn_msg *)data;
+ struct cn_ctl_msg *ctl;
+ struct cn_ctl_entry *ent;
+ u32 size;
+
+ if (msg->len < sizeof(*ctl)) {
+ printk(KERN_ERR "Wrong connector request size %u, must be >= %u.\n",
+ msg->len, sizeof(*ctl));
+ return;
+ }
+
+ ctl = (struct cn_ctl_msg *)msg->data;
+
+ size = sizeof(*ctl) + (ctl->idx_notify_num + ctl->val_notify_num)*sizeof(struct cn_notify_req);
+
+ if (msg->len != size) {
+ printk(KERN_ERR "Wrong connector request size %u, must be == %u.\n",
+ msg->len, size);
+ return;
+ }
+
+ if (ctl->len + sizeof(*ctl) != msg->len) {
+ printk(KERN_ERR "Wrong message: msg->len=%u must be equal to inner_len=%u [+%u].\n",
+ msg->len, ctl->len, sizeof(*ctl));
+ return;
+ }
+
+ /*
+ * Remove notification.
+ */
+ if (ctl->group == 0) {
+ struct cn_ctl_entry *n;
+
+ spin_lock(¬ify_lock);
+ list_for_each_entry_safe(ent, n, ¬ify_list, notify_entry) {
+ if (cn_ctl_msg_equals(ent->msg, ctl)) {
+ list_del(&ent->notify_entry);
+ kfree(ent);
+ }
+ }
+ spin_unlock(¬ify_lock);
+
+ return;
+ }
+
+ size += sizeof(*ent);
+
+ ent = kmalloc(size, GFP_ATOMIC);
+ if (!ent) {
+ printk(KERN_ERR "Failed to allocate %d bytes for new notify entry.\n", size);
+ return;
+ }
+
+ memset(ent, 0, size);
+
+ ent->msg = (struct cn_ctl_msg *)(ent + 1);
+
+ memcpy(ent->msg, ctl, size - sizeof(*ent));
+
+ spin_lock(¬ify_lock);
+ list_add(&ent->notify_entry, ¬ify_list);
+ spin_unlock(¬ify_lock);
+
+ {
+ int i;
+ struct cn_notify_req *req;
+
+ printk("Notify group %x for idx: ", ctl->group);
+
+ req = (struct cn_notify_req *)ctl->data;
+ for (i=0; i<ctl->idx_notify_num; ++i, ++req) {
+ printk("%u-%u ", req->first, req->first+req->range-1);
+ }
+
+ printk("\nNotify group %x for val: ", ctl->group);
+
+ for (i=0; i<ctl->val_notify_num; ++i, ++req) {
+ printk("%u-%u ", req->first, req->first+req->range-1);
+ }
+ printk("\n");
+ }
+}
+
+static int cn_init(void)
+{
+ struct cn_dev *dev = &cdev;
+
+ dev->input = cn_input;
+ dev->id.idx = cn_idx;
+ dev->id.val = cn_val;
+
+ dev->nls = netlink_kernel_create(unit, dev->input);
+ if (!dev->nls) {
+ printk(KERN_ERR "Failed to create new netlink socket(%u).\n",
+ unit);
+ return -EIO;
+ }
+
+ dev->cbdev = cn_queue_alloc_dev("cqueue", dev->nls);
+ if (!dev->cbdev) {
+ if (dev->nls->sk_socket)
+ sock_release(dev->nls->sk_socket);
+ return -EINVAL;
+ }
+
+ return cn_add_callback(&dev->id, "connector", &cn_callback);
+}
+
+static void cn_fini(void)
+{
+ struct cn_dev *dev = &cdev;
+
+ cn_del_callback(&dev->id);
+ cn_queue_free_dev(dev->cbdev);
+ if (dev->nls->sk_socket)
+ sock_release(dev->nls->sk_socket);
+}
+
+module_init(cn_init);
+module_exit(cn_fini);
+
+EXPORT_SYMBOL(cn_add_callback);
+EXPORT_SYMBOL(cn_del_callback);
+EXPORT_SYMBOL(cn_netlink_send);
diff -Nru /tmp/empty/connector.h linux-2.6/drivers/connector/connector.h
--- /tmp/empty/connector.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/connector/connector.h 2004-09-26 00:14:16.000000000 +0400
@@ -0,0 +1,81 @@
+/*
+ * connector.h
+ *
+ * 2004 Copyright (c) Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ * All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CONNECTOR_H
+#define __CONNECTOR_H
+
+#include "../connector/cn_queue.h"
+
+#define CONNECTOR_MAX_MSG_SIZE 1024
+
+struct cn_msg
+{
+ struct cb_id id;
+
+ __u32 seq;
+ __u32 ack;
+
+ __u32 len; /* Length of the following data */
+ __u8 data[0];
+};
+
+struct cn_notify_req
+{
+ __u32 first;
+ __u32 range;
+};
+
+struct cn_ctl_msg
+{
+ __u32 idx_notify_num;
+ __u32 val_notify_num;
+ __u32 group;
+ __u32 len;
+ __u8 data[0];
+};
+
+#ifdef __KERNEL__
+
+#include <net/sock.h>
+
+struct cn_ctl_entry
+{
+ struct list_head notify_entry;
+ struct cn_ctl_msg *msg;
+};
+
+struct cn_dev
+{
+ struct cb_id id;
+
+ u32 seq, groups;
+ struct sock *nls;
+ void (*input)(struct sock *sk, int len);
+
+ struct cn_queue_dev *cbdev;
+};
+
+int cn_add_callback(struct cb_id *, char *, void (* callback)(void *));
+void cn_del_callback(struct cb_id *);
+void cn_netlink_send(struct cn_msg *, u32);
+
+#endif /* __KERNEL__ */
+#endif /* __CONNECTOR_H */
[-- Attachment #3: Kconfig.connector.patch --]
[-- Type: application/octet-stream, Size: 292 bytes --]
--- linux-2.6/drivers/Kconfig.orig 2004-09-26 13:34:48.000000000 +0400
+++ linux-2.6/drivers/Kconfig 2004-09-26 13:34:57.000000000 +0400
@@ -44,6 +44,8 @@
source "drivers/w1/Kconfig"
+source "drivers/connector/Kconfig"
+
source "drivers/misc/Kconfig"
source "drivers/media/Kconfig"
[-- Attachment #4: Kconfig.crypto.patch --]
[-- Type: application/octet-stream, Size: 296 bytes --]
--- linux-2.6/drivers/Kconfig.nocrypto 2004-10-30 09:05:52.000000000 +0400
+++ linux-2.6/drivers/Kconfig 2004-10-30 09:06:11.000000000 +0400
@@ -42,6 +42,8 @@
source "drivers/i2c/Kconfig"
+source "drivers/acrypto/Kconfig"
+
source "drivers/w1/Kconfig"
source "drivers/connector/Kconfig"
[-- Attachment #5: Makefile.connector.patch --]
[-- Type: application/octet-stream, Size: 378 bytes --]
--- linux-2.6/drivers/Makefile.orig 2004-09-25 23:47:08.000000000 +0400
+++ linux-2.6/drivers/Makefile 2004-09-26 13:34:25.000000000 +0400
@@ -44,6 +44,7 @@
obj-$(CONFIG_I2O) += message/
obj-$(CONFIG_I2C) += i2c/
obj-$(CONFIG_W1) += w1/
+obj-$(CONFIG_CONNECTOR) += connector/
obj-$(CONFIG_PHONE) += telephony/
obj-$(CONFIG_MD) += md/
obj-$(CONFIG_BT) += bluetooth/
[-- Attachment #6: Makefile.crypto.patch --]
[-- Type: application/octet-stream, Size: 367 bytes --]
--- linux-2.6/drivers/Makefile.nocrypto 2004-10-30 09:05:02.000000000 +0400
+++ linux-2.6/drivers/Makefile 2004-10-30 09:05:31.000000000 +0400
@@ -49,5 +49,6 @@
obj-$(CONFIG_GAMEPORT) += input/gameport/
obj-$(CONFIG_I2O) += message/
obj-$(CONFIG_I2C) += i2c/
+obj-$(CONFIG_ACRYPTO) += acrypto/
obj-$(CONFIG_W1) += w1/
obj-$(CONFIG_CONNECTOR) += connector/
[-- Attachment #7: acrypto.patch --]
[-- Type: application/octet-stream, Size: 89095 bytes --]
diff -Nru /tmp/empty/Makefile linux-2.6/drivers/acrypto/Makefile
--- /tmp/empty/Makefile 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/Makefile 2004-10-30 09:03:14.000000000 +0400
@@ -0,0 +1,5 @@
+obj-$(CONFIG_ACRYPTO) += acrypto.o
+obj-$(CONFIG_SIMPLE_LB) += simple_lb.o
+acrypto-objs := crypto_main.o crypto_lb.o crypto_dev.o crypto_conn.o crypto_stat.o
+
+obj-$(CONFIG_VIA_PADLOCK) += via-padlock/
diff -Nru /tmp/empty/acrypto.h linux-2.6/drivers/acrypto/acrypto.h
--- /tmp/empty/acrypto.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/acrypto.h 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,247 @@
+/*
+ * acrypto.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __ACRYPTO_H
+#define __ACRYPTO_H
+
+#define SCACHE_NAMELEN 32
+
+struct crypto_session_initializer;
+struct crypto_data;
+typedef void (*crypto_callback_t)(struct crypto_session_initializer *, struct crypto_data *);
+
+#if 0
+#define SESSION_COMPLETED 0
+#define SESSION_FINISHED 1
+#define SESSION_STARTED 2
+#define SESSION_PROCESSED 3
+
+#define session_completed(s) (test_bit(SESSION_COMPLETED, (unsigned long *)&s->ci.flags))
+#define complete_session(s) do { set_bit(SESSION_COMPLETED, (unsigned long *)&s->ci.flags); }while (0)
+#define uncomplete_session(s) do { clear_bit(SESSION_COMPLETED, (unsigned long *)&s->ci.flags); } while (0)
+
+#define session_started(s) (test_bit(SESSION_STARTED, (unsigned long *)&s->ci.flags))
+#define start_session(s) do { set_bit(SESSION_STARTED, (unsigned long *)&s->ci.flags); } while (0)
+#define unstart_session(s) do { clear_bit(SESSION_STARTED, (unsigned long *)&s->ci.flags); } while (0)
+
+#define session_finished(s) (test_bit(SESSION_FINISHED, (unsigned long *)&s->ci.flags))
+#define finish_session(s) do { set_bit(SESSION_FINISHED, (unsigned long *)&s->ci.flags); } while (0)
+#define unfinish_session(s) do { clear_bit(SESSION_FINISHED, (unsigned long *)&s->ci.flags); } while (0)
+
+#define session_is_processed(s) (test_bit(SESSION_PROCESSED, (unsigned long *)&s->ci.flags))
+#define start_process_session(s) do { set_bit(SESSION_PROCESSED, (unsigned long *)&s->ci.flags); } while (0)
+#define stop_process_session(s) do { clear_bit(SESSION_PROCESSED, (unsigned long *)&s->ci.flags); } while (0)
+#else
+
+#define SESSION_COMPLETED (1<<15)
+#define SESSION_FINISHED (1<<14)
+#define SESSION_STARTED (1<<13)
+#define SESSION_PROCESSED (1<<12)
+#define SESSION_BINDED (1<<11)
+#define SESSION_BROKEN (1<<10)
+
+#define session_completed(s) (s->ci.flags & SESSION_COMPLETED)
+#define complete_session(s) do {s->ci.flags |= SESSION_COMPLETED;} while(0)
+#define uncomplete_session(s) do {s->ci.flags &= ~SESSION_COMPLETED;} while (0)
+
+#define session_finished(s) (s->ci.flags & SESSION_FINISHED)
+#define finish_session(s) do {s->ci.flags |= SESSION_FINISHED;} while(0)
+#define unfinish_session(s) do {s->ci.flags &= ~SESSION_FINISHED;} while (0)
+
+#define session_started(s) (s->ci.flags & SESSION_STARTED)
+#define start_session(s) do {s->ci.flags |= SESSION_STARTED;} while(0)
+#define unstart_session(s) do {s->ci.flags &= ~SESSION_STARTED;} while (0)
+
+#define session_is_processed(s) (s->ci.flags & SESSION_PROCESSED)
+#define start_process_session(s) do {s->ci.flags |= SESSION_PROCESSED;} while(0)
+#define stop_process_session(s) do {s->ci.flags &= ~SESSION_PROCESSED;} while (0)
+
+#define session_binded(s) (s->ci.flags & SESSION_BINDED)
+#define bind_session(s) do {s->ci.flags |= SESSION_BINDED;} while(0)
+#define unbind_session(s) do {s->ci.flags &= ~SESSION_BINDED;} while (0)
+#define sci_binded(ci) (ci->flags & SESSION_BINDED)
+
+#define session_broken(s) (s->ci.flags & SESSION_BROKEN)
+#define broke_session(s) do {s->ci.flags |= SESSION_BROKEN;} while(0)
+#define unbroke_session(s) do {s->ci.flags &= ~SESSION_BROKEN;} while (0)
+
+#endif
+
+struct crypto_device_stat
+{
+ __u64 scompleted;
+ __u64 sfinished;
+ __u64 sstarted;
+ __u64 kmem_failed;
+};
+
+#ifdef __KERNEL__
+
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/device.h>
+#include <linux/workqueue.h>
+
+#include <asm/scatterlist.h>
+
+#define DEBUG
+#ifdef DEBUG
+#define dprintk(f, a...) printk(KERN_EMERG f, ##a)
+#else
+#define dprintk(f, a...)
+#endif
+
+#define CRYPTO_MAX_PRIV_SIZE 1024
+
+#define DEVICE_BROKEN (1<<0)
+
+#define device_broken(dev) (dev->flags & DEVICE_BROKEN)
+#define broke_device(dev) do {dev->flags |= DEVICE_BROKEN;} while(0)
+#define repair_device(dev) do {dev->flags &= ~DEVICE_BROKEN;} while(0)
+
+struct crypto_capability
+{
+ u16 operation;
+ u16 type;
+ u16 mode;
+ u16 qlen;
+};
+
+struct crypto_session_initializer
+{
+ u16 operation;
+ u16 type;
+ u16 mode;
+ u16 priority;
+
+ u64 id;
+ u64 dev_id;
+
+ u32 flags;
+
+ u32 bdev;
+
+ crypto_callback_t callback;
+};
+
+struct crypto_data
+{
+ int sg_src_num;
+ struct scatterlist sg_src;
+ struct scatterlist sg_dst;
+ struct scatterlist sg_key;
+ struct scatterlist sg_iv;
+
+ void *priv;
+ unsigned int priv_size;
+};
+
+struct crypto_device
+{
+ char name[SCACHE_NAMELEN];
+
+ spinlock_t session_lock;
+ struct list_head session_list;
+
+ u64 sid;
+ spinlock_t lock;
+
+ atomic_t refcnt;
+
+ u32 flags;
+
+ u32 id;
+
+ struct list_head cdev_entry;
+
+ void (*data_ready)(struct crypto_device *);
+
+ struct device_driver *driver;
+ struct device device;
+ struct class_device class_device;
+ struct completion dev_released;
+
+ spinlock_t stat_lock;
+ struct crypto_device_stat stat;
+
+ struct crypto_capability *cap;
+ int cap_number;
+};
+
+struct crypto_route_head
+{
+ struct crypto_route *next;
+ struct crypto_route *prev;
+
+ __u32 qlen;
+ spinlock_t lock;
+};
+
+struct crypto_route
+{
+ struct crypto_route *next;
+ struct crypto_route *prev;
+
+ struct crypto_route_head *list;
+ struct crypto_device *dev;
+
+ struct crypto_session_initializer ci;
+};
+
+
+struct crypto_session
+{
+ struct list_head dev_queue_entry;
+ struct list_head main_queue_entry;
+
+ struct crypto_session_initializer ci;
+
+ struct crypto_data data;
+
+ spinlock_t lock;
+
+ struct work_struct work;
+
+ struct crypto_route_head route_list;
+};
+
+struct crypto_session *crypto_session_alloc(struct crypto_session_initializer *, struct crypto_data *);
+inline void crypto_session_dequeue_main(struct crypto_session *);
+inline void __crypto_session_dequeue_main(struct crypto_session *);
+void crypto_session_dequeue_route(struct crypto_session *);
+
+inline void crypto_device_get(struct crypto_device *);
+inline void crypto_device_put(struct crypto_device *);
+inline struct crypto_device *crypto_device_get_name(char *);
+
+int __crypto_device_add(struct crypto_device *);
+int crypto_device_add(struct crypto_device *);
+void __crypto_device_remove(struct crypto_device *);
+void crypto_device_remove(struct crypto_device *);
+int match_initializer(struct crypto_device *, struct crypto_session_initializer *);
+
+inline void crypto_session_insert_main(struct crypto_device *dev, struct crypto_session *s);
+inline void crypto_session_insert(struct crypto_device *dev, struct crypto_session *s);
+
+#endif /* __KERNEL__ */
+#endif /* __ACRYPTO_H */
diff -Nru /tmp/empty/crypto_conn.c linux-2.6/drivers/acrypto/crypto_conn.c
--- /tmp/empty/crypto_conn.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_conn.c 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,194 @@
+/*
+ * crypto_conn.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/vmalloc.h>
+
+#include "acrypto.h"
+#include "crypto_lb.h"
+
+#include "../connector/connector.h"
+
+#include "crypto_conn.h"
+
+static struct cb_id crypto_conn_id = { 0xdead, 0x0000 };
+static char crypto_conn_name[] = "crconn";
+
+static void crypto_conn_callback(void *data)
+{
+ struct cn_msg *msg, *reply;
+ struct crypto_conn_data *d, *cmd;
+ struct crypto_device *dev;
+ u32 sessions;
+
+ msg = (struct cn_msg *)data;
+ d = (struct crypto_conn_data *)msg->data;
+
+ if (msg->len < sizeof(*d))
+ {
+ dprintk(KERN_ERR "Wrong message to crypto connector: msg->len=%u < %u.\n",
+ msg->len, sizeof(*d));
+ return;
+ }
+
+ if (msg->len != sizeof(*d) + d->len)
+ {
+ dprintk(KERN_ERR "Wrong message to crypto connector: msg->len=%u != %u.\n",
+ msg->len, sizeof(*d) + d->len);
+ return;
+ }
+
+ dev = crypto_device_get_name(d->name);
+ if (!dev)
+ {
+ dprintk(KERN_INFO "Crypto device %s was not found.\n", d->name);
+ return;
+ }
+
+ switch (d->cmd)
+ {
+ case CRYPTO_CONN_READ_SESSIONS:
+ reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + sizeof(sessions), GFP_ATOMIC);
+ if (reply) {
+ memcpy(reply, msg, sizeof(*reply));
+ reply->len = sizeof(*cmd) + sizeof(sessions);
+
+ /*
+ * See protocol description in connector.c
+ */
+ reply->ack++;
+
+ cmd = (struct crypto_conn_data *)(reply + 1);
+ memcpy(cmd, d, sizeof(*cmd));
+ cmd->len = sizeof(sessions);
+
+ sessions = atomic_read(&dev->refcnt);
+
+ memcpy(cmd+1, &sessions, sizeof(sessions));
+
+ cn_netlink_send(reply, 0);
+
+ kfree(reply);
+ } else
+ dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
+ sizeof(*msg) + sizeof(*cmd), d->cmd);
+ break;
+ case CRYPTO_CONN_DUMP_QUEUE:
+ reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + 1024*sizeof(struct crypto_session_initializer), GFP_ATOMIC);
+ if (reply) {
+ struct crypto_session *s;
+ struct crypto_session_initializer *ptr;
+
+ memcpy(reply, msg, sizeof(*reply));
+
+ /*
+ * See protocol description in connector.c
+ */
+ reply->ack++;
+
+ cmd = (struct crypto_conn_data *)(reply + 1);
+ memcpy(cmd, d, sizeof(*cmd));
+
+ ptr = (struct crypto_session_initializer *)(cmd+1);
+
+ sessions = 0;
+ spin_lock_bh(&dev->session_lock);
+ list_for_each_entry(s, &dev->session_list, dev_queue_entry)
+ {
+ memcpy(ptr, &s->ci, sizeof(*ptr));
+ sessions++;
+ ptr++;
+
+ if (sessions >= 1024)
+ break;
+ }
+ spin_unlock_bh(&dev->session_lock);
+
+ cmd->len = sizeof(*ptr)*sessions;
+ reply->len = sizeof(*cmd) + cmd->len;
+
+ cn_netlink_send(reply, 0);
+
+ kfree(reply);
+ } else
+ dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
+ sizeof(*msg) + sizeof(*cmd), d->cmd);
+ break;
+ case CRYPTO_GET_STAT:
+ reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + sizeof(struct crypto_device_stat), GFP_ATOMIC);
+ if (reply) {
+ struct crypto_device_stat *ptr;
+
+ memcpy(reply, msg, sizeof(*reply));
+ reply->len = sizeof(*cmd) + sizeof(*ptr);
+
+ /*
+ * See protocol description in connector.c
+ */
+ reply->ack++;
+
+ cmd = (struct crypto_conn_data *)(reply + 1);
+ memcpy(cmd, d, sizeof(*cmd));
+ cmd->len = sizeof(*ptr);
+
+ ptr = (struct crypto_device_stat *)(cmd+1);
+ memcpy(ptr, &dev->stat, sizeof(*ptr));
+
+ cn_netlink_send(reply, 0);
+
+ kfree(reply);
+ } else
+ dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
+ sizeof(*msg) + sizeof(*cmd), d->cmd);
+ break;
+ default:
+ dprintk(KERN_ERR "Wrong operation 0x%04x for crypto connector.\n",
+ d->cmd);
+ return;
+ }
+
+ crypto_device_put(dev);
+}
+
+int crypto_conn_init(void)
+{
+ int err;
+
+ err = cn_add_callback(&crypto_conn_id, crypto_conn_name, crypto_conn_callback);
+ if (err)
+ return err;
+
+ dprintk(KERN_INFO "Crypto connector callback is registered.\n");
+
+ return 0;
+}
+
+void crypto_conn_fini(void)
+{
+ cn_del_callback(&crypto_conn_id);
+ dprintk(KERN_INFO "Crypto connector callback is unregistered.\n");
+}
diff -Nru /tmp/empty/crypto_conn.h linux-2.6/drivers/acrypto/crypto_conn.h
--- /tmp/empty/crypto_conn.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_conn.h 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,45 @@
+/*
+ * crypto_conn.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CRYPTO_CONN_H
+#define __CRYPTO_CONN_H
+
+#include "acrypto.h"
+
+#define CRYPTO_CONN_READ_SESSIONS 0
+#define CRYPTO_CONN_DUMP_QUEUE 1
+#define CRYPTO_GET_STAT 2
+
+struct crypto_conn_data
+{
+ char name[SCACHE_NAMELEN];
+ __u16 cmd;
+ __u16 len;
+ __u8 data[0];
+};
+
+#ifdef __KERNEL__
+
+int crypto_conn_init(void);
+void crypto_conn_fini(void);
+
+#endif /* __KERNEL__ */
+#endif /* __CRYPTO_CONN_H */
diff -Nru /tmp/empty/crypto_def.h linux-2.6/drivers/acrypto/crypto_def.h
--- /tmp/empty/crypto_def.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_def.h 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,38 @@
+/*
+ * crypto_def.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CRYPTO_DEF_H
+#define __CRYPTO_DEF_H
+
+#define CRYPTO_OP_DECRYPT 0
+#define CRYPTO_OP_ENCRYPT 1
+#define CRYPTO_OP_HMAC 2
+
+#define CRYPTO_MODE_ECB 0
+#define CRYPTO_MODE_CBC 1
+#define CRYPTO_MODE_CFB 2
+#define CRYPTO_MODE_OFB 3
+
+#define CRYPTO_TYPE_AES_128 0
+#define CRYPTO_TYPE_AES_192 1
+#define CRYPTO_TYPE_AES_256 2
+
+#endif /* __CRYPTO_DEF_H */
diff -Nru /tmp/empty/crypto_dev.c linux-2.6/drivers/acrypto/crypto_dev.c
--- /tmp/empty/crypto_dev.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_dev.c 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,383 @@
+/*
+ * crypto_dev.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/device.h>
+
+#include "acrypto.h"
+
+static LIST_HEAD(cdev_list);
+static spinlock_t cdev_lock = SPIN_LOCK_UNLOCKED;
+static u32 cdev_ids;
+
+struct list_head *crypto_device_list = &cdev_list;
+spinlock_t *crypto_device_lock = &cdev_lock;
+
+static int crypto_match(struct device *dev, struct device_driver *drv)
+{
+ return 1;
+}
+
+static int crypto_probe(struct device *dev)
+{
+ return -ENODEV;
+}
+
+static int crypto_remove(struct device *dev)
+{
+ return 0;
+}
+
+static void crypto_release(struct device *dev)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, device);
+
+ complete(&d->dev_released);
+}
+
+static void crypto_class_release(struct class *class)
+{
+}
+
+static void crypto_class_release_device(struct class_device *class_dev)
+{
+}
+
+struct class crypto_class =
+{
+ .name = "acrypto",
+ .class_release = crypto_class_release,
+ .release = crypto_class_release_device
+};
+
+struct bus_type crypto_bus_type =
+{
+ .name = "acrypto",
+ .match = crypto_match
+};
+
+struct device_driver crypto_driver = {
+ .name = "crypto_driver",
+ .bus = &crypto_bus_type,
+ .probe = crypto_probe,
+ .remove = crypto_remove,
+};
+
+struct device crypto_dev = {
+ .parent = NULL,
+ .bus = &crypto_bus_type,
+ .bus_id = "Asynchronous crypto",
+ .driver = &crypto_driver,
+ .release = &crypto_release
+};
+
+static ssize_t sessions_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%d\n", atomic_read(&d->refcnt));
+}
+static ssize_t name_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%s\n", d->name);
+}
+static ssize_t devices_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d;
+ int off = 0;
+
+ spin_lock_bh(&cdev_lock);
+ list_for_each_entry(d, &cdev_list, cdev_entry)
+ {
+ off += sprintf(buf+off, "%s ", d->name);
+ }
+ spin_unlock_bh(&cdev_lock);
+
+ if (!off)
+ off = sprintf(buf, "No devices registered yet.");
+
+ off += sprintf(buf+off, "\n");
+
+ return off;
+}
+
+static ssize_t kmem_failed_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%llu\n", d->stat.kmem_failed);
+}
+static ssize_t sstarted_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%llu\n", d->stat.sstarted);
+}
+static ssize_t sfinished_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%llu\n", d->stat.sfinished);
+}
+static ssize_t scompleted_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%llu\n", d->stat.scompleted);
+}
+
+static CLASS_DEVICE_ATTR(sessions, 0444, sessions_show, NULL);
+static CLASS_DEVICE_ATTR(name, 0444, name_show, NULL);
+CLASS_DEVICE_ATTR(devices, 0444, devices_show, NULL);
+static CLASS_DEVICE_ATTR(scompleted, 0444, scompleted_show, NULL);
+static CLASS_DEVICE_ATTR(sstarted, 0444, sstarted_show, NULL);
+static CLASS_DEVICE_ATTR(sfinished, 0444, sfinished_show, NULL);
+static CLASS_DEVICE_ATTR(kmem_failed, 0444, kmem_failed_show, NULL);
+
+static inline int compare_device(struct crypto_device *d1, struct crypto_device *d2)
+{
+ if (!strncmp(d1->name, d2->name, sizeof(d1->name)))
+ return 1;
+
+ return 0;
+}
+
+static void create_device_attributes(struct crypto_device *dev)
+{
+ class_device_create_file(&dev->class_device, &class_device_attr_sessions);
+ class_device_create_file(&dev->class_device, &class_device_attr_name);
+ class_device_create_file(&dev->class_device, &class_device_attr_scompleted);
+ class_device_create_file(&dev->class_device, &class_device_attr_sstarted);
+ class_device_create_file(&dev->class_device, &class_device_attr_sfinished);
+ class_device_create_file(&dev->class_device, &class_device_attr_kmem_failed);
+}
+
+static void remove_device_attributes(struct crypto_device *dev)
+{
+ class_device_remove_file(&dev->class_device, &class_device_attr_sessions);
+ class_device_remove_file(&dev->class_device, &class_device_attr_name);
+ class_device_remove_file(&dev->class_device, &class_device_attr_scompleted);
+ class_device_remove_file(&dev->class_device, &class_device_attr_sstarted);
+ class_device_remove_file(&dev->class_device, &class_device_attr_sfinished);
+ class_device_remove_file(&dev->class_device, &class_device_attr_kmem_failed);
+}
+
+static int __match_initializer(struct crypto_capability *cap, struct crypto_session_initializer *ci)
+{
+ if ( cap->operation == ci->operation &&
+ cap->type == ci->type &&
+ cap->mode == (ci->mode & 0x1fff))
+ return 1;
+
+ return 0;
+}
+
+int match_initializer(struct crypto_device *dev, struct crypto_session_initializer *ci)
+{
+ int i;
+
+ for (i=0; i<dev->cap_number; ++i)
+ {
+ struct crypto_capability *cap = &dev->cap[i];
+
+ if (__match_initializer(cap, ci))
+ {
+ if (cap->qlen >= atomic_read(&dev->refcnt) + 1)
+ return 1;
+ }
+ }
+
+ return 0;
+}
+
+
+inline void crypto_device_get(struct crypto_device *dev)
+{
+ atomic_inc(&dev->refcnt);
+}
+
+inline struct crypto_device *crypto_device_get_name(char *name)
+{
+ struct crypto_device *dev;
+ int found = 0;
+
+ spin_lock_bh(&cdev_lock);
+ list_for_each_entry(dev, &cdev_list, cdev_entry)
+ {
+ if (!strcmp(dev->name, name))
+ {
+ found = 1;
+ crypto_device_get(dev);
+ break;
+ }
+ }
+ spin_unlock_bh(&cdev_lock);
+
+ if (!found)
+ return NULL;
+
+ return dev;
+}
+
+inline void crypto_device_put(struct crypto_device *dev)
+{
+ atomic_dec(&dev->refcnt);
+}
+
+
+int __crypto_device_add(struct crypto_device *dev)
+{
+ int err;
+
+ memset(&dev->stat, 0, sizeof(dev->stat));
+ spin_lock_init(&dev->stat_lock);
+ spin_lock_init(&dev->lock);
+ spin_lock_init(&dev->session_lock);
+ INIT_LIST_HEAD(&dev->session_list);
+ atomic_set(&dev->refcnt, 0);
+ dev->sid = 0;
+ dev->flags = 0;
+ init_completion(&dev->dev_released);
+ memcpy(&dev->device, &crypto_dev, sizeof(struct device));
+ dev->driver = &crypto_driver;
+
+ snprintf(dev->device.bus_id, sizeof(dev->device.bus_id), "%s", dev->name);
+ err = device_register(&dev->device);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto device %s: err=%d.\n",
+ dev->name, err);
+ return err;
+ }
+
+ snprintf(dev->class_device.class_id, sizeof(dev->class_device.class_id), "%s", dev->name);
+ dev->class_device.dev = &dev->device;
+ dev->class_device.class = &crypto_class;
+
+ err = class_device_register(&dev->class_device);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto class device %s: err=%d.\n",
+ dev->name, err);
+ device_unregister(&dev->device);
+ return err;
+ }
+
+ create_device_attributes(dev);
+
+ return 0;
+}
+
+void __crypto_device_remove(struct crypto_device *dev)
+{
+ remove_device_attributes(dev);
+ class_device_unregister(&dev->class_device);
+ device_unregister(&dev->device);
+}
+
+int crypto_device_add(struct crypto_device *dev)
+{
+ int err;
+
+ err = __crypto_device_add(dev);
+ if (err)
+ return err;
+
+ spin_lock_bh(&cdev_lock);
+ list_add(&dev->cdev_entry, &cdev_list);
+ dev->id = ++cdev_ids;
+ spin_unlock_bh(&cdev_lock);
+
+ dprintk(KERN_INFO "Crypto device %s was registered with ID=%x.\n", dev->name, dev->id);
+
+ return 0;
+}
+
+void crypto_device_remove(struct crypto_device *dev)
+{
+ struct crypto_device *__dev, *n;
+ unsigned long flags;
+
+ __crypto_device_remove(dev);
+
+ spin_lock_irqsave(&cdev_lock, flags);
+ list_for_each_entry_safe(__dev, n, &cdev_list, cdev_entry)
+ {
+ if (compare_device(__dev, dev))
+ {
+ list_del(&__dev->cdev_entry);
+ spin_unlock_irqrestore(&cdev_lock, flags);
+
+ /*
+ * In test cases or when crypto device driver is not written correctly,
+ * it's ->data_ready() method will not be callen anymore
+ * after device is removed from crypto device list.
+ *
+ * For such cases we either should provide ->flush() call
+ * or properly write ->data_ready() method.
+ */
+
+ while (atomic_read(&__dev->refcnt))
+ {
+
+ dprintk(KERN_INFO "Waiting for %s to become free: refcnt=%d.\n",
+ __dev->name, atomic_read(&dev->refcnt));
+
+ /*
+ * Hack zone: you need to write good ->data_ready()
+ * and crypto device driver itself.
+ *
+ * Driver shoud not buzz if it has pending sessions
+ * in it's queue and it was removed from global device list.
+ *
+ * Although I can workaround it here, for example by
+ * flushing the whole queue and drop all pending sessions.
+ */
+
+ __dev->data_ready(__dev);
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(HZ);
+ }
+
+ dprintk(KERN_ERR "Crypto device %s was unregistered.\n", dev->name);
+ return;
+ }
+ }
+ spin_unlock_irqrestore(&cdev_lock, flags);
+
+ dprintk(KERN_ERR "Crypto device %s was not registered.\n", dev->name);
+}
+
+EXPORT_SYMBOL(crypto_device_add);
+EXPORT_SYMBOL(crypto_device_remove);
+EXPORT_SYMBOL(crypto_device_get);
+EXPORT_SYMBOL(crypto_device_get_name);
+EXPORT_SYMBOL(crypto_device_put);
+EXPORT_SYMBOL(match_initializer);
diff -Nru /tmp/empty/crypto_lb.c linux-2.6/drivers/acrypto/crypto_lb.c
--- /tmp/empty/crypto_lb.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_lb.c 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,655 @@
+/*
+ * crypto_lb.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/workqueue.h>
+#include <linux/err.h>
+
+#include "acrypto.h"
+#include "crypto_lb.h"
+#include "crypto_stat.h"
+#include "crypto_route.h"
+
+static LIST_HEAD(crypto_lb_list);
+static spinlock_t crypto_lb_lock = SPIN_LOCK_UNLOCKED;
+static int lb_num = 0;
+static struct crypto_lb *current_lb, *default_lb;
+static struct completion thread_exited;
+static int need_exit;
+static struct workqueue_struct *crypto_lb_queue;
+static DECLARE_WAIT_QUEUE_HEAD(crypto_lb_wait_queue);
+
+extern struct list_head *crypto_device_list;
+extern spinlock_t *crypto_device_lock;
+
+extern int force_lb_remove;
+extern struct crypto_device main_crypto_device;
+
+static inline int lb_is_current(struct crypto_lb *l)
+{
+ return (l->crypto_device_list != NULL && l->crypto_device_lock != NULL);
+}
+
+static inline int lb_is_default(struct crypto_lb *l)
+{
+ return (l == default_lb);
+}
+
+static inline void __lb_set_current(struct crypto_lb *l)
+{
+ struct crypto_lb *c = current_lb;
+
+ if (c)
+ {
+ l->crypto_device_list = crypto_device_list;
+ l->crypto_device_lock = crypto_device_lock;
+ current_lb = l;
+ c->crypto_device_list = NULL;
+ c->crypto_device_lock = NULL;
+ }
+ else
+ {
+ l->crypto_device_list = crypto_device_list;
+ l->crypto_device_lock = crypto_device_lock;
+ current_lb = l;
+ }
+}
+
+static inline void lb_set_current(struct crypto_lb *l)
+{
+ struct crypto_lb *c = current_lb;
+
+ if (c)
+ {
+ spin_lock_bh(&c->lock);
+ __lb_set_current(l);
+ spin_unlock_bh(&c->lock);
+ }
+ else
+ __lb_set_current(l);
+}
+
+static inline void __lb_set_default(struct crypto_lb *l)
+{
+ default_lb = l;
+}
+
+static inline void lb_set_default(struct crypto_lb *l)
+{
+ struct crypto_lb *c = default_lb;
+
+ if (c)
+ {
+ spin_lock_bh(&c->lock);
+ __lb_set_default(l);
+ spin_unlock_bh(&c->lock);
+ }
+ else
+ __lb_set_default(l);
+}
+
+
+static int crypto_lb_match(struct device *dev, struct device_driver *drv)
+{
+ return 1;
+}
+
+static int crypto_lb_probe(struct device *dev)
+{
+ return -ENODEV;
+}
+
+static int crypto_lb_remove(struct device *dev)
+{
+ return 0;
+}
+
+static void crypto_lb_release(struct device *dev)
+{
+ struct crypto_lb *d = container_of(dev, struct crypto_lb, device);
+
+ complete(&d->dev_released);
+}
+
+static void crypto_lb_class_release(struct class *class)
+{
+}
+
+static void crypto_lb_class_release_device(struct class_device *class_dev)
+{
+}
+
+struct class crypto_lb_class =
+{
+ .name = "crypto_lb",
+ .class_release = crypto_lb_class_release,
+ .release = crypto_lb_class_release_device
+};
+
+struct bus_type crypto_lb_bus_type =
+{
+ .name = "crypto_lb",
+ .match = crypto_lb_match
+};
+
+struct device_driver crypto_lb_driver = {
+ .name = "crypto_lb_driver",
+ .bus = &crypto_lb_bus_type,
+ .probe = crypto_lb_probe,
+ .remove = crypto_lb_remove,
+};
+
+struct device crypto_lb_dev = {
+ .parent = NULL,
+ .bus = &crypto_lb_bus_type,
+ .bus_id = "crypto load balancer",
+ .driver = &crypto_lb_driver,
+ .release = &crypto_lb_release
+};
+
+static ssize_t name_show(struct class_device *dev, char *buf)
+{
+ struct crypto_lb *lb = container_of(dev, struct crypto_lb, class_device);
+
+ return sprintf(buf, "%s\n", lb->name);
+}
+
+static ssize_t current_show(struct class_device *dev, char *buf)
+{
+ struct crypto_lb *lb;
+ int off = 0;
+
+ spin_lock_bh(&crypto_lb_lock);
+ list_for_each_entry(lb, &crypto_lb_list, lb_entry)
+ {
+ if (lb_is_current(lb))
+ off += sprintf(buf+off, "[");
+ if (lb_is_default(lb))
+ off += sprintf(buf+off, "(");
+ off += sprintf(buf+off, "%s", lb->name);
+ if (lb_is_default(lb))
+ off += sprintf(buf+off, ")");
+ if (lb_is_current(lb))
+ off += sprintf(buf+off, "]");
+ }
+ spin_unlock_bh(&crypto_lb_lock);
+
+ if (!off)
+ off = sprintf(buf, "No load balancers regitered yet.");
+
+ off += sprintf(buf+off, "\n");
+
+ return off;
+}
+static ssize_t current_store(struct class_device *dev, const char *buf, size_t count)
+{
+ struct crypto_lb *lb;
+
+ spin_lock_bh(&crypto_lb_lock);
+ list_for_each_entry(lb, &crypto_lb_list, lb_entry)
+ {
+ if (count == strlen(lb->name) && !strcmp(buf, lb->name))
+ {
+ lb_set_current(lb);
+ lb_set_default(lb);
+
+ dprintk(KERN_INFO "Load balancer %s is set as current and default.\n",
+ lb->name);
+
+ break;
+ }
+ }
+ spin_unlock_bh(&crypto_lb_lock);
+
+ return count;
+}
+
+static CLASS_DEVICE_ATTR(name, 0444, name_show, NULL);
+CLASS_DEVICE_ATTR(lbs, 0644, current_show, current_store);
+
+static void create_device_attributes(struct crypto_lb *lb)
+{
+ class_device_create_file(&lb->class_device, &class_device_attr_name);
+}
+
+static void remove_device_attributes(struct crypto_lb *lb)
+{
+ class_device_remove_file(&lb->class_device, &class_device_attr_name);
+}
+
+static inline int compare_lb(struct crypto_lb *l1, struct crypto_lb *l2)
+{
+ if (!strncmp(l1->name, l2->name, sizeof(l1->name)))
+ return 1;
+
+ return 0;
+}
+
+inline void crypto_lb_rehash(void)
+{
+ if (!current_lb)
+ return;
+
+ spin_lock_bh(¤t_lb->lock);
+ current_lb->rehash(current_lb);
+ spin_unlock_bh(¤t_lb->lock);
+
+ wake_up_interruptible(&crypto_lb_wait_queue);
+}
+
+struct crypto_device *crypto_lb_find_device(struct crypto_session_initializer *ci)
+{
+ struct crypto_device *dev;
+
+ if (!current_lb)
+ return NULL;
+
+ if (sci_binded(ci))
+ {
+ int found = 0;
+
+ spin_lock_bh(crypto_device_lock);
+ list_for_each_entry(dev, crypto_device_list, cdev_entry)
+ {
+ if (dev->id == ci->bdev)
+ {
+ found = 1;
+ break;
+ }
+ }
+ spin_unlock_bh(crypto_device_lock);
+
+ return (found)?dev:NULL;
+ }
+
+ spin_lock_bh(¤t_lb->lock);
+
+ current_lb->rehash(current_lb);
+
+ spin_lock(crypto_device_lock);
+
+ dev = current_lb->find_device(current_lb, ci);
+ if (dev)
+ crypto_device_get(dev);
+
+ spin_unlock(crypto_device_lock);
+ spin_unlock_bh(¤t_lb->lock);
+
+ wake_up_interruptible(&crypto_lb_wait_queue);
+
+ return dev;
+}
+
+static int __crypto_lb_register(struct crypto_lb *lb)
+{
+ int err;
+
+ spin_lock_init(&lb->lock);
+
+ init_completion(&lb->dev_released);
+ memcpy(&lb->device, &crypto_lb_dev, sizeof(struct device));
+ lb->driver = &crypto_lb_driver;
+
+ snprintf(lb->device.bus_id, sizeof(lb->device.bus_id), "%s", lb->name);
+ err = device_register(&lb->device);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto load balancer device %s: err=%d.\n",
+ lb->name, err);
+ return err;
+ }
+
+ snprintf(lb->class_device.class_id, sizeof(lb->class_device.class_id), "%s", lb->name);
+ lb->class_device.dev = &lb->device;
+ lb->class_device.class = &crypto_lb_class;
+
+ err = class_device_register(&lb->class_device);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto load balancer class device %s: err=%d.\n",
+ lb->name, err);
+ device_unregister(&lb->device);
+ return err;
+ }
+
+ create_device_attributes(lb);
+ wake_up_interruptible(&crypto_lb_wait_queue);
+
+ return 0;
+
+}
+
+static void __crypto_lb_unregister(struct crypto_lb *lb)
+{
+ wake_up_interruptible(&crypto_lb_wait_queue);
+ remove_device_attributes(lb);
+ class_device_unregister(&lb->class_device);
+ device_unregister(&lb->device);
+}
+
+int crypto_lb_register(struct crypto_lb *lb, int set_current, int set_default)
+{
+ struct crypto_lb *__lb;
+ int err;
+
+ spin_lock_bh(&crypto_lb_lock);
+ list_for_each_entry(__lb, &crypto_lb_list, lb_entry)
+ {
+ if (unlikely(compare_lb(__lb, lb)))
+ {
+ spin_unlock_bh(&crypto_lb_lock);
+ dprintk(KERN_ERR "Crypto load balancer %s is already registered.\n", lb->name);
+ return -EINVAL;
+ }
+ }
+
+ list_add(&lb->lb_entry, &crypto_lb_list);
+ spin_unlock_bh(&crypto_lb_lock);
+
+ err = __crypto_lb_register(lb);
+ if (err)
+ {
+ spin_lock_bh(&crypto_lb_lock);
+ list_del(&lb->lb_entry);
+ spin_unlock_bh(&crypto_lb_lock);
+
+ return err;
+ }
+
+ if (!default_lb || set_default)
+ lb_set_default(lb);
+
+ if (!current_lb || set_current)
+ lb_set_current(lb);
+
+ dprintk(KERN_INFO "Crypto load balancer %s was registered and set to be [%s.%s].\n",
+ lb->name,
+ (lb_is_current(lb))?"current":"not current",
+ (lb_is_default(lb))?"default":"not default");
+
+ lb_num++;
+
+ return 0;
+}
+
+void crypto_lb_unregister(struct crypto_lb *lb)
+{
+ struct crypto_lb *__lb, *n;
+
+ if (lb_num == 1)
+ {
+ dprintk(KERN_INFO "You are removing crypto load balancer %s which is current and default.\n"
+ "There is no other crypto load balancers. "
+ "Removing %s delayed untill new load balancer is registered.\n",
+ lb->name, (force_lb_remove)?"is not":"is");
+ while (lb_num == 1 && !force_lb_remove)
+ {
+ set_current_state(TASK_INTERRUPTIBLE);
+ schedule_timeout(HZ);
+
+ if (signal_pending(current))
+ flush_signals(current);
+ }
+ }
+
+ __crypto_lb_unregister(lb);
+
+ spin_lock_bh(&crypto_lb_lock);
+ list_for_each_entry_safe(__lb, n, &crypto_lb_list, lb_entry)
+ {
+ if (compare_lb(__lb, lb))
+ {
+ lb_num--;
+ list_del(&__lb->lb_entry);
+
+ dprintk(KERN_ERR "Crypto load balancer %s was unregistered.\n", lb->name);
+ }
+ else if (lb_num)
+ {
+ if (lb_is_default(lb))
+ lb_set_default(__lb);
+ if (lb_is_current(lb))
+ lb_set_current(default_lb);
+ }
+ }
+ spin_unlock_bh(&crypto_lb_lock);
+}
+
+static void crypto_lb_queue_wrapper(void *data)
+{
+ struct crypto_device *dev = &main_crypto_device;
+ struct crypto_session *s = (struct crypto_session *)data;
+
+ dprintk(KERN_INFO "%s: Calling callback for session %llu [%llu] flags=%x, "
+ "op=%04u, type=%04x, mode=%04x, priority=%04x\n",
+ __func__,
+ s->ci.id, s->ci.dev_id, s->ci.flags,
+ s->ci.operation, s->ci.type, s->ci.mode, s->ci.priority);
+
+ /*
+ * Prevent races with crypto devices
+ * which may change flags of the sessions in theirs queues.
+ */
+ spin_lock(&s->lock);
+ crypto_stat_finish_inc(s);
+ finish_session(s);
+ unstart_session(s);
+ spin_unlock(&s->lock);
+
+ s->ci.callback(&s->ci, &s->data);
+
+ if (session_finished(s))
+ {
+ crypto_session_dequeue_route(s);
+ kfree(s);
+ }
+ else
+ {
+ /*
+ * Special case: crypto consumer marks session as "not finished"
+ * in it's callback - it means that crypto consumer wants
+ * this session to be processed further,
+ * for example crypto consumer can add new route and then
+ * mark session as "not finished".
+ */
+
+ uncomplete_session(s);
+ unstart_session(s);
+ crypto_session_insert_main(dev, s);
+ }
+
+ wake_up_interruptible(&crypto_lb_wait_queue);
+}
+
+static void crypto_lb_process_next_route(struct crypto_session *s)
+{
+ struct crypto_route *rt;
+ struct crypto_device *dev;
+
+ rt = crypto_route_dequeue(s);
+ if (rt)
+ {
+ dev = rt->dev;
+
+ spin_lock_bh(&dev->session_lock);
+ list_del(&s->dev_queue_entry);
+ spin_unlock_bh(&dev->session_lock);
+ crypto_route_free(rt);
+
+ dev = crypto_route_get_current_device(s);
+ if (dev)
+ {
+ dprintk(KERN_INFO "%s: processing new route to %s.\n",
+ __func__, dev->name);
+
+ memcpy(&s->ci, &rt->ci, sizeof(s->ci));
+ crypto_session_insert(dev, s);
+
+ /*
+ * Reference to this device was already hold when
+ * new routing was added.
+ */
+ crypto_device_put(dev);
+ }
+ }
+}
+
+int crypto_lb_thread(void *data)
+{
+ struct crypto_session *s, *n;
+ struct crypto_device *dev = (struct crypto_device *)data;
+
+ daemonize("%s", dev->name);
+ allow_signal(SIGTERM);
+
+ while (!need_exit)
+ {
+ spin_lock_bh(&dev->session_lock);
+ list_for_each_entry_safe(s, n, &dev->session_list, main_queue_entry)
+ {
+ printk("session %llu [%llu]: flags=%x, route_num=%d, %s,%s,%s,%s.\n",
+ s->ci.id, s->ci.dev_id, s->ci.flags,
+ crypto_route_queue_len(s),
+ (session_completed(s))?"completed": "not completed",
+ (session_finished(s))?"finished": "not finished",
+ (session_started(s))?"started": "not started",
+ (session_is_processed(s))?"is being processed": "is not being processed");
+
+ if (!spin_trylock(&s->lock))
+ continue;
+
+ if (session_is_processed(s))
+ goto unlock;
+ if (session_started(s))
+ goto unlock;
+
+ if (session_completed(s))
+ {
+ uncomplete_session(s);
+
+ if (crypto_route_queue_len(s) > 1)
+ {
+ crypto_lb_process_next_route(s);
+ }
+ else
+ {
+ start_session(s);
+ crypto_stat_start_inc(s);
+
+ __crypto_session_dequeue_main(s);
+ spin_unlock(&s->lock);
+
+ INIT_WORK(&s->work, &crypto_lb_queue_wrapper, s);
+ queue_work(crypto_lb_queue, &s->work);
+ continue;
+ }
+ }
+unlock:
+ spin_unlock(&s->lock);
+ }
+ spin_unlock_bh(&dev->session_lock);
+
+ interruptible_sleep_on_timeout(&crypto_lb_wait_queue, 1000);
+ }
+
+ flush_workqueue(crypto_lb_queue);
+ complete_and_exit(&thread_exited, 0);
+}
+
+int crypto_lb_init(void)
+{
+ int err;
+ long pid;
+
+ err = bus_register(&crypto_lb_bus_type);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto load balancer bus: err=%d.\n", err);
+ goto err_out_exit;
+ }
+
+ err = driver_register(&crypto_lb_driver);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto load balancer driver: err=%d.\n", err);
+ goto err_out_bus_unregister;
+ }
+
+ crypto_lb_class.class_dev_attrs = &class_device_attr_lbs;
+
+ err = class_register(&crypto_lb_class);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto load balancer class: err=%d.\n", err);
+ goto err_out_driver_unregister;
+ }
+
+ crypto_lb_queue = create_workqueue("clbq");
+ if (!crypto_lb_queue)
+ {
+ dprintk(KERN_ERR "Failed to create crypto load balaner work queue.\n");
+ goto err_out_class_unregister;
+ }
+
+ init_completion(&thread_exited);
+ pid = kernel_thread(crypto_lb_thread, &main_crypto_device, CLONE_FS | CLONE_FILES);
+ if (IS_ERR((void *)pid))
+ {
+ dprintk(KERN_ERR "Failed to create kernel load balancing thread.\n");
+ goto err_out_destroy_workqueue;
+ }
+
+
+ return 0;
+
+err_out_destroy_workqueue:
+ destroy_workqueue(crypto_lb_queue);
+err_out_class_unregister:
+ class_unregister(&crypto_lb_class);
+err_out_driver_unregister:
+ driver_unregister(&crypto_lb_driver);
+err_out_bus_unregister:
+ bus_unregister(&crypto_lb_bus_type);
+err_out_exit:
+ return err;
+}
+
+void crypto_lb_fini(void)
+{
+ need_exit = 1;
+ wait_for_completion(&thread_exited);
+ destroy_workqueue(crypto_lb_queue);
+ class_unregister(&crypto_lb_class);
+ driver_unregister(&crypto_lb_driver);
+ bus_unregister(&crypto_lb_bus_type);
+}
+
+EXPORT_SYMBOL(crypto_lb_register);
+EXPORT_SYMBOL(crypto_lb_unregister);
+EXPORT_SYMBOL(crypto_lb_rehash);
+EXPORT_SYMBOL(crypto_lb_find_device);
diff -Nru /tmp/empty/crypto_lb.h linux-2.6/drivers/acrypto/crypto_lb.h
--- /tmp/empty/crypto_lb.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_lb.h 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,59 @@
+/*
+ * crypto_lb.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CRYPTO_LB_H
+#define __CRYPTO_LB_H
+
+#include "acrypto.h"
+
+#define CRYPTO_LB_NAMELEN 32
+
+struct crypto_lb
+{
+ struct list_head lb_entry;
+
+ char name[CRYPTO_LB_NAMELEN];
+
+ void (*rehash)(struct crypto_lb *);
+ struct crypto_device * (*find_device)(struct crypto_lb *, struct crypto_session_initializer *);
+
+ spinlock_t lock;
+
+ spinlock_t *crypto_device_lock;
+ struct list_head *crypto_device_list;
+
+ struct device_driver *driver;
+ struct device device;
+ struct class_device class_device;
+ struct completion dev_released;
+
+};
+
+int crypto_lb_register(struct crypto_lb *lb, int set_current, int set_default);
+void crypto_lb_unregister(struct crypto_lb *lb);
+
+inline void crypto_lb_rehash(void);
+struct crypto_device *crypto_lb_find_device(struct crypto_session_initializer *ci);
+
+int crypto_lb_init(void);
+void crypto_lb_fini(void);
+
+#endif /* __CRYPTO_LB_H */
diff -Nru /tmp/empty/crypto_main.c linux-2.6/drivers/acrypto/crypto_main.c
--- /tmp/empty/crypto_main.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_main.c 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,323 @@
+/*
+ * crypto_main.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+
+#include "acrypto.h"
+#include "crypto_lb.h"
+#include "crypto_conn.h"
+#include "crypto_route.h"
+
+int force_lb_remove;
+module_param(force_lb_remove, int, 0);
+
+struct crypto_device main_crypto_device;
+
+extern struct bus_type crypto_bus_type;
+extern struct device_driver crypto_driver;
+extern struct class crypto_class;
+extern struct device crypto_dev;
+
+extern struct class_device_attribute class_device_attr_devices;
+extern struct class_device_attribute class_device_attr_lbs;
+
+static inline void dump_ci(struct crypto_session_initializer *ci)
+{
+ dprintk("%llu [%llu] op=%04u, type=%04x, mode=%04x, priority=%04x",
+ ci->id, ci->dev_id,
+ ci->operation, ci->type, ci->mode, ci->priority);
+}
+
+static void __crypto_session_insert(struct crypto_device *dev, struct crypto_session *s)
+{
+ struct crypto_session *__s;
+
+ if (unlikely(list_empty(&dev->session_list)))
+ {
+ list_add(&s->dev_queue_entry, &dev->session_list);
+ }
+ else
+ {
+ int inserted = 0;
+
+ list_for_each_entry(__s, &dev->session_list, dev_queue_entry)
+ {
+ if (__s->ci.priority < s->ci.priority)
+ {
+ list_add_tail(&s->dev_queue_entry, &__s->dev_queue_entry);
+ inserted = 1;
+ break;
+ }
+ }
+
+ if (!inserted)
+ list_add_tail(&s->dev_queue_entry, &dev->session_list);
+ }
+
+ dump_ci(&s->ci);
+ dprintk(" added to crypto device %s [%d].\n", dev->name, atomic_read(&dev->refcnt));
+}
+
+inline void crypto_session_insert_main(struct crypto_device *dev, struct crypto_session *s)
+{
+ struct crypto_session *__s;
+
+ spin_lock_bh(&dev->session_lock);
+ crypto_device_get(dev);
+ if (unlikely(list_empty(&dev->session_list)))
+ {
+ list_add(&s->main_queue_entry, &dev->session_list);
+ }
+ else
+ {
+ int inserted = 0;
+
+ list_for_each_entry(__s, &dev->session_list, main_queue_entry)
+ {
+ if (__s->ci.priority < s->ci.priority)
+ {
+ list_add_tail(&s->main_queue_entry, &__s->main_queue_entry);
+ inserted = 1;
+ break;
+ }
+ }
+
+ if (!inserted)
+ list_add_tail(&s->main_queue_entry, &dev->session_list);
+ }
+ spin_unlock_bh(&dev->session_lock);
+
+ dump_ci(&s->ci);
+ dprintk(" added to main crypto device %s [%d].\n", dev->name, atomic_read(&dev->refcnt));
+}
+
+inline void crypto_session_insert(struct crypto_device *dev, struct crypto_session *s)
+{
+ spin_lock_bh(&dev->session_lock);
+ __crypto_session_insert(dev, s);
+ spin_unlock_bh(&dev->session_lock);
+
+ if (dev->data_ready)
+ dev->data_ready(dev);
+}
+
+struct crypto_session *crypto_session_alloc(struct crypto_session_initializer *ci, struct crypto_data *d)
+{
+ struct crypto_device *dev = &main_crypto_device;
+ struct crypto_device *ldev;
+ struct crypto_session *s;
+ int err;
+
+ if (d->priv_size > CRYPTO_MAX_PRIV_SIZE)
+ {
+ dprintk("priv_size %u is too big, maximum allowed %u.\n", d->priv_size, CRYPTO_MAX_PRIV_SIZE);
+ return NULL;
+ }
+
+ ldev = crypto_lb_find_device(ci);
+ if (!ldev)
+ {
+ dprintk("Cannot find suitable device.\n");
+ return NULL;
+ }
+
+ s = kmalloc(sizeof(*s) + d->priv_size, GFP_ATOMIC);
+ if (!s)
+ {
+ ldev->stat.kmem_failed++;
+ goto err_out_device_put;
+ }
+
+ memset(s, 0xAB, sizeof(*s));
+
+ crypto_route_head_init(&s->route_list);
+
+ spin_lock_init(&s->lock);
+ memcpy(&s->ci, ci, sizeof(s->ci));
+ memcpy(&s->data, d, sizeof(s->data));
+ if (d->priv_size)
+ {
+ s->data.priv = s+1;
+ if (d->priv)
+ memcpy(s->data.priv, d->priv, d->priv_size);
+ }
+ else
+ s->data.priv = NULL;
+
+ s->ci.id = dev->sid++;
+ s->ci.dev_id = ldev->sid++;
+ s->ci.flags = 0;
+#if 1
+ err = crypto_route_add(ldev, s, ci);
+ if (err)
+ {
+ dprintk("Can not add route to device %s.\n", ldev->name);
+ goto err_out_session_free;
+ }
+
+ crypto_session_insert(ldev, s);
+ crypto_device_put(ldev);
+#endif
+ crypto_session_insert_main(dev, s);
+
+ return s;
+
+err_out_session_free:
+ crypto_device_put(ldev);
+err_out_device_put:
+ kfree(s);
+
+ return NULL;
+}
+
+void crypto_session_dequeue_route(struct crypto_session *s)
+{
+ struct crypto_route *rt;
+ struct crypto_device *dev;
+
+ BUG_ON(crypto_route_queue_len(s) > 1);
+
+ while ((rt = crypto_route_dequeue(s)))
+ {
+ dev = rt->dev;
+
+ dprintk(KERN_INFO "Removing route entry for device %s.\n", dev->name);
+
+ spin_lock_bh(&dev->session_lock);
+ list_del(&s->dev_queue_entry);
+ spin_unlock_bh(&dev->session_lock);
+
+ crypto_route_free(rt);
+ }
+}
+
+inline void __crypto_session_dequeue_main(struct crypto_session *s)
+{
+ struct crypto_device *dev = &main_crypto_device;
+
+ list_del(&s->main_queue_entry);
+ crypto_device_put(dev);
+}
+
+inline void crypto_session_dequeue_main(struct crypto_session *s)
+{
+ struct crypto_device *dev = &main_crypto_device;
+
+ spin_lock_bh(&dev->session_lock);
+ __crypto_session_dequeue_main(s);
+ spin_unlock_bh(&dev->session_lock);
+}
+
+int __devinit cmain_init(void)
+{
+ struct crypto_device *dev = &main_crypto_device;
+ int err;
+
+ snprintf(dev->name, sizeof(dev->name), "crypto_sessions");
+
+ err = bus_register(&crypto_bus_type);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto bus: err=%d.\n", err);
+ return err;
+ }
+
+ err = driver_register(&crypto_driver);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto driver: err=%d.\n", err);
+ goto err_out_bus_unregister;
+ }
+
+ err = class_register(&crypto_class);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto class: err=%d.\n", err);
+ goto err_out_driver_unregister;
+ }
+
+ err = crypto_lb_init();
+ if (err)
+ goto err_out_class_unregister;
+
+ err = crypto_conn_init();
+ if (err)
+ goto err_out_crypto_lb_fini;
+
+ err = __crypto_device_add(dev);
+ if (err)
+ goto err_out_crypto_conn_fini;
+
+ err = class_device_create_file(&dev->class_device, &class_device_attr_devices);
+ if (err)
+ dprintk("Failed to create \"devices\" attribute: err=%d.\n", err);
+ err = class_device_create_file(&dev->class_device, &class_device_attr_lbs);
+ if (err)
+ dprintk("Failed to create \"lbs\" attribute: err=%d.\n", err);
+
+
+ return 0;
+
+err_out_crypto_conn_fini:
+ crypto_conn_fini();
+err_out_crypto_lb_fini:
+ crypto_lb_fini();
+err_out_class_unregister:
+ class_unregister(&crypto_class);
+err_out_driver_unregister:
+ driver_unregister(&crypto_driver);
+err_out_bus_unregister:
+ bus_unregister(&crypto_bus_type);
+
+ return err;
+}
+
+void __devexit cmain_fini(void)
+{
+ struct crypto_device *dev = &main_crypto_device;
+
+ class_device_remove_file(&dev->class_device, &class_device_attr_devices);
+ class_device_remove_file(&dev->class_device, &class_device_attr_lbs);
+ __crypto_device_remove(dev);
+
+ crypto_conn_fini();
+ crypto_lb_fini();
+
+ class_unregister(&crypto_class);
+ driver_unregister(&crypto_driver);
+ bus_unregister(&crypto_bus_type);
+}
+
+module_init(cmain_init);
+module_exit(cmain_fini);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
+MODULE_DESCRIPTION("Asynchronous crypto layer.");
+
+EXPORT_SYMBOL(crypto_session_alloc);
diff -Nru /tmp/empty/crypto_route.h linux-2.6/drivers/acrypto/crypto_route.h
--- /tmp/empty/crypto_route.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_route.h 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,215 @@
+/*
+ * crypto_route.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CRYPTO_ROUTE_H
+#define __CRYPTO_ROUTE_H
+
+#include <linux/types.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include "acrypto.h"
+
+static inline struct crypto_route *crypto_route_alloc(struct crypto_device *dev, struct crypto_session_initializer *ci)
+{
+ struct crypto_route *rt;
+
+ crypto_device_get(dev);
+ if (!match_initializer(dev, ci))
+ {
+ crypto_device_put(dev);
+ return NULL;
+ }
+
+ rt = kmalloc(sizeof(*rt), GFP_ATOMIC);
+ if (!rt)
+ {
+ crypto_device_put(dev);
+ return NULL;
+ }
+
+ memset(rt, 0, sizeof(*rt));
+ memcpy(&rt->ci, ci, sizeof(*ci));
+
+ rt->dev = dev;
+
+ return rt;
+}
+
+static inline void crypto_route_free(struct crypto_route *rt)
+{
+ crypto_device_put(rt->dev);
+ rt->dev = NULL;
+ kfree(rt);
+}
+
+static inline void __crypto_route_del(struct crypto_route *rt, struct crypto_route_head *list)
+{
+ struct crypto_route *next, *prev;
+
+ list->qlen--;
+ next = rt->next;
+ prev = rt->prev;
+ rt->next = rt->prev = NULL;
+ rt->list = NULL;
+ next->prev = prev;
+ prev->next = next;
+}
+
+static inline void crypto_route_del(struct crypto_route *rt)
+{
+ struct crypto_route_head *list = rt->list;
+
+ if (list)
+ {
+ spin_lock_bh(&list->lock);
+ if (list == rt->list)
+ __crypto_route_del(rt, rt->list);
+ spin_unlock_bh(&list->lock);
+
+ crypto_route_free(rt);
+ }
+}
+
+static inline struct crypto_route *__crypto_route_dequeue(struct crypto_route_head *list)
+{
+ struct crypto_route *next, *prev, *result;
+
+ prev = (struct crypto_route *) list;
+ next = prev->next;
+ result = NULL;
+ if (next != prev) {
+ result = next;
+ next = next->next;
+ list->qlen--;
+ next->prev = prev;
+ prev->next = next;
+ result->next = result->prev = NULL;
+ result->list = NULL;
+ }
+ return result;
+}
+
+static inline struct crypto_route *crypto_route_dequeue(struct crypto_session *s)
+{
+ struct crypto_route *rt;
+
+ spin_lock_bh(&s->route_list.lock);
+ rt = __crypto_route_dequeue(&s->route_list);
+ spin_unlock_bh(&s->route_list.lock);
+
+ return rt;
+}
+
+static inline void __crypto_route_queue(struct crypto_route *rt, struct crypto_route_head *list)
+{
+ struct crypto_route *prev, *next;
+
+ rt->list = list;
+ list->qlen++;
+ next = (struct crypto_route *)list;
+ prev = next->prev;
+ rt->next = next;
+ rt->prev = prev;
+ next->prev = prev->next = rt;
+}
+
+static inline void crypto_route_queue(struct crypto_route *rt, struct crypto_session *s)
+{
+ spin_lock_bh(&s->route_list.lock);
+ __crypto_route_queue(rt, &s->route_list);
+ spin_unlock_bh(&s->route_list.lock);
+}
+
+static inline int crypto_route_add(struct crypto_device *dev, struct crypto_session *s, struct crypto_session_initializer *ci)
+{
+ struct crypto_route *rt;
+
+ rt = crypto_route_alloc(dev, ci);
+ if (!rt)
+ return -ENOMEM;
+
+ crypto_route_queue(rt, s);
+
+ return 0;
+}
+
+static inline int crypto_route_queue_len(struct crypto_session *s)
+{
+ return s->route_list.qlen;
+}
+
+static inline void crypto_route_head_init(struct crypto_route_head *list)
+{
+ spin_lock_init(&list->lock);
+ list->prev = list->next = (struct crypto_route *)list;
+ list->qlen = 0;
+}
+
+static inline struct crypto_route *__crypto_route_current(struct crypto_route_head *list)
+{
+ struct crypto_route *next, *prev, *result;
+
+ prev = (struct crypto_route *) list;
+ next = prev->next;
+ result = NULL;
+ if (next != prev)
+ result = next;
+
+ return result;
+}
+
+static inline struct crypto_route *crypto_route_current(struct crypto_session *s)
+{
+ struct crypto_route_head *list;
+ struct crypto_route *rt = NULL;
+
+ list = &s->route_list;
+
+ if (list)
+ {
+ spin_lock_bh(&list->lock);
+ rt = __crypto_route_current(list);
+ spin_unlock_bh(&list->lock);
+ }
+
+ return rt;
+}
+
+static inline struct crypto_device *crypto_route_get_current_device(struct crypto_session *s)
+{
+ struct crypto_route *rt = NULL;
+ struct crypto_device *dev = NULL;
+ struct crypto_route_head *list = &s->route_list;
+
+ spin_lock_bh(&list->lock);
+ rt = __crypto_route_current(list);
+ if (rt)
+ {
+ dev = rt->dev;
+ crypto_device_get(dev);
+ }
+ spin_unlock_bh(&list->lock);
+
+ return dev;
+}
+
+#endif /* __CRYPTO_ROUTE_H */
diff -Nru /tmp/empty/crypto_stat.c linux-2.6/drivers/acrypto/crypto_stat.c
--- /tmp/empty/crypto_stat.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_stat.c 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,81 @@
+/*
+ * crypto_stat.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+
+#include "acrypto.h"
+#include "crypto_route.h"
+
+inline void crypto_stat_start_inc(struct crypto_session *s)
+{
+ struct crypto_device *dev;
+
+ dev = crypto_route_get_current_device(s);
+ if (dev)
+ {
+ spin_lock_bh(&dev->stat_lock);
+ dev->stat.sstarted++;
+ spin_unlock_bh(&dev->stat_lock);
+
+ crypto_device_put(dev);
+ }
+}
+
+inline void crypto_stat_finish_inc(struct crypto_session *s)
+{
+ struct crypto_device *dev;
+
+ dev = crypto_route_get_current_device(s);
+ if (dev)
+ {
+ spin_lock_bh(&dev->stat_lock);
+ dev->stat.sfinished++;
+ spin_unlock_bh(&dev->stat_lock);
+
+ crypto_device_put(dev);
+ }
+}
+
+inline void crypto_stat_complete_inc(struct crypto_session *s)
+{
+ struct crypto_device *dev;
+
+ dev = crypto_route_get_current_device(s);
+ if (dev)
+ {
+ spin_lock_bh(&dev->stat_lock);
+ dev->stat.scompleted++;
+ spin_unlock_bh(&dev->stat_lock);
+
+ crypto_device_put(dev);
+ }
+}
+
+EXPORT_SYMBOL(crypto_stat_start_inc);
+EXPORT_SYMBOL(crypto_stat_finish_inc);
+EXPORT_SYMBOL(crypto_stat_complete_inc);
diff -Nru /tmp/empty/crypto_stat.h linux-2.6/drivers/acrypto/crypto_stat.h
--- /tmp/empty/crypto_stat.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_stat.h 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,31 @@
+/*
+ * crypto_stat.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CRYPTO_STAT_H
+#define __CRYPTO_STAT_H
+
+#include "acrypto.h"
+
+inline void crypto_stat_start_inc(struct crypto_session *s);
+inline void crypto_stat_finish_inc(struct crypto_session *s);
+inline void crypto_stat_complete_inc(struct crypto_session *s);
+
+#endif /* __CRYPTO_STAT_H */
diff -Nru /tmp/empty/simple_lb.c linux-2.6/drivers/acrypto/simple_lb.c
--- /tmp/empty/simple_lb.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/simple_lb.c 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,90 @@
+/*
+ * simple_lb.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include "crypto_lb.h"
+
+static void simple_lb_rehash(struct crypto_lb *);
+static struct crypto_device *simple_lb_find_device(struct crypto_lb *, struct crypto_session_initializer *);
+
+struct crypto_lb simple_lb =
+{
+ .name = "simple_lb",
+ .rehash = simple_lb_rehash,
+ .find_device = simple_lb_find_device
+};
+
+static void simple_lb_rehash(struct crypto_lb *lb)
+{
+ BUG_ON(!lb->crypto_device_list);
+ BUG_ON(!lb->crypto_device_lock);
+}
+
+static struct crypto_device *simple_lb_find_device(struct crypto_lb *lb, struct crypto_session_initializer *ci)
+{
+ struct crypto_device *dev, *__dev;
+ int min = 0x7ffffff;
+
+ __dev = NULL;
+ list_for_each_entry(dev, lb->crypto_device_list, cdev_entry)
+ {
+ if (device_broken(dev))
+ continue;
+ if (!match_initializer(dev, ci))
+ continue;
+
+ if (atomic_read(&dev->refcnt) < min)
+ {
+ min = atomic_read(&dev->refcnt);
+ __dev = dev;
+ }
+ }
+
+ return __dev;
+}
+
+int __devinit simple_lb_init(void)
+{
+ dprintk(KERN_INFO "Registering simple crypto load balancer.\n");
+
+ return crypto_lb_register(&simple_lb, 1, 1);
+}
+
+void __devexit simple_lb_fini(void)
+{
+ dprintk(KERN_INFO "Unregistering simple crypto load balancer.\n");
+
+ crypto_lb_unregister(&simple_lb);
+}
+
+module_init(simple_lb_init);
+module_exit(simple_lb_fini);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
+MODULE_DESCRIPTION("Simple crypto load balancer.");
diff -Nru /tmp/empty/via-padlock/Makefile linux-2.6/drivers/acrypto/via-padlock/Makefile
--- /tmp/empty/via-padlock/Makefile 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/via-padlock/Makefile 2004-10-30 09:03:37.000000000 +0400
@@ -0,0 +1,2 @@
+obj-$(CONFIG_VIA_PADLOCK) += padlock.o
+padlock-objs := padlock-aes.o padlock-generic.o
diff -Nru /tmp/empty/via-padlock/padlock-aes.c linux-2.6/drivers/acrypto/via-padlock/padlock-aes.c
--- /tmp/empty/via-padlock/padlock-aes.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/via-padlock/padlock-aes.c 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,552 @@
+/*
+ * Cryptographic API.
+ *
+ * Support for VIA PadLock hardware crypto engine.
+ *
+ * Linux developers:
+ * Michal Ludvig <mludvig@suse.cz>
+ *
+ * Key expansion routine taken from crypto/aes.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * ---------------------------------------------------------------------------
+ * Copyright (c) 2002, Dr Brian Gladman <brg@gladman.me.uk>, Worcester, UK.
+ * All rights reserved.
+ *
+ * LICENSE TERMS
+ *
+ * The free distribution and use of this software in both source and binary
+ * form is allowed (with or without changes) provided that:
+ *
+ * 1. distributions of this source code include the above copyright
+ * notice, this list of conditions and the following disclaimer;
+ *
+ * 2. distributions in binary form include the above copyright
+ * notice, this list of conditions and the following disclaimer
+ * in the documentation and/or other associated materials;
+ *
+ * 3. the copyright holder's name is not used to endorse products
+ * built using this software without specific written permission.
+ *
+ * ALTERNATIVELY, provided that this notice is retained in full, this product
+ * may be distributed under the terms of the GNU General Public License (GPL),
+ * in which case the provisions of the GPL apply INSTEAD OF those given above.
+ *
+ * DISCLAIMER
+ *
+ * This software is provided 'as is' with no explicit or implied warranties
+ * in respect of its properties, including, but not limited to, correctness
+ * and/or fitness for purpose.
+ * ---------------------------------------------------------------------------
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <linux/crypto.h>
+#include <asm/byteorder.h>
+#include <linux/mm.h>
+
+#include <asm/scatterlist.h>
+
+#include "padlock.h"
+
+#include "../crypto_def.h"
+#include "../acrypto.h"
+#include "../crypto_stat.h"
+
+static inline int aes_hw_extkey_available (u8 key_len);
+
+static inline
+u32 generic_rotr32 (const u32 x, const unsigned bits)
+{
+ const unsigned n = bits % 32;
+ return (x >> n) | (x << (32 - n));
+}
+
+static inline
+u32 generic_rotl32 (const u32 x, const unsigned bits)
+{
+ const unsigned n = bits % 32;
+ return (x << n) | (x >> (32 - n));
+}
+
+#define rotl generic_rotl32
+#define rotr generic_rotr32
+
+/*
+ * #define byte(x, nr) ((unsigned char)((x) >> (nr*8)))
+ */
+inline static u8
+byte(const u32 x, const unsigned n)
+{
+ return x >> (n << 3);
+}
+
+#define u32_in(x) le32_to_cpu(*(const u32 *)(x))
+#define u32_out(to, from) (*(u32 *)(to) = cpu_to_le32(from))
+
+static u8 pow_tab[256];
+static u8 log_tab[256];
+static u8 sbx_tab[256];
+static u8 isb_tab[256];
+static u32 rco_tab[10];
+static u32 ft_tab[4][256];
+static u32 it_tab[4][256];
+
+static u32 fl_tab[4][256];
+static u32 il_tab[4][256];
+
+static inline u8
+f_mult (u8 a, u8 b)
+{
+ u8 aa = log_tab[a], cc = aa + log_tab[b];
+
+ return pow_tab[cc + (cc < aa ? 1 : 0)];
+}
+
+#define ff_mult(a,b) (a && b ? f_mult(a, b) : 0)
+
+#define f_rn(bo, bi, n, k) \
+ bo[n] = ft_tab[0][byte(bi[n],0)] ^ \
+ ft_tab[1][byte(bi[(n + 1) & 3],1)] ^ \
+ ft_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
+ ft_tab[3][byte(bi[(n + 3) & 3],3)] ^ *(k + n)
+
+#define i_rn(bo, bi, n, k) \
+ bo[n] = it_tab[0][byte(bi[n],0)] ^ \
+ it_tab[1][byte(bi[(n + 3) & 3],1)] ^ \
+ it_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
+ it_tab[3][byte(bi[(n + 1) & 3],3)] ^ *(k + n)
+
+#define ls_box(x) \
+ ( fl_tab[0][byte(x, 0)] ^ \
+ fl_tab[1][byte(x, 1)] ^ \
+ fl_tab[2][byte(x, 2)] ^ \
+ fl_tab[3][byte(x, 3)] )
+
+#define f_rl(bo, bi, n, k) \
+ bo[n] = fl_tab[0][byte(bi[n],0)] ^ \
+ fl_tab[1][byte(bi[(n + 1) & 3],1)] ^ \
+ fl_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
+ fl_tab[3][byte(bi[(n + 3) & 3],3)] ^ *(k + n)
+
+#define i_rl(bo, bi, n, k) \
+ bo[n] = il_tab[0][byte(bi[n],0)] ^ \
+ il_tab[1][byte(bi[(n + 3) & 3],1)] ^ \
+ il_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
+ il_tab[3][byte(bi[(n + 1) & 3],3)] ^ *(k + n)
+
+static void
+gen_tabs (void)
+{
+ u32 i, t;
+ u8 p, q;
+
+ /* log and power tables for GF(2**8) finite field with
+ 0x011b as modular polynomial - the simplest prmitive
+ root is 0x03, used here to generate the tables */
+
+ for (i = 0, p = 1; i < 256; ++i) {
+ pow_tab[i] = (u8) p;
+ log_tab[p] = (u8) i;
+
+ p ^= (p << 1) ^ (p & 0x80 ? 0x01b : 0);
+ }
+
+ log_tab[1] = 0;
+
+ for (i = 0, p = 1; i < 10; ++i) {
+ rco_tab[i] = p;
+
+ p = (p << 1) ^ (p & 0x80 ? 0x01b : 0);
+ }
+
+ for (i = 0; i < 256; ++i) {
+ p = (i ? pow_tab[255 - log_tab[i]] : 0);
+ q = ((p >> 7) | (p << 1)) ^ ((p >> 6) | (p << 2));
+ p ^= 0x63 ^ q ^ ((q >> 6) | (q << 2));
+ sbx_tab[i] = p;
+ isb_tab[p] = (u8) i;
+ }
+
+ for (i = 0; i < 256; ++i) {
+ p = sbx_tab[i];
+
+ t = p;
+ fl_tab[0][i] = t;
+ fl_tab[1][i] = rotl (t, 8);
+ fl_tab[2][i] = rotl (t, 16);
+ fl_tab[3][i] = rotl (t, 24);
+
+ t = ((u32) ff_mult (2, p)) |
+ ((u32) p << 8) |
+ ((u32) p << 16) | ((u32) ff_mult (3, p) << 24);
+
+ ft_tab[0][i] = t;
+ ft_tab[1][i] = rotl (t, 8);
+ ft_tab[2][i] = rotl (t, 16);
+ ft_tab[3][i] = rotl (t, 24);
+
+ p = isb_tab[i];
+
+ t = p;
+ il_tab[0][i] = t;
+ il_tab[1][i] = rotl (t, 8);
+ il_tab[2][i] = rotl (t, 16);
+ il_tab[3][i] = rotl (t, 24);
+
+ t = ((u32) ff_mult (14, p)) |
+ ((u32) ff_mult (9, p) << 8) |
+ ((u32) ff_mult (13, p) << 16) |
+ ((u32) ff_mult (11, p) << 24);
+
+ it_tab[0][i] = t;
+ it_tab[1][i] = rotl (t, 8);
+ it_tab[2][i] = rotl (t, 16);
+ it_tab[3][i] = rotl (t, 24);
+ }
+}
+
+#define star_x(x) (((x) & 0x7f7f7f7f) << 1) ^ ((((x) & 0x80808080) >> 7) * 0x1b)
+
+#define imix_col(y,x) \
+ u = star_x(x); \
+ v = star_x(u); \
+ w = star_x(v); \
+ t = w ^ (x); \
+ (y) = u ^ v ^ w; \
+ (y) ^= rotr(u ^ t, 8) ^ \
+ rotr(v ^ t, 16) ^ \
+ rotr(t,24)
+
+/* initialise the key schedule from the user supplied key */
+
+#define loop4(i) \
+{ t = rotr(t, 8); t = ls_box(t) ^ rco_tab[i]; \
+ t ^= E_KEY[4 * i]; E_KEY[4 * i + 4] = t; \
+ t ^= E_KEY[4 * i + 1]; E_KEY[4 * i + 5] = t; \
+ t ^= E_KEY[4 * i + 2]; E_KEY[4 * i + 6] = t; \
+ t ^= E_KEY[4 * i + 3]; E_KEY[4 * i + 7] = t; \
+}
+
+#define loop6(i) \
+{ t = rotr(t, 8); t = ls_box(t) ^ rco_tab[i]; \
+ t ^= E_KEY[6 * i]; E_KEY[6 * i + 6] = t; \
+ t ^= E_KEY[6 * i + 1]; E_KEY[6 * i + 7] = t; \
+ t ^= E_KEY[6 * i + 2]; E_KEY[6 * i + 8] = t; \
+ t ^= E_KEY[6 * i + 3]; E_KEY[6 * i + 9] = t; \
+ t ^= E_KEY[6 * i + 4]; E_KEY[6 * i + 10] = t; \
+ t ^= E_KEY[6 * i + 5]; E_KEY[6 * i + 11] = t; \
+}
+
+#define loop8(i) \
+{ t = rotr(t, 8); ; t = ls_box(t) ^ rco_tab[i]; \
+ t ^= E_KEY[8 * i]; E_KEY[8 * i + 8] = t; \
+ t ^= E_KEY[8 * i + 1]; E_KEY[8 * i + 9] = t; \
+ t ^= E_KEY[8 * i + 2]; E_KEY[8 * i + 10] = t; \
+ t ^= E_KEY[8 * i + 3]; E_KEY[8 * i + 11] = t; \
+ t = E_KEY[8 * i + 4] ^ ls_box(t); \
+ E_KEY[8 * i + 12] = t; \
+ t ^= E_KEY[8 * i + 5]; E_KEY[8 * i + 13] = t; \
+ t ^= E_KEY[8 * i + 6]; E_KEY[8 * i + 14] = t; \
+ t ^= E_KEY[8 * i + 7]; E_KEY[8 * i + 15] = t; \
+}
+
+static int
+aes_set_key(void *ctx_arg, const u8 *in_key, unsigned int key_len)
+{
+ struct aes_ctx *ctx = ctx_arg;
+ u32 i, t, u, v, w;
+ u32 P[AES_EXTENDED_KEY_SIZE];
+ u32 rounds;
+
+ if (key_len != 16 && key_len != 24 && key_len != 32) {
+ return -EINVAL;
+ }
+
+ ctx->key_length = key_len;
+
+ ctx->E = ctx->e_data;
+ ctx->D = ctx->d_data;
+
+ /* Ensure 16-Bytes alignmentation of keys for VIA PadLock. */
+ if ((int)(ctx->e_data) & 0x0F)
+ ctx->E += 4 - (((int)(ctx->e_data) & 0x0F) / sizeof (ctx->e_data[0]));
+
+ if ((int)(ctx->d_data) & 0x0F)
+ ctx->D += 4 - (((int)(ctx->d_data) & 0x0F) / sizeof (ctx->d_data[0]));
+
+ E_KEY[0] = u32_in (in_key);
+ E_KEY[1] = u32_in (in_key + 4);
+ E_KEY[2] = u32_in (in_key + 8);
+ E_KEY[3] = u32_in (in_key + 12);
+
+ /* Don't generate extended keys if the hardware can do it. */
+ if (aes_hw_extkey_available(key_len))
+ return 0;
+
+ switch (key_len) {
+ case 16:
+ t = E_KEY[3];
+ for (i = 0; i < 10; ++i)
+ loop4 (i);
+ break;
+
+ case 24:
+ E_KEY[4] = u32_in (in_key + 16);
+ t = E_KEY[5] = u32_in (in_key + 20);
+ for (i = 0; i < 8; ++i)
+ loop6 (i);
+ break;
+
+ case 32:
+ E_KEY[4] = u32_in (in_key + 16);
+ E_KEY[5] = u32_in (in_key + 20);
+ E_KEY[6] = u32_in (in_key + 24);
+ t = E_KEY[7] = u32_in (in_key + 28);
+ for (i = 0; i < 7; ++i)
+ loop8 (i);
+ break;
+ }
+
+ D_KEY[0] = E_KEY[0];
+ D_KEY[1] = E_KEY[1];
+ D_KEY[2] = E_KEY[2];
+ D_KEY[3] = E_KEY[3];
+
+ for (i = 4; i < key_len + 24; ++i) {
+ imix_col (D_KEY[i], E_KEY[i]);
+ }
+
+ /* PadLock needs a different format of the decryption key. */
+ rounds = 10 + (key_len - 16) / 4;
+
+ for (i = 0; i < rounds; i++) {
+ P[((i + 1) * 4) + 0] = D_KEY[((rounds - i - 1) * 4) + 0];
+ P[((i + 1) * 4) + 1] = D_KEY[((rounds - i - 1) * 4) + 1];
+ P[((i + 1) * 4) + 2] = D_KEY[((rounds - i - 1) * 4) + 2];
+ P[((i + 1) * 4) + 3] = D_KEY[((rounds - i - 1) * 4) + 3];
+ }
+
+ P[0] = E_KEY[(rounds * 4) + 0];
+ P[1] = E_KEY[(rounds * 4) + 1];
+ P[2] = E_KEY[(rounds * 4) + 2];
+ P[3] = E_KEY[(rounds * 4) + 3];
+
+ memcpy(D_KEY, P, AES_EXTENDED_KEY_SIZE_B);
+
+ return 0;
+}
+
+/* Tells whether the ACE is capable to generate
+ the extended key for a given key_len. */
+static inline int aes_hw_extkey_available(u8 key_len)
+{
+ /* TODO: We should check the actual CPU model/stepping
+ as it's likely that the capability will be
+ added in the next CPU revisions. */
+ if (key_len == 16)
+ return 1;
+ return 0;
+}
+
+static void aes_padlock(void *ctx_arg, u8 *out_arg, const u8 *in_arg,
+ const u8 *iv_arg, size_t nbytes, int encdec,
+ int mode)
+{
+ struct aes_ctx *ctx = ctx_arg;
+ char bigbuf[sizeof(union cword) + 16];
+ union cword *cword;
+ void *key;
+
+ if (((long)bigbuf) & 0x0F)
+ cword = (void*)(bigbuf + 16 - ((long)bigbuf & 0x0F));
+ else
+ cword = (void*)bigbuf;
+
+ /* Prepare Control word. */
+ memset (cword, 0, sizeof(union cword));
+ cword->b.encdec = !encdec; /* in the rest of cryptoapi ENC=1/DEC=0 */
+ cword->b.rounds = 10 + (ctx->key_length - 16) / 4;
+ cword->b.ksize = (ctx->key_length - 16) / 8;
+
+ /* Is the hardware capable to generate the extended key? */
+ if (!aes_hw_extkey_available(ctx->key_length))
+ cword->b.keygen = 1;
+
+ /* ctx->E starts with a plain key - if the hardware is capable
+ to generate the extended key itself we must supply
+ the plain key for both Encryption and Decryption. */
+ if (encdec == CRYPTO_OP_ENCRYPT || cword->b.keygen == 0)
+ key = ctx->E;
+ else
+ key = ctx->D;
+
+ padlock_aligner(out_arg, in_arg, iv_arg, key, cword,
+ nbytes, AES_BLOCK_SIZE, encdec, mode);
+}
+
+static void aes_padlock_ecb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
+ size_t nbytes, int encdec)
+{
+ aes_padlock(ctx, dst, src, NULL, nbytes, encdec, CRYPTO_MODE_ECB);
+}
+
+static void aes_padlock_cbc(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
+ size_t nbytes, int encdec)
+{
+ aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_CBC);
+}
+
+static void aes_padlock_cfb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
+ size_t nbytes, int encdec)
+{
+ aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_CFB);
+}
+
+static void aes_padlock_ofb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
+ size_t nbytes, int encdec)
+{
+ aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_OFB);
+}
+
+static struct crypto_capability padlock_caps[] =
+{
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_OFB, 1000},
+
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_OFB, 1000},
+
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_OFB, 1000},
+
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_OFB, 1000},
+
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_OFB, 1000},
+
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_OFB, 1000},
+};
+static int padlock_cap_number = sizeof(padlock_caps)/sizeof(padlock_caps[0]);
+
+static void padlock_data_ready(struct crypto_device *dev);
+static int padlock_data_ready_reentry;
+
+static struct crypto_device padlock_device =
+{
+ .name = "via-padlock",
+ .data_ready = padlock_data_ready,
+ .cap = &padlock_caps[0],
+};
+
+static void process_session(struct crypto_session *s)
+{
+ int err;
+ u8 *key, *dst, *src, *iv;
+ size_t size, keylen;
+
+ key = ((u8 *)page_address(s->data.sg_key.page)) + s->data.sg_key.offset;
+ keylen = s->data.sg_key.length;
+ dst = ((u8 *)page_address(s->data.sg_dst.page)) + s->data.sg_dst.offset;
+ src = ((u8 *)page_address(s->data.sg_src.page)) + s->data.sg_src.offset;
+ size = s->data.sg_src.length;
+ iv = ((u8 *)page_address(s->data.sg_iv.page)) + s->data.sg_iv.offset;
+
+ err = aes_set_key(s->data.priv, key, keylen);
+ if (err)
+ return;
+
+ switch (s->ci.mode)
+ {
+ case CRYPTO_MODE_ECB:
+ aes_padlock_ecb(s->data.priv, dst, src, iv, size, s->ci.operation);
+ break;
+ case CRYPTO_MODE_CBC:
+ aes_padlock_cbc(s->data.priv, dst, src, iv, size, s->ci.operation);
+ break;
+ case CRYPTO_MODE_CFB:
+ aes_padlock_cfb(s->data.priv, dst, src, iv, size, s->ci.operation);
+ break;
+ case CRYPTO_MODE_OFB:
+ aes_padlock_ofb(s->data.priv, dst, src, iv, size, s->ci.operation);
+ break;
+ }
+
+ s->data.sg_dst.length = size;
+
+ return;
+}
+
+static void padlock_data_ready(struct crypto_device *dev)
+{
+ struct crypto_session *s, *n;
+
+ if (padlock_data_ready_reentry)
+ return;
+
+ padlock_data_ready_reentry++;
+ list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry)
+ {
+ if (!session_completed(s))
+ {
+ start_process_session(s);
+ process_session(s);
+ crypto_stat_complete_inc(s);
+ complete_session(s);
+ stop_process_session(s);
+ }
+ }
+ padlock_data_ready_reentry--;
+}
+
+int padlock_init_aes(void)
+{
+ u32 cpuid, edx;
+ u32 val = 0xC0000000;
+
+ cpuid = cpuid_eax(val);
+ edx = cpuid_edx(val);
+ printk("val=%x, cpuid=%x, edx=%x.\n", val, cpuid, edx);
+ if (cpuid >= val + 1)
+ {
+ printk("Board supports ACE.\n");
+ }
+ else
+ {
+ printk("Board does not support ACE.\n");
+ return -ENODEV;
+ }
+
+ printk(KERN_NOTICE "Using VIA PadLock ACE for AES algorithm (multiblock).\n");
+
+ padlock_device.cap_number = padlock_cap_number;
+
+ gen_tabs();
+ return crypto_device_add(&padlock_device);
+}
+
+void padlock_fini_aes(void)
+{
+ crypto_device_remove(&padlock_device);
+}
diff -Nru /tmp/empty/via-padlock/padlock-generic.c linux-2.6/drivers/acrypto/via-padlock/padlock-generic.c
--- /tmp/empty/via-padlock/padlock-generic.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/via-padlock/padlock-generic.c 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,194 @@
+/*
+ * Cryptographic API.
+ *
+ * Support for VIA PadLock hardware crypto engine.
+ *
+ * Linux developers:
+ * Michal Ludvig <mludvig@suse.cz>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <linux/crypto.h>
+#include <asm/byteorder.h>
+
+#include "padlock.h"
+#include "../acrypto.h"
+#include "../crypto_def.h"
+
+#define PFX "padlock: "
+
+typedef void (xcrypt_t)(u8 *input, u8 *output, u8 *key, u8 *iv,
+ void *control_word, u32 count);
+
+static inline void padlock_xcrypt_ecb(u8 *input, u8 *output, u8 *key,
+ u8 *iv, void *control_word, u32 count)
+{
+ asm volatile ("pushfl; popfl"); /* enforce key reload. */
+ asm volatile (".byte 0xf3,0x0f,0xa7,0xc8" /* rep xcryptecb */
+ : "=m"(*output), "+S"(input), "+D"(output)
+ : "d"(control_word), "b"(key), "c"(count));
+}
+
+static inline void padlock_xcrypt_cbc(u8 *input, u8 *output, u8 *key,
+ u8 *iv, void *control_word, u32 count)
+{
+ asm volatile ("pushfl; popfl"); /* enforce key reload. */
+ asm volatile (".byte 0xf3,0x0f,0xa7,0xd0" /* rep xcryptcbc */
+ : "=m"(*output), "+S"(input), "+D"(output)
+ : "d"(control_word), "b"(key), "c"(count), "a"(iv));
+}
+
+static inline void padlock_xcrypt_cfb(u8 *input, u8 *output, u8 *key,
+ u8 *iv, void *control_word, u32 count)
+{
+ asm volatile ("pushfl; popfl"); /* enforce key reload. */
+ asm volatile (".byte 0xf3,0x0f,0xa7,0xe0" /* rep xcryptcfb */
+ : "=m"(*output), "+S"(input), "+D"(output)
+ : "d"(control_word), "b"(key), "c"(count), "a"(iv));
+}
+
+static inline void padlock_xcrypt_ofb(u8 *input, u8 *output, u8 *key,
+ u8 *iv, void *control_word, u32 count)
+{
+ asm volatile ("pushfl; popfl"); /* enforce key reload. */
+ asm volatile (".byte 0xf3,0x0f,0xa7,0xe8" /* rep xcryptofb */
+ : "=m"(*output), "+S"(input), "+D"(output)
+ : "d"(control_word), "b"(key), "c"(count), "a"(iv));
+}
+
+void *crypto_aligned_kmalloc(size_t size, int mode, size_t alignment, void **index)
+{
+ char *ptr;
+
+ ptr = kmalloc(size + alignment, mode);
+ *index = ptr;
+ if (alignment > 1 && ((long)ptr & (alignment - 1))) {
+ ptr += alignment - ((long)ptr & (alignment - 1));
+ }
+
+ return ptr;
+}
+
+void padlock_aligner(u8 *out_arg, const u8 *in_arg, const u8 *iv_arg,
+ void *key, union cword *cword,
+ size_t nbytes, size_t blocksize,
+ int encdec, int mode)
+{
+ /* Don't blindly modify this structure - the items must
+ fit on 16-Bytes boundaries! */
+ struct padlock_xcrypt_data {
+ u8 iv[blocksize]; /* Initialization vector */
+ };
+
+ u8 *in, *out, *iv;
+ void *index = NULL;
+ char bigbuf[sizeof(struct padlock_xcrypt_data) + 16];
+ struct padlock_xcrypt_data *data;
+
+ /* Place 'data' at the first 16-Bytes aligned address in 'bigbuf'. */
+ if (((long)bigbuf) & 0x0F)
+ data = (void*)(bigbuf + 16 - ((long)bigbuf & 0x0F));
+ else
+ data = (void*)bigbuf;
+
+ if (((long)in_arg) & 0x0F) {
+ in = crypto_aligned_kmalloc(nbytes, GFP_KERNEL, 16, &index);
+ memcpy(in, in_arg, nbytes);
+ }
+ else
+ in = (u8*)in_arg;
+
+ if (((long)out_arg) & 0x0F) {
+ if (index)
+ out = in; /* xcrypt can work "in place" */
+ else
+ out = crypto_aligned_kmalloc(nbytes, GFP_KERNEL, 16, &index);
+ }
+ else
+ out = out_arg;
+
+ /* Always make a local copy of IV - xcrypt may change it! */
+ iv = data->iv;
+ if (iv_arg)
+ memcpy(iv, iv_arg, blocksize);
+
+
+ dprintk("data=%p\n", data);
+ dprintk("in=%p\n", in);
+ dprintk("out=%p\n", out);
+ dprintk("iv=%p\n", iv);
+ dprintk("nbytes=%d, blocksize=%d.\n", nbytes, blocksize);
+
+ switch (mode) {
+ case CRYPTO_MODE_ECB:
+ padlock_xcrypt_ecb(in, out, key, iv, cword, nbytes/blocksize);
+ break;
+
+ case CRYPTO_MODE_CBC:
+ padlock_xcrypt_cbc(in, out, key, iv, cword, nbytes/blocksize);
+ break;
+
+ case CRYPTO_MODE_CFB:
+ padlock_xcrypt_cfb(in, out, key, iv, cword, nbytes/blocksize);
+ break;
+
+ case CRYPTO_MODE_OFB:
+ padlock_xcrypt_ofb(in, out, key, iv, cword, nbytes/blocksize);
+ break;
+
+ default:
+ BUG();
+ }
+
+ /* Copy the 16-Byte aligned output to the caller's buffer. */
+ if (out != out_arg)
+ memcpy(out_arg, out, nbytes);
+
+ if (index)
+ kfree(index);
+}
+
+static int __init padlock_init(void)
+{
+ int ret = -ENOSYS;
+#if 0
+ if (!cpu_has_xcrypt) {
+ printk(KERN_ERR PFX "VIA PadLock not detected.\n");
+ return -ENODEV;
+ }
+
+ if (!cpu_has_xcrypt_enabled) {
+ printk(KERN_ERR PFX "VIA PadLock detected, but not enabled. Hmm, strange...\n");
+ return -ENODEV;
+ }
+#endif
+ if ((ret = padlock_init_aes())) {
+ printk(KERN_ERR PFX "VIA PadLock AES initialization failed.\n");
+ return ret;
+ }
+
+ if (ret == -ENOSYS)
+ printk(KERN_ERR PFX "Hmm, VIA PadLock was compiled without any algorithm.\n");
+
+ return ret;
+}
+
+static void __exit padlock_fini(void)
+{
+ padlock_fini_aes();
+}
+
+module_init(padlock_init);
+module_exit(padlock_fini);
+
+MODULE_DESCRIPTION("VIA PadLock crypto engine support.");
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_AUTHOR("Michal Ludvig");
diff -Nru /tmp/empty/via-padlock/padlock.h linux-2.6/drivers/acrypto/via-padlock/padlock.h
--- /tmp/empty/via-padlock/padlock.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/via-padlock/padlock.h 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,71 @@
+/*
+ * Cryptographic API.
+ *
+ * Copyright (c) 2004 Michal Ludvig <mludvig@suse.cz>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ *
+ */
+
+#ifndef _CRYPTO_PADLOCK_H
+#define _CRYPTO_PADLOCK_H
+
+#define AES_MIN_KEY_SIZE 16 /* in u8 units */
+#define AES_MAX_KEY_SIZE 32 /* ditto */
+#define AES_BLOCK_SIZE 16 /* ditto */
+#define AES_EXTENDED_KEY_SIZE 64 /* in u32 units */
+#define AES_EXTENDED_KEY_SIZE_B (AES_EXTENDED_KEY_SIZE * sizeof(u32))
+
+struct aes_ctx {
+ u32 e_data[AES_EXTENDED_KEY_SIZE+4];
+ u32 d_data[AES_EXTENDED_KEY_SIZE+4];
+ int key_length;
+ u32 *E;
+ u32 *D;
+};
+
+#define E_KEY ctx->E
+#define D_KEY ctx->D
+
+
+/* Control word. */
+#if 1
+union cword {
+ u32 cword[4];
+ struct {
+ int rounds:4;
+ int algo:3;
+ int keygen:1;
+ int interm:1;
+ int encdec:1;
+ int ksize:2;
+ } b;
+};
+#else
+union cword {
+ u32 cword[4];
+ struct {
+ unsigned rounds:4,
+ algo:3,
+ keygen:1,
+ interm:1,
+ encdec:1,
+ ksize:2;
+ } b;
+};
+#endif
+
+#define PFX "padlock: "
+
+void padlock_aligner(u8 *out_arg, const u8 *in_arg, const u8 *iv_arg,
+ void *key, union cword *cword,
+ size_t nbytes, size_t blocksize,
+ int encdec, int mode);
+
+int padlock_init_aes(void);
+void padlock_fini_aes(void);
+
+#endif /* _CRYPTO_PADLOCK_H */
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 17:46 ` James Morris
@ 2004-10-30 5:20 ` Evgeniy Polyakov
2004-10-31 5:43 ` James Morris
0 siblings, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-30 5:20 UTC (permalink / raw)
To: James Morris; +Cc: netdev, cryptoapi
On Fri, 29 Oct 2004 13:46:04 -0400 (EDT)
James Morris <jmorris@redhat.com> wrote:
> On Fri, 29 Oct 2004, James Morris wrote:
>
> > On Fri, 29 Oct 2004, Evgeniy Polyakov wrote:
> >
> > > I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
> >
>
> Also, please consider EXPORT_SYMBOL_GPL instead of EXPORT_SYMBOL.
I'm not insisting, but why?
>
> - James
> --
> James Morris
> <jmorris@redhat.com>
>
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 19:56 ` Sam Leffler
@ 2004-10-30 6:16 ` Christoph Hellwig
0 siblings, 0 replies; 57+ messages in thread
From: Christoph Hellwig @ 2004-10-30 6:16 UTC (permalink / raw)
To: Sam Leffler; +Cc: johnpol, netdev, cryptoapi
On Fri, Oct 29, 2004 at 12:56:15PM -0700, Sam Leffler wrote:
> I realize this suggestion will be met with the usual boo's and catcalls
> but it'd also be nice if all systems could share a common user-level api
> so, for example, you could just _use_ the support in openssl that's been
> around for several years.
Does FreeBSD have a less broken user API than OpenBSD? The OpenBSD API
had been implemented for Linux and had been rejected dueto execessive
bogusness.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 5:19 ` Evgeniy Polyakov
@ 2004-10-30 8:34 ` Evgeniy Polyakov
2004-10-30 8:36 ` Evgeniy Polyakov
2004-10-30 16:57 ` Michal Ludvig
1 sibling, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-30 8:34 UTC (permalink / raw)
To: johnpol; +Cc: James Morris, netdev, cryptoapi
On Sat, 30 Oct 2004 09:19:32 +0400
Evgeniy Polyakov <johnpol@2ka.mipt.ru> wrote:
> On Fri, 29 Oct 2004 13:44:24 -0400 (EDT)
> James Morris <jmorris@redhat.com> wrote:
>
> > On Fri, 29 Oct 2004, Evgeniy Polyakov wrote:
> >
> > > I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
> >
> > Could you please resend as a patch against the latest bk snapshot?
> >
> > i.e. output of 'diff -purN tree-a tree-b'
> >
>
> Attached following patches:
>
> Kconfig.connector.patch
> Kconfig.crypto.patch
> Makefile.connector.patch
> Makefile.crypto.patch
> acrypto.patch
> connector.patch
>
> Any other files are examples and do not included here.
Forget Kconfig for acrypto.
> >
> > - James
> > --
> > James Morris
> > <jmorris@redhat.com>
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 8:34 ` Evgeniy Polyakov
@ 2004-10-30 8:36 ` Evgeniy Polyakov
0 siblings, 0 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-30 8:36 UTC (permalink / raw)
To: johnpol; +Cc: James Morris, cryptoapi, netdev
[-- Attachment #1: Type: text/plain, Size: 528 bytes --]
Now attached needed file.
--- /dev/null 2004-09-17 14:58:06.000000000 +0400
+++ linux-2.6/drivers/acrypto/Kconfig 2004-10-30 12:32:53.000000000 +0400
@@ -0,0 +1,15 @@
+menu "Asynchronous crypto layer"
+
+config ACRYPTO
+ tristate "Asynchronous crypto layer"
+ depends on CONNECTOR && CRYPTO
+ ---help---
+ Asynchronous crypto layer.
+
+config SIMPLE_LB
+ tristate "Simple load balancer"
+ depends on ACRYPTO
+ ---help---
+ Simple load balancer.
+endmenu
+
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
[-- Attachment #2: acrypto.patch.addon --]
[-- Type: application/octet-stream, Size: 432 bytes --]
--- /dev/null 2004-09-17 14:58:06.000000000 +0400
+++ linux-2.6/drivers/acrypto/Kconfig 2004-10-30 12:32:53.000000000 +0400
@@ -0,0 +1,15 @@
+menu "Asynchronous crypto layer"
+
+config ACRYPTO
+ tristate "Asynchronous crypto layer"
+ depends on CONNECTOR && CRYPTO
+ ---help---
+ Asynchronous crypto layer.
+
+config SIMPLE_LB
+ tristate "Simple load balancer"
+ depends on ACRYPTO
+ ---help---
+ Simple load balancer.
+endmenu
+
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 5:19 ` Evgeniy Polyakov
2004-10-30 8:34 ` Evgeniy Polyakov
@ 2004-10-30 16:57 ` Michal Ludvig
2004-10-30 17:40 ` Evgeniy Polyakov
2004-10-30 19:42 ` James Morris
1 sibling, 2 replies; 57+ messages in thread
From: Michal Ludvig @ 2004-10-30 16:57 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: netdev, cryptoapi
On Sat, 30 Oct 2004, Evgeniy Polyakov wrote:
> On Fri, 29 Oct 2004 13:44:24 -0400 (EDT)
> James Morris <jmorris@redhat.com> wrote:
>
> > On Fri, 29 Oct 2004, Evgeniy Polyakov wrote:
> >
> > > I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
> >
> > Could you please resend as a patch against the latest bk snapshot?
> >
> > i.e. output of 'diff -purN tree-a tree-b'
> >
>
> Attached following patches:
>
> Kconfig.connector.patch
> Kconfig.crypto.patch
> Makefile.connector.patch
> Makefile.crypto.patch
> acrypto.patch
> connector.patch
Hi Evgeniy,
I have compiled and booted with your patches. Not too much
playing around so far. Anyway some quick observations:
- "rmmod simple_lb" hangs because:
| <6>You are removing crypto load balancer simple_lb which is current and
| default.
| There is no other crypto load balancers. Removing is delayed untill new
| load balancer is registered.
Of course I can't re-add it because the module is still there. In this
case rmmod should fail with a gentle message (-EBUSY) instead of hang. Or
let it go if not in use.
- In the kernel config ACRYPTO depends on CONNECTOR, but is above it in
the menuconfig.
- I would choose a different directory strucrure. Instead of
drivers/acrypto and drivers/connector I'd put the device-independent
parrts info e.g. crypto/acrypto and the drivers (e.g. Fcrypt or PadLock)
into drivers/crypto.
- For other interested: concatenated patch with some very minor tweaks is
at http://www.logix.cz/michal/dl/acrypto/acrypto.diff
My changes are in mludvig.diff in the same directory.
It applies cleanly to BitKeeper tree as well.
Michal Ludvig
--
* A mouse is a device used to point at the xterm you want to type in.
* Personal homepage - http://www.logix.cz/michal
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 16:57 ` Michal Ludvig
@ 2004-10-30 17:40 ` Evgeniy Polyakov
2004-10-30 20:17 ` Michal Ludvig
2004-10-30 19:42 ` James Morris
1 sibling, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-30 17:40 UTC (permalink / raw)
To: Michal Ludvig; +Cc: netdev, cryptoapi
On Sat, 30 Oct 2004 18:57:20 +0200 (CEST)
Michal Ludvig <michal@logix.cz> wrote:
> On Sat, 30 Oct 2004, Evgeniy Polyakov wrote:
>
> > On Fri, 29 Oct 2004 13:44:24 -0400 (EDT)
> > James Morris <jmorris@redhat.com> wrote:
> >
> > > On Fri, 29 Oct 2004, Evgeniy Polyakov wrote:
> > >
> > > > I'm pleased to announce asynchronous crypto layer for Linux kernel 2.6.
> > >
> > > Could you please resend as a patch against the latest bk snapshot?
> > >
> > > i.e. output of 'diff -purN tree-a tree-b'
> > >
> >
> > Attached following patches:
> >
> > Kconfig.connector.patch
> > Kconfig.crypto.patch
> > Makefile.connector.patch
> > Makefile.crypto.patch
> > acrypto.patch
> > connector.patch
>
> Hi Evgeniy,
>
> I have compiled and booted with your patches. Not too much
> playing around so far. Anyway some quick observations:
I've wrote mega block device which can work(without hang) only
with 10mb "disks", but what did you expect from midnight hack after
several Staropramen's :)
Give me several minutes to complete async_provider.c and I will post
numbers, which I expect will be the same for both sync and async
driver since I do not have neither SMP(which should benefit I believe)
nor crypto card here.
> - "rmmod simple_lb" hangs because:
> | <6>You are removing crypto load balancer simple_lb which is current and
> | default.
> | There is no other crypto load balancers. Removing is delayed untill new
> | load balancer is registered.
> Of course I can't re-add it because the module is still there. In this
> case rmmod should fail with a gentle message (-EBUSY) instead of hang. Or
> let it go if not in use.
acrypto module has parameter force_lb_remove which if is set allows to remove
last crypto load balancer.
But ut has atricky moment - since any crypto_session_alloc() may occur
asyncronously then we can not say if there are any load balancer users
without some locks. Currently if there is no any crypto load balancer, then
acrypto will catch BUG_ON().
I will think of it some more.
> - In the kernel config ACRYPTO depends on CONNECTOR, but is above it in
> the menuconfig.
>
> - I would choose a different directory strucrure. Instead of
> drivers/acrypto and drivers/connector I'd put the device-independent
> parrts info e.g. crypto/acrypto and the drivers (e.g. Fcrypt or PadLock)
> into drivers/crypto.
connector is independent module - it is used instead of ioctl for
diagnostic. So I think it should live in drivers/connector.
I do not have any objection against your tree schema for crypto stuff.
> - For other interested: concatenated patch with some very minor tweaks is
> at http://www.logix.cz/michal/dl/acrypto/acrypto.diff
> My changes are in mludvig.diff in the same directory.
> It applies cleanly to BitKeeper tree as well.
I totally agree with your changes, thank you.
Note: KERN_EMERG was put into dprintk to allow netconsole send this data
by default.
> Michal Ludvig
> --
> * A mouse is a device used to point at the xterm you want to type in.
> * Personal homepage - http://www.logix.cz/michal
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 16:57 ` Michal Ludvig
2004-10-30 17:40 ` Evgeniy Polyakov
@ 2004-10-30 19:42 ` James Morris
1 sibling, 0 replies; 57+ messages in thread
From: James Morris @ 2004-10-30 19:42 UTC (permalink / raw)
To: Michal Ludvig; +Cc: Evgeniy Polyakov, netdev, cryptoapi
On Sat, 30 Oct 2004, Michal Ludvig wrote:
> - I would choose a different directory strucrure. Instead of
> drivers/acrypto and drivers/connector I'd put the device-independent
> parrts info e.g. crypto/acrypto and the drivers (e.g. Fcrypt or PadLock)
Agreed, do we need a separate acrypto subdirectory? Why not just put the
async code directly under crypto.
- James
--
James Morris
<jmorris@redhat.com>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 17:40 ` Evgeniy Polyakov
@ 2004-10-30 20:17 ` Michal Ludvig
2004-10-30 20:56 ` Evgeniy Polyakov
0 siblings, 1 reply; 57+ messages in thread
From: Michal Ludvig @ 2004-10-30 20:17 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: netdev, cryptoapi
On Sat, 30 Oct 2004, Evgeniy Polyakov wrote:
> On Sat, 30 Oct 2004 18:57:20 +0200 (CEST)
> Michal Ludvig <michal@logix.cz> wrote:
>
> > I have compiled and booted with your patches. Not too much
> > playing around so far. Anyway some quick observations:
>
> I've wrote mega block device which can work(without hang) only
> with 10mb "disks", but what did you expect from midnight hack after
> several Staropramen's :)
10MB is probably too small to get some reliable numbers. The testing
should last for at least couple of seconds...
> Give me several minutes to complete async_provider.c and I will post
> numbers, which I expect will be the same for both sync and async
> driver since I do not have neither SMP(which should benefit I believe)
> nor crypto card here.
At least try it with a SMP kernel (on UP machine) to catch the worst
spindeadlocks etc. In the ideal case the acrypto core should know that
there are two (four, ...) CPUs and all of them can encrypt in paralel.
> > - "rmmod simple_lb" hangs because:
> > | <6>You are removing crypto load balancer simple_lb which is current and
> > | default.
> > | There is no other crypto load balancers. Removing is delayed untill new
> > | load balancer is registered.
> > Of course I can't re-add it because the module is still there. In this
> > case rmmod should fail with a gentle message (-EBUSY) instead of hang. Or
> > let it go if not in use.
>
> acrypto module has parameter force_lb_remove which if is set allows to remove
> last crypto load balancer.
> But ut has atricky moment - since any crypto_session_alloc() may occur
> asyncronously then we can not say if there are any load balancer users
> without some locks. Currently if there is no any crypto load balancer, then
> acrypto will catch BUG_ON().
BUG_ON()? Shouldn't crypto_session_alloc() fail instead? BUG_ON() is for
catching conditions that really shouldn't happen in normal operation.
Missing module with a load balancer won't be that unusual...
Or should the acrypto.ko module prerequire a balancer to get it loaded
automatically?
Michal Ludvig
--
* A mouse is a device used to point at the xterm you want to type in.
* Personal homepage - http://www.logix.cz/michal
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 14:06 ` Evgeniy Polyakov
2004-10-29 14:03 ` Michal Ludvig
2004-10-29 15:08 ` jamal
@ 2004-10-30 20:35 ` Eugene Surovegin
2004-10-30 21:04 ` Evgeniy Polyakov
2 siblings, 1 reply; 57+ messages in thread
From: Eugene Surovegin @ 2004-10-30 20:35 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: hadi, netdev, cryptoapi
On Fri, Oct 29, 2004 at 06:06:52PM +0400, Evgeniy Polyakov wrote:
> If we have a hardware accelerator chip, than we _already_ have improvements
> with even the worst async crypto layer, since software and hardware
> will work in parrallel.
This is not true.
For example, if chip request setup and PCI transfer takes more than
just using sw implementation. This is reality for AES and short
packets.
--
Eugene
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 6:22 Asynchronous crypto layer Evgeniy Polyakov
` (2 preceding siblings ...)
2004-10-29 19:56 ` Sam Leffler
@ 2004-10-30 20:37 ` Evgeniy Polyakov
2004-10-31 16:05 ` James Morris
` (2 subsequent siblings)
6 siblings, 0 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-30 20:37 UTC (permalink / raw)
To: jamal, Michal Ludvig; +Cc: netdev, cryptoapi
[-- Attachment #1: Type: text/plain, Size: 841 bytes --]
Sorry, no numbers today.
I've attached block device driver, synchronous driver which is needed
for block device and asycnhronous one, first two works as expected, but
either I too tiread after climbing or one of the two, but
asynchronous driver hangs on mounting.
Michal, please test block device with your PCI adapter with attached
scripts and synchronous driver adapter and report numbers.
Tomorrow I will finish software asynchronous driver and then we can
test it on SMP where I expect some speed increase.
Thank you.
P.S. While digging into my test trees I found very usefull patch that
I forget to apply. It converts _bh locking into _irq{save,restore}
since session allocation and processing can occure with any kind
of interrupt state. Patch is also attached.
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
[-- Attachment #2: Makefile --]
[-- Type: application/octet-stream, Size: 304 bytes --]
obj-m := bd.o
obj-m += sync_provider.o
obj-m += async_provider.o
#KDIR := /lib/modules/$(shell uname -r)/build
KDIR := /usr/local/src/linux/soekris/tmp/linux-2.6
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
clean:
rm -f *.o *.ko *.mod.* .*.cmd *~
rm -rf .tmp_versions
[-- Attachment #3: async_provider.c --]
[-- Type: text/x-csrc, Size: 6563 bytes --]
/*
* async_provider.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/err.h>
#include <linux/crypto.h>
#include <linux/mm.h>
#include <linux/blkdev.h>
#include "../crypto/acrypto.h"
#include "../crypto/crypto_stat.h"
#include "../crypto/crypto_def.h"
static void prov_data_ready(struct crypto_device *);
static void async_callback(struct crypto_session_initializer *, struct crypto_data *);
static struct crypto_capability prov_caps[] = {
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 1000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 1000},
};
static int prov_cap_number = sizeof(prov_caps)/sizeof(prov_caps[0]);
static struct completion thread_exited;
static DECLARE_WAIT_QUEUE_HEAD(async_wait_queue);
static int need_exit;
static struct crypto_tfm *tfm;
struct crypto_session *sp;
static char async_algo[] = "aes";
static char async_key[] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f};
static struct crypto_session_initializer ci =
{
.operation = CRYPTO_OP_ENCRYPT,
.mode = CRYPTO_MODE_ECB,
.type = CRYPTO_TYPE_AES_128,
.callback = async_callback,
};
static struct crypto_device pdev =
{
.name = "async_provider",
.data_ready = prov_data_ready,
.cap = &prov_caps[0],
};
static void async_callback(struct crypto_session_initializer *ci, struct crypto_data *data)
{
struct request *req = data->priv;
dprintk("%s: req=%p\n", __func__, req);
(unsigned long)req->special--;
if (((unsigned long)req->special) == 0)
{
spinlock_t *l = req->q->queue_lock;
dprintk("Finishing request for req=%p.\n", req);
spin_lock_irq(l);
end_that_request_last(req);
spin_unlock_irq(l);
}
}
static __inline__ void async_prepare(struct crypto_data *data, void *src, void *dst, unsigned long size, void *priv)
{
struct request *req = priv;
(unsigned long)req->special++;
dprintk("%s: req=%p, refcnt=%lu.\n",
__func__, req, (unsigned long)req->special);
data->sg_src.page = virt_to_page(src);
data->sg_src.offset = offset_in_page(src);
data->sg_src.length = size;
data->sg_dst.page = virt_to_page(dst);
data->sg_dst.offset = offset_in_page(dst);
data->sg_dst.length = size;
data->sg_key.page = virt_to_page(async_key);
data->sg_key.offset = offset_in_page(async_key);
data->sg_key.length = sizeof(async_key);
data->priv = priv;
data->priv_size = 0;
}
void bd_encrypt(void *src, void *dst, unsigned long size, void *priv)
{
struct crypto_session *session;
struct crypto_data data;
async_prepare(&data, src, dst, size, priv);
ci.operation = CRYPTO_OP_ENCRYPT;
session = crypto_session_alloc(&ci, &data);
if (!session)
WARN_ON(1);
wake_up(&async_wait_queue);
}
void bd_decrypt(void *src, void *dst, unsigned long size, void *priv)
{
struct crypto_session *session;
struct crypto_data data;
async_prepare(&data, src, dst, size, priv);
ci.operation = CRYPTO_OP_DECRYPT;
session = crypto_session_alloc(&ci, &data);
if (!session)
WARN_ON(1);
wake_up(&async_wait_queue);
}
static void prov_data_ready(struct crypto_device *dev)
{
struct crypto_session *s, *n;
list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry)
{
if (!session_completed(s))
{
sp = s;
wake_up(&async_wait_queue);
break;
}
}
}
static int async_thread(void *data)
{
struct crypto_device *dev = (struct crypto_device *)data;
daemonize("%s", dev->name);
allow_signal(SIGTERM);
while (!need_exit)
{
interruptible_sleep_on(&async_wait_queue);
if (need_exit)
break;
if (!sp)
continue;
start_process_session(sp);
if (sp->ci.operation == CRYPTO_OP_ENCRYPT)
crypto_cipher_encrypt(tfm, &sp->data.sg_dst, &sp->data.sg_src, sp->data.sg_src.length);
else
crypto_cipher_decrypt(tfm, &sp->data.sg_dst, &sp->data.sg_src, sp->data.sg_src.length);
dprintk("Completing session %llu [%llu] in %s.\n",
sp->ci.id, sp->ci.dev_id, pdev.name);
//crypto_stat_complete_inc(sp);
complete_session(sp);
stop_process_session(sp);
prov_data_ready(dev);
sp = NULL;
}
complete_and_exit(&thread_exited, 0);
}
int prov_init(void)
{
int err, pid;
tfm = crypto_alloc_tfm(async_algo, 0);
if (!tfm)
{
printk(KERN_ERR "Failed to allocate SHA1 tfm.\n");
return -EINVAL;
}
err = crypto_cipher_setkey(tfm, async_key, sizeof(async_key));
if (err)
{
dprintk("Failed to set key [keylen=%d]: err=%d.\n",
sizeof(async_key), err);
goto err_out_free_tfm;
}
init_completion(&thread_exited);
pid = kernel_thread(async_thread, &pdev, CLONE_FS | CLONE_FILES);
if (IS_ERR((void *)pid))
{
err = -EINVAL;
dprintk(KERN_ERR "Failed to create kernel load balancing thread.\n");
goto err_out_free_tfm;
}
pdev.cap_number = prov_cap_number;
err = crypto_device_add(&pdev);
if (err)
goto err_out_remove_thread;
dprintk(KERN_INFO "Test crypto provider module %s is loaded.\n", pdev.name);
return 0;
err_out_remove_thread:
need_exit = 1;
wake_up(&async_wait_queue);
wait_for_completion(&thread_exited);
err_out_free_tfm:
crypto_free_tfm(tfm);
return err;
}
void prov_fini(void)
{
need_exit = 1;
wake_up(&async_wait_queue);
wait_for_completion(&thread_exited);
crypto_device_remove(&pdev);
crypto_free_tfm(tfm);
dprintk(KERN_INFO "Test crypto provider module %s is unloaded.\n", pdev.name);
}
module_init(prov_init);
module_exit(prov_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Test crypto module provider.");
EXPORT_SYMBOL(bd_encrypt);
EXPORT_SYMBOL(bd_decrypt);
[-- Attachment #4: bd.c --]
[-- Type: text/x-csrc, Size: 6536 bytes --]
/*
* bd.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/spinlock.h>
#include <linux/blkdev.h>
#include <linux/fs.h>
#include <linux/bio.h>
#include <linux/init.h>
//#define BD_DEBUG
#ifdef BD_DEBUG
#define dprintk(f, a...) printk(KERN_EMERG f, ##a)
#else
#define dprintk(f, a...) do {} while(0)
#endif
#define BD_READ 0
#define BD_WRITE 1
static char bd_name[] = "bd";
static unsigned int bd_major = 123;
module_param(bd_major, uint, 0);
static unsigned int bd_sector_size = 512;
module_param(bd_sector_size, uint, 0);
static unsigned int bd_max_request_size = 1024*1024*50;
module_param(bd_max_request_size, uint, 0);
static void *bd_request_data;
static spinlock_t bd_lock = SPIN_LOCK_UNLOCKED;
static struct gendisk *bd_disk;
extern void bd_encrypt(void *src, void *dst, unsigned long size, void *priv);
extern void bd_decrypt(void *src, void *dst, unsigned long size, void *priv);
static int bd_ioctl(struct inode *, struct file *, unsigned int, unsigned long);
static struct block_device_operations bd_fops =
{
.owner = THIS_MODULE,
.ioctl = bd_ioctl,
};
static int bd_ioctl(struct inode *inode, struct file *file,
unsigned int cmd, unsigned long arg)
{
dprintk("%s: cmd=%u, arg=%ld.\n", __func__, cmd, arg);
return 0;
}
static int bd_process_data(int cmd, void *buf, unsigned long off, unsigned int size, void *priv)
{
if (size > bd_max_request_size)
{
dprintk("Too big size %u, setting to %u.\n",
size, bd_max_request_size);
size = bd_max_request_size;
}
if (off + size > bd_max_request_size)
{
dprintk("Too big size %u or offset %lu, exiting.\n",
size, off);
size = bd_max_request_size;
}
switch (cmd)
{
case BD_READ:
bd_decrypt(((char *)bd_request_data) + off, buf, size, priv);
break;
case BD_WRITE:
bd_encrypt(buf, ((char *)bd_request_data) + off, size, priv);
break;
}
dprintk("%s finished cmd=%s, size=%u, off=%lu.\n",
__func__,
(cmd == BD_WRITE)?"WRITE":"READ",
size, off);
return 0;
}
static __inline__ int bd_process_bvec(int cmd, struct bio_vec *bvec, unsigned long off, void *priv)
{
int err;
char *kaddr;
void *buf;
unsigned int size;
kaddr = page_address(bvec->bv_page);
buf = kaddr + bvec->bv_offset;
size = bvec->bv_len;
err = bd_process_data(cmd, buf, off, size, priv);
return err;
}
static void bd_request(request_queue_t * q)
{
struct request *req;
unsigned long off, size;
int err, cmd, bad_bio;
struct bio *bio;
while ((req = elv_next_request(q)) != NULL)
{
blkdev_dequeue_request(req);
(unsigned long)req->special = 0;
off = req->sector * bd_sector_size;
size = req->nr_sectors * bd_sector_size;
cmd = (rq_data_dir(req) == 1)?BD_WRITE:BD_READ;
dprintk("%s: TRANSFER: cmd=%s, off=%lu [sector=%ld], size=%lu [nr_sectors=%lu], flags=%lx.\n",
bd_name,
(cmd == BD_WRITE)?"WRITE":"READ",
off, req->sector,
size, req->nr_sectors,
req->flags);
bad_bio = 0;
if (!(req->flags & REQ_CMD))
{
dprintk("%s: wrong command.\n", bd_name);
goto err_out_wrong_command;
}
#if 1
rq_for_each_bio(bio, req)
{
struct bio_vec *bvec;
int i;
dprintk("BIO: bi_size=%u, bvec=%p.\n", bio->bi_size, bio->bi_io_vec);
if (!bio->bi_size || !bio->bi_io_vec)
{
bad_bio = 1;
continue;
}
bio_for_each_segment(bvec, bio, i) {
dprintk("%s: i=%d, req=%p, size=%d.\n",
bd_disk->disk_name, i,
req, bvec->bv_len);
err = bd_process_bvec(cmd, bvec, off, req);
if (err)
{
bad_bio = 1;
continue;
}
off += bvec->bv_len;
size -= bvec->bv_len;
}
//bio->bi_end_io = NULL;
//bio_endio(bio, bio->bi_size, (bad_bio)?-EIO:0);
}
#else
err = bd_process_data(cmd, req->buffer, off, size);
if (err)
bad_bio = 1;
#endif
dprintk("cmd=%s has been processed: err=%d\n",
(cmd == BD_WRITE)?"WRITE":"READ",
req->errors);
err_out_wrong_command:
if (!end_that_request_first(req, (bad_bio)?0:1, req->nr_sectors));
//end_that_request_last(req);
}
}
static void bd_setup(struct gendisk *d)
{
request_queue_t *q;
d->major = bd_major;
d->first_minor = 0;
d->fops = &bd_fops;
d->private_data = NULL;
d->flags = GENHD_FL_SUPPRESS_PARTITION_INFO;
sprintf(d->disk_name, "%s%d", bd_name, 0);
q = d->queue;
blk_queue_hardsect_size(q, bd_sector_size);
set_capacity(d, bd_max_request_size/bd_sector_size);
add_disk(d);
}
int __devinit bd_init(void)
{
int err;
bd_request_data = vmalloc(bd_max_request_size);
if (!bd_request_data)
{
dprintk("Failed to allocate %d bytes for %s requests.\n",
bd_max_request_size, bd_name);
return -ENOMEM;
}
err = register_blkdev(bd_major, bd_name);
if (err)
{
dprintk("Failed to register blkdev with major %u: err=%d.\n",
bd_major, err);
return err;
}
bd_disk = alloc_disk(1);
if (!bd_disk)
{
dprintk("Failed to allocate a disk.\n");
goto err_out_unregister_blkdev;
}
bd_disk->queue = blk_init_queue(bd_request, &bd_lock);
if (!bd_disk->queue)
{
dprintk("Failed to initialize blk queue: err=%d.\n", err);
goto err_out_free_disk;
}
bd_setup(bd_disk);
dprintk("%s has beed successfully added.\n", bd_name);
return 0;
err_out_free_disk:
put_disk(bd_disk);
err_out_unregister_blkdev:
unregister_blkdev(bd_major, "bd");
vfree(bd_request_data);
return -EINVAL;
}
void __devexit bd_fini(void)
{
del_gendisk(bd_disk);
blk_cleanup_queue(bd_disk->queue);
put_disk(bd_disk);
unregister_blkdev(bd_major, "bd");
vfree(bd_request_data);
dprintk("%s has beed successfully removed.\n", bd_name);
}
module_init(bd_init);
module_exit(bd_fini);
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_LICENSE("GPL");
[-- Attachment #5: sync_provider.c --]
[-- Type: text/x-csrc, Size: 3206 bytes --]
/*
* sync_provider.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/spinlock.h>
#include <linux/blkdev.h>
#include <linux/fs.h>
#include <linux/bio.h>
#include <linux/init.h>
#include <linux/crypto.h>
#include <linux/hardirq.h>
#include <asm/scatterlist.h>
//#define SYNC_DEBUG
#ifdef SYNC_DEBUG
#define dprintk(f, a...) printk(KERN_EMERG f, ##a)
#else
#define dprintk(f, a...) do {} while(0)
#endif
#define OP_ENCRYPT 0
#define OP_DECRYPT 1
static char sync_algo[] = "aes";
static char sync_key[] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f};
static struct crypto_tfm *tfm;
void bd_encrypt(void *src, void *dst, unsigned long size, void *priv)
{
struct scatterlist s, d;
s.page = virt_to_page(src);
s.offset = offset_in_page(src);
s.length = size;
d.page = virt_to_page(dst);
d.offset = offset_in_page(dst);
d.length = size;
/*
* A hack to fool crypto layer - it thinks that we can sleep here...
* Sigh.
*
* preempt_count() |= SOFTIRQ_MASK;
*/
preempt_count() |= SOFTIRQ_MASK;
crypto_cipher_encrypt(tfm, &d, &s, size);
preempt_count() &= ~SOFTIRQ_MASK;
}
void bd_decrypt(void *src, void *dst, unsigned long size, void *priv)
{
struct scatterlist s, d;
s.page = virt_to_page(src);
s.offset = offset_in_page(src);
s.length = size;
d.page = virt_to_page(dst);
d.offset = offset_in_page(dst);
d.length = size;
/*
* A hack to fool crypto layer - it thinks that we can sleep here...
* Sigh.
*
* preempt_count() |= SOFTIRQ_MASK;
*/
preempt_count() |= SOFTIRQ_MASK;
crypto_cipher_decrypt(tfm, &d, &s, size);
preempt_count() &= ~SOFTIRQ_MASK;
}
int __devinit sync_init(void)
{
int err;
err = -ENODEV;
tfm = crypto_alloc_tfm(sync_algo, 0);
if (!tfm)
{
dprintk("Failed to allocate %s tfm.\n", sync_algo);
goto err_out_exit;
}
err = crypto_cipher_setkey(tfm, sync_key, sizeof(sync_key));
if (err)
{
dprintk("Failed to set key [keylen=%d]: err=%d.\n",
sizeof(sync_key), err);
goto err_out_free_tfm;
}
return 0;
err_out_free_tfm:
crypto_free_tfm(tfm);
err_out_exit:
return err;
}
void __devexit sync_fini(void)
{
crypto_free_tfm(tfm);
}
module_init(sync_init);
module_exit(sync_fini);
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_LICENSE("GPL");
EXPORT_SYMBOL(bd_encrypt);
EXPORT_SYMBOL(bd_decrypt);
[-- Attachment #6: bd_i --]
[-- Type: application/octet-stream, Size: 390 bytes --]
mod_path=/lib/modules/`uname -r`/kernel
insmod $mod_path/arch/i386/crypto/aes-i586.ko
#insmod $mod_path/drivers/block/sync_provider.ko
insmod $mod_path/drivers/block/async_provider.ko
insmod $mod_path/drivers/block/bd.ko bd_max_request_size=52428800
mkfs.ext2 /dev/bd0
mount /dev/bd0 /mnt
#time dd if=/dev/zero of=/mnt/file bs=1M count=10
#time dd if=/dev/zero of=/dev/bd0 bs=1M count=30
[-- Attachment #7: crypto --]
[-- Type: application/octet-stream, Size: 483 bytes --]
insmod /lib/modules/2.6.9-rc4/kernel/drivers/connector/cn.ko
insmod /lib/modules/2.6.9-rc4/kernel/drivers/crypto/acrypto.ko force_lb_remove=1
insmod /lib/modules/2.6.9-rc4/kernel/drivers/crypto/simple_lb.ko
#insmod /lib/modules/2.6.9-rc4/kernel/drivers/crypto/provider.ko
#insmod /lib/modules/2.6.9-rc4/kernel/drivers/crypto/slow_provider.ko
#insmod /lib/modules/2.6.9-rc4/kernel/drivers/crypto/consumer.ko
#insmod /lib/modules/2.6.9-rc4/kernel/drivers/crypto/slow_consumer.ko
[-- Attachment #8: start --]
[-- Type: application/octet-stream, Size: 152 bytes --]
rand=$RANDOM
file=/mnt/$rand
for ((i=0; i<10; ++i)); do
echo -en "."
dd if=/dev/zero of=$file bs=1M count=10 > /dev/null 2>&1
rm -f $file
done
echo
[-- Attachment #9: crypto.patch --]
[-- Type: application/octet-stream, Size: 14474 bytes --]
--- orig/crypto_lb.c
+++ mod/crypto_lb.c
@@ -83,12 +83,13 @@
static inline void lb_set_current(struct crypto_lb *l)
{
struct crypto_lb *c = current_lb;
+ unsigned long flags;
if (c)
{
- spin_lock_bh(&c->lock);
+ spin_lock_irqsave(&c->lock, flags);
__lb_set_current(l);
- spin_unlock_bh(&c->lock);
+ spin_unlock_irqrestore(&c->lock, flags);
}
else
__lb_set_current(l);
@@ -102,12 +103,13 @@
static inline void lb_set_default(struct crypto_lb *l)
{
struct crypto_lb *c = default_lb;
+ unsigned long flags;
if (c)
{
- spin_lock_bh(&c->lock);
+ spin_lock_irqsave(&c->lock, flags);
__lb_set_default(l);
- spin_unlock_bh(&c->lock);
+ spin_unlock_irqrestore(&c->lock, flags);
}
else
__lb_set_default(l);
@@ -183,8 +185,9 @@
{
struct crypto_lb *lb;
int off = 0;
+ unsigned long flags;
- spin_lock_bh(&crypto_lb_lock);
+ spin_lock_irqsave(&crypto_lb_lock, flags);
list_for_each_entry(lb, &crypto_lb_list, lb_entry)
{
if (lb_is_current(lb))
@@ -197,7 +200,7 @@
if (lb_is_current(lb))
off += sprintf(buf+off, "]");
}
- spin_unlock_bh(&crypto_lb_lock);
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
if (!off)
off = sprintf(buf, "No load balancers regitered yet.");
@@ -209,8 +212,9 @@
static ssize_t current_store(struct class_device *dev, const char *buf, size_t count)
{
struct crypto_lb *lb;
+ unsigned long flags;
- spin_lock_bh(&crypto_lb_lock);
+ spin_lock_irqsave(&crypto_lb_lock, flags);
list_for_each_entry(lb, &crypto_lb_list, lb_entry)
{
if (count == strlen(lb->name) && !strcmp(buf, lb->name))
@@ -224,7 +228,7 @@
break;
}
}
- spin_unlock_bh(&crypto_lb_lock);
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
return count;
}
@@ -252,12 +256,14 @@
inline void crypto_lb_rehash(void)
{
+ unsigned long flags;
+
if (!current_lb)
return;
- spin_lock_bh(¤t_lb->lock);
+ spin_lock_irqsave(¤t_lb->lock, flags);
current_lb->rehash(current_lb);
- spin_unlock_bh(¤t_lb->lock);
+ spin_unlock_irqrestore(¤t_lb->lock, flags);
wake_up_interruptible(&crypto_lb_wait_queue);
}
@@ -265,6 +271,7 @@
struct crypto_device *crypto_lb_find_device(struct crypto_session_initializer *ci)
{
struct crypto_device *dev;
+ unsigned long flags;
if (!current_lb)
return NULL;
@@ -273,7 +280,7 @@
{
int found = 0;
- spin_lock_bh(crypto_device_lock);
+ spin_lock_irqsave(crypto_device_lock, flags);
list_for_each_entry(dev, crypto_device_list, cdev_entry)
{
if (dev->id == ci->bdev)
@@ -282,12 +289,12 @@
break;
}
}
- spin_unlock_bh(crypto_device_lock);
+ spin_unlock_irqrestore(crypto_device_lock, flags);
return (found)?dev:NULL;
}
- spin_lock_bh(¤t_lb->lock);
+ spin_lock_irqsave(¤t_lb->lock, flags);
current_lb->rehash(current_lb);
@@ -298,7 +305,7 @@
crypto_device_get(dev);
spin_unlock(crypto_device_lock);
- spin_unlock_bh(¤t_lb->lock);
+ spin_unlock_irqrestore(¤t_lb->lock, flags);
wake_up_interruptible(&crypto_lb_wait_queue);
@@ -356,8 +363,9 @@
{
struct crypto_lb *__lb;
int err;
+ unsigned long flags;
- spin_lock_bh(&crypto_lb_lock);
+ spin_lock_irqsave(&crypto_lb_lock, flags);
list_for_each_entry(__lb, &crypto_lb_list, lb_entry)
{
if (unlikely(compare_lb(__lb, lb)))
@@ -369,14 +377,14 @@
}
list_add(&lb->lb_entry, &crypto_lb_list);
- spin_unlock_bh(&crypto_lb_lock);
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
err = __crypto_lb_register(lb);
if (err)
{
- spin_lock_bh(&crypto_lb_lock);
+ spin_lock_irqsave(&crypto_lb_lock, flags);
list_del(&lb->lb_entry);
- spin_unlock_bh(&crypto_lb_lock);
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
return err;
}
@@ -400,6 +408,7 @@
void crypto_lb_unregister(struct crypto_lb *lb)
{
struct crypto_lb *__lb, *n;
+ unsigned long flags;
if (lb_num == 1)
{
@@ -419,7 +428,7 @@
__crypto_lb_unregister(lb);
- spin_lock_bh(&crypto_lb_lock);
+ spin_lock_irqsave(&crypto_lb_lock, flags);
list_for_each_entry_safe(__lb, n, &crypto_lb_list, lb_entry)
{
if (compare_lb(__lb, lb))
@@ -437,7 +446,7 @@
lb_set_current(default_lb);
}
}
- spin_unlock_bh(&crypto_lb_lock);
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
}
static void crypto_lb_queue_wrapper(void *data)
@@ -490,15 +499,16 @@
{
struct crypto_route *rt;
struct crypto_device *dev;
+ unsigned long flags;
rt = crypto_route_dequeue(s);
if (rt)
{
dev = rt->dev;
- spin_lock_bh(&dev->session_lock);
+ spin_lock_irqsave(&dev->session_lock, flags);
list_del(&s->dev_queue_entry);
- spin_unlock_bh(&dev->session_lock);
+ spin_unlock_irqrestore(&dev->session_lock, flags);
crypto_route_free(rt);
dev = crypto_route_get_current_device(s);
@@ -523,13 +533,14 @@
{
struct crypto_session *s, *n;
struct crypto_device *dev = (struct crypto_device *)data;
+ unsigned long flags;
daemonize("%s", dev->name);
allow_signal(SIGTERM);
while (!need_exit)
{
- spin_lock_bh(&dev->session_lock);
+ spin_lock_irqsave(&dev->session_lock, flags);
list_for_each_entry_safe(s, n, &dev->session_list, main_queue_entry)
{
printk("session %llu [%llu]: flags=%x, route_num=%d, %s,%s,%s,%s.\n",
@@ -572,7 +583,7 @@
unlock:
spin_unlock(&s->lock);
}
- spin_unlock_bh(&dev->session_lock);
+ spin_unlock_irqrestore(&dev->session_lock, flags);
interruptible_sleep_on_timeout(&crypto_lb_wait_queue, 1000);
}
--- orig/crypto_dev.c
+++ mod/crypto_dev.c
@@ -112,13 +112,14 @@
{
struct crypto_device *d;
int off = 0;
+ unsigned long flags;
- spin_lock_bh(&cdev_lock);
+ spin_lock_irqsave(&cdev_lock, flags);
list_for_each_entry(d, &cdev_list, cdev_entry)
{
off += sprintf(buf+off, "%s ", d->name);
}
- spin_unlock_bh(&cdev_lock);
+ spin_unlock_irqrestore(&cdev_lock, flags);
if (!off)
off = sprintf(buf, "No devices registered yet.");
@@ -227,8 +228,9 @@
{
struct crypto_device *dev;
int found = 0;
+ unsigned long flags;
- spin_lock_bh(&cdev_lock);
+ spin_lock_irqsave(&cdev_lock, flags);
list_for_each_entry(dev, &cdev_list, cdev_entry)
{
if (!strcmp(dev->name, name))
@@ -238,7 +240,7 @@
break;
}
}
- spin_unlock_bh(&cdev_lock);
+ spin_unlock_irqrestore(&cdev_lock, flags);
if (!found)
return NULL;
@@ -305,15 +307,16 @@
int crypto_device_add(struct crypto_device *dev)
{
int err;
+ unsigned long flags;
err = __crypto_device_add(dev);
if (err)
return err;
- spin_lock_bh(&cdev_lock);
+ spin_lock_irqsave(&cdev_lock, flags);
list_add(&dev->cdev_entry, &cdev_list);
dev->id = ++cdev_ids;
- spin_unlock_bh(&cdev_lock);
+ spin_unlock_irqrestore(&cdev_lock, flags);
dprintk(KERN_INFO "Crypto device %s was registered with ID=%x.\n", dev->name, dev->id);
--- orig/crypto_main.c
+++ mod/crypto_main.c
@@ -86,8 +86,9 @@
inline void crypto_session_insert_main(struct crypto_device *dev, struct crypto_session *s)
{
struct crypto_session *__s;
+ unsigned long flags;
- spin_lock_bh(&dev->session_lock);
+ spin_lock_irqsave(&dev->session_lock, flags);
crypto_device_get(dev);
if (unlikely(list_empty(&dev->session_list)))
{
@@ -110,7 +111,7 @@
if (!inserted)
list_add_tail(&s->main_queue_entry, &dev->session_list);
}
- spin_unlock_bh(&dev->session_lock);
+ spin_unlock_irqrestore(&dev->session_lock, flags);
dump_ci(&s->ci);
dprintk(" added to main crypto device %s [%d].\n", dev->name, atomic_read(&dev->refcnt));
@@ -118,9 +119,11 @@
inline void crypto_session_insert(struct crypto_device *dev, struct crypto_session *s)
{
- spin_lock_bh(&dev->session_lock);
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->session_lock, flags);
__crypto_session_insert(dev, s);
- spin_unlock_bh(&dev->session_lock);
+ spin_unlock_irqrestore(&dev->session_lock, flags);
if (dev->data_ready)
dev->data_ready(dev);
@@ -199,6 +202,7 @@
{
struct crypto_route *rt;
struct crypto_device *dev;
+ unsigned long flags;
BUG_ON(crypto_route_queue_len(s) > 1);
@@ -208,9 +212,9 @@
dprintk(KERN_INFO "Removing route entry for device %s.\n", dev->name);
- spin_lock_bh(&dev->session_lock);
+ spin_lock_irqsave(&dev->session_lock, flags);
list_del(&s->dev_queue_entry);
- spin_unlock_bh(&dev->session_lock);
+ spin_unlock_irqrestore(&dev->session_lock, flags);
crypto_route_free(rt);
}
@@ -227,10 +231,11 @@
inline void crypto_session_dequeue_main(struct crypto_session *s)
{
struct crypto_device *dev = &main_crypto_device;
+ unsigned long flags;
- spin_lock_bh(&dev->session_lock);
+ spin_lock_irqsave(&dev->session_lock, flags);
__crypto_session_dequeue_main(s);
- spin_unlock_bh(&dev->session_lock);
+ spin_unlock_irqrestore(&dev->session_lock, flags);
}
int __devinit cmain_init(void)
--- orig/crypto_route.h
+++ mod/crypto_route.h
@@ -77,13 +77,14 @@
static inline void crypto_route_del(struct crypto_route *rt)
{
struct crypto_route_head *list = rt->list;
+ unsigned long flags;
if (list)
{
- spin_lock_bh(&list->lock);
+ spin_lock_irqsave(&list->lock, flags);
if (list == rt->list)
__crypto_route_del(rt, rt->list);
- spin_unlock_bh(&list->lock);
+ spin_unlock_irqrestore(&list->lock, flags);
crypto_route_free(rt);
}
@@ -111,10 +112,11 @@
static inline struct crypto_route *crypto_route_dequeue(struct crypto_session *s)
{
struct crypto_route *rt;
+ unsigned long flags;
- spin_lock_bh(&s->route_list.lock);
+ spin_lock_irqsave(&s->route_list.lock, flags);
rt = __crypto_route_dequeue(&s->route_list);
- spin_unlock_bh(&s->route_list.lock);
+ spin_unlock_irqrestore(&s->route_list.lock, flags);
return rt;
}
@@ -134,9 +136,11 @@
static inline void crypto_route_queue(struct crypto_route *rt, struct crypto_session *s)
{
- spin_lock_bh(&s->route_list.lock);
+ unsigned long flags;
+
+ spin_lock_irqsave(&s->route_list.lock, flags);
__crypto_route_queue(rt, &s->route_list);
- spin_unlock_bh(&s->route_list.lock);
+ spin_unlock_irqrestore(&s->route_list.lock, flags);
}
static inline int crypto_route_add(struct crypto_device *dev, struct crypto_session *s, struct crypto_session_initializer *ci)
@@ -181,14 +185,15 @@
{
struct crypto_route_head *list;
struct crypto_route *rt = NULL;
+ unsigned long flags;
list = &s->route_list;
if (list)
{
- spin_lock_bh(&list->lock);
+ spin_lock_irqsave(&list->lock, flags);
rt = __crypto_route_current(list);
- spin_unlock_bh(&list->lock);
+ spin_unlock_irqrestore(&list->lock, flags);
}
return rt;
@@ -199,16 +204,16 @@
struct crypto_route *rt = NULL;
struct crypto_device *dev = NULL;
struct crypto_route_head *list = &s->route_list;
+ unsigned long flags;
- spin_lock_bh(&list->lock);
+ spin_lock_irqsave(&list->lock, flags);
rt = __crypto_route_current(list);
if (rt)
{
dev = rt->dev;
crypto_device_get(dev);
}
- spin_unlock_bh(&list->lock);
-
+ spin_unlock_irqrestore(&list->lock, flags);
return dev;
}
--- orig/crypto_stat.c
+++ mod/crypto_stat.c
@@ -34,13 +34,14 @@
inline void crypto_stat_start_inc(struct crypto_session *s)
{
struct crypto_device *dev;
+ unsigned long flags;
dev = crypto_route_get_current_device(s);
if (dev)
{
- spin_lock_bh(&dev->stat_lock);
+ spin_lock_irqsave(&dev->stat_lock, flags);
dev->stat.sstarted++;
- spin_unlock_bh(&dev->stat_lock);
+ spin_unlock_irqrestore(&dev->stat_lock, flags);
crypto_device_put(dev);
}
@@ -49,13 +50,14 @@
inline void crypto_stat_finish_inc(struct crypto_session *s)
{
struct crypto_device *dev;
+ unsigned long flags;
dev = crypto_route_get_current_device(s);
if (dev)
{
- spin_lock_bh(&dev->stat_lock);
+ spin_lock_irqsave(&dev->stat_lock, flags);
dev->stat.sfinished++;
- spin_unlock_bh(&dev->stat_lock);
+ spin_unlock_irqrestore(&dev->stat_lock, flags);
crypto_device_put(dev);
}
@@ -64,13 +66,14 @@
inline void crypto_stat_complete_inc(struct crypto_session *s)
{
struct crypto_device *dev;
+ unsigned long flags;
dev = crypto_route_get_current_device(s);
if (dev)
{
- spin_lock_bh(&dev->stat_lock);
+ spin_lock_irqsave(&dev->stat_lock, flags);
dev->stat.scompleted++;
- spin_unlock_bh(&dev->stat_lock);
+ spin_unlock_irqrestore(&dev->stat_lock, flags);
crypto_device_put(dev);
}
* looking for johnpol@2ka.mipt.ru-2004/crypto--main--0--patch-52 to compare with
* comparing to johnpol@2ka.mipt.ru-2004/crypto--main--0--patch-52
M crypto_main.c
M acrypto.h
M crypto_lb.c
M crypto_conn.c
* modified files
--- orig/acrypto.h
+++ mod/acrypto.h
@@ -104,7 +104,7 @@
#include <asm/scatterlist.h>
-#define DEBUG
+//#define DEBUG
#ifdef DEBUG
#define dprintk(f, a...) printk(KERN_EMERG f, ##a)
#else
--- orig/crypto_conn.c
+++ mod/crypto_conn.c
@@ -44,6 +44,7 @@
struct crypto_conn_data *d, *cmd;
struct crypto_device *dev;
u32 sessions;
+ unsigned long flags;
msg = (struct cn_msg *)data;
d = (struct crypto_conn_data *)msg->data;
@@ -116,7 +117,7 @@
ptr = (struct crypto_session_initializer *)(cmd+1);
sessions = 0;
- spin_lock_bh(&dev->session_lock);
+ spin_lock_irqsave(&dev->session_lock, flags);
list_for_each_entry(s, &dev->session_list, dev_queue_entry)
{
memcpy(ptr, &s->ci, sizeof(*ptr));
@@ -126,7 +127,7 @@
if (sessions >= 1024)
break;
}
- spin_unlock_bh(&dev->session_lock);
+ spin_unlock_irqrestore(&dev->session_lock, flags);
cmd->len = sizeof(*ptr)*sessions;
reply->len = sizeof(*cmd) + cmd->len;
--- orig/crypto_lb.c
+++ mod/crypto_lb.c
@@ -370,7 +370,7 @@
{
if (unlikely(compare_lb(__lb, lb)))
{
- spin_unlock_bh(&crypto_lb_lock);
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
dprintk(KERN_ERR "Crypto load balancer %s is already registered.\n", lb->name);
return -EINVAL;
}
@@ -543,7 +543,7 @@
spin_lock_irqsave(&dev->session_lock, flags);
list_for_each_entry_safe(s, n, &dev->session_list, main_queue_entry)
{
- printk("session %llu [%llu]: flags=%x, route_num=%d, %s,%s,%s,%s.\n",
+ dprintk("session %llu [%llu]: flags=%x, route_num=%d, %s,%s,%s,%s.\n",
s->ci.id, s->ci.dev_id, s->ci.flags,
crypto_route_queue_len(s),
(session_completed(s))?"completed": "not completed",
--- orig/crypto_main.c
+++ mod/crypto_main.c
@@ -169,8 +169,6 @@
if (d->priv)
memcpy(s->data.priv, d->priv, d->priv_size);
}
- else
- s->data.priv = NULL;
s->ci.id = dev->sid++;
s->ci.dev_id = ldev->sid++;
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 15:27 ` Evgeniy Polyakov
@ 2004-10-30 20:39 ` Eugene Surovegin
2004-10-30 21:17 ` Evgeniy Polyakov
0 siblings, 1 reply; 57+ messages in thread
From: Eugene Surovegin @ 2004-10-30 20:39 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: Michal Ludvig, netdev, cryptoapi
On Fri, Oct 29, 2004 at 07:27:41PM +0400, Evgeniy Polyakov wrote:
> but it was designed to be usefull only in custom MX box,
This is not true. I coded it using Hifn PCI card plugged in 440GP eval
board as well in P4 2.6 GHz box.
Driver isn't perfect, it's quite simple, I suspended all work on it
because without async crypto it doesn't make a lot of sense.
> so it has some design notes that I do not agree with.
And they are (I know there are problems, just curious which ones you
"don't agree with") ?
--
Euege
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 20:17 ` Michal Ludvig
@ 2004-10-30 20:56 ` Evgeniy Polyakov
0 siblings, 0 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-30 20:56 UTC (permalink / raw)
To: Michal Ludvig; +Cc: netdev, cryptoapi
On Sat, 30 Oct 2004 22:17:25 +0200 (CEST)
Michal Ludvig <michal@logix.cz> wrote:
> On Sat, 30 Oct 2004, Evgeniy Polyakov wrote:
>
> > On Sat, 30 Oct 2004 18:57:20 +0200 (CEST)
> > Michal Ludvig <michal@logix.cz> wrote:
> >
> > > I have compiled and booted with your patches. Not too much
> > > playing around so far. Anyway some quick observations:
> >
> > I've wrote mega block device which can work(without hang) only
> > with 10mb "disks", but what did you expect from midnight hack after
> > several Staropramen's :)
>
> 10MB is probably too small to get some reliable numbers. The testing
> should last for at least couple of seconds...
It was repeated of course.
On one of my very little test machine 10 times of 10mb in a two threads
took about 40-42 secs.
Test was like this:
for ((i=0; i<10; ++i)); do
dd if=/dev/zero of=/mnt/file bs=1M count=10
rm -f /mnt/file
done
Where /mnt is /dev/bd0 mount point.
Above script was run from two processes.
Above block device is just _very_ simple in-memory-only block device.
I decided to not change loopback or ramdisk, it was done to remove disk
latencies and have more fair results.
> > Give me several minutes to complete async_provider.c and I will post
> > numbers, which I expect will be the same for both sync and async
> > driver since I do not have neither SMP(which should benefit I believe)
> > nor crypto card here.
>
> At least try it with a SMP kernel (on UP machine) to catch the worst
> spindeadlocks etc. In the ideal case the acrypto core should know that
> there are two (four, ...) CPUs and all of them can encrypt in paralel.
I will get access to the real SMP machine only ater tomorrow.
Of course I test acrypto with preempt and SMP kernel to find different
kind locking issues.
acrypto's callbacks are run from queue_work context so they can be scheduled to
diferent CPU, but noone forbids to run 2/4/any synchronous threads and
register them as crypto providers with the same capabilities.
It is very easy with acrypto to balance such kind of load by design.
> > > - "rmmod simple_lb" hangs because:
> > > | <6>You are removing crypto load balancer simple_lb which is current and
> > > | default.
> > > | There is no other crypto load balancers. Removing is delayed untill new
> > > | load balancer is registered.
> > > Of course I can't re-add it because the module is still there. In this
> > > case rmmod should fail with a gentle message (-EBUSY) instead of hang. Or
> > > let it go if not in use.
> >
> > acrypto module has parameter force_lb_remove which if is set allows to remove
> > last crypto load balancer.
> > But ut has atricky moment - since any crypto_session_alloc() may occur
> > asyncronously then we can not say if there are any load balancer users
> > without some locks. Currently if there is no any crypto load balancer, then
> > acrypto will catch BUG_ON().
>
> BUG_ON()? Shouldn't crypto_session_alloc() fail instead? BUG_ON() is for
> catching conditions that really shouldn't happen in normal operation.
> Missing module with a load balancer won't be that unusual...
It is impossible and very bad condition when sessions want to be allocated
but there are no even load balancer.
To be honest it is _supposed_ to not happen, that is why it is BUG_ON() there,
but you are right it is too rude, I will create a patch tomorrow which
will allow crypto_session_alloc() fail if there is no load balancer
like it is done when there is no appropriate crypto device.
> Or should the acrypto.ko module prerequire a balancer to get it loaded
> automatically?
My first idea was to insert simple_lb as part of acrypto.
And then any other load balancer can be removed with fallback to simple_lb,
but it is not fair enough I think.
But it is still can be easily implemented, if we decide to do it.
> Michal Ludvig
> --
> * A mouse is a device used to point at the xterm you want to type in.
> * Personal homepage - http://www.logix.cz/michal
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 21:04 ` Evgeniy Polyakov
@ 2004-10-30 20:56 ` Eugene Surovegin
2004-10-30 21:24 ` Evgeniy Polyakov
0 siblings, 1 reply; 57+ messages in thread
From: Eugene Surovegin @ 2004-10-30 20:56 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: hadi, netdev, cryptoapi
On Sun, Oct 31, 2004 at 01:04:15AM +0400, Evgeniy Polyakov wrote:
> On Sat, 30 Oct 2004 13:35:50 -0700
> Eugene Surovegin <ebs@ebshome.net> wrote:
>
> > On Fri, Oct 29, 2004 at 06:06:52PM +0400, Evgeniy Polyakov wrote:
> > > If we have a hardware accelerator chip, than we _already_ have improvements
> > > with even the worst async crypto layer, since software and hardware
> > > will work in parrallel.
> >
> > This is not true.
> >
> > For example, if chip request setup and PCI transfer takes more than
> > just using sw implementation. This is reality for AES and short
> > packets.
>
> You have dataflow of packets, if invoce expenses take less time
> then you will win.
Yes, but as I said for short packets, AES and fast box for example
this might not be true.
> More than 1kbyte of data to be encrypted already beats time
> need for software encryption on 266 mhz.
OK, what about 64 byte packets? I'm not saying hw crypto is useless,
what I'm saying is it's not that obvious that having hw crypto will
help in _all_ situations.
> And what about asynchronous crypto?
Asynchronous? Never heard of it. Did you mean asymmetric?
If yes, ok, I _think_ hw crypto will help, provided we have kernel API
for such stuff.
--
Eugene
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 20:35 ` Eugene Surovegin
@ 2004-10-30 21:04 ` Evgeniy Polyakov
2004-10-30 20:56 ` Eugene Surovegin
0 siblings, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-30 21:04 UTC (permalink / raw)
To: Eugene Surovegin; +Cc: hadi, netdev, cryptoapi
On Sat, 30 Oct 2004 13:35:50 -0700
Eugene Surovegin <ebs@ebshome.net> wrote:
> On Fri, Oct 29, 2004 at 06:06:52PM +0400, Evgeniy Polyakov wrote:
> > If we have a hardware accelerator chip, than we _already_ have improvements
> > with even the worst async crypto layer, since software and hardware
> > will work in parrallel.
>
> This is not true.
>
> For example, if chip request setup and PCI transfer takes more than
> just using sw implementation. This is reality for AES and short
> packets.
You have dataflow of packets, if invoce expenses take less time
then you will win.
More than 1kbyte of data to be encrypted already beats time
need for software encryption on 266 mhz.
What hardware requires PCI transfer that will take more time
than it's encryption?
And what about asynchronous crypto?
Kernel currently even can not generate RSA keys, although
necessity of such feature is under the question.
>
> --
> Eugene
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 21:17 ` Evgeniy Polyakov
@ 2004-10-30 21:09 ` Eugene Surovegin
2004-10-30 21:46 ` Evgeniy Polyakov
0 siblings, 1 reply; 57+ messages in thread
From: Eugene Surovegin @ 2004-10-30 21:09 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: Michal Ludvig, netdev, cryptoapi
On Sun, Oct 31, 2004 at 01:17:19AM +0400, Evgeniy Polyakov wrote:
> On Sat, 30 Oct 2004 13:39:06 -0700
> Eugene Surovegin <ebs@ebshome.net> wrote:
>
> > On Fri, Oct 29, 2004 at 07:27:41PM +0400, Evgeniy Polyakov wrote:
> > > but it was designed to be usefull only in custom MX box,
> >
> > This is not true. I coded it using Hifn PCI card plugged in 440GP eval
> > board as well in P4 2.6 GHz box.
> >
> > Driver isn't perfect, it's quite simple, I suspended all work on it
> > because without async crypto it doesn't make a lot of sense.
> >
> > > so it has some design notes that I do not agree with.
> >
> > And they are (I know there are problems, just curious which ones you
> > "don't agree with") ?
>
> It supports only one card,
Yes, and comment says it trivially fixed :)
> it has very nontrivial locking,
locking is really simple, it was obviously done for current sync
crypto layer.
> which may(I'm not sure after just finished diagonal review) lead to
> too big latencies,
Huh?
> hifn_cipher() can not work with any other crypto design,
Well, this funny one :). Of course it cannot, because none existed
when I wrote it.
If your async stuff will be accepted into the official kernel tree
I'll re-write it ASAP.
> I doubt it can work with other card capabilities other then you
> have implemented without major rewrite.
Hmm, what do you mean "other" card? It's written for particular
Hifn chip. I wrote it for IPSec acceleration in mind. Some algorithms
aren't supported (compression for example), because we don't use
them.
> I'm quite sure it is good driver for current crypto schema, but if
> acrypto will be committed it could not work without complete rewrite.
Yes, that's true :). That's the reason I never officially announced it,
because current one is quite trivial, and there are a lot of things
which can be improved upon, after async crypto is in, and IPSec uses
it.
--
Eugene
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 20:39 ` Eugene Surovegin
@ 2004-10-30 21:17 ` Evgeniy Polyakov
2004-10-30 21:09 ` Eugene Surovegin
0 siblings, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-30 21:17 UTC (permalink / raw)
To: Eugene Surovegin; +Cc: Michal Ludvig, netdev, cryptoapi
On Sat, 30 Oct 2004 13:39:06 -0700
Eugene Surovegin <ebs@ebshome.net> wrote:
> On Fri, Oct 29, 2004 at 07:27:41PM +0400, Evgeniy Polyakov wrote:
> > but it was designed to be usefull only in custom MX box,
>
> This is not true. I coded it using Hifn PCI card plugged in 440GP eval
> board as well in P4 2.6 GHz box.
>
> Driver isn't perfect, it's quite simple, I suspended all work on it
> because without async crypto it doesn't make a lot of sense.
>
> > so it has some design notes that I do not agree with.
>
> And they are (I know there are problems, just curious which ones you
> "don't agree with") ?
It supports only one card, it has very nontrivial locking, which may(I'm
not sure after just finished diagonal review) lead to too big latencies,
hifn_cipher() can not work with any other crypto design, I doubt it can
work with other card capabilities other then you have implemented without
major rewrite.
I'm quite sure it is good driver for current crypto schema, but if
acrypto will be committed it could not work without complete rewrite.
> --
> Euege
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 20:56 ` Eugene Surovegin
@ 2004-10-30 21:24 ` Evgeniy Polyakov
2004-10-30 23:41 ` jamal
0 siblings, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-30 21:24 UTC (permalink / raw)
To: Eugene Surovegin; +Cc: hadi, netdev, cryptoapi
On Sat, 30 Oct 2004 13:56:31 -0700
Eugene Surovegin <ebs@ebshome.net> wrote:
> On Sun, Oct 31, 2004 at 01:04:15AM +0400, Evgeniy Polyakov wrote:
> > On Sat, 30 Oct 2004 13:35:50 -0700
> > Eugene Surovegin <ebs@ebshome.net> wrote:
> >
> > > On Fri, Oct 29, 2004 at 06:06:52PM +0400, Evgeniy Polyakov wrote:
> > > > If we have a hardware accelerator chip, than we _already_ have improvements
> > > > with even the worst async crypto layer, since software and hardware
> > > > will work in parrallel.
> > >
> > > This is not true.
> > >
> > > For example, if chip request setup and PCI transfer takes more than
> > > just using sw implementation. This is reality for AES and short
> > > packets.
> >
> > You have dataflow of packets, if invoce expenses take less time
> > then you will win.
>
> Yes, but as I said for short packets, AES and fast box for example
> this might not be true.
>
> > More than 1kbyte of data to be encrypted already beats time
> > need for software encryption on 266 mhz.
>
> OK, what about 64 byte packets? I'm not saying hw crypto is useless,
> what I'm saying is it's not that obvious that having hw crypto will
> help in _all_ situations.
Sure.
For such cases I offered "rate" or "speed" parameter in my first e-mail,
but it can be implicitly implemented by queue length.
With 1 big datflow of packets with high priority and little dataflow of
packets with little priority if we will catch high priority packets by SW
and thus can have situation when we will _never_ catch low-priority packets,
with even very slow HW we can route all those low-priority packets into it,
at least they will be processed sometimes...
> > And what about asynchronous crypto?
>
> Asynchronous? Never heard of it. Did you mean asymmetric?
:)
> If yes, ok, I _think_ hw crypto will help, provided we have kernel API
> for such stuff.
>
> --
> Eugene
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 21:09 ` Eugene Surovegin
@ 2004-10-30 21:46 ` Evgeniy Polyakov
0 siblings, 0 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-30 21:46 UTC (permalink / raw)
To: Eugene Surovegin; +Cc: Michal Ludvig, netdev, cryptoapi
On Sat, 30 Oct 2004 14:09:15 -0700
Eugene Surovegin <ebs@ebshome.net> wrote:
> On Sun, Oct 31, 2004 at 01:17:19AM +0400, Evgeniy Polyakov wrote:
> > On Sat, 30 Oct 2004 13:39:06 -0700
> > Eugene Surovegin <ebs@ebshome.net> wrote:
> >
> > > On Fri, Oct 29, 2004 at 07:27:41PM +0400, Evgeniy Polyakov wrote:
> > > > but it was designed to be usefull only in custom MX box,
> > >
> > > This is not true. I coded it using Hifn PCI card plugged in 440GP eval
> > > board as well in P4 2.6 GHz box.
> > >
> > > Driver isn't perfect, it's quite simple, I suspended all work on it
> > > because without async crypto it doesn't make a lot of sense.
> > >
> > > > so it has some design notes that I do not agree with.
> > >
> > > And they are (I know there are problems, just curious which ones you
> > > "don't agree with") ?
> >
> > It supports only one card,
>
> Yes, and comment says it trivially fixed :)
>
> > it has very nontrivial locking,
>
> locking is really simple, it was obviously done for current sync
> crypto layer.
Current locking just can not be used tu support several sessions to be
processed in a parallel, although hardware support it, not?
Probably it is not driver problem, but what do we try to figure out?
With current crypto it can work - good, with acrypto it must be completely
rewritten.
> > which may(I'm not sure after just finished diagonal review) lead to
> > too big latencies,
>
> Huh?
neither will hifn_cipher() busy-wait until request processing will be finished
if it happens in interrupt context?
> > hifn_cipher() can not work with any other crypto design,
>
> Well, this funny one :). Of course it cannot, because none existed
> when I wrote it.
It was written _only_ for syncronous events.
I think only HW initialization bits can be copied into future version,
but again, what do we discuss and/or accuse? :)
> If your async stuff will be accepted into the official kernel tree
> I'll re-write it ASAP.
>
> > I doubt it can work with other card capabilities other then you
> > have implemented without major rewrite.
>
> Hmm, what do you mean "other" card? It's written for particular
> Hifn chip. I wrote it for IPSec acceleration in mind. Some algorithms
> aren't supported (compression for example), because we don't use
> them.
I mean other capabilities not card.
> > I'm quite sure it is good driver for current crypto schema, but if
> > acrypto will be committed it could not work without complete rewrite.
>
> Yes, that's true :). That's the reason I never officially announced it,
> because current one is quite trivial, and there are a lot of things
> which can be improved upon, after async crypto is in, and IPSec uses
> it.
>
> --
> Eugene
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 21:24 ` Evgeniy Polyakov
@ 2004-10-30 23:41 ` jamal
2004-10-31 9:13 ` Evgeniy Polyakov
0 siblings, 1 reply; 57+ messages in thread
From: jamal @ 2004-10-30 23:41 UTC (permalink / raw)
To: johnpol; +Cc: Eugene Surovegin, netdev, cryptoapi
On Sat, 2004-10-30 at 17:24, Evgeniy Polyakov wrote:
> On Sat, 30 Oct 2004 13:56:31 -0700
> Eugene Surovegin <ebs@ebshome.net> wrote:
> > OK, what about 64 byte packets? I'm not saying hw crypto is useless,
> > what I'm saying is it's not that obvious that having hw crypto will
> > help in _all_ situations.
>
> Sure.
> For such cases I offered "rate" or "speed" parameter in my first e-mail,
> but it can be implicitly implemented by queue length.
> With 1 big datflow of packets with high priority and little dataflow of
> packets with little priority if we will catch high priority packets by SW
> and thus can have situation when we will _never_ catch low-priority packets,
> with even very slow HW we can route all those low-priority packets into it,
> at least they will be processed sometimes...
Can you explain the "rate" or "speed" parameter ?
I havent studied your code, however, what Eugene is pointing out is
valuable detail/feedback.
You should have in your queuing towards the crypto chip ability to
batch. i.e sort of nagle-like "wait until we have 10 packets/20KB or 20
jiffies" before you send everything in the queue to the chip.
As he points out (and i am sure he can back it with data ;->), that
given the setup cost, packet size, algo and CPU and bus speed, it may
not make sense to use the chip at all ;->
cheers,
jamal
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 5:20 ` Evgeniy Polyakov
@ 2004-10-31 5:43 ` James Morris
0 siblings, 0 replies; 57+ messages in thread
From: James Morris @ 2004-10-31 5:43 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: netdev, cryptoapi
On Sat, 30 Oct 2004, Evgeniy Polyakov wrote:
> > Also, please consider EXPORT_SYMBOL_GPL instead of EXPORT_SYMBOL.
>
> I'm not insisting, but why?
The existing crypto API uses EXPORT_SYMBOL_GPL, and I would hope that
extensions to it do the same. This area is contentious in general, but
ultimately, I think we should document that these APIs are not intended to
be used by proprietary modules to do things like plug into the kernel and
extend it in proprietary ways.
- James
--
James Morris
<jmorris@redhat.com>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 20:00 ` Sam Leffler
@ 2004-10-31 6:09 ` James Morris
2004-10-31 6:35 ` Sam Leffler
0 siblings, 1 reply; 57+ messages in thread
From: James Morris @ 2004-10-31 6:09 UTC (permalink / raw)
To: Sam Leffler; +Cc: johnpol, jamal, netdev, cryptoapi, David S. Miller
On Fri, 29 Oct 2004, Sam Leffler wrote:
> And a paper I wrote explains how I fixed many of the problems with the
> openbsd code in freebsd.
>
> http://www.usenix.org/publications/library/proceedings/bsdcon03/tech/leffler_crypto/leffler_crpto_html/
Thanks, that's a useful reference.
For handling overload of crypto hardware, I was thinking of falling back
to software processing (really, I guess this is just load balancing
between hardware and software).
- James
--
James Morris
<jmorris@redhat.com>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-31 6:09 ` James Morris
@ 2004-10-31 6:35 ` Sam Leffler
0 siblings, 0 replies; 57+ messages in thread
From: Sam Leffler @ 2004-10-31 6:35 UTC (permalink / raw)
To: James Morris; +Cc: johnpol, jamal, netdev, cryptoapi, David S. Miller
James Morris wrote:
> On Fri, 29 Oct 2004, Sam Leffler wrote:
>
>
>>And a paper I wrote explains how I fixed many of the problems with the
>>openbsd code in freebsd.
>>
>>http://www.usenix.org/publications/library/proceedings/bsdcon03/tech/leffler_crypto/leffler_crpto_html/
>
>
> Thanks, that's a useful reference.
>
> For handling overload of crypto hardware, I was thinking of falling back
> to software processing (really, I guess this is just load balancing
> between hardware and software).
I did a prototype scheduler (never committed) that took into account
per-device performance metrics and outstanding operations. The s/w
crypto driver was just part of the mix. As a result fallback on
overload and using s/w crypto for small operands (where the setup
overhead is too costly) just fell out.
Sam
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-30 23:41 ` jamal
@ 2004-10-31 9:13 ` Evgeniy Polyakov
2004-10-31 10:46 ` Michal Ludvig
2004-10-31 14:56 ` jamal
0 siblings, 2 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-10-31 9:13 UTC (permalink / raw)
To: hadi; +Cc: Eugene Surovegin, netdev, cryptoapi
On 30 Oct 2004 19:41:27 -0400
jamal <hadi@cyberus.ca> wrote:
> On Sat, 2004-10-30 at 17:24, Evgeniy Polyakov wrote:
> > On Sat, 30 Oct 2004 13:56:31 -0700
> > Eugene Surovegin <ebs@ebshome.net> wrote:
>
>
> > > OK, what about 64 byte packets? I'm not saying hw crypto is useless,
> > > what I'm saying is it's not that obvious that having hw crypto will
> > > help in _all_ situations.
> >
> > Sure.
> > For such cases I offered "rate" or "speed" parameter in my first e-mail,
> > but it can be implicitly implemented by queue length.
> > With 1 big datflow of packets with high priority and little dataflow of
> > packets with little priority if we will catch high priority packets by SW
> > and thus can have situation when we will _never_ catch low-priority packets,
> > with even very slow HW we can route all those low-priority packets into it,
> > at least they will be processed sometimes...
>
> Can you explain the "rate" or "speed" parameter ?
Driver writer can set "rate" parameter to any number from 0 to 64k -
and it will show speed of this driver in current mode/type/operation.
For example:
struct crypto_capability cap[] = {
{
.mode = CRYPTO_MODE_EBC,
.operation = CRYPTO_OP_ENCRYPT,
.type = CRYPTO_TYPE_AES_128,
.qlen = 1000,
.rate = 10000
},
{
.mode = CRYPTO_MODE_EBC,
.operation = CRYPTO_OP_ENCRYPT,
.type = CRYPTO_TYPE_DES,
.qlen = 1000,
.rate = 65000
},
and so on.
That mean that this driver perform des 6 time faster than aes, but
it should be fair numbers and somehow measured and compared to other
drivers.
This also can be achieved by qlen parameter - if driver writer sets
it to bigger walue then in this mode driver/hardware works faster.
But driver writer can set qlen in a too big value just because
it want it to be such without any means of driver/hardware capabilities.
It is not forbidden.
And the last and the most interesting one is following:
we create per session initialiser parameter "session_processin_time"
which will be sum of the time slices when driver performed operation
on session, since we alredy has "scomplete" paramenter which is equal
to the amount of completed(processed) session then we have _fair_ speed
of the driver/hardware in a given mode/operation/type.
Of course load blancer should select device with the lowest
session_processing_time/scompleted.
I think third variant is what we want. I will think of it some more
and will implement soon.
> I havent studied your code, however, what Eugene is pointing out is
> valuable detail/feedback.
>
> You should have in your queuing towards the crypto chip ability to
> batch. i.e sort of nagle-like "wait until we have 10 packets/20KB or 20
> jiffies" before you send everything in the queue to the chip.
That is exactly how crypto driver should be written.
Driver has it's queue and number of session in it, so it and only it
can decide when begin to process them in the most effective way.
> As he points out (and i am sure he can back it with data ;->), that
> given the setup cost, packet size, algo and CPU and bus speed, it may
> not make sense to use the chip at all ;->
Michal has numbers - pure hardware beats soft in a certain setups
in a fully synchronous schema, let's work SW and HW in parallel.
Of course SW can encrypt 64 byte faster than it will be transfered to
old ISA crypto card, but it worth to do it for compressing with LZW
9000 bytes jumbo frame.
>
> cheers,
> jamal
>
>
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-31 9:13 ` Evgeniy Polyakov
@ 2004-10-31 10:46 ` Michal Ludvig
2004-10-31 15:03 ` jamal
2004-10-31 15:36 ` Michael Richardson
2004-10-31 14:56 ` jamal
1 sibling, 2 replies; 57+ messages in thread
From: Michal Ludvig @ 2004-10-31 10:46 UTC (permalink / raw)
To: johnpol; +Cc: hadi, netdev, cryptoapi, Eugene Surovegin
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Evgeniy Polyakov wrote:
>
>>As he points out (and i am sure he can back it with data ;->), that
>>given the setup cost, packet size, algo and CPU and bus speed, it may
>>not make sense to use the chip at all ;->
>
> Michal has numbers - pure hardware beats soft in a certain setups
> in a fully synchronous schema, let's work SW and HW in parallel.
Yes, I have *some* numbers, but consider that they are for quite
eligible setup - encrypting ~1.5k IPsec packets. I should retry with a
much smaller MTU to see the difference...
I think it won't be the programmer but the system administrator who will
have to correctly set priorities and constraints for different
hardware/software engines for the particular system. With a slow CPU it
may be worth to offload even small blocks to hardware, with a fast one
it may be worth to set HW and SW as equal, etc.
Michal Ludvig
- --
SUSE Labs mludvig@suse.cz
(+420) 296.545.373 http://www.suse.cz
Personal homepage http://www.logix.cz/michal
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFBhMKDDDolCcRbIhgRAqhBAKCCYwol3N3pd69yBI1HyEEFH0diOACeOqw6
2yCxYNf91eEG1Omi8uQ+Un4=
=i1UL
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-31 9:13 ` Evgeniy Polyakov
2004-10-31 10:46 ` Michal Ludvig
@ 2004-10-31 14:56 ` jamal
2004-11-01 5:12 ` Evgeniy Polyakov
1 sibling, 1 reply; 57+ messages in thread
From: jamal @ 2004-10-31 14:56 UTC (permalink / raw)
To: johnpol; +Cc: Eugene Surovegin, netdev, cryptoapi
On Sun, 2004-10-31 at 04:13, Evgeniy Polyakov wrote:
> On 30 Oct 2004 19:41:27 -0400
> jamal <hadi@cyberus.ca> wrote:
> > Can you explain the "rate" or "speed" parameter ?
>
> Driver writer can set "rate" parameter to any number from 0 to 64k -
> and it will show speed of this driver in current mode/type/operation.
[..]
>
> That mean that this driver perform des 6 time faster than aes, but
> it should be fair numbers and somehow measured and compared to other
> drivers.
>
So you have some init code that does testing? Or is this factor of six
part of the spec provided by chip vendor?
> This also can be achieved by qlen parameter - if driver writer sets
> it to bigger walue then in this mode driver/hardware works faster.
> But driver writer can set qlen in a too big value just because
> it want it to be such without any means of driver/hardware capabilities.
> It is not forbidden.
>
It is no different than say the way you will do ethernet drivers.
DMA ring sizes and link speeds. harder for ethernet drivers if link
speeds change (Linux net scheduling assumes fixed speed ;->)
> And the last and the most interesting one is following:
> we create per session initialiser parameter "session_processin_time"
> which will be sum of the time slices when driver performed operation
> on session, since we alredy has "scomplete" paramenter which is equal
> to the amount of completed(processed) session then we have _fair_ speed
> of the driver/hardware in a given mode/operation/type.
>
> Of course load blancer should select device with the lowest
> session_processing_time/scompleted.
>
> I think third variant is what we want. I will think of it some more
> and will implement soon.
>
I think you should be able to have multiple, configurable LB algos.
> > I havent studied your code, however, what Eugene is pointing out is
> > valuable detail/feedback.
> >
> > You should have in your queuing towards the crypto chip ability to
> > batch. i.e sort of nagle-like "wait until we have 10 packets/20KB or 20
> > jiffies" before you send everything in the queue to the chip.
>
> That is exactly how crypto driver should be written.
> Driver has it's queue and number of session in it, so it and only it
> can decide when begin to process them in the most effective way.
>
This should be above driver, really.
You should have one or more queues where the scheduler feeds off and
shoves to hardware. Perhaps several levels:
->LB-+
+-> device1 scheduler --> driver queues/rings.
|
|
+-> device2 scheduler --> driver queues/rings.
|
.
.
+-> devicen scheduler --> driver queues/rings.
If you look at the linux traffic control, the stuff below LB is how it
behaves. That should be generic enough to _not_ sit in the driver.
This allows for adding smart algorithms to it; fe: qos, rate limiting,
feedback to LB so it could make smarter decisions etc.
> > As he points out (and i am sure he can back it with data ;->), that
> > given the setup cost, packet size, algo and CPU and bus speed, it may
> > not make sense to use the chip at all ;->
>
> Michal has numbers - pure hardware beats soft in a certain setups
> in a fully synchronous schema, let's work SW and HW in parallel.
>
> Of course SW can encrypt 64 byte faster than it will be transfered to
> old ISA crypto card, but it worth to do it for compressing with LZW
> 9000 bytes jumbo frame.
>
Would be interesting. I have seen the numbers from Eugene and they are
quiet intriguing - but they are for the sync mode.
cheers,
jamal
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-31 10:46 ` Michal Ludvig
@ 2004-10-31 15:03 ` jamal
2004-10-31 16:07 ` James Morris
2004-11-01 6:01 ` Evgeniy Polyakov
2004-10-31 15:36 ` Michael Richardson
1 sibling, 2 replies; 57+ messages in thread
From: jamal @ 2004-10-31 15:03 UTC (permalink / raw)
To: Michal Ludvig; +Cc: johnpol, netdev, cryptoapi, Eugene Surovegin
On Sun, 2004-10-31 at 05:46, Michal Ludvig wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> Yes, I have *some* numbers, but consider that they are for quite
> eligible setup - encrypting ~1.5k IPsec packets. I should retry with a
> much smaller MTU to see the difference...
>
You should try with different packet sizes for different hardware, or
s/ware drivers with and without async; with and without batching.
packet sizes 64,256,512,1024,1500 bytes. batch sizes, 1,2,4,8,16,..
> I think it won't be the programmer but the system administrator who will
> have to correctly set priorities and constraints for different
> hardware/software engines for the particular system.
Why is the admin involved in such decision making?
> With a slow CPU it
> may be worth to offload even small blocks to hardware, with a fast one
> it may be worth to set HW and SW as equal, etc.
>
I think the system should discover all this at runtime.
If the driver says its busy, you dont give it more work.
Clearly giving it more data is beneficial; hence before it gets busy
you give it enough to overcome the setup cost.
You should have qos (start with simple strict priority); and the
preference could be given to large packets etc as long as you dont
introduce reordering.
Come to think of it, this would be really easily doable if the crypto
device appeared to the system as a netdevice.
cheers,
jamal
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-31 10:46 ` Michal Ludvig
2004-10-31 15:03 ` jamal
@ 2004-10-31 15:36 ` Michael Richardson
2004-10-31 16:09 ` James Morris
1 sibling, 1 reply; 57+ messages in thread
From: Michael Richardson @ 2004-10-31 15:36 UTC (permalink / raw)
To: Michal Ludvig; +Cc: johnpol, hadi, netdev, cryptoapi, Eugene Surovegin
-----BEGIN PGP SIGNED MESSAGE-----
>>>>> "Michal" == Michal Ludvig <mludvig@suse.cz> writes:
Michal> I think it won't be the programmer but the system
Michal> administrator who will have to correctly set priorities and
Michal> constraints for different hardware/software engines for the
Michal> particular system. With a slow CPU it may be worth to
Michal> offload even small blocks to hardware, with a fast one it
Michal> may be worth to set HW and SW as equal, etc.
yes.
There are lots of situations where, yes, a 2Ghz CPU can do IPsec much
faster and with lower latency and with less jitter than hardware.
However, the CPU might have other important things to do. Things that
make the owner money.
This will become more and more obvious as IPsec is used host-to-host
for all sorts of traffic.
- --
] "Elmo went to the wrong fundraiser" - The Simpson | firewalls [
] Michael Richardson, Xelerance Corporation, Ottawa, ON |net architect[
] mcr@xelerance.com http://www.sandelman.ottawa.on.ca/mcr/ |device driver[
] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Finger me for keys
iQCVAwUBQYUGZoqHRg3pndX9AQED8wP/SuLbcQimL6HleUQJq/ZtFTKBqKPGyQWg
s7BrN/ZWCHPhS+9X3Pe3kioab2EC77VajHVLP45xDTX2Jb2MLyWzCGbMrJGZvmHT
ZXUNSWpyy8XZtM807ibajlbRT8J8cKgkz9/J1r4hh4JdoxQFlbxH7PmEaO/qws+g
WIcQ2cdXDOA=
=Q8L1
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 6:22 Asynchronous crypto layer Evgeniy Polyakov
` (3 preceding siblings ...)
2004-10-30 20:37 ` Evgeniy Polyakov
@ 2004-10-31 16:05 ` James Morris
2004-11-01 5:58 ` Evgeniy Polyakov
2004-11-02 15:24 ` Evgeniy Polyakov
2004-11-02 16:12 ` Evgeniy Polyakov
6 siblings, 1 reply; 57+ messages in thread
From: James Morris @ 2004-10-31 16:05 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: netdev, cryptoapi, David S. Miller
> Please review and comment, I am open for discussion.
Here is an initial review, based on the updated patch posted my Michael
Ludwig.
It's great to see this code being written rather than just discussed.
Have you seen earlier discussions on hardware crypto requirements, some of
which are summarized at http://samba.org/~jamesm/crypto/hardware_notes.txt
, and further discussed on the cryptoapi list?
Briefly, the main components required for hardware crypto support are:
a) Crypto driver API
This is for registering all types of crypto drivers, including the
software algorithms and various hardware devices. A feature of this API
would be to communicate various capabilities and features of the driver to
the core crypto code. Your code appears to at least partially provide
such an API, although I'm not clear on exactly what it's suitable for all
types of crypto drivers. It looks to be heading in the right direction.
The existing software drivers should be converted to the new API and the
old API removed.
b) Kernel crypto API
This refers to the API used by general kernel code which needs to invoke
crypto processing. Currently we have a simple synchronous API.
For hardware support we need an asynchronous API, which for example, would
allow a caller to:
- allocate a crypto session, specifying preferences, algorithms to be
used etc.
- manage crypto session parameters (e.g. key info etc).
- batch and submit crypto operations
- receive completed operation results
I would imagine that we would deprecate the existing sync API, convert
existing users to the async API then eventually remove the sync API.
This areas needs more analysis and development.
c) Async engine: scheduling/batching/routing etc.
This seems to be the core of what you've developed so far. I'm not sure
if we need pluggable load balancers. How would the sysadmin select the
correct one? The simpler this code is the better.
d) User API via filesystem.
The user API issue has been discussed on the crypto API list previously.
I have outlined some ideas for a pseudo filesytem API, although there are
still some issues to be resolved.
Overall I think it's a good start. There are some chicken & egg type
problems when you don't have GPL drivers, hardware or an existing async
API, so I'd imagine that this will all continue to evolve: with more
hardware we can write/incorporate more drivers, with more drivers we can
further refine the async support and API etc.
Code Review.
Firstly, please follow Documentation/CodingStyle more closely. Perhaps
run Lindent over your code and have a look at the differences.
It's much easier to review and test the code if it is supplied as a patch
against a recent Linus kernel, rather than a collection of files. If you
update your tree, please regenerate the patch rather than just send new
files.
Here are some issues that I noticed:
What is main_crypto_device? Is this a placeholder for when there are no
other devices registered?
Async & sync drivers need to both be treated as crypto drivers.
static inline void crypto_route_free(struct crypto_route *rt)
{
crypto_device_put(rt->dev);
rt->dev = NULL;
kfree(rt);
}
Why do you set rt->dev to NULL here? It should not still be referenceable
at this point.
__crypto_route_del() etc:
Why are you rolling your own list management and not using the list.h
functions?
+struct crypto_device_stat
+{
+ __u64 scompleted;
+ __u64 sfinished;
...
Please see how networking stats are implemented (e.g. netif_rx_stats) and
previous netdev discussions on 64-bit counters. Please only use __u64 and
similar when exporting structures to userspace.
Struct ordring:
+ u16 priority;
+
+ u64 id;
+ u64 dev_id;
+
+ u32 flags;
The struct will be better aligned if you put the smaller fields first
(although there may be cases where you want cache-hot items at the front).
Also, why the gaps between fields?
__crypto_device_add():
Please split device initialization and linking into distinct functions.
The initialization should probably be done in e.g. crypto_device_alloc()
instead of this.
+ struct crypto_device *__dev, *n;
Please use a name other than __dev here. Underscored variable names are
generally used for things like local variables in macros, not local
temporary variables in functions.
void crypto_device_remove(struct crypto_device *dev):
Don't loop waiting for the device to become free. Also, you're scheduling
inside a spinlock. This code needs to be re-thought in general to ensure
that a device is always destroyed safely.
crypto_lb_unregister():
I think this has already been raised. Please don't do things like this.
I'm not sure we need loadable load balancers yet, although in any case,
if we always need one loaded, then make a default one that cannot be
unloaded or replaces the last lb.
Where is crypto_session_alloc() used? It's difficult to evaluate the code
more deeply without seeing such primitives in use.
Thanks for doing this work and hope this helps,
- James
--
James Morris
<jmorris@redhat.com>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-31 15:03 ` jamal
@ 2004-10-31 16:07 ` James Morris
2004-11-01 6:01 ` Evgeniy Polyakov
1 sibling, 0 replies; 57+ messages in thread
From: James Morris @ 2004-10-31 16:07 UTC (permalink / raw)
To: jamal; +Cc: Michal Ludvig, johnpol, netdev, cryptoapi, Eugene Surovegin
On 31 Oct 2004, jamal wrote:
> Come to think of it, this would be really easily doable if the crypto
> device appeared to the system as a netdevice.
Should netdevices be generalized into some kind of generic packet
munchers? :-)
- James
--
James Morris
<jmorris@redhat.com>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-31 15:36 ` Michael Richardson
@ 2004-10-31 16:09 ` James Morris
0 siblings, 0 replies; 57+ messages in thread
From: James Morris @ 2004-10-31 16:09 UTC (permalink / raw)
To: Michael Richardson
Cc: Michal Ludvig, johnpol, hadi, netdev, cryptoapi, Eugene Surovegin
On Sun, 31 Oct 2004, Michael Richardson wrote:
> There are lots of situations where, yes, a 2Ghz CPU can do IPsec much
> faster and with lower latency and with less jitter than hardware.
>
> However, the CPU might have other important things to do. Things that
> make the owner money.
I guess control over this should be tunable, perhaps just a boolean of
whether to try and aim for raw speed or for reduced CPU contention.
- James
--
James Morris
<jmorris@redhat.com>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-31 14:56 ` jamal
@ 2004-11-01 5:12 ` Evgeniy Polyakov
0 siblings, 0 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-11-01 5:12 UTC (permalink / raw)
To: hadi; +Cc: netdev, cryptoapi, Eugene Surovegin
[-- Attachment #1: Type: text/plain, Size: 5609 bytes --]
On Sun, 2004-10-31 at 17:56, jamal wrote:
> On Sun, 2004-10-31 at 04:13, Evgeniy Polyakov wrote:
> > On 30 Oct 2004 19:41:27 -0400
> > jamal <hadi@cyberus.ca> wrote:
>
> > > Can you explain the "rate" or "speed" parameter ?
> >
> > Driver writer can set "rate" parameter to any number from 0 to 64k -
> > and it will show speed of this driver in current mode/type/operation.
> [..]
> >
> > That mean that this driver perform des 6 time faster than aes, but
> > it should be fair numbers and somehow measured and compared to other
> > drivers.
> >
>
> So you have some init code that does testing? Or is this factor of six
> part of the spec provided by chip vendor?
Chip vendor, but what if vendor lies or it was measured in a different
setup than other vendor?
And what about software speeds?
> > This also can be achieved by qlen parameter - if driver writer sets
> > it to bigger walue then in this mode driver/hardware works faster.
> > But driver writer can set qlen in a too big value just because
> > it want it to be such without any means of driver/hardware capabilities.
> > It is not forbidden.
> >
>
> It is no different than say the way you will do ethernet drivers.
> DMA ring sizes and link speeds. harder for ethernet drivers if link
> speeds change (Linux net scheduling assumes fixed speed ;->)
>
> > And the last and the most interesting one is following:
> > we create per session initialiser parameter "session_processin_time"
> > which will be sum of the time slices when driver performed operation
> > on session, since we alredy has "scomplete" paramenter which is equal
> > to the amount of completed(processed) session then we have _fair_ speed
> > of the driver/hardware in a given mode/operation/type.
> >
> > Of course load blancer should select device with the lowest
> > session_processing_time/scompleted.
> >
> > I think third variant is what we want. I will think of it some more
> > and will implement soon.
> >
>
> I think you should be able to have multiple, configurable LB algos.
Second is already implemented.
Third I posted to my TODO.
Should we add "rate" or will use "qlen" instead?
> > > I havent studied your code, however, what Eugene is pointing out is
> > > valuable detail/feedback.
> > >
> > > You should have in your queuing towards the crypto chip ability to
> > > batch. i.e sort of nagle-like "wait until we have 10 packets/20KB or 20
> > > jiffies" before you send everything in the queue to the chip.
> >
> > That is exactly how crypto driver should be written.
> > Driver has it's queue and number of session in it, so it and only it
> > can decide when begin to process them in the most effective way.
> >
>
> This should be above driver, really.
>
> You should have one or more queues where the scheduler feeds off and
> shoves to hardware. Perhaps several levels:
There are already several queues - one per crypto device.
Scheduler decides what device should handle given session,
and put session into selected device's queue and calls ->data_ready()
device's callback.
It is device that can handle several session per one "travel" of it's
queue.
Or do you say that scheduler can hold session until number of such
session becomes more than threshold and then post them at once to some
device's queue?
Why it is needed? That can be implemented easily by splitting
crypto_session_alloc() into two parts like James sugests, but I do not
see a reason for it. What if no session will be allocated in the near
future? Then first session will sit in a vacuum for a long while it
could be already processed even non effectively.
> ->LB-+
> +-> device1 scheduler --> driver queues/rings.
> |
> |
> +-> device2 scheduler --> driver queues/rings.
> |
> .
> .
> +-> devicen scheduler --> driver queues/rings.
>
> If you look at the linux traffic control, the stuff below LB is how it
> behaves. That should be generic enough to _not_ sit in the driver.
> This allows for adding smart algorithms to it; fe: qos, rate limiting,
> feedback to LB so it could make smarter decisions etc.
It does not sit in the driver.
Ethernet device has hard_start_xmit() which takes only one skb just
because hardware can not handle several packets at once.
Crypto devices [often] have ability to handle several sessions at once,
that is why struct crypto_device has queue from which device can take
sessions more than one in a time.
Sessions are placed into device's queue by scheduler which has some
algorithms inside(like qos), but then driver can get one session and
process it - thus it will looks like netdevice, but it also can take
several sessions and process them at once, netdev just can not do it.
> > > As he points out (and i am sure he can back it with data ;->), that
> > > given the setup cost, packet size, algo and CPU and bus speed, it may
> > > not make sense to use the chip at all ;->
> >
> > Michal has numbers - pure hardware beats soft in a certain setups
> > in a fully synchronous schema, let's work SW and HW in parallel.
> >
> > Of course SW can encrypt 64 byte faster than it will be transfered to
> > old ISA crypto card, but it worth to do it for compressing with LZW
> > 9000 bytes jumbo frame.
> >
>
> Would be interesting. I have seen the numbers from Eugene and they are
> quiet intriguing - but they are for the sync mode.
>
> cheers,
> jamal
>
--
Evgeniy Polyakov
Crash is better than data corruption. -- Art Grabowski
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-31 16:05 ` James Morris
@ 2004-11-01 5:58 ` Evgeniy Polyakov
2004-12-14 6:56 ` James Morris
0 siblings, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-11-01 5:58 UTC (permalink / raw)
To: James Morris; +Cc: netdev, cryptoapi, David S. Miller
[-- Attachment #1: Type: text/plain, Size: 9712 bytes --]
On Sun, 2004-10-31 at 19:05, James Morris wrote:
> > Please review and comment, I am open for discussion.
>
> Here is an initial review, based on the updated patch posted my Michael
> Ludwig.
>
> It's great to see this code being written rather than just discussed.
>
> Have you seen earlier discussions on hardware crypto requirements, some of
> which are summarized at http://samba.org/~jamesm/crypto/hardware_notes.txt
> , and further discussed on the cryptoapi list?
Yes, I have read it and many others.
I believe I've gathered all usefull features and implement them in a
right direction.
> Briefly, the main components required for hardware crypto support are:
>
> a) Crypto driver API
>
> This is for registering all types of crypto drivers, including the
> software algorithms and various hardware devices. A feature of this API
> would be to communicate various capabilities and features of the driver to
> the core crypto code. Your code appears to at least partially provide
> such an API, although I'm not clear on exactly what it's suitable for all
> types of crypto drivers. It looks to be heading in the right direction.
>
> The existing software drivers should be converted to the new API and the
> old API removed.
Or we can create some kind of bridge between old sync codebase and new
async API. Peaple are using sync API, async can live with it without
problems.
> b) Kernel crypto API
>
> This refers to the API used by general kernel code which needs to invoke
> crypto processing. Currently we have a simple synchronous API.
>
> For hardware support we need an asynchronous API, which for example, would
> allow a caller to:
>
> - allocate a crypto session, specifying preferences, algorithms to be
> used etc.
> - manage crypto session parameters (e.g. key info etc).
> - batch and submit crypto operations
> - receive completed operation results
I've put first tree into one function crypto_session_alloc(), although
there are mechanisms to manage crypto session after it is allocated.
> I would imagine that we would deprecate the existing sync API, convert
> existing users to the async API then eventually remove the sync API.
>
> This areas needs more analysis and development.
>
>
> c) Async engine: scheduling/batching/routing etc.
>
> This seems to be the core of what you've developed so far. I'm not sure
> if we need pluggable load balancers. How would the sysadmin select the
> correct one? The simpler this code is the better.
By following command:
echo -en "simple_lb" > /sys/class/crypto_lb/lbs
or by sending [still not implemented] connector's(netlink) command.
>
> d) User API via filesystem.
>
> The user API issue has been discussed on the crypto API list previously.
> I have outlined some ideas for a pseudo filesytem API, although there are
> still some issues to be resolved.
>
>
> Overall I think it's a good start. There are some chicken & egg type
> problems when you don't have GPL drivers, hardware or an existing async
> API, so I'd imagine that this will all continue to evolve: with more
> hardware we can write/incorporate more drivers, with more drivers we can
> further refine the async support and API etc.
That is true, but I think not all parts of API should be exported as GPL
only.
> Code Review.
>
> Firstly, please follow Documentation/CodingStyle more closely. Perhaps
> run Lindent over your code and have a look at the differences.
I waited this :)
> It's much easier to review and test the code if it is supplied as a patch
> against a recent Linus kernel, rather than a collection of files. If you
> update your tree, please regenerate the patch rather than just send new
> files.
Sure. Files were sent for review, since they can be compiled as stand
alone module.
>
> Here are some issues that I noticed:
>
>
> What is main_crypto_device? Is this a placeholder for when there are no
> other devices registered?
main_crypto_device is just virtual device into which all sessions are
placed along with specific crypto device queue.
it is usefull for cases when some driver decides that HW is broken and
marks itself like not working, then scheduler _may_ (simple_lb does not
have such feature) and actually _should_ move all sessions that are
placed into broken device's queue into other devices or drop them(call
callback with special session flag).
It is also usefull for the following situation:
consider situation when several slow devices has it's queues with some
sessions in it, then we add new fast crypto device, bu there are no new
sessions that can be pleced into it. Scheduler may decide to move
sessions from different slow device into new fast one.
All above cases can be easier to implement if we already have one queue
with all sessions in it, and scheduler should not go through all crypto
device's queues.
> Async & sync drivers need to both be treated as crypto drivers.
>
>
> static inline void crypto_route_free(struct crypto_route *rt)
> {
> crypto_device_put(rt->dev);
> rt->dev = NULL;
> kfree(rt);
> }
>
> Why do you set rt->dev to NULL here? It should not still be referenceable
> at this point.
crypto_route_free() can be called only from crypto_route_del() which
unlinks given route thus noone can reference rt->dev(but of course
asnyone can have it's private copy, obtained under the lock).
Any route access is guarded by route list lock.
>
> __crypto_route_del() etc:
>
> Why are you rolling your own list management and not using the list.h
> functions?
It is more convenient here to use queue like sk_buf does.
>
> +struct crypto_device_stat
> +{
> + __u64 scompleted;
> + __u64 sfinished;
> ...
>
> Please see how networking stats are implemented (e.g. netif_rx_stats) and
> previous netdev discussions on 64-bit counters. Please only use __u64 and
> similar when exporting structures to userspace.
I use __ prefix for any parameter that is exported to userspace.
I've seen several attempts to convert network statistic to 64bit values,
with backward compatibility it is not an easy task. Let's do not
catch this again.
> Struct ordring:
>
> + u16 priority;
> +
> + u64 id;
> + u64 dev_id;
> +
> + u32 flags;
>
> The struct will be better aligned if you put the smaller fields first
> (although there may be cases where you want cache-hot items at the front).
> Also, why the gaps between fields?
>
All they are created with cache alignment in mind - above cut is
following:
struct crypto_session_initializer
{
u16 operation;
u16 type;
u16 mode;
u16 priority;
u64 id;
u64 dev_id;
u32 flags;
u32 bdev;
crypto_callback_t callback;
};
>
> __crypto_device_add():
>
> Please split device initialization and linking into distinct functions.
> The initialization should probably be done in e.g. crypto_device_alloc()
> instead of this.
No problem. I gave it such name since __crypto_device_add() is called
only from crypto_device_add() (and main_crypto_device initialisation
routing).
> + struct crypto_device *__dev, *n;
>
> Please use a name other than __dev here. Underscored variable names are
> generally used for things like local variables in macros, not local
> temporary variables in functions.
Ok.
> void crypto_device_remove(struct crypto_device *dev):
>
> Don't loop waiting for the device to become free. Also, you're scheduling
> inside a spinlock. This code needs to be re-thought in general to ensure
> that a device is always destroyed safely.
No, lock is already freed there.
The sequence is following:
grab the lock.
search given device
unlink it
free the lock // after it nobody can add new sessions to the given
device
wait until device becomes free - it should be called from device's
module_exit() function, thus we are in a process context.
while waiting(device finishes it's sessions processing) we
reschedule itself.
return.
> crypto_lb_unregister():
>
> I think this has already been raised. Please don't do things like this.
> I'm not sure we need loadable load balancers yet, although in any case,
> if we always need one loaded, then make a default one that cannot be
> unloaded or replaces the last lb.
We have several IO schedulers.
I believe different hardware setups will requere different load
balancers - for example for setups with only fast devices we can use
just simple_lb, for setups with mixed devices we need to think of
session moving between them.
Simple_lb is fast but it does not have some features, advanced1_lb for
example can have some tricky algorithm to select crypto device and/or
have some thoughts about session moving...
But I think that we need one unloadable crypto load balancer which will
be used if all others are removed/unused.
I add it to my TODO.
>
> Where is crypto_session_alloc() used? It's difficult to evaluate the code
> more deeply without seeing such primitives in use.
I've sent several files in a first e-mail that are called like
consumer.c slow_consumer.c - it is crypto consumers, actually they
just call crypto_session_alloc() with appropriate parameters...
>
> Thanks for doing this work and hope this helps,
Thank you.
>
> - James
--
Evgeniy Polyakov
Crash is better than data corruption. -- Art Grabowski
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-31 15:03 ` jamal
2004-10-31 16:07 ` James Morris
@ 2004-11-01 6:01 ` Evgeniy Polyakov
1 sibling, 0 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-11-01 6:01 UTC (permalink / raw)
To: hadi; +Cc: Michal Ludvig, netdev, cryptoapi, Eugene Surovegin
[-- Attachment #1: Type: text/plain, Size: 2078 bytes --]
On Sun, 2004-10-31 at 18:03, jamal wrote:
> On Sun, 2004-10-31 at 05:46, Michal Ludvig wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
>
> > Yes, I have *some* numbers, but consider that they are for quite
> > eligible setup - encrypting ~1.5k IPsec packets. I should retry with a
> > much smaller MTU to see the difference...
> >
>
> You should try with different packet sizes for different hardware, or
> s/ware drivers with and without async; with and without batching.
> packet sizes 64,256,512,1024,1500 bytes. batch sizes, 1,2,4,8,16,..
>
> > I think it won't be the programmer but the system administrator who will
> > have to correctly set priorities and constraints for different
> > hardware/software engines for the particular system.
>
> Why is the admin involved in such decision making?
>
> > With a slow CPU it
> > may be worth to offload even small blocks to hardware, with a fast one
> > it may be worth to set HW and SW as equal, etc.
> >
>
> I think the system should discover all this at runtime.
> If the driver says its busy, you dont give it more work.
> Clearly giving it more data is beneficial; hence before it gets busy
> you give it enough to overcome the setup cost.
> You should have qos (start with simple strict priority); and the
> preference could be given to large packets etc as long as you dont
> introduce reordering.
Crypto session priority already exists - sessions are placed into
the queues in order of it's priority.
It is supposed that crypto device driver will get them in this order
too.
> Come to think of it, this would be really easily doable if the crypto
> device appeared to the system as a netdevice.
:) First of all it would be good to stabilize what we already have.
> cheers,
> jamal
>
> _______________________________________________
>
> Subscription: http://lists.logix.cz/mailman/listinfo/cryptoapi
> List archive: http://lists.logix.cz/pipermail/cryptoapi
--
Evgeniy Polyakov
Crash is better than data corruption. -- Art Grabowski
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 6:22 Asynchronous crypto layer Evgeniy Polyakov
` (4 preceding siblings ...)
2004-10-31 16:05 ` James Morris
@ 2004-11-02 15:24 ` Evgeniy Polyakov
2004-11-02 16:12 ` Evgeniy Polyakov
6 siblings, 0 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-11-02 15:24 UTC (permalink / raw)
To: jamal, James Morris, Michal Ludvig; +Cc: netdev, cryptoapi
[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]
Here we go.
Numbers.
Actually nothing excitement:
since ecryption/decryption was made in a one thread
(please see dirty hack in sync_provider.c:89 to allow
crypto_cipher_{en,de}crypt() work in _bh context, BUG()
in crypto_yeld() can be reproduced by removing hack).
UP with HT badly broken Xeon(800 mhz) 2 parallel running scripts:
rnd=$RANDOM
file=/mnt/$rnd
for ((i=0; i<20; ++i)); do
echo -en "."
dd if=/dev/zero of=$file bs=1M count=40 > /dev/null 2>&1
rm -f $file
done
echo
Pure sync mode:
0.07user 14.87system 0:16.49elapsed 90%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6840minor)pagefaults 0swaps
Pseudo async mode(data ecnryption/decryption is done synchronous thread,
only pre/post processing is asynchronous):
0.08user 11.66system 0:17.03elapsed 68%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6870minor)pagefaults 0swaps
Attached synchronous and asynchronous crypto providers and purely in-memory
block device.
Usage:
Async mode:
cd crypto
sudo ./i
cd ../block
sudo ./i
term1$ sudo time ./start
term2$ sudo time ./start
Sync mode:
cd block
sudo ./r
cd ../crypto
sudo ./r
cd ../block
sudo ./i 1
term1$ sudo time ./start
term2$ sudo time ./start
I will send updated asynchronous crypto patch in a next e-mail.
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
[-- Attachment #2: async_provider.c --]
[-- Type: text/x-csrc, Size: 6770 bytes --]
/*
* async_provider.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/err.h>
#include <linux/crypto.h>
#include <linux/mm.h>
#include <linux/blkdev.h>
#include "../crypto/acrypto.h"
#include "../crypto/crypto_stat.h"
#include "../crypto/crypto_def.h"
static void prov_data_ready(struct crypto_device *);
static void async_callback(struct crypto_session_initializer *, struct crypto_data *);
static struct crypto_capability prov_caps[] = {
{CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 10000},
{CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 10000},
};
static int prov_cap_number = sizeof(prov_caps)/sizeof(prov_caps[0]);
static struct completion thread_exited;
static DECLARE_WAIT_QUEUE_HEAD(async_wait_queue);
static int need_exit;
static struct crypto_tfm *tfm;
static char async_algo[] = "aes";
static char async_key[] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f};
static struct crypto_session_initializer ci =
{
.operation = CRYPTO_OP_ENCRYPT,
.mode = CRYPTO_MODE_ECB,
.type = CRYPTO_TYPE_AES_128,
.callback = async_callback,
};
static struct crypto_device pdev =
{
.name = "async_provider",
.data_ready = prov_data_ready,
.cap = &prov_caps[0],
};
static void async_callback(struct crypto_session_initializer *ci, struct crypto_data *data)
{
struct request *req = data->priv;
dprintk("%s: req=%p, req->q=%p.\n", __func__, req, req->q);
(unsigned long)req->special--;
if ((unsigned long)req->special == 0 && req->q)
{
spinlock_t *l = req->q->queue_lock;
dprintk("Finishing request for req=%p [%lu].\n", req, (unsigned long)req->special);
spin_lock_irq(l);
(unsigned long)req->special = 0;
if (!end_that_request_first(req, 1, req->nr_sectors));
end_that_request_last(req);
spin_unlock_irq(l);
}
}
static __inline__ void async_prepare(struct crypto_data *data, void *src, void *dst, unsigned long size, void *priv)
{
struct request *req = priv;
(unsigned long)req->special++;
dprintk("%s: req=%p, refcnt=%lu.\n",
__func__, req, (unsigned long)req->special);
data->sg_src.page = virt_to_page(src);
data->sg_src.offset = offset_in_page(src);
data->sg_src.length = size;
data->sg_dst.page = virt_to_page(dst);
data->sg_dst.offset = offset_in_page(dst);
data->sg_dst.length = size;
data->sg_key.page = virt_to_page(async_key);
data->sg_key.offset = offset_in_page(async_key);
data->sg_key.length = sizeof(async_key);
data->priv = priv;
data->priv_size = 0;
}
void bd_encrypt(void *src, void *dst, u64 size, void *priv)
{
struct crypto_data data;
struct crypto_session *sp;
async_prepare(&data, src, dst, size, priv);
ci.operation = CRYPTO_OP_ENCRYPT;
sp = crypto_session_alloc(&ci, &data);
BUG_ON(!sp);
wake_up_interruptible(&async_wait_queue);
}
void bd_decrypt(void *src, void *dst, u64 size, void *priv)
{
struct crypto_data data;
struct crypto_session *sp;
async_prepare(&data, src, dst, size, priv);
ci.operation = CRYPTO_OP_DECRYPT;
sp = crypto_session_alloc(&ci, &data);
BUG_ON(!sp);
wake_up_interruptible(&async_wait_queue);
}
static void prov_data_ready(struct crypto_device *dev)
{
wake_up_interruptible(&async_wait_queue);
}
static int async_thread(void *data)
{
struct crypto_device *dev = (struct crypto_device *)data;
struct crypto_session *s, *n;
daemonize("%s", dev->name);
allow_signal(SIGTERM);
while (!need_exit)
{
int num, pnum;
interruptible_sleep_on_timeout(&async_wait_queue, 100);
if (need_exit)
break;
num = pnum = 0;
list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry)
{
num++;
if (session_completed(s))
continue;
pnum++;
start_process_session(s);
if (s->ci.operation == CRYPTO_OP_ENCRYPT)
crypto_cipher_encrypt(tfm, &s->data.sg_dst, &s->data.sg_src, s->data.sg_src.length);
else
crypto_cipher_decrypt(tfm, &s->data.sg_dst, &s->data.sg_src, s->data.sg_src.length);
dprintk("%lu: Completing session %llu [%llu] in %s.\n",
jiffies, s->ci.id, s->ci.dev_id, pdev.name);
crypto_stat_complete_inc(s);
complete_session(s);
stop_process_session(s);
}
dprintk("%lu: %s: %d sessions, %d processed.\n", jiffies, dev->name, num, pnum);
}
complete_and_exit(&thread_exited, 0);
}
int prov_init(void)
{
int err, pid;
tfm = crypto_alloc_tfm(async_algo, 0);
if (!tfm)
{
dprintk(KERN_ERR "Failed to allocate %s tfm.\n", async_algo);
return -EINVAL;
}
err = crypto_cipher_setkey(tfm, async_key, sizeof(async_key));
if (err)
{
dprintk("Failed to set key [keylen=%d]: err=%d.\n",
sizeof(async_key), err);
goto err_out_free_tfm;
}
init_completion(&thread_exited);
pid = kernel_thread(async_thread, &pdev, CLONE_FS | CLONE_FILES);
if (IS_ERR((void *)pid))
{
err = -EINVAL;
dprintk(KERN_ERR "Failed to create kernel load balancing thread.\n");
goto err_out_free_tfm;
}
pdev.cap_number = prov_cap_number;
err = crypto_device_add(&pdev);
if (err)
goto err_out_remove_thread;
dprintk(KERN_INFO "Test crypto provider module %s is loaded.\n", pdev.name);
return 0;
err_out_remove_thread:
need_exit = 1;
wake_up(&async_wait_queue);
wait_for_completion(&thread_exited);
err_out_free_tfm:
crypto_free_tfm(tfm);
return err;
}
void prov_fini(void)
{
need_exit = 1;
wake_up(&async_wait_queue);
wait_for_completion(&thread_exited);
crypto_device_remove(&pdev);
crypto_free_tfm(tfm);
dprintk(KERN_INFO "Test crypto provider module %s is unloaded.\n", pdev.name);
}
module_init(prov_init);
module_exit(prov_fini);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_DESCRIPTION("Test crypto module provider.");
EXPORT_SYMBOL(bd_encrypt);
EXPORT_SYMBOL(bd_decrypt);
[-- Attachment #3: Makefile --]
[-- Type: application/octet-stream, Size: 242 bytes --]
obj-m := bd.o
KDIR := /lib/modules/$(shell uname -r)/build
#KDIR := /usr/local/src/linux-2.6/linux-2.6
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
clean:
rm -f *.o *.ko *.mod.* .*.cmd *~
rm -rf .tmp_versions
[-- Attachment #4: bd.c --]
[-- Type: text/x-csrc, Size: 6573 bytes --]
/*
* bd.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/spinlock.h>
#include <linux/blkdev.h>
#include <linux/fs.h>
#include <linux/bio.h>
#include <linux/init.h>
//#define BD_DEBUG
#ifdef BD_DEBUG
#define dprintk(f, a...) printk(KERN_EMERG f, ##a)
#else
#define dprintk(f, a...) do {} while(0)
#endif
#define BD_READ 0
#define BD_WRITE 1
static char bd_name[] = "bd";
static unsigned int bd_major = 123;
module_param(bd_major, uint, 0);
static unsigned int bd_sector_size = 512;
module_param(bd_sector_size, uint, 0);
static unsigned int bd_max_request_size = 1024*1024*50;
module_param(bd_max_request_size, uint, 0);
static int sync_mode = 0;
module_param(sync_mode, uint, 0);
static void *bd_request_data;
static spinlock_t bd_lock = SPIN_LOCK_UNLOCKED;
static struct gendisk *bd_disk;
static unsigned long req_counter = 1;
extern void bd_encrypt(void *src, void *dst, u64 size, void *priv);
extern void bd_decrypt(void *src, void *dst, u64 size, void *priv);
static int bd_ioctl(struct inode *, struct file *, unsigned int, unsigned long);
static struct block_device_operations bd_fops =
{
.owner = THIS_MODULE,
.ioctl = bd_ioctl,
};
static int bd_ioctl(struct inode *inode, struct file *file,
unsigned int cmd, unsigned long arg)
{
dprintk("%s: cmd=%u, arg=%ld.\n", __func__, cmd, arg);
return 0;
}
static int bd_process_data(int cmd, void *buf, u64 off, u64 size, void *priv)
{
if (size > bd_max_request_size)
{
dprintk("Too big size %llu, setting to %u.\n",
size, bd_max_request_size);
size = bd_max_request_size;
}
if (off + size > bd_max_request_size)
{
dprintk("Too big size %llu or offset %llu, exiting.\n",
size, off);
size = bd_max_request_size;
}
switch (cmd)
{
case BD_READ:
bd_decrypt(((char *)bd_request_data) + off, buf, size, priv);
break;
case BD_WRITE:
bd_encrypt(buf, ((char *)bd_request_data) + off, size, priv);
break;
}
dprintk("%s finished cmd=%s, size=%llu, off=%llu.\n",
__func__,
(cmd == BD_WRITE)?"WRITE":"READ",
size, off);
return 0;
}
static __inline__ int bd_process_bvec(int cmd, struct bio_vec *bvec, u64 off, void *priv)
{
int err;
char *kaddr;
void *buf;
u64 size;
kaddr = page_address(bvec->bv_page);
buf = kaddr + bvec->bv_offset;
size = bvec->bv_len;
err = bd_process_data(cmd, buf, off, size, priv);
return err;
}
static void bd_request(request_queue_t * q)
{
struct request *req;
u64 size, off;
int err, cmd, bad_bio;
struct bio *bio;
while ((req = elv_next_request(q)) != NULL)
{
blkdev_dequeue_request(req);
(unsigned long)req->special = 0;
off = req->sector * bd_sector_size;
size = req->nr_sectors * bd_sector_size;
cmd = (rq_data_dir(req) == 1)?BD_WRITE:BD_READ;
dprintk("%s: TRANSFER: req=%p [%lu], cmd=%s, off=%llu [sector=%llu], size=%llu [nr_sectors=%lu], flags=%lx.\n",
bd_name, req, (unsigned long)req->special,
(cmd == BD_WRITE)?"WRITE":"READ",
off, req->sector,
size, req->nr_sectors,
req->flags);
bad_bio = 0;
if (!(req->flags & REQ_CMD))
{
dprintk("%s: wrong command.\n", bd_name);
goto err_out_wrong_command;
}
rq_for_each_bio(bio, req)
{
struct bio_vec *bvec;
int i;
dprintk("BIO: bi_size=%u, bvec=%p.\n", bio->bi_size, bio->bi_io_vec);
if (!bio->bi_size || !bio->bi_io_vec)
{
bad_bio = 1;
continue;
}
bio_for_each_segment(bvec, bio, i) {
dprintk("%s: i=%d, req=%p, size=%d.\n",
bd_disk->disk_name, i,
req, bvec->bv_len);
err = bd_process_bvec(cmd, bvec, off, req);
if (err)
{
bad_bio = 1;
continue;
}
off += bvec->bv_len;
size -= bvec->bv_len;
}
/*
* bio->bi_end_io = NULL;
* bio_endio(bio, bio->bi_size, (bad_bio)?-EIO:0);
*/
}
dprintk("cmd=%s has been processed: err=%d\n",
(cmd == BD_WRITE)?"WRITE":"READ",
req->errors);
err_out_wrong_command:
if (sync_mode)
{
if (!end_that_request_first(req, (bad_bio)?0:1, req->nr_sectors));
end_that_request_last(req);
}
}
}
static void bd_setup(struct gendisk *d)
{
request_queue_t *q;
d->major = bd_major;
d->first_minor = 0;
d->fops = &bd_fops;
d->private_data = NULL;
d->flags = GENHD_FL_SUPPRESS_PARTITION_INFO;
sprintf(d->disk_name, "%s%d", bd_name, 0);
q = d->queue;
blk_queue_hardsect_size(q, bd_sector_size);
set_capacity(d, bd_max_request_size/bd_sector_size);
add_disk(d);
}
int __devinit bd_init(void)
{
int err;
bd_request_data = vmalloc(bd_max_request_size);
if (!bd_request_data)
{
dprintk("Failed to allocate %d bytes for %s requests.\n",
bd_max_request_size, bd_name);
return -ENOMEM;
}
err = register_blkdev(bd_major, bd_name);
if (err)
{
dprintk("Failed to register blkdev with major %u: err=%d.\n",
bd_major, err);
return err;
}
bd_disk = alloc_disk(1);
if (!bd_disk)
{
dprintk("Failed to allocate a disk.\n");
goto err_out_unregister_blkdev;
}
bd_disk->queue = blk_init_queue(bd_request, &bd_lock);
if (!bd_disk->queue)
{
dprintk("Failed to initialize blk queue: err=%d.\n", err);
goto err_out_free_disk;
}
bd_setup(bd_disk);
dprintk("%s has beed successfully added.\n", bd_name);
return 0;
err_out_free_disk:
put_disk(bd_disk);
err_out_unregister_blkdev:
unregister_blkdev(bd_major, "bd");
vfree(bd_request_data);
return -EINVAL;
}
void __devexit bd_fini(void)
{
del_gendisk(bd_disk);
blk_cleanup_queue(bd_disk->queue);
put_disk(bd_disk);
unregister_blkdev(bd_major, "bd");
vfree(bd_request_data);
dprintk("%s has beed successfully removed.\n", bd_name);
}
module_init(bd_init);
module_exit(bd_fini);
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_LICENSE("GPL");
[-- Attachment #5: i --]
[-- Type: application/octet-stream, Size: 319 bytes --]
#!/bin/sh
num=$#
dev_file=/dev/bd0
sync_mode=0;
if [ $num != 0 ]; then
sync_mode=$1
fi
if [ $sync_mode != 0 ]; then
insmod ./sync_provider.ko
else
insmod ./async_provider.ko
fi
insmod ./bd.ko sync_mode=$sync_mode
if [ ! -f $dev_file ]; then
mknod $dev_file b 123 0
fi
mkfs.ext2 $dev_file
mount $dev_file /mnt
[-- Attachment #6: r --]
[-- Type: application/octet-stream, Size: 81 bytes --]
dev_file=/dev/bd0
umount /mnt
rmmod bd
rmmod async_provider
rmmod sync_provider
[-- Attachment #7: sync_provider.c --]
[-- Type: text/x-csrc, Size: 3186 bytes --]
/*
* sync_provider.c
*
* Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/spinlock.h>
#include <linux/blkdev.h>
#include <linux/fs.h>
#include <linux/bio.h>
#include <linux/init.h>
#include <linux/crypto.h>
#include <linux/hardirq.h>
#include <asm/scatterlist.h>
//#define SYNC_DEBUG
#ifdef SYNC_DEBUG
#define dprintk(f, a...) printk(KERN_EMERG f, ##a)
#else
#define dprintk(f, a...) do {} while(0)
#endif
#define OP_ENCRYPT 0
#define OP_DECRYPT 1
static char sync_algo[] = "aes";
static char sync_key[] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f};
static struct crypto_tfm *tfm;
void bd_encrypt(void *src, void *dst, u64 size, void *priv)
{
struct scatterlist s, d;
s.page = virt_to_page(src);
s.offset = offset_in_page(src);
s.length = size;
d.page = virt_to_page(dst);
d.offset = offset_in_page(dst);
d.length = size;
/*
* A hack to fool crypto layer - it thinks that we can sleep here...
* Sigh.
*
* preempt_count() |= SOFTIRQ_MASK;
*/
preempt_count() |= SOFTIRQ_MASK;
crypto_cipher_encrypt(tfm, &d, &s, size);
preempt_count() &= ~SOFTIRQ_MASK;
}
void bd_decrypt(void *src, void *dst, u64 size, void *priv)
{
struct scatterlist s, d;
s.page = virt_to_page(src);
s.offset = offset_in_page(src);
s.length = size;
d.page = virt_to_page(dst);
d.offset = offset_in_page(dst);
d.length = size;
/*
* A hack to fool crypto layer - it thinks that we can sleep here...
* Sigh.
*
* preempt_count() |= SOFTIRQ_MASK;
*/
preempt_count() |= SOFTIRQ_MASK;
crypto_cipher_decrypt(tfm, &d, &s, size);
preempt_count() &= ~SOFTIRQ_MASK;
}
int __devinit sync_init(void)
{
int err;
err = -ENODEV;
tfm = crypto_alloc_tfm(sync_algo, 0);
if (!tfm)
{
dprintk("Failed to allocate %s tfm.\n", sync_algo);
goto err_out_exit;
}
err = crypto_cipher_setkey(tfm, sync_key, sizeof(sync_key));
if (err)
{
dprintk("Failed to set key [keylen=%d]: err=%d.\n",
sizeof(sync_key), err);
goto err_out_free_tfm;
}
return 0;
err_out_free_tfm:
crypto_free_tfm(tfm);
err_out_exit:
return err;
}
void __devexit sync_fini(void)
{
crypto_free_tfm(tfm);
}
module_init(sync_init);
module_exit(sync_fini);
MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
MODULE_LICENSE("GPL");
EXPORT_SYMBOL(bd_encrypt);
EXPORT_SYMBOL(bd_decrypt);
[-- Attachment #8: start --]
[-- Type: application/octet-stream, Size: 150 bytes --]
rnd=$RANDOM
file=/mnt/$rnd
for ((i=0; i<20; ++i)); do
echo -en "."
dd if=/dev/zero of=$file bs=1M count=40 > /dev/null 2>&1
rm -f $file
done
echo
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-10-29 6:22 Asynchronous crypto layer Evgeniy Polyakov
` (5 preceding siblings ...)
2004-11-02 15:24 ` Evgeniy Polyakov
@ 2004-11-02 16:12 ` Evgeniy Polyakov
2004-12-14 7:23 ` James Morris
6 siblings, 1 reply; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-11-02 16:12 UTC (permalink / raw)
To: jamal, James Morris, Michal Ludvig; +Cc: netdev, cryptoapi
[-- Attachment #1: Type: text/plain, Size: 1326 bytes --]
This patch implements several features discussed previously:
- crypto_session_alloc() split. I did not change crypto_session_alloc()
name, but added
struct crypto_session *crypto_session_create(
struct crypto_initizlier *, struct crypto_data *)
and void crypto_session_add(struct crypto_session *s);
Thus now we have timeslice for crypto routing manipulations.
- Added per crypto_initializer ptime and scomp fields.
The former is total time sessions with given crypto_initializer
being processed, it is in jifies.
The latter is total number of session with given crypto_initializer
already have been processed. Sum of all scomp above is equal to
dev->stat.scompleted - total number io completed sessions
in given device.
- Added crypto_wake_lb() call - this function is called when device
finishes session processing(by calling stop_process_session()).
It wakes crypto_lb_thread() which efficiently reduces latancies.
- Added struct crypto_data * parameter into crypto_lb_find_device().
Load balancer now has ability to select device based not only
on device's capabilities, but on packet's characteristics too.
- If none crypto load balancer registered yet, then just return NULL
from crypto_session_{create,alloc}(), not BUG_ON().
Evgeniy Polyakov
Only failure makes us experts. -- Theo de Raadt
[-- Attachment #2: crypto.patch --]
[-- Type: application/octet-stream, Size: 92536 bytes --]
diff -Nru /tmp/empty/Kconfig linux-2.6/drivers/acrypto/Kconfig
--- /tmp/empty/Kconfig 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/Kconfig 2004-10-30 12:32:53.000000000 +0400
@@ -0,0 +1,15 @@
+menu "Asynchronous crypto layer"
+
+config ACRYPTO
+ tristate "Asynchronous crypto layer"
+ depends on CONNECTOR && CRYPTO
+ ---help---
+ Asynchronous crypto layer.
+
+config SIMPLE_LB
+ tristate "Simple load balancer"
+ depends on ACRYPTO
+ ---help---
+ Simple load balancer.
+endmenu
+
diff -Nru /tmp/empty/Makefile linux-2.6/drivers/acrypto/Makefile
--- /tmp/empty/Makefile 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/Makefile 2004-10-30 09:03:14.000000000 +0400
@@ -0,0 +1,5 @@
+obj-$(CONFIG_ACRYPTO) += acrypto.o
+obj-$(CONFIG_SIMPLE_LB) += simple_lb.o
+acrypto-objs := crypto_main.o crypto_lb.o crypto_dev.o crypto_conn.o crypto_stat.o
+
+obj-$(CONFIG_VIA_PADLOCK) += via-padlock/
diff -Nru /tmp/empty/acrypto.h linux-2.6/drivers/acrypto/acrypto.h
--- /tmp/empty/acrypto.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/acrypto.h 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,231 @@
+/*
+ * acrypto.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __ACRYPTO_H
+#define __ACRYPTO_H
+
+#define SCACHE_NAMELEN 32
+
+struct crypto_session_initializer;
+struct crypto_data;
+typedef void (*crypto_callback_t)(struct crypto_session_initializer *, struct crypto_data *);
+
+extern void crypto_wake_lb(void);
+
+#define SESSION_COMPLETED (1<<15)
+#define SESSION_FINISHED (1<<14)
+#define SESSION_STARTED (1<<13)
+#define SESSION_PROCESSED (1<<12)
+#define SESSION_BINDED (1<<11)
+#define SESSION_BROKEN (1<<10)
+
+#define session_completed(s) (s->ci.flags & SESSION_COMPLETED)
+#define complete_session(s) do {s->ci.flags |= SESSION_COMPLETED;} while(0)
+#define uncomplete_session(s) do {s->ci.flags &= ~SESSION_COMPLETED;} while (0)
+
+#define session_finished(s) (s->ci.flags & SESSION_FINISHED)
+#define finish_session(s) do {s->ci.flags |= SESSION_FINISHED;} while(0)
+#define unfinish_session(s) do {s->ci.flags &= ~SESSION_FINISHED;} while (0)
+
+#define session_started(s) (s->ci.flags & SESSION_STARTED)
+#define start_session(s) do {s->ci.flags |= SESSION_STARTED;} while(0)
+#define unstart_session(s) do {s->ci.flags &= ~SESSION_STARTED;} while (0)
+
+#define session_is_processed(s) (s->ci.flags & SESSION_PROCESSED)
+#define start_process_session(s) do {s->ci.flags |= SESSION_PROCESSED; s->ci.ptime = jiffies;} while(0)
+#define stop_process_session(s) do {s->ci.flags &= ~SESSION_PROCESSED; s->ci.ptime = jiffies - s->ci.ptime; crypto_wake_lb();} while (0)
+
+#define session_binded(s) (s->ci.flags & SESSION_BINDED)
+#define bind_session(s) do {s->ci.flags |= SESSION_BINDED;} while(0)
+#define unbind_session(s) do {s->ci.flags &= ~SESSION_BINDED;} while (0)
+#define sci_binded(ci) (ci->flags & SESSION_BINDED)
+
+#define session_broken(s) (s->ci.flags & SESSION_BROKEN)
+#define broke_session(s) do {s->ci.flags |= SESSION_BROKEN;} while(0)
+#define unbroke_session(s) do {s->ci.flags &= ~SESSION_BROKEN;} while (0)
+
+struct crypto_device_stat
+{
+ __u64 scompleted;
+ __u64 sfinished;
+ __u64 sstarted;
+ __u64 kmem_failed;
+};
+
+#ifdef __KERNEL__
+
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/device.h>
+#include <linux/workqueue.h>
+
+#include <asm/scatterlist.h>
+
+//#define DEBUG
+#ifdef DEBUG
+#define dprintk(f, a...) printk(KERN_EMERG f, ##a)
+#else
+#define dprintk(f, a...)
+#endif
+
+#define CRYPTO_MAX_PRIV_SIZE 1024
+
+#define DEVICE_BROKEN (1<<0)
+
+#define device_broken(dev) (dev->flags & DEVICE_BROKEN)
+#define broke_device(dev) do {dev->flags |= DEVICE_BROKEN;} while(0)
+#define repair_device(dev) do {dev->flags &= ~DEVICE_BROKEN;} while(0)
+
+struct crypto_capability
+{
+ u16 operation;
+ u16 type;
+ u16 mode;
+ u16 qlen;
+ u64 ptime;
+ u64 scomp;
+};
+
+struct crypto_session_initializer
+{
+ u16 operation;
+ u16 type;
+ u16 mode;
+ u16 priority;
+
+ u64 id;
+ u64 dev_id;
+
+ u32 flags;
+
+ u32 bdev;
+
+ u64 ptime;
+
+ crypto_callback_t callback;
+};
+
+struct crypto_data
+{
+ int sg_src_num;
+ struct scatterlist sg_src;
+ struct scatterlist sg_dst;
+ struct scatterlist sg_key;
+ struct scatterlist sg_iv;
+
+ void *priv;
+ unsigned int priv_size;
+};
+
+struct crypto_device
+{
+ char name[SCACHE_NAMELEN];
+
+ spinlock_t session_lock;
+ struct list_head session_list;
+
+ u64 sid;
+ spinlock_t lock;
+
+ atomic_t refcnt;
+
+ u32 flags;
+
+ u32 id;
+
+ struct list_head cdev_entry;
+
+ void (*data_ready)(struct crypto_device *);
+
+ struct device_driver *driver;
+ struct device device;
+ struct class_device class_device;
+ struct completion dev_released;
+
+ spinlock_t stat_lock;
+ struct crypto_device_stat stat;
+
+ struct crypto_capability *cap;
+ int cap_number;
+};
+
+struct crypto_route_head
+{
+ struct crypto_route *next;
+ struct crypto_route *prev;
+
+ __u32 qlen;
+ spinlock_t lock;
+};
+
+struct crypto_route
+{
+ struct crypto_route *next;
+ struct crypto_route *prev;
+
+ struct crypto_route_head *list;
+ struct crypto_device *dev;
+
+ struct crypto_session_initializer ci;
+};
+
+
+struct crypto_session
+{
+ struct list_head dev_queue_entry;
+ struct list_head main_queue_entry;
+
+ struct crypto_session_initializer ci;
+
+ struct crypto_data data;
+
+ spinlock_t lock;
+
+ struct work_struct work;
+
+ struct crypto_route_head route_list;
+};
+
+struct crypto_session *crypto_session_alloc(struct crypto_session_initializer *, struct crypto_data *);
+struct crypto_session *crypto_session_create(struct crypto_session_initializer *, struct crypto_data *);
+void crypto_session_add(struct crypto_session *);
+inline void crypto_session_dequeue_main(struct crypto_session *);
+inline void __crypto_session_dequeue_main(struct crypto_session *);
+void crypto_session_dequeue_route(struct crypto_session *);
+
+inline void crypto_device_get(struct crypto_device *);
+inline void crypto_device_put(struct crypto_device *);
+inline struct crypto_device *crypto_device_get_name(char *);
+
+int __crypto_device_add(struct crypto_device *);
+int crypto_device_add(struct crypto_device *);
+void __crypto_device_remove(struct crypto_device *);
+void crypto_device_remove(struct crypto_device *);
+int match_initializer(struct crypto_device *, struct crypto_session_initializer *);
+int __match_initializer(struct crypto_capability *, struct crypto_session_initializer *);
+
+inline void crypto_session_insert_main(struct crypto_device *dev, struct crypto_session *s);
+inline void crypto_session_insert(struct crypto_device *dev, struct crypto_session *s);
+
+#endif /* __KERNEL__ */
+#endif /* __ACRYPTO_H */
diff -Nru /tmp/empty/crypto_conn.c linux-2.6/drivers/acrypto/crypto_conn.c
--- /tmp/empty/crypto_conn.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_conn.c 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,195 @@
+/*
+ * crypto_conn.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/vmalloc.h>
+
+#include "acrypto.h"
+#include "crypto_lb.h"
+
+#include "../connector/connector.h"
+
+#include "crypto_conn.h"
+
+static struct cb_id crypto_conn_id = { 0xdead, 0x0000 };
+static char crypto_conn_name[] = "crconn";
+
+static void crypto_conn_callback(void *data)
+{
+ struct cn_msg *msg, *reply;
+ struct crypto_conn_data *d, *cmd;
+ struct crypto_device *dev;
+ u32 sessions;
+ unsigned long flags;
+
+ msg = (struct cn_msg *)data;
+ d = (struct crypto_conn_data *)msg->data;
+
+ if (msg->len < sizeof(*d))
+ {
+ dprintk(KERN_ERR "Wrong message to crypto connector: msg->len=%u < %u.\n",
+ msg->len, sizeof(*d));
+ return;
+ }
+
+ if (msg->len != sizeof(*d) + d->len)
+ {
+ dprintk(KERN_ERR "Wrong message to crypto connector: msg->len=%u != %u.\n",
+ msg->len, sizeof(*d) + d->len);
+ return;
+ }
+
+ dev = crypto_device_get_name(d->name);
+ if (!dev)
+ {
+ dprintk(KERN_INFO "Crypto device %s was not found.\n", d->name);
+ return;
+ }
+
+ switch (d->cmd)
+ {
+ case CRYPTO_CONN_READ_SESSIONS:
+ reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + sizeof(sessions), GFP_ATOMIC);
+ if (reply) {
+ memcpy(reply, msg, sizeof(*reply));
+ reply->len = sizeof(*cmd) + sizeof(sessions);
+
+ /*
+ * See protocol description in connector.c
+ */
+ reply->ack++;
+
+ cmd = (struct crypto_conn_data *)(reply + 1);
+ memcpy(cmd, d, sizeof(*cmd));
+ cmd->len = sizeof(sessions);
+
+ sessions = atomic_read(&dev->refcnt);
+
+ memcpy(cmd+1, &sessions, sizeof(sessions));
+
+ cn_netlink_send(reply, 0);
+
+ kfree(reply);
+ } else
+ dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
+ sizeof(*msg) + sizeof(*cmd), d->cmd);
+ break;
+ case CRYPTO_CONN_DUMP_QUEUE:
+ reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + 1024*sizeof(struct crypto_session_initializer), GFP_ATOMIC);
+ if (reply) {
+ struct crypto_session *s;
+ struct crypto_session_initializer *ptr;
+
+ memcpy(reply, msg, sizeof(*reply));
+
+ /*
+ * See protocol description in connector.c
+ */
+ reply->ack++;
+
+ cmd = (struct crypto_conn_data *)(reply + 1);
+ memcpy(cmd, d, sizeof(*cmd));
+
+ ptr = (struct crypto_session_initializer *)(cmd+1);
+
+ sessions = 0;
+ spin_lock_irqsave(&dev->session_lock, flags);
+ list_for_each_entry(s, &dev->session_list, dev_queue_entry)
+ {
+ memcpy(ptr, &s->ci, sizeof(*ptr));
+ sessions++;
+ ptr++;
+
+ if (sessions >= 1024)
+ break;
+ }
+ spin_unlock_irqrestore(&dev->session_lock, flags);
+
+ cmd->len = sizeof(*ptr)*sessions;
+ reply->len = sizeof(*cmd) + cmd->len;
+
+ cn_netlink_send(reply, 0);
+
+ kfree(reply);
+ } else
+ dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
+ sizeof(*msg) + sizeof(*cmd), d->cmd);
+ break;
+ case CRYPTO_GET_STAT:
+ reply = kmalloc(sizeof(*msg) + sizeof(*cmd) + sizeof(struct crypto_device_stat), GFP_ATOMIC);
+ if (reply) {
+ struct crypto_device_stat *ptr;
+
+ memcpy(reply, msg, sizeof(*reply));
+ reply->len = sizeof(*cmd) + sizeof(*ptr);
+
+ /*
+ * See protocol description in connector.c
+ */
+ reply->ack++;
+
+ cmd = (struct crypto_conn_data *)(reply + 1);
+ memcpy(cmd, d, sizeof(*cmd));
+ cmd->len = sizeof(*ptr);
+
+ ptr = (struct crypto_device_stat *)(cmd+1);
+ memcpy(ptr, &dev->stat, sizeof(*ptr));
+
+ cn_netlink_send(reply, 0);
+
+ kfree(reply);
+ } else
+ dprintk(KERN_ERR "Failed to allocate %d bytes in reply to comamnd 0x%x.\n",
+ sizeof(*msg) + sizeof(*cmd), d->cmd);
+ break;
+ default:
+ dprintk(KERN_ERR "Wrong operation 0x%04x for crypto connector.\n",
+ d->cmd);
+ return;
+ }
+
+ crypto_device_put(dev);
+}
+
+int crypto_conn_init(void)
+{
+ int err;
+
+ err = cn_add_callback(&crypto_conn_id, crypto_conn_name, crypto_conn_callback);
+ if (err)
+ return err;
+
+ dprintk(KERN_INFO "Crypto connector callback is registered.\n");
+
+ return 0;
+}
+
+void crypto_conn_fini(void)
+{
+ cn_del_callback(&crypto_conn_id);
+ dprintk(KERN_INFO "Crypto connector callback is unregistered.\n");
+}
diff -Nru /tmp/empty/crypto_conn.h linux-2.6/drivers/acrypto/crypto_conn.h
--- /tmp/empty/crypto_conn.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_conn.h 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,45 @@
+/*
+ * crypto_conn.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CRYPTO_CONN_H
+#define __CRYPTO_CONN_H
+
+#include "acrypto.h"
+
+#define CRYPTO_CONN_READ_SESSIONS 0
+#define CRYPTO_CONN_DUMP_QUEUE 1
+#define CRYPTO_GET_STAT 2
+
+struct crypto_conn_data
+{
+ char name[SCACHE_NAMELEN];
+ __u16 cmd;
+ __u16 len;
+ __u8 data[0];
+};
+
+#ifdef __KERNEL__
+
+int crypto_conn_init(void);
+void crypto_conn_fini(void);
+
+#endif /* __KERNEL__ */
+#endif /* __CRYPTO_CONN_H */
diff -Nru /tmp/empty/crypto_def.h linux-2.6/drivers/acrypto/crypto_def.h
--- /tmp/empty/crypto_def.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_def.h 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,38 @@
+/*
+ * crypto_def.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CRYPTO_DEF_H
+#define __CRYPTO_DEF_H
+
+#define CRYPTO_OP_DECRYPT 0
+#define CRYPTO_OP_ENCRYPT 1
+#define CRYPTO_OP_HMAC 2
+
+#define CRYPTO_MODE_ECB 0
+#define CRYPTO_MODE_CBC 1
+#define CRYPTO_MODE_CFB 2
+#define CRYPTO_MODE_OFB 3
+
+#define CRYPTO_TYPE_AES_128 0
+#define CRYPTO_TYPE_AES_192 1
+#define CRYPTO_TYPE_AES_256 2
+
+#endif /* __CRYPTO_DEF_H */
diff -Nru /tmp/empty/crypto_dev.c linux-2.6/drivers/acrypto/crypto_dev.c
--- /tmp/empty/crypto_dev.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_dev.c 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,387 @@
+/*
+ * crypto_dev.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/device.h>
+
+#include "acrypto.h"
+
+static LIST_HEAD(cdev_list);
+static spinlock_t cdev_lock = SPIN_LOCK_UNLOCKED;
+static u32 cdev_ids;
+
+struct list_head *crypto_device_list = &cdev_list;
+spinlock_t *crypto_device_lock = &cdev_lock;
+
+static int crypto_match(struct device *dev, struct device_driver *drv)
+{
+ return 1;
+}
+
+static int crypto_probe(struct device *dev)
+{
+ return -ENODEV;
+}
+
+static int crypto_remove(struct device *dev)
+{
+ return 0;
+}
+
+static void crypto_release(struct device *dev)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, device);
+
+ complete(&d->dev_released);
+}
+
+static void crypto_class_release(struct class *class)
+{
+}
+
+static void crypto_class_release_device(struct class_device *class_dev)
+{
+}
+
+struct class crypto_class =
+{
+ .name = "acrypto",
+ .class_release = crypto_class_release,
+ .release = crypto_class_release_device
+};
+
+struct bus_type crypto_bus_type =
+{
+ .name = "acrypto",
+ .match = crypto_match
+};
+
+struct device_driver crypto_driver = {
+ .name = "crypto_driver",
+ .bus = &crypto_bus_type,
+ .probe = crypto_probe,
+ .remove = crypto_remove,
+};
+
+struct device crypto_dev = {
+ .parent = NULL,
+ .bus = &crypto_bus_type,
+ .bus_id = "Asynchronous crypto",
+ .driver = &crypto_driver,
+ .release = &crypto_release
+};
+
+static ssize_t sessions_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%d\n", atomic_read(&d->refcnt));
+}
+static ssize_t name_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%s\n", d->name);
+}
+static ssize_t devices_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d;
+ int off = 0;
+ unsigned long flags;
+
+ spin_lock_irqsave(&cdev_lock, flags);
+ list_for_each_entry(d, &cdev_list, cdev_entry)
+ {
+ off += sprintf(buf+off, "%s ", d->name);
+ }
+ spin_unlock_irqrestore(&cdev_lock, flags);
+
+ if (!off)
+ off = sprintf(buf, "No devices registered yet.");
+
+ off += sprintf(buf+off, "\n");
+
+ return off;
+}
+
+static ssize_t kmem_failed_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%llu\n", d->stat.kmem_failed);
+}
+static ssize_t sstarted_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%llu\n", d->stat.sstarted);
+}
+static ssize_t sfinished_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%llu\n", d->stat.sfinished);
+}
+static ssize_t scompleted_show(struct class_device *dev, char *buf)
+{
+ struct crypto_device *d = container_of(dev, struct crypto_device, class_device);
+
+ return sprintf(buf, "%llu\n", d->stat.scompleted);
+}
+
+static CLASS_DEVICE_ATTR(sessions, 0444, sessions_show, NULL);
+static CLASS_DEVICE_ATTR(name, 0444, name_show, NULL);
+CLASS_DEVICE_ATTR(devices, 0444, devices_show, NULL);
+static CLASS_DEVICE_ATTR(scompleted, 0444, scompleted_show, NULL);
+static CLASS_DEVICE_ATTR(sstarted, 0444, sstarted_show, NULL);
+static CLASS_DEVICE_ATTR(sfinished, 0444, sfinished_show, NULL);
+static CLASS_DEVICE_ATTR(kmem_failed, 0444, kmem_failed_show, NULL);
+
+static inline int compare_device(struct crypto_device *d1, struct crypto_device *d2)
+{
+ if (!strncmp(d1->name, d2->name, sizeof(d1->name)))
+ return 1;
+
+ return 0;
+}
+
+static void create_device_attributes(struct crypto_device *dev)
+{
+ class_device_create_file(&dev->class_device, &class_device_attr_sessions);
+ class_device_create_file(&dev->class_device, &class_device_attr_name);
+ class_device_create_file(&dev->class_device, &class_device_attr_scompleted);
+ class_device_create_file(&dev->class_device, &class_device_attr_sstarted);
+ class_device_create_file(&dev->class_device, &class_device_attr_sfinished);
+ class_device_create_file(&dev->class_device, &class_device_attr_kmem_failed);
+}
+
+static void remove_device_attributes(struct crypto_device *dev)
+{
+ class_device_remove_file(&dev->class_device, &class_device_attr_sessions);
+ class_device_remove_file(&dev->class_device, &class_device_attr_name);
+ class_device_remove_file(&dev->class_device, &class_device_attr_scompleted);
+ class_device_remove_file(&dev->class_device, &class_device_attr_sstarted);
+ class_device_remove_file(&dev->class_device, &class_device_attr_sfinished);
+ class_device_remove_file(&dev->class_device, &class_device_attr_kmem_failed);
+}
+
+int __match_initializer(struct crypto_capability *cap, struct crypto_session_initializer *ci)
+{
+ if ( cap->operation == ci->operation &&
+ cap->type == ci->type &&
+ cap->mode == (ci->mode & 0x1fff))
+ return 1;
+
+ return 0;
+}
+
+int match_initializer(struct crypto_device *dev, struct crypto_session_initializer *ci)
+{
+ int i;
+
+ for (i=0; i<dev->cap_number; ++i)
+ {
+ struct crypto_capability *cap = &dev->cap[i];
+
+ if (__match_initializer(cap, ci))
+ {
+ if (cap->qlen >= atomic_read(&dev->refcnt) + 1)
+ return 1;
+ }
+ }
+
+ return 0;
+}
+
+
+inline void crypto_device_get(struct crypto_device *dev)
+{
+ atomic_inc(&dev->refcnt);
+}
+
+inline struct crypto_device *crypto_device_get_name(char *name)
+{
+ struct crypto_device *dev;
+ int found = 0;
+ unsigned long flags;
+
+ spin_lock_irqsave(&cdev_lock, flags);
+ list_for_each_entry(dev, &cdev_list, cdev_entry)
+ {
+ if (!strcmp(dev->name, name))
+ {
+ found = 1;
+ crypto_device_get(dev);
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&cdev_lock, flags);
+
+ if (!found)
+ return NULL;
+
+ return dev;
+}
+
+inline void crypto_device_put(struct crypto_device *dev)
+{
+ atomic_dec(&dev->refcnt);
+}
+
+
+int __crypto_device_add(struct crypto_device *dev)
+{
+ int err;
+
+ memset(&dev->stat, 0, sizeof(dev->stat));
+ spin_lock_init(&dev->stat_lock);
+ spin_lock_init(&dev->lock);
+ spin_lock_init(&dev->session_lock);
+ INIT_LIST_HEAD(&dev->session_list);
+ atomic_set(&dev->refcnt, 0);
+ dev->sid = 0;
+ dev->flags = 0;
+ init_completion(&dev->dev_released);
+ memcpy(&dev->device, &crypto_dev, sizeof(struct device));
+ dev->driver = &crypto_driver;
+
+ snprintf(dev->device.bus_id, sizeof(dev->device.bus_id), "%s", dev->name);
+ err = device_register(&dev->device);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto device %s: err=%d.\n",
+ dev->name, err);
+ return err;
+ }
+
+ snprintf(dev->class_device.class_id, sizeof(dev->class_device.class_id), "%s", dev->name);
+ dev->class_device.dev = &dev->device;
+ dev->class_device.class = &crypto_class;
+
+ err = class_device_register(&dev->class_device);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto class device %s: err=%d.\n",
+ dev->name, err);
+ device_unregister(&dev->device);
+ return err;
+ }
+
+ create_device_attributes(dev);
+
+ return 0;
+}
+
+void __crypto_device_remove(struct crypto_device *dev)
+{
+ remove_device_attributes(dev);
+ class_device_unregister(&dev->class_device);
+ device_unregister(&dev->device);
+}
+
+int crypto_device_add(struct crypto_device *dev)
+{
+ int err;
+ unsigned long flags;
+
+ err = __crypto_device_add(dev);
+ if (err)
+ return err;
+
+ spin_lock_irqsave(&cdev_lock, flags);
+ list_add(&dev->cdev_entry, &cdev_list);
+ dev->id = ++cdev_ids;
+ spin_unlock_irqrestore(&cdev_lock, flags);
+
+ dprintk(KERN_INFO "Crypto device %s was registered with ID=%x.\n", dev->name, dev->id);
+
+ return 0;
+}
+
+void crypto_device_remove(struct crypto_device *dev)
+{
+ struct crypto_device *__dev, *n;
+ unsigned long flags;
+
+ __crypto_device_remove(dev);
+
+ spin_lock_irqsave(&cdev_lock, flags);
+ list_for_each_entry_safe(__dev, n, &cdev_list, cdev_entry)
+ {
+ if (compare_device(__dev, dev))
+ {
+ list_del(&__dev->cdev_entry);
+ spin_unlock_irqrestore(&cdev_lock, flags);
+
+ /*
+ * In test cases or when crypto device driver is not written correctly,
+ * it's ->data_ready() method will not be callen anymore
+ * after device is removed from crypto device list.
+ *
+ * For such cases we either should provide ->flush() call
+ * or properly write ->data_ready() method.
+ */
+
+ while (atomic_read(&__dev->refcnt))
+ {
+
+ dprintk(KERN_INFO "Waiting for %s to become free: refcnt=%d.\n",
+ __dev->name, atomic_read(&dev->refcnt));
+
+ /*
+ * Hack zone: you need to write good ->data_ready()
+ * and crypto device driver itself.
+ *
+ * Driver shoud not buzz if it has pending sessions
+ * in it's queue and it was removed from global device list.
+ *
+ * Although I can workaround it here, for example by
+ * flushing the whole queue and drop all pending sessions.
+ */
+
+ __dev->data_ready(__dev);
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(HZ);
+ }
+
+ dprintk(KERN_ERR "Crypto device %s was unregistered.\n", dev->name);
+ return;
+ }
+ }
+ spin_unlock_irqrestore(&cdev_lock, flags);
+
+ dprintk(KERN_ERR "Crypto device %s was not registered.\n", dev->name);
+}
+
+EXPORT_SYMBOL(crypto_device_add);
+EXPORT_SYMBOL(crypto_device_remove);
+EXPORT_SYMBOL(crypto_device_get);
+EXPORT_SYMBOL(crypto_device_get_name);
+EXPORT_SYMBOL(crypto_device_put);
+EXPORT_SYMBOL(match_initializer);
+EXPORT_SYMBOL(__match_initializer);
diff -Nru /tmp/empty/crypto_lb.c linux-2.6/drivers/acrypto/crypto_lb.c
--- /tmp/empty/crypto_lb.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_lb.c 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,673 @@
+/*
+ * crypto_lb.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/workqueue.h>
+#include <linux/err.h>
+
+#include "acrypto.h"
+#include "crypto_lb.h"
+#include "crypto_stat.h"
+#include "crypto_route.h"
+
+static LIST_HEAD(crypto_lb_list);
+static spinlock_t crypto_lb_lock = SPIN_LOCK_UNLOCKED;
+static int lb_num = 0;
+static struct crypto_lb *current_lb, *default_lb;
+static struct completion thread_exited;
+static int need_exit;
+static struct workqueue_struct *crypto_lb_queue;
+static DECLARE_WAIT_QUEUE_HEAD(crypto_lb_wait_queue);
+
+extern struct list_head *crypto_device_list;
+extern spinlock_t *crypto_device_lock;
+
+extern int force_lb_remove;
+extern struct crypto_device main_crypto_device;
+
+static inline int lb_is_current(struct crypto_lb *l)
+{
+ return (l->crypto_device_list != NULL && l->crypto_device_lock != NULL);
+}
+
+static inline int lb_is_default(struct crypto_lb *l)
+{
+ return (l == default_lb);
+}
+
+static inline void __lb_set_current(struct crypto_lb *l)
+{
+ struct crypto_lb *c = current_lb;
+
+ if (c)
+ {
+ l->crypto_device_list = crypto_device_list;
+ l->crypto_device_lock = crypto_device_lock;
+ current_lb = l;
+ c->crypto_device_list = NULL;
+ c->crypto_device_lock = NULL;
+ }
+ else
+ {
+ l->crypto_device_list = crypto_device_list;
+ l->crypto_device_lock = crypto_device_lock;
+ current_lb = l;
+ }
+}
+
+static inline void lb_set_current(struct crypto_lb *l)
+{
+ struct crypto_lb *c = current_lb;
+ unsigned long flags;
+
+ if (c)
+ {
+ spin_lock_irqsave(&c->lock, flags);
+ __lb_set_current(l);
+ spin_unlock_irqrestore(&c->lock, flags);
+ }
+ else
+ __lb_set_current(l);
+}
+
+static inline void __lb_set_default(struct crypto_lb *l)
+{
+ default_lb = l;
+}
+
+static inline void lb_set_default(struct crypto_lb *l)
+{
+ struct crypto_lb *c = default_lb;
+ unsigned long flags;
+
+ if (c)
+ {
+ spin_lock_irqsave(&c->lock, flags);
+ __lb_set_default(l);
+ spin_unlock_irqrestore(&c->lock, flags);
+ }
+ else
+ __lb_set_default(l);
+}
+
+
+static int crypto_lb_match(struct device *dev, struct device_driver *drv)
+{
+ return 1;
+}
+
+static int crypto_lb_probe(struct device *dev)
+{
+ return -ENODEV;
+}
+
+static int crypto_lb_remove(struct device *dev)
+{
+ return 0;
+}
+
+static void crypto_lb_release(struct device *dev)
+{
+ struct crypto_lb *d = container_of(dev, struct crypto_lb, device);
+
+ complete(&d->dev_released);
+}
+
+static void crypto_lb_class_release(struct class *class)
+{
+}
+
+static void crypto_lb_class_release_device(struct class_device *class_dev)
+{
+}
+
+struct class crypto_lb_class =
+{
+ .name = "crypto_lb",
+ .class_release = crypto_lb_class_release,
+ .release = crypto_lb_class_release_device
+};
+
+struct bus_type crypto_lb_bus_type =
+{
+ .name = "crypto_lb",
+ .match = crypto_lb_match
+};
+
+struct device_driver crypto_lb_driver = {
+ .name = "crypto_lb_driver",
+ .bus = &crypto_lb_bus_type,
+ .probe = crypto_lb_probe,
+ .remove = crypto_lb_remove,
+};
+
+struct device crypto_lb_dev = {
+ .parent = NULL,
+ .bus = &crypto_lb_bus_type,
+ .bus_id = "crypto load balancer",
+ .driver = &crypto_lb_driver,
+ .release = &crypto_lb_release
+};
+
+static ssize_t name_show(struct class_device *dev, char *buf)
+{
+ struct crypto_lb *lb = container_of(dev, struct crypto_lb, class_device);
+
+ return sprintf(buf, "%s\n", lb->name);
+}
+
+static ssize_t current_show(struct class_device *dev, char *buf)
+{
+ struct crypto_lb *lb;
+ int off = 0;
+ unsigned long flags;
+
+ spin_lock_irqsave(&crypto_lb_lock, flags);
+ list_for_each_entry(lb, &crypto_lb_list, lb_entry)
+ {
+ if (lb_is_current(lb))
+ off += sprintf(buf+off, "[");
+ if (lb_is_default(lb))
+ off += sprintf(buf+off, "(");
+ off += sprintf(buf+off, "%s", lb->name);
+ if (lb_is_default(lb))
+ off += sprintf(buf+off, ")");
+ if (lb_is_current(lb))
+ off += sprintf(buf+off, "]");
+ }
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
+
+ if (!off)
+ off = sprintf(buf, "No load balancers regitered yet.");
+
+ off += sprintf(buf+off, "\n");
+
+ return off;
+}
+static ssize_t current_store(struct class_device *dev, const char *buf, size_t count)
+{
+ struct crypto_lb *lb;
+ unsigned long flags;
+
+ spin_lock_irqsave(&crypto_lb_lock, flags);
+ list_for_each_entry(lb, &crypto_lb_list, lb_entry)
+ {
+ if (count == strlen(lb->name) && !strcmp(buf, lb->name))
+ {
+ lb_set_current(lb);
+ lb_set_default(lb);
+
+ dprintk(KERN_INFO "Load balancer %s is set as current and default.\n",
+ lb->name);
+
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
+
+ return count;
+}
+
+static CLASS_DEVICE_ATTR(name, 0444, name_show, NULL);
+CLASS_DEVICE_ATTR(lbs, 0644, current_show, current_store);
+
+static void create_device_attributes(struct crypto_lb *lb)
+{
+ class_device_create_file(&lb->class_device, &class_device_attr_name);
+}
+
+static void remove_device_attributes(struct crypto_lb *lb)
+{
+ class_device_remove_file(&lb->class_device, &class_device_attr_name);
+}
+
+static inline int compare_lb(struct crypto_lb *l1, struct crypto_lb *l2)
+{
+ if (!strncmp(l1->name, l2->name, sizeof(l1->name)))
+ return 1;
+
+ return 0;
+}
+
+inline void crypto_lb_rehash(void)
+{
+ unsigned long flags;
+
+ if (!current_lb)
+ return;
+
+ spin_lock_irqsave(¤t_lb->lock, flags);
+ current_lb->rehash(current_lb);
+ spin_unlock_irqrestore(¤t_lb->lock, flags);
+
+ wake_up_interruptible(&crypto_lb_wait_queue);
+}
+
+struct crypto_device *crypto_lb_find_device(struct crypto_session_initializer *ci, struct crypto_data *data)
+{
+ struct crypto_device *dev;
+ unsigned long flags;
+
+ if (!current_lb)
+ return NULL;
+
+ if (sci_binded(ci))
+ {
+ int found = 0;
+
+ spin_lock_irqsave(crypto_device_lock, flags);
+ list_for_each_entry(dev, crypto_device_list, cdev_entry)
+ {
+ if (dev->id == ci->bdev)
+ {
+ found = 1;
+ break;
+ }
+ }
+ spin_unlock_irqrestore(crypto_device_lock, flags);
+
+ return (found)?dev:NULL;
+ }
+
+ spin_lock_irqsave(¤t_lb->lock, flags);
+
+ current_lb->rehash(current_lb);
+
+ spin_lock(crypto_device_lock);
+
+ dev = current_lb->find_device(current_lb, ci, data);
+ if (dev)
+ crypto_device_get(dev);
+
+ spin_unlock(crypto_device_lock);
+ spin_unlock_irqrestore(¤t_lb->lock, flags);
+
+ wake_up_interruptible(&crypto_lb_wait_queue);
+
+ return dev;
+}
+
+static int __crypto_lb_register(struct crypto_lb *lb)
+{
+ int err;
+
+ spin_lock_init(&lb->lock);
+
+ init_completion(&lb->dev_released);
+ memcpy(&lb->device, &crypto_lb_dev, sizeof(struct device));
+ lb->driver = &crypto_lb_driver;
+
+ snprintf(lb->device.bus_id, sizeof(lb->device.bus_id), "%s", lb->name);
+ err = device_register(&lb->device);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto load balancer device %s: err=%d.\n",
+ lb->name, err);
+ return err;
+ }
+
+ snprintf(lb->class_device.class_id, sizeof(lb->class_device.class_id), "%s", lb->name);
+ lb->class_device.dev = &lb->device;
+ lb->class_device.class = &crypto_lb_class;
+
+ err = class_device_register(&lb->class_device);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto load balancer class device %s: err=%d.\n",
+ lb->name, err);
+ device_unregister(&lb->device);
+ return err;
+ }
+
+ create_device_attributes(lb);
+ wake_up_interruptible(&crypto_lb_wait_queue);
+
+ return 0;
+
+}
+
+static void __crypto_lb_unregister(struct crypto_lb *lb)
+{
+ wake_up_interruptible(&crypto_lb_wait_queue);
+ remove_device_attributes(lb);
+ class_device_unregister(&lb->class_device);
+ device_unregister(&lb->device);
+}
+
+int crypto_lb_register(struct crypto_lb *lb, int set_current, int set_default)
+{
+ struct crypto_lb *__lb;
+ int err;
+ unsigned long flags;
+
+ spin_lock_irqsave(&crypto_lb_lock, flags);
+ list_for_each_entry(__lb, &crypto_lb_list, lb_entry)
+ {
+ if (unlikely(compare_lb(__lb, lb)))
+ {
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
+ dprintk(KERN_ERR "Crypto load balancer %s is already registered.\n", lb->name);
+ return -EINVAL;
+ }
+ }
+
+ list_add(&lb->lb_entry, &crypto_lb_list);
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
+
+ err = __crypto_lb_register(lb);
+ if (err)
+ {
+ spin_lock_irqsave(&crypto_lb_lock, flags);
+ list_del(&lb->lb_entry);
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
+
+ return err;
+ }
+
+ if (!default_lb || set_default)
+ lb_set_default(lb);
+
+ if (!current_lb || set_current)
+ lb_set_current(lb);
+
+ dprintk(KERN_INFO "Crypto load balancer %s was registered and set to be [%s.%s].\n",
+ lb->name,
+ (lb_is_current(lb))?"current":"not current",
+ (lb_is_default(lb))?"default":"not default");
+
+ lb_num++;
+
+ return 0;
+}
+
+void crypto_lb_unregister(struct crypto_lb *lb)
+{
+ struct crypto_lb *__lb, *n;
+ unsigned long flags;
+
+ if (lb_num == 1)
+ {
+ dprintk(KERN_INFO "You are removing crypto load balancer %s which is current and default.\n"
+ "There is no other crypto load balancers. "
+ "Removing %s delayed untill new load balancer is registered.\n",
+ lb->name, (force_lb_remove)?"is not":"is");
+ while (lb_num == 1 && !force_lb_remove)
+ {
+ set_current_state(TASK_INTERRUPTIBLE);
+ schedule_timeout(HZ);
+
+ if (signal_pending(current))
+ flush_signals(current);
+ }
+ }
+
+ __crypto_lb_unregister(lb);
+
+ spin_lock_irqsave(&crypto_lb_lock, flags);
+ list_for_each_entry_safe(__lb, n, &crypto_lb_list, lb_entry)
+ {
+ if (compare_lb(__lb, lb))
+ {
+ lb_num--;
+ list_del(&__lb->lb_entry);
+
+ dprintk(KERN_ERR "Crypto load balancer %s was unregistered.\n", lb->name);
+ }
+ else if (lb_num)
+ {
+ if (lb_is_default(lb))
+ lb_set_default(__lb);
+ if (lb_is_current(lb))
+ lb_set_current(default_lb);
+ }
+ }
+ spin_unlock_irqrestore(&crypto_lb_lock, flags);
+}
+
+static void crypto_lb_queue_wrapper(void *data)
+{
+ struct crypto_device *dev = &main_crypto_device;
+ struct crypto_session *s = (struct crypto_session *)data;
+
+ dprintk(KERN_INFO "%s: Calling callback for session %llu [%llu] flags=%x, "
+ "op=%04u, type=%04x, mode=%04x, priority=%04x\n",
+ __func__,
+ s->ci.id, s->ci.dev_id, s->ci.flags,
+ s->ci.operation, s->ci.type, s->ci.mode, s->ci.priority);
+
+ /*
+ * Prevent races with crypto devices
+ * which may change flags of the sessions in theirs queues.
+ */
+ spin_lock(&s->lock);
+ crypto_stat_finish_inc(s);
+ finish_session(s);
+ unstart_session(s);
+ spin_unlock(&s->lock);
+
+ s->ci.callback(&s->ci, &s->data);
+
+ if (session_finished(s))
+ {
+ crypto_session_dequeue_route(s);
+ kfree(s);
+ }
+ else
+ {
+ /*
+ * Special case: crypto consumer marks session as "not finished"
+ * in it's callback - it means that crypto consumer wants
+ * this session to be processed further,
+ * for example crypto consumer can add new route and then
+ * mark session as "not finished".
+ */
+
+ uncomplete_session(s);
+ unstart_session(s);
+ crypto_session_insert_main(dev, s);
+ }
+
+ wake_up_interruptible(&crypto_lb_wait_queue);
+}
+
+static void crypto_lb_process_next_route(struct crypto_session *s)
+{
+ struct crypto_route *rt;
+ struct crypto_device *dev;
+ unsigned long flags;
+
+ rt = crypto_route_dequeue(s);
+ if (rt)
+ {
+ dev = rt->dev;
+
+ spin_lock_irqsave(&dev->session_lock, flags);
+ list_del(&s->dev_queue_entry);
+ spin_unlock_irqrestore(&dev->session_lock, flags);
+ crypto_route_free(rt);
+
+ dev = crypto_route_get_current_device(s);
+ if (dev)
+ {
+ dprintk(KERN_INFO "%s: processing new route to %s.\n",
+ __func__, dev->name);
+
+ memcpy(&s->ci, &rt->ci, sizeof(s->ci));
+ crypto_session_insert(dev, s);
+
+ /*
+ * Reference to this device was already hold when
+ * new routing was added.
+ */
+ crypto_device_put(dev);
+ }
+ }
+}
+
+void crypto_wake_lb(void)
+{
+ wake_up_interruptible(&crypto_lb_wait_queue);
+}
+
+int crypto_lb_thread(void *data)
+{
+ struct crypto_session *s, *n;
+ struct crypto_device *dev = (struct crypto_device *)data;
+ unsigned long flags;
+
+ daemonize("%s", dev->name);
+ allow_signal(SIGTERM);
+
+ while (!need_exit)
+ {
+ spin_lock_irqsave(&dev->session_lock, flags);
+ list_for_each_entry_safe(s, n, &dev->session_list, main_queue_entry)
+ {
+ dprintk("session %llu [%llu]: flags=%x, route_num=%d, %s,%s,%s,%s.\n",
+ s->ci.id, s->ci.dev_id, s->ci.flags,
+ crypto_route_queue_len(s),
+ (session_completed(s))?"completed": "not completed",
+ (session_finished(s))?"finished": "not finished",
+ (session_started(s))?"started": "not started",
+ (session_is_processed(s))?"is being processed": "is not being processed");
+
+ if (!spin_trylock(&s->lock))
+ continue;
+
+ if (session_is_processed(s))
+ goto unlock;
+ if (session_started(s))
+ goto unlock;
+
+ if (session_completed(s))
+ {
+ crypto_stat_ptime_inc(s);
+ uncomplete_session(s);
+
+ if (crypto_route_queue_len(s) > 1)
+ {
+ crypto_lb_process_next_route(s);
+ }
+ else
+ {
+ start_session(s);
+ crypto_stat_start_inc(s);
+
+ __crypto_session_dequeue_main(s);
+ spin_unlock(&s->lock);
+
+ INIT_WORK(&s->work, &crypto_lb_queue_wrapper, s);
+ queue_work(crypto_lb_queue, &s->work);
+ continue;
+ }
+ }
+unlock:
+ spin_unlock(&s->lock);
+ }
+ spin_unlock_irqrestore(&dev->session_lock, flags);
+
+ interruptible_sleep_on_timeout(&crypto_lb_wait_queue, 10);
+ }
+
+ flush_workqueue(crypto_lb_queue);
+ complete_and_exit(&thread_exited, 0);
+}
+
+int crypto_lb_init(void)
+{
+ int err;
+ long pid;
+
+ err = bus_register(&crypto_lb_bus_type);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto load balancer bus: err=%d.\n", err);
+ goto err_out_exit;
+ }
+
+ err = driver_register(&crypto_lb_driver);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto load balancer driver: err=%d.\n", err);
+ goto err_out_bus_unregister;
+ }
+
+ crypto_lb_class.class_dev_attrs = &class_device_attr_lbs;
+
+ err = class_register(&crypto_lb_class);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto load balancer class: err=%d.\n", err);
+ goto err_out_driver_unregister;
+ }
+
+ crypto_lb_queue = create_workqueue("clbq");
+ if (!crypto_lb_queue)
+ {
+ dprintk(KERN_ERR "Failed to create crypto load balaner work queue.\n");
+ goto err_out_class_unregister;
+ }
+
+ init_completion(&thread_exited);
+ pid = kernel_thread(crypto_lb_thread, &main_crypto_device, CLONE_FS | CLONE_FILES);
+ if (IS_ERR((void *)pid))
+ {
+ dprintk(KERN_ERR "Failed to create kernel load balancing thread.\n");
+ goto err_out_destroy_workqueue;
+ }
+
+
+ return 0;
+
+err_out_destroy_workqueue:
+ destroy_workqueue(crypto_lb_queue);
+err_out_class_unregister:
+ class_unregister(&crypto_lb_class);
+err_out_driver_unregister:
+ driver_unregister(&crypto_lb_driver);
+err_out_bus_unregister:
+ bus_unregister(&crypto_lb_bus_type);
+err_out_exit:
+ return err;
+}
+
+void crypto_lb_fini(void)
+{
+ need_exit = 1;
+ wait_for_completion(&thread_exited);
+ destroy_workqueue(crypto_lb_queue);
+ class_unregister(&crypto_lb_class);
+ driver_unregister(&crypto_lb_driver);
+ bus_unregister(&crypto_lb_bus_type);
+}
+
+EXPORT_SYMBOL(crypto_lb_register);
+EXPORT_SYMBOL(crypto_lb_unregister);
+EXPORT_SYMBOL(crypto_lb_rehash);
+EXPORT_SYMBOL(crypto_lb_find_device);
+EXPORT_SYMBOL(crypto_wake_lb);
diff -Nru /tmp/empty/crypto_lb.h linux-2.6/drivers/acrypto/crypto_lb.h
--- /tmp/empty/crypto_lb.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_lb.h 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,61 @@
+/*
+ * crypto_lb.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CRYPTO_LB_H
+#define __CRYPTO_LB_H
+
+#include "acrypto.h"
+
+#define CRYPTO_LB_NAMELEN 32
+
+struct crypto_lb
+{
+ struct list_head lb_entry;
+
+ char name[CRYPTO_LB_NAMELEN];
+
+ void (*rehash)(struct crypto_lb *);
+ struct crypto_device * (*find_device)(struct crypto_lb *, struct crypto_session_initializer *, struct crypto_data *);
+
+ spinlock_t lock;
+
+ spinlock_t *crypto_device_lock;
+ struct list_head *crypto_device_list;
+
+ struct device_driver *driver;
+ struct device device;
+ struct class_device class_device;
+ struct completion dev_released;
+
+};
+
+int crypto_lb_register(struct crypto_lb *lb, int set_current, int set_default);
+void crypto_lb_unregister(struct crypto_lb *lb);
+
+inline void crypto_lb_rehash(void);
+struct crypto_device *crypto_lb_find_device(struct crypto_session_initializer *ci, struct crypto_data *data);
+
+void crypto_wake_lb(void);
+
+int crypto_lb_init(void);
+void crypto_lb_fini(void);
+
+#endif /* __CRYPTO_LB_H */
diff -Nru /tmp/empty/crypto_main.c linux-2.6/drivers/acrypto/crypto_main.c
--- /tmp/empty/crypto_main.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_main.c 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,359 @@
+/*
+ * crypto_main.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+
+#include "acrypto.h"
+#include "crypto_lb.h"
+#include "crypto_conn.h"
+#include "crypto_route.h"
+
+int force_lb_remove;
+module_param(force_lb_remove, int, 0);
+
+struct crypto_device main_crypto_device;
+
+extern struct bus_type crypto_bus_type;
+extern struct device_driver crypto_driver;
+extern struct class crypto_class;
+extern struct device crypto_dev;
+
+extern struct class_device_attribute class_device_attr_devices;
+extern struct class_device_attribute class_device_attr_lbs;
+
+static inline void dump_ci(struct crypto_session_initializer *ci)
+{
+ dprintk("%llu [%llu] op=%04u, type=%04x, mode=%04x, priority=%04x",
+ ci->id, ci->dev_id,
+ ci->operation, ci->type, ci->mode, ci->priority);
+}
+
+static void __crypto_session_insert(struct crypto_device *dev, struct crypto_session *s)
+{
+ struct crypto_session *__s;
+
+ if (unlikely(list_empty(&dev->session_list)))
+ {
+ list_add(&s->dev_queue_entry, &dev->session_list);
+ }
+ else
+ {
+ int inserted = 0;
+
+ list_for_each_entry(__s, &dev->session_list, dev_queue_entry)
+ {
+ if (__s->ci.priority < s->ci.priority)
+ {
+ list_add_tail(&s->dev_queue_entry, &__s->dev_queue_entry);
+ inserted = 1;
+ break;
+ }
+ }
+
+ if (!inserted)
+ list_add_tail(&s->dev_queue_entry, &dev->session_list);
+ }
+
+ dump_ci(&s->ci);
+ dprintk(" added to crypto device %s [%d].\n", dev->name, atomic_read(&dev->refcnt));
+}
+
+inline void crypto_session_insert_main(struct crypto_device *dev, struct crypto_session *s)
+{
+ struct crypto_session *__s;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->session_lock, flags);
+ crypto_device_get(dev);
+ if (unlikely(list_empty(&dev->session_list)))
+ {
+ list_add(&s->main_queue_entry, &dev->session_list);
+ }
+ else
+ {
+ int inserted = 0;
+
+ list_for_each_entry(__s, &dev->session_list, main_queue_entry)
+ {
+ if (__s->ci.priority < s->ci.priority)
+ {
+ list_add_tail(&s->main_queue_entry, &__s->main_queue_entry);
+ inserted = 1;
+ break;
+ }
+ }
+
+ if (!inserted)
+ list_add_tail(&s->main_queue_entry, &dev->session_list);
+ }
+ spin_unlock_irqrestore(&dev->session_lock, flags);
+
+ dump_ci(&s->ci);
+ dprintk(" added to main crypto device %s [%d].\n", dev->name, atomic_read(&dev->refcnt));
+}
+
+inline void crypto_session_insert(struct crypto_device *dev, struct crypto_session *s)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->session_lock, flags);
+ __crypto_session_insert(dev, s);
+ spin_unlock_irqrestore(&dev->session_lock, flags);
+
+ if (dev->data_ready)
+ dev->data_ready(dev);
+}
+
+struct crypto_session *crypto_session_create(struct crypto_session_initializer *ci, struct crypto_data *d)
+{
+ struct crypto_device *dev = &main_crypto_device;
+ struct crypto_device *ldev;
+ struct crypto_session *s;
+ int err;
+
+ if (d->priv_size > CRYPTO_MAX_PRIV_SIZE)
+ {
+ dprintk("priv_size %u is too big, maximum allowed %u.\n", d->priv_size, CRYPTO_MAX_PRIV_SIZE);
+ return NULL;
+ }
+
+ ldev = crypto_lb_find_device(ci, d);
+ if (!ldev)
+ {
+ dprintk("Cannot find suitable device.\n");
+ return NULL;
+ }
+
+ s = kmalloc(sizeof(*s) + d->priv_size, GFP_ATOMIC);
+ if (!s)
+ {
+ ldev->stat.kmem_failed++;
+ goto err_out_device_put;
+ }
+
+ memset(s, 0xAB, sizeof(*s));
+
+ crypto_route_head_init(&s->route_list);
+
+ spin_lock_init(&s->lock);
+ memcpy(&s->ci, ci, sizeof(s->ci));
+ memcpy(&s->data, d, sizeof(s->data));
+ if (d->priv_size)
+ {
+ s->data.priv = s+1;
+ if (d->priv)
+ memcpy(s->data.priv, d->priv, d->priv_size);
+ }
+
+ s->ci.id = dev->sid++;
+ s->ci.dev_id = ldev->sid++;
+ s->ci.flags = 0;
+
+ err = crypto_route_add(ldev, s, ci);
+ if (err)
+ {
+ dprintk("Can not add route to device %s.\n", ldev->name);
+ goto err_out_session_free;
+ }
+
+ return s;
+
+err_out_session_free:
+ crypto_device_put(ldev);
+err_out_device_put:
+ kfree(s);
+
+ return NULL;
+}
+
+void crypto_session_add(struct crypto_session *s)
+{
+ struct crypto_device *ldev;
+ struct crypto_device *dev = &main_crypto_device;
+
+ ldev = crypto_route_get_current_device(s);
+ BUG_ON(!ldev); /* This can not happen. */
+
+ crypto_session_insert(ldev, s);
+
+ /*
+ * Crypto device's refcnt was incremented in
+ * crypto_lb_find_device(),
+ * get_current_device(),
+ * crypto_session_insert(),
+ * thus we need to decrement it's refcnt twice.
+ */
+ crypto_device_put(ldev);
+ crypto_device_put(ldev);
+
+ crypto_session_insert_main(dev, s);
+}
+
+struct crypto_session *crypto_session_alloc(struct crypto_session_initializer *ci, struct crypto_data *d)
+{
+ struct crypto_session *s;
+
+ s = crypto_session_create(ci, d);
+ if (!s)
+ return NULL;
+
+ crypto_session_add(s);
+
+ return s;
+}
+
+void crypto_session_dequeue_route(struct crypto_session *s)
+{
+ struct crypto_route *rt;
+ struct crypto_device *dev;
+ unsigned long flags;
+
+ BUG_ON(crypto_route_queue_len(s) > 1);
+
+ while ((rt = crypto_route_dequeue(s)))
+ {
+ dev = rt->dev;
+
+ dprintk(KERN_INFO "Removing route entry for device %s.\n", dev->name);
+
+ spin_lock_irqsave(&dev->session_lock, flags);
+ list_del(&s->dev_queue_entry);
+ spin_unlock_irqrestore(&dev->session_lock, flags);
+
+ crypto_route_free(rt);
+ }
+}
+
+inline void __crypto_session_dequeue_main(struct crypto_session *s)
+{
+ struct crypto_device *dev = &main_crypto_device;
+
+ list_del(&s->main_queue_entry);
+ crypto_device_put(dev);
+}
+
+inline void crypto_session_dequeue_main(struct crypto_session *s)
+{
+ struct crypto_device *dev = &main_crypto_device;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->session_lock, flags);
+ __crypto_session_dequeue_main(s);
+ spin_unlock_irqrestore(&dev->session_lock, flags);
+}
+
+int __devinit cmain_init(void)
+{
+ struct crypto_device *dev = &main_crypto_device;
+ int err;
+
+ snprintf(dev->name, sizeof(dev->name), "crypto_sessions");
+
+ err = bus_register(&crypto_bus_type);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto bus: err=%d.\n", err);
+ return err;
+ }
+
+ err = driver_register(&crypto_driver);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto driver: err=%d.\n", err);
+ goto err_out_bus_unregister;
+ }
+
+ err = class_register(&crypto_class);
+ if (err)
+ {
+ dprintk(KERN_ERR "Failed to register crypto class: err=%d.\n", err);
+ goto err_out_driver_unregister;
+ }
+
+ err = crypto_lb_init();
+ if (err)
+ goto err_out_class_unregister;
+
+ err = crypto_conn_init();
+ if (err)
+ goto err_out_crypto_lb_fini;
+
+ err = __crypto_device_add(dev);
+ if (err)
+ goto err_out_crypto_conn_fini;
+
+ err = class_device_create_file(&dev->class_device, &class_device_attr_devices);
+ if (err)
+ dprintk("Failed to create \"devices\" attribute: err=%d.\n", err);
+ err = class_device_create_file(&dev->class_device, &class_device_attr_lbs);
+ if (err)
+ dprintk("Failed to create \"lbs\" attribute: err=%d.\n", err);
+
+
+ return 0;
+
+err_out_crypto_conn_fini:
+ crypto_conn_fini();
+err_out_crypto_lb_fini:
+ crypto_lb_fini();
+err_out_class_unregister:
+ class_unregister(&crypto_class);
+err_out_driver_unregister:
+ driver_unregister(&crypto_driver);
+err_out_bus_unregister:
+ bus_unregister(&crypto_bus_type);
+
+ return err;
+}
+
+void __devexit cmain_fini(void)
+{
+ struct crypto_device *dev = &main_crypto_device;
+
+ class_device_remove_file(&dev->class_device, &class_device_attr_devices);
+ class_device_remove_file(&dev->class_device, &class_device_attr_lbs);
+ __crypto_device_remove(dev);
+
+ crypto_conn_fini();
+ crypto_lb_fini();
+
+ class_unregister(&crypto_class);
+ driver_unregister(&crypto_driver);
+ bus_unregister(&crypto_bus_type);
+}
+
+module_init(cmain_init);
+module_exit(cmain_fini);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
+MODULE_DESCRIPTION("Asynchronous crypto layer.");
+
+EXPORT_SYMBOL(crypto_session_alloc);
+EXPORT_SYMBOL(crypto_session_create);
+EXPORT_SYMBOL(crypto_session_add);
diff -Nru /tmp/empty/crypto_route.h linux-2.6/drivers/acrypto/crypto_route.h
--- /tmp/empty/crypto_route.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_route.h 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,220 @@
+/*
+ * crypto_route.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CRYPTO_ROUTE_H
+#define __CRYPTO_ROUTE_H
+
+#include <linux/types.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include "acrypto.h"
+
+static inline struct crypto_route *crypto_route_alloc(struct crypto_device *dev, struct crypto_session_initializer *ci)
+{
+ struct crypto_route *rt;
+
+ crypto_device_get(dev);
+ if (!match_initializer(dev, ci))
+ {
+ crypto_device_put(dev);
+ return NULL;
+ }
+
+ rt = kmalloc(sizeof(*rt), GFP_ATOMIC);
+ if (!rt)
+ {
+ crypto_device_put(dev);
+ return NULL;
+ }
+
+ memset(rt, 0, sizeof(*rt));
+ memcpy(&rt->ci, ci, sizeof(*ci));
+
+ rt->dev = dev;
+
+ return rt;
+}
+
+static inline void crypto_route_free(struct crypto_route *rt)
+{
+ crypto_device_put(rt->dev);
+ rt->dev = NULL;
+ kfree(rt);
+}
+
+static inline void __crypto_route_del(struct crypto_route *rt, struct crypto_route_head *list)
+{
+ struct crypto_route *next, *prev;
+
+ list->qlen--;
+ next = rt->next;
+ prev = rt->prev;
+ rt->next = rt->prev = NULL;
+ rt->list = NULL;
+ next->prev = prev;
+ prev->next = next;
+}
+
+static inline void crypto_route_del(struct crypto_route *rt)
+{
+ struct crypto_route_head *list = rt->list;
+ unsigned long flags;
+
+ if (list)
+ {
+ spin_lock_irqsave(&list->lock, flags);
+ if (list == rt->list)
+ __crypto_route_del(rt, rt->list);
+ spin_unlock_irqrestore(&list->lock, flags);
+
+ crypto_route_free(rt);
+ }
+}
+
+static inline struct crypto_route *__crypto_route_dequeue(struct crypto_route_head *list)
+{
+ struct crypto_route *next, *prev, *result;
+
+ prev = (struct crypto_route *) list;
+ next = prev->next;
+ result = NULL;
+ if (next != prev) {
+ result = next;
+ next = next->next;
+ list->qlen--;
+ next->prev = prev;
+ prev->next = next;
+ result->next = result->prev = NULL;
+ result->list = NULL;
+ }
+ return result;
+}
+
+static inline struct crypto_route *crypto_route_dequeue(struct crypto_session *s)
+{
+ struct crypto_route *rt;
+ unsigned long flags;
+
+ spin_lock_irqsave(&s->route_list.lock, flags);
+ rt = __crypto_route_dequeue(&s->route_list);
+ spin_unlock_irqrestore(&s->route_list.lock, flags);
+
+ return rt;
+}
+
+static inline void __crypto_route_queue(struct crypto_route *rt, struct crypto_route_head *list)
+{
+ struct crypto_route *prev, *next;
+
+ rt->list = list;
+ list->qlen++;
+ next = (struct crypto_route *)list;
+ prev = next->prev;
+ rt->next = next;
+ rt->prev = prev;
+ next->prev = prev->next = rt;
+}
+
+static inline void crypto_route_queue(struct crypto_route *rt, struct crypto_session *s)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&s->route_list.lock, flags);
+ __crypto_route_queue(rt, &s->route_list);
+ spin_unlock_irqrestore(&s->route_list.lock, flags);
+}
+
+static inline int crypto_route_add(struct crypto_device *dev, struct crypto_session *s, struct crypto_session_initializer *ci)
+{
+ struct crypto_route *rt;
+
+ rt = crypto_route_alloc(dev, ci);
+ if (!rt)
+ return -ENOMEM;
+
+ crypto_route_queue(rt, s);
+
+ return 0;
+}
+
+static inline int crypto_route_queue_len(struct crypto_session *s)
+{
+ return s->route_list.qlen;
+}
+
+static inline void crypto_route_head_init(struct crypto_route_head *list)
+{
+ spin_lock_init(&list->lock);
+ list->prev = list->next = (struct crypto_route *)list;
+ list->qlen = 0;
+}
+
+static inline struct crypto_route *__crypto_route_current(struct crypto_route_head *list)
+{
+ struct crypto_route *next, *prev, *result;
+
+ prev = (struct crypto_route *) list;
+ next = prev->next;
+ result = NULL;
+ if (next != prev)
+ result = next;
+
+ return result;
+}
+
+static inline struct crypto_route *crypto_route_current(struct crypto_session *s)
+{
+ struct crypto_route_head *list;
+ struct crypto_route *rt = NULL;
+ unsigned long flags;
+
+ list = &s->route_list;
+
+ if (list)
+ {
+ spin_lock_irqsave(&list->lock, flags);
+ rt = __crypto_route_current(list);
+ spin_unlock_irqrestore(&list->lock, flags);
+ }
+
+ return rt;
+}
+
+static inline struct crypto_device *crypto_route_get_current_device(struct crypto_session *s)
+{
+ struct crypto_route *rt = NULL;
+ struct crypto_device *dev = NULL;
+ struct crypto_route_head *list = &s->route_list;
+ unsigned long flags;
+
+ spin_lock_irqsave(&list->lock, flags);
+ rt = __crypto_route_current(list);
+ if (rt)
+ {
+ dev = rt->dev;
+ crypto_device_get(dev);
+ }
+ spin_unlock_irqrestore(&list->lock, flags);
+ return dev;
+}
+
+#endif /* __CRYPTO_ROUTE_H */
diff -Nru /tmp/empty/crypto_stat.c linux-2.6/drivers/acrypto/crypto_stat.c
--- /tmp/empty/crypto_stat.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_stat.c 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,110 @@
+/*
+ * crypto_stat.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+
+#include "acrypto.h"
+#include "crypto_route.h"
+
+inline void crypto_stat_start_inc(struct crypto_session *s)
+{
+ struct crypto_device *dev;
+ unsigned long flags;
+
+ dev = crypto_route_get_current_device(s);
+ if (dev)
+ {
+ spin_lock_irqsave(&dev->stat_lock, flags);
+ dev->stat.sstarted++;
+ spin_unlock_irqrestore(&dev->stat_lock, flags);
+
+ crypto_device_put(dev);
+ }
+}
+
+inline void crypto_stat_finish_inc(struct crypto_session *s)
+{
+ struct crypto_device *dev;
+ unsigned long flags;
+
+ dev = crypto_route_get_current_device(s);
+ if (dev)
+ {
+ spin_lock_irqsave(&dev->stat_lock, flags);
+ dev->stat.sfinished++;
+ spin_unlock_irqrestore(&dev->stat_lock, flags);
+
+ crypto_device_put(dev);
+ }
+}
+
+inline void crypto_stat_complete_inc(struct crypto_session *s)
+{
+ struct crypto_device *dev;
+ unsigned long flags;
+
+ dev = crypto_route_get_current_device(s);
+ if (dev)
+ {
+ spin_lock_irqsave(&dev->stat_lock, flags);
+ dev->stat.scompleted++;
+ spin_unlock_irqrestore(&dev->stat_lock, flags);
+
+ crypto_device_put(dev);
+ }
+}
+
+inline void crypto_stat_ptime_inc(struct crypto_session *s)
+{
+ struct crypto_device *dev;
+ unsigned long flags;
+
+ dev = crypto_route_get_current_device(s);
+ if (dev)
+ {
+ int i;
+
+ spin_lock_irqsave(&dev->stat_lock, flags);
+ for (i=0; i<dev->cap_number; ++i)
+ {
+ if (__match_initializer(&dev->cap[i], &s->ci))
+ {
+ dev->cap[i].ptime += s->ci.ptime;
+ dev->cap[i].scomp++;
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&dev->stat_lock, flags);
+
+ crypto_device_put(dev);
+ }
+}
+
+EXPORT_SYMBOL(crypto_stat_start_inc);
+EXPORT_SYMBOL(crypto_stat_finish_inc);
+EXPORT_SYMBOL(crypto_stat_complete_inc);
diff -Nru /tmp/empty/crypto_stat.h linux-2.6/drivers/acrypto/crypto_stat.h
--- /tmp/empty/crypto_stat.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/crypto_stat.h 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,32 @@
+/*
+ * crypto_stat.h
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __CRYPTO_STAT_H
+#define __CRYPTO_STAT_H
+
+#include "acrypto.h"
+
+inline void crypto_stat_start_inc(struct crypto_session *s);
+inline void crypto_stat_finish_inc(struct crypto_session *s);
+inline void crypto_stat_complete_inc(struct crypto_session *s);
+inline void crypto_stat_ptime_inc(struct crypto_session *s);
+
+#endif /* __CRYPTO_STAT_H */
diff -Nru /tmp/empty/simple_lb.c linux-2.6/drivers/acrypto/simple_lb.c
--- /tmp/empty/simple_lb.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/simple_lb.c 2004-11-02 18:44:21.000000000 +0300
@@ -0,0 +1,88 @@
+/*
+ * simple_lb.c
+ *
+ * Copyright (c) 2004 Evgeniy Polyakov <johnpol@2ka.mipt.ru>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include "crypto_lb.h"
+
+static void simple_lb_rehash(struct crypto_lb *);
+static struct crypto_device *simple_lb_find_device(struct crypto_lb *, struct crypto_session_initializer *, struct crypto_data *);
+
+struct crypto_lb simple_lb =
+{
+ .name = "simple_lb",
+ .rehash = simple_lb_rehash,
+ .find_device = simple_lb_find_device
+};
+
+static void simple_lb_rehash(struct crypto_lb *lb)
+{
+}
+
+static struct crypto_device *simple_lb_find_device(struct crypto_lb *lb, struct crypto_session_initializer *ci, struct crypto_data *data)
+{
+ struct crypto_device *dev, *__dev;
+ int min = 0x7ffffff;
+
+ __dev = NULL;
+ list_for_each_entry(dev, lb->crypto_device_list, cdev_entry)
+ {
+ if (device_broken(dev))
+ continue;
+ if (!match_initializer(dev, ci))
+ continue;
+
+ if (atomic_read(&dev->refcnt) < min)
+ {
+ min = atomic_read(&dev->refcnt);
+ __dev = dev;
+ }
+ }
+
+ return __dev;
+}
+
+int __devinit simple_lb_init(void)
+{
+ dprintk(KERN_INFO "Registering simple crypto load balancer.\n");
+
+ return crypto_lb_register(&simple_lb, 1, 1);
+}
+
+void __devexit simple_lb_fini(void)
+{
+ dprintk(KERN_INFO "Unregistering simple crypto load balancer.\n");
+
+ crypto_lb_unregister(&simple_lb);
+}
+
+module_init(simple_lb_init);
+module_exit(simple_lb_fini);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Evgeniy Polyakov <johnpol@2ka.mipt.ru>");
+MODULE_DESCRIPTION("Simple crypto load balancer.");
diff -Nru /tmp/empty/via-padlock/Makefile linux-2.6/drivers/acrypto/via-padlock/Makefile
--- /tmp/empty/via-padlock/Makefile 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/via-padlock/Makefile 2004-10-30 09:03:37.000000000 +0400
@@ -0,0 +1,2 @@
+obj-$(CONFIG_VIA_PADLOCK) += padlock.o
+padlock-objs := padlock-aes.o padlock-generic.o
diff -Nru /tmp/empty/via-padlock/padlock-aes.c linux-2.6/drivers/acrypto/via-padlock/padlock-aes.c
--- /tmp/empty/via-padlock/padlock-aes.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/via-padlock/padlock-aes.c 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,552 @@
+/*
+ * Cryptographic API.
+ *
+ * Support for VIA PadLock hardware crypto engine.
+ *
+ * Linux developers:
+ * Michal Ludvig <mludvig@suse.cz>
+ *
+ * Key expansion routine taken from crypto/aes.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * ---------------------------------------------------------------------------
+ * Copyright (c) 2002, Dr Brian Gladman <brg@gladman.me.uk>, Worcester, UK.
+ * All rights reserved.
+ *
+ * LICENSE TERMS
+ *
+ * The free distribution and use of this software in both source and binary
+ * form is allowed (with or without changes) provided that:
+ *
+ * 1. distributions of this source code include the above copyright
+ * notice, this list of conditions and the following disclaimer;
+ *
+ * 2. distributions in binary form include the above copyright
+ * notice, this list of conditions and the following disclaimer
+ * in the documentation and/or other associated materials;
+ *
+ * 3. the copyright holder's name is not used to endorse products
+ * built using this software without specific written permission.
+ *
+ * ALTERNATIVELY, provided that this notice is retained in full, this product
+ * may be distributed under the terms of the GNU General Public License (GPL),
+ * in which case the provisions of the GPL apply INSTEAD OF those given above.
+ *
+ * DISCLAIMER
+ *
+ * This software is provided 'as is' with no explicit or implied warranties
+ * in respect of its properties, including, but not limited to, correctness
+ * and/or fitness for purpose.
+ * ---------------------------------------------------------------------------
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <linux/crypto.h>
+#include <asm/byteorder.h>
+#include <linux/mm.h>
+
+#include <asm/scatterlist.h>
+
+#include "padlock.h"
+
+#include "../crypto_def.h"
+#include "../acrypto.h"
+#include "../crypto_stat.h"
+
+static inline int aes_hw_extkey_available (u8 key_len);
+
+static inline
+u32 generic_rotr32 (const u32 x, const unsigned bits)
+{
+ const unsigned n = bits % 32;
+ return (x >> n) | (x << (32 - n));
+}
+
+static inline
+u32 generic_rotl32 (const u32 x, const unsigned bits)
+{
+ const unsigned n = bits % 32;
+ return (x << n) | (x >> (32 - n));
+}
+
+#define rotl generic_rotl32
+#define rotr generic_rotr32
+
+/*
+ * #define byte(x, nr) ((unsigned char)((x) >> (nr*8)))
+ */
+inline static u8
+byte(const u32 x, const unsigned n)
+{
+ return x >> (n << 3);
+}
+
+#define u32_in(x) le32_to_cpu(*(const u32 *)(x))
+#define u32_out(to, from) (*(u32 *)(to) = cpu_to_le32(from))
+
+static u8 pow_tab[256];
+static u8 log_tab[256];
+static u8 sbx_tab[256];
+static u8 isb_tab[256];
+static u32 rco_tab[10];
+static u32 ft_tab[4][256];
+static u32 it_tab[4][256];
+
+static u32 fl_tab[4][256];
+static u32 il_tab[4][256];
+
+static inline u8
+f_mult (u8 a, u8 b)
+{
+ u8 aa = log_tab[a], cc = aa + log_tab[b];
+
+ return pow_tab[cc + (cc < aa ? 1 : 0)];
+}
+
+#define ff_mult(a,b) (a && b ? f_mult(a, b) : 0)
+
+#define f_rn(bo, bi, n, k) \
+ bo[n] = ft_tab[0][byte(bi[n],0)] ^ \
+ ft_tab[1][byte(bi[(n + 1) & 3],1)] ^ \
+ ft_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
+ ft_tab[3][byte(bi[(n + 3) & 3],3)] ^ *(k + n)
+
+#define i_rn(bo, bi, n, k) \
+ bo[n] = it_tab[0][byte(bi[n],0)] ^ \
+ it_tab[1][byte(bi[(n + 3) & 3],1)] ^ \
+ it_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
+ it_tab[3][byte(bi[(n + 1) & 3],3)] ^ *(k + n)
+
+#define ls_box(x) \
+ ( fl_tab[0][byte(x, 0)] ^ \
+ fl_tab[1][byte(x, 1)] ^ \
+ fl_tab[2][byte(x, 2)] ^ \
+ fl_tab[3][byte(x, 3)] )
+
+#define f_rl(bo, bi, n, k) \
+ bo[n] = fl_tab[0][byte(bi[n],0)] ^ \
+ fl_tab[1][byte(bi[(n + 1) & 3],1)] ^ \
+ fl_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
+ fl_tab[3][byte(bi[(n + 3) & 3],3)] ^ *(k + n)
+
+#define i_rl(bo, bi, n, k) \
+ bo[n] = il_tab[0][byte(bi[n],0)] ^ \
+ il_tab[1][byte(bi[(n + 3) & 3],1)] ^ \
+ il_tab[2][byte(bi[(n + 2) & 3],2)] ^ \
+ il_tab[3][byte(bi[(n + 1) & 3],3)] ^ *(k + n)
+
+static void
+gen_tabs (void)
+{
+ u32 i, t;
+ u8 p, q;
+
+ /* log and power tables for GF(2**8) finite field with
+ 0x011b as modular polynomial - the simplest prmitive
+ root is 0x03, used here to generate the tables */
+
+ for (i = 0, p = 1; i < 256; ++i) {
+ pow_tab[i] = (u8) p;
+ log_tab[p] = (u8) i;
+
+ p ^= (p << 1) ^ (p & 0x80 ? 0x01b : 0);
+ }
+
+ log_tab[1] = 0;
+
+ for (i = 0, p = 1; i < 10; ++i) {
+ rco_tab[i] = p;
+
+ p = (p << 1) ^ (p & 0x80 ? 0x01b : 0);
+ }
+
+ for (i = 0; i < 256; ++i) {
+ p = (i ? pow_tab[255 - log_tab[i]] : 0);
+ q = ((p >> 7) | (p << 1)) ^ ((p >> 6) | (p << 2));
+ p ^= 0x63 ^ q ^ ((q >> 6) | (q << 2));
+ sbx_tab[i] = p;
+ isb_tab[p] = (u8) i;
+ }
+
+ for (i = 0; i < 256; ++i) {
+ p = sbx_tab[i];
+
+ t = p;
+ fl_tab[0][i] = t;
+ fl_tab[1][i] = rotl (t, 8);
+ fl_tab[2][i] = rotl (t, 16);
+ fl_tab[3][i] = rotl (t, 24);
+
+ t = ((u32) ff_mult (2, p)) |
+ ((u32) p << 8) |
+ ((u32) p << 16) | ((u32) ff_mult (3, p) << 24);
+
+ ft_tab[0][i] = t;
+ ft_tab[1][i] = rotl (t, 8);
+ ft_tab[2][i] = rotl (t, 16);
+ ft_tab[3][i] = rotl (t, 24);
+
+ p = isb_tab[i];
+
+ t = p;
+ il_tab[0][i] = t;
+ il_tab[1][i] = rotl (t, 8);
+ il_tab[2][i] = rotl (t, 16);
+ il_tab[3][i] = rotl (t, 24);
+
+ t = ((u32) ff_mult (14, p)) |
+ ((u32) ff_mult (9, p) << 8) |
+ ((u32) ff_mult (13, p) << 16) |
+ ((u32) ff_mult (11, p) << 24);
+
+ it_tab[0][i] = t;
+ it_tab[1][i] = rotl (t, 8);
+ it_tab[2][i] = rotl (t, 16);
+ it_tab[3][i] = rotl (t, 24);
+ }
+}
+
+#define star_x(x) (((x) & 0x7f7f7f7f) << 1) ^ ((((x) & 0x80808080) >> 7) * 0x1b)
+
+#define imix_col(y,x) \
+ u = star_x(x); \
+ v = star_x(u); \
+ w = star_x(v); \
+ t = w ^ (x); \
+ (y) = u ^ v ^ w; \
+ (y) ^= rotr(u ^ t, 8) ^ \
+ rotr(v ^ t, 16) ^ \
+ rotr(t,24)
+
+/* initialise the key schedule from the user supplied key */
+
+#define loop4(i) \
+{ t = rotr(t, 8); t = ls_box(t) ^ rco_tab[i]; \
+ t ^= E_KEY[4 * i]; E_KEY[4 * i + 4] = t; \
+ t ^= E_KEY[4 * i + 1]; E_KEY[4 * i + 5] = t; \
+ t ^= E_KEY[4 * i + 2]; E_KEY[4 * i + 6] = t; \
+ t ^= E_KEY[4 * i + 3]; E_KEY[4 * i + 7] = t; \
+}
+
+#define loop6(i) \
+{ t = rotr(t, 8); t = ls_box(t) ^ rco_tab[i]; \
+ t ^= E_KEY[6 * i]; E_KEY[6 * i + 6] = t; \
+ t ^= E_KEY[6 * i + 1]; E_KEY[6 * i + 7] = t; \
+ t ^= E_KEY[6 * i + 2]; E_KEY[6 * i + 8] = t; \
+ t ^= E_KEY[6 * i + 3]; E_KEY[6 * i + 9] = t; \
+ t ^= E_KEY[6 * i + 4]; E_KEY[6 * i + 10] = t; \
+ t ^= E_KEY[6 * i + 5]; E_KEY[6 * i + 11] = t; \
+}
+
+#define loop8(i) \
+{ t = rotr(t, 8); ; t = ls_box(t) ^ rco_tab[i]; \
+ t ^= E_KEY[8 * i]; E_KEY[8 * i + 8] = t; \
+ t ^= E_KEY[8 * i + 1]; E_KEY[8 * i + 9] = t; \
+ t ^= E_KEY[8 * i + 2]; E_KEY[8 * i + 10] = t; \
+ t ^= E_KEY[8 * i + 3]; E_KEY[8 * i + 11] = t; \
+ t = E_KEY[8 * i + 4] ^ ls_box(t); \
+ E_KEY[8 * i + 12] = t; \
+ t ^= E_KEY[8 * i + 5]; E_KEY[8 * i + 13] = t; \
+ t ^= E_KEY[8 * i + 6]; E_KEY[8 * i + 14] = t; \
+ t ^= E_KEY[8 * i + 7]; E_KEY[8 * i + 15] = t; \
+}
+
+static int
+aes_set_key(void *ctx_arg, const u8 *in_key, unsigned int key_len)
+{
+ struct aes_ctx *ctx = ctx_arg;
+ u32 i, t, u, v, w;
+ u32 P[AES_EXTENDED_KEY_SIZE];
+ u32 rounds;
+
+ if (key_len != 16 && key_len != 24 && key_len != 32) {
+ return -EINVAL;
+ }
+
+ ctx->key_length = key_len;
+
+ ctx->E = ctx->e_data;
+ ctx->D = ctx->d_data;
+
+ /* Ensure 16-Bytes alignmentation of keys for VIA PadLock. */
+ if ((int)(ctx->e_data) & 0x0F)
+ ctx->E += 4 - (((int)(ctx->e_data) & 0x0F) / sizeof (ctx->e_data[0]));
+
+ if ((int)(ctx->d_data) & 0x0F)
+ ctx->D += 4 - (((int)(ctx->d_data) & 0x0F) / sizeof (ctx->d_data[0]));
+
+ E_KEY[0] = u32_in (in_key);
+ E_KEY[1] = u32_in (in_key + 4);
+ E_KEY[2] = u32_in (in_key + 8);
+ E_KEY[3] = u32_in (in_key + 12);
+
+ /* Don't generate extended keys if the hardware can do it. */
+ if (aes_hw_extkey_available(key_len))
+ return 0;
+
+ switch (key_len) {
+ case 16:
+ t = E_KEY[3];
+ for (i = 0; i < 10; ++i)
+ loop4 (i);
+ break;
+
+ case 24:
+ E_KEY[4] = u32_in (in_key + 16);
+ t = E_KEY[5] = u32_in (in_key + 20);
+ for (i = 0; i < 8; ++i)
+ loop6 (i);
+ break;
+
+ case 32:
+ E_KEY[4] = u32_in (in_key + 16);
+ E_KEY[5] = u32_in (in_key + 20);
+ E_KEY[6] = u32_in (in_key + 24);
+ t = E_KEY[7] = u32_in (in_key + 28);
+ for (i = 0; i < 7; ++i)
+ loop8 (i);
+ break;
+ }
+
+ D_KEY[0] = E_KEY[0];
+ D_KEY[1] = E_KEY[1];
+ D_KEY[2] = E_KEY[2];
+ D_KEY[3] = E_KEY[3];
+
+ for (i = 4; i < key_len + 24; ++i) {
+ imix_col (D_KEY[i], E_KEY[i]);
+ }
+
+ /* PadLock needs a different format of the decryption key. */
+ rounds = 10 + (key_len - 16) / 4;
+
+ for (i = 0; i < rounds; i++) {
+ P[((i + 1) * 4) + 0] = D_KEY[((rounds - i - 1) * 4) + 0];
+ P[((i + 1) * 4) + 1] = D_KEY[((rounds - i - 1) * 4) + 1];
+ P[((i + 1) * 4) + 2] = D_KEY[((rounds - i - 1) * 4) + 2];
+ P[((i + 1) * 4) + 3] = D_KEY[((rounds - i - 1) * 4) + 3];
+ }
+
+ P[0] = E_KEY[(rounds * 4) + 0];
+ P[1] = E_KEY[(rounds * 4) + 1];
+ P[2] = E_KEY[(rounds * 4) + 2];
+ P[3] = E_KEY[(rounds * 4) + 3];
+
+ memcpy(D_KEY, P, AES_EXTENDED_KEY_SIZE_B);
+
+ return 0;
+}
+
+/* Tells whether the ACE is capable to generate
+ the extended key for a given key_len. */
+static inline int aes_hw_extkey_available(u8 key_len)
+{
+ /* TODO: We should check the actual CPU model/stepping
+ as it's likely that the capability will be
+ added in the next CPU revisions. */
+ if (key_len == 16)
+ return 1;
+ return 0;
+}
+
+static void aes_padlock(void *ctx_arg, u8 *out_arg, const u8 *in_arg,
+ const u8 *iv_arg, size_t nbytes, int encdec,
+ int mode)
+{
+ struct aes_ctx *ctx = ctx_arg;
+ char bigbuf[sizeof(union cword) + 16];
+ union cword *cword;
+ void *key;
+
+ if (((long)bigbuf) & 0x0F)
+ cword = (void*)(bigbuf + 16 - ((long)bigbuf & 0x0F));
+ else
+ cword = (void*)bigbuf;
+
+ /* Prepare Control word. */
+ memset (cword, 0, sizeof(union cword));
+ cword->b.encdec = !encdec; /* in the rest of cryptoapi ENC=1/DEC=0 */
+ cword->b.rounds = 10 + (ctx->key_length - 16) / 4;
+ cword->b.ksize = (ctx->key_length - 16) / 8;
+
+ /* Is the hardware capable to generate the extended key? */
+ if (!aes_hw_extkey_available(ctx->key_length))
+ cword->b.keygen = 1;
+
+ /* ctx->E starts with a plain key - if the hardware is capable
+ to generate the extended key itself we must supply
+ the plain key for both Encryption and Decryption. */
+ if (encdec == CRYPTO_OP_ENCRYPT || cword->b.keygen == 0)
+ key = ctx->E;
+ else
+ key = ctx->D;
+
+ padlock_aligner(out_arg, in_arg, iv_arg, key, cword,
+ nbytes, AES_BLOCK_SIZE, encdec, mode);
+}
+
+static void aes_padlock_ecb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
+ size_t nbytes, int encdec)
+{
+ aes_padlock(ctx, dst, src, NULL, nbytes, encdec, CRYPTO_MODE_ECB);
+}
+
+static void aes_padlock_cbc(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
+ size_t nbytes, int encdec)
+{
+ aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_CBC);
+}
+
+static void aes_padlock_cfb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
+ size_t nbytes, int encdec)
+{
+ aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_CFB);
+}
+
+static void aes_padlock_ofb(void *ctx, u8 *dst, const u8 *src, const u8 *iv,
+ size_t nbytes, int encdec)
+{
+ aes_padlock(ctx, dst, src, iv, nbytes, encdec, CRYPTO_MODE_OFB);
+}
+
+static struct crypto_capability padlock_caps[] =
+{
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_OFB, 1000},
+
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_OFB, 1000},
+
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_ENCRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_OFB, 1000},
+
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_128, CRYPTO_MODE_OFB, 1000},
+
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_192, CRYPTO_MODE_OFB, 1000},
+
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_ECB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CBC, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_CFB, 1000},
+ {CRYPTO_OP_DECRYPT, CRYPTO_TYPE_AES_256, CRYPTO_MODE_OFB, 1000},
+};
+static int padlock_cap_number = sizeof(padlock_caps)/sizeof(padlock_caps[0]);
+
+static void padlock_data_ready(struct crypto_device *dev);
+static int padlock_data_ready_reentry;
+
+static struct crypto_device padlock_device =
+{
+ .name = "via-padlock",
+ .data_ready = padlock_data_ready,
+ .cap = &padlock_caps[0],
+};
+
+static void process_session(struct crypto_session *s)
+{
+ int err;
+ u8 *key, *dst, *src, *iv;
+ size_t size, keylen;
+
+ key = ((u8 *)page_address(s->data.sg_key.page)) + s->data.sg_key.offset;
+ keylen = s->data.sg_key.length;
+ dst = ((u8 *)page_address(s->data.sg_dst.page)) + s->data.sg_dst.offset;
+ src = ((u8 *)page_address(s->data.sg_src.page)) + s->data.sg_src.offset;
+ size = s->data.sg_src.length;
+ iv = ((u8 *)page_address(s->data.sg_iv.page)) + s->data.sg_iv.offset;
+
+ err = aes_set_key(s->data.priv, key, keylen);
+ if (err)
+ return;
+
+ switch (s->ci.mode)
+ {
+ case CRYPTO_MODE_ECB:
+ aes_padlock_ecb(s->data.priv, dst, src, iv, size, s->ci.operation);
+ break;
+ case CRYPTO_MODE_CBC:
+ aes_padlock_cbc(s->data.priv, dst, src, iv, size, s->ci.operation);
+ break;
+ case CRYPTO_MODE_CFB:
+ aes_padlock_cfb(s->data.priv, dst, src, iv, size, s->ci.operation);
+ break;
+ case CRYPTO_MODE_OFB:
+ aes_padlock_ofb(s->data.priv, dst, src, iv, size, s->ci.operation);
+ break;
+ }
+
+ s->data.sg_dst.length = size;
+
+ return;
+}
+
+static void padlock_data_ready(struct crypto_device *dev)
+{
+ struct crypto_session *s, *n;
+
+ if (padlock_data_ready_reentry)
+ return;
+
+ padlock_data_ready_reentry++;
+ list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry)
+ {
+ if (!session_completed(s))
+ {
+ start_process_session(s);
+ process_session(s);
+ crypto_stat_complete_inc(s);
+ complete_session(s);
+ stop_process_session(s);
+ }
+ }
+ padlock_data_ready_reentry--;
+}
+
+int padlock_init_aes(void)
+{
+ u32 cpuid, edx;
+ u32 val = 0xC0000000;
+
+ cpuid = cpuid_eax(val);
+ edx = cpuid_edx(val);
+ printk("val=%x, cpuid=%x, edx=%x.\n", val, cpuid, edx);
+ if (cpuid >= val + 1)
+ {
+ printk("Board supports ACE.\n");
+ }
+ else
+ {
+ printk("Board does not support ACE.\n");
+ return -ENODEV;
+ }
+
+ printk(KERN_NOTICE "Using VIA PadLock ACE for AES algorithm (multiblock).\n");
+
+ padlock_device.cap_number = padlock_cap_number;
+
+ gen_tabs();
+ return crypto_device_add(&padlock_device);
+}
+
+void padlock_fini_aes(void)
+{
+ crypto_device_remove(&padlock_device);
+}
diff -Nru /tmp/empty/via-padlock/padlock-generic.c linux-2.6/drivers/acrypto/via-padlock/padlock-generic.c
--- /tmp/empty/via-padlock/padlock-generic.c 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/via-padlock/padlock-generic.c 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,194 @@
+/*
+ * Cryptographic API.
+ *
+ * Support for VIA PadLock hardware crypto engine.
+ *
+ * Linux developers:
+ * Michal Ludvig <mludvig@suse.cz>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <linux/crypto.h>
+#include <asm/byteorder.h>
+
+#include "padlock.h"
+#include "../acrypto.h"
+#include "../crypto_def.h"
+
+#define PFX "padlock: "
+
+typedef void (xcrypt_t)(u8 *input, u8 *output, u8 *key, u8 *iv,
+ void *control_word, u32 count);
+
+static inline void padlock_xcrypt_ecb(u8 *input, u8 *output, u8 *key,
+ u8 *iv, void *control_word, u32 count)
+{
+ asm volatile ("pushfl; popfl"); /* enforce key reload. */
+ asm volatile (".byte 0xf3,0x0f,0xa7,0xc8" /* rep xcryptecb */
+ : "=m"(*output), "+S"(input), "+D"(output)
+ : "d"(control_word), "b"(key), "c"(count));
+}
+
+static inline void padlock_xcrypt_cbc(u8 *input, u8 *output, u8 *key,
+ u8 *iv, void *control_word, u32 count)
+{
+ asm volatile ("pushfl; popfl"); /* enforce key reload. */
+ asm volatile (".byte 0xf3,0x0f,0xa7,0xd0" /* rep xcryptcbc */
+ : "=m"(*output), "+S"(input), "+D"(output)
+ : "d"(control_word), "b"(key), "c"(count), "a"(iv));
+}
+
+static inline void padlock_xcrypt_cfb(u8 *input, u8 *output, u8 *key,
+ u8 *iv, void *control_word, u32 count)
+{
+ asm volatile ("pushfl; popfl"); /* enforce key reload. */
+ asm volatile (".byte 0xf3,0x0f,0xa7,0xe0" /* rep xcryptcfb */
+ : "=m"(*output), "+S"(input), "+D"(output)
+ : "d"(control_word), "b"(key), "c"(count), "a"(iv));
+}
+
+static inline void padlock_xcrypt_ofb(u8 *input, u8 *output, u8 *key,
+ u8 *iv, void *control_word, u32 count)
+{
+ asm volatile ("pushfl; popfl"); /* enforce key reload. */
+ asm volatile (".byte 0xf3,0x0f,0xa7,0xe8" /* rep xcryptofb */
+ : "=m"(*output), "+S"(input), "+D"(output)
+ : "d"(control_word), "b"(key), "c"(count), "a"(iv));
+}
+
+void *crypto_aligned_kmalloc(size_t size, int mode, size_t alignment, void **index)
+{
+ char *ptr;
+
+ ptr = kmalloc(size + alignment, mode);
+ *index = ptr;
+ if (alignment > 1 && ((long)ptr & (alignment - 1))) {
+ ptr += alignment - ((long)ptr & (alignment - 1));
+ }
+
+ return ptr;
+}
+
+void padlock_aligner(u8 *out_arg, const u8 *in_arg, const u8 *iv_arg,
+ void *key, union cword *cword,
+ size_t nbytes, size_t blocksize,
+ int encdec, int mode)
+{
+ /* Don't blindly modify this structure - the items must
+ fit on 16-Bytes boundaries! */
+ struct padlock_xcrypt_data {
+ u8 iv[blocksize]; /* Initialization vector */
+ };
+
+ u8 *in, *out, *iv;
+ void *index = NULL;
+ char bigbuf[sizeof(struct padlock_xcrypt_data) + 16];
+ struct padlock_xcrypt_data *data;
+
+ /* Place 'data' at the first 16-Bytes aligned address in 'bigbuf'. */
+ if (((long)bigbuf) & 0x0F)
+ data = (void*)(bigbuf + 16 - ((long)bigbuf & 0x0F));
+ else
+ data = (void*)bigbuf;
+
+ if (((long)in_arg) & 0x0F) {
+ in = crypto_aligned_kmalloc(nbytes, GFP_KERNEL, 16, &index);
+ memcpy(in, in_arg, nbytes);
+ }
+ else
+ in = (u8*)in_arg;
+
+ if (((long)out_arg) & 0x0F) {
+ if (index)
+ out = in; /* xcrypt can work "in place" */
+ else
+ out = crypto_aligned_kmalloc(nbytes, GFP_KERNEL, 16, &index);
+ }
+ else
+ out = out_arg;
+
+ /* Always make a local copy of IV - xcrypt may change it! */
+ iv = data->iv;
+ if (iv_arg)
+ memcpy(iv, iv_arg, blocksize);
+
+
+ dprintk("data=%p\n", data);
+ dprintk("in=%p\n", in);
+ dprintk("out=%p\n", out);
+ dprintk("iv=%p\n", iv);
+ dprintk("nbytes=%d, blocksize=%d.\n", nbytes, blocksize);
+
+ switch (mode) {
+ case CRYPTO_MODE_ECB:
+ padlock_xcrypt_ecb(in, out, key, iv, cword, nbytes/blocksize);
+ break;
+
+ case CRYPTO_MODE_CBC:
+ padlock_xcrypt_cbc(in, out, key, iv, cword, nbytes/blocksize);
+ break;
+
+ case CRYPTO_MODE_CFB:
+ padlock_xcrypt_cfb(in, out, key, iv, cword, nbytes/blocksize);
+ break;
+
+ case CRYPTO_MODE_OFB:
+ padlock_xcrypt_ofb(in, out, key, iv, cword, nbytes/blocksize);
+ break;
+
+ default:
+ BUG();
+ }
+
+ /* Copy the 16-Byte aligned output to the caller's buffer. */
+ if (out != out_arg)
+ memcpy(out_arg, out, nbytes);
+
+ if (index)
+ kfree(index);
+}
+
+static int __init padlock_init(void)
+{
+ int ret = -ENOSYS;
+#if 0
+ if (!cpu_has_xcrypt) {
+ printk(KERN_ERR PFX "VIA PadLock not detected.\n");
+ return -ENODEV;
+ }
+
+ if (!cpu_has_xcrypt_enabled) {
+ printk(KERN_ERR PFX "VIA PadLock detected, but not enabled. Hmm, strange...\n");
+ return -ENODEV;
+ }
+#endif
+ if ((ret = padlock_init_aes())) {
+ printk(KERN_ERR PFX "VIA PadLock AES initialization failed.\n");
+ return ret;
+ }
+
+ if (ret == -ENOSYS)
+ printk(KERN_ERR PFX "Hmm, VIA PadLock was compiled without any algorithm.\n");
+
+ return ret;
+}
+
+static void __exit padlock_fini(void)
+{
+ padlock_fini_aes();
+}
+
+module_init(padlock_init);
+module_exit(padlock_fini);
+
+MODULE_DESCRIPTION("VIA PadLock crypto engine support.");
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_AUTHOR("Michal Ludvig");
diff -Nru /tmp/empty/via-padlock/padlock.h linux-2.6/drivers/acrypto/via-padlock/padlock.h
--- /tmp/empty/via-padlock/padlock.h 1970-01-01 03:00:00.000000000 +0300
+++ linux-2.6/drivers/acrypto/via-padlock/padlock.h 2004-10-30 09:01:56.000000000 +0400
@@ -0,0 +1,71 @@
+/*
+ * Cryptographic API.
+ *
+ * Copyright (c) 2004 Michal Ludvig <mludvig@suse.cz>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ *
+ */
+
+#ifndef _CRYPTO_PADLOCK_H
+#define _CRYPTO_PADLOCK_H
+
+#define AES_MIN_KEY_SIZE 16 /* in u8 units */
+#define AES_MAX_KEY_SIZE 32 /* ditto */
+#define AES_BLOCK_SIZE 16 /* ditto */
+#define AES_EXTENDED_KEY_SIZE 64 /* in u32 units */
+#define AES_EXTENDED_KEY_SIZE_B (AES_EXTENDED_KEY_SIZE * sizeof(u32))
+
+struct aes_ctx {
+ u32 e_data[AES_EXTENDED_KEY_SIZE+4];
+ u32 d_data[AES_EXTENDED_KEY_SIZE+4];
+ int key_length;
+ u32 *E;
+ u32 *D;
+};
+
+#define E_KEY ctx->E
+#define D_KEY ctx->D
+
+
+/* Control word. */
+#if 1
+union cword {
+ u32 cword[4];
+ struct {
+ int rounds:4;
+ int algo:3;
+ int keygen:1;
+ int interm:1;
+ int encdec:1;
+ int ksize:2;
+ } b;
+};
+#else
+union cword {
+ u32 cword[4];
+ struct {
+ unsigned rounds:4,
+ algo:3,
+ keygen:1,
+ interm:1,
+ encdec:1,
+ ksize:2;
+ } b;
+};
+#endif
+
+#define PFX "padlock: "
+
+void padlock_aligner(u8 *out_arg, const u8 *in_arg, const u8 *iv_arg,
+ void *key, union cword *cword,
+ size_t nbytes, size_t blocksize,
+ int encdec, int mode);
+
+int padlock_init_aes(void);
+void padlock_fini_aes(void);
+
+#endif /* _CRYPTO_PADLOCK_H */
--- linux-2.6/drivers/Kconfig.nocrypto 2004-10-30 09:05:52.000000000 +0400
+++ linux-2.6/drivers/Kconfig 2004-11-02 18:45:36.000000000 +0300
@@ -46,3 +46,5 @@
source "drivers/connector/Kconfig"
+source "drivers/acrypto/Kconfig"
+
--- linux-2.6/drivers/Kconfig.nocrypto 2004-11-02 18:53:10.000000000 +0300
+++ linux-2.6/drivers/Kconfig 2004-11-02 18:53:34.000000000 +0300
@@ -46,6 +46,8 @@
source "drivers/connector/Kconfig"
+source "drivers/acrypto/Kconfig"
+
source "drivers/misc/Kconfig"
source "drivers/media/Kconfig"
--- linux-2.6/drivers/Makefile.nocrypto 2004-11-02 18:54:05.000000000 +0300
+++ linux-2.6/drivers/Makefile 2004-11-02 18:54:17.000000000 +0300
@@ -51,6 +51,7 @@
obj-$(CONFIG_I2C) += i2c/
obj-$(CONFIG_W1) += w1/
obj-$(CONFIG_CONNECTOR) += connector/
+obj-$(CONFIG_CONNECTOR) += acrypto/
obj-$(CONFIG_PHONE) += telephony/
obj-$(CONFIG_MD) += md/
obj-$(CONFIG_BT) += bluetooth/
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-11-01 5:58 ` Evgeniy Polyakov
@ 2004-12-14 6:56 ` James Morris
2004-12-14 7:58 ` Evgeniy Polyakov
0 siblings, 1 reply; 57+ messages in thread
From: James Morris @ 2004-12-14 6:56 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: netdev, cryptoapi, David S. Miller
On Mon, 1 Nov 2004, Evgeniy Polyakov wrote:
[Sorry for taking so long to get back to this].
> > c) Async engine: scheduling/batching/routing etc.
> >
> > This seems to be the core of what you've developed so far. I'm not sure
> > if we need pluggable load balancers. How would the sysadmin select the
> > correct one? The simpler this code is the better.
>
> By following command:
> echo -en "simple_lb" > /sys/class/crypto_lb/lbs
> or by sending [still not implemented] connector's(netlink) command.
The real question was how would the sysadmin know which is the best load
balancer to configure. Start simple, make one good load balancer, and
ponly create a pluggable framework only if a demonstratable need arises.
> > Overall I think it's a good start. There are some chicken & egg type
> > problems when you don't have GPL drivers, hardware or an existing async
> > API, so I'd imagine that this will all continue to evolve: with more
> > hardware we can write/incorporate more drivers, with more drivers we can
> > further refine the async support and API etc.
>
> That is true, but I think not all parts of API should be exported as GPL
> only.
Why do you think this?
> > Here are some issues that I noticed:
> >
> >
> > What is main_crypto_device? Is this a placeholder for when there are no
> > other devices registered?
>
> main_crypto_device is just virtual device into which all sessions are
> placed along with specific crypto device queue.
> it is usefull for cases when some driver decides that HW is broken and
> marks itself like not working, then scheduler _may_ (simple_lb does not
> have such feature) and actually _should_ move all sessions that are
> placed into broken device's queue into other devices or drop them(call
> callback with special session flag).
No reason to drop them, just fall back to software.
> > Async & sync drivers need to both be treated as crypto drivers.
> >
> >
> > static inline void crypto_route_free(struct crypto_route *rt)
> > {
> > crypto_device_put(rt->dev);
> > rt->dev = NULL;
> > kfree(rt);
> > }
> >
> > Why do you set rt->dev to NULL here? It should not still be referenceable
> > at this point.
>
> crypto_route_free() can be called only from crypto_route_del() which
> unlinks given route thus noone can reference rt->dev(but of course
> asnyone can have it's private copy, obtained under the lock).
> Any route access is guarded by route list lock.
No, if you're about to kfree(rt), you cannot also be able to dereference
it.
> > __crypto_route_del() etc:
> >
> > Why are you rolling your own list management and not using the list.h
> > functions?
>
> It is more convenient here to use queue like sk_buf does.
Use list.h or put what you need into list.h.
> > +struct crypto_device_stat
> > +{
> > + __u64 scompleted;
> > + __u64 sfinished;
> > ...
> >
> > Please see how networking stats are implemented (e.g. netif_rx_stats) and
> > previous netdev discussions on 64-bit counters. Please only use __u64 and
> > similar when exporting structures to userspace.
>
> I use __ prefix for any parameter that is exported to userspace.
But this structure is not.
> I've seen several attempts to convert network statistic to 64bit values,
> with backward compatibility it is not an easy task. Let's do not
> catch this again.
To clarifu: the best way to do this is to use per-cpu counters of unsigned
long and add them together in userspace.
- James
--
James Morris
<jmorris@redhat.com>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-11-02 16:12 ` Evgeniy Polyakov
@ 2004-12-14 7:23 ` James Morris
2004-12-14 8:07 ` Evgeniy Polyakov
0 siblings, 1 reply; 57+ messages in thread
From: James Morris @ 2004-12-14 7:23 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: jamal, Michal Ludvig, netdev, cryptoapi
On Tue, 2 Nov 2004, Evgeniy Polyakov wrote:
Some feedback:
+#define session_completed(s) (s->ci.flags & SESSION_COMPLETED)
+#define complete_session(s) do {s->ci.flags |= SESSION_COMPLETED;} while(0)
+#define uncomplete_session(s) do {s->ci.flags &= ~SESSION_COMPLETED;} while (0)
etc.
Please use static inlines for all of these things.
Are these supposed to be exported to userspace? Why?
+struct crypto_conn_data
+{
+ char name[SCACHE_NAMELEN];
+ __u16 cmd;
Again, exporting to userspace?
+ list_for_each_entry_safe(__dev, n, &cdev_list, cdev_entry)
+ {
+ if (compare_device(__dev, dev))
+ {
Incorrect coding style (and many more).
As mentioned before, pluggable load balacers are not needed now, and
complicate the code.
- James
--
James Morris
<jmorris@redhat.com>
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-12-14 6:56 ` James Morris
@ 2004-12-14 7:58 ` Evgeniy Polyakov
0 siblings, 0 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-12-14 7:58 UTC (permalink / raw)
To: James Morris; +Cc: netdev, cryptoapi, David S. Miller
[-- Attachment #1: Type: text/plain, Size: 5611 bytes --]
On Tue, 2004-12-14 at 01:56 -0500, James Morris wrote:
> On Mon, 1 Nov 2004, Evgeniy Polyakov wrote:
>
> [Sorry for taking so long to get back to this].
>
> > > c) Async engine: scheduling/batching/routing etc.
> > >
> > > This seems to be the core of what you've developed so far. I'm not sure
> > > if we need pluggable load balancers. How would the sysadmin select the
> > > correct one? The simpler this code is the better.
> >
> > By following command:
> > echo -en "simple_lb" > /sys/class/crypto_lb/lbs
> > or by sending [still not implemented] connector's(netlink) command.
>
> The real question was how would the sysadmin know which is the best load
> balancer to configure. Start simple, make one good load balancer, and
> ponly create a pluggable framework only if a demonstratable need arises.
By reading documentation which will be written some day...
If it has only SW crypto it does not need scheduler at all - only very
simple switch,
but if system has diferent kinds of HW crypto cards, and sysadmin knows
about it's
capabilities it can be better to turn more advanced scheduler on - for
example one
which can take packet size into account.
It will be a bit slower while searching needed device, but will have big
win
when doing actual crypto processing.
> > > Overall I think it's a good start. There are some chicken & egg type
> > > problems when you don't have GPL drivers, hardware or an existing async
> > > API, so I'd imagine that this will all continue to evolve: with more
> > > hardware we can write/incorporate more drivers, with more drivers we can
> > > further refine the async support and API etc.
> >
> > That is true, but I think not all parts of API should be exported as GPL
> > only.
>
> Why do you think this?
In my current acrypto tree I export all symbols as GPL except
crypto_session_alloc().
Lets greedy scums use at least bits of new functionality. :)
>
> > > Here are some issues that I noticed:
> > >
> > >
> > > What is main_crypto_device? Is this a placeholder for when there are no
> > > other devices registered?
> >
> > main_crypto_device is just virtual device into which all sessions are
> > placed along with specific crypto device queue.
> > it is usefull for cases when some driver decides that HW is broken and
> > marks itself like not working, then scheduler _may_ (simple_lb does not
> > have such feature) and actually _should_ move all sessions that are
> > placed into broken device's queue into other devices or drop them(call
> > callback with special session flag).
>
> No reason to drop them, just fall back to software.
SW is just one of the crypto devices, consider emebedded system without
gigantic processor time which can be spent doing crypto processing.
Or if it is asymmetric crypto?
If SW exists and is loaded into acrypto it _will_ be called if needed.
> > > Async & sync drivers need to both be treated as crypto drivers.
> > >
> > >
> > > static inline void crypto_route_free(struct crypto_route *rt)
> > > {
> > > crypto_device_put(rt->dev);
> > > rt->dev = NULL;
> > > kfree(rt);
> > > }
> > >
> > > Why do you set rt->dev to NULL here? It should not still be referenceable
> > > at this point.
> >
> > crypto_route_free() can be called only from crypto_route_del() which
> > unlinks given route thus noone can reference rt->dev(but of course
> > asnyone can have it's private copy, obtained under the lock).
> > Any route access is guarded by route list lock.
>
> No, if you're about to kfree(rt), you cannot also be able to dereference
> it.
I mean reference to device, not route.
Crypto route in this point can be accessed only through crypto_route_del
().
Any routing _must_ be accessed _only_ by crypto_route_* functions from
crypto_route.h which are always performed with proper locking.
> > > __crypto_route_del() etc:
> > >
> > > Why are you rolling your own list management and not using the list.h
> > > functions?
> >
> > It is more convenient here to use queue like sk_buf does.
>
> Use list.h or put what you need into list.h.
list.h will never have struct sk_buff_head or any other queueing
primitives.
crypto_route is not a list, it is a queue.
> > > +struct crypto_device_stat
> > > +{
> > > + __u64 scompleted;
> > > + __u64 sfinished;
> > > ...
> > >
> > > Please see how networking stats are implemented (e.g. netif_rx_stats) and
> > > previous netdev discussions on 64-bit counters. Please only use __u64 and
> > > similar when exporting structures to userspace.
> >
> > I use __ prefix for any parameter that is exported to userspace.
>
> But this structure is not.
struct crypto_device_stat
{
__u64 scompleted;
__u64 sfinished;
__u64 sstarted;
__u64 kmem_failed;
};
Probably version mismatch...
> > I've seen several attempts to convert network statistic to 64bit values,
> > with backward compatibility it is not an easy task. Let's do not
> > catch this again.
>
> To clarifu: the best way to do this is to use per-cpu counters of unsigned
> long and add them together in userspace.
It is the fastest way sure,
current acrypto statistic code is not fast, it takes a stat lock to
guarantee
that fields are changed atomically since some (per
session_crypto_initializer )
of them are used in scheduler.
>
> - James
--
Evgeniy Polyakov
Crash is better than data corruption -- Arthur Grabowski
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: Asynchronous crypto layer.
2004-12-14 7:23 ` James Morris
@ 2004-12-14 8:07 ` Evgeniy Polyakov
0 siblings, 0 replies; 57+ messages in thread
From: Evgeniy Polyakov @ 2004-12-14 8:07 UTC (permalink / raw)
To: James Morris; +Cc: jamal, Michal Ludvig, netdev, cryptoapi
[-- Attachment #1: Type: text/plain, Size: 1485 bytes --]
On Tue, 2004-12-14 at 02:23 -0500, James Morris wrote:
> On Tue, 2 Nov 2004, Evgeniy Polyakov wrote:
>
> Some feedback:
>
> +#define session_completed(s) (s->ci.flags & SESSION_COMPLETED)
> +#define complete_session(s) do {s->ci.flags |= SESSION_COMPLETED;} while(0)
> +#define uncomplete_session(s) do {s->ci.flags &= ~SESSION_COMPLETED;} while (0)
> etc.
>
> Please use static inlines for all of these things.
> Are these supposed to be exported to userspace? Why?
No, they are not supposed to be exported, I will convert them.
>
> +struct crypto_conn_data
> +{
> + char name[SCACHE_NAMELEN];
> + __u16 cmd;
>
> Again, exporting to userspace?
Yes, it is part of the acrypto control over connector(netlink) protocol.
>
> + list_for_each_entry_safe(__dev, n, &cdev_list, cdev_entry)
> + {
> + if (compare_device(__dev, dev))
> + {
>
> Incorrect coding style (and many more).
Sigh... :)
<very sadly> I know, know, and will change it. </very sadly>
> As mentioned before, pluggable load balacers are not needed now, and
> complicate the code.
I am still not giving up :) -
we have several TCP congestion models, we have different IO schedulers,
we can tune routing code(although we can not change hash to tree in
runtime).
>
> - James
--
Evgeniy Polyakov
Crash is better than data corruption -- Arthur Grabowski
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 57+ messages in thread
end of thread, other threads:[~2004-12-14 8:07 UTC | newest]
Thread overview: 57+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2004-10-29 6:22 Asynchronous crypto layer Evgeniy Polyakov
2004-10-29 12:42 ` jamal
2004-10-29 14:06 ` Evgeniy Polyakov
2004-10-29 14:03 ` Michal Ludvig
2004-10-29 14:36 ` Evgeniy Polyakov
2004-10-29 14:53 ` Michal Ludvig
2004-10-29 15:11 ` jamal
2004-10-29 15:27 ` Evgeniy Polyakov
2004-10-30 20:39 ` Eugene Surovegin
2004-10-30 21:17 ` Evgeniy Polyakov
2004-10-30 21:09 ` Eugene Surovegin
2004-10-30 21:46 ` Evgeniy Polyakov
2004-10-29 15:08 ` jamal
2004-10-29 15:37 ` Evgeniy Polyakov
2004-10-29 15:28 ` Michal Ludvig
2004-10-29 16:16 ` James Morris
2004-10-29 16:45 ` Evgeniy Polyakov
2004-10-29 20:00 ` Sam Leffler
2004-10-31 6:09 ` James Morris
2004-10-31 6:35 ` Sam Leffler
2004-10-30 20:35 ` Eugene Surovegin
2004-10-30 21:04 ` Evgeniy Polyakov
2004-10-30 20:56 ` Eugene Surovegin
2004-10-30 21:24 ` Evgeniy Polyakov
2004-10-30 23:41 ` jamal
2004-10-31 9:13 ` Evgeniy Polyakov
2004-10-31 10:46 ` Michal Ludvig
2004-10-31 15:03 ` jamal
2004-10-31 16:07 ` James Morris
2004-11-01 6:01 ` Evgeniy Polyakov
2004-10-31 15:36 ` Michael Richardson
2004-10-31 16:09 ` James Morris
2004-10-31 14:56 ` jamal
2004-11-01 5:12 ` Evgeniy Polyakov
2004-10-29 17:44 ` James Morris
2004-10-29 17:46 ` James Morris
2004-10-30 5:20 ` Evgeniy Polyakov
2004-10-31 5:43 ` James Morris
2004-10-30 5:19 ` Evgeniy Polyakov
2004-10-30 8:34 ` Evgeniy Polyakov
2004-10-30 8:36 ` Evgeniy Polyakov
2004-10-30 16:57 ` Michal Ludvig
2004-10-30 17:40 ` Evgeniy Polyakov
2004-10-30 20:17 ` Michal Ludvig
2004-10-30 20:56 ` Evgeniy Polyakov
2004-10-30 19:42 ` James Morris
2004-10-29 19:56 ` Sam Leffler
2004-10-30 6:16 ` Christoph Hellwig
2004-10-30 20:37 ` Evgeniy Polyakov
2004-10-31 16:05 ` James Morris
2004-11-01 5:58 ` Evgeniy Polyakov
2004-12-14 6:56 ` James Morris
2004-12-14 7:58 ` Evgeniy Polyakov
2004-11-02 15:24 ` Evgeniy Polyakov
2004-11-02 16:12 ` Evgeniy Polyakov
2004-12-14 7:23 ` James Morris
2004-12-14 8:07 ` Evgeniy Polyakov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).